CogLM: Tracking Cognitive Development of Large Language Models
Piaget's Theory of Cognitive Development (PTC) posits that the development of cognitive levels forms the foundation for human learning across various abilities. As Large Language Models (LLMs) have recently shown remarkable abilities across a wide variety of tasks, we are curious about the cogn...
Saved in:
Main Authors | , , , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
17.08.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Piaget's Theory of Cognitive Development (PTC) posits that the development of
cognitive levels forms the foundation for human learning across various
abilities. As Large Language Models (LLMs) have recently shown remarkable
abilities across a wide variety of tasks, we are curious about the cognitive
levels of current LLMs: to what extent they have developed and how this
development has been achieved. To this end, we construct a benchmark CogLM
(Cognitive Ability Evaluation for Language Model) based on PTC to assess the
cognitive levels of LLMs. CogLM comprises 1,220 questions spanning 10 cognitive
abilities crafted by more than 20 human experts, providing a comprehensive
testbed for the cognitive levels of LLMs. Through extensive experiments across
multiple mainstream LLMs with CogLM, we find that: (1) Human-like cognitive
abilities have emerged in advanced LLMs (GPT-4), comparable to those of a
20-year-old human. (2) The parameter size and optimization objective are two
key factors affecting the cognitive levels of LLMs. (3) The performance on
downstream tasks is positively correlated with the level of cognitive
abilities. These findings fill the gap in research on the cognitive abilities
of LLMs, tracing the development of LLMs from a cognitive perspective and
guiding the future direction of their evolution. |
---|---|
DOI: | 10.48550/arxiv.2408.09150 |