Concept Formation

Abstract

Concept formation, a form of thinking, is the process by which the mind abstracts categories from experience, grouping distinct objects into kinds that support inference and communication. Early theory treated a concept as a definition discovered by testing hypotheses, but most natural categories proved to have graded structure, organised around typical members rather than defining features. Competing formal accounts explain how categories are learned: prototype models store a summary of each class, exemplar models store the individual instances, and Bayesian models treat learning as inference over hypotheses. Evidence that rule-based and similarity-based learning depend on partly separable brain systems has recast the debate as one about multiple mechanisms rather than a single rule. This article surveys the definition, the principal theories, and the neural basis of concept formation.

Keywords: concept formation, categorization, prototype, exemplar, category learning

To treat two different things as the same in some respect is the first act of thought. A child who has seen a handful of dogs comes to expect the next four-legged barker to fetch rather than pounce, and in doing so has formed a concept: a mental grouping that reaches beyond the instances that produced it. Concept formation is the study of how such groupings are built from experience and how they are used once built. The problem is deceptively hard, because the members of a natural category rarely share a tidy set of defining features, and the same objects can be carved into different categories depending on what the learner is trying to do. How the mind settles on useful kinds, and what it stores when it does, is among the oldest questions in cognitive psychology and one of the most actively modelled.

What Concept Formation Is

A concept is a mental representation of a category: a way of treating a potentially unlimited set of distinct things as instances of one kind. Its value lies in inference. Once an object is categorised as a bird, a great deal that was never observed of that particular object — that it likely has hollow bones, lays eggs, descends from dinosaurs — becomes available for free. Concepts are therefore the units on which most of higher cognition runs, the terms of thought and the referents of words, and forming them well is what lets finite experience generalise to an open future.

The central difficulty is that the categories worth having are not defined by necessary and sufficient conditions. The classical theory, inherited from Aristotle, held that a concept is a definition: a list of features an object must have to belong. But no such list picks out the everyday category game, or chair, or dog, without either excluding clear members or admitting clear non-members. What the members of a natural category share is not a common essence but a network of overlapping resemblances, so that membership is a matter of degree rather than a yes-or-no fact (Rosch & Mervis, 1975). Concept formation is thus not the discovery of a definition but the extraction of structure from a set of imperfectly similar examples.

Rules and Hypotheses

The first experimental programme to treat concept formation as an information-processing problem framed it as the testing of hypotheses. In the founding study, participants shown instances labelled positive or negative for an unknown concept were understood to entertain candidate rules — red, or red and square — and to revise them as feedback arrived, choosing among strategies that traded thoroughness against memory load (Bruner, Goodnow, & Austin, 1956). The concept, in this view, was a logical rule over features, and learning it was a search through the space of possible rules.

That framing predicts that a category's difficulty should track the logical complexity of the rule that defines it, and a classic experiment confirmed a version of the prediction. Using categories built from three binary dimensions, learners mastered a rule that depended on a single dimension fastest, a rule requiring an exclusive-or of two dimensions more slowly, and a rule with no compact logical description slowest of all (Shepard, Hovland, & Jenkins, 1961). The difficulty ordering became a benchmark that every later model of category learning has had to reproduce. But the rule-testing account struggled with the categories people actually use, whose members resist any compact rule, and this tension drove the field toward representations built on similarity rather than logic.

Prototypes and Exemplars

The evidence that categories have graded rather than all-or-none structure came first from studies of abstraction. When people are trained on distortions of a dot pattern they never actually see — the prototype — they later classify that unseen prototype as accurately as the training items, and sometimes more so, as though the mind had averaged the instances into a central tendency (Posner & Keele, 1968). Extending this to natural categories, participants judged some members far more typical than others: a robin is a better bird than a penguin, and the more typical members are named faster, learned earlier, and produced first (Rosch, 1975). Typicality is the psychological signature of graded structure, and no classical definition can explain it.

Prototype theory accounts for this by proposing that what the learner stores is a single summary representation of the category — its central tendency — against which new items are compared. A rival account keeps the graded structure but denies the abstraction: the exemplar view holds that the mind stores the individual instances themselves and classifies a new item by its summed similarity to all remembered members of each category (Medin & Schaffer, 1978). Both predict typicality effects, because an item near the centre of a category is close to the prototype and also, on average, close to many exemplars, but they diverge sharply when categories have irregular structure, and much of the field's subsequent work has been an effort to tell them apart.

Formal Models of Categorization

The exemplar view was made precise as the generalized context model, which places stimuli in a multidimensional psychological space, measures the distance between them, and turns distance into similarity through an exponentially decaying function (Nosofsky, 1986). Its decisive addition was selective attention: the space can be stretched along dimensions that matter for the current categorisation and compressed along those that do not, so learning a concept includes learning where to look. With attention weights fitted, the model reproduces the classic difficulty orderings and the fine detail of individual classification choices, and it set the quantitative standard the whole field now works to.

Later models added the missing dynamics. ALCOVE embedded the same similarity computation in a connectionist network trained by error correction, so that attention and category associations are learned gradually from feedback rather than fitted after the fact (Kruschke, 1992). SUSTAIN added the ability to grow: it begins with a single cluster and recruits a new one only when a surprising item cannot be absorbed, which lets one architecture behave like a prototype model for simple categories and like an exemplar model for complex ones (Love, Medin, & Gureckis, 2004). A different tradition recast the whole problem as statistical inference, in which generalising from examples is Bayesian reasoning over a hypothesis space of candidate categories, weighting each hypothesis by its prior and its fit to the evidence (Tenenbaum & Griffiths, 2001). These accounts differ in mechanism but agree that concept formation is the disciplined extraction of structure, not the memorisation of a rule.

Multiple Systems

A single similarity computation, however flexible, may not be the whole story. The multiple-systems view holds that humans learn categories with at least two competing subsystems: an explicit, verbalisable system that forms and tests logical rules, and an implicit, procedural system that gradually associates regions of perceptual space with responses (Ashby, Alfonso-Reese, Turken, & Waldron, 1998). The explicit system dominates when a category can be captured by a rule on a single obvious dimension; the implicit system takes over when the category boundary cuts across dimensions in a way no verbal rule describes.

The evidence for a division is behavioural before it is neural. Categories learnable by a one-dimensional rule and categories requiring integration across dimensions dissociate under manipulations that should not matter if a single mechanism were at work: a delay between response and feedback, or a demanding secondary task, selectively impairs one kind of learning and spares the other. Read this way, the prototype-versus-exemplar debate is partly a false dichotomy, because different structures engage different systems, and the question shifts from which model is right to which system a given category recruits.

Even the multiple-systems view, however, still grounds categorisation in similarity, and a deeper critique questions whether similarity is enough. Because any two things share indefinitely many features, unconstrained similarity cannot by itself explain why some groupings strike people as coherent and others as arbitrary; what does the constraining, on this account, is the learner's background knowledge — the intuitive theories that say which features matter and how they hang together (Murphy & Medin, 1985). A category like things to remove from a burning house coheres not through perceptual resemblance but through a goal, and even everyday kinds are held together partly by naive theories of biology or mechanics. The theory-theory does not overturn the similarity models so much as bound them, marking the point at which concept formation shades into the broader use of knowledge in thought.

Concept Learning in the Brain

The multiple-systems proposal makes a neural prediction, and imaging and patient work broadly bear it out: category learning draws on a distributed network in which the prefrontal cortex, the medial temporal lobe, and the basal ganglia play distinguishable roles, with the striatum supporting the slow, feedback-driven learning of perceptual categories and prefrontal regions supporting explicit rules (Seger & Miller, 2010). No single locus is the seat of concepts; what the brain provides is a set of interacting systems matched to the different structures categories can have.

More recent work has watched representations change as concepts are acquired. Multivariate analysis of hippocampal activity shows that the neural representation of an object is dynamically updated as a learner discovers which of its features are relevant, warping representational space in the same way selective attention warps the psychological space of formal models (Mack, Love, & Preston, 2016). Studies contrasting prototype and exemplar learning localise abstract, generalised category representations to the ventromedial prefrontal cortex and hippocampus, distinct from the storage of individual instances (Bowman & Zeithamova, 2018), and later work tracks both kinds of representation forming in parallel across the course of learning (Bowman, Iwashita, & Zeithamova, 2020). The convergent picture, synthesised across methods, is that the brain implements several of the mechanisms the competing models describe rather than adjudicating between them (Zeithamova et al., 2019).

Worked Example

Consider a learner who has seen two small categories in a two-dimensional feature space, each dimension running from 0 to 5. Category A contains the items at (1, 1) and (2, 1); Category B contains the items at (4, 4) and (5, 4). A new item appears at (2, 2). How should an exemplar model classify it?

The generalized context model computes the distance from the new item to each stored exemplar, converts distance to similarity, and sums the similarities within each category. Using a city-block metric with equal attention weights of 0.5 on each dimension, the distance to A's first exemplar (1, 1) is 0.5|2−1| + 0.5|2−1| = 1.0, and similarity is e^(−1.0) = 0.368; to A's second exemplar (2, 1) the distance is 0.5, giving e^(−0.5) = 0.607. Summed, Category A draws 0.975. The new item is far from B: the distances to (4, 4) and (5, 4) are 2.0 and 2.5, giving similarities 0.135 and 0.082, so Category B draws 0.217. The probability of choosing A is its share of the total similarity, 0.975 / (0.975 + 0.217) = 0.82.

The number itself matters less than what moves it. Because attention weights multiply the distance on each dimension, shifting attention toward the dimension that actually separates the categories — here the vertical one, on which A sits low and B sits high — would stretch the gap and drive the probability still higher, while attending to the uninformative dimension would blur the categories together. Selective attention is therefore not a detail of the fit but the mechanism by which the same instances yield different concepts, which is why every serious model of concept formation now carries an attentional parameter.

Discussion

The trajectory of the field is a steady retreat from the idea that a concept is a definition. The classical view fell to the discovery of graded structure; the rule-testing account fell to the categories people actually use; and the long contest between prototype and exemplar models resolved, in the end, into the recognition that both describe real mechanisms the brain can deploy. What survives across every account is the primacy of similarity in a psychological space whose shape is itself learned, and the insistence that concept formation generalises — that its whole purpose is to license inferences about instances never seen. Table 1 sets the principal accounts side by side, with what each claims is stored and where each is strongest.

That the debate resolved into pluralism rather than a winner is itself the lesson. A category with a clean one-dimensional rule and a category defined by a tangle of correlated features are different computational problems, and it would be strange if one representation solved both. The mind appears to hold several solutions at once and to lean on whichever the structure rewards, which is why the neural evidence finds not a concept centre but a network (Seger & Miller, 2010). Concept formation, understood this way, is less a single faculty than a coalition of mechanisms bound by a common goal: to turn the particulars of experience into kinds that pay off in prediction.

Current Directions

The sharpest current challenge comes from the comparison with machine learning. Deep networks now classify images at human accuracy, but they need thousands of labelled examples to do what a person does from one or two, and the gap has renewed interest in the mechanisms of rapid abstraction. One influential response models human concept learning as probabilistic program induction: a concept is represented not as a point in feature space but as a small generative program that can produce new instances, and learning is inferring that program from a handful of examples — an approach that matches human one-shot generalisation on handwritten characters where standard classifiers fail (Lake, Salakhutdinov, & Tenenbaum, 2015). The proposal reframes the old prototype-exemplar question by suggesting that what is stored is neither a summary nor a set of instances but a procedure for generating the category.

A second front uses neural decoding to settle the storage debate empirically rather than by model fit. By tracking the emergence of prototype and exemplar representations across learning, recent imaging work shows that the two are not mutually exclusive but form in parallel, with their relative strength depending on category structure and on the individual learner (Bowman, Iwashita, & Zeithamova, 2020). The methods are beginning to answer, in the brain, a question that a century of behaviour could only constrain, and they point toward the same pluralism the theory has converged on.

Common Misconceptions

A concept is a definition that all members satisfy.

Most natural categories have no set of necessary and sufficient features; their members share overlapping resemblances rather than a common essence, and membership is graded (Rosch & Mervis, 1975). The classical definitional view fails for the categories people actually use.

All members of a category are equally good examples.

Typicality is graded: some members are judged better, named faster, and learned earlier than others, and this graded structure is the central fact any theory must explain (Rosch, 1975).

Forming a concept means storing an average.

Storing a single central tendency is only the prototype account; exemplar models classify by similarity to remembered individual instances without ever abstracting an average, and they often fit the data better for irregular categories (Medin & Schaffer, 1978).

Glossary

Basic level.

The level of a category hierarchy, such as dog between animal and collie, at which categories are most differentiated and most readily used.

Bayesian inference.

Updating a prior distribution over candidate hypotheses into a posterior in proportion to how well each explains the observed examples.

Categorization.

The act of assigning an object or event to a class, treating it as an instance of a kind rather than a unique particular.

Classical theory.

The view that a concept is a definition specifying features necessary and jointly sufficient for membership; falsified for most natural categories.

Concept.

A mental representation of a category that groups distinct instances into a kind and supports inference about members never observed.

Exemplar model.

A theory holding that a category is represented by its stored individual instances, and a new item is classified by its summed similarity to them.

Family resemblance.

A structure in which category members share overlapping features without any single feature common to all, as faces in a family resemble one another pairwise.

Generalized context model.

A formal exemplar model that classifies by similarity in a psychological space, with selective attention weighting each dimension.

Hypothesis testing.

An account of concept learning as the successive proposal and revision of candidate rules over features in light of labelled feedback.

Multiple-systems theory.

The proposal that category learning is served by separable subsystems — an explicit rule-based one and an implicit procedural one — matched to different category structures.

Prototype.

A summary representation of a category's central tendency, against which new items are compared under prototype theory.

Psychological space.

A multidimensional representation in which stimuli are points and perceived similarity falls off with distance, the arena in which similarity models operate.

Rule-based category.

A category whose membership can be captured by an easily verbalised logical rule, typically over one salient dimension.

Selective attention.

The weighting of stimulus dimensions so that relevant ones count more toward similarity, stretching the psychological space along them and compressing it along the rest.

Similarity.

The graded psychological closeness between two representations, typically a decaying function of their distance in psychological space, and the currency of prototype and exemplar models.

Typicality.

The degree to which an instance is judged a good example of its category; the behavioural marker of graded category structure.

Key Researchers

F. Gregory Ashby (b. 1953). Distinguished Professor of Psychological and Brain Sciences at the University of California, Santa Barbara; he proposed the multiple-systems theory (COVIS) separating explicit rule-based from implicit procedural category learning. Faculty Page - ORCID

Jerome S. Bruner (1915-2016). Psychologist at Harvard University, Oxford, and New York University; his early study of concept attainment framed categorisation as active hypothesis testing over features. Wikipedia

John K. Kruschke (b. 1958). Provost Professor at Indiana University Bloomington; he built ALCOVE, the connectionist exemplar model that learns selective attention by error correction. Faculty Page - ORCID

Brenden M. Lake (b. 1984). Associate Professor at Princeton University; he modelled human one-shot concept learning as probabilistic program induction, matching people where standard classifiers fail. Faculty Page - ORCID

Bradley C. Love (b. 1970). Professor of Cognitive and Decision Sciences at University College London; he developed the SUSTAIN clustering model and traced hippocampal representational change during learning. Faculty Page - ORCID

Douglas L. Medin (b. 1944). Professor Emeritus of Psychology at Northwestern University; he introduced the context (exemplar) theory of classification and co-developed SUSTAIN. Wikipedia - Faculty Page - ORCID

Robert M. Nosofsky (b. 1956). Distinguished Professor at Indiana University Bloomington; he formalised the generalized context model, linking selective attention to the identification-categorization relationship. Faculty Page - ORCID

Michael I. Posner (b. 1936). Professor Emeritus of Psychology at the University of Oregon; his dot-pattern studies with Steven Keele gave prototype abstraction its classic empirical basis. National Medal of Science laureate. Wikipedia - Faculty Page

Eleanor Rosch (b. 1938). Professor Emerita of Psychology at the University of California, Berkeley; she established prototype theory and the graded, family-resemblance structure of natural categories. Wikipedia - Faculty Page

Joshua B. Tenenbaum (b. 1972). Professor of Brain and Cognitive Sciences at the Massachusetts Institute of Technology; he recast generalisation as Bayesian inference and co-authored probabilistic program induction. Wikipedia - Faculty Page - ORCID

Dagmar Zeithamova (b. 1978). Associate Professor of Psychology at the University of Oregon; she uses neural decoding to dissociate prototype and exemplar representations and localise concept generalisation. Faculty Page - ORCID

Frequently Asked Questions

What is concept formation?
Concept formation is the process by which the mind abstracts a category from encountered instances, building a mental grouping that lets finite experience generalise to new members and support inference about them (Rosch & Mervis, 1975).

Why did the classical definitional view of concepts fail?
Most natural categories have no set of necessary and sufficient features that includes every member and excludes every non-member; their members instead share overlapping resemblances, so membership is graded rather than all-or-none (Rosch & Mervis, 1975).

What is the difference between a prototype and an exemplar model?
A prototype model stores one summary representation of a category's central tendency, whereas an exemplar model stores the individual instances and classifies a new item by its summed similarity to all of them (Medin & Schaffer, 1978).

What is typicality?
Typicality is the graded degree to which an instance is a good example of its category; more typical members are judged better, named faster, and learned earlier, and it is the behavioural signature of graded structure (Rosch, 1975).

What role does attention play in learning a concept?
Selective attention weights the stimulus dimensions that matter for a categorisation, stretching the psychological space along them and compressing it along irrelevant ones, so learning where to look is part of learning the concept (Nosofsky, 1986).

Does the brain use one system to learn categories?
No; rule-based and similarity-based category learning recruit partly separable systems, with prefrontal regions supporting explicit rules and the striatum supporting slow feedback-driven learning (Ashby et al., 1998).

How can people learn a concept from just one or two examples?
One account models concept learning as inferring a small generative program that can produce new instances, which supports human-level generalisation from a single example where standard classifiers require thousands (Lake et al., 2015).

Where in the brain are concepts represented?
Category learning draws on a distributed network; abstract, generalised category representations have been localised to the ventromedial prefrontal cortex and hippocampus, distinct from the storage of individual instances (Bowman & Zeithamova, 2018).

References