Semantics

Abstract

Semantics is the study of meaning — in cognitive psychology, how meaning is represented in the mind, retrieved from memory, and combined into sentences and discourse. This article covers the four accounts that organize the field: network models that explain retrieval through spreading activation, feature and prototype accounts that make category membership graded, distributional models that derive meaning from word co-occurrence, and neurocognitive work that locates meaning in a distributed cortical system with a semantic hub. What is contested is how these accounts relate — whether they are rivals or complementary levels of one explanation, a question their growing convergence is beginning to answer. Interactive demonstrations let the reader spread activation through a semantic network, compute typicality as feature overlap, and measure the cosine similarity between word vectors.

Keywords: meaning, semantic memory, spreading activation, prototype, distributional semantics

Semantics is the branch of the study of meaning concerned with what expressions signify and how those significations are structured. For cognitive psychology the interest is not the abstract logical content of a sentence but the mental representation that lets a person understand a word, verify a fact, or grasp a paragraph. An early and influential program treated word meaning as decomposable into more primitive semantic features combined by projection rules, an attempt to specify the structure of a semantic theory precisely enough to be tested (Katz & Fodor, 1963). That decompositional impulse — meaning as structure rather than an unanalyzable whole — runs through most of the psychology of meaning that followed, even where later models replaced discrete features with graded activations or continuous vectors.

What Semantics Is

Semantics addresses meaning at several grain sizes: the meaning of a single word (lexical semantics), the meaning built by combining words (compositional semantics), and the meaning of extended text (discourse semantics). Cognitive psychology treats each as a question about representation and process — what is stored for a concept, and what operations retrieve and combine it. The enterprise is distinct from the study of reference in logic because its object is the mental content a comprehender actually uses, which is often richer, vaguer, and more graded than a logician's truth conditions.

A foundational distinction separates the semantic store from the episodic store. Semantic memory holds general, context-free knowledge — that a canary is a bird, that Paris is a city — whereas episodic memory holds events tagged with the time and place of their occurrence (Tulving, 1972). Meaning, on this view, is the content of semantic memory: a body of knowledge abstracted away from the particular occasions on which it was acquired. Most of the models below are, in effect, competing proposals for how that context-free knowledge is organized. The store itself is treated at length under semantic memory; the present article concerns the structure of the meanings it holds.

Semantic Memory as a Network

The first mechanistic model of semantic memory represented concepts as nodes in a hierarchy, with category membership and properties stored on labeled links and inherited downward to save storage — bird stores has wings, and canary inherits it rather than restating it (Collins & Quillian, 1969). The model made a sharp, testable prediction: verifying a sentence should take longer the more hierarchical links must be traversed, so A canary is an animal should be slower than A canary is a bird. Reaction-time data broadly confirmed this, giving the psychology of meaning one of its first quantitative signatures. Figure 1 shows the hierarchy and the properties stored at each level. The strict hierarchy nonetheless failed on details — A canary is a bird is verified faster than An ostrich is a bird though both cross one link — and on typicality effects the model could not accommodate.

Figure 1

The Hierarchical Network Model of Semantic Memory

The successor account kept the network but abandoned the strict hierarchy in favor of spreading activation: concepts are nodes connected by links whose length reflects semantic relatedness, and activating one node sends activation outward to its neighbors, decaying with distance (Collins & Loftus, 1975). Retrieval is the arrival of sufficient activation at a node. The model's central empirical support is semantic priming: a word is recognized faster when preceded by a related word, because activation has already spread to it. In the lexical-decision task, deciding that nurse is a word is faster after doctor than after an unrelated prime, a facilitation first demonstrated by Meyer and Schvaneveldt (1971) and since replicated in hundreds of studies. The demonstration below shows activation spreading through a small semantic network and decaying as it travels.

Semantic Features and Prototypes

A parallel tradition represents a concept not as a node in a graph but as a set of semantic features — a canary is +animate, +feathered, +sings, +small. Decompositional theories held that word meaning just is such a feature structure (Katz & Fodor, 1963), and feature representations remain central because they connect meaning to perception and action: many features are sensory or functional properties. Large empirical feature production norms, in which many participants list the properties they associate with each of hundreds of concepts, have made this representation quantitative, revealing that concepts differ in how many features they share and that shared features predict semantic similarity and confusability (McRae et al., 2005).

Features also explain why categories have graded structure. Classical theory treated a category as a set defined by necessary and sufficient conditions, so that membership is all-or-none. Against this, prototype theory showed that categories are organized around a prototype — a central tendency of the category's features — and that members vary in typicality according to how many features they share with it (Rosch, 1975). A robin is a typical bird and an ostrich an atypical one; typical members are categorized faster, named first, and learned earlier. Typicality is the graded phenomenon the strict hierarchy could not capture, and it is now a benchmark any model of meaning must reproduce. This structure is treated in its own right under prototype theory. The demonstration below computes typicality as feature overlap and shows how it predicts verification time.

Distributional Semantics

A third tradition derives meaning not from hand-specified features but from the statistics of language use, formalizing the distributional hypothesis: words that occur in similar contexts tend to have similar meanings. An early behavioral precursor measured the connotative meaning of words along dimensions extracted from ratings — evaluation, potency, and activity — using the semantic differential, showing that affective meaning occupies a low-dimensional space (Osgood, Suci & Tannenbaum, 1957). The idea is generalized by semantic differential methods still in use. Distributional models make the same dimensional move for denotative meaning: each word becomes a vector in a high-dimensional space, and similarity of meaning becomes proximity of vectors, quantified by the cosine of the angle between them.

Latent Semantic Analysis was the first such model to match human performance at scale, deriving a vector for every word from a large corpus by factoring a word-by-document co-occurrence matrix, and thereby learning enough of the meaning of English to pass a synonym test at the level of a foreign applicant (Landauer & Dumais, 1997). The framework matured into the vector-space models now standard in psychology and computational linguistics, whose cognitive interpretation — what they capture, and what they miss — has been carefully mapped (Günther, Rinaldi & Marelli, 2019). Recent work shows these spaces contain far more structured knowledge than expected: semantic projection onto an interpretable axis recovers human judgments of object features — size, danger, wealth — directly from word embeddings, closing part of the gap between distributional and feature-based accounts (Grand et al., 2022). The demonstration below places words in a two-dimensional semantic space and computes the cosine similarity between any two.

Semantics in the Brain

Where meaning is realized in the brain has become answerable with neuroimaging, and the answer is a distributed system rather than a single center. A meta-analysis of 120 functional-imaging studies identified a left-lateralized network — angular gyrus, lateral and ventral temporal cortex, and prefrontal regions — that is reliably engaged by semantic processing across tasks (Binder et al., 2009). Voxel-wise modeling of the cortex during natural listening then showed that word meaning is mapped continuously and broadly: semantically related concepts activate neighboring cortical patches, tiling much of the cortex in semantic maps that are remarkably consistent across individuals (Huth et al., 2016).

Reconciling a distributed representation with the coherence of concepts is the work of the hub-and-spoke account of controlled semantic cognition, on which modality-specific spokes (visual, auditory, motor, linguistic) are bound together by a modality-general hub in the anterior temporal lobe, while control processes shape retrieval to suit the task (Lambon Ralph et al., 2017). The computational logic of such a system was worked out earlier in the parallel-distributed-processing framework, which showed that a network learning distributed representations reproduces the developmental and breakdown patterns of human semantic knowledge (McClelland & Rogers, 2003). Meaning at the largest grain — the meaning of connected text — is built by integrating these lexical meanings with world knowledge, as in the construction-integration model, in which a network of word and proposition meanings is constructed from the text and then settles into a coherent interpretation (Kintsch, 1988).

Worked Example

To see how a vector model quantifies meaning, consider three words placed in a two-dimensional semantic space whose axes might be read informally as animacy and size. Suppose their coordinates are dog = (5, 4), cat = (4, 3), and car = (1, 5). Distributional similarity is the cosine of the angle between two vectors: the dot product divided by the product of the vectors' lengths.

For dog and cat, the dot product is (5 × 4) + (4 × 3) = 20 + 12 = 32. The lengths are √(5² + 4²) = √41 ≈ 6.40 and √(4² + 3²) = √25 = 5. The cosine is 32 ÷ (6.40 × 5) = 32 ÷ 32.0 ≈ 1.00 — the two vectors point in almost the same direction, so the words are judged highly similar.

For dog and car, the dot product is (5 × 1) + (4 × 5) = 5 + 20 = 25. The lengths are 6.40 and √(1² + 5²) = √26 ≈ 5.10. The cosine is 25 ÷ (6.40 × 5.10) = 25 ÷ 32.6 ≈ 0.77. The lower cosine places car farther from dog than cat is, matching the intuition that a car shares less meaning with a dog than a cat does. Note that cosine ignores vector length and measures only direction, which is why a rare word and a frequent one can still count as near-synonyms — the same computation the VectorSemanticsDemo above performs for any pair of words.

Discussion

The four accounts are less rivals than descriptions of the same capacity at different levels. Network models are explicit about process — how activation flows and why priming occurs — but agnostic about what a concept's content is. Feature and prototype accounts are explicit about content and graded structure but say less about the dynamics of retrieval. Distributional models are explicit about learning — how meaning could be acquired from experience without innate features — but their vectors are hard to interpret. Neurocognitive models supply the implementation, and the hub-and-spoke architecture in particular shows how a distributed feature representation can be bound into the coherent concepts the other models presuppose (Lambon Ralph et al., 2017).

The convergence is now the interesting story. Feature norms and distributional vectors, developed independently, predict overlapping portions of human similarity judgments, and semantic projection recovers feature-like dimensions from purely distributional embeddings (Grand et al., 2022), suggesting that the co-occurrence statistics a distributional model exploits are, in part, a trace of the perceptual and functional features a feature model posits. What no single account yet delivers is composition — how the meanings of words combine into the meaning of a novel sentence — which remains, as it was for Katz and Fodor (1963), the hardest problem in the psychology of meaning.

Table 1. The four accounts of meaning compared. Each isolates a different aspect of the same capacity — process, content, learning, and implementation — which is why the field increasingly treats them as complementary levels rather than competitors.

Current Directions

The most active front is the encounter between cognitive semantics and large-scale language models. Distributional vectors have grown from the static, one-vector-per-word spaces of Latent Semantic Analysis into contextual representations that assign a different vector to a word in each sentence, and psychologists are now asking how far these capture human meaning and where they diverge (Günther, Rinaldi & Marelli, 2019). The finding that interpretable knowledge can be projected out of embeddings (Grand et al., 2022) has turned these models from black boxes into objects of psychological study in their own right, used to predict typicality, feature listings, and priming.

A second direction is the fine-grained mapping of meaning onto cortex. Semantic maps recovered from natural language (Huth et al., 2016) are being combined with the hub-and-spoke framework (Lambon Ralph et al., 2017) to ask how graded, distributed cortical representations give rise to discrete concepts and how control processes select the task-relevant aspects of a meaning. The unifying question across both directions is whether a single representational format — a high-dimensional space grounded in sensory and motor experience — can subsume networks, features, and prototypes as special cases, which would make the four traditions genuinely one theory.

Common Misconceptions

A concept has a fixed definition — a list of necessary and sufficient features.

Most everyday concepts have no such definition. Category membership is graded around a prototype, and members vary in typicality according to their feature overlap with it, which is why a robin is a better bird than an ostrich (Rosch, 1975).

Semantic memory and episodic memory are the same store.

They are dissociable. Semantic memory holds context-free general knowledge; episodic memory holds events tagged with when and where they occurred, and the two can be independently impaired (Tulving, 1972).

Meaning lives in one language center in the brain.

Semantic knowledge is distributed across a broad cortical network and even tiles much of the cortex in consistent semantic maps, rather than residing in a single region (Binder et al., 2009; Huth et al., 2016).

A computer cannot learn word meaning without being told the features.

Distributional models learn a great deal of meaning from raw co-occurrence statistics alone, matching human performance on synonym tests and recovering interpretable feature dimensions from the learned space (Landauer & Dumais, 1997; Grand et al., 2022).

Glossary

Categorization.

The assignment of an object or idea to a class on the basis of its features; graded rather than all-or-none for most natural categories.

Compositional semantics.

The study of how the meanings of words combine to form the meaning of phrases and sentences.

Connotative meaning.

The affective, associative meaning of a word, distinct from its denotation; measured along dimensions such as evaluation, potency, and activity.

Construction-integration model.

Kintsch's account of discourse comprehension in which a network of word and proposition meanings is constructed from a text and then settles into a coherent interpretation.

Cosine similarity.

A measure of the similarity of two vectors given by the cosine of the angle between them; used to quantify semantic similarity in vector-space models.

Decompositional semantics.

The view that word meaning can be analyzed into a structured set of more primitive semantic features.

Distributional hypothesis.

The principle that words occurring in similar linguistic contexts tend to have similar meanings.

Latent Semantic Analysis.

An early distributional model that derives word meaning by factoring a word-by-document co-occurrence matrix from a large corpus.

Lexical decision task.

A task requiring a rapid word/non-word judgment; the standard measure of semantic priming, in which related primes speed responses.

Prototype.

The central tendency of a category's features, against which the typicality of individual members is judged.

Semantic feature.

An elementary component of a concept's meaning, often a sensory or functional property, that can be shared across concepts.

Semantic memory.

The store of general, context-free knowledge about the world and the meanings of words, distinct from memory for events.

Semantic network.

A model of meaning in which concepts are nodes connected by labeled links whose length reflects semantic relatedness.

Semantic priming.

The facilitation in recognizing a word when it is preceded by a semantically related word, taken as evidence of spreading activation.

Spreading activation.

The process by which activation applied to one node in a semantic network flows to connected nodes, decaying with distance.

Typicality.

The degree to which a category member shares features with the prototype; typical members are categorized and named faster.

Vector-space model.

A representation in which each word is a point in a high-dimensional space and semantic similarity is the proximity of points.

Key Researchers

Jeffrey R. Binder

Professor of Neurology at the Medical College of Wisconsin; his meta-analysis of 120 neuroimaging studies localized the distributed cortical network that supports semantic processing. ORCID - Google Scholar - Faculty Page

Evelina Fedorenko

(b. 1980). Associate Professor in Brain and Cognitive Sciences at MIT; investigates the language network and semantic representation, including semantic projection from word embeddings. ORCID - Wikipedia - Google Scholar - Faculty Page

Walter Kintsch

(1932-2023). Distinguished Professor Emeritus of Psychology at the University of Colorado Boulder; author of the construction-integration model of discourse comprehension and a developer of Latent Semantic Analysis. Wikipedia - Faculty Page

James L. McClelland

(b. 1948). Lucie Stern Professor at Stanford University; co-originator of the parallel-distributed-processing approach to semantic cognition. ORCID - Wikipedia - Google Scholar - Faculty Page

Charles E. Osgood

(1916-1991). Psychologist at the University of Illinois; introduced the semantic differential and the dimensional measurement of connotative meaning. Wikipedia - Faculty Page

Matthew A. Lambon Ralph

Director of the MRC Cognition and Brain Sciences Unit at the University of Cambridge; architect of the controlled-semantic-cognition, or hub-and-spoke, framework. Google Scholar - Faculty Page

Eleanor Rosch

(b. 1938). Professor Emerita of Psychology at the University of California, Berkeley; her prototype theory established the graded typicality structure of semantic categories. Wikipedia - Faculty Page

Endel Tulving

(1927-2023). Cognitive psychologist at the University of Toronto; drew the foundational distinction between episodic and semantic memory. Wikipedia

Frequently Asked Questions

What is semantics in cognitive psychology?

Semantics is the study of meaning, and in cognitive psychology it concerns how meaning is represented in the mind, retrieved from memory, and combined into the meaning of sentences and discourse. It treats meaning as mental content a comprehender uses, drawn largely from a store of general knowledge distinct from memory for events (Tulving, 1972).

What is the difference between semantics and semantic memory?

Semantics is the study of meaning in general; semantic memory is the specific memory store that holds context-free general knowledge and word meanings. Most psychological models of semantics are proposals for how the content of semantic memory is organized and retrieved (Collins & Loftus, 1975).

What is spreading activation?

Spreading activation is the process by which activating one concept in a semantic network sends activation to connected concepts, decaying with distance. It explains semantic priming, the finding that a word is recognized faster after a related word than after an unrelated one (Collins & Loftus, 1975).

Why are some category members more typical than others?

Prototype theory holds that a category is organized around a central tendency of its features, and members vary in typicality by how many features they share with it. Typical members such as a robin among birds are categorized and named faster than atypical ones (Rosch, 1975).

How can a computer learn word meaning from text alone?

Distributional models exploit the principle that words in similar contexts have similar meanings, deriving a vector for each word from co-occurrence statistics. Latent Semantic Analysis learned enough of English this way to pass a synonym test at a creditable level (Landauer & Dumais, 1997).

Where is meaning stored in the brain?

Semantic knowledge is not housed in one center but distributed across a broad cortical network, with a modality-general hub in the anterior temporal lobe binding sensory and motor features into coherent concepts (Lambon Ralph et al., 2017).

How do word meanings combine into sentence meaning?

Comprehension of connected text integrates individual word meanings with world knowledge. In the construction-integration model, a network of word and proposition meanings is built from the text and then settles into a coherent interpretation (Kintsch, 1988).

Do feature-based and vector-based models of meaning agree?

Increasingly, yes. Feature production norms and distributional vectors predict overlapping parts of human similarity judgment, and interpretable feature dimensions such as size or danger can be projected directly out of word embeddings (Grand et al., 2022).

References