Manual Communication

Abstract

Manual communication is the conveyance of language through the hands, face, and body rather than the voice, encompassing natural sign languages, fingerspelling, manually coded spoken languages, and cued speech. Its central lesson for cognitive science is that full human language is modality-independent: natural sign languages have autonomous phonology, morphology, and syntax, are acquired on the same developmental timetable as speech, and are processed by the same largely left-lateralized brain network. Signed and spoken languages diverge mainly where the channel differs, exploiting space and simultaneity for grammar and showing more pervasive iconicity than speech. Like spoken language, signed language is governed by a critical period, so late first exposure leaves lasting deficits. This article surveys the systems, their cognitive and neural organization, and the evidence that the mind builds language from whatever signal it is given.

Keywords: sign language, fingerspelling, language acquisition, iconicity, neurolinguistics

Manual communication covers every system in which the primary signal is produced by the hands and seen rather than heard. The term is an umbrella: at one end sit the natural sign languages that arise in Deaf communities, full languages with their own grammar that no one designed; at the other sit invented codes that borrow a spoken language's words or spell them letter by letter. The scientific interest is disproportionate to the size of the populations involved, because a language carried entirely in a visual-manual channel is a natural experiment on what language is. If the properties that define human language survive a complete change of medium, those properties belong to the mind, not to the mouth and ear. The evidence of the last six decades is that they largely do (Stokoe, 1960; Sandler & Lillo-Martin, 2006).

What Manual Communication Is

Manual communication is defined by its channel, not by its users or its grammar. The signal is articulated by the hands, arms, torso, and face and is received through vision (or, for deafblind communication, through touch). This single channel criterion collects together systems that are otherwise very different in kind, and much of the confusion surrounding the topic comes from treating them as one thing. A natural sign language is a language that emerged in a community of deaf signers without being planned; American Sign Language (ASL), British Sign Language (BSL), and Langue des Signes Française are mutually unintelligible languages, as different from one another as the spoken languages of the same countries. A manually coded language, by contrast, is an engineered system that maps the words and often the grammar of a spoken language onto the hands, so that Signing Exact English is English rendered sign-by-sign rather than a language of its own. Fingerspelling uses a manual alphabet to spell out the written form of words and functions within signing as a way to represent names and borrowings. Cued speech is different again: a small set of handshapes placed near the mouth disambiguates the sounds that lipreading alone leaves ambiguous, so it is an aid to perceiving a spoken language rather than a language or even a sign system.

The scientific weight of the topic rests almost entirely on the first category. When Stokoe argued in 1960 that the signs of ASL decompose into a small inventory of contrastive formational units — handshape, location, and movement — he was claiming that sign has a phonology: a level of meaningless units combined into meaningful ones, the design feature once thought to require speech (Stokoe, 1960). That claim, resisted at the time, has been confirmed and extended into a full account of signed language structure (Sandler & Lillo-Martin, 2006). The rest of this article treats manual communication primarily through natural sign language, because that is where the questions of cognitive science actually bite.

Types of Manual Communication

MeSH indexes manual communication under nonverbal communication and under correction of hearing impairment, and lists two narrower descriptors beneath it. The two are a reminder that an indexing classification tracks how a literature is filed, not how a cognitive scientist would carve the domain: lipreading is grouped here although it is a visual-oral skill rather than a manual one, and the far larger topic of natural sign language sits as a single child. The subtypes below are not mutually exclusive — a signed conversation routinely interleaves fingerspelling, and skilled perception of signing draws on the face and lips as well as the hands — and the list is how the literature is organized rather than a claim about natural joints in the mind.

A Spectrum from Gesture to Grammar

The systems of manual communication can be ordered along a single dimension: how much of a spoken language's structure they carry, and how much they constitute a structure of their own. At one extreme is co-speech gesture, the manual movement that hearing speakers produce alongside talk; it is tightly integrated with speech but is not itself a language and lacks a stable vocabulary or combinatorial grammar (Goldin-Meadow & Brentari, 2017). At the other extreme is a natural sign language, which is grammatically autonomous and owes nothing structural to the surrounding spoken language. The engineered codes fall in between: cued speech and manually coded English exist precisely to carry spoken-language structure through the eye, and they are parasitic on that spoken language by design.

Figure 1

The Manual Communication Spectrum

The distinction matters cognitively because the two ends make different demands. A manually coded system must be learned as a second encoding of a language the user already knows, and its sign-by-sign rendering of a spoken grammar is slow and cumbersome in practice, which is part of why deaf communities gravitate to natural sign languages for everyday use. A natural sign language is acquired as a first language and is processed with the fluency and automaticity of any native tongue.

The Signing Brain

If language were built on the auditory and vocal systems, damage to the classical left-hemisphere language areas should spare a language that uses neither. It does not. Lesions to left perisylvian cortex produce sign-language aphasias that mirror the spoken aphasias — fluent and non-fluent types, with comprehension and production dissociating along the same lines — while right-hemisphere damage leaves grammatical signing largely intact even as it disrupts the use of signing space for non-linguistic purposes (Campbell, MacSweeney, & Waters, 2008). Neuroimaging confirms the lesion picture: perceiving and producing sign recruits a left-lateralized fronto-temporal network closely overlapping the one that serves speech, a convergence summarized across studies in reviews of the signing brain (MacSweeney, Capek, Campbell, & Woll, 2008) and quantified in a meta-analysis of sign-language neuroimaging that localizes the shared core to left inferior frontal and posterior temporal regions (Trettenbrein, Papitto, Friederici, & Zaccarella, 2021).

The differences are as informative as the similarities. Because signed languages encode grammatical relations in space and recruit the visual system for perception, they engage parietal and some right-hemisphere regions more than spoken languages do, and the balance of shared and modality-specific recruitment is an active target of current work on the neurobiology of sign (Emmorey, 2021). The headline, however, is robust: the brain treats a language of the hands as a language first and a thing of the hands second. Language is a property of the human cerebral cortex's organization, not of any one sensory surface, and sign-language aphasia is the clinical proof.

Acquisition and the Critical Period

Deaf children born to signing parents acquire sign on the same schedule, and through the same stages, as hearing children acquire speech. They even babble manually: infants exposed to sign produce rhythmic, reduced-form hand activity that is structurally distinct from ordinary motor activity and parallels vocal babbling in timing and function, evidence that the babbling stage reflects a language capacity seeking a channel rather than a quirk of the vocal tract (Petitto & Marentette, 1991). The milestones of language development — first signs, first combinations, the grammatical explosion — track the spoken timetable closely.

The majority of deaf children, however, are born to hearing parents and meet accessible language late, which turns sign acquisition into the clearest available test of the critical-period hypothesis. The logic is powerful because, unlike spoken-language learners, these learners vary widely in the age of their first language exposure, disentangling age from years of practice. The results are consistent: eventual proficiency in sign falls as the age of first exposure rises, even decades later and even controlling for total experience, so a late first language leaves a permanent grammatical signature (Newport, 1990; Mayberry & Eichen, 1991). The demonstration below reproduces the shape of that relationship.

Most striking is what the emergence of a new sign language reveals about the child's contribution. When deaf children in Nicaragua were first brought together in the 1970s and 1980s, each arrived with only the idiosyncratic homesign — the structured gesture system a deaf child invents at home without accessible language — that they had developed in isolation. From that pool of inconsistent input, successive cohorts of young signers regularized a grammar their adult models did not have, inventing spatial agreement and other devices generation by generation (Senghas & Coppola, 2001). Language acquisition is not merely the absorption of a code from the environment; the young learner actively imposes structure, and a critical period is the window in which that structuring power is at its height.

Iconicity and the Form of Signs

Spoken words are overwhelmingly arbitrary: nothing about the sound of dog resembles a dog. Signs are arbitrary too, but far less uniformly so — the visual-manual channel can depict as well as symbolize, and many signs preserve a motivated resemblance to their meaning. For decades iconicity was treated as an embarrassment to be explained away, on the assumption that a real language must be arbitrary. The modern view reverses this: iconicity is a general design resource of language, more available to the eye than to the ear, and it shapes how signs are learned, processed, and remembered rather than disqualifying them as linguistic (Perniss, Thompson, & Vigliocco, 2010). Large-scale lexical databases now rate thousands of signs for iconicity alongside their frequency and phonological properties, making the variable measurable rather than anecdotal (Caselli, Sehyr, Cohen-Goldberg, & Emmorey, 2017).

Iconicity is also partial and conventionalized, which is why a non-signer cannot simply read a signed conversation. A sign may be transparent once its meaning is known yet unguessable in advance, and the same concept is signed differently in different sign languages. The demonstration below lets the degree of iconicity vary and shows how it tracks the chance that a naive viewer recovers a sign's meaning.

Rate, Simultaneity, and Working Memory

A single sign takes roughly twice as long to produce as a single spoken word, which by a naive accounting should make signing half as fast. It does not. Signed and spoken narrations of the same material convey propositions at approximately the same rate, because sign offsets its slower articulation with greater simultaneity: where speech must string morphemes in sequence, sign can layer them, inflecting a verb for its subject, object, aspect, and manner in one spatially modulated movement (Klima & Bellugi, 1979). The channel thus reshapes the grammar — spoken languages lean on sequence, signed languages exploit the extra spatial and simultaneous bandwidth of the hands and face.

The same modality difference reaches into memory. Immediate serial recall of signs shows the same capacity-limited, order-sensitive profile as recall of words, implicating a sign-based rehearsal mechanism analogous to the speech-based phonological loop of verbal working memory. The architecture of short-term verbal memory, in other words, is modality-general at the level of its organization while being tuned to the specific signal it holds.

Worked Example

Consider the propositional-rate finding in concrete terms, using representative figures in the range reported by Klima and Bellugi (1979). Suppose a short narrative contains 12 propositions. Rendered in English speech it runs to about 50 words delivered at roughly 4.7 words per second; rendered in ASL it runs to about 24 signs delivered at roughly 2.3 signs per second.

Speech duration is 50 ÷ 4.7 ≈ 10.6 s, giving 12 ÷ 10.6 ≈ 1.13 propositions per second. Signing duration is 24 ÷ 2.3 ≈ 10.4 s, giving 12 ÷ 10.4 ≈ 1.15 propositions per second. The two rates are within two percent of each other even though the signer produces fewer than half as many lexical units per second as the speaker.

The arithmetic exposes the mechanism. Speech needs about 4.2 words per proposition (50 ÷ 12) but sign needs only about 2.0 signs per proposition (24 ÷ 12), because each sign carries, on average, roughly twice the grammatical load through simultaneous inflection. Halving the units per proposition exactly compensates for halving the units per second, so the information rate converges. Modality changes the packaging of a proposition without changing how fast the mind delivers propositions.

Discussion

The study of manual communication has settled one large question and opened several smaller ones. The settled question is foundational: human language does not depend on speech. A system carried entirely in the visual-manual channel can satisfy every criterion once thought diagnostic of language — a combinatorial phonology, a productive morphology and syntax, native acquisition on the standard timetable, processing by the standard left-hemisphere network, and vulnerability to the standard critical period. Any theory of language that is really a theory of speech is thereby refuted, and the competence that underlies language must be specified abstractly enough to be neutral between the mouth and the hand (Sandler & Lillo-Martin, 2006).

The open questions concern the imprint of the channel. Modality is not inert: it pushes signed grammars toward spatial and simultaneous encoding, admits more iconicity, and recruits a partly distinct neural periphery around a shared core. Disentangling what is universal to language from what each channel contributes is the productive tension in the field, and it requires exactly the comparison that sign makes possible. The practical stakes are high as well. Because the critical period is real and most deaf children are born to hearing parents, the timing of access to an accessible first language is a developmental variable with lasting cognitive consequences, which makes early exposure to a natural sign language a matter of cognitive development and not only of culture.

Current Directions

Three strands are especially active. The first is quantitative and lexical: large, rated databases such as ASL-LEX have moved sign-language psycholinguistics from small hand-built stimulus sets to properties measured across thousands of signs, letting researchers model how frequency, iconicity, and phonological neighborhood density jointly shape recognition (Caselli et al., 2017). The second is neurobiological: meta-analytic and high-resolution imaging is refining exactly which regions are truly shared with speech and which are modality-specific, replacing the broad claim of overlap with a region-by-region accounting (Trettenbrein et al., 2021; Emmorey, 2021). The third reframes the whole domain: rather than asking whether sign is more like gesture or more like speech, current theory treats gesture, sign, and speech as a single integrated system of human communication, with the hands contributing both gradient, imagistic gesture and discrete, categorical sign, often at once (Goldin-Meadow & Brentari, 2017). Each strand depends on the comparison of modalities that manual communication uniquely affords.

Common Misconceptions

Sign language is a visual form of the surrounding spoken language.

A natural sign language has its own grammar and is not English, or any spoken language, on the hands. American Sign Language is historically related to French Sign Language and is mutually unintelligible with British Sign Language, even though the United States and Britain share a spoken language (Sandler & Lillo-Martin, 2006). The belief persists because the engineered manually coded systems, which are a spoken language rendered sign-by-sign, are often mistaken for the natural language used by Deaf communities.

Sign language is universal.

There is no single worldwide sign language; dozens of unrelated sign languages exist, and new ones arise whenever a community of deaf signers forms, as the independent emergence of Nicaraguan Sign Language showed (Senghas & Coppola, 2001). The impression of universality comes from iconicity: because some signs resemble their meaning, outsiders overestimate how much of a signed conversation is transparent.

Signs are just pictures or pantomime, so sign is not a real language.

Signs decompose into a small inventory of contrastive formational units — handshape, location, movement — combined by rule, exactly the meaningless-units-into-meaningful-units design of a phonology (Stokoe, 1960). Iconicity is a resource the visual channel makes available, not a substitute for grammatical structure (Perniss et al., 2010); a motivated form can still be a fully conventional, rule-governed linguistic sign.

Glossary

Critical period.

A maturational window during which first-language acquisition proceeds most completely, after which eventual proficiency declines with the age of first exposure.

Cued speech.

A system of handshapes placed near the mouth that disambiguates the phonemes left ambiguous by lipreading; an aid to perceiving a spoken language, not a language itself.

Fingerspelling.

Representing the written letters of a word with a manual alphabet, used within signing for names, technical terms, and borrowings.

Handshape.

The configuration of the fingers and thumb in a sign; one of the contrastive formational parameters of signed phonology.

Homesign.

The structured gesture system a deaf child without accessible language input invents to communicate with hearing family; more organized than gesture, less than a shared language.

Iconicity.

A motivated resemblance between the form of a sign or word and its meaning; a general resource of language more pervasive in the visual-manual channel.

Language.

A combinatorial communicative system with a phonology, morphology, and syntax, acquired naturally in childhood; realizable in either the vocal or the manual channel.

Manual alphabet.

The set of handshapes, one per written letter, used in fingerspelling; distinct from the signs of the language itself.

Manually coded language.

An engineered system that maps the vocabulary and grammar of a spoken language onto signs, such as Signing Exact English; parasitic on that spoken language rather than autonomous.

Modality.

The sensory-motor channel in which a language is produced and perceived — auditory-vocal for speech, visual-manual for sign.

Natural sign language.

A language that emerged in a Deaf community without planning, with full grammatical structure; for example American, British, or Nicaraguan Sign Language.

Phonology (of sign).

The level at which signs are built from a small set of meaningless contrastive units — handshape, location, and movement — combined by rule.

Sign language.

A natural language of the visual-manual channel; the central and most studied form of manual communication.

Signing space.

The region in front of the signer in which locations are assigned to referents and used for grammatical agreement and spatial description.

Simultaneity.

The layering of several morphemes or grammatical features within one signed movement, the visual-manual counterpart to speech's reliance on sequence.

Key Researchers

Ursula Bellugi

(1931-2022). Formerly of the Salk Institute for Biological Studies; with Edward Klima she established the cognitive and neural study of sign language, demonstrating its grammatical structure and its propositional-rate equivalence with speech. Wikipedia - Wikidata

Karen Emmorey

(b. 1958). Distinguished Professor at San Diego State University; a leading figure in the cognitive neuroscience of sign language and the construction of large-scale signed-language databases. ORCID - Wikipedia - Google Scholar - Faculty Page

Susan Goldin-Meadow

. Beardsley Ruml Distinguished Service Professor at the University of Chicago; her work on homesign and on gesture reframed the relationship between the hands, sign, and language. ORCID - Wikipedia - Faculty Page

Mairéad MacSweeney

. Deputy Director of the Institute of Cognitive Neuroscience at University College London; she uses neuroimaging to map the signing brain and its overlap with the spoken-language network. ORCID - Faculty Page

Rachel Mayberry

. Professor of Linguistics at the University of California, San Diego; her studies of age of acquisition established the lasting cost of late first-language exposure in sign. Faculty Page - Wikipedia

William C. Stokoe

(1919-2000). Linguist at Gallaudet University who demonstrated in 1960 that American Sign Language has a phonological structure, founding the modern linguistic study of sign. Wikipedia - Wikidata

Frequently Asked Questions

Is sign language the same everywhere in the world?

No. There is no universal sign language; many unrelated sign languages exist, and new ones can emerge whenever a community of deaf signers forms, as with Nicaraguan Sign Language (Senghas & Coppola, 2001). American and British Sign Language are mutually unintelligible despite a shared spoken language.

Is a natural sign language a real language or a code for a spoken one?

It is a full language in its own right, with an autonomous phonology, morphology, and syntax, and it is not a visual rendering of any spoken language (Sandler & Lillo-Martin, 2006). Systems that do encode a spoken language on the hands, such as Signing Exact English, are a separate, engineered category.

Does the brain process sign language the same way it processes speech?

Largely yes: signing recruits the same left-lateralized fronto-temporal network as speech, and left-hemisphere damage produces sign-language aphasias parallel to the spoken ones (Campbell, MacSweeney, & Waters, 2008). The visual-spatial demands of sign add some parietal and right-hemisphere involvement (Emmorey, 2021).

Why do signs often look like what they mean?

The visual-manual channel can depict as well as symbolize, so signs show more iconicity than spoken words, but iconicity is a general design resource of language rather than a sign that sign is not linguistic (Perniss, Thompson, & Vigliocco, 2010). Many signs are still arbitrary and must be learned.

Does signing take longer than speaking to say the same thing?

No. A single sign takes about twice as long as a word to produce, but signed and spoken narrations convey propositions at about the same rate, because sign layers morphemes simultaneously where speech must sequence them (Klima & Bellugi, 1979).

Does the age at which a child first learns sign matter?

Yes, decisively. Eventual proficiency declines as the age of first exposure rises, even controlling for years of use, which is strong evidence for a critical period for language that is independent of modality (Newport, 1990; Mayberry & Eichen, 1991).

Do deaf infants babble?

Infants exposed to sign produce manual babbling, a rhythmic reduced-form hand activity distinct from ordinary movement, paralleling vocal babbling, which suggests the babbling stage reflects an emerging language capacity seeking any available channel (Petitto & Marentette, 1991).

Is fingerspelling a separate language from sign language?

No. Fingerspelling uses a manual alphabet to spell written words and operates within a sign language, typically for names, technical terms, and borrowings; it is a component of signing, not a language on its own (Stokoe, 1960).

References