Agonistic Behavior

Abstract

Agonistic behavior, which MeSH classifies under aggression, is the whole behavioral complex — threat, attack, defense, appeasement, and retreat — that animals of the same species direct at one another during contests over resources, territory, mates, or rank. The term names a functional system larger than aggression alone, because submission and flight are as much a part of a fight as attack. This article develops the construct as ethology and behavioral ecology now treat it: a decision problem in which each contestant assesses an opponent and chooses how far to escalate. It traces the term's origin, the ritualization of threat displays, the game-theoretic turn that explained why most contests end without injury, and the assessment models that predict a fight's length and winner. A worked example derives the Hawk-Dove evolutionarily stable strategy, and three interactive demonstrations make the ideas concrete.

Keywords: agonistic behavior, ritualization, evolutionarily stable strategy, hawk-dove game, resource holding potential

Agonistic behavior is the complete repertoire of contest behavior between members of one species: not only the threats and attacks of aggression but also the appeasement signals, submissive postures, and retreats that end a fight (Scott & Fredericson, 1951). The category is deliberately wider than aggression, because the behavior that de-escalates a contest belongs to the same functional system as the behavior that starts it; a complete account of a fight has to explain the surrender as well as the strike. This is why the ethological literature prefers agonistic to aggressive when the object of study is the contest itself rather than the attack in isolation.

The modern treatment is strategic. Where early ethology catalogued the displays, the frameworks that now dominate model a contest as a decision in which each animal estimates its opponent's fighting ability and its own, weighs the value of the contested resource against the risk of injury, and chooses how far to escalate (Maynard Smith & Price, 1973; Parker, 1974). The sections below follow that logic from the naming of the behavioral complex, through the ritualized displays that carry its information, to the game theory and assessment models that explain why conflicts of interest so rarely end in death.

What Agonistic Behavior Is

Agonistic behavior is any behavior associated with conflict between two members of the same species, comprising threat, attack, and defense together with the appeasement, submission, and flight that terminate a contest (Scott & Fredericson, 1951). Scott and Fredericson introduced the term in their study of fighting in mice and rats to solve a classification problem: the behaviors that start a fight and the behaviors that end it form one adaptive system, and a vocabulary that named only aggression split that system in two. Agonistic, from the Greek agōn (contest), was chosen to hold the whole set together — the winner's attack and the loser's retreat are two outcomes of a single encounter, not two unrelated acts.

Scott's later review formalized the components into a small set of behavioral categories that recur across the vertebrates: threat, by which one animal signals readiness to fight; active fighting, the attack and defense proper; submission, by which a loser signals withdrawal from the contest; and escape (Scott, 1966). The crucial observation is that these categories are mutually inhibiting and sequenced: a submissive display turns off a winner's attack, so that in most natural contests fighting is brief and injury rare. This settling of disputes by signal rather than damage is the central puzzle the later theoretical work was built to explain — why natural selection, which might be expected to favor ever more dangerous fighters, instead so often produces contests that stop short of serious harm.

Ritualization and Threat Displays

The threats that open an agonistic encounter are not raw preludes to attack but ritualized signals, and Tinbergen's analysis of their origin is foundational. Many displays are derived activities: movements that originally served some other function — an intention movement to fly away, a redirected peck, a thermoregulatory feather-raising — have, over evolutionary time, been exaggerated, formalized, and stereotyped into conspicuous signals whose function is now communication (Tinbergen, 1952). Tinbergen called this process emancipation: the display is freed from its original causal context and comes under the control of the motivational system it now serves. The upright threat posture of a gull, derived from the preparation to strike, has become a signal read by rivals rather than an incipient blow.

Ritualization makes displays informative precisely by making them stereotyped and exaggerated. A signal that varied continuously with underlying motivation would be ambiguous; a signal driven to a conspicuous, typical form is unambiguous and hard to miss, which is what a contest needs if it is to be settled by information rather than by damage. This is why agonistic displays across species share a family resemblance — piloerection, lateral presentation, amplified size — and why they so often appear antithetical to the appeasement signals that oppose them. The displays carry the information on which the strategic decisions of the following sections depend: an animal can only decide whether to escalate if its opponent's threat tells it something reliable about what will happen if it does.

The Game Theory of Animal Conflict

The decisive theoretical advance was to treat a contest as a game. Maynard Smith and Price asked why animal conflicts are typically limited war — settled by conventional displays rather than escalated to injury — when a mutant that always fought all-out might seem to win every contest (Maynard Smith & Price, 1973). Their answer introduced the evolutionarily stable strategy (ESS): a strategy that, if adopted by most of a population, cannot be invaded by any rare alternative. The all-out fighter is not an ESS, because in a population of all-out fighters the costs of injury are so high that a more cautious strategy does better. The ESS concept, which Maynard Smith then set out formally, became the organizing idea of the whole field, and it explained the limited character of agonistic behavior as an equilibrium rather than as restraint for the good of the species (Maynard Smith, 1974).

The canonical illustration is the Hawk-Dove game. Two strategies compete for a resource of value V: a Hawk escalates and fights until injured or until its opponent retreats, while a Dove displays but retreats if its opponent escalates. When two Hawks meet, one wins V and the other pays the injury cost C, so each expects (V − C)/2; a Hawk meeting a Dove takes V uncontested; two Doves share the resource, each expecting V/2. When C exceeds V, neither pure strategy is stable: a population of Hawks is invadable by Doves and vice versa, and the ESS is a mixture in which Hawk is played with probability V/C. The equilibrium is maintained by frequency dependence — Hawks do well when rare and badly when common — and it predicts exactly the mix of escalation and restraint that agonistic behavior displays.

A mixed ESS is wasteful, and Maynard Smith and Parker showed that contests can be settled far more cheaply when the contestants differ in some way both can perceive (Maynard Smith & Parker, 1976). An asymmetry in resource holding potential or in resource value gives each animal a cue to condition its behavior on, so that the stronger or more motivated contestant escalates and the other concedes. More surprisingly, even an uncorrelated asymmetry — one that bears on neither fighting ability nor payoff, such as which animal arrived first — can settle a contest if both contestants respect it, because the convention escalate if owner, retreat if intruder is itself an ESS. This Bourgeois strategy is the game-theoretic basis for the widely observed respect for ownership in territorial contests, and it shifts the explanatory burden from the fight itself onto the asymmetries the contestants can read.

Assessment and Resource Holding Potential

The Hawk-Dove game assigns strategies at random, but real contestants differ in fighting ability and can gather information about each other during a contest. Parker introduced resource holding potential (RHP) — an individual's absolute fighting ability, set by size, weaponry, and condition — and argued that contests are settled by assessment: each animal estimates its own RHP and its opponent's, and the one that judges itself weaker retreats (Parker, 1974). Assessment turns a fight into an information-gathering process, and it predicts that contests should be decided by the RHP difference between rivals, with the weaker contestant giving up before injury is necessary.

How assessment proceeds was formalized by Enquist and Leimar's sequential assessment model, in which contestants sample information about relative RHP through repeated bouts of escalating cost, accumulating evidence until one is confident enough of its disadvantage to withdraw (Enquist & Leimar, 1983). The model makes a sharp, testable prediction: contest duration should decrease as the RHP difference between rivals increases, because well-matched opponents need more sampling to resolve a close contest, while a lopsided one is settled quickly. A complementary strand concerns the honesty of the signals assessment relies on. Clutton-Brock and Albon showed that the roaring contests of red deer stags function as honest advertisement: roaring rate is costly and correlates with fighting ability, so a stag that cannot match a rival's roaring concedes without fighting, and roaring is an honest index of RHP precisely because it is hard to fake (Clutton-Brock & Albon, 1979).

Worked Example

The Hawk-Dove game yields its evolutionarily stable strategy by a short calculation. Let the contested resource have value V = 50 and let injury cost C = 100, both in units of fitness. The expected payoffs are set by the four pairings: Hawk versus Hawk gives (V − C)/2 = (50 − 100)/2 = −25; Hawk versus Dove gives V = 50; Dove versus Hawk gives 0; and Dove versus Dove gives V/2 = 25.

Write p for the proportion of Hawks in the population. The expected payoff to playing Hawk is W(H) = p(V − C)/2 + (1 − p)V, and to playing Dove is W(D) = p·0 + (1 − p)V/2. At the ESS the two payoffs are equal, so no individual can do better by switching. Setting W(H) = W(D):

p(V − C)/2 + (1 − p)V = (1 − p)V/2.

Multiplying through by 2 and simplifying, p(V − C) + (1 − p)V = 0, which reduces to V − pC = 0, so the equilibrium proportion of Hawks is p\ = V/C = 50/100 = 0.5. Half the population plays Hawk. Substituting back, the expected payoff at equilibrium is W = (1 − 0.5)(50)/2 = 12.5 for both strategies, confirming that W(H) = 0.5(−25) + 0.5(50) = 12.5 equals W(D) = 0.5(25) = 12.5. The equilibrium is stable because the payoff to Hawk falls as p rises: when Hawks are rarer than V/C they do better than Doves and increase, when commoner they do worse and decline, so the population is driven back to p\ = V/C. The HawkDoveDemo lets the reader vary V and C and watch the ESS and its payoff update; when C ≤ V the demo shows Hawk becoming the pure ESS, because escalation no longer costs more than the prize is worth.

Discussion

The study of agonistic behavior moved in a century from description to strategy. Scott and Fredericson's contribution was conceptual — naming the contest system as a whole so that submission and flight could be studied alongside attack — and Tinbergen's was mechanistic, explaining where the displays that carry a contest come from and how selection shapes them into signals (Scott & Fredericson, 1951; Tinbergen, 1952). The game-theoretic turn then supplied the missing functional logic: the limited, conventional character of most agonistic behavior is not restraint for the species' benefit but an evolutionarily stable equilibrium, in which the cost of injury makes unlimited escalation self-defeating (Maynard Smith & Price, 1973). The assessment models completed the picture by predicting the observable course of a contest — who wins and how long it lasts — from the fighting abilities the contestants bring and the information they gather (Parker, 1974; Enquist & Leimar, 1983).

The open problems concern what contestants actually assess. The sequential assessment model assumes mutual assessment of relative RHP, but an animal might instead escalate purely on its own state until a self-set threshold of cost is reached — a self-assessment or war-of-attrition process that predicts the same broad pattern by a different mechanism. Distinguishing mutual from self-assessment empirically has proved difficult, because the two make overlapping predictions about contest duration, and the measurement choices that separate them are easy to get wrong. The honesty of agonistic signals likewise remains an active question: a signal is honest only to the extent that cheating is too costly, and the conditions that enforce that cost are not the same across species or across the different signals used within a single contest.

Current Directions

Contemporary work has reopened the assessment question with sharper tools. Chapin, Peixoto, and Briffa argued that much of the published support for mutual assessment rests on statistical artifacts — correlations between contest duration and RHP that arise even under pure self-assessment — and set out a corrected analytical framework for distinguishing the strategies, a direct methodological critique of how the field has tested its central models (Chapin et al., 2019). Their intervention bears on decades of contest data, much of which may have been read as mutual assessment on inadequate grounds.

Two further fronts widen the construct. Briffa and Lane drew attention to skill — the efficient, well-timed use of fighting ability — as a neglected component of fighting success that RHP, defined by size and weaponry alone, does not capture, so that two animals of equal RHP can differ systematically in who wins (Briffa & Lane, 2017). Reichert and Quinn, meanwhile, brought cognition explicitly into the contest, reviewing how learning, memory, and individual differences in information processing shape assessment and the decision to escalate, and arguing that the strategic models should be integrated with the mechanisms of animal cognition rather than treated as a separate level of explanation (Reichert & Quinn, 2017). Together these lines move the field from the strategy a contestant should play toward the abilities and the mind that let it play one.

Common Misconceptions

Agonistic behavior is just a technical word for aggression.

It is deliberately broader: the term was coined to include the appeasement, submission, and retreat that end a contest alongside the threat and attack that start it, because all belong to one functional system (Scott & Fredericson, 1951).

Animals limit their fighting for the good of the species.

The conventional, non-injurious character of most contests is an evolutionarily stable equilibrium among individuals, not group-level restraint; unlimited escalation is simply not a strategy that resists invasion when injury is costly (Maynard Smith & Price, 1973).

A longer fight means the contestants are more aggressive.

Under assessment models, duration tracks how closely matched the rivals are: well-matched opponents fight longer because resolving a close contest takes more information, so a long fight signals similar fighting ability, not greater ferocity (Enquist & Leimar, 1983).

Glossary

Aggression.

Behavior directed toward another with the intent to harm; the offensive component of agonistic behavior, narrower than the whole contest system.

Agonistic behavior.

The complete repertoire of within-species contest behavior, comprising threat, attack, defense, appeasement, submission, and retreat.

Appeasement.

A signal or posture by which a loser inhibits a winner's attack and de-escalates a contest.

Dominance hierarchy.

A stable ranking of individuals established and maintained through repeated agonistic encounters, which reduces the frequency of overt fighting.

Emancipation.

The evolutionary freeing of a display from its original causal context so that it comes to serve communication under a new motivational control.

Escalation.

The increase in the intensity, duration, or risk of a contest as contestants commit progressively more to winning it.

Evolutionarily stable strategy.

A strategy that, once common in a population, cannot be invaded by any rare alternative strategy.

Hawk-Dove game.

A model contest between an escalating strategy (Hawk) and a displaying-then-retreating strategy (Dove), whose equilibrium explains limited fighting.

Honest advertisement.

A signal that reliably indicates an attribute such as fighting ability because it is too costly to fake, as with the roaring of red deer stags.

Resource holding potential (RHP).

An individual's absolute fighting ability, determined by size, weaponry, and condition, which contestants assess to decide a fight.

Ritualization.

The evolutionary exaggeration and stereotyping of a movement into a conspicuous signal, typically from an activity with an unrelated original function.

Sequential assessment model.

A model in which contestants sample information about relative fighting ability through escalating bouts until one withdraws, predicting shorter fights between mismatched rivals.

Submission.

A loser's signal of withdrawal from a contest, which terminates the winner's attack.

Threat display.

A ritualized signal of readiness to fight, shown early in a contest to influence an opponent before escalation.

Key Researchers

Mark Briffa

(University of Plymouth). Professor of animal behavior whose modern analyses of contest dynamics, the role of skill in fighting ability, and the statistics of assessment strategies have reshaped how animal contests are measured and interpreted. ORCID - University of Plymouth

Tim Clutton-Brock

(University of Cambridge). Professor of animal ecology whose study of red deer established the roaring contests of stags as honest advertisement of fighting ability, a landmark in the empirical test of assessment theory. ORCID - Wikipedia - Wikidata

Geoffrey A. Parker

(University of Liverpool). Emeritus professor of evolutionary biology who introduced resource holding potential and the assessment theory of fighting, and whose ESS modeling shaped the study of animal contests. Google Scholar - Wikipedia - Wikidata

John Paul Scott

(1909–2000). Behavior geneticist at the Jackson Laboratory and Bowling Green State University who, with Emil Fredericson, coined the term agonistic behavior and established the comparative study of aggression and social development in mice and dogs. Wikipedia - Wikidata

John Maynard Smith

(1920–2004). Evolutionary biologist at the University of Sussex who, with George Price, applied game theory to animal conflict and introduced the evolutionarily stable strategy and the Hawk-Dove model. Wikipedia - Wikidata

Nikolaas Tinbergen

(1907–1988). Co-founder of ethology at the University of Oxford and Nobel laureate whose analysis of derived activities and ritualization explained the evolutionary origin of threat and other displays. Wikipedia - Wikidata

Frequently Asked Questions

What is agonistic behavior?

Agonistic behavior is the whole repertoire of contest behavior between members of the same species, including threat, attack, defense, appeasement, submission, and retreat, treated as one functional system rather than as attack alone (Scott & Fredericson, 1951).

How is agonistic behavior different from aggression?

Aggression names only the offensive component, behavior intended to harm, whereas agonistic behavior also includes the submissive and fleeing behavior that ends a contest, because the surrender and the strike belong to the same adaptive system (Scott & Fredericson, 1951).

Where does the term agonistic come from?

Scott and Fredericson introduced it in 1951, from the Greek agōn meaning contest, to name the complete behavioral system of a fight in their studies of mice and rats (Scott & Fredericson, 1951).

What is ritualization?

Ritualization is the evolutionary process by which a movement derived from some other activity is exaggerated and stereotyped into a conspicuous signal, so that a threat display becomes a reliable communication emancipated from its original function (Tinbergen, 1952).

Why don't animals usually fight to the death?

Because unlimited escalation is not an evolutionarily stable strategy when injury is costly: a population of all-out fighters can be invaded by more cautious ones, so the stable outcome is a mixture of escalation and restraint (Maynard Smith & Price, 1973).

What is the Hawk-Dove game?

It is a model contest between a Hawk strategy that escalates and a Dove strategy that displays then retreats; when injury costs more than the resource is worth, the evolutionarily stable strategy is to play Hawk with probability equal to the resource value divided by the injury cost (Maynard Smith, 1974).

What is resource holding potential?

Resource holding potential is an individual's absolute fighting ability, set by size, weaponry, and condition, which contestants assess in themselves and their rivals to decide whether to escalate or retreat (Parker, 1974).

What determines how long an animal contest lasts?

Under the sequential assessment model, contest duration tracks how closely matched the rivals are: well-matched opponents fight longer because resolving a close contest requires more information, while a mismatch is settled quickly (Enquist & Leimar, 1983).

References