Exploratory Behavior

Abstract

Exploratory behavior is a type of behavior: the investigation of novel objects, places, and information that no immediate biological need compels, which cognitive psychology studies as the behavioral expression of curiosity and the drive to reduce uncertainty. This article sets out that account: how novelty and complexity arouse curiosity along an inverted-U, how latent learning builds cognitive maps during apparently aimless wandering, how the brain arbitrates the explore–exploit trade-off through directed and random strategies, and how information itself acquires a value that guides attention and search. The same machinery that drives a rat through a maze drives a scientist through a problem—a link to motivation that MeSH itself records. Three demonstrations make the novelty-preference curve, the explore–exploit trade-off, and information-gap curiosity concrete.

Keywords: exploratory behavior, curiosity, explore–exploit trade-off, information-seeking

Exploratory behavior is the heading the National Library of Medicine uses for the activity of investigating a novel environment or stimulus, and it names one of the deepest puzzles in the psychology of motivation: why an animal with no hunger to satisfy and no threat to escape will nonetheless spend energy and accept risk to inspect what it does not yet understand. Cognitive psychology treats this not as idle restlessness but as the visible output of systems built to gather information—to learn the layout of a world before that knowledge is needed, and to resolve uncertainty for its own sake. The central question of the field is how a nervous system assigns value to the unknown, and how it balances the pull of what is already known to pay against the promise of what might be learned by looking further.

What Exploratory Behavior Is

Within the Medical Subject Headings, Exploratory Behavior sits beneath the broad parent heading behavior as the investigation of a novel environment or stimulus; MeSH also files it under motivation, and that dual placement captures the phenomenon exactly, because exploration is a behavior whose cause is motivational rather than reflexive. Its defining feature is that it is not drawn out by any consummatory need: the animal is not eating, mating, or fleeing, yet it moves, sniffs, manipulates, and looks. This is what made exploration a theoretical problem for early drive-reduction accounts of motivation, which held that behavior exists only to reduce a bodily deficit. Exploration reduces no such deficit, and its persistence forced psychology to admit a motivation to acquire information as a primary one in its own right (Berlyne, 1966).

The cognitive account treats exploratory behavior as the output of systems that evolved to gather and organize knowledge in advance of its use. On this view the forager inspecting a new patch, the infant mouthing an unfamiliar object, and the adult reading past the point of practical need are running the same program: sampling the environment to build an internal model that will pay off later (Hills et al., 2015). The sections that follow trace that program from its motivational root upward—first the curiosity that novelty and complexity arouse, then the latent learning that silent exploration produces, then the formal trade-off that governs when to explore at all, and finally the value the brain places on information itself.

Curiosity: Novelty, Complexity, and the Inverted-U

The modern psychology of exploration begins with Daniel Berlyne’s analysis of what makes a stimulus interesting. He argued that a family of collative properties—novelty, complexity, surprise, incongruity, and ambiguity—determine a stimulus’s arousal potential, and that these properties, by requiring the organism to compare and collate what it perceives against what it expects, drive the curiosity that motivates exploration (Berlyne, 1950). Crucially, the relationship between arousal potential and the reward value of a stimulus is not monotonic but an inverted-U: stimuli of intermediate novelty or complexity are preferred and explored most, while those that are too simple are dull and those that are too novel or complex are confusing or aversive (Berlyne, 1966).

A complementary account locates curiosity in a gap. Loewenstein’s information-gap theory holds that curiosity arises when attention fixes on a discrepancy between what one knows and what one wants to know, and that the resulting feeling of deprivation motivates behavior to close the gap—which explains why curiosity is sharpest not when one knows nothing but when one knows just enough to feel the absence of the rest (Loewenstein, 1994). The information-gap view and Berlyne’s arousal view both predict that curiosity peaks at intermediate levels of the unknown, and a recent rational analysis unifies them by showing that a learner maximizing expected information gain should indeed prefer stimuli of moderate novelty and complexity (Dubey & Griffiths, 2020). The demonstration below moves a stimulus along the dimensions of novelty and complexity and traces how preference rises and then falls—the signature inverted-U.

The inverted-U of curiosity

Novelty and complexity set a stimulus’s arousal potential. Preference is not a straight line in arousal potential but an inverted-U: it rises as a stimulus becomes more interesting, then falls as it becomes confusing or aversive. Move the sliders and watch where the stimulus sits on the curve.

Cognitive Maps and Latent Learning

If curiosity explains why animals explore, Edward Tolman’s work explains what exploration accomplishes even when it earns no reward. Against the prevailing view that learning requires reinforcement, Tolman argued that animals exploring a maze acquire a cognitive map—an internal representation of the spatial layout and the relationships among places—rather than a mere chain of stimulus–response connections (Tolman, 1948). The decisive evidence was latent learning: rats allowed to wander an unrewarded maze for several days showed little change in their errors, but when food was finally introduced at the goal, their performance dropped almost immediately to the level of rats rewarded from the start, as though the knowledge gathered during free exploration had been there all along, waiting for a reason to be used.

Latent learning is central to the cognitive theory of exploration because it dissociates learning from performance and from reward. The exploring animal is building a model of its world during behavior that a reinforcement-only account must call pointless, and that model becomes behavior the moment a goal makes it relevant. Exploration, in this light, is an investment: knowledge is acquired now, at some cost and no immediate benefit, against the expectation that the environment will later pose a problem the map can solve. Figure 1 shows the classic latent-learning result that anchored this argument.

The Explore–Exploit Trade-off

Every agent acting in an uncertain world faces a recurring decision: take the option that currently looks best, or try an alternative whose value is less certain but might prove better. This is the explore–exploit trade-off, and it has no free solution—every choice to explore forgoes known reward, and every choice to exploit forgoes possible information. Cognitive neuroscience has shown that the human brain treats this as a genuine control problem: functional imaging reveals that exploratory choices recruit frontopolar cortex and intraparietal sulcus, distinct from the regions tracking the value of exploitation, implying a dedicated mechanism that can switch behavior into an exploratory mode (Daw et al., 2006). A broader analysis frames the trade-off as a problem of regulating the balance between exploiting a current goal and exploring alternatives, and ties it to the neuromodulator norepinephrine and to anterior cingulate and prefrontal control (Cohen, McClure, & Yu, 2007).

The decisive behavioral advance was to show that humans do not explore in one way but two. Directed exploration adds an uncertainty bonus to an option’s value, systematically favoring the option about which one knows least; random exploration instead injects noise into choice, sometimes selecting the lower-valued option simply because decisions are not deterministic. A task that manipulates information and horizon independently shows that people use both, and that they are dissociable—directed exploration scales with how much there is to learn, while random exploration scales with decision noise (Wilson et al., 2014). The demonstration below is a two-armed bandit in which an uncertainty bonus and a choice-temperature can be tuned separately, so that each is seen to push behavior toward the uncertain option in its own characteristic way.

Directed and random exploration

Arm A pays 0.60 and is well known; arm B pays 0.50 but is uncertain. The uncertainty bonus is directed exploration—it raises the value of the option you know least. The temperature is random exploration—it flattens choice toward chance. Each drives you to B differently.

Information as a Reward: Seeking and Attention

The final element of the account is the claim that information is not merely a means to reward but carries value of its own—an epistemic value, the worth of an action in terms of the information it is expected to yield, as distinct from the instrumental value of the reward it secures. On this view the brain computes the expected informational gain of a potential observation and treats uncertainty reduction as intrinsically rewarding, which lets a unified currency govern where the eyes move, what the hand reaches for, and what a person chooses to find out (Gottlieb et al., 2013). This reframes attention and gaze as sampling decisions: an organism points its sensors at whatever promises the most information, so that curiosity and selective attention become two faces of the same information-seeking system.

There is direct neural and behavioral support for information acting like reward. States of high curiosity enhance memory not only for the specific thing one is curious about but for incidental material encountered in that state, an effect accompanied by activity in the dopaminergic midbrain and hippocampus—the same circuit that supports reward-motivated learning (Gruber, Gelman, & Ranganath, 2014). Reviewing the field, Kidd and Hayden argue that curiosity is best understood as a drive to obtain information whose satisfaction is rewarding, and that its characteristic inverted-U over novelty reflects an underlying computation of expected learning (Kidd & Hayden, 2015). The demonstration below shows how the value of an observation depends on how much uncertainty it stands to resolve, peaking when one is maximally unsure.

Information as the value of uncertainty reduction

Suppose you are about to observe a yes/no fact you currently believe is true with probability p. The information the observation yields is the entropy of that belief: it is greatest when you are most unsure (p = 0.5) and vanishes when you are already certain. This is why curiosity peaks at intermediate knowledge, not at ignorance.

Worked Example

Consider how directed and random exploration each change a choice between two options, using the rule behind the second demonstration. Let two bandit arms have estimated values Q_A = 0.6 and Q_B = 0.5, and let choice be governed by the softmax rule P(A) = e^Q_A/τ / (e^Q_A/τ + e^Q_B/τ), where the temperature τ controls random exploration. With a low temperature τ = 0.1 the agent mostly exploits: the exponents are 6.0 and 5.0, giving P(A) = e⁶ / (e⁶ + e⁵) = 403.43 / (403.43 + 148.41) = 0.731, so arm A, the higher-valued option, is chosen about three-quarters of the time. Raise the temperature to τ = 0.5 and the exponents fall to 1.2 and 1.0, giving P(A) = 3.320 / (3.320 + 2.718) = 0.550—behavior now close to a coin flip, the hallmark of random exploration driven by decision noise.

Directed exploration works differently. Suppose arm B has been sampled far less, so its value is more uncertain: let the uncertainty be σ_A = 0.1 and σ_B = 0.4, and let the agent add an uncertainty bonus γσ to each value with γ = 0.5. The bonus-adjusted values become Q_A′ = 0.6 + 0.5 × 0.1 = 0.65 and Q_B′ = 0.5 + 0.5 × 0.4 = 0.70, so the less valued but more uncertain arm B now has the higher adjusted value. Applying the same low temperature τ = 0.1, the exponents are 6.5 and 7.0 and P(B) = e^7.0 / (e^6.5 + e^7.0) = 1096.63 / (665.14 + 1096.63) = 0.622: the agent now prefers B, and does so deliberately, because it is the option worth learning about. The lesson is quantitative: random exploration flattens the choice distribution toward chance, while directed exploration shifts the preferred option itself toward whatever is most uncertain—two mechanically distinct routes to the same adaptive end of trying something new.

Discussion

Exploratory behavior is a clarifying case for cognitive psychology because it reveals a motivation that classical drive theory could not accommodate and that modern decision theory makes precise. Each piece of the account is a piece of normal cognition: the arousal that collative properties generate; the latent model-building that turns wandering into knowledge; the formal arbitration between known and unknown payoffs; the valuation of information that steers attention. Together they portray exploration not as the absence of goal-directed behavior but as a goal-directed behavior whose goal is knowledge itself.

Table 2

Cognitive mechanisms in exploratory behavior and what each explains.

Note. Each mechanism is drawn from a section above; together they portray exploratory behavior as the joint product of curiosity, latent model-building, formal arbitration of the explore–exploit trade-off, and the valuation of information.

The account has open tensions. The two theories of curiosity—arousal-based and gap-based—are not fully reconciled, and whether the inverted-U reflects a single computation of expected learning or several interacting processes remains debated. The relationship between the directed and random strategies and the latent model-building of Tolman’s tradition is also not settled: it is unclear whether a single model-based system produces both the map and the uncertainty estimates that drive directed exploration, or whether these are separate mechanisms that happen to cooperate.

Current Directions

The contemporary field is increasingly computational, treating exploration as approximate solutions to a problem that is formally intractable. Decomposing human choice on bandit tasks, Gershman has shown that people combine directed and random exploration and that each can be identified with a specific term in a Bayesian decision rule—an uncertainty bonus for the directed component and value-scaled noise for the random one—so that the two strategies Wilson and colleagues isolated behaviorally fall out of a single normative analysis (Gershman, 2018). A complementary line asks how exploration scales to the vast option spaces of real life, where no agent could sample every possibility: here generalization does the work, as people use function learning to infer the value of untried options from similar tried ones, exploring on the basis of a learned value landscape rather than option-by-option uncertainty (Wu et al., 2018).

These threads are drawn together in an emerging consensus that exploration rests on an algorithmic architecture—a small set of interacting strategies, some directed and model-based, some random and cheap, that the brain selects among according to the structure of the problem and the resources available (Schulz & Gershman, 2019). At the same time the psychology of curiosity has been placed on a rational footing: modeling curiosity as the pursuit of expected information gain reconciles the long-competing novelty and complexity accounts and predicts the inverted-U directly from what an optimal learner should want to sample (Dubey & Griffiths, 2020). The direction of travel is toward a single framework in which curiosity, attention, and exploratory choice are expressions of one underlying computation over the value of information.

Cognitive Implications

Read as cognition, exploratory behavior illustrates three themes that reach well beyond the maze. First, it establishes information as a primary motivator: organisms will work to reduce uncertainty with no extrinsic payoff, so any complete theory of motivation must grant the acquisition of knowledge a value of its own alongside food, safety, and reward (Berlyne, 1966; Gottlieb et al., 2013). Second, it shows that learning and performance are distinct: the latent-learning result proves that much of what exploration accomplishes is invisible until a goal calls on it, a dissociation that reshaped how psychology thinks about the relationship between knowing and doing (Tolman, 1948). Third, it demonstrates that adaptive behavior requires deliberately suboptimal choices in the short run: an agent that only ever exploited its current best estimate would never discover a better option, so exploration is the mechanism by which a system trades present reward for future knowledge (Cohen, McClure, & Yu, 2007; Gershman, 2018). In each, exploration is a window onto how a finite mind copes intelligently with an uncertain and only partly known world.

Common Misconceptions

Exploration is just random, aimless activity.

Much exploration is highly structured. People use directed exploration that deliberately targets the option they know least, and even the random component is value-scaled noise rather than pure chance; both serve the adaptive end of gathering information (Wilson et al., 2014; Gershman, 2018).

The more novel or complex a thing is, the more curiosity it provokes.

Curiosity follows an inverted-U, not a straight line. Interest rises with novelty and complexity up to a point and then falls, so the most explored stimuli are of intermediate difficulty—too little is boring, too much is confusing or aversive (Berlyne, 1966; Kidd & Hayden, 2015).

Nothing is learned during exploration unless it is rewarded.

Latent learning refutes this. Animals exploring an unrewarded maze build a cognitive map whose existence is revealed only when reward is later introduced and performance improves at once—knowledge was acquired without reinforcement and lay dormant until it became useful (Tolman, 1948).

Glossary

Arousal potential.

In Berlyne’s theory, the capacity of a stimulus to raise arousal, determined by collative properties such as novelty, complexity, and surprise; its relation to reward value follows an inverted-U.

Cognitive map.

An internal representation of the spatial layout and relational structure of an environment, built during exploration and used to guide flexible, goal-directed navigation.

Collative properties.

Stimulus features—novelty, complexity, surprise, incongruity, ambiguity—that require comparison of what is perceived against what is expected, and that drive curiosity and exploration.

Curiosity.

The motivational state that drives information-seeking; a drive to resolve uncertainty whose satisfaction is rewarding and which peaks at intermediate levels of the unknown.

Directed exploration.

A strategy that adds an uncertainty bonus to an option’s value, systematically favoring the least-known option; scales with how much there is to learn.

Epistemic value.

The worth of an action in terms of the information it is expected to yield, as distinct from its instrumental value in obtaining extrinsic reward; what makes uncertainty reduction intrinsically motivating.

Exploration–exploitation trade-off.

The decision between taking the option that currently appears best (exploitation) and sampling an uncertain alternative that may prove better (exploration); it has no cost-free solution.

Exploratory behavior.

The investigation of novel objects, places, and information not compelled by an immediate biological need; the behavioral expression of curiosity and the drive to reduce uncertainty.

Information gap.

In Loewenstein’s theory, the perceived discrepancy between what one knows and what one wants to know; attention to the gap produces the feeling of deprivation that motivates curiosity.

Information-seeking.

Behavior directed at obtaining observations that reduce uncertainty, governed by the expected informational gain of a potential sample; it unifies gaze, attention, and curiosity.

Latent learning.

Learning that occurs without reinforcement and remains unexpressed in performance until a goal makes it useful; the decisive evidence for cognitive maps.

Novelty.

The degree to which a stimulus departs from what has been experienced; a principal collative property and a central driver of exploration, most attractive at intermediate levels.

Optimal foraging.

The analysis of search as a resource-maximizing problem; its logic of when to leave a depleting patch is formally analogous to the explore–exploit trade-off in mind and society.

Random exploration.

A strategy that injects noise into choice so that lower-valued options are sometimes selected; scales with decision noise (temperature) rather than with information.

Softmax choice rule.

A decision rule that converts option values into choice probabilities through an exponential weighted by a temperature parameter; higher temperature yields more random, exploratory choice.

Uncertainty.

The imprecision of an estimate of an option’s value; its reduction is the goal of exploration, and directed exploration treats it as a bonus to be sought.

Key Researchers

Daniel E. Berlyne

. Professor of Psychology at the University of Toronto (1924–1976); the founder of the experimental psychology of curiosity and exploration, he introduced the collative properties and the inverted-U relation between arousal potential and reward that still organize the field. Wikipedia

Samuel J. Gershman

. Professor of Psychology at Harvard University; his computational work decomposes human exploration into directed and random components derived from a single Bayesian decision rule, placing the explore–exploit trade-off on a normative footing. ORCID · Google Scholar · Faculty page

Jacqueline Gottlieb

. Professor of Neuroscience at Columbia University’s Zuckerman Institute; she established the view of information-seeking as a valuation process, showing how the brain assigns value to uncertainty-reducing observations and uses it to steer attention and gaze. ORCID · Wikipedia · Google Scholar · Faculty page

Celeste Kidd

. Professor of Psychology at the University of California, Berkeley; her research on curiosity, attention, and belief formation develops the account of curiosity as a rational drive to obtain information whose deployment follows the expected value of learning. Wikipedia · Google Scholar · Faculty page

Edward C. Tolman

. Professor of Psychology at the University of California, Berkeley (1886–1959); his latent-learning experiments and the concept of the cognitive map showed that exploration builds internal representations of the world without reward, founding the cognitive tradition in the study of learning. Wikipedia

Frequently Asked Questions

What is exploratory behavior in cognitive psychology?

It is the investigation of novel objects, places, and information that no immediate biological need compels—the sense in which the National Library of Medicine uses the term—studied as the behavioral output of curiosity and an active drive to reduce uncertainty rather than as aimless activity (Berlyne, 1966; Hills et al., 2015).

Why do animals and people explore when there is no reward for it?

Because information itself is valuable. Exploration builds a model of the environment that pays off later, and the brain treats uncertainty reduction as intrinsically rewarding, so curiosity can motivate behavior with no extrinsic payoff (Gottlieb et al., 2013; Tolman, 1948).

What is the inverted-U relationship in curiosity?

Berlyne showed that a stimulus’s attractiveness rises with its novelty, complexity, and surprise up to a point and then declines, so exploration is strongest for stimuli of intermediate novelty—too little is dull and too much is aversive (Berlyne, 1950; Berlyne, 1966).

What is a cognitive map?

It is an internal representation of the layout and relationships of an environment, built during exploration. Tolman inferred its existence from latent learning: rats that explored a maze without reward navigated efficiently as soon as reward was introduced, as though they already knew the way (Tolman, 1948).

What is the explore–exploit trade-off?

It is the choice between taking the option that currently looks best and sampling an uncertain alternative that might be better. Exploiting forgoes possible information; exploring forgoes known reward; and the brain appears to have dedicated machinery for switching between them (Daw et al., 2006; Cohen, McClure, & Yu, 2007).

How do directed and random exploration differ?

Directed exploration adds an uncertainty bonus to an option’s value, favoring the option one knows least; random exploration injects noise into choice so that lower-valued options are sometimes selected. They are dissociable, and people use both (Wilson et al., 2014; Gershman, 2018).

Does curiosity help memory?

Yes. States of high curiosity enhance learning not only of the material one is curious about but of incidental information encountered at the same time, an effect linked to activity in the dopaminergic midbrain and hippocampus (Gruber, Gelman, & Ranganath, 2014; Kidd & Hayden, 2015).

How does exploration work when there are too many options to try them all?

Through generalization. People learn a value function over the space of options and use it to infer the worth of untried choices from similar tried ones, exploring on the basis of a learned value landscape rather than evaluating every possibility (Wu et al., 2018; Schulz & Gershman, 2019).

References