Operant Conditioning

Abstract

Operant conditioning is a form of psychological conditioning in which behaviour changes because of the consequences it produces: responses followed by reinforcement become more frequent, and responses followed by punishment become less so. The paradigm grew from Thorndike's experiments on animal learning and Skinner's experimental analysis of behaviour, and its laboratory core includes schedules of reinforcement, the matching law for choice, and biological limits on what a species learns easily. Later work split instrumental behaviour into goal-directed actions and habits, a distinction well supported in rats but contested in humans, and tied reinforcement learning to dopamine prediction-error signals. Applications run from token economies on psychiatric wards to early intensive behavioural intervention for autism, where the pooled evidence is of low quality. Several founding theories have since been revised or remain contested.

Keywords: operant conditioning, reinforcement, punishment, schedules of reinforcement, law of effect

Operant conditioning is learning in which the consequence of a response alters how often that response occurs again. Its experimental history opens with Thorndike's 1898 monograph, an experimental study of associative processes in animals (Thorndike, 1898). More than a century later, Staddon and Cerutti's review of the field concentrated on two large classes of operant behaviour: how animals time the intervals between reinforcers, and how they choose among alternatives (Staddon & Cerutti, 2003).

The paradigm matters well beyond the animal laboratory. Its vocabulary of reinforcement, punishment, schedules and extinction was carried onto psychiatric wards and into autism treatment, and its theory is still moving. In a theory paper without new data, Killeen and Jacobs argued that no stimulus is a reinforcer in its own right: reinforcement is a relation between what a setting affords and what an organism is disposed to do (Killeen & Jacobs, 2017).

What Operant Conditioning Is

Operant conditioning is learning in which a response is strengthened or weakened by what follows it. Its standard unit of analysis is the three-term contingency: a discriminative stimulus sets the occasion, a response is made, and a consequence follows that changes how often the response recurs in that setting (Figure 1). Killeen and Jacobs placed that contingency inside a wider frame of motivational operations and behavioural systems, arguing that whether food or anything else reinforces depends on the animal's dispositions and on what the situation affords, so that being a reinforcer is a relation and not an intrinsic property of a stimulus (Killeen & Jacobs, 2017). Their case rests on argument rather than on new experiments.

The same process goes by the name instrumental learning, which MeSH records as an entry term for the descriptor (also in the inverted form Learning, Instrumental). Associative-learning research tends to prefer the instrumental label: Adams and Dickinson's devaluation experiments with hungry rats, for example, were analysed in terms of the instrumental contingency between lever pressing and food (Adams & Dickinson, 1981).

The contrast with classical conditioning turns on response dependence. In a classical procedure the significant event arrives whatever the animal does; in an operant procedure it arrives only when the response is made. The boundary is not always sharp in practice. Across four rat experiments framed as avoidance learning, Garcia and Koelling found that rats linked a taste readily with later illness and an audiovisual cue readily with shock, a selectivity of cue to consequence, and, having made the audiovisual cue depend on licking just as taste does, they concluded that cues are selected according to the nature of the reinforcer that follows (Garcia & Koelling, 1966). At the neural level, too, Schultz's review treated Pavlovian and operant learning together when describing how dopamine neurons code reward (Schultz, 2016).

Types of Operant Conditioning

MeSH files operant conditioning under its parent descriptor, psychological conditioning, and lists a single narrower descriptor beneath it (Table 1); subtypes in this hierarchy need not be mutually exclusive, and the hierarchy itself is an indexing classification for the literature rather than a theory of how learning is organised.

Avoidance learning is the case in which a response escapes or forestalls an aversive event, and one of its best-known findings is a failure of it: dogs given inescapable shock in a harness, compared with yoked controls, later performed poorly at escape/avoidance learning in a shuttle box (Seligman & Maier, 1967). Two-process learning theory, as reviewed by Rescorla and Solomon, holds that Pavlovian conditioned responses strongly mediate instrumental responding, though they judged the evidence from measuring the two kinds of response concurrently to be weak and the evidence from interaction studies to be stronger (Rescorla & Solomon, 1967).

Thorndike and the Law of Effect

The law of effect, the principle that consequences strengthen or weaken the responses that produce them, is the oldest theory in the field, and a recent theoretical proposal would revise how it is understood. Its experimental roots lie in Thorndike's 1898 monograph on the associative processes of animals (Thorndike, 1898), and Nevin and Grace dated the formal law to 1911 when they set out to reinterpret it (Nevin & Grace, 2000; Thorndike, 1911/2017).

Skinner took the empirical programme in a deliberately atheoretical direction. His 1950 paper asked whether theories of learning were needed at all and examined several kinds of theory, physiological theorising among them (Skinner, 1950). Two years earlier he had published a short report on superstition in the pigeon (Skinner, 1948), and with Ferster he later produced a book-length treatment of schedules of reinforcement (Ferster & Skinner, 1957).

The law itself was reworked three times in ways that matter here. Herrnstein's 1970 paper on the law of effect drew schedule data together into a quantitative account in which matching governed choice on concurrent schedules and contrast effects appeared on multiple schedules, so that a response's strength depended on reinforcement relative to other reinforcement (Herrnstein, 1970). Nevin and Grace recast the law in terms of behavioural momentum, discussed below. Killeen and Jacobs moved the explanatory weight from the stimulus to the relation between organism and setting (Killeen & Jacobs, 2017). On that last reading the law of effect survives as a description of how consequences act, while the older idea that reinforcing power belongs to particular stimuli has been revised. Baum went further still, contending that reinforcement is at best an incomplete concept and very probably a mistaken one and proposing that three others, allocation, induction and correlation, could serve in its place; his case is a conceptual argument that brings no new data (Baum, 2012).

Reinforcement and Punishment

Reinforcement is any consequence that raises the future rate of a response, and punishment any consequence that lowers it. Each can work by adding or by removing something, which yields four contingencies. Positive reinforcement adds a desirable event, negative reinforcement removes an aversive one, positive punishment adds an aversive event, and negative punishment removes a desirable one. The words positive and negative refer to adding and subtracting, not to good and bad.

Positive reinforcement was the subject of Premack's 1959 paper, the first part of an attempt to state empirical laws of behaviour (Premack, 1959). Timberlake and Allison's response-deprivation account ties instrumental performance to whether a contingency holds the contingent response below its operant baseline, and on their analysis instrumental performance is set not by the probability differential at the heart of the Premack principle but by response deprivation, leaving the empirical law of effect incomplete (Timberlake & Allison, 1974). Reinforcers need not be primary. In six experiments on a mental hospital ward, Ayllon and Azrin used tokens as conditioned reinforcers and found that token reinforcement maintained adaptive behaviour among psychotic patients, although the gains lasted only while the procedures were in force (Ayllon & Azrin, 1965).

Negative reinforcement is the contingency most often misread. Iwata's 1987 review described its applied use as an emerging technology, covering both behaviour maintained by negative reinforcement and negative reinforcement deliberately used as therapy, while noting that a comprehensive body of applied research did not yet exist (Iwata, 1987).

Evidence on punishment in human development comes largely from correlational studies of parents' physical discipline, and it does not test whether punishment causes the outcomes it accompanies. Gershoff's meta-analysis and theoretical review found parental corporal punishment associated with every one of 11 child behaviours and experiences examined, among them immediate compliance, aggression, moral internalization and mental health, and the review proposed a process-context model of how corporal punishment might cause particular outcomes, weighed alternative explanations, and listed seven major issues for future research (Gershoff, 2002). A later set of random-effects meta-analyses, covering 75 studies, 111 unique effect sizes and 160,927 children, found spanking associated with detrimental outcomes on 13 of 17 measures, with the other four non-significant (Gershoff & Grogan-Kaylor, 2016). Those estimates were bivariate, without covariates, came only from peer-reviewed English-language studies, and most studies were cross-sectional or retrospective; the few experimental and longitudinal studies included gave effect sizes not significantly different from the cross-sectional ones, and the authors judged randomized experiments of spanking difficult if not ethically impossible (Gershoff & Grogan-Kaylor, 2016).

Reinforcement also has costs when the behaviour was already enjoyed for its own sake. Pooling 128 studies, a meta-analysis concluded that tangible rewards erode intrinsic motivation: free-choice behaviour fell when rewards were engagement-contingent (d = -0.40), completion-contingent (d = -0.36) or performance-contingent (d = -0.28), self-reported interest showed smaller declines, and positive feedback raised motivation (d = 0.33) (Deci et al., 1999). In children, tangible rewards tended to do more harm and verbal rewards tended to do less good than in college students, and effects on self-reported interest were smaller than effects on free-choice behaviour (Deci et al., 1999). An earlier meta-analysis of 96 between-groups experiments had reached the opposite overall verdict, finding that reward did not lower intrinsic motivation; its one negative effect came from expected tangible rewards given simply for doing a task, and that effect was minimal and appeared on time spent on the task (Cameron & Pierce, 1994). Its within-subject analysis rested on only five studies (Cameron & Pierce, 1994).

The demonstration below lets the four contingencies be crossed: a stimulus is either added or removed after a response, and the response rate rises or falls accordingly.

Schedules of Reinforcement

A reinforcement schedule is the rule that decides which responses are reinforced. Ratio schedules reinforce after a number of responses and interval schedules after a period of time, each in fixed or variable form, and the four simple types generate characteristic patterns on a cumulative record. Ferster and Skinner's 1957 volume catalogued schedules of reinforcement at book length (Ferster & Skinner, 1957).

Interval schedules became a laboratory for the study of timing. Staddon and Cerutti's review discussed interval timing through proportional timing, Weber's law and linear waiting, and saw prospects for a unified approach to timing, choice and reinforcement schedules (Staddon & Cerutti, 2003). A separate line of work asked how firmly a schedule maintains behaviour when it is disrupted. Nevin and Grace argued that resistance to change and preference both reflect one underlying construct, which they likened to mass in physics, and named the approach behavioural momentum (Nevin & Grace, 2000). Nevin and Shahan later carried behavioral momentum theory into applied work, relating persistence to the reinforcers obtained in a stimulus context and drawing out what this means for resurgence after extinction, for noncontingent reinforcement and for differential reinforcement of alternative behavior (Nevin & Shahan, 2011). They cautioned that alternative reinforcement can backfire and that using the theory in practice means mapping its terms onto each setting (Nevin & Shahan, 2011).

The demonstration below runs the four simple schedules side by side so their response patterns can be compared.

Choice and the Matching Law

The matching law holds that, when two responses are reinforced on concurrent schedules, the share of responses each receives tracks its share of reinforcement; its standing is contested, and no appraisal more recent than 2003 was available for this article. Herrnstein's 1961 experiment gave pigeons two response keys on a concurrent schedule and related the relative frequency of responding on each key to the relative frequency of reinforcement it delivered (Herrnstein, 1961). In 1970 he extended the relation into a general account of response strength across concurrent, multiple and single schedules (Herrnstein, 1970).

The law did not go unchallenged. In a theoretical analysis of choice data, Baum proposed a generalized matching law that treats strict matching as the special case where both of its parameters equal 1 (a = 1, k = 1) and that captures two departures from it, undermatching and bias (Baum, 1974). Baum regarded the reasons for undermatching as obscure, and he offered its possible link to discrimination only as a suggestion (Baum, 1974). Staddon and Cerutti's review of operant choice, which also covered concurrent chain schedules and self-control, discussed problems for the matching law as well as its extensions (Staddon & Cerutti, 2003). Matching thus remains a widely used description of aggregate choice whose status as a fundamental law is unsettled. The Worked Example below sets out the arithmetic, and the demonstration lets the reinforcement rates be varied.

Biological Constraints

Operant conditioning does not work equally well on every response in every species. Writing from field experience of training many new species, Breland and Breland reported that instinctive behaviour patterns intruded on conditioned performance, and they echoed a growing view that the white rat could not reveal everything about behaviour (Breland & Breland, 1961); the surviving abstract is truncated, so the details rest on an excerpt. Garcia and Koelling's rat experiments pointed the same way from a different angle: taste was readily associated with illness and audiovisual cues with shock, which implies that some cue-consequence pairings are learned far more readily than others (Garcia & Koelling, 1966). Both lines of evidence limit the early hope that any arbitrary response could be shaped with equal ease by any reinforcer.

Actions and Habits

The distinction between goal-directed actions and habits is firmly established in rats, while in humans its standing is contested. Its test is reinforcer devaluation. In two experiments, Adams and Dickinson trained hungry rats to press a lever for food and then made the food aversive; the devalued reinforcer produced less responding in an extinction test, a result that held against a control aversion of comparable strength (Adams & Dickinson, 1981). The abstract gives no sample size.

Dickinson then showed that training history matters. Limited training left responding goal-directed, sensitive to devaluation, while extended training produced habits that persisted after the reward was devalued (Dickinson, 1985). He tied this autonomy to the loss of a correlation between performance and reward rather than to the amount of training as such, and the evidence concerned simple food-rewarded activity (Dickinson, 1985). Balleine and Dickinson later argued that goal-directed action rests on two kinds of learning, about the contingency between action and outcome and about the incentive value of the outcome, and pointed to the prelimbic prefrontal cortex and insular cortex as substrates, on lesion evidence they described as preliminary and from rats only (Balleine & Dickinson, 1998).

Recent work has complicated the picture from two directions. Reviewing experiments from his own laboratory, all in rats, Bouton argued that habit is reversible and depends on context and attention, with a predictable reinforcer favouring habit, although lever-insertion procedures produced habit despite uncertainty and why lever insertion behaved differently from a tone was not fully understood (Bouton, 2021). In human life, Wood and Rünger's review cast habit as an efficient default that complements deliberate goal pursuit, with links to stress and addiction (Wood & Rünger, 2016). The direct test in people, however, has not held up. Across five experiments with healthy adults, using outcome devaluation, a slips-of-action task, an avoidance learning task and a replication of the Tricomi et al. snack task, de Wit and colleagues found no support for the claim that overtraining produces habits, and the critical Group x Value interaction was non-significant (de Wit et al., 2018). The authors noted that the original human demonstration was a single study, that their briefly trained groups might not have reached asymptote, and that the tasks might be insensitive to training (de Wit et al., 2018). The human action-habit distinction is therefore unresolved: the rat evidence is strong, and the human evidence is a single positive study against five failures.

Neural Substrates

The reward prediction error hypothesis, which holds that dopamine neurons signal the difference between the reward received and the reward predicted, is supported. Schultz, Dayan and Montague proposed in a combined theory and review paper that primate dopaminergic neurons apparently encode such errors, providing a substrate for adaptive control by prediction (Schultz et al., 1997). Schultz's 2016 narrative review extended the account across humans, monkeys and rodents, relating positive and negative prediction errors to temporal difference learning and the Rescorla-Wagner model and to reward processing in economic terms (Schultz, 2016). The review noted that in humans, neuroimaging picks up the prediction-error signal in striatal and frontal terminal areas and not in the midbrain where the dopamine cell bodies sit, and the mechanisms behind gambling and food addiction remain less well understood (Schultz, 2016).

Learned helplessness shows a theory reversed by its own originators, and its standing is revised. The original finding was that dogs exposed to inescapable shock later failed to escape shock that they could have avoided (Seligman & Maier, 1967). Fifty years on, Maier and Seligman concluded from neuroscience that passivity in the face of prolonged aversive events is the unlearned default, governed by serotonergic activity in the dorsal raphe nucleus, and that what animals learn is control, detected by the medial prefrontal cortex (Maier & Seligman, 2016). They described their original theory as reversed and treated the implications for depression as speculative (Maier & Seligman, 2016).

Applications

Operant principles reached the clinic first as the token economy. Ayllon and Azrin's ward programme reinforced adaptive behaviour with tokens, and the behaviour declined once the procedures were withdrawn, a reminder that operant gains depend on the contingency staying in place (Ayllon & Azrin, 1965). Iwata's review argued that negative reinforcement, too, could be used as a therapeutic tool within applied behavior analysis (Iwata, 1987).

The most debated application is intensive treatment for autism. Lovaas compared 19 young autistic children given intensive behavioural treatment with 40 controls and reported that 47% of the treated group reached normal intellectual and educational functioning, while control outcomes were much poorer (Lovaas, 1987); only the abstract was available for this article. A Cochrane review three decades later pooled five studies, one randomised controlled trial and four controlled clinical trials, with 219 children with autism spectrum disorders who were under six when treatment began; the review describes the intervention as typically delivered at 20 to 40 hours per week (Reichow et al., 2018). Early intensive behavioural intervention was associated with better adaptive behaviour (mean difference 9.58 on the Vineland Adaptive Behavior Scale, 95% CI 5.57 to 13.60), higher IQ (mean difference 15.44, 95% CI 9.29 to 21.59) and better language, but effects on autism symptom severity and problem behaviour were not significant (Reichow et al., 2018). The authors rated the evidence low or very low in quality under the GRADE approach, because the studies were small, mostly non-randomised and at high risk of bias (Reichow et al., 2018).

Table 2 summarises the theories discussed in this article and where each stands.

Note. Status gives the most recent appraisal among the sources cited in this article. Where no appraisal from the last 15 years was available, the cell says so.

Discussion

Operant conditioning explains a great deal with a small set of ideas. The three-term contingency, the four contingencies of reinforcement and punishment, and the schedules that govern them account for how consequences shape behaviour in the laboratory, and quantitative relations such as matching describe choice among alternatives with considerable economy. The law of effect has survived more than a century, and one recent proposal would recast it as a statement about relations between organism and setting rather than about stimuli with built-in power, though that proposal is so far argued rather than tested.

The paradigm's limits are now as well documented as its reach. Biology constrains it: instinctive patterns intrude on trained performance, and some cue-consequence pairings are learned far more readily than others. Cognition constrains it too. Devaluation experiments show that much instrumental behaviour in rats is goal-directed, sensitive to the current value of its outcome, and that only some becomes habitual. The neural account has also moved: dopamine prediction-error signals supply a candidate substrate for reinforcement learning, and learned helplessness has been rewritten so that control, not passivity, is what animals learn.

Two questions remain open. The first is whether the action-habit distinction, robust in rats, describes human behaviour at all, given that a single human demonstration has been followed by five failures to induce habits by overtraining. Settling it would need tasks shown to be sensitive to training and adequately long training, two possible gaps that de Wit and colleagues raised about their own experiments. The second is the gap between strong animal evidence and weaker applied evidence. Token economies showed effects that faded when the contingency was withdrawn; the punishment literature is mostly correlational and unable to settle the direction of cause; and the pooled evidence for early intensive intervention in autism, though favourable on adaptive behaviour and IQ, rests on small, mostly non-randomised studies of low quality. For the clinical questions, randomised trials of adequate size, rather than further laboratory demonstrations, are what is needed; for spanking, the meta-analysts judged such experiments difficult if not ethically impossible.

Current Directions

Recent theory has questioned what a reinforcer is. Killeen and Jacobs argued that reinforcing value lies in the relation between an organism's dispositions and the affordances of a setting, not in the stimulus (Killeen & Jacobs, 2017), a proposal that is so far argued rather than tested.

The habit debate is the most active empirical front. Five failed attempts to induce habits in healthy adults (de Wit et al., 2018) sit beside rat work showing that habit is reversible and depends on context and the predictability of the reinforcer (Bouton, 2021), and beside a review that treats habit as a pervasive and efficient feature of everyday human behaviour (Wood & Rünger, 2016). Whether these strands describe one phenomenon is unresolved.

At the neural level, prediction-error coding by dopamine neurons has been described across humans, monkeys and rodents (Schultz, 2016), and the neuroscience of learned helplessness has led its originators to reverse their theory (Maier & Seligman, 2016). A rival account of dopamine comes from Berridge and Robinson, who combined a literature review with experiments on rats whose accumbens and neostriatal dopamine was depleted by up to 99% and whose taste reactivity was measured; they concluded that dopamine may govern the incentive salience of rewards, a matter of wanting rather than of liking (hedonic impact), and that dopamine is not needed for the predictive associations of hedonic reward learning (Berridge & Robinson, 1998). In applied research, meta-analyses of spanking found consistent associations with detrimental child outcomes in correlational data (Gershoff & Grogan-Kaylor, 2016), and a Cochrane review found favourable but low-quality evidence for early intensive behavioural intervention in autism (Reichow et al., 2018).

Commonly Confused With

Classical Conditioning

The test is whether the consequence depends on what the organism does. In operant conditioning the consequence follows only when the response occurs, and so it changes how often that response is made. In classical conditioning the consequence arrives whether or not the organism responds. The two are confused because in both an animal links a cue with a consequence; in rat experiments that found such links to be selective (Garcia & Koelling, 1966), both cues depended equally on the response, so the selectivity tracked the kind of consequence, not response dependence.

Common Misconceptions

Negative reinforcement is another name for punishment.

Negative reinforcement strengthens a response by removing an aversive event; punishment weakens one. Applied behaviour analysts have studied behaviour maintained by negative reinforcement and have used the contingency as a form of therapy (Iwata, 1987). The confusion is lexical: negative means subtracted in the technical sense, not bad.

Rewards always increase motivation.

A meta-analysis of 128 studies found that tangible rewards undermined intrinsic motivation, and that they tended to do more harm in children than in college students (Deci et al., 1999). The belief persists because rewards do raise the rate of the rewarded behaviour while the reward lasts.

Any behaviour can be conditioned equally easily in any species.

Animal trainers working with many species found that instinctive behaviour patterns intruded on conditioned performance, and echoed a growing view that the white rat could not reveal everything about behaviour (Breland & Breland, 1961). The belief grew from the generality of early laboratory results, which were obtained with a few species and a few arbitrary responses.

Glossary

Avoidance learning.

Learning a response that escapes or prevents an aversive event; the one narrower descriptor MeSH lists under operant conditioning.

Concurrent schedule.

An arrangement in which two or more responses, such as pecks on two keys, are each reinforced on its own schedule at the same time; the standard procedure for studying choice.

Conditioned reinforcer.

A stimulus, such as a token, that reinforces because of its association with other reinforcers rather than in its own right.

Cumulative record.

A running graph of total responses against time, whose slope shows response rate and whose shape reveals the pattern a schedule produces.

Discriminative stimulus.

The first term of the three-term contingency: a stimulus in whose presence a response is followed by a particular consequence.

Extinction.

The withholding of reinforcement for a previously reinforced response, and the decline in responding that follows; also the test phase of devaluation experiments.

Fixed-interval schedule.

A schedule that reinforces the first response after a set period has elapsed since the last reinforcer.

Fixed-ratio schedule.

A schedule that reinforces every nth response, for a constant n.

Goal-directed action.

Instrumental behaviour controlled by the current value of its outcome and by the action-outcome contingency, shown in rats by sensitivity to reinforcer devaluation.

Habit.

Responding that persists after its outcome is devalued, produced by extended training in rats; whether overtraining produces it in humans is contested.

Instrumental learning.

A synonym for operant conditioning, listed by MeSH as an entry term, stressing that the response is instrumental in producing its consequence.

Law of effect.

The principle, traced to Thorndike, that consequences strengthen or weaken the responses that produce them; recent theory proposes revising it.

Learned helplessness.

A later failure to escape avoidable shock after exposure to inescapable shock, first shown in dogs; its originators now hold that passivity is the unlearned default and control is what is learned.

Matching law.

The relation in which each response's share of total responding equals its share of total reinforcement on concurrent schedules; a contested law.

Negative punishment.

Removal of a desirable event after a response, which lowers the response's future rate.

Negative reinforcement.

Removal of an aversive event after a response, which raises the response's future rate; not a form of punishment.

Positive punishment.

Addition of an aversive event after a response, which lowers the response's future rate.

Positive reinforcement.

Addition of a desirable event after a response, which raises the response's future rate.

Reinforcer devaluation.

Reducing the value of a reinforcer after training, for example by pairing food with illness, to test whether responding still depends on the outcome's value.

Reward prediction error.

The difference between a reward received and the reward predicted, positive when the outcome is better than expected and negative when worse; a signal attributed to dopamine neurons.

Token economy.

A programme in which tokens earned for target behaviour serve as conditioned reinforcers; on a psychiatric ward its gains lasted only while the programme ran.

Variable-interval schedule.

A schedule that reinforces the first response after a period that varies around a mean.

Variable-ratio schedule.

A schedule that reinforces after a number of responses that varies around a mean.

Key Researchers

Bernard W. Balleine

Scientia Professor in the School of Psychology at UNSW Sydney; with Anthony Dickinson he analysed goal-directed instrumental action and its cortical substrates. ORCID - Faculty Page

Mark E. Bouton

Learning theorist in the Department of Psychological Science at the University of Vermont; his laboratory has studied context, extinction and the switch between habit and goal-directed behaviour. ORCID

Anthony Dickinson

(b. 1944). Emeritus Professor of Comparative Psychology at the University of Cambridge; his reinforcer-devaluation experiments distinguished goal-directed actions from habits. Wikidata - Wikipedia - Faculty Page

Elizabeth T. Gershoff

Professor of Human Development and Family Sciences at The University of Texas at Austin; author of meta-analyses of corporal punishment and child outcomes. Wikidata - Wikipedia

Richard J. Herrnstein

(1930–1994). Psychologist at Harvard University who formulated the matching law relating response allocation to relative reinforcement. Wikidata - Wikipedia

Wolfram Schultz

Professor of Neuroscience at the University of Cambridge; his single-neuron recordings in monkeys found phasic dopamine responses to rewards and to cues during learning tasks (Schultz et al., 1993), work that underpins the reward prediction error account. ORCID - Wikidata - Wikipedia

B. F. Skinner

(1904–1990). Psychologist at Harvard University who developed the experimental analysis of operant behaviour and, with Charles Ferster, catalogued schedules of reinforcement. Wikidata - Wikipedia

Edward L. Thorndike

(1874–1949). Psychologist at Teachers College, Columbia University, whose experiments on animal learning led to the law of effect. Wikidata - Wikipedia

Frequently Asked Questions

What is operant conditioning?

Operant conditioning is learning in which the consequences of a response change how often that response occurs, and Killeen and Jacobs have argued that a reinforcer is a relation between organism and setting rather than a fixed property of a stimulus (Killeen & Jacobs, 2017).

How does operant conditioning differ from classical conditioning?

In operant conditioning the consequence depends on the response, whereas in classical conditioning it does not, although rat experiments that made an audiovisual cue depend on licking, as taste does, found that which cue an animal links with a consequence depends on the kind of consequence (Garcia & Koelling, 1966).

What is negative reinforcement?

Negative reinforcement strengthens a response by removing an aversive event, and applied behaviour analysts have studied both behaviour maintained this way and its deliberate use as therapy (Iwata, 1987).

What are the schedules of reinforcement?

Schedules of reinforcement are rules that set which responses earn reinforcement, by count or by elapsed time, in fixed or variable form, and Ferster and Skinner devoted a book to cataloguing them (Ferster & Skinner, 1957).

Who discovered operant conditioning?

Its experimental study began with Thorndike, whose 1898 monograph examined associative processes in animals and led to the law of effect (Thorndike, 1898), and Skinner later developed it into the experimental analysis of behaviour, the subject of his 1938 book on the behavior of organisms (Skinner, 1938).

Does punishment work?

Across 75 studies covering 160,927 children, meta-analyses linked spanking to worse outcomes on 13 of 17 measures, but the effects were bivariate, most of the studies were cross-sectional or retrospective, and the authors noted that settling the direction of effect needs longitudinal or experimental designs, though the few included gave effect sizes similar to the cross-sectional ones (Gershoff & Grogan-Kaylor, 2016).

Do rewards reduce motivation?

A meta-analysis of 128 studies found that tangible rewards undermined intrinsic motivation, tending to do more harm in children than in college students and with smaller effects on self-reported interest than on free-choice behaviour (Deci et al., 1999), although an earlier meta-analysis found that reward did not lower intrinsic motivation overall (Cameron & Pierce, 1994).

Is operant conditioning used in therapy?

Early intensive behavioural intervention for autism was associated with gains in adaptive behaviour and IQ in a Cochrane review of five studies, though the evidence was rated low or very low in quality (Reichow et al., 2018).

References