What Is Dopamine Actually Doing?
It is not the pleasure chemical. Dopamine neurons fire when things go better than expected, and go silent when a reward arrives exactly on schedule.
The surprise signal, not the pleasure signal
Junior level — plain language, no maths
Dopamine gets described as the brain's pleasure chemical, and that description is wrong in a way that matters. The experiment that settled it is simple enough to run in an afternoon. Give a thirsty monkey a drop of juice with no warning, and the dopamine neurons fire a burst. So far the pleasure story survives.
Now do it a hundred times, but switch a light on two seconds before every drop. Something strange happens. The burst at the juice gets smaller and smaller until it disappears completely - and a new burst appears at the light. The monkey still likes the juice. It drinks it just as eagerly. But the neurons have stopped reacting to it.
What they are reporting is not how good something is. It is how much better it is than expected. An unexpected drop is a pleasant surprise, so there is a burst. A drop you saw coming two seconds ago is exactly what you predicted, so there is nothing to report. The light, on the other hand, is genuinely new information - it means juice is on its way - and that is what gets the burst now.
The proof is in what happens when you break your promise. Switch the light on and then give no juice, and at the exact moment the drop should have arrived, the neurons go below their normal rate. Nothing bad happened; something good merely failed to happen. Press the button in the simulation and watch the dip appear.
Things worth knowing
- The monkeys in Wolfram Schultz's experiments drank the juice just as eagerly after the dopamine response to it had vanished entirely. Liking it and signalling it are separate things.
- An expected reward that fails to arrive drives dopamine below baseline at precisely the moment it was due - which means the brain is timing the prediction, not just making it.
- Unpredictable rewards keep the burst alive forever, because they are never fully predicted. That is the mechanism a slot machine is built around.
Reward prediction error and temporal difference learning
Student level — the core equations
The quantity dopamine neurons encode has a precise definition borrowed from reinforcement learning: the reward prediction error, \(\delta = r - V\), where \(r\) is what arrived and \(V\) is what was predicted. Positive \(\delta\) means better than expected and produces a burst; zero produces nothing; negative produces a pause below baseline. Learning is then simply \(V \leftarrow V + \alpha\delta\) - move the prediction towards reality by some fraction of the error. This is the Rescorla-Wagner rule, proposed from animal learning data in 1972, and dopamine neurons appear to implement it.
The temporal structure needs the fuller version. In temporal difference learning the error is \(\delta_t = r_t + \gamma V(s_{t+1}) - V(s_t)\): a state is valuable if it leads to valuable states. This is what makes the response migrate backwards in time to the earliest reliable predictor, exactly as observed, and it explains the dip at reward omission without any extra assumptions - \(V(s_{t+1})\) collapses to zero when the expected reward does not appear.
The correspondence is unusually tight for neuroscience. TD learning was developed as a computational solution to a credit-assignment problem, and the match to dopamine firing was noticed afterwards - a prediction, not a fit. Dopamine also acts as a plasticity signal rather than merely a report: it gates long-term potentiation at corticostriatal synapses, so the error signal directly modifies the weights that produced the prediction.
The distinction that follows is between wanting and liking, and it is experimentally sharp. Dopamine-depleted rats still show normal hedonic facial reactions to sucrose - they like it fine - but will not work for it. Dopamine assigns incentive salience: it makes cues motivating. That is why addiction is better described as pathological wanting than as excessive pleasure, and why people report craving substances they no longer enjoy.
Key Formulas
| Prediction error | \(\delta = r - V\) | the burst size |
|---|---|---|
| Rescorla-Wagner | \(V \leftarrow V + \alpha\,\delta\) | |
| Temporal difference | \(\delta_t = r_t + \gamma V(s_{t+1}) - V(s_t)\) | |
| Value of a state | \(V(s) = \mathbb{E}\!\left[\textstyle\sum_{k} \gamma^{k} r_{t+k}\right]\) | |
| At omission | \(\delta = 0 - V = -V\) | the dip below baseline |
Things worth knowing
- TD learning was invented to solve a machine learning problem. The match to dopamine firing was found afterwards - one of the cleanest cases of a computational theory predicting a biological measurement.
- Rats with dopamine depleted still make the same pleasure faces at sugar, but will not cross a room for it. Wanting and liking are separable, and dopamine carries the wanting.
- Dopamine does not just report the error - it gates plasticity at the synapses that made the prediction. The signal and the learning rule share the same molecule.
Distributional coding, tonic versus phasic, and where the simple story breaks
Scholar level — full mathematical depth
01The canonical account, and its evidence
Schultz's recordings established the three signatures - burst to unpredicted reward, transfer to the earliest predictor, dip at omission - and optogenetics later supplied causality: artificially bursting dopamine neurons at the moment of reward delivery produces learning as if the reward had been better than it was, and suppressing them produces extinction. The signal also scales with probability and magnitude in the way \(\delta\) requires, and shows the correct asymmetry, since firing rates are floored at zero while bursts are not - negative errors are encoded by pause duration rather than by rate.
02Distributional reinforcement learning
The most interesting recent result is that the population does not encode a single scalar. Individual dopamine neurons have different reversal points - the reward magnitude at which they switch from excitation to inhibition - and differ systematically in how asymmetrically they weight positive against negative errors. That is precisely the structure required by distributional RL, in which a population of value predictors with different optimism parameters collectively represents the full probability distribution over future reward rather than its mean. The distribution can be decoded from the population, which means the brain is tracking risk, not just expectation.
03Tonic, phasic, and vigour
Phasic bursts on a hundred-millisecond scale carry prediction error; slowly varying tonic concentration appears to carry something else. Niv's account ties tonic dopamine to the average reward rate, which in optimal control sets the opportunity cost of time and therefore the optimal vigour - how fast to act. It explains the otherwise puzzling pairing of bradykinesia in Parkinson's disease with intact motor capability, and the effect of stimulants on response speed rather than on accuracy. It also explains why the same molecule looks like a learning signal in one experiment and a motor signal in another: different timescales, different quantities.
04Where the simple story is wrong
Dopamine neurons are not homogeneous. A substantial subpopulation, particularly in the ventral tegmental area projecting to the tail of the striatum, responds to salient events including aversive ones and novel neutral stimuli - not a reward prediction error by any reading. Dorsal striatal dopamine release tracks position and movement kinematics in ways that a scalar error cannot explain, and recent work shows release waves travelling across the striatum with spatial structure. The field is converging on a view in which projection target, not cell identity, determines what the signal means.
05Addiction, and what dopamine actually predicts
Redish's model treats addictive drugs as producing a prediction error that cannot be cancelled by learning, because the drug acts on the error signal pharmacologically rather than through the reward pathway that the prediction would learn about. The value of drug-predicting states therefore grows without bound, which matches the clinical picture better than any account based on pleasure - tolerance reduces the hedonic effect while craving increases, and cue-induced relapse occurs long after withdrawal ends. The practical implication is that the target for treatment is the cue-value association, not the pleasure.
Key Formulas
| TD error | \(\delta_t = r_t + \gamma V(s_{t+1}) - V(s_t)\) | |
|---|---|---|
| Asymmetric update | \(V_i \leftarrow V_i + \alpha_i^{+}[\delta]_{+} - \alpha_i^{-}[\delta]_{-}\) | |
| Expectile | \(\tau_i = \dfrac{\alpha_i^{+}}{\alpha_i^{+}+\alpha_i^{-}}\) | the neuron's optimism |
| Average reward | \(\rho = \lim_{T\to\infty}\tfrac{1}{T}\textstyle\sum_{t} r_t\) | tonic dopamine, vigour |
| Optimal vigour | \(\tau^{*} = \sqrt{C_v/\rho}\) | act faster when reward is dense |
Things worth knowing
- Individual dopamine neurons have different reversal points and different optimism - exactly the structure distributional reinforcement learning requires. The distribution over future reward can be decoded from the population.
- Tonic dopamine appears to set vigour by encoding the average reward rate, which in optimal control is the opportunity cost of time. It explains why Parkinsonian slowness coexists with intact motor ability.
- Addictive drugs produce a prediction error the system cannot learn away, because they act on the error signal itself. Value grows without bound - which is craving without enjoyment.