Dopamine neurons

Dopamine neuronal response

Economic utility formalises the subjective nature of reward value mathematically. Meaningful reward neurons should code utility rather than physical amount. The dopamine reward prediction error (RPE) signal follows nonlinear utility with unpredicted reward (left) and gambles (right: stronger dopamine UPE response at steeper utility slope despite same physical RPE).

Dopamine optogenetics

Medium efficacy but high specificity (left). Optogenetic stimulation adds to the dopamine response to juice reward (top center), which enhances the dopamine response to a reward predicting stimulus, compared to juice alone (centre bottom) and enhances the choice probability for that stimulus (right).

Response components

Neuronal responses to external events can have multiple components (center), as shown in frontal cortex (top left). An external reward is registered in the brain first by its physical impact and then by its value. Dopamine responses show similar two components (right). Their first component reflects salience and is enhanced by several factors, which explains some non-reward responses (blue box). The benefit is early reward detection (maroon box).

Probability

According to the frequentist approach, probability reflects the frequency of past events (top left). Reinforcement learning can implement that approach (top right). Monkeys learn to estimate probabilities in choice between a fixed known reward probability and different probabilities indicated by new stimuli (bottom left). Dopamine responses to the new stimuli show an indiscriminate novelty response that declines with stimulus experience and is followed shortly later by a differential value response that develops with experience (bottom right).

Aversive stimuli

Like rewards, punishers have multiple components that are processed separately (top left). Following psychophysical value assessment (bottom left), a stronger punisher elicits a weaker dopamine excitation, indicating that the excitation reflects punisher salience rather than negative outcome value (top right). Thus, punishers elcit two-component dopamine responses similar to rewards (bottom right). Without such controls, an apparent dopamine excitation from a punisher may simply reflect the first, salience component.

Multiple functions

Large movements engaging plenty of muscles and sensory receptors in monkeys are associated with heterogeneous dopamine increases or decreases at various task events (left). Similar dopamine changes occur in rodents during their behavioural engagement (bottom) These dopamine activities seem to reflect general behavioural activation rather than intercorrelated specific movement parameters or reward expectation. By contrast, more restricted behavioural activity fails to affect dopamine impulse activity in monkeys (right). Thus, dopamine neurons show at least two types of behavioural relationships (top): fast phasic reward prediction error (RPE) signalling, and slower behavioural activation with trial start, movement (and associated visual and somatosensory stimulation), reward consumption and everything that excites the animal.