Feeding, Motivation and Reward

Switch on one small group of hypothalamic neurons with a pulse of light, and a mouse that has just eaten its fill starts eating again. Aponte, Atasoy and Sternson reported this in 2011, in well-fed mice whose arcuate AgRP neurons carried a light-gated channel. That one experiment raises the two questions of this lesson. How does the brain register the need for food? And how does it learn which cues and actions lead to food, or to any reward?

The idea: hunger is read from a hormone, not found in a centre

For decades feeding was explained by two “centres”. In rats, Hetherington and Ranson (1940) found that lesions of the ventromedial hypothalamus produced overeating and obesity, and Anand and Brobeck (1951) found that lesions of the lateral hypothalamus stopped animals eating. A lesion destroys everything in one place, including fibres passing through, so these labels described where damage changed feeding, not what the cells there compute. They have been superseded.

The better clue came from genetics. Two mouse strains, obese (ob) and diabetes (db), were both obese. Douglas Coleman used parabiosis, surgically joining two animals so they exchange some blood. His 1973 reading of the results: an ob/ob mouse lacks a circulating factor that limits eating, and a db/db mouse makes the factor but cannot respond to it. A normal mouse joined to a db/db mouse ate little and starved.

Jeffrey Friedman’s laboratory found the ob gene by positional cloning (Zhang and colleagues, 1994; the method is in linkage and mapping). Its product, named leptin, is a protein of about 16 kDa made by fat and present in mouse and human plasma. Daily injections of recombinant leptin reduced the body weight of ob/ob mice by 30% after two weeks, and did not reduce the weight of db/db mice (Halaas and colleagues, 1995). The db gene turned out to encode the leptin receptor, which signals through the pathway taught in JAK-STAT signalling.

Think of leptin as a fuel gauge rather than a brake. It reports stored energy. Most people with common obesity have high leptin, not too little, so the simple replacement that worked in ob/ob mice, and in rare people born unable to make leptin, does not reliably treat common obesity.

The idea: two arcuate populations push feeding in opposite directions

The arcuate nucleus is a cluster of neurons at the base of the hypothalamus, next to the median eminence, where circulating hormones reach neurons easily (a nucleus, in the brain, is a cluster of cell bodies with a shared role; the hypothalamus lesson maps the region). Two intermingled populations there matter most.

• AgRP neurons make agouti-related peptide, coexpress neuropeptide Y (NPY), and release GABA. AgRP is an antagonist of the melanocortin-4 receptor (MC4R).
• POMC neurons make pro-opiomelanocortin, which is cleaved into alpha-MSH, an agonist at MC4R (a G-protein-coupled receptor; see GPCRs and G proteins). Alpha-MSH acting on MC4R reduces feeding.

Leptin acts on both. In recordings from identified POMC neurons in mouse brain slices, leptin depolarized the cells through a nonspecific cation conductance and reduced their inhibitory input from local NPY/GABA neurons (Cowley and colleagues, 2001).

Now read the manipulations by what each can show. Exciting AgRP neurons with channelrhodopsin (20 Hz, 10 ms light pulses) was sufficient to evoke feeding in well-fed mice. It still evoked feeding in Ay mice, whose melanocortin receptors are already blocked, so AgRP neurons do not act only by opposing alpha-MSH. Prolonged POMC stimulation reduced intake, and that effect required melanocortin signalling (Aponte and colleagues, 2011). For necessity, Luquet and colleagues (2005) killed AgRP neurons with diphtheria toxin: adult mice rapidly starved, while ablation in newborn mice had minimal effects on feeding, which suggests that developing circuits compensate.

Calcium recordings (fibre photometry) add timing. In awake mice, nutrients infused into the stomach inhibited AgRP neurons rapidly, scaling with calories, while leptin changed their activity over hours (Beutler and colleagues, 2017). Brief AgRP stimulation that ended before food arrived still increased later eating (Chen and colleagues, 2016). Whether AgRP activity feels unpleasant or rewarding to the mouse depends on the test used, so that question stays open.

The idea: dopamine neurons signal an error in prediction

Hunger makes food valuable. Something must also learn which cues predict it. Dopamine neurons in the ventral tegmental area (VTA) and substantia nigra of the midbrain project to the striatum, including the nucleus accumbens, and to frontal cortex (transmitter systems covers their release and reuptake).

Wolfram Schultz’s group recorded single dopamine neurons in behaving monkeys. About three quarters of recorded neurons in two monkeys fired a brief burst when juice arrived without warning. After the monkeys learned that a sound predicted the juice, the burst came at the sound rather than at the juice (Mirenowicz and Schultz, 1994). When an expected reward was left out, firing dipped below baseline at the time it should have come; a reward at an unexpected time evoked a burst (Hollerman and Schultz, 1998).

That pattern is a reward prediction error: the difference between the reward received and the reward predicted. The simplest learning rule built on it is the Rescorla-Wagner rule.

delta(n) = r - V(n); V(n+1) = V(n) + alpha x delta(n)

Here V is the cue’s learned value, r the reward, and alpha (between 0 and 1) the learning rate. The temporal-difference rule, delta(t) = r(t) + gamma V(t+1) - V(t), gives the same answer at the end of a trial, where nothing follows and V(t+1) = 0 (Schultz, Dayan and Montague, 1997).

Worked problem. Take alpha = 0.30, a reward of r = 1 unit on every trial, and V = 0 at the start. These are model units, not spikes per second.

1. Trial 1: delta = 1 - 0 = 1.00. New V = 0 + 0.30 x 1.00 = 0.30.
2. Trial 2: delta = 1 - 0.30 = 0.70. New V = 0.30 + 0.30 x 0.70 = 0.51.
3. Trial 3: delta = 1 - 0.51 = 0.49. New V = 0.51 + 0.30 x 0.49 = 0.657.
4. The error left after n trials is 0.7 to the power n, so V = 1 - 0.7^n. After nine trials, V = 1 - 0.040 = 0.96.

The error at the reward shrinks as the cue’s value grows. In the temporal-difference reading, the cue itself arrives unpredicted, so the error at the cue grows toward V: the burst moves from reward to cue, as in the monkeys. Now omit the reward after long training, with V about 0.95: delta = 0 - 0.95 = -0.95, the dip at the expected time. An unpredicted reward (V = 0) gives +1.

Two cautions. A recording shows that firing goes with an error; it does not show that the burst teaches. And the code is not uniform: not every dopamine neuron fits it, and in rats, accumbens dopamine release tracked reward expectation without matching changes in VTA firing (Mohebi and colleagues, 2019). How dopamine changes striatal synapses is in synaptic plasticity and the basal ganglia.

Design problem

Recordings cannot show that a dopamine burst causes learning. Design an experiment that tests whether an imposed burst is sufficient to make an animal learn about a cue it would otherwise ignore.

The idea: wanting is not liking

A common shortcut treats dopamine as the brain’s signal for pleasure. The evidence separates two things. Wanting, or incentive salience, is the motivational pull of a reward and its cues. Liking is the hedonic impact of the reward itself (Berridge and Robinson, 1998).

Liking can be measured in animals through taste reactivity: tongue protrusions to sweet, gapes to bitter, scored from video. Rats whose forebrain dopamine was depleted with a neurotoxin stopped seeking food yet kept normal positive reactions to sugar in the mouth. In the other direction, mice with dopamine transporter (DAT) expression at about 10% of normal, and about 70% more extracellular striatal dopamine, reached sweet food sooner in a runway, because they paused less (their running speed itself did not differ), but showed no increase in liking reactions to sucrose (Peciña and colleagues, 2003). Excess dopamine raised wanting, not liking. A lifelong mutation is not an acute drug, and a mouse’s facial reaction is not a human report, so keep the scope.

The idea: in addiction, wanting can grow while liking does not

Addictive drugs raise dopamine signalling by different routes, such as blocking DAT or reversing it (see transmitter systems). Incentive-sensitization theory (Robinson and Berridge) proposes that in susceptible individuals, repeated drug use makes the circuits that assign incentive salience hypersensitive, so drug wanting, especially when triggered by cues, can grow out of proportion to drug liking. It explains why craving can persist after the drug has stopped being enjoyable.

It is a theory, and addiction is not only dopamine. Other accounts stress habit learning in the dorsal striatum, weakened prefrontal control, and withdrawal and stress systems, and genes, environment and drug history all shape who becomes addicted. Forward, attention and decisions shows how learned values guide choice.