|
Yonatan Loewenstein (and H. S. Seung)
Howard Hughes Medical Institute & Department of Brain & Cognitive
Sciences, MIT, USA
Wednesday 20 July 2005
16:00
4th Floor Seminar Room, Alexandra House,
17 Queen Square, London WC1N 3AR
Stochastic choice and operant matching in a neuronal model
According
to Herrnstein's famous matching law of repeated choice behavior, the probability
of choosing an alternative is proportional to the reward derived from that
alternative. Although this empirical law of behavior has been studied
extensively by psychologists, economists, and neuroscientists, the question of
its neural
basis has received little attention.
Here we
show that a classic neural model of decision making, when combined with Hebbian
synaptic plasticity, reproduces matching behavior. Choices between alternatives
are mediated by lateral inhibition between decision neurons, while stochasticity
is generated by fluctuations in the activity of input neurons. For exponentially
distributed inputs, the choice probabilities are proportional to the strengths
of the synapses from the input neurons to the decision neurons. Therefore,
plasticity of these synapses leads to changes in choice probabilities over time.
In particular, we demonstrate equivalence between neural and behavioral models,
by showing that two forms of Hebbian plasticity cause our neural model to
reproduce two well-known reinforcement learning models that lead to matching
behavior. These two models can be distinguished behaviorally by their dependence
of speed of convergence on the overall reward rate, and by the way they learn to
play games. In both models, the mean activity of an input neuron, conditioned on
choice of the corresponding alternative, is a decreasing function of that
alternative's choice probability. The two models can be distinguished
physiologically by measuring how the mean activity of a decision neuron,
conditioned on choice of the corresponding alternative, depends on the reward
rates of other alternatives.