• Home
• Staff & Students
• Vacancies

Kenji Doya

 

 

http://www.cns.atr.jp/~doya/okinawa/

 

 

( Okinawa Institute of Science and Technology (OIST) )

 

Wednesday 2nd May 2012

16:00

 

B10 Seminar Room, Basement,

Alexandra House, 17 Queen Square, London, WC1N 3AR

 

 

Multiple Strategies for Decision and Learning Kenji Doya Okinawa Institute of Science and Technology

 

 

The theory of reinforcement learning prescribes two major strategies of decision making: model-based and model-free. The dichotomy has been helpful in searching for the computational bases for goal-directed and habitual behaviors, conscious and unconscious actions, and so forth (e.g., Daw et al., 2005). Both model-based and model-free strategies, however, have several different algorithms for implementation and we are assessing which of them are used in what stage of learning, and by what neural mechanisms.

 

In the analysis of rats' binary choice sequences under probabilistic reward, we had earlier showed that action value-based models predicted rats' choices better than higher-order Markov models (Ito & Doya, 2009). However, recent analysis of a new set of data revealed that finite state-based models predict rats' choices better than action value-based models. Analysis of striatal neural firing revealed both action value-coding and finite state-coding neurons in the dorsomedial striatum.

 

In a human fMRI experiment requiring multiple steps of actions to the goal, we found the evidence of model-based action planning in the intermediate stage of learning, when the activities were seen in not only the cortical areas (parietal, premotor, prefrontal), but also the anterior basal ganglia and the lateral cerebellum. Behaviors in both early and late stages of learning can be considered as model-free, but the performance as well as the brain activation during the two stages were largely different.

 

The results suggest the need for considering at least two different sub-types of model-free decision making, namely, value-based and procedure-based. The neural implementation of model-based decision making may not just involve the prefrontal cortex, but also the basal ganglia and the cerebellum (Doya, 1999).