|
|
|
 |
|
Flexible Shaping: How learning in small steps helps
Kai Krueger
Gatsby Computational Neuroscience Unit, UCL , UK
Animals and humans enjoy large, flexible, repertoires of learned behaviours. Although some are acquired through solitary interaction with an environment, a rich and interesting class is taught by such means as abstract verbal explanation, imitation or shaping, which is the focus of this work. Shaping typically involves a teacher decomposing a single, complex, task into multiple simpler ones, and thus providing a possible path for learning. Although shaping is critical for training animals to do sophisticated behavioural tasks, it has so far been mostly ignored in computational modeling, often in favour of architectural elaborations.
Here, we consider shaping in the context of the 12-AX task, a paradigmatic cognitive problem with an hierarchical structure. First we establish the feasibility of a particular shaping path, and consider what about a network allows shaping to be effective. Then we show that shaping has special benefits in the face of long temporal credit assignment paths. Finally, we analyse the effects of shaping on the ability of the network to deal with task variations, including the effects of changing stimulus statistics while keeping the reward rules constant and task shifting. Even the very simple sort of shaping we consider turns out to be highly beneficial.