40. Modularisation through the lense of Shaping

Gatsby, University College London, London, UK

An important solution to the problem of acquiring temporally and structurally rich and complex cognitive capabilities is modularization, or divide and conquer. We, and indeed other species, are able to learn simple elements and recombine them in multifarious ways in order to address sophisticated challenges. However, despite some important suggestions, most computational models have focused on uniform, rather than modularized learning, that is, taking on the full complexity of tasks starting from a naive state. Not only does this make learning of a single problem more difficult, but it also fails to generate sub-solutions that can be re-used later. Evidence for the nature and importance of modularity comes from the behavioral procedures used for training subjects to solve complex tasks. Subjects are shaped, i.e., are led step-by-step to acquire elemental sub-components before being presented with full tasks.

Previously, we used a computational model to elucidate shaping019s substantial beneficial effects on learning. We demonstrated this in a hierarchical, conditional one-back memory-based cognitive task called 12-AX, which we continue to employ here. However, in that study, we solved by hand one of the critical problems in making shaping work, namely the allocation mechanism that creates new network resources for each stage of shaping. Here, we explore algorithmic replacements for this homunculus which are based on ideas about the cortical and subcortical processing of surprise. Changes in tasks over the course of shaping can be detected based on sudden increases in the observed error (a form of unexpected uncertainty); this provides a signal that new resources should be allocated. We show that a mechanisms based on this can allow shaping to work well for standard learning; in certain cases such as reversal learning can even out perform the manual allocation previously employed.