Where do the priors come from? The role of structured representations in Bayesian models of learning and reasoning
Rational models have gone far by assuming that learning, reasoning, perception, decision-making and other cognitive processes can be understood as approximations to optimal statistical inference -- often some form of Bayesian inference. With appropriately chosen prior probabilities, Bayesian models can explain many aspects of behavior that have previously resisted a principled unifying account. But these successes also raise a challenge, which has been rightly voiced by critics of the approach. Where do the priors come from? Are they merely a choice of the modeler or assumed to be built-in constraints -- in which case they seem like a potentially endless source of flexibility? Or can we give some principled account of how human learners acquire appropriate priors from their experience in the world? If the latter, how do we avoid an infinite regress: where do the priors for the priors come from....?
I will argue that Bayesian modelers can give compelling answers to these questions, but only if they embrace an idea that has typically been cast in opposition to probabilistic approaches to learning and inference. The key is to understand how rational statistical mechanisms of inference interact with structured symbolic representations of abstract knowledge. I will present a hierarchical Bayesian framework for inductive learning, in which Bayesian inference operates over structured representations of knowledge at multiple levels of abstraction, with each level defining priors for the level below. As representations become more abstract, they become simpler and more general -- and more plausible candidates for built-in cognitive architecture. In many cases, a higher-level knowledge representation may be thought of as a kind of "intuitive theory" for a domain of entities, properties, and relations. The hierarchical Bayesian analysis shows how abstract knowledge of domain structure can generate strong priors for guiding generalization at lower levels, and how that abstract knowledge may itself be learned through rational statistical means. I will discuss applications of the framework to learning and reasoning in several domains, such as natural kind categories and their properties, social relations and causal relations.
