Complementary priors were first proposed by Hinton et al (2006) as priors in greedy layer-wise learning of deep belief networks. They can be thought of as surrogate priors which are used in learning each layer then discarded when learning further layers. In this talk we explore an alternative approach to surrogate priors using nonparametric Bayesian modelling. In particular, we treat the learning of each layer in the deep belief network as a semiparametric modelling problem, with a nonparametric DP mixture prior on the latent variables but a parametric conditional model for the observed variables given latent variables.
BACK
