Restricted Boltzmann Machines (RBMs) are a key building block of deep belief networks. RBM parameters are typically trained to maximize likelihood. Exact likelihood training is intractable and therefore requires approximations such as contrastive divergence. Instead, we propose to use Bayesian learning methods to estimate the full posterior over RBM parameters. We apply message passing using a deterministic approximation called Power EP, which has been shown to be faster than sampling-based methods for a given accuracy. At test time, model averaging over the parameter posterior can be done analytically, so is very efficient. This framework also provides a way to estimate the evidence, which can be used to perform model selection, such as selecting the number of hidden units in the RBM.
BACK
