A New Learning Algorithm for Mean Field Boltzmann Machines

Max Welling and G.E. Hinton
Gatsby Computational Neuroscience Unit

GCNU TR 2001-002

Abstract

We present a new learning algorithm for Mean Field Boltzmann Machines based on the contrastive divergence optimization criterion. In addition to mini-mizing the divergence between the data distribution and the equilibrium dis-tribution that the network believes in, we maximize the divergence between one-step reconstructions of the data and the equilibrium distribution. This eliminates the need to estimate equilibrium statistics, so we do not need to ap-proximate the multimodal probablility distribution of the free network with the unimodal mean field distribution. We test the learning algorithm on the classification of digits.

---

Download: [ps.gz] or [pdf]