Abstract
This paper presents a novel practical framework for Bayesian model
averaging and model selection in probabilistic graphical models. Our approach
approximates full posterior distributions over model parameters and structures, as well as
latent variables, in an analytical manner. These posteriors fall out of a free-form
optimization procedure, which naturally incorporates conjugate priors. Unlike in
large sample approximations, the posteriors are generally non-Gaussian and no Hessian
needs to be computed. Predictive quantities are obtained analytically. The
resulting algorithm generalizes the standard Expectation Maximization algorithm, and its
convergence is guaranteed. We demonstrate that this approach can be applied to a
large class of models in several domains, including mixture models and source separation.
Download [ps] [pdf]