Machine Learning II (2008/2009)

Advanced Markov Chain Monte Carlo

This week we will have a coding weekend, where you will form teams and code up:

• A Gibbs sampler for a mixture model;
• Annealed importance sampling (AIS);
• Reversible jump MCMC (RJMCMC).

The idea is to do Bayesian model averaging. You can use AIS to estimate the evidence of data given the model size (number of components). From this you can estimate the posterior distribution over models. Independently, you can use RJMCMC to directly get samples from the posterior. As both approaches are pretty different, if you can get both to agree on the posterior over model size, there is a good chance that you would have gotten the programming for both correct.

On Friday 130-300pm we will split you up into groups of three students and set you up with work package. Yee Whye and Maneesh will give introductory lectures. You will have Friday and over the weekend to work on this. Both will be available to answer questions via email, and Yee Whye will be available on Monday and Tuesday at Gatsby. We will reconvene at 11am on Tuesday to discuss results and wrap up.

Slides:
[rjmcmc.pdf] [ais.pdf]

Package:
[tgz] [zip]

Reading material:

• Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Peter J. Green. Biometrika 82(4):711-732 (1995).
• Annealed Importance Sampling. Radford M. Neal. Statistics and Computing 11:125-139 (2001).

• The Harmonic Mean of the Likelihood: Worst Monte Carlo Method Ever. Radford Neal's blog.
• Evaluating probabilities under high-dimensional latent variable models. Iain Murray and Ruslan Salakhutdinov. NIPS 21 (2009).
• Marginal Likelihood from the Gibbs Output. Siddhartha Chib. JASA 90(432):1313-1321 (1995).
• Bayesian Inference in Hidden Markov Models through the Reversible Jump Markov Chain Monte Carlo Method. Christian P. Robert, Tobias Ryden and D. M. Titterington. JRSSB 62(1):57-75 (2000).
• A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model. Sonia Jain and Radford M. Neal. JCGS 13(1):158-182 (2004).