Machine Learning II (2007/2008)

4/18 4/25 Nonparametric Bayes

We introduce the nonparametric Bayesian models popular in machine learning in this two lectures.

In the first lecture we start with an introduction to the Dirichlet process. We start with some intuition concerning the need for nonparametric models, and the idea of partitions of data. From there the various perspectives on the Dirichlet process are presented, specifically the Chinese restaurant process and the stick-breaking construction. We then cover the hierarchical Dirichlet process, the nested Dirichlet process, and the Indian Buffet process.

Slides:

• Nonparametric Bayesian Modelling.

Reading material:

• Introduction to Dirichlet Processes by Yee Whye.
• Mike Jordan's NIPS 2005 tutorial notes.
• Zoubin Ghahramani's UAI 2005 tutorial notes.
• Hierarchical Dirichlet Processes. Teh et al (2006) JASA 101(476):1566-1581.
• Nested Dirichlet Processes. Rodriguez et al (2006) Duke Statistics Technical Report.
• Indian Buffet Processes. Ghahramani (2007) Bayesian Statistics 8.

In the second week students will present the following papers:

• Infinite Relational Model. Kemp et al (2006) UAI.
• Infinite Hidden Markov Model. Beal et al (2001) NIPS.
• Nested Chinese Restaurant Processes. Blei et al (2003) NIPS.
• Dirichlet Diffusion Trees. Neal (2003) Bayesian Statistics 7.