Machine Learning II (2008/2009)

Dates Fridays
6 March - 29 May 2009
Time 13:00-15:00
Organizers Yee Whye Teh (main contact) and Maneesh Sahani.
Location 4th Floor Seminar Room, Alexandra House, 17 Queen Square, London [directions]
About the course

This is a topics-oriented course covering important material left out in the Gatsby Probabilistic and Unsupervised Learning course, and is targeted at students interested in machine learning research. The aim of the course is to impart both a broader knowledge of the field, as well as in-depth knowledge of certain current advanced techniques as used in machine learning.

The course is run as a number of independent classes covering a range of topics. Some modules are run as taught lectures, others seminar style (presented by students), or as coding days (morning lecture followed by a day of implementing the algorithm). Topics vary from year to year. See 2007/2008 course here.

The course is run primarily for new Gatsby machine learning students for whom it is mandatory. Students, postdocs and faculty from both within and outside the unit are welcome to attend, but should contact Yee Whye in advance.


A good background in statistics, calculus, linear algebra, and computer science, as well as familiarity with the material in the Probabilistic and Unsupervised Learning course. The cribsheet [pdf] [ps] gives some useful mathematical knowledge required for the course. Knowledge of Matlab, Octave or some other development language will be required for the coding days.


There is no required textbook. However, the following are excellent introductory texts for many of the topics covered here.

David J.C. MacKay (2003) Information Theory, Inference, and Learning Algorithms, Cambridge University Press. (also available online).
Christopher M. Bishop (2006) Pattern Recognition and Machine Learning, Springer-Verlag.
Stephen Boyd and Lieven Vandenberghe (2004) Convex Optimization, Cambridge University Press. (also available online).
Bernhard Scholkopf and Alex Smola (2002) Learning with Kernels, MIT Press.
Specific reading are listed in the schedule below. Information on specific modules will be updated later.
Topics Bayesian Nonparametrics (x4)
Computational Learning Theory
VC Theory and Concentration of Measures
Hilbert Space Learning Framework
Advanced Monte Carlo Sampling (x2)
Choice Models
Deep Learning
Lecture schedule
3/06: Introduction to Bayesian Nonparametrics Yee Whye
3/13: Foundations of Nonparametric Bayes Peter Orbanz
3/20: Foundations of Nonparametric Bayes Peter Orbanz
3/27: Foundations of Nonparametric Bayes Peter Orbanz
4/03: Coding Weekend: Annealed Importance Sampling and Reversible Jump MCMC Yee Whye and Maneesh
4/10: No class (Easter)
4/17: Computational Learning Theory Risi
4/24: VC Theory and Concentration of Measures Risi
5/01: Hilbert Space Learning Framework Risi
5/08: Choice Models Dilan
5/15: Gradient Descent and other Optimization Methods Nicolas Le Roux
5/27: Note time change. Bayesian/Frequentist Debate n/a
5/29: TBA TBA
To attend Please contact Yee Whye.