Unsupervised Learning (2006)

Dates Mondays & Thursdays
2 October - 14 December 2006
Time 11:00-13:00
Lecturer Maneesh Sahani
TA Kai Krueger
Location 4th Floor Seminar Room, Alexandra House, 17 Queen Square, London [directions]
About the course

This course provides students with an in-depth introduction to statistical modelling and unsupervised learning techniques. It presents probabilistic approaches to modelling and their relation to coding theory and Bayesian statistics. A variety of latent variable models will be covered including mixture models (used for clustering), dimensionality reduction methods, time series models such as hidden Markov models which are used in speech recognition and bioinformatics, independent components analysis, hierarchical models, and nonlinear models. The course will present the foundations of probabilistic graphical models (e.g. Bayesian networks and Markov networks) as an overarching framework for unsupervised modelling. We will cover Markov chain Monte Carlo sampling methods and variational approximations for inference. Time permitting, students will also learn about other topics in probabilistic (or Bayesian) machine learning.

The course is run primarily for new Gatsby students for whom it is mandatory, and for students on the CS MSc in Intelligent Systems, for whom it is optional. Students, postdocs and faculty from outside the unit and this programme are welcome to attend, but should contact the unit in advance.


A good background in statistics, calculus, linear algebra, and computer science. You should thoroughly review the maths in the following cribsheet [pdf] [ps] before the start of the course. You must either know Matlab or Octave, be taking a class on Matlab/Octave, or be willing to learn it on your own. Any student or researcher at UCL meeting these requirements is welcome to attend the lectures.


There is no required textbook. However, the following in an excellent sources for many of the topics covered here.

David J.C. MacKay (2003) Information Theory, Inference, and Learning Algorithms, Cambridge University Press. (also available online)
Lecture schedule
2/10: Introduction 5/10: Statistical Foundations
9/10: Latent Variable Models 12/10: The EM algorithm
16/10: no lecture 19/10: no lecture
23/10: EM continued 26/10: Latent time-series models
30/10: Time-series continued    
    2/11: Introduction to graphical models
6/11: no lecture 9/11: no lecture
13/11: Graphical models contd. [screen version] 16/11: continued
20/11: Hierarchical and Nonlinear Models 23/11: Sampling and Monte Carlo
27/11: continued 30/11: Variational approximations
4/12: continued 7/12: no lecture
11/12: no lecture 14/12: Bayesian Model Comparison
50% of the total mark is based on coursework assignments (the other 50% being based on the final written examination).

All assignments (coursework) are to be handed in to the Gatsby Unit, not to the CS department. Please hand in all assignments at the beginning of lecture on the due date to either Maneesh or Kai. Late assignments will be penalised. If you are unable to come to class, you can also hand in assignments to Ms. Rachel Howes at the Alexandra House 4th floor reception.

Late Assignment Policy: Assignments that are handed in late will be penalised as follows: 10% penalty per day for every weekday late, until the answers are discussed in a review session. NO CREDIT will be given for assignments that are handed in after answers are discussed in the review session.

Assignment 1 due 12 Oct 2006
Assignment 2        [binarydigits.txt, bindigit.m] due 2 Nov 2006
Assignment 3        [geyser.txt, data1.txt] due 16 Nov 2006
Assignment 4 due 4 Dec 2006
Assignment 5        [images.jpg, genimages.m, Mstep.m] ` due 14 Dec 2006

To attend Please contact the unit.