Gatsby Computational Neuroscience Unit

This course (offered as two successive modules to MSc students) provides an in-depth introduction to statistical modelling, unsupervised, and some supervised learning techniques. It presents probabilistic approaches to modelling and their relation to coding theory and Bayesian statistics. A variety of latent variable models will be covered including mixture models (used for clustering), dimensionality reduction methods, time series models such as hidden Markov models which are used in speech recognition and bioinformatics, Gaussian process models, independent components analysis, hierarchical models, and nonlinear models. The course will present the foundations of probabilistic graphical models (e.g. Bayesian networks and Markov networks) as an overarching framework for unsupervised modelling. We will cover Markov chain Monte Carlo sampling methods and variational approximations for inference. Time permitting, students will also learn about other topics in probabilistic (or Bayesian) machine learning.

The complete course forms a component of the Gatsby PhD programme, and is mandatory for Gatsby students. The two component modules are also available to students on Machine Learning related MSc programmes. Note that MSc student may only take the second part (Approximate Inference and Learning in Probabilistic Models) if they complete the first.

UCL Students, postdocs and faculty from outside the unit and these programmes are welcome to attend the lectures as long as space is available. If you are not a student on either MSc programme above, please do not register officially for the course without consulting the course administrator.

A good background in statistics, calculus, linear algebra, and computer science. You should thoroughly review the maths in the following cribsheet [pdf] [ps] before the start of the course. This quiz may help you check where you stand. You must know Matlab, Octave or Python/NumPy.

There is no required textbook. However, the following are excellent sources for many of the topics covered here.

David J.C. MacKay (2003) Information Theory, Inference, and Learning Algorithms, Cambridge University Press.

Christopher M. Bishop (2006) Pattern Recognition and Machine Learning, Springer-Verlag.

David Barber (2012) Bayesian Reasoning and Machine Learning, Cambridge University Press.

Kevin Patrick Murphy (2012) Machine Learning: a Probabilistic Perspective, MIT Press.

Daphne Koller and Nir Friedman (2009) Probabilistic Graphical Models, MIT Press. (This contains a more extensive treatment of graphical models, good for reference)

Some of our work will depend numerical computation. A very useful reference in this area is:

Gene H. Golub and Charles F. Van Loan (2013) Matrix Computations, 4th ed, Johns Hopkins Univ. Press

Specific recommendations for additional reading can be found here. New material may be assembled as course progresses.

Assignments (coursework) are due before the beginning of lecture on the date specified (except Assignement 6, which is due at 5pm). They must be submitted online if you are registered for the course through an MSc programme. Late assignments will be penalised For Gatsby, SWC and related students, they should be handed in by email or on paper to a Gatsby TA.

Late Assignment Policy: we will be using the official UCL-wide late policy.