UCL logo
skip to navigation. skip to content.

Gatsby Computational Neuroscience Unit




UCL Home
  • UCL Home
  • UCL Gatsby Computational Neuroscience Unit
UCL Gatsby Unit
  • introduction
  • people
  • research
  • publications
  • courses
  • phd programme
  • events
  • directions
  • greater gatsby
  • vacancies
  • Internal
  • ucl

 

 

  • Home
  • Staff & Students
  • Vacancies

 

Probabilistic and Unsupervised Learning
Approximate Inference and Learning in Probabilistic Models
(2013)

Dates Mondays & Thursdays
30 September - 4 December 2013
Time 11:00-13:00
Tutorials Wednesdays 9:00-11:00
Lecturer Maneesh Sahani
TAs Vincent Adam, Tom Haines, Tom Furmston
Location Basement B10 Seminar Room, Alexandra House, 17 Queen Square, London WC1N 3AR
About the course

This course (offered as two successive modules to MSc students) provides an in-depth introduction to statistical modelling, unsupervised, and some supervised learning techniques. It presents probabilistic approaches to modelling and their relation to coding theory and Bayesian statistics. A variety of latent variable models will be covered including mixture models (used for clustering), dimensionality reduction methods, time series models such as hidden Markov models which are used in speech recognition and bioinformatics, Gaussian process models, independent components analysis, hierarchical models, and nonlinear models. The course will present the foundations of probabilistic graphical models (e.g. Bayesian networks and Markov networks) as an overarching framework for unsupervised modelling. We will cover Markov chain Monte Carlo sampling methods and variational approximations for inference. Time permitting, students will also learn about other topics in probabilistic (or Bayesian) machine learning.

The complete course forms a component of the Gatsby PhD programme, and is mandatory for Gatsby students. The two component modules are also available to students on the UCL MSc in Machine Learning and UCL MSc in Computational Statistics and Machine Learning. The first part (GI18: Probabilistic and Unsupervised Learning) may be used to fill a core requirement in each Masters programme. The second part (GI16: Approximate Inference and Learning in Probabilistic Models) is an optional module, but is only available to students who have completed GI18.

Students, postdocs and faculty from outside the unit and these programmes are welcome to attend, but shouldcontact the unit in advance.

Prerequisites

A good background in statistics, calculus, linear algebra, and computer science. You should thoroughly review the maths in the following cribsheet [pdf] [ps] before the start of the course. This quiz may help you check where you stand. You must either know Matlab or Octave, be taking a class on Matlab/Octave, or be willing to learn it on your own. Any student or researcher at UCL meeting these requirements is welcome to attend the lectures.

Text

There is no required textbook. However, the following are excellent sources for many of the topics covered here.

David J.C. MacKay (2003) Information Theory, Inference, and Learning Algorithms, Cambridge University Press.
Christopher M. Bishop (2006) Pattern Recognition and Machine Learning, Springer-Verlag.
David Barber (2012) Bayesian Reasoning and Machine Learning, Cambridge University Press.
Kevin Patrick Murphy (2012) Machine Learning: a Probabilistic Perspective, MIT Press.
Daphne Koller and Nir Friedman (2009) Probabilistic Graphical Models, MIT Press. (This contains a more extensive treatment of graphical models, good for reference)
Reading material

Specific recommendations can be found here and will be updated as course progresses.

Lecture schedule and slides Lecture slides and assignments will be posted here as the course progresses. Note that the content and sequence of lectures may vary slightly from those of previous years. Also note that, following the usual Gatsby teaching pattern, there are lectures scheduled during "reading week".

 

GI18
30/9: Intro / screen version 3/10: Intro contd., Latent Variables / screen version
7/10: Latent Variables contd. 10/10: tutorial
14/10: Expectation Maximisation / screen version 17/10: tutorial
21/10: Latent chain models / screen version 24/10: Latent chain models contd.
28/10: no lecture 31/10: Graphical models / screen version
4/11: Graphical models contd. 7/11: Model selection and GPs / screen version
GI16
11/11: Intractable models / screen version 14/11: Variational methods / screen version
18/11: Expectation Propagation / screen version 21/11: Belief Propagation / screen version
25/11: Convexity-based Methods / screen version 28/11: Sampling / screen version
2/12: Sampling contd. 5/12: no lecture
9/12: no lecture 12/12: no lecture

Assignments
50% of the total mark (for each module) is based on coursework assignments (the other 50% being based on the final written examination).

All assignments (coursework) are to be handed in to the Gatsby Unit, not to the CS department. Please hand in all assignments at the beginning of lecture on the due date to the lecturer or to the TAs. Late assignments will be penalised. If you are unable to attend the lecture, you can also hand in assignments to Mr. Barry Fong at the Alexandra House 4th floor reception.

Late Assignment Policy: we will be using the official UCL-wide late policy. Marks will be reduced by 5 for assignments up to one day late, 15 for assignments between one and seven days late (or before the answers are discussed in tutorial, whichever comes first), and will receive no credit thereafter.

Assignment 0 - maths quiz to be discussed in tutorial 2 Oct 2013
Assignment 1        [binarydigits.txt,  bindigit.m] due 14 Oct 2013
Assignment 2        [geyser.txt] due 4 Nov 2013
Assignment 2 bonus question
[ssm_spins.txt ssm_spins_test.txt ssm_kalman.m ]
due 18 Nov 2013
Assignment 3        [co2.txt] due 14 Nov 2013
Assignments 4 and 5        [images.jpg, genimages.m, Mstep.m] due 9 Dec 2013
Assignment 6        [message.txt, symbols.txt, asst6code.tgz] due 13 Jan 2014

To attend Please  contact  the unit.


 

 

  • Disclaimer
  • Freedom of Information
  • Accessibility
  • Privacy
  • Advanced Search
  • Contact Us
Gatsby Computational Neuroscience Unit - Alexandra House - 17 Queen Square - London - WC1N 3AR - Telephone: +44 (0)20 7679 1176

© UCL 1999–20112011