Probabilistic and Unsupervised Learning - Additional material

You'll find here some additional material to help you understand the course. We'll also provide you with the slides used during tutorials. Please note that we will not publish corrections for assignments online.

Additional reading

Please be warned that these are *recommended* reading.
For example, you shouldn't expect the assignments or exams to cover only the chapters in Bishop that we list here. The same is true for any item of this list, you are expected to find additional material about the lecture by yourself when needed.

These are compiled from previous years pages and courses.

Topic Additional reading
Statistical Foundations
  1. Cribsheet by Iain Murray
  2. Matrix Identities by Sam Rowei
  3. Matrix Cookbook by K. Petersen and M. Pedersen
  4. Tom Minka's notes on matrix algebra
  5. Bishop: Chapter 1 and 2.
  6. David MacKay's book: Chapters 1-3 and 22-23. Appendices B, C
  7. Nuances of Probability Theory by Tom Minka.
  8. Probability Theory: The Logic of Science by ET Jaynes
  9. Probability and Statistics Online Reference
Latent Variable Models
  1. Bishop: Chapters 3 and 12.
  2. David MacKay's book: Chapters 20-22, 34.
  3. Max Welling's Class Notes on PCA and FA [pdf]
  1. Bishop: Chapter 9
Time series models
  1. Bishop: Chapter 13
  2. Minka, T. (1999) From Hidden Markov Models to Linear Dynamical Systems
  3. Welling notes on HMM and Kalman Filter
  4. Rabiner's tutorial on HMM
Graphical models
  1. Bishop: Chapter 8
  2. MacKay: Chapter 16, 26
  3. David Heckermann's Tutorial on Graphical Models
Exact Bayes
  1. Bishop: Chapter 6, 4.4, 3.4-3.5
  2. MacKay: Chapter 27-28, 45
  3. Carl Rasmussen's tutorial slides on Gaussian Processes
  4. A great textbook about Gaussian Processes, available online: Gaussian processes for Machine Learning by Rasmussen and Williams
Expectation Propagation
  1. Divergence measures and message passing (Minka 2005)
  2. Expectation propagation for exponential families (Seeger 2008)
  3. Graphical models, exponential families and variational inference (Wainwright, Jordan 2008)
  4. Notes on EP by Lloyd Elliott.


Here are the slides used during tutorials (when available).
Corrections of assignments will be discussed during the tutorials but not posted here.

Tutorial dateSlides
6.10Lloyd's slides, Loic's slides
13.10Lloyd's slides
20.10Loic's slides
18.11Lloyd's slides