Notes

This page contains a collection of notes written on various subjects. These are often rather incomplete documents, for which I apologise. Nevertheless, I think there are some interesting nuggets in them that make them worth sharing.

• A review of parameter learning methods based on approximate versions of the method of moments
• There is a folk-theorem which says, "Variational approximations are always more compact than the true distribution". In this note we provide counter examples which disprove this folk theorem.
• Probilistic Amplitude and Frequency Demodulation: On the equivalence of the Dynamic Harmonic Model and Bayesian Spectral Estimation.
• Population codes: a literature review including distributional population codes (DPCs), DDPCs, and Alex Pouget and Peter Latham's work.
• A useful tensor notation: which can handle the operators like: Entry-wise product, diag, Kronecker product, vec, MATLABs reshape etc.
• K-Step Learning or Contrastive Divergence: Some notes on contrastive divergence for the Machine Learning Journal Club.
• Free Energies: Various free energies in statistical physics and inference.
• PCA 1: An in depth look at probablistic principal components analysis
• PCA 2: Another version of probablistic principal components analysis
• The Wiener Filter: Bayesian and non-Bayesian theory and some illustrative applications in neuroscience.
• The Kalman Fitler: An attempt to give some intuition about linear Gaussian state space models via a simple 1d example. Followed by a section on the relation to Gaussian Processes and the Wiener filter.
• The 2 Envelopes Paradox: You can pick one of two envelopes. One contains twice the contents of the other. You choose one, open it, and see it contains $Y. Should you change your choice of envelope?
• In the stamp problem you have two types of stamps worth A pence and B pence and you can put as many as you like onto an envelope. The task is to find the expression for the largest total value that it is impossible to make N(A,B). e.g. if A=7, B=5, the largest unmakeable value is N=23.

Return to main page