Machine Learning Research

In many machine perception problems, the set of inputs which are likely to be encountered is limited. Consequently, generic and untuned representations are wasteful because they allocate representational resources to atypical inputs. Moreover, generic representations do not take noise and other distortions into account, and are therefore comparatively brittle. My research programme addresses these two key limitations by developing adaptive and robust representations for visual and audio signals. Establishing a good representation is arguably the key step in building successful computer vision and audition algorithms, and so these methods have wide applicability.


Modern machine learning approaches sit at the core of my research programme because they provide automatic methods for adapting representations to match the statistics of the input, and because they handle noise corrupted signals gracefully by maintaining a representation of the associated uncertainty. More specifically, I use the Bayesian approach through which uncertainty is handled using the rules of probability.

My machine learning projects include:


Bayesian Signal Processing. I view Machine Learning and Signal Processing as two sides of the same coin: they are both interested in making inferences from data. Traditionally signal processing has focussed on efficient, and therefore often feedforward methods for processing the data. Whereas machine learning has focussed on more complicated and more computationally intensive methods. I have established concrete theoretical connections between the two fields. For instance, I have shown that classical signal processing methods for time-frequency analysis and demodulation, are equivalent to Bayesian inference problems. This break through has allowed techniques from both fields to be combined, thereby improving on both approaches.


Approximate inference. The quantities of interest in Bayesian inference are often hard to compute mathematically or on a computer (i.e. they are often analytically and computationally intractable). Therefore, a large part of a Bayesian's time is spent devising fast and accurate approximations. I use and develop a suite of different approximation methods including, variational free-energy methods, expectation propagation, Markov chain Monte Carlo, and moment-matching schemes.


Circular Variables. Circular variables (i.e. angular variables that range between between π and -π) show up all over the place. They arise naturally in signal processing, neuroscience, brain recording data, geophysics, engineering, and any place where complex variables are used. Often they form a time-series like a succession of wind direction measurements, a set of joint angle measurements during a movement, or a time-varying complex variable such as a wavelet coefficient. I build statistical models for time-series of circular variables which can remove noise and impute missing data (e.g. data which is corrupted due to a faulty sensor) and which can adaptively process the data in an efficient manner.


Related papers

Related talks



Return to main page