Neuroscience Research

Computer vision and computer hearing researchers must pay attention to biology. There are two main reasons for this. First, signals often have to be processed for consumption by the human perceptual system, and we need to study that system to figure out the best format of the information. For example, the fact that televisions, cameras and other displays and sensors are based around the RGB-color model is a direct consequence of the fact that the eye contains three different color sensitive cone photoreceptors. Similarly, the MP3 format used to code music, speech and other audio signals is based upon perceptual experiments that determine which combinations of sounds people can distinguish, and which they cannot. A quiet sound can be masked by a loud sound, for instance, and so the quiet sound need not be coded, and it can be thrown away thereby making the file more compressed.


There is a second reason why computer vision and computer hearing researchers must pay attention to biology: Biological systems routinely solve complicated tasks like visual and audio recognition with a level of performance that far exceeds that possible by machine. By figuring out the principles underpinning the success of these biological systems, we can hope to build better artificial systems. Currently, although many artificial systems are based upon principles which appear to be used by the brain (e.g. object recognition systems and machine hearing systems), I believe that engineers and biologists still have a lot to learn from one another.

My research in Computational Neuroscience has sought to understand the principles underpinning computations performed by the brain. (That's Computational neuroscience with a Big-C.) For example, the black and white figure on the right shows the signal to which a neuron in primary visual cortex responds most strongly - the neuron is sensitive to an oriented edge. The colour plots below the figure show the preferred stimuli for five other neurons, which also take the form of oriented edges. It appears as if primary visual cortex is detecting the edges present in the incoming images - but why is this a sensible thing to do?


Jones & Palmer, Journal of Neurophysiology, 1987

Ringach, Journal of Neurophysiology, 2002

One influential idea is that images are composed of basic structural primitives, things like line segments or oriented edges, and it is the presence or absence of these structural primitives which neurons are representing. One way of testing this idea is to determine the form of these structural primitives from a pile of images, and to compare the structural primitives to the neurons in visual cortex. How can we do this?

Presumbably, there are a vast number of structural primitives used across all images. For example, line segments at all possible orientations, scales, and positions. However, in any one image only a small number will be present. That is, the structural primitives' activites are sparse variables which are either highly active or highly inactive. This signature can be used to learn the structural primitives: we find the set of structural primitives which describe a set of images in the sparsest way possible.

Amazingly, when this is done, the resulting structural primitives (shown on the right in grey) are found to be oriented edges which closely resemble the features to which neurons in visual cortex are sensitive too. The conclusion is that visual cortex is decomposing images in terms of the structural primitives out of which images are composed, and that these primitives are oriented edges.

Similar representations, inspired by visual cortex, are now used in computer vision systems.


Olshausen and Field, Nature, 1996; Turner et al, submitted, 2012

Related papers

Related talks



Return to main page