Unravelling the way neurons encode and decode sensory information is one of the challenges of neuroscience. Understanding the mechanism of translation between the visual experience in the physical world and the neural code is a key part of this and remains an open question, as well as the identification of the essential features to encode and decode information. The usual approach to this topic is to investigate encoding strategies, i.e. how an image can be efficiently represented using neural activity, or to perform an identification task, in which the aim is to identify which of a predefined set of stimuli elicited a specific neural response pattern. A few studies have employed a more challenging approach, namely the development of an algorithm for reconstructing visual stimuli given the neural response activity pattern, measured with fMRI recordings in humans. However, fMRI data currently lack the spatial and temporal resolution provided by in vivo techniques, such as two-photon calcium imaging. Moreover, the final result of such approach, produced by a weighted average of images taken from a previously defined set, makes it more like an identification task. Another study [1] used electrophysiological data, although recorded from the LGN, whose encoding strategies might differ from those of V1. In this work, we present a technique for visual stimuli reconstruction which is based on the application of a forward receptive field based model to estimate the predicted neural response given a specific input. An inverse model is then used to recover which stimulus gave rise to a certain response pattern. We search for the optimal reconstruction in the pixel space. A training and a test dataset are used to determine the parameters of the forward model and to evaluate the quality of the reconstruction, respectively. We apply the algorithm to decode two-photon calcium imaging data from mouse V1 and we quantify its performances using a structural similarity measure between the true and the reconstructed image. The use of an explicit forward model allows us to test which properties of the system are the most important in terms of the quality of the reconstruction and therefore for information transmission. To do this, we use different receptive field models with various degrees of biological accuracy, such as the Gabor and the Berkeley Wavelet Transform, and we compare their performance. Having access to a spatial resolution at the level of individual neurons, allows us to investigate the effect on the reconstruction performance of the spatial pooling of the neural response, as occurs in fMRI data, in comparison to a labelled line code. The algorithm is currently used for single frame reconstruction, however the framework is suitable for extending this algorithm to the online decoding of moving images.

[1] G.B. Stanley, F.F. Li and D. Yang, J. Neurosci. 19(18):8036-8042 (1999).