Bayesian Retrieval in Associative Memories
with Storage Errors.
Fritz Sommer Peter Dayan
IEEE Transactions in Neural Networks, 9, 705-713.
Abstract
It is well known that for finite-sized networks, one-step retrieval in
the autoassociative Willshaw net is a suboptimal way to extract the
information stored in the synapses. Iterative retrieval strategies are
much better, but have hitherto only had heuristic justification. We
show how they emerge naturally from considerations of probabilistic
inference under conditions of noisy and partial input and a corrupted
weight matrix. We start from the conditional probability distribution
over possible patterns for retrieval. We develop two approximate, but
tractable, iterative retrieval methods. One performs maximum
likelihood inference to find the single most likely pattern, using the
conditional probability as a Lyapunov function for retrieval. The
second method makes a mean field assumption to optimize a tractable
estimate of the full conditional probability distribution. In the
absence of storage errors, both models become very similar to the
Willshaw model.
pdf