Optimising Synaptic Learning Rules in
Linear Associative Memories
Peter Dayan David Willshaw
Biological Cybernetics, 65, 253-265.
Abstract
Associative matrix memories with real-values synapses have been
studied in many forms. The authors consider how the signal/noise ratio
for associations depends on the form of the learning rule, and they
show that a covariance rule is optimal. Two other rules, which have
been suggested in the neurobiology literature, are asymptotically
optimal in the limit of sparse coding. The results appear to
contradict a line of reasoning particularly prevalent in the physics
community. It turns out that the apparent conflict is due to the
adoption of different underlying models. Ironically, they perform
identically at their coincident optima. The authors give details of
the mathematical results, and discuss some other possible derivations
and definitions of the signal/noise ratio.
pdf