Up
Previous
Next
Novel approach to estimation of entropies of discrete variables with
applications to neural coding
Ilya Nemenman1 and William Bialek2
1UCSB
2Princeton University
We will review a recently introduced estimator of entropies for
undersampled discrete nonmetric variables, which is based on the Occam
factors inspired averaging over a set of popular Dirichlet priors. We
will analyze performance of the estimator numerically for simple toy
problems and analytically for various asymptotic regimes. One
conclusion is that, at least for some classes of probability
distributions involved, the estimator is nearly unbiased and performs
well even with no a priori assumptions about the cardinalities of
variables being studied. We will present some preliminary results from
applications of the method to the estimation of information in spike
trains from the H1 neuron in a fly visual system.