Novel approach to estimation of entropies of discrete variables with applications to neural coding

Ilya Nemenman1 and William Bialek2

2Princeton University

We will review a recently introduced estimator of entropies for undersampled discrete nonmetric variables, which is based on the Occam factors inspired averaging over a set of popular Dirichlet priors. We will analyze performance of the estimator numerically for simple toy problems and analytically for various asymptotic regimes. One conclusion is that, at least for some classes of probability distributions involved, the estimator is nearly unbiased and performs well even with no a priori assumptions about the cardinalities of variables being studied. We will present some preliminary results from applications of the method to the estimation of information in spike trains from the H1 neuron in a fly visual system.