We propose to learn the structure of data based on the simple idea of learning to discriminate data from artificially generated noise. Such learning is basically unsupervised, although we formulate it as logistic regression. The method can be shown to estimate a parametric probabilistic model for the data. Furthermore, the probabilistic model does not need to be normalized (i.e. it can be energy-based) because the normalization parameter can actually be estimated just like the other parameters. We apply the method to learning two- and three-layer networks from natural images, where the noise is white and gaussian.
BACK
