9th May 2005 — Iterative fitting and scaling
Iain covered “Iterative Proportional Fitting” (IPF) and “Generalized Iterative Scaling” (GIS).
Original papers: GIS, “Generalized Iterative Scaling for Log-Linear Models” J. N. Darroch; D. Ratcliff The Annals of Mathematical Statistics, Vol. 43, No. 5. (Oct., 1972), pp. 1470-1480.; IPF, “On a least squares adjustment of a sampled frequency table when the expected marginal totals are known” WE Deming, FF Stephan, Annals of Mathematical Statistics, 1940.
Tom Minka runs comparisons of many algorithms including GIS variants on logistic regression.
We alluded to Yee Whye Teh and Max Welling’s Unified propagation and scaling. Is this an area where intelligent message passing can beat gradient methods?