25th October 2005 — Unconstrained continuous optimization
Ed and Iain will present some methods for unconstrained continuous optimization.
Neural networks for pattern recognition, Chris Bishop, 1995. Chapter 7.
- An alternative viewpoint, focussing on stochastic gradient optimization, is provided by: Genevieve B. Orr, Klaus-Robert Müller (Eds.): Neural Networks: Tricks of the Trade, this book is an outgrowth of a 1996 NIPS workshop. Lecture Notes in Computer Science 1524 Springer 1998, ISBN 3-540-65311-2
- The Nelder-Mead Simplex/Amoeba method, and other things covered above, are in Numerical recipes in C, William H. Press, Saul A. Teukolsky, William T. Vetterlin and Brian P. Flannery, Cambridge University Press, 2002.
- The documentation for the Matlab Neural net toolbox has a section on optimization experiments, and comes to some vague conclusions.
- We mentioned: Beyond Newton’s method, Thomas P. Minka, 2002, available online.