25th October 2005 — Unconstrained continuous optimization
Ed and Iain will present some methods
for unconstrained continuous optimization.
Neural networks for
pattern recognition, Chris Bishop, 1995. Chapter 7.
- An alternative viewpoint, focussing on stochastic gradient optimization, is
Genevieve B. Orr, Klaus-Robert Müller (Eds.): Neural
Networks: Tricks of the Trade, this book is an outgrowth of a 1996 NIPS
workshop. Lecture Notes in Computer Science 1524 Springer 1998, ISBN
- The Nelder-Mead Simplex/Amoeba method, and other things covered above, are in
Numerical recipes in
C, William H. Press, Saul A. Teukolsky, William T. Vetterlin and Brian P.
Flannery, Cambridge University Press, 2002.
- The documentation for the Matlab Neural
net toolbox has a section on optimization experiments,
and comes to some vague conclusions.
- We mentioned:
Newton’s method, Thomas P. Minka, 2002, available online.