Machine Learning II (2007/2008)

5/09 Convex Optimization

Convex optimization has in recent years become an indispensible technique in many branches of machine learning. As opposed to general optimization problems, globally optimal solutions can be found efficiently for convex problems.

This lecture is meant to introduce some of the basic concepts of convex analysis and optimization: convex sets, convex functions, convex optimization problems, and duality theory. I hope to also go through derivations of SVMs and/or Wainwright et al's tree-structured upper bound on the log partition functions of MRFs.

Reading material:

• Boyd and Vandenberghe
  • slides
  • book
• Wainwright, Jaakkola, Willsky. UAI 2002. A New Class of Upper Bounds on the Log Partition Function.