Math 273, Section 1, Fall 2009

Optimization, Calculus of Variations, and Control Theory

Lecture Meeting Time: MWF 12.00PM - 12:50PM.
Lecture Location: MS 5127.

Instructor: Luminita A. Vese
Office: MS 7620 D
Office hours: Mon 4-6pm, or by appointment.
E-mail: lvese@math.ucla.edu

General Course Description: Application of abstract mathematical theory to optimization problems of calculus of variations and control theory. Abstract nonlinear programming and applications to control systems described by ordinary differential equations, partial differential equations, and functional differential equations. Dynamic programming.

References:
  • J. Nocedal and S.J. Wright, Numerical Optimization, Springer Series in Operations REsearch, Springer 1999 (1st or 2nd edition).
    Online access restricted to UC campuses (2nd edition)
  • I. Ekeland and R. Temam, Convex Analysis and Variational Problems, SIAM, 1999 (new edition), placed on rserve at SEL Library.
  • E. Zeidler, Nonlinear Functional Analysis and its Applications, Vol. III, Variational Methods and Optimization , Springer-Verlag 1984.
  • P.E. Gill, W. Murray, and M.H. Wright, Practical Optimization, Academic Press 1981.
  • R.T. Rockafellar, Convex Analysis, Princeton University Press 1970.
  • J.-B. Hiriart-Urruty, C. Lemarechal, Fundamentals of Convex Analysis, Springer 2001.
  • S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004 (especially Chapters 9, 10 and 11).
  • M. Giaquinta, S. Hildebrandt, Calculus of variations, Springer, 1996 (two volumes).
  • D. Luenberger, Optimization by Vector Space Methods , John Wiley & Sons, 1969.
  • Dimitri P. Bertsekas, with Angelia Nedic and Asuman E. Ozdaglar, Convex Analysis and Optimization .
  • L.C. Evans, Partial Differential Equations , Chapter 8.
  • H. Attouch, G. Buttazzo, and G. Michaille, Variational Analysis in Sobolev and BV Spaces: applications to PDE's and optimization, MPS-SIAM 2006.

    Specific topics will be listed here:
  • Formulation of a general finite dimensional optimization problem with constraints; objective function, equality or inequality constraints, feasible region, example.
  • Gateaux-differentiability, computation of Euler-Lagrange equation for F(u)=int_{x0}^{x1} L(x,u,u')dx in one dimension; associated gradient descent method for a general problem "Min F(u)" for u in V that decreases the objective function (associated time-dependent Euler-Lagrange equation).
  • Finite dimensional unconstrained optimization: recall of Taylor's theorem; 1st and 2nd order necessary conditions; sufficient conditions for local minimizer; case of convex functions and global minimizers (pages 13-17 from Nocedal and Wright).
  • Examples of non-smooth optimization problems and how to transform them into smooth problems (from P.E. Gill, W. Murray, and M.H. Wright, Practical Optimization, pages 96-98).
  • Descent methods: definition of descent directions and steepest descent directions, step length alpha, computation of steepest descent direction for various norms (Euclidean, general quadratic norm, and l1-norm), exact line search and backtracking line search (from Chapter 9 Convex Optimization and part from Nocedal-Wright).
  • Newton and Quasi-Newton methods
  • Constrained optimization: log barrier, augmented Lagrangian, etc
  • Duality techniques in the continuous case (Ekeland-Temam)
  • ...
  • Sobolev gradients
  • ...

  • Sections on finite dimensional optimization covered from Nocedal and Wright: 2.1, 3.1, 3.2, 3.3, 7.1, 8.1, 10, 10.2, Summary of Ch. 12, 15, 17.
    Additional topics are covered from Gill-Murray-Wright, and from Boyd-Vandenberghe (9.1.2, 9.2, 9.3, 9.4, 9.5).

    Links:
  • Matlab Optimization Toolbox (check with our computer office to see which machines, if any, have the Matlab optimization toolbox installed).
  • Optimization Online
  • Optimization Technology Center (DOE and Northwestern)
  • SIAM Activity Group on Optimization
  • Numerical Recipies
  • NEOS Guide
  • Convex Analysis and Optimization by Dimitri P. Bertsekas
  • CCA Computational Convex Analysis numerical library by Yves Lucet

    Assignments Policy: There will be several homework assignments with theoretical and computational questions.

    Examinations: There will be one take-home final exam.

    Grading Policy: Hw and Projects 65%, Final 35%

  • Summary of optimality conditions
  • Notes on Stable and Normal Problems (following Ekeland-Temam)
  • Connections with the finite dimensional case

    Homework Assignments, Projects & Practice Problems:
  • HW #1, due on Friday, October 16
    Latex file
  • HW #2, due on Monday, November 2
    Latex file
  • HW #3, due on Monday, November 16
    Latex file
  • HW #4, due on Wednesday, November 25, or on Monday, November 30
    Latex file
  • HW #5, due on December 4 (or during the week of finals)