Math 273, Section 1, Fall 2009

Optimization, Calculus of Variations, and Control Theory

Lecture Meeting Time: MF 4.00PM - 5:15PM.
Lecture Location: MS 5137.

Instructor: Luminita A. Vese
Office: MS 7620 D
Office hours: (subject to change) Monday 5.15pm-6.15pm; Friday 5.15pm-6.15pm, or by appointment.
E-mail: lvese@math.ucla.edu

General Course Description: Application of abstract mathematical theory to optimization problems of calculus of variations and control theory. Abstract nonlinear programming and applications to control systems described by ordinary differential equations, partial differential equations, and functional differential equations. Dynamic programming.

References:
  • J. Nocedal and S.J. Wright, Numerical Optimization, Springer Series in Operations REsearch, Springer 1999 (1st or 2nd edition).
    Online access restricted to UC campuses (2nd edition)
  • I. Ekeland and R. Temam, Convex Analysis and Variational Problems, SIAM, 1999 (new edition), placed on rserve at SEL Library.
  • E. Zeidler, Nonlinear Functional Analysis and its Applications, Vol. III, Variational Methods and Optimization , Springer-Verlag 1984.
  • P.E. Gill, W. Murray, and M.H. Wright, Practical Optimization, Academic Press 1981.
  • R.T. Rockafellar, Convex Analysis, Princeton University Press 1970.
  • J.-B. Hiriart-Urruty, C. Lemarechal, Fundamentals of Convex Analysis, Springer 2001.
  • S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004 (especially Chapters 9, 10 and 11).
  • M. Giaquinta, S. Hildebrandt, Calculus of variations, Springer, 1996 (two volumes).
  • D. Luenberger, Optimization by Vector Space Methods , John Wiley & Sons, 1969.
  • Dimitri P. Bertsekas, with Angelia Nedic and Asuman E. Ozdaglar, Convex Analysis and Optimization .
  • L.C. Evans, Partial Differential Equations , Chapter 8.
  • H. Attouch, G. Buttazzo, and G. Michaille, Variational Analysis in Sobolev and BV Spaces: applications to PDE's and optimization, MPS-SIAM 2006.

    Specific topics will be listed here:
    First half of the course is devoted to numerical nonlinear optimization. Sections on finite dimensional optimization covered from Nocedal and Wright: 2.1, 3.1, 3.2, 3.3, 7.1, 8.1, 10, 10.2, Summary of Ch. 12, 15, 17. Additional topics are covered from Gill-Murray-Wright, and from Boyd-Vandenberghe (9.1.2, 9.2, 9.3, 9.4, 9.5).
  • Formulation of a general finite dimensional optimization problem with constraints; objective function, equality or inequality constraints, feasible region, example.
  • Finite dimensional unconstrained optimization: recall of Taylor's theorem; 1st and 2nd order necessary conditions; sufficient conditions for local minimizer; case of convex functions and global minimizers (pages 13-17 from Nocedal and Wright).
  • Examples of non-smooth optimization problems and how to transform them into smooth problems (from P.E. Gill, W. Murray, and M.H. Wright, Practical Optimization, pages 96-98).
  • Descent methods: definition of descent directions and steepest descent directions, step length alpha, computation of steepest descent direction for various norms (Euclidean, general quadratic norm, and l1-norm), exact line search and backtracking line search (from Chapter 9 Convex Optimization and part from Nocedal-Wright).
  • Newton and Quasi-Newton methods
  • Constrained optimization: quadratic penalty method, log barrier, augmented Lagrangian. The second half of the course will be devoted to abstract formulations in calculus of variations and applications to minimization problems in Sobolev spaces. Several sections from Ekeland-Temam will be presented.
  • Abstract minimization problems, applications, duality techniques in the continuous case (Ekeland-Temam)
  • Gateaux-differentiability, computation of Euler-Lagrange equation for F(u)=int_{x0}^{x1} L(x,u,u')dx in one dimension; associated gradient descent method for a general problem "Min F(u)" for u in V that decreases the objective function (associated time-dependent Euler-Lagrange equation).
  • Applications to abstract minimization problems and to minimization problems on Sobolev spaces
  • Sobolev gradients
  • Shape Optimization

    Links:
  • Matlab Optimization Toolbox (check with our computer office to see which machines, if any, have the Matlab optimization toolbox installed).
  • Optimization Online
  • Optimization Technology Center (DOE and Northwestern)
  • SIAM Activity Group on Optimization
  • Numerical Recipies
  • NEOS Guide
  • Convex Analysis and Optimization by Dimitri P. Bertsekas
  • CCA Computational Convex Analysis numerical library by Yves Lucet

    Assignments Policy: There will be several homework assignments with theoretical and computational questions.

    Examinations: There will be several homeworks and one take-home final exam.

  • Summary of optimality conditions
  • Notes on Stable and Normal Problems (following Ekeland-Temam)
  • Connections with the finite dimensional case

    Homework Assignments, Projects & Practice Problems:
  • HW #1, due on Monday, October 11
    Latex file
  • HW #2, due on Monday, October 25
    Latex file
  • HW #3, due on Monday, November 8
    Latex file
  • HW #4, due on Monday, November 22nd
    Latex file
  • HW #5, due on December 3rd, or during the week of finals.
    Latex file