Math 273, Section 1, Fall 2007

Optimization, Calculus of Variations, and Control Theory

Lecture Meeting Time: MWF 3.00PM - 3:50PM.
Lecture Location: MS 5127 (NOTE: new location)

Instructor: Luminita A. Vese
Office: MS 7620 D
Office hours: Mon 4-5pm, Wed 5-6pm, and Fri 5-6pm, or by appointment.


General Course Description: Application of abstract mathematical theory to optimization problems of calculus of variations and control theory. Abstract nonlinear programming and applications to control systems described by ordinary differential equations, partial differential equations, and functional differential equations. Dynamic programming.

  • I. Ekeland and R. Temam, Convex Analysis and Variational Problems, SIAM, 1999 (new edition).
  • E. Zeidler, Nonlinear Functional Analysis and its Applications, Vol. III, Variational Methods and Optimization , Springer-Verlag 1984.
  • P.E. Gill, W. Murray, and M.H. Wright, Practical Optimization, Academic Press 1981.
  • J. Nocedal and S.J. Wright, Numerical Optimization, Springer Series in Operations REsearch, Springer 1999.
  • R.T. Rockafellar, Convex Analysis, Princeton University Press 1970.
  • J.-B. Hiriart-Urruty, C. Lemarechal, Fundamentals of Convex Analysis, Springer 2001.
  • S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004 (especially Chapters 9, 10 and 11).
  • M. Giaquinta, S. Hildebrandt, Calculus of variations, Springer, 1996 (two volumes).
  • D. Luenberger, Optimization by Vector Space Methods , John Wiley & Sons, 1969.
  • Dimitri P. Bertsekas, with Angelia Nedic and Asuman E. Ozdaglar, Convex Analysis and Optimization .
  • L.C. Evans, Partial Differential Equations , Chapter 8.

    Specific topics will be listed here:
  • Formulation of a general finite dimensional optimization problem with constraints; objective function, equality or inequality constraints, feasible region, example.
  • Gateaux-differentiability, computation of Euler-Lagrange equation for F(u)=int_{x0}^{x1} L(x,u,u')dx in one dimension; associated gradient descent method for a general problem "Min F(u)" for u in V that decreases the objective function (associated time-dependent Euler-Lagrange equation).
  • Finite dimensional unconstrained optimization: recall of Taylor's theorem; 1st and 2nd order necessary conditions; sufficient conditions for local minimizer; case of convex functions and global minimizers (pages 13-17 from Nocedal and Wright).
  • Examples of non-smooth optimization problems and how to transform them into smooth problems (from P.E. Gill, W. Murray, and M.H. Wright, Practical Optimization, pages 96-98).
  • Descent methods: definition of descent directions and steepest descent directions, step length alpha, computation of steepest descent direction for various norms (Euclidean, general quadratic norm, and l1-norm), exact line search and backtracking line search (from Chapter 9 Convex Optimization and part from Nocedal-Wright).
  • Newton and Quasi-Newton methods
  • Constrained optimization: log barrier, augmented Lagrangian, etc
  • Duality techniques in the continuous case (Ekeland-Temam)
  • ...
  • Sobolev gradients
  • ...

  • Matlab Optimization Toolbox (check with our computer office to see which machines, if any, have the Matlab optimization toolbox installed).
  • Optimization Online
  • Optimization Technology Center (DOE and Northwestern)
  • SIAM Activity Group on Optimization
  • Numerical Recipies
  • NEOS Guide
  • Convex Analysis and Optimization by Dimitri P. Bertsekas

    Assignments Policy: There will be several homework assignments with theoretical and computational questions.

    Examinations: There will be one take-home final exam.

    Grading Policy: Hw and Projects 50%, Final 50%

    Homework Assignments, Projects & Practice Problems:
  • HW #1, due on Monday, October 8
  • HW #2, due no later than Wednesday, October 24
  • HW #3, due on November 7 or on November 9
  • HW #4, due on Monday, November 19
  • HW #5, due on Friday, December 7