## Math 285J: Operators, Splitting, and First-Order Optimization Algorithms## LinksLocation MS 6201 First meeting on Friday, September 23rd, 3pm. If necessary, we will change the time for future meetings Q&A and lecture notes on piazza.com. Send me an email if you need the access.
## TopicsOverview of optimization, operators, splitting, and first-order algorithm Subgradients and subgradient algorithms for nonsmooth optimization Proximal operators and related algorithms Convergence of fixed-point iterations Monotone operators and properties Operator splitting methods Duality and dual algorithms Primal-dual algorithms Coordinate update algorithms Parallel versions algorithms for structured problems Convex optimization convergence analysis Stochastic approximation algorithms Asynchronous parallel algorithms Nonconvex optimization algorithms Distributed and decentralized algoritms
Lecture slides and notes will be provided. ## Mild workloadVoluntary exercises on each topic, not graded Notes taking for one topic Reading some papers in one topic, either alone or with another student in a group, and giving a 20-minute presentation No exams
## Final projectsSlide Latex template. Suggested but not mandatory Each group has either one or two members. Justification (e.g., difficult tasks) is needed for any group of three or more. Presentations are scheduled for Monday Nov 28th and Wednesday Nov 30th. If you need more time, I can schedule your presentation in the final week. Please either pick your own paper (written by you or others) or one of the following candidate papers
## Candidate papers:Stochastic Three-Composite Convex Minimization Yurtsever, Alp; Vu, Cong Bang; Cevher, Volkan Infimal convolution of data discrepancies for mixed noise removal Luca Calatroni, Juan Carlos De Los Reyes, Carola-Bibiane Schonlieb A first-order primal-dual algorithm with linesearch Yura Malitsky, Thomas Pock Disciplined Multi-Convex Programming Xinyue Shen, Steven Diamond, Madeleine Udell, Yuantao Gu, Stephen Boyd Generalized Kalman Smoothing: Modeling and Algorithms A.Y. Aravkin, J.V. Burke, L. Ljung, A. Lozano, G. Pillonetto Block-proximal methods with spatially adapted acceleration Tuomo Valkonen A descent Lemma beyond Lipschitz gradient continuity: first-order methods revisited and applications HH Bauschke, J Bolte, M Teboulle On Convergence Rate of Distributed Stochastic Gradient Algorithm for Convex Optimization with Inequality Constraints Deming Yuan, Daniel W. C. Ho, and Yiguang Hong On the convergence rate of the three operator splitting scheme Fabian Pedregosa Learning Fast Approximations of Sparse Coding Karol Gregor and Yann LeCun Asynchronous Doubly Stochastic Proximal Optimization with Variance Reduction Bin Gu, Zhouyuan Huo, Heng Huang Training Neural Networks Without Gradients: A Scalable ADMM Approach Gavin Taylor, Ryan Burmeister, Zheng Xu, Bharat Singh, Ankit Patel, Tom Goldstein Efficient Algorithms for Large-scale Generalized Eigenvector Computation and Canonical Correlation Analysis Rong Ge, Chi Jin, Sham M. Kakade, Praneeth Netrapalli, Aaron Sidford Level-Set Methods For Convex Optimization A. Y. Aravkin, J. V. Burke, D. Drusvyatskiy, M. P. Friedlander, S. Roy Fast Stochastic Methods for Nonsmooth Nonconvex Optimization Sashank J. Reddi, Suvrit Sra, Barnabas Poczos, Alex Smola Regularized Nonlinear Acceleration Damien Scieur, Alexandre d'Aspremont, Francis Bach The non-convex Burer-Monteiro approach works on smooth semidefinite programs Nicolas Boumal, Vladislav Voroninski, Afonso S. Bandeira A Primal-Dual Type Algorithm with the O(1/t) Convergence Rate for Large Scale Constrained Convex Programs Hao Yu, Michael J. Neely Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization Xiao Wang, Shiqian Ma, Donald Goldfarb, Wei Liu Second Order Stochastic Optimization in Linear Time Naman Agarwal, Brian Bullins, Elad Hazan Cyclic Coordinate Update Algorithms for Fixed-Point Problems: Analysis and Applications Yat Tin Chow, Tianyu Wu, Wotao Yin Coordinate Friendly Structures, Algorithms and Applications Zhimen Peng, Tianyu Wu, Yangyang Xu, Ming Yan, and Wotao Yin
« Back |