A Primer on Coordinate Descent Algorithms

Hao-Jun Michael Shi, Shenyinying Tu, Yangyang Xu, and Wotao Yin

Last updated: September 30, 2016


This monograph presents a class of algorithms called coordinate descent algorithms for mathematicians, statisticians, and engineers outside the field of optimization. This particular class of algorithms has recently gained popularity due to their effectiveness in solving large-scale optimization problems in machine learning, compressed sensing, image processing, and computational statistics. Coordinate descent algorithms solve optimization problems by successively minimizing along each coordinate or coordinate hyperplane, which is ideal for parallelized and distributed computing. Avoiding detailed technicalities and proofs, this monograph gives relevant theory and examples for practitioners to effectively apply coordinate descent to modern problems in data science and engineering.

To keep the primer up-to-date, we intend to publish this monograph only after no additional topics need to be added and we foresee no further major advances in the area.


H.-J. Shi, S. Tu, Y. Xu, and W. Yin, A Primer on Coordinate Descent Algorithms, UCLA CAM 16-67, 2016.

« Back