Expander Graph and Communication-Efficient Decentralized Optimization

Yat-Tin Chow, Wei Shi, Tianyu Wu, and Wotao Yin

Published in Asilomar’16

Overview

In this paper, we discuss how to design the graph topology to reduce the communication complexity of algorithms for decentralized optimization. Our goal is to minimize the total communication that is needed to achieve a prescribed accuracy. We discover that expander graphs are near-optimal choices. (Roughly speaking, these graphs can be both sparse and possessing good spectral properties.)

We propose three approaches to construct expander graphs for different specified numbers of nodes and node degrees. Our numerical results show that the performance of decentralized optimization is significantly better on expander graphs than other regular graphs

Citation

Y.-T. Chow, W. Shi, T. Wu, and W. Yin, Expander Graph and Communication-Efficient Decentralized Optimization. Signals, Systems and Computers, 2016 50th Asilomar Conference on, Pacific Grove, CA. DOI: 10.1109/ACSSC.2016.7869675


« Back