Communication-Efficient Asynchronous Stochastic Frank-Wolfe over Nuclear-norm Balls [article]

Jiacheng Zhuo, Qi Lei, Alexandros G. Dimakis, Constantine Caramanis
2019 arXiv   pre-print
Large-scale machine learning training suffers from two prior challenges, specifically for nuclear-norm constrained problems with distributed systems: the synchronization slowdown due to the straggling workers, and high communication costs. In this work, we propose an asynchronous Stochastic Frank Wolfe (SFW-asyn) method, which, for the first time, solves the two problems simultaneously, while successfully maintaining the same convergence rate as the vanilla SFW. We implement our algorithm in
more » ... hon (with MPI) to run on Amazon EC2, and demonstrate that SFW-asyn yields speed-ups almost linear to the number of machines compared to the vanilla SFW.
arXiv:1910.07703v1 fatcat:kaxmvnegunak3lr2l7smmnglzi