Filters








5 Hits in 4.2 sec

SPARQ-SGD: Event-Triggered and Compressed Communication in Decentralized Stochastic Optimization [article]

Navjot Singh, Deepesh Data, Jemin George, Suhas Diggavi
2020 arXiv   pre-print
In this paper, we propose and analyze SPARQ-SGD, which is an event-triggered and compressed algorithm for decentralized training of large-scale machine learning models.  ...  We prove that the SPARQ-SGD converges as O(1/nT) and O(1/√(nT)) in the strongly-convex and non-convex settings, respectively, demonstrating that such aggressive compression, including event-triggered communication  ...  The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory  ... 
arXiv:1910.14280v2 fatcat:oxyl532m7fdgblydpk2fegbhee

Communication Efficient Tensor Factorization for Decentralized Healthcare Networks [article]

Jing Ma, Qiuchen Zhang, Jian Lou, Li Xiong, Sivasubramanium Bhavani, Joyce C. Ho
2021 arXiv   pre-print
In this paper, we propose CiderTF, a communication-efficient decentralized generalized tensor factorization, which reduces the uplink communication cost by leveraging a four-level communication reduction  ...  ., are converted to meaningful and interpretable medical concepts.  ...  , CTSA Award UL1TR002378, and Cisco Research University Award #2738379.  ... 
arXiv:2109.01718v1 fatcat:b6eamlzq3vchrmz4iugiyxtnmi

LENA: Communication-Efficient Distributed Learning with Self-Triggered Gradient Uploads

Hossein Shokri Ghadikolaei, Sebastian U. Stich, Martin Jaggi
2021 International Conference on Artificial Intelligence and Statistics  
Every node locally accumulates an error vector in memory and self-triggers the upload of the memory contents to the parameter server using a significance filter.  ...  In distributed optimization, parameter updates from the gradient computing node devices have to be aggregated in every iteration on the orchestrating server.  ...  error feedback and extended LAG to a decentralized setting in SPARQ-SGD.  ... 
dblp:conf/aistats/GhadikolaeiSJ21 fatcat:mslvii426vezvocstsf7za23ju

A Decentralized Parallel Algorithm for Training Generative Adversarial Nets [article]

Mingrui Liu, Wei Zhang, Youssef Mroueh, Xiaodong Cui, Jerret Ross, Tianbao Yang, Payel Das
2020 arXiv   pre-print
The main difficulty lies at handling the nonconvex-nonconcave min-max optimization and the decentralized communication simultaneously.  ...  In this paper, we address this difficulty by designing the first gradient-based decentralized parallel algorithm which allows workers to have multiple rounds of communications in one iteration and to update  ...  Sparq-sgd: Event-triggered and compressed communication in decentralized stochastic optimization. arXiv preprint arXiv:1910.14280, 2019.  ... 
arXiv:1910.12999v6 fatcat:xbhl2e6oyvhdnc3swaq2cq5vza

SQuARM-SGD: Communication-Efficient Momentum SGD for Decentralized Optimization [article]

Navjot Singh, Deepesh Data, Jemin George, Suhas Diggavi
2020 arXiv   pre-print
We propose and analyze SQuARM-SGD, a decentralized training algorithm, employing momentum and compressed communication between nodes regulated by a locally computable triggering rule.  ...  In this paper, we study communication-efficient decentralized training of large-scale machine learning models over a network.  ...  SPARQ-SGD: Event-triggered and compressed communication in decentralized stochastic optimization. arXiv preprint arXiv:1910.14280, 2019. [SDJ13] Georg S. Seyboth, Dimos V.  ... 
arXiv:2005.07041v2 fatcat:wn7e64caf5bgxjnzs7vrws4xcq