10 Hits in 2.7 sec

ByRDiE: Byzantine-resilient distributed coordinate descent for decentralized learning [article]

Zhixiong Yang, Waheed U. Bajwa
2019 arXiv   pre-print
In this paper, an algorithm termed Byzantine-resilient distributed coordinate descent (ByRDiE) is developed and analyzed that enables distributed learning in the presence of Byzantine failures.  ...  While Byzantine fault tolerance has a rich history, existing work does not translate into efficient and practical algorithms for high-dimensional learning in fully distributed (also known as decentralized  ...  BYZANTINE-RESILIENT DISTRIBUTED COORDINATE DESCENT FOR DECENTRALIZED LEARNING In distributed learning, one would ideally like to solve the empirical risk minimization (ERM) problem w opt = arg min w∈R  ... 
arXiv:1708.08155v4 fatcat:fmv3hbw2rzhvvgmeio4umjpcny

Adversary-resilient Distributed and Decentralized Statistical Inference and Machine Learning [article]

Zhixiong Yang, Arpita Gang, Waheed U. Bajwa
2020 arXiv   pre-print
Driven in part by the world's growing appetite for data-driven decision making, however, securing of distributed/decentralized frameworks for inference and learning against adversarial threats remains  ...  While the last few decades have witnessed a huge body of work devoted to inference and learning in distributed and decentralized setups, much of this work assumes a non-adversarial setting in which individual  ...  This limitation of [4] , [38] is overcome in [39] , which proposes an algorithm termed Byzantine-resilient decentralized coordinate descent (ByRDiE) for vector-valued decentralized learning in the  ... 
arXiv:1908.08649v2 fatcat:de356dvwinfv5g5njo64qmzpvi

BRIDGE: Byzantine-resilient Decentralized Gradient Descent [article]

Cheng Fang, Zhixiong Yang, Waheed U. Bajwa
2022 arXiv   pre-print
In this paper, a scalable, Byzantine-resilient decentralized machine learning framework termed Byzantine-resilient decentralized gradient descent (BRIDGE) is introduced.  ...  But the study of Byzantine resilience within decentralized learning, in contrast to distributed learning, is still in its infancy.  ...  More importantly, however, the coordinate-descent nature of ByRDiE makes it slow and inefficient for learning of large-scale models.  ... 
arXiv:1908.08098v2 fatcat:uh7wdsvotzbkppmele2pchk5kq

Byzantine Fault Tolerance in Distributed Machine Learning : a Survey [article]

Djamila Bouhata, Hamouma Moumen
2022 arXiv   pre-print
Byzantine Fault Tolerance (BFT) is among the most challenging problems in Distributed Machine Learning (DML).  ...  Mainly in first-order optimization methods, especially Stochastic Gradient Descent (SGD). We highlight the key techniques as well as fundamental approaches.  ...  Weaknesses: ByzantinePGD is still near-optimal in the high-dimensional setting ByRDiE Yang and Bajwa [156] proposed the Byzantine resilient distributed coordinate descent (ByRDiE) algorithm to deal with  ... 
arXiv:2205.02572v1 fatcat:h2hkcgz3w5cvrnro6whl2rpvby

Byzantine-resilient Decentralized Stochastic Gradient Descent [article]

Shangwei Guo, Tianwei Zhang, Han Yu, Xiaofei Xie, Lei Ma, Tao Xiang, Yang Liu
2021 arXiv   pre-print
In this paper, we present an in-depth study towards the Byzantine resilience of decentralized learning systems with two contributions.  ...  Specifically, UBAR provides a Uniform Byzantine-resilient Aggregation Rule for benign nodes to select the useful parameter updates and filter out the malicious ones in each training iteration.  ...  While ByRDiE is designed for the coordinate descent optimization algorithm, BRIDGE is used in decentralized learning systems with SGD.  ... 
arXiv:2002.08569v4 fatcat:jqsjyln3tnhjleiq4aic6f3yta

2019 Index IEEE Transactions on Signal and Information Processing over Networks Vol. 5

2019 IEEE Transactions on Signal and Information Processing over Networks  
., +, TSIPN March 2019 61-69 Fault tolerance ByRDiE: Byzantine-Resilient Distributed Coordinate Descent for Decentralized Learning.  ...  ., +, TSIPN Sept. 2019 479-494 ByRDiE: Byzantine-Resilient Distributed Coordinate Descent for Decen- tralized Learning.  ... 
doi:10.1109/tsipn.2019.2959414 fatcat:ixpx5rg5l5hshkt2ppvie3afqe

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications [article]

S. Hu, X. Chen, W. Ni, E. Hossain, X. Wang
2020 arXiv   pre-print
Distributed machine learning (DML) techniques, such as federated learning, partitioned learning, and distributed reinforcement learning, have been increasingly applied to wireless communications.  ...  There is a clear gap in the existing literature in that the DML techniques are yet to be systematically reviewed for their applicability to wireless systems.  ...  Next, BYRDIE uses the Byzantine-resilient approach to solve each scalar-valued subproblem.  ... 
arXiv:2012.01489v1 fatcat:pdauhq4xbbepvf26clhpqnc2ci

Approximate Byzantine Fault-Tolerance in Distributed Optimization [article]

Shuo Liu, Nirupam Gupta, Nitin H. Vaidya
2021 arXiv   pre-print
In case when the agents' cost functions are differentiable, we obtain conditions for (f,ϵ)-resilience of the distributed gradient-descent method when equipped with robust gradient aggregation.  ...  This paper considers the problem of Byzantine fault-tolerance in distributed multi-agent optimization.  ...  Byrdie: Byzantine-resilient distributed coordinate descent for decentralized learning, 2017. [54] Yin, D., Chen, Y., Kannan, R., and Bartlett, P.  ... 
arXiv:2101.09337v4 fatcat:jlhclmf2ljhzvlaf6almnqdyri

Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning) [article]

El-Mahdi El-Mhamdi, Sadegh Farhadkhani, Rachid Guerraoui, Arsany Guirguis, Lê Nguyên Hoang, Sébastien Rouault
2021 arXiv   pre-print
The second, based on reliable broadcast and coordinate-wise trimmed mean, achieves optimal Byzantine resilience, i.e., n ≥ 3f+1.  ...  We study Byzantine collaborative learning, where n nodes seek to collectively learn from each others' local data. The data distribution may vary from one node to another.  ...  Acknowledgments and Disclosure of Funding We thank Rafaël Pinot and Nirupam Gupta for their useful comments.  ... 
arXiv:2008.00742v5 fatcat:3e3qelv44nex5dlludxbfj7dfm

Secure Distributed Training at Scale [article]

Eduard Gorbunov, Alexander Borzunov, Michael Diskin, Max Ryabinin
2021 arXiv   pre-print
In this work, we propose a novel protocol for secure (Byzantine-tolerant) decentralized training that emphasizes communication efficiency.  ...  Training in presence of such peers requires specialized distributed training algorithms with Byzantine tolerance.  ...  Byrdie: Byzantine-resilient distributed coordinate descent for decentralized learning. IEEE Transactions on Signal and Information Processing over Networks, 5 (4):611-627, 2019b.  ... 
arXiv:2106.11257v2 fatcat:whcd527c6bf2pknucgdise4ope