10 Hits in 2.3 sec

ByRDiE: Byzantine-resilient distributed coordinate descent for decentralized learning [article]

Zhixiong Yang, Waheed U. Bajwa
2019 arXiv   pre-print
In this paper, an algorithm termed Byzantine-resilient distributed coordinate descent (ByRDiE) is developed and analyzed that enables distributed learning in the presence of Byzantine failures.  ...  Distributed machine learning algorithms enable learning of models from datasets that are distributed over a network without gathering the data at a centralized location.  ...  Byzantine-resilient algorithms for scalar averaging distributed consensus were studied in [29] .  ... 
arXiv:1708.08155v4 fatcat:fmv3hbw2rzhvvgmeio4umjpcny

Adversary-resilient Distributed and Decentralized Statistical Inference and Machine Learning [article]

Zhixiong Yang, Arpita Gang, Waheed U. Bajwa
2020 arXiv   pre-print
As a result, we now have a plethora of algorithmic approaches that guarantee robustness of distributed and/or decentralized inference and learning under different adversarial threat models.  ...  While the last few decades have witnessed a huge body of work devoted to inference and learning in distributed and decentralized setups, much of this work assumes a non-adversarial setting in which individual  ...  Byzantine-resilient distributed machine learning.  ... 
arXiv:1908.08649v2 fatcat:de356dvwinfv5g5njo64qmzpvi

BRIDGE: Byzantine-resilient Decentralized Gradient Descent [article]

Cheng Fang, Zhixiong Yang, Waheed U. Bajwa
2022 arXiv   pre-print
But the study of Byzantine resilience within decentralized learning, in contrast to distributed learning, is still in its infancy.  ...  In this paper, a scalable, Byzantine-resilient decentralized machine learning framework termed Byzantine-resilient decentralized gradient descent (BRIDGE) is introduced.  ...  As such, despite the plethora of work on Byzantine-resilient distributed learning, the problem of Byzantine-resilient decentralized learning-with the exception of a handful of works discussed in the following-largely  ... 
arXiv:1908.08098v2 fatcat:uh7wdsvotzbkppmele2pchk5kq

Byzantine Fault Tolerance in Distributed Machine Learning : a Survey [article]

Djamila Bouhata, Hamouma Moumen
2022 arXiv   pre-print
Byzantine Fault Tolerance (BFT) is among the most challenging problems in Distributed Machine Learning (DML).  ...  Byzantine failures are still difficult to tackle due to their unrestricted nature; as a result, the possibility of generating arbitrary data.  ...  The authors also prove the resilience of ByRDiE to cope with Byzantine network failures for distributed convex and non-convex learning tasks.  ... 
arXiv:2205.02572v1 fatcat:h2hkcgz3w5cvrnro6whl2rpvby

Byzantine-resilient Decentralized Stochastic Gradient Descent [article]

Shangwei Guo, Tianwei Zhang, Han Yu, Xiaofei Xie, Lei Ma, Tao Xiang, Yang Liu
2021 arXiv   pre-print
Second, from the defense perspective, we propose UBAR, a novel algorithm to enhance decentralized learning with Byzantine Fault Tolerance.  ...  In this paper, we present an in-depth study towards the Byzantine resilience of decentralized learning systems with two contributions.  ...  Byzantine-resilient Centralized Learning A centralized learning system consists of a Parameter Server (PS) and multiple distributed worker nodes, as shown in Figure 1(a) .  ... 
arXiv:2002.08569v4 fatcat:jqsjyln3tnhjleiq4aic6f3yta

2019 Index IEEE Transactions on Signal and Information Processing over Networks Vol. 5

2019 IEEE Transactions on Signal and Information Processing over Networks  
., +, TSIPN Sept. 2019 479-494 ByRDiE: Byzantine-Resilient Distributed Coordinate Descent for Decen- tralized Learning.  ...  ., +, TSIPN Sept. 2019 479-494 ByRDiE: Byzantine-Resilient Distributed Coordinate Descent for Decen- tralized Learning.  ... 
doi:10.1109/tsipn.2019.2959414 fatcat:ixpx5rg5l5hshkt2ppvie3afqe

Approximate Byzantine Fault-Tolerance in Distributed Optimization [article]

Shuo Liu, Nirupam Gupta, Nitin H. Vaidya
2021 arXiv   pre-print
In this problem, each agent has a local cost function, and in the fault-free case, the goal is to design a distributed algorithm that allows all the agents to find a minimum point of all the agents' aggregate  ...  This paper considers the problem of Byzantine fault-tolerance in distributed multi-agent optimization.  ...  Research reported in this paper is also supported in part by a Fritz Fellowship from Georgetown University.  ... 
arXiv:2101.09337v4 fatcat:jlhclmf2ljhzvlaf6almnqdyri

Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning) [article]

El-Mahdi El-Mhamdi, Sadegh Farhadkhani, Rachid Guerraoui, Arsany Guirguis, Lê Nguyên Hoang, Sébastien Rouault
2021 arXiv   pre-print
Each of these algorithms induces an optimal Byzantine collaborative learning protocol.  ...  We study Byzantine collaborative learning, where n nodes seek to collectively learn from each others' local data. The data distribution may vary from one node to another.  ...  This work has been supported in part by the Swiss National Science Foundation projects: 200021_182542, Machine learning and 200021_200477, Controlling the spread of Epidemics.  ... 
arXiv:2008.00742v5 fatcat:3e3qelv44nex5dlludxbfj7dfm

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications [article]

S. Hu, X. Chen, W. Ni, E. Hossain, X. Wang
2020 arXiv   pre-print
Distributed machine learning (DML) techniques, such as federated learning, partitioned learning, and distributed reinforcement learning, have been increasingly applied to wireless communications.  ...  This survey bridges the gap by providing a contemporary and comprehensive survey of DML techniques with a focus on wireless networks.  ...  In [183] and [184] , a Byzantine-resilient algorithm for distributed learning, named BYRDIE, is developed to avoid sharing primitive data between workers, so that ML tasks can be accomplished in a completely  ... 
arXiv:2012.01489v1 fatcat:pdauhq4xbbepvf26clhpqnc2ci

Secure Distributed Training at Scale [article]

Eduard Gorbunov, Alexander Borzunov, Michael Diskin, Max Ryabinin
2021 arXiv   pre-print
As a result, it can be infeasible to apply such algorithms to large-scale distributed deep learning, where models can have billions of parameters.  ...  Training in presence of such peers requires specialized distributed training algorithms with Byzantine tolerance.  ...  Byrdie: Byzantine-resilient distributed coordinate descent for decentralized learning. IEEE Transactions on Signal and Information Processing over Networks, 5 (4):611-627, 2019b.  ... 
arXiv:2106.11257v2 fatcat:whcd527c6bf2pknucgdise4ope