8 Hits in 3.8 sec

Byzantine Fault-Tolerance in Federated Local SGD under 2f-Redundancy [article]

Nirupam Gupta, Thinh T. Doan, Nitin Vaidya
2021 arXiv   pre-print
We show that, under 2f-redundancy, the federated local SGD algorithm with CE can indeed obtain exact fault-tolerance in the deterministic setting when the non-faulty agents can accurately compute gradients  ...  We consider the problem of Byzantine fault-tolerance in federated machine learning. In this problem, the system comprises multiple agents each with local data, and a trusted centralized coordinator.  ...  SUMMARY In this paper, we have considered the problem of Byzantine fault-tolerance in the federated local stochastic gradientdescent method.  ... 
arXiv:2108.11769v1 fatcat:s36f3k7rkzdujib5efwo3uppbm

Byzantine Fault Tolerance in Distributed Machine Learning : a Survey [article]

Djamila Bouhata, Hamouma Moumen
2022 arXiv   pre-print
Byzantine Fault Tolerance (BFT) is among the most challenging problems in Distributed Machine Learning (DML).  ...  In this paper, we present a survey of recent works surrounding BFT in DML. Mainly in first-order optimization methods, especially Stochastic Gradient Descent (SGD).  ...  Byzantine fault tolerance in Federated Learning complements this survey and represents our future work.  ... 
arXiv:2205.02572v1 fatcat:h2hkcgz3w5cvrnro6whl2rpvby

A Survey on Fault-tolerance in Distributed Optimization and Machine Learning [article]

Shuo Liu
2021 arXiv   pre-print
This survey investigates the current state of fault-tolerance research in distributed optimization, and aims to provide an overview of the existing studies on both fault-tolerant distributed optimization  ...  With the rapid expansion of the scale of distributed systems, resilient distributed algorithms for optimization are needed, in order to mitigate system failures, communication issues, or even malicious  ...  [95] proposed Byzantine-resilient secure aggregation (BREA) framework to achieve both privacy-preservation and fault-tolerance in federated learning.  ... 
arXiv:2106.08545v2 fatcat:g6fys4icrbbr5k3bd3ycylaptu

Byzantine Fault-Tolerant Distributed Machine Learning Using Stochastic Gradient Descent (SGD) and Norm-Based Comparative Gradient Elimination (CGE) [article]

Nirupam Gupta, Shuo Liu, Nitin H. Vaidya
2021 arXiv   pre-print
This paper considers the Byzantine fault-tolerance problem in distributed stochastic gradient descent (D-SGD) method - a popular algorithm for distributed multi-agent machine learning.  ...  We show that the CGE gradient-filter guarantees fault-tolerance against a bounded fraction of Byzantine agents under standard stochastic assumptions, and is computationally simpler compared to many existing  ...  . • A new algorithm and its fault-tolerance property: We show that our algorithm, D-SGD method with CGE gradient-filter, guarantees Byzantine fault-tolerance under standard assumptions [7] .  ... 
arXiv:2008.04699v2 fatcat:v3bvhnb4vvffrp3t7dsawb3etu

Secure Distributed Training at Scale [article]

Eduard Gorbunov, Alexander Borzunov, Michael Diskin, Max Ryabinin
2021 arXiv   pre-print
Training in presence of such peers requires specialized distributed training algorithms with Byzantine tolerance.  ...  In this work, we propose a novel protocol for secure (Byzantine-tolerant) decentralized training that emphasizes communication efficiency.  ...  Byzantine fault-tolerance in decentralized optimization under 2f-redundancy. In 2021 American Control Conference (ACC), pp. 3632-3637. IEEE, 2021.  ... 
arXiv:2106.11257v2 fatcat:whcd527c6bf2pknucgdise4ope

Distributed Momentum for Byzantine-resilient Learning [article]

El-Mahdi El-Mhamdi, Rachid Guerraoui, Sébastien Rouault
2020 arXiv   pre-print
We then provide an extensive experimental demonstration of the robustness effect of worker-side momentum on distributed SGD.  ...  In a distributed setting, momentum can be implemented either at the server or the worker side.  ...  For instance, in the context of federated learning, recent work has shown that Byzantine fault tolerance serves as a good basis to study poisoning (Bagdasaryan et al., 2018; Sun et al., 2019) .  ... 
arXiv:2003.00010v2 fatcat:ykr3ay2jinbd3co3zfpd4lfefe

Asynchronous Fully-Decentralized SGD in the Cluster-Based Model [article]

Hagit Attiya, Noa Schiller
2022 arXiv   pre-print
This paper presents fault-tolerant asynchronous Stochastic Gradient Descent (SGD) algorithms.  ...  (This holds under standard assumptions on Q.) In this case, the algorithm obtains the same convergence rate as sequential SGD, up to a logarithmic factor.  ...  It was shown that 2𝑓 -redundancy is a necessary and sufficient condition for 𝑓 -resilient Byzantine deterministic optimization, both exact [18] and approximate [24] .  ... 
arXiv:2202.10862v2 fatcat:virh2gkm5vfqvg7h6jgnzzowkm

Privacy and Robustness in Federated Learning: Attacks and Defenses [article]

Lingjuan Lyu, Han Yu, Xingjun Ma, Chen Chen, Lichao Sun, Jun Zhao, Qiang Yang, Philip S. Yu
2022 arXiv   pre-print
Recently, federated learning (FL) has emerged as an alternative solution and continue to thrive in this new reality.  ...  In this paper, we conduct the first comprehensive survey on this topic.  ...  Defenses against Untargeted Attacks For Byzantine-resilient aggregation, an algorithm is Byzantine fault tolerant [22] if its convergence is robust even when a large portion of participants are adversarial  ... 
arXiv:2012.06337v3 fatcat:f5aflxnsdrdcdf4kvoa6yzseqq