114 Hits in 5.2 sec

Byzantine Fault Tolerance in Distributed Machine Learning : a Survey [article]

Djamila Bouhata, Hamouma Moumen
2022 arXiv   pre-print
Byzantine Fault Tolerance (BFT) is among the most challenging problems in Distributed Machine Learning (DML).  ...  Mainly in first-order optimization methods, especially Stochastic Gradient Descent (SGD). We highlight the key techniques as well as fundamental approaches.  ...  The proposed encoding scheme extends efficiently to the data streaming model and achieves stochastic gradient descent (SGD) Byzantine-resilient.  ... 
arXiv:2205.02572v1 fatcat:h2hkcgz3w5cvrnro6whl2rpvby

Data Encoding for Byzantine-Resilient Distributed Optimization [article]

Deepesh Data, Linqi Song, Suhas Diggavi
2020 arXiv   pre-print
Our encoding scheme extends efficiently to the data streaming model and for stochastic gradient descent (SGD). We also give experimental results to show the efficacy of our proposed schemes.  ...  We study distributed optimization in the presence of Byzantine adversaries, where both data and computation are distributed among m worker machines, t of which may be corrupt.  ...  Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on.  ... 
arXiv:1907.02664v2 fatcat:ctptgj7gpfelraygvetdxrpy4m

A Survey on Fault-tolerance in Distributed Optimization and Machine Learning [article]

Shuo Liu
2021 arXiv   pre-print
With the rapid expansion of the scale of distributed systems, resilient distributed algorithms for optimization are needed, in order to mitigate system failures, communication issues, or even malicious  ...  The robustness of distributed optimization is an emerging field of study, motivated by various applications of distributed optimization including distributed machine learning, distributed sensing, and  ...  gradient descent or distributed stochastic gradient descent) [19, 49, 68] .  ... 
arXiv:2106.08545v2 fatcat:g6fys4icrbbr5k3bd3ycylaptu

Fast and Robust Distributed Learning in High Dimension [article]

El-Mahdi El-Mhamdi and Rachid Guerraoui and Sébastien Rouault
2021 arXiv   pre-print
, the fastest (but non Byzantine resilient) rule for distributed machine learning.  ...  Could a gradient aggregation rule (GAR) for distributed machine learning be both robust and fast? This paper answers by the affirmative through multi-Bulyan.  ...  For instance, Stochastic Gradient Descent (SGD), an algorithm which is the workhorse of today's machine learning.  ... 
arXiv:1905.04374v2 fatcat:lqmumop6r5cylm3h6opqhm6kwm

Byzantine-Tolerant Machine Learning [article]

Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, Julien Stainer
2017 arXiv   pre-print
The growth of data, the need for scalability and the complexity of models used in modern machine learning calls for distributed implementations.  ...  In this paper, we study the robustness to Byzantine failures at the fundamental level of stochastic gradient descent (SGD), the heart of most machine learning algorithms.  ...  The authors would like to thank to Lê Nguyen Hoang for fruitful discussion and inputs.  ... 
arXiv:1703.02757v1 fatcat:o5ktjd24p5gppakkreep5ezvtu

Adaptive Verifiable Coded Computing: Towards Fast, Secure and Private Distributed Machine Learning [article]

Tingting Tang, Ramy E. Ali, Hanieh Hashemi, Tynan Gangwani, Salman Avestimehr, Murali Annavaram
2022 arXiv   pre-print
Stragglers, Byzantine workers, and data privacy are the main bottlenecks in distributed cloud computing. Some prior works proposed coded computing strategies to jointly address all three challenges.  ...  AVCC leverages coded computing just for handling stragglers and privacy, and then uses an orthogonal approach that leverages verifiable computing to mitigate Byzantine workers.  ...  In such approaches, a primary server encodes the data Distributed machine learning using cloud resources is and distributes the encoded data over  ... 
arXiv:2107.12958v2 fatcat:f4zr6cymjray3mwdcin2dsyqoi

Solon: Communication-efficient Byzantine-resilient Distributed Training via Redundant Gradients [article]

Lingjiao Chen, Leshang Chen, Hongyi Wang, Susan Davidson, Edgar Dobriban
2021 arXiv   pre-print
There has been a growing need to provide Byzantine-resilience in distributed model training.  ...  In this paper, we propose Solon, an algorithmic framework that exploits gradient redundancy to provide communication efficiency and Byzantine robustness simultaneously.  ...  Frank Seide, Hao Fu, Jasha Droppo, Gang Li, and Dong Yu. 1-bit stochastic gradient descent and its application to data-parallel distributed training of speech dnns.  ... 
arXiv:2110.01595v2 fatcat:bfvzjjz5kzfqxfsutqxg7yp4ky

ByGARS: Byzantine SGD with Arbitrary Number of Attackers [article]

Jayanth Regatti, Hao Chen, Abhishek Gupta
2020 arXiv   pre-print
We propose two novel stochastic gradient descent algorithms, ByGARS and ByGARS++, for distributed machine learning in the presence of any number of Byzantine adversaries.  ...  This reputation score is then used for aggregating the gradients for stochastic gradient descent.  ...  Conclusion We devise a novel, Byzantine resilient stochastic gradient aggregation algorithm for distributed machine learning with arbitrary number of adversarial workers.  ... 
arXiv:2006.13421v2 fatcat:tiz7nkd6uvgctojqahcfrwwqwi

Byzantine-Resilient Secure Federated Learning [article]

Jinhyun So, Basak Guler, A. Salman Avestimehr
2021 arXiv   pre-print
Towards addressing this challenge, this paper presents the first single-server Byzantine-resilient secure aggregation framework (BREA) for secure federated learning.  ...  This presents a major challenge for the resilience of the model against adversarial (Byzantine) users, who can manipulate the global model by modifying their local models or datasets.  ...  In concurrent work, a Byzantine-robust secure gradient descent algorithm has been proposed for a two-server model in [30] , however, unlike federated learning (which is based on a single-server architecture  ... 
arXiv:2007.11115v2 fatcat:4skcnakvcvbabhttlqlvhk62cm

Robust Federated Recommendation System [article]

Chen Chen, Jingfeng Zhang, Anthony K. H. Tung, Mohan Kankanhalli, Gang Chen
2020 arXiv   pre-print
Theoretically, we justify our robust learning strategy by our proposed definition of Byzantine resilience.  ...  We argue that the key to Byzantine detection is monitoring of gradients of the model parameters of clients.  ...  Machine learning with adversaries: Byzantine tolerant gradient descent.  ... 
arXiv:2006.08259v1 fatcat:boav3q2s5zgv5o3u5v5sxya6ti

DRACO: Byzantine-resilient Distributed Training via Redundant Gradients [article]

Lingjiao Chen and Hongyi Wang and Zachary Charles and Dimitris Papailiopoulos
2018 arXiv   pre-print
In this work, we present DRACO, a scalable framework for robust distributed training that uses ideas from coding theory.  ...  Distributed model training is vulnerable to byzantine system failures and adversarial compute nodes, i.e., nodes that use malicious updates to corrupt the global model stored at a parameter server (PS)  ...  Acknowledgement This work was supported in part by a gift from Google and AWS Cloud Credits for Research from Amazon.  ... 
arXiv:1803.09877v4 fatcat:6gdrzlwtlzewfleq6jcrguqaya

Privacy-preserving Graph Analytics: Secure Generation and Federated Learning [article]

Dongqi Fu, Jingrui He, Hanghang Tong, Ross Maciejewski
2022 arXiv   pre-print
For each direction, we identify both "quick wins" and "hard problems". Towards the end, we demonstrate a user interface that can facilitate model explanation, interpretation, and visualization.  ...  Directly motivated by security-related applications from the Homeland Security Enterprise, we focus on the privacy-preserving analysis of graph data, which provides the crucial capacity to represent rich  ...  To be specific, in [17] , the privacy scheme is added to the gradient descent phase of the generation learning process.  ... 
arXiv:2207.00048v1 fatcat:sjv4mhx7qrbi5jq5g2c7kgxqp4

Byzantine-Resilient SGD in High Dimensions on Heterogeneous Data [article]

Deepesh Data, Suhas Diggavi
2020 arXiv   pre-print
We study distributed stochastic gradient descent (SGD) in the master-worker architecture under Byzantine attacks.  ...  We also propose and analyze a Byzantine-resilient SGD algorithm with gradient compression, where workers send k random coordinates of their gradients.  ...  to perform data encoding across different nodes.  ... 
arXiv:2005.07866v1 fatcat:ixlbiuqipfcztchkwaewrej77q

Byzantine Fault Tolerant Distributed Linear Regression [article]

Nirupam Gupta, Nitin H. Vaidya
2019 arXiv   pre-print
We show that the server can achieve this objective, in a deterministic manner, by robustifying the original distributed gradient descent method using norm based filters, namely 'norm filtering' and 'norm-cap  ...  The proposed algorithms differ from each other in the assumptions made for their correctness, and the gradient filter they use.  ...  In [12] , the authors propose a data encoding scheme for tolerating Byzantine faulty workers.  ... 
arXiv:1903.08752v2 fatcat:c6zgvwkxe5g5tb3pkpu7yfima4

Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy [article]

Qian Yu, Songze Li, Netanel Raviv, Seyed Mohammadreza Mousavi Kalan, Mahdi Soltanolkotabi, Salman Avestimehr
2019 arXiv   pre-print
We propose Lagrange Coded Computing (LCC), a new framework to simultaneously provide (1) resiliency against stragglers that may prolong computations; (2) security against Byzantine (or malicious) workers  ...  adversaries, and providing data privacy against the maximum number of colluding workers.  ...  Gradient descent (GD) solves this problem by iteratively moving the weight along the negative gradient direction, which is in iteration-t computed as 2X (Xw (t) − y).  ... 
arXiv:1806.00939v4 fatcat:qy4eglfqa5c65eiwt42pfs572u
« Previous Showing results 1 — 15 out of 114 results