4 Hits in 2.8 sec

A Unified Coded Deep Neural Network Training Strategy based on Generalized PolyDot codes

Sanghamitra Dutta, Ziqian Bai, Haewon Jeong, Tze Meng Low, Pulkit Grover
2018 2018 IEEE International Symposium on Information Theory (ISIT)  
Second, we demonstrate that Generalized PolyDot can be used for training large Deep Neural Networks (DNNs) on unreliable nodes prone to soft-errors.  ...  First, we propose a novel coded matrix multiplication technique called Generalized PolyDot codes that advances on existing methods for coded matrix multiplication under storage and communication constraints  ...  unified coded computing strategy for DNN training.  ... 
doi:10.1109/isit.2018.8437852 dblp:conf/isit/DuttaBJLG18 fatcat:cugezh5vrzdx5m2ufinzcqbfi4

A Survey of Coded Distributed Computing [article]

Jer Shyuan Ng, Wei Yang Bryan Lim, Nguyen Cong Luong, Zehui Xiong, Alia Asheralieva, Dusit Niyato, Cyril Leung, Chunyan Miao
2020 arXiv   pre-print
To address these issues, coded distributed computing (CDC), i.e., a combination of coding theoretic techniques and distributed computing, has been recently proposed as a promising solution.  ...  Furthermore, a distributed computing network may include straggling nodes that run intermittently slower.  ...  However, this unified coded DNN training strategy may not be relevant to the training of other neural networks which have large number of non-linear functions.  ... 
arXiv:2008.09048v1 fatcat:riy4dxvuc5ae3krz7lf25zkg6m

Private and Secure Distributed Matrix Multiplication with Flexible Communication Load [article]

Malihe Aliasgari, Osvaldo Simeone, Joerg Kliewer
2019 arXiv   pre-print
For this problem, we introduce a novel class of secure codes, referred to as secure generalized PolyDot (SGPD) codes, that generalize state-of-the-art non-secure codes for matrix multiplication.  ...  For this model, we present a variant of generalized PolyDot codes that can guarantee both secrecy of one matrix and privacy for the identity of the other matrix for the case of no colluding servers.  ...  In [14] , GPD codes are used to design a unified coded computing strategy for the training of deep neural networks.  ... 
arXiv:1909.00407v2 fatcat:wpybzzsylzf27bqvrhu3ju5374

Coded Distributed Computing with Partial Recovery [article]

Emre Ozfatura and Sennur Ulukus and Deniz Gunduz
2021 arXiv   pre-print
We then extend this approach to distributed implementation of more general computation tasks by proposing a coded communication scheme with partial recovery, where the results of subtasks computed by the  ...  In this paper, we first introduce a novel coded matrix-vector multiplication scheme, called coded computation with partial recovery (CCPR), which benefits from the advantages of both coded and uncoded  ...  Grover, “A unified coded PMLR, 10–15 Jul 2018, pp. 5610–5619. deep neural network training strategy based on generalized polydot [10] W. Halbawi, N.  ... 
arXiv:2007.02191v2 fatcat:a7rx5w4yrbhnbnh4uf2s2h3eke