A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf
.
Tackling Data Heterogeneity: A New Unified Framework for Decentralized SGD with Sample-induced Topology
[article]
2022
arXiv
pre-print
We develop a general framework unifying several gradient-based stochastic optimization methods for empirical risk minimization problems both in centralized and distributed scenarios. The framework hinges on the introduction of an augmented graph consisting of nodes modeling the samples and edges modeling both the inter-device communication and intra-device stochastic gradient computation. By designing properly the topology of the augmented graph, we are able to recover as special cases the
arXiv:2207.03730v1
fatcat:v4wtrwohfjcrjdym2v6gcxqjd4