Filters








310 Hits in 1e+01 sec

Direct Federated Neural Architecture Search [article]

Anubhav Garg, Amit Kumar Saha, Debo Dutta
2020 arXiv   pre-print
We apply this idea to Federated Learning (FL), wherein predefined neural network models are trained on the client/device data.  ...  NAS is promising for FL which can search for global and personalized models automatically for the non-IID data.  ...  If the clients' data is small and IID, the single global model learnt via federated learning should perform better than local models.  ... 
arXiv:2010.06223v3 fatcat:2lqnsn5frzhmvbevonubux6hcq

FDNAS: Improving Data Privacy and Model Diversity in AutoML [article]

Chunhui Zhang, Yongyuan Liang, Xiaoming Yuan, Lei Cheng
2020 arXiv   pre-print
In particular, how to efficiently search the optimal neural architecture directly from massive non-iid data of clients in a federated manner remains to be a hard nut to crack.  ...  non-iid data of clients.  ...  More importantly, in federated learning, the difference in distribution between proxy data and target data will be larger due to the presence of non-iid data.  ... 
arXiv:2011.03372v1 fatcat:akhyxyaqsfe7hdj3ctg2vdyskq

Towards Tailored Models on Private AIoT Devices: Federated Direct Neural Architecture Search [article]

Chunhui Zhang, Xiaoming Yuan, Qianyun Zhang, Guangxu Zhu, Lei Cheng, Ning Zhang
2022 arXiv   pre-print
In particular, how to efficiently search the optimal neural architecture directly from massive non-independent and identically distributed (non-IID) data among AIoT devices in a federated manner is a hard  ...  non- IID data across devices.  ...  ., learning rate decay) to demystify their impacts on non-IID data [14] . In addition, a global data-sharing strategy is proposed to improve the accuracy of the algorithm on non-IID data [37] .  ... 
arXiv:2202.11490v1 fatcat:tkbiffmfmrdtjfmt3uq53ajpm4

FedML: A Research Library and Benchmark for Federated Machine Learning [article]

Chaoyang He, Songze Li, Jinhyun So, Xiao Zeng, Mi Zhang, Hongyi Wang, Xiaoyang Wang, Praneeth Vepakomma, Abhishek Singh, Hang Qiu, Xinghua Zhu, Jianzong Wang (+8 others)
2020 arXiv   pre-print
Federated learning (FL) is a rapidly growing research field in machine learning.  ...  FedML supports three computing paradigms: on-device training for edge devices, distributed computing, and single-machine simulation.  ...  Federated Neural Architecture Search (FedNAS).  ... 
arXiv:2007.13518v4 fatcat:tyoav4xm3bgqbdy2gctnjfeb5i

Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search [article]

Chaoyang He, Murali Annavaram, Salman Avestimehr
2021 arXiv   pre-print
However, this predefined architecture may not be the optimal choice because it may not fit data with non-identical and independent distribution (non-IID).  ...  Our experiments on non-IID dataset show that the architecture searched by FedNAS can outperform the manually predefined architecture.  ...  Conclusion We study automating federated learning (AutoFL) via Neural Architecture Search (NAS) by proposing a Federated NAS (FedNAS) algorithm that can help scatter workers collaboratively searching for  ... 
arXiv:2004.08546v4 fatcat:eny4ndhbvbhtxico5mowjccuqa

ASFGNN: Automated Separated-Federated Graph Neural Network [article]

Longfei Zheng, Jun Zhou, Chaochao Chen, Bingzhe Wu, Li Wang, Benyu Zhang
2020 arXiv   pre-print
Specifically, to solve the data Non-IID problem, we first propose a separated-federated GNN learning model, which decouples the training of GNN into two parts: the message passing part that is done by  ...  However, in practice, such data are usually isolated by different data owners (clients) and thus are likely to be Non-Independent and Identically Distributed (Non-IID).  ...  Experiments on real world datasets demonstrated that our model significantly outperformed the federated GNN learning on the isolated Non-IID data.  ... 
arXiv:2011.03248v1 fatcat:gmrl2mbczvhq3oeeviw54wjbea

Training Keyword Spotting Models on Non-IID Data with Federated Learning

Andrew Hard, Kurt Partridge, Cameron Nguyen, Niranjan Subrahmanya, Aishanee Shah, Pai Zhu, Ignacio Lopez Moreno, Rajiv Mathews
2020 Interspeech 2020  
To overcome the algorithmic constraints associated with fitting on-device data (which are inherently non-independent and identically distributed), we conduct thorough empirical studies of optimization  ...  We demonstrate that a production-quality keyword-spotting model can be trained on-device using federated learning and achieve comparable false accept and false reject rates to a centrally-trained model  ...  Optimizers and learning rate schedules Optimization techniques were explored for non-IID training. First, the algorithms described in Section 3 were tuned via grid searches.  ... 
doi:10.21437/interspeech.2020-3023 dblp:conf/interspeech/HardPNSSZLM20 fatcat:5ggpsfoi25fkld7c7wdqkfafde

Training Keyword Spotting Models on Non-IID Data with Federated Learning [article]

Andrew Hard, Kurt Partridge, Cameron Nguyen, Niranjan Subrahmanya, Aishanee Shah, Pai Zhu, Ignacio Lopez Moreno, Rajiv Mathews
2020 arXiv   pre-print
To overcome the algorithmic constraints associated with fitting on-device data (which are inherently non-independent and identically distributed), we conduct thorough empirical studies of optimization  ...  We demonstrate that a production-quality keyword-spotting model can be trained on-device using federated learning and achieve comparable false accept and false reject rates to a centrally-trained model  ...  The authors would like to thank Google Research colleagues for providing the FL framework, Manzil Zaheer for his optimizer expertise, and Daniel Park for SpecAugment discussions. References  ... 
arXiv:2005.10406v2 fatcat:yhfuwayucjandmnawwvtdclwie

Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges [article]

Latif U. Khan, Walid Saad, Zhu Han, Ekram Hossain, Choong Seon Hong
2021 arXiv   pre-print
To overcome this challenge, federated learning can be a promising solution that enables on-device machine learning without the need to migrate the private end-user data to a central cloud.  ...  In this paper, first, we present the recent advances of federated learning towards enabling federated learning-powered IoT applications.  ...  Multi-branch networks For fixed hardware design, we can search different neural architectures for optimal out of available ones.  ... 
arXiv:2009.13012v2 fatcat:4oqifqi5czfyxiqe7gjewmuzsq

Federated Neural Architecture Search [article]

Jinliang Yuan, Mengwei Xu, Yuxin Zhao, Kaigui Bian, Gang Huang, Xuanzhe Liu, Shangguang Wang
2022 arXiv   pre-print
In this work, we propose an automatic neural architecture search into the decentralized training, as a new DNN training paradigm called Federated Neural Architecture Search, namely federated NAS.  ...  FedNAS fully exploits the key opportunity of insufficient model candidate re-training during the architecture search process, and incorporates three key optimizations: parallel candidates training on partial  ...  This becomes a bottleneck of practically deploying federated learning, given the increasingly important role of neural architecture search (or NAS) in launching deep learning in reality.  ... 
arXiv:2002.06352v5 fatcat:6l76motzp5bpbb5jnhfktvzvgm

Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning [article]

Liangqiong Qu, Yuyin Zhou, Paul Pu Liang, Yingda Xia, Feifei Wang, Ehsan Adeli, Li Fei-Fei, Daniel Rubin
2022 arXiv   pre-print
Concretely, we conduct the first rigorous empirical investigation of different neural architectures across a range of federated algorithms, real-world benchmarks, and heterogeneous data splits.  ...  In this paper, we demonstrate that self-attention-based architectures (e.g., Transformers) are more robust to distribution shifts and hence improve federated learning over heterogeneous data.  ...  Transformers in Federated Learning In this section, we present background on Transformer architectures and federated learning methods. Vision Architectures CNN.  ... 
arXiv:2106.06047v2 fatcat:nlbpw53xxnek5ilys7ga7cwfdy

SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learning [article]

Sixing Yu, Phuong Nguyen, Waqwoya Abebe, Wei Qian, Ali Anwar, Ali Jannesari
2022 arXiv   pre-print
Federated learning (FL) facilitates the training and deploying AI models on edge devices.  ...  Preserving user data privacy in FL introduces several challenges, including expensive communication costs, limited resources, and data heterogeneity.  ...  Lastly, training data on client edge devices depends on the user's unique usage causing an overall non-IID [3] , [23] user dataset.  ... 
arXiv:2111.14345v2 fatcat:wogvgqhkhjbj3k644sxv4zhife

Federated Graph Neural Networks: Overview, Techniques and Challenges [article]

Rui Liu, Han Yu
2022 arXiv   pre-print
This has led to the rapid development of federated graph neural networks (FedGNNs) research in recent years.  ...  We propose a unique 3-tiered taxonomy of the FedGNNs literature to provide a clear view into how GNNs work in the context of Federated Learning (FL).  ...  ., 2021] , inductive federated learning scheme is proposed to deal with spatio-temporal data leveraging an alternating optimization procedure.  ... 
arXiv:2202.07256v1 fatcat:pkq76uijxjetxmsyqxm5uxn5zu

Federated Learning Meets Natural Language Processing: A Survey [article]

Ming Liu, Stella Ho, Mengqi Wang, Longxiang Gao, Yuan Jin, He Zhang
2021 arXiv   pre-print
Federated Learning aims to learn machine learning models from multiple decentralized edge devices (e.g. mobiles) or servers without sacrificing local data privacy.  ...  However, both big deep neural and language models are trained with huge amounts of data which often lies on the server side.  ...  We organize the survey as follows: in Section 2, basic federated learning concepts, frameworks, optimization toward non-IID data, privacy are discussed. Section 3 reviews federated learning in NLP.  ... 
arXiv:2107.12603v1 fatcat:ebi4i6jnxbhihe7zuqx4uposbm

A Survey on Surrogate-assisted Efficient Neural Architecture Search [article]

Shiqing Liu, Haoyu Zhang, Yaochu Jin
2022 arXiv   pre-print
Neural architecture search (NAS) has become increasingly popular in the deep learning community recently, mainly because it can provide an opportunity to allow interested users without rich expertise to  ...  benefit from the success of deep neural networks (DNNs).  ...  [150] proposed FedNAS, a distributed NAS algorithm to search for optimal architectures in federated learning with non-IID data distribution.  ... 
arXiv:2206.01520v2 fatcat:4artwgoyw5fzph2m25xanowywe
« Previous Showing results 1 — 15 out of 310 results