A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Filters
On Biased Compression for Distributed Learning
[article]
2021
arXiv
pre-print
Preserved Fulltext
Other Versions
We show for the first time that biased compressors can lead to linear convergence rates both in the single node and distributed settings. ...
In the last few years, various communication compression techniques have emerged as an indispensable tool helping to alleviate the communication bottleneck in distributed learning. ...
On Biased Compression for Distributed Learning
Aleksandr Beznosikov∗ Samuel Horváth ...
arXiv:2002.12410v2
fatcat:n45gzaj5z5fy5mbwa5ux66mxku
Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Although meta-learning has emerged as an approach for endowing neural networks with useful inductive biases, agents trained by meta-learning may acquire very different strategies from humans. ...
We show that co-training these agents on predicting representations from natural language task descriptions and from programs induced to generate such tasks guides them toward human-like inductive biases ...
By analogy, library-learned concepts allow for compression of programs (Fig. 4B ). ...
arXiv:2205.11558v1
fatcat:wqa4h3ogabbhpn4y26gcfaqdxq
Semantic Compression of Episodic Memories
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We formalise the compression of episodes in the normative framework of information theory and argue that semantic memory provides the distortion function for compression of experiences. ...
Recent advances and insights from machine learning allow us to approximate semantic compression in naturalistic domains and contrast the resulting deviations in compressed episodes with memory errors observed ...
Acknowledgements The authors thank the anonymous reviewers for useful comments and Ferenc Huszár for discussions. ...
arXiv:1806.07990v1
fatcat:3xuv6pk3dbh57gfsoqeqpvv56q
CosSGD: Communication-Efficient Federated Learning with a Simple Cosine-Based Quantization
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Further, our approach is highly suitable for federated learning problems since it has low computational complexity and requires only a little additional data to recover the compressed information. ...
Federated learning is a promising framework to mitigate data privacy and computation concerns. ...
The biased linear quantization has an error bound of bv 2 s −1 = cos(b θ ) 2 s −1 • v 2 for all the gradients on [−b v , b v ]. ...
arXiv:2012.08241v2
fatcat:pes6bfrkxveohcouipxa4uwdga
Debiasing Methods in Natural Language Understanding Make Bias More Accessible
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Model robustness to bias is often determined by the generalization on carefully designed out-of-distribution datasets. ...
We propose a general probing-based framework that allows for post-hoc interpretation of biases in language models, and use an information-theoretic approach to measure the extractability of certain biases ...
We thank Victor Sanh and Prasetya Ajie Utama for their feedback and help reproducing the models. We also thank the anonymous reviewers for their useful suggestions. ...
arXiv:2109.04095v1
fatcat:lrt56s2e55gzrjacb3plloiqei
Robust Simulation-Based Inference in Cosmology with Bayesian Neural Networks
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Simulation-based inference (SBI) is rapidly establishing itself as a standard machine learning technique for analyzing data in cosmological surveys. ...
outside the training distribution, which is mostly unconstrained. ...
The reason for this hyperparameter is that the covariance matrix estimated by SWA will depend on the learning rate. ...
arXiv:2207.08435v2
fatcat:a4dk3wqzhrgdriwaialxtgn43a
Generalization of prior information for rapid Bayesian time estimation
2016
Proceedings of the National Academy of Sciences of the United States of America
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
For example, a variety of biases in visual perception have been shown to be consistent with reliance on prior knowledge regarding statistical regularities in the environment, such as the distribution of ...
Similarly, Gekas et al. (24) showed that distinct priors for motion direction can be learned for sets of dot stimuli of different colors. ...
We thank Thomas Veale for assisting with data collection. This work was supported by Wellcome Trust Research Fellowship WT097387 (to N.W.R.). ...
doi:10.1073/pnas.1610706114
pmid:28007982
pmcid:PMC5240697
fatcat:egonw364j5autac2whrm4pi7ii
De-biasing facial detection system using VAE
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
The proposed approach uses generative models which are best suited for learning underlying features(latent variables) from the dataset and by using these learned features models try to reduce the threats ...
And then we train models on two datasets and compare the results. ...
Ultimately gives an algorithm for debiasing against learned biases based on the unsupervised latent space. ...
arXiv:2204.09556v1
fatcat:oh4t7ianwvdixjyktw2sdk754e
DESP: Deep Enhanced Sampling of Proteins' Conformation Spaces Using AI-Inspired Biasing Forces
2021
Frontiers in Molecular Biosciences
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
and deep neural networks (DNNs), in which biasing potentials for guiding the MD simulations are derived from the KL divergence between the DNN-learned latent space vectors of [a] the most recently sampled ...
In this work, we present Deep Enhanced Sampling of Proteins' Conformation Spaces Using AI-Inspired Biasing Forces (DESP), a technique for enhanced sampling that combines molecular dynamics (MD) simulations ...
on the distribution of the molecule's conformations in that compressed representation space. ...
doi:10.3389/fmolb.2021.587151
pmid:34026817
pmcid:PMC8132871
fatcat:4bmn2eyeyrdhrehzvvuxoutipm
FEDZIP: A Compression Framework for Communication-Efficient Federated Learning
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
It assigns the learning process independently to each client. First, clients locally train a machine learning model based on local data. ...
FedZip outperforms state-of-the-art compression frameworks and reaches compression rates up to 1085x, and preserves up to 99% of bandwidth and 99% of energy for clients during communication. ...
Table 1 : 1 Comparison of state-of-the-art distributed learning methods with compression and FedZip. ...
arXiv:2102.01593v1
fatcat:24gyfd4etzbwxjm2psqry4mace
Language as a Latent Variable: Discrete Generative Models for Sentence Compression
[article]
2016
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We formulate a variational auto-encoder for inference in this model and apply it to the task of compressing sentences. ...
In our empirical evaluation we show that generative formulations of both abstractive and extractive compression yield state-of-the-art results when trained on a large amount of supervised data. ...
Although we introduce a biased estimator by using pointer networks, it is still very difficult for the compression model to generate reasonable natural language sentences at the early stage of learning ...
arXiv:1609.07317v2
fatcat:fppddfdenfgkhjaadassg4dxcy
Language as a Latent Variable: Discrete Generative Models for Sentence Compression
2016
Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We formulate a variational auto-encoder for inference in this model and apply it to the task of compressing sentences. ...
In our empirical evaluation we show that generative formulations of both abstractive and extractive compression yield state-of-the-art results when trained on a large amount of supervised data. ...
Although we introduce a biased estimator by using pointer networks, it is still very difficult for the compression model to generate reasonable natural language sentences at the early stage of learning ...
doi:10.18653/v1/d16-1031
dblp:conf/emnlp/MiaoB16
fatcat:hdvrqfdv6rhddg3cniw5e7ft7m
Meta-Learning Sparse Compression Networks
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Secondly, introduce the first method allowing for sparsification to be employed in the inner-loop of commonly used Meta-Learning algorithms, drastically improving both compression and the computational ...
Recent work on such Implicit Neural Representations (INRs) has shown that - following careful architecture search - INRs can outperform established compression methods such as JPEG (e.g. ...
Their cooperation sets a high standard for scientific collaboration and should be applauded. ...
arXiv:2205.08957v2
fatcat:zgioj3pwijgztjr3c577cklucy
Autoencoders, Kernels, and Multilayer Perceptrons for Electron Micrograph Restoration and Compression
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
TEM autoencoders have been trained for 1×, 4×, 16× and 64× compression, STEM autoencoders for 1×, 4× and 16× compression and TEM+STEM autoencoders for 1×, 2×, 4×, 8×, 16×, 32× and 64× compression. ...
We present 14 autoencoders, 15 kernels and 14 multilayer perceptrons for electron micrograph restoration and compression. ...
Minima subtraction was applied to increase the similarity of input intensity distributions, making it easier for the autoencoders to learn. ...
arXiv:1808.09916v1
fatcat:canyqapil5aonkdvedth6ia2zu
Trends and Advancements in Deep Neural Network Communication
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This paper gives an overview over the recent advancements and challenges in this new field of research at the intersection of machine learning and communications. ...
To address the challenges of these distributed environments, a wide range of training and evaluation schemes have been developed, which require the communication of neural network parametrizations. ...
An established technique to reduce the impact of biased compression on the convergence speed is error accumulation. ...
arXiv:2003.03320v1
fatcat:tgs7b6nbovflngyuxhxwkmxhv4
« Previous
Showing results 1 — 15 out of 44,692 results