A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Filters
Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization
[article]
2021
arXiv
pre-print
Preserved Fulltext
Other Versions
We first show that standard fine-tuning after pre-training destroys some of this structure. We then propose composed fine-tuning, which trains a predictor composed with the pre-trained denoiser. ...
We prove for two-layer ReLU networks that composed fine-tuning significantly reduces the complexity of the predictor, thus improving generalization. ...
Effect of pre-training objectives Naively, better pre-trained denoisers should improve the gains with composed fine-tuning. ...
arXiv:2006.16205v3
fatcat:6cg3pl4bv5gxdgeiqrwe3fmpku
User-specific Adaptive Fine-tuning for Cross-domain Recommendations
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
However, current methods are mainly based on the global fine-tuning strategy: the decision of which layers of the pre-trained model to freeze or fine-tune is taken for all users in the target domain. ...
As such, we propose a novel User-specific Adaptive Fine-tuning method (UAF), selecting which layers of the pre-trained network to fine-tune, on a per-user basis. ...
Similar to [40] , [41] , we generate all freezing/fine-tuning policies for all residual blocks at once for the trade-off of efficiency and accuracy. ...
arXiv:2106.07864v2
fatcat:2juhc4smfbcf5b5ogrl2ovkum4
Learning Transferable Features for Speech Emotion Recognition
2017
Proceedings of the on Thematic Workshops of ACM Multimedia 2017 - Thematic Workshops '17
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
In this paper, we propose a deep architecture that jointly exploits a convolutional network for extracting domain-shared features and a long short-term memory network for classifying emotions using domain-specific ...
We evaluate several domain adaptation approaches, and we perform an ablation study to understand which source domains add the most to the overall recognition effectiveness for a given target domain. ...
ACKNOWLEDGEMENTS We thank the partial support given by the Brazilian National Institute of Science and Technology for the Web (grant MCT-CNPq 573871/2008-6), Project: Models, Algorithms and Systems for ...
doi:10.1145/3126686.3126735
dblp:conf/mm/MarczewskiVZ17
fatcat:l6b3h6yxp5eddoi6ist42xwewy
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
BART is particularly effective when fine tuned for text generation but also works well for comprehension tasks. ...
We present BART, a denoising autoencoder for pretraining sequence-to-sequence models. ...
Future work should explore new methods for corrupting documents for pre-training, perhaps tailoring them to specific end tasks. ...
arXiv:1910.13461v1
fatcat:gxhwv2jwlbayhhonjdfksrcpou
Masked Autoencoders Are Scalable Vision Learners
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This paper shows that masked autoencoders (MAE) are scalable self-supervised learners for computer vision. ...
Coupling these two designs enables us to train large models efficiently and effectively: we accelerate training (by 3x or more) and improve accuracy. ...
Our MAE pre-training does not use it. For any specialized fine-tuning. ...
arXiv:2111.06377v3
fatcat:4d7762easfdcniz4jvqedqizqy
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
2017
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
We propose split-brain autoencoders, a straightforward modification of the traditional autoencoder architecture, for unsupervised representation learning. ...
Each sub-network is trained to perform a difficult taskpredicting one subset of the data channels from another. Together, the sub-networks extract features from the entire input signal. ...
Acknowledgements We thank members of the Berkeley Artificial Intelligence Research Lab (BAIR), in particular Andrew Owens, for helpful discussions, as well as Saurabh Gupta for help with RGB-D experiments ...
doi:10.1109/cvpr.2017.76
dblp:conf/cvpr/ZhangIE17
fatcat:boaqjncxxrcklavpco4b5mrku4
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
[article]
2017
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We propose split-brain autoencoders, a straightforward modification of the traditional autoencoder architecture, for unsupervised representation learning. ...
Each sub-network is trained to perform a difficult task -- predicting one subset of the data channels from another. Together, the sub-networks extract features from the entire input signal. ...
Acknowledgements We thank members of the Berkeley Artificial Intelligence Research Lab (BAIR), in particular Andrew Owens, for helpful discussions, as well as Saurabh Gupta for help with RGB-D experiments ...
arXiv:1611.09842v3
fatcat:lja7ousv2zgo3hqtxafz7xsyvm
SPIRAL: Self-supervised Perturbation-Invariant Representation Learning for Speech Pre-Training
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
training solely in the fine-tuning stage. ...
We introduce a new approach for speech pre-training named SPIRAL which works by learning denoising representation of perturbed data in a teacher-student framework. ...
pre-training is more effective than solely applying multi-condition training in the fine-tuning stage.We presume SPIRAL as a general pre-training method, which can apply to other modalities such as images ...
arXiv:2201.10207v3
fatcat:qyalg2exynbjflttbmggdh4qlm
Reusing Monolingual Pre-Trained Models by Cross-Connecting Seq2seq Models for Machine Translation
2021
Applied Sciences
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
This work uses sequence-to-sequence (seq2seq) models pre-trained on monolingual corpora for machine translation. ...
We pre-train two seq2seq models with monolingual corpora for the source and target languages, then combine the encoder of the source language model and the decoder of the target language model, i.e., the ...
The fluency of each pre-trained model is preserved intact.
Cross-Connection and Fine-Tuning We compose the pre-trained models for fine-tuning. ...
doi:10.3390/app11188737
fatcat:oepw35gjp5fubcotgnqjxvofqm
Laplacian Denoising Autoencoder
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this paper, we propose to learn data representations with a novel type of denoising autoencoder, where the noisy input data is generated by corrupting latent clean data in the gradient domain. ...
While deep neural networks have been shown to perform remarkably well in many machine learning tasks, labeling a large amount of ground truth data for supervised training is usually very costly to scale ...
The learned weights of our model trained on ImageNet are transferred to a standard AlexNet for the evaluation. We then fine-tune the model on the PASCAL VOC trainval set and test on the test set. ...
arXiv:2003.13623v1
fatcat:uel6btlupzbarey7rvr6i6ya7i
Online Learning of Deep Hybrid Architectures for Semi-supervised Categorization
[chapter]
2015
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We (1) compare this architecture against a hybrid denoising autoencoder version of itself as well as several other models and (2) investigate training in the context of an incremental learning procedure ...
The model's training algorithm is built on principles developed from hybrid discriminative Boltzmann machines and composes deep architectures in a greedy fashion. ...
We thank Hugo Larochelle and Roberto Calandra for their correspondence and advice, although all shortcomings of the presented work are ours. ...
doi:10.1007/978-3-319-23528-8_32
fatcat:znn2gsczwzhd3kypvnfg2meayu
Deep Learning Approaches for Detecting Freezing of Gait in Parkinson's Disease Patients through On-Body Acceleration Sensors
2020
Sensors
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Also, several machine and deep learning approaches for FOG detection are evaluated using a leave-one-subject-out (LOSO) cross-validation. ...
Freezing of gait (FOG) is one of the most incapacitating motor symptoms in Parkinson's disease (PD). The occurrence of FOG reduces the patients' quality of live and leads to falls. ...
Architecture for ML Denoiser Autoencoder For the novelty detection based in DL, a denoiser autoencoder composed of seven fully connected layers, according to the scheme presented in Figure 2 , was tested ...
doi:10.3390/s20071895
pmid:32235373
fatcat:7hfljuaik5fljfuod5ubesbsci
Test-Time Adaptable Neural Networks for Robust Medical Image Segmentation
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We employ an independently trained denoising autoencoder (DAE) in order to model such an implicit prior on plausible anatomical segmentation labels. ...
The proposed test-time adaptation consistently provides performance improvement, demonstrating the promise and generality of the approach. ...
We also thank NVIDIA corporation for their GPU donation. ...
arXiv:2004.04668v3
fatcat:lxh3hsoeifhffmx6pgwtrdwwkq
IPMiner: hidden ncRNA-protein interaction sequential pattern mining with stacked autoencoder for accurate computational prediction
2016
BMC Genomics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
Finally, stacked ensembling is used to integrate different predictors to further improve the prediction performance. ...
In addition, we further apply IPMiner for large-scale prediction of ncRNA-protein network, that achieves promising prediction performance. ...
Acknowledgements We thanks for the two anonymous reviewers's comments on this study. ...
doi:10.1186/s12864-016-2931-8
pmid:27506469
pmcid:PMC4979166
fatcat:ibwcovlqcndo7jlbxjp6xphpvi
Representation Learning Through Latent Canonicalizations
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We dub these transformations latent canonicalizers, as they aim to modify the value of a factor to a pre-determined (but arbitrary) canonical value (e.g., recoloring the image foreground to black). ...
We seek to learn a representation on a large annotated data source that generalizes to a target domain using limited new supervision. ...
Transfer Learning and Few-shot Learning: Alternatively, synthetic data can be used for pre-training followed by fine-tuning on real data (Richter et al., 2016 )-a form of transfer learning. ...
arXiv:2002.11829v1
fatcat:sqvupdu2bvgl7azfjtsfw5kdxe
« Previous
Showing results 1 — 15 out of 129 results