Filters








74 Hits in 3.9 sec

Self-Distillation Mixup Training for Non-autoregressive Neural Machine Translation [article]

Jiaxin Guo, Minghan Wang, Daimeng Wei, Hengchao Shang, Yuxia Wang, Zongyao Li, Zhengzhe Yu, Zhanglin Wu, Yimeng Chen, Chang Su, Min Zhang, Lizhi Lei (+2 others)
2021 arXiv   pre-print
Recently, non-autoregressive (NAT) models predict outputs in parallel, achieving substantial improvements in generation speed compared to autoregressive (AT) models.  ...  An effective training strategy to improve the performance of AT models is Self-Distillation Mixup (SDM) Training, which pre-trains a model on raw data, generates distilled data by the pre-trained model  ...  Recently, non-autoregressive (NAT) models generate all outputs in parallel.  ... 
arXiv:2112.11640v1 fatcat:vfwfjbwb7va6nh7z5e4a7pdjzm

Unsupervised Cross-Domain Singing Voice Conversion [article]

Adam Polyak, Lior Wolf, Yossi Adi, Yaniv Taigman
2020 arXiv   pre-print
Our method utilizes both an acoustic model, trained for the task of automatic speech recognition, together with melody extracted features to drive a waveform-based generator.  ...  The proposed generative architecture is invariant to the speaker's identity and can be trained to generate target singers from unlabeled training data, using either speech or singing sources.  ...  The model is trained with a mixup batch every 3 steps after 100K steps of training had passed.  ... 
arXiv:2008.02830v1 fatcat:gmiltidfofe2bdqqxkjwtv4hhe

Unsupervised Cross-Domain Singing Voice Conversion

Adam Polyak, Lior Wolf, Yossi Adi, Yaniv Taigman
2020 Interspeech 2020  
Our method utilizes both an acoustic model, trained for the task of automatic speech recognition, together with melody extracted features to drive a waveform-based generator.  ...  The proposed generative architecture is invariant to the speaker's identity and can be trained to generate target singers from unlabeled training data, using either speech or singing sources.  ...  The model is trained with a mixup batch every 3 steps after 100K steps of training had passed.  ... 
doi:10.21437/interspeech.2020-1862 dblp:conf/interspeech/PolyakWAT20 fatcat:xcnchkemkreorihgfysy2wdh6i

Multi-SpectroGAN: High-Diversity and High-Fidelity Spectrogram Generation with Adversarial Style Combination for Speech Synthesis [article]

Sang-Hoon Lee, Hyun-Wook Yoon, Hyeong-Rae Noh, Ji-Hoon Kim, Seong-Whan Lee
2020 arXiv   pre-print
This leads to better guidance for generator training.  ...  Because adversarial feedback alone is not sufficient to train the generator, current models still require the reconstruction loss compared with the ground-truth and the generated mel-spectrogram directly  ...  To solve these problems, several non-autoregressive models have been proposed for faster generation.  ... 
arXiv:2012.07267v1 fatcat:cms2ugs23jhpvnneq57yjrk2fy

Adversarial Learning for Improved Onsets and Frames Music Transcription

Jong Wook Kim, Juan Bello
2019 Zenodo  
Our approach is generic and applicable to any transcription model based on multi-label predictions, which are very common in music signal analysis.  ...  To address this issue, we introduce an adversarial training scheme that operates directly on the time-frequency representations and makes the output distribution closer to the ground-truth.  ...  To avoid this, a non-saturating variant of GAN is suggested in [16] where the generator is trained with the following optimization objective instead: max G Ez log D(G(z)). (5) The non-saturating GAN  ... 
doi:10.5281/zenodo.3527898 fatcat:yu2oeub5treudj2gzgpsb2zs5q

COUnty aggRegation mixup AuGmEntation (COURAGE) COVID-19 prediction

Siawpeng Er, Shihao Yang, Tuo Zhao
2021 Scientific Reports  
This paper proposes a method named COURAGE (COUnty aggRegation mixup AuGmEntation) to generate a short-term prediction of 2-week-ahead COVID-19 related deaths for each county in the United States, leveraging  ...  One way to improve the model's prediction on emerging trends is to improve its generalization with augmented input data.  ...  With a more drastic change in the trend, a model trained with mixup data augmentation generalizes better, hence outperforming both Naive and County models.  ... 
doi:10.1038/s41598-021-93545-6 pmid:34253768 pmcid:PMC8275764 fatcat:kvdpmlhbnvaaxhpxunalia7jqi

Deep Learning in Neuroimaging: Overcoming Challenges With Emerging Approaches

Jason Smucny, Ge Shi, Ian Davidson
2022 Frontiers in Psychiatry  
We first introduce two methods that can reduce the amount of training data required to develop accurate models.  ...  incorporate knowledge learned from one data source (e.g., fMRI data from one site) and apply it toward learning from a second data source (e.g., data from another site), and (2) data augmentation (via Mixup  ...  in Total Brief Psychiatric Rating score after 1 year of treatment ("Improver")] with that of a patient with a poor clinical outcome ("Non-Improver").  ... 
doi:10.3389/fpsyt.2022.912600 pmid:35722548 pmcid:PMC9200984 fatcat:fiztnwk3vbh7lpyy7wkts6f2a4

COUnty aggRegation mixup AuGmEntation (COURAGE) COVID-19 Prediction [article]

Siawpeng Er, Shihao Yang, Tuo Zhao
2021 arXiv   pre-print
This paper proposes a method named COURAGE (COUnty aggRegation mixup AuGmEntation) to generate a short-term prediction of 2-week-ahead COVID-19 related deaths for each county in the United States, leveraging  ...  One way to improve the model's prediction on emerging trends is to improve its generalization with augmented input data.  ...  With a more drastic change in the trend, a model trained with mixup data augmentation generalizes better, hence outperforming both Naive and County models.  ... 
arXiv:2105.00620v2 fatcat:6whft7azkfaw5oacy3lc4tuphe

PitchNet: Unsupervised Singing Voice Conversion with Pitch Adversarial Network [article]

Chengqi Deng, Chengzhu Yu, Heng Lu, Chao Weng, Dong Yu
2020 arXiv   pre-print
Recent work shows that unsupervised singing voice conversion can be achieved with an autoencoder-based approach [1].  ...  Our evaluation shows that the proposed method can greatly improve the quality of the converted singing voice (2.92 vs 3.75 in MOS).  ...  During the training process, backtranslation and mixup [1] were employed to improve the conversion.  ... 
arXiv:1912.01852v2 fatcat:fpkpq62n2fhmdgvz2rxxzxpjrm

Data Augmentation for Deep Graph Learning: A Survey [article]

Kaize Ding, Zhe Xu, Hanghang Tong, Huan Liu
2022 arXiv   pre-print
However, conventional data augmentation methods can hardly handle graph-structured data which is defined on non-Euclidean space with multi-modality.  ...  ., 2018] where the former models a graph in an autoregressive manner and the latter trains a generator using the Wasserstein generative adversarial net objective.  ...  ., 2021] performs semantic-level feature mixup by constructing semantic relation spaces and edge mixup with an edge predictor trained on two well-designed context-based self-supervised tasks, which is  ... 
arXiv:2202.08235v1 fatcat:5jutv3soenfh3ikbrgeckeynw4

Towards Domain-Agnostic Contrastive Learning [article]

Vikas Verma, Minh-Thang Luong, Kenji Kawaguchi, Hieu Pham, Quoc V. Le
2021 arXiv   pre-print
Our results show that DACL not only outperforms other domain-agnostic noising methods, such as Gaussian-noise, but also combines well with domain-specific methods, such as SimCLR, to improve self-supervised  ...  Key to our approach is the use of Mixup noise to create similar and dissimilar examples by mixing data samples differently either at the input or hidden-state levels.  ...  that Mixup-noise has better generalization bounds than Gaussian-noise. • We show that using other forms of data-dependent noise (geometric-mixup, binary-mixup) can further improve the performance of DACL  ... 
arXiv:2011.04419v2 fatcat:jaep2yi2rndfxcowehpgtcgjue

Autoregressive Perturbations for Data Poisoning [article]

Pedro Sandoval-Segura, Vasu Singla, Jonas Geiping, Micah Goldblum, Tom Goldstein, David W. Jacobs
2022 arXiv   pre-print
In this work, we introduce autoregressive (AR) poisoning, a method that can generate poisoned data without access to the broader dataset.  ...  Unfortunately, existing methods require knowledge of both the target architecture and the complete dataset so that a surrogate network can be trained, the parameters of which are used to generate the attack  ...  Note that these AR perturbations are fast to generate, do not require a pre-trained surrogate model, and can be generated independently from the data. Why do Autoregressive Perturbations Work?  ... 
arXiv:2206.03693v2 fatcat:cbfnhusaorba5cmd6paacolcxi

Multi-scale Attention Flow for Probabilistic Time Series Forecasting [article]

Shibo Feng and Ke Xu and Jiaxiang Wu and Pengcheng Wu and Fan Lin and Peilin Zhao
2022 arXiv   pre-print
Additionally, compared with autoregressive modeling methods, our model avoids the influence of cumulative error and does not increase the time complexity.  ...  In this work, we proposed a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF), where we integrate multi-scale attention and relative position information  ...  Additionally, compared with the autoregressive modeling methods, we avoid the cumulative error of multi-step forecasting and greatly improve the parallelism with the non-autoregressive way.  ... 
arXiv:2205.07493v1 fatcat:cnk5eebcmredjff2kvj3mfxq5e

Distance Metric-Based Learning with Interpolated Latent Features for Location Classification in Endoscopy Image and Video [article]

Mohammad Reza Mohebbian, Khan A. Wahid, Anh Dinh, Paul Babyn
2021 arXiv   pre-print
The manifold mixup process improves few-shot learning by increasing the number of training epochs while reducing overfitting, as well as providing more accurate decision boundaries.  ...  The proposed method is also compared with various methods trained on categorical cross-entropy loss and produced better results which show that proposed method has potential to be used for endoscopy image  ...  Besides, the impact of manifold mixup scheme on performance is also investigated. For this purpose, the SNN without manifold mixup is trained and compared with the proposed method.  ... 
arXiv:2103.08504v2 fatcat:uvmcft3bufbcha2owktoztpu34

DeLoRes: Decorrelating Latent Spaces for Low-Resource Audio Representation Learning [article]

Sreyan Ghosh and Ashish Seth and and Deepak Mittal and Maneesh Singh and S. Umesh
2022 arXiv   pre-print
Inspired from the Barlow Twins objective function, we propose to learn embeddings that are invariant to distortions of an input audio sample, while making sure that they contain non-redundant information  ...  Inspired by the recent progress in self-supervised learning for computer vision, in this paper we introduce DeLoRes, a new general-purpose audio representation learning approach.  ...  Acknowledgments We thank Centre For Development Of Advanced Computing (CDAC), as part of the National Language Translation Mission for providing us with the compute resources required for the experiments  ... 
arXiv:2203.13628v3 fatcat:bmpxhdekmvaxzl3m5ie73gixn4
« Previous Showing results 1 — 15 out of 74 results