4,180 Hits in 5.4 sec

Checkpoint Ensembles: Ensemble Methods from a Single Training Process [article]

Hugh Chen and Scott Lundberg and Su-In Lee
2017 arXiv   pre-print
We present the checkpoint ensembles method that can learn ensemble models on a single training process.  ...  Checkpoint ensembles improve performance by averaging the predictions from "checkpoints" of the best models within single training process.  ...  We present and analyze a method to capture effects of traditional ensemble methods within a single training process. Checkpoint ensembles (CE) provide the following benefits: 1.  ... 
arXiv:1710.03282v1 fatcat:lns5xy347rhpzehsjgpb3a4bgi

Classification of lung nodules in CT scans using three-dimensional deep convolutional neural networks with a checkpoint ensemble method

Hwejin Jung, Bumsoo Kim, Inyeop Lee, Junhyun Lee, Jaewoo Kang
2018 BMC Medical Imaging  
In addition, we use an alternative ensemble method called the checkpoint ensemble method to boost performance.  ...  Conclusion: The result demonstrates that our method of using a 3D DCNN with shortcut connections, a 3D DCNN with dense connections, and the checkpoint ensemble method is effective for capturing 3D features  ...  In addition, we apply a checkpoint ensemble method to boost nodule classification performance.  ... 
doi:10.1186/s12880-018-0286-0 fatcat:w7jjzdugpbe5lglmgdc5obvsjm

Auto-Ensemble: An Adaptive Learning Rate Scheduling based Deep Learning Model Ensembling [article]

Jun Yang, Fei Wang
2020 arXiv   pre-print
Ensembling deep learning models is a shortcut to promote its implementation in new scenarios, which can avoid tuning neural networks, losses and training algorithms from scratch.  ...  When the number of lo-cal optimal solutions tends to be saturated, all the collected checkpoints are used for ensemble. Our method is universal, it can be applied to various scenarios.  ...  Checkpoint ensemble provides a method to collect abundant models within one single training process [15] .  ... 
arXiv:2003.11266v1 fatcat:tc6rz5gl4jbdxebdcrsaqipylm

Boost Neural Networks by Checkpoints [article]

Feng Wang, Guoyizhe Wei, Qiao Liu, Jinxiang Ou, Xian Wei, Hairong Lv
2021 arXiv   pre-print
Several recent works attempt to save and ensemble the checkpoints of DNNs, which only requires the same computational cost as training a single network.  ...  In this paper, we propose a novel method to ensemble the checkpoints, where a boosting scheme is utilized to accelerate model convergence and maximize the checkpoint diversity.  ...  Similarly, another method FGE (Fast Geometric Ensembling) [Garipov et al., 2018 ] copies a trained model and further fine-tunes it with a cyclical learning rate, saving checkpoints and ensembling them  ... 
arXiv:2110.00959v2 fatcat:e4hkignpifhehga2glkhsksrwi

Auto-Ensemble: An Adaptive Learning Rate Scheduling based Deep Learning Model Ensembling

Jun Yang, Fei Wang
2020 IEEE Access  
Checkpoint ensemble provides a method to collect abundant models within one single training process [16] .  ...  After having collected a checkpoint of model, the learning rate rises to escape from it, and start a new searching process. Finally, all the collected model checkpoints are used for ensemble.  ...  By collecting the checkpoint of models, the ensemble accuracy can greatly exceed accuracy of single model.  ... 
doi:10.1109/access.2020.3041525 fatcat:7n3wiqdvevf3bg7olwmaksacja

Training Data Subset Search with Ensemble Active Learning [article]

Kashyap Chitta, Jose M. Alvarez, Elmar Haussmann, Clement Farabet
2020 arXiv   pre-print
We do this with ensembles of hundreds of models, obtained at a minimal computational cost by reusing intermediate training checkpoints.  ...  In this paper, we propose to scale up ensemble Active Learning (AL) methods to perform acquisition at a large scale (10k to 500k samples at a time).  ...  If we train a single DNN, we only obtain a single sample from the distribution p(θ|L).  ... 
arXiv:1905.12737v3 fatcat:xj2zrozkurcxxnovr6tcaq263m

Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks

Tim Whitaker, Darrell Whitley
We introduce a fast, low-cost method for creating diverse ensembles of neural networks without needing to train multiple models from scratch. We do this by first training a single parent network.  ...  We then briefly train each child network for a small number of epochs, which now converge significantly faster when compared to training from scratch.  ...  Evolutionary Ensembles take a single network and generate explicit child networks according to a perturbative process.  ... 
doi:10.1609/aaai.v36i8.20842 fatcat:sl6ioyshk5gz5enl3yeoqwbfnq

Sparse MoEs meet Efficient Ensembles [article]

James Urquhart Allingham, Florian Wenzel, Zelda E Mariet, Basil Mustafa, Joan Puigcerver, Neil Houlsby, Ghassen Jerfel, Vincent Fortuin, Balaji Lakshminarayanan, Jasper Snoek, Dustin Tran, Carlos Riquelme Ruiz (+1 others)
2021 arXiv   pre-print
Then, we present partitioned batch ensembles, an efficient ensemble of sparse MoEs that takes the best of both classes of models.  ...  We study the interplay of two popular classes of such models: ensembles of neural networks and sparse mixture of experts (sparse MoEs).  ...  This choice was motivated by the fact that like ViT, V-MoE, and pBE, down-DE requires a only a single upstream checkpoint, which all of the methods more comparable.  ... 
arXiv:2110.03360v1 fatcat:iaycbzllqjfixp5f3pql33clpa

Ensemble Distillation for Structured Prediction: Calibrated, Accurate, Fast-Choose Three [article]

Steven Reich, David Mueller, Nicholas Andrews
2021 arXiv   pre-print
single model during test-time.  ...  Modern neural networks do not always produce well-calibrated predictions, even when trained with a proper scoring function such as cross-entropy.  ...  In this section, we explore whether these findings can be mirrored by an ensemble which is built from a single optimization trajectory, built from multiple checkpoints.  ... 
arXiv:2010.06721v2 fatcat:k7hurbf6hzdjphvvgzcuc5ueie

Sequential Bayesian Neural Subnetwork Ensembles [article]

Sanket Jantre, Sandeep Madireddy, Shrijita Bhattacharya, Tapabrata Maiti, Prasanna Balaprakash
2022 arXiv   pre-print
We propose sequential ensembling of dynamic Bayesian neural subnetworks that systematically reduce model complexity through sparsity-inducing priors and generate diverse ensembles in a single forward pass  ...  Furthermore, we found that our approach produced the most diverse ensembles compared to the approaches with a single forward pass and even compared to the approaches with multiple forward passes in some  ...  Our ensembling strategy produces diverse set of base learners from a single end-to-end training process. It consists of an exploration phase followed by M exploitation phases.  ... 
arXiv:2206.00794v1 fatcat:zke4v34plrgvnfsodhqimqbpti

Assessing the Benefits of Model Ensembles in Neural Re-Ranking for Passage Retrieval [article]

Luís Borges, Bruno Martins, Jamie Callan
2021 arXiv   pre-print
Starting from relatively standard neural models, we use a previous technique named Fast Geometric Ensembling to generate multiple model instances from particular training schedules, then focusing or attention  ...  Our work aimed at experimentally assessing the benefits of model ensembling within the context of neural methods for passage reranking.  ...  This research was supported by Fundação para a Ciência e Tecnologia (FCT), through the Ph.D. scholarship with reference SFRH/BD/150497/2019, and the INESC-ID multi-annual funding from the PIDDAC programme  ... 
arXiv:2101.08705v1 fatcat:jhnyk3eksvawnljdj6p6s4lxfq

DynE: Dynamic Ensemble Decoding for Multi-Document Summarization [article]

Chris Hokamp, Demian Gholipour Ghalandari, Nghia The Pham, John Glover
2020 arXiv   pre-print
into a single output sequence.  ...  In this work we propose a simple decoding methodology which ensembles the output of multiple instances of the same model on different inputs.  ...  A simple ensembling method is presented that allows models trained for single inputs to be used in multi-input settings. 2.  ... 
arXiv:2006.08748v1 fatcat:dnrx2f4tczbr7dsf4wo3lpztli

Ensembling Graph Predictions for AMR Parsing [article]

Hoang Thanh Lam, Gabriele Picco, Yufang Hou, Young-Suk Lee, Lam M. Nguyen, Dzung T. Phan, Vanessa López, Ramon Fernandez Astudillo
2022 arXiv   pre-print
On the other hand, ensemble methods combine predictions from multiple models to create a new one that is more robust and accurate than individual predictions.  ...  In many machine learning tasks, models are trained to predict structure data such as graphs.  ...  Ensemble of AMR predictions from a single type of model is studied in [Zhou et al., 2021] where the authors demonstrated that by combining predictions from three different model's checkpoints they gain  ... 
arXiv:2110.09131v2 fatcat:5jhi7fvasrgrpf226vwf6743mi

Frustratingly Easy Model Ensemble for Abstractive Summarization

Hayato Kobayashi
2018 Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing  
In this paper, we propose an alternative, simple but effective unsupervised ensemble method, post-ensemble, that combines multiple models by selecting a majority-like output in post-processing.  ...  Experimental results on a newsheadline-generation task show that the proposed method performs better than the current ensemble methods.  ...  The first one, self-ensemble, is a method of extracting models from "checkpoints" saved in each epoch in a training. We prepared the models of self-ensemble by using 10 checkpoints from 4-13 epochs.  ... 
doi:10.18653/v1/d18-1449 dblp:conf/emnlp/Kobayashi18 fatcat:xioycfexuver3m2quz2n3azgli

Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs [article]

Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, Andrew Gordon Wilson
2018 arXiv   pre-print
Using FGE we can train high-performing ensembles in the time required to train a single model.  ...  Inspired by this new geometric insight, we also propose a new ensembling method entitled Fast Geometric Ensembling (FGE).  ...  to train a single model.  ... 
arXiv:1802.10026v4 fatcat:ayy7sz37trhd5aw64vvii4pfwi
« Previous Showing results 1 — 15 out of 4,180 results