A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
Checkpoint Ensembles: Ensemble Methods from a Single Training Process
[article]
2017
arXiv
pre-print
Preserved Fulltext
We present the checkpoint ensembles method that can learn ensemble models on a single training process. ...
Checkpoint ensembles improve performance by averaging the predictions from "checkpoints" of the best models within single training process. ...
We present and analyze a method to capture effects of traditional ensemble methods within a single training process. Checkpoint ensembles (CE) provide the following benefits: 1. ...
arXiv:1710.03282v1
fatcat:lns5xy347rhpzehsjgpb3a4bgi
Classification of lung nodules in CT scans using three-dimensional deep convolutional neural networks with a checkpoint ensemble method
2018
BMC Medical Imaging
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
In addition, we use an alternative ensemble method called the checkpoint ensemble method to boost performance. ...
Conclusion: The result demonstrates that our method of using a 3D DCNN with shortcut connections, a 3D DCNN with dense connections, and the checkpoint ensemble method is effective for capturing 3D features ...
In addition, we apply a checkpoint ensemble method to boost nodule classification performance. ...
doi:10.1186/s12880-018-0286-0
fatcat:w7jjzdugpbe5lglmgdc5obvsjm
Auto-Ensemble: An Adaptive Learning Rate Scheduling based Deep Learning Model Ensembling
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Ensembling deep learning models is a shortcut to promote its implementation in new scenarios, which can avoid tuning neural networks, losses and training algorithms from scratch. ...
When the number of lo-cal optimal solutions tends to be saturated, all the collected checkpoints are used for ensemble. Our method is universal, it can be applied to various scenarios. ...
Checkpoint ensemble provides a method to collect abundant models within one single training process [15] . ...
arXiv:2003.11266v1
fatcat:tc6rz5gl4jbdxebdcrsaqipylm
Boost Neural Networks by Checkpoints
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2110.00959v1 fatcat:qcoh2xaa2zdvlb6rqpp63ikywm
Several recent works attempt to save and ensemble the checkpoints of DNNs, which only requires the same computational cost as training a single network. ...
In this paper, we propose a novel method to ensemble the checkpoints, where a boosting scheme is utilized to accelerate model convergence and maximize the checkpoint diversity. ...
Similarly, another method FGE (Fast Geometric Ensembling) [Garipov et al., 2018 ] copies a trained model and further fine-tunes it with a cyclical learning rate, saving checkpoints and ensembling them ...
arXiv:2110.00959v2
fatcat:e4hkignpifhehga2glkhsksrwi
Auto-Ensemble: An Adaptive Learning Rate Scheduling based Deep Learning Model Ensembling
2020
IEEE Access
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Checkpoint ensemble provides a method to collect abundant models within one single training process [16] . ...
After having collected a checkpoint of model, the learning rate rises to escape from it, and start a new searching process. Finally, all the collected model checkpoints are used for ensemble. ...
By collecting the checkpoint of models, the ensemble accuracy can greatly exceed accuracy of single model. ...
doi:10.1109/access.2020.3041525
fatcat:7n3wiqdvevf3bg7olwmaksacja
Training Data Subset Search with Ensemble Active Learning
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We do this with ensembles of hundreds of models, obtained at a minimal computational cost by reusing intermediate training checkpoints. ...
In this paper, we propose to scale up ensemble Active Learning (AL) methods to perform acquisition at a large scale (10k to 500k samples at a time). ...
If we train a single DNN, we only obtain a single sample from the distribution p(θ|L). ...
arXiv:1905.12737v3
fatcat:xj2zrozkurcxxnovr6tcaq263m
Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks
2022
PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
We introduce a fast, low-cost method for creating diverse ensembles of neural networks without needing to train multiple models from scratch. We do this by first training a single parent network. ...
We then briefly train each child network for a small number of epochs, which now converge significantly faster when compared to training from scratch. ...
Evolutionary Ensembles take a single network and generate explicit child networks according to a perturbative process. ...
doi:10.1609/aaai.v36i8.20842
fatcat:sl6ioyshk5gz5enl3yeoqwbfnq
Sparse MoEs meet Efficient Ensembles
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Then, we present partitioned batch ensembles, an efficient ensemble of sparse MoEs that takes the best of both classes of models. ...
We study the interplay of two popular classes of such models: ensembles of neural networks and sparse mixture of experts (sparse MoEs). ...
This choice was motivated by the fact that like ViT, V-MoE, and pBE, down-DE requires a only a single upstream checkpoint, which all of the methods more comparable. ...
arXiv:2110.03360v1
fatcat:iaycbzllqjfixp5f3pql33clpa
Ensemble Distillation for Structured Prediction: Calibrated, Accurate, Fast-Choose Three
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
single model during test-time. ...
Modern neural networks do not always produce well-calibrated predictions, even when trained with a proper scoring function such as cross-entropy. ...
In this section, we explore whether these findings can be mirrored by an ensemble which is built from a single optimization trajectory, built from multiple checkpoints. ...
arXiv:2010.06721v2
fatcat:k7hurbf6hzdjphvvgzcuc5ueie
Sequential Bayesian Neural Subnetwork Ensembles
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
We propose sequential ensembling of dynamic Bayesian neural subnetworks that systematically reduce model complexity through sparsity-inducing priors and generate diverse ensembles in a single forward pass ...
Furthermore, we found that our approach produced the most diverse ensembles compared to the approaches with a single forward pass and even compared to the approaches with multiple forward passes in some ...
Our ensembling strategy produces diverse set of base learners from a single end-to-end training process. It consists of an exploration phase followed by M exploitation phases. ...
arXiv:2206.00794v1
fatcat:zke4v34plrgvnfsodhqimqbpti
Assessing the Benefits of Model Ensembles in Neural Re-Ranking for Passage Retrieval
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Starting from relatively standard neural models, we use a previous technique named Fast Geometric Ensembling to generate multiple model instances from particular training schedules, then focusing or attention ...
Our work aimed at experimentally assessing the benefits of model ensembling within the context of neural methods for passage reranking. ...
This research was supported by Fundação para a Ciência e Tecnologia (FCT), through the Ph.D. scholarship with reference SFRH/BD/150497/2019, and the INESC-ID multi-annual funding from the PIDDAC programme ...
arXiv:2101.08705v1
fatcat:jhnyk3eksvawnljdj6p6s4lxfq
DynE: Dynamic Ensemble Decoding for Multi-Document Summarization
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
into a single output sequence. ...
In this work we propose a simple decoding methodology which ensembles the output of multiple instances of the same model on different inputs. ...
A simple ensembling method is presented that allows models trained for single inputs to be used in multi-input settings. 2. ...
arXiv:2006.08748v1
fatcat:dnrx2f4tczbr7dsf4wo3lpztli
Ensembling Graph Predictions for AMR Parsing
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
On the other hand, ensemble methods combine predictions from multiple models to create a new one that is more robust and accurate than individual predictions. ...
In many machine learning tasks, models are trained to predict structure data such as graphs. ...
Ensemble of AMR predictions from a single type of model is studied in [Zhou et al., 2021] where the authors demonstrated that by combining predictions from three different model's checkpoints they gain ...
arXiv:2110.09131v2
fatcat:5jhi7fvasrgrpf226vwf6743mi
Frustratingly Easy Model Ensemble for Abstractive Summarization
2018
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In this paper, we propose an alternative, simple but effective unsupervised ensemble method, post-ensemble, that combines multiple models by selecting a majority-like output in post-processing. ...
Experimental results on a newsheadline-generation task show that the proposed method performs better than the current ensemble methods. ...
The first one, self-ensemble, is a method of extracting models from "checkpoints" saved in each epoch in a training. We prepared the models of self-ensemble by using 10 checkpoints from 4-13 epochs. ...
doi:10.18653/v1/d18-1449
dblp:conf/emnlp/Kobayashi18
fatcat:xioycfexuver3m2quz2n3azgli
Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Using FGE we can train high-performing ensembles in the time required to train a single model. ...
Inspired by this new geometric insight, we also propose a new ensembling method entitled Fast Geometric Ensembling (FGE). ...
to train a single model. ...
arXiv:1802.10026v4
fatcat:ayy7sz37trhd5aw64vvii4pfwi
« Previous
Showing results 1 — 15 out of 4,180 results