The Internet Archive has a preservation copy of this work in our general collections.
The file type is application/pdf.
Filters
Metric-Free Natural Gradient for Joint-Training of Boltzmann Machines
[article]
2013
arXiv
pre-print
Preserved Fulltext
Other Versions
This paper introduces the Metric-Free Natural Gradient (MFNG) algorithm for training Boltzmann Machines. ...
the natural gradient metric L. ...
our Metric-Free Natural Gradient (MFNG) algorithm is suitable for joint-training of complex Boltzmann Machines. ...
arXiv:1301.3545v2
fatcat:3g5h55fg7bbjldmckgzff56jwu
Natural gradient, fitness modelling and model selection: A unifying perspective
2013
2013 IEEE Congress on Evolutionary Computation
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
This equivalence allows to simultaneously perform model selection and robust estimation of the natural gradient. ...
In this paper Stochastic Relaxation is used to provide theoretical results on Estimation of Distribution Algorithms (EDAs). ...
This leads to two different algorithms, Stochastic Gradient Descent (SGD), based on the regular gradient, and Stochastic Natural Gradient Descent (SNGD), based on the natural gradient, reported in Algorithm ...
doi:10.1109/cec.2013.6557608
dblp:conf/cec/MalagoMP13
fatcat:e3smb2pcojhmvi7ootvlszxg2y
Training Restricted Boltzmann Machines
2015
Künstliche Intelligenz
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2018; you can also visit the original URL.
The file type is application/pdf.
Therefore, training algorithms such as Contrastive Divergence (CD) and learning based on Parallel Tempering (PT) rely on Markov chain Monte Carlo methods to approximate the gradient. ...
It is based on likelihood maximization, but the likelihood and its gradient are computationally intractable. ...
An Analysis of Centered RBMs An undesired property of training RBMs based on the loglikelihood gradient is that the learning procedure is not invariant to the data representation. ...
doi:10.1007/s13218-015-0371-2
fatcat:pi56jeamzbartjca2aj6zybwlu
An Overview of Restricted Boltzmann Machines
2019
Journal of the Indian Institute of Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The aim of this article is to give a tutorial introduction to the restricted Boltzmann machines and to review the evolution of this model. ...
A Boltzmann machine (BM) is a model of pairwise interacting units where each unit updates its state over time in a probabilistic way depending on the states of the neighboring units. ...
The H-F method is used to design natural gradient descent for learning Boltzmann machines 11 . ...
doi:10.1007/s41745-019-0102-z
fatcat:fmg33s4tzvdonjqopkt7td2gny
Unsupervised Learning of Distributions of Binary Vectors Using 2-Layer Networks
1991
Neural Information Processing Systems
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2016; you can also visit the original URL.
The file type is application/pdf.
The first learning algorithm is the standard gradient ascent heuristic for computing maximum likelihood estimates for the parameters (i.e. weights and thresholds) of the modeL Here we give a closed form ...
We give experimental results for these learning methods on synthetic data and natural data from the domain of handwritten digits. ...
The process of learning an unknown distribution from examples is usually called denszty estimation or parameter estimation in statistics, depending on the nature of the class of distributions used as models ...
dblp:conf/nips/FreundH91
fatcat:ozaqjdzylfaidbow4hvbgrfzxu
Restricted Boltzmann Machines for galaxy morphology classification with a quantum annealer
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
During this exploration, we analyzed the steps required for Boltzmann sampling with the D-Wave 2000Q, including a study of temperature estimation, and examined the impact of qubit noise by comparing and ...
The methods we compare include Quantum Annealing (QA), Markov Chain Monte Carlo (MCMC) Gibbs Sampling, and Simulated Annealing (SA) as well as machine learning algorithms like gradient boosted decision ...
However, outside of these rather special training scenarios, RBMs (regardless of the classical or quantum nature of the training algorithm) did not outperform the gradient boosted tree algorithm. ...
arXiv:1911.06259v2
fatcat:jehwnvv4xbgullsvys73gqo6jq
A Two-Stage Pretraining Algorithm for Deep Boltzmann Machines
[chapter]
2013
Lecture Notes in Computer Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
In this paper, we propose a novel pretraining algorithm that consists of two stages; obtaining approximate posterior distributions over hidden units from a simpler model and maximizing the variational ...
It has been shown empirically that it is difficult to train a DBM with approximate maximumlikelihood learning using the stochastic gradient unlike its simpler special case, restricted Boltzmann machine ...
The proposed scheme is based on an observation that training DBMs consists of two separate stages; approximating a posterior distribution over hidden units and updating parameters to maximize the lower-bound ...
doi:10.1007/978-3-642-40728-4_14
fatcat:63wyviu3z5fdxce25zzb5givxe
Benchmarking Quantum Hardware for Training of Fully Visible Boltzmann Machines
[article]
2016
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Here we report on the ability of recent QA hardware to accelerate training of fully visible Boltzmann machines. ...
We characterize the sampling distribution of QA hardware, and show that in many cases, the quantum distributions differ significantly from classical Boltzmann distributions. ...
Training and test sets had size 5×10 5 points each, generated by an exact Boltzmann sampler. On these problems,P k (s s s|θ θ θ learn ) provides better fits than B(s s s|θ θ θ learn ). ...
arXiv:1611.04528v1
fatcat:6rt5auq4b5capafe5qjc5hsvai
A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
The goal of this survey article is to present background about the Gumbel-max trick, and to provide a structured overview of its extensions to ease algorithm selection. ...
Over the past years, the machine learning community has proposed several extensions of this trick to facilitate, e.g., drawing multiple samples, sampling from structured domains, or gradient estimation ...
Huijben gratefully acknowledges support from Onera Health and the project 'OP-SLEEP'. The project 'OP-SLEEP' is made possible by the European Regional Development Fund, in the context of OPZuid. W. ...
arXiv:2110.01515v2
fatcat:ixtrm2va3vfthmwq76wmw6k5bm
Quantum Inspired Training for Boltzmann Machines
[article]
2015
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
We present an efficient classical algorithm for training deep Boltzmann machines (DBMs) that uses rejection sampling in concert with variational approximations to estimate the gradients of the training ...
Finally our algorithm can train full Boltzmann machines and scales more favorably with the number of layers in a DBM than greedy contrastive divergence training. ...
The algorithm utilizes subroutines Q and Q that provide an estimate of the probability of a given configuration of hidden and visible units. ...
arXiv:1507.02642v1
fatcat:wl3lgye55jdyplefpngpewiymu
Efficient Learning of Restricted Boltzmann Machines Using Covariance Estimates
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
One of the terms in the gradient, which involves expectation w.r.t. the model distribution, is intractable and is obtained through an MCMC estimate. ...
Learning RBMs using standard algorithms such as CD(k) involves gradient descent on the negative log-likelihood. ...
., 2013) H-F algorithm is used to design natural gradient descent for learning Boltzmann machines. ...
arXiv:1810.10777v2
fatcat:4vgfgjqh7rhcdc23m7ooy2afem
Parallel tempering is efficient for learning restricted Boltzmann machines
2010
The 2010 International Joint Conference on Neural Networks (IJCNN)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
We propose to use an advanced Monte Carlo method called parallel tempering instead, and show experimentally that it works efficiently. The authors are with the ...
While contrastive divergence learning has been considered an efficient way to learn an RBM, it has a drawback due to a biased approximation in the learning gradient. ...
Acknowledgements This work was supported by the honours programme of the department, by the Academy of Finland and by the IST Program of the European Community, under the PASCAL2 Network of Excellence. ...
doi:10.1109/ijcnn.2010.5596837
dblp:conf/ijcnn/ChoRI10
fatcat:yuimhaah4jemzncxnlk5rgqawm
How to Pretrain Deep Boltzmann Machines in Two Stages
[chapter]
2015
Springer Series in Bio-/Neuroinformatics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
In this paper, we propose a novel pretraining algorithm that consists of two stages; obtaining approximate posterior distributions over hidden units from a simpler model and maximizing the variational ...
It has been shown empirically that it is difficult to train a DBM with approximate maximum-likelihood learning using the stochastic gradient unlike its simpler special case, restricted Boltzmann machine ...
Acknowledgements This work was supported by "the Academy of Finland (Finnish Centre of Excellence in Computational Inference Research COIN, 251170)". ...
doi:10.1007/978-3-319-09903-3_10
fatcat:rnh2f3meevhdroawotiigdnvfm
Modeling spectral envelopes using deep conditional restricted Boltzmann machines for statistical parametric speech synthesis
2016
2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
Experimental results show that our proposed method can produce more natural speech sounds than the hidden Markov model (HMM)-based, DNN-based, and DMDNbased synthesis methods. ...
At training stage, the DNN part and the CRBM part of the DCRBM are pre-trained successively and then a unified fine-tuning of all model parameters is conducted. ...
After random initialization, the model parameters of both systems were estimated by backpropagation with a mini-batch-based stochastic gradient descent (SGD) algorithm. 3 When building the system using ...
doi:10.1109/icassp.2016.7472654
dblp:conf/icassp/YinLHD16
fatcat:qxl6ptwssvdkrolg47tiim7kfm
A hybrid strategy for Gilbert' s channel characterization using gradient and annealing techniques
1998
International Journal of Systems Science
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit the original URL.
The file type is application/pdf.
This paper presents a hybrid algorithm for estimating Gilbert's channel model parameters from an experimental error-gap distribution. ...
A stochastic simulated annealing algorithm is applied to determine automatically a set ofgood starting points, which are then used by the deterministic gradient algorithm for faster convergence to the ...
Acknowledgments This work was supported by the National Science Council, Taiwan, under grant NSC85-2221-E009-029. (A 6) ...
doi:10.1080/00207729808929549
fatcat:iwsvlevsyvdqbow3pxar6wuisu
« Previous
Showing results 1 — 15 out of 26,081 results