30,409 Hits in 4.2 sec

Unsupervised learning with stochastic gradient

Harold Szu, Ivica Kopriva
2005 Neurocomputing  
A stochastic gradient is formulated based on deterministic gradient augmented with Cauchy simulated annealing capable to reach a global minimum with a convergence speed significantly faster when simulated  ...  A stochastic gradient was successfully applied to solve inverse space-variant imaging problems on a concurrent pixel-bypixel basis with the unknown mixing matrix (imaging point spread function) varying  ...  That motivated our efforts to formulate stochastic gradient descent learning as a combination of the gradient descent and stochastic Cauchy annealing.  ... 
doi:10.1016/j.neucom.2004.11.010 fatcat:gkmkkmfkujhrjlzze5ygyugpsm

Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning [article]

Dong Liu, Chengjian Sun, Chenyang Yang, Lajos Hanzo
2020 arXiv   pre-print
When the mathematical model of the environment is incomplete, we introduce reinforced-unsupervised learning algorithms that learn the model by interacting with the environment.  ...  In this article, we introduce unsupervised and reinforced-unsupervised learning frameworks for solving both variable and functional optimization problems without the supervision of the optimal solutions  ...  We note that for the stochastic policy, there is no substantial difference between the model-free and the modelbased unsupervised learning, since the gradients of the OF and CFs are not required for the  ... 
arXiv:2001.00784v1 fatcat:c7f4kqg5vrb6leofxk6ixaeorm

Why Does Unsupervised Pre-training Help Deep Learning?

Dumitru Erhan, Yoshua Bengio, Aaron C. Courville, Pierre-Antoine Manzagol, Pascal Vincent, Samy Bengio
2010 Journal of machine learning research  
The best results obtained on supervised learning tasks involve an unsupervised learning component, usually in an unsupervised pre-training phase.  ...  Much recent research has been devoted to learning algorithms for deep architectures such as Deep Belief Networks and stacks of auto-encoder variants, with impressive results obtained in several areas,  ...  This is a natural effect of stochastic gradient descent with a constant learning rate (which gives exponentially more weight to recent examples).  ... 
dblp:journals/jmlr/ErhanBCMVB10 fatcat:x5qjobqpsfhy3ndz452vzwwz5y

Application of Stochastic Gradient Descent Algorithm in Evaluating the Performance Contribution of Employees

R. M. V. S. Ratnayake, K. G. U Perera, G. S. K. Wickramanayaka, C. S. Gunasekara, K. C. J. K. Samarawickrama
2014 IOSR Journal of Business and Management  
This paper presents the applicability of stochastic gradient descent algorithm as an approach for evaluation of performance contribution.  ...  Under that, performance evaluation is established in corporate, department and team levels with the intention of creating an employee incentive scheme.  ...  Training data of an unsupervised learning problem does not consist such labelled inputs corresponded with the targets.  ... 
doi:10.9790/487x-16637780 fatcat:nkritehgdzchzpykcvcspld2by

Unsupervised Sequence Classification using Sequential Output Statistics [article]

Yu Liu, Jianshu Chen, Li Deng
2017 arXiv   pre-print
., language models) could be obtained independently of input data and thus with low or no cost. To address the problem, we propose an unsupervised learning cost function and study its properties.  ...  Although it is harder to optimize in its functional form, a stochastic primal-dual gradient method is developed to effectively solve the problem.  ...  Specifically, we minimize L with respect to the primal variable θ by stochastic gradient descent and maximize L with respect to the dual variable V by stochastic gradient ascent.  ... 
arXiv:1702.07817v2 fatcat:zwrwooo5dbhmlontkwvvqsqova

Ladder Networks: Learning under Massive Label Deficit

Behroz Mirza, Tahir Syed, Jamshed Memon, Yameen Malik
2017 International Journal of Advanced Computer Science and Applications  
Advancement in deep unsupervised learning are finally bringing machine learning close to natural learning, which happens with as few as one labeled instance.  ...  This work discusses how the ladder network model successfully combines supervised and unsupervised learning taking it beyond the pre-training realm.  ...  Ladder networks combine supervised learning with unsupervised learning in deep neural networks.  ... 
doi:10.14569/ijacsa.2017.080769 fatcat:ix4dlifjc5hwjka6hohdrswz2m

Doc2hash: Learning Discrete Latent variables for Documents Retrieval

Yifei Zhang, Hao Zhu
2019 North American Chapter of the Association for Computational Linguistics  
However, the discrete stochastic layer is usually incompatible with the backpropagation in the training stage and thus causes a gradient flow problem.  ...  In this paper, we propose a method, Doc2hash, that solves the gradient flow problem of the discrete stochastic layer by using continuous relaxation on priors, and trains the generative model in an end-to-end  ...  Therefore, it produces low-variance biased gradients of the stochastic layer in the backpropagation.  ... 
doi:10.18653/v1/n19-1232 dblp:conf/naacl/ZhangZ19 fatcat:2a6b2orh4zghpd7gesjjt76hle

Comparing Unsupervised Word Translation Methods Step by Step

Mareike Hartmann, Yova Kementchedjhieva, Anders Søgaard
2019 Neural Information Processing Systems  
, and transfer learning.  ...  In the unsupervised regime, an initial seed dictionary is learned in the absence of any known correspondences between words, through distribution matching, and the seed dictionary is then used to supervise  ...  GANs with stochastic dictionary induction provides a new state of the art for unsupervised word translation.  ... 
dblp:conf/nips/HartmannKS19 fatcat:jbnapc2la5fptb5oa5ffplsvuy

Hierarchical Deep Learning Architecture For 10K Objects Classification [article]

Atul Laxman Katole, Krishna Prasad Yellapragada, Amish Kumar Bedi, Sehaj Singh Kalra, Mynepalli Siva Chaitanya
2015 arXiv   pre-print
Also we propose a blend of leaf level models trained with either supervised or unsupervised learning approaches.  ...  These architectures extract & learn the real world hierarchical visual features utilizing supervised & unsupervised learning approaches respectively.  ...  supervised gradient descent.Utilizing unsupervised learning we have trained a leaf model Leaf-4 that consists of man-made artifacts with 235 categories.  ... 
arXiv:1509.01951v1 fatcat:zm6636eulfgcpgn3qgai36tpke

Geometry-aware Domain Adaptation for Unsupervised Alignment of Word Embeddings [article]

Pratik Jawanpuria, Mayank Meghwanshi, Bamdev Mishra
2020 arXiv   pre-print
We propose a novel manifold based geometric approach for learning unsupervised alignment of word embeddings between the source and the target languages.  ...  The rich geometry of the doubly stochastic manifold allows to employ efficient Riemannian conjugate gradient algorithm for the proposed formulation.  ...  Learning unsupervised cross-lingual mapping may be viewed as an instance of the more general unsupervised domain adaptation problem (Ben-David et al., 2007; Gopalan et al., 2011; Sun et al., 2016; Mahadevan  ... 
arXiv:2004.08243v2 fatcat:icp7v4fvwvgwlmpobhxxov5zsq

Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991) [article]

Juergen Schmidhuber
2020 arXiv   pre-print
One network learns to generate a probability distribution over outputs, the other learns to predict effects of the outputs. Each network minimizes the objective function maximized by the other.  ...  I review unsupervised or self-supervised neural networks playing minimax games in game-theoretic settings: (i) Artificial Curiosity (AC, 1990) is based on two such networks.  ...  The 1990 paper [61] describes gradient-based learning methods for both C and M.  ... 
arXiv:1906.04493v3 fatcat:rigrb7az65hjvpvwt6kcbldh3y

Deep learning and face recognition: the state of the art

Stephen Balaban
2019 arXiv   pre-print
Deep Neural Networks (DNNs) have established themselves as a dominant technique in machine learning.  ...  In this talk and accompanying paper, I attempt to provide a review and summary of the deep learning techniques used in the state-of-the-art.  ...  Online stochastic gradient descent: B = 1. 2. Mini-batch stochastic gradient descent: B > 1 but B < |x|. 3. "Batch" gradient descent, B = |x|.  ... 
arXiv:1902.03524v1 fatcat:6z2n44siwbht3a22c4kgc6mghu

The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training

Dumitru Erhan, Pierre-Antoine Manzagol, Yoshua Bengio, Samy Bengio, Pascal Vincent
2009 Journal of machine learning research  
The experiments confirm and clarify the advantage of unsupervised pre-training.  ...  We empirically show the influence of pre-training with respect to architecture depth, model capacity, and number of training examples.  ...  Stochastic gradient descent θ ⇐ θ − • ∂KL(x||x)/∂θ is then performed with learning rate , for a fixed number of pre-training iterations.  ... 
dblp:journals/jmlr/ErhanMBBV09 fatcat:zbjsmisy7nhgtl5g62uoovy6ju

Bayesian Optimization for Machine Learning : A Practical Guidebook [article]

Ian Dewancker, Michael McCourt, Scott Clark
2016 arXiv   pre-print
machine learning applications.  ...  We outline four example machine learning problems that can be solved using open source machine learning libraries, and highlight the benefits of using Bayesian optimization in the context of these common  ...  Unsupervised Learning Unsupervised learning algorithms are designed with the hope of capturing some useful latent structure in data.  ... 
arXiv:1612.04858v1 fatcat:cncz6ky4zrakjitnh7o5ed6fju

Heavy-Tailed Symmetric Stochastic Neighbor Embedding

Zhirong Yang, Irwin King, Zenglin Xu, Erkki Oja
2009 Neural Information Processing Systems  
Moreover, it uses a gradient descent algorithm that may require users to tune parameters such as the learning step size, momentum, etc., in finding its optimum.  ...  With this generalization, we are presented with two difficulties.  ...  Compared with an earlier method Stochastic Neighbor Embedding (SNE) [6] , SSNE uses a symmetrized cost function with simpler gradients.  ... 
dblp:conf/nips/YangKXO09 fatcat:iotx2rzsgbegtean6gpc4b4jmu
« Previous Showing results 1 — 15 out of 30,409 results