78,964 Hits in 3.7 sec

Sequence modeling with mixtures of conditional maximum entropy distributions

D. Pavlov
Third IEEE International Conference on Data Mining  
We present a novel approach to modeling sequences using mixtures of conditional maximum entropy distributions.  ...  We demonstrate how our mixture of conditional maximum entropy models can be learned from data using the EM algorithm that scales linearly in the dimensions of the data and the number of mixture components  ...  Mixture of Conditional Maxent Models As we pointed out in the introduction, the primary motivation for considering the mixture of maximum entropy models comes from the desire to model unobserved, hidden  ... 
doi:10.1109/icdm.2003.1250927 dblp:conf/icdm/Pavlov03 fatcat:wmwa7t7d5rbu3gziodqz3oucca

Maximum Causal Tsallis Entropy Imitation Learning [article]

Kyungjae Lee and Sungjoon Choi and Songhwai Oh
2018 arXiv   pre-print
Third, we propose a maximum causal Tsallis entropy imitation learning (MCTEIL) algorithm with a sparse mixture density network (sparse MDN) by modeling mixture weights using a sparsemax distribution.  ...  In particular, we show that the causal Tsallis entropy of an MDN encourages exploration and efficient mixture utilization while Boltzmann Gibbs entropy is less effective.  ...  Optimality Condition of Maximum Causal Tsallis Entropy We show that the optimal policy of the problem (6) is a sparsemax distribution using the KKT conditions.  ... 
arXiv:1805.08336v2 fatcat:gyqyph2subbinj5wjagvaoq4f4

Learning Mixture Models With the Regularized Latent Maximum Entropy Principle

S. Wang, D. Schuurmans, F. Peng, Y. Zhao
2004 IEEE Transactions on Neural Networks  
This paper presents a new approach to estimating mixture models based on a recent inference principle we have proposed: the latent maximum entropy principle (LME).  ...  We show that a regularized version of LME (RLME), is effective at estimating mixture models.  ...  . ; 1, or any Poisson distribution as a default conditional model.  ... 
doi:10.1109/tnn.2004.828755 pmid:15461082 fatcat:br5lrv7x4fb4pkpotsbskr7zea

Influence of the Thermodynamic Properties of a Sprayed Liquid on the Droplet-Air Mixture Parameters in the Framework of the Maximum Entropy Model

R.V. Kolodnytska, P.P. Moskvin, S.I. Skurativskyi, Ye.S. Syroid
2017 Ukrainian Journal of Physics  
The influence of the thermodynamic properties of a sprayed liquid on the size and velocity distribution functions for droplets in the droplet-air mixture has been analyzed, by using the maximum entropy  ...  K e y w o r d s: maximum entropy method, droplet-air and disperse systems, droplet distribution functions, biofuel.  ...  Equilibrium models describing the formation of a droplet-air mixture include, first of all, the models that are based on the maximum entropy principle [4] [5] [6] .  ... 
doi:10.15407/ujpe62.03.0230 fatcat:dzo5smssjjedvgw42sv3lofycu

A joint maximum-entropy model for binary neural population patterns and continuous signals

Sebastian Gerwinn, Philipp Berens, Matthias Bethge
2009 Neural Information Processing Systems  
Second-order maximum-entropy models have recently gained much interest for describing the statistics of binary spike trains.  ...  Therefore, extending the framework of maximum-entropy models to continuous variables allows us to gain novel insights into the relationship between the firing patterns of neural ensembles and the stimuli  ...  This work is supported by the German Ministry of Education, Science, Research and Technology through the Bernstein award to MB (BMBF; FKZ: 01GQ0601), the Werner-Reichardt Centre for Integrative Neuroscience  ... 
dblp:conf/nips/GerwinnBB09 fatcat:fcqzbgvc65dpxj4e5sloosle3m

Semi-supervised Learning by Entropy Minimization

Yves Grandvalet, Yoshua Bengio
2004 Neural Information Processing Systems  
The method challenges mixture models when the data are sampled from the distribution class spanned by the generative model.  ...  The performances are definitely in favor of minimum entropy regularization when generative models are misspecified, and the weighting of unlabeled data provides robustness to the violation of the "cluster  ...  Test errors of minimum entropy logistic regression (•) and mixture models (+).  ... 
dblp:conf/nips/GrandvaletB04 fatcat:bsycp7up6vafndkfbgrjyrvv7y

A Maximum Entropy Test for Evaluating Higher-Order Correlations in Spike Counts

Arno Onken, Valentin Dragoi, Klaus Obermayer, Emery N. Brown
2012 PLoS Computational Biology  
Using mutual information as a divergence measure we find that there are spike count bin sizes at which the maximum entropy hypothesis can be rejected for a substantial number of neuronal pairs.  ...  We construct a family of reference distributions: maximum entropy distributions, which are constrained only by marginals and by linear correlations as quantified by the Pearson correlation coefficient.  ...  For this purpose, we apply a maximum entropy model subject to a set of constraints.  ... 
doi:10.1371/journal.pcbi.1002539 pmid:22685392 pmcid:PMC3369943 fatcat:os4gjw7nqrgl3ale36dhfltwpy

Relaxation models of phase transition flows

Philippe Helluy, Nicolas Seguin
2006 Mathematical Modelling and Numerical Analysis  
These equations of state are particularly suitable for a use in a relaxation finite volume scheme. The approach is based on a constrained convex optimization problem on the mixture entropy.  ...  It is valid for both miscible and immiscible mixtures. We also propose a rough pressure law for modelling a super-critical fluid. Mathematics Subject Classification. 76M12, 65M12.  ...  The concavity of s and the positivity of T are important conditions for a proper modelling. But the positivity of the pressure is absolutely not necessary.  ... 
doi:10.1051/m2an:2006015 fatcat:3pbwxktuzjhpbawbu4hoipvstm


Alexander I. Balunov
The method is based on an extended version of the maximum entropy principle.  ...  The informational entropy of complex experiment involving conditional entropy and conditional probabilities is used as the likelihood criterion.  ...  As it has been mentioned before, the maximum likelihood criterion, along with Shannon's entropy, also includes conditional entropy and conditional probabilities while accounting the athermal mixture properties  ... 
doi:10.6060/ivkkt.20206301.6072 fatcat:ypungylbmraejbvetnsdop22wu

Audio classification based on maximum entropy model

Zhe Feng, Yaqian Zhou, Lide Wu, Zongge Li
2003 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698)  
In this paper, we present a novel audio classification method based on Maximum Entropy Model.  ...  It is one of the key components in audio and video applications. In prior work, the accuracy under complicated condition is not satisfactory enough and the results highly depend on the dataset.  ...  MAXIMUM ENTROPY MODEL Basic Idea The basic idea of maximum entropy model is to model anything that is known and to assume nothing that is unknown.  ... 
doi:10.1109/icme.2003.1221025 dblp:conf/icmcs/FengZWL03 fatcat:flturkgwkngb5jg7lfja3crkpi

MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning

Diego Granziol, Binxin Ru, Stefan Zohren, Xiaowen Dong, Michael Osborne, Stephen Roberts
2019 Entropy  
In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations.  ...  Efficient approximation lies at the heart of large-scale machine learning problems.  ...  Similarly, we learn the maximum entropy spectral density for the Gaussian mixture and then approximate the entropy of the Gaussian mixture via the entropy of the maximum entropy spectral density, which  ... 
doi:10.3390/e21060551 pmid:33267265 fatcat:4lximq4lhveqtguxpeb7xplpvy

Collaborative Filtering with Maximum Entropy

D. Pavlov, E. Manavoglu, D.M. Pennock, C. Lee Giles
2004 IEEE Intelligent Systems  
We describe a novel maximum entropy (maxent) approach for generating online recommendations as a user navigates through a collection of documents.  ...  We show that our maxent algorithm is arguably one of the most accurate recommenders, as compared to such techniques as correlation, mixture of Markov models, mixture of multinomial models, individual similarity-based  ...  Our recent work [7] suggests that for difficult prediction problems improvement beyond the plain maximum entropy models can be sought by employing the mixtures of maximum entropy models.  ... 
doi:10.1109/mis.2004.59 fatcat:k5u2sfbbfbe3velfrylh75tzfm

Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models

Gideon Mann, Ryan T. McDonald, Mehryar Mohri, Nathan Silberman, Dan Walker
2009 Neural Information Processing Systems  
Training conditional maximum entropy models on massive data sets requires significant computational resources.  ...  We analyze and compare the CPU and network time complexity of each of these methods and present a theoretical analysis of conditional maxent models, including a study of the convergence of the mixture  ...  Acknowledgments We thank Yishay Mansour for his comments on an earlier version of this paper.  ... 
dblp:conf/nips/MannMMSW09 fatcat:np7lab25qfezfehwer45whmgha

An Entropy Maximization Approach to Optimal Model Selection in Gaussian Mixtures [chapter]

Antonio Peñalver, Juan M. Sáez, Francisco Escolano
2003 Lecture Notes in Computer Science  
We propose a criterion based on the entropy of the pdf (probability density function) associated to each kernel to measure the quality of a given mixture model, and a modification of the classical EM algorithm  ...  In this paper we address the problem of estimating the parameters of a Gaussian mixture model.  ...  Conclusions and Future Work In this paper we have presented a method for finding the optimal number of kernels in a Gaussian mixture based on maximum entropy.  ... 
doi:10.1007/978-3-540-24586-5_53 fatcat:umvmvlkjtzdzvn5v7up75wotca

Page 1239 of Mathematical Reviews Vol. , Issue 2003B [page]

2003 Mathematical Reviews  
mixture models.  ...  One result of studying the local geometry is that it unifies the convex and differential geometric theories of mixture models.  ... 
« Previous Showing results 1 — 15 out of 78,964 results