A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is `application/pdf`

.

## Filters

##
###
Upper Bound on Normalized Maximum Likelihood Codes for Gaussian Mixture Models
[article]

2018
*
arXiv
*
pre-print

This paper shows that the

arXiv:1709.00925v2
fatcat:u6fdh6v5rbdzdigthflcnqrlka
*normalized**maximum**likelihood*~(NML)*code*-length calculated in [1] is*an**upper**bound**on*the NML*code*-length strictly calculated*for*the*Gaussian**Mixture**Model*. ... When we use this*upper**bound**on*the NML*code*-length, we must change the scale of the data sequence to satisfy the restricted domain. ... Here, we use the*Gaussian**Model*Class N (µ, Σ), µ ∈ R m , Σ ∈ R m×m , and calculate the*normalized**maximum**likelihood*(NML)*code*-length*for*the*Gaussian**Model*. ...##
###
Rate Distortion Behavior of Sparse Sources

2012
*
IEEE Transactions on Information Theory
*

The former lead to low-and high-rate

doi:10.1109/tit.2012.2201335
fatcat:i7x3mjprujgdvhtb6o4bx6a4q4
*upper**bounds**on*mean squared error , while the latter yields lower and*upper**bounds**on*source entropy, thereby characterizing asymptotic behavior. ... These*bounding*techniques are applied to two source*models*:*Gaussian**mixtures*and power laws matching the approximately scale-invariant decay of wavelet coefficients. ... Telatar*for*helpful discussions, as well as the anonymous reviewers and the associate editor*for*their valuable suggestions and comments, which greatly helped improving the presentation of this material ...##
###
Technical report: Training Mixture Density Networks with full covariance matrices
[article]

2020
*
arXiv
*
pre-print

In the standard formulation,

arXiv:2003.05739v1
fatcat:wdylwz7flrbefbavxbigearvz4
*an*MDN takes some input and outputs parameters*for*a*Gaussian**mixture**model*with restrictions*on*the*mixture*components' covariance. ...*Mixture*Density Networks are a tried and tested tool*for**modelling*conditional probability distributions. As such, they constitute a great baseline*for*novel approaches to this problem. ... Then the latter network can be trained via back-propagation through the GMM block, using the negative log-*likelihood*loss function which is also supplied. ...##
###
Bit error rate estimation for turbo decoding

2003
*
IEEE International Symposium on Information Theory, 2003. Proceedings.
*

We

doi:10.1109/isit.2003.1228454
fatcat:26uqgm5r5zam3jbus6rorlwua4
*model*the log-*likelihood*ratios as a*mixture*of two*Gaussian*random variables and derive estimators*for*the mean and variance of these distributions, which can be used to estimate BER. ... We propose a method*for**on*-line estimation of Bit Error Rate during turbo decoding. ... We have derived estimators*for*the parameters of the LLR distribution by*modelling*LLRs as a*Gaussian**mixture*. ...##
###
Bit error rate estimation for turbo decoding

2009
*
IEEE Transactions on Communications
*

We

doi:10.1109/tcomm.2009.03.060410
fatcat:q5ye7ehgpfcd3oo74v6kvo7yn4
*model*the log-*likelihood*ratios as a*mixture*of two*Gaussian*random variables and derive estimators*for*the mean and variance of these distributions, which can be used to estimate BER. ... We propose a method*for**on*-line estimation of Bit Error Rate during turbo decoding. ... We have derived estimators*for*the parameters of the LLR distribution by*modelling*LLRs as a*Gaussian**mixture*. ...##
###
The minimum description length principle in coding and modeling

1998
*
IEEE Transactions on Information Theory
*

The

doi:10.1109/18.720554
fatcat:s7ylg53uvzhufabbucrzq4m26q
*normalized*maximized*likelihood*,*mixture*, and predictive*codings*are each shown to achieve the stochastic complexity to within asymptotically vanishing terms. ... Context tree*modeling*, density estimation, and*model*selection in*Gaussian*linear regression serve as examples. ... the desired asymptotic equivalence of the Jeffreys*mixture*and*normalized**maximum**likelihood*. ...##
###
The Minimum Description Length Principle in Coding and Modeling
[chapter]

2009
*
Information Theory
*

The

doi:10.1109/9780470544907.ch25
fatcat:b5bnutcdg5cyvetrcohul7omvm
*normalized*maximized*likelihood*,*mixture*, and predictive*codings*are each shown to achieve the stochastic complexity to within asymptotically vanishing terms. ... Context tree*modeling*, density estimation, and*model*selection in*Gaussian*linear regression serve as examples. ... the desired asymptotic equivalence of the Jeffreys*mixture*and*normalized**maximum**likelihood*. ...##
###
Complexity of simple nonlogarithmic loss functions

2003
*
IEEE Transactions on Information Theory
*

Index Terms--loss functions, complexity,

doi:10.1109/tit.2002.807281
fatcat:uv5p7pk2njarzbywzp2t4bhlha
*maximum*entropy, min-max*bounds*, prediction*bound*. ... The loss complexity*for*nonlogarithmic loss functions is defined analogously to the stochastic complexity*for*logarithmic loss functions such that its mean provides*an*achievable lower*bound**for*estimation ... If the integral is finite, we can define the*normalized**maximum*-*likelihood*(NML)*model*, [2] , [10] (9) where (11) We derive next a few important properties of the*models*in the class*for*a simple ...##
###
A Tutorial on VAEs: From Bayes' Rule to Lossless Compression
[article]

2020
*
arXiv
*
pre-print

The Variational Auto-Encoder (VAE) is a simple, efficient, and popular deep

arXiv:2006.10273v2
fatcat:qevuyc6vrrhg7lcvlkkerowixi
*maximum**likelihood**model*. Though usage of VAEs is widespread, the derivation of the VAE is not as widely understood. ... Finally, we will visualize the capabilities and limitations of VAEs using a*code*example (with*an*accompanying Jupyter notebook)*on*toy 2D data. ... VLAE [7] shows that the negative ELBO −L(x) is*an**upper**bound**on*this number by constructing a*code*to describe p gt (x) that*on*average uses −L(x) bits. ...##
###
PARAFAC-Based Blind Identification of Underdetermined Mixtures Using Gaussian Mixture Model

2013
*
Circuits, systems, and signal processing
*

The GMM-PARAFAC algorithm uses

doi:10.1007/s00034-013-9719-8
fatcat:k443mbfxafaofd6ckyohbt52nq
*Gaussian**mixture**model*(GMM) to*model*non-*Gaussianity*of the independent sources. ... In order to reduce the computation complexity, the mixing matrix is estimated by maximizing a tight*upper**bound*of the*likelihood*instead of the log-*likelihood*itself. ... Acknowledgements The authors would like to thank Lieven De Lathauwer*for*sharing with us the Matlab*codes*of the FOOBI algorithm. ...##
###
Density-Preserving Hierarchical EM Algorithm: Simplifying Gaussian Mixture Models for Approximate Inference

2018
*
IEEE Transactions on Pattern Analysis and Machine Intelligence
*

We propose

doi:10.1109/tpami.2018.2845371
pmid:29994194
fatcat:5qm3jc4j7fesrg7j4wjixvtqne
*an*algorithm*for*simplifying a finite*mixture**model*into a reduced*mixture**model*with fewer*mixture*components. ...*For*recursive Bayesian filtering, we propose*an*efficient algorithm*for*approximating*an*arbitrary*likelihood*function as a sum of scaled*Gaussian*. ... and*codes*in [66] ; K Zhang, L Zhang and MH Yang*for*the*codes*in [64] ; MA Brubaker, A Geiger and R Urtasun*for*the dataset and*codes**for*selflocalization in [21] . ...##
###
Learning deep kernels for exponential family densities
[article]

2021
*
arXiv
*
pre-print

Compared to deep density

arXiv:1811.08357v4
fatcat:f66ynommebdwdc7zradtnewfiu
*models*fit via*maximum**likelihood*, our approach provides a complementary set of strengths and tradeoffs: in empirical studies, the former can yield higher*likelihoods*, whereas ... This gives a very rich class of density*models*, capable of fitting complex structures*on*moderate-dimensional problems. ...*For*each*model*fit*on*each distribution, we report the*normalized*log*likelihood*(left) and Fisher divergence (right). ...##
###
Independent Component Analysis by Entropy Bound Minimization

2010
*
IEEE Transactions on Signal Processing
*

We show that such

doi:10.1109/tsp.2010.2055859
fatcat:4rg3si2k7jb6dm3djtfblwmaoe
*an*estimator exists*for*a wide class of measuring functions, and provide a number of design examples to demonstrate its flexible nature. ... A novel (differential) entropy estimator is introduced where the*maximum*entropy*bound*is used to approximate the entropy given the observations, and is computed using a numerical procedure thus resulting ... to the*upper**bound*of , i.e.,*for*, we have By solving the above inequality, we obtain*On*the other hand, there is*an*such that*for*, the*upper**bound*of is no greater than the lower*bound*of , i.e. ...##
###
Iterated logarithmic expansions of the pathwise code lengths for exponential families

2000
*
IEEE Transactions on Information Theory
*

*For*exponential families we obtain pathwise expansions, to the constant order, of the predictive and

*mixture*

*code*lengths used in MDL. The results are useful

*for*understanding di erent MDL forms. ... Rissanen's Minimum Description Length (MDL) principle is a statistical

*modeling*principle motivated by

*coding*theory. ... This modi ed two-stage

*code*is called

*Normalized*

*Maximum*

*Likelihood*(NML)

*code*. ...

##
###
Likelihood Inference in the Errors-in-Variables Model

1996
*
Journal of Multivariate Analysis
*

We consider estimation and con dence regions

doi:10.1006/jmva.1996.0055
fatcat:vbm72woeqzg7nlgrdwsgxc56l4
*for*the parameters and based*on*the observations X 1 ; Y 1 ; : : : ; X n ; Y n in the errors-in-variables*model*X i = Z i + e i and Y i = + Z i +f i*for**normal*... Similarly we study the con dence regions obtained from the*likelihood*ratio statistic*for*the*mixture**model*and show that these are asymptotically consistent both in the*mixture*case and in the case that ... Theorem 1.1 and its proof shows that the latter is true*for*the*maximum**likelihood*estimator*for*the*mixture**model*. ...
« Previous

*Showing results 1 — 15 out of 11,815 results*