Filters








167,348 Hits in 3.5 sec

Estimating Entropy of Distributions in Constant Space [article]

Jayadev Acharya, Sourbh Bhadane, Piotr Indyk, Ziteng Sun
2019 arXiv   pre-print
We consider the task of estimating the entropy of k-ary distributions from samples in the streaming model, where space is limited.  ...  Our main contribution is an algorithm that requires O(k log (1/ε)^2/ε^3) samples and a constant O(1) memory words of space and outputs a ±ε estimate of H(p).  ...  We initiate the study of distribution entropy estimation with space limitations, with the goal of understanding the space-sample trade-offs.  ... 
arXiv:1911.07976v1 fatcat:yom3mxqtsbgo7dwudqdwxqohha

Computing Accurate Probabilistic Estimates of One-D Entropy from Equiprobable Random Samples [article]

Hoshin V Gupta, Mohammed Reza Ehsani, Tirthankar Roy, Maria A Sans-Fuentes, Uwe Ehret, Ali Behrangi
2021 arXiv   pre-print
We develop a simple Quantile Spacing (QS) method for accurate probabilistic estimation of one-dimensional entropy from equiprobable random samples, and compare it with the popular Bin-Counting (BC) method  ...  Bootstrapping is used to approximate the sampling variability distribution of the resulting entropy estimate, and is shown to accurately reflect the true uncertainty.  ...  The QS and BC algorithms used in this work are freely accessible for non-commercial use at https://github.com/rehsani/Entropy accessed on 20 February 2021.  ... 
arXiv:2102.12675v1 fatcat:kztvvxdqorhctolpzzt3kfl4ke

A Stochastic Non-Homogeneous Constant Elasticity of Substitution Production Function as an Inverse Problem: A Non-Extensive Entropy Estimation Approach

S. Bwanakare
2013 Acta Physica Polonica. A  
To apply the approach, we select a stochastic non-homogeneous constant elasticity of substitution aggregated production function of the 27 EU countries which we estimate maximizing a non-extensive entropy  ...  The document proposes a new entropy-based approach for estimating the parameters of nonlinear and complex models, i.e. those whose no transformation renders linear in parameters.  ...  (2) τ e constant elasticity of substitution, ε t stands for the random disturbance with unknown distribution.  ... 
doi:10.12693/aphyspola.123.502 fatcat:p2sndzicgbhs3ojgb325zn5a2y

Computing Accurate Probabilistic Estimates of One-D Entropy from Equiprobable Random Samples

Hoshin V. Gupta, Mohammad Reza Ehsani, Tirthankar Roy, Maria A. Sans-Fuentes, Uwe Ehret, Ali Behrangi
2021 Entropy  
We develop a simple Quantile Spacing (QS) method for accurate probabilistic estimation of one-dimensional entropy from equiprobable random samples, and compare it with the popular Bin-Counting (BC) and  ...  Bootstrapping is used to approximate the sampling variability distribution of the resulting entropy estimate, and is shown to accurately reflect the true uncertainty.  ...  The QS and BC algorithms used in this work are freely accessible for non-commercial use at https://github.com/rehsani/Entropy accessed on 20 February 2021.  ... 
doi:10.3390/e23060740 pmid:34208344 pmcid:PMC8231182 fatcat:tyaulcs5nzfadfymdj54c4sphu

Non-Extensive Entropy Econometrics: New Statistical Features of Constant Elasticity of Substitution-Related Models

Second Bwanakare
2014 Entropy  
We estimated three stochastically distinct models of constant elasticity of substitution (CES) class functions as non-linear inverse problem and showed that these PL related functions should have a closed  ...  The approach leads to robust estimation and to new findings about the true stochastic nature of this class of nonlinear-up until now-analytically intractable functions.  ...  Acknowledgments The author gratefully acknowledges University of Information Technology and Management in Rzeszow (Poland) for having financed this research.  ... 
doi:10.3390/e16052713 fatcat:hy5hvpuy5rctpo5smnzz2lzaci

Optimal Partitions for Nonparametric Multivariate Entropy Estimation [article]

Z. Keskin
2021 arXiv   pre-print
Such optimal partitions were observed to be more accurate than existing techniques in estimating entropies of correlated bivariate Gaussian distributions with known theoretical values, across varying sample  ...  This paper demonstrates that the application of rotation operations can improve entropy estimates by aligning the geometry of the partition to the sample distribution.  ...  ., x N }, the uncertainty of the distribution can be estimated from the partitioned sample space, by calculating the entropy of the probability density estimator using a histogram approach.  ... 
arXiv:2112.06299v1 fatcat:35dyo3snuvhwlakyk5ljx3v4sa

Streaming and sublinear approximation of entropy and information distances

Sudipto Guha, Andrew McGregor, Suresh Venkatasubramanian
2006 Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm - SODA '06  
In a data stream setting (sublinear space), we give the first algorithm for estimating the entropy of a distribution.  ...  Our algorithm runs in polylogarithmic space and yields an asymptotic constant factor approximation scheme.  ...  We consider the problem of estimating the entropy H of a distribution, providing optimal (up to constants) upper bounds for testing entropy.  ... 
doi:10.1145/1109557.1109637 fatcat:rxplmcvhijg5fluzcbcthblcdy

Projective Power Entropy and Maximum Tsallis Entropy Distributions

Shinto Eguchi, Osamu Komori, Shogo Kato
2011 Entropy  
A close relation of the entropy with the Lebesgue space L p and the dual L q is explored, in which the escort distribution associates with an interesting property.  ...  We discuss the statistical estimation by minimization of the empirical loss associated with the projective power entropy.  ...  Model of Maximum Entropy Distributions We will elucidate a dualistic structure between the maximum entropy model on H γ , defined in (2) and the minimum cross entropy estimator on C γ , defined in (1)  ... 
doi:10.3390/e13101746 fatcat:e24fizjbm5b4ffb3dcazafziwe

Streaming and Sublinear Approximation of Entropy and Information Distances [article]

Sudipto Guha, Andrew McGregor, Suresh Venkatasubramanian
2005 arXiv   pre-print
In a stream setting (sublinear space), we give the first algorithm for estimating the entropy of a distribution.  ...  Efficient estimation of these distances is a key component in algorithms for manipulating distributions.  ...  We consider the problem of estimating the entropy H of a distribution, providing optimal (upto constants) upper bounds for testing entropy.  ... 
arXiv:cs/0508122v2 fatcat:6t3pruhej5h5deybzpl5xqf6nq

A frequency-domain entropy-based detector for robust spectrum sensing in cognitive radio networks

Ya Zhang, Qin Zhang, Tommaso Melodia
2010 IEEE Communications Letters  
The entropy of the sensed signal is estimated in the frequency domain with a probability space partitioned into fixed dimensions.  ...  To counteract noise uncertainty, a new entropy-based spectrum sensing scheme is introduced in this letter.  ...  The entropy of the measured signal is estimated in the frequency domain with the probability space partitioned into fixed dimensions.  ... 
doi:10.1109/lcomm.2010.06.091954 fatcat:zu6vhpobnve6xfkyl74luqnhee

Association Rule Mining using Maximum Entropy [article]

Rasmus Pagh, Morten Stöckel
2015 arXiv   pre-print
We consider the use of maximum entropy probability estimates, which give a principled way of extrapolating probabilities of events that do not even occur in the data set!  ...  entropy estimates, and 2) Maximum entropy estimates based on a small number of samples are provably tightly concentrated around the true maximum entropy frequency that arises if we let the number of samples  ...  We note that a similar proof of how to find the maxent estimate appears in [8] , however we give explicit constants in Equation (14) whereas their proof shows existence of the constants.  ... 
arXiv:1501.02143v1 fatcat:zfbnapgaffdwhahhmd2jgwqnlq

Estimation of entropy measures for categorical variables with spatial correlation [article]

Linda Altieri, Daniela Cocchi, Giulia Roli
2019 arXiv   pre-print
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space.  ...  We propose a path for spatial entropy estimation: instead of correcting the global entropy estimator, we focus on improving the estimation of its components, i.e. the probabilities, in order to account  ...  In the case of categorical variables following, e.g., a multinomial distribution, the crucial point is to estimate the distribution parameters.  ... 
arXiv:1911.03685v1 fatcat:wvlbsnpmlndz7bcst7qp7ueoe4

Geodesic Entropic Graphs for Dimension and Entropy Estimation in Manifold Learning

J.A. Costa, A.O. Hero
2004 IEEE Transactions on Signal Processing  
In this paper, we consider the closely related problem of estimating the manifold's intrinsic dimension and the intrinsic entropy of the sample points.  ...  In the manifold learning problem, one seeks to discover a smooth low dimensional surface, i.e., a manifold embedded in a higher dimensional linear vector space, based on a set of measured sample points  ...  Almal for their help in acquiring and processing these face images.  ... 
doi:10.1109/tsp.2004.831130 fatcat:lon4gmz5wzfyva7o2rv2eizjsa

Detecting temperature fluctuations at equilibrium [article]

Purushottam D. Dixit
2015 arXiv   pre-print
Numerical evidence using an analytically tractable model shows that the effects of temperature fluctuations can be detected in equilibrium and dynamical properties of the phase space of the small system  ...  We interpret this ambiguity as resulting from a stochastically fluctuating temperature coupled with the phase space variables giving rise to a broad temperature distribution.  ...  Acknowledgment: We thank Ken Dill, Steve Pressé, Dilip Asthagiri for a critical reading of the manuscript.  ... 
arXiv:1503.01808v1 fatcat:2oqq7so6v5afloe5fiavcp7nim

5 Estimating the CGE Model through the Maximum Entropy Principle [chapter]

2019 Non-Extensive Entropy Econometrics for Low Frequency Series  
In their reference work, Arndt et al. (2002) present a new approach to estimating parameters of a CGE model through maximum entropy. That approach is pursued here.  ...  Whatever class of CGE model is in use, it displays consistent drawbacks, such as the lack of efficient methodology for the estimates of behavioural parameters (e.g., trade parameters), less realistic economic  ...  Application: Non-Extensive Entropy and Constant Elasticity of Substitution-Based Models The example below shows the possibility of carrying out robust estimation of a stochastic constant elasticity of  ... 
doi:10.1515/9783110605914-018 fatcat:y7gv3atblvdd7pmf4bwurtjhzu
« Previous Showing results 1 — 15 out of 167,348 results