16,938 Hits in 3.6 sec

Non-parametric entropy estimators based on simple linear regression

Hideitsu Hino, Kensuke Koshijima, Noboru Murata
2015 Computational Statistics & Data Analysis  
Other entropy estimators which directly estimate entropy by linear regression, are also proposed.  ...  Simple linear regression is utilized to estimate the values of density function and its second derivative at a point.  ...  In this paper, we propose novel non-parametric entropy estimators based on the second order expansion of probability mass function and simple linear regression.  ... 
doi:10.1016/j.csda.2015.03.011 fatcat:2cobnxrtsrho3i263eteraq4aq

Entropy Balancing for Continuous Treatments

Stefan Tübbicke
2021 Journal of Econometric Methods  
Empirical Monte-Carlo simulations suggest that treatment effect estimates using EBCT display favorable properties in terms of bias and root mean squared error, especially when balance on higher moments  ...  To facilitate the estimation of causal effects in this setting, the present paper introduces entropy balancing for continuous treatments (EBCT) – an intuitive and user-friendly automated covariate balancing  ...  Lastly, the paper provides credible simulation evidence on the performance of effect estimates based on EBCT and comparison methods using parametric and non-parametric regression techniques.  ... 
doi:10.1515/jem-2021-0002 fatcat:xaurm4j5sndbzp3oahoq4ffuri

An Entropy Estimator Based on Polynomial Regression with Poisson Error Structure [chapter]

Hideitsu Hino, Shotaro Akaho, Noboru Murata
2016 Lecture Notes in Computer Science  
A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point.  ...  Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators.  ...  There are a large number of non-parametric entropy estimation methods.  ... 
doi:10.1007/978-3-319-46672-9_2 fatcat:gsbo6hysp5gftotvbvctkfpr7y

New Developments in Statistical Information Theory Based on Entropy and Divergence Measures

Leandro Pardo
2019 Entropy  
In the last decades the interest in statistical methods based on information measures and particularly in pseudodistances or divergences has grown substantially [...]  ...  This Special Issue focuses on original and new research based on minimum divergence estimators, divergence statistics as well as parametric tests based on pseudodistances or divergences, from a theoretical  ...  In "Robust and Sparse Regression via γ-Divergence", T. Kawashima and H. Fujisawa [23] study robust and sparse regression based on the γ-divergence.  ... 
doi:10.3390/e21040391 pmid:33267105 fatcat:jc37cuc4yjamhhdlddrppmkzuu

Dimensionality reduction based on non-parametric mutual information

Lev Faivishevsky, Jacob Goldberger
2012 Neurocomputing  
Next we provide a nonlinear regression algorithm based on the proposed dimensionality reduction approach.  ...  The algorithm utilizes the recently introduced MeanNN estimator for differential entropy. We show that the estimator is an appropriate tool for the dimensionality reduction task.  ...  Non-parametric estimators for differential entropy Our dimensionality reduction method is based on a smooth non-parametric approximation of differential entropy which is reviewed below.  ... 
doi:10.1016/j.neucom.2011.07.028 fatcat:cuv43q63kjfthhk33vfv3bgb2e

Page 7015 of Mathematical Reviews Vol. , Issue 99j [page]

1999 Mathematical Reviews  
); Mukaidono, Masao (J-MEIJ2-C; Kawasaki) Gaussian clustering method based on maximum-fuzzy-entropy interpretation.  ...  This together with the possibility of considering weighted LS estimators with arbitrary weights [A. Pazman, Non- linear statistical models, Kluwer Acad.  ... 

Growth and convergence: A profile of distribution dynamics and mobility

Esfandiar Maasoumi, Jeff Racine, Thanasis Stengos
2007 Journal of Econometrics  
We propose new concepts and measures of "convergence," or "divergence" that are based on entropy distances and dominance relations between groups of countries over time.  ...  We update the sample period to include the most recent decade of data available, and we offer traditional parametric and new nonparametric estimates of the most widely used growth regressions for two important  ...  inferences on more robust nonparametric estimators. 2 The Traditional Parametric Setting.  ... 
doi:10.1016/j.jeconom.2005.11.012 fatcat:qehoj3u2vvcvpblhkjt4azou2m

Comparing Entropy and Beta as Measures of Risk in Asset Pricing

Galina Deeva
2017 Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis  
Three different non-parametric estimation procedures are considered to evaluate financial entropy, namely kernel density estimated Shannon entropy, kernel density estimated Rényi entropy and maximum likelihood  ...  The comparison is provided based on the European stock market data, for which the basic risk-return trade-off is generally negative.  ...  II summarizes the results of linear regressions of risk premium on risk measure in three data frequencies: daily, weekly and monthly.  ... 
doi:10.11118/actaun201765061889 fatcat:zkxdzgchm5gy3ctwvdmypk2uee

Mutual information based dimensionality reduction with application to non-linear regression

Lev Faivishevsky, Jacob Goldberger
2010 2010 IEEE International Workshop on Machine Learning for Signal Processing  
Next we provide a nonlinear regression algorithm based on the proposed dimensionality reduction approach.  ...  In this paper we introduce a supervised linear dimensionality reduction algorithm which is based on finding a projected input space that maximizes mutual information between input and output values.  ...  We demonstrated a simple nonlinear regression algorithm MIPR that is based on MIDR.  ... 
doi:10.1109/mlsp.2010.5589176 fatcat:7qhylegog5g2nndimvlg7evafu

Partially adaptive estimation via the maximum entropy densities

Ximing Wu, Thanasis Stengos
2005 Econometrics Journal  
We propose a partially adaptive estimator based on information theoretic maximum entropy estimates of the error distribution.  ...  The maximum entropy (maxent) densities have simple yet flexible functional forms to nest most of the mathematical distributions.  ...  Bickel (1982) extended this to linear regression and other models for i.i.d. errors. Manski (1984) studied adaptive estimation in non-linear models.  ... 
doi:10.1111/j.1368-423x.2005.00168.x fatcat:uxvrjd5sevbd3dfqh4mk647td4

Modified Entropy Estimator Using a Haar Matrix to Fit the Weibull Growth Model

Basim Shlaibah Msallam, Saifaldin Hashim Kamar
2020 Journal of Southwest Jiaotong University  
Therefore, in this paper, a new method for estimating the parameters of the four-parameter Weibull growth model was proposed using the Generalized Maximum Entropy function by fitting data based on the  ...  It is well known that the Generalized Maximum Entropy method can be used to fit linear regression models, especially as they are not restricted by the conditions to be verified as are other classical methods  ...  The obtained non-parametric function estimator included modified wavelength coefficients based on the original wavelength coefficients.  ... 
doi:10.35741/issn.0258-2724.55.1.37 fatcat:jrdu6evdcjfsrazl7fy43zshfy

Variable Selection with Copula Entropy [article]

Jian Ma
2020 arXiv   pre-print
In this paper, a Copula Entropy (CE) based method for variable selection which use CE based ranks to select variables is proposed. The method is both model-free and tuning-free.  ...  models and Adaptive LASSO, were conducted on the UCI heart disease data.  ...  Under the blessing of Theorem 1, Ma and Sun [21] proposed a simple and elegant non-parametric method for estimating CE (MI) from data which comprises of only two steps * : 1.  ... 
arXiv:1910.12389v2 fatcat:hm7kbxyhbnbwxo5yitcovsaldm

Page 7830 of Mathematical Reviews Vol. , Issue 99k [page]

1999 Mathematical Reviews  
Tessitore, Non-parametric regression and non- linear data analysis: an overview (531-538); Yadolah Dodge and Valentin Rousson, Multivariate L; mean (539-546); Jean-Francois Durand, Nicolas Molinari and  ...  Preisser, Re- sistant fits for regression with correlated outcomes: an estimating equations approach (415-431); Pranab Kumar Sen, Robust non- parametrics in mixed-MANOCOVA models (433-451); Haibo Zhou  ... 

Lower bounds on minimax rates for nonparametric regression with additive sparsity and smoothness

Garvesh Raskutti, Martin J. Wainwright, Bin Yu
2009 Neural Information Processing Systems  
the minimax rate for estimating the regression function with respect to squared-L 2 (P) error.  ...  We study minimax rates for estimating high-dimensional nonparametric regression models with sparse additive structure and smoothness constraints.  ...  It would also be interesting to develop a more complete understanding of whether computationally efficient algorithms [7, 12, 9] based on regularization achieve the lower bounds on the minimax rate derived  ... 
dblp:conf/nips/RaskuttiWY09 fatcat:exjsszuepfh7rc3ubydk3nyzau

A mutual information approach to calculating nonlinearity

Reginald Smith
2015 Stat  
This technique, which gives an exact value for the proportion of linear dependence, is then compared with another common test for linearity, the Brock, Dechert and Scheinkman (BDS) test.  ...  A new method to measure nonlinear dependence between two variables is described using mutual information to analyze the separate linear and nonlinear components of dependence.  ...  An overview of the effect of bin sizes on entropy estimation is given in [25] .  ... 
doi:10.1002/sta4.96 fatcat:anvb573g2bby3duxgsvy2cgufm
« Previous Showing results 1 — 15 out of 16,938 results