Fast Multidimensional Entropy Estimation by $k$-d Partitioning

Dan Stowell, Mark D. Plumbley
2009 IEEE Signal Processing Letters  
We describe a non-parametric estimator for the differential entropy of a multidimensional distribution, given a limited set of data points, by a recursive rectilinear partitioning. The estimator uses an adaptive partitioning method and runs in Θ N log N time, with low memory requirements. In experiments using known distributions, the estimator is several orders of magnitude faster than other estimators, with only modest increase in bias and variance.
doi:10.1109/lsp.2009.2017346 fatcat:ubylrtowtzgkhahsgdn4d5bdvy