Filters








13,290 Hits in 3.9 sec

Relative Error Streaming Quantiles [article]

Graham Cormode and Zohar Karnin and Edo Liberty and Justin Thaler and Pavel Veselý
2020 arXiv   pre-print
Approximating ranks, quantiles, and distributions over streaming data is a central task in data analysis and monitoring.  ...  This paper presents a randomized sketch of size O(log^1.5(ε n)/ε) that achieves a 1±ε multiplicative error guarantee, without prior knowledge of the stream length or dependence on the size of the data  ...  Other related works that do not fully solve the relative error quantiles problem are as follows.  ... 
arXiv:2004.01668v2 fatcat:iqdva7dzczhd7bpev2iqwtov3u

Two Maximum Entropy-Based Algorithms for Running Quantile Estimation in Nonstationary Data Streams

Ognjen Arandjelovic, Duc-Son Pham, Svetha Venkatesh
2015 IEEE transactions on circuits and systems for video technology (Print)  
In this paper we specifically address the problem of estimating the running quantile of a data stream when the memory for storing observations is limited.  ...  The need to estimate a particular quantile of a distribution is an important problem which frequently arises in many computer vision and signal processing applications.  ...  relative error Ground truth quantile estimate (e) Uniform histogram [7] Fig. 12. Running estimate of the 0.95-quantile on data stream 3.  ... 
doi:10.1109/tcsvt.2014.2376137 fatcat:dvfxlgknvbe6hj4ytewsfzzf2y

Two maximum entropy based algorithms for running quantile estimation in non-stationary data streams [article]

Ognjen Arandjelovic, Duc-Son Pham, Svetha Venkatesh
2014 arXiv   pre-print
In this paper we specifically address the problem of estimating the running quantile of a data stream when the memory for storing observations is limited.  ...  The need to estimate a particular quantile of a distribution is an important problem which frequently arises in many computer vision and signal processing applications.  ...  relative error Ground truth quantile estimate (e) Data stream 2, 0.99-quantile, 500 binsFig. 14. and S.  ... 
arXiv:1411.2250v1 fatcat:d4wcbwhsgjbwjkhp4vfcp77pzm

Frugal Streaming for Estimating Quantiles [chapter]

Qiang Ma, S. Muthukrishnan, Mark Sandler
2013 Lecture Notes in Computer Science  
Modern applications require processing streams of data for estimating statistical quantities such as quantiles with small amount of memory.  ...  For stochastic streams where data items are drawn from a distribution independently, we analyze and show that the algorithm finds an approximation to the quantile rapidly and remains stably close to it  ...  (b) relative mass error for (a). (c) 90-% quantile estimation. (d) relative mass error for (c). Fig. 3 . 3 Evaluation on one stream generated from three Cauchy distributions. (a) Median estimation.  ... 
doi:10.1007/978-3-642-40273-9_7 fatcat:zucsmbeigrcptops2xykmazzxa

Evaluation of Summarization Schemes for Learning in Streams [chapter]

Alec Pawling, Nitesh V. Chawla, Amitabh Chaudhary
2006 Lecture Notes in Computer Science  
We present a time-and-memory-efficient discretization technique based on computing ε-approximate exponential frequency quantiles, and prove bounds on the worst-case error introduced in computing information  ...  , under a variety of streaming scenarios for real and artificial datasets.  ...  We consider two cases: the first will result in both relative and absolute error terms, and the second in just a relative error term. In Case 1, assume p > 1/2.  ... 
doi:10.1007/11871637_34 fatcat:sgoibqvuobfcbaur3emgnw3u5q

The adaptable buffer algorithm for high quantile estimation in non-stationary data streams [article]

Ognjen Arandjelovic, Duc-Son Pham, Svetha Venkatesh
2015 arXiv   pre-print
In this paper we specifically address the problem of estimating the running quantile of a data stream with non-stationary stochasticity when the memory for storing observations is limited.  ...  We make several major contributions: (i) we derive an important theoretical result which shows that the change in the quantile of a stream is constrained regardless of the stochastic properties of data  ...  Stream 1 Stream 2 Stream 3 Method Bins Relative Absolute Relative Absolute Relative Absolute L 1 error L ∞ error L 1 error L ∞ error L 1 error L ∞ error Targeted adaptable 500 2.1% 1.00e11  ... 
arXiv:1504.05302v1 fatcat:z2w3jl445fbknjajmjjfdmil6i

Approximate Quantiles for Datacenter Telemetry Monitoring [article]

Gangmuk Lim, Mohamed Hassan, Ze Jin, Stavros Volos, Myeongjae Jeon
2019 arXiv   pre-print
AOMG estimates quantiles with high throughput and less than 5% relative value error across a wide range of use cases while state-of-the-art algorithms either have a high relative value error (9.3-137.0%  ...  Low value error for tail quantiles is achieved by retaining a few tail values per subwindow.  ...  average relative value error for different quantiles all falls below 5%.  ... 
arXiv:1906.00228v3 fatcat:kprpfzs7l5hcxj453f47leyc5u

UDDSketch: Accurate Tracking of Quantiles in Data Streams [article]

Italo Epicoco, Catiuscia Melle, Massimo Cafaro, Marco Pulimeno, Giuseppe Morleo
2020 arXiv   pre-print
We present UDDSketch (Uniform DDSketch), a novel sketch for fast and accurate tracking of quantiles in data streams.  ...  On the contrary, UDDSketch is designed so that accuracy guarantees can be given over the full range of quantiles and for arbitrary distribution in input.  ...  Figure 4 :Figure 5 :Figure 6 : 456 Relative errors on quantiles (boxplots), varying the number of buckets. Dataset: pareto, α : Relative errors on quantiles (boxplots), varying the number of buckets.  ... 
arXiv:2004.08604v1 fatcat:mr2qxbvzvnf3zkwinj4kkhawiy

UDDSketch: Accurate Tracking of Quantiles in Data Streams

Italo Epicoco, Catiuscia Melle, Massimo Cafaro, Marco Pulimeno, Giuseppe Morleo
2020 IEEE Access  
We present UDDSketch (Uniform DDSketch), a novel sketch for fast and accurate tracking of quantiles in data streams.  ...  The Gigascope streaming database [3] relies on quantiles for monitoring network applications and systems.  ...  .: UDDSketch: Accurate Tracking of Quantiles in Data Streams FIGURE 6 . 6 Relative errors on quantiles (boxplots), varying the number of buckets.  ... 
doi:10.1109/access.2020.3015599 fatcat:d77dz7u3vfbnlaq7t7nk5p6gvm

Stream quantiles via maximal entropy histograms [article]

Ognjen Arandjelovic, Ducson Pham, Svetha Venkatesh
2014 arXiv   pre-print
We address the problem of estimating the running quantile of a data stream when the memory for storing observations is limited.  ...  We (i) highlight the limitations of approaches previously described in the literature which make them unsuitable for non-stationary streams, (ii) describe a novel principle for the utilization of the available  ...  This table shows the variation in the mean relative error as well as the largest absolute error of the quantile estimate for the proposed data-aligned histogram-based algorithm as the number of available  ... 
arXiv:1409.7289v1 fatcat:6gczx5spwbeddkyg7cpgd67lyu

Targeted Adaptable Sample for Accurate and Efficient Quantile Estimation in Non-Stationary Data Streams

Ognjen Arandjelović
2019 Machine Learning and Knowledge Extraction  
We make several major contributions: (i) we derive an important theoretical result that shows that the change in the quantile of a stream is constrained regardless of the stochastic properties of data;  ...  In this paper, we specifically address the problem of estimating the running quantile of a data stream with non-stationary stochasticity when the absolute (rather than asymptotic) memory for storing observations  ...  Stream 1 Stream 2 Stream 3 Method Bins Relative Absolute Relative Absolute Relative Absolute L 1 Error L ∞ Error L 1 Error L ∞ Error L 1 Error L ∞ Error Targeted adaptable 500 2.1%  ... 
doi:10.3390/make1030049 fatcat:ngngo5fr6vd6bpjukzw6zzwea4

Frugal Streaming for Estimating Quantiles:One (or two) memory suffices [article]

Qiang Ma, S. Muthukrishnan, Mark Sandler
2014 arXiv   pre-print
Modern applications require processing streams of data for estimating statistical quantities such as quantiles with small amount of memory.  ...  For stochastic streams where data items are drawn from a distribution independently, we analyze and show that the algorithm finds an approximation to the quantile rapidly and remains stably close to it  ...  Instead of evaluating the absolute error of quantile estimation, we evaluate how far the estimate is from the true quantiles, the relative mass error.  ... 
arXiv:1407.1121v1 fatcat:lchb6d3oczblzav7nv2jztaqkm

A permutation test for quantile regression

Brian S. Cade, Jon D. Richards
2006 Journal of Agricultural Biological and Environmental Statistics  
A drop in dispersion, F -ratio like, permutation test (D) for linear quantile regression estimates (0 ≤ τ ≤ 1) had relative power ≥1 compared to quantile rank score tests (T ) for hypotheses on parameters  ...  extreme quantiles where one or the other maintained better error rates.  ...  PERFORMANCE RELATIVE TO RANK SCORE TESTS We compared the ability of the drop in dispersion D test to maintain valid Type I error rates relative to the conventional rank score T test (Cade et al. 2006)  ... 
doi:10.1198/108571106x96835 fatcat:i2fm6htptze5hjux3n4nud46om

Space-efficient estimation of empirical tail dependence coefficients for bivariate data streams [article]

Alastair Gregory, Kaushik Jana
2019 arXiv   pre-print
The approximation, which has stream-length invariant error bounds, utilises recent work on the development of a summary for bivariate empirical copula functions.  ...  Modifications to the space-efficient bivariate copula approximation, presented in this paper, allow the error of approximations to the tail dependence coefficients to remain stream-length invariant.  ...  An approximation to the biased quantiles should have error relative to the quantile query, such that the approximation to should have an error of ± (or if required, ± (1 − ) by symmetry) rather than the  ... 
arXiv:1902.03586v3 fatcat:7hzeijr6grexlk6sx6js6rigcy

Efficient approximation of correlated sums on data streams

R. Ananthakrishna, A. Das, J. Gehrke, F. Korn, S. Muthukrishnan, D. Srivastava
2003 IEEE Transactions on Knowledge and Data Engineering  
Finally, we prove that, when AGG(x) is a quantile (which cannot be computed over a data stream in limited space), the error of a CS-aggregate can be arbitrarily large.  ...  quantiles.  ...  Recent work on computing approximate quantiles has employed an analogous error metric where φ-quantiles are determined to within a rank precision of n, rather than a "relative" rank precision of φn (see  ... 
doi:10.1109/tkde.2003.1198391 fatcat:zsbwekhtfrdrbelowcrkrt2qpy
« Previous Showing results 1 — 15 out of 13,290 results