Load Shedding for Window Joins on Multiple Data Streams

Yan-Nei Law, Carlo Zaniolo
2007 2007 IEEE 23rd International Conference on Data Engineering Workshop  
We consider the problem of semantic load shedding for continuous queries containing window joins on multiple data streams and propose a robust approach that is effective with the different semantic accuracy criteria that are required in different applications. In fact, our approach can be used to (i) maximize the number of output tuples produced by joins, and (ii) optimize the accuracy of complex aggregates estimates under uniform random sampling. We first consider the problem of computing
more » ... al subsets of approximate window joins over multiple data streams. Previously proposed approaches are based on multiple pairwise joins and, in their load-shedding decisions, disregard the content of streams outside the joined pairs. To overcome these limitations, we optimize our load-shedding policy using various predictors of the productivity of each tuple in the window. To minimize processing costs, we use a fastand-light sketching technique to estimate the productivity of the tuples. We then show that our method can be generalized to produce statistically accurate samples, as needed in, e.g., the computation of averages, quantiles, and stream mining queries. Tests performed on both synthetic and reallife data demonstrate that our method outperforms previous approaches, while requiring comparable amounts of time and space.
doi:10.1109/icdew.2007.4401054 dblp:conf/icde/LawZ07 fatcat:bgwn2csw55em3p7sp7xqho4whu