11,159 Hits in 5.6 sec

An Efficient Query Optimizer with Materialized Intermediate Views in Distributed and Cloud Environment

Archana Bachhav, Vilas Kharat, Madhukar Shelar
2021 Tehnički glasnik  
In this research, we have come up with an algorithm for query optimization, which aims at storing intermediate results of the queries and use these by-products for execution of future queries.  ...  Extensive experiments have been carried out with the help of simulation model to test the algorithm efficiency  ...  [38] presented history aware optimizer that archives intermediate results of queries so as to reuse them during future query execution.  ... 
doi:10.31803/tg-20210205094356 fatcat:mlo2wvvtrba2rlpmwar5o6khbm

Don't Trash your Intermediate Results, Cache 'em [article]

Prasan Roy, Krithi Ramamritham, S. Seshadri, Pradeep Shenoy, S. Sudarshan
2000 arXiv   pre-print
In this paper we describe a caching system called Exchequer which incorporates several novel features including optimization aware cache maintenance and the use of a cache aware optimizer.  ...  Query response times can be greatly improved by caching final/intermediate results of previous queries, and using them to answer later queries.  ...  A by-product of the multi-query optimizer is an optimal plan for a set of queries given a set of materialized views. This proves useful for cache-cognizant optimization as well.  ... 
arXiv:cs/0003005v1 fatcat:4rgkqvlukbfrfic53nlps4cyhm

Revisiting Reuse in Main Memory Database Systems [article]

Kayhan Dursun, Carsten Binnig, Ugur Cetintemel, Tim Kraska
2016 arXiv   pre-print
As queries arrive, our reuse-aware optimizer reasons about the reuse opportunities for hash tables, employing cost models that take into account hash table statistics together with the CPU and data movement  ...  Experimental results, based on our HashStash prototype demonstrate performance gains of 2× for typical analytical workloads with no additional overhead for materializing intermediates.  ...  optimizers that rewrite queries for materialized views.  ... 
arXiv:1608.05678v1 fatcat:lqibmggfvreyxkxdv7uijnfigy

Revisiting Reuse in Main Memory Database Systems

Kayhan Dursun, Carsten Binnig, Ugur Cetintemel, TIm Kraska
2017 Proceedings of the 2017 ACM International Conference on Management of Data - SIGMOD '17  
As queries arrive, our reuseaware optimizer reasons about the reuse opportunities for hash tables, employing cost models that take into account hash table statistics together with the CPU and data movement  ...  Experimental results, based on our prototype implementation, demonstrate performance gains of 2× for typical analytical workloads with no additional overhead for materializing intermediates.  ...  for materialized views.  ... 
doi:10.1145/3035918.3035957 dblp:conf/sigmod/DursunBCK17 fatcat:o7j5b6un3fethe53csln6yhbby

Enabling Privacy in Provenance-Aware Workflow Systems

Susan B. Davidson, Sanjeev Khanna, Val Tannen, Sudeepa Roy, Yi Chen, Tova Milo, Julia Stoyanovich
2011 Conference on Innovative Data Systems Research  
First, standard, non-privacy preserving workflow management systems use various indexing structures or materialized views to speed up query processing.  ...  Second, to achieve privacy, one needs to generate query results with respect to user access privileges (view).  ... 
dblp:conf/cidr/DavidsonKTRCMS11 fatcat:zaondzsqv5a3plcyh3leshnui4

Continuous cloud-scale query optimization and processing

Nicolas Bruno, Sapna Jain, Jingren Zhou
2013 Proceedings of the VLDB Endowment  
Highlevel scripting languages free developers from understanding various system trade-offs, but introduce new challenges for query optimization.  ...  We continuously monitor query execution, collect actual runtime statistics, and adapt parallel execution plans as the query executes.  ...  Recall that the optimizer treats intermediate results as materialized views, which are represented with traditional read operations.  ... 
doi:10.14778/2536222.2536223 fatcat:cqu63teeoneynjxqxsq7som2sy

Query Optimization for Databases in Cloud Environment: A Survey

Archana Bachhav, Vilas Kharat, Madhukar Shelar
2017 International Journal of Database Theory and Application  
 Catching Intermediate Queries or Results  Materialization  Pipelining Some of these techniques work on reducing communication cost, some focus on reducing execution time of the query and some focus  ...  Either use centralized or distributed platforms with traditional SQL or MapReduce technique query optimization is obligatory to enhance high performance in large scale data systems.  ...  History aware query optimizer [16] is based on archiving intermediate results. It is capable of matching views which are the byproducts of previously executed queries.  ... 
doi:10.14257/ijdta.2017.10.6.01 fatcat:6wdddszmffd7pomsn3bhtfhyim

Recycling in pipelined query evaluation

F. Nagel, P. Boncz, S. D. Viglas
2013 2013 IEEE 29th International Conference on Data Engineering (ICDE)  
The DBA is kept in the loop to make explicit sharing decisions by identifying and/or defining materialized views.  ...  Materializing intermediate results can consume significant system resources, therefore most other database systems avoid this where possible, following a pipelined query architecture instead.  ...  Multi-query optimization and materialized views have similar goals and use structures similar to the recycler graph. In multi-query optimization (MQO) [15] , queries are submitted in batches.  ... 
doi:10.1109/icde.2013.6544837 dblp:conf/icde/NagelBV13 fatcat:fkdoiz35qfcttleiaz4ftaqgcu

Scalable Multi-Query Execution using Reinforcement Learning

Panagiotis Sioulas, Anastasia Ailamaki
2021 Proceedings of the 2021 International Conference on Management of Data  
RouLette scales by replacing sharing-aware optimization with adaptive query processing, and it chooses opportunities to explore and exploit by using reinforcement learning.  ...  strategy scales poorly to large query counts, as it requires expensive sharing-aware optimization, whereas the latter detects only a subset of the opportunities.  ...  Rather than follow an optimize-then-execute approach, RouLette uses runtime adaptation to move sharing-aware optimization out of the critical path, restoring scalability.  ... 
doi:10.1145/3448016.3452799 fatcat:sjt5akmfufc37honzmywh7e5jq

Investigating privacy-aware distributed query evaluation

Nicholas L. Farnan, Adam J. Lee, Ting Yu
2010 Proceedings of the 9th annual ACM workshop on Privacy in the electronic society - WPES '10  
In this paper, we explore the degree to which query intension is leaked to remote data sources when a variety of existing query processing and view materialization techniques are used.  ...  Historically, privacy and efficiency have largely been at odds with one another when querying remote data sources: traditional query optimization techniques provide efficient retrieval by exporting information  ...  Incorporating History.  ... 
doi:10.1145/1866919.1866926 dblp:conf/wpes/FarnanLY10 fatcat:h4y7fzwq4zaf3m5ehqoep7adde

Computation Reuse in Analytics Job Service at Microsoft

Alekh Jindal, Sriram Rao, Shi Qiao, Hiren Patel, Zhicheng Yin, Jieming Di, Malay Bag, Marc Friedman, Yifung Lin, Konstantinos Karanasos
2018 Proceedings of the 2018 International Conference on Management of Data - SIGMOD '18  
of each overlapping computation, and (iii) we create materialized views in an online fashion, without requiring an offline phase to materialize the overlapping computations.  ...  The key aspects of our system are the following: (i) we reuse computations by creating materialized views over recurring workloads, i.e., periodically executing jobs that have the same script templates  ...  Others have looked at history-aware query optimization with materialized intermediate views [38] and at allocating the cache fairly amongst multiple cloud tenants [28] .  ... 
doi:10.1145/3183713.3190656 dblp:conf/sigmod/JindalQPYDBFLKR18 fatcat:46raoj4saba7vdj7kjneo6vf3m

GProM - A Swiss Army Knife for Your Provenance Needs

Bahareh Sadat Arab, Su Feng, Boris Glavic, Seokki Lee, Xing Niu, Qitian Zeng
2018 IEEE Data Engineering Bulletin  
optimization.  ...  The system supports diverse provenance and annotation management tasks through query instrumentation, i.e., compiling a declarative frontend language with provenance-specific features into the query language  ...  Opinions, findings and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.  ... 
dblp:journals/debu/ArabFGLNZ17 fatcat:ndr6mcq2kva6nj6brotrwa6u3y


Jeff LeFevre, Jagan Sankaranarayanan, Hakan Hacigumus, Junichi Tatemura, Neoklis Polyzotis, Michael J. Carey
2014 Proceedings of the 2014 ACM SIGMOD international conference on Management of data - SIGMOD '14  
Our method, called MISO for MultISstore Online tuning, is adaptive, lightweight, and works in an online fashion utilizing only the by-products of query processing, which we term as opportunistic views.  ...  Multistore systems utilize multiple distinct data stores such as Hadoop's HDFS and an RDBMS for query processing by allowing a query to access data and computation in both stores.  ...  ., HV) materialize intermediate results during query execution, and we retain these byproducts as opportunistic materialized views.  ... 
doi:10.1145/2588555.2588568 dblp:conf/sigmod/LeFevreSHTPC14a fatcat:xrnnhpc6tjgwvpaxgs3zcrfjyq

Adaptive query processing

A. Deshpande, J. M. Hellerstein, V. Raman
2006 Proceedings of the 2006 ACM SIGMOD international conference on Management of data - SIGMOD '06  
Our goal with this paper is to be a "value-add" over the existing papers on the material, providing not only a brief overview of each technique, but also a basic framework for understanding the field of  ...  that the traditional optimize-then-execute paradigm is insufficient.  ...  Acknowledgments We would like to express our thanks to our collaborators in the field of adaptive query processing, especially Sirish Chandrasekaran, Mike Franklin, Peter Haas, Alon Halevy, Joe Hellerstein  ... 
doi:10.1145/1142473.1142603 dblp:conf/sigmod/DeshpandeHR06 fatcat:geklxibgbzdjtix2axmtuhuewu

Database architecture evolution

Stefan Manegold, Martin L. Kersten, Peter Boncz
2009 Proceedings of the VLDB Endowment  
However, the operator-at-a-time paradigm with full materialization of all intermediates pursued in MonetDB provides a hook for easier materialized view capturing.  ...  The next improvement came from materialized views, which gives the query optimizer alternatives to reduce access cost.  ... 
doi:10.14778/1687553.1687618 fatcat:az3viqkxx5hf7gmu5zoih67hgq
« Previous Showing results 1 — 15 out of 11,159 results