Filters








10 Hits in 1.1 sec

NeuroCard: One Cardinality Estimator for All Tables [article]

Zongheng Yang, Amog Kamsetty, Sifei Luan, Eric Liang, Yan Duan, Xi Chen, Ion Stoica
2020 arXiv   pre-print
Query optimizers rely on accurate cardinality estimates to produce good execution plans.  ...  Despite decades of research, existing cardinality estimators are inaccurate for complex queries, due to making lossy modeling assumptions and not capturing inter-table correlations.  ...  ACKNOWLEDGMENTS We thank Joe Hellerstein for fruitful discussions and guidance, and Michael Whittaker, Richard Liaw, and Chenggang Wu for their insightful comments on this paper.  ... 
arXiv:2006.08109v2 fatcat:hdjtxxeduzhwjdjxa4uamgtlqi

Cardinality Estimation in DBMS: A Comprehensive Benchmark Evaluation [article]

Yuxing Han, Ziniu Wu, Peizhi Wu, Rong Zhu, Jingyi Yang, Liang Wei Tan, Kai Zeng, Gao Cong, Yanzhao Qin, Andreas Pfadler, Zhengping Qian, Jingren Zhou (+2 others)
2021 arXiv   pre-print
Cardinality estimation (CardEst) plays a significant role in generating high-quality query plans for a query optimizer in DBMS.  ...  We have made all of the benchmark data and evaluation code publicly available at https://github.com/Nathaniel-Han/End-to-End-CardEst-Benchmark.  ...  of) full outer join of all tables has poor scalability and low accuracy for numerous tables in a DB.  ... 
arXiv:2109.05877v3 fatcat:ufqtoe6f7nbmxcfotdlbqga7ja

Uncertainty-aware Cardinality Estimation by Neural Network Gaussian Process [article]

Kangfei Zhao, Jeffrey Xu Yu, Zongyan He, Hao Zhang
2021 arXiv   pre-print
In this paper, we study cardinality estimation for SQL queries with a focus on uncertainty, which we believe is important in database systems when dealing with a large number of user queries on various  ...  The approach we explore is different from the direction of deploying sophisticated DL models in database systems to build cardinality estimators.  ...  Acknowledgement We thank Zongheng Yang, the author of NeuroCard [66] for his help in testing the estimator.  ... 
arXiv:2107.08706v1 fatcat:t35s7o5x6ve5hdw2mcxxd32jki

FLAT: Fast, Lightweight and Accurate Method for Cardinality Estimation [article]

Rong Zhu, Ziniu Wu, Yuxing Han, Kai Zeng, Andreas Pfadler, Zhengping Qian, Jingren Zhou, Bin Cui
2021 arXiv   pre-print
It can estimate cardinality for both single table queries and multi table join queries.  ...  Query optimizers rely on accurate cardinality estimation (CardEst) to produce good execution plans.  ...  When a query touches tables in multiple models, all local probabilities are corrected and merged together to estimate the final cardinality.  ... 
arXiv:2011.09022v5 fatcat:z5sdcuqgafgulnwgnx6u44sciu

A Unified Deep Model of Learning from both Data and Queries for Cardinality Estimation [article]

Peizhi Wu, Gao Cong
2021 arXiv   pre-print
Cardinality estimation is a fundamental problem in database systems.  ...  To capture the rich joint data distributions of a relational table, most of the existing work either uses data as unsupervised information or uses query workload as supervised information.  ...  We would like to thank Zizhong Meng (NTU) for helping with some of the experiments, and the anonymous reviewers for providing constructive feedback and valuable suggestions.  ... 
arXiv:2107.12295v1 fatcat:hmvedppunnbihfc5m72estcweu

BayesCard: Revitilizing Bayesian Frameworks for Cardinality Estimation [article]

Ziniu Wu, Amir Shaikhha, Rong Zhu, Kai Zeng, Yuxing Han, Jingren Zhou
2021 arXiv   pre-print
Cardinality estimation (CardEst) is an essential component in query optimizers and a fundamental problem in DBMS.  ...  Our experimental results on several single-table and multi-table benchmarks indicate BayesCard's superiority over existing state-of-the-art CardEst methods: BayesCard achieves comparable or better accuracy  ...  In our testing, we randomly sample 1% of all tuples for CardEst. 3). Naru/NeuroCard [52, 53] are DAR-based CardEst methods for single table and multi-table join queries, respectively. 4).  ... 
arXiv:2012.14743v2 fatcat:b2wf3gs5x5antmrces4jatpafq

AutoML: From Methodology to Application

Yaliang Li, Zhen Wang, Yuexiang Xie, Bolin Ding, Kai Zeng, Ce Zhang
2021 Proceedings of the 30th ACM International Conference on Information & Knowledge Management  
For each topic, we will motivate it with application examples from industry, illustrate the state-of-the-art methodologies, and discuss some future research directions based on our experience from industry  ...  Machine Learning methods have been adopted for a wide range of real-world applications, ranging from social networks, online image/video-sharing platforms, and e-commerce to education, healthcare, etc.  ...  On the other hand, data-driven cardinality estimation methods directly model the joint data distribution of all tables in the database.  ... 
doi:10.1145/3459637.3483279 fatcat:cu5pqsaulrah7kzvvz6oihaaci

Are We Ready For Learned Cardinality Estimation? [article]

Xiaoying Wang, Changbo Qu, Weiyuan Wu, Jiannan Wang, Qingqing Zhou
2021 arXiv   pre-print
Cardinality estimation is a fundamental but long unresolved problem in query optimization.  ...  Recently, multiple papers from different research groups consistently report that learned models have the potential to replace existing cardinality estimators.  ...  Unlike JOB, we focus on single table cardinality estimation.  ... 
arXiv:2012.06743v3 fatcat:5elzqkv4djfpjdw3naoetug4xa

Dependency Structures in Differentially Coded Cardiovascular Time Series

Tatjana Tasic, Sladjana Jovanovic, Omer Mohamoud, Tamara Skoric, Nina Japundzic-Zigon, Dragana Bajic
2017 Computational and Mathematical Methods in Medicine  
The latter ones were additionally binary encoded for a joint conditional entropy application.  ...  This paper analyses temporal dependency in the time series recorded from aging rats, the healthy ones and those with early developed hypertension.  ...  Suave Lobodzinski, UCLA, for the inspiring discussion during and after Neurocard 2016.  ... 
doi:10.1155/2017/2082351 pmid:28127384 pmcid:PMC5240046 fatcat:zrxxving2jez5fvm5o3ew6vf2q

Phoebe: A Learning-based Checkpoint Optimizer [article]

Yiwen Zhu, Matteo Interlandi, Abhishek Roy, Krishnadhan Das, Hiren Patel, Malay Bag, Hitesh Sharma, Alekh Jindal
2021 pre-print
failures, and worse query optimizer estimates being examples of issues that we are facing at Microsoft.  ...  For each stage of a job, Phoebe makes accurate predictions for: (1) the execution time, (2) the output size, and (3) the start/end time taking into account the inter-stage dependencies.  ...  Table 1 : 1 Cost model features Feature Group Feature Name Feature De- scription Query Optimizer Features Estimated Cost, Estimated Input Cardinality, Estimated Exclusive Cost, Estimated Cardinality  ... 
doi:10.14778/3476249.3476298 arXiv:2110.02313v1 fatcat:x4x3kuq2w5di7czzl6kbdqruw4