Data Profiling

Ziawasch Abedjan, Lukasz Golab, Felix Naumann
2017 Proceedings of the 2017 ACM International Conference on Management of Data - SIGMOD '17  
One of the crucial requirements before consuming datasets for any application is to understand the dataset at hand and its metadata. The process of metadata discovery is known as data profiling. Profiling activities range from ad-hoc approaches, such as eye-balling random subsets of the data or formulating aggregation queries, to systematic inference of structural information and statistics of a dataset using dedicated profiling tools. In this tutorial, we highlight the importance of data
more » ... ing as part of any data-related usecase, and we discuss the area of data profiling by classifying data profiling tasks and reviewing the state-of-the-art data profiling systems and techniques. In particular, we discuss hard problems in data profiling, such as algorithms for dependency discovery and profiling algorithms for dynamic data and streams. We also pay special attention to visualizing and interpreting the results of data profiling. We conclude with directions for future research in the area of data profiling. This tutorial is based on our survey on profiling relational data [2] .
doi:10.1145/3035918.3054772 dblp:conf/sigmod/AbedjanGN17 fatcat:dwqqb6w6pzfu7l5nkz3m67oxsq