SMARTINT: using mined attribute dependencies to integrate fragmented web databases

Ravi Gummadi, Anupam Khulbe, Aravind Kalavagattu, Sanil Salvi, Subbarao Kambhampati
2011 Journal of Intelligent Information Systems  
Many web databases can be seen as providing partial and overlapping information about entities in the world. To answer queries effectively, we need to integrate the information about the individual entities that are fragmented over multiple sources. At first blush this is just the inverse of traditional database normalization problem -rather than go from a universal relation to normalized tables, we want to reconstruct the universal relation given the tables (sources). The standard way of
more » ... tructing the entities will involve joining the tables. Unfortunately, because of the autonomous and decentralized way in which the sources are populated, they often do not have Primary Key -Foreign Key relations. While tables do share attributes, direct joins over these shared attributes can result in reconstruction of many spurious entities thus seriously compromising precision. We present a unified approach that supports intelligent retrieval over fragmented web databases by mining and using intertable dependencies. Experiments with the prototype implementation, SMARTINT, show that its retrieval strikes a good balance between precision and recall. See arxiv.org/abs/1101.5334 for a longer version of this paper
doi:10.1007/s10844-011-0169-0 fatcat:voy66c2sijacjb7gaceqx4o7p4