Query-driven indexing for peer-to-peer text retrieval

Gleb Skobeltsyn, Toan Luu, Karl Aberer, Martin Rajman, Ivana Podnar Zarko
2007 Proceedings of the 16th international conference on World Wide Web - WWW '07  
We describe a query-driven indexing framework for scalable text retrieval over structured P2P networks. To cope with the bandwidth consumption problem that has been identified as the major obstacle for full-text retrieval in P2P networks, we truncate posting lists associated with indexing features to a constant size storing only top-k ranked document references. To compensate for the loss of information caused by the truncation, we extend the set of indexing features with carefully chosen term
more » ... ets. Indexing term sets are selected based on the query statistics extracted from query logs, thus we index only such combinations that are a) frequently present in user queries and b) non-redundant w.r.t the rest of the index. The distributed index is compact and efficient as it constantly evolves adapting to the current query popularity distribution. Moreover, it is possible to control the tradeoff between the storage/bandwidth requirements and the quality of query answering by tuning the indexing parameters. Our theoretical analysis and experimental results indicate that we can indeed achieve scalable P2P text retrieval for very large document collections and deliver good retrieval performance.
doi:10.1145/1242572.1242757 dblp:conf/www/SkobeltsynLARZ07 fatcat:j75qmwpnlfh5xmlc7oaghbjpwi