7,489 Hits in 7.4 sec

A term-based inverted index partitioning model for efficient distributed query processing

B. Barla Cambazoglu, Enver Kayaaslan, Simon Jonassen, Cevdet Aykanat
2013 ACM Transactions on the Web  
A term-based inverted index partitioning model for efficient distributed query processing.  ...  In a shared-nothing, distributed text retrieval system, queries are processed over an inverted index that is partitioned among a number of index servers.  ...  The focus of this work is on efficient query processing on distributed text retrieval systems where the index is partitioned based on terms.  ... 
doi:10.1145/2516633.2516637 fatcat:hkuqmjmbvzakpmc4degshrx26q

Exploiting Parallelism in Query Processing for Web Document Search Using Shared-Memory and Cluster-Based Architectures

Amal Elsayed Aboutabl
2013 Computer and Information Science  
Results show that in terms of average response time, speedup and efficiency, the proposed model for parallel query processing on shared-memory architecture outperforms that on cluster-based architecture  ...  In this paper, we present a model for parallel query processing. Then, this model is extended particularly for usage on shared-memory and cluster parallel architectures.  ...  Index Partitioning Schemes Inverted Indices Currently, search engines use the inverted index data structure for faster and more efficient query processing (Zobel & Moffiat, 2006) .  ... 
doi:10.5539/cis.v6n3p125 fatcat:3mpcqs3odbgbpagtvzhk4x7vue

Effect of Inverted Index Partitioning Schemes on Performance of Query Processing in Parallel Text Retrieval Systems [chapter]

B. Barla Cambazoglu, Aytul Catal, Cevdet Aykanat
2006 Lecture Notes in Computer Science  
In general, the index can be partitioned based on either the terms or documents in the collection, and the way the partitioning is done greatly affects the query processing performance of the parallel  ...  Performance results are reported for a large (30 GB) document collection using an MPI-based parallel query processing implementation.  ...  This is due to the more balanced distribution of the query processing load on index servers in the case of document-based partitioning.  ... 
doi:10.1007/11902140_75 fatcat:p5xepgwmxbg75fjxbq26zsmrky

Load balancing distributed inverted files

Mauricio Marin, Carlos Gomez
2007 Proceedings of the 9th annual ACM international workshop on Web information and data management - WIDM '07  
This paper present a comparison of scheduling algorithms applied to the context of load balancing the query traffic on distributed inverted files.  ...  We propose a novel method to formulate the cost of query processing so that these algorithms can be used to schedule queries onto processors.  ...  INTRODUCTION Cluster based search engines use distributed inverted files [10] for dealing efficiently with high traffic of user queries.  ... 
doi:10.1145/1316902.1316912 dblp:conf/widm/MarenG07 fatcat:tg763relkfapzo7xlgesinn5ey

ImprovingWeb Searches with Distributed Buckets Structures

V. Costa, A. Printista, M. Marin
2006 2006 Fourth Latin American Web Congress  
This article compares several strategies for searching in Web engines and we present the bucket algorithms to improve the efficiency of a classical index data structure for parallel textual database.  ...  The design of the server that processes the queries, is effected on top of the Bulk Synchronous-BSP model of parallel computing, to study how query performance is affected by the index organization.  ...  , for the partitioned inverted files implemented under the BSP model, and for our Figure 3 . 3 Figure 3.  ... 
doi:10.1109/la-web.2006.18 dblp:conf/la-web/CostaPM06 fatcat:xokhjcsybfcgxjfup6dbuh6zmy

BigDataGrapes D3.3 - Distributed Indexing Components

Rossano Venturini, Raffaele Perego, Milena Yankova, Pythagoras Karampiperis
2018 Zenodo  
Specifically, we investigated the efficiency and effectiveness dimensions of indexes for RDF triples based on inverted indexes, and designed a novel compression technique for making these indexes more  ...  The goal of the BDG Distributed Indexing activity is to develop novel methodologies and components for realizing efficient indexing over distributed big data batch and cross-streaming sources.  ...  Inverted index compression is essential to make efficient use of the memory hierarchy, thus maximizing query processing speed.  ... 
doi:10.5281/zenodo.1481838 fatcat:62zdz3anmzaffozlxyrzfeoiay

Efficiency Considerations for Scalable Information Retrieval Servers

Ophir Frieder, David A. Grossman, Abdur Chowdhury, Gideon Frieder
2000 Journal of Digital Information  
Given the importance of index processing, in addition to a general overview, we include some recent index maintenance results.  ...  We review a variety of techniques to improve efficiency in information retrieval.  ...  Acknowledgment We thank Nazli Goharian and Don Kraft for their insightful comments that dramatically improved the quality and clarity of this paper.  ... 
dblp:journals/jodi/FriederGCF00 fatcat:sfwv57h4grd7djvk6zx33e7hdi

Making Peer-to-Peer Keyword Searching Feasible Using Multi-level Partitioning [chapter]

Shuming Shi, Guangwen Yang, Dingxing Wang, Jin Yu, Shaogang Qu, Ming Chen
2005 Lecture Notes in Computer Science  
The state-of-the-art keyword searching techniques for unstructured and structured P2P systems are query flooding and inverted list intersection respectively.  ...  We propose in this paper a new index partitioning and building scheme, multi-level partitioning (MLP), and discuss its implementation on top of P2P networks.  ...  Acknowledgments We thank Zheng Zhang at Microsoft Research Asia for useful discussions. We thank the anonymous reviewers for their comments.  ... 
doi:10.1007/978-3-540-30183-7_15 fatcat:23gie66hgvgpfki2lfqtz6pbba

Efficient query processing in distributed search engines

Simon Jonassen
2012 SIGIR Forum  
We present an efficient self-skipping inverted index designed for modern index compression methods and several query processing optimizations.  ...  In this thesis, we elaborate on query processing efficiency, address several problems within partitioned query processing, pruning and caching and propose several novel techniques: First, we look at term-wise  ...  Rocha-Junior for the useful advices and comments on the paper. Acknowledgments. This work was done while the second author was an intern at Yahoo!  ... 
doi:10.1145/2492189.2492201 fatcat:uwasxhngrfgntemkhawyv3te64

WINGS: A Parallel Indexer for Web Contents [chapter]

Fabrizio Silvestri, Salvatore Orlando, Raffaele Perego
2004 Lecture Notes in Computer Science  
By exploiting both data and pipeline parallelism, our prototype indexer efficiently builds a partitioned inverted compressed index, a suitable data structure commonly utilized by modern Web Search Engines  ...  In this paper we discuss the design of a parallel indexer for Web documents.  ...  Fig. 1 . 1 Construction of a distributed index based on the document partition paradigm, according to which each local inverted index only refers to a partition of the whole document collection.  ... 
doi:10.1007/978-3-540-24685-5_33 fatcat:myzngisv7zcotoydmfquvjbotu

Parallel computing in information retrieval – an updated review

A. Macfarlane, S.E. Robertson, J.A. Mccann
1997 Journal of Documentation  
We give a description of the retrieval models used in parallel Information Processing.. We describe areas of research which we believe are needed.  ...  A. DAP (Distributed Array of Processors). The AMT (formally ICL) DAP is a SIMD class architecture.  ...  ACKNOWLEDGEMENTS This research is supported by the Department for Education and Employment, grant number IS96/4197.  ... 
doi:10.1108/eum0000000007201 fatcat:2zuwtehixbd6xk33hwb3j43nse

Optimized Inverted List Assignment in Distributed Search Engine Architectures

Jiangong Zhang, Torsten Suel
2007 2007 IEEE International Parallel and Distributed Processing Symposium  
We study efficient query processing in distributed web search engines with global index organization.  ...  The main performance bottleneck in this case is due to the large amount of index data that is exchanged between nodes during the processing of a query, and previous work has proposed several techniques  ...  We also thank Xiaohui Long for his work on the indexing and query execution software used in our experiments.  ... 
doi:10.1109/ipdps.2007.370231 dblp:conf/ipps/ZhangS07 fatcat:3fp7n7obarfupjwnqm4ehtg5mi

A pipelined architecture for distributed text query evaluation

Alistair Moffat, William Webber, Justin Zobel, Ricardo Baeza-Yates
2006 Information retrieval (Boston)  
In a term-partitioned system, each of the processors hosts a subset of the inverted lists that make up the index of the collection, and serves them to a central machine as they are required for query evaluation  ...  Two principal query-evaluation methodologies have been described for clusterbased implementation of distributed information retrieval systems: document partitioning and term partitioning.  ...  Pipelined query processing with a term-partitioned index.  ... 
doi:10.1007/s10791-006-9014-4 fatcat:4rac6uumrzelrnfmq4dmn44czm

A Parallel Framework for In-Memory Construction of Term-Partitioned Inverted Indexes

T. Kucukyilmaz, A. Turk, C. Aykanat
2012 Computer journal  
Construction of term-based, partitioned, parallel inverted indexes is a communication intensive task and suitable for memory-based modeling.  ...  In this paper, we provide an efficient parallel framework for in-memory construction of term-based partitioned, inverted indexes.  ...  The final term-partitioned inverted index will also be stored in a distributed fashion in order to allow both inter-and intra-query parallelism on query processing.  ... 
doi:10.1093/comjnl/bxr133 fatcat:wqrz6taydnhjbkhhcip56j5zxy

The Potential of Learned Index Structures for Index Compression [article]

Harrie Oosterhuis, J. Shane Culpepper, Maarten de Rijke
2018 arXiv   pre-print
Inverted indexes are vital in providing fast key-word-based search.  ...  Our work shows that learned models have great potential in inverted indexing, and this direction seems to be a promising area for future research.  ...  This work was partially supported by the Netherlands Organisation for Scientific Research (NWO) under project nr. 612.001.551 and the Australian Research Council's Discovery Projects Scheme (DP170102231  ... 
arXiv:1811.06678v1 fatcat:3dceboshujfntozgpxyrrekmzi
« Previous Showing results 1 — 15 out of 7,489 results