Building efficient multi-threaded search nodes

Carolina Bonacic, Carlos García, Mauricio Marin, Manuel Prieto-Matias, Francisco Tirado
2010 Proceedings of the 19th ACM international conference on Information and knowledge management - CIKM '10  
Search nodes are single-purpose components of large Web search engines and their efficient implementation is critical to sustain thousands of queries per second and guarantee individual query response times within a fraction of a second. Current technology trends indicate that search nodes ought to be implemented as multithreaded multi-core systems. The straightforward solution that system designers can apply in this case is simply to follow standard practice by deploying one asynchronous
more » ... per active query in the node and attaching each thread to a different core. Each concurrent thread is responsible for sequentially processing a single query at a time. The only potential source of read/write conflicts among threads are the accesses to the different application caches present in the search node. However, new Web applications pose much more demanding requirements in terms of read/write conflicts than recent past applications since now data updates must take place concurrently with query processing. Insisting on the same paradigm of concurrent threads now augmented with a transaction concurrency control protocol is a feasible solution. In this paper we propose a more efficient and much simpler solution which has the additional advantage of enabling a very efficient administration of application caches. We propose performing relaxed bulksynchronous parallelism at multi-core level.
doi:10.1145/1871437.1871595 dblp:conf/cikm/BonacicGMPT10 fatcat:h2n3pr6xt5c7dddrsgotkfdp2e