Scalability and efficiency challenges in large-scale web search engines

Ricardo Baeza-Yates, B. Barla Cambazoglu
2014 Proceedings of the 23rd International Conference on World Wide Web - WWW '14 Companion  
The main goals of a web search engine are quality, e ciency, and scalability. In this tutorial, we focus on the last two goals, providing a fairly comprehensive overview of the scalability and e ciency challenges in large-scale web search engines. In particular, the tutorial provides an in-depth architectural overview of a web search engine, mainly focusing on the web crawling, indexing, and query processing components. The scalability and e ciency issues encountered in these components are
more » ... ented at four di↵erent granularities: at the level of a single computer, a cluster of computers, a single data center, and a multi-center search engine. The tutorial also points at open research problems and provides recommendations to researchers who are new to the field.
doi:10.1145/2567948.2577271 dblp:conf/www/Baeza-YatesC14 fatcat:ygasqlbobjbrzipvtd2kj4babq