Querying at Internet scale

Brent Chun, Ion Stoica, Joseph M. Hellerstein, Ryan Huebsch, Shawn R. Jeffery, Boon Thau Loo, Sam Mardanbeigi, Timothy Roscoe, Sean Rhea, Scott Shenker
2004 Proceedings of the 2004 ACM SIGMOD international conference on Management of data - SIGMOD '04  
We are developing a distributed query processor called PIER, which is designed to run on the scale of the entire Internet. PIER utilizes a Distributed Hash Table ( DHT) as its communication substrate in order to achieve scalability, reliability, decentralized control, and load balancing. PIER enhances DHTs with declarative and algebraic query interfaces, and underneath those interfaces implements multihop, in-network versions of joins, aggregation, recursion, and query/result dissemination.
more » ... dissemination. PIER is currently being used for diverse applications, including network monitoring, keyword-based filesharing search, and network topology mapping. We will demonstrate PIER's functionality by showing system monitoring queries running on PlanetLab, a testbed of over 300 machines distributed across the globe.
doi:10.1145/1007568.1007705 dblp:conf/sigmod/ChunRHHJLMRSS04 fatcat:uqgjp2rmcnbrvjwsxsuncbg4ou