A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2006; you can also visit the original URL.
The file type is application/pdf.
Distributed location aware web crawling
2004
Alternate track papers & posters of the 13th international conference on World Wide Web - WWW Alt. '04
Preserved Fulltext
Distributed crawling has shown that it can overcome important limitations of the today's crawling paradigm. However, the optimal benefits of this approach are usually limited to the sites hosting the crawler. In this work, we propose a location-aware method, called IPMicra, that utilizes an IP address hierarchy, and allows crawling of links in a near optimal location aware manner.
doi:10.1145/1010432.1010594
fatcat:j4nwtxhpxzet3ifoq3iwyxbbqu