A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is
Design of a Migrating Crawler Based on a Novel URL Scheduling Mechanism using AHP
International Journal of Rough Sets and Data Analysis
In order to manage the vast information available on web, crawler plays a significant role. The working of crawler should be optimized to get maximum and unique information from the World Wide Web. In this paper, architecture of migrating crawler is proposed which is based on URL ordering, URL scheduling and document redundancy elimination mechanism. The proposed ordering technique is based on URL structure, which plays a crucial role in utilizing the web efficiently. Scheduling ensures thatdoi:10.4018/ijrsda.2017010106 fatcat:43k7w3lknjbvpjroufwhemti3a