The World Wide Web (WWW) is being prolonged by an impulsive speed. As a result, search engines encounter many challenges such as yielding accurate and conversant results to the users, and responding them in an appropriate timely manner. A crawler is a program that downloads and stores Web pages, often for a Web search engine. Web crawler (also known as a Web spider or Web robot) is a program or automated script which browses the World Wide Web crawlers are mainly used to create a copy of all

more »

... visited pages for later processing by a search engine, which will index the downloaded pages to provide fast searches. We have concluded that the advantage with Path-mounting crawler is that they are very effective in finding isolated resources, or resources for which no inbound link would have been found in regular crawling.