Using Exclusive Web Crawlers to Store Better Results in Search Engines'Database

Ali Tourani, Amir Seyed Danesh
<span title="2013-04-30">2013</span> <i title="Academy and Industry Research Collaboration Center (AIRCC)"> <a target="_blank" rel="noopener" href="" style="color: black;">International journal of Web &amp; Semantic Technology</a> </i> &nbsp;
Crawler-based search engines are the mostly used search engines among web and Internet users , involve web crawling, storing in database, ranking, indexing and displaying to the user. But it is noteworthy that because of increasing changes in web sites search engines suffer high time and transfers costs which are consumed to investigate the existence of each page in database while crawling, updating database and even investigating its existence in any crawling operations. "Exclusive Web
more &raquo; ... proposes guidelines for crawling features, links, media and other elements and to store crawling results in a certain table in its database on the web. With doing this, search engines store each site's tables in their databases and implement their ranking results on them. Thus, accuracy of data in every table (and its being up-to-date) is ensured and no 404 result is shown in search results since, in fact, this data crawler crawls data entered by webmaster and the database stores whatever he wants to display. Keywords exclusive crawler, search engine, crawler, database 1.Introduction Searching is a daily activity of many people. Individuals deal with this task every day. The need to search enhances along with the increasing demand for more knowledge and information. Also, in computer science search and communication are among most important items so that they can be seen in most applications, programs and even on the web and Internet. Nowadays the Internet contains millions of information containing pages. Knowing a site's address users can connect it and use its resources and information. In order to reach this huge information resource human has began to increasingly use search engines. For this reason search engines repeat their crawling, ranking and indexing operations every day. [12] Among these daily operations crawling specifies the most basic function of a search engine. The biggest search engines of the world do this operation in offline mode (query is not accomplished while entering but it is already done) [16] and its optimization does not influence response received by the user but better results can be embedded in the database through optimization. [17] Making webmasters responsible for storing operations through "exclusive crawlers" it can be expected that better results are observed in output sent to the user by filling the search engine's
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="">doi:10.5121/ijwest.2013.4201</a> <a target="_blank" rel="external noopener" href="">fatcat:7bkmfo67jjhstfhisqns5q3gmu</a> </span>
<a target="_blank" rel="noopener" href="" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href=""> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / </button> </a>