Myanmar Web Pages Crawler

Su Mon Khine, Yadana Thein
2015 International Journal on Web Service Computing  
Nowadays web pages are implemented in various kinds of languages on the Web and web crawlers are important for search engine. Language specific crawlers are crawlers that traverse and collect the relative web pages using the successive URls of web page. There are very few research areas in crawling for Myanmar Language web sites. Most of the language specific crawlers are based on n-gram character sequences which require training documents. The proposed crawler differs from those crawlers. The
more » ... hose crawlers. The proposed crawler searches and retrieves Myanmar web pages for Myanmar Language search engine. The proposed crawler detects the Myanmar character and rule-based syllable threshold is used to judgment the relevance of the pages. According to experimental results, the proposed crawler has better performance, achieves successful accuracy and storage space for search engines are lesser since it only crawls the relevant documents for Myanmar web sites. KEYWORDS Language specific crawler, Myanmar Language, rule-based syllable segmentation.
doi:10.5121/ijwsc.2015.6101 fatcat:xrq2s6vfi5fibecxj35scw5iqu