Dependency Structure Applied to Language Modeling for Information Retrieval

Changki Lee, Gary Geunbae Lee, Myung Gil Jang
2006 ETRI Journal  
In this paper, we propose a new language model, namely, a dependency structure language model, for information retrieval to compensate for the weaknesses of unigram and bigram language models. The dependency structure language model is based on the first-order dependency model and the dependency parse tree generated by a linguistic parser. So, long-distance dependencies can be naturally captured by the dependency structure language model. We carried out extensive experiments to verify the
more » ... to verify the proposed model, where the dependency structure model gives a better performance than recently proposed language models and the Okapi BM25 method, and the dependency structure is more effective than unigram and bigram in language modeling for information retrieval.
doi:10.4218/etrij.06.0105.0020 fatcat:hvnu2jm4zngujm2m7oe6x2442y