A Comparative Study of Parameter Estimation Methods for Statistical Natural Language Processing

Jianfeng Gao, Galen Andrew, Mark Johnson, Kristina Toutanova
2007 Annual Meeting of the Association for Computational Linguistics  
This paper presents a comparative study of five parameter estimation algorithms on four NLP tasks. Three of the five algorithms are well-known in the computational linguistics community: Maximum Entropy (ME) estimation with L 2 regularization, the Averaged Perceptron (AP), and Boosting. We also investigate ME estimation with L 1 regularization using a novel optimization algorithm, and BLasso, which is a version of Boosting with Lasso (L 1 ) regularization. We first investigate all of our
more » ... ors on two re-ranking tasks: a parse selection task and a language model (LM) adaptation task. Then we apply the best of these estimators to two additional tasks involving conditional sequence models: a Conditional Markov Model (CMM) for part of speech tagging and a Conditional Random Field (CRF) for Chinese word segmentation. Our experiments show that across tasks, three of the estimators -ME estimation with L 1 or L 2 regularization, and APare in a near statistical tie for first place.
dblp:conf/acl/GaoAJT07 fatcat:wkd6dgqsovbohmjlxac4tyuyjy