Context and learning in novelty detection

Barry Schiffman, Kathleen R. McKeown
2005 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05   unpublished
We demonstrate the value of using context in a new-information detection system that achieved the highest precision scores at the Text Retrieval Conference's Novelty Track in 2004. In order to determine whether information within a sentence has been seen in material read previously, our system integrates information about the context of the sentence with novel words and named entities within the sentence, and uses a specialized learning algorithm to tune the system parameters.
doi:10.3115/1220575.1220665 fatcat:h6x4snvpa5blvkh5xore77c5cm