Incident threading for news passages

Ao Feng, James Allan
2009 Proceeding of the 18th ACM conference on Information and knowledge management - CIKM '09  
With an overwhelming volume of news reports currently available, there is an increasing need for automatic techniques to analyze and present news to a general reader in a meaningful and efficient manner. We explore incident threading as a possible solution to this problem. All text that describes the occurrence of a real-world happening is merged into a news incident, and incidents are organized in a network with dependencies of predefined types. Earlier attempts at this problem have assumed
more » ... t a news story covers a single topic. We move beyond that limitation to introduce passage threading, which processes news at the passage level. First we develop a new testbed for this research and extend the evaluation methods to address new granularity issues. Then a three-stage algorithm is described that identifies on-subject passages, groups them into incidents, and establishes links between related incidents. Finally, we observe significant improvement over earlier work when we optimize the harmonic mean of the appropriate evaluation measures. The resulting performance exceeds the level that a calibration study shows is necessary to support a reading comprehension task.
doi:10.1145/1645953.1646118 dblp:conf/cikm/FengA09 fatcat:kfultzg2zjgpreqh6z3zbpajo4