TREC Incident Streams: Finding Actionable Information on Social Media

Richard McCreadie, Cody Buntain, Ian Soboroff
2019 International Conference on Information Systems for Crisis Response and Management  
The Text Retrieval Conference (TREC) Incident Streams track is a new initiative that aims to mature social media-based emergency response technology. This initiative advances the state of the art in this area through an evaluation challenge, which attracts researchers and developers from across the globe. The 2018 edition of the track provides a standardized evaluation methodology, an ontology of emergency-relevant social media information types, proposes a scale for information criticality,
more » ... releases a dataset containing fifteen test events and approximately 20,000 labeled tweets. Analysis of this dataset reveals a significant amount of actionable information on social media during emergencies (> 10%). While this data is valuable for emergency response efforts, analysis of the 39 state-of-the-art systems demonstrate a performance gap in identifying this data. We therefore find the current state-of-the-art is insufficient for emergency responders' requirements, particularly for rare actionable information for which there is little prior training data available.
dblp:conf/iscram/McCreadieBS19 fatcat:lvkg2v4flzak3p5xkoqwh7jaoa