Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages [article]

Muhammad Imran, Prasenjit Mitra, Carlos Castillo
<span title="2016-05-31">2016</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Microblogging platforms such as Twitter provide active communication channels during mass convergence and emergency events such as earthquakes, typhoons. During the sudden onset of a crisis situation, affected people post useful information on Twitter that can be used for situational awareness and other humanitarian disaster response efforts, if processed timely and effectively. Processing social media information pose multiple challenges such as parsing noisy, brief and informal messages,
more &raquo; ... ing information categories from the incoming stream of messages and classifying them into different classes among others. One of the basic necessities of many of these tasks is the availability of data, in particular human-annotated data. In this paper, we present human-annotated Twitter corpora collected during 19 different crises that took place between 2013 and 2015. To demonstrate the utility of the annotations, we train machine learning classifiers. Moreover, we publish first largest word2vec word embeddings trained on 52 million crisis-related tweets. To deal with tweets language issues, we present human-annotated normalized lexical resources for different lexical variations.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1605.05894v2">arXiv:1605.05894v2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/7fbntmastvebzil23ozkrev4iq">fatcat:7fbntmastvebzil23ozkrev4iq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20191014121417/https://arxiv.org/pdf/1605.05894v2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/c6/63/c6632a818621ddca523d6b6fe0749dd114c23d60.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1605.05894v2" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>