Extracting Possessions from Social Media: Images Complement Language

Dhivya Chinnappa, Srikala Murugan, Eduardo Blanco
2019 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)  
This paper describes a new dataset and experiments to determine whether authors of tweets possess the objects they tweet about. We work with 5,000 tweets and show that both humans and neural networks benefit from images in addition to text. We also introduce a simple yet effective strategy to incorporate visual information into any neural network beyond weights from pretrained networks. Specifically, we consider the tags identified in an image as an additional textual input, and leverage
more » ... ned word embeddings as usually done with regular text. Experimental results show this novel strategy is beneficial.
doi:10.18653/v1/d19-1061 dblp:conf/emnlp/ChinnappaMB19 fatcat:dhbhwqghlrhl7moypl4yvuegum