Cheap and easy entity evaluation

Ben Hachey, Joel Nothman, Will Radford
2014 Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)  
The AIDA-YAGO dataset is a popular target for whole-document entity recognition and disambiguation, despite lacking a shared evaluation tool. We review evaluation regimens in the literature while comparing the output of three approaches, and identify research opportunities. This utilises our open, accessible evaluation tool. We exemplify a new paradigm of distributed, shared evaluation, in which evaluation software and standardised, versioned system outputs are provided online.
doi:10.3115/v1/p14-2076 dblp:conf/acl/HacheyNR14 fatcat:55pyai2ldnasvaf5tfe65yzcte