SUM-QE: a BERT-based Summary Quality Estimation Model

Stratos Xenouleas, Prodromos Malakasiotis, Marianna Apidianaki, Ion Androutsopoulos
2019 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)  
We propose SUM-QE, a novel Quality Estimation model for summarization based on BERT. The model addresses linguistic quality aspects that are only indirectly captured by content-based approaches to summary evaluation, without involving comparison with human references. SUM-QE achieves very high correlations with human ratings, outperforming simpler models addressing these linguistic aspects. Predictions of the SUM-QE model can be used for system development, and to inform users of the quality of
more » ... s of the quality of automatically produced summaries and other types of generated text.
doi:10.18653/v1/d19-1618 dblp:conf/emnlp/XenouleasMAA19 fatcat:74dzq5oppjgxxb6zf53vwu5rli