Next Sentence Prediction helps Implicit Discourse Relation Classification within and across Domains

Wei Shi, Vera Demberg
2019 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)  
Implicit discourse relation classification is one of the most difficult tasks in discourse parsing. Previous studies have generally focused on extracting better representations of the relational arguments. In order to solve the task, it is however additionally necessary to capture what events are expected to cause or follow each other. Current discourse relation classifiers fall short in this respect. We here show that this shortcoming can be effectively addressed by using the bidirectional
more » ... e bidirectional encoder representation from transformers (BERT) proposed by Devlin et al. (2019) , which were trained on a nextsentence prediction task, and thus encode a representation of likely next sentences. The BERT-based model outperforms the current state of the art in 11-way classification by 8% points on the standard PDTB dataset. Our experiments also demonstrate that the model can be successfully ported to other domains: on the BioDRB dataset, the model outperforms the state of the art system around 15% points.
doi:10.18653/v1/d19-1586 dblp:conf/emnlp/ShiD19 fatcat:rvqmcvdijfdfxondi2ctst5yhq