Filters








93,321 Hits in 4.4 sec

A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets [article]

Changchang Zeng, Shaobo Li, Qin Li, Jie Hu, Jianjun Hu
2020 arXiv   pre-print
To address the current lack of comprehensive survey of existing MRC tasks, evaluation metrics, and datasets, herein, (1) we analyze 57 MRC tasks and datasets and propose a more precise classification method  ...  At present, a lot of MRC models have already surpassed human performance on various benchmark datasets despite the obvious giant gap between existing MRC models and genuine human-level reading comprehension  ...  Conflicts of Interest: The authors declare no conflict of interest.  ... 
arXiv:2006.11880v2 fatcat:auup4gvsuzf4dkjb2n6nzyfl3m

A Survey on Machine Reading Comprehension—Tasks, Evaluation Metrics and Benchmark Datasets

Changchang Zeng, Shaobo Li, Qin Li, Jie Hu, Jianjun Hu
2020 Applied Sciences  
To address the current lack of comprehensive survey of existing MRC tasks, evaluation metrics, and datasets, herein, (1) we analyze 57 MRC tasks and datasets and propose a more precise classification method  ...  At present, a lot of MRC models have already surpassed human performance on various benchmark datasets despite the obvious giant gap between existing MRC models and genuine human-level reading comprehension  ...  Conflicts of Interest: The authors declare no conflict of interest.  ... 
doi:10.3390/app10217640 fatcat:e6rioqqdnbdqpimpxzvowhgv7a

A Multilingual Modeling Method for Span-Extraction Reading Comprehension [article]

Gaochen Wu, Bin Xu, Dejie Chang, Bangchang Liu
2021 arXiv   pre-print
Span-extraction reading comprehension models have made tremendous advances enabled by the availability of large-scale, high-quality training datasets.  ...  Despite such rapid progress and widespread application, extractive reading comprehension datasets in languages other than English remain scarce, and creating such a sufficient amount of training data for  ...  This work was also supported by the Ministry of Science and Technology via grant 2017YFB1401903 and 2018YFB1005101.  ... 
arXiv:2105.14880v1 fatcat:57m24zap4ffvhe4qv4qngnxrcu

ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension [article]

Dheeru Dua, Ananth Gottumukkala, Alon Talmor, Sameer Singh, Matt Gardner
2019 arXiv   pre-print
Given the availability of many such datasets, comprehensive and reliable evaluation is tedious and time-consuming for researchers working on this problem.  ...  Reading comprehension is one of the crucial tasks for furthering research in natural language understanding.  ...  We believe this strategy of evaluating on many datasets, including distribution-shifted synthetic examples, will lead the field towards more robust and comprehensive reading comprehension models.  ... 
arXiv:1912.12598v1 fatcat:qmdmyoj73zhfldcllcybxqr3da

Sogou Machine Reading Comprehension Toolkit [article]

Jindou Wu, Yunlun Yang, Chao Deng, Hongyi Tang, Bingning Wang, Haoze Sun, Ting Yao, Qi Zhang
2019 arXiv   pre-print
In this paper, we present a Sogou Machine Reading Comprehension (SMRC) toolkit that can be used to provide the fast and efficient development of modern machine comprehension models, including both published  ...  Machine reading comprehension have been intensively studied in recent years, and neural network-based models have shown dominant performances.  ...  Because of the renaissance of neural networks and accessibility of large-scale datasets, great progress has recently been made in reading comprehension.  ... 
arXiv:1903.11848v2 fatcat:dojjqtknvfc33jer7b2khjj56m

JEC-QA: A Legal-Domain Question Answering Dataset

Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, Maosong Sun
2020 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
The examination is a comprehensive evaluation of professional skills for legal practitioners. College students are required to pass the examination to be certified as a lawyer or a judge.  ...  We will release JEC-QA and our baselines to help improve the reasoning ability of machine comprehension models. You can access the dataset from http://jecqa.thunlp.org/.  ...  Acknowledgements This work is supported by the National Key Research and Development Program of China (No. 2018YFC0831900) and the National Natural Science Foundation of China (NSFC No. 61572273, 61661146007  ... 
doi:10.1609/aaai.v34i05.6519 fatcat:5zbbp4df25drvnnrh2qmde27ai

JEC-QA: A Legal-Domain Question Answering Dataset [article]

Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, Maosong Sun
2019 arXiv   pre-print
The examination is a comprehensive evaluation of professional skills for legal practitioners. College students are required to pass the examination to be certified as a lawyer or a judge.  ...  We will release JEC-QA and our baselines to help improve the reasoning ability of machine comprehension models. You can access the dataset from http://jecqa.thunlp.org/.  ...  Acknowledgements This work is supported by the National Key Research and Development Program of China (No. 2018YFC0831900) and the National Natural Science Foundation of China (NSFC No. 61572273, 61661146007  ... 
arXiv:1911.12011v1 fatcat:eklv2ozujrafnjwlzzyva234wy

Benchmarking Machine Reading Comprehension: A Psychological Perspective [article]

Saku Sugawara, Pontus Stenetorp, Akiko Aizawa
2021 arXiv   pre-print
However, the conventional task design of MRC lacks explainability beyond the model interpretation, i.e., reading comprehension by a model cannot be explained in human terms.  ...  We conclude that future datasets should (i) evaluate the capability of the model for constructing a coherent and grounded representation to understand context-dependent situations and (ii) ensure substantive  ...  Acknowledgments The authors would like to thank Xanh Ho for helping create the dataset list and the anonymous reviewers for their insightful comments.  ... 
arXiv:2004.01912v2 fatcat:lyypngwm4vbk7igfcjfmhkn5ja

Improving Low-resource Reading Comprehension via Cross-lingual Transposition Rethinking [article]

Gaochen Wu, Bin Xu, Yuxin Qin, Fei Kong, Bangchang Liu, Hongwen Zhao, Dejie Chang
2021 arXiv   pre-print
Extractive Reading Comprehension (ERC) has made tremendous advances enabled by the availability of large-scale high-quality ERC training data.  ...  To address this issue, we propose a Cross-Lingual Transposition ReThinking (XLTT) model by modelling existing high-quality extractive reading comprehension datasets in a multilingual environment.  ...  This work was supported by the Ministry of Science and Technology via grant 2017YFB1401903 and 2018YFB1005101. This work was also supported by Beijing MoreHealth Technology Group Co. Ltd.  ... 
arXiv:2107.05002v2 fatcat:on63mhbnrnebrjbvs4rn6rlkx4

Improving Machine Reading Comprehension with Multi-Task Learning and Self-Training

Jianquan Ouyang, Mengen Fu
2022 Mathematics  
Therefore, to meet the comprehensive requirements in such application situations, we construct a multi-task fusion training reading comprehension model based on the BERT pre-training model.  ...  We evaluated the SQuAD2.0 and CAIL2019 datasets. The experiments show that our model can efficiently handle different tasks.  ...  The results of different sized labeled datasets for self-training improvement. (a) SQuAD 2.0 dataset. (b) CAIL2019 Reading Comprehension dataset.  ... 
doi:10.3390/math10030310 fatcat:4gc3cwac5bd3tck57pnicdbfse

RACE: Large-scale ReAding Comprehension Dataset From Examinations [article]

Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, Eduard Hovy
2017 arXiv   pre-print
We present RACE, a new dataset for benchmark evaluation of methods in the reading comprehension task.  ...  We hope this new dataset can serve as a valuable resource for research and evaluation in machine comprehension.  ...  This comprehensiveness of topic/style coverage makes RACE a desirable resource for evaluating the reading comprehension ability of machine learning systems in general.  ... 
arXiv:1704.04683v5 fatcat:gn4lj7iet5fnfcpe6uqtqjxqxe

RACE: Large-scale ReAding Comprehension Dataset From Examinations

Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, Eduard Hovy
2017 Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing  
We present RACE, a new dataset for benchmark evaluation of methods in the reading comprehension task.  ...  We hope this new dataset can serve as a valuable resource for research and evaluation in machine comprehension. The dataset is freely available at  ...  This comprehensiveness of topic/style coverage makes RACE a desirable resource for evaluating the reading comprehension ability of machine learning systems in general.  ... 
doi:10.18653/v1/d17-1082 dblp:conf/emnlp/LaiXLYH17 fatcat:tabtuvvt6jburauztydtga4zji

BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels [article]

Yimin Jing, Deyi Xiong, Yan Zhen
2019 arXiv   pre-print
We also observe that answering questions of novels requires reading comprehension skills of coreference resolution, multi-sentence reasoning, and understanding of implicit causality, etc.  ...  This paper presents BiPaR, a bilingual parallel novel-style machine reading comprehension (MRC) dataset, developed to support multilingual and cross-lingual reading comprehension.  ...  Acknowledgments The present research was supported by the National Natural Science Foundation of China (Grant No. 61622209).  ... 
arXiv:1910.05040v1 fatcat:owk5qfc67ne73mjx5quy5que5y

Situation and Behavior Understanding by Trope Detection on Films [article]

Chen-Hsi Chang, Hung-Ting Su, Jui-heng Hsu, Yu-Siang Wang, Yu-Cheng Chang, Zhe Yu Liu, Ya-Liang Chang, Wen-Feng Cheng, Ke-Jyun Wang, Winston H. Hsu
2021 arXiv   pre-print
We present a multi-stream comprehension network (MulCom) leveraging multi-level attention of words, sentences, and role relations.  ...  Existing machine comprehension datasets assume sentence-level input, lack of casual or motivational inferences, or could be answered with question-answer bias.  ...  We also thank Igor Morawski and Yun-Hsuan Liu for participating the human evaluation.  ... 
arXiv:2101.07632v2 fatcat:tjwm6t2sgnfbnh5kbxhvlzjdtq

A Multi-answer Multi-task Framework for Real-world Machine Reading Comprehension

Jiahua Liu, Wan Wei, Maosong Sun, Hao Chen, Yantao Du, Dekang Lin
2018 Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing  
The task of machine reading comprehension (MRC) has evolved from answering simple questions from well-edited text to answering real questions from users out of web data.  ...  Minimum Risk Training is applied to solve the multi-occurrence problem of a single answer.  ...  multi-answer A reading comprehension model is typically trained as an extractor of an answer span from a candidate passage.  ... 
doi:10.18653/v1/d18-1235 dblp:conf/emnlp/LiuWSCDL18 fatcat:6ssnwqnwczck7ow2a3qsulqeey
« Previous Showing results 1 — 15 out of 93,321 results