Filters








195,117 Hits in 9.9 sec

Performance issues and error analysis in an open-domain question answering system

Dan Moldovan, Marius Paşca, Sanda Harabagiu, Mihai Surdeanu
2003 ACM Transactions on Information Systems  
This paper presents an in-depth analysis of a state-of-the-art Question Answering system.  ...  Several scenarios are examined: (1) the performance of each module in a serial baseline system, (2) the impact of feedbacks and the insertion of a logic prover, and (3) the impact of various retrieval  ...  INTRODUCTION Aiming at returning brief answers in response to natural language questions, open-domain Question Answering (QA) systems represent an advanced application of natural language processing.  ... 
doi:10.1145/763693.763694 fatcat:xvwxby75bfgbrb42c33ufadoay

Performance issues and error analysis in an open-domain Question Answering system

Dan Moldovan, Marius Paşca, Sanda Harabagiu, Mihai Surdeanu
2001 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02  
This paper presents an in-depth analysis of a state-of-the-art Question Answering system.  ...  Several scenarios are examined: (1) the performance of each module in a serial baseline system, (2) the impact of feedbacks and the insertion of a logic prover, and (3) the impact of various retrieval  ...  INTRODUCTION Aiming at returning brief answers in response to natural language questions, open-domain Question Answering (QA) systems represent an advanced application of natural language processing.  ... 
doi:10.3115/1073083.1073091 dblp:conf/acl/MoldovanPHS02 fatcat:yykmw4tjsjbohkocbyzewsrboa

Questions and Answers: Theoretical and Applied Perspectives

Raffaella Bernardi, Bonnie Webber
2007 Journal of Applied Logic  
question analysis, answer extraction and knowledge engineering.  ...  Two of the papers in this special issue describe successful open domain QA system that take advantage of logic based approaches.  ...  Finally, they provide an extensive error analysis explaining the meaning of their results and highlighting issues for further research.  ... 
doi:10.1016/j.jal.2005.12.008 fatcat:rak2bsvqize5jgvhgavvxclxo4

Adapting and evaluating a deep learning language model for clinical why-question answering

Andrew Wen, Mohamed Y Elwazir, Sungrim Moon, Jungwei Fan
2020 JAMIA Open  
Discussion The error analysis suggested that the model did not really perform deep reasoning and that clinical why-QA might warrant more sophisticated solutions.  ...  The evaluation focused on: (1) comparing the merits from different training data and (2) error analysis. Results The best model achieved an accuracy of 0.707 (or 0.760 by partial match).  ...  Tasks for Challenges in NLP for Clinical Data organized by Dr Ozlem Uzuner, i2b2 and SUNY.  ... 
doi:10.1093/jamiaopen/ooz072 pmid:32607483 pmcid:PMC7309262 fatcat:fwqxont7bnckrdgq7vmpmfcsx4

Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset [article]

Xiang Yue, Bernal Jimenez Gutierrez, Huan Sun
2020 arXiv   pre-print
In this paper, we provide an in-depth analysis of this dataset and the clinical reading comprehension (CliniRC) task.  ...  From our qualitative analysis, we find that (i) emrQA answers are often incomplete, and (ii) emrQA questions are often answerable without using domain knowledge.  ...  This research was sponsored in part by the Patient-Centered Outcomes Research Institute Funding ME-2017C1-6413, the Army Research Office under cooperative agreements W911NF-17-1-0412, NSF Grant IIS1815674  ... 
arXiv:2005.00574v1 fatcat:qdylyryzdrabfobkkajdqcnryu

Impact of Question Difficulty on Engagement and Learning [chapter]

Jan Papoušek, Vít Stanislav, Radek Pelánek
2016 Lecture Notes in Computer Science  
We study the impact of question difficulty on learners' engagement and learning using an experiment with an open online educational system for adaptive practice of geography.  ...  These results stress the necessity of careful formalization of goals and optimization criteria of open online education systems.  ...  Discussion We performed an experiment with varied difficulty of items in a widely used open online educational system.  ... 
doi:10.1007/978-3-319-39583-8_28 fatcat:a53wu5rjy5dvvf7w2mpu7rx5za

Comparative Analysis of Neural QA models on SQuAD

Soumya Wadhwa, Khyathi Chandu, Eric Nyberg
2018 Proceedings of the Workshop on Machine Reading for Question Answering  
As a first step towards achieving generalization across multiple domains, we attempt to understand and compare the peculiarities of existing end-to-end neural models on the Stanford Question Answering  ...  The task of Question Answering has gained prominence in the past few decades for testing the ability of machines to understand natural language.  ...  Acknowledgments We would like to thank Chaitanya Malaviya and Abhishek Chinni for their valuable feedback, and the Language Technologies Institute at CMU for the GPU resources used in this work.  ... 
doi:10.18653/v1/w18-2610 dblp:conf/acl/WadhwaCN18 fatcat:m2xdejnjlfdujfykzdblurztqm

MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languages [article]

Akari Asai, Shayne Longpre, Jungo Kasai, Chia-Hsuan Lee, Rui Zhang, Junjie Hu, Ikuya Yamada, Jonathan H. Clark, Eunsol Choi
2022 arXiv   pre-print
We present the results of the Workshop on Multilingual Information Access (MIA) 2022 Shared Task, evaluating cross-lingual open-retrieval question answering (QA) systems in 16 typologically diverse languages  ...  In this task, we adapted two large-scale cross-lingual open-retrieval QA datasets in 14 typologically diverse languages, and newly annotated open-retrieval QA data in 2 underrepresented languages: Tagalog  ...  We thank Maraim Masoud for her help in error analysis.  ... 
arXiv:2207.00758v1 fatcat:upipd6sp5vhcjfmmmjcmpmaqtm

Comparative Analysis of Neural QA models on SQuAD [article]

Soumya Wadhwa and Khyathi Raghavi Chandu and Eric Nyberg
2018 arXiv   pre-print
As a first step towards achieving generalization across multiple domains, we attempt to understand and compare the peculiarities of existing end-to-end neural models on the Stanford Question Answering  ...  The task of Question Answering has gained prominence in the past few decades for testing the ability of machines to understand natural language.  ...  Acknowledgments We would like to thank Chaitanya Malaviya and Abhishek Chinni for their valuable feedback, and the Language Technologies Institute at CMU for the GPU resources used in this work.  ... 
arXiv:1806.06972v1 fatcat:3zcv6ikta5d57dzmfpul22gixa

Adapting and evaluating a deep learning language model for clinical why-question answering [article]

Andrew Wen, Mohamed Y. Elwazir, Sungrim Moon, Jungwei Fan
2019 arXiv   pre-print
Discussion: The error analysis suggested that the model did not really perform deep reasoning and that clinical why-QA might warrant more sophisticated solutions.  ...  Materials and Methods: Bidirectional encoder representations from transformers (BERT) models were trained with varying data sources to perform SQuAD 2.0 style why-question answering (why-QA) on clinical  ...  More importantly, our error analysis helped understand the data issues, the system behavior, and areas to improve on.  ... 
arXiv:1911.05604v1 fatcat:ezcrnepe5bgfncff44a447kury

Robustness Testing of Embedded Software Systems: An Industrial Interview Study

Syed Muhammad Ali Shah, Daniel Sundmark, Birgitta Lindstrom, Sten F. Andler
2016 IEEE Access  
The motivation behind this study is to provide a knowledge base of the state of the practice in robustness testing of embedded software systems and to compare this to the state of the art.  ...  The study is a first step in capturing the state of the practice of robustness testing of embedded software systems. The results can be used by both researchers and practitioners.  ...  The authors are thankful to the entire TOCSYC team for involvement in the early design of this study and providing us feedback on study instruments and data analysis.  ... 
doi:10.1109/access.2016.2544951 fatcat:dpyas22hvjcmjfmrudbz2tpvji

Supervised Hierarchical Classification for Student Answer Scoring [article]

Itziar Aldabe, Oier Lopez de Lacalle, Iñigo Lopez-Gazpio, Montse Maritxalar
2015 arXiv   pre-print
It finally discusses the motivations and fundamentals of such an approach.  ...  This paper describes a hierarchical system that predicts one label at a time for automated student response analysis.  ...  Acknowledgments This project was partially funded by the national program for university professor and philosophy doctor training (FPU13/00501).  ... 
arXiv:1507.03462v1 fatcat:mu3ygdiqr5dcdpaqmw7zeiogwa

Framing QA as Building and Ranking Intersentence Answer Justifications

Peter Jansen, Rebecca Sharp, Mihai Surdeanu, Peter Clark
2017 Computational Linguistics  
We evaluate our method on 1,000 multiple-choice questions from elementary school science exams, and empirically demonstrate that it performs better than several strong baselines, including neural network  ...  We propose a question answering (QA) approach for standardized science exams that both identifies correct answers and produces compelling human-readable justifications for why those answers are correct  ...  This interest has been disclosed to the University of Arizona Institutional Review Committee and is being managed in accordance with its conflict of interest policies.  ... 
doi:10.1162/coli_a_00287 fatcat:kqssixxjpbavdc6nimqknvtn74

The use of C++ exception handling constructs: A comprehensive study

Rodrigo Bonifacio, Fausto Carvalho, Guilherme N. Ramos, Uira Kulesza, Roberta Coelho
2015 2015 IEEE 15th International Working Conference on Source Code Analysis and Manipulation (SCAM)  
In addition, the survey respondents consider that incompatibility with existing C code and libraries, extra performance costs (in terms of response time and size of the compiled code), and lack of expertise  ...  Firstly, we carried out a static analysis investigation to understand how developers employ the exception handling construct of C++, considering 65 open-source systems (which comprise 34 million lines  ...  In addition, we carried out a qualitative text analysis [27] for interpreting SQ7, which is an open-ended question where the respondents could fill in detailed answers regarding their perceptions on  ... 
doi:10.1109/scam.2015.7335398 dblp:conf/scam/BonifacioCRKC15 fatcat:4cqmzupobnhbfikpsdo2jshjmm

A Comparison of Hard Filters and Soft Evidence for Answer Typing in Watson [chapter]

Chris Welty, J. William Murdock, Aditya Kalyanpur, James Fan
2012 Lecture Notes in Computer Science  
Such a strategy is poorly suited to an open domain in which there is an extremely broad range of types of answers, and the most frequently occurring types cover only a small fraction of all answers.  ...  In this paper we present an alternative approach called TyCor, that employs soft filtering of candidates using multiple strategies and sources.  ...  We have presented an approach to handling answer types in open domain question answering systems that is more open and flexible than the commonly used type-andgenerate approach.  ... 
doi:10.1007/978-3-642-35173-0_16 fatcat:pm4xhxgu5ngxpedvejc5qlp7tm
« Previous Showing results 1 — 15 out of 195,117 results