Filters








110,162 Hits in 3.2 sec

Question-answering by predictive annotation

John Prager, Eric Brown, Anni Coden, Dragomir Radev
2000 Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '00  
We present a new technique for question answering called Predictive Annotation. Predictive Annotation identifies potential answers to questions in text, annotates them accordingly and indexes them.  ...  This technique, along with a complementary analysis of questions, passage-level ranking and answer selection, produces a system effective at answering natural-language fact-seeking questions posed against  ...  Dealing with How and Why How and Why questions are difficult for all questionanswering systems; in particular, Predictive Annotation is by design primarily for fact-seeking questions rather than those  ... 
doi:10.1145/345508.345574 dblp:conf/sigir/PragerBCR00 fatcat:ar4gxpsigfel7hjhhdm25op2de

ParaShoot: A Hebrew Question Answering Dataset [article]

Omri Keren, Omer Levy
2021 arXiv   pre-print
The dataset follows the format and crowdsourcing methodology of SQuAD, and contains approximately 3000 annotated examples, similar to other question-answering datasets in low-resource languages.  ...  In this work, we present ParaShoot, the first question answering dataset in modern Hebrew.  ...  Acknowledgements This work was supported by the Tel Aviv University Data Science Center, Len Blavatnik and the Blavatnik Family foundation, the Alon Scholarship, Intel Corporation, and the Yandex Initiative  ... 
arXiv:2109.11314v1 fatcat:tvzivzosmfblheaww3gobeu4aq

Ditch the Gold Standard: Re-evaluating Conversational Question Answering [article]

Huihan Li, Tianyu Gao, Manan Goenka, Danqi Chen
2022 arXiv   pre-print
Conversational question answering aims to provide natural-language answers to users in information-seeking conversations.  ...  We further investigate how to improve automatic evaluations, and propose a question rewriting mechanism based on predicted history, which better correlates with human judgments.  ...  This research is supported by a Graduate Fellowship at Princeton University and the James Mi *91 Research Innovation Fund for Data Science.  ... 
arXiv:2112.08812v2 fatcat:g7soz4ai6veyrjifukjh7yjaae

Question-Answer Driven Semantic Role Labeling: Using Natural Language to Annotate Natural Language

Luheng He, Mike Lewis, Luke Zettlemoyer
2015 Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing  
It also allows for scalable data collection by annotators with very little training and no linguistic expertise.  ...  We gather data in two domains, newswire text and Wikipedia articles, and introduce simple classifierbased models for predicting which questions to ask and what their answers should be.  ...  We would also like to thank our freelance workers on oDesk/Upwork for their annotation and the anonymous reviewers for their valuable feedback.  ... 
doi:10.18653/v1/d15-1076 dblp:conf/emnlp/HeLZ15 fatcat:z7u2wookb5ayflqih6pix3epiq

An entropy clustering approach for assessing visual question difficulty

Kento Terao, Toru Tamaki, Bisser Raytchev, Kazufumi Kaneda, Shin'Ichi Satoh
2020 IEEE Access  
We propose to cluster the entropy values of the predicted answer distributions obtained by three different models: a baseline method that takes as input images and questions, and two variants that take  ...  We propose a novel approach to identify the difficulty of visual questions for Visual Question Answering (VQA) without direct supervision or annotations to the difficulty.  ...  Histograms of visual questions (that are correctly answered by models) with numbers of unique answers of ground truth annotations, and max overlap of predicted answers by 9 models.  ... 
doi:10.1109/access.2020.3022063 fatcat:rt2k3hj4kngolfcsmxrlhq5c7e

TED-Q: TED Talks and the Questions they Evoke

Matthijs Westera, Laia Mayol, Hannah Rohde
2020 International Conference on Language Resources and Evaluation  
We present a new dataset of TED-talks annotated with the questions they evoke and, where available, the answers to these questions.  ...  question answering).  ...  This work was supported in part by a Leverhulme Trust Prize in Languages and Literatures to H. Rohde.  ... 
dblp:conf/lrec/WesteraMR20 fatcat:bkk2w2nz2zf5jgai2jeyi45bhq

Which visual questions are difficult to answer? Analysis with Entropy of Answer Distributions [article]

Kento Terao, Toru Tamaki, Bisser Raytchev, Kazufumi Kaneda, Shun'ichi Satoh
2020 arXiv   pre-print
We propose to cluster the entropy values of the predicted answer distributions obtained by three different models: a baseline method that takes as input images and questions, and two variants that take  ...  We propose a novel approach to identify the difficulty of visual questions for Visual Question Answering (VQA) without direct supervision or annotations to the difficulty.  ...  This work was supported by JSPS KAKENHI grant number JP16H06540.  ... 
arXiv:2004.05595v1 fatcat:ag4uhdfyrrfvxohlrvc7sleqne

ChiMed: A Chinese Medical Corpus for Question Answering

Yuanhe Tian, Weicheng Ma, Fei Xia, Yan Song
2019 Proceedings of the 18th BioNLP Workshop and Shared Task  
Question answering (QA) is a challenging task in natural language processing (NLP), especially when it is applied to specific domains.  ...  In this study, we first collect a large-scale Chinese medical QA corpus called ChiMed; second we annotate a small fraction of the corpus to check the quality of the answers; third, we extract two datasets  ...  In the forum, the questions are asked by web users and all the answers are provided by accredited physicians.  ... 
doi:10.18653/v1/w19-5027 dblp:conf/bionlp/TianMXS19 fatcat:2jvznf4znjhq3gwbnykmblrt4u

Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval [article]

Akari Asai, Eunsol Choi
2021 arXiv   pre-print
When provided with a gold paragraph and knowing when to abstain from answering, existing models easily outperform a human annotator. However, predicting answerability itself remains challenging.  ...  We manually annotate 800 unanswerable examples across six languages on what makes them challenging to answer.  ...  We thank Vitaly Nikolaev for helping with the Russian data annotation. We also thank the authors of RikiNet and ETC for their cooperation on analyzing their system outputs.  ... 
arXiv:2010.11915v2 fatcat:sx4cqkvktfc6zpaispn4reynfe

SQuAD2-CR: Semi-supervised Annotation for Cause and Rationales for Unanswerability in SQuAD 2.0

Gyeongbok Lee, Seung-won Hwang, Hyunsouk Cho
2020 International Conference on Language Resources and Evaluation  
However, despite the super-human accuracy of existing models on such datasets, it is still unclear how the model predicts the answerability of the question, potentially due to the absence of a shared annotation  ...  Specifically, we annotate (1) explanation on why the most plausible answer span cannot be the answer and (2) which part of the question causes unanswerability.  ...  Rationales annotates fine-grained reason as red why the plausible answer cannot be entailed by the question. nale) annotations in our dataset.  ... 
dblp:conf/lrec/LeeHC20 fatcat:3cqh4siesjb5bkwj6nsxuxynvi

AmbigQA: Answering Ambiguous Open-domain Questions [article]

Sewon Min, Julian Michael, Hannaneh Hajishirzi, Luke Zettlemoyer
2020 arXiv   pre-print
Ambiguity is inherent to open-domain question answering; especially when exploring new topics, it can be difficult to ask questions that have a single, unambiguous answer.  ...  In this paper, we introduce AmbigQA, a new open-domain question answering task which involves finding every plausible answer, and then rewriting the question for each one to resolve the ambiguity.  ...  Acknowledgments This research was supported by ONR N00014-18-1-2826, DARPA N66001-19-2-403, the NSF (IIS-1252835, IIS-1562364), an Allen Distinguished Investigator Award, and the Sloan Fellowship.  ... 
arXiv:2004.10645v2 fatcat:6okxsochibdhjdf5o5u6uagglu

Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluation [article]

Jannis Bulian, Christian Buck, Wojciech Gajewski, Benjamin Boerschinger, Tal Schuster
2022 arXiv   pre-print
The predictions of question answering (QA) systems are typically evaluated against manually annotated finite sets of one or more answers.  ...  Finally, we also demonstrate the practical utility of AE and BEM on the concrete application of minimal accurate prediction sets, reducing the number of required answers by up to 2.6 times.  ...  Acknowledgments We thank Adam Fisch for helpful feedback on the conformal prediction experiments, and Costanza Conforti for helping us to add the dataset to TFDS.  ... 
arXiv:2202.07654v1 fatcat:gzgobwedzbfilhhpg5zhu2fdxa

ExpMRC: Explainability Evaluation for Machine Reading Comprehension [article]

Yiming Cui, Ting Liu, Wanxiang Che, Zhigang Chen, Shijin Wang
2021 arXiv   pre-print
However, it is necessary to provide both answer prediction and its explanation to further improve the MRC system's reliability, especially for real-life applications.  ...  The MRC systems are required to give not only the correct answer but also its explanation.  ...  Finally, they should verify that we can pick out the correct answer by only reading the evidence and question to ensure that the annotation is valid.  ... 
arXiv:2105.04126v1 fatcat:doqicprlazhvzllafhpurcitgu

A BERT Baseline for the Natural Questions [article]

Chris Alberti, Kenton Lee, Michael Collins
2019 arXiv   pre-print
Our model is based on BERT and reduces the gap between the model F1 scores reported in the original dataset paper and the human upper bound by 30% and 50% relative for the long and short answer tasks respectively  ...  This technical note describes a new baseline for the Natural Questions.  ...  If the long answer annotation is non-empty, but the short answer annotation is empty, then the annotated passage answers the question but no explicit short answer could be found.  ... 
arXiv:1901.08634v3 fatcat:lrzmoqpkmbdopckwghhpe7y77u

Calibrating Trust of Multi-Hop Question Answering Systems with Decompositional Probes [article]

Kaige Xie, Sarah Wiegreffe, Mark Riedl
2022 arXiv   pre-print
Through human participant studies, we verify that exposing the decomposition probes and answers to the probes to users can increase their ability to predict system performance on a question instance basis  ...  Recent work in multi-hop QA has shown that performance can be boosted by first decomposing the questions into simpler, single-hop questions.  ...  Annotator performance metrics at predicting answer correctness, averaged across all 30 participants, are presented in Table 6 , along with annotator performance on the same subset given SILVER sub-questions  ... 
arXiv:2204.07693v1 fatcat:3kukcifhhvcehm6mxeir6ze454
« Previous Showing results 1 — 15 out of 110,162 results