Filters








3,156,333 Hits in 3.3 sec

Evaluating Information Extraction [chapter]

Andrea Esuli, Fabrizio Sebastiani
2010 Lecture Notes in Computer Science  
The issue of how to experimentally evaluate information extraction (IE) systems has received hardly any satisfactory solution in the literature.  ...  We describe the properties of this models, also by presenting the result of a re-evaluation of the results of the CoNLL'03 and CoNLL'02 Shared Tasks on Named Entity Extraction. 1 Christopher Manning, Hal  ...  , and evaluation campaigns), devotes surprisingly little space to discussing the mathematical measures used in evaluating IE systems; and the same happens for a recent survey on information extraction  ... 
doi:10.1007/978-3-642-15998-5_12 fatcat:cpeh7ompgvfcrlav336ljerfvq

TIPSTER information extraction evaluation

Elaine Marsh
1996 Proceedings of a workshop on held at Baltimore, Maryland October 13-15, 1998 -  
EVALUATION TASKS MUC-7 consisted of six information extraction tasks.  ...  be done to put production-level information extraction systems on users' desks.  ... 
doi:10.3115/1119089.1119125 dblp:conf/tipster/Marsh98 fatcat:bk4osnxt45hxfigbk6glr6uxam

Evaluating an Information Extraction System

W. Lehnert, C. Cardie, D. Fisher, J. McCarthy, E. Riloff, S. Soderland, Steven L. Lytinen
1994 Integrated Computer-Aided Engineering  
by the Foreign Broadcast Information Service.  ...  1 We note that earlier attempts at comparative performance evaluation took place at the MUCK1 and MUCK2 message understanding conferences in 1987 and 1989.  ... 
doi:10.3233/ica-1994-1601 fatcat:wp5nyseokbdrpjmq4omadvuh7m

Evaluating machine learning for information extraction

Neil Ireson, Fabio Ciravegna, Mary Elaine Califf, Dayne Freitag, Nicholas Kushmerick, Alberto Lavelli
2005 Proceedings of the 22nd international conference on Machine learning - ICML '05  
Comparative evaluation of Machine Learning (ML) systems used for Information Extraction (IE) has suffered from various inconsistencies in experimental procedures.  ...  This paper reports on the results of the Pascal Challenge on Evaluating Machine Learning for Information Extraction, which provides a standardised corpus, set of tasks, and evaluation methodology.  ...  Introduction As part of text understanding, Information Extraction (IE) is a long standing goal of Artificial Intelligence (AI).  ... 
doi:10.1145/1102351.1102395 dblp:conf/icml/IresonCCFKL05 fatcat:b763xnjt3fentihsu53e2bbw6i

Holistic query evaluation over information extraction pipelines

Ekaterini Ioannou, Minos Garofalakis
2017 Proceedings of the VLDB Endowment  
Holistic Query Evaluation over Information Extraction Pipelines. PVLDB, 11(2): 217 -229, 2017.  ...  We introduce holistic in-database query processing over information extraction pipelines.  ...  To successfully perform resolution in highly heterogeneous and unstructured data, the Information Extraction (IE) community uses information extraction pipelines.  ... 
doi:10.14778/3149193.3149201 fatcat:64nf75uzincwvd3xrfxgmtqp3i

Tipster/MUC-5 information extraction system evaluation

Beth M. Sundheim
1993 Proceedings of a workshop on held at Fredericksburg, Virginia September 19-23, 1993 -  
Tipster Phase 1 extraction system evaluation s Evaluation Criteria and Metric s In assessing the performance of the information extraction systems, we are interested in knowing the classe s of errors  ...  The final evaluation included not only the Tipster-supported information extraction contractors but thirteen other participants as well.  ... 
doi:10.3115/1119149.1119168 dblp:conf/tipster/Sundheim93a fatcat:qm4mk3a4kzdrxgb6lpabju23u4

Information Extraction from Microblogs

Prashant Bhardwaj, Partha Pakray
2016 Forum for Information Retrieval Evaluation  
The paper attempts to extract information from the database of the tweets collected from Twitter.  ...  The data can be used to extract much meaningful information that could be used to develop technologies for future use.  ...  This paper presents the experiments carried out at National Institute of Technology Agartala as part of the participation in the Forum for Information Retrieval Evaluation (FIRE) 2016 in Information Extraction  ... 
dblp:conf/fire/BhardwajP16 fatcat:psonj7gt5jftxesp5yzxgwxhou

Open Information Extraction on Scientific Text: An Evaluation [article]

Paul Groth and Michael Lauruhn and Antony Scerri and Ron Daniel Jr
2018 arXiv   pre-print
Open Information Extraction (OIE) is the task of the unsupervised creation of structured information from text.  ...  In this article, we evaluate the performance of OIE on scientific texts originating from 10 different disciplines.  ...  been extracted from a sentence; • uninformative extractions that omit critical information from a sentence; In our evaluation, we take a different approach.  ... 
arXiv:1802.05574v2 fatcat:g354xnerxrch7geekdkmuml3rm

Metrics for Evaluation of Ontology-based Information Extraction

Diana Maynard, Wim Peters, Yaoyong Li
2006 The Web Conference  
mechanism, and comparison of different evaluation metrics.  ...  This paper discusses existing evaluation metrics, and proposes a new method for evaluating the ontology population task, which is general enough to be used in a variety of situations, yet more precise  ...  information extraction systems.  ... 
dblp:conf/www/MaynardPL06 fatcat:vc4kqfrpsvdpfpmi66txmcvrcy

Evaluating the Quality of Automatically Extracted Synonymy Information

A. Kumaran, Ranbeer Makin, Vijay Pattisapu, Shaik Sharif, Lucy Vanderwende
2008 Journal for Language Technology and Computational Linguistics  
Evaluating the Quality of Automatically Extracted Synonymy Information Automatic extraction of semantic information, if successful, offers to languages with little or poor resources, the prospects of creating  ...  In this paper we explore the automatic extraction of synonymy information from large corpora using two complementary techniques: a generic broad-coverage parser for generation of bits of semantic information  ...  Quantitative Evaluation of Synthesized Synsets We performed LSA on the Brown Corpus (9) to extract a 15-dimensional words space for computing similarity.  ... 
dblp:journals/ldvf/KumaranMPSV08 fatcat:ycdzbwudnzbwbcn7d72lx7yz3a

Key Information Extraction From Documents: Evaluation And Generator [article]

Oliver Bensch, Mirela Popa, Constantin Spille
2021 arXiv   pre-print
An existing information extraction model "Chargrid" (Katti et al., 2019) was reconstructed and the impact of a bounding box regression decoder, as well as the impact of an NLP pre-processing step was evaluated  ...  for information extraction from documents.  ...  Therefore, no benchmark dataset is currently available for comparing or evaluating existing models that extract information from documents.  ... 
arXiv:2106.14624v1 fatcat:k2maqss64jbe7hjnmribmjahqi

Fact distribution in Information Extraction

Mark Stevenson
2007 Language Resources and Evaluation  
Several recent Information Extraction (IE) systems have been restricted to the identification facts which are described within a single sentence.  ...  Therefore systems which are evaluated only on facts described within single sentences are being tested against a limited portion of the relevant information in the text and it is difficult to compare their  ...  Any mistakes are my own. paper.tex; 2/01/2007; 11:09; p.22 Fact Distribution in Information Extraction  ... 
doi:10.1007/s10579-006-9014-4 fatcat:pkx7e3y65jeurglmlcnq5fwkkq

BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation [article]

Kiril Gashteovski, Mingying Yu, Bhushan Kotnis, Carolin Lawrence, Mathias Niepert, Goran Glavaš
2022 arXiv   pre-print
., it takes into account informational equivalence of extractions: our gold standard consists of fact synsets, clusters in which we exhaustively list all acceptable surface forms of the same fact.  ...  Intrinsic evaluations of OIE systems are carried out either manually -- with human evaluators judging the correctness of extractions -- or automatically, on standardized benchmarks.  ...  A.1.4 Attribution Clauses Extractions that indicate attribution of another core piece of information should be placed in separate fact synset, because they indicate a separate piece of information with  ... 
arXiv:2109.06850v2 fatcat:bjm65itajrctzl4vhzzvqgiahe

Real Time Information Extraction from Microblog

Sandip Modha, Chintak Mandalia, Krati Agrawal, Deepali Verma, Prasenjit Majumder
2016 Forum for Information Retrieval Evaluation  
The main objective of the track is to identify Information Retrieval methodologies to retrieve important information from Twitter posted during the disasters.  ...  This paper present the participation of Information Retrieval Lab(IR LAB DA-IICT Gandhinagar) in FIRE 2016 Microblog Track.  ...  INTRODUCTION Social media, like Twitter, is a massive source of real-time information.  ... 
dblp:conf/fire/ModhaMAVM16 fatcat:2kvn2uryyfa6jm4p5dhu37mhkm

Detecting Opinionated Sentences by Extracting Context Information

Xinfan Meng, Houfeng Wang
2008 NTCIR Conference on Evaluation of Information Access Technologies  
In the subtask of opinionated sentence detection, two methods aiming to extract the context information of each sentence are proposed. Maximum Entropy model is used to predict the polarity class.  ...  Conditional Random Field Model Conditional Random Field (CRF) is a sequential labeling model, which is a more aggressive way to extract the context information.  ...  We participated in all 5 evaluation subtasks in both Traditional Chinese side and Simplified Chinese side.  ... 
dblp:conf/ntcir/MengW08 fatcat:xgktdbxdbjc2xndyfnvgkzp56q
« Previous Showing results 1 — 15 out of 3,156,333 results