402 Hits in 3.4 sec

Why generative phrase models underperform surface heuristics

John DeNero, Dan Gillick, James Zhang, Dan Klein
2006 Proceedings of the Workshop on Statistical Machine Translation - StatMT '06   unpublished
We investigate why weights from generative models underperform heuristic estimates in phrasebased machine translation.  ...  We first propose a simple generative, phrase-based model and verify that its estimates are inferior to those given by surface statistics.  ...  One particularly surprising result is that a simple heuristic extraction algorithm based on surface statistics of a word-aligned training set outperformed the phrase-based generative model proposed by  ... 
doi:10.3115/1654650.1654656 fatcat:qo5f3zarm5bnfajzesp765by6m

Abstractive Summarization: A Survey of the State of the Art

Hui Lin, Vincent Ng
Originally developed for machine translation, neural methods provide a viable framework for obtaining an abstract representation of the meaning of an input text and generating informative, fluent, and  ...  This could explain why extractive summarizers underperform the RLbased abstractive summarizers: RL can address an abstractive summarizer's weakness in making sentence-level decisions.  ...  Surface realization Surface realization aims to combine the candidates selected in content selection using grammatical/syntactic rules to generate a summary.  ... 
doi:10.1609/aaai.v33i01.33019815 fatcat:ufoyvxeuundelfoul5kzbjeutq

DSCo-NG: A Practical Language Modeling Approach for Time Series Classification [chapter]

Daoyuan Li, Tegawendé F. Bissyandé, Jacques Klein, Yves Le Traon
2016 Lecture Notes in Computer Science  
Our previous work, Domain Series Corpus (DSCo), compresses time series into symbolic strings and takes advantage of language modeling techniques to extract from the training set knowledge about different  ...  propose DSCo-NG, which reduces DSCo's complexity and offers an efficient (linear time complexity and low memory footprint), accurate (performance comparable to approaches working on uncompressed data) and generic  ...  This is a major reason why DSCo-NG greatly underperforms 1NN for the WordSynonyms dataset, which has many (25) classes but very few (267) training instances.  ... 
doi:10.1007/978-3-319-46349-0_1 fatcat:r74kcumiifhrlph3rrge4qqt7y

Generative Models can Help Writers without Writing for Them

Kenneth C. Arnold, April M. Volzer, Noah G. Madrid
2021 International Conference on Intelligent User Interfaces  
Computational models of language have the exciting potential to help writers generate and express their ideas.  ...  We present early explorations of two new types of interactions with generative language models; both share the design goal of keeping the writer in ultimate control while providing generative assistance  ...  We are grateful to the contributors to the Huggingface Transformers project, especially the Helsinki NLP group, for making easy-to-use APIs for pre-trained models.  ... 
dblp:conf/iui/ArnoldVM21 fatcat:ydzrb7rrvvaczbd2ajbw26t37a

Statistical Deep Parsing for Spanish: Abridged Version

Luis Chiruzzo
2022 CLEI Electronic Journal  
HPSG is a deep linguistic formalism that combines syntactic and semantic information in the same representation, and is capable of elegantly modeling many linguistic phenomena.  ...  The rather weak statistical model, than only takes in consideration partial information from the supertags, might be one reason why these models are underperforming.  ...  It is possible that many valid trees are generated in the process, so we also need a probabilistic model for determining which of the trees is the most likely one.  ... 
doi:10.19153/cleiej.25.1.2 fatcat:k5jdrcbqcjc5vccmw5x4egckke

Linguistically Annotated Reordering: Evaluation and Analysis

Deyi Xiong, Min Zhang, Aiti Aw, Haizhou Li
2010 Computational Linguistics  
system translations, and summarize syntactic reordering patterns that are captured by reordering models.  ...  Linguistic knowledge plays an important role in phrase movement in statistical machine translation.  ...  If it is not necessary, do the heuristic selection rules impose any bias on the reordering model?  ... 
doi:10.1162/coli_a_00009 fatcat:kzn6ydkakzecparatgjgtcnyfq

A Survey of Word Reordering in Statistical Machine Translation: Computational Models and Language Phenomena

Arianna Bisazza, Marcello Federico
2016 Computational Linguistics  
advanced reordering modeling.  ...  We then question why some approaches are more successful than others in different language pairs.  ...  Model type indicates whether a model is trained in a generative (gener.) or discriminative (discr.) way.  ... 
doi:10.1162/coli_a_00245 fatcat:bgq57yklijgllhn42trnobaexa

Latent Relation Language Models

Hiroaki Hayashi, Zecong Hu, Chenyan Xiong, Graham Neubig
In this paper, we propose Latent Relation Language Models (LRLMs), a class of language models that parameterizes the joint distribution over the words in a document and the entities that occur therein  ...  This model has a number of attractive properties: it not only improves language modeling performance, but is also able to annotate the posterior probability of entity spans for a given text through relations  ...  However, our model often favors word-based generation for common phrases even if related entities exist.  ... 
doi:10.1609/aaai.v34i05.6298 fatcat:zkcvm3b7tnarng7dy4bghyhyim

NegatER: Unsupervised Discovery of Negatives in Commonsense Knowledge Bases [article]

Tara Safavi, Jing Zhu, Danai Koutra
2021 arXiv   pre-print
As a first step toward the latter, this paper proposes NegatER, a framework that ranks potential negatives in commonsense KBs using a contextual language model (LM).  ...  , coherent, and informative -- leading to statistically significant accuracy improvements in a challenging KB completion task and confirming that the positive knowledge in LMs can be "re-purposed" to generate  ...  To make COMET generate negatives, we prepend a "not" token to each positive head phrase X + h and generate 10 tail phrases X COMET t for the modified head/relation prefix using beam search.  ... 
arXiv:2011.07497v2 fatcat:g6evjmoeu5cvdgw2m6rnu636bu

Reconnecting interpretation to reasoning through individual differences

Keith Stenning, Richard Cox
2006 Quarterly Journal of Experimental Psychology  
Grice's theory taken as a broad framework for credulous discourse processing in which participants construct speakers' "intended models" of discourses can reconcile these results, purchasing continuity  ...  We conclude that most participants do not understand deductive tasks as experimenters intend, and just as there is no single logical model of reasoning, so there is no reason to expect a single "fundamental  ...  In the exceptional problem where the source identified by the heuristic cannot be so used, they underperform their peers on the canonically ordered problem but outperform them on the noncanonical member  ... 
doi:10.1080/17470210500198759 pmid:16846971 fatcat:mgf4beid5bfrlo3gkn56ws6gdq

Getting Past the Language Gap: Innovations in Machine Translation [chapter]

Rodolfo Delmonte
2012 Mobile Speech and Advanced Natural Language Solutions  
Translation Models and the Problem of Overfitting It is possible to distinguish between generative translation models (essentially, the IBM models), and the other half to various discriminative models.  ...  The authors take into consideration the "collocation"-like ability of adjacent words to appear in a phrase. Different phrase segmentation will generate different translation results.  ...  The model is also capable of estimating phrase correspondences automatically without heuristic rules.  ... 
doi:10.1007/978-1-4614-6018-3_6 fatcat:2njkc6meabhaxosl4wircumfjm

Précis of Simple heuristics that make us smart

Peter M. Todd, Gerd Gigerenzer
2000 Behavioral and Brain Sciences  
These simple heuristics perform comparably to more complex algorithms, particularly when generalizing to new data -that is, simplicity leads to robustness.  ...  decision making for choice, elimination models for categorization, and satisficing heuristics for sequential search.  ...  Why and when do simple heuristics work?  ... 
doi:10.1017/s0140525x00003447 fatcat:43gpji75ifbv3jej4n2xef25vi

Précis of Simple heuristics that make us smart

P M Todd, G Gigerenzer
2000 Behavioral and Brain Sciences  
decision making for choice, elimination models for categorization, and satisficing heuristics for sequential search.  ...  These simple heuristics perform comparably to more complex algorithms, particularly when generalizing to new data--that is, simplicity leads to robustness.  ...  Even in the cases they describe, the models often seriously underperform other models such as a neural network (Table 11-1 ).  ... 
pmid:11301545 fatcat:2rcestruqzeurjcgbvj2akmfbm

Unsupervised Sub-tree Alignment for Tree-to-Tree Translation

T. Xiao, J. Zhu
2013 The Journal of Artificial Intelligence Research  
Unlike previous work, we do not resort to surface heuristics or expensive annotated data, but instead derive an unsupervised model to infer the syntactic correspondence between two languages.  ...  With tree binarization and fuzzy decoding, it even outperforms a state-of-the-art hierarchical phrase-based system.  ...  We attribute this to the better use of syntactic information on both language sides in our model, which are generally ignored in traditional models based on surface heuristics and word alignments.  ... 
doi:10.1613/jair.4033 fatcat:tvkuj36omvdivkbstmnmdcxc4u

The NarrativeQA Reading Comprehension Challenge [article]

Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, Edward Grefenstette
2017 arXiv   pre-print
We show that although humans solve the tasks easily, standard RC models struggle on the tasks presented here. We provide an analysis of the dataset and the challenges it presents.  ...  The data is human generated, and the answers can be phrases or sentences.  ...  While the answers are multiword phrases, the spans are generally short and rarely cross sentence boundaries.  ... 
arXiv:1712.07040v1 fatcat:4vjf4yx6f5gadjpynnxb2mrp5y
« Previous Showing results 1 — 15 out of 402 results