Filters








9 Hits in 5.0 sec

Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling [article]

Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen, Benjamin Van Durme (+2 others)
2019 arXiv   pre-print
language modeling.  ...  Our primary results support the use language modeling, especially when combined with pretraining on additional labeled-data tasks.  ...  This relative weakness on sentence structure is somewhat surprising given the finding in Zhang & Bowman (2018) that language model pretraining is helpful for tasks involving sentence structure.  ... 
arXiv:1812.10860v5 fatcat:74vvlqd6r5gdhekrlge2jg7nri

Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling

Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen (+4 others)
2019 Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics  
language modeling.  ...  Our primary results support the use language modeling, especially when combined with pretraining on additional labeled-data tasks.  ...  This * This paper supercedes "Looking for ELMo's Friends: Sentence-Level Pretraining Beyond Language Modeling", an earlier version of this work by the same authors.  ... 
doi:10.18653/v1/p19-1439 dblp:conf/acl/WangHXPMPKTHYJC19 fatcat:rqhxn6jtgjcepclspzad3odir4

Linguistic Knowledge and Transferability of Contextual Representations

Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, Noah A. Smith
2019 Proceedings of the 2019 Conference of the North  
For any given task, pretraining on a closely related task yields better performance than language model pretraining (which is better on average) when the pretraining dataset is fixed.  ...  However, language model pretraining on more data gives the best results.  ...  We also thank the members of the Noah's ARK group at the University of Washington, the researchers at the Allen Institute for Artificial Intelligence, and the anonymous reviewers for their valuable feedback  ... 
doi:10.18653/v1/n19-1112 dblp:conf/naacl/Liu0BPS19 fatcat:uxpjqb7v3vgtjpxyz3ansjwubm

Linguistic Knowledge and Transferability of Contextual Representations [article]

Nelson F. Liu and Matt Gardner and Yonatan Belinkov and Matthew E. Peters and Noah A. Smith
2019 arXiv   pre-print
For any given task, pretraining on a closely related task yields better performance than language model pretraining (which is better on average) when the pretraining dataset is fixed.  ...  However, language model pretraining on more data gives the best results.  ...  We also thank the members of the Noah's ARK group at the University of Washington, the researchers at the Allen Institute for Artificial Intelligence, and the anonymous reviewers for their valuable feedback  ... 
arXiv:1903.08855v5 fatcat:74y4i5pf3resnj34mlkl2jrium

HellaSwag: Can a Machine Really Finish Your Sentence?

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, Yejin Choi
2019 Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics  
Our construction of HellaSwag, and its resulting di culty, sheds light on the inner workings of deep pretrained models.  ...  Though its questions are trivial for humans (°95% accuracy), state-of-the-art models struggle ( †48%).  ...  Acknowledgments We thank the reviewers, as well as Jesse Thomason, for their helpful feedback. We thank the Mechanical Turk workers for their great work during dataset collection.  ... 
doi:10.18653/v1/p19-1472 dblp:conf/acl/ZellersHBFC19 fatcat:ycefrzvajrd4hponvuapbt6lt4

HellaSwag: Can a Machine Really Finish Your Sentence? [article]

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, Yejin Choi
2019 arXiv   pre-print
Our construction of HellaSwag, and its resulting difficulty, sheds light on the inner workings of deep pretrained models.  ...  Though its questions are trivial for humans (>95% accuracy), state-of-the-art models struggle (<48%).  ...  Acknowledgments We thank the reviewers, as well as Jesse Thomason, for their helpful feedback. We thank the Mechanical Turk workers for their great work during dataset collection.  ... 
arXiv:1905.07830v1 fatcat:swxasusty5drrnn2mair44ntji

Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches [article]

Shane Storks, Qiaozi Gao, Joyce Y. Chai
2020 arXiv   pre-print
In the NLP community, recent years have seen a surge of research activities that address machines' ability to perform deep language understanding which goes beyond what is explicitly stated in text, rather  ...  As these benchmarks become instrumental and a driving force for the NLP research community, this paper aims to provide an overview of recent benchmarks, relevant knowledge resources, and state-of-the-art  ...  Acknowledgements We would like to thank the anonymous reviewers for their greatly helpful comments and suggestions.  ... 
arXiv:1904.01172v3 fatcat:minzpxrrwfebdipu55udwe5dxq

DeepHelp: Deep Learning for Shout Crisis Text Conversations [article]

Daniel Cahn
2021 arXiv   pre-print
model for using imperfect machine learning models to estimate population parameters from a biased training set.  ...  We produce three metrics for conversation success and evaluate the validity and usefulness for each.  ...  Ariele Noble, the Shout coaches and crisis volunteers I spoke with and the entire team at Shout and MHI for making the Shout dataset available for this research project, for providing a constant source  ... 
arXiv:2110.13244v1 fatcat:ka2hq4u7w5de7am666mmkyxgia

Measuring and Comparing Social Bias in Static and Contextual Word Embeddings

Alan Cueva Mora
2022
Word embeddings have been considered one of the biggest breakthroughs of deep learning for natural language processing.  ...  models GloVe and Word2Vec.  ...  I would like to thank Dr Emma Murphy and Dr Luca Longo for helping me with the coordination and the design of the proposal of this research project respectively.  ... 
doi:10.21427/9fea-6f46 fatcat:twsfbuz5bvea5mdh6svs7rvzxi