Filters








44 Hits in 6.6 sec

Robustness Gym: Unifying the NLP Evaluation Landscape [article]

Karan Goel, Nazneen Rajani, Jesse Vig, Samson Tan, Jason Wu, Stephan Zheng, Caiming Xiong, Mohit Bansal, Christopher Ré
2021 arXiv   pre-print
In this work, we identify challenges with evaluating NLP systems and propose a solution in the form of Robustness Gym (RG), a simple and extensible evaluation toolkit that unifies 4 standard evaluation  ...  Robustness Gym can be found at https://robustnessgym.com/  ...  KG and NR conceived the idea of Robustness Gym. KG, NR, and JV made significant overall contributions to the toolkit. ST and JW ran initial experiments on some NLP tasks.  ... 
arXiv:2101.04840v1 fatcat:gky3ggyvavgr3atajqjnhgs52i

Robustness Gym: Unifying the NLP Evaluation Landscape

Karan Goel, Nazneen Fatema Rajani, Jesse Vig, Zachary Taschdjian, Mohit Bansal, Christopher Ré
2021 Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations   unpublished
In this work, we identify challenges with evaluating NLP systems and propose a solution in the form of Robustness Gym (RG), 1 a simple and extensible evaluation toolkit that unifies 4 standard evaluation  ...  Robustness Gym is under active development and we welcome feedback & contributions from the community. 2. Transformations. Perturb data to check that the model responds correctly to changes.  ...  In response to these challenges, we introduce Robustness Gym (RG), a simple, extensible and unified toolkit for evaluating robustness and sharing findings ( Figure 1 ).  ... 
doi:10.18653/v1/2021.naacl-demos.6 fatcat:unrw4gmueffrznwvfx7pn4vfem

Thinking Beyond Distributions in Testing Machine Learned Models [article]

Negar Rostamzadeh, Ben Hutchinson, Christina Greer, Vinodkumar Prabhakaran
2021 arXiv   pre-print
While recent work on robustness and fairness testing within the ML community has pointed to the importance of testing against distributional shifts, these efforts also focus on estimating the likelihood  ...  as the training dataset.  ...  Robustness gym: Unifying the NLP evaluation landscape.  ... 
arXiv:2112.03057v1 fatcat:af2ih6jo5vdhrgvouhygmx4wfe

Identifying Adversarial Attacks on Text Classifiers [article]

Zhouhang Xie, Jonathan Brophy, Adam Noack, Wencong You, Kalyani Asthana, Carter Perkins, Sabrina Reis, Sameer Singh, Daniel Lowd
2022 arXiv   pre-print
The landscape of adversarial attacks against text classifiers continues to grow, with new attacks developed every year and many of them available in standard toolkits, such as TextAttack and OpenAttack  ...  In response, there is a growing body of work on robust learning, which reduces vulnerability to these attacks, though sometimes at a high cost in compute time or accuracy.  ...  This work benefited from access to the University of Oregon high performance computer, Talapas.  ... 
arXiv:2201.08555v1 fatcat:bknr7chhaza2bhnrwveufhot2m

NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation [article]

Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann, Aadesh Gupta, Zhenhao Li, Saad Mahamood, Abinaya Mahendiran, Simon Mille, Ashish Srivastava, Samson Tan, Tongshuang Wu, Jascha Sohl-Dickstein (+113 others)
2021 arXiv   pre-print
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on.  ...  The infrastructure, datacards and robustness analysis results are available publicly on the NL-Augmenter repository ().  ...  Robust- semantic role labeling using parameterized neigh- ness Gym: Unifying the NLP evaluation landscape. borhood memory adaptation.  ... 
arXiv:2112.02721v1 fatcat:uqizuxc4wzgxnnfsc6azh6ckpq

A Compression-Based Method for Detecting Anomalies in Textual Data

Gonzalo de la Torre-Abaitua, Luis F. Lago-Fernández, David Arroyo
2021 Entropy  
The results obtained show the validity and flexibility of our approach in different security scenarios with a low configuration burden.  ...  Defence mechanisms are generally articulated around tools that trace and store information in several ways, the simplest one being the generation of plain text files coined as security logs.  ...  now chilling out and cleaning the gym oiss easy!!  ... 
doi:10.3390/e23050618 pmid:34065721 pmcid:PMC8156803 fatcat:uyvkt7ntnnhzfaomroihi35kwy

Reinforcement Learning in Practice: Opportunities and Challenges [article]

Yuxi Li
2022 arXiv   pre-print
Various groups of readers, like researchers, engineers, students, managers, investors, officers, and people wanting to know more about the field, may find the article interesting.  ...  The article is based on both historical and recent research papers, surveys, tutorials, talks, blogs, books, (panel) discussions, and workshops/conferences.  ...  Powell (2019) proposes a unified framework based on stochastic control.  ... 
arXiv:2202.11296v2 fatcat:xdtsmme22rfpfn6rgfotcspnhy

ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound [article]

Yan-Bo Lin, Jie Lei, Mohit Bansal, Gedas Bertasius
2022 arXiv   pre-print
Our method, named ECLIPSE (Efficient CLIP with Sound Encoding), adapts the popular CLIP model to an audiovisual video setting, by adding a unified audiovisual transformer block that captures complementary  ...  cues from the video and audio streams.  ...  While these approaches are effective in NLP, they are still very costly in the video domain due to the high dimensionality of video inputs.  ... 
arXiv:2204.02874v3 fatcat:rxkjg5r22zgxbp24kuvqq2lvfi

Deep Reinforcement Learning: An Overview [article]

Yuxi Li
2018 arXiv   pre-print
Hu et al. (2017) unified GANs and Variational Autoencoders (VAEs).  ...  Its open source based on OpenAI Gym is available at https://github.com/siemens/industrialbenchmark.  ... 
arXiv:1701.07274v6 fatcat:x2es3yf3crhqblbbskhxelxf2q

Deep Reinforcement Learning [article]

Yuxi Li
2018 arXiv   pre-print
After that, we discuss RL applications, including games, robotics, natural language processing (NLP), computer vision, finance, business management, healthcare, education, energy, transportation, computer  ...  Lanctot et al. (2017) observe that independent RL, in which each agent learns by interacting with the environment, oblivious to other agents, can overfit the learned policies to other agents' policies  ...  The authors present an implementation with centralized training for decentralized execution, as discussed below. The authors experiment with grid world coordination, a partially observable game,  ... 
arXiv:1810.06339v1 fatcat:kp7atz5pdbeqta352e6b3nmuhy

Creativity in Motion: Examining the Creative Potential System and Enriched Movement Activities as a Way to Ignite It

Veronique Richard, Darren Holder, John Cairney
2021 Frontiers in Psychology  
To address this limitation, the first goal of this paper is to review the creativity science literature to identify the elements that underpin the realization of an individual's creative potential.  ...  The summary of the literature is presented using a framework which highlights the interactions between environmental elements (i.e., cultural values, social interactions, and material world) and actors  ...  The authors also want to thank Makenzie Thomas for her help in the design of the Figures.  ... 
doi:10.3389/fpsyg.2021.690710 pmid:34659006 pmcid:PMC8514639 fatcat:nltotzdobjflrhftkongokrprq

Explainable AI: A Review of Machine Learning Interpretability Methods

Pantelis Linardatos, Vasilis Papastefanopoulos, Sotiris Kotsiantis
2020 Entropy  
, ultimately, the way that they come to decisions.  ...  As a result, scientific interest in the field of Explainable Artificial Intelligence (XAI), a field that is concerned with the development of new methods that explain and interpret machine learning models  ...  and robust than the existing methods.  ... 
doi:10.3390/e23010018 pmid:33375658 pmcid:PMC7824368 fatcat:gv42gcovm5cxzl2kmdsluiegdi

Deep Learning in Mobile and Wireless Networking: A Survey [article]

Chaoyun Zhang, Paul Patras, Hamed Haddadi
2019 arXiv   pre-print
In this paper we bridge the gap between deep learning and mobile and wireless networking research, by presenting a comprehensive survey of the crossovers between the two areas.  ...  The rapid uptake of mobile devices and the rising popularity of mobile applications and services pose unprecedented demands on mobile and wireless networking infrastructure.  ...  However, at the moment, the centralized approach dominates the WSN data analysis landscape.  ... 
arXiv:1803.04311v3 fatcat:awuvyviarvbr5kd5ilqndpfsde

Deep Learning in Mobile and Wireless Networking: A Survey

Chaoyun Zhang, Paul Patras, Hamed Haddadi
2019 IEEE Communications Surveys and Tutorials  
In this paper we bridge the gap between deep learning and mobile and wireless networking research, by presenting a comprehensive survey of the crossovers between the two areas.  ...  The rapid uptake of mobile devices and the rising popularity of mobile applications and services pose unprecedented demands on mobile and wireless networking infrastructure.  ...  However, at the moment, the centralized approach dominates the WSN data analysis landscape.  ... 
doi:10.1109/comst.2019.2904897 fatcat:xmmrndjbsfdetpa5ef5e3v4xda

Opportunities and Challenges of Automatic Speech Recognition Systems for Low-Resource Language Speakers

Thomas Reitmaier, Electra Wallington, Dani Kalarikalayil Raju, Ondrej Klejch, Jennifer Pearson, Matt Jones, Peter Bell, Simon Robinson
2022 CHI Conference on Human Factors in Computing Systems  
Yiza nemifuno xa ubuya Transcription: isiXhosa Figure 1: Artist's representation of the Automatic Speech Recognition systems we developed and feld-tested in partnership with two communities in South Africa  ...  ACKNOWLEDGMENTS We would like to thank Minah and Manik as well as the workshop participants in Langa & Dharavi for their contribution to this work.  ...  We had initially hoped that the Langa workshops would provide a rich 'in-context' data-set that we could utilise as a robust test data-set to evaluate against.  ... 
doi:10.1145/3491102.3517639 fatcat:2fz45jcstfadrblp3mi3aczcmi
« Previous Showing results 1 — 15 out of 44 results