Filters








11 Hits in 2.5 sec

CLOSURE: Assessing Systematic Generalization of CLEVR Models [article]

Dzmitry Bahdanau, Harm de Vries, Timothy J. O'Donnell, Shikhar Murty, Philippe Beaudoin, Yoshua Bengio, Aaron Courville
2020 arXiv   pre-print
Our experiments on the thereby constructed CLOSURE benchmark show that state-of-the-art models often do not exhibit systematicity after being trained on CLEVR.  ...  In this work, we study how systematic the generalization of such models is, that is to which extent they are capable of handling novel combinations of known linguistic constructs.  ...  Zero-shot Generalization In our first set of experiments, we assess zeroshot systematic generalization of models trained on CLEVR by measuring their performance on the CLOSURE tests.  ... 
arXiv:1912.05783v2 fatcat:ayy5otncezfuhmgv23zakq7ham

Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering [article]

Ben Bogin, Sanjay Subramanian, Matt Gardner, Jonathan Berant
2020 arXiv   pre-print
as well as on CLOSURE, a dataset that focuses on systematic generalization for grounded question answering.  ...  However, state-of-the-art models in grounded question answering often do not explicitly perform decomposition, leading to difficulties in generalization to out-of-distribution examples.  ...  This work was completed in partial fulfillment for the Ph.D degree of Ben Bogin.  ... 
arXiv:2007.00266v3 fatcat:vnxuxacz7fhljfab5kcacsgvem

Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering

Ben Bogin, Sanjay Subramanian, Matt Gardner, Jonathan Berant
2021 Transactions of the Association for Computational Linguistics  
However, state-of-the-art models in grounded question answering often do not explicitly perform decomposition, leading to difficulties in generalization to out-of-distribution examples.  ...  as well as on C losure, a dataset that focuses on systematic generalization for grounded question answering.  ...  This work was completed in partial fulfillment for the Ph.D. degree of Ben Bogin.  ... 
doi:10.1162/tacl_a_00361 fatcat:dnwv7pl6hrdbzkf4tutdf6ixqy

CLEVR Parser: A Graph Parser Library for Geometric Learning on Language Grounded Image Scenes [article]

Raeid Saqur, Ameet Deshpande
2020 arXiv   pre-print
We present a graph parser library for CLEVR, that provides functionalities for object-centric attributes and relationships extraction, and construction of structural graph representations for dual modalities  ...  The CLEVR dataset has been used extensively in language grounded visual reasoning in Machine Learning (ML) and Natural Language Processing (NLP) domains.  ...  International Journal of Computer Vision, 123(1):32-73. Dzmitry Bahdanau, Philippe Beaudoin, and Aaron Courville. 2019. CLOSURE : Assessing Systematic Generalization of CLEVR Models.  ... 
arXiv:2009.09154v2 fatcat:us54vlnxtzgf3bf4ggnvbldneu

Learning Natural Language Generation from Scratch [article]

Alice Martin Donati, Guillaume Quispe, Charles Ollion, Sylvain Le Corff, Florian Strub, Olivier Pietquin
2021 arXiv   pre-print
AsRL methods unsuccessfully scale to large action spaces, we dynamically truncate the vocabulary spaceusing a generic language model.  ...  To our knowledge, it is the first approach that successfullylearns a language generation policy (almost) from scratch.  ...  Closure: Assessing systematic generalization of clevr models. Visually Grounded Interaction and Lan- guage (ViGIL). Banerjee, S. and Lavie, A. (2005).  ... 
arXiv:2109.09371v1 fatcat:2ks2fmyjz5bgrmzclj5hfb25mq

Transformer Module Networks for Systematic Generalization in Visual Question Answering [article]

Moyuru Yamada, Vanessa D'Amario, Kentaro Takemoto, Xavier Boix, Tomotake Sasaki
2022
TMNs achieve state-of-the-art systematic generalization performance in three VQA datasets, namely, CLEVR-CoGenT, CLOSURE and GQA-SGL, in some cases improving more than 30% over standard Transformers.  ...  However, when we evaluate them on systematic generalization, i.e., handling novel combinations of known concepts, their performance degrades.  ...  Acknowledgements and Disclosure of Funding We would like to thank Pawan Sinha and Tomaso Poggio for warm encouragement and insightful advice.  ... 
doi:10.48550/arxiv.2201.11316 fatcat:7pd5vpopmnfspoea5zzzzqgyjq

ReaSCAN: Compositional Reasoning in Language Grounding [article]

Zhengxuan Wu, Elisa Kreiss, Desmond C. Ong, Christopher Potts
2021 arXiv   pre-print
We assess two models on ReaSCAN: a multi-modal baseline and a state-of-the-art graph convolutional neural model.  ...  This suggests that ReaSCAN can serve as a valuable benchmark for advancing our understanding of models' compositional generalization and reasoning capabilities.  ...  For instance, [31] proposed CLOSURE, a set of unseen testing splits for the CLEVR dataset [10] which contains synthetically generated natural-looking questions about 3D geometric objects.  ... 
arXiv:2109.08994v1 fatcat:n5di3kebxnhdtkivxxjyr6nrl4

Compositional Scene Representation Learning via Reconstruction: A Survey [article]

Jinyang Yuan, Tonglin Chen, Bin Li, Xiangyang Xue
2022 arXiv   pre-print
In this survey, we first outline the current progress on this research topic, including development history and categorizations of existing methods from the perspectives of modeling of visual scenes and  ...  Complex visual scenes are composed of relatively simple visual concepts, and have the property of combinatorial explosion.  ...  research prospects, and future directions include learning with implicit 3D modeling, learning with embodied agents, and continual learning of visual concepts.  ... 
arXiv:2202.07135v2 fatcat:ihyu5as6tjgs5e4lsqvhxvreiu

Mind the Context: The Impact of Contextualization in Neural Module Networks for Grounding Visual Referring Expressions

Arjun Akula, Spandana Gella, Keze Wang, Song-Chun Zhu, Siva Reddy
2021 Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing   unpublished
Clo- SURE, NLVR2 and a new contrast set CC-Ref+ sure: Assessing systematic generalization of clevr demonstrate that our proposed method enhances models.  ...  sure: Assessing systematic generalization of clevr models. ArXiv preprint, abs/1912.05783. Arjun R Akula. 2015.  ... 
doi:10.18653/v1/2021.emnlp-main.516 fatcat:vczu774jjzbovpoksfe7ofdxei

Learning to Recombine and Resample Data for Compositional Generalization [article]

Ekin Akyürek, Afra Feyza Akyürek, Jacob Andreas
2021 arXiv   pre-print
R&R has two components: recombination of original training examples via a prototype-based generative model and resampling of generated examples to encourage extrapolation.  ...  Flexible neural sequence models outperform grammar- and automaton-based counterparts on a variety of tasks.  ...  Closure: Assessing systematic generalization of clevr models. arXiv preprint arXiv:1912.05783, 2019a.  ... 
arXiv:2010.03706v6 fatcat:a5i5yv35qfc5lh3jmy5qdtvxjy

Deep Reinforcement Learning [article]

Yuxi Li
2018 arXiv   pre-print
We start with background of artificial intelligence, machine learning, deep learning, and reinforcement learning (RL), with resources.  ...  Next we discuss RL core elements, including value function, policy, reward, model, exploration vs. exploitation, and representation.  ...  PSRO/DCH generalizes previous algorithms, like independent RL, iterative best response, double oracle, and fictitious play.  ... 
arXiv:1810.06339v1 fatcat:kp7atz5pdbeqta352e6b3nmuhy