Filters








123,251 Hits in 5.8 sec

Transformers as Soft Reasoners over Language

Peter Clark, Oyvind Tafjord, Kyle Richardson
2020 Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence  
These findings are significant as it suggests a new role for transformers, namely as limited "soft theorem provers" operating over explicit theories in language.  ...  We train transformers to reason (or emulate reasoning) over these sentences using synthetically generated data.  ...  For languages such as English and Chinese, there exist manually labeled datasets on a variety of tasks, trained over which neural models for NLP can rival human performance.  ... 
doi:10.24963/ijcai.2020/533 dblp:conf/ijcai/QinN0C20 fatcat:q7eyzc2nhfdrvjwwbnza7f6wta

Transformers as Soft Reasoners over Language [article]

Peter Clark, Oyvind Tafjord, Kyle Richardson
2020 arXiv   pre-print
These findings are significant as it suggests a new role for transformers, namely as limited "soft theorem provers" operating over explicit theories in language.  ...  We train transformers to reason (or emulate reasoning) over these sentences using synthetically generated data.  ...  The result suggests a new role for transformers, namely as a kind of limited "soft theorem prover" over language ( Figure 2 ).  ... 
arXiv:2002.05867v2 fatcat:xzensbe54rh2xak4cajgmtufcu

Weak Supervision helps Emergence of Word-Object Alignment and improves Vision-Language Tasks [article]

Corentin Kervadec, Grigory Antipov, Moez Baccouche, Christian Wolf
2019 arXiv   pre-print
The large adoption of the self-attention (i.e. transformer model) and BERT-like training principles has recently resulted in a number of high performing models on a large panoply of vision-and-language  ...  In particular , we integrate an object-word alignment loss into SOTA vision-language reasoning models and evaluate it on two tasks VQA and Language-driven Comparison of Images.  ...  scores are normalized over the objects such as: n objects j A * ij = 1 (18) Hence the ground truth soft alignment score A * i of word si is a probability distribution over the set of visual objects detected  ... 
arXiv:1912.03063v1 fatcat:ete5vlbecfh67e324zfeo7qahu

Semantic-Based Development of Service-Oriented Systems [chapter]

Martin Wirsing, Allan Clark, Stephen Gilmore, Matthias Hölzl, Alexander Knapp, Nora Koch, Andreas Schroeder
2006 Lecture Notes in Computer Science  
This includes service-oriented extensions to the UML, a mathematical basis formed by a family of process calculi, a language for expressing context-dependent soft constraints and preferences, qualitative  ...  and quantitative analysis methods, and model transformations from UML to process calculi.  ...  Soft constraints are a promising way to specify and implement reasoning mechanisms. In the case study, soft constraints are used for service selection only.  ... 
doi:10.1007/11888116_3 fatcat:lkiwczznjzfbrcy5fcgifbyeaq

Decision Procedures in Soft, Hard and Bio-ware - Follow Up (Dagstuhl Seminar 11272)

Nikolaj Bjorner, Robert Nieuwenhuis, Helmut Veith, Andrei Voronkov, Marc Herbstritt
2011 Dagstuhl Reports  
This report documents the program and the outcomes of Dagstuhl Seminar 11272 Decision Procedures in Soft, Hard and Bio-ware (Follow Up).  ...  It was held as a follow-on for a seminar 10161, of the same title, that took place in late April 2010 during the initial eruption of Eyjafjallajökull.  ...  Hampi constraints may contain context-free-language definitions, regular-language definitions and operations, the membership predicate and equations over string terms (word equations).  ... 
doi:10.4230/dagrep.1.7.23 dblp:journals/dagstuhl-reports/BjornerNVV11 fatcat:wfkdnqhh6nbdbf5jgtirejs44u

How to Design Sample and Computationally Efficient VQA Models [article]

Karan Samel, Zelin Zhao, Binghong Chen, Kuan Wang, Robin Luo, Le Song
2021 arXiv   pre-print
In multi-modal reasoning tasks, such as visual question answering (VQA), there have been many modeling and training paradigms tested.  ...  We extend existing models to leverage these soft programs and scene graphs to train on question answer pairs in an end-to-end manner.  ...  Each method type handles the vision, language, reasoning and training in different fashions. LXMERT. Reasoning over images based on questions is carried out via Transformer like deep architecture.  ... 
arXiv:2103.11537v1 fatcat:6qngtsgicfcsnj6ishngrdaufe

Distilled Dual-Encoder Model for Vision-Language Understanding [article]

Zekun Wang, Wenhui Wang, Haichao Zhu, Ming Liu, Bing Qin, Furu Wei
2021 arXiv   pre-print
We propose a cross-modal attention distillation framework to train a dual-encoder model for vision-language understanding tasks, such as visual reasoning and visual question answering.  ...  However, the shallow interaction module used in dual-encoder models is insufficient to handle complex vision-language understanding tasks.  ...  Vision-Language Understanding For visionlanguage understanding tasks such as visual reasoning and VQA, we fine-tune the student model with cross-modal attention distillation and soft labels losses.  ... 
arXiv:2112.08723v1 fatcat:lkmjcqn4q5fsjextn4fy37lj2a

Evaluating NLP Systems On a Novel Cloze Task: Judging the Plausibility of Possible Fillers in Instructional Texts [article]

Zizhao Hu, Ravikiran Chanumolu, Xingyu Lin, Nayela Ayaz, Vincent Chi
2021 arXiv   pre-print
Cloze task is a widely used task to evaluate an NLP system's language understanding ability.  ...  Soft-max: Add a soft-max layer to the MLM and use the soft-max probability as scores. (10) BERT + Naive Bayes Classifier iii.  ...  Lastly, we use an Electra model as the transformer encoder as it is faster to train.  ... 
arXiv:2112.01867v1 fatcat:t6vxakipz5bllc5z5eoqeknbke

PhD Abstracts

GRAHAM HUTTON
2020 Journal of functional programming  
As a service to the community, twice per year the Journal of Functional Programming publishes the abstracts from PhD dissertations completed during the previous year.  ...  Formalization of Transform Methods Using Higher-Order-Logic Theorem Proving Transform methods, such as the Laplace and the Fourier transforms, are widely used for analyzing the differential equations modeling  ...  Erlang is a message-passing concurrent, functional programming language based on the actor model. These and other features make it especially appropriate for distributed, soft real-time applications.  ... 
doi:10.1017/s0956796820000167 fatcat:dxjjb46ngvbc5pteeekly6rqem

Transformations in the Translation of Hard and Soft News Reports into Arabic

2022 International Journal of Arabic-English Studies  
This study examines local transformations as a translation strategy of the news report genre from English into Arabic.  ...  Results show that deletion is by far the most frequently used transformation in both soft and hard news.  ...  Over 70% of the headlines modified in both hard and soft news came from BBC texts.  ... 
doi:10.33806/ijaes2000.22.2.3 fatcat:jqmmagwsjjcjbi2gfydbs6fsmq

Constrained physical design tuning

Nicolas Bruno, Surajit Chaudhuri
2008 Proceedings of the VLDB Endowment  
We build upon an existing transformation-based framework to effectively incorporate constraints in the search space.  ...  To overcome this limitation, we introduce a constraint language that is simple yet powerful enough to express many important scenarios.  ...  Our language supports a rich set of functions over these data types.  ... 
doi:10.14778/1453856.1453863 fatcat:sokpw77hqzet3kvfttchyvy5e4

Constrained physical design tuning

Nicolas Bruno, Surajit Chaudhuri
2009 The VLDB journal  
We build upon an existing transformation-based framework to effectively incorporate constraints in the search space.  ...  To overcome this limitation, we introduce a constraint language that is simple yet powerful enough to express many important scenarios.  ...  Our language supports a rich set of functions over these data types.  ... 
doi:10.1007/s00778-009-0154-1 fatcat:chrfjjzxtndvjoytir3hb6bldi

Logically Consistent Adversarial Attacks for Soft Theorem Provers [article]

Alexander Gaskell, Yishu Miao, Lucia Specia, Francesca Toni
2022 arXiv   pre-print
Recent efforts within the AI community have yielded impressive results towards "soft theorem proving" over natural language sentences using language models.  ...  We propose a novel, generative adversarial framework for probing and improving these models' reasoning capabilities.  ...  They demonstrate that Transformers can act as "soft theorem provers" over natural language by solving a logical deduction task.  ... 
arXiv:2205.00047v1 fatcat:jjtwzez75fbzpfkkvtrspyxgdm

Formal Verification of SLA Transformations

Vatche Ishakian, Andrei Lapets, Azer Bestavros, Assaf Kfoury
2011 2011 IEEE World Congress on Services  
This paper demonstrates the promise of such an approach by proposing a type-theoretic framework for the representation and safe transformation of SLAs.  ...  Extensive experimental results support the conclusion that the flexibility afforded by safe SLA transformations has the potential to yield significant savings.  ...  A DOMAIN-SPECIFIC LANGUAGE FOR REAL-TIME SLAS As we hinted earlier, SLA transformations must be based on a given "calculus".  ... 
doi:10.1109/services.2011.16 dblp:conf/services/IshakianLBK11 fatcat:yfmchycb4res3lwee7q64pymfy

Modeling Coreference Relations in Visual Dialog [article]

Mingxiao Li, Marie-Francine Moens
2022 arXiv   pre-print
Experimental results on the VisDial v1.0 dataset shows that our model, which integrates two novel and linguistically inspired soft constraints in a deep transformer neural architecture, obtains new state-of-the-art  ...  In this paper, based on linguistic knowledge and discourse features of human dialog we propose two soft constraints that can improve the model's ability of resolving coreferences in dialog in an unsupervised  ...  As the transformer encoder receives a sequence of tokens as input, to feed both image and language into the model, we simply concatenate the language sequence embedding and image region sequence representation  ... 
arXiv:2203.02986v1 fatcat:ojwsi27afbdpnbv5fdab3ux3ye
« Previous Showing results 1 — 15 out of 123,251 results