A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Filters
KAT: A Knowledge Augmented Transformer for Vision-and-Language
[article]
2022
arXiv
pre-print
Preserved Fulltext
Other Versions
knowledge used, and how the reasoning processes over implicit and explicit knowledge should be integrated. ...
Our approach integrates implicit and explicit knowledge in an end to end encoder-decoder architecture, while still jointly reasoning over both knowledge sources during answer generation. ...
We thank Zhe Gan, Zhengyuan Yang, Lijuan Wang from cognition service team of Microsoft for their work and their generous helps and feedback for the project. ...
arXiv:2112.08614v2
fatcat:fvqo2qkodrhdjabphbzrfhljqy
Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
One dataset that is mostly used in evaluating knowledge-based VQA is OK-VQA, but it lacks a gold standard knowledge corpus for retrieval. ...
To address this issue, we collect a natural language knowledge base that can be used for any VQA system. Moreover, we propose a Visual Retriever-Reader pipeline to approach knowledge-based VQA. ...
Acknowledgements The authors acknowledge support from the NSF grant 1816039, DARPA grant W911NF2020006, DARPA grant FA875019C0003, and ONR award N00014-20-1-2332; and thank the reviewers for their feedback ...
arXiv:2109.04014v1
fatcat:rnm2ghrosbd4xkctt4jnozfndu
A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
In this paper, we call for a paradigm shift for the OK-VQA task, which transforms the image into plain text, so that we can enable knowledge passage retrieval, and generative question-answering in the ...
This paradigm takes advantage of the sheer volume of gigantic knowledge bases and the richness of pre-trained language models. ...
Open-Domain Question Answering in NLP Opendomain question answering (Open-Domain QA) has been popular in the NLP community in recent years. ...
arXiv:2201.05299v1
fatcat:cbr5icvimbh7xfj7m2vxmpjyma
A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This paper examines the inclusion of domain-knowledge by means of changes to: the input, the loss-function, and the architecture of deep networks. ...
In many such instances, machine-based model construction may benefit significantly from being provided with human-knowledge of the domain encoded in a sufficiently precise form. ...
Krisp: Integrating implicit and symbolic
[31] Ellis, K., Morales, L., Meyer, M. S., Solar-Lezama, knowledge for open-domain knowledge-based vqa. In
A. & Tenenbaum, J. B. ...
arXiv:2107.10295v4
fatcat:ifkgq3cptbapfamr3dld57uruu
Low-resource Learning with Knowledge Graphs: A Comprehensive Survey
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 pre-print arXiv:2112.10006v2 fatcat:4gytqdlalncmhjgbsaitozph4mOther Versions
Among all the low-resource learning studies, many prefer to utilize some auxiliary information in the form of Knowledge Graph (KG), which is becoming more and more popular for knowledge representation, ...
dividing them into different paradigms such as the mapping-based, the data augmentation, the propagation-based and the optimization-based. ...
KRISP: Integrating implicit and symbolic knowledge for
open-domain knowledge-based VQA. ...
arXiv:2112.10006v3
fatcat:wkz6gjx4r5gvlhh673p3rqsmgi
Towards Knowledge-capable AI: Agents that See, Speak, Act and Know
2022
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Then we develop a method on that dataset which combines two types of knowledge: knowledge graphs and implicit knowledge from large language models. ...
We introduce a benchmark for vision and language that requires models with the capability to bring in and reason about knowledge about the world. ...
Abhinav has been a dogged advocate during my career, providing advice, support and guidance throught my PhD. ...
doi:10.1184/r1/19552225.v1
fatcat:rxhy64wyqfb4rjnizil6hrffge
Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering
2021
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
unpublished
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
One dataset that is mostly used in evaluating knowledge-based VQA is OK-VQA, but it lacks a gold standard knowledge corpus for retrieval. ...
To address this issue, we collect a natural language knowledge base that can be used for any VQA system. Moreover, we propose a Visual Retriever-Reader pipeline to approach knowledge-based VQA. ...
Acknowledgements The authors acknowledge support from the NSF grant 1816039, DARPA grant W911NF2020006, DARPA grant FA875019C0003, and ONR award N00014-20-1-2332; and thank the reviewers for their feedback ...
doi:10.18653/v1/2021.emnlp-main.517
fatcat:eacsuzciajb2npa2fholawrkpu
Integrating common sense knowledge in visual question answering systems
[article]
2022
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Aditionally, This thesis work comprehensively studies existing approaches for knowledge-based VQA architectures and analyze their shortcomings. ...
Finally, a novel pipeline-based architecture to integrate common sense knowledge into VQA systems, using an automatic know [...] ...
Thang Vu for his valuable input during the entire duration of my thesis. ...
doi:10.18419/opus-11965
fatcat:gkdhgd25lrdwzosc5dvw2kjexa
Low-resource Learning with Knowledge Graphs: A Comprehensive Survey
[article]
2021
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Among all the low-resource learning studies, many prefer to utilize some auxiliary information in the form of Knowledge Graph (KG), which is becoming more and more popular for knowledge representation, ...
dividing them into different paradigms such as the mapping-based, the data augmentation, the propagation-based and the optimization-based. ...
ACKNOWLEDGMENTS This work was supported by the SIRIUS Centre for Scalable Data Access (Research Council of Norway, project 237889), eBay, Samsung Research UK, Siemens AG, and the EPSRC projects OASIS ( ...
doi:10.48550/arxiv.2112.10006
fatcat:e2pdfnugbfad5nd4s525ge3tru