A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
End-to-End Entity Classification on Multimodal Knowledge Graphs
[article]
2020
arXiv
pre-print
Preserved Fulltext
End-to-end multimodal learning on knowledge graphs has been left largely unaddressed. ...
We propose a multimodal message passing network which not only learns end-to-end from the structure of graphs, but also from their possibly divers set of multimodal node features. ...
Acknowledgments We express our gratitude to Wouter Beek from Triply 9 for helping us obtain two of the three knowledge graphs that make up the Dutch Monument Graph. ...
arXiv:2003.12383v1
fatcat:z5qqcjqaurftzakgf3lagie25y
A Garbage Detection and Classification Method Based on Visual Scene Understanding in the Home Environment
2021
Complexity
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
This method uses knowledge graphs to store and model items in the scene in the form of images, videos, texts, and other multimodal forms. ...
The ESA attention mechanism is added to the backbone network part of the YOLOv5 network, aiming to improve the feature extraction ability of the network, combining with the built multimodal knowledge graph ...
Multimodal Knowledge Graph. ...
doi:10.1155/2021/1055604
fatcat:i3uvswukarf7bkfpflt7yoaesa
Fake News Detection via Knowledge-driven Multimodal Graph Convolutional Networks
2020
Proceedings of the 2020 International Conference on Multimedia Retrieval
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In conclusion, the contributions of our work are as follows: • We propose an end-to-end Knowledge-driven Multimodal Graph Convolutional Network to model the semantic-level representations for fake news ...
correct entities T in the knowledge graph they refer to. ...
doi:10.1145/3372278.3390713
dblp:conf/mir/WangQHFX20
fatcat:bdtdwo3pwbhm5bbipy7w3x6idi
Combining and Curating Automated Metadata from Multiple Sources
2020
Zenodo
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This presentation was featured in the DataTV 2020 Webinar on September 14, 2020. ...
• Legacy metadata provided by INA and Yle, (+1600 hours of media) was
processed and consolidated according to the EBUCore/MeMAD ontology
and put into the MeMAD Knowledge Graph
○ Knowledge Graph is ...
Named entity recognition and disambiguation
9. Semantic enrichment
10. ...
doi:10.5281/zenodo.4058122
fatcat:hczzvlatonddtlwg7u6rairlnq
Fine-Grained Chemical Entity Typing with Multimodal Knowledge Representation
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
How to extract detailed knowledge about chemical reactions from the core chemistry literature is a new emerging challenge that has not been well studied. ...
We introduce a new benchmark data set (CHEMET) to facilitate the study of the new task and propose a novel multi-modal representation learning framework to solve the problem of fine-grained chemical entity ...
. • We propose and evaluate a novel method that utilizes multimodal knowledge to enrich entity mention repre- sentation. ...
arXiv:2108.12899v1
fatcat:cib35n544vdo5pj34ztrm7jowy
Literature mining for context-specific molecular relations using multimodal representations (COMMODAR)
2020
BMC Bioinformatics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
The models based on multiple modalities outperformed those solely based on the linguistic modality. ...
We applied COMMODAR to the 14 million PubMed abstracts and extracted 9214 context-specific molecular relations. ...
About this supplement This article has been published as part of BMC Bioinformatics, Volume 21 Supplement 5, 2020: Proceedings of the 13th International Workshop on Data and Text Mining in Biomedical Informatics ...
doi:10.1186/s12859-020-3396-y
pmid:33106154
fatcat:s3oha7endfcqhnxifzdckxxi5a
Understanding the Gist of Images - Ranking of Concepts for Multimedia Indexing
[article]
2018
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Thus, the presented end-to-end setting outperforms DBM and competes with Hashing-based methods. ...
Our pipeline benefits from external knowledge and two subsequent learning- to-rank (l2r) settings. The first l2r produces a ranking of concepts rep- resenting the respective multimedia instance. ...
In an end-to-end approach we have demonstrated that multimodal modelling -even if one modality comes from noisy object detectors -performs better than a single modality approach. ...
arXiv:1809.08593v1
fatcat:4kqujgxphrgrjc2ukzidnk3pse
Knowledge Extraction And Representation Learning For Music Recommendation And Classification
2017
Zenodo
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
To this end, we first focus on the problem of linking music-related texts with online knowledge repositories and on the automated construction of music knowledge bases. ...
We focus on the semantic enrichment of descriptions associated to musical items (e.g., artists biographies, album reviews, metadata), and the exploitation of multimodal data (e.g., text, audio, images) ...
To this end, we present MuMu, a new large-scale multimodal dataset for multilabel music genre classification. ...
doi:10.5281/zenodo.1048497
fatcat:kdh5jhvocbh3riwln6n2f756su
Knowledge Extraction And Representation Learning For Music Recommendation And Classification
2017
Zenodo
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
To this end, we first focus on the problem of linking music-related texts with online knowledge repositories and on the automated construction of music knowledge bases. ...
We focus on the semantic enrichment of descriptions associated to musical items (e.g., artists biographies, album reviews, metadata), and the exploitation of multimodal data (e.g., text, audio, images) ...
To this end, we present MuMu, a new large-scale multimodal dataset for multilabel music genre classification. ...
doi:10.5281/zenodo.1100973
fatcat:yfpmc6qxbbakjp6qzvywyoaoci
A Multimodal Translation-Based Approach for Knowledge Graph Representation Learning
2018
Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
to the KG entities. ...
We compared the performance of our approach to other baselines on two standard tasks, namely knowledge graph completion and triple classification, using our as well as the WN9-IMG dataset. 1 The results ...
(5) to combine the linguistic and visual embeddings into a single one. To that end, we identified three methods for multimodal representation learning and adapted them to KG entities. ...
doi:10.18653/v1/s18-2027
dblp:conf/starsem/SergiehBGR18
fatcat:dftmpkrljbecjfoqrksnklwyni
Boosting Entity-aware Image Captioning with Multi-modal Knowledge Graph
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
To tackle these challenges, we propose a novel approach that constructs a multi-modal knowledge graph to associate the visual objects with named entities and capture the relationship between entities simultaneously ...
Entity-aware image captioning aims to describe named entities and events related to the image by utilizing the background knowledge in the associated article. ...
To obtain rich representations of the entities in the multimodal knowledge graph, we encode the nodes in the multimodal knowledge graph using a two-layer graph attention network (GAT) [25] . ...
arXiv:2107.11970v1
fatcat:drxuorjiobfuzeqdv4cvq3upzm
Multimodal Classification of Violent Online Political Extremism Content with Graph Convolutional Networks
2017
Proceedings of the on Thematic Workshops of ACM Multimedia 2017 - Thematic Workshops '17
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
were recently proven effective in classification tasks on knowledge graphs. ...
In this paper we present a multimodal approach to categorizing user posts based on their discussion topic. ...
The authors would like to thank Stefan Verbruggen and Gijs Koot from TNO for their help with collecting the data as well as Prof.dr. ...
doi:10.1145/3126686.3126776
dblp:conf/mm/RudinacGW17
fatcat:kxpmaws5yzeovijqlg57rsd3xe
LingYi: Medical Conversational Question Answering System based on Multi-modal Knowledge Graphs
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
This paper presents a medical conversational question answering (CQA) system based on the multi-modal knowledge graph, namely "LingYi", which is designed as a pipeline framework to maintain high flexibility ...
To conduct knowledge-grounded dialogues with patients, we first construct a Chinese Medical Multi-Modal Knowledge Graph (CM3KG) and collect a large-scale Chinese Medical CQA (CMCQA) dataset. ...
First, the CRM maps the medical entities obtained in the disambiguation module to specific attributes on the knowledge graph, and stores the past entites into the dictionary. ...
arXiv:2204.09220v1
fatcat:t7ai6qosmzac7nbehowfmy4fg4
Current Approaches and Applications in Natural Language Processing
2022
Applied Sciences
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Artificial Intelligence has gained a lot of popularity in recent years thanks to the advent of, mainly, Deep Learning techniques [...] ...
[10] proposes an approach to entity linking (associate mentions in documents to existing entities in a knowledge graph) that profits from structural information of the graph, so correlation information ...
Linguistic-based methods have been surpassed by end-to-end architectures, where no prior knowledge on language is needed, although only when a massive amount of data is available. ...
doi:10.3390/app12104859
fatcat:yhoyyoqcazflrbx7veksnkrrdq
AR-BERT: Aspect-relation enhanced Aspect-level Sentiment Classification with Multi-modal Explanations
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Existing models do not incorporate information on aspect-aspect relations in knowledge graphs (KGs), e.g. DBpedia. ...
Two main challenges stem from inaccurate disambiguation of aspects to KG entities, and the inability to learn aspect representations from the large KGs in joint training with ALSC models. ...
Hence, we seek to design a postthoc global explanation model for predicting multimodal explanations from both context words, and KG-entities. ...
arXiv:2108.11656v2
fatcat:xpymls6pdjbunctvvcvl66jgp4
« Previous
Showing results 1 — 15 out of 4,914 results