Towards Using Semantic-Web Technologies for Multi-Modal Knowledge Graph Construction

Matthias Baumgartner, Luca Rossetto, Abraham Bernstein
2020
While a multitude of approaches for extracting semantic information from multimedia documents has emerged in recent years, isolating any form of holistic semantic representation from a larger type of document, such as a movie, is not yet feasible. In this paper we present our approaches used in the first instance of the Deep Video Understanding Challenge, using a combination of several multimodal detectors and an integration scheme informed by methods from the semantic web context in order to
more » ... ntext in order to determine the capabilities limitations of currently available methods for the extraction of semantic relations between the characters and locations relevant to the narrative of a movie. ABSTRACT While a multitude of approaches for extracting semantic information from multimedia documents has emerged in recent years, isolating any form of holistic semantic representation from a larger type of document, such as a movie, is not yet feasible. In this paper we present our approaches used in the first instance of the Deep Video Understanding Challenge, using a combination of several multi-modal detectors and an integration scheme informed by methods from the semantic web context in order to determine the capabilities limitations of currently available methods for the extraction of semantic relations between the characters and locations relevant to the narrative of a movie.
doi:10.5167/uzh-190784 fatcat:wo47lhzwlnfnljch7uyzvv762u