Filters








5 Hits in 5.9 sec

AI2D-RST: A multimodal corpus of 1000 primary school science diagrams [article]

Tuomo Hiippala and Malihe Alikhani and Jonas Haverinen and Timo Kalliokoski and Evanfiya Logacheva and Serafina Orekhova and Aino Tuomainen and Matthew Stone and John A. Bateman
2020 arXiv   pre-print
This article introduces AI2D-RST, a multimodal corpus of 1000 English-language diagrams that represent topics in primary school natural sciences, such as food webs, life cycles, moon phases and human physiology  ...  Building on the segmentation of diagram layouts in AI2D, the AI2D-RST corpus presents a new multi-layer annotation schema that provides a rich description of their multimodal structure.  ...  In this article, we make a novel contribution to the study of diagrams by presenting AI2D-RST, a corpus of 1000 English-language diagrams that represent topics in primary school natural sciences.  ... 
arXiv:1912.03879v2 fatcat:bzorpln4enhmzct3jhjabkoxie

AI2D-RST: a multimodal corpus of 1000 primary school science diagrams

Tuomo Hiippala, Malihe Alikhani, Jonas Haverinen, Timo Kalliokoski, Evanfiya Logacheva, Serafina Orekhova, Aino Tuomainen, Matthew Stone, John A. Bateman
2020 Language Resources and Evaluation  
AbstractThis article introduces AI2D-RST, a multimodal corpus of 1000 English-language diagrams that represent topics in primary school natural sciences, such as food webs, life cycles, moon phases and  ...  Building on the segmentation of diagram layouts in AI2D, the AI2D-RST corpus presents a new multi-layer annotation schema that provides a rich description of their multimodal structure.  ...  corpus of 1000 primary school science diagrams AI2D-RST discourse structure graph 1.  ... 
doi:10.1007/s10579-020-09517-1 fatcat:el3xk2art5f3fjcmvkg3prwtce

Classifying Diagrams and Their Parts using Graph Neural Networks: A Comparison of Crowd-Sourced and Expert Annotations [article]

Tuomo Hiippala
2019 arXiv   pre-print
This article compares two multimodal resources that consist of diagrams which describe topics in elementary school natural sciences.  ...  This article reports on two experiments that evaluate how effectively crowd-sourced and expert-annotated graphs can represent the multimodal structure of diagrams for representation learning using various  ...  Here the aim is to evaluate to what extent graph neural networks can learn about the generic structure of primary school science diagrams from the graph-based representations in AI2D and AI2D-RST.  ... 
arXiv:1912.02866v1 fatcat:u4snprup2jhkvazmoew5rhibf4

Semiotically-grounded distant viewing of diagrams: insights from two multimodal corpora [article]

Tuomo Hiippala, John A. Bateman
2021 arXiv   pre-print
In this article, we bring together theories of multimodal communication and computational methods to study how primary school science diagrams combine multiple expressive resources.  ...  diagrams as a mode of communication.  ...  The AI2D-RST corpus covers 1000 diagrams from AI2D.  ... 
arXiv:2103.04692v1 fatcat:c4pv6vinmrbybcgg3gkxhcqdiu

Introducing the diagrammatic semiotic mode [article]

Tuomo Hiippala, John A. Bateman
2022 arXiv   pre-print
within the field of multimodality.  ...  We illustrate the proposed approach using two recent diagram corpora and show how a multimodal approach supports the empirical analysis of diagrammatic representations, especially in identifying diagrammatic  ...  AI2D-RST -a multimodally-motivated annotation schema The second corpus considered here, AI2D-RST, covers 1000 diagrams taken from the AI2D corpus and is annotated using a new schema by experts specifically  ... 
arXiv:2001.11224v2 fatcat:ntmpmximt5hjbdt2xnd3kthfjq