Filters








742 Hits in 6.9 sec

Deconstructing multimodality: visual properties and visual context in human semantic processing

Christopher Davis, Luana Bulat, Anita Lilla Vero, Ekaterina Shutova
2019 Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*  
This choice of task allows us to assess the importance of each type of visual information in human semantic processing.  ...  For instance, some concepts may be better characterised by their own (internal) visual properties and others by the (external) visual context, in which they appear.  ... 
doi:10.18653/v1/s19-1013 dblp:conf/starsem/DavisBVS19 fatcat:mcroikgusbb3pk3clzcpb2jwyu

Reading paths and visual perception in multimodal research, psychology and brain sciences

Tuomo Hiippala
2012 Journal of Pragmatics  
This paper argues that the concept of a reading path in multimodal research can be improved by previous research on visual perception in psychology and brain sciences, and particularly by the work done  ...  Finally, the paper presents a hypothesis on the relationship of visual perception and multimodal semiosis, which may be evaluated using the proposed method combining multimodal analysis and eye-tracking  ...  Acknowledgements The author would like to thank -in no particular order -the following individuals who commented on earlier drafts of this paper: John Bateman, Ivan Berazhny, Eija Ventola and the two anonymous  ... 
doi:10.1016/j.pragma.2011.12.008 fatcat:5vjdfzrrtzcw3asiksvtypibru

Multimodality in human interaction

Charles Goodwin
2010 Calidoscópio  
Second, the processes through which archaeologists acquire the ability to see relevant structure in the dirt they are excavating, and construct the documents, such as maps, that animate the discourse of  ...  Human action is built by actively combining materials with intrinsically different properties into situated contextual confi gurations where they can mutually elaborate each other to create a whole that  ...  The talk in line 14 is semantically and syntactically far beyond anything that Child could say on his own.  ... 
doi:10.4013/cld.2010.82.01 fatcat:os5jp4lwwbe2pcdqomcrpfpruy

Multimodal Metaphor as the Source of Parodic Integration in Hip Hop Music Videos

Gordana Čupković, Silvana Dunat
2020 Narodna Umjetnost  
This paper deals with multimodal metaphors as the basis of parodic integration in selected videos and album covers by rap artist Krešo Bengalka and his band Kiša metaka.  ...  The study focuses on multimodal integration and disintegration and on the reversal of the conventional way of representing both the relation between the interior and exterior and the relation between the  ...  Instead, we would like to show that parody, revealed in a multimodal context, exhibits semantic processes akin to those discussed in cognitive linguistics, which means that perhaps further investigation  ... 
doi:10.15176/vol57no204 fatcat:cgdafwwtrzay7cwp42rvxgotde

Interaction Design as a Multimodal Conversation [chapter]

Arminda Lopes
2010 IFIP Advances in Information and Communication Technology  
This paper explores the concept of design as a multimodal conversation, in the context of observed case studies within interdisciplinary collaboration.  ...  Five case studies were video recorded and the verbal and non-verbal designers' behaviours as well as the designers' interactions with the artefacts were analysed.  ...  I am very grateful to the management team of the Leonardo Network in Culture, Creativity and Interaction Design and to White Rose Network for Affective Communication in Consumer Products and Exhibition  ... 
doi:10.1007/978-3-642-11762-6_19 fatcat:kjfgxgnvene7pfwu4ouya4g5vu

Context, Contextualisation and (Multimodal) Text

Yves Gambier
2021 Studies About Languages  
What does re-contextualisation imply, and how does context apply to (multimodal) text in a digital environment?  ...  We will consider to what extent context and contextualisation are related, when translation is defined as a context-dependent meaning-making process.  ...  The decline of objectivism and positivism is parallel to the abandonment of context-free grammars, the increase in references to the concept of context, especially in discursive and cognitive semantics  ... 
doi:10.5755/j01.sal.1.39.28965 fatcat:c6iqm6ekireebl7vglw7ad3qem

Sentic Avatar: Multimodal Affective Conversational Agent with Common Sense [chapter]

Erik Cambria, Isabelle Hupont, Amir Hussain, Eva Cerezo, Sandra Baldassarri
2011 Lecture Notes in Computer Science  
multimodal animation engine for managing virtual agents and 3D scenarios.  ...  In this paper we present a novel architecture for the development of intelligent multimodal affective interfaces.  ...  The model is particularly useful to recognize, understand and express emotions in the context of human-computer interaction (HCI).  ... 
doi:10.1007/978-3-642-18184-9_8 fatcat:ptwanhtyoveabp2hsnxwwzjqdy

A review of affective computing: From unimodal analysis to multimodal fusion

Soujanya Poria, Erik Cambria, Rajiv Bajpai, Amir Hussain
2017 Information Fusion  
In this paper, we focus mainly on the use of audio, visual and text information for multimodal affect analysis, since around 90% of the relevant literature appears to cover these three modalities.  ...  Multimodality is defined by the presence of more than one modality or channel, e.g., visual, audio, text, gestures, and eye gage.  ...  ] on multimodal emotion recognition, which mainly focuses on identifying challenges involved in collecting and processing audio, visual and audio-visual (i.e., multimodal) data.  ... 
doi:10.1016/j.inffus.2017.02.003 fatcat:ytebhjxlz5bvxcdghg4wxbvr6a

Museum Audio Description: Multimodal and 'Multisensory' Translation: A Case Study from the British Museum

Monica Randaccio
2018 Linguistics and Literature Studies  
Then, a general overview of audio description and its process creation are briefly illustrated in their strengths and limitations.  ...  Both studies see the museum and its audio description as an interactive multimodal communicative event but the former focusses more on the grammar of multimodality, whereas the latter emphasises aspects  ...  This choice is therefore a psychological process expressing a certain attitude of the human mind towards the visual world.  ... 
doi:10.13189/lls.2018.060604 fatcat:d6ck5fzn4bamhizufljncna7ea

Towards an intelligent framework for multimodal affective data analysis

Soujanya Poria, Erik Cambria, Amir Hussain, Guang-Bin Huang
2015 Neural Networks  
In this paper, we propose a novel multimodal information extraction agent, which infers and aggregates the semantic and affective information associated with usergenerated multimodal data in contexts such  ...  as e-learning, e-health, automatic video content tagging and human-computer interaction.  ...  Video opinions provide multimodal data in terms of vocal and visual modality.  ... 
doi:10.1016/j.neunet.2014.10.005 pmid:25523041 fatcat:xu4k5ywowfb2jgmuwgsdpozfzq

A Socio Semiotic Approach to Multimodal Discourse of Selected Nollywood Film Advertisement Posters

Monsurat Aramide Nurudeen, Ebenezer Oluseun Ogungbe, Moshood Zakariyah
2021 Studies in Pragmatics and Discourse Analysis  
The study concludes by emphasizing the importance of the synergy of both linguistic and visual multimodal resources or modes of signification in the successful meaning-making and meaning-comprehension  ...  in the study of visual communication.  ...  Hence, the need for a viable field of study to account for the complexities in human interaction is what multimodality or multimodal discourse analysis addresses.  ... 
doi:10.48185/spda.v2i2.364 fatcat:rnfpu3sv2remvbsrqrq6csvt5i

Locating the Semiotic Power of Multimodality

Glynda A. Hull, Mark Evan Nelson
2005 Written Communication  
texts play in real-world contexts, and they offer one example of how such investigations might be approached.  ...  Our sincere thanks to Randy Young, the multimodal composer whose work we featured in this article, and to Mike Rose, Chris Haas, and Mira-Lisa Katz for their helpful commentary.  ...  context.  ... 
doi:10.1177/0741088304274170 fatcat:7pxuklbdencv7j42tv2ckqnlyq

People with Motor and Mobility Impairment: Innovative Multimodal Interfaces to Wheelchairs [chapter]

Andreas Holzinger, Alexander K. Nischelwitzer
2006 Lecture Notes in Computer Science  
While Media Lab is in charge of the implementation of the technical innovations, interface design and the human-computer-interaction, the Art Museum Development Department coordinates the development of  ...  In addition, the Art Museum Development Department takes part in teaching, planning the MUMMI curricula and its objectives, and evaluation of the demonstrations.  ...  In multimodal interface speech, signing, gesture, pen could be a vital part of interaction between human and the computer.  ... 
doi:10.1007/11788713_144 fatcat:56ce7t6nfzdunkvoxze4qxrul4

Authoring the Readerly Experience: On Materiality, Multimodality and Singularity of Literature in Nicola Barker's 'H(a)ppy'

2021 Avant  
of subverting authority in postmodern fiction and visions of the reader in multimodal texts.  ...  This article offers an overview of typographical and experimental strategies employed in contemporary novel writing in the context of author-ing, authority, freedom and experience programming.  ...  in the reading process.  ... 
doi:10.26913/avant.2021.03.09 fatcat:eohorbrjtza67d3hikcprcjg54

Carpool Karaoke: deconstructing the directly lived experience of hearing oneself singing

George Rossolatos
2016 Social Semiotics  
The various ways whereby spatial conditions afford to monumentalize culture and to appropriate geographically demarcated places in terms of individual and collective meaning structures has been amply documented  ...  in urban cultural studies.  ...  in this multimodal spectacle (cf.  ... 
doi:10.1080/10350330.2016.1261480 fatcat:vcql4qr26vbszfl2tquvj6krza
« Previous Showing results 1 — 15 out of 742 results