Filters








88,682 Hits in 4.1 sec

Modeling Multimodal Communication as a Complex System [chapter]

Kristinn R. Thórisson
Modeling Communication with Robots and Virtual Humans  
Keywords: Multimodal realtime communication, computational model, complex natural system, heterogeneous large system, abstract module, theory of dialogue.  ...  Consequently, models of communication must encompass a large subset of the functions and couplings that make up the real system, calling for a powerful methodology for integrating information from multiple  ...  European Community Framework Programme.  ... 
doi:10.1007/978-3-540-79037-2_8 dblp:conf/zif/Thorisson06 fatcat:alzgbd66mvbo7m2vz2z7aa26dq

Multiagent-Based Modeling and Simulation of a Coal Multimodal Transport System

Yan Yang, Qiang Zhou, Ke Chen
2022 IEEE Access  
The CMTS is a complex nonlinear dynamic system with many participants with complex and uncertain links.  ...  Therefore, it is necessary to analyze the multimodal transportation nodes, build a coal multimodal transportation model, measure the network throughput through simulation experiments, and identify system  ...  The research group has a profound background in simulation modeling research on coal terminals and multimodal transport projects and is good at finding system bottlenecks and researching system optimization  ... 
doi:10.1109/access.2022.3184728 fatcat:c3ep7biqerdkfnz6aqberx4arm

MILKI-PSY Cloud : Facilitating multimodal learning analytics by explainable AI and blockchain

Michal Piotr Slupczynski, Ralf Klamma
2021 Multimodal Immersive Learning Systems 2021 : Proceedings of the First International Workshop on Multimodal Immersive Learning Systems (MILeS 2021)  
In particular, complex psycho-motor skills like dancing or operating a complex machine are profiting from MMLA.  ...  We propose an approach for the acquisition, storage, processing and presentation of multimodal learning analytics data using machine learning and blockchain as services to reach explainable artificial  ...  This architecture serves as a reference model of our system structure.  ... 
doi:10.18154/rwth-2022-01582 fatcat:z4uhikkz6zboxcqjkukoe3f7be

Modeling multimodal human-computer interaction

Z. Obrenovic, D. Starcevic
2004 Computer  
. 5 Modeling at different levels of abstraction clearly benefits multimodal systems, as implementation platforms typically use primitives that are far removed from concepts such as modality or multimodal  ...  As the "Multimodal Interaction" sidebar describes, although many approaches exist, sound and practical solutions for developing multimodal systems are still lacking.  ...  We then introduced UML extensions for modeling basic modalities and describing complex multimodal systems.  ... 
doi:10.1109/mc.2004.139 fatcat:gkavi73fzrexrgcxnscysin4vu

When Language Evolution Meets Multimodality: Current Status and Challenges Toward Multimodal Computational Models

Patrizia Grifoni, Arianna D'Ulizia, Fernando Ferri
2021 IEEE Access  
It discusses the key challenges towards multimodal language evolution modelling by envisioning a conceptual framework to design the multimodal grounding and the language learning processes, as well as  ...  a multimodal language evolution model.  ...  They concluded that spoken and gestural languages have evolved together as one integrated multimodal communication system.  ... 
doi:10.1109/access.2021.3061756 fatcat:f2gutl4pnvdkli7zdxoubzhzka

An introduction to multimodal communication

James P. Higham, Eileen A. Hebets
2013 Behavioral Ecology and Sociobiology  
We discuss the history of the study of complexity in animal communication, issues relating to defining and classifying multimodal signals, and particular issues to consider with multimodal (as opposed  ...  The present manuscript serves as both an introduction to this special issue, as well as an introduction to multimodal communication more generally.  ...  Acknowledgments We are very grateful to Esteban Fernandez-Juricic and two anonymous reviewers for comments on a previous draft of this manuscript.  ... 
doi:10.1007/s00265-013-1590-x fatcat:krg6qy7odzgzza42rdptzwssfa

A Multimodal Approach to Design of Aircraft Cockpit Displays

Mlađan Jovanović, Dušan Starčević, Mirko Petrović
2011 JITA - Journal of Information Technology and Applications (Banja Luka) - APEIRON  
Command tables impose high load on human visual senses for displaying flight information such as altitude, attitude, vertical speed, airspeed, heading and engine power.  ...  Operator/pilot forms a mental model as a comprehensive understanding of a system and its dynamics.  ...  User communicates with the system through set of communication channels which use different modalities, such as visual display, audio, and tactile feedback, to engage human perceptual, cognitive, and communication  ... 
doi:10.7251/jit1101016j fatcat:yy5ng3wjlvbm5cgulh5oe57fv4

Book ReviewMultimodality in Language and Speech Systems Björn Granström, David House, and Inger Karlsson (editors) (Royal Institute of Technology, Stockholm) Dordrecht: Kluwer Academic Publishers (Text, speech and language technology series, edited by Nancy Ide and Jean Véronis, volume 19), 2002, ix+241 pp; hardbound, ISBN 0-4020-0635-7, $82.00, £56.00, €89.00

Michael Johnston
2003 Computational Linguistics  
A mind model for multimodal communicative creatures and humanoids. International Journal of Applied Artificial Intelligence, 13(4—-5):449-486.  ...  Like Allwood’s, the chapter by Bernsen, “Multimodality in Language and Speech Systems—From Theory to Design Support Tool,” provides a framework that can be used in the analysis of multimodal communication  ... 
doi:10.1162/coli.2003.29.2.321 fatcat:cqmeuwmd6ngrbnrpzfri54tuji

MIS: Multimodal Interaction Services in a cloud perspective [article]

Patrizia Grifoni, Fernando Ferri, Maria Chiara Caschera, Arianna D'Ulizia, Mauro Mazzei
2017 arXiv   pre-print
The benefits of cloud computing for multimodal interaction have been identified by emphasizing the flexibility and scalability of a SOA, and its characteristics to provide a more holistic view of interaction  ...  It can offer adequate computational resources to manage the complexity implied by the use of the five senses when involved in human machine interaction.  ...  modeled by SOA technologies as each HCI function can be considered as a service.  ... 
arXiv:1704.00972v1 fatcat:hld4rrlbyrbi7bii6yskvezf2m

Towards Symmetric Multimodality: Fusion and Fission of Speech, Gesture, and Facial Expression [chapter]

Wolfgang Wahlster
2003 Lecture Notes in Computer Science  
A dialogue system with symmetric multimodality must not only understand and represent the user's multimodal input, but also its own multimodal output.  ...  We present the SmartKom system, that provides full symmetric multimodality in a mixed-initiative dialogue system with an embodied conversational agent.  ...  In SMARTKOM, we have developed M3L (Multimodal Markup Language) as a complete XML language that covers all data interfaces within this complex multimodal dialog system.  ... 
doi:10.1007/978-3-540-39451-8_1 fatcat:sqdnkrur2zcpdmiainiy2ydybm

Boundaries of Multimodality in Virtual Information Environment Design [chapter]

J. Kalaykova, M. Pankina
2005 Communication Trends in the Post-Literacy Era: Polylingualism, Multimodality and Multiculturalism As Preconditions for New Creativity  
environment design and raise the question of human limits and potentialities in the perception of communicative images and the issue of communication in the virtual environment as a goal of design.  ...  With reference to the current scientific discourse, we explore insights into the identification and interpretation of meanings in their multiplicity as a characteristic feature of objects of virtual information  ...  A design product considered as a system of multimodal elements may be quite simple: "The apparent complexity of our behavior over time is largely a reflection of the complexity of the environment in which  ... 
doi:10.15826/b978-5-7996-3081-2.24 fatcat:hqd6nkodhfhkpfxgbx454gymvm

Multimodal Emotional Understanding in Robotics [chapter]

Juanpablo Heredia, Yudith Cardinale, Irvin Dongo, Ana Aguilera, Jose Diaz-Amado
2022 Ambient Intelligence and Smart Environments  
This paper presents the implementation of a previous proposed multi-modal emotional system in the context of social robotics; that works on a server and bases its prediction in four modalities as inputs  ...  In this sense, an out-of-robot approach and a multimodal approach can be the solution.  ...  [11] present a multimodal system that employs different inputs such as speech, facial expression, body gesture, eye movement, and psychological signals.  ... 
doi:10.3233/aise220020 fatcat:juxjzq2vofebneutqdngm7nhem

A Framework for a Priori Evaluation of Multimodal User Interfaces Supporting Cooperation [chapter]

Magnus Larsson, Gilles Coppin, Franck Poirier, Olivier Grisvard
2013 Human–Computer Interaction Series  
as well as multimodal.  ...  We believe that our framework will be a welcome help for novice designers of cooperative multimodal systems when making expert decisions in choice of modality or combinations of modalities.  The complexity  ... 
doi:10.1007/978-1-4471-5499-0_7 dblp:series/hci/LarssonCPG13 fatcat:ywfsdeuoljgrjjvvkg5tjsjdge

Supporting children's social communication skills through interactive narratives with virtual characters

Mary Ellen Foster, Katerina Avramides, Sara Bernardini, Jingying Chen, Christopher Frauenberger, Oliver Lemon, Kaska Porayska-Pomsta
2010 Proceedings of the international conference on Multimedia - MM '10  
The development of social communication skills in children relies on multimodal aspects of communication such as gaze, facial expression, and gesture.  ...  We also describe how the structure underlying this system is currently being extended into a general framework for the development of interactive multimodal systems.  ...  Supporting such inter-module communication is a complex technical task, and one where a general-purpose framework is useful: not only does it simplify the process of developing such a system, by providing  ... 
doi:10.1145/1873951.1874163 dblp:conf/mm/FosterABCFLP10 fatcat:a3os5qunu5d4neyymdyyutnzkq

A review on manipulation skill acquisition through teleoperation‐based learning from demonstration

Weiyong Si, Ning Wang, Chenguang Yang
2021 Cognitive Computation and Systems  
Especially, the learning from demonstration method has been exploited widely and successfully in the robotic community, and it is regarded as a promising direction to realise the manipulation skill learning  ...  This review, therefore, aims to provide an overview of immersive teleoperation for skill learning and generalisation to deal with complex manipulation tasks.  ...  | PRELIMINARIES Generally, a multimodal teleoperation system for robot skill learning includes the following parts: human operator with interactive interfaces, a communication module, teleoperation  ... 
doi:10.1049/ccs2.12005 fatcat:wxyourkvrvcqlh6aht6g5fi3sy
« Previous Showing results 1 — 15 out of 88,682 results