1,687 Hits in 5.9 sec

Multi-Modal Audio, Video and Physiological Sensor Learning for Continuous Emotion Prediction

Kevin Brady, Youngjune Gwon, Pooya Khorrami, Elizabeth Godoy, William Campbell, Charlie Dagli, Thomas S. Huang
2016 Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge - AVEC '16  
It presents the opportunity for investigating multimodal solutions that include audio, video, and physiological sensor signals.  ...  It includes a number of technical contributions, including the development of novel high-and low-level features for modeling emotion in the audio, video, and physiological channels.  ...  INTRODUCTION 1 The 2016 Audio-Visual Emotion Challenge (AVEC 2016) [10] aims to compare multimedia processing and machine learning methods for automatic speech, video, and physiological analysis of human  ... 
doi:10.1145/2988257.2988264 dblp:conf/mm/BradyGKGCDH16 fatcat:l6ncvm2qibfthnoabp6poxkcuu

Multistep deep system for multimodal emotion detection with invalid data in the Internet of Things

Min-Jia Li, Lun Xie, Ze-Ping Lv, Juan Li, Zhi-liang Wang
2020 IEEE Access  
The feature from invalid modal data is replaced through the imputation method to compensate for the impact of invalid data on emotion detection.  ...  Furthermore, considering the spatiotemporal information, the features of video and physiological signals are extracted by specific deep neural networks in the MSD system.  ...  physiological signals as the input signal of multi-modal fusion.  ... 
doi:10.1109/access.2020.3029288 fatcat:mictqr7hmbarbb5w2vmgvg6iti

A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances [article]

Yan Wang, Wei Song, Wei Tao, Antonio Liotta, Dawei Yang, Xinlei Li, Shuyong Gao, Yixuan Sun, Weifeng Ge, Wei Zhang, Wenqiang Zhang
2022 arXiv   pre-print
baseline dataset, fusion strategies for multimodal affective analysis, and unsupervised learning models.  ...  ., textual, audio, and visual data) and physiological signals (e.g., EEG and ECG signals). Physical-based affect recognition caters to more researchers due to multiple public databases.  ...  (visual-audio, text-audio and visual-audio-text modalities), multi-physiological approaches, and physicalphysiological approaches.  ... 
arXiv:2203.06935v3 fatcat:h4t3omkzjvcejn2kpvxns7n2qe

AV+EC 2015

Fabien Ringeval, Björn Schuller, Michel Valstar, Shashank Jaiswal, Erik Marchi, Denis Lalanne, Roddy Cowie, Maja Pantic
2015 Proceedings of the 5th International Workshop on Audio/Visual Emotion Challenge - AVEC '15  
We present the first Audio-Visual + Emotion recognition Challenge and workshop (AV + EC 2015) aimed at comparison of multimedia processing and machine learning methods for automatic audio, visual and physiological  ...  The goal of the Challenge is to provide a common benchmark test set for multimodal information processing and to bring together the audio, video and physiological emotion recognition communities, to compare  ...  in automatic continuous affect recognition from audio, video and, for the first time ever, physiological data.  ... 
doi:10.1145/2808196.2811642 dblp:conf/mm/RingevalSVJMLCP15 fatcat:og3k5mutvjcijb4xieg63u4phq

Decoupling Temporal Dynamics for Naturalistic Affect Recognition in a Two-Stage Regression Framework

Yona Falinie A. Gaus, Hongying Meng, Asim Jan
2017 2017 3rd IEEE International Conference on Cybernetics (CYBCONF)  
The results shows the proposed framework can capture temporal information at the prediction level, and outperform state-of-theart approaches in continuous affective recognition.  ...  Automatic continuous affect recognition from multiple modalities is one of the most active research areas in affective computing.  ...  The corpus consists of multimodal data, such as; audio, video, and physiological signal.  ... 
doi:10.1109/cybconf.2017.7985772 dblp:conf/cybconf/GausMJ17 fatcat:jxxbewad45d3hen6pjwptnlhim

Affective computing using speech and eye gaze: a review and bimodal system proposal for continuous affect prediction [article]

Jonny O'Dwyer, Niall Murray, Ronan Flynn
2018 arXiv   pre-print
Such multi-modal affective computing systems are advantageous for emotion assessment of individuals in audio-video communication environments such as teleconferencing, healthcare, and education.  ...  This work presents a review of the literature within the emotion classification and continuous affect prediction sub-fields of affective computing for both speech and eye gaze modalities.  ...  The authors used Kalman filter fusion of audio, video, and physiological modalities for their multi-modal submission on the test set.  ... 
arXiv:1805.06652v1 fatcat:mkwhbbocxnhtropgsjj5v3fo5u

Linear and Non-Linear Multimodal Fusion for Continuous Affect Estimation In-the-Wild

Yona Falinie A. Gaus, Hongying Meng
2018 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)  
In addressing this regression problem, the advantages of the each modality, such as audio, video and text, have been frequently explored but in an isolated way.  ...  Exponent Weighted Decision Fusion and Multi-Gene Genetic Programming.  ...  Recent developments of sensors like camera and microphone have led to a renewed interest in emotion recognition, from recognizing discrete basic emotion to recognizing continuous emotion, or continuous  ... 
doi:10.1109/fg.2018.00079 dblp:conf/fgr/GausM18 fatcat:p44l7hhgtfecddykm4ectr2f5i

AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challenge [article]

Michel Valstar, Jonathan Gratch, Bjorn Schuller, Fabien Ringeval, Denis Lalanne, Mercedes Torres Torres, Stefan Scherer, Guiota Stratou, Roddy Cowie, Maja Pantic
2016 arXiv   pre-print
The goal of the Challenge is to provide a common benchmark test set for multi-modal information processing and to bring together the depression and emotion recognition communities, as well as the audio  ...  for automatic audio, visual and physiological depression and emotion analysis, with all participants competing under strictly the same conditions.  ...  audio, video, and physiological analysis of emotion and depression, with all participants competing under strictly the same conditions.  ... 
arXiv:1605.01600v4 fatcat:j5bbsbjijzbgxh5zpclfksr4vu

Detecting expressions with multimodal transformers [article]

Srinivas Parthasarathy, Shiva Sundaram
2020 arXiv   pre-print
This study investigates deep-learning algorithms for audio-visual detection of user's expression.  ...  Ablation studies show the significance of the visual modality for the expression detection on the Aff-Wild2 database.  ...  Also, the proliferation of audio and video sensors in most devices make these appealing for expression detection.  ... 
arXiv:2012.00063v1 fatcat:zbcl4s76zvf4lasf5dz6fcvfrq

Inferring Depression and Affect from Application Dependent Meta Knowledge

Markus Kächele, Martin Schels, Friedhelm Schwenker
2014 Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge - AVEC '14  
It comprises classification results and considerations for both the continuous affect recognition sub-challenge and also the depression recognition subchallenge.  ...  task and label dependent templates to infer the respective emotional states.  ...  The authors of this paper are partially funded by the Transregional Collaborative Research Centre SFB/TRR 62 "Companion-Technology for Cognitive Technical Systems" funded by the German Research Foundation  ... 
doi:10.1145/2661806.2661813 dblp:conf/mm/KacheleSS14 fatcat:2qcepxw62fhqtoguynov4d7b3i

Emotion Recognition from Multiple Modalities: Fundamentals and Methodologies [article]

Sicheng Zhao, Guoli Jia, Jufeng Yang, Guiguang Ding, Kurt Keutzer
2021 arXiv   pre-print
Furthermore, we present some representative approaches on representation learning of each affective modality, feature fusion of different affective modalities, classifier optimization for MER, and domain  ...  In this tutorial, we discuss several key aspects of multi-modal emotion recognition (MER). We begin with a brief introduction on widely used emotion representation models and affective modalities.  ...  That means temporal, spatial, and multi-channel representations can be learned and utilized to recognize the emotions in videos. IV.  ... 
arXiv:2108.10152v1 fatcat:hwnq7hoiqba3pdf6aakcxjq33i

Recognition of Students' Mental States in Discussion Based on Multimodal Data and its Application to Educational Support

Shimeng Peng, Katashi Nagao
2021 IEEE Access  
Shigeki Ohira for his help in collecting the ground truth measures of the experiments, and the members from Nagao Lab for participating in the data collection process.  ...  The authors would also like to thank Andy Gao for his help in proofreading the contents of this paper.  ...  Some of these works used uni-variate modality signals, that is, video [11] , audio [4] , [12] , and physiological measures [13] .  ... 
doi:10.1109/access.2021.3054176 fatcat:uekzibqbkfcz5liehqzleg6k2y

EmoBed: Strengthening Monomodal Emotion Recognition via Training with Crossmodal Emotion Embeddings

Han Jing, Zhang Zixing, Ren Zhao, Schuller Björn
2019 Zenodo  
The monomodal emotion recognition systems normally independently explore the prominent features for the emotions of interest, from one specific modality, such as audio, video, image, text, or physiology  ...  fusion (c)), and multi-task learning (d).  ...  Besides, she co-chaired the 7th Audio/Visual Emotion Challenge (AVEC) and workshop in 2017, and served as a program committee member of the 8th AVEC challenge and workshop in 2018.  ... 
doi:10.5281/zenodo.3661155 fatcat:7tpenwbfqnho3cypcjsmdxul7m

An Event Driven Fusion Approach for Enjoyment Recognition in Real-time

Florian Lingenfelser, Johannes Wagner, Elisabeth André, Gary McKeown, Will Curran
2014 Proceedings of the ACM International Conference on Multimedia - MM '14  
Fusion of multi-modal signals is a natural and interesting way to improve automatic classification of emotions transported in social signals.  ...  Throughout most present studies, uni-modal affect recognition as well as multi-modal fusion, decisions are forced for fixed annotation segments across all modalities.  ...  Table 2 gives insight into the capability of event detection recognizers for the audio and video modality.  ... 
doi:10.1145/2647868.2654924 dblp:conf/mm/LingenfelserWAMC14 fatcat:wmwmjpuz4zb3djszou42wc56eq

A Survey on Human Emotion Recognition Approaches, Databases and Applications

C. Vinola, K. Vimaladevi
2015 ELCVIA Electronic Letters on Computer Vision and Image Analysis  
The modalities and approaches used for affect detection vary and contribute to accuracy and efficacy in detecting emotions of human beings.  ...  This paper presents the various emotion classification and recognition systems which implement methods aiming at improving Human Machine Interaction.  ...  Emotion recognition databases As a result of wide-ranging research in emotion identification, multiple databases have emerged covering different modalities including image, video, audio and physiological  ... 
doi:10.5565/rev/elcvia.795 fatcat:wlaxn2xgczhm7dhkekk2atfcuq
« Previous Showing results 1 — 15 out of 1,687 results