Filters








405 Hits in 4.0 sec

Multimodal Memorability: Modeling Effects of Semantics and Decay on Video Memorability [article]

Anelise Newman, Camilo Fosco, Vincent Casser, Allen Lee, Barry McNamara, Aude Oliva
2020 arXiv   pre-print
Based on our findings we propose a new mathematical formulation of memorability decay, resulting in a model that is able to produce the first quantitative estimation of how a video decays in memory over  ...  Towards this goal, we develop a predictive model of human visual event memory and how those memories decay over time.  ...  We thank Zoya Bylinskii and Phillip Isola for their useful discussions and Alex Lascelles and Mathew Monfort for helping with the dataset.  ... 
arXiv:2009.02568v1 fatcat:canniflh4vfy5ee7dt277a7g3a

Leveraging Audio Gestalt to Predict Media Memorability [article]

Lorin Sweeney, Graham Healy, Alan F. Smeaton
2020 arXiv   pre-print
We used audio gestalt to estimate the influence of the audio modality on overall video memorability, and accordingly inform which combination of features would best predict a given video's memorability  ...  Our approach is a multimodal deep learning-based late fusion that combines visual, semantic, and auditory features.  ...  Predicting Media Memorability MediaEval'20, 14-15 December 2020, Online  ... 
arXiv:2012.15635v1 fatcat:imiqmgq6g5aote7brb6abjr5ce

Topic-Oriented Text Features Can Match Visual Deep Models of Video Memorability

Ricardo Kleinlein, Cristina Luna-Jiménez, David Arias-Cuadrado, Javier Ferreiros, Fernando Fernández-Martínez
2021 Applied Sciences  
Our results suggest that text descriptions expressed in natural language might be effective in embodying the visual semantics required to model video memorability.  ...  Recent studies have shown that the elements of an image, as well as their mutual semantic dependencies, provide a strong clue as to whether a video clip will be recalled on a second viewing or not.  ...  Conflicts of Interest: The authors declare no conflict of interest.  ... 
doi:10.3390/app11167406 fatcat:mltshpchxfbonmz3c5bcxjzkoa

M2P2: Multimodal Persuasion Prediction using Adaptive Fusion [article]

Chongyang Bai, Haipeng Chen, Srijan Kumar, Jure Leskovec, V.S. Subrahmanian
2021 arXiv   pre-print
Past studies on DOP fail to leverage two important aspects of multimodal data: 1) multiple modalities are often semantically aligned, and 2) different modalities may provide diverse information for prediction  ...  unimodal reference models.  ...  Past studies on DOP fail to leverage two important aspects of multimodal data: 1) multiple modalities are often semantically aligned, and 2) different modalities may provide diverse information for prediction  ... 
arXiv:2006.11405v2 fatcat:zxvsvxhjxzc2nl56wjd3snj2s4

Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts [article]

Julia Kruk, Jonah Lubin, Karan Sikka, Xiao Lin, Dan Jurafsky, Ajay Divakaran
2019 arXiv   pre-print
Computing author intent from multimodal data like Instagram posts requires modeling a complex relationship between text and image.  ...  meanings of the image and caption, and the semiotic relationship between the signified meanings of the image and caption.  ...  Disclaimer: The views, opinions, and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S  ... 
arXiv:1904.09073v3 fatcat:tgcqmlrpxfesrpmljhj7ka5j6i

TopicBERT: A Transformer transfer learning based memory-graph approach for multimodal streaming social media topic detection [article]

Meysam Asgari-Chenaghlu, Mohammad-Reza Feizi-Derakhshi, Leili farzinvash, Mohammad-Ali Balafar, Cina Motamed
2020 arXiv   pre-print
Transformer in one hand, provides the semantic relation between words in different contexts.  ...  Many of these researches are based on detection and tracking of hot topics and trending social media events that help revealing many unanswered questions.  ...  Overlearning effect is practicing memorization beyond required repetitions. This effect ensures that the memorized information will be more resistant to disruption or loss.  ... 
arXiv:2008.06877v1 fatcat:rri3j5hrgvfrxigwkn2543l334

MultiBench: Multiscale Benchmarks for Multimodal Representation Learning [article]

Paul Pu Liang, Yiwei Lyu, Xiang Fan, Zetian Wu, Yun Cheng, Jason Wu, Leslie Chen, Peter Wu, Michelle A. Lee, Yuke Zhu, Ruslan Salakhutdinov, Louis-Philippe Morency
2021 arXiv   pre-print
Therefore, MultiBench presents a milestone in unifying disjoint efforts in multimodal research and paves the way towards a better understanding of the capabilities and limitations of multimodal models,  ...  Simply applying methods proposed in different research areas can improve the state-of-the-art performance on 9/15 datasets.  ...  We believe that multimodal models are able to successfully rely on the other modality when one is  ... 
arXiv:2107.07502v2 fatcat:ls47dr7lpfhkbfry4r6dtqjtua

D2.1 Libraries and tools for multimodal content analysis

Doukhan; David, Danny Francis, Benoit Huet, Sami Keronen, Mikko Kurimo, Jorma Laaksonen, Tiina Lindh-Knuutila, Bernard Merialdo, Mats Sjöberg, Umut Sulubacak, Jörg Tiedemann, Kim Viljanen
2018 Zenodo  
This deliverable describes a joint collection of libraries and tools for multimodal content analysis created by the MeMAD project partners.  ...  As part of this deliverable, the open source components have been gathered into a joint software collection of tools and libraries publicly available on GitHub.  ...  1 Acknowledgements Computational resources were provided by the Aalto Science-IT project and the CSC -IT Center for Science, Finland.  ... 
doi:10.5281/zenodo.3697989 fatcat:bde5x3yggzb2jk2fh2mu6t5wxy

Modeling multimodal integration patterns and performance in seniors

Benfang Xiao, Rebecca Lunsford, Rachel Coulston, Matt Wesson, Sharon Oviatt
2003 Proceedings of the 5th international conference on Multimodal interfaces - ICMI '03  
The development of adaptive multimodal processing techniques will require empirical guidance from quantitative modeling on key aspects of individual differences, especially as users engage in different  ...  Multimodal interfaces are designed with a focus on flexibility, although very few currently are capable of adapting to major sources of user, task, or environmental variation.  ...  Thanks also to Jim Ann Carter and Lesley Carmichael for graphics and editing assistance, and to members of CHCC for many insightful discussions.  ... 
doi:10.1145/958432.958480 dblp:conf/icmi/XiaoLCWO03 fatcat:rwns2g7hjzfllk2lt4fpfhamau

Modeling multimodal integration patterns and performance in seniors

Benfang Xiao, Rebecca Lunsford, Rachel Coulston, Matt Wesson, Sharon Oviatt
2003 Proceedings of the 5th international conference on Multimodal interfaces - ICMI '03  
The development of adaptive multimodal processing techniques will require empirical guidance from quantitative modeling on key aspects of individual differences, especially as users engage in different  ...  Multimodal interfaces are designed with a focus on flexibility, although very few currently are capable of adapting to major sources of user, task, or environmental variation.  ...  Thanks also to Jim Ann Carter and Lesley Carmichael for graphics and editing assistance, and to members of CHCC for many insightful discussions.  ... 
doi:10.1145/958468.958480 fatcat:knszbs65efeltoh6c3ch4irn6m

Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts

Julia Kruk, Jonah Lubin, Karan Sikka, Xiao Lin, Dan Jurafsky, Ajay Divakaran
2019 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)  
Computing author intent from multimodal data like Instagram posts requires modeling a complex relationship between text and image.  ...  meanings of the image and caption, and the semiotic relationship between the signified meanings of the image and caption.  ...  Disclaimer: The views, opinions, and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S  ... 
doi:10.18653/v1/d19-1469 dblp:conf/emnlp/KrukLSLJD19 fatcat:phsgnay6gffmxcuskozymcn3om

Multimodal Events and Moving Locations: Eye Movements of Adults and 6-Month-Olds Reveal Dynamic Spatial Indexing

Daniel C. Richardson, Natasha Z. Kirkham
2004 Journal of experimental psychology. General  
Experiments 2 and 3 showed that infants are capable of both binding multimodal events to locations and tracking those locations when they move.  ...  The ability to keep track of locations in a dynamic, multimodal environment is crucial for successful interactions with other people and objects.  ...  One method participants could use was to look at the model area and memorize the pattern; each block in turn could be located in the resource area and then placed in the workspace.  ... 
doi:10.1037/0096-3445.133.1.46 pmid:14979751 fatcat:bia4u2r7n5gzvcx2i4tavfqe6i

Multimodal Approach of Speech Emotion Recognition Using Multi-Level Multi-Head Fusion Attention based Recurrent Neural Network

Ngoc-Huynh Ho, Hyung-Jeong Yang, Soo-Hyung Kim, Gueesang Lee
2020 IEEE Access  
Our experimental results on the three databases: Interactive Emotional Motion Capture (IEMOCAP), Multimodal EmotionLines Dataset (MELD), and CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI  ...  This paper presents a multimodal approach for speech emotion recognition based on Multi-Level Multi-Head Fusion Attention mechanism and recurrent neural network (RNN).  ...  Table 3 presents the performance of the four emotional classification on the Improvised and Mixed scenarios for the thee models: only audio, only text, and multimodal.  ... 
doi:10.1109/access.2020.2984368 fatcat:ghtmbc65brgqdki5kekxvqlvyy

A Virtual Agent as Vocabulary Trainer: Iconic Gestures Help to Improve Learners' Memory Performance [chapter]

Kirsten Bergmann, Manuela Macedonia
2013 Lecture Notes in Computer Science  
effects and long-term decay effects.  ...  Research has repeatedly demonstrated that performing iconic gestures together with novel words has a beneficial effect on learning performance.  ...  Acknowledgements This research is supported by the Deutsche Forschungsgemeinschaft (DFG) in the Collaborative Research Center 673 "Alignment in Communication" and the Center of Excellence 277 "Cognitive  ... 
doi:10.1007/978-3-642-40415-3_12 fatcat:2xlv55kg5rbsfktnos7in72l4a

Multimodal strategies of emotional governance: a critical analysis of 'nudge' tactics in health policy

Jane Mulderrig
2017 Critical Discourse Studies  
Employing multimodal critical discourse analysis (Machin and Mayr 2012; Machin and Mayr 2013) I analyse twenty six TV adverts broadcast since 2009 and identify three intersecting multimodal techniques:  ...  Drawing on the concept of 'governmentality', I assess what kinds of roles, relations and responsibilities for citizens and state this implies.  ...  To this end I draw on the model of multimodal critical discourse analysis proposed by Machin and Mayr (2012; Machin and Mayr 2013) .  ... 
doi:10.1080/17405904.2017.1382381 fatcat:rweoqr7bbva3rmpcego3nr4xfa
« Previous Showing results 1 — 15 out of 405 results