2,698 Hits in 18.1 sec

Fusion of Multimodal Information in Music Content Analysis *

Slim Essid, Gaël Richard
Music is often processed through its acoustic realization.  ...  Part of this work was also conducted with the support from the European Commission with the 3Dlife Network of Excellence 8 .  ...  Acknowledgments This article is largely based on the works of several students mostly from Telecom ParisTech.  ... 

Music Emotion Research Based on Reinforcement Learning and Multimodal Information

Yue Hu, Naeem Jan
2022 Journal of Mathematics  
In this paper, a multimodal fusion algorithm for music emotion analysis is proposed, and a dynamic model based on reinforcement learning is constructed to improve the analysis accuracy.  ...  Music is an important carrier of emotion and an indispensable factor in people's daily life.  ...  Music Emotion Feature Analysis Method Based on Multimodal Fusion Multimodal information fusion is an information processing process that comprehensively utilizes natural language processing, semantic analysis  ... 
doi:10.1155/2022/2446399 fatcat:74fl266uenccno7p4rrdwhxure

Table of Contents

2019 IEEE transactions on multimedia  
Porikli 510 Speech, Language, and Audio in Video Analysis; Music in Multimedia Creating a Multitrack Classical Music Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applications  ...  Su 363 Multimodal Perception, Integration, and Multisensory Fusion The Labeled Multiple Canonical Correlation Analysis for Information Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  ... 
doi:10.1109/tmm.2019.2892660 fatcat:35egnbnnofbkpnmrujdvtpjj7m

Query-Document-Dependent Fusion: A Case Study of Multimodal Music Retrieval

Zhonghua Li, Bingjun Zhang, Yi Yu, Jialie Shen, Ye Wang
2013 IEEE transactions on multimedia  
We thus propose a general multimodal fusion framework, query-document-dependent fusion (QDDF), which derives the optimal fusion strategy for each query-document pair via intelligent content analysis of  ...  We found that document-dependent weights are instrumental in enhancing multimedia fusion performance. In addition, efficiency analysis demonstrates the scalability of QDDF over large data sets.  ...  ., metadata, content), multimodal fusion, which combines multiple complementary modalities, has become an effective approach to boost information retrieval performance.  ... 
doi:10.1109/tmm.2013.2280437 fatcat:7io6qmo4bre5vagrtaaap7lgcm

Fusion of electroencephalographic dynamics and musical contents for estimating emotional responses in music listening

Yuan-Pin Lin, Yi-Hsuan Yang, Tzyy-Ping Jung
2014 Frontiers in Neuroscience  
This study aimed to assess the applicability of a multimodal approach by leveraging the EEG dynamics and acoustic characteristics of musical contents for the classification of emotional valence and arousal  ...  The present study not only provided principles for constructing an EEG-based multimodal approach, but also revealed the fundamental insights into the interplay of the brain activity and musical contents  ...  Table 4 4 | The informative musical features in the subject-independent multimodal approach.  ... 
doi:10.3389/fnins.2014.00094 pmid:24822035 pmcid:PMC4013455 fatcat:rahfs7a5dfguxptjoq3k7hluty

Document dependent fusion in multimodal music retrieval

Zhonghua Li, Bingjun Zhang, Ye Wang
2011 Proceedings of the 19th ACM international conference on Multimedia - MM '11  
In this paper, we propose a novel multimodal fusion framework, document dependent fusion (DDF), which derives the optimal combination strategy for each individual document in the fusion process.  ...  Experiments are conducted on a 17174-song music database to compare the retrieval accuracy of traditional query independent fusion and query dependent fusion approaches, and that obtained after integrating  ...  ., music, images, text documents) contain information or cues in different modalities, multimodal fusion, which aims to combine these modalities to better meet users' information needs, has been regarded  ... 
doi:10.1145/2072298.2071949 dblp:conf/mm/LiZW11 fatcat:gp2dheuw5nfghjt5cnprphes2a

Multimodal music information processing and retrieval: survey and future challenges [article]

Federico Simonetta, Stavros Ntalampiras, Federico Avanzini
2019 arXiv   pre-print
Subsequently, we analyze existing information fusion approaches, and we conclude with the set of challenges that Music Information Retrieval and Sound and Music Computing research communities should focus  ...  Towards improving the performance in various music information processing tasks, recent studies exploit different modalities able to capture diverse aspects of music.  ...  In this paper, we review the existing literature about Music Information Retrieval techniques which exploit multiple descriptions of music to the end of multimodal fusion [12] .  ... 
arXiv:1902.05347v1 fatcat:i2indkxk3vcmxajn6ajkh56wva

Multimodal Music Processing (Dagstuhl Seminar 11041)

Meinard Müller, Masataka Goto, Simon Dixon, Marc Herbstritt
2011 Dagstuhl Reports  
List of Authors  ...  of research in multimodal music processing and music information retrieval.  ...  Essid and Richard first give an overview of general fusion principles and then discuss various case studies that highlight how video, acoustic, and sensor information can be fused in an integrated analysis  ... 
doi:10.4230/dagrep.1.1.68 dblp:journals/dagstuhl-reports/MullerGD11 fatcat:qpuvecey6jhgdmp344uujkmcja

Affective Computing for Large-scale Heterogeneous Multimedia Data

Sicheng Zhao, Shangfei Wang, Mohammad Soleymani, Dhiraj Joshi, Qiang Ji
2019 ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP)  
The wide popularity of digital photography and social networks has generated a rapidly growing volume of multimedia data (i.e., image, music, and video), resulting in a great demand for managing, retrieving  ...  We then summarize and compare the representative methods on AC of different multimedia types, i.e., images, music, videos, and multimodal data, with the focus on both handcrafted features-based methods  ...  Data Fusion In total, there are two fusion strategies for multimodal information: feature-level fusion and decisionlevel fusion.  ... 
doi:10.1145/3363560 fatcat:m56udtjlxrauvmj6d5z2r2zdeu

Multimodal Content Representation and Similarity Ranking of Movies [article]

Konstantinos Bougiatiotis, Theodore Giannakopoulos
2017 arXiv   pre-print
We assert movie similarities, as propagated by the singular modalities and fusion models, in the form of recommendation rankings.  ...  In this paper we examine the existence of correlation between movie similarity and low level features from respective movie content.  ...  Audio Analysis The audio signal is a very important channel of information with regards to a movie's content: music tracks, musical background themes, sound effects, speech, sound events, they all play  ... 
arXiv:1702.04815v2 fatcat:fwhcj6af3fb5veabr672uzb7lq

Multimodal Music Emotion Recognition Method Based on the Combination of Knowledge Distillation and Transfer Learning

Guiying Tong, Baiyuan Ding
2022 Scientific Programming  
This paper proposes a multimodal method based on the combination of knowledge distillation and music style transfer learning and verifies the effectiveness of the method on 20,000 songs.  ...  Experiments show that compared with traditional methods, such as single audio, single lyric, and single audio with multimodal lyric methods, the method proposed in this paper has significantly improved  ...  Acknowledgments is work was supported by the research project on 2021 teaching quality and reform of Huizhou University, name: Research on Guzheng Teaching Mode Reform Based on Application-Oriented Talent  ... 
doi:10.1155/2022/2802573 fatcat:l2k2cern7rdi7gn55vsa25mriy

Musical Instrument Recognition in User-generated Videos using a Multimodal Convolutional Neural Network Architecture

Olga Slizovskaia, Emilia Gómez, Gloria Haro
2017 Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval - ICMR '17  
Our approach does not only focus on the analysis of audio information, but we exploit the multimodal information embedded in the audio and visual domains.  ...  Musical instrument recognition from music signals is a well-known task in the music information retrieval (MIR) eld, where current approaches rely on the analysis of the good-quality audio material. is  ...  While the most important information comes from audio, visual content also plays an important role in detecting musical instruments in videos. us, di erent taxonomies for musical instruments rely on audio  ... 
doi:10.1145/3078971.3079002 dblp:conf/mir/SlizovskaiaGH17 fatcat:obngh3odxzai7jhwoslsiedh7a

Enhanced Movie Content Similarity Based on Textual, Auditory and Visual Information [article]

Konstantinos Bougiatiotis, Theodore Giannakopoulos
2017 arXiv   pre-print
In this paper we examine the ability of low-level multimodal features to extract movie similarity, in the context of a content-based movie recommendation approach.  ...  In particular, we demonstrate the extraction of multimodal representation models of movies, based on textual information from subtitles, as well as cues from the audio and visual channels.  ...  Audio Analysis The audio signal is a very important channel of information with regards to a movie's content: music tracks, musical background themes, sound effects, speech, acoustic events, they all play  ... 
arXiv:1711.03889v1 fatcat:smgf6bbvuvaejauk64tgemj3iq

Audio-Textual Classification System Design for Arabic News Videos

Amal Dandashi, Jihad Jihad Al Ja'am, Sebti Foufou
2016 International Journal of Computing and Information Sciences  
We propose a system design that implements multimodal video classification such that annotations and caption processing is excluded. The domain targetted is the news domain.  ...  The majority of work aimed at classficiation of Arabic videos is based on textual annotation or closed caption text extraction and processing.  ...  The findings achieved herein are solely the responsibility of the authors.  ... 
doi:10.21700/ijcis.2016.124 fatcat:jgid6kup7jh5xdt472l3gzualy

Music-Video Emotion Analysis Using Late Fusion of Multimodal

Yagya Raj PANDEYA, Joonwhoan LEE
2019 DEStech Transactions on Computer Science and Engineering  
We also use late fusion to fuse the learned features of audio and video network. The proposed network performs better for music-video emotion analysis.  ...  In this research we make music-video emotion dataset and extract music and video features from pre-trained neural networks.  ...  The various music-video emotion expresses various high-level semantics of human moods. This makes the automatic music-video emotion content analysis highly challenging for computer.  ... 
doi:10.12783/dtcse/iteee2019/28738 fatcat:r5wgfksvmzeldeqtlzm3jqcrti
« Previous Showing results 1 — 15 out of 2,698 results