Object Category Detection Using Audio-Visual Cues [chapter]

Jie Luo, Barbara Caputo, Alon Zweig, Jörg-Hendrik Bach, Jörn Anemüller
Lecture Notes in Computer Science  
Categorization is one of the fundamental building blocks of cognitive systems. Object categorization has traditionally been addressed in the vision domain, even though cognitive agents are intrinsically multimodal. Indeed, biological systems combine several modalities in order to achieve robust categorization. In this paper we propose a multimodal approach to object category detection, using audio and visual information. The auditory channel is modeled on biologically motivated spectral
more » ... via a discriminative classifier. The visual channel is modeled by a state of the art part based model. Multimodality is achieved using two fusion schemes, one high level and the other low level. Experiments on six different object categories, under increasingly difficult conditions, show strengths and weaknesses of the two approaches, and clearly underline the open challenges for multimodal category detection. IDIAP-RR 07-58 3
doi:10.1007/978-3-540-79547-6_52 dblp:conf/icvs/LuoCZBA08 fatcat:bfophp746vdn5bslyciaqp3lu4