Multimedia Search Without Visual Analysis: The Value of Linguistic and Contextual Information

Franciska M. G. de Jong, Thijs Westerveld, Arjen P. de Vries
2007 IEEE transactions on circuits and systems for video technology (Print)  
This paper addresses the focus of this special issue by analyzing the potential contribution of linguistic content and other nonimage aspects to the processing of audiovisual data. It summarizes the various ways in which linguistic content analysis contributes to enhancing the semantic annotation of multimedia content, and, as a consequence, to improving the effectiveness of conceptual media access tools. A number of techniques are presented, including the time-alignment of textual resources,
more » ... dio and speech processing, content reduction and reasoning tools, and the exploitation of surface features. Index Terms-Context, language processing, multimedia search, semantic metadata, speech and audio analysis, surface features. 1051-8215/$25.00 © 2007 IEEE Arjen P. de Vries received the Ph.D. degree in computer science from the University of Twente, Enschede, The Netherlands, in 1999, for his work on the integration of content management in database systems. He is especially interested in the design of database systems that support search in multimedia digital libraries. He has worked on a variety of research topics, including (multimedia) information retrieval, database architecture, query processing, retrieval system evaluation, and ambient intelligence. He works as a postdoctoral researcher at
doi:10.1109/tcsvt.2007.890834 fatcat:fiybaunbjzdvddksqewudk7n7m