A Robust Passage Retrieval Algorithm for Video Question Answering

Yu-Chieh Wu, Jie-Chi Yang
2008 IEEE transactions on circuits and systems for video technology (Print)  
In this paper, we present a robust passage retrieval algorithm to extend the conventional text question answering (Q/A) to videos. Users interact with our videoQ/A system through natural language queries, while the top-ranked passage fragments with associated video clips are returned as answers. We compare our method with five of the high-performance ranking algorithms that are portable to different languages and domains. The experiments were evaluated with 75.3 h of Chinese videos and 253
more » ... ions. The experimental results showed that our method outperformed the second best retrieval model (language models) in relatively 1.43% in mean reciprocal rank (MRR) score and 11.36% when employing a Chinese word segmentation tool. By adopting the initial retrieval results from the retrieval models, our method yields an improvement of at least 5.94% improvement in MRR score. This makes it very attractive for the Asia-like languages since the use of a well-developed word tokenizer is unnecessary. Index Terms-Multimedia retrieval, question answering (Q/A), video question answering (videoQ/A).
doi:10.1109/tcsvt.2008.2002831 fatcat:cffpxmodh5b2tau7ga4iovxho4