Content-Based Video Search over 1 Million Videos with 1 Core in 1 Second

Shoou-I Yu, Lu Jiang, Zhongwen Xu, Yi Yang, Alexander G. Hauptmann
2015 Proceedings of the 5th ACM on International Conference on Multimedia Retrieval - ICMR '15  
Many content-based video search (CBVS) systems have been proposed to analyze the rapidly-increasing amount of user-generated videos on the Internet. Though the accuracy of CBVS systems have drastically improved, these high accuracy systems tend to be too inefficient for interactive search. Therefore, to strive for realtime web-scale CBVS, we perform a comprehensive study on the different components in a CBVS system to understand the tradeoffs between accuracy and speed of each component.
more » ... ons investigated include exploring different low-level and semanticsbased features, testing different compression factors and approximations during video search, and understanding the time v.s. accuracy trade-off of reranking. Extensive experiments on data sets consisting of more than 1,000 hours of video showed that through a combination of effective features, highly compressed representations, and one iteration of reranking, our proposed system can achieve an 10,000-fold speedup while retaining 80% accuracy of a state-of-the-art CBVS system. We further performed search over 1 million videos and demonstrated that our system can complete the search in 0.975 seconds with a single core, which potentially opens the door to interactive web-scale CBVS for the general public.
doi:10.1145/2671188.2749398 dblp:conf/mir/YuJXYH15 fatcat:p2ios43a75anjjkuswkrrfuk44