Fast video segment retrieval by Sort-Merge feature selection, boundary refinement, and lazy evaluation

Yan Liu, John R Kender
2003 Computer Vision and Image Understanding  
We present a fast video retrieval system with three novel characteristics. First, it exploits the methods of machine learning to construct automatically a hierarchy of small subsets of features that are progressively more useful for indexing. These subsets are induced by a new heuristic method called Sort-Merge feature selection, which exploits a novel combination of Fastmap for dimensionality reduction and Mahalanobis distance for likelihood determination. Second, because these induced feature
more » ... sets form a hierarchy with increasing classification accuracy, video segments can be segmented and categorized simultaneously in a coarse-fine manner that efficiently and progressively detects and refines their temporal boundaries. Third, the feature set hierarchy enables an efficient implementation of query systems by the approach of lazy evaluation, in which new queries are used to refine the retrieval index in real-time. We analyze the performance of these methods, and demonstrate them in the domain of a 75minute instructional video and a 30-minute baseball video.
doi:10.1016/j.cviu.2003.06.003 fatcat:xs2ln4dzwvcclilwwer2bob2pe