Model-based 2D&3D dominant motion estimation for mosaicing and video representation

H.S. Sawhney, S. Ayer, M. Gorkani
Proceedings of IEEE International Conference on Computer Vision  
It is fairly common in video sequences that a mostly xed background (scene) is imaged with or without independently moving objects. The dominant background changes in the image plane mostly due to camera operations and motion (zoom, pan, tilt, track etc.). In this paper we address the problem of computation of the dominant image transformation over time and demonstrate how this can be e ectively used for e cient video representation through video mosaicing and image registration. We formulate
more » ... e problem of dominant component estimation as that of model-based robust estimation using M-estimators with direct, multi-resolution methods. In addition to 2D a ne and plane projective models, that have been used in the past, for describing image motion using direct methods, we also employ a true 3D model of motion and scene structure imaged with uncalibrated cameras. This model parameterizes the image motion as that due to a planar component and a parallax component. For rigid 3D scenes imaged under camera motion only, least squares (LS) methods with the plane and parallax parameterization are also presented. Furthermore, in the context of robust estimation, in contrast with previous approaches for similar problems, our algorithm employs an automatic computation of a scale parameter that is crucial in rejecting the non-dominant components as outliers. Results of dominant model approach for separation of the background from moving objects are presented. Also the results of registration using 3D models with LS methods are shown. Furthermore, both the computed 2D and 3D models are used to demonstrate multi-frame registration, and to create a compact representation of a video sequence as a single expanded-view frame using image mosaicing. This representation could be useful for video coding as well as indexing based on the created representative frame. 2
doi:10.1109/iccv.1995.466886 dblp:conf/iccv/SawhneyAG95 fatcat:poe5bj4hybflzksujqoxstkg2u