Invertible motion blur in video

Amit Agrawal, Yi Xu, Ramesh Raskar
2009 ACM Transactions on Graphics  
We show that motion blur in successive video frames is invertible even if the point-spread function (PSF) due to motion smear in a single photo is non-invertible. Blurred photos exhibit nulls (zeros) in the frequency transform of the PSF, leading to an ill-posed de-convolution. Hardware solutions to avoid this require specialized devices such as the coded exposure camera or accelerating sensor motion. We employ ordinary video cameras and introduce the notion of null-filling along with
more » ... rtibility of multiple blur-functions. The key idea is to record the same object with varying PSFs, so that the nulls in the frequency component of one frame can be filled by other frames. The combined frequency transform becomes null-free, making deblurring wellposed. We achieve jointly invertible blur simply by changing the exposure time of successive frames. We address the problem of automatic deblurring of objects moving with constant velocity by solving the four critical components: preservation of all spatial frequencies, segmentation of moving parts, motion estimation of moving parts, and non-degradation of the static parts of the scene. We demonstrate several challenging cases of object motion blur including textured backgrounds and partial occluders. Siggraph 2009 This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Figure 1: By simply varying the exposure time for video frames, multi-image deblurring can be made invertible. (Left) Varying exposure photos of a moving car. Notice the change in illumination and the blur size in the captured photos. (Right) The foreground object is automatically rectified, segmented, deblurred, and composed onto the background using the varying exposure video. Novel renderings, such as motion streaks, can be generated by linear combination of the deblurred image and the blurred images. Abstract We show that motion blur in successive video frames is invertible even if the point-spread function (PSF) due to motion smear in a single photo is non-invertible. Blurred photos exhibit nulls (zeros) in the frequency transform of the PSF, leading to an ill-posed deconvolution. Hardware solutions to avoid this require specialized devices such as the coded exposure camera or accelerating sensor motion. We employ ordinary video cameras and introduce the notion of null-filling along with joint-invertibility of multiple blurfunctions. The key idea is to record the same object with varying PSFs, so that the nulls in the frequency component of one frame can be filled by other frames. The combined frequency transform becomes null-free, making deblurring well-posed. We achieve jointlyinvertible blur simply by changing the exposure time of successive frames. We address the problem of automatic deblurring of objects moving with constant velocity by solving the four critical components: preservation of all spatial frequencies, segmentation of moving parts, motion estimation of moving parts, and non-degradation of the static parts of the scene. We demonstrate several challenging cases of object motion blur including textured backgrounds and partial occluders.
doi:10.1145/1531326.1531401 fatcat:hfw3onivdresxhkq6gl5glwbta