Multiscale modeling and estimation of motion fields for video coding

P. Moulin, R. Krishnamurthy, J.W. Woods
1997 IEEE Transactions on Image Processing  
We present a systematic approach to forward{motion{compensated predictive video coding. The rst step is the de nition of a exible model that compactly represents motion elds. The inhomogeneity and spatial coherence properties of motion elds are captured using linear multiscale models. One possible design is based on linear nite elements and yields a multiscale extension of the Triangle Motion Compensation (TMC) method. The second step is the choice of a computational technique that identi es
more » ... coe cients of the linear model. We study a modi ed optical ow technique and minimize a cost function closely related to Horn and Schunck's criterion. The cost function balances accuracy and complexity of the motion{compensated predictor and is viewed as a measure of goodness of the motion eld. It determines not only the coe cients of the model, but also the quantization method. We formulate the estimation and quantization problems jointly as a discrete optimization problem and solve it using a fast multiscale relaxation algorithm. A hierarchical extension of the algorithm allows proper handling of large displacements. Simulations on a variety of video sequences have produced improvements over TMC and over the half{pel{accuracy, full{search block matching algorithm, in excess of 0.5 dB in average. The results are visually superior as well. In particular, the reconstructed video is entirely free of blocking artifacts. This work has been presented in part at ISCAS'93, IMDSP'93 and ICIP'95. y Work partly performed while P. Moulin was with Bell Communications Research in Morristown, NJ. z Work partly performed during an internship at Bell Communications Research in Morristown, NJ.
doi:10.1109/83.650115 pmid:18285232 fatcat:bz47zzejhjcj7nzv6rt66b4mka