View Synthesis Prediction in the 3-D Video Coding Extensions of AVC and HEVC

Feng Zou, Dong Tian, Anthony Vetro, Huifang Sun, Oscar C. Au, Shinya Shimizu
2014 IEEE transactions on circuits and systems for video technology (Print)  
Advanced multiview video systems are able to generate intermediate viewpoints of a 3D scene. To enable low complexity free view generation, texture and its associated depth are used as input data for each viewpoint. To improve the coding efficiency of such content, view synthesis prediction (VSP) is proposed to further reduce inter-view redundancy in addition to traditional disparity compensated prediction (DCP). This paper describes and analyzes rate-distortion optimized VSP designs, which
more » ... adopted in the 3D extensions of both AVC and HEVC. In particular, we propose a novel backward-VSP scheme using a derived disparity vector, as well as efficient signaling methods in the context of AVC and HEVC. Additionally, we put forward a novel depth-assisted motion vector prediction method to optimize the coding efficiency. A thorough analysis of coding performance is provided using different VSP schemes and configurations. Experimental results demonstrate average bit rate reductions of 2.5% and 1.2% in AVC and HEVC coding frameworks, respectively, with up to 23.1% bit rate reduction for dependent views. IEEE Transactions on Circuits and Systems for Video Technology This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Abstract-Advanced multiview video systems are able to generate intermediate viewpoints of a 3D scene. To enable low complexity free view generation, texture and its associated depth are used as input data for each viewpoint. To improve the coding efficiency of such content, view synthesis prediction (VSP) is proposed to further reduce inter-view redundancy in addition to traditional disparity compensated prediction (DCP). This paper describes and analyzes rate-distortion optimized VSP designs, which were adopted in the 3D extensions of both AVC and HEVC. In particular, we propose a novel backward-VSP scheme using a derived disparity vector, as well as efficient signalling methods in the context of AVC and HEVC. Additionally, we put forward a novel depth-assisted motion vector prediction method to optimize the coding efficiency. A thorough analysis of coding performance is provided using different VSP schemes and configurations. Experimental results demonstrate average bit rate reductions of 2.5% and 1.2% in AVC and HEVC coding frameworks, respectively, with up to 23.1% bit rate reduction for dependent views. Index Terms-3D, video coding, depth, view synthesis prediction
doi:10.1109/tcsvt.2014.2313891 fatcat:bjiiw3skhfg4neq57ebfhbld74