Reference frame selection for loss-resilient depth map coding in multiview video conferencing

Bruno Macchiavello, Camilo Dorea, Edson M. Hung, Gene Cheung, Wai-Tian Tan, Robert L. Stevenson, Amir Said, Onur G. Guleryuz
2012 Visual Information Processing and Communication III  
Multiview video in "texture-plus-depth" format enables decoder to synthesize freely chosen intermediate views for enhanced visual experience. Nevertheless, transmission of multiple texture and depth maps over bandwidthconstrained and loss-prone networks is challenging, especially for conferencing applications with stringent deadlines. In this paper, we examine the problem of loss-resilient coding of depth maps by exploiting two observations. First, different depth macroblocks have significantly
more » ... have significantly different error sensitivities with respect to the reconstructed images. Second, unlike texture, the relative overhead of using reference pictures with large prediction distance is low for depth maps. This motivates our approach of assigning a weight to represent the varying error sensitivity of each macroblock and using such weights to guide selection of reference frames. Results show that (1) errors in depth maps in sequence with high motion yields significant drop in quality in reconstructed images, and (2) that the proposed scheme can efficiently maintain the quality of reconstructed images even at relatively high packet loss rates of 3-5%. Keywords: Multiview video, depth-image-based rendering, loss resiliency * Depth maps can be estimated from texture maps using stereo matching algorithms, or captured explicitly using time-of-flight cameras.
doi:10.1117/12.909462 dblp:conf/vipc/MacchiavelloDHC12 fatcat:scodmsec55b5rguooj52c5236e