Display adaptive 3D content remapping

Belen Masia, Gordon Wetzstein, Carlos Aliaga, Ramesh Raskar, Diego Gutierrez
2013 Computers & graphics  
Glasses-free automultiscopic displays are on the verge of becoming a standard technology in consumer products. These displays are capable of producing the illusion of 3D content without the need of any additional eyewear. However, due to limitations in angular resolution, they can only show a limited depth of field, which translates into blurred-out areas whenever an object extrudes beyond a certain depth. Moreover, the blurring is device-specific, due to the different constraints of each
more » ... y. We introduce a novel display-adaptive light field retargeting method, to provide high-quality, blur-free viewing experiences of the same content on a variety of display types, ranging from hand-held devices to movie theaters. We pose the problem as an optimization, which aims at modifying the original light field so that the displayed content appears sharp while preserving the original perception of depth. In particular, we run the optimization on the central view and use warping to synthesize the rest of the light field. We validate our method using existing objective metrics for both image quality (blur) and perceived depth. The proposed framework can also be applied to retargeting disparities in stereoscopic image displays, supporting both dichotomous and non-dichotomous comfort zones. Within the last years, stereoscopic and automultiscopic dis-2 plays have started to enter the consumer market from all an-3 gles. These displays can show three-dimensional objects that 4 appear to be floating in front of or behind the physical screen, 5 even without the use of additional eyewear. Capable of elec-6 tronically switching between a full-resolution 2D and a lower-7 resolution 3D mode, parallax barrier technology [1] is dominant 8 for hand-held and tablet-sized devices, while medium-sized dis-9 plays most often employ arrays of microlenses [2]. Although 10 most cinema screens today are stereoscopic and rely on addi-11 tional eyewear, large-scale automultiscopic projection systems 12 are an emerging technology [3]. Each technology has its own 13 particular characteristics, including field of view, depth of field, 14 contrast, resolution, and screen size. Counterintuitively, pro-15 duced content is usually targeted toward a single display con-16 figuration, making labor-intense, manual post-processing of the 17 recorded or rendered data necessary. 18 19 Display-adaptive content retargeting is common practice for 20 attributes such as image size, dynamic range (tone mapping), 21 color gamut, and spatial resolution [4]. In order to counteract 22 3D objects extruding from the physical display enclosure ap-36 pear blurred out (see Figs. 1, left, and 2 for a real photograph 37 and a simulation showing the effect, respectively). We propose 38 here a framework that remaps the disparities in a 3D scene to 39 fit the DOF constraints of a target display by means of an opti-40 mization scheme that leverages perceptual models of the human 41 visual system. Our optimization approach runs on the central 42 view of an input light field and uses warping to synthesize the 43 rest of the views. 44 45 Contributions. Our nonlinear optimization framework for 46 3D content retargeting specifically provides the following con-47 tributions: 48 • We propose a solution to handle the intrinsic trade-off 49 between the spatial frequency that can be shown and the 50 perceived depth of a given scene. This is a fundamental 51 limitation of automultiscopic displays (see Section 3). 52 • We combine exact formulations of display-specific depth 53 of field limitations with models of human perception, to 54 find an optimized solution. In particular, we consider the 55 frequency-dependent sensitivity to contrast of the human 56 visual system, and the sensitivity to binocular disparity. 57 Based on this combination, a first objective term min-58 imizes the perceived luminance and contrast difference 59 between the original and the displayed scene, effectively 60 minimizing DOF blur, while a second term strives to pre-61 serve the perceived depth. 62 • We validate our results with existing state-of-the-art, ob-63 jective metrics for both image quality and perceived depth. 64 • We show how our framework can be easily extended to 65 113 lem of depth remapping of light field information to the specific 114 constraints of each display. 115 116 Generally speaking, content remapping is a standard ap-117 proach to adapt spatial and temporal resolution, contrast, col-118 ors, and sizes of images to a display having limited capabilities 119 in any of these dimensions [4]. For the particular case of dispar-120 ity remapping, Lang et al. [6] define a set of non-linear disparity 121 remapping operators, and propose a new stereoscopic warping 122 technique for the generation of the remapped stereo pairs. A 123 metric to assess the magnitude of perceived changes in binocu-124 lar disparity is introduced by Didyk et al. [8], who also inves-125 tigate the use of the Cornsweet illusion to enhance perceived 126 depth [24]. Recently, the original disparity metric has been fur-127 ther refined including the effect of luminance-contrast [9]. Kim 128 and colleagues [7] develop a a novel framework for flexible ma-129 nipulation of binocular parallax, where a new stereo pair is cre-130 ated from two non-linear cuts of the EPI volume corresponding 131 to multi-perspective images [25]. Inspired by Lang and col-132 leagues [6], they explore linear and non-linear global remap-133 ping functions, and also non-linear disparity gradient compres-134 sion. Here we focus on a remapping function that incorporates 135 the specific depth of field limitations of the target display [26]. 136
doi:10.1016/j.cag.2013.06.004 fatcat:6yn34hyfrzgyhnltkb5vwr7rou