Signal Processing for Stereoscopic and Multi-View 3D Displays [chapter]

Atanas Boev, Robert Bregovic, Atanas Gotchev
2013 Handbook of Signal Processing Systems  
Displays which aim at visualizing 3D scenes with realistic depth are known as "3D displays". Due to technical limitations and design decisions, such displays might create visible distortions, which are interpreted by the human visual system as artifacts. This book chapter overviews a number of signal processing techniques for decreasing the visibility of artifacts on 3D displays. It begins by identifying the properties of a scene which the brain utilizes for perceiving depth. Further, operation
more » ... principles of the most popular types of 3D displays are explained. A signal processing channel is proposed as a general model reflecting these principles. The model is applied in analyzing how visual quality is influenced by display distortions. The analysis allows identifying a set of optical properties which are directly related with the perceived quality. A methodology for measuring these properties and creating a quality profile of a 3D display is discussed. A comparative study introducing the measurement results on the visual quality and position of the sweet spots of a number of 3D displays of different types is presented. Based on knowledge of 3D artifact visibility and understanding of distortions introduced by 3D displays, a number of signal processing techniques for artifact mitigation are overviewed. These include a methodology for passband optimization which addresses typical 3D display artifacts (e.g. Moiré, fixedpattern-noise and ghosting), a framework for design of tunable anti-aliasing filters and a set of real-time algorithms for view-point based optimization. An ideal 3D display would attempt creating a light field being a perfect visual replica of a 3D scene. Such a replica, however, would also include components which are not visible to human eyes. These components can be considered redundant and can be omitted from the scene representation. The result is a visuallyindistinguishable replica of the scene. Furthermore, the typical display use case does not require the scene to react to external light sources or to allow the observ-A. Boev, R. Bregović, and A. Gotchev, "Signal processing for stereoscopic and multi-view 3D displays," chapter in Handbook of signal processing systems, 2nd edition, edited by S. Bhattacharyya, E. Deprettere, R. Leupers, and J. Takala. Springer 2013, pp. 3-47. (final submitted version) er to walk through object in the scene. Thus, some visual information (e.g. light distribution within scene objects) is unnecessary. Removing this information produces a redundancy-free replica of the scene. In a typical use case a redundancyfree replica is also a visually indistinguishable representation of the scene under the use case constraints. Failure in creating redundancy-free and visuallyindistinguishable replica leads to visible distortions. In order to avoid this one needs to know which light properties are important and which scene features are relevant for perceiving the scene in 3D. Visual Perception of Depth Vision in general can be separated into two parts -visual perception and visual cognition. In studies of human vision, visual perception and properties of early vision are subjects of anatomy and neurophysiology [1, p. 2] [2], and visual cognition, as a higher level brain function, is a subject of psychology [1, p. 387] [3] . Visual perception involves a number of optical and neural transformations. The eye can change its refractive power in order to focus on objects at various distances. The process is known as accommodation and the refractive power is measured in diopters. The light entering the eye is focused onto the retina which contains photosensitive receptors tuned to various spectral components (frequencies). The density of the photoreceptors has its maximum close to the optical center of the eye. The area with the highest photoreceptor density is known as the fovea. There are four types of photoreceptor cells -rods, L-cones, M-cones and S-coneswhich allow detection of light with wavelengths between 370 and 730nm. The cones can be thought of (to a crude approximation) as sensitive to red, green and blue color components of the light. The rods are responsible for the low-light vision and are generally ignored in HVS modeling. Rather than perceiving continuous spectrum, the HVS encodes the color information as a combination of three color components; the process is known as color perception. The combination of the iris controlling the amount of light entering the eye, and the sensitivity adaptation of the retina allow the eye to work over a wide range of intensities (between 10 ି and 10 ଼ cd/m 2 ). The eye is sensitive to luminance difference (i.e. contrast) rather than absolute luminance values. This visual property is known as light adaptation. However, the HVS has different contrast sensitivity for patterns with different density and orientation [1] . The ability to perceive visual information through two distinctive eyes is known as binocular vision. The eyes of a human are separated horizontally and have distance between pupils (also known as interpupilar distance, IPD) of approximately 65 mm on average [2] . Such positioning allows each eye to perceive the world from a different perspective, as shown in Fig. 1 . The luminance, color and contrast perception occur in each eye separately and the visual information is fed through the optical nerve to the so-called lateral geniculate nucleus (LGN) [1]. The LGN de-correlates the binocular information and produces a single, fused representation of the scene. The fused image appears as if observed from a point A. Boev, R. Bregović, and A. Gotchev, "Signal processing for stereoscopic and multi-view 3D displays," chapter in Handbook of signal processing systems, 2nd edition, edited by S. Bhattacharyya, E. Deprettere, R. Leupers, and J. Takala. Springer 2013, pp. 3-47. (final submitted version)
doi:10.1007/978-1-4614-6859-2_1 fatcat:46yllwirivaklfkme7ivocethu