Steered Mixture-of-Experts Approximation of Spherical Image Data

Ruben Verhack, Nilesh Madhu, Glenn Van Wallendael, Peter Lambert, Thomas Sikora
2018 2018 26th European Signal Processing Conference (EUSIPCO)  
Steered Mixture-of-Experts (SMoE) is a novel framework for approximating multidimensional image modalities. Our goal is to provide full Six Degrees-of-Freedom capabilities for camera captured content. Previous research concerned only limited translational movement for which the 4D light field representation is sufficient. However, our goal is to arrive at a representation that allows for unlimited translational-rotational freedom, i.e. our goal is to approximate the full 5D plenoptic function.
more » ... lenoptic function. Until now, SMoE was only applied on Euclidean spaces. However, the plenoptic function contains two spherical coordinate dimensions. In this paper, we propose a methodology to extend the SMoE framework to spherical dimensions. Furthermore, we propose a method to reduce the parameter space to the same two dimensional Euclidean space as for planar 2D images by using a projection of the covariance matrices onto tangent spaces perpendicular to the unit sphere. Finally, we propose a novel training technique for spherical dimensions based on these observations. Experiments performed on omnidirectional 360°images show that the introduction of the dimensionalityreduction projection step results in very low quality loss.
doi:10.23919/eusipco.2018.8553065 dblp:conf/eusipco/VerhackMWLS18 fatcat:c36irer2e5fpzcbamdg55viuja