### On Weighting and Choosing Constraints for Optimally Reconstructing the Geometry of Image Triplets [chapter]

Wolfgang Förstner
2000 Lecture Notes in Computer Science
Optimally reconstructing the geometry of image triplets from point correspondences requires a proper weighting or selection of the used constraints between observed coordinates and unknown parameters. By analysing the ML-estimation process the paper solves a set of yet unsolved problems: (1) The minimal set of four linearily independent trilinearities (Shashua 1995 , Hartley 1995) actually imposes only three constraints onto the geometry of the image triplet. The seeming contradiction between
more » ... e number of used constraints, three vs. four, can be explained naturally using the normal equations. (2) Direct application of such an estimation suggests a pseudoinverse of a 4 × 4-matix having rank 3 which contains the covariance matrix of the homologeous image points to be the optimal weight matrix. (3) Instead of using this singluar weight matrix one could select three linearily dependent constraints. This is discussed for the two classical cases of forward and lateral motion, and clarifies the algebraic analyis of dependencies between trilinear constraints by Faugeras 1995. Results of an image sequence with 800 images and an Euclidean parametrization of the trifocal tensor demonstrate the feasibility of the approach. Motivation and Problem Image triplets reveal quite some advantage over image pairs for geometric image analysis. Though the geometry of the image triplet is studied quite well, implementing an optimal estimation procedure for recovering the orientation and calibration of the three images from point, and possibly line, correspondencies still has to cope with a number of problems. The Task This paper discusses the role of the trilinear constraints between observed coordinates and unknown parameters [12, 13, 2, 8, 16] within an optimal estimation process for the orientation of the image triplet and shows an application within image sequence analysis. The task formally can be described as following. We assume to have observed J sets (P (x , y ), P (x , y ), P (x , y