Appearance-based face recognition and light-fields

R. Gross, I. Matthews, S. Baker
2004 IEEE Transactions on Pattern Analysis and Machine Intelligence  
Face recognition algorithms perform very unreliably when the pose of the probe face is different from the stored face. Unfortunately, this is a common situation where face recognition is required without the explicit co-operation of the subject. This underlying reason for the lack of performance is that typical feature vectors vary more with pose than with identity. There have been two types of approach to this problem: first, a full 3D head model can be estimated based on just one image, and
more » ... -rendered at all possible poses before searching the database [1]. Although this method produces good results, it is very computationally intensive, and currently too slow for practical use. Second, matching can be treated as a missing data problem -the single test and probe images are assumed to be parts of larger data vector containing the face viewed from all poses. The missing information is estimated from the visible data, using knowledge of the covariance structure. The complete vector is used for the matching process. This type of learning approach is considerably faster than constructing a full 3D model, but recognition performance is inferior. The emphasis in these algorithms is on creating a model which can predict how a given face will appear under different poses. Our algorithm takes a different approach. We aim to construct a single feature which does not vary with pose. This seems a natural formulation for a recognition task. Our approach is non-linear so that signal is preserved but the unwanted variation removed. We propose a non-linear many-to-one mapping from a conventional feature space to a new space constructed so that each individual has a unique feature vector regardless of pose. We model the converse process (in which the invariant vector generates the conventional feature vector) as a linear transform, in the presence of Gaussian noise. This transformation varies with pose. We learn the parameters for the linear transformation at each pose, as well as the noise parameters, using training data in which the pose is known. For a new feature face, we can now form a posterior distribution over the underlying pose-invariant vector. Recognition is performed by comparing these posterior distributions for the probe and test faces. We investigate the effect of the dimensionality of the invariant vector. We demonstrate that our system produces favorable results compared to contemporary methods. [1] V. Blanz, S. Romdhani and T. Vetter, "Face identification across different poses and illumination with a 3D morphable model, " Proc. 5
doi:10.1109/tpami.2004.1265861 pmid:15382650 fatcat:c4uwuts2xrfdjizpcmzohhm5a4