Open set face recognition using transduction

Fayin Li, H. Wechsler
2005 IEEE Transactions on Pattern Analysis and Machine Intelligence  
This paper motivates and describes a novel realization of transductive inference that can address the Open Set face recognition task. Open Set operates under the assumption that not all the test probes have mates in the gallery. It either detects the presence of some biometric signature within the gallery and finds its identity or rejects it, i.e., it provides for the "none of the above" answer. The main contribution of the paper is Open Set TCM -kNN (Transduction Confidence Machine -k Nearest
more » ... eighbors), which is suitable for multi-class authentication operational scenarios that have to include a rejection option for classes never enrolled in the gallery. Open Set TCM -kNN, driven by the relation between transduction and Kolmogorov complexity, provides a local estimation of the likelihood ratio needed for detection tasks. We provide extensive experimental data to show the feasibility, robustness, and comparative advantages of Open Set TCM -kNN on Open Set identification and watch list (surveillance) tasks using challenging FERET data. Last, we analyze the error structure driven by the fact that most of the errors in identification are due to a relatively small number of face patterns. Open Set TCM -kNN is shown to be suitable for PSEI (pattern specific error inhomogeneities) error analysis in order to identify difficult to recognize faces. PSEI analysis improves biometric performance by removing a small number of those difficult to recognize faces responsible for much of the original error in performance and/or by using data fusion. the choice is among several (tentative) classifications, each of them leading to different (re)partitionings of the original ID(entity) face space. The paper introduces the Open Set TCM -kNN (Transduction Confidence Machine -k Nearest Neighbors), which is a novel realization of transductive inference that is suitable for open set multi-class classification. Open Set TCM -kNN, driven by the relation between transduction and Kolmogorov complexity, provides a local estimation of the likelihood ratio required for detection tasks and provides for the needed 3 rejection option. The paper provides extensive experimental data to show the feasibility, robustness, and comparative advantages of Open Set TCM -kNN against existing methods using challenging FERET data. The outline for the paper is as follows. Sect. 2 familiarizes the reader with face recognition and the protocols used. Sects. 3 and 4 discuss the learning theory behind transduction. In particular, Sect. 4 describes the TCM (Transduction Confidence Machine) (Proedrou, 2001). Sect. 5 includes one of the main contributions of this paper, Open Set TCM-kNN. It expands on TCM and provides for a novel transductive algorithm that is suitable for the open set multi-class recognition problem that includes a rejection option. Sect. 6 describes the experimental design set up, while Sects. 7 and 8 describe the use of Open Set TCM-kNN for open set and watch list / surveillance biometric recognition tasks. Sects. 9 and 10 deal with error analysis and how to use the uneven contributions to errors made by face patterns for effective data fusion. The conclusions are presented in Sect. 11. Face Recognition Tasks and Performance Evaluation Protocols The generic (on-line) biometric system used herein is shown in Fig. 1 . The match component compares the biometric information extracted from the sample face exemplar and the signature stored in the reference (signature) template(s). One has to compare then an output score with a predefined threshold value. The comparison may be against a single template (for verification), or against a list of candidate templates (for identification). The face space, i.e., the basis needed to generate the templates, is derived using face images acquired ahead of time and independent of those that would be later on enrolled or tested (see top of Fig. 1 ). FERET (Phillips et al., 1998) and BANCA (Bailly-Bailliere et al., 2003) , the standard evaluation protocols in use today, are briefly described next. FERET undertakes algorithmic (technology) and scenario evaluations. It works by considering target (gallery) T and query
doi:10.1109/tpami.2005.224 pmid:16285369 fatcat:rtuxfrzm2zew3o63g4hracynha