Context-Aware Person Identification in Personal Photo Collections

Neil O'Hare, Alan F. Smeaton
2009 IEEE transactions on multimedia  
Identifying the people in photos is an important need for users of photo management systems. We present MediAssist, one such system which facilitates browsing, searching and semi-automatic annotation of personal photos, using analysis of both image content and the context in which the photo is captured. This semiautomatic annotation includes annotation of the identity of people in photos. In this paper, we focus on such person annotation, and propose person identification techniques based on a
more » ... ombination of context and content. We propose language modelling and nearest neighbor approaches to context-based person identification, in addition to novel face color and image color content-based features (used alongside face recognition and body patch features). We conduct a comprehensive empirical study of these techniques using the real private photo collections of a number of users, and show that combining context-and content-based analysis improves performance over content or context alone. Index Terms-Context and content, person identification, personal photo management. Neil O'Hare is a Postdoctoral Researcher in the Centre for Digital Video Processing in Dublin City University, Dublin, Ireland. His research interests are in the area of multimedia information management, in particular the management of personal photos using a combination of context-and content-based analysis. Alan Smeaton (M'05) is a Professor of Computing at Dublin City University, Dublin, Ireland, where he is the Director of the Centre for Digital Video Processing and Deputy Director of the CLARITY CSET. His research interests cover information management of all kinds, including managing video archives and managing information gathered from sensor networks.
doi:10.1109/tmm.2008.2009679 fatcat:k45tkigy6fekrfk5mbn2byte64