Automatic recognition of eye blinking in spontaneously occurring behavior

Jeffrey F. Cohn, Jing Xiao, Tsuyoshi Moriyama, Zara Ambadar, Takeo Kanade
2003 Behavoir research methods, instruments & computers  
Previous research in automatic facial expression recognition has been limited to recognition of gross expression categories (e.g., joy or anger) in posed facial behavior under well-controlled conditions (e.g., frontal pose and minimal out-of-plane head motion). We have developed a system that detects discrete and important facial actions, (e.g., eye blinking), in spontaneously occurring facial behavior with non-frontal pose, moderate out-of-plane head motion, and occlusion. The system recovers
more » ... D motion parameters, stabilizes facial regions, extracts motion and appearance information, and recognizes discrete facial actions in spontaneous facial behavior. We tested the system in video data from a 2-person interview. Subjects were ethnically diverse, action units occurred during speech, and out-of-plane motion and occlusion from head motion and glasses were common. The video data were originally collected to answer substantive questions in psychology, and represent a substantial challenge to automated AU recognition. In analysis of 335 single and multiple blinks and non-blinks, the system achieved 98% accuracy.
doi:10.3758/bf03195519 pmid:14587550 fatcat:7qdygif5qvabxktrqyexysa5aq