Description In this video, we present our latest work on facial expression analysis, synthesis and face recognition [3, 5, 4, 1, 2] . The advent of new technologies that allow the capture of massive amounts of high resolution, high frame rate face data, leads us to propose data-driven face models that accurately describe the appearance of faces under unknown pose and illumination conditions as well as to track subtle geometry changes that occur during expressions. To start with, high quality

more »

... se point clouds of nonrigid geometry moving at video speeds are acquired using a phase-shifting based structure light ranging technique. We use a generic face mesh model to register different dynamic face scans for expression analysis purposes. We have developed two tracking methods for such data. The first one is a hierarchical scheme where a generic face mesh is first deformed to fit the data at a coarse level. Then using a Free-Form-Deformation based shape-registration algorithm, the fitting is refined, thereby establishing dense correspondences between face scans. A recent, more automatic tracking method uses harmonic maps with interior feature correspondence constraints. As an example, on the "big smile" expression for a female subject, our method provides highly accurate facial expression tracking, even in the presence of topology changes and large head motion. The close-up view displays how our method tracks even subtleties very accurately. To provide a measure for the accuracy of our tracking method, we compare the re-rendered face model with the original 3D scan data. Texture is assigned to the face model from the first data frame and then applied to tracking results for subsequent frames, generating the re-rendered sequence. In this video, we also demonstrate our results for expression transfer among different subjects. We reduce the dimensionality of our data onto a lower dimensional space manifold and then decompose it into style and content parameters. This allows us to transfer subtle expression information (in the form of a style vector) between individuals to synthesize new expressions, as well as smoothly morph geometry and motion. To achieve face synthesis and relighting from one sin-gle input image under arbitrary unknown lighting and pose, we created a statistical model of shape and spherical harmonic appearance information, which allows us to convert any single image of a face into a different pose and illumination. Experimental results show that using only a single image of a face under unknown lighting, we can achieve high recognition rates and generate photo-realistic images of the face under a wide range of illumination conditions, including multiple sources of illumination. Finally, we demonstrate the accuracy of our face modeling methods through an integrated example of image-driven re-targeting and relighting of facial expressions, where transfer of expression and illumination information between different individuals is possible. Acknowledgements We are grateful to