Unwrap mosaics

Alex Rav-Acha, Pushmeet Kohli, Carsten Rother, Andrew Fitzgibbon
2008 ACM Transactions on Graphics  
Two input video frames Unwrap mosaic Edited mosaic Output video Figure 1 : Editing a video of a talking head. The input images are automatically converted to an "unwrap mosaic" without an intervening 3D reconstruction. Painting the mosaic and re-rendering the video allows us to add virtual make-up (eyebrows, moustache, and rouge on the cheeks) to the actor, as if to a texture map on a deformable 3D surface. Abstract We introduce a new representation for video which facilitates a number of
more » ... editing tasks. The representation has some of the power of a full reconstruction of 3D surface models from video, but is designed to be easy to recover from a priori unseen and uncalibrated footage. By modelling the image-formation process as a 2D-to-2D transformation from an object's texture map to the image, modulated by an object-space occlusion mask, we can recover a representation which we term the "unwrap mosaic". Many editing operations can be performed on the unwrap mosaic, and then re-composited into the original sequence, for example resizing objects, repainting textures, copying/cutting/pasting objects, and attaching effects layers to deforming objects.
doi:10.1145/1360612.1360616 fatcat:yazep25nenduxnppzqmfm5i5z4