Video Imagination from a Single Image with Transformation Generation

Baoyang Chen, Wenmin Wang, Jinzhuo Wang
2017 Proceedings of the on Thematic Workshops of ACM Multimedia 2017 - Thematic Workshops '17  
In this work, we focus on a challenging task: synthesizing multiple imaginary videos given a single image. Major problems come from high dimensionality of pixel space and the ambiguity of potential motions. To overcome those problems, we propose a new framework that produce imaginary videos by transformation generation. The generated transformations are applied to the original image in a novel volumetric merge network to reconstruct frames in imaginary video. Through sampling di erent latent
more » ... iables, our method can output di erent imaginary video samples. The framework is trained in an adversarial way with unsupervised learning. For evaluation, we propose a new assessment metric RIQA. In experiments, we test on 3 datasets varying from synthetic data to natural scene. Our framework achieves promising performance in image quality assessment. The visual inspection indicates that it can successfully generate diverse ve-frame videos in acceptable perceptual quality.
doi:10.1145/3126686.3126737 dblp:conf/mm/ChenWW17a fatcat:gykx6n5d35a4xjeaf225mb6upq