756 Hits in 4.4 sec

Temporally Coherent Video Harmonization Using Adversarial Networks [article]

Haozhi Huang, Senzhe Xu, Junxiong Cai, Wei Liu, Shimin Hu
2018 arXiv   pre-print
Specifically, we train a convolutional neural network in an adversarial way, exploiting a pixel-wise disharmony discriminator to achieve more realistic harmonized results and introducing a temporal loss  ...  supervised training of the proposed adversarial network.  ...  RELATED WORK Our work attempts to generate a temporally consistent harmonized video by an adversarial network.  ... 
arXiv:1809.01372v1 fatcat:zflqx4tlw5gdvdfo7df6dvixs4

FoleyGAN: Visually Guided Generative Adversarial Network-Based Synchronous Sound Generation in Silent Videos [article]

Sanchita Ghose, John J. Prevost
2021 arXiv   pre-print
In this research we introduce a novel task of guiding a class conditioned generative adversarial network with the temporal visual information of a video input for visual to sound generation task adapting  ...  The detailed methods are explained in following paragraphs. 1) Video Action Class Prediction: We use a fused network comprised of CNN and multiscale temporal relation network (TRN), proposed in [16]  ...  In this research we introduce a novel task of guiding a class conditioned generative adversarial network with the temporal visual information of a video input for visual to sound generation task adapting  ... 
arXiv:2107.09262v1 fatcat:lqyve5czjzdf3ihtcr7vohffyi

Dilated Temporal Relational Adversarial Network for Generic Video Summarization [article]

Yujia Zhang, Michael Kampffmeyer, Xiaodan Liang, Dingwen Zhang, Min Tan, Eric P. Xing
2019 arXiv   pre-print
We propose a novel Dilated Temporal Relational Generative Adversarial Network (DTR-GAN) to achieve frame-level video summarization.  ...  The generator uses this unit to effectively exploit global multi-scale temporal context to select key frames and to complement the commonly used Bi-LSTM.  ...  coherence).  ... 
arXiv:1804.11228v2 fatcat:mp2cjt646zgtdjqcgwq5ggm5su

MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment [article]

Hao-Wen Dong, Wen-Yi Hsiao, Li-Chia Yang, Yi-Hsuan Yang
2017 arXiv   pre-print
Generating music has a few notable differences from generating images and videos. First, music is an art of time, necessitating a temporal model.  ...  In this paper, we propose three models for symbolic multi-track music generation under the framework of generative adversarial networks (GANs).  ...  Recent years have seen major progress in generating images, videos and text, notably using generative adversarial networks (GANs) (Goodfellow et al. 2014; Radford, Metz, and Chintala 2016; Vondrick, Pirsiavash  ... 
arXiv:1709.06298v2 fatcat:l5g3ey34lfdo3i6mag5dapoanq

A GAN-based temporally stable shading model for fast animation of photorealistic hair

Zhi Qiao, Takashi Kanai
2021 Computational Visual Media  
We use two constraints to ensure temporal coherence and highlight stability. Our approach outperforms and is computationally more efficient than previous methods.  ...  A temporal coherence comparison is given in a video in Electronic Supplementary Material.  ...  This method achieves temporal coherence for video style transfer. Chen et al. [23] Fig. 2 Mapping approaches.  ... 
doi:10.1007/s41095-020-0201-9 fatcat:jtq6z3czsbcyffinkvttz7yrki

Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation [article]

Lingjie Liu, Weipeng Xu, Marc Habermann, Michael Zollhoefer, Florian Bernard, Hyeongwoo Kim, Wenping Wang, Christian Theobalt
2021 arXiv   pre-print
Synthesizing realistic videos of humans using neural networks has been a popular alternative to the conventional graphics-based rendering pipeline due to its high efficiency.  ...  Given the pose information, the first CNN predicts a dynamic texture map that contains time-coherent high-frequency details, and the second CNN conditions the generation of the final video on the temporally  ...  Conditional Generative Adversarial Networks.  ... 
arXiv:2001.04947v3 fatcat:ppii2ilexze7nkejshrohlky4u

Deep Video Portraits [article]

Hyeongwoo Kim, Pablo Garrido, Ayush Tewari, Weipeng Xu, Justus Thies, Matthias Nießner, Patrick Pérez, Christian Richardt, Michael Zollhöfer, Christian Theobalt
2018 arXiv   pre-print
We present a novel approach that enables photo-realistic re-animation of portrait videos using only an input video.  ...  The realism in this rendering-to-video transfer is achieved by careful adversarial training, and as a result, we can create modified target videos that mimic the behavior of the synthetically-created input  ...  For temporally coherent results, our network works on space-time volumes of conditioning inputs.  ... 
arXiv:1805.11714v1 fatcat:44phmzr42rayrdkbbwpvwyy3bi

Learning Blind Video Temporal Consistency [article]

Wei-Sheng Lai, Jia-Bin Huang, Oliver Wang, Eli Shechtman, Ersin Yumer, Ming-Hsuan Yang
2018 arXiv   pre-print
In this paper, we present an efficient end-to-end approach based on deep recurrent network for enforcing temporal consistency in a video.  ...  We train the proposed network by minimizing both short-term and long-term temporal losses as well as the perceptual loss to strike a balance between temporal stability and perceptual similarity with the  ...  One approach for achieving temporally coherent results is to explicitly embed flow-based temporal consistency loss in the design and training of the networks.  ... 
arXiv:1808.00449v1 fatcat:7mfcib6k4zbsxebyhhus7rmn4u

Self-Enhanced Convolutional Network for Facial Video Hallucination [article]

Chaowei Fang, Guanbin Li, Xiaoguang Han, Yizhou Yu
2019 arXiv   pre-print
Taking advantage of high inter-frame dependency in videos, we propose a self-enhanced convolutional network for facial video hallucination.  ...  However, the direct migration of existing methods to video is still difficult to achieve good performance due to its lack of alignment and consistency modelling in temporal domain.  ...  On the other hand, generative adversarial models [31] are widely used in face super-resolution. UR-DGN [32] is claimed to be the first face SR method using generative adversarial network.  ... 
arXiv:1911.11136v1 fatcat:xavmtdckufa6ngargrf4lspv5a

2020 Index IEEE Transactions on Image Processing Vol. 29

2020 IEEE Transactions on Image Processing  
., +, TIP 2020 9689-9702 Temporally Coherent Video Harmonization Using Adversarial Networks.  ...  Ren, X., +, TIP 2020 7497-7510 Temporally Coherent Video Harmonization Using Adversarial Networks.  ... 
doi:10.1109/tip.2020.3046056 fatcat:24m6k2elprf2nfmucbjzhvzk3m

Future Video Synthesis with Object Motion Prediction [article]

Yue Wu, Rongrong Gao, Jaesik Park, Qifeng Chen
2020 arXiv   pre-print
We present an approach to predict future video frames given a sequence of continuous video frames in the past.  ...  The anticipated appearances are combined to create a reasonable video in the future. With this procedure, our method exhibits much less tearing or distortion artifact compared to other approaches.  ...  We also use two discriminators to ensure the locations of predicted objects are spatially and temporally coherent.  ... 
arXiv:2004.00542v2 fatcat:6wpozllu3nhubnl3yubdllntlu

DVI: Depth Guided Video Inpainting for Autonomous Driving [article]

Miao Liao, Feixiang Lu, Dingfu Zhou, Sibo Zhang, Wei Li, Ruigang Yang
2020 arXiv   pre-print
To our knowledge, we are the first to fuse multiple videos for video inpainting.  ...  Furthermore, we are able to fuse multiple videos through 3D point cloud registration, making it possible to inpaint a target video with multiple source videos.  ...  The emergence of deep learning, especially Generative Adversarial Networks (GAN), has provided us a powerful tool for inpainting.  ... 
arXiv:2007.08854v1 fatcat:7zitrsnbbrf57oiccarkhgcwci

SalSum: Saliency-based Video Summarization using Generative Adversarial Networks [article]

George Pantazis, George Dimas, Dimitris K. Iakovidis
2020 arXiv   pre-print
In this paper, we propose a novel VS method based on a Generative Adversarial Network (GAN) model pre-trained with human eye fixations.  ...  The huge amount of video data produced daily by camera-based systems, such as surveilance, medical and telecommunication systems, emerges the need for effective video summarization (VS) methods.  ...  Specifically, SalGAN is a generative adversarial network (GAN) trained with a joint loss function consisting of two losses, a recostructional and an adversarial loss.  ... 
arXiv:2011.10432v1 fatcat:645u5rfa3fcfhpit7d2g4w7xzu

FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning [article]

Chenxu Zhang, Yifan Zhao, Yifei Huang, Ming Zeng, Saifeng Ni, Madhukar Budagavi, Xiaohu Guo
2021 arXiv   pre-print
Then, our Rendering-to-Video network takes the rendered face images and the attention map of eye blinks as input to generate the photo-realistic output video frames.  ...  To model such complicated relationships among different face attributes with input audio, we propose a FACe Implicit Attribute Learning Generative Adversarial Network (FACIAL-GAN), which integrates the  ...  To ensure temporal coherency, we use a window of size 2N w with the current frame at the center of the window. By following Chan et al.'  ... 
arXiv:2108.07938v1 fatcat:2nodbkvg3rh3fifbfnc2byowjy

Natural Language Description of Videos for Smart Surveillance

Aniqa Dilawari, Muhammad Usman Ghani Khan, Yasser D. Al-Otaibi, Zahoor-ur Rehman, Atta-ur Rahman, Yunyoung Nam
2021 Applied Sciences  
This framework is based on the multitask learning of high-level features (HLFs) using a convolutional neural network (CNN) and natural language generation (NLG) through bidirectional recurrent networks  ...  Problems related to these videos are twofold: (1) understanding the contents of video streams, and (2) conversion of the video contents to condensed formats, such as textual interpretations and summaries  ...  However, these deep features cannot be used for videos due to the absence of temporal information.  ... 
doi:10.3390/app11093730 fatcat:62o2odoqn5gajk46grynsxaa74
« Previous Showing results 1 — 15 out of 756 results