9 Hits in 3.2 sec

SDCNet: Video Prediction Using Spatially-Displaced Convolution [article]

Fitsum A. Reda, Guilin Liu, Kevin J. Shih, Robert Kirby, Jon Barker, David Tarjan, Andrew Tao, Bryan Catanzaro
2021 arXiv   pre-print
Here, we spatially-displaced convolution (SDC) module for video frame prediction.  ...  We present an approach for high-resolution video frame prediction by conditioning on both past frames and past optical flows.  ...  Here, we present spatially-displaced convolution (SDC) module for video frame prediction.  ... 
arXiv:1811.00684v2 fatcat:kg7yswg2arawvl252uuldhpjw4

Generation and Simulation of Yeast Microscopy Imagery with Deep Learning [article]

Christoph Reich
2021 arXiv   pre-print
To tackle this simulation task an advanced future frame prediction model is introduced. The proposed models are trained and tested on a novel dataset that is presented in this thesis.  ...  [19] SDC-Net The paper "SDC-Net: Video prediction using spatially-displaced convolution " by Reda et al. [22] proposes a novel approach to the problem of future frame prediction.  ...  The spatially-displaced convolution CUDA/PyTorch [110] implementation is taken from the official SDC-Net [22] repository.  ... 
arXiv:2103.11834v4 fatcat:x6jljdhd4vcx7a3wjxj6igs57q

End-to-end Optimized Video Compression with MV-Residual Prediction [article]

XiangJi Wu, Ziwen Zhang, Jie Feng, Lei Zhou, Junmin Wu
2020 arXiv   pre-print
Specially, the spatially-displaced convolution is applied for video frame prediction, in which a motion kernel for each pixel is learned to generate predicted pixel by applying the kernel at a displaced  ...  Finally, novel rate allocation and post-processing strategies are used to produce the final compressed bits, considering the bits constraint of the challenge.  ...  The motion kernels can be learnt by spatially-displaced convolutions to predict pixels in the P-frame by applying the kernels at a displaced locations in the source image.  ... 
arXiv:2005.12945v1 fatcat:xcibnnubn5dqnm42j2juot7iri

PseudoProp: Robust Pseudo-Label Generation for Semi-Supervised Object Detection in Autonomous Driving Systems [article]

Shu Hu, Chun-Hao Liu, Jayanta Dutta, Ming-Ching Chang, Siwei Lyu, Naveen Ramakrishnan
2022 arXiv   pre-print
detections using such generated pseudo-labels.  ...  In this paper, we propose a new approach, PseudoProp, to generate robust pseudo-labels by leveraging motion continuity in video frames.  ...  Video Motion The spatially-displaced convolution network (SDC-Net) in [23] can predict future video frames based on a two-stage process of first estimating motion then predicting frames.  ... 
arXiv:2203.05983v2 fatcat:ygk7v75nlbgvtp2i4gyyoopkqy

A Review on Deep Learning Techniques for Video Prediction [article]

Sergiu Oprea, Pablo Martinez-Gonzalez, Alberto Garcia-Garcia, John Alejandro Castro-Vargas, Sergio Orts-Escolano, Jose Garcia-Rodriguez, Antonis Argyros
2020 arXiv   pre-print
We firstly define the video prediction fundamentals, as well as mandatory background concepts and the most used datasets.  ...  In light of the success of deep learning in computer vision, deep-learning-based video prediction emerged as a promising research direction.  ...  [155] proposed the Spatially Displaced Convolution (SDC) module that synthesizes highresolution images applying a learned per-pixel motion vector and kernel at a displaced location in the source image  ... 
arXiv:2004.05214v2 fatcat:weerbkanmjb4dn6wkn5o4b5aia

Affine Transformation-Based Deep Frame Prediction [article]

Hyomin Choi, Ivan V. Bajić
2021 arXiv   pre-print
The predicted frame is used as a reference for coding the current frame.  ...  We propose a neural network model to estimate the current frame from two reference frames, using affine transformation and adaptive spatially-varying filters.  ...  [26] proposed a spatial-displaced convolution network (SDC-Net) that applies smaller ASFs to adaptively-chosen positions, rather than collocated areas, in reference frames.  ... 
arXiv:2009.05666v2 fatcat:xd6umwi3wrdnhav7we4h5wr4f4

Disentangling Propagation and Generation for Video Prediction [article]

Hang Gao, Huazhe Xu, Qi-Zhi Cai, Ruth Wang, Fisher Yu, Trevor Darrell
2019 arXiv   pre-print
Prior approaches to video prediction typically learn either to warp or to hallucinate future pixels, but not both.  ...  In this paper, we describe a computational model for high-fidelity video prediction which disentangles motion-specific propagation from motion-agnostic generation.  ...  Catanzaro, “Sdc-net: Video predic- tion using spatially-displaced convolution,” arXiv preprint arXiv:1811.00684, 2018. 3 [35] Y. Wang, Y. Yang, Z. Yang, L. Zhao, P. Wang, and W.  ... 
arXiv:1812.00452v2 fatcat:ukz6uht6pbb4tb3qxc5yshpah4

MotionRNN: A Flexible Model for Video Prediction with Spacetime-Varying Motions [article]

Haixu Wu, Zhiyu Yao, Jianmin Wang, Mingsheng Long
2021 arXiv   pre-print
This paper tackles video prediction from a new dimension of predicting spacetime-varying motions that are incessantly changing across both space and time.  ...  The second is that we apply the MotionGRU to RNN-based predictive models and indicate a new flexible video prediction architecture with a Motion Highway that can significantly improve the ability to predict  ...  SDC-Net [20] learns the transformation kernel and kernel offsets between frames based on the optical flow.  ... 
arXiv:2103.02243v3 fatcat:w7xcxbv3sbdudaykejkwlwsn3a

Learning Semantic-Aware Dynamics for Video Prediction [article]

Xinzhu Bei, Yanchao Yang, Stefano Soatto
2021 arXiv   pre-print
The appearance of the scene is warped from past frames using the predicted motion in co-visible regions; dis-occluded regions are synthesized with content-aware inpainting utilizing the predicted scene  ...  We propose an architecture and training scheme to predict video frames by explicitly modeling dis-occlusions and capturing the evolution of semantically consistent regions in the video.  ...  SDC-Net [31] applies flow guided spatially-displaced convolutions, while [15] predicts with dynamic filters that depend on the inputs.  ... 
arXiv:2104.09762v1 fatcat:rzbewbus4zftpn6cu4e6asfoi4