Filters








220 Hits in 2.9 sec

Personalized Face Modeling for Improved Face Reconstruction and Motion Retargeting [article]

Bindita Chaudhuri, Noranart Vesdapunt, Linda Shapiro, Baoyuan Wang
2020 arXiv   pre-print
Traditional methods for image-based 3D face reconstruction and facial motion retargeting fit a 3D morphable model (3DMM) to the face, which has limited modeling capacity and fail to generalize well to  ...  in more accurate face reconstruction and facial motion retargeting compared to state-of-the-art methods.  ...  Acknowledgements: We thank the anonymous reviewers for their constructive feedback, Muscle Wu, Wenbin Zhu and Zeyu Chen for helping, and Alex Colburn for valuable discussions.  ... 
arXiv:2007.06759v2 fatcat:3nzh2hcqdjcc7jbxveluovqqcy

A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos [article]

Thiago L. Gomes and Renato Martins and João Ferreira and Rafael Azevedo and Guilherme Torres and Erickson R. Nascimento
2021 arXiv   pre-print
We also present a new video retargeting benchmark dataset composed of different videos with annotated human motions to evaluate the task of synthesizing people's videos, which can be used as a common base  ...  The dataset and retargeting code are publicly available to the community at: https://www.verlab.dcc.ufmg.br/retargeting-motion.  ...  The authors thank CAPES, CNPq, and FAPEMIG for funding this work. We also thank NVIDIA for the donation of a Titan XP GPU used in this research.  ... 
arXiv:2103.15596v2 fatcat:them7wrlvfchpgy36mtxmfzm2q

Do As I Do: Transferring Human Motion and Appearance between Monocular Videos with Spatial and Temporal Constraints

Thiago L. Gomes, Renato Martins, Joao Ferreira, Erickson R. Nascimento
2020 2020 IEEE Winter Conference on Applications of Computer Vision (WACV)  
In this paper, we propose a unifying formulation for transferring appearance and retargeting human motion from monocular videos that regards all these aspects.  ...  Our method synthesizes new videos of people in a different context where they were initially recorded.  ...  The authors would like to thank CAPES, CNPq, and FAPEMIG for funding different parts of this work. We also thank NVIDIA Corporation for the donation of a Titan XP GPU used in this research.  ... 
doi:10.1109/wacv45572.2020.9093395 dblp:conf/wacv/GomesMFN20 fatcat:mwmpbph2ozbmfieitekagcwcpq

S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling [article]

Ze Yang, Shenlong Wang, Sivabalan Manivasagam, Zeng Huang, Wei-Chiu Ma, Xinchen Yan, Ersin Yumer, Raquel Urtasun
2021 arXiv   pre-print
Constructing and animating humans is an important component for building virtual worlds in a wide variety of applications such as virtual reality or robotics testing in simulation.  ...  As there are exponentially many variations of humans with different shape, pose and clothing, it is critical to develop methods that can automatically reconstruct and animate humans at scale from real  ...  Additional Details 3D U-Net Architecture: We adopt the 3D U-Net architecture from [17] , where the encoder consists of 8 convolution layers.  ... 
arXiv:2101.06571v1 fatcat:l2rob7uxurcf7fakfcqntuz6tu

Do As I Do: Transferring Human Motion and Appearance between Monocular Videos with Spatial and Temporal Constraints [article]

Thiago L. Gomes and Renato Martins and João Ferreira and Erickson R. Nascimento
2020 arXiv   pre-print
In this paper, we propose a unifying formulation for transferring appearance and retargeting human motion from monocular videos that regards all these aspects.  ...  Our method synthesizes new videos of people in a different context where they were initially recorded.  ...  We also thank NVIDIA Corporation for the donation of a Titan XP GPU used in this research.  ... 
arXiv:2001.02606v2 fatcat:m2cposgyqneqvnxsxznrtr4akm

MetaPix: Few-Shot Video Retargeting [article]

Jessica Lee, Deva Ramanan, Rohit Girdhar
2020 arXiv   pre-print
We address the task of unsupervised retargeting of human actions from one video to another. We consider the challenging setting where only a few frames of the target is available.  ...  To do so, we make use of meta-learning to discover effective strategies for on-the-fly personalization.  ...  Acknowledgements: This research is based upon work supported in part by NSF Grant 1618903, the Intel Science and Technology Center for Visual Cloud Systems (ISTC-VCS), and Google.  ... 
arXiv:1910.04742v2 fatcat:rdvimi3hijg4zo5bitz2yrsd34

Expression transfer for facial sketch animation

Yang Yang, Nanning Zheng, Yuehu Liu, Shaoyi Du, Yuanqi Su, Yoshifumi Nishio
2011 Signal Processing  
Based on the NET model, we present a hierarchical method to animate facial sketches. The motion vectors on the source face are adjusted from coarse to fine on the target face.  ...  Without any expression example obtained from target faces, our approach can transfer expressions by motion retargetting to facial sketches.  ...  The idea of our animation method is similar as the motion retargetting.  ... 
doi:10.1016/j.sigpro.2011.04.020 fatcat:xjpyy7m2s5hn7lc752uhcosn5e

Unsupervised Co-part Segmentation through Assembly [article]

Qingzhe Gao, Bin Wang, Libin Liu, Baoquan Chen
2021 arXiv   pre-print
For the training stage, we leverage motion information embedded in videos and explicitly extract latent representations to segment meaningful object parts.  ...  More importantly, we introduce a dual procedure of part-assembly to form a closed loop with part-segmentation, enabling an effective self-supervision.  ...  Acknowledgements We thank the anonymous reviewers for their constructive comments.  ... 
arXiv:2106.05897v1 fatcat:mk57vltdbzhe3ii3sd54vzhe2e

Learning-Based Automation of Robotic Assembly for Smart Manufacturing

Sanghoon Ji, Sukhan Lee, Sujeong Yoo, Ilhong Suh, Inso Kwon, Frank C. Park, Sanghyoung Lee, Hongseok Kim
2021 Proceedings of the IEEE  
modeling and simulated retargeting.  ...  We validate the proposed approach by building a prototype of the automated robotic assembly system for a power breaker and an electronic set-top box.  ...  Note that robot skill can be modeled either by DMP for a task with milder uncertainties and variations, e.g., an insertion task, or by DCNN for a task with higher uncertainties and variations, e.g., a  ... 
doi:10.1109/jproc.2021.3063154 fatcat:3sggy6vmobbppmijygotvigvey

Human Action Transfer Based on 3D Model Reconstruction

Shanyan Guan, Shuo Wen, Dexin Yang, Bingbing Ni, Wendong Zhang, Jun Tang, Xiaokang Yang
2019 PROCEEDINGS OF THE THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE TWENTY-EIGHTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE  
We present a practical and effective method for human action transfer. Given a sequence of source action and limited target information, we aim to transfer motion from source to target.  ...  Although recent works based on GAN or VAE achieved impressive results for action transfer in 2D, there still exists a lot of problems which cannot be avoided, such as distorted and discontinuous human  ...  Table 1: Transfer results comparison with Variational U-net.  ... 
doi:10.1609/aaai.v33i01.33018352 fatcat:yg2zoebs75buleufhqwm3zrrky

PuppeteerGAN: Arbitrary Portrait Animation With Semantic-Aware Appearance Transformation

Zhuo Chen, Chaoyue Wang, Bo Yuan, Dacheng Tao
2020 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
Similar to U-net [30] , the coloring network (right bottom) includes an encoder, a generator and several proposed warp-based semantic-aware skip-connections (WASIC).  ...  As shown in Fig. 3 -right, the structure of the coloring network is based on U-net [30] . We replace the single convolution layers with residual blocks [13] in both encoder and decoder.  ... 
doi:10.1109/cvpr42600.2020.01353 dblp:conf/cvpr/ChenWYT20 fatcat:hwbvjljwpbfxrlmkftrlzopmaa

Recycle-GAN: Unsupervised Video Retargeting [chapter]

Aayush Bansal, Shugao Ma, Deva Ramanan, Yaser Sheikh
2018 Lecture Notes in Computer Science  
We introduce a data-driven approach for unsupervised video retargeting that translates content from one domain to another while preserving the style native to a domain, i.e., if contents of John Oliver's  ...  In this work, we first study the advantages of using spatiotemporal constraints over spatial constraints for effective retargeting.  ...  To implement our temporal predictors P X and P Y , we concatenate the last two frames as input to a network whose architecture is identical to U-Net architecture [23, 37] . 4E x p e r i m e n t s We  ... 
doi:10.1007/978-3-030-01228-1_8 fatcat:y35ontbezngepnysioqzs6leiu

Towards Disentangled Representations for Human Retargeting by Multi-view Learning [article]

Chao Yang, Xiaofeng Liu, Qingming Tang, C.-C. Jay Kuo
2019 arXiv   pre-print
We study the problem of learning disentangled representations for data across multiple domains and its applications in human retargeting.  ...  To this end, we present a novel multi-view learning approach that leverages various data sources such as images, keypoints, and poses.  ...  [36] devises a framework based on variational u-net that separates appearances and poses.  ... 
arXiv:1912.06265v1 fatcat:o2ah4eih6bahhdkn37iygcmjxy

Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders [article]

Jing Li, Di Kang, Wenjie Pei, Xuefei Zhe, Ying Zhang, Zhenyu He, Linchao Bao
2021 arXiv   pre-print
In order to overcome this problem, we propose a novel conditional variational autoencoder (VAE) that explicitly models one-to-many audio-to-motion mapping by splitting the cross-modal latent code into  ...  However, splitting the latent code into two parts poses training difficulties for the VAE model.  ...  As a result, Speech2Gesture [12] proposes a more powerful fully convolutional network, consisting of a 8-layer CNN audio encoder and a 16-layer 1D U-Net decoder, to translate log-mel audio feature to  ... 
arXiv:2108.06720v1 fatcat:cjognac7gbdavkfut2qu2kichu

Recycle-GAN: Unsupervised Video Retargeting [article]

Aayush Bansal, Shugao Ma, Deva Ramanan, Yaser Sheikh
2018 arXiv   pre-print
We introduce a data-driven approach for unsupervised video retargeting that translates content from one domain to another while preserving the style native to a domain, i.e., if contents of John Oliver's  ...  In this work, we first study the advantages of using spatiotemporal constraints over spatial constraints for effective retargeting.  ...  To implement our temporal predictors P X and P Y , we concatenate the last two frames as input to a network whose architecture is identical to U-Net architecture [23, 37] .  ... 
arXiv:1808.05174v1 fatcat:2vychysagvf43molzagxxtvxqi
« Previous Showing results 1 — 15 out of 220 results