2,940 Hits in 4.1 sec

Self-Supervised Approach for Facial Movement Based Optical Flow [article]

Muhannad Alkaddour, Usman Tariq, Abhinav Dhall
2021 arXiv   pre-print
However, deep learning-based optical flow techniques do not perform well for non-rigid movements such as those found in faces, primarily due to lack of the training data representing the fine facial motion  ...  The aim of this work is threefold: (1) exploring self-supervised techniques to generate optical flow ground truth for face images; (2) computing baseline results on the effects of using face data to train  ...  CONCLUSION AND FUTURE WORK In this paper, we explore the possibility of using a facial expression dataset to learn optical flow representations based on a self-supervised technique.  ... 
arXiv:2105.01256v1 fatcat:kdpdlzmuwzablgqsziuuhnt33i

Self-Supervised Regional and Temporal Auxiliary Tasks for Facial Action Unit Recognition [article]

Jingwei Yan and Jingjing Wang and Qiang Li and Chunmao Wang and Shiliang Pu
2021 arXiv   pre-print
Meanwhile, a single image based optical flow estimation task is proposed to leverage the dynamic change of facial muscles and encode the motion information into the global feature representation.  ...  Based on these two self-supervised auxiliary tasks, local features, mutual relation and motion cues of AUs are better captured in the backbone network with the proposed regional and temporal based auxiliary  ...  Here we propose a self-supervised task of optical flow estimation based on the static facial image to model the temporal change of facial muscles elegantly and effectively.  ... 
arXiv:2107.14399v1 fatcat:bb3f65jiujcnjae5eampgsk2wa

Synthesising Facial Macro- and Micro-Expressions Using Reference Guided Style Transfer

Chuin Hong Yap, Ryan Cunningham, Adrian K. Davison, Moi Hoon Yap
2021 Journal of Imaging  
Additionally, optical flow is used to visually compare the original facial movements and the transferred facial movements.  ...  We evaluate SAMM-SYNTH by conducting an analysis based on the facial action units detected by OpenFace.  ...  We recommend to use optical flow for facial motion transfer analysis, to visualise the facial movements and its intensity. 4.  ... 
doi:10.3390/jimaging7080142 pmid:34460778 fatcat:4lydmjwp5vgupnw7ccoctkgtwe

Deformation Flow Based Two-Stream Network for Lip Reading [article]

Jingyun Xiao, Shuang Yang, Yuanhang Zhang, Shiguang Shan, Xilin Chen
2020 arXiv   pre-print
Specifically, we introduce a Deformation Flow Network (DFN) to learn the deformation flow between adjacent frames, which directly captures the motion information within the lip region.  ...  Lip reading is the task of recognizing the speech content by analyzing movements in the lip region when people are speaking.  ...  However, it aims to learn the facial attributes in a self-supervised manner.  ... 
arXiv:2003.05709v2 fatcat:o7man6lolzhvzlkxewld74xo7u

Hand Motion Capture and Tracking in 3D Animation [article]

J. Condell, G. Moore, J. Moore
2006 Computer Graphics and Visual Computing  
Motion capture is a technique of digitally recording the movements of real entities, usually humans.  ...  In this context it has been widely used for both cinema and video games.  ...  Hand Tracking using Optical Flow The two main methods for hand tracking are appearance-based methods and model-based methods.  ... 
doi:10.2312/localchapterevents/tpcg/tpcg06/105-109 dblp:conf/tpcg/CondellMM06 fatcat:3curey4nmbb7vojbmjw5y2gmc4

Robust Representation and Recognition of Facial Emotions Using Extreme Sparse Learning

Seyedehsamaneh Shojaeilangari, Wei-Yun Yau, Karthik Nandakumar, Jun Li, Eam Khwang Teoh
2015 IEEE Transactions on Image Processing  
ACKNOWLEDGMENT This research is supported by the Agency for Science, Technology and Research (A*STAR), Singapore. The authors would like to thank the reviewers for their valuable comments.  ...  Fig. 1 . 1 Optical flow correction for head movement.  ...  flow (Utot) illustrated in blue and the head movement optical flow (U head ) indicated in green; (d) emotion related optical flow (Uemo) illustrated in red; (e) Utot of mouth region; and (f) Uemo of mouth  ... 
doi:10.1109/tip.2015.2416634 pmid:25823034 fatcat:lwhs5wv6gjgnzig7fb7ucmsrgq

Box Supervised Video Segmentation Proposal Network [article]

Tanveer Hannan, Rajat Koner, Jonathan Kobold, Matthias Schubert
2022 arXiv   pre-print
However, self-supervised approaches pose a significant performance gap.  ...  Video Object Segmentation (VOS) has been targeted by various fully-supervised and self-supervised approaches.  ...  Self-supervised VOS provides an impressive approach that avoids the need for expensive frame-wise object masks.  ... 
arXiv:2202.07025v2 fatcat:alq22qzyjvfhpi5eyxbt7zbcza

Learning Facial Representations from the Cycle-consistency of Face [article]

Jia-Ren Chang, Yong-Sheng Chen, Wei-Chen Chiu
2021 arXiv   pre-print
At test time, we use the linear protocol scheme for evaluating facial representations on various tasks, including facial expression recognition and head pose regression.  ...  At training time, our model learns to disentangle two distinct facial representations to be useful for performing cycle-consistent face reconstruction.  ...  We are grateful to the National Center for High-performance Computing for providing computing services and facilities.  ... 
arXiv:2108.03427v1 fatcat:7eajqmwwofe7pb4spkuqv7yiu4

Deep Learning for Micro-expression Recognition: A Survey [article]

Yante Li, Jinsheng Wei, Yang Liu, Janne Kauttonen, Guoying Zhao
2021 arXiv   pre-print
This survey defines a new taxonomy for the field, encompassing all aspects of MER based on DL. For each aspect, the basic approaches and advanced developments are summarized and discussed.  ...  Early methods for MER mainly based on traditional appearance and geometry features.  ...  Currently, many MER approaches utilize optical flow to represent the micro-facial movement and reduce the identity characteristic [57] , [100] , [138] , [154] .  ... 
arXiv:2107.02823v4 fatcat:w2bqbvxw4zbc7jjnqmx2gmh5aa

Animating Arbitrary Objects via Deep Motion Transfer

Aliaksandr Siarohin, Stephane Lathuiliere, Sergey Tulyakov, Elisa Ricci, Nicu Sebe
2019 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
This paper introduces a novel deep learning framework for image animation.  ...  We demonstrate the effectiveness of our method on several benchmark datasets, spanning a wide variety of object appearances, and show that our approach outperforms stateof-the-art image animation and video  ...  The optical flow computed by M is used by G to handle misalignments between x and x ′ . The model is learned with a self-supervised learning scheme.  ... 
doi:10.1109/cvpr.2019.00248 dblp:conf/cvpr/SiarohinLT0S19 fatcat:quvi2lioefd3jlmaczmfgfd634

Two-Stream Attention Network for Pain Recognition from Video Sequences

Patrick Thiam, Hans A. Kestler, Friedhelm Schwenker
2020 Sensors  
) and Optical Flow Images (OFIs).  ...  Several approaches have been proposed for the analysis of pain-related facial expressions.  ...  Acknowledgments: We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Tesla K40 GPU used for this research.  ... 
doi:10.3390/s20030839 pmid:32033240 pmcid:PMC7038688 fatcat:acsqlqmpnbdxpihnqyav3wfe6y

Self-Supervised Representation Learning From Videos for Facial Action Unit Detection

Yong Li, Jiabei Zeng, Shiguang Shan, Xilin Chen
2019 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
Inspired by the fact that facial actions are the movements of facial muscles, we depict the movements as the transformation between two face images in different frames and use it as the self-supervisory  ...  Experimental results also demonstrate that the learned representation is discriminative for AU detection, where TCAE outperforms or is comparable with the state-of-the-art self-supervised learning methods  ...  We also thank Zijia Lu for discussion. J. Zeng, S. Shan, and X. Chen are the corresponding co-authors of this paper.  ... 
doi:10.1109/cvpr.2019.01118 dblp:conf/cvpr/LiZSC19 fatcat:6dtuad7yp5emfcmxjwcfpib3cm

Challenges and Opportunities for Machine Learning Classification of Behavior and Mental State from Images [article]

Peter Washington, Cezmi Onur Mutlu, Aaron Kline, Kelley Paskov, Nate Tyler Stockham, Brianna Chrisman, Nick Deveau, Mourya Surhabi, Nick Haber, Dennis P. Wall
2022 arXiv   pre-print
While CV classifiers for traditional and structured classification tasks can be developed with standard machine learning pipelines for supervised learning consisting of data labeling, preprocessing, and  ...  training a convolutional neural network, there are several pain points which arise when attempting this process for behavioral phenotyping.  ...  In particular, self-supervision on 2D keypoint detections learned features useful for 3D body joint predictions, self-supervision on 2D optical flow vectors learned features useful for 3D mesh vertex displacement  ... 
arXiv:2201.11197v1 fatcat:fvhzvvctn5drtooemem4u6qloi

ICE-GAN: Identity-aware and Capsule-Enhanced GAN with Graph-based Reasoning for Micro-Expression Recognition and Synthesis [article]

Jianhui Yu, Chaoyi Zhang, Yang Song, Weidong Cai
2021 arXiv   pre-print
The short detection window, the subtle facial muscle movements, and the limited training samples make micro-expression recognition challenging.  ...  To this end, we propose a novel Identity-aware and Capsule-Enhanced Generative Adversarial Network with graph-based reasoning (ICE-GAN), introducing micro-expression synthesis as an auxiliary task to assist  ...  For instance, optical flow images are generated via GAN to enlarge the database in [29] , and real facial images are produced based on AU intensity in [30] .  ... 
arXiv:2005.04370v2 fatcat:sx5oxoutbjfdfkgc3wblwqefoe

Towards Machine Recognition of Facial Expressions of Pain in Horses

Pia Haubro Andersen, Sofia Broomé, Maheen Rashid, Johan Lundblad, Katrina Ask, Zhenghong Li, Elin Hernlund, Marie Rhodin, Hedvig Kjellström
2021 Animals  
Automated recognition of human facial expressions of pain and emotions is to a certain degree a solved problem, using approaches based on computer vision and machine learning.  ...  One involves the use of a manual, but relatively objective, classification system for facial activity (Facial Action Coding System), where data are analyzed for pain expressions after coding using machine  ...  Acknowledgments: Deborah Forster, University of California at San Diego, US, is thanked for her invaluable input and for guiding the interdisciplinary research start-up in this field.  ... 
doi:10.3390/ani11061643 fatcat:asosn37bhvenlkgrdgcgkmvazq
« Previous Showing results 1 — 15 out of 2,940 results