Filters








247 Hits in 7.8 sec

Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation [article]

Anurag Ranjan, Varun Jampani, Lukas Balles, Kihwan Kim, Deqing Sun, Jonas Wulff, Michael J. Black
2019 arXiv   pre-print
We address the unsupervised learning of several interconnected problems in low-level vision: single view depth prediction, camera motion estimation, optical flow, and segmentation of a video into the static  ...  of the static scene structure, and the optical flow of moving objects.  ...  We thank Georgios Pavlakos for helping us with several revisions of the paper. We thank Joel Janai for preparing optical flow visualizations, and Clément Gorard for his Make3d evaluation code.  ... 
arXiv:1805.09806v3 fatcat:pcs3v67funbpvdfqtdx47gu2y4

Joint Unsupervised Learning of Optical Flow and Depth by Watching Stereo Videos [article]

Yang Wang, Zhenheng Yang, Peng Wang, Yi Yang, Chenxu Luo, Wei Xu
2018 arXiv   pre-print
Specifically, given two consecutive stereo image pairs from a video, we first estimate depth, camera ego-motion and optical flow from three neural networks.  ...  Learning depth and optical flow via deep neural networks by watching videos has made significant progress recently.  ...  It learns to estimate depth, optical flow, camera motion, and motion segmentation from two consecutive stereo pairs in an unsupervised manner.  ... 
arXiv:1810.03654v1 fatcat:kejv4uquazao3ilf4ijetbfzjq

Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding [article]

Chenxu Luo, Zhenheng Yang, Peng Wang, Yang Wang, Wei Xu, Ram Nevatia, Alan Yuille
2019 arXiv   pre-print
Performance on the five tasks of depth estimation, optical flow estimation, odometry, moving object segmentation and scene flow estimation shows that our approach outperforms other SoTA methods.  ...  Specifically, during training, given two consecutive frames from a video, we adopt three parallel networks to predict the camera motion (MotionNet), dense depth map (DepthNet), and per-pixel optical flow  ...  the joint learning of depth, motion and flow networks.  ... 
arXiv:1810.06125v2 fatcat:gisehl6ee5cbbd74w3n63zkha4

Learning Residual Flow as Dynamic Motion from Stereo Videos [article]

Seokju Lee, Sunghoon Im, Stephen Lin, In So Kweon
2019 arXiv   pre-print
Our unsupervised learning framework jointly reasons about the camera motion, optical flow, and 3D motion of moving objects.  ...  Based on rigid projective geometry, the estimated stereo depth is used to guide the camera motion estimation, and the depth and camera motion are used to guide the residual flow estimation.  ...  CONCLUSION We have presented a joint learning framework that predicts camera motion, optical flow, and the 3D motion of moving objects in an unsupervised manner.  ... 
arXiv:1909.06999v1 fatcat:gl4q6sdsuvav3e6mmojugwl5oq

Self-Supervised Relative Depth Learning for Urban Scene Understanding [article]

Huaizu Jiang, Erik Learned-Miller, Gustav Larsson, Michael Maire, Greg Shakhnarovich
2018 arXiv   pre-print
The relative depth training images are automatically derived from simple videos of cars moving through a scene, using recent motion segmentation techniques, and no human-provided labels.  ...  This proxy task of predicting relative depth from a single image induces features in the network that result in large improvements in a set of downstream tasks including semantic segmentation, joint road  ...  Acknowledgement The experiments were performed using equipment obtained under a grant from the Collaborative R&D Fund managed by the Massachusetts Tech Collaborative.  ... 
arXiv:1712.04850v2 fatcat:5gglwn4vfrajdgoqygqdjtpnfi

Self-Supervised Relative Depth Learning for Urban Scene Understanding [chapter]

Huaizu Jiang, Gustav Larsson, Michael Maire, Greg Shakhnarovich, Erik Learned-Miller
2018 Lecture Notes in Computer Science  
The relative depth training images are automatically derived from simple videos of cars moving through a scene, using recent motion segmentation techniques, and no human-provided labels.  ...  The proxy task of predicting relative depth from a single image induces features in the network that result in large improvements in a set of downstream tasks including semantic segmentation, joint road  ...  The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the  ... 
doi:10.1007/978-3-030-01252-6_2 fatcat:nzdkxoorlncjxoof5vndwy3ftq

Unsupervised Learning of Monocular Depth and Ego-Motion with Optical Flow Features and Multiple Constraints

Baigan Zhao, Yingping Huang, Wenyan Ci, Xing Hu
2022 Sensors  
This paper proposes a novel unsupervised learning framework for depth recovery and camera ego-motion estimation from monocular video.  ...  The framework exploits the optical flow (OF) property to jointly train the depth and the ego-motion models.  ...  Conflicts of Interest: The authors declare no conflict of interest.  ... 
doi:10.3390/s22041383 pmid:35214285 pmcid:PMC8963015 fatcat:tat3vfv3tneb3g4nvsmyujz6ii

Unsupervised Learning of Depth, Optical Flow and Pose with Occlusion from 3D Geometry [article]

Guangming Wang, Chi Zhang, Hesheng Wang, Jingchuan Wang, Yong Wang, Xinlei Wang
2020 arXiv   pre-print
In the occluded region, as depth and camera motion can provide more reliable motion estimation, they can be used to instruct unsupervised learning of optical flow.  ...  In joint unsupervised training of depth and pose, we can segment the occluded region explicitly.  ...  JOINT UNSUPERVISED TRAINING OF DEPTH-POSE AND OPTICAL FLOW NETWORKS Our system depends on three networks to estimate monocular depth and to obtain the ego-motion and optical flow between two consecutive  ... 
arXiv:2003.00766v2 fatcat:3x76yj6njjfprjts2pveecs37e

Learning with privileged information via adversarial discriminative modality distillation [article]

Nuno C. Garcia, Pietro Morerio, Vittorio Murino
2018 arXiv   pre-print
We consider the practical case of learning representations from depth and RGB videos, while relying only on RGB data at test time.  ...  We propose a new approach to train a hallucination network that learns to distill depth information via adversarial learning, resulting in a clean approach without several losses to balance or hyperparameters  ...  ACKNOWLEDGMENTS The authors would like to thank Riccardo Volpi for useful discussion on adversarial training and GANs.  ... 
arXiv:1810.08437v1 fatcat:emrj23ga3ngprlp2zmtxvms3qy

2020 Index IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 42

2021 IEEE Transactions on Pattern Analysis and Machine Intelligence  
., and Nishino, K., Recognizing Material Properties from Images; 1981-1995 Sebe, N., see Pilzer, A., 2380-2395 Seddik, M., see Tamaazousti, Y., 2212-2224 Shah, M., see Kalayeh, M.M., TPAMI June 2020  ...  ., +, TPAMI June 2020 1483-1500 Unsupervised Learning of a Hierarchical Spiking Neural Network for Optical Flow Estimation: From Events to Global Motion Perception.  ...  ., +, TPAMI Oct. 2020 2396-2409 + Check author entry for coauthors Unsupervised Learning of a Hierarchical Spiking Neural Network for Optical Flow Estimation: From Events to Global Motion Perception.  ... 
doi:10.1109/tpami.2020.3036557 fatcat:3j6s2l53x5eqxnlsptsgbjeebe

Monocular Depth Estimation Based On Deep Learning: An Overview [article]

Chaoqiang Zhao, Qiyu Sun, Chongzhen Zhang, Yang Tang, Feng Qian
2020 arXiv   pre-print
Traditional depth estimation methods, like structure from motion and stereo vision matching, are built on feature correspondences of multiple viewpoints.  ...  With the rapid development of deep neural networks, monocular depth estimation based on deep learning has been widely studied recently and achieved promising performance in accuracy.  ...  framework of unsupervised and semisupervised methods based on GAN. Fig. 2 . 2 (a) The generator of raw GAN TABLE SEMANTIC SEGMENTATION, MOTION SEGMENTATION, OPTICAL FLOW, ETC.  ... 
arXiv:2003.06620v1 fatcat:l5ei3ognova6xkyppflef5nqsq

Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies [article]

Yu Huang, Yue Chen
2020 arXiv   pre-print
Due to the limited space, we focus the analysis on several key areas, i.e. 2D and 3D object detection in perception, depth estimation from cameras, multiple sensor fusion on the data, feature and task  ...  Almost at the same time, deep learning has made breakthrough by several pioneers, three of them (also called fathers of deep learning), Hinton, Bengio and LeCun, won ACM Turin Award in 2019.  ...  G. 2-D Optic Flow and 3-D Scene Flow Optic flow is pixel-level motion, from which the local object motion and global camera motion are estimated.  ... 
arXiv:2006.06091v3 fatcat:nhdgivmtrzcarp463xzqvnxlwq

Deep Learning based Monocular Depth Prediction: Datasets, Methods and Applications [article]

Qing Li, Jiasong Zhu, Jun Liu, Rui Cao, Qingquan Li, Sen Jia, Guoping Qiu
2020 arXiv   pre-print
In this survey, we first introduce the datasets for depth estimation, and then give a comprehensive introduction of the methods from three perspectives: supervised learning-based methods, unsupervised  ...  Recently, monocular depth estimation has obtained great progress owing to the rapid development of deep learning techniques.  ...  Ranjan et al. [120] design a competitive and collaborative scheme to predict depth, ego-motion, optical flow and static and moving objects segmentation simultaneously.They divide the tasks into static  ... 
arXiv:2011.04123v1 fatcat:by6swdegvvdrxk73ti46k2rj2e

2021 Index IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 43

2022 IEEE Transactions on Pattern Analysis and Machine Intelligence  
The primary entry includes the coauthors' names, the title of the paper or other item, and its location, specified by the publication abbreviation, year, month, and inclusive pagination.  ...  The Subject Index contains entries describing the item under all appropriate subject headings, plus the first author's name, the publication abbreviation, month, and year, and inclusive pages.  ...  ., +, TPAMI Jan. 2021 269-283 Self-Paced Collaborative and Adversarial Network for Unsupervised Domain Adaptation.  ... 
doi:10.1109/tpami.2021.3126216 fatcat:h6bdbf2tdngefjgj76cudpoyia

2020 Index IEEE Transactions on Image Processing Vol. 29

2020 IEEE Transactions on Image Processing  
., +, TIP 2020 3845-3858 Unsupervised Learning of Optical Flow With CNN-Based Non-Local Filtering.  ...  ., +, TIP 2020 8916-8929 Unsupervised Learning of Optical Flow With CNN-Based Non-Local Filtering.  ... 
doi:10.1109/tip.2020.3046056 fatcat:24m6k2elprf2nfmucbjzhvzk3m
« Previous Showing results 1 — 15 out of 247 results