6,446 Hits in 5.4 sec

Cycle-Consistent Generative Rendering for 2D-3D Modality Translation [article]

Tristan Aumentado-Armstrong, Alex Levinshtein, Stavros Tsogkas, Konstantinos G. Derpanis, Allan D. Jepson
2020 arXiv   pre-print
We can thus translate between the 2D visual and 3D structural modalities of a given object.  ...  For humans, visual understanding is inherently generative: given a 3D shape, we can postulate how it would look in the world; given a 2D image, we can infer the 3D structure that likely gave rise to it  ...  Overview of the cycle-consistent modality translation model.  ... 
arXiv:2011.08026v1 fatcat:irpf4btl6vdvvldr6zgqgfwxpi

MM-Hand: 3D-Aware Multi-Modal Guided Hand Generative Network for 3D Hand Pose Synthesis [article]

Zhenyu Wu, Duc Hoang, Shih-Yao Lin, Yusheng Xie, Liangjian Chen, Yen-Yu Lin, Zhangyang Wang, Wei Fan
2020 arXiv   pre-print
We propose a 3D-aware multi-modal guided hand generative network (MM-Hand), together with a novel geometry-based curriculum learning strategy.  ...  Moreover, the augmented data can consistently improve the quantitative performance of the state-of-the-art 3D hand pose estimators on two benchmark datasets.  ...  CycleGAN learns unpaired image-to-image translation, by enforcing cycle consistency to push the source domain of 2D pose maps to be consistent with the target domain of realistic hand images.  ... 
arXiv:2010.01158v1 fatcat:j42oeeqlanct7mtwgozazmnk6e

Transfer Learning from an Artificial Radiograph-landmark Dataset for Registration of the Anatomic Skull Model to Dual Fluoroscopic X-ray Images [article]

Chaochao Zhou, Thomas Cha, Yun Peng, Guoan Li
2021 arXiv   pre-print
Landmarks on the X-rays experiencing GAN style translation were detected by the ResNet, and were used in triangulation optimization for 3D-to-2D registration of the skull in actual dual-fluoroscope images  ...  They were used to train a residual network (ResNet) for landmark detection and a cycle generative adversarial network (GAN) to eliminate the style difference between DRRs and actual X-rays.  ...  Acknowledgements 345 We are thankful for the support from National Institutes of Health (1R03AG056897), USA. 346  ... 
arXiv:2108.06466v1 fatcat:mrj62z7jyvawvnomoyxmlv7ii4

A^3DSegNet: Anatomy-aware artifact disentanglement and segmentation network for unpaired segmentation, artifact reduction, and modality translation [article]

Yuanyuan Lyu, Haofu Liao, Heqin Zhu, S. Kevin Zhou
2021 arXiv   pre-print
Finally, we stack 2D slices together and build 3D networks on top to obtain final 3D segmentation result.  ...  each task and, remarkably, it achieves an average Dice coefficient of 0.926 for unpaired 3D CBCT vertebra segmentation.  ...  translation tasks. 2D segmentation.  ... 
arXiv:2001.00339v3 fatcat:v7orgxdqrrfj3b7oldzdtwurmu

Blending Generative Adversarial Image Synthesis with Rendering for Computer Graphics [article]

Ekim Yurtsever, Dongfang Yang, Ibrahim Mert Koc, Keith A. Redmill
2020 arXiv   pre-print
Conventional computer graphics pipelines require detailed 3D models, meshes, textures, and rendering engines to generate 2D images from 3D scenes. These processes are labor-intensive.  ...  Our main idea is straightforward: Given a 3D scene, render only important objects of interest and use generative adversarial processes for synthesizing the rest of the image.  ...  First, G(x) generates fakeŷ, then F (G(x)) translates the fakeŷ back tox. If the cycle is consistent, then x ≈x. x → G(x) → F (G(x)) ≈ x.  ... 
arXiv:2007.15820v1 fatcat:3q4meccswjatfaslykqvniuite

A Survey on GAN-Based Data Augmentation for Hand Pose Estimation Problem

Farnaz Farahanipad, Mohammad Rezaei, Mohammad Sadegh Nasr, Farhad Kamangar, Vassilis Athitsos
2022 Technologies  
dataset with different modalities.  ...  Next, we present related hand pose datasets and performance comparison of some of these methods for the hand pose estimation problem.  ...  for generating data in different modalities.  ... 
doi:10.3390/technologies10020043 fatcat:6ljh7d4ijrfsdgigm5zn4lg54y

Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Images [article]

Hang Zhou, Jihao Liu, Ziwei Liu, Yu Liu, Xiaogang Wang
2020 arXiv   pre-print
Our key insight is that rotating faces in the 3D space back and forth, and re-rendering them to the 2D plane can serve as a strong self-supervision.  ...  Since the 3D rotation-and-render on faces can be applied to arbitrary angles without losing details, our approach is extremely suitable for in-the-wild scenarios (i.e. no paired data are available), where  ...  We thank Hao Shao for helpful discussions.  ... 
arXiv:2003.08124v1 fatcat:zsveqwdvgraj7m2d7wfq6citja

Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition [article]

Hassan Abu Alhaija, Siva Karthik Mustikovela, Justus Thies, Varun Jampani, Matthias Nießner, Andreas Geiger, Carsten Rother
2021 arXiv   pre-print
More specifically, we propose an autoencoder for joint generation of realistic images from synthetic 3D models while simultaneously decomposing real images into their intrinsic shape and appearance properties  ...  While several supervised methods have been proposed for this task, acquiring a dataset of images with accurately aligned 3D models is very difficult.  ...  In the following, we detail our cycle consistency losses and the novel shared adversarial losses. Cycle Consistency Rendering Cycle.  ... 
arXiv:2006.16011v3 fatcat:xjksrt7jyjhk7ehl5xpga75k6y

Eidolon: Visualization and Computational Framework for Multi-modal Biomedical Data Analysis [chapter]

Eric Kerfoot, Lauren Fovargue, Simone Rivolo, Wenzhe Shi, Daniel Rueckert, David Nordsletten, Jack Lee, Radomir Chabiniok, Reza Razavi
2016 Lecture Notes in Computer Science  
Biomedical research, combining multi-modal image and geometry data, presents unique challenges for data visualization, processing, and quantitative analysis.  ...  Medical imaging provides rich information, from anatomical to deformation, but extracting this to a coherent picture across image modalities with preserved quality is not trivial.  ...  Acknowledgements This research was partly supported by the National Institute for Health Research (NIHR) Biomedical Research Centre (BRC), and by the NIHR Healthcare Technology Co-operative for Cardiovascular  ... 
doi:10.1007/978-3-319-43775-0_39 fatcat:iktzb4ta5bhonlag6uoffmnbqu

Technical Principles of Transthoracic Three-Dimensional Echocardiography [chapter]

Stein Inge Rabben
2010 Textbook of Real-Time Three Dimensional Echocardiography  
Today a typical 2D array transducer consists of 2,000-3,000 elements arranged in rows and columns.  ...  Section 2.5 covers the different image displays (slice, volume and surface renderings) made available with 3D ultrasound.  ...  Special thanks to Jøger Hansegård and Jan Yee for help with making the figures and to Svein Brekke who let me adapt three of his figures (beamforming, subvolume stitching and geometrical distortion).  ... 
doi:10.1007/978-1-84996-495-1_2 fatcat:iudt2cuo3rfbbadvxkxwt7s6ga

A Comprehensive Study on Deep Learning-Based 3D Hand Pose Estimation Methods

Theocharis Chatzis, Andreas Stergioulas, Dimitrios Konstantinidis, Kosmas Dimitropoulos, Petros Daras
2020 Applied Sciences  
In this survey, we provide a comprehensive study of the most representative deep learning-based methods in literature and propose a new taxonomy heavily based on the input data modality, being RGB, depth  ...  The field of 3D hand pose estimation has been gaining a lot of attention recently, due to its significance in several applications that require human-computer interaction (HCI).  ...  The translation GAN was trained using the cycle consistency with a geometric consistency loss that preserved the hand pose during translation.  ... 
doi:10.3390/app10196850 fatcat:hgyqkoyetbbilncksguarqz3bq

Neural Mesh Refiner for 6-DoF Pose Estimation [article]

Di Wu, Yihao Chen, Xianbiao Qi, Yongjian Yu, Weixuan Chen, Rong Xiao
2020 arXiv   pre-print
This paper bridges the gap between 2D mask generation and 3D location prediction via a differentiable neural mesh renderer.  ...  However, due to a lack of geometrical scene understanding from the directly regressed pose estimation, there are misalignments between the rendered mesh from the 3D object and the 2D instance segmentation  ...  The goal is to match 3D mesh generated from R and T with 2D mask predictions M.  ... 
arXiv:2003.07561v3 fatcat:3kst5u2mn5dm3npmduyolkls4e

Three-dimensional display of cortical anatomy and vasculature: Magnetic resonance angiography versus multimodality integration

Christopher J. Henri, G. Bruce Pike, D. Louis Collins, Terence M. Peters
1991 Journal of digital imaging  
Here, true 3D data are acquired and displayed using volume or surface rendering techniques.  ...  We present two methods for acquiring and viewing integrated three-dimensional (3D) images of cerebral vasculature and cortical anatomy.  ...  Marrett for their interest and assistance in this work.  ... 
doi:10.1007/bf03173871 pmid:2029569 fatcat:d3vvafr2jjgnlob2c2yluksivi

Unsupervised Video-to-Video Translation [article]

Dina Bashkirova, Ben Usman, Kate Saenko
2018 arXiv   pre-print
translation using existing image-to-image translation networks, and propose a spatio-temporal 3D translator as an alternative solution to this problem.  ...  We evaluate our 3D method on multiple synthetic datasets, such as moving colorized digits, as well as the realistic segmentation-to-video GTA dataset and a new CT-to-MRI volumetric images translation dataset  ...  Frame-wise translation (2D) produces plausible images, but does not preserve temporal consistency.  ... 
arXiv:1806.03698v1 fatcat:4rwttfxyt5cq5o4gbbk5r2mu6e

Egocentric and exocentric teleoperation interface using real-time, 3D video projection

François Ferland, François Pomerleau, Chon Tam Le Dinh, François Michaud
2009 Proceedings of the 4th ACM/IEEE international conference on Human robot interaction - HRI '09  
The user interface is the central element of a telepresence robotic system and its visualization modalities greatly affect the operator's situation awareness, and thus its performance.  ...  Our system, which combines a 3D reconstruction of the environment using laser range finder readings with two video projection methods, allows the operator to easily switch from ego-to exocentric viewpoints  ...  gratefully acknowledge the contribution of the Natural Sciences and Engineering Research Council of Canada (NSERC), the Canada Research Chairs (CRC) and the Fondation of the Université de Sherbrooke for  ... 
doi:10.1145/1514095.1514105 dblp:conf/hri/FerlandPDM09 fatcat:wobnn4rt2rbwzmbqeb7athb5fq
« Previous Showing results 1 — 15 out of 6,446 results