Filters








6,862 Hits in 3.5 sec

Learned Semantic Multi-Sensor Depth Map Fusion [article]

Denys Rozumnyi, Ian Cherabier, Marc Pollefeys, Martin R. Oswald
2019 arXiv   pre-print
In this paper, we are generalizing this classic method in multiple ways: 1) Semantics: Semantic information enriches the scene representation and is incorporated into the fusion process. 2) Multi-Sensor  ...  Our method learns sensor or algorithm properties jointly with semantic depth fusion and scene completion and can also be used as an expert system, e.g. to unify the strengths of various photometric stereo  ...  accurate semantic depth map fusion.  ... 
arXiv:1909.00703v1 fatcat:p36xx7jrdvbi5gjirs6mxefkwm

Learned Semantic Multi-Sensor Depth Map Fusion

Denys Rozumnyi, Ian Cherabier, Marc Pollefeys, Martin Oswald
2019 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)  
In this paper, we are generalizing this classic method in multiple ways: 1) Semantics: Semantic information enriches the scene representation and is incorporated into the fusion process. 2) Multi-Sensor  ...  Our method learns sensor or algorithm properties jointly with semantic depth fusion and scene completion and can also be used as an expert system, e.g. to unify the strengths of various photometric stereo  ...  accurate semantic depth map fusion.  ... 
doi:10.1109/iccvw.2019.00264 dblp:conf/iccvw/RozumnyiCPO19 fatcat:fcnynqxognampf45ckxc2h32ly

An Outline of Multi-Sensor Fusion Methods for Mobile Agents Indoor Navigation

Yuanhao Qu, Minghao Yang, Jiaqing Zhang, Wu Xie, Baohua Qiang, Jinlong Chen
2021 Sensors  
the mainstream technologies of multi-sensor fusion methods, including various combinations of sensors and several widely recognized multi-modal sensor datasets.  ...  In spite of the high performance of the single-sensor navigation method, multi-sensor fusion methods still potentially improve the perception and navigation abilities of mobile agents.  ...  The fusion of RGB images and depth information helps semantic SLAM, mainly embodied in the semantic segmentation of environmental targets, while those based on deep learning are the common methods.  ... 
doi:10.3390/s21051605 pmid:33668886 pmcid:PMC7956205 fatcat:7twh225phbbupjd4fv2fw6twum

Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review [article]

Yaodong Cui, Ren Chen, Wenbo Chu, Long Chen, Daxin Tian, Ying Li, Dongpu Cao
2020 arXiv   pre-print
Followed by in-depth reviews of camera-LiDAR fusion methods in depth completion, object detection, semantic segmentation, tracking and online cross-sensor calibration, which are organized based on their  ...  In particular, the camera-LiDAR fusion is becoming an emerging research theme. However, so far there has been no critical review that focuses on deep-learning-based camera-LiDAR fusion methods.  ...  To achieve this goal, we can adopt multiple fusion modules with different sensor inputs, or a multi-path fusion module of asynchronous multi-modal data.  ... 
arXiv:2004.05224v2 fatcat:6dgjxdnx5baxxaohfwec7idoi4

Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges [article]

Di Feng, Christian Haase-Schuetz, Lars Rosenbaum, Heinz Hertlein, Claudius Glaeser, Fabian Timm, Werner Wiesbeck, Klaus Dietmayer
2020 arXiv   pre-print
This review paper attempts to systematically summarize methodologies and discuss challenges for deep multi-modal object detection and semantic segmentation in autonomous driving.  ...  Recent advancements in perception for autonomous driving are driven by deep learning.  ...  We first provide background information on multi-modal sensors, test vehicles, and modern deep learning approaches in object detection and semantic segmentation in Sec. II.  ... 
arXiv:1902.07830v4 fatcat:or6enjxktnamdmh2yekejjr4re

SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion [article]

Danish Nazir, Marcus Liwicki, Didier Stricker, Muhammad Zeshan Afzal
2022 arXiv   pre-print
In addition, we also propose to apply semantic-aware multi-modal attention-based fusion block (SAMMAFB) to fuse features between all three branches.  ...  The predicted dense depth map of color-guided branch along-with semantic image and sparse depth map is passed as input to semantic-guided branch for estimating semantic depth.  ...  SEMANTIC-AWARE MULTI-MODAL ATTENTION-BASED FUSION BLOCK In order to perform fusion between RGB, semantic, and depth features, we propose to apply Semantic-aware Multi-Modal Attention-based Fusion Block  ... 
arXiv:2204.13635v1 fatcat:oxtd5e7xjvcavmodbpruvcwgt4

Multi-Task Multi-Sensor Fusion for 3D Object Detection

Ming Liang, Bin Yang, Yun Chen, Rui Hu, Raquel Urtasun
2019 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  
In this paper we propose to exploit multiple related tasks for accurate multi-sensor 3D object detection.  ...  Our experiments show that all these tasks are complementary and help the network learn better representations by fusing information at various levels.  ...  Point-wise and ROI-wise feature fusion are applied to achieve full multi-sensor fusion, while multi-task learning provides additional map prior and geometric clues enabling better representation learning  ... 
doi:10.1109/cvpr.2019.00752 dblp:conf/cvpr/LiangYCHU19 fatcat:j2u7e6v2ebgjhmz6z5etjpw7sy

A survey of LiDAR and camera fusion enhancement

Huazan Zhong, Hao Wang, Zhengrong Wu, Chen Zhang, Yongwei Zheng, Tao Tang
2021 Procedia Computer Science  
This paper briefly reviews the fusion and enhancement systems between both two sensors in the field of depth completion, 3D object detection, 2D\3D semantic segmentation and 3D object tracking.  ...  This paper briefly reviews the fusion and enhancement systems between both two sensors in the field of depth completion, 3D object detection, 2D\3D semantic segmentation and 3D object tracking.  ...  Valada et al. 8 designed a multi-level feature-level fusion model of different depths to promote semantic segmentation.  ... 
doi:10.1016/j.procs.2021.02.100 fatcat:3t65dnpajzcyfpv3lak3mor76a

A Survey of Simultaneous Localization and Mapping with an Envision in 6G Wireless Networks [article]

Baichuan Huang, Jun Zhao, Jingbin Liu
2020 arXiv   pre-print
For Lidar and visual fused SLAM, the paper highlights the multi-sensors calibration, the fusion in hardware, data, task layer.  ...  For Lidar or visual SLAM, the survey illustrates the basic type and product of sensors, open source system in sort and history, deep learning embedded, the challenge and future.  ...  The key contribution is a multi-resolution depth estimation and spatial smoothing process. • Kinect Fusion: it (RGB-D) is almost the first 3D reconstruction system with depth camera [99] [100]. • RTAB-MAP  ... 
arXiv:1909.05214v4 fatcat:itnluvkewfd6fel7x65wdgig3e

The Fusion Strategy of 2D and 3D Information Based on Deep Learning: A Review

Jianghong Zhao, Yinrui Wang, Yuee Cao, Ming Guo, Xianfeng Huang, Ruiju Zhang, Xintong Dou, Xinyu Niu, Yuanyuan Cui, Jun Wang
2021 Remote Sensing  
Recently, researchers have realized a number of achievements involving deep-learning-based neural networks for the tasks of segmentation and detection based on 2D images, 3D point clouds, etc.  ...  Using 2D and 3D information fusion for the advantages of compensation and accuracy improvement has become a hot research topic.  ...  The multi-task and multi-sensor fusion method created by Liang et al. [198] is an end-toend network which takes the advantages of both point-wise and ROI-wise feature fusion.  ... 
doi:10.3390/rs13204029 fatcat:onnjeqvwb5gsjcrhdaq6hiekru

Multi-modal Sensor Fusion-Based Deep Neural Network for End-to-end Autonomous Driving with Scene Understanding [article]

Zhiyu Huang, Chen Lv, Yang Xing, Jingda Wu
2020 arXiv   pre-print
The designed end-to-end deep neural network takes the visual image and associated depth information as inputs in an early fusion level and outputs the pixel-wise semantic segmentation as scene understanding  ...  deep neural network with multimodal sensor fusion.  ...  The system is similar to the multi-task learning model [14] with removal of the depth estimation.  ... 
arXiv:2005.09202v2 fatcat:fnr5xcrsvvflfikly7tejhvgpu

Atlas: End-to-End 3D Scene Reconstruction from Posed Images [article]

Zak Murez, Tarrence van As, James Bartolozzi, Ayan Sinha, Vijay Badrinarayanan, Andrew Rabinovich
2020 arXiv   pre-print
We compare our 3D semantic segmentation to prior methods that use a depth sensor since no previous work attempts the problem with only RGB input.  ...  Traditional approaches to 3D reconstruction rely on an intermediate representation of depth maps prior to estimating a full 3D model of a scene.  ...  TSDF refinement starts by fusing depth maps from a depth sensor into an initial voxel volume using TSDF fusion [10] , in which each voxel stores the truncated signed distance to the nearest surface.  ... 
arXiv:2003.10432v3 fatcat:g3p5j6sdxbb5lb3aoaeqngxfcy

Evaluation of Multimodal Semantic Segmentation using RGB-D Data [article]

Jiesi Hu, Ganning Zhao, Suya You, C. C. Jay Kuo
2021 arXiv   pre-print
In addition, we also develop new strategies of multi-datasets learning to improve the detection and recognition of unseen objects.  ...  This work reports our efforts on the evaluation of a state-of-the-art approach for semantic segmentation with multiple RGB and depth sensing data.  ...  Using multi-sensors such as RGB camera, depth camera, and LiDAR sensors can make the perception model more robust under various conditions.  ... 
arXiv:2103.16758v1 fatcat:mfqahmpchvcwdohiumyq5iuywq

Multi-Modal 3D Object Detection in Autonomous Driving: a Survey [article]

Yingjie Wang, Qiuyu Mao, Hanqi Zhu, Yu Zhang, Jianmin Ji, Yanyong Zhang
2021 arXiv   pre-print
Then we present in-depth reviews of recent multi-modal 3D detection networks by considering the following three aspects of the fusion: fusion location, fusion data representation, and fusion granularity  ...  So far, there has been no indepth review that focuses on multi-sensor fusion based perception.  ...  MMF Through image and LiDAR sensor fusion to learn multiple related tasks, Realize 3D and 2D target detection, ground estimation and depth completion at the same time.  ... 
arXiv:2106.12735v2 fatcat:5twzbk4yhrcfzddp7zghnsivna

Real-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-driving Images [article]

Lei Sun, Kailun Yang, Xinxin Hu, Weijian Hu, Kaiwei Wang
2020 arXiv   pre-print
However, few real-time RGB-D fusion semantic segmentation studies are carried out despite readily accessible depth information nowadays.  ...  In this paper, we propose a real-time fusion semantic segmentation network termed RFNet that effectively exploits complementary cross-modal information.  ...  RGB-D Semantic Segmentation High-quality dense depth maps from depth sensors like Kinect and RealSense boost the development of indoor semantic segmentation.  ... 
arXiv:2002.10570v2 fatcat:tfnxpu5mxbclla5icoe25vwys4
« Previous Showing results 1 — 15 out of 6,862 results