A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf
.
Filters
Multi-Modal Fusion Transformer for End-to-End Autonomous Driving
[article]
2021
arXiv
pre-print
Therefore, we propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. ...
Geometry alone may therefore be insufficient for effectively fusing representations in end-to-end driving models. ...
Related Work Multi-Modal Autonomous Driving: Recent multi-modal methods for end-to-end driving [58, 65, 51, 3] have shown that complementing RGB images with depth and semantics has the potential to improve ...
arXiv:2104.09224v1
fatcat:au3nqx7kwfds7kqymx2jceq5ga
Software/Hardware Co-design for Multi-modal Multi-task Learning in Autonomous Systems
[article]
2021
arXiv
pre-print
., multi-modal data from different sensors, requiring diverse data preprocessing, sensor fusion, and feature aggregation. ...
Therefore, autonomous systems essentially require multi-modal multi-task (MMMT) learning which must be aware of hardware performance and implementation strategies. ...
[7] propose an end-to-end model for multi-modal sensor fusion with visual and depth information of images. Multi-task learning (MTL). ...
arXiv:2104.04000v1
fatcat:vf673pujtvhg7nyqrlppmsk2fa
Navigating an Automated Driving Vehicle via the Early Fusion of Multi-Modality
2022
Sensors
This paper focuses on the early fusion of multi-modality and demonstrates how it outperforms a single modality using the CARLA simulator. ...
The latter is less well-studied but is becoming more popular since it is easier to use. This article focuses on end-to-end autonomous driving, using RGB pictures as the primary sensor input data. ...
Conclusions This paper presents a comparison of single-and multi-modal perception data for end-to-end driving. ...
doi:10.3390/s22041425
pmid:35214327
pmcid:PMC8878300
fatcat:g7ti567yn5h6jbxbazzeoprnt4
Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer
[article]
2022
arXiv
pre-print
In this paper, we propose a safety-enhanced autonomous driving framework, named Interpretable Sensor Fusion Transformer(InterFuser), to fully process and fuse information from multi-modal multi-view sensors ...
However, reasoning from a global context requires access to sensors of multiple types and adequate fusion of multi-modal sensor signals, which is difficult to achieve. ...
Conclusion We present InterFuser, a new design for autonomous driving based on an interpretable sensor fusion transformer. ...
arXiv:2207.14024v2
fatcat:y7n2cr2tl5gyhofmfj2ap2rruq
3D Object Detection for Autonomous Driving: A Review and New Outlooks
[article]
2022
arXiv
pre-print
Autonomous driving, in recent years, has been receiving increasing attention for its potential to relieve drivers' burdens and improve the safety of driving. ...
In modern autonomous driving pipelines, the perception system is an indispensable component, aiming to accurately estimate the status of surrounding environments and provide reliable observations for prediction ...
End-to-end learning for autonomous driving. ...
arXiv:2206.09474v1
fatcat:3skws77uqngjtpo6mycpo4dhny
Deep Multi-modal Object Detection for Autonomous Driving
2021
2021 18th International Multi-Conference on Systems, Signals & Devices (SSD)
In this paper, we present methods that have been proposed in the literature for the different deep multi-modal perception techniques. ...
Robust perception in autonomous vehicles is a huge challenge that is the main tool for detecting and tracking the different kinds of objects around the vehicle. ...
CARLA (Intel): an open-source simulator for autonomous driving research. CARLA is a platform for the evaluation of autonomous urban driving systems. ...
doi:10.1109/ssd52085.2021.9429355
fatcat:ek3qpf3dzbed7iyvkcwk6mv4hu
TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving
[article]
2022
arXiv
pre-print
How should we integrate representations from complementary sensors for autonomous driving? Geometry-based fusion has shown promise for perception (e.g. object detection, motion forecasting). ...
However, in the context of end-to-end driving, we find that imitation learning based on existing sensor fusion methods underperforms in complex driving scenarios with a high density of dynamic agents. ...
RELATED WORK Multi-Modal Autonomous Driving: Recent multi-modal methods for end-to-end driving [35] , [42] - [45] have shown that complementing RGB images with depth and semantics has the potential ...
arXiv:2205.15997v1
fatcat:nyteapdbr5dqbadxliletwnzcy
Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
[article]
2020
arXiv
pre-print
This review paper attempts to systematically summarize methodologies and discuss challenges for deep multi-modal object detection and semantic segmentation in autonomous driving. ...
To this end, we first provide an overview of on-board sensors on test vehicles, open datasets, and background information for object detection and semantic segmentation in autonomous driving research. ...
ACKNOWLEDGMENT We thank Fabian Duffhauss for collecting literature and reviewing the paper. ...
arXiv:1902.07830v4
fatcat:or6enjxktnamdmh2yekejjr4re
FUTR3D: A Unified Sensor Fusion Framework for 3D Detection
[article]
2022
arXiv
pre-print
FUTR3D employs a query-based Modality-Agnostic Feature Sampler (MAFS), together with a transformer decoder with a set-to-set loss for 3D detection, thus avoiding using late fusion heuristics and post-processing ...
In this work, we propose the first unified end-to-end sensor fusion framework for 3D detection, named FUTR3D, which can be used in (almost) any sensor configuration. ...
systems for autonomous driving. ...
arXiv:2203.10642v1
fatcat:6ggsokiqoves3pzqsybi3hikuq
Multi-Modality Cascaded Fusion Technology for Autonomous Driving
[article]
2020
arXiv
pre-print
Multi-modality fusion is the guarantee of the stability of autonomous driving systems. ...
Last, the proposed step-by-step cascaded fusion framework is more interpretable and flexible compared to the end-toend fusion methods. ...
In autonomous driving, thanks to the arise of deep learning, not only single sensor research has made great progress [10, 13, 11, 18] , such as camera, LiDAR, multi-modality fusion has also gain increasing ...
arXiv:2002.03138v1
fatcat:szse6ak5ffemvmmc6mrwc6ucly
Autonomous Navigation in Complex Environments with Deep Multimodal Fusion Network
[article]
2020
arXiv
pre-print
We further show that the use of multiple modalities is essential for autonomous navigation in complex environments. ...
We then propose a Navigation Multimodal Fusion Network (NMFNet) which has three branches to effectively handle three visual modalities: laser, RGB images, and point cloud data. ...
In [27] , Bojarski et al. proposed the first end-to-end navigation system for autonomous car using 2D images. Smolyanskiy et al. ...
arXiv:2007.15945v1
fatcat:aoyrmjgue5fl5jpnx7xsr3jsgm
Multi-Modal Fusion for Sensorimotor Coordination in Steering Angle Prediction
[article]
2022
arXiv
pre-print
This work explores the fusion of frame-based RGB and event data for learning end-to-end lateral control by predicting steering angle. ...
To this end, we propose DRFuser, a novel convolutional encoder-decoder architecture for learning end-to-end lateral control. ...
However, the vision modality that is frame-based RGB cameras has shown impressive results for end-to-end driving [3] [4] [5] [6] . ...
arXiv:2202.05500v1
fatcat:yk7e4qskxfashdbpawbkt27up4
Robust Sensor Fusion Algorithms Against Voice Command Attacks in Autonomous Vehicles
[article]
2021
arXiv
pre-print
With recent advances in autonomous driving, Voice Control Systems have become increasingly adopted as human-vehicle interaction methods. ...
To this end, we propose a novel multimodal deep learning classification system to defend against inaudible command attacks. ...
On the other hand, multi-modality sensor fusion algorithms have been widely adopted for Unmanned Aerial Vehicle (UAV) landing and autonomous driving needs to provide requested robustness and safety assurance ...
arXiv:2104.09872v3
fatcat:orgbuv7b4reyzpv76s5amtnxzy
MMFN: Multi-Modal-Fusion-Net for End-to-End Driving
[article]
2022
arXiv
pre-print
Inspired by the fact that humans use diverse sensory organs to perceive the world, sensors with different modalities are deployed in end-to-end driving to obtain the global context of the 3D scene. ...
In previous works, camera and LiDAR inputs are fused through transformers for better driving performance. ...
We also thank the anonymous reviewers for their constructive comments. ...
arXiv:2207.00186v2
fatcat:7fa3oo54ybc5dpsgoy5533hhtq
BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation
[article]
2022
arXiv
pre-print
Multi-sensor fusion is essential for an accurate and reliable autonomous driving system. Recent approaches are based on point-level fusion: augmenting the LiDAR point cloud with camera features. ...
In this paper, we break this deeply-rooted convention with BEVFusion, an efficient and generic multi-task multi-sensor fusion framework. ...
We would like to thank Xuanyao Chen and Brady Zhou for their guidance on detection and segmentation evaluation, and Yingfei Liu and Tiancai Wang for their helpful discussions. ...
arXiv:2205.13542v2
fatcat:qtunylgozjcvrdrjzdk23xjpve
« Previous
Showing results 1 — 15 out of 6,526 results