Multi-Sensor Image Fusion: A Survey of the State of the Art

Bing Li, Yong Xian, Daqiao Zhang, Juan Su, Xiaoxiang Hu, Weilin Guo
2021 Journal of Computer and Communications  
Image fusion has been developing into an important area of research. In remote sensing, the use of the same image sensor in different working modes, or different image sensors, can provide reinforcing or complementary information. Therefore, it is highly valuable to fuse outputs from multiple sensors (or the same sensor in different working modes) to improve the overall performance of the remote images, which are very useful for human visual perception and image processing task. Accordingly, in
more » ... this paper, we first provide a comprehensive survey of the state of the art of multi-sensor image fusion methods in terms of three aspects: pixel-level fusion, feature-level fusion and decision-level fusion. An overview of existing fusion strategies is then introduced, after which the existing fusion quality measures are summarized. Finally, this review analyzes the development trends in fusion algorithms that may attract researchers to further explore the research in this field. the scene [1]. Different sensor images have different advantages and disadvantages. For example, visible images can provide texture detail, along with high spatial resolution and high definition in a manner consistent with the human visual system, but do not work well in all-day/night conditions. By contrast, infrared images can distinguish targets from their backgrounds using differences in radiation, and work well in all-day/night conditions, but the infrared images are more blurred [2]. Moreover, Synthetic Aperture Radar (SAR) can better reflect the B. Li et al. Communications fusion schemes is provided in [23]. According to the adapted transformation strategy, Li et al. investigated various pixel-level image fusion algorithms, summarized the existing fusion performance evaluation methods and unresolved issues, and analyzed and summarized the main challenges encountered in different image fusion applications. Different from previous surveys, the purpose of this paper is to introduce the relevant fusion methods and applications recently introduced, which can provide new insights into the development of current image fusion theory and application [20]. In addition, the special issue [24] published in the Information Fusion journal by Goshtasby and Nikolov is an excellent source that tracks the development of image fusion methods. Furthermore, some research applying image fusion methods to certain specific application fields has also been published in recent years [25]-[30]. Taking remote sensing as an example, [25] summarized the early proposed remote sensing image fusion algorithms, [26] conducted a critical comparison of recently proposed remote sensing image fusion methods, [27] reviewed the current multi-source remote sensing data fusion techniques and discussed their future trends and challenges. In other applications, such as the medical imaging field, a practical list of methods was provided in [28], which also summarized the broad scientific challenges facing the field of medical image fusion. Based on the reported comparative results, recent image fusion and performance assessment algorithms were reviewed and categorized, after which a comprehensive evaluation of 40 fusion algorithms from recently published results was conducted to demonstrate their significance in terms of statistical analyses within their respective applications [29]. The remainder of this paper is organized as follows. Section 2 briefly reviews the popular and state-of-the-art fusion methods at different levels (namely the pixel-level, feature-level, and decision-level). Section 3 presents an overview of the fusion strategy. Moreover, an overview of the fusion performance assessment metrics is introduced in Section 4. Finally, some future trends and conclusions are summarized in Section 5. The novelty of the work in this paper can be summarized as follows: 1) this paper summarizes the existing multi-sensor image fusion algorithms, fusion strategies and evaluation indicators relatively completely, which has a reliable reference value for the subsequent image fusion researches; 2) unlike other image fusion reviews, this paper also summarizes image fusion algorithms based on the emerging theory-deep learning; 3) coupled with the analysis of the development trends in the field of image fusion, the paper provides a reference for researchers to further explore the research in this direction, which can promote the innovative development of this field. Multi-Sensor Image Fusion According to the fusion stage in the processing flow, image fusion can be broadly categorized into pixel-level fusion, feature-level fusion and decision-level fu- B. Li et al. Communications sion depending on the degree of information abstraction. In this section, we present a comprehensive review of multi-sensor image fusion methods of the pixel-level fusion, feature level fusion, and decision-level fusion varieties. B. Li et al.
doi:10.4236/jcc.2021.96005 fatcat:ulm4hbfy5ndkdne6fng6ncyn3i