A Novel Multitemporal Deep Fusion Network (MDFN) for Short-term Multitemporal HR Images Classification

Yongjie Zheng, Sicong Liu, Qian Du, Hui Zhao, Xiaohua Tong, Michele Dalponte
2021 IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing  
High-resolution (HR) satellite images, due to the technical constraints on spectral and spatial resolutions, usually contain only several broad spectral bands but with a very high spatial resolution. This provides rich spatial details of the objects on the Earth surface, while their spectral discrimination is relatively low. Recently, the increase of the satellite revisit times made it possible to acquire more frequent data coverage for finer classification. In this paper, we proposed a novel
more » ... ltitemporal deep fusion network (MDFN) for short-term multitemporal HR images classification. Specifically, a two-branch structure of MDFN is designed, which includes a long short-term memory (LSTM) and a convolutional neural network (CNN). The LSTM branch is mainly used to learn the joint expression of different temporal-spectral features. For the CNN branch, the 3D convolution is firstly applied along the temporal and spectral dimensions to jointly learn the temporal-spatial and spectral-spatial information, respectively, and then the 2D convolution is performed along the spatial dimension to further extract the spatial context information. Finally, features generated from the two different branches are fused to obtain the discriminative high-level semantic information for classification. Experimental results carried on two real multitemporal HR remote sensing data sets demonstrate that the proposed MDFN provides better classification performance over the state-of-the-art methods, and it also shows the potentiality to use short-term multitemporal HR images for more accurate Land Use/Land Cover (LULC) mapping.
doi:10.1109/jstars.2021.3119942 fatcat:edgi3fvdf5d5pf7yn2fo4bqv7q