Single satellite imagery simultaneous super-resolution and colorization using multi-task deep neural networks

Heng Liu, Zilin Fu, Jungong Han, Ling Shao, Hongshen Liu
2018 Journal of Visual Communication and Image Representation  
8 Satellite imagery is a kind of typical remote sensing data, which holds preponderance in large area imaging and strong macro integrity. However, for most commercial space usages, such as virtual display of urban traffic flow, virtual interaction of environmental resources, one drawback of satellite imagery is its low spatial resolution, failing to provide the clear image details. Moreover, in recent years, synthesizing the color for grayscale satellite imagery or recovering the original color
more » ... of camouflage sensitive regions becomes an urgent requirement for large spatial objects virtual reality interaction. In this work, unlike existing works which solve these two problems separately, we focus on achieving image super-resolution (SR) and image colorization synchronously. Based on multi-task learning, we provide a novel deep neural network model to fulfill single satellite imagery SR and colorization simultaneously. By feeding back the color feature representations into the SR network and jointly optimizing such two tasks, our deep model successfully achieves the mutual cooperation between imagery reconstruction and image colorization. To avoid color bias, we not only adopt the non-satellite imagery Preprint submitted to Journal of visual communication and image representationMarch 1, 2018 to enrich the color diversity of satellite image, but also recalculate the prior color distribution and the valid color range based on the mixed data. We evaluate the proposed model on satellite images from different data sets, such as RSSCN7 and AID. Both the evaluations and comparisons reveal that the proposed multi-task deep learning approach is superior to the state-of-the-art methods, where image SR and colorization can be accomplished simultaneously and efficiently. 9 neural networks; Multi-task learning 10 *: We realize and train its caffe version. Highlights  We propose a multi-task deep neural model to achieve satellite imagery SR and colorization simultaneously. To the best of our knowledge, this is the first work which explores to achieve satellite imagery SR and colorization cooperatively.  We incorporate natural images with satellite data to enrich the color diversity in imagery colorization and we manage to realize the expectation color distribution learning to avoid color bias in colorization.  We introduce a novel multi-scale deep encoder-decoder symmetrical network for satellite imagery SR, where a residual structure is adopted to improve the imagery reconstruction performance.
doi:10.1016/j.jvcir.2018.02.016 fatcat:v5gmsy4r5nhp3og2itnfblkgvi