Dual camera snapshot hyperspectral imaging system via physics informed learning [article]

Hui Xie, Zhuang Zhao, Jing Han, Yi Zhang, Lianfa Bai, Jun Lu
2021 arXiv   pre-print
We consider using the system's optical imaging process with convolutional neural networks (CNNs) to solve the snapshot hyperspectral imaging reconstruction problem, which uses a dual-camera system to capture the three-dimensional hyperspectral images (HSIs) in a compressed way. Various methods using CNNs have been developed in recent years to reconstruct HSIs, but most of the supervised deep learning methods aimed to fit a brute-force mapping relationship between the captured compressed image
more » ... d standard HSIs. Thus, the learned mapping would be invalid when the observation data deviate from the training data. Especially, we usually don't have ground truth in real-life scenarios. In this paper, we present a self-supervised dual-camera equipment with an untrained physics-informed CNNs framework. Extensive simulation and experimental results show that our method without training can be adapted to a wide imaging environment with good performance. Furthermore, compared with the training-based methods, our system can be constantly fine-tuned and self-improved in real-life scenarios.
arXiv:2109.02643v2 fatcat:5wirlbqrnzhszmryoc3ku4nz5i