In space image processing using AI embedded on system on module: example of OPS-SAT cloud segmentation

Frédéric Férésin, Erwann Kervennic, Yves Bobichon, Edgar Lemaire, Nassim Abderrahmane, Gaétan Bahl, Ingrid Grenet, Matthieu Moretti, Michael Benguigui
2021 Zenodo  
During the OBPDC-2020 conference held last year, we presented the publication "Onboard image processing using AI to reduce data transmission: example of OPS-SAT cloud segmentation". In this paper, we explained how we implemented three Artificial Neural Networks (ANNs) on OPS-SAT FPGA to perform cloud segmentation based on: - A classical LeNet-5 architecture, - A fully convolutional architecture, - A hybrid convolutional / spiking architecture. Cloud segmentation is a useful onboard service to
more » ... lter unnecessary data and to preserve the limited storage and bandwidth of nanosats. This service is also compatible with OPS-SAT spatial resolution and the number of logic cells within its Cyclone V FPGA. In the OBPDC-2020 paper, we detailed several challenges we had to tackle to achieve OPS-SAT implementation, specifically: - Dataset engineering, which was made difficult by the fact that no actual OPS-SAT images were available at the time of ANN trainings, - ANN architectures selection, which was almost completely driven by the execution target capability and required to come up with tiny designs, - Hardware acceleration of the trained ANNs, using a VHDL based solution specifically developed to target OPS-SAT FPGA on Cyclone-V System on Chip. In the continuity of the OBPDC-2020 paper, we propose for the OBDP-2021 conference to report the in-flight inferences of our ANNs on FPGA that is probably a world first. We will discuss the different parameters affecting the overall performance measured onboard OPS-SAT, while presenting, at least for one reference ANN, the impact of the different deployment steps on the inference metrics (in full precision on CPU, quantified on the validation board, and in-flight). Then we will propose relevant improvements. We will especially analyze the generalization capability of the trained ANNs on real OPS-SAT images. Since these images display a wide variety of solar irradiance and geometry, we tested different kinds of pre- [...]
doi:10.5281/zenodo.5574959 fatcat:bjioh3f2gnfktp2xw2jfysklje