This research paper presents a novel high bit rate and spectral efficient 20 Gb/s mode division multiplexed (MDM) radio over free space optical communication (MDM-RoFSO) system. 10 Gb/s MZM/QAM-16 modulated data streams, each from two distinct channels, are transported over two spirals phased Hermite Gaussian (HG) laser modes, HG00 and HG01 respectively. These two HG channels are multiplexed using MDM at a 10 GHz RF signal, amplified and transmitted over the atmospheric link at an optical

more »

... ngth of 1550 nm. The proposed MDM-RoFSO system is simulated for MZM modulation over two distinct HG modes in clear sky, dense haze, dense fog, and strong rain weather conditions. The performance of the system is evaluated using performance metrics such as BER, eye diagram, Q factor, and link transmission range. The simulation results show that the HG00 mode is more robust and the extended transmission range is achieved from (7 to 24.4) km for (24.4 to 2.98) Q factor for clear sky to strong rain conditions. This transmission range is further extended by the factor of (2.9 to 1.7) km when the QAM-16 modulation is incorporated in composite (HG00 + HG01) mode and then transmitted in similar weather conditions for the acceptable BER of < 10− 5. Finally, the simulation results confirm that the QAM-16 modulated composite HG mode is able to mitigate the atmospheric weather conditions and travel a longer transmission distance compared to individual HG mode transmission.