Machine Learning For Performance Improvement of Long-Haul End-to-End Optical Transmission Systems

Abstract

The thesis focuses on addressing the challenges faced by optical fiber networks in keeping up with the growing demand for data transfer, especially with the advent of 5G/6G and the Internet of Things (IoT). The rapid expansion in data transfer requirements highlights the limitations of current optical fiber networks and the necessity for improvements in data encoding techniques, spectrumutilization, and signal clarity over long distances. The thesis contributes to this field by developing new methods for applying the Nonlinear Fourier Transform (NFT) to continuous signals, improving signal processing algorithms, and using Machine learning (ML) to understand complex patterns and make data-driven decisions to optimize optical communication systems. The work is divided into two primary sections. The first section delves into advanced NFT techniques, including their application in optical fiber channel modeling for single and dualpolarization systems, signal processing with a sliding window technique combined with NFT, exploring solitonic components in optical signals, and the use of neural networks for NFT to work with noisy signals. The second section is dedicated to the role of ML in optimizing optical communication systems, discussing the new High-Performance COMmunication library (Hp-Com) framework for simulating optical channels, the use of Gradient Boosting for nonlinear equalization, studying received symbol distributions using the GaussianMixtureModel, and summarizing findings with insights for future research. The thesis outlines the creation of innovative techniques to improve optical fiber systems, thus aiding the continued development of the digital world by handling the ever-increasing demands for data transmission.