Machine learning-based nonlinear Fourier transform for finite-genus solutions: implementation and application in fibre-optic communications

Abstract

The constantly growing demand for fibre-optic communication traffic motivates researchers to develop new data transmission approaches. The nonlinear Fourier transform (NFT) technique effectively linearizes an information channel and has the potential to overcome the nonlinear capacity limit. However, this method has not been studied thoroughly, especially its counterpart – the periodic NFT. In the context of fibre-optic communication, the periodic NFT is closely related to the finite-genus solutions of the nonlinear Schrodinger equation (NLSE). Previously, analysis of data transmission systems with finite-genus solutions was performed, but the capacity was underestimated due to the restrictions of the periodic NFT. This thesis is devoted to developing the NFT for finite-genus solutions, avoiding any limitations, and providing a fair analysis of the corresponding communication systems. The complete NFT framework for finite-genus solutions to the NLSE is developed in the thesis. The Riemann-Hilbert problem (RHP) parametrization of finite-genus solutions is exploited. Among the operations constituting the NFT, the inverse problem and the evolution of scattering data are defined in the RHP method, while solving the direct problem is limited. This transformation is performed with a convolutional neural network that lifts existing restrictions. With this neural network-based direct transform, the NFT framework for finite-genus solutions becomes complete. Having such NFT tools in hand, fair performance estimations of fibre-optic communications with finite-genus solutions data carriers are performed. Numerical simulations of the near-real communication systems are implemented, but the computational complexity of the NFT algorithms is disregarded. In such a system, additional distortions are caused by deviation from the original NLSE model. Applying a convolutional neural network at the receiver to compensate for these impairments while simultaneously recovering the scattering data provides high spectral efficiency comparable to conventional NFT techniques.