Deep Learning Methods for Nonlinearity Mitigation in Coherent Fiber-Optic Communication Links


Nowadays, the demand for telecommunication services is rapidly growing. To meet this everincreasing connectivity demand telecommunication industry needs to maintain the exponential growth of capacity supply. One of the central efforts in this initiative is directed towards coherent fiber-optic communication systems, the backbone of modern telecommunication infrastructure. Nonlinear distortions, i.e., the ones dependent on the transmitted signal, are widely considered to be one of the major limiting factors of these systems. When mitigating these distortions, we can’t rely on the pre-recorded information about channel properties, which is often missing or incorrect, and, therefore, have to resort to adaptive mitigation techniques, learning the link properties by themselves. Unfortunately, the existing practical approaches are suboptimal: they assume weak nonlinear distortion and propose its compensation via a cascade of separately trained sub-optimal algorithms. Deep learning, a subclass of machine learning very popular nowadays, proposes a way to address these problems. First, deep learning solutions can approximate well an arbitrary nonlinear function without making any prior assumptions about it. Second, deep learning solutions can effectively optimize a cluster of single-purpose algorithms, which leads them to a global performance optimum. In this thesis, two deep-learning solutions for nonlinearity mitigation in high-baudrate coherent fiber-optic communication links are proposed. The first one is the data augmentation technique for improving the training of supervised-learned algorithms for the compensation of nonlinear distortion. Data augmentation encircles a set of approaches for enhancing the size and the quality of training datasets so that they can lead us to better supervised learned models. This thesis shows that specially designed data augmentation techniques can be a very efficient tool for the development of powerful supervised-learned nonlinearity compensation algorithms. In various testcases studied both numerically and experimentally, the suggested augmentation is shown to lead to the reduction of up to 6× in the size of the dataset required to achieve the desired performance and a nearly 2× reduction in the training complexity of a nonlinearity compensation algorithm. The proposed approach is generic and can be applied to enhance a multitude of supervised-learned nonlinearity compensation techniques. The second one is the end-to-end learning procedure enabling optimization of the joint probabilistic and geometric shaping of symbol sequences. In a general end-to-end learning approach, the whole system is implemented as a single trainable NN from bits-in to bits-out. The novelty of the proposed approach is in using cost-effective channel model based on the perturbation theory and the refined symbol probabilities training procedure. The learned constellation shaping demonstrates a considerable mutual information gains in single-channel 64 GBd transmission through both single-span 170 km and multi-span 30x80 km single-mode fiber links. The suggested end-to-end learning procedure is applicable to an arbitrary coherent fiber-optic communication link.

Additional Information: Copyright © Vladislav Neskorniuk, 2022. Vladislav Neskorniuk asserts his moral right to be identified as the author of this thesis. This copy of the thesis has been supplied on condition that anyone who consults it is understood to recognise that its copyright rests with its author and that no quotation from the thesis and no information derived from it may be published without appropriate permission or acknowledgement. If you have discovered material in Aston Publications Explorer which is unlawful e.g. breaches copyright, (either yours or that of a third party) or any other law, including but not limited to those relating to patent, trademark, confidentiality, data protection, obscenity, defamation, libel, then please read our Takedown Policy and contact the service immediately. Author's contribution statement and attribution: Chapters 3.1, 3.2, 3.3.1, and Figures 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9 are licensed under the Creative Commons Attribution CC BY 4.0 license [] and taken from the article, 'Vladislav Neskorniuk, Andrea Carnio, Domenico Marsella, Sergei K Turitsyn, Jaroslaw E Prilepsky, and Vahid Aref. Memory-aware end-to-end learning of channel distortions in optical coherent communications. Optics Express, 31(1):1–20, 2023.,' which I co-authored as a leading and corresponding author. Figure 3.10 is taken from the conference paper, 'Vladislav Neskorniuk, Andrea Carnio, Vinod Bajaj, Domenico Marsella, Sergei K Turitsyn, Jaroslaw E Prilepsky, and Vahid Aref. End-to-end deep learning of long-haul coherent optical fiber communications via regular perturbation model. In 2021 European Conference on Optical Communication (ECOC), pages 1–4. IEEE, 2021.,' which I also co-authored as a leading and corresponding author. I have obtained all the results presented in this chapter, drafted its text, and prepared all illustrations, besides Figure 3.10, by myself. The method proposed in this chapter was developed by me together with the co-authors of these articles. The computer code used in the presented research was written by myself, except for the RP model, written jointly with Andrea Carnio. This research was done under the supervision of Dr. Vahid Aref, Prof. Sergei K. Turitsyn, and Dr. Jaroslaw E. Prilepsky. In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of Aston University’s products or services. Internal or personal use of this material is permitted. If interested in reprinting/republishing IEEE copyrighted material for advertising or promotional purposes or for creating new collective works for resale or redistribution, please go to to learn how to obtain a License from RightsLink.
Institution: Aston University
Uncontrolled Keywords: Telecommunications,nonlinear optics,coherent detection,digital signal processing,machine learning,deep learning,end-to-end learning,constellation shaping
Last Modified: 08 Dec 2023 09:00
Date Deposited: 24 Jul 2023 15:54
Completed Date: 2022-11
Authors: Neskorniuk, Vladislav (ORCID Profile 0000-0002-3358-153X)

Export / Share Citation


Additional statistics for this record