Deep Neural Network-aided Soft-Demapping in Coherent Optical Systems: Regression versus Classification


We examine here what type of predictive modelling, classification, or regression, using neural networks (NN), fits better the task of soft-demapping based post-processing in coherent optical communications, where the transmission channel is nonlinear and dispersive. For the first time, we present possible drawbacks in using each type of predictive task in a machine learning context, considering the nonlinear coherent optical channel equalization/soft-demapping problem. We study two types of equalizers based on the feed-forward and recurrent NNs, for several transmission scenarios, in linear and nonlinear regimes of the optical channel. We point out that even though from the information theory perspective the cross-entropy loss (classification) is the most suitable option for our problem, the NN models based on the cross-entropy loss function can severely suffer from learning problems. The latter translates into the fact that regression-based learning is typically superior in terms of delivering higher Q-factor and achievable information rates. In short, we show by empirical evidence that loss functions based on cross-entropy may not be necessarily the most suitable option for training communication systems in practical scenarios when overfitting- and vanishing gradients-related problems come into play.

Publication DOI:
Divisions: College of Engineering & Physical Sciences > Aston Institute of Photonics Technology (AIPT)
College of Engineering & Physical Sciences
Additional Information: This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see
Uncontrolled Keywords: Neural networks,classification,coherent detection,digital signal processing,nonlinear equalizer,optical communications,regression,Electrical and Electronic Engineering
Publication ISSN: 1558-0857
Last Modified: 22 May 2024 16:23
Date Deposited: 19 Oct 2022 07:54
Full Text Link:
Related URLs: https://ieeexpl ... ocument/9915441 (Publisher URL)
http://www.scop ... tnerID=8YFLogxK (Scopus URL)
PURE Output Type: Article
Published Date: 2022-12
Published Online Date: 2022-10-10
Accepted Date: 2022-09-21
Authors: Freire, Pedro J. (ORCID Profile 0000-0003-3145-1018)
Prilepsky, Jaroslaw E. (ORCID Profile 0000-0002-3035-4112)
Osadchuk, Yevhenii (ORCID Profile 0000-0003-3145-1018)
Turitsyn, Sergei K. (ORCID Profile 0000-0003-0101-3834)
Aref, Vahid



Version: Published Version

License: Creative Commons Attribution

| Preview


Version: Accepted Version

Access Restriction: Restricted to Registered users only

License: Creative Commons Attribution

Export / Share Citation


Additional statistics for this record