Dynamics of on-line learning in radial basis function networks


On-line learning is examined for the radial basis function network, an important and practical type of neural network. The evolution of generalization error is calculated within a framework which allows the phenomena of the learning process, such as the specialization of the hidden units, to be analyzed. The distinct stages of training are elucidated, and the role of the learning rate described. The three most important stages of training, the symmetric phase, the symmetry-breaking phase, and the convergence phase, are analyzed in detail; the convergence phase analysis allows derivation of maximal and optimal learning rates. As well as finding the evolution of the mean system parameters, the variances of these parameters are derived and shown to be typically small. Finally, the analytic results are strongly confirmed by simulations.

Publication DOI: https://doi.org/10.1103/PhysRevE.56.907
Divisions: College of Engineering & Physical Sciences > Systems analytics research institute (SARI)
Additional Information: Copyright of the American Physical Society
Uncontrolled Keywords: on-line learning,radial basis function network,neural network,error,learning process,symmetric phase,symmetry-breaking phase,convergence phase,Mathematical Physics,Physics and Astronomy(all),Condensed Matter Physics,Statistical and Nonlinear Physics
Publication ISSN: 1550-2376
Last Modified: 02 Jan 2024 08:03
Date Deposited: 23 Jul 2009 11:28
Full Text Link:
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
http://prola.ap ... E/v56/i1/p907_1 (Publisher URL)
PURE Output Type: Article
Published Date: 1997
Authors: Freeman, Jason
Saad, David (ORCID Profile 0000-0001-9821-2623)



Version: Accepted Version

Export / Share Citation


Additional statistics for this record