Globally optimal parameters for on-line learning in multilayer neural networks

Abstract

We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.

Publication DOI: https://doi.org/10.1103/PhysRevLett.79.2578
Divisions: College of Engineering & Physical Sciences > School of Informatics and Digital Engineering > Mathematics
College of Engineering & Physical Sciences > Systems analytics research institute (SARI)
Additional Information: Copyright of the American Physical Society
Uncontrolled Keywords: on-line learning,multilayer neural networks,learning rates,training algorithms
Publication ISSN: 1079-7114
Full Text Link:
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
http://prola.ap ... v79/i13/p2578_1 (Publisher URL)
PURE Output Type: Article
Published Date: 1997-09-29
Authors: Saad, David (ORCID Profile 0000-0001-9821-2623)
Rattray, Magnus

Download

[img]

Version: Accepted Version

| Preview

Export / Share Citation


Statistics

Additional statistics for this record