Globally optimal on-line learning rules for multi-layer neural networks


We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

Divisions: College of Engineering & Physical Sciences > Systems analytics research institute (SARI)
Additional Information: Copyright of the Institute of Physics.
Publication ISSN: 0305-4470
Last Modified: 29 Nov 2023 10:00
Date Deposited: 11 Mar 2019 17:28
Full Text Link: 10.1088/0305-4470/30/22/005
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
http://iopscien ... ect=.iopscience (Publisher URL)
PURE Output Type: Article
Published Date: 1997-11-21
Authors: Rattray, Magnus
Saad, David



Version: Accepted Version

| Preview

Export / Share Citation


Additional statistics for this record