Globally optimal on-line learning rules for multi-layer neural networks

Rattray, Magnus and Saad, David Globally optimal on-line learning rules for multi-layer neural networks. Journal of Physics A: Mathematical and General, 30 (22), L771-L776.


We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

Publication DOI:
Divisions: Engineering & Applied Sciences > Mathematics
Engineering & Applied Sciences > Non-linearity and complexity research group
Related URLs:
Additional Information: Copyright of the Institute of Physics.
Uncontrolled Keywords: globally optimal on-line learning,soft committee machine,error,locally optimal rule,Physics and Astronomy(all),Statistical and Nonlinear Physics,Mathematical Physics



Export / Share Citation


Additional statistics for this record