Adaptive back-propagation in on-line learning of multilayer networks

West, Ansgar H L and Saad, David (1996). Adaptive back-propagation in on-line learning of multilayer networks. IN: Proceedings of the neural information processing systems. Touretzky, David S; Mozer, Michael C and Hasselmo, Michael E. (eds) Boston: MIT.

Abstract

An adaptive back-propagation algorithm is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, both numerical studies and a rigorous analysis show that the adaptive back-propagation method results in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

Divisions: Engineering & Applied Sciences > Mathematics
Engineering & Applied Sciences > Systems analytics research institute (SARI)
Additional Information: Copyright of the Massachusetts Institute of Technology Press (MIT Press)
Event Title: Neural Information Processing Systems 95
Event Type: Other
Event Dates: 1996-01-01 - 1996-01-01
Uncontrolled Keywords: adaptive back-propagation,algorithm,gradient descent,neural networks,statistical
Full Text Link:
Related URLs: http://mitpress ... type=2&tid=8421 (Publisher URL)
Published Date: 1996
Authors: West, Ansgar H L
Saad, David ( 0000-0001-9821-2623)

Download

[img]

Version: Published Version


Export / Share Citation


Statistics

Additional statistics for this record