Transients and asymptotics of natural gradient learning

Rattray, Magnus and Saad, David (1998). Transients and asymptotics of natural gradient learning. IN: Proceedings of the 8th International Conference on Artificial Neural Networks. Niklasson, L.; Boden, M. and Ziemke, T. (eds) Springer.

Abstract

We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical mechanics framework which is appropriate for large input dimension. We find significant improvement over standard gradient descent in both the transient and asymptotic phases of learning.

Publication DOI: https://doi.org/10.1007/978-1-4471-1599-1_21
Divisions: Engineering & Applied Sciences > Mathematics
Engineering & Applied Sciences > Systems analytics research institute (SARI)
Additional Information: The original publication is available at www.springerlink.com
Uncontrolled Keywords: natural gradient,statistical mechanics,gradient descent,transient,asymptotic
Full Text Link:
Related URLs: https://link.sp ... -4471-1599-1_21 (Publisher URL)
Published Date: 1998-09-01
Authors: Rattray, Magnus
Saad, David ( 0000-0001-9821-2623)

Download

[img]

Version: Accepted Version


Export / Share Citation


Statistics

Additional statistics for this record