Transients and asymptotics of natural gradient learning


We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical mechanics framework which is appropriate for large input dimension. We find significant improvement over standard gradient descent in both the transient and asymptotic phases of learning.

Publication DOI:
Divisions: College of Engineering & Physical Sciences > Systems analytics research institute (SARI)
Additional Information: The original publication is available at
Uncontrolled Keywords: natural gradient,statistical mechanics,gradient descent,transient,asymptotic
ISBN: 3540762639
Last Modified: 27 Dec 2023 09:54
Date Deposited: 22 Sep 2009 13:30
Full Text Link:
Related URLs: https://link.sp ... -4471-1599-1_21 (Publisher URL)
PURE Output Type: Chapter
Published Date: 1998-09-01
Authors: Rattray, Magnus
Saad, David (ORCID Profile 0000-0001-9821-2623)



Version: Accepted Version

Export / Share Citation


Additional statistics for this record