Analysis of natural gradient descent for multilayer neural networks

Rattray, M. and Saad, D. (1999). Analysis of natural gradient descent for multilayer neural networks. Physical Review E, 59 (4), pp. 4523-4532.


Natural gradient descent (NGD) is an on-line algorithm for redefining the steepest descent direction. An analysis of NGD for training a multilayer neural network is presented using statistical mechanics. The performance can be significantly improved using NGD algorithm and can be used for both the transient and asymptotic stages of learning.

Divisions: Engineering & Applied Sciences > Mathematics
Engineering & Applied Sciences > Systems analytics research institute (SARI)
Additional Information: ©1999 The American Physical Society
Full Text Link:
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
https://journal ... hysRevE.59.4523 (Publisher URL)
Published Date: 1999-04-01
Authors: Rattray, M.
Saad, D. ( 0000-0001-9821-2623)



Version: Published Version

| Preview

Export / Share Citation


Additional statistics for this record