Learning with regularizers in multilayer neural networks

Saad, David and Rattray, Magnus (1998). Learning with regularizers in multilayer neural networks. Physical Review E, 57 (2), pp. 2170-2176.

Abstract

We study the effect of regularization in an on-line gradient-descent learning scenario for a general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labelled by a two-layer teacher network with an arbitrary number of hidden units which may be corrupted by Gaussian output noise. We examine the effect of weight decay regularization on the dynamical evolution of the order parameters and generalization error in various phases of the learning process, in both noiseless and noisy scenarios.

Divisions: Engineering & Applied Sciences > Mathematics
Engineering & Applied Sciences > Systems analytics research institute (SARI)
Additional Information: Copyright of the American Physical Society
Uncontrolled Keywords: on-line gradient-descent learning scenario,Gaussian,noise,weight decay,error
Full Text Link:
Related URLs: http://prola.ap ... /v57/i2/p2170_1 (Publisher URL)
Published Date: 1998-02
Authors: Saad, David ( 0000-0001-9821-2623)
Rattray, Magnus

Download

[img]

Version: Accepted Version

| Preview

Export / Share Citation


Statistics

Additional statistics for this record