Learning with noise and regularizers in multilayer neural networks

Saad, David and Solla, Sara A. (1996). Learning with noise and regularizers in multilayer neural networks. Advances in Neural Information Processing Systems, 9 , pp. 260-266.

Abstract

We study the effect of two types of noise, data noise and model noise, in an on-line gradient-descent learning scenario for general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labeled by a two-layer teacher network with an arbitrary number of hidden units. Data is then corrupted by Gaussian noise affecting either the output or the model itself. We examine the effect of both types of noise on the evolution of order parameters and the generalization error in various phases of the learning process.

Divisions: Engineering & Applied Sciences > Mathematics
Engineering & Applied Sciences > Non-linearity and complexity research group
Additional Information: Copiright of Massachusetts Institute of Technology Press (MIT Press)
Uncontrolled Keywords: noise,data noise,model noise,gradient-descent learning,vectors,gaussian noise,error
Published Date: 1996

Download

[img]

Export / Share Citation


Statistics

Additional statistics for this record