On-line Learning in a Changing Environment

Abstract

On-line learning of non-stationary tasks by two-layer neural networks is studied within the framework of statistical mechanics. A fully connected network with K hidden units and fixed hidden-to-output weights (a soft committee machine) learns a non-stationary task represented by a network of similar architecture having M hidden nodes. The network is trained via gradient descent (standard back-propagation) on randomly drawn inputs and the corresponding outputs generated by the teacher network representing the task. This work employs a general framework for the dynamics of on-line learning obtained earlier for the fixed environment case. We describe a general task non-stationarity and investigate the learning process in these learning scenarios where on-line methods have been found to be most useful. The dynamics are first analysed for K = M = 2 which is the building block of the general case (any K and M). The learning processes of stationary and non-stationary tasks are found to be qualitatively similar. However, for non-stationarities the transient stage of the dynamics becomes shorter and there is some residual error after convergence. These phases are investigated both numerically and analytically. The insight gained from the non-stationary case leads to a new learning rule which seems to be more efficient than basic gradient descent in escaping the symmetric subspace related to the transient part of the dynamics. These effects are studied in arbitrary realisable scenarios (K = M).