Using ancillary statistics in on-line learning algorithms

Zhu, Huaiyu and Rohwer, Richard (1996). Using ancillary statistics in on-line learning algorithms. IN: Proceedings of the 1996 International Conference on Neural Information Processing. Amari, S I; Xu, L; Chan, L W; King, I and Leung, K S (eds) Springer.

Abstract

Neural networks are usually curved statistical models. They do not have finite dimensional sufficient statistics, so on-line learning on the model itself inevitably loses information. In this paper we propose a new scheme for training curved models, inspired by the ideas of ancillary statistics and adaptive critics. At each point estimate an auxiliary flat model (exponential family) is built to locally accommodate both the usual statistic (tangent to the model) and an ancillary statistic (normal to the model). The auxiliary model plays a role in determining credit assignment analogous to that played by an adaptive critic in solving temporal problems. The method is illustrated with the Cauchy model and the algorithm is proved to be asymptotically efficient.

Divisions: Aston University (General)
Additional Information: The original publication is available at www.springerlink.com
Event Title: Proc. 1996 International Conference on Neural Information Processing
Event Type: Other
Event Dates: 1996-01-01 - 1996-01-01
Uncontrolled Keywords: Neural networks,curved models,auxiliary,ancillary statistic,Cauchy,algorithm
Full Text Link:
Related URLs: http://www.spri ... 8-981-3083-05-9 (Publisher URL)
Published Date: 1996
Authors: Zhu, Huaiyu
Rohwer, Richard

Download

[img]

Version: Accepted Version


Export / Share Citation


Statistics

Additional statistics for this record