Using ancillary statistics in on-line learning algorithms


Neural networks are usually curved statistical models. They do not have finite dimensional sufficient statistics, so on-line learning on the model itself inevitably loses information. In this paper we propose a new scheme for training curved models, inspired by the ideas of ancillary statistics and adaptive critics. At each point estimate an auxiliary flat model (exponential family) is built to locally accommodate both the usual statistic (tangent to the model) and an ancillary statistic (normal to the model). The auxiliary model plays a role in determining credit assignment analogous to that played by an adaptive critic in solving temporal problems. The method is illustrated with the Cauchy model and the algorithm is proved to be asymptotically efficient.

Divisions: Aston University (General)
Additional Information: The original publication is available at
Event Title: Proc. 1996 International Conference on Neural Information Processing
Event Type: Other
Event Dates: 1996-01-01 - 1996-01-01
Uncontrolled Keywords: Neural networks,curved models,auxiliary,ancillary statistic,Cauchy,algorithm
ISBN: 9789813083059
Last Modified: 08 Dec 2023 12:36
Date Deposited: 16 Jul 2009 11:20
Full Text Link:
Related URLs: http://www.spri ... 8-981-3083-05-9 (Publisher URL)
PURE Output Type: Chapter
Published Date: 1996
Authors: Zhu, Huaiyu
Rohwer, Richard



Version: Accepted Version

Export / Share Citation


Additional statistics for this record