Zhu, Huaiyu and Rohwer, Richard (1996). Using ancillary statistics in on-line learning algorithms. IN: Proceedings of the 1996 International Conference on Neural Information Processing. Amari, S I; Xu, L; Chan, L W; King, I and Leung, K S (eds) Springer.
Abstract
Neural networks are usually curved statistical models. They do not have finite dimensional sufficient statistics, so on-line learning on the model itself inevitably loses information. In this paper we propose a new scheme for training curved models, inspired by the ideas of ancillary statistics and adaptive critics. At each point estimate an auxiliary flat model (exponential family) is built to locally accommodate both the usual statistic (tangent to the model) and an ancillary statistic (normal to the model). The auxiliary model plays a role in determining credit assignment analogous to that played by an adaptive critic in solving temporal problems. The method is illustrated with the Cauchy model and the algorithm is proved to be asymptotically efficient.
Divisions: | Aston University (General) |
---|---|
Additional Information: | The original publication is available at www.springerlink.com |
Event Title: | Proc. 1996 International Conference on Neural Information Processing |
Event Type: | Other |
Event Dates: | 1996-01-01 - 1996-01-01 |
Uncontrolled Keywords: | Neural networks,curved models,auxiliary,ancillary statistic,Cauchy,algorithm |
ISBN: | 9789813083059 |
Last Modified: | 05 Aug 2024 08:55 |
Date Deposited: | 16 Jul 2009 11:20 |
Full Text Link: | |
Related URLs: |
http://www.spri ... 8-981-3083-05-9
(Publisher URL) |
PURE Output Type: | Chapter |
Published Date: | 1996 |
Authors: |
Zhu, Huaiyu
Rohwer, Richard |