Bayesian invariant measurements of generalisation


The problem of evaluating different learning rules and other statistical estimators is analysed. A new general theory of statistical inference is developed by combining Bayesian decision theory with information geometry. It is coherent and invariant. For each sample a unique ideal estimate exists and is given by an average over the posterior. An optimal estimate within a model is given by a projection of the ideal estimate. The ideal estimate is a sufficient statistic of the posterior, so practical learning rules are functions of the ideal estimator. If the sole purpose of learning is to extract information from the data, the learning rule must also approximate the ideal estimator. This framework is applicable to both Bayesian and non-Bayesian methods, with arbitrary statistical models, and to supervised, unsupervised and reinforcement learning schemes.

Divisions: Aston University (General)
Additional Information: Copyright of Springer Verlag. The original publication is available at
Uncontrolled Keywords: learning rules,statistical estimators,statistical inference,decision theory,information geometry,Bayesian,non-Bayesian
Publication ISSN: 1573-773X
Last Modified: 23 May 2024 07:07
Date Deposited: 06 Jul 2009 11:36
Full Text Link:
Related URLs: http://www.spri ... aaede73120&pi=0 (Publisher URL)
PURE Output Type: Article
Published Date: 1995-12
Authors: Zhu, Huaiyu
Rohwer, Richard



Version: Published Version

Export / Share Citation


Additional statistics for this record