Storage capacity of ultrametric committee machines

Abstract

The problem of computing the storage capacity of a feed-forward network, with L hidden layers, N inputs, and K units in the first hidden layer, is analyzed using techniques from statistical mechanics. We found that the storage capacity strongly depends on the network architecture αc ∼ (log K)1-1/2L and that the number of units K limits the number of possible hidden layers L through the relationship 2L - 1 < 2log K.

Publication DOI: https://doi.org/10.1088/1751-8113/47/6/065002
Divisions: College of Engineering & Physical Sciences > Systems analytics research institute (SARI)
Additional Information: © 2014 IOP Publishing
Uncontrolled Keywords: 84.35.+i,adaptive systems PACS numbers: 89.70.,replica trick,storage capacity,Mathematical Physics,Physics and Astronomy(all),Statistical and Nonlinear Physics,Modelling and Simulation,Statistics and Probability
Publication ISSN: 1751-8121
Last Modified: 26 Feb 2024 08:13
Date Deposited: 18 Mar 2015 11:35
Full Text Link: http://iopscien ... 21/47/6/065002/
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
PURE Output Type: Article
Published Date: 2014-02-14
Published Online Date: 2014-01-27
Authors: Neirotti, J.P. (ORCID Profile 0000-0002-2409-8917)

Download

[img]

Version: Accepted Version


Export / Share Citation


Statistics

Additional statistics for this record