Space of Functions Computed by Deep-Layered Machines


We study the space of functions computed by random-layered machines, including deep neural networks and Boolean circuits. Investigating the distribution of Boolean functions computed on the recurrent and layer-dependent architectures, we find that it is the same in both models. Depending on the initial conditions and computing elements used, we characterize the space of functions computed at the large depth limit and show that the macroscopic entropy of Boolean functions is either monotonically increasing or decreasing with the growing depth.

Publication DOI:
Divisions: College of Engineering & Physical Sciences
College of Engineering & Physical Sciences > Aston Institute of Urban Technology and the Environment (ASTUTE)
College of Engineering & Physical Sciences > Systems analytics research institute (SARI)
Additional Information: © 2020 American Physical Society. Space of Functions Computed by Deep-Layered Machines. Alexander Mozeika, Bo Li, and David Saad. Phys. Rev. Lett. 125, 168301 – Published 12 October 2020 Funding: B. L. and D. S. acknowledge support from the Leverhulme Trust (RPG-2018-092), European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie Grant Agreement No. 835913. D. S. acknowledges support from the EPSRC program grant TRANSNET (EP/R035342/1).
Uncontrolled Keywords: Physics and Astronomy(all)
Publication ISSN: 1079-7114
Last Modified: 22 Jan 2024 08:57
Date Deposited: 29 Oct 2020 09:13
Full Text Link: https://arxiv.o ... /abs/2004.08930
Related URLs: https://journal ... Lett.125.168301 (Publisher URL)
http://www.scop ... tnerID=8YFLogxK (Scopus URL)
PURE Output Type: Article
Published Date: 2020-10-12
Accepted Date: 2020-09-09
Authors: Mozeika, Alexander
Li, Bo (ORCID Profile 0000-0001-9743-9447)
Saad, David (ORCID Profile 0000-0001-9821-2623)



Version: Published Version

| Preview

Export / Share Citation


Additional statistics for this record