Li, Bo and Saad, David (2018). Exploring the Function Space of Deep-Learning Machines. Physical Review Letters, 120 (24),
Abstract
The function space of deep-learning machines is investigated by studying growth in the entropy of functions of a given error with respect to a reference function, realized by a deep-learning machine. Using physics-inspired methods we study both sparsely and densely connected architectures to discover a layerwise convergence of candidate functions, marked by a corresponding reduction in entropy when approaching the reference function, gain insight into the importance of having a large number of layers, and observe phase transitions as the error increases.
Publication DOI: | https://doi.org/10.1103/PhysRevLett.120.248301 |
---|---|
Divisions: | College of Engineering & Physical Sciences > Systems analytics research institute (SARI) |
Additional Information: | © 2018 The American Physical Society. Exploring the Function Space of Deep-Learning Machines. Bo Li and David Saad. Phys. Rev. Lett. 120, 248301 – Published 12 June 2018 |
Uncontrolled Keywords: | deep-learning, statistical physics, machine learning,General Physics and Astronomy |
Publication ISSN: | 1079-7114 |
Last Modified: | 06 Aug 2024 08:01 |
Date Deposited: | 03 Jul 2018 07:50 |
Full Text Link: |
https://arxiv.o ... /abs/1708.01422 |
Related URLs: |
http://www.scop ... tnerID=8YFLogxK
(Scopus URL) |
PURE Output Type: | Article |
Published Date: | 2018-06-12 |
Published Online Date: | 2018-06-12 |
Accepted Date: | 2018-05-11 |
Authors: |
Li, Bo
Saad, David ( ![]() |