Exploring the Function Space of Deep-Learning Machines


The function space of deep-learning machines is investigated by studying growth in the entropy of functions of a given error with respect to a reference function, realized by a deep-learning machine. Using physics-inspired methods we study both sparsely and densely connected architectures to discover a layerwise convergence of candidate functions, marked by a corresponding reduction in entropy when approaching the reference function, gain insight into the importance of having a large number of layers, and observe phase transitions as the error increases.

Publication DOI: https://doi.org/10.1103/PhysRevLett.120.248301
Divisions: College of Engineering & Physical Sciences > Systems analytics research institute (SARI)
Additional Information: © 2018 The American Physical Society. Exploring the Function Space of Deep-Learning Machines. Bo Li and David Saad. Phys. Rev. Lett. 120, 248301 – Published 12 June 2018
Uncontrolled Keywords: deep-learning, statistical physics, machine learning,Physics and Astronomy(all)
Publication ISSN: 1079-7114
Last Modified: 03 Jun 2024 07:33
Date Deposited: 03 Jul 2018 07:50
Full Text Link: https://arxiv.o ... /abs/1708.01422
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
PURE Output Type: Article
Published Date: 2018-06-12
Published Online Date: 2018-06-12
Accepted Date: 2018-05-11
Authors: Li, Bo
Saad, David (ORCID Profile 0000-0001-9821-2623)



Version: Accepted Version

| Preview

Export / Share Citation


Additional statistics for this record