Exploring the Function Space of Deep-Learning Machines

Li, Bo and Saad, David (2018). Exploring the Function Space of Deep-Learning Machines. Physical Review Letters, 120 (24),

Abstract

The function space of deep-learning machines is investigated by studying growth in the entropy of functions of a given error with respect to a reference function, realized by a deep-learning machine. Using physics-inspired methods we study both sparsely and densely connected architectures to discover a layerwise convergence of candidate functions, marked by a corresponding reduction in entropy when approaching the reference function, gain insight into the importance of having a large number of layers, and observe phase transitions as the error increases.

Publication DOI: https://doi.org/10.1103/PhysRevLett.120.248301
Divisions: Engineering & Applied Sciences > Systems analytics research institute (SARI)
Engineering & Applied Sciences > Mathematics
Additional Information: © 2018 The American Physical Society. Exploring the Function Space of Deep-Learning Machines. Bo Li and David Saad. Phys. Rev. Lett. 120, 248301 – Published 12 June 2018
Uncontrolled Keywords: deep-learning, statistical physics, machine learning,Physics and Astronomy(all)
Full Text Link: https://arxiv.o ... /abs/1708.01422
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
Published Online Date: 2018-06-12
Published Date: 2018-06-12
Authors: Li, Bo
Saad, David ( 0000-0001-9821-2623)

Download

[img]

Version: Accepted Version

| Preview

Export / Share Citation


Statistics

Additional statistics for this record