How Deep Are Deep Gaussian Processes?
Abstract
Recent research has shown the potential utility of deep Gaussian processes. These deep structures are probability distributions, designed through hierarchical construction, which are conditionally Gaussian. In this paper, the current published body of work is placed in a common framework and, through recursion, several classes of deep Gaussian processes are defined. The resulting samples generated from a deep Gaussian process have a Markovian structure with respect to the depth parameter, and the effective depth of the resulting process is interpreted in terms of the ergodicity, or non-ergodicity, of the resulting Markov chain. For the classes of deep Gaussian processes introduced, we provide results concerning their ergodicity and hence their effective depth. We also demonstrate how these processes may be used for inference; in particular we show how a Metropolis-within-Gibbs construction across the levels of the hierarchy can be used to derive sampling tools which are robust to the level of resolution used to represent the functions on a computer. For illustration, we consider the effect of ergodicity in some simple numerical examples.
Additional Information
© 2018 Matthew M. Dunlop, Mark A. Girolami, Andrew M. Stuart and Aretha L. Teckentrup. License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at http://jmlr.org/papers/v19/18-015.html. Submitted 1/18; Revised 8/18; Published 9/18. MG is supported by EPSRC grants [EP/R034710/1, EP/R018413/1, EP/R004889/1, EP/P020720/1], an EPSRC Established Career Fellowship EP/J016934/3, a Royal Academy of Engineering Research Chair, and The Lloyds Register Foundation Programme on Data Centric Engineering. AMS is supported by AFOSR Grant FA9550-17-1-0185 and by US National Science Foundation (NSF) grant DMS 1818977. ALT is partially supported by The Alan Turing Institute under the EPSRC grant EP/N510129/1.Attached Files
Published - 18-015.pdf
Submitted - 1711.11280.pdf
Files
Name | Size | Download all |
---|---|---|
md5:15ec872d7b65b4752c0f36f8e4d4a485
|
2.0 MB | Preview Download |
md5:84a4bb8ecf8c4ca370552a07704fc6a3
|
3.7 MB | Preview Download |
Additional details
- Eprint ID
- 90763
- Resolver ID
- CaltechAUTHORS:20181108-140320751
- Engineering and Physical Sciences Research Council (EPSRC)
- EP/R034710/1
- Engineering and Physical Sciences Research Council (EPSRC)
- EP/R018413/1
- Engineering and Physical Sciences Research Council (EPSRC)
- EP/R004889/1
- Engineering and Physical Sciences Research Council (EPSRC)
- EP/P020720/1
- Engineering and Physical Sciences Research Council (EPSRC)
- EP/J016934/3
- Royal Academy of Engineering
- Lloyds Register Foundation
- Air Force Office of Scientific Research (AFOSR)
- FA9550-17-1-0185
- NSF
- DMS-1818977
- Alan Turing Institute
- Engineering and Physical Sciences Research Council (EPSRC)
- EP/N510129/1
- Created
-
2018-11-09Created from EPrint's datestamp field
- Updated
-
2023-06-02Created from EPrint's last_modified field