Logo image
Back
Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting
Conference proceeding   Open access   Peer reviewed

Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting

Adam D. Cobb and Brian Jalaian
Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence (UAI 2021), Vol.161, pp.675-685
Proceedings of Machine Learning Research
Conference on Uncertainty in Artificial Intelligence (UAI 2021), 37th (Virtual, 07/27/2021–07/30/2021)
2021
Web of Science ID: WOS:001237128000063

Metrics

1 File views/ downloads
3 Record Views
38 Times Cited - Web of Science

Abstract

Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) approach that exhibits favourable exploration properties in high-dimensional models such as neural networks. Unfortunately, HMC has limited use in large-data regimes and little work has explored suitable approaches that aim to preserve the entire Hamiltonian. In our work, we introduce a new symmetric integration scheme for split HMC that does not rely on stochastic gradients. We show that our new formulation is more efficient than previous approaches and is easy to implement with a single GPU. As a result, we are able to perform full HMC over common deep learning architectures using entire data sets. In addition, when we compare with stochastic gradient MCMC, we show that our method achieves better performance in both accuracy and uncertainty quantification. Our approach demonstrates HMC as a feasible option when considering inference schemes for large-scale machine learning problems.
pdf
Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting387.26 kBDownloadView
Published (Version of record)Conference proceeding pdf Open Access
pdf
Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting1.13 MBDownloadView
Preprint (supplemental)Proceedings pre-print pdf Open Access
pdf
Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting Supplementary Material677.48 kBDownloadView
Published (Version of record) (supplemental)Supplementary info pdf Open Access
url
Scaling Hamiltonian Monte Carlo inference for Bayesian neural networks with symmetric splittingView
Published (Version of record)link to proceedings paper Open
url
Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric SplittingView
Preprint (supplemental)link to proceedings pre-print CC BY V4.0,  Open

Related links

Details

Logo image