On Connecting Stochastic Gradient MCMC and Differential Privacy
- Creators
- Li, Bai
- Chen, Changyou
- Liu, Hao
- Carin, Lawrence
Abstract
Concerns related to data security and confidentiality have been raised when applying machine learning to real-world applications. Differential privacy provides a principled and rigorous privacy guarantee for machine learning models. While it is common to inject noise to design a model satisfying a required differential-privacy property, it is generally hard to balance the trade-off between privacy and utility. We show that stochastic gradient Markov chain Monte Carlo (SG-MCMC) – a class of scalable Bayesian posterior sampling algorithms – satisfies strong differential privacy, when carefully chosen stepsizes are employed. We develop theory on the performance of the proposed differentially-private SG-MCMC method. We conduct experiments to support our analysis, and show that a standard SG-MCMC sampler with minor modification can reach state-of-the-art performance in terms of both privacy and utility on Bayesian learning.
Additional Information
© 2019 by the author(s). This research was supported in part by DARPA, DOE, NIH, NSF and ONR. We thank Ruiyi Zhang for providing the code base.Attached Files
Published - li19a.pdf
Submitted - 1712.09097.pdf
Supplemental Material - li19a-supp.pdf
Files
Additional details
- Eprint ID
- 101725
- Resolver ID
- CaltechAUTHORS:20200305-132433478
- Defense Advanced Research Projects Agency (DARPA)
- Department of Energy (DOE)
- NIH
- NSF
- Office of Naval Research (ONR)
- Created
-
2020-03-05Created from EPrint's datestamp field
- Updated
-
2023-06-02Created from EPrint's last_modified field