A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf
.
Privacy Regularization: Joint Privacy-Utility Optimization in Language Models
[article]
2021
arXiv
pre-print
Neural language models are known to have a high capacity for memorization of training samples. This may have serious privacy implications when training models on user content such as email correspondence. Differential privacy (DP), a popular choice to train models with privacy guarantees, comes with significant costs in terms of utility degradation and disparate impact on subgroups of users. In this work, we introduce two privacy-preserving regularization methods for training language models
arXiv:2103.07567v2
fatcat:kma3aa7wejdobom2ef5yhkn4he