作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程

• •    

基于个性化梯度裁剪的联邦学习隐私保护算法

  • 出版日期:2024-10-09 发布日期:2024-10-09

Privacy-preserving algorithm for federated learning based on personalized gradient clipping

  • Online:2024-10-09 Published:2024-10-09

摘要: 联邦学习作为目前深度学习最为常用的隐私保护框架,被众多机构广泛应用。此框架中的各个参与方通过上传模型参数数据实现本地数据不离本地,达到共享数据的目的。但在联邦学习中各个参与方频繁上传及接收参数时易出现隐私泄露问题。为解决这一问题,提出一种基于个性化梯度裁剪的联邦学习隐私保护算法(AADP_FL),该算法根据参与方不同网络层历史数据的L1范数计算出各层的裁剪阈值,对梯度数据进行裁剪以限制梯度范围,预防梯度爆炸及梯度消失。同时计算各层的贡献度,根据各层贡献度为每层分配隐私预算,进而添加个性化噪声。参与方在上传数据时加入适量的噪声,以掩盖上传数据的具体内容,进而隐藏各个参与者的贡献率,保护各个参与方的数据安全。经过一系列实验证明,本算法的准确率有较大提升,相较于常用的个性化梯度裁剪方法准确率提升3.5%以上,对比于传统的联邦学习框架本算法也能保持较高的准确率。在保持较高准确率的同时经证明能严格保护参与方数据的隐私安全,使得模型性能与数据隐私达到均衡状态。

Abstract: Federated learning, as the most commonly used privacy protection framework in deep learning, is widely applied by many institutions. The various participants in this framework achieve the goal of sharing data by uploading model parameter data without leaving the local data. However, in federated learning, there is a risk of privacy leakage when various parties frequently upload and receive parameters. To address this issue, a personalized gradient pruning based federated learning privacy preserving algorithm (AADP-FL) is proposed. This algorithm calculates the pruning threshold for each layer based on the L1 norm of historical data from different network layers of the participating parties. The gradient data is then pruned to limit the gradient range and prevent gradient explosion and vanishing. Simultaneously calculate the contribution of each layer, allocate privacy budgets for each layer based on their contribution, and then add personalized noise. Participants add an appropriate amount of noise when uploading data to conceal the specific content of the uploaded data, thereby hiding the contribution rate of each participant and protecting the data security of each participant. After a series of experiments, it has been proven that the accuracy of this algorithm has been greatly improved, with an accuracy increase of over 3.5% compared to commonly used personalized gradient cropping methods. Compared with traditional federated learning frameworks, this algorithm can also maintain a high accuracy. It has been proven to strictly protect the privacy and security of participant data while maintaining high accuracy, achieving a balance between model performance and data privacy.