基于个性化梯度裁剪的联邦学习隐私保护算法

doi:10.19678/j.issn.1000-3428.0069644

摘要/Abstract

摘要： 联邦学习作为目前深度学习最为常用的隐私保护框架,被众多机构广泛应用。此框架中的各个参与方通过上传模型参数数据实现本地数据不离本地,达到共享数据的目的。但在联邦学习中各个参与方频繁上传及接收参数时易出现隐私泄露问题。为解决这一问题,提出一种基于个性化梯度裁剪的联邦学习隐私保护算法(AADP_FL)。该算法根据参与方不同网络层历史数据的L1范数计算出各层的裁剪阈值,对梯度数据进行裁剪以限制梯度范围,预防梯度爆炸及梯度消失。同时计算各层的贡献度,根据各层贡献度为每层分配隐私预算,进而添加个性化噪声。参与方在上传数据时加入适量的噪声,以掩盖上传数据的具体内容,进而隐藏各个参与者的贡献率,保护各个参与方的数据安全。经过一系列实验证明,AADP_FL算法的准确率相较于常用的个性化梯度裁剪方法提升3.5百分点以上,相比于传统的联邦学习框架也能保持较高的准确率。同时,该算法在保持较高准确率的同时能严格保护参与方数据的隐私安全,使得模型性能与数据隐私性达到均衡状态。

关键词: 联邦学习, 隐私保护, 差分隐私, 隐私预算, 个性化梯度裁剪

Abstract: Federated learning, as the most commonly used privacy protection framework in deep learning, is widely applied by many institutions. The various participants in this framework achieve the goal of sharing data by uploading model parameter data without leaving the local data. However, in federated learning, privacy leakage occurs when various parties frequently upload and receive parameters. To address this issue, a personalized gradient clipping-based federated learning privacy preserving algorithm (AADP-FL) is proposed. This algorithm calculates the clipping threshold for each layer based on the L1 norm of historical data from different network layers of the participants. The gradient data is then clipped to limit the gradient range and prevent gradient explosion and vanishing gradients. Simultaneously, the contribution of each layer is calculated, privacy budgets are allocated for each layer based on their contribution, and then personalized noise is added. Participants add an appropriate amount of noise when uploading data to conceal the specific content, thereby hiding the contribution rate of each participant and improving the data security for each participant. A series of experiments reveal that the accuracy of this algorithm is superior compared to the commonly used personalized gradient clipping methods, with an accuracy increase of over 3.5 percentage points. This algorithm can also maintain a high accuracy compared with traditional federated learning frameworks. It can effectively protect the privacy of participant data while maintaining high accuracy, achieving a balance between model performance and data privacy.

Key words: federated learning, privacy protection, differential privacy, privacy budget, personalized gradient clipping

中图分类号:

TP309

曹天涯, 张雨静, 贾俊杰, 张宇帆, 邓晓飞. 基于个性化梯度裁剪的联邦学习隐私保护算法[J]. 计算机工程, 2026, 52(2): 265-274.

CAO Tianya, ZHANG Yujing, JIA Junjie, ZHANG Yufan, DENG Xiaofei. Privacy Protection Algorithm for Federated Learning Based on Personalized Gradient Clipping[J]. Computer Engineering, 2026, 52(2): 265-274.

https://www.ecice06.com/CN/Y2026/V52/I2/265

参考文献

[1] MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proceedings of International Conference on Artificial Intelligence and Statistics. Fort Lauderdale, USA: PMLR, 2017: 1273-1282.
[2] 邹赛兰, 李卓, 陈昕. 面向分层联邦学习的传输优化研究[J]. 计算机科学, 2022, 49(12): 5-16. ZOU S L, LI Z, CHEN X. Research on transmission optimization for hierarchical federated learning[J]. Computer Science, 2022, 49(12): 5-16. (in Chinese)
[3] SONG M K, WANG Z B, ZHANG Z F, et al. Analyzing user-level privacy attack against federated learning[J]. IEEE Journal on Selected Areas in Communications, 2020, 38(10): 2430-2444.
[4] MELIS L, SONG C Z, CRISTOFARO E, et al. Exploiting unintended feature leakage in collaborative learning[C]//Proceedings of the IEEE Symposium on Security and Privacy. San Francisco, USA: IEEE Press, 2019: 691-706.
[5] HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep models under the GAN: information leakage from collaborative deep learning[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York, USA: ACM, 2017: 603-618.
[6] 秦宝东, 杨国栋, 马宇涵. 一种基于异步联邦学习的安全聚合机制[J]. 西安邮电大学学报, 2023, 28(1): 50-61. QIN B D, YANG G D, MA Y H. A secure aggregation mechanism based on asynchronous federated learning[J]. Journal of Xi’an University of Posts and Telecommunications, 2023, 28(1): 50-61. (in Chinese)
[7] TRUEX S, BARACALDO N, ANWAR A, et al. A hybrid approach to privacy-preserving federated learning[J]. Informatik Spektrum, 2019, 42(5): 356-357.
[8] XU G W, LI H W, ZHANG Y, et al. Privacy-preserving federated deep learning with irregular users[J]. IEEE Transactions on Dependable and Secure Computing, 2020, 19(2): 1364-1381.
[9] ZIGOMITROS A, CASINO F, SOLANAS A, et al. A survey on privacy properties for data publishing of relational data[J]. IEEE Access, 2020, 8: 51071-51099.
[10] DWORK C. Differential privacy[C]//Proceedings of the 33rd International Conference on Automata, Languages and Programming. Berlin, Germany: Springer, 2006: 1-12.
[11] LIU X Y, LI H W, XU G W, et al. Privacy-enhanced federated learning against poisoning adversaries[J]. IEEE Transactions on Information Forensics and Security, 2021, 16: 4574-4588.
[12] PHONG L T, AONO Y, HAYASHI T, et al. Privacy-preserving deep learning via additively homomorphic encryption[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(5): 1333-1345.
[13] 康海燕, 冀源蕊. 基于本地化差分隐私的联邦学习方法研究[J]. 通信学报, 2022, 43(10): 94-105. KANG H Y, JI Y L. Research on federated learning method based on local differential privacy[J]. Journal of Communications, 2022, 43(10): 94-105. (in Chinese)
[14] BU Z Q, DONG J S, LONG Q, et al. Deep learning with Gaussian differential privacy[J]. Harvard Data Science Review, 2020(23): 10-1162.
[15] ABADI M, CHU A, GOODFELLOW I, et al. Deep learning with differential privacy[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York, USA: ACM, 2016: 308-318.
[16] 王方伟, 谢美云, 李青茹, 等. 自适应裁剪的差分隐私联邦学习框架[J]. 西安电子科技大学学报, 2023, 50(4): 111-120. WANG F W, XIE M Y, LI Q R, et al. An adaptive clipping framework for differential privacy-based federated learning[J]. Journal of Xidian University, 2023, 50(4): 111-120. (in Chinese)
[17] LIU W Y, CHENG J H, WANG X L. Hybrid differential privacy based federated learning for Internet of Things[J]. Journal of System Architecture, 2022, 124: 1-15.
[18] 张晓龙, 罗文华. 利用动态裁剪差分隐私实现联邦学习入侵检测[J]. 小型微型计算机系统, 2024, 45(6): 1474-1481. ZHANG X L, LUO W H. Implementing federated learning intrusion detection using dynamic clipping differential privacy[J]. Journal of Chinese Computer Systems, 2024, 45(6): 1474-1481. (in Chinese)
[19] 杨达森. 交通轨迹数据发布差分隐私保护算法研究[D]. 广州: 广东工业大学, 2020. YANG D S. Research on differential privacy protection algorithms for traffic trajectory data publication[D]. Guangzhou: Guangdong University of Technology, 2020. (in Chinese)
[20] 余晟兴, 陈钟. 基于同态加密的高效安全联邦学习聚合框架[J]. 通信学报, 2023, 44(1): 14-28. YU S X, CHEN Z. An efficient and secure federated learning aggregation framework based on homomorphic encryption[J]. Journal of Communications, 2023, 44(1): 14-28. (in Chinese)
[21] HU H, PENG R, TAI Y W, et al. Network trimming: a data-driven neuron pruning approach towards efficient deep architectures[EB/OL].[2024-01-02]. https://arxiv.org/pdf/1607.03250.
[22] 尚涛, 赵铮, 舒王伟, 等. 基于等差隐私预算分配的大数据决策树算法[J]. 工程科学与技术, 2019, 51(2): 130-136. SHANG T, ZHAO Z, SHU W W, et al. Big data decision tree algorithm based on arithmetic differential privacy budget allocation[J]. Advanced Engineering Sciences, 2019, 51(2): 130-136. (in Chinese)
[23] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[24] KRIZHEVSKY A, HINTON G. Learning multiple layers of features from tiny images[J]. Handbook of Systemic Autoimmune Diseases, 2009,1(4): 1-60.
[25] 孙磊, 杨宇, 毛秀青, 等. 基于空间特征的生成对抗网络数据生成方法[J]. 电子与信息学报, 2023, 45(6): 1959-1969. SUN L, YANG Y, MAO X Q, et al. A data generation method based on generative adversarial networks with spatial characteristics[J]. Journal of Electronics & Information Technology, 2023, 45(6): 1959-1969. (in Chinese)
[26] 于群, 沈志恒, 孙飞飞, 等. 面向云计算应用的用电负荷数据差分隐私保护方法[J]. 电力自动化设备, 2022, 42(7): 68-75. YU Q, SHEN Z H, SUN F F, et al. Differential privacy protection method for power consumption load data oriented to cloud computing applications[J].Electric Power Automation Equipment, 2022, 42(7): 68-75. (in Chinese)
[27] 刘艺璇, 陈红, 刘宇涵, 等. 联邦学习中的隐私保护技术[J].软件学报.2022,33(3): 1057-1092. LIU Y X, CHEN H, LIU Y H, et al. Privacy-preserving Techniques in Federated Learning[J]. Journal of Software, 2021, 33(3): 1057-1092. (in Chinese)

选择文件类型/文献管理软件名称

选择包含的内容