基于个性化梯度裁剪的联邦学习隐私保护算法

doi:10.19678/j.issn.1000-3428.0069644

摘要/Abstract

摘要：

联邦学习作为目前深度学习最为常用的隐私保护框架, 被众多机构广泛应用。此框架中的各个参与方通过上传模型参数数据实现本地数据不离本地, 达到共享数据的目的。但在联邦学习中各个参与方频繁上传及接收参数时易出现隐私泄露问题。为解决这一问题, 提出一种基于个性化梯度裁剪的联邦学习隐私保护算法(AADP_FL)。该算法根据参与方不同网络层历史数据的L1范数计算出各层的裁剪阈值, 对梯度数据进行裁剪以限制梯度范围, 预防梯度爆炸及梯度消失。同时计算各层的贡献度, 根据各层贡献度为每层分配隐私预算, 进而添加个性化噪声。参与方在上传数据时加入适量的噪声, 以掩盖上传数据的具体内容, 进而隐藏各个参与者的贡献率, 保护各个参与方的数据安全。经过一系列实验证明, AADP_FL算法的准确率相较于常用的个性化梯度裁剪方法提升3.5百分点以上, 相比于传统的联邦学习框架也能保持较高的准确率。同时, 该算法在保持较高准确率的同时能严格保护参与方数据的隐私安全, 使得模型性能与数据隐私性达到均衡状态。

关键词: 联邦学习, 隐私保护, 差分隐私, 隐私预算, 个性化梯度裁剪

Abstract:

Federated learning, as the most commonly used privacy protection framework in deep learning, is widely applied by many institutions. The various participants in this framework achieve the goal of sharing data by uploading model parameter data without leaving the local data. However, in federated learning, privacy leakage occurs when various parties frequently upload and receive parameters. To address this issue, a personalized gradient clipping-based federated learning privacy preserving algorithm (AADP-FL) is proposed. This algorithm calculates the clipping threshold for each layer based on the L1 norm of historical data from different network layers of the participants. The gradient data is then clipped to limit the gradient range and prevent gradient explosion and vanishing gradients. Simultaneously, the contribution of each layer is calculated, privacy budgets are allocated for each layer based on their contribution, and then personalized noise is added. Participants add an appropriate amount of noise when uploading data to conceal the specific content, thereby hiding the contribution rate of each participant and improving the data security for each participant. A series of experiments reveal that the accuracy of this algorithm is superior compared to the commonly used personalized gradient clipping methods, with an accuracy increase of over 3.5 percentage points. This algorithm can also maintain a high accuracy compared with traditional federated learning frameworks. It can effectively protect the privacy of participant data while maintaining high accuracy, achieving a balance between model performance and data privacy.

Key words: federated learning, privacy protection, differential privacy, privacy budget, personalized gradient clipping

曹天涯, 张雨静, 贾俊杰, 张宇帆, 邓晓飞. 基于个性化梯度裁剪的联邦学习隐私保护算法[J]. 计算机工程, 2026, 52(2): 265-274.

CAO Tianya, ZHANG Yujing, JIA Junjie, ZHANG Yufan, DENG Xiaofei. Privacy Protection Algorithm for Federated Learning Based on Personalized Gradient Clipping[J]. Computer Engineering, 2026, 52(2): 265-274.

https://www.ecice06.com/CN/Y2026/V52/I2/265

图/表 6

图1 自适应梯度裁剪方法框架

Fig.1 Framework of adaptive gradient clipping method

图2 MNIST数据集下使用不同p -百分位数的结果对比

Fig.2 Comparison of results using different p -percentiles in the MNIST dataset

图3 不同数据集下使用不同p -百分位数的结果对比

Fig.3 Comparison of results using different p -percentiles under different datasets

图4 不同网络结构下使用不同p -百分位数的结果对比

Fig.4 Comparison of results using different p -percentiles under different network structures

图5 不同通信轮次下不同算法的结果比较

Fig.5 Comparison of results of different algorithms under different communication rounds

参考文献 27

1	MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proceedings of International Conference on Artificial Intelligence and Statistics. Fort Lauderdale, USA: PMLR, 2017: 1273-1282.
2	邹赛兰, 李卓, 陈昕. 面向分层联邦学习的传输优化研究. 计算机科学, 2022, 49(12): 5- 16.
	ZOU S L , LI Z , CHEN X . Research on transmission optimization for hierarchical federated learning. Computer Science, 2022, 49(12): 5- 16.
3	SONG M K , WANG Z B , ZHANG Z F , et al. Analyzing user-level privacy attack against federated learning. IEEE Journal on Selected Areas in Communications, 2020, 38(10): 2430- 2444. doi: 10.1109/JSAC.2020.3000372
4	MELIS L, SONG C Z, CRISTOFARO E, et al. Exploiting unintended feature leakage in collaborative learning[C]//Proceedings of the IEEE Symposium on Security and Privacy. San Francisco, USA: IEEE Press, 2019: 691-706.
5	HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep models under the GAN: information leakage from collaborative deep learning[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York, USA: ACM, 2017: 603-618.
6	秦宝东, 杨国栋, 马宇涵. 一种基于异步联邦学习的安全聚合机制. 西安邮电大学学报, 2023, 28(1): 50- 61.
	QIN B D , YANG G D , MA Y H . A secure aggregation mechanism based on asynchronous federated learning. Journal of Xi'an University of Posts and Telecommunications, 2023, 28(1): 50- 61.
7	TRUEX S , BARACALDO N , ANWAR A , et al. A hybrid approach to privacy-preserving federated learning. Informatik Spektrum, 2019, 42(5): 356- 357. doi: 10.1007/s00287-019-01205-x
8	XU G W , LI H W , ZHANG Y , et al. Privacy-preserving federated deep learning with irregular users. IEEE Transactions on Dependable and Secure Computing, 2020, 19(2): 1364- 1381.
9	ZIGOMITROS A , CASINO F , SOLANAS A , et al. A survey on privacy properties for data publishing of relational data. IEEE Access, 2020, 8, 51071- 51099. doi: 10.1109/ACCESS.2020.2980235
10	DWORK C. Differential privacy[C]//Proceedings of the 33rd International Conference on Automata, Languages and Programming. Berlin, Germany: Springer, 2006: 1-12.
11	LIU X Y , LI H W , XU G W , et al. Privacy-enhanced federated learning against poisoning adversaries. IEEE Transactions on Information Forensics and Security, 2021, 16, 4574- 4588. doi: 10.1109/TIFS.2021.3108434
12	PHONG L T , AONO Y , HAYASHI T , et al. Privacy-preserving deep learning via additively homomorphic encryption. IEEE Transactions on Information Forensics and Security, 2018, 13(5): 1333- 1345. doi: 10.1109/TIFS.2017.2787987
13	康海燕, 冀源蕊. 基于本地化差分隐私的联邦学习方法研究. 通信学报, 2022, 43(10): 94- 105.
	KANG H Y , JI Y L . Research on federated learning method based on local differential privacy. Journal of Communications, 2022, 43(10): 94- 105.
14	BU Z Q , DONG J S , LONG Q , et al. Deep learning with Gaussian differential privacy. Harvard Data Science Review, 2020(23): 10- 1162.
15	ABADI M, CHU A, GOODFELLOW I, et al. Deep learning with differential privacy[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York, USA: ACM, 2016: 308-318.
16	王方伟, 谢美云, 李青茹, 等. 自适应裁剪的差分隐私联邦学习框架. 西安电子科技大学学报, 2023, 50(4): 111- 120.
	WANG F W , XIE M Y , LI Q R , et al. An adaptive clipping framework for differential privacy-based federated learning. Journal of Xidian University, 2023, 50(4): 111- 120.
17	LIU W Y , CHENG J H , WANG X L . Hybrid differential privacy based federated learning for Internet of Things. Journal of System Architecture, 2022, 124, 1- 15.
18	张晓龙, 罗文华. 利用动态裁剪差分隐私实现联邦学习入侵检测. 小型微型计算机系统, 2024, 45(6): 1474- 1481.
	ZHANG X L , LUO W H . Implementing federated learning intrusion detection using dynamic clipping differential privacy. Journal of Chinese Computer Systems, 2024, 45(6): 1474- 1481.
19	杨达森. 交通轨迹数据发布差分隐私保护算法研究[D]. 广州: 广东工业大学, 2020.
	YANG D S. Research on differential privacy protection algorithms for traffic trajectory data publication[D]. Guangzhou: Guangdong University of Technology, 2020. (in Chinese)
20	余晟兴, 陈钟. 基于同态加密的高效安全联邦学习聚合框架. 通信学报, 2023, 44(1): 14- 28.
	YU S X , CHEN Z . An efficient and secure federated learning aggregation framework based on homomorphic encryption. Journal of Communications, 2023, 44(1): 14- 28.
21	HU H, PENG R, TAI Y W, et al. Network trimming: a data-driven neuron pruning approach towards efficient deep architectures[EB/OL]. [2024-01-02]. https://arxiv.org/pdf/1607.03250.
22	尚涛, 赵铮, 舒王伟, 等. 基于等差隐私预算分配的大数据决策树算法. 工程科学与技术, 2019, 51(2): 130- 136.
	SHANG T , ZHAO Z , SHU W W , et al. Big data decision tree algorithm based on arithmetic differential privacy budget allocation. Advanced Engineering Sciences, 2019, 51(2): 130- 136.
23	LECUN Y , BOTTOU L , BENGIO Y , et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278- 2324. doi: 10.1109/5.726791
24	KRIZHEVSKY A , HINTON G . Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 2009, 1(4): 1- 60.
25	孙磊, 杨宇, 毛秀青, 等. 基于空间特征的生成对抗网络数据生成方法. 电子与信息学报, 2023, 45(6): 1959- 1969.
	SUN L , YANG Y , MAO X Q , et al. A data generation method based on generative adversarial networks with spatial characteristics. Journal of Electronics & Information Technology, 2023, 45(6): 1959- 1969.
26	于群, 沈志恒, 孙飞飞, 等. 面向云计算应用的用电负荷数据差分隐私保护方法. 电力自动化设备, 2022, 42(7): 68- 75.
	YU Q , SHEN Z H , SUN F F , et al. Differential privacy protection method for power consumption load data oriented to cloud computing applications. Electric Power Automation Equipment, 2022, 42(7): 68- 75.
27	刘艺璇, 陈红, 刘宇涵, 等. 联邦学习中的隐私保护技术. 软件学报, 2022, 33(3): 1057- 1092.
	LIU Y X , CHEN H , LIU Y H , et al. Privacy-preserving Techniques in Federated Learning. Journal of Software, 2021, 33(3): 1057- 1092.

[1]	陈先意, 糜慧, 何俊杰, 付章杰. 基于结构嵌入的可溯源联邦学习版权保护方法[J]. 计算机工程, 2026, 52(2): 253-264.
[2]	毕昌兵, 田有亮. 车联网中基于身份签名的匿名可追溯消息认证方案[J]. 计算机工程, 2025, 51(9): 158-165.
[3]	雷一凡, 陈晓红. 隐私保护的去中心联邦多视图聚类[J]. 计算机工程, 2025, 51(7): 180-189.
[4]	姚玉鹏, 魏立斐, 张蕾. 一种隐私保护的抗投毒攻击联邦学习方案[J]. 计算机工程, 2025, 51(6): 223-235.
[5]	施永辉, 代琪, 陈丽芳, 韩阳. 基于自然最近邻的联邦聚合算法[J]. 计算机工程, 2025, 51(6): 236-244.
[6]	黄舒琳, 章志明, 杨伟. 基于环结构的无线传感器网络基站位置隐私保护路由协议[J]. 计算机工程, 2025, 51(4): 198-207.
[7]	吴小红, 李佩, 顾永跟, 陶杰. 基于EMD最优匹配的分层联邦学习算法[J]. 计算机工程, 2025, 51(2): 170-178.
[8]	吴若岚, 陈玉玲, 豆慧, 张洋文, 龙钟. 抗攻击的联邦学习隐私保护算法[J]. 计算机工程, 2025, 51(2): 179-187.
[9]	苗炜捷, 吴文渊. 基于同态加密的隐私保护逻辑回归模型训练方案[J]. 计算机工程, 2025, 51(12): 68-81.
[10]	王华华, 黄烨霞, 李玲, 王嘉程. 无蜂窝网络中的联邦学习用户调度与资源优化[J]. 计算机工程, 2025, 51(12): 255-267.
[11]	牛渲文, 杜晔, 杨明松, 李昂, 黎妹红. 基于层次联邦与动态权重的卫星网络异常检测方法[J]. 计算机工程, 2025, 51(12): 210-220.
[12]	王圆圆, 王世谦, 王涵, 郭正宾, 胡显承. 基于纵向联邦学习的能源排放跨界智能分析[J]. 计算机工程, 2025, 51(1): 164-173.
[13]	陈先意, 丁思哲, 王康, 闫雷鸣, 付章杰. 一种支持安全联邦学习的主动保护模型水印框架[J]. 计算机工程, 2025, 51(1): 138-147.
[14]	潘恩元, 钟原, 李平. 联邦异质性数据下半监督颈椎MRI分割模型[J]. 计算机工程, 2024, 50(9): 367-376.
[15]	李红娇, 王宝金, 王朝晖, 胡仁豪. 基于模型相似度与本地损失的双重客户端选择算法[J]. 计算机工程, 2024, 50(8): 153-164.

选择文件类型/文献管理软件名称

选择包含的内容