融合参数个性化的自适应差分隐私联邦学习

doi:10.19678/j.issn.1000-3428.0253490

摘要/Abstract

摘要： 个性化联邦学习通过共享训练参数而非数据进行模型训练，但仍易受推理攻击，因此广泛应用差分隐私技术进行防护。针对传统差分隐私个性化联邦学习（Differentially Private Personalized Federated Learning，DP-PFL）中静态模型划分和统一噪声的局限，本文提出了一种融合参数个性化的自适应差分隐私联邦学习框架DP-FedADC。首先，该框架利用设计的参数自适应划分（Adaptive Parameter Partitioning，APP）实现模型参数的动态分析，并自适应划分个性化参数与共享参数。在此基础上，设计差异化更新策略（Differentiated Parameter Update，DPU），通过对不同类型参数施加差异化正则约束，稳定关键参数更新并缓解梯度裁剪对优化方向的影响。其次，提出客户端级自适应隐私预算分配策略（Client-level Adaptive Privacy Budget Allocation，CAPBA），根据客户端个性化参数比例动态调整隐私预算，使高敏感度客户端获得更严格的隐私保护，同时避免对全局收敛起主导作用的参数施加过度噪声扰动，从而抑制隐私噪声在训练后期的累积效应。在MNIST、CIFAR-10、Fashion-Mnist数据集上的实验表明，在严格差分隐私约束下，DP-FedADC显著提升了分类准确率和领域泛化性能，其测试准确率相较基线方法最高提升约2%-4%，且损失值收敛至更低区间。实验结果验证了所提出框架在差分隐私联邦学习场景下的有效性与鲁棒性。

Abstract: Federated learning trains models by sharing model parameters rather than raw data, but it remains vulnerable to inference attacks, which motivates the integration of differential privacy techniques. To address the limitations of static parameter partitioning and uniform noise injection in conventional Differentially Private Federated Learning (DP-FL), this paper proposes an adaptive differentially private federated learning framework with parameter personalization, termed DP-FedADC. The framework introduces Adaptive Parameter Partitioning (APP) to dynamically analyze model parameters and to separate personalized parameters from shared parameters according to their importance. Based on this partitioning, a Differentiated Parameter Update (DPU) strategy is designed to apply distinct regularization constraints to different parameter types, which stabilizes critical parameter updates and mitigates the distortion of optimization directions caused by gradient clipping. In addition, a Client-level Adaptive Privacy Budget Allocation (CAPBA) strategy is proposed to dynamically adjust privacy budgets according to the proportion of personalized parameters at each client, enabling stronger protection for high-sensitivity clients while avoiding excessive noise perturbation on parameters that dominate global convergence. Experiments conducted on MNIST, CIFAR-10, and Fashion-MNIST demonstrate that under strict differential privacy constraints, DP-FedADC consistently improves classification accuracy, convergence speed, and training stability. Compared with existing baselines, the proposed method achieves up to a 2%–4% improvement in test accuracy and converges to a lower loss range, validating its effectiveness and robustness in differentially private federated learning scenarios.

王炯炯, 张淑芬, 代家佳, 张晗瑞, 张益. 融合参数个性化的自适应差分隐私联邦学习[J]. 计算机工程, doi: 10.19678/j.issn.1000-3428.0253490.

WANG Jiongjiong, ZHANG Shufen, DAI Jiajia, ZHANG Hanrui, ZHANG Yi. Adaptive Differential Privacy Federated Learning with Integrated Parameter Personalization[J]. Computer Engineering, doi: 10.19678/j.issn.1000-3428.0253490.

参考文献

[1] KAIROUZ P, MCMAHAN H B, AVENT B, et al. Advances and open problems in federated learning[J]. Foundations and trends® in machine learning, 2021, 14(1–2）: 1-210.
[2] LI T, SAHU A K, TALWALWAR A, et al. Federated learning: Challenges, methods, and future directions[J]. IEEE signal processing magazine, 2020, 37(3）: 50-60.
[3] HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep models under the GAN: Information leakage from collaborative deep learning[C]//Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. 2017: 603-618.
[4] DWORK C, MCSHERRY F, NISSIM K, et al. Calibrating noise to sensitivity in private data analysis[C]//Theory of cryptography conference. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006: 265-284.
[5] CHENG A, WANG P, ZHANG X S, et al. Differentially private federated learning with local regularization and sparsification[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 10122-10131.
[6] 李阳,姜毅,陈帅,等. 多层超网络聚合的个性化联邦学习算法[J/OL].计算机工程,1-11[2025-12-23].https://doi.org/10.19678/j.issn.1000-3428.0070679. LI Y, JIANG Y, CHENG S, et al. Computer Engineering, Personalized federated learning algorithm based on multi-layer hypernetwork model aggregation 1-11[2025-12-23]. https://doi.org/10.19678/j.issn.1000-3428.0070679. （in Chinese）
[7] LIANG Z, WANG B, GU Q, et al. Differentially private federated learning with Laplacian smoothing[J]. Applied and Computational Harmonic Analysis, 2024, 72: 101660.
[8] GEYER R C, KLEIN T, NABI M. Differentially private federated learning: A client level perspective[J]. arxiv preprint arxiv:1712.07557, 2017.
[9] 曹天涯,张雨静,贾俊杰,等.基于个性化梯度裁剪的联邦学习隐私保护算法[J/OL].计算机工程,1-12[2025-12-23].https://doi.org/10.19678/j.issn.1000-3428.0069644. CAO T Y, ZHANG Y J, JIA J J, et al. Computer Engineering, Privacy-preserving algorithm for federated learning based on personalized gradient clipping. 1-12[2025-12-23]. https://doi.org/10.19678/j.issn.1000-3428.0069644. （in Chinese）
[10] 郭子赟,田有亮,李梦倩.基于激励机制的自适应隐私保护联邦学习方案[J/OL].计算机工程,1-16[2025-12-23].https://doi.org/10.19678/j.issn.1000-3428.0070751. GUO Z B, TIAN Y L, LI M Q. Computer Engineering, Adaptive Privacy-Preserving Federated Learning Scheme Based on Incentive Mechanism. 1-16[2025-12-23]. https://doi.org/10.19678/j.issn.1000-3428.0070751.（in Chinese）
[11] SHAN F, LU Y, LI S, et al. Efficient adaptive defense scheme for differential privacy in federated learning[J]. Journal of Information Security and Applications, 2025, 89: 103992.
[12] LEE S, ZHANG T, AVESTIMEHR A S. Layer-wise adaptive model aggregation for scalable federated learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37(7）: 8491-8499.
[13] TAN A Z, YU H, CUI L, et al. Towards personalized federated learning[J]. IEEE transactions on neural networks and learning systems, 2022, 34(12）: 9587-9603.
[14] ARIVAZHAGAN M G, AGGARWAL V, SINGH A K, et al. Federated learning with personalization layers[J]. arxiv preprint arxiv:1912.00818, 2019.
[15] HUANG Y, CHU L, ZHOU Z, et al. Personalized cross-silo federated learning on non-iid data[C]//Proceedings of the AAAI conference on artificial intelligence. 2021, 35(9）: 7865-7873.
[16] SUN G, MENDIETA M, LUO J, et al. Fedperfix: Towards partial model personalization of vision transformers in federated learning[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2023: 4988-4998.
[17] ARIVAZHAGAN M G, AGGARWAL V, SINGH A K, et al. Federated learning with personalization layers[J]. arxiv preprint arxiv:1912.00818, 2019.
[18] COLLINS L, HASSANI H, MOKHTARI A, et al. Exploiting shared representations for personalized federated learning[C]//International conference on machine learning. PMLR, 2021: 2089-2099.
[19] HUANG C, CHEN X, ZHANG Y, et al. Fedcrl: Personalized federated learning with contrastive shared representations for label heterogeneity in non-iid data[J]. arxiv preprint arxiv:2404.17916, 2024.
[20] LI X, JIANG M, ZHANG X, et al. Fedbn: Federated learning on non-iid features via local batch normalization[J]. arxiv preprint arxiv:2102.07623, 2021.
[21] ZHANG J, HUA Y, WANG H, et al. Fedala: Adaptive local aggregation for personalized federated learning[C]//Proceedings of the AAAI conference on artificial intelligence. 2023, 37(9）: 11237-11244.
[22] WEI K, LI J, MA C, et al. Personalized federated learning with differential privacy and convergence guarantee[J]. IEEE Transactions on Information Forensics and Security, 2023, 18: 4488-4503.
[23] YANG Y, HUI B, YUAN H, et al. {PrivateFL}: Accurate, differentially private federated learning via personalized data transformation[C]//32nd USENIX Security Symposium (USENIX Security 23）. 2023: 1595-1612.
[24] FU J, CHEN Z, HAN X. Adap dp-fl: Differentially private federated learning with adaptive noise[C]//2022 IEEE international conference on trust, security and privacy in computing and communications (TrustCom）. IEEE, 2022: 656-663.
[25] FAN T, CUI Z. Adaptive differential privacy preserving based on multi‐objective optimization in deep neural networks[J]. Concurrency and Computation: Practice and Experience, 2021, 33(20）: e6367.
[26] WANG Z, YU X, HUANG Q, et al. An adaptive differential privacy method based on federated learning[J]. arxiv preprint arxiv:2408.08909, 2024.
[27] GEYER R C, KLEIN T, NABI M. Differentially private federated learning: A client level perspective[J]. arxiv preprint arxiv:1712.07557, 2017.
[28] LI T, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks[J]. Proceedings of Machine learning and systems, 2020, 2: 429-450.
[29] LING X, FU J, WANG K, et al. Ali-dpfl: Differentially private federated learning with adaptive local iterations[C]//2024 IEEE 25th International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM）. IEEE, 2024: 349-358.
[30] Zhao X, Cui Q, Li W, et al. Convergence-privacy-fairness trade-off in personalized federated learning[J]. IEEE Transactions on Machine Learning in Communications and Networking, 2025, 3: 246-262.
[31] Li Y, FU L, WANG T, et al. Clients collaborate: Flexible differentially private federated learning with guaranteed improvement of utility-privacy trade-off[J]. arXiv preprint arXiv:2402.07002, 2024.
[32] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 2002, 86(11): 2278-2324.
[33] KRIZHEVSKY A, HINTON G. Learning multiple layers of features from tiny images.(2009)[EB/OL].(2009-9)
[34] H. XIAO, K. RASUL, and R. VOLLGRAF, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017.

选择文件类型/文献管理软件名称

选择包含的内容