Personalized Federated Learning Algorithm Based on Multi-Layer Hypernetwork Aggregation

doi:10.19678/j.issn.1000-3428.0070679

Abstract

Abstract:

Personalized Federated Learning (pFL) algorithms have significant advantages in handling non-Independent and Identically Distributed (non-IID) datasets and enabling client-side model personalization. Hypernetwork-based pFL utilizes the client's own hypernetwork to achieve a personalized client model. However, the effect of sharing client-side hypernetwork parameters and client-side data on the accuracy of client-side personalized models remains unclear. A personalized Federated learning with Multi-layer Hypernetwork (pFedMHN) framework is proposed to optimize client models through the weighted aggregation of local and global hypernetworks. The server learns a global hypernetwork and each client's multi-layer local hypernetworks and then aggregates them. Clients use these aggregated hypernetwork parameters to iteratively update their models, resulting in more accurate personalized models. The experimental results show that on general public datasets, the pFedMHN outperforms the four benchmark algorithms in terms of accuracy, effectively solving the problems of data heterogeneity and model accuracy faced during personalized federated learning on non-IID datasets and achieving a more accurate personalized model for clients by utilizing hypernetwork parameters and client data sharing.

Key words: federated learning, federated averaging, hypernetwork, personalization model, model aggregation

摘要：

个性化联邦学习(pFL)算法在处理非独立同分布(non-IID)数据集和客户端模型个性化方面有着巨大优势。而基于超网络的pFL利用客户端各自的超网络实现了客户端模型的个性化训练。然而, 针对客户端超网络参数和客户端数据的共享对客户端个性化模型准确率的影响, 提出多层超网络个性化联邦学习(pFedMHN)框架, 利用局部和全局超网络完成客户端超网络模型的加权聚合, 进而优化客户端模型。在服务器端学习全局超网络和每个客户端的多层局部超网络, 并加权聚合得到客户端超网络, 在客户端利用超网络参数迭代更新客户端模型, 超网络参数的共享实现了客户端更精确的个性化模型。实验结果表明, 在通用公开数据集上, pFedMHN的准确率优于基准算法, 有效解决了在non-IID数据集上个性化联邦学习中数据异构性和模型准确性的问题, 利用超网络参数和客户端数据共享实现了客户端更精确的个性化模型。

关键词: 联邦学习, 联邦平均, 超网络, 个性化模型, 模型聚合

LI Yang, JIANG Yi, CHEN Shuai, YAN Shichao, WANG Lei, MA Li. Personalized Federated Learning Algorithm Based on Multi-Layer Hypernetwork Aggregation[J]. Computer Engineering, 2026, 52(9): 133-142.

李阳, 姜毅, 陈帅, 闫世超, 王磊, 马礼. 多层超网络聚合的个性化联邦学习算法[J]. 计算机工程, 2026, 52(9): 133-142.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0070679

https://www.ecice06.com/EN/Y2026/V52/I9/133

Figures/Tables 8

Fig.1 Overall framework of pFedMHN algorithm

Fig.2 Framework of pFedMHN with hypernetwork

Fig.3 The test accuracy among different algorithms under different datasets

Fig.4 Time consumption among different algorithms under different datasets

Fig.5 Comparison of model storage sizes for different algorithms

References 33

1	FALLAH A , MOKHTARI A , OZDAGLAR A . Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach. Advances in Neural Information Processing Systems, 2020, 33, 3557- 3568.
2	顾永跟, 高凌轩, 吴小红, 等. 非独立同分布下联邦半监督学习的数据分享研究. 计算机工程, 2024, 50 (6): 188- 196. doi: 10.19678/j.issn.1000-3428.0067926
	GU Y G , GAO L X , WU X H , et al. Research on data sharing for federated semi-supervised learning with non-IID. Computer Engineering, 2024, 50 (6): 188- 196. doi: 10.19678/j.issn.1000-3428.0067926
3	HUANG Y T, CHU L Y, ZHOU Z R, et al. Personalized cross-silo federated learning on non-IID data[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/2007.03797.
4	LI T, HU S Y, BEIRAMI A, et al. Ditto: fair and robust federated learning through personalization[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/2012.04221.
5	MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/1602.05629.
6	DUAN M M , LIU D , CHEN X Z , et al. Self-balancing federated learning with global imbalanced data in mobile systems. IEEE Transactions on Parallel and Distributed Systems, 2020, 32 (1): 59- 71.
7	张少波, 张激勇, 朱更明, 等. 基于Bregman散度和差分隐私的个性化联邦学习方法. 软件学报, 2024, 35 (11): 5249- 5262.
	ZHANG S B , ZHANG J Y , ZHU G M , et al. Personalized federated learning method based on Bregman divergence and differential privacy. Journal of Software, 2024, 35 (11): 5249- 5262.
8	GHOSH A , CHUNG J C , YIN D , et al. An efficient framework for clustered federated learning. IEEE Transactions on Information Theory, 2022, 68 (12): 8076- 8091. doi: 10.1109/TIT.2022.3192506
9	SATTLER F , MÜLLER K R , SAMEK W . Clustered federated learning: model-agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32 (8): 3710- 3722.
10	COLLINS L, HASSANI H, MOKHTARI A, et al. Exploiting shared representations for personalized federated learning[C]//Proceedings of International Conference on Machine Learning. [S. l. ]: AAAI Press, 2021: 2089-2099.
11	SHAMSIAN A, NAVON A, FETAYA E, et al. Personalized federated learning using hypernetworks[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/2103.04628.
12	MA X S, ZHANG J, GUO S, et al. Layer-wised model aggregation for personalized federated learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2022: 10092-10101.
13	XING S K, NING Z H, ZHOU J, et al. N-FedAvg: novel federated average algorithm based on FedAvg[C]//Proceedings of the 14th International Conference on Communication Software and Networks (ICCSN). Washington D.C., USA: IEEE Press, 2022: 187-196.
14	ZHANG H , WU T T , CHENG S Y , et al. CC-FedAvg: computationally customized federated averaging. IEEE Internet of Things Journal, 2023, 11 (3): 4826- 4841.
15	ZHOU Y H , YE Q , LÜ J C . Communication-efficient federated learning with compensated overlap-FedAvg. IEEE Transactions on Parallel and Distributed Systems, 2021, 33 (1): 192- 205.
16	SUN T , LI D S , WANG B . Decentralized federated averaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45 (4): 4289- 4301.
17	LAI W L, YAN Q. Federated learning for detecting COVID-19 in chest CT images: a lightweight federated learning approach[C]//Proceedings of the 4th International Conference on Frontiers Technology of Information and Computer (ICFTIC). Washington D.C., USA: IEEE Press, 2022: 146-149.
18	金彪, 李逸康, 姚志强, 等. GenFedRL: 面向深度强化学习智能体的通用联邦强化学习框架. 通信学报, 2023, 44 (6): 183- 197.
	JIN B , LI Y K , YAO Z Q , et al. GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents. Journal of Communications, 2023, 44 (6): 183- 197.
19	ACHITUVE I, SHAMSIAN A, NAVON A, et al. Personalized federated learning with Gaussian processes[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/2106.15482.
20	YANG X Y, HUANG W K, YE M. Dynamic personalized federated learning with adaptive differential privacy[C]//Proceedings of Conference on Neural Information Processing Systems (NeurIPS). New York, USA: [s. n], 2023: 72181-72192.
21	YANG F E, WANG C Y, WANG Y C F. Efficient model personalization in federated learning via client-specificprompt generation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2023: 19159-19168.
22	YUAN L Q , SU L , WANG Z R . Federated transfer-ordered-personalized learning for driver monitoring application. IEEE Internet of Things Journal, 2023, 10 (20): 18292- 18301. doi: 10.1109/JIOT.2023.3279273
23	朱素霞, 顾玢珂, 孙广路. 基于相似度加速的自适应聚类联邦学习. 通信学报, 2024, 45 (3): 197- 207.
	ZHU S X , GU F K , SUN G L . Adaptive clustering federated learning via similarity acceleration. Journal of Communications, 2024, 45 (3): 197- 207.
24	汤凌韬, 王迪, 刘盛云. 面向非独立同分布数据的联邦学习数据增强方案. 通信学报, 2023, 44 (1): 164- 176.
	TANG L T , WANG D , LIU S Y . Data augmentation scheme for federated learning targeting non-IID data. Journal of Communications, 2023, 44 (1): 164- 176.
25	SHAMSIAN A, NAVON A, FETAYA E, et al. Personalized federated learning using hypernetworks[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/2103.04628.
26	YANG Z Y , XIA W J , LU Z X , et al. Hypernetwork-based physics-driven personalized federated learning for CT imaging. IEEE Transactions on Neural Networks and Learning Systems, 2023, 99, 1- 15.
27	LI H X , CAI Z Y , WANG J Y , et al. FedTP: federated learning by transformer personalization. IEEE Transactions on Neural Networks and Learning Systems, 2023, 35 (10): 13426- 13440.
28	DARLOW L N, CROWLEY E J, ANTONIOU A, et al. CINIC-10 is not ImageNet or CIFAR-10[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/1810.03505.
29	SINGLA S, SINGLA S, FEIZI S. Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/2108.04062.
30	COHEN G, AFSHAR S, TAPSON J, et al. EMNIST: extending MNIST to handwritten letters[C]//Proceedings of International Joint Conference on Neural Networks (IJCNN). Washington D.C., USA: IEEE Press, 2017: 2921-2926.
31	XIAO H, RASUL K, VOLLGRAF R. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/1708.07747.
32	MCMAHAN B H, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/1602.05629.
33	LI X X, JIANG M R, ZHANG X F, et al. FedBN: federated learning on non-IID features via local batch normalization[EB/OL]. [2024-11-01]. https://arxiv.org/pdf/2102.07623.

[1]	LIU Tao, MAN Dapeng, XU Chen, LÜ Jiguang, FENG Zhu, ZENG Fanyi, ZHOU Xue, YANG Wu. Learnable Trigger Backdoor Attacks for Federated Learning [J]. Computer Engineering, 2026, 52(9): 306-319.
[2]	SONG Shuhan, TIAN Youliang, WANG Shuai. Secure Aggregation Method for Hierarchical Federated Learning with Non-interactive Verification [J]. Computer Engineering, 2026, 52(8): 149-162.
[3]	CAO Tianya, ZHANG Yufan, JIA Junjie. A Federal Learning Privacy Protection Algorithm with Effectively Improve Data Availability [J]. Computer Engineering, 2026, 52(6): 249-257.
[4]	LI Xuexiang, ZHENG Yongli, ZHANG Yize, DUAN Pengsong. Review of Traffic Analysis Methods Based on Machine Learning and Pre-trained Model [J]. Computer Engineering, 2026, 52(6): 53-67.
[5]	LIU Haijun, FU Xiaodong. Federated Learning on Long-Tail Data Combining Rotational Self-Supervision and CLIP Guidance [J]. Computer Engineering, 2026, 52(5): 129-138.
[6]	WANG Tian, LI Guo, MEI Yaxin, ZHONG Wentao. Survey on Sensor—Cloud and Edge Computing (Invited) [J]. Computer Engineering, 2026, 52(5): 3-42.
[7]	NIU Shufen, WANG Ning, ZHOU Xusheng, KONG Weiying, CHEN Lihua. Secure Federated Learning Scheme Based on Secret Sharing and Homomorphic Encryption in Smart Healthcare [J]. Computer Engineering, 2026, 52(4): 302-312.
[8]	YIN Hengjie, ZHENG Keqing, KE Jiannan, DONG Yunquan. Local Momentum Accelerated Based Non-IID Federated Learning Method [J]. Computer Engineering, 2026, 52(4): 103-110.
[9]	CHEN Xianyi, MI Hui, HE Junjie, FU Zhangjie. Traceable Federated Learning Copyright Protection Method Based on Structural Embedding [J]. Computer Engineering, 2026, 52(2): 253-264.
[10]	CAO Tianya, ZHANG Yujing, JIA Junjie, ZHANG Yufan, DENG Xiaofei. Privacy Protection Algorithm for Federated Learning Based on Personalized Gradient Clipping [J]. Computer Engineering, 2026, 52(2): 265-274.
[11]	LEI Yifan, CHEN Xiaohong. Privacy-Preserving Decentralized Federated Multi-View Clustering [J]. Computer Engineering, 2025, 51(7): 180-189.
[12]	YAO Yupeng, WEI Lifei, ZHANG Lei. A Privacy-Preserving Federated Learning Scheme Against Poisoning Attack [J]. Computer Engineering, 2025, 51(6): 223-235.
[13]	SHI Yonghui, DAI Qi, CHEN Lifang, HAN Yang. Federated Aggregation Algorithm Based on Natural Nearest Neighbors [J]. Computer Engineering, 2025, 51(6): 236-244.
[14]	WU Ruolan, CHEN Yuling, DOU Hui, ZHANG Yangwen, LONG Zhong. Privacy Preserving Algorithm Using Federated Learning Against Attacks [J]. Computer Engineering, 2025, 51(2): 179-187.
[15]	WU Xiaohong, LI Pei, GU Yonggen, TAO Jie. Hierarchical Federated Learning Algorithm Based on EMD Optimal Matching [J]. Computer Engineering, 2025, 51(2): 170-178.

Please choose a citation manager

Content to export