A Cloud-Edge-End Hierarchical Federated Compression Algorithm for Mobile Devices

doi:10.19678/j.issn.1000-3428.0252967

Abstract

Abstract: In the real cloud-edge-end hierarchical federated learning scenario (such as the Internet of vehicles environment), the terminal equipment will switch edge servers due to the movement of physical location, resulting in the gradient update of the newly connected edge servers after the terminal moves based on different versions of different edge nodes, which will lead to the deviation of aggregation results and the decline of training efficiency. Existing studies only consider the reallocation of device belonging edge nodes after global aggregation, and the traditional asynchronous federated learning algorithm is difficult to adapt to the scene of frequent device movement, and it is difficult to solve the cross domain problem of mobile terminals. To address this, this research proposes FedSAQ(Federated Learning with Staleness-aware Adaptive Quantization), a mobile-oriented hierarchical federated compression algorithm for cloud-edge-end architectures: firstly, the aging coefficient is calculated by calculating the round difference between the download model and the upload gradient of the terminal device, measuring the difference between the local model and the edge node model, and then the aggregation weight of the edge server is adjusted according to the coefficient, and the adaptive quantization gradient compression algorithm based on aging is adopted, which can effectively use the cross domain training results and reduce the communication overhead. Compared with the benchmark algorithm, the model accuracy has been improved by 0.6% -5%, and communication overhead has been reduced by up to 50%.

摘要： 在现实的云边端分层联邦学习场景（如车联网环境）中，终端设备因物理位置的移动会切换边缘服务器，导致终端移动后新连接的边缘服务器的梯度更新会基于不同边缘节点的不同版本模型，从而引发聚合结果偏差、训练效率下降等问题。现有研究仅考虑了在全局聚合后重新分配设备归属边缘节点，且传统的异步联邦学习算法难以适应设备频繁移动的场景，难以解决移动终端跨域的问题。为此，该研究提出了面向移动设备的云边端分层联邦压缩算法FedSAQ（Federated Learning with Staleness-aware Adaptive Quantization）：该算法首先通过计算终端设备下载模型与上传梯度的轮次差、衡量本地模型与边缘节点模型的差异计算陈旧度系数，然后根据该系数调整边缘服务器的聚合权重，并采用基于陈旧度的自适应量化梯度压缩算法，能有效利用跨域的训练结果并减少通信开销，与基准算法相比，模型精度提升了0.6%-5%，通信开销降低了至多50%。

Huang Ying, Ouyang Peng, Hou Yingwei, Wu Weigang. A Cloud-Edge-End Hierarchical Federated Compression Algorithm for Mobile Devices[J]. Computer Engineering, doi: 10.19678/j.issn.1000-3428.0252967.

黄颖, 欧阳鹏, 侯英威, 吴维刚. 面向移动设备的云边端分层联邦压缩算法[J]. 计算机工程, doi: 10.19678/j.issn.1000-3428.0252967.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0252967

References

[1] Wang Z , Xu H , Liu J ,et al.Resource-Efficient Federated Learning with Hierarchical Aggregation in Edge Computing[C]//IEEE INFOCOM 2021 - IEEE Conference on Computer Communications.IEEE, 2021.DOI:10.1109/INFOCOM42981.2021.9488756.
[1] Xu B , Xia W , Wen W ,et al.Adaptive Hierarchical Federated Learning Over Wireless Networks[J].IEEE Transactions on Vehicular Technology, 2022, 71:2070-2083.DOI:10.1109/TVT.2021.3135541.
[2] Liu L , Zhang J , Song S ,et al.Hierarchical Quantized Federated Learning: Convergence Analysis and System Design[J]. 2021.DOI:10.48550/arXiv.2103.14272.
[3] Wu Q, Chen X, Ouyang T, et al. HiFlash: Communication-Efficient Hierarchical Federated Learning With Adaptive Staleness Control and Heterogeneity-Aware Client-Edge Association[J/OL]. IEEE Transactions on Parallel and Distributed Systems, 2023, 34(5): 1560-1579. DOI:10.1109/tpds.2023.3238049.
[4] Deng Y, Lyu F, Xia T, et al. A Communication-Efficient Hierarchical Federated Learning Framework via Shaping Data Distribution at Edge[J/OL]. IEEE/ACM Transactions on Networking, 2024, 32(3): 2600-2615. DOI:10.1109/tnet.2024.3363916.
[5] Ma Q, Xu Y, Xu H, et al. FedUC: A Unified Clustering Approach for Hierarchical Federated Learning[J/OL]. IEEE Transactions on Mobile Computing, 2024, 23(10): 9737-9756. DOI:10.1109/tmc.2024.3366947.
[6] Luo L, Zhang C, Yu H, et al. Communication-Efficient Federated Learning With Adaptive Aggregation for Heterogeneous Client-Edge-Cloud Network[J/OL]. IEEE Transactions on Services Computing, 2024, 17(6): 3241-3255. DOI:10.1109/tsc.2024.3399649.
[7] Zhou H, Zheng Y, Huang H, et al. Toward Robust Hierarchical Federated Learning in Internet of Vehicles[J/OL]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(5): 5600-5614. DOI:10.1109/tits.2023.3243003.
[8] Yang X, Liwang M, Wang X, et al. HEART: Achieving Timely Multi-Model Training for Vehicle-Edge-Cloud-Integrated Hierarchical Federated Learning[A/OL]. arXiv, 2025[2025-05-23]. http://arxiv.org/abs/2501.09934. DOI:10.48550/arXiv.2501.09934.
[9] 冯奕铭, 钱珍, 李光辉, 代成龙. 异构边缘环境下自适应分层联邦学习协同优化方法[J]. 计算机研究与发展, 2025, 62(6): 1416-1433. DOI: 10.7544/issn1000-1239.202550146 Feng Yiming, Qian Zhen, Li Guanghui, Dai Chenglong. Synergistic Optimization Method for Adaptive Hierarchical Federated Learning in Heterogeneous Edge Environments[J]. Journal of Computer Research and Development, 2025, 62(6): 1416-1433. DOI: 10.7544/issn1000-1239.202550146
[10] He, Zixiao, et al. "FedDT: A Communication-Efficient Federated Learning via Knowledge Distillation and Ternary Compression."[J] Electronics 14.11 (2025): 2183.
[11] Haddadpour, F., Kamani, M. M., Mokhtari, A., & Mahdavi, M. (2021, March). Federated learning with compression: Unified analysis and sharp guarantees. In International Conference on Artificial Intelligence and Statistics[C] (pp. 2350-2358). PMLR.
[12] Abad, Mehdi Salehi Heydar, et al. "Hierarchical federated learning across heterogeneous cellular networks." [C]ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020.
[13] 王汝言, 陈伟, 张普宁, 吴大鹏, 杨志刚. 异构物联网下资源高效的分层协同联邦学习方法[J]. 电子与信息学报, 2023, 45(8): 2847-2855. doi: 10.11999/JEIT220914 WANG Ruyan, CHEN Wei, ZHANG Puning, WU Dapeng, YANG Zhigang. Resource-Efficient Hierarchical Collaborative Federated Learning in Heterogeneous Internet of Things[J]. Journal of Electronics & Information Technology, 2023, 45(8): 2847-2855. doi: 10.11999/JEIT220914
[14] McMahan H B, Moore E, Ramage D, et al. Communication-Efficient Learning of Deep Networks from Decentralized Data[A/OL]. arXiv, 2023[2025-05-24]. http://arxiv.org/abs/1602.05629. DOI:10.48550/arXiv.1602.05629.
[15] Liu L, Zhang J, Song S H, et al. Edge-assisted hierarchical federated learning with non-iid data[J]. arXiv preprint arXiv:1905.06641, 2019.
[16] Xie C , Koyejo S , Gupta I .Asynchronous Federated Optimization[J]. 2019.DOI:10.48550/arXiv.1903.03934.
[17] Yosinski J , Clune J , Bengio Y ,et al.How transferable are features in deep neural networks?.2014[2025-05-24].
[18] Krizhevsky A .One weird trick for parallelizing convolutional neural networks[J].Eprint Arxiv, 2014.DOI:http://hgpu.org/?p=11941.
[19] Oh J , Kim S , Yun S Y .FedBABU: Towards Enhanced Representation for Federated Image Classification[J]. 2021.DOI:10.48550/arXiv.2106.06042.
[20] Nguyen J , Malik K , Zhan H ,et al.Federated Learning with Buffered Asynchronous Aggregation[J]. 2021.DOI:10.48550/arXiv.2106.06639.
[21] Zhou C , Tian H , Zhang H ,et al.TEA-fed: time-efficient asynchronous federated learning for edge computing[C]//CF '21: Computing Frontiers Conference.2021.DOI:10.1145/3457388.3458655.
[22] Chen Y , Sun X , Jin Y .Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted Aggregation[J].IEEE Trans Neural Netw Learn Syst, 2020(10).DOI:10.1109/TNNLS.2019.2953131.
[23] Shi G , Li L , Wang J ,et al.HySync: Hybrid Federated Learning with Effective Synchronization[J].IEEE, 2020.DOI:10.1109/HPCC-SmartCity-DSS50907.2020.00080.
[24] Yurochkin M , Agarwal M , Ghosh S ,et al.Bayesian Nonparametric Federated Learning of Neural Networks[J]. 2019.DOI:10.48550/arXiv.1905.12022.

Please choose a citation manager

Content to export