一种鲁棒的半监督联邦学习系统

doi:10.19678/j.issn.1000-3428.0061911

计算机工程 ›› 2022, Vol. 48 ›› Issue (6): 107-114,123. doi: 10.19678/j.issn.1000-3428.0061911

一种鲁棒的半监督联邦学习系统

王树芬¹, 张哲², 马士尧², 陈俞强³, 伍一²

1. 哈尔滨石油学院信息工程学院, 哈尔滨 150028;
2. 黑龙江大学数据科学与技术学院, 哈尔滨 150080;
3. 广州航海学院信息与通信工程学院, 广州 510725

收稿日期:2021-06-15 修回日期:2021-07-23 发布日期:2021-08-12
作者简介:王树芬(1982—),女,副教授、硕士,主研方向为边缘计算、联邦学习;张哲、马士尧,硕士研究生;陈俞强,教授、博士;伍一(通信作者),教授。
基金资助:
国家自然科学基金“基于DIBR绘制3D图像认证的关键技术研究”（61702224）。

A Robust Semi-Supervised Federated Learning System

WANG Shufen¹, ZHANG Zhe², MA Shiyao², CHEN Yuqiang³, WU Yi²

1. School of Information Engineering, Harbin Institute of Petroleum, Harbin 150028, China;
2. School of Data Science and Technology, Heilongjiang University, Harbin 150080, China;
3. School of Information and Communication Engineering, Guangzhou Maritime University, Guangzhou 510725, China

Received:2021-06-15 Revised:2021-07-23 Published:2021-08-12

摘要/Abstract

摘要： 联邦学习允许边缘设备或客户端将数据存储在本地来合作训练共享的全局模型。主流联邦学习系统通常基于客户端本地数据有标签这一假设，然而客户端数据一般没有真实标签，且数据可用性和数据异构性是联邦学习系统面临的主要挑战。针对客户端本地数据无标签的场景，设计一种鲁棒的半监督联邦学习系统。利用FedMix方法分析全局模型迭代之间的隐式关系，将在标签数据和无标签数据上学习到的监督模型和无监督模型进行分离学习。采用FedLoss聚合方法缓解客户端之间数据的非独立同分布（non-IID）对全局模型收敛速度和稳定性的影响，根据客户端模型损失函数值动态调整局部模型在全局模型中所占的权重。在CIFAR-10数据集上的实验结果表明，该系统的分类准确率相比于主流联邦学习系统约提升了3个百分点，并且对不同non-IID水平的客户端数据更具鲁棒性。

关键词: 联邦学习, 半监督联邦学习, 数据异构性, 一致性损失, 鲁棒性

Abstract: Federated Learning(FL) allows edge devices or clients to cooperatively train a shared global model by storing data locally.Mainstream FL systems are typically based on the assumption that client-side local data contain labels;however, client-side data generally do not contain abundant real labels.Meanwhile, data availability and heterogeneity are the main challenges encountered by FL systems.A robust Semi-Supervised Federated Learning(SSFL) system is designed for scenarios where client local data are unlabeled.The FedMix method is used to analyze implicit relationships between global model iterations, whereas supervised and unsupervised models are learned separately on labeled and unlabeled data.The FedLoss aggregation method is used to alleviate the effect of not Identically and Independently Distributed(non-IID) data between clients on the convergence speed and stability of the global model, and the weight of the local model in the global model is dynamically adjusted based on the loss function value of the client model.Experimental results on the CIFAR-10 dataset show that the classification accuracy of this system is approximately 3 percentage points higher than that of the mainstream FL system, and that it is more robust to client data of different non-IID levels.

Key words: Federated Learning(FL), Semi-Supervised Federated Learning(SSFL), data heterogeneity, consistency loss, robustness

中图分类号:

TP301.6

王树芬, 张哲, 马士尧, 陈俞强, 伍一. 一种鲁棒的半监督联邦学习系统[J]. 计算机工程, 2022, 48(6): 107-114,123.

WANG Shufen, ZHANG Zhe, MA Shiyao, CHEN Yuqiang, WU Yi. A Robust Semi-Supervised Federated Learning System[J]. Computer Engineering, 2022, 48(6): 107-114,123.

http://www.ecice06.com/CN/Y2022/V48/I6/107

图/表 8

20220625174817

20220625174820

20220625174824

20220625174828

20220625174832

20220625174835

20220625174839

20220625174843

参考文献

[1] MCMAHAN H B, MOORE E, RAMAGE D, et al.Communication-efficient learning of deep networks from decentralized data[EB/OL].[2021-05-13].https://arxiv.org/abs/1602.05629.
[2] 杨文琦, 章阳, 聂江天, 等.基于联邦学习的无线网络节点能量与信息管理策略[J].计算机工程, 2022, 48(1):188-196, 203. YANG W Q, ZHANG Y, NIE J T, et al.Energy and information management strategy based on federated learning for wireless network nodes[J].Computer Engineering, 2022, 48(1):188-196, 203.(in Chinese)
[3] 温亚兰, 陈美娟.融合联邦学习与区块链的医疗数据共享方案[J].计算机工程, 2022, 48(5):145-153, 161. WEN Y L, CHEN M J.Medical data sharing scheme combined with federal learning and blockchain[J].Computer Engineering, 2022, 48(5):145-153, 161.(in Chinese)
[4] 赵健, 张鑫褆, 李佳明, 等.群体智能2.0研究综述[J].计算机工程, 2019, 45(12):1-7. ZHAO J, ZHANG X T, LI J M, et al.Research review of crowd intelligence 2.0[J].Computer Engineering, 2019, 45(12):1-7.(in Chinese)
[5] LIU Y, YU J J Q, KANG J W, et al.Privacy-preserving traffic flow prediction:a federated learning approach[J].IEEE Internet of Things Journal, 2020, 7(8):7751-7763.
[6] 彭红艳, 凌娇, 覃少华, 等.面向边缘计算的属性加密方案[J].计算机工程, 2021, 47(1):37-43. PENG H Y, LING J, QIN S H, et al.Attribute-based encryption scheme for edge computing[J].Computer Engineering, 2021, 47(1):37-43.(in Chinese)
[7] LIU Y, YUAN X L, ZHAO R H, et al.RC-SSFL:towards robust and communication-efficient semi-supervised federated learning system[EB/OL].[2021-05-31].https://arxiv.org/abs/2012.04432.
[8] ITAHARA S, NISHIO T, KODA Y, et al.Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-IID private data[EB/OL].[2021-05-31].https://arxiv.org/abs/2008. 06180v2.
[9] JIN Y L, WEI X G, LIU Y, et al.Towards utilizing unlabeled data in federated learning:a survey and prospective[EB/OL].[2021-05-31].https://arxiv.org/abs/2002.11545?context=cs.
[10] JEONG W, YOON J, YANG E, et al.Federated semi-supervised learning with inter-client consistency & disjoint learning[EB/OL].[2021-05-31].https://arxiv.org/abs/2006.12097.
[11] ZHU X J, GOLDBERG A B.Introduction to semi-supervised learning[J].Synthesis Lectures on Artificial Intelligence and Machine Learning, 2009, 3(1):105-130.
[12] LONG Z W, CHE L W, WANG Y Q, et al.FedSemi:an adaptive federated semi-supervised learning framework[EB/OL].[2021-05-31].https://arxiv.org/abs/2012.03292.
[13] PARK S, PARK J K, SHIN S J, et al.Adversarial dropout for supervised and semi-supervised learning[EB/OL].[2021-05-31].https://arxiv.org/abs/1707.03631.
[14] LI X X, JIANG M R, ZHANG X F, et al.FedBN:federated learning on non-IID features via local batch normalization[EB/OL].[2021-05-31].https://arxiv.org/abs/2102. 07623.
[15] LEE D H.Pseudo-label:the simple and efficient semi-supervised learning method for deep neural networks[EB/OL].[2021-05-31].http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.664.3543&rep=rep1&type=pdf.
[16] LAINE S, AILA T.Temporal ensembling for semi-supervised learning[EB/OL].[2021-05-31].https://arxiv.org/abs/1610.02242.
[17] TARVAINEN A, VALPOLA H.Mean teachers are better role models:weight-averaged consistency targets improve semi-supervised deep learning results[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.New York, USA:ACM Press, 2017:1195-1204.
[18] ZHANG Z M, YANG Y Q, YAO Z W, et al.Improving semi-supervised federated learning by reducing the gradient diversity of models[C]//Proceedings of 2021 IEEE International Conference on Big Data.Washington D.C., USA:IEEE Press, 2021:1214-1225.
[19] XIE Q Z, DAI Z H, HOVY E, et al.Unsupervised data augmentation for consistency training[EB/OL].[2021-05-31].https://arxiv.org/abs/1904.12848.
[20] LI Q B, DIAO Y Q, CHEN Q, et al.Federated learning on non-IID data silos:an experimental study[EB/OL].[2021-05-31].https://arxiv.org/abs/2102.02079.
[21] LI T, SAHU A K, ZAHEER M, et al.Federated optimization in heterogeneous networks[EB/OL].[2021-05-31].https://arxiv.org/abs/1812.06127.
[22] WANG J Y, LIU Q H, LIANG H, et al.Tackling the objective inconsistency problem in heterogeneous federated optimization[EB/OL].[2021-05-31].https://arxiv.org/abs/2007.07481v1.
[23] BERTHELOT D, CARLINI N, GOODFELLOW I, et al.MixMatch:a holistic approach to semi-supervised learning[EB/OL].[2021-05-31].https://arxiv.org/abs/1905. 02249.
[24] SOHN K, BERTHELOT D, LI C L, et al.FixMatch:simplifying semi-supervised learning with consistency and confidence[EB/OL].[2021-05-31].https://arxiv.org/abs/2001.07685.
[25] YUROCHKIN M, AGARWAL M, GHOSH S, et al.Bayesian nonparametric federated learning of neural networks[EB/OL].[2021-05-31].https://arxiv.org/abs/1905.12022.
[26] HSU T M H, QI H, BROWN M.Measuring the effects of non-identical data distribution for federated visual classification[EB/OL].[2021-05-31].https://arxiv.org/abs/1909.06335v1.
[27] 张曼, 闫飞, 阎高伟, 等.基于狄利克雷问题的路网控制子区动态划分[J].计算机工程, 2020, 46(12):21-26, 35. ZHANG M, YAN F, YAN G W, et al.Dynamic partition of control sub-regions in road network based on Dirichlet problem[J].Computer Engineering, 2020, 46(12):21-26, 35.(in Chinese)
[28] LIU Y, YUAN X L, XIONG Z H, et al.Federated learning for 6G communications:challenges, methods, and future directions[J].China Communications, 2020(8):105-118.

选择文件类型/文献管理软件名称

选择包含的内容

一种鲁棒的半监督联邦学习系统

A Robust Semi-Supervised Federated Learning System

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	郑美光, 杨泳. 基于互信息软聚类的个性化联邦学习算法[J]. 计算机工程, 2023, 49(8): 20-28.
[2]	温依霖, 赵乃良, 曾艳, 韩猛, 岳鲁鹏, 张纪林. 基于本地模型质量的客户端选择方法[J]. 计算机工程, 2023, 49(6): 131-143.
[3]	陈何雄, 罗宇薇, 韦云凯, 郭威, 杭菲璐, 何映军, 杨宁. 基于联邦学习的SDN异常流量协同检测技术[J]. 计算机工程, 2023, 49(3): 168-176.
[4]	李尤慧子, 俞海涛, 殷昱煜, 高洪皓. 基于超级账本的集群联邦优化模型[J]. 计算机工程, 2023, 49(1): 22-30.
[5]	刘金硕, 詹岱依, 邓娟, 王丽娜. 基于深度神经网络和联邦学习的网络入侵检测[J]. 计算机工程, 2023, 49(1): 15-21,30.
[6]	周全兴, 李秋贤, 丁红发, 樊玫玫. 基于博弈论优化的高效联邦学习方案[J]. 计算机工程, 2022, 48(8): 144-151,159.
[7]	陈乃月, 金一, 李浥东, 蔡露鑫, 魏圆梦. 基于区块链的公平性联邦学习模型[J]. 计算机工程, 2022, 48(6): 33-41.
[8]	温亚兰, 陈美娟. 融合联邦学习与区块链的医疗数据共享方案[J]. 计算机工程, 2022, 48(5): 145-153,161.
[9]	胡彬, 王晓军, 张雷. 一种半监督对抗鲁棒模型无关元学习方法[J]. 计算机工程, 2022, 48(12): 112-118.
[10]	柏财通, 崔翛龙, 李爱. 基于本地蒸馏联邦学习的鲁棒语音识别技术[J]. 计算机工程, 2022, 48(10): 103-109.
[11]	杨文琦, 章阳, 聂江天, 杨和林, 康嘉文, 熊泽辉. 基于联邦学习的无线网络节点能量与信息管理策略[J]. 计算机工程, 2022, 48(1): 188-196,203.
[12]	刘迪洋, 张震, 张进. 基于社区结构的复杂网络鲁棒性优化策略[J]. 计算机工程, 2021, 47(8): 84-92.
[13]	廖俊帆, 顾益军, 张培晶, 廖茜. 端到端说话人辨认的对抗样本应用比较研究[J]. 计算机工程, 2021, 47(6): 132-141.
[14]	李浩, 张晓强. 基于单线程的无锚点目标检测模型[J]. 计算机工程, 2021, 47(5): 229-235,243.
[15]	田钰杰, 管有庆, 龚锐. 一种鲁棒的多特征点云分类分割深度神经网络[J]. 计算机工程, 2021, 47(11): 234-240.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

一种鲁棒的半监督联邦学习系统

A Robust Semi-Supervised Federated Learning System

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献

相关文章 15

编辑推荐

Metrics

本文评价