基于超级账本的集群联邦优化模型

doi:10.19678/j.issn.1000-3428.0064301

计算机工程 ›› 2023, Vol. 49 ›› Issue (1): 22-30. doi: 10.19678/j.issn.1000-3428.0064301

基于超级账本的集群联邦优化模型

李尤慧子¹, 俞海涛¹, 殷昱煜¹, 高洪皓^2,3

1. 杭州电子科技大学计算机学院, 杭州 310018;
2. 上海大学计算机工程与科学学院, 上海 200444;
3. 韩国嘉泉大学计算机工程系, 城南 461701

收稿日期:2022-03-25 修回日期:2022-10-14 发布日期:2022-11-16
作者简介:李尤慧子(1989-),女,副教授、博士,主研方向为边缘计算、隐私保护;俞海涛,硕士研究生;殷昱煜、高洪皓,教授、博士。
基金资助:
浙江省“尖兵”“领雁”研发攻关计划“基于异质多模态数据融合的卒中后认知功能障碍智能诊疗技术研究与应用示范”（2022C03043）；浙江省自然科学基金“车联网中全路径轨迹隐私保护关键技术研究”（LY22F020018）。

Cluster Federated Optimization Model Based on Hyperledger Fabric

LI Youhuizi¹, YU Haitao¹, YIN Yuyu¹, GAO Honghao^2,3

1. School of Computer Science, Hangzhou Dianzi University, Hangzhou 310018, China;
2. School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China;
3. Department of Computer Engineering, Gachon University, Seongnam 461701, Republic of Korea

Received:2022-03-25 Revised:2022-10-14 Published:2022-11-16

摘要/Abstract

摘要： 联邦学习作为分布式机器学习框架，在数据不离开本地的情况下，通过共享模型参数达到协作训练的目标，一定程度上解决了隐私保护问题，但其存在中心参数服务器无法应对单点故障、潜在恶意客户端梯度攻击、客户端数据偏态分布导致训练性能低下等问题。将去中心化的区块链技术与联邦学习相结合，提出基于超级账本的集群联邦优化模型。以超级账本作为分布式训练的架构基础，客户端初始化后在本地训练向超级账本传输模型参数及分布信息，通过聚类优化联邦学习模型在客户端数据非独立同分布下的训练表现。在此基础上，随机选举客户端成为领导者，由领导者代替中央服务器的功能，领导者根据分布相似度和余弦相似度聚类并下载模型参数聚合，最后客户端获取聚合模型继续迭代训练。以EMNIST数据集为例，数据非独立同分布情况下该模型平均准确率为79.26%，较FedAvg提高17.26%，在保证准确率的前提下，较集群联邦学习训练至收敛的通信轮次减少36.3%。

关键词: 区块链, 超级账本, 联邦学习, 隐私保护, 智能合约

Abstract: As a distributed machine learning framework, Federated Learning(FL) achieves collaborative training by sharing model parameters without leaving the data locally and ensures privacy protection to a certain extent. However, several challenges are encountered in FL, such as potential malicious client gradient attacks, inability of the central parameter server to cope with a single point of failure, and poor training performance due to skewed client data distribution. In response to the above-mentioned problems, the decentralized blockchain technology is combined with FL and a cluster federated optimization model is proposed based on Hyperledger Fabric. The model uses Hyperledger Fabric as the architectural basis for distributed training. After the client is initialized, the local training transmits model parameters and distribution information to the Hyperledger Fabric. The training performance of FL under the Non-IID distribution of client data is optimized through clustering; then, a client is randomly elected to become the leader; the leader replaces the central server. The leader clusters and downloads the model parameter aggregation according to the distribution similarity and cosine similarity. The client obtains the aggregation model and iterative training continues. On the EMNIST dataset, the average accuracy of the proposed model is 79.26% under Non-IID distribution of data, which is 17.26% higher than that of FedAvg. For a high rate, it requires 36.3% less time than the communication round of Cluster Federated Learning(CFL) training to convergence.

Key words: blockchain, Hyperledger Fabric, Federated Learning(FL), privacy protection, smart contract

中图分类号:

TP391

李尤慧子, 俞海涛, 殷昱煜, 高洪皓. 基于超级账本的集群联邦优化模型[J]. 计算机工程, 2023, 49(1): 22-30.

LI Youhuizi, YU Haitao, YIN Yuyu, GAO Honghao. Cluster Federated Optimization Model Based on Hyperledger Fabric[J]. Computer Engineering, 2023, 49(1): 22-30.

http://www.ecice06.com/CN/Y2023/V49/I1/22

图/表 9

20230701174807

20230701174810

20230701174814

20230701174817

20230701174820

20230701174825

20230701174828

20230701174831

20230701174834

参考文献

[1] RADFORD A, NARASIMHAN K, SALIMANS T, et al.Improving language understanding by generative pre-training[EB/OL].[2022-04-27].https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf.
[2] RADFORD A, WU J, CHILD R, et al.Language models are unsupervised multitask learners[EB/OL].[2022-04-27].https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf.
[3] BROWN T B, MANN B, RYDER N, et al.Language models are few-shot learners[EB/OL].[2022-04-27].https://arxiv.org/pdf/2005.14165.pdf.
[4] WARNAT-HERRESTHAL S, SCHULTZE H, SHASTRY K L, et al.Swarm learning for decentralized and confidential clinical machine learning[J].Nature, 2021, 594(7862):265-270.
[5] JIANG J C, KANTARCI B, OKTUG S, et al.Federated learning in smart city sensing:challenges and opportunities[J].Sensors(Basel, Switzerland), 2020, 20(21):6230.
[6] LIM W Y B, LUONG N C, HOANG D T, et al.Federated learning in mobile edge networks:a comprehensive survey[J].IEEE Communications Surveys & Tutorials, 2020, 22(3):2031-2063.
[7] YANG Q, LIU Y, CHEN T J, et al.Federated machine learning[J].ACM Transactions on Intelligent Systems and Technology, 2019, 10(2):1-19.
[8] ZHU L, LIU Z, HAN S.Deep leakage from gradients[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems.New York, USA:ACM Press, 2019:14774-14784.
[9] MELIS L, SONG C Z, DE CRISTOFARO E, et al.Exploiting unintended feature leakage in collaborative learning[C]//Proceedings of IEEE Symposium on Security and Privacy.Washington D.C., USA:IEEE Press, 2019:691-706.
[10] YAN X D, CUI B J, XU Y, et al.A method of information protection for collaborative deep learning under GAN model attack[J].IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2021, 18(3):871-881.
[11] HITAJ B, ATENIESE G, PEREZ-CRUZ F.Deep models under the GAN:information leakage from collaborative deep learning[C]//Proceedings of 2017 ACM SIGSAC Conference on Computer and Communications Security.New York, USA:ACM Press, 2017:603-618.
[12] PYRGELIS A, TRONCOSO C, DE CRISTOFARO E.Knock knock who's there?Membership inference on aggregate location data[EB/OL].[2022-04-27].https://arxiv.org/pdf/1708.06145.pdf.
[13] SATTLER F, WIEDEMANN S, MÜLLER K R, et al.Robust and communication-efficient federated learning from non-i.i.d.data[J].IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(9):3400-3413.
[14] SAHU A K, LI T, SANJABI M, et al.On the convergence of federated optimization in heterogeneous networks[EB/OL].[2022-04-27].https://arxiv.org/pdf/1812. 06127.pdf.
[15] LI X, HUANG K, YANG W, et al.On the convergence of FedAvg on non-iid data[EB/OL].[2022-04-27].https://arxiv.org/pdf/1907.02189.pdf.
[16] MCMAHAN H B, MOORE E, RAMAGE D, et al.Communication-efficient learning of deep networks from decentralized data[EB/OL].[2022-04-27].https://arxiv.org/abs/1602.05629.
[17] ZHAO Y, LI M, LAI L, et al.Federated learning with non-IID data[EB/OL].[2022-04-27].https://arxiv.org/pdf/1806.00582.pdf.
[18] 朱建明, 张沁楠, 高胜, 等.基于区块链的隐私保护可信联邦学习模型[J].计算机学报, 2021, 44(12):2464-2484. ZHU J M, ZHANG Q N, GAO S, et al.Privacy preserving and trustworthy federated learning model based on blockchain[J].Chinese Journal of Computers, 2021, 44(12):2464-2484.(in Chinese)
[19] SATOSHI N.Bitcoin:a peer-to-peer electronic cash system[EB/OL].[2022-04-27].https://bitcoin.org/en/bitcoin-paper.
[20] The Technical Working Group China.Fabric-documentation[EB/OL].[2022-04-27].https://hyperledger-fabric.readthedocs.io/zh_CN/release-2.2/index.html.
[21] KONEČNÝ J, MCMAHAN B, RAMAGE D, et al.Federated optimization:distributed optimization beyond the datacenter[EB/OL].[2022-04-27].https://arxiv.org/pdf/1511.03575.pdf.
[22] GHOSH A, CHUNG J, YIN D, et al.An efficient framework for clustered federated learning[J].Advances in Neural Information Processing Systems, 2020, 33:19586-19597.
[23] LI X C, ZHAN D C.FedRS:federated learning with Restricted Softmax for label distribution non-IID data[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining.New York, USA:ACM Press, 2021:995-1005.
[24] SATTLER F, MÜLLER K R, SAMEK W.Clustered federated learning:model-agnostic distributed multitask optimization under privacy constraints[J].IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(8):3710-3722.
[25] DUAN M M, LIU D, JI X Y, et al.Flexible clustered federated learning for client-level data distribution shift[J].IEEE Transactions on Parallel and Distributed Systems, 2022, 33(11):2661-2674.
[26] BAO X, SU C, XIONG Y, et al.FLChain:a blockchain for auditable federated learning with trust and incentive[C]//Proceedings of the 5th International Conference on Big Data Computing and Communications.Washington D.C., USA:IEEE Press, 2019:151-159.
[27] LECUN Y, BOTTOU L, BENGIO Y, et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE, 1998, 86(11):2278-2324.

选择文件类型/文献管理软件名称

选择包含的内容

基于超级账本的集群联邦优化模型

Cluster Federated Optimization Model Based on Hyperledger Fabric

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	王群, 李馥娟, 倪雪莉, 夏玲玲, 梁广俊. 区块链数据形成与隐私威胁[J]. 计算机工程, 2023, 49(8): 1-12.
[2]	郑美光, 杨泳. 基于互信息软聚类的个性化联邦学习算法[J]. 计算机工程, 2023, 49(8): 20-28.
[3]	邱天晨, 郑小盈, 祝永新, 封松林. 面向非独立同分布数据的联邦学习架构[J]. 计算机工程, 2023, 49(7): 110-117.
[4]	陈福安, 柳毅. 基于区块链的V2G无证书环签密隐私保护方案[J]. 计算机工程, 2023, 49(6): 34-41,52.
[5]	温依霖, 赵乃良, 曾艳, 韩猛, 岳鲁鹏, 张纪林. 基于本地模型质量的客户端选择方法[J]. 计算机工程, 2023, 49(6): 131-143.
[6]	潘雪, 袁凌云, 黄敏敏. 基于区块链和域信任度的物联网跨域信任评估模型[J]. 计算机工程, 2023, 49(5): 181-190.
[7]	高健博, 张家硕, 李青山, 陈钟. RegLang监管合约规则冲突检测方法[J]. 计算机工程, 2023, 49(5): 12-21,28.
[8]	杨越佳, 华蓓, 钟志威, 高咪. 基于同态加密的隐私保护逻辑回归协同计算[J]. 计算机工程, 2023, 49(4): 23-31.
[9]	唐敏, 张宇浩, 邓国强. 一种高效的非交互式隐私保护逻辑回归模型[J]. 计算机工程, 2023, 49(4): 32-42,51.
[10]	王群, 李馥娟, 倪雪莉, 夏玲玲, 梁广俊. 区块链隐私保护机制研究[J]. 计算机工程, 2023, 49(4): 1-13.
[11]	黄金荣, 刘百祥, 张亮, 张展鹏. 基于智能合约和非同质化代币的去中心化匿名身份认证模型[J]. 计算机工程, 2023, 49(4): 14-22.
[12]	白首圳, 陈美娟. 面向工业互联网的区块链分层分片研究[J]. 计算机工程, 2023, 49(3): 58-66,79.
[13]	陈何雄, 罗宇薇, 韦云凯, 郭威, 杭菲璐, 何映军, 杨宁. 基于联邦学习的SDN异常流量协同检测技术[J]. 计算机工程, 2023, 49(3): 168-176.
[14]	杨俊明, 尹超, 杨铮. 基于激励驱动的轻量级隐私保护位置证明协议[J]. 计算机工程, 2023, 49(3): 151-160.
[15]	张晓均, 刘庆, 郑爽, 王鑫, 薛婧婷, 王世雄. 支持隐私保护的可验证云端数据分享方案[J]. 计算机工程, 2023, 49(3): 49-57.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于超级账本的集群联邦优化模型

Cluster Federated Optimization Model Based on Hyperledger Fabric

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献

相关文章 15

编辑推荐

Metrics

本文评价