基于Fabric的海量交易数据上链预处理机制

doi:10.19678/j.issn.1000-3428.0067004

摘要/Abstract

摘要：

Hyperledger Fabric是一种国内外广泛使用的联盟链框架，在基于Fabric技术的一些业务中具有协同组织众多、交易操作频繁、事务冲突增加等特点。Fabric采用的多版本并发控制技术能够在一定程度上解决部分交易冲突，提升系统并发性，但其机制不完善，会出现部分交易数据无法正常上链存储的问题。为了实现海量交易数据完整、高效、可信的上链存储，提出一种基于Fabric预言机的数据上链预处理机制。设计海量数据冲突预处理(MCPP)方法，通过检测、监听、延时提交、事务加锁、重排序缓存等方式实现主键冲突交易数据的完整上链。引入数据传输保障措施，在传输过程中利用非对称加密技术防止恶意节点伪造认证信息，确保交易数据链外处理前后的一致性。通过理论分析和实验结果表明，该机制可有效解决联盟链平台中海量交易数据上链时的并发冲突问题，当交易数据规模达到1 000和10 000时，MCPP的时效性比LMLS提高了38%和21.4%，且成功率接近100%，具有高效性和安全性，同时在无并发冲突情况下不影响Fabric系统性能。

关键词: 联盟链, Hyperledger Fabric平台, 预言机, 海量交易数据, 并发冲突, 数据传输

Abstract:

Hyperledger Fabric is an alliance chain framework widely adopted both domestically and internationally. It exhibits characteristics such as numerous participating organizations, frequent transaction operations, and increased transaction conflicts in certain businesses based on Fabric technology. The multi-version concurrency control technology used in Fabric can partially resolve transaction conflicts as well as enhance system concurrency. However, this mechanism is imperfect and certain transaction data cannot be properly stored on the chain. To achieve complete, efficient, and trustworthy up-chain storage of massive transaction data, a data preprocessing mechanism based on the Fabric oracle machine is proposed. The Massive Conflict Preprocessing(MCPP) method is designed to ensure the integrity of transaction data with primary key conflicts through techniques including detection, monitoring, delayed submission, transaction locking, and reordering caching. Data transmission protection measures are introduced to utilize asymmetric encryption technology during transmission, preventing malicious nodes from forging authentication information and ensuring consistency before and after off-chain processing of transaction data. Theoretical analysis and experimental results demonstrate that this mechanism can effectively address concurrent conflict issues regarding up-chain massive transaction data in alliance chain platforms. When the transaction data scales reach 1 000 and 10 000, the MCPP method achieves time efficiency improvements of 38% and 21.4%, respectively, compared with the LMLS algorithm, with a success rate close to 100%. Thus, the proposed method exhibits efficiency and security, and does not impact Fabric system performance when concurrent conflicts do not occur.

Key words: alliance chain, Hyperledger Fabric platform, oracle machine, massive transaction data, concurrency conflict, data transmission

刘颖, 马玉鹏, 赵凡, 王轶, 蒋同海. 基于Fabric的海量交易数据上链预处理机制[J]. 计算机工程, 2024, 50(1): 39-49.

Ying LIU, Yupeng MA, Fan ZHAO, Yi WANG, Tonghai JIANG. Fabric-based Up-chain Preprocessing Mechanism for Mass Transaction Data[J]. Computer Engineering, 2024, 50(1): 39-49.

https://www.ecice06.com/CN/Y2024/V50/I1/39

图/表 13

图1 Fabric冲突处理原理

Fig.1 Principle of handling conflicts in Fabric

图2 整体架构

Fig.2 Overall architecture

图3 MCPP冲突处理流程

Fig.3 Procedure of MCPP for handling conflicts

图4 主键加锁原理

Fig.4 Primary key locking principle

图5 依赖关系图

Fig.5 Dependency graph

图6 数据传输保障措施

Fig.6 Data transmission guarantee measure

图7 不同交易数据总量的时延对比

Fig.7 Comparison of delay of total amount of different transaction data

图8 不同交易到达率下总时延和交易成功率对比

Fig.8 Comparison of total delay and transaction success rate under different transaction arrival rates

图9 常规冲突时延分析

Fig.9 Delay analysis of conventional conflicts

图10 MCPP与LMLS运行时间对比

Fig.10 Comparison of running time of MCPP and LMLS

参考文献 33

1	袁勇, 王飞跃. 区块链技术发展现状与展望. 自动化学报, 2016, 42 (4): 481- 494.
	YUAN Y, WANG F Y. Blockchain: the state of the art and future trends. Acta Automatica Sinica, 2016, 42 (4): 481- 494.
2	ZHANG K W, JACOBSEN H A. Towards dependable, scalable, and pervasive distributed ledgers with blockchains[C]//Proceedings of the 38th International Conference on Distributed Computing Systems. Washington D. C., USA: IEEE Press, 2018: 1337-1346.
3	曾诗钦, 霍如, 黄韬, 等. 区块链技术研究综述: 原理、进展与应用. 通信学报, 2020, 41 (1): 134- 151.
	ZENG S Q, HUO R, HUANG T, et al. Survey of blockchain: principle, progress and application. Journal on Communications, 2020, 41 (1): 134- 151.
4	ANDROULAKI E, BARGER A, BORTNIKOV V, et al. Hyperledger Fabric: a distributed operating system for permissioned blockchains[C]//Proceedings of the 13th EuroSys Conference. New York, USA: ACM Press, 2018: 1-15.
5	RAUCHS M, BLANDIN A, BEAR K, et al. 2nd global enterprise blockchain benchmarking study[EB/OL]. [2023-01-11]. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=3461765.
6	刘懿中, 刘建伟, 张宗洋, 等. 区块链共识机制研究综述. 密码学报, 2019, 6 (4): 395- 432.
	LIU Y Z, LIU J W, ZHANG Z Y, et al. Overview on blockchain consensus mechanisms. Journal of Cryptologic Research, 2019, 6 (4): 395- 432.
7	胡凯, 白晓敏, 高灵超, 等. 智能合约的形式化验证方法. 信息安全研究, 2016, 2 (12): 1080- 1089.
	HU K, BAI X M, GAO L C, et al. Formal verification method of smart contract. Journal of Information Security Research, 2016, 2 (12): 1080- 1089.
8	HEISS J, EBERHARDT J, TAI S. From oracles to trustworthy data on-chaining systems[C]//Proceedings of IEEE International Conference on Blockchain. Washington D. C., USA: IEEE Press, 2020: 496-503.
9	SHARMA A, SCHUHKNECHT F M, AGRAWAL D, et al. Blurring the lines between blockchains and database systems: the case of Hhyperledger Fabric[C]//Proceedings of 2019 International Conference on Management of Data. New York, USA: ACM Press, 2019: 105-122.
10	ISTVÁN Z, SORNIOTTI A, VUKOLIĆ M. StreamChain: do blockchains need blocks?[C]//Proceedings of the 2nd Workshop on Scalable and Resilient Infrastructures for Distributed Ledgers. New York, USA: ACM Press, 2018: 1-6.
11	RUAN P C, LOGHIN D, TA Q T, et al. A transactional perspective on execute-order-validate blockchains[C]//Proceedings of 2020 ACM SIGMOD International Conference on Management of Data. New York, USA: ACM Press, 2020: 543-557.
12	XU L, CHEN W, LI Z X, et al. Locking mechanism for concurrency conflicts on Hyperledger Fabric[M]. Berlin, Germany: Springer International Publishing, 2019: 32-47.
13	毕丹阳, 张钰雯, 毕雅晴. 基于预言机的可信数据上链技术. 信息通信技术与政策, 2021, 47 (9): 79- 84.
	BI D Y, ZHANG Y W, BI Y Q. Trusted data uplink technology based on oracle machine. Information and Communications Technology and Policy, 2021, 47 (9): 79- 84.
14	SATA B, BERLANGA A, CHANEL C P C, et al. Connecting AI-based oracles to blockchains via an auditable auction protocol[C]//Proceedings of the 3rd Conference on Blockchain Research & Applications for Innovative Networks and Services. Washington D. C., USA: IEEE Press, 2021: 23-24.
15	ADLER J, BERRYHILL R, VENERIS A, et al. Astraea: a decentralized blockchain oracle[C]//Proceedings of IEEE International Conference on Internet of Things(iThings) and IEEE Green Computing and Communications(GreenCom) and IEEE Cyber, Physical and Social Computing(CPSCom) and IEEE Smart Data(SmartData). Washington D. C., USA: IEEE Press, 2019: 1145-1152.
16	CAI Y X, FRAGKOS G, TSIROPOULOU E E, et al. A truth-inducing sybil resistant decentralized blockchain oracle[C]//Proceedings of the 2nd Conference on Blockchain Research & Applications for Innovative Networks and Services(BRAINS). Washington D. C., USA: IEEE Press, 2020: 128-135.
17	SOBER M, SCAFFINO G, SPANRING C, et al. A voting-based blockchain interoperability oracle[C]//Proceedings of IEEE International Conference on Blockchain. Washington D. C., USA: IEEE Press, 2022: 160-169.
18	NELATURU K, ADLER J, MERLINI M, et al. On public crowdsource-based mechanisms for a decentralized blockchain oracle. IEEE Transactions on Engineering Management, 2020, 67 (4): 1444- 1458. doi: 10.1109/TEM.2020.2993673
19	MERLINI M, VEIRA N, BERRYHILL R, et al. On public decentralized ledger oracles via a paired-question protocol[C]//Proceedings of IEEE International Conference on Blockchain and Cryptocurrency. Washington D. C., USA: IEEE Press, 2019: 337-344.
20	GAO Z P, ZHUANG Z J, LIN Y J, et al. Select-storage: a new oracle design pattern on blockchain[C]//Proceedings of the 20th International Conference on Trust, Security and Privacy in Computing and Communications(TrustCom). Washington D. C., USA: IEEE Press, 2022: 1177-1184.
21	WANG S, LU H, SUN X K, et al. A novel blockchain oracle implementation scheme based on application specific knowledge engines[C]//Proceedings of IEEE International Conference on Service Operations and Logistics, and Informatics(SOLI). Washington D. C., USA: IEEE Press, 2020: 258-262.
22	ZHANG C, ZHU L H, XU C, et al. PRVB: achieving privacy-preserving and reliable vehicular crowdsensing via blockchain oracle. IEEE Transactions on Vehicular Technology, 2021, 70 (1): 831- 843. doi: 10.1109/TVT.2020.3046027
23	WANG S H, WU C C, LIANG Y C, et al. ProMutator: detecting vulnerable price oracles in DeFi by mutated transactions[C]//Proceedings of IEEE European Symposium on Security and Privacy Workshops(EuroS & PW). Washington D. C., USA: IEEE Press, 2021: 380-385.
24	KUMAR M, NIKHIL N, SINGH R. Decentralising finance using decentralised blockchain oracles[C]//Proceedings of International Conference for Emerging Technology(INCET). Washington D. C., USA: IEEE Press, 2020: 1-4.
25	GOEL S, SINGH A, GARG R, et al. Resource fairness and prioritization of transactions in permissioned blockchain systems(industry track)[C]//Proceedings of the 19th International Middleware Conference Industry. New York, USA: ACM Press, 2018: 46-53.
26	GORENFLO C, LEE S, GOLAB L, et al. FastFabric: scaling Hyperledger Fabric to 20 000 transactions per second. International Journal of Network Management, 2020, 30 (5): e2099. doi: 10.1002/nem.2099
27	THAKKAR P, NATHAN S, VISWANATHAN B. Performance benchmarking and optimizing Hyperledger Fabric blockchain platform[C]//Proceedings of the 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems. Washington D. C., USA: IEEE Press, 2018: 264-276.
28	SHARMA A, SCHUHKNECHT F M, AGRAWAL D, et al. How to databasify a blockchain: the case of Hyperledger Fabric[EB/OL]. [2023-01-11]. https://arxiv.org/abs/1810.13177.
29	SOUSA J, BESSANI A, VUKOLIC M. A Byzantine fault-tolerant ordering service for the Hyperledger Fabric blockchain platform[C]//Proceedings of the 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. Washington D. C., USA: IEEE Press, 2018: 51-58.
30	RAMAN R K, VACULIN R, HIND M, et al. Trusted multi-party computation and verifiable simulations: a scalable blockchain approach[EB/OL]. [2023-01-11]. https://arxiv.org/abs/1809.08438.
31	NASIRIFARD P, MAYER R, JACOBSEN H A. FabricCRDT: a conflict-free replicated data types approach to permissioned blockchains[C]//Proceedings of the 20th International Middleware Conference. New York, USA: ACM Press, 2019: 110-122.
32	THAKKAR P, NATARAJAN S. Scaling Hyperledger Fabric using pipelined execution and sparse Peers[EB/OL]. [2023-01-11]. https://arxiv.org/abs/2003.05113.
33	GAO Z P, LI H L, XIAO K L, et al. Cross-chain oracle based data migration mechanism in heterogeneous blockchains[C]//Proceedings of the 40th International Conference on Distributed Computing Systems. Washington D. C., USA: IEEE Press, 2021: 1263-1268.

[1]	崔怀勇, 张绍华, 李超, 戴炳荣. 一种基于Schnorr签名的区块链预言机改进方案[J]. 计算机工程, 2024, 50(1): 166-173.
[2]	黄保华, 郑慧颖, 屈锡, 陈宁江. 联盟链高效存储访问控制方案[J]. 计算机工程, 2023, 49(8): 37-45.
[3]	王春东, 王翔宇. 多层次实用拜占庭容错算法改进[J]. 计算机工程, 2023, 49(8): 29-36.
[4]	何建江, 陈玉玲. 基于DLIN加密的可监管联盟链隐私保护方案[J]. 计算机工程, 2023, 49(6): 170-179.
[5]	张凌云, 陈玉玲. 基于格上密文策略属性基加密的联盟链数据共享方案[J]. 计算机工程, 2023, 49(11): 30-39.
[6]	刘陕南, 张荣华, 刘长征. 基于分组和信用分级的PBFT共识算法改进方案[J]. 计算机工程, 2023, 49(11): 143-149.
[7]	梁嘉诚, 余江, 王洪波, 刘渊, 王晓锋. 基于RDMA的高性能单向数据采集技术研究[J]. 计算机工程, 2023, 49(10): 31-40.
[8]	贺鹏飞, 范鹏飞, 尹千慧, 王中训, 张桐敬, 梁大伟. 基于负载均衡算法的Hyperledger Fabric共识机制研究[J]. 计算机工程, 2022, 48(11): 170-176.
[9]	方燚飚, 周创明, 雷晓莉, 宋亚飞, 高娜. 基于区块链技术的供应链交易系统设计[J]. 计算机工程, 2021, 47(6): 23-31.
[10]	任智, 杨迪, 胡春, 朱克兰. 一种高信道利用率的无人机自组网单阈值接入协议[J]. 计算机工程, 2021, 47(2): 206-211.
[11]	窦凤鸽, 曹素珍, 马佳佳, 丁晓晖, 王彩芬. 指定使用者的多服务器多关键字可搜索加密方案[J]. 计算机工程, 2021, 47(11): 144-149,157.
[12]	刘美佳, 张箐. 基于分布式集群架构的遥感数据传输机制[J]. 计算机工程, 2021, 47(10): 180-185.
[13]	段靓, 吕鑫, 刘凡. 基于信任委托的区块链分层共识优化[J]. 计算机工程, 2020, 46(10): 120-130,136.
[14]	王彩芬,赵冰,刘超,成玉丹,许钦百. 基于整数多项式环的多对一全同态加密算法[J]. 计算机工程, 2019, 45(4): 130-135.
[15]	刘东东,李勇,徐栋,阮持光,鲁亚凯,刘江兵. 一种负载均衡的RPL多路径数据传输机制[J]. 计算机工程, 2019, 45(1): 67-72,77.

选择文件类型/文献管理软件名称

选择包含的内容