基于FPGA的软硬件协同纠删码编码加速方案

doi:10.19678/j.issn.1000-3428.0066809

摘要/Abstract

摘要：

纠删码容错技术已广泛应用于分布式存储系统，相较于多副本容错技术能显著降低数据存储成本，并且具有更高的数据通信可靠性和安全性，但在数据存储过程中不可避免地会引入额外的计算开销并增加编码时延，导致数据写入吞吐量降低。针对该问题，提出一种基于现场可编程门列阵（FPGA）的纠删码编码加速方案。首先，利用FPGA的高速并行计算优势对纠删码算法进行硬件加速，并实现并行处理和时序优化。然后，针对上位机与FPGA之间因传输速率和处理速率不一致造成内存中的数据溢出问题，在FPGA上拓展了片外DDR3接口用于数据缓存，提高了通信可靠性，并利用DDR3的随机存取特点实现对数据块的分片。最后，设计基于FPGA的纠删码编码硬件加速架构进行实验验证。实验结果表明，与主流Jerasure 2.0开源纠删码库相比，该方案的数据写入吞吐量提升了2.7~93.0倍，尤其对于小文件的编码写入性能提升更为显著。

关键词: 纠删码, 现场可编程门阵列, 硬件加速, 分布式存储, 模块化设计

Abstract:

The erasure code fault-tolerant technology is widely used in current distributed storage systems. Compared with the multicopy fault-tolerant technology, the erasure code can not only significantly reduce the data storage costs, but can also provide improved data reliability and security. However, during the data storage process, the erasure code technology inevitably introduces additional computing overhead, increases the coding delay, and decreases the data write throughput. This study proposes an acceleration scheme of erasure code encoding based on the Field Programmable Gate Array(FPGA) to decrease the coding delay and increase the data write throughput. The specific work includes the following aspects. First, the advantages of high-speed parallel computing of the FPGA is used to speed up the erasure code algorithm and achieve parallel processing and timing optimization. Next, the off-chip DDR3 interface is expanded on the FPGA for data cache, which improves the reliability of communication, to solve the problem of data overflow in memory caused by the inconsistency of the transmission rate and processing rate between the upper computer and FPGA. In addition, the random access feature of DDR3 is used to fragment the data blocks. Finally, a hardware acceleration architecture based on the FPGA for the erasure code is designed for experimental verification. The experimental results show that the FPGA-based acceleration scheme improves the data write throughput by 2.7-93.0 times compared with the current mainstream open-source erasure code library Jerasure 2.0, particularly for the encoding and writing performance of smaller files.

Key words: erasure code, Field Programmable Gate Array(FPGA), hardware acceleration, distributed storage, modular design

杨思捷, 陈俊奇, 王勇, 李树林. 基于FPGA的软硬件协同纠删码编码加速方案[J]. 计算机工程, 2024, 50(2): 224-231.

Sijie YANG, Junqi CHEN, Yong WANG, Shulin LI. FPGA-based Software and Hardware Cooperative Acceleration Scheme of Erasure Code Encoding[J]. Computer Engineering, 2024, 50(2): 224-231.

http://www.ecice06.com/CN/Y2024/V50/I2/224

图/表 12

图1 基于FPGA的RS码编码硬件加速架构

Fig.1 FPGA-based hardware acceleration architecture of RS code encoding

图2 RS码编码器框架

Fig.2 Framework of RS code encoder

图3 纠删码算法模块的主要接口时序

Fig.3 Timing of main interfaces of erasure code algorithm module

图4 DDR3地址结构

Fig.4 DDR3 address structure

图5 RS（8，4）编码过程

Fig.5 RS(8, 4) encoding process

图6 时钟周期数记录数据包

Fig.6 Clock cycle number recording data packet

图7 平均编码速率对比曲线

Fig.7 Comparison curve of average encoding rate

图8 平均数据写入吞吐量对比

Fig.8 Comparison of average data write throughput

参考文献 27

1	POLATO I, RÉ R, GOLDMAN A, et al. A comprehensive view of Hadoop research—a systematic literature review. Journal of Network and Computer Applications, 2014, 46, 1- 25. doi: 10.1016/j.jnca.2014.07.022
2	ANILKUMAR C, SUBRAMANIAN S. A novel predicate based access control scheme for cloud environment using open stack swift storage. Peer-to-Peer Networking and Applications, 2021, 14(4): 2372- 2384. doi: 10.1007/s12083-020-00961-y
3	柯文龙, 王勇, 叶苗, 等. Ceph云存储网络中一种业务优先级区分的多播流调度方法. 通信学报, 2020, 41(11): 40- 51.
	KE W L, WANG Y, YE M, et al. Priority differentiated multicast flow scheduling method in Ceph cloud storage network. Journal on Communications, 2020, 41(11): 40- 51.
4	王勇, 叶苗, 何倩, 等. 基于软件定义网络和多属性决策的Ceph存储系统节点选择方法. 计算机学报, 2019, 42(2): 93- 108.
	WANG Y, YE M, HE Q, et al. A new node selecting approach in Ceph storage system based on software defined network and multi-attributes decision-making model. Chinese Journal of Computers, 2019, 42(2): 93- 108.
5	CHEN J Q, WANG Y, YE M, et al. A load-aware multistripe concurrent update scheme in erasure-coded storage system[EB/OL]. [2022-12-13]. https://downloads.hindawi.com/journals/wcmc/2022/5392474.pdf.
6	REED I S, SOLOMON G. Polynomial codes over certain finite fields. Journal of the Society for Industrial and Applied Mathematics, 1960, 8(2): 300- 304. doi: 10.1137/0108018
7	王意洁, 许方亮, 裴晓强. 分布式存储中的纠删码容错技术研究. 计算机学报, 2017, 40(1): 236- 255.
	WANG Y J, XU F L, PEI X Q. Research on erasure code-based fault-tolerant technology for distributed storage. Chinese Journal of Computers, 2017, 40(1): 236- 255.
8	PLANK J, GREENAN K. Jerasure: a library in C facilitating erasure coding for storage application[EB/OL]. [2022-12-13]. https://www.researchgate.net/publication/228932918_Jerasure_A_library_in_CC_facilitating_erasure_coding_for_storage_applications.
9	TANG Y J, ZHANG X M. Fast en/decoding of Reed-Solomon codes for failure recovery. IEEE Transactions on Computers, 2022, 71(3): 724- 735. doi: 10.1109/TC.2021.3060701
10	CAI H, CHENG M Q, FAN C L, et al. Optimal locally repairable systematic codes based on packings. IEEE Transactions on Communications, 2019, 67(1): 39- 49. doi: 10.1109/TCOMM.2018.2869800
11	CHEN X, MA X. Optimized recovery algorithms for RDP(p, 3)code. IEEE Communications Letters, 2018, 22(12): 2443- 2446. doi: 10.1109/LCOMM.2018.2875468
12	HOU H X, LEE P P C. A new construction of EVENODD codes with lower computational complexity. IEEE Communications Letters, 2018, 22(6): 1120- 1123. doi: 10.1109/LCOMM.2018.2820007
13	BLAUM M, HETZLER S R. Array codes with local properties. IEEE Transactions on Information Theory, 2020, 66(6): 3675- 3690. doi: 10.1109/TIT.2019.2951693
14	鲍禹含, 付印金, 陈卫卫. 多云存储关键技术研究进展. 计算机工程, 2020, 46(10): 18-32, 40. doi: 10.19678/j.issn.1000-3428.0058345
	BAO Y H, FU Y J, CHEN W W. Research progress on key technologies of multi-cloud storage. Computer Engineering, 2020, 46(10): 18-32, 40. doi: 10.19678/j.issn.1000-3428.0058345
15	王晞阳, 陈继林, 李猛, 等. FPGA架构上面向稀疏矩阵求解的静态调度算法. 计算机工程, 2022, 48(7): 199-205, 213. doi: 10.19678/j.issn.1000-3428.0062198
	WANG X Y, CHEN J L, LI M, et al. Static scheduling algorithm for solving sparse matrixes on FPGA architecture. Computer Engineering, 2022, 48(7): 199-205, 213. doi: 10.19678/j.issn.1000-3428.0062198
16	BELYAEV A, POPERECHNY P. Reed-Solomon encoder design by means of the digital filtration[C]//Proceedings of IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering. Washington D. C. , USA: IEEE Press, 2019: 2204-2207.
17	GAO Z, ZHANG L L, CHENG Y H, et al. Design of FPGA-implemented Reed-Solomon Erasure Code(RS-EC) decoders with fault detection and location on user memory. IEEE Transactions on Very Large Scale Integration(VLSI) Systems, 2021, 29(6): 1073- 1082. doi: 10.1109/TVLSI.2021.3066804
18	ZHANG C, FORCHHAMMER S, ANDERSEN J D, et al. Fast SD-hamming decoding in FPGA for high-speed concatenated FEC for optical communication[C]//Proceedings of 2020 IEEE Global Communications Conference. Washington D. C. , USA: IEEE Press, 2020: 1-6.
19	KIM M. Arithmetic gauge theory: a brief introduction. Modern Physics Letters A, 2018, 33(29): 1830012. doi: 10.1142/S0217732318300124
20	CHEN H B, FU S. Parallel erasure coding: exploring task parallelism in erasure coding for enhanced bandwidth and energy efficiency[C]//Proceedings of IEEE International Conference on Networking, Architecture and Storage. Washington D. C. , USA: IEEE Press, 2016: 1-4.
21	孙黎, 苏宇, 张弛, 等. 分布式存储系统中的纠删码容错方法研究. 计算机工程, 2019, 45(11): 74- 80. doi: 10.19678/j.issn.1000-3428.0052959
	SUN L, SU Y, ZHANG C, et al. Research on fault-tolerant method of erasure code for distributed storage system. Computer Engineering, 2019, 45(11): 74- 80. doi: 10.19678/j.issn.1000-3428.0052959
22	LIU C J, WANG Q, CHU X W, et al. G-CRS: GPU accelerated cauchy Reed-Solomon coding. IEEE Transactions on Parallel and Distributed Systems, 2018, 29(7): 1484- 1498. doi: 10.1109/TPDS.2018.2791438
23	许佳豪. 基于廉价GPU的LRC编码加速优化的设计与实现[D]. 武汉: 华中科技大学, 2020.
	XU J H. Design and implementation of LRC acceleration optimization based on cheap GPU[D]. Wuhan: Huazhong University of Science and Technology, 2020. (in Chinese)
24	CHEN G Y, ZHOU H Y, SHEN X P, et al. OpenCL-based erasure coding on heterogeneous architectures[C]//Proceedings of the 27th International Conference on Application-specific Systems, Architectures and Processors. Washington D. C. , USA: IEEE Press, 2016: 33-40.
25	王先鹏. 基于纠删码的SSD自适应容错方案及其性能优化[D]. 武汉: 华中科技大学, 2020.
	WANG X P. Adaptive fault-tolerant scheme and performance optimization of SSD based on erasure coding[D]. Wuhan: Huazhong University of Science and Technology, 2020. (in Chinese)
26	CHINIAH A, MUNGUR A. Dynamic Erasure Coding Policy Allocation(DECPA) in Hadoop 3.0[C]//Proceedings of the 6th IEEE International Conference on Cyber Security and Cloud Computing(CSCloud)/2019 5th IEEE International Conference on Edge Computing and Scalable Cloud(EdgeCom). Washington D. C. , USA: IEEE Press, 2019: 29-33.
27	孙伟东. 分布式存储中基于纠删码的数据容错技术研究[D]. 长沙: 国防科学技术大学, 2015.
	SUN W D. Research on erasure code based data fault-tolerant technology in distributed storage[D]. Changsha: National University of Defense Technology, 2015. (in Chinese)

[1]	陈锐, 孙羽菲, 郭强, 隋轶丞, 周振辉, 石昌青, 张玉志. OclDNN:一种可应用于TensorFlow的通用DNN库[J]. 计算机工程, 2023, 49(4): 138-148.
[2]	洪起润, 王琴. 基于帧间数据复用的稀疏CNN加速器设计[J]. 计算机工程, 2023, 49(12): 55-62.
[3]	陈逸, 刘博生, 徐永祺, 武继刚. 混合精度频域卷积神经网络FPGA加速器设计[J]. 计算机工程, 2023, 49(12): 1-9.
[4]	黄正伟, 刘宏伟, 徐渊. 用于IToF传感器的极低功耗RISC-V专用处理器设计[J]. 计算机工程, 2022, 48(9): 146-154.
[5]	奚智雯, 蔡晶晶, 阳文敏, 柴志雷. 基于微服务架构FPGA云平台的并发请求调度机制[J]. 计算机工程, 2022, 48(7): 206-213.
[6]	巩杰, 赵烁, 何虎, 邓宁. 基于FPGA的量化CNN加速系统设计[J]. 计算机工程, 2022, 48(3): 170-174,196.
[7]	杨珂, 张帆, 郭威, 赵博, 穆清. 一种拟态存储元数据随机性问题解决方法[J]. 计算机工程, 2022, 48(2): 140-146,155.
[8]	黄瑞, 金光浩, 李磊, 姜文超, 宋庆增. 轻量化神经网络加速器的设计与实现[J]. 计算机工程, 2021, 47(9): 185-190,196.
[9]	赵晨园, 李文新, 张庆熙. 一种改进的实时半全局立体匹配算法及硬件实现[J]. 计算机工程, 2021, 47(9): 162-170.
[10]	廖汉松, 吴朝晖, 李斌. 基于RISC-V的卷积神经网络专用指令集处理器[J]. 计算机工程, 2021, 47(7): 196-204.
[11]	狄新凯, 杨海钢. 基于FPGA的稀疏化卷积神经网络加速器[J]. 计算机工程, 2021, 47(7): 189-195,204.
[12]	石永泉, 景乃锋. 基于FPGA模拟的阻变神经网络加速器评估方法[J]. 计算机工程, 2021, 47(12): 209-214.
[13]	吴健凤, 郑博文, 聂一, 柴志雷. 基于OpenCL的3DES算法FPGA加速器[J]. 计算机工程, 2021, 47(12): 147-155,162.
[14]	张浩, 魏敬和. 高效率PLB2AXI总线桥的设计与验证[J]. 计算机工程, 2020, 46(8): 228-234.
[15]	王浩亮, 廉玉忠, 王丽莉. 面向电子证照共享的区块链技术方案研究与实现[J]. 计算机工程, 2020, 46(8): 277-283.

选择文件类型/文献管理软件名称

选择包含的内容