Research and Implementation of Computational Storage System for High-Energy Physics

doi:10.19678/j.issn.1000-3428.0070060

Abstract

Abstract:

In high-energy physics experiments, data processing typically involves a compute-storage separated computing model. During the computation process, data must be transferred between the computing and storage nodes. The continuous growth in experimental data and data analysis demands has led to data transfer bottlenecks, reducing the overall processing efficiency of these systems. This paper proposes a computational storage system for high-energy physics. First, the storage software EOS is extended. Computational storage plugins are introduced by building on the original architecture. After parsing user commands, the storage server executes local computations based on the file I/O, thereby reducing data movement, alleviating network pressure, and enhancing data processing efficiency. Second, a computational storage server based on a Central Processing Unit-Field Programmable Gate Array (CPU-FPGA) heterogeneous computing architecture is constructed. Considering the lower computational complexity of I/O-intensive tasks, tasks suitable for parallel computing are offloaded to the FPGA via the PCIe bus, thereby extending the computational capabilities of the storage server. Experimental evaluations show that the computational storage system eliminates queuing time and network latency, thereby shortening the overall execution time of computational tasks. Moreover, leveraging FPGA-based hardware acceleration effectively compensates for the weak computing performance of CPUs in storage servers, thereby enhancing the algorithmic versatility of computational storage devices. In tests based on decoding by LHAASO, the computational storage system achieves a speedup of approximately sixfold.

Key words: computational storage, heterogeneous computing, Field Programmable Gate Array (FPGA), big data, high-energy physics

摘要：

高能物理实验数据处理普遍采用存算分离的计算模式, 计算过程中需要在计算节点和存储节点间传输数据。实验数据和数据分析需求的不断增长造成了数据传输瓶颈, 降低了系统整体的处理效率。针对上述问题提出面向高能物理的可计算存储系统。首先, 对存储软件EOS进行扩展, 在原架构的基础上增加可计算存储插件, 存储服务器解析用户命令后, 在文件I/O的基础上执行本地计算, 减少数据移动, 缓解网络压力, 提升数据处理效率。然后, 构建基于中央处理器-现场可编程门阵列(CPU-FPGA)异构计算架构的可计算存储服务器。针对I/O密集型任务计算复杂度较低的特点, 将适合并行计算的任务通过PCIe总线卸载到FPGA中, 扩展了存储服务器的计算能力。对系统的实验评估结果表明, 可计算存储系统能有效消除排队时间和网络延迟, 进而缩短计算任务的整体执行时间。基于FPGA的硬件加速, 有效弥补了存储服务器中CPU计算性能较弱的缺陷, 提升了可计算存储设备的算法通用性。在基于LHAASO的解码作业的测试中, 可计算存储系统实现了约6倍的速度提升。

关键词: 可计算存储, 异构计算, 现场可编程门阵列, 大数据, 高能物理

GAO Yu, CHENG Yaodong, ZHANG Minxing, CHENG Yaosong, BI Yujiang. Research and Implementation of Computational Storage System for High-Energy Physics[J]. Computer Engineering, 2026, 52(3): 299-307.

高宇, 程耀东, 张敏行, 程垚松, 毕玉江. 面向高能物理的可计算存储系统研究与实现[J]. 计算机工程, 2026, 52(3): 299-307.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0070060

https://www.ecice06.com/EN/Y2026/V52/I3/299

Figures/Tables 10

Fig.1 Computational storage architecture for high-energy physics

Fig.2 EOS software architecture

Fig.3 Computational storage architecture based on EOS

Fig.4 CSSFST execution flow

Fig.5 Computational storage server system architecture

Fig.6 Schematic diagram of FPGA function module

Fig.7 Decode time-consuming comparison in different modes

Fig.8 Time-consuming testing of compression algorithms

Fig.9 The execution process of the decode job

References 25

1	程耀东, 石京燕, 陈刚. 高能物理计算环境概述. 科研信息化技术与应用, 2014, 5 (3): 3- 10. doi: 10.11871/j.issn.1674-9480.2014.03.001
	CHENG Y D , SHI J Y , CHEN G . Overview of high energy physics computing environment. Research Information Technology and Application, 2014, 5 (3): 3- 10. doi: 10.11871/j.issn.1674-9480.2014.03.001
2	PHILIPS S. Lustre: building a file system for 1 000-node clusters[C]//Proceedings of the 2003 Linux Symposium. Washington D. C., USA: IEEE Press, 2003: 380-386.
3	PETERS A , SINDRILARU E , ADDE G . EOS as the present and future solution for data storage at CERN. Journal of Physics: Conference Series, 2015, 664 (4): 042042. doi: 10.1088/1742-6596/664/4/042042
4	郭昕婕, 王光燿, 王绍迪. 存内计算芯片研究进展及应用. 电子与信息学报, 2023, 45 (5): 1888- 1898. doi: 10.11999/JEIT220420
	GUO X J , WANG G Y , WANG S D . Research progress and application of memory computing chips. Journal of Electronics and Information, 2023, 45 (5): 1888- 1898. doi: 10.11999/JEIT220420
5	方旭东, 吴俊杰. 基于忆阻器的计算存储融合体系结构研究进展. 计算机工程与科学, 2020, 42 (11): 1929- 1940. doi: 10.3969/j.issn.1007-130X.2020.11.003
	FANG X D , WU J J . Advance in memristor-based computing storage fusion architecture. Computer Engineering & Science, 2020, 42 (11): 1929- 1940. doi: 10.3969/j.issn.1007-130X.2020.11.003
6	李迦雳, 刘铎, 陈咸彰, 等. 基于闪存存储的近数据处理技术综述. 集成技术, 2022, 11 (3): 23- 41. doi: 10.12146/j.issn.2095-3135.20211019001
	LI J L , LIU D , CHEN X Z , et al. A survey of flash memory based near-data processing technology. Journal of Integration Technology, 2022, 11 (3): 23- 41. doi: 10.12146/j.issn.2095-3135.20211019001
7	SNIA Standard. Computational storage architecture and programming Model[EB/OL]. [2024-06-01]. https://www.snia.org/sites/default/files/technical-work/computational/release/SNIA-Computational-Storage-Architecture-and-Programming-Model-1.0.pdf.
8	LEE J H , ZHANG H , LAGRANGE V , et al. SmartSSD: FPGA accelerated near-storage data analytics on SSD. IEEE Computer Architecture Letters, 2020, 19 (2): 110- 113. doi: 10.1109/LCA.2020.3009347
9	QIAO W K, OH J, GUO L C, et al. FANS: FPGA-accelerated near-storage sorting[C]//Proceedings of the 29th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines. Orlando, USA: IEEE Press, 2021: 106-114.
10	CAO W, LIU Y, CHENG Z S, et al. POLARDB meets computational storage: efficiently support analytical workloads in cloud-native relational database[C]//Proceedings of the 18th USENIX Conference on File and Storage Technologies. Washington D. C., USA: IEEE Press, 2020: 29-41.
11	KWON D, KIM D, BOO J, et al. A fast and flexible hardware-based virtualization mechanism for computational storage devices[C]//Proceedings of the 2021 USENIX Annual Technical Conference. Washington D. C., USA: IEEE Press, 2021: 729-743.
12	YANG Z, LU Y Y, LIAO X J, et al. λ-I/O: a unified I/O stack for computational storage[C]//Proceedings of the 21st USENIX Conference on File and Storage Technologies. Washington D. C., USA: IEEE Press, 2023: 347-362.
13	LeFEVRE J , MALTZAHN C . SkyhookDM: data processing in Ceph with programmable storage. USENIX Magazine, 2020, 45 (2): 13- 18. URL
14	TORABZADEHKASHI M , REZAEI S , HEYDARIGORJI A , et al. Computational storage: an efficient and scalable platform for big data and HPC applications. Journal of Big Data, 2019, 6 (1): 100. doi: 10.1186/s40537-019-0265-5
15	DEAN J , GHEMAWAT S . MapReduce: simplified data processing on large clusters. Communications of the ACM, 2008, 51 (1): 107- 113. doi: 10.1145/1327452.1327492
16	杨思捷, 陈俊奇, 王勇, 等. 基于FPGA的软硬件协同纠删码编码加速方案. 计算机工程, 2024, 50 (2): 224- 231. doi: 10.19678/j.issn.1000-3428.0066809
	YANG S J , CHEN J Q , WANG Y , et al. FPGA-based software and hardware cooperative acceleration scheme of erasure code encoding. Computer Engineering, 2024, 50 (2): 224- 231. doi: 10.19678/j.issn.1000-3428.0066809
17	关明晓, 刘嘉堃, 张鸿锐, 等. 基于FPGA误差可控的浮点运算加速器研究. 计算机工程, 2024, 50 (5): 291- 297. doi: 10.19678/j.issn.1000-3428.0067233
	GUAN M X , LIU J K , ZHANG H R , et al. Study of FPGA-based error-controllable floating-point operation accelerators. Computer Engineering, 2024, 50 (5): 291- 297. doi: 10.19678/j.issn.1000-3428.0067233
18	PUTNAM A , CAULFIELD A M , CHUNG E S , et al. A reconfigurable fabric for accelerating large-scale datacenter services. ACM SIGARCH Computer Architecture News, 2014, 42 (3): 13- 24. doi: 10.1145/2678373.2665678
19	MAAZOUZ M , TOUBAL A , BENGHERBIA B , et al. FPGA implementation of a chaos-based image encryption algorithm. Journal of King Saud University-Computer and Information Sciences, 2022, 34 (10): 9926- 9941. doi: 10.1016/j.jksuci.2021.12.022
20	HEINTZ A, RAZAVIMALEKI V, DUARTE J, et al. Accelerated charged particle tracking with graph neural networks on FPGAs[EB/OL]. [2024-06-01]. https://arxiv.org/abs/2012.01563.
21	WANG J N , TONG W Q , ZHI X L . Model parallelism optimization for CNN FPGA accelerator. Algorithms, 2023, 16 (2): 110. doi: 10.3390/a16020110
22	刘怡俊, 曹宇, 叶武剑, 等. 基于FPGA并行加速的脉冲神经网络在线学习硬件结构的设计与实现. 华南理工大学学报(自然科学版), 2023, 51 (5): 104- 113. doi: 10.12141/j.issn.1000-565X.220623
	LIU Y J , CAO Y , YE W J , et al. Design and implementation of online learning hardware structure of pulsed neural network based on FPGA parallel acceleration. Journal of South China University of Technology(Natural Science Edition), 2023, 51 (5): 104- 113. doi: 10.12141/j.issn.1000-565X.220623
23	DORIGO A , ELMER P , FURANO F , et al. XROOTD: a highly scalable architecture for data access. WSEAS Transactions on Computers, 2005, 1 (4): 348- 353. URL
24	CHENG Y D , LI H B , BI Y J , et al. Construction and application of LHAASO data processing platform. Radiation Detection Technology and Methods, 2022, 6 (3): 418- 426. doi: 10.1007/s41605-022-00328-2
25	BRUN R , RADEMAKERS F . ROOT: an object oriented data analysis framework. Nuclear Instruments and Methods in Physics Research, 1997, 389 (1/2): 81- 86.

[1]	PENG Yun, WANG Yubing, LIANG Lei, SONG Yue, QIU Cheng, LEI Yuxin, JIA Peng, MIAO Guoqing, QIN Li, WANG Lijun. Winograd Heterogeneous Sampling Window Convolution Acceleration Operator [J]. Computer Engineering, 2025, 51(9): 71-79.
[2]	SHANG Jiandong, XIONG Wei, HUA Haobo, SONG Zhaolu, GUO Hengliang, ZHANG Jun. Heterogeneous Implementation of Fluid-Structure Interaction Immersed Boundary Algorithm for Deep Compute Unit [J]. Computer Engineering, 2025, 51(7): 263-274.
[3]	PAN Shunjie, YU Junyang, WANG Longge, LI Han, ZHAI Rui. Spark Adaptive Cache Optimization Strategy Based on the Reuse Degree of RDD [J]. Computer Engineering, 2025, 51(7): 190-198.
[4]	ZHANG Ming, GUO Wenkang, WANG Haifeng. Design of Heterogeneous Graph Computing System for Large-Scale Dynamic Graph [J]. Computer Engineering, 2025, 51(3): 197-207.
[5]	LI Yunjiayun, AN Junshe, ZHOU Li. Design and Verification of Autonomous and Controllable IP Core for Satellite-Borne FlexRay Data Bus [J]. Computer Engineering, 2025, 51(12): 311-323.
[6]	WANG Yukun, XU Xingjian, MENG Fanjun, SONG Huiyuan. Objective Question Difficulty Prediction Model Based on Multi-Feature Attention Bidirectional Recurrent Neural Network [J]. Computer Engineering, 2025, 51(10): 130-139.
[7]	YANG Tailong, ZHAO Hongpeng, ZHANG Lei. Singular Value Decomposition Method Based on Domestic Heterogeneous Platforms [J]. Computer Engineering, 2024, 50(9): 216-225.
[8]	Xiaohua ZHOU, Yuanchun ZHOU, Zhen MENG, Xuezhi WANG. Large-Scale Open Remote Sensing Images Map Rendering and Cache Optimization [J]. Computer Engineering, 2024, 50(7): 227-239.
[9]	Yi SUN, Huimei WANG, Ming XIAN, Hang XIANG. Research on Heterogeneous Computing Scheduling Strategy for Kubeflow [J]. Computer Engineering, 2024, 50(2): 25-32.
[10]	YAN Changyu, ZHANG Lei. Hybrid List-Scheduling Algorithm Based on Task Duplication and Pre-Scheduling [J]. Computer Engineering, 2024, 50(12): 124-132.
[11]	WANG Xindi, YANG Su, ZHANG Siyuan, LUO Wuyang, LI Jie, LIU Hui. Urban Fire Risk Prediction Based on Spatial-Temporal Big Data and Satellite Images [J]. Computer Engineering, 2023, 49(6): 242-249.
[12]	LIU Pengfei, ZHU Jianchen, WAN Liangyi, JIANG Bo. Research on Hyperspectral Remote Sensing Image Classification Using Low-Power Heterogeneous Computing Architecture [J]. Computer Engineering, 2022, 48(12): 9-15,23.
[13]	DI Xinkai, YANG Haigang. FPGA-based Accelerator for Sparse Convolutional Neutral Network [J]. Computer Engineering, 2021, 47(7): 189-195,204.
[14]	GUO Wei, XIE Guangwei, ZHANG Fan, LI Min. Design and Implementation of a Mimic Architecture for Distributed Storage System [J]. Computer Engineering, 2020, 46(6): 12-19.
[15]	WANG Zhihui, WANG Xiaodong. Research on Text Classification Methods Based on Neural Network [J]. Computer Engineering, 2020, 46(3): 11-17.

Please choose a citation manager

Content to export