摘要：

高性能数据采集技术是提高数据分析效率的重要前提。为解决当前数据采集技术中安全性低、传输时延高、CPU开销大的问题，设计一种基于远程直接内存访问的高性能单向数据采集（ODAR）架构，提高数据采集过程中的安全性和传输性能。针对传输时数据正确性问题，基于可靠性的数据封装策略，设计动态内存优化策略解决内存分配时存在的时延问题，并提出基于优先级的数据传输调度算法解决高吞吐量数据传输中存在的带宽利用率低的问题。实验结果表明，相对于基于UDP协议实现的单向数据采集技术，ODAR架构的吞吐量平均提高了57.01%，传输时延与CPU开销平均降低了61.27%与68.01%，并且大幅提高了数据传输的准确率，内存分配时的时延平均降低了80.15%，网卡带宽利用率平均提高了33.03%。

关键词: 单向数据采集, 远程直接内存访问, 消息中间件, 动态内存优化, 数据传输调度

Abstract:

High-performance data acquisition technology is an important prerequisite to improving data analysis efficiency. In this paper, a type of high-performance one-way data acquisition architecture called the One-way Data Acquisition by RDMA(ODAR) based on Remote Direct Memory Access(RDMA) is designed to improve the security and transmission performance in the data acquisition process and solve low security, high transmission delay, and high Computer Processing Unit(CPU) overhead in current data acquisition technology. Furthermore, a data encapsulation strategy based on reliability is designed to solve the data correctness problem during transmission, and a dynamic memory optimization approach is proposed to solve the memory allocation delay. A priority-based data transmission-scheduling algorithm is proposed to solve the low bandwidth utilization in high throughput data transmission. The experiment shows that on average, compared with the one-way data acquisition technology based on the User Datagram Protocol(UDP), the throughput of the ODAR architecture increases by 57.01%, the transmission delay decreases by 61.27%, the CPU overhead decreases by 68.01%, and the data transmission accuracy is significantly improved. On average, the time delay of the memory allocation decreases by 80.15%, and the bandwidth utilization of the network card increases by 33.03%.

Key words: one-way data acquisition, Remote Direct Memory Access(RDMA), message middleware, dynamic memory optimization, data transmission scheduling