DPDK-Based Optimization Approach for High Speed Storage I/O

doi:10.19678/j.issn.1000-3428.0058139

Computer Engineering ›› 2021, Vol. 47 ›› Issue (7): 205-211,217. doi: 10.19678/j.issn.1000-3428.0058139

• Computer Architecture and Software Technology • Previous Articles Next Articles

DPDK-Based Optimization Approach for High Speed Storage I/O

ZHU Wenjun¹, XU Zhuang¹, QIN Jiajia¹, LI Peng^1,2

1. School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210023, China;
2. Jiangsu High Technology Research Key Laboratory for Wireless Sensor Networks, Nanjing 210023, China

Received:2020-04-22 Revised:2020-06-28 Published:2020-07-07

基于DPDK的高速存储I/O优化方法

朱文俊¹, 徐壮¹, 秦家佳¹, 李鹏^1,2

1. 南京邮电大学计算机学院, 南京 210023;
2. 江苏省高性能计算与智能处理工程研究中心, 南京 210023

作者简介:朱文俊(1992-),男,硕士研究生,主研方向为网络安全;徐壮、秦家佳,硕士研究生;李鹏,教授、博士。
基金资助:
国家重点研发计划（2018YFB1003201）；国家自然科学基金（61672296，61602261，61872196，61872194，61902196）；江苏省科技支撑计划项目（BE2017166，BE2019740）；江苏省高等学校自然科学研究重大专项（18KJA520008）；江苏省“六大人才高峰”高层次人才项目（RJFW-111）。

Abstract

Abstract: In the Redis storage process,I/O is a key factor that influences storage performance.Inaccurate default parameter configuration or manual parameter configuration will restrict the performance of storage and increase the delay.To address the problem,an optimization method for storage I/O is proposed. The method fully considers the impact of parameters of each stage on storage performance.Based on the optimization principle of DPDK,the processing characteristics are analyzed,and the layered model is established to effectively predict the storage performance to find the optimal parameter configuration.Experimental results show that the GTS algorithm can effectively improve the storage throughput compared with the default parameters,and reduce delay compared with the ATH approach in write-intensive conditions.

Key words: DPDK technology, performance tuning, memory database, parameter configuration, ATH algorithm

摘要： 网络I/O在Redis存储过程中是限制存储性能的关键因素，而默认参数或人工参数配置会制约存储性能。针对参数配置不当导致存储吞吐性能下降及时延较高的问题，提出一种存储I/O优化方法GTS。考虑各阶段参数对存储性能的影响，在DPDK的优化原理基础上通过分析处理特性，采用分层模型策略实现对存储性能预测，从而寻找出最优参数调优方案。实验结果表明，与默认参数相比，GTS方法能够有效提升存储吞吐量，且在写密集下较ATH算法具有更低的时延。

关键词: DPDK技术, 性能调优, 内存数据库, 参数配置, ATH算法

CLC Number:

TP391

ZHU Wenjun, XU Zhuang, QIN Jiajia, LI Peng. DPDK-Based Optimization Approach for High Speed Storage I/O[J]. Computer Engineering, 2021, 47(7): 205-211,217.

朱文俊, 徐壮, 秦家佳, 李鹏. 基于DPDK的高速存储I/O优化方法[J]. 计算机工程, 2021, 47(7): 205-211,217.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0058139

http://www.ecice06.com/EN/Y2021/V47/I7/205

Figures/Tables 9

References

[1] 唐宏,柴卓原,任平.DPDK应用基础[M].北京:人民邮电出版社,2016. TANG H,CHAI Z Y,REN P.DPDK application basis[M].Beijing:Posts & Telecom Press,2016.(in Chinese)
[2] MIAO M,CHENG W X,REN F Y,et al.Smart batching:a load-sensitive self-tuning packet I/O using dynamic batch sizing[C]//Proceedings of the 18th IEEE International Conference on High Performance Computing and Communications.Washington D.C.,USA:IEEE Press,2016:726-733.
[3] DECANDIA G,HASTORUN D,JAMPANI M.Dynamo:Amazon's highly available key-value store[J].ACM SIGOPS Operating Systems Review,2007,41(6):205-220.
[4] CHENG W X,REN F Y,JIANG W C,et al.Modeling and analyzing latency in the memcached system[C]//Proceedings of the 37th IEEE International Conference on Distributed Computing Systems.Washington D.C.,USA:IEEE Press,2017:346-358.
[5] WANG K,LIU G L,ZHAI M,et al.Building an efficient storage model of spatial-temporal information based on HBase[J].Spatial Science,2018,64(5):1-17.
[6] LEE T,KIM Y,HWANG E.Abnormal payment transaction detection scheme based on scalable architecture and Redis cluster[C]//Proceedings of International Conference on Platform Technology and Service.Jeju,South Korea:[s.n.],2018:1-6.
[7] 李翀,刘利娜,刘学敏,等.一种高效的Redis Cluster的分布式缓存系统[J].计算机系统应用,2018,27(10):91-98. LI C,LIU L N,LIU X M,et al.High efficient distributed cache system based on Redis cluster[J].Computer Systems &Applications,2018,27(10):91-98.(in Chinese)
[8] CHEN S S,TANG X X,WANG H W,et al.Towards scalable and reliable in-memory storage system:a case study with Redis[C]//Proceedings of 2016 IEEE Trustcom/BigDataSE/ISPA.Washington D.C.,USA:IEEE Press,2016:1660-1667.
[9] DANIEL T,PETER S D,ROBERT O.PKTGEN:measuring performance on high speed networks[J].Computer Communications,2016,82:39-48.
[10] AHMAD H,PRASATH V,MOHAMMED A,et al.An improved genetic algorithm with a new initialization mechanism based on regression techniques[J].Information,2018,9(7):167-196.
[11] DING Y,ZHAO Y,MA X B.A novel response-surface-based method in storage performance prediction[C]//Proceedings of Annual Reliability and Maintainability Symposium.Orlando,USA:[s.n.],2019:1-5.
[12] DIAO Y X,HELLERSTEIN J L,PAREKH S,et al.Managing Web server performance with AutoTune agents[J].IBM Systems Journal,2010,42(1):136-149.
[13] TIAN X H,DAI S P,DU Z Z,et al.BigDataBench-S:an open-source scientific big data benchmark suite[C]//Proceedings of IEEE International Parallel and Distributed Processing Symposium.Lake Buena Vista,USA:[s.n.],2017:1068-1077.
[14] 屠雪真,屠要峰,陈小强.一种优化的Key-Value型NoSQL系统[J].计算机工程,2019,45(6):52-59. TU X Z,TU Y F,CHEN X Q.An optimized Key-Value NoSQL system[J].Computer Engineering,2019,45(6):52-59.(in Chinese)
[15] BAO X Q,LIU L,XIAO N,et al.Policy-driven configuration management for NoSQL[C]//Proceedings of ACM International Conference on Cloud Computing.New York,USA:ACM Press,2015:245-252.
[16] JAMSHIDI P,CASALE G.An uncertainty-aware approach to optimal configuration of stream processing systems[C]//Proceedings of the 24th IEEE International Symposium on Modeling,Analysis and Simulation of Computer and Telecommunication Systems.Washington D.C.,USA:IEEE Press,2016:39-48.
[17] CHEN H F,JIANG G F,ZHANG H,et al.Boosting the performance of computing systems through adaptive configuration tuning[C]//Proceedings of the 2009 ACM Symposium on Applied Computing.New York,USA:ACM Press,2009:1045-1049.
[18] QI C X,BI Y M,LI Y.Improved BP neural network algorithm model based on chaos genetic algorithm[C]//Proceedings of the 3rd IEEE International Conference on Control Science and Systems Engineering.Washington D.C.,USA:IEEE Press,2017:679-682.
[19] XIONG W,BEI Z D,XU C Z,et al.ATH:auto-tuning HBase's configuration via ensemble learning[J].IEEE Access,2017,5:13157-13170.
[20] BAO L,LIU X,XU Z H,et al.AutoConfig:automatic configuration tuning for distributed message systems[C]//Proceedings of the 33rd IEEE/ACM International Conference on Automated Software Engineering.Montpellier,France:[s.n.],2018:29-40.
[21] 黄健宏.Redis设计与实现[M].北京:机械工业出版社,2014. HUANG J H.The design and implementation of Redis[M].Beijing:China Machine Press,2014.(in Chinese)
[22] Intel.Intel DPDK:programmers guide[EB/OL].[2020-03-10].http://dpdk.org/doc/guides/prog_guide.
[23] ABUBAKAR Y,ADEVI T S,AUTA I G.Performance evaluation of NoSQL systems using YCSB in a resource austere environment[J].International Journal of Applied Information Systems,2014,7(8):23-27.

Please choose a citation manager

Content to export