基于Kubernetes的异构任务调度方法

doi:10.19678/j.issn.1000-3428.0069437

摘要/Abstract

摘要：

Kubernetes作为容器编排的主流工具, 可支持自动部署、服务发现以及负载均衡, 且具有高可用性、高效能的特点。然而, Kubernetes采用的最佳适应算法或最小负载法等调度策略忽略了节点的异构性和性能的差异性。此外, Kubernetes工具仅考虑CPU资源和内存资源且预先设置统一权重机制, 容易导致负载不均衡、性能下降以及无法满足精细化调度等问题。针对这些问题, 提出了一种基于多维度资源的异构任务调度(A-KCSS)算法, 该算法基于集群的异构计算资源, 增加磁盘输入/输出(I/O)、网络I/O负载以及GPU资源作为评价指标进行过滤和筛选, 更全面地考虑了node的异构性。此外, 引入一种基于多维度资源因素的权重计算模型, 依据待调度任务的资源需求, 计算待调度任务各维度资源因素的权重值, 结合集群node的实时资源利用率计算出每一个node的评分, 依据评分进行优先级排序, 并选择优先级最高的node进行调度。通过实验在Kubernetes集群上测试了A-KCSS算法的性能, 该算法与Kubernetes默认的调度算法以及Kubernetes容器调度(KCSS)算法相比, 平均响应时间分别减少10%和4%, 吞吐量分别提高30%和15%, 可用性分别提高40%和30%, 负载均衡性分别提高23%和18%, 集群的整体性能得到提高。

关键词: 异构任务, 任务调度, Kubernetes, 负载均衡, 多维度资源, 逼近理想解法

Abstract:

As a mainstream tool for container orchestration, Kubernetes can support automatic deployment, service discovery, and load balancing. It is known for its high availability and performance. However, scheduling strategies such as the best adaptation algorithm or the minimum negative cut method ignore the heterogeneity and energy differences of nodes. In addition, Kubernetes tools only consider CPU and memory resources and set a unified weight mechanism in advance, which can easily lead to problems such as load imbalance, performance degradation, and the inability to satisfy refined scheduling. To address these existing problems, this study proposes a heterogeneous task scheduling algorithm based on multi-dimensional resources, namely A-KCSS. The A-KCSS algorithm is based on the heterogeneous computing resources of a cluster. It adds disk Input/Output (I/O), network I/O load, and GPU resources as evaluation indicators for filtering and screening, and it more comprehensively considers the heterogeneity of nodes. This study also introduces a weight calculation model based on multi-dimensional resource factors. Based on the resource requirements of the task to be scheduled, the weight value of each dimension of the resource factor of the task to be scheduled is calculated, and the score of each node is calculated based on the real-time resource utilization of the cluster. Nodes are prioritized according to the score, and the node with the highest priority is selected for scheduling. The performance of the A-KCSS algorithm is experimentally verified on the Kubernetes cluster. Compared with the default scheduling algorithm and the KCSS algorithm, the average response time is reduced by 10% and 4%, the throughput is increased by 30% and 15%, the availability is improved by 40% and 30%, and the load balancing performance is increased by 23% and 18%, respectively, thereby improving the overall cluster performance.

Key words: heterogeneous tasks, task scheduling, Kubernetes, load balancing, multi-dimensional resources, approximation to the ideal solution

朱亚州, 杜平川, 柴志雷. 基于Kubernetes的异构任务调度方法[J]. 计算机工程, 2025, 51(12): 337-345.

ZHU Yazhou, DU Pingchuan, CHAI Zhilei. Heterogeneous Task Scheduling Method Based on Kubernetes[J]. Computer Engineering, 2025, 51(12): 337-345.

https://www.ecice06.com/CN/Y2025/V51/I12/337

图/表 11

图1 Kubernetes调度原理

Fig.1 Scheduling principle of Kubernetes

图2 请求数对应的成功率对比

Fig.2 Comparison of success rate varying with request number

图3 负载均衡率对比

Fig.3 Comparison of load balancing rates

参考文献 28

1	POULTON N . The Kubernetes book. North Charleston: Independently Published, 2023.
2	赵飞鸿, 孙立峰. 一种基于深度强化学习的资源调度方法. 计算机科学与应用, 2021, 11, 2008.
	ZHAO F H , SUN L F . A resource scheduling method based on deep reinforcement learning. Computer Science and Application, 2021, 11, 2008.
3	刘瑞奇, 李博扬, 高玉金, 等. 新型分布式计算系统中的异构任务调度框架. 软件学报, 2021, 33 (3): 1005- 1017.
	LIU R Q , LI B Y , GAO Y J , et al. Heterogeneous task scheduling framework in a new distributed computing system. Journal of Software, 2021, 33 (3): 1005- 1017.
4	DUSIA A, YANG Y, TAUFER M. Network quality of service in docker containers[C]//Proceedings of the IEEE International Conference on Cluster Computing. Washington D. C., USA: IEEE Press, 2015: 527-528.
5	LE T N, SUN X, CHOWDHURY M, et al. AlloX: compute allocation in hybrid clusters[C]//Proceedings of the Fifteenth European Conference on Computer Systems. Palo Alto, USA: AAAI Press, 2020: 1-16.
6	TOWNEND P, CLEMENT S, BURDETT D, et al. Invited paper: improving data center efficiency through holistic scheduling in Kubernetes[C]//Proceedings of the IEEE International Conference on Service-Oriented System Engineering (SOSE). Washington D. C., USA: IEEE Press, 2019: 156-166.
7	LIN M , XI J Q , BAI W H , et al. Ant colony algorithm for multi-objective optimization of container-based microservice scheduling in cloud. IEEE Access, 2019, 7, 83088- 83100. doi: 10.1109/ACCESS.2019.2924414
8	ZHANG W G, MA X L, ZHANG J Z. Research on Kubernetes' resource scheduling scheme[C]//Proceedings of the 8th International Conference on Communication and Network Security. New York, USA: ACM, 2018: 144-148.
9	OLEGHE O . Container placement and migration in edge computing: concept and scheduling models. IEEE Access, 2021, 9, 68028- 68043. doi: 10.1109/ACCESS.2021.3077550
10	何丰. 基于Kubernetes的异构容器云平台的系统架构与方法优化[D]. 杭州: 杭州电子科技大学, 2022.
	HE F. Optimization of system architecture and method of heterogeneous container cloud platform based on Kubernetes[D]. Hangzhou: Hangzhou Dianzi University, 2022. (in Chinese)
11	MENOUER T . KCSS: Kubernetes container scheduling strategy. The Journal of Supercomputing, 2021, 77 (5): 4267- 4293. doi: 10.1007/s11227-020-03427-3
12	SENJAB K , ABBAS S , AHMED N , et al. A survey of Kubernetes scheduling algorithms. Journal of Cloud Computing, 2023, 12 (1): 87. doi: 10.1186/s13677-023-00471-1
13	PENG Y H, BAO Y X, CHEN Y R, et al. Optimus: an efficient dynamic resource scheduler for deep learning clusters[C]//Proceedings of the 13th EuroSys Conference. New York, USA: ACM, 2018: 1-14.
14	MAO Y , FU Y Q , ZHENG W J , et al. Speculative container scheduling for deep learning applications in a Kubernetes cluster. IEEE Systems Journal, 2022, 16 (3): 3770- 3781. doi: 10.1109/JSYST.2021.3129974
15	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究. 计算机工程, 2024, 50 (9): 82- 91.
	LI J J , DONG J G , LI K . Research on cluster energy-saving strategies based on Kubernetes. Computer Engineering, 2024, 50 (9): 82- 91.
16	TAHERIZADEH S , STANKOVSKI V , CHO J H . Dynamic multi-level auto-scaling rules for containerized applications. The Computer Journal, 2019, 62 (2): 174- 197. doi: 10.1093/comjnl/bxy043
17	RATTIHALLI G, GOVINDARAJU M, LU H, et al. Exploring potential for non-disruptive vertical auto scaling and resource estimation in Kubernetes[C]//Proceedings of the IEEE 12th International Conference on Cloud Computing (CLOUD). Washington D. C., USA: IEEE Press, 2019: 33-40.
18	孔德瑾, 姚晓玲. 面向5G边缘计算的Kubernetes资源调度策略. 计算机工程, 2021, 47 (2): 32- 38. doi: 10.19678/j.issn.1000-3428.0058047
	KONG D J , YAO X L . Kubernetes resource scheduling strategy for 5G edge computing. Computer Engineering, 2021, 47 (2): 32- 38. doi: 10.19678/j.issn.1000-3428.0058047
19	PANDEY V , KOMAL , DINCER H . A review on TOPSIS method and its extensions for different applications with recent development. Soft Computing, 2023, 27 (23): 18011- 18039. doi: 10.1007/s00500-023-09011-0
20	DIAS KNOB L A, KAYSER C H, FERRETO T. Improving container deployment in edge computing using the infrastructure aware scheduling algorithm[C]//Proceedings of the IEEE Symposium on Computers and Communications (ISCC). Washington D. C., USA: IEEE Press, 2021: 1-6.
21	DEWI L P, NOERTJAHYANA A, PALIT H N, et al. Server scalability using Kubernetes[C]//Proceedings of the 4th Technology Innovation Management and Engineering Science International Conference (TIMES-iCON). Washington D. C., USA: IEEE Press, 2019: 1-4.
22	刘志彬, 黄秋兰, 胡庆宝, 等. Kubernetes异构资源细粒度调度策略的设计与实现. 计算机工程, 2023, 49 (2): 31-36, 45. doi: 10.19678/j.issn.1000-3428.0063709
	LIU Z B , HUANG Q L , HU Q B , et al. Design and implementation of fine-grained scheduling strategies for heterogeneous Kubernetes resources. Computer Engineering, 2023, 49 (2): 31-36, 45. doi: 10.19678/j.issn.1000-3428.0063709
23	ZHU L , LI J J , LIU Z J , et al. A multi-resource scheduling scheme of Kubernetes for IIoT. Journal of Systems Engineering and Electronics, 2022, 33 (3): 683- 692. doi: 10.23919/JSEE.2022.000063
24	HSIEH T H, HO K Y, TSAI M Y, et al. Usage-oriented resource allocation strategy in edge computing environments[C]//Proceedings of Advances in Internet, Data and Web Technologies. Berlin, Germany: Springer International Publishing, 2020: 351-360.
25	耿棒棒, 王勇. 基于改进秃鹰搜索算法的Kubernetes资源调度应用. 计算机系统应用, 2023, 32 (4): 187- 196.
	GENG B B , WANG Y . A Kubernetes resource scheduling application based on the improved Vulture search algorithm. Computer System Applications, 2023, 32 (4): 187- 196.
26	ZHU C P, HAN B, ZHAO Y L. A comparative study of spark on the bare metal and Kubernetes[C]//Proceedings of the 6th International Conference on Big Data and Information Analytics (BigDIA). Washington D. C., USA: IEEE Press, 2020: 117-124.
27	ARIZA-PORRAS C , KUZNETSOV V , LEGGER F , et al. The evolution of the CMS monitoring infrastructure. EPJ Web of Conferences, 2021, 251, 02004.
28	PILYAI X, IRSYAD R N, ZAINI I N, et al. Implementation and benchmarking of Kubernetes horizontal pod autoscaling method to event-driven messaging system[C]//Proceedings of Advances on Broad-Band and Wireless Computing, Communication and Applications. Berlin, Germany: Springer Nature Switzerland, 2023: 45-56.

[1]	黄尧, 柴志雷. 基于通信和拓扑感知的SNN分区与映射算法[J]. 计算机工程, 2025, 51(5): 219-228.
[2]	冯爽, 江波, 徐宏. 基于进化自适应蝙蝠算法的异构多核处理器任务调度[J]. 计算机工程, 2025, 51(5): 249-256.
[3]	王克文, 张维庭, 孙童. 空天地一体化算力网络资源调度机制[J]. 计算机工程, 2025, 51(5): 52-61.
[4]	张明, 郭文康, 王海峰. 面向大规模动态图的异构图计算系统设计[J]. 计算机工程, 2025, 51(3): 197-207.
[5]	聂雷, 胡字升, 鲍海洲. 基于RSU辅助和自适应分簇的异构车载网络选择方法[J]. 计算机工程, 2025, 51(3): 162-171.
[6]	魏德宾, 乔维维, 张怡. 基于麻雀搜索算法的软件定义卫星网络控制器部署[J]. 计算机工程, 2025, 51(3): 172-179.
[7]	崔焕庆, 吴一凡, 董柯桢, 周升庆. 基于标签传播与沙丘猫群优化的属性图划分算法[J]. 计算机工程, 2025, 51(12): 180-188.
[8]	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究[J]. 计算机工程, 2024, 50(9): 82-91.
[9]	严长宇, 张磊. 基于任务复制与预调度的混合列表调度算法[J]. 计算机工程, 2024, 50(12): 124-132.
[10]	彭世明, 林士飏, 贾硕, 杨苗会. 基于负载预测的多目标优化任务卸载策略[J]. 计算机工程, 2024, 50(1): 206-215.
[11]	刘向举, 赵犇, 方贤进, 徐杨洋. SDN中基于过程优化的动态负载均衡策略[J]. 计算机工程, 2023, 49(8): 137-145.
[12]	刘志彬, 黄秋兰, 胡庆宝, 程耀东, 胡誉, 田浩来. Kubernetes异构资源细粒度调度策略的设计与实现[J]. 计算机工程, 2023, 49(2): 31-36,45.
[13]	王奎宇, 宋晓勤, 缪娟娟, 张昕婷, 雷磊. 基于SDN的高性能QoS保障低轨道卫星星间路由算法[J]. 计算机工程, 2022, 48(5): 185-190,199.
[14]	贺鹏飞, 范鹏飞, 尹千慧, 王中训, 张桐敬, 梁大伟. 基于负载均衡算法的Hyperledger Fabric共识机制研究[J]. 计算机工程, 2022, 48(11): 170-176.
[15]	李亚朋, 庞建民, 徐金龙, 聂凯. 一种针对线性循环结构的非线性静态调度策略[J]. 计算机工程, 2022, 48(1): 155-162.

选择文件类型/文献管理软件名称

选择包含的内容