一种智能多路径路由及子流分配协同算法

doi:10.19678/j.issn.1000-3428.0059488

计算机工程 ›› 2021, Vol. 47 ›› Issue (9): 136-144,152. doi: 10.19678/j.issn.1000-3428.0059488

一种智能多路径路由及子流分配协同算法

徐啸^1,2, 顾玲丽^1,2, 陈建平^1,2, 傅启明^1,2,3, 陆悠^1,2,3

1. 苏州科技大学电子与信息工程学院, 江苏苏州 215009;
2. 苏州科技大学江苏省建筑智慧节能重点实验室, 江苏苏州 215009;
3. 苏州科技大学天平学院电子与信息工程学院, 江苏苏州 215011

收稿日期:2020-09-10 修回日期:2020-10-13 发布日期:2020-11-02
作者简介:徐啸(1997-),男,硕士研究生,主研方向为多路径传输控制、强化学习;顾玲丽,硕士研究生;陈建平,教授、博士;傅启明,讲师、博士;陆悠(通信作者),副教授、博士。
基金资助:
国家自然科学基金（61876217，61876121，61772357，61750110519）；江苏省重点研发计划（BE2017663）；江苏省高等学校自然科学研究面上项目（18KJB520045）。

An Intelligent Cooperative Algorithm for Multi-Path Routing and Subflow Allocation

XU Xiao^1,2, GU Lingli^1,2, CHEN Jianping^1,2, FU Qiming^1,2,3, LU You^1,2,3

1. School of Electronical and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, China;
2. Jiangsu Province Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, China;
3. School of Electronical and Information Engineering, Tianping College of Suzhou University of Science and Technology, Suzhou, Jiangsu 215011, China

Received:2020-09-10 Revised:2020-10-13 Published:2020-11-02

摘要/Abstract

摘要： 传统单一路径的传输机制难以满足当前以智慧城市为代表的新一代应用对时延、丢包率等网络性能的要求，而现有多路径传输机制在路由算法及子流分配等方面不能根据网络实时状态调整且互相缺乏协同。引入强化学习理论并结合软件定义网络，提出多路径路由及子流分配协同算法。基于Q-learning设计多路径路由算法，并从策略协同角度对其进行改进，实现路由与子流分配的相互协同。在此基础上，通过Q-value的回环消除方法保证路由准确性并提高算法收敛速度。实验结果表明，该算法在网络负载动态变化过程中能实时调整最佳的多路径路由及子流分配协同策略，提高了传输成功率。

关键词: 多路径路由, 强化学习, 协同决策, 流量调度, 软件定义网络

Abstract: The new generation of applications such as smart cities put forward higher requirements for network performance, including lower delay and packet loss rate.However, the traditional single-path transmission mechanism fail to meet the new requirements, while the existing multi-path transmission technology is limited by some problems in routing and subflow allocation, such as the difficulty in adjusting strategies according to the real-time network conditions and the lack of cooperation.An intelligent cooperative algorithm for multi-path routing and subflow allocation is proposed, which is based on the reinforcement learning theory and software defined network.For this algorithm, a multi-path routing method based on Q-learning is designed and improved from the perspective of strategy coordination to realize mutual coordination between routing and subflow allocation. On this basis, a loop elimination method based on Q-value is used to ensure the validity of routing decision and to improve the convergence speed of the algorithm.The experimental results show that the proposed algorithm can adjust the optimal cooperative strategy for multi-path routing and subflow allocation in real time according to the dynamically changing network loads.It improves the transmission success rate.

Key words: multi-path routing, reinforcement learning, collaborative decision-making, traffic scheduling, Software Defined Network(SDN)

中图分类号:

TP393

徐啸, 顾玲丽, 陈建平, 傅启明, 陆悠. 一种智能多路径路由及子流分配协同算法[J]. 计算机工程, 2021, 47(9): 136-144,152.

XU Xiao, GU Lingli, CHEN Jianping, FU Qiming, LU You. An Intelligent Cooperative Algorithm for Multi-Path Routing and Subflow Allocation[J]. Computer Engineering, 2021, 47(9): 136-144,152.

http://www.ecice06.com/CN/Y2021/V47/I9/136

图/表 15

20210917190838

20210917190842

20210917190845

20210917190849

20210917190853

20210917190857

20210917190901

20210917190905

20210917190911

20210917190915

20210917190919

20210917190923

20210917190927

20210917190931

20210917190935

参考文献

[1] 江卓, 吴茜, 李贺武, 等.互联网端到端多路径传输跨层优化研究综述[J].软件学报, 2019, 30(2):112-132. JIANG Z, WU Q, LI H W, et al.Survey on internet end-to-end multipath transfer research with cross-layer optimization[J].Journal of Software, 2019, 30(2):112-132.(in Chinese)
[2] 王竹, 袁青云, 郝凡凡, 等.基于链路容量的多路径拥塞控制算法[J].通信学报, 2020, 41(5):59-71. WANG Z, YUAN Q Y, HAO F F, et al.Multipath congestion control algorithm based on link capacity[J].Journal on Communications, 2020, 41(5):59-71.(in Chinese)
[3] 杨洋, 杨家海, 温皓森.基于时隙传输的数据中心路由算法设计[J].软件学报, 2018, 29(8):2485-2500. YANG Y, YANG J H, WEN H S.Routing algorithm design based on timeslot of transmission for data centers[J].Journal of Software, 2018, 29(8):2485-2500.(in Chinese)
[4] ALHARBI F, FEI Z.An SDN architecture for improving throughput of large flows using multipath TCP[C]//Proceedings of the 5th IEEE International Conference on Cyber Security and Cloud Computing.Washington D.C., USA:IEEE Press, 2018:1-6.
[5] DU P, PANG F, BRAUN T, et al.Traffic optimization in software defined naval network for satellite communications[C]//Proceedings of 2017 IEEE Military Communications Conference.Washington D.C., USA:IEEE Press, 2017:12-15.
[6] ZHAO P, ZHAO W, LIU Q.Multi-objective link-separation multipath selection using k max-min for software-defined MANETs[C]//Proceedings of the 15th International Conference on Mobile Ad-Hoc and Sensor Networks.Washington D.C., USA:IEEE Press, 2019:12-18.
[7] DUAN J, WANG Z, WU C.Responsive multipath TCP in SDN-based datacenters[C]//Proceedings of 2015 IEEE International Conference on Communications.Washington D.C., USA:IEEE Press, 2015:5296-5301.
[8] ZHAO J, LIU J, WANG H, et al.Multipath TCP for datacenters:from energy efficiency perspective[C]//Proceedings of 2017 IEEE Conference on Computer Communications.Washington D.C., USA:IEEE Press, 2017:1-9.
[9] NAM H, CALIN D, SCHULZRINNE H.Towards dynamic MPTCP path control using SDN[C]//Proceedings of 2016 IEEE NetSoft Conference.Washington D.C., USA:IEEE Press, 2016:286-294.
[10] TARIQ S, BASSIOUNI M.QAMO-SDN:QoS aware multipath TCP for software defined optical networks[C]//Proceedings of the 12th Annual IEEE Consumer Communications and Networking Conference.Washington D.C., USA:IEEE Press, 2015:485-491.
[11] DHALIWAL H K, LUNG C H.Load balancing using ECMP in multi-stage clos topology in a datacenter[C]//Proceedings of 2018 IEEE Conference on Dependable and Secure Computing.Washington D.C., USA:IEEE Press, 2018:1-7.
[12] RHAMDANI F, SUWASTIKA N A, NUGROHO M A.Equal-cost multipath routing in data center network based on software defined network[C]//Proceedings of the 6th International Conference on Information and Communication Technology.Washington D.C., USA:IEEE Press, 2018:222-226.
[13] 陈章迎, 朱尚明.基于重叠网络的多路径路由设计和实现[J].通信学报, 2018, 39(S1):200-205. CHEN Z Y, ZHU S M.Designing and implementation of multi-path QoS routing algorithm based on overlay network[J].Journal on Communications, 2018, 39(S1):200-205.(in Chinese)
[14] 高先明, 王宝生, 邓文平.SDRS:集中与分布控制相结合的弹性多路径路由机制[J].计算机学报, 2018, 41(9):1976-1989. GAO X M, WANG B S, DENG W P.SDRS:elastic multi-path routing mechanism with integration of centralization control and distribution control[J].Chinese Journal of Computers, 2018, 41(9):1976-1989.(in Chinese)
[15] YE J L, CHEN C, CHU Y H.A weighted ECMP load balancing scheme for data centers using P4 switches[C]//Proceedings of 2018 IEEE International Conference on Cloud Networking.Washington D.C., USA:IEEE Press, 2018:1-4.
[16] MARCO C, GABOR R, MICHAEL S.Oblivious routing in IP networks[J].IEEE/ACM Transactions on Networking, 2018:1-14.
[17] 黄建洋, 兰巨龙, 胡宇翔, 等.一种基于分段路由的多路径流传输机制[J].电子学报, 2018, 46(6):1488-1495. HUANG J Y, LAN J L, HU Y X, et al.A segment routing based multipath flow transmission mechanism[J].Acta Electronica Sinica, 2018, 46(6):1488-1495.(in Chinese)
[18] 蔡岳平, 王昌平.软件定义数据中心网络混合路由机制[J].通信学报, 2016, 37(4):44-52. CAI Y P, WANG C P.Software defined data center network with hybrid routing[J].Journal on Communications, 2016, 37(4):44-52.(in Chinese)
[19] XU J W, WU M Q, HOU X L.A traffic scheduling scheme for data center networks based on SDN[C]//Proceedings of the 5th International Conference on Computer and Communications.Washington D.C., USA:IEEE Press, 2019:1417-1422.
[20] HUANG X, CHENG S, CAO K, et al.A survey of deployment solutions and optimization strategies for hybrid SDN networks[J].IEEE Communications Surveys & Tutorials, 2019, 21(2):1483-1507.
[21] 潘成胜, 刘勇, 石怀峰, 等.SDN架构下的空间信息网络业务识别技术[J].计算机工程, 2019, 45(4):18-24. PAN C S, LIU Y, SHI H F, et al.Spatial information network business identification technology under SDN architecture[J].Computer Engineering, 2019, 45(4):18-24.(in Chinese)
[22] 彭大芹, 赖香武, 刘艳林.基于SDN的胖树数据中心网络多路径路由算法[J].计算机工程, 2018, 44(4):41-45, 65. PENG D Q, LAI X W, LIU Y L.Multi-path routing algorithm for fat-tree data center network based on SDN[J].Computer Engineering, 2018, 44(4):41-45, 65.(in Chinese)
[23] MAKSIC N.Two-phase load balancing for data center networks using OpenFlow[C]//Proceedings of the 25th Telecommunication Forum.Washington D.C., USA:IEEE Press, 2017:1-4.
[24] NARAYAN D G, MUDENAGUDI U.A joint approach to routing metrics and application-aware rate adaptation in multi-radio wireless mesh networks[J].Journal of High Speed Networks, 2017, 23(2):93-108.
[25] 刘辰屹, 徐明伟, 耿男, 等.基于机器学习的智能路由算法综述[J].计算机研究与发展, 2020, 57(4):671-687. LIU C Y, XU M W, GENG N, et al.A survey on machine learning based routing algorithms[J].Journal of Computer Research and Development, 2020, 57(4):671-687.(in Chinese)
[26] LUONG N C, HOANG D T, GONG S, et al.Applications of deep reinforcement learning in communications and networking:a survey[J].IEEE Communications Surveys and Tutorials, 2019, 21(4):3133-3174.

选择文件类型/文献管理软件名称

选择包含的内容

一种智能多路径路由及子流分配协同算法

An Intelligent Cooperative Algorithm for Multi-Path Routing and Subflow Allocation

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	赵徐炎, 崔允贺, 钱清, 郭春, 申国伟. 基于虚拟扩展网络的虚拟机及路由路径联合部署[J]. 计算机工程, 2023, 49(8): 154-162.
[2]	张冠莹, 伊鹏, 李丹, 朱棣, 毛明. 面向大规模网络的服务功能链部署方法[J]. 计算机工程, 2023, 49(8): 122-129.
[3]	刘向举, 赵犇, 方贤进, 徐杨洋. SDN中基于过程优化的动态负载均衡策略[J]. 计算机工程, 2023, 49(8): 137-145.
[4]	梅晶, 戴龙宝, 童钊, 邓昕, 王嘉珂. 资源约束下基于Lyapunov优化的自适应卸载算法[J]. 计算机工程, 2023, 49(7): 34-46.
[5]	蔡丽娇, 秦进, 陈双. 远离旧区域和避免回路的强化探索方法[J]. 计算机工程, 2023, 49(7): 118-124.
[6]	李强, 仪晋辉, 杜婷婷, 王胜春. 移动边缘计算中基于A3C的依赖任务卸载与资源分配[J]. 计算机工程, 2023, 49(6): 42-52.
[7]	金志军, 王浩, 方宝富. 稀疏场景下基于理性好奇心的多智能体强化学习[J]. 计算机工程, 2023, 49(5): 302-309.
[8]	陈何雄, 罗宇薇, 韦云凯, 郭威, 杭菲璐, 何映军, 杨宁. 基于联邦学习的SDN异常流量协同检测技术[J]. 计算机工程, 2023, 49(3): 168-176.
[9]	饶东宁, 罗南岳. 基于多任务强化学习的堆垛机调度与库位推荐[J]. 计算机工程, 2023, 49(2): 279-287,295.
[10]	宋健, 王子磊. 基于值分解的多目标多智能体深度强化学习方法[J]. 计算机工程, 2023, 49(1): 31-40.
[11]	吴仍裕, 周强, 于海龙, 王亚沙. 基于深度强化学习的深圳市急救车调度算法[J]. 计算机工程, 2022, 48(9): 298-304.
[12]	赵寅甫, 冯正勇. 基于深度强化学习的机械臂控制快速训练方法[J]. 计算机工程, 2022, 48(8): 113-120.
[13]	厉子凡, 王浩, 方宝富. 一种基于多步竞争网络的多智能体协作方法[J]. 计算机工程, 2022, 48(5): 74-81.
[14]	王奎宇, 宋晓勤, 缪娟娟, 张昕婷, 雷磊. 基于SDN的高性能QoS保障低轨道卫星星间路由算法[J]. 计算机工程, 2022, 48(5): 185-190,199.
[15]	叶和元, 韩俐, 孙士民. SDN中基于蚁群优化的网络测量节点选择算法[J]. 计算机工程, 2022, 48(5): 191-199.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

一种智能多路径路由及子流分配协同算法

An Intelligent Cooperative Algorithm for Multi-Path Routing and Subflow Allocation

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献

相关文章 15

编辑推荐

Metrics

本文评价