一种智能多路径路由及子流分配协同算法

doi:10.19678/j.issn.1000-3428.0059488

计算机工程 ›› 2021, Vol. 47 ›› Issue (9): 136-144,152. doi: 10.19678/j.issn.1000-3428.0059488

一种智能多路径路由及子流分配协同算法

徐啸^1,2, 顾玲丽^1,2, 陈建平^1,2, 傅启明^1,2,3, 陆悠^1,2,3

1. 苏州科技大学电子与信息工程学院, 江苏苏州 215009;
2. 苏州科技大学江苏省建筑智慧节能重点实验室, 江苏苏州 215009;
3. 苏州科技大学天平学院电子与信息工程学院, 江苏苏州 215011

收稿日期:2020-09-10 修回日期:2020-10-13 发布日期:2020-11-02
作者简介:徐啸(1997-),男,硕士研究生,主研方向为多路径传输控制、强化学习;顾玲丽,硕士研究生;陈建平,教授、博士;傅启明,讲师、博士;陆悠(通信作者),副教授、博士。
基金资助:
国家自然科学基金（61876217，61876121，61772357，61750110519）；江苏省重点研发计划（BE2017663）；江苏省高等学校自然科学研究面上项目（18KJB520045）。

An Intelligent Cooperative Algorithm for Multi-Path Routing and Subflow Allocation

XU Xiao^1,2, GU Lingli^1,2, CHEN Jianping^1,2, FU Qiming^1,2,3, LU You^1,2,3

1. School of Electronical and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, China;
2. Jiangsu Province Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, China;
3. School of Electronical and Information Engineering, Tianping College of Suzhou University of Science and Technology, Suzhou, Jiangsu 215011, China

Received:2020-09-10 Revised:2020-10-13 Published:2020-11-02

摘要/Abstract

摘要： 传统单一路径的传输机制难以满足当前以智慧城市为代表的新一代应用对时延、丢包率等网络性能的要求，而现有多路径传输机制在路由算法及子流分配等方面不能根据网络实时状态调整且互相缺乏协同。引入强化学习理论并结合软件定义网络，提出多路径路由及子流分配协同算法。基于Q-learning设计多路径路由算法，并从策略协同角度对其进行改进，实现路由与子流分配的相互协同。在此基础上，通过Q-value的回环消除方法保证路由准确性并提高算法收敛速度。实验结果表明，该算法在网络负载动态变化过程中能实时调整最佳的多路径路由及子流分配协同策略，提高了传输成功率。

关键词: 多路径路由, 强化学习, 协同决策, 流量调度, 软件定义网络

Abstract: The new generation of applications such as smart cities put forward higher requirements for network performance, including lower delay and packet loss rate.However, the traditional single-path transmission mechanism fail to meet the new requirements, while the existing multi-path transmission technology is limited by some problems in routing and subflow allocation, such as the difficulty in adjusting strategies according to the real-time network conditions and the lack of cooperation.An intelligent cooperative algorithm for multi-path routing and subflow allocation is proposed, which is based on the reinforcement learning theory and software defined network.For this algorithm, a multi-path routing method based on Q-learning is designed and improved from the perspective of strategy coordination to realize mutual coordination between routing and subflow allocation. On this basis, a loop elimination method based on Q-value is used to ensure the validity of routing decision and to improve the convergence speed of the algorithm.The experimental results show that the proposed algorithm can adjust the optimal cooperative strategy for multi-path routing and subflow allocation in real time according to the dynamically changing network loads.It improves the transmission success rate.

Key words: multi-path routing, reinforcement learning, collaborative decision-making, traffic scheduling, Software Defined Network(SDN)

中图分类号:

TP393

徐啸, 顾玲丽, 陈建平, 傅启明, 陆悠. 一种智能多路径路由及子流分配协同算法[J]. 计算机工程, 2021, 47(9): 136-144,152.

XU Xiao, GU Lingli, CHEN Jianping, FU Qiming, LU You. An Intelligent Cooperative Algorithm for Multi-Path Routing and Subflow Allocation[J]. Computer Engineering, 2021, 47(9): 136-144,152.

https://www.ecice06.com/CN/Y2021/V47/I9/136

图/表 15

20210917190838

20210917190842

20210917190845

20210917190849

20210917190853

20210917190857

20210917190901

20210917190905

20210917190911

20210917190915

20210917190919

20210917190923

20210917190927

20210917190931

20210917190935

参考文献

[1] 江卓, 吴茜, 李贺武, 等.互联网端到端多路径传输跨层优化研究综述[J].软件学报, 2019, 30(2):112-132. JIANG Z, WU Q, LI H W, et al.Survey on internet end-to-end multipath transfer research with cross-layer optimization[J].Journal of Software, 2019, 30(2):112-132.(in Chinese)
[2] 王竹, 袁青云, 郝凡凡, 等.基于链路容量的多路径拥塞控制算法[J].通信学报, 2020, 41(5):59-71. WANG Z, YUAN Q Y, HAO F F, et al.Multipath congestion control algorithm based on link capacity[J].Journal on Communications, 2020, 41(5):59-71.(in Chinese)
[3] 杨洋, 杨家海, 温皓森.基于时隙传输的数据中心路由算法设计[J].软件学报, 2018, 29(8):2485-2500. YANG Y, YANG J H, WEN H S.Routing algorithm design based on timeslot of transmission for data centers[J].Journal of Software, 2018, 29(8):2485-2500.(in Chinese)
[4] ALHARBI F, FEI Z.An SDN architecture for improving throughput of large flows using multipath TCP[C]//Proceedings of the 5th IEEE International Conference on Cyber Security and Cloud Computing.Washington D.C., USA:IEEE Press, 2018:1-6.
[5] DU P, PANG F, BRAUN T, et al.Traffic optimization in software defined naval network for satellite communications[C]//Proceedings of 2017 IEEE Military Communications Conference.Washington D.C., USA:IEEE Press, 2017:12-15.
[6] ZHAO P, ZHAO W, LIU Q.Multi-objective link-separation multipath selection using k max-min for software-defined MANETs[C]//Proceedings of the 15th International Conference on Mobile Ad-Hoc and Sensor Networks.Washington D.C., USA:IEEE Press, 2019:12-18.
[7] DUAN J, WANG Z, WU C.Responsive multipath TCP in SDN-based datacenters[C]//Proceedings of 2015 IEEE International Conference on Communications.Washington D.C., USA:IEEE Press, 2015:5296-5301.
[8] ZHAO J, LIU J, WANG H, et al.Multipath TCP for datacenters:from energy efficiency perspective[C]//Proceedings of 2017 IEEE Conference on Computer Communications.Washington D.C., USA:IEEE Press, 2017:1-9.
[9] NAM H, CALIN D, SCHULZRINNE H.Towards dynamic MPTCP path control using SDN[C]//Proceedings of 2016 IEEE NetSoft Conference.Washington D.C., USA:IEEE Press, 2016:286-294.
[10] TARIQ S, BASSIOUNI M.QAMO-SDN:QoS aware multipath TCP for software defined optical networks[C]//Proceedings of the 12th Annual IEEE Consumer Communications and Networking Conference.Washington D.C., USA:IEEE Press, 2015:485-491.
[11] DHALIWAL H K, LUNG C H.Load balancing using ECMP in multi-stage clos topology in a datacenter[C]//Proceedings of 2018 IEEE Conference on Dependable and Secure Computing.Washington D.C., USA:IEEE Press, 2018:1-7.
[12] RHAMDANI F, SUWASTIKA N A, NUGROHO M A.Equal-cost multipath routing in data center network based on software defined network[C]//Proceedings of the 6th International Conference on Information and Communication Technology.Washington D.C., USA:IEEE Press, 2018:222-226.
[13] 陈章迎, 朱尚明.基于重叠网络的多路径路由设计和实现[J].通信学报, 2018, 39(S1):200-205. CHEN Z Y, ZHU S M.Designing and implementation of multi-path QoS routing algorithm based on overlay network[J].Journal on Communications, 2018, 39(S1):200-205.(in Chinese)
[14] 高先明, 王宝生, 邓文平.SDRS:集中与分布控制相结合的弹性多路径路由机制[J].计算机学报, 2018, 41(9):1976-1989. GAO X M, WANG B S, DENG W P.SDRS:elastic multi-path routing mechanism with integration of centralization control and distribution control[J].Chinese Journal of Computers, 2018, 41(9):1976-1989.(in Chinese)
[15] YE J L, CHEN C, CHU Y H.A weighted ECMP load balancing scheme for data centers using P4 switches[C]//Proceedings of 2018 IEEE International Conference on Cloud Networking.Washington D.C., USA:IEEE Press, 2018:1-4.
[16] MARCO C, GABOR R, MICHAEL S.Oblivious routing in IP networks[J].IEEE/ACM Transactions on Networking, 2018:1-14.
[17] 黄建洋, 兰巨龙, 胡宇翔, 等.一种基于分段路由的多路径流传输机制[J].电子学报, 2018, 46(6):1488-1495. HUANG J Y, LAN J L, HU Y X, et al.A segment routing based multipath flow transmission mechanism[J].Acta Electronica Sinica, 2018, 46(6):1488-1495.(in Chinese)
[18] 蔡岳平, 王昌平.软件定义数据中心网络混合路由机制[J].通信学报, 2016, 37(4):44-52. CAI Y P, WANG C P.Software defined data center network with hybrid routing[J].Journal on Communications, 2016, 37(4):44-52.(in Chinese)
[19] XU J W, WU M Q, HOU X L.A traffic scheduling scheme for data center networks based on SDN[C]//Proceedings of the 5th International Conference on Computer and Communications.Washington D.C., USA:IEEE Press, 2019:1417-1422.
[20] HUANG X, CHENG S, CAO K, et al.A survey of deployment solutions and optimization strategies for hybrid SDN networks[J].IEEE Communications Surveys & Tutorials, 2019, 21(2):1483-1507.
[21] 潘成胜, 刘勇, 石怀峰, 等.SDN架构下的空间信息网络业务识别技术[J].计算机工程, 2019, 45(4):18-24. PAN C S, LIU Y, SHI H F, et al.Spatial information network business identification technology under SDN architecture[J].Computer Engineering, 2019, 45(4):18-24.(in Chinese)
[22] 彭大芹, 赖香武, 刘艳林.基于SDN的胖树数据中心网络多路径路由算法[J].计算机工程, 2018, 44(4):41-45, 65. PENG D Q, LAI X W, LIU Y L.Multi-path routing algorithm for fat-tree data center network based on SDN[J].Computer Engineering, 2018, 44(4):41-45, 65.(in Chinese)
[23] MAKSIC N.Two-phase load balancing for data center networks using OpenFlow[C]//Proceedings of the 25th Telecommunication Forum.Washington D.C., USA:IEEE Press, 2017:1-4.
[24] NARAYAN D G, MUDENAGUDI U.A joint approach to routing metrics and application-aware rate adaptation in multi-radio wireless mesh networks[J].Journal of High Speed Networks, 2017, 23(2):93-108.
[25] 刘辰屹, 徐明伟, 耿男, 等.基于机器学习的智能路由算法综述[J].计算机研究与发展, 2020, 57(4):671-687. LIU C Y, XU M W, GENG N, et al.A survey on machine learning based routing algorithms[J].Journal of Computer Research and Development, 2020, 57(4):671-687.(in Chinese)
[26] LUONG N C, HOANG D T, GONG S, et al.Applications of deep reinforcement learning in communications and networking:a survey[J].IEEE Communications Surveys and Tutorials, 2019, 21(4):3133-3174.

选择文件类型/文献管理软件名称

选择包含的内容

一种智能多路径路由及子流分配协同算法

An Intelligent Cooperative Algorithm for Multi-Path Routing and Subflow Allocation

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	崔允贺, 赵建朋, 杨鸿珍, 李显超. 基于曲线决策融合的SDN饱和攻击检测方法[J]. 计算机工程, 2024, 50(8): 123-132.
[2]	石琼, 段辉, 师智斌. 基于深度强化学习的可信任务卸载方案[J]. 计算机工程, 2024, 50(8): 142-152.
[3]	钱清, 龙永, 蒋忠远, 段春红, 王宏. 基于深度强化学习的自适应图像隐写算法[J]. 计算机工程, 2024, 50(8): 319-327.
[4]	高家豪, 胡创业, 丁男, 刘战东. 智能网联汽车中联合驾驶风格的交通流数据有效性分析[J]. 计算机工程, 2024, 50(6): 367-376.
[5]	孙文洁, 李宗民, 孙浩淼. 基于图神经网络的多智能体强化学习值函数分解方法[J]. 计算机工程, 2024, 50(5): 62-70.
[6]	傅明建, 郭福强. 基于深度强化学习的无信号灯路口决策研究[J]. 计算机工程, 2024, 50(5): 91-99.
[7]	张斯力, 李梓健, 蔡瑞初, 郝志峰, 闫玉光. 基于因果机制约束的强化推荐系统[J]. 计算机工程, 2024, 50(5): 279-290.
[8]	冯雄波, 黄于欣, 赖华, 高玉梦. 基于多策略强化学习的低资源跨语言摘要方法研究[J]. 计算机工程, 2024, 50(2): 68-77.
[9]	杜海军, 余粟. 基于时空图注意力网络的服务机器人动态避障[J]. 计算机工程, 2024, 50(2): 105-112.
[10]	郭雷, 荆山, 魏亮, 赵川. 基于软件定义网络的Crossfire攻击防御方法[J]. 计算机工程, 2024, 50(10): 216-227.
[11]	张俊娜, 韩超臣, 陈家伟, 赵晓焱, 袁培燕. 一种联合边缘服务器部署与服务放置的方法[J]. 计算机工程, 2024, 50(10): 266-280.
[12]	孔月萍, 杨世海, 段梅梅, 丁泽诚, 方凯杰. 基于混合动作强化学习的电动汽车聚合商决策优化算法[J]. 计算机工程, 2024, 50(10): 418-428.
[13]	蔡梓越, 谭北海, 余荣, 黄旭民, 王思明. 面向6G物联网设备协同的区块链动态分片[J]. 计算机工程, 2024, 50(1): 50-59.
[14]	张晓天, 王雅文, 谢志庆, 金大海, 宫云战. 面向类集成测试序列确定的强化学习方法[J]. 计算机工程, 2024, 50(1): 68-78.
[15]	王少桐, 况立群, 韩慧妍, 熊风光, 薛红新. 基于优势后见经验回放的强化学习导航方法[J]. 计算机工程, 2024, 50(1): 313-319.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

一种智能多路径路由及子流分配协同算法

An Intelligent Cooperative Algorithm for Multi-Path Routing and Subflow Allocation

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献

相关文章 15

编辑推荐

Metrics

本文评价