作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2023, Vol. 49 ›› Issue (9): 199-207, 216. doi: 10.19678/j.issn.1000-3428.0066301

• 移动互联与通信技术 • 上一篇    下一篇

基于深度强化学习的无线网络智能路由算法

孔凌辉, 饶哲恒, 徐彦彦*, 潘少明   

  1. 武汉大学 测绘遥感信息工程国家重点实验室, 武汉 430079
  • 收稿日期:2022-11-21 出版日期:2023-09-15 发布日期:2023-09-14
  • 通讯作者: 徐彦彦
  • 作者简介:

    孔凌辉(1997—),男,硕士研究生,主研方向为异构网络通信

    饶哲恒,博士

    潘少明,副教授、博士

  • 基金资助:
    国家重点研发计划(2022YFB3903404); 国家自然科学基金(42271431); 国家自然科学基金(42271425)

Intelligent Routing Algorithm for Wireless Networks Based on Deep Reinforcement Learning

Linghui KONG, Zheheng RAO, Yanyan XU*, Shaoming PAN   

  1. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
  • Received:2022-11-21 Online:2023-09-15 Published:2023-09-14
  • Contact: Yanyan XU

摘要:

基于深度强化学习(DRL)的智能路由算法因兼具深度学习的感知能力和强化学习的决策能力,成为路由决策的重要发展方向。然而,现有基于深度强化学习的智能路由算法无法适应无线网络中动态变化的网络拓扑结构,难以在网络拓扑动态变化时做出恰当的路由决策。针对该问题,提出一种结合消息传递神经网络(MPNN)和DRL的智能路由算法MPNN-DQN。利用MPNN对不规则的网络拓扑进行学习,使其在网络拓扑动态变化时仍然能够做出有效的决策。设计基于k阶邻居信息聚合的逐跳路由生成方法,使得模型在保证决策效果的同时提升算法的可扩展性,能够更广泛地适用于中大型网络拓扑。实验结果表明,相比GCN、DRSIR、DQN等路由算法,该算法具有较优的平均时延、丢包率和网络吞吐量指标,在Germany、GBN和synth50这3种不同的网络场景下,该算法的吞吐量提升3.27%~23.03%,具有较强适应动态网络拓扑的能力。

关键词: 深度强化学习, 消息传递神经网络, 邻居信息聚合, 智能路由, 动态变化的网络拓扑

Abstract:

Intelligent routing algorithm based on Deep Reinforcement Learning(DRL) has become an important development direction for intelligent routing algorithms due to its combination of deep learning perception ability and reinforcement learning decision-making ability.However, existing DRL-based intelligent routing algorithms cannot adapt to the dynamically changing network topology in wireless networks, making it difficult to make appropriate routing decisions.To address this issue, this paper proposes an intelligent routing algorithm called MPNN-DQN, which combines the Message Passing Neural Network (MPNN) and DRL.MPNN-DQN uses MPNN to learn irregular network topology, enabling it to make effective decisions even when the network topology changes dynamically.Moreover, a hop-by-hop routing generation method based on k-order neighbor information aggregation is designed to improve the scalability of the algorithm while ensuring decision-making effectiveness; thus, the algorithm can be better applied to medium- to large-sized network topologies.Experimental results show that compared to routing algorithms such as GCN, DRSIR, and DQN, MPNN-DQN has superior average latency, packet loss rate, and network throughput indicators.In three different network scenarios, Germany, GBN, and synth50, the throughput of the proposed algorithm has been improved by 3.27%-23.03%, and has strong adaptability to dynamic network topologies.

Key words: Deep Reinforcement Learning(DRL), Message Passing Neural Network(MPNN), neighbor information aggregation, intelligent routing, dynamically changing network topology