计算机工程

• •    

基于函数逼近的强化学习优化FANET路由算法

  

  • 发布日期:2020-12-07

Reinforcement Learning Optimization FANET Routing Algorithm based on Function Approximation

  • Published:2020-12-07

摘要: 针对节点高速运动状态下,飞行自组网( Flying Ad hoc network, FANET)路由协议链路维护困难的问题,本文提出一 种基于强化学习的自适应优化链路状态路由算法。该路由算法基于强化学习中 Q 学习算法,通过感知动态环境下节点邻居数 量变化程度和业务负载程度,构建节点收发数据能力的价值函数,求解最优 HELLO 时隙,以提高节点链路发现与维护能力。 其次通过改进 Kanerva 编码算法的状态相似度机制,降低算法的复杂度,增加算法的稳定性。仿真实验表明:该路由算法能 有效的提升网络的吞吐量,降低路由维护的开销,且具有自学习的特性,适用于高动态环境下的无人机组网。

Abstract: In this paper, we propose an adaptive optimization link state routing algorithm based on Reinforcement Learning (RL) to solve the problem of difficulty in maintaining the link of the FANET routing protocol when the node is moving at a high speed. Firstly, we modify the interval of the broadcast HELLO message by using Q-learning in RL. And then, according to sense the degree of change under the number of neighbors and the degree of business load in a dynamic environment, we maximize the node's ability to send and consider receive data as a value function of the optimal HELLO interval. Furthermore, we improve the state similarity mechanism of the Kanerva coding algorithm to reduce the algorithm complexity while increasing the stability, it is more suitable to describe the FANET network environment with large state space and high complexity. Finally, simulation experiments show that the routing algorithm can effectively improve the throughput of the network, reduce the overhead of routine maintenance, and has the characteristics of self-learning and fast response. It is suitable for UAVs networking in a highly dynamic environment.