基于启发式Q学习的FANET可信路由算法

doi:10.19678/j.issn.1000-3428.0062147

摘要/Abstract

摘要： 无人机自组织网络（FANET）是实现无人机自主集群的关键技术，其通过各无人机节点来完成协同通信。但节点的高机动性、网络结构的开放性造成FANET拓扑变化频繁，容易遭受恶意攻击。为此，提出一种基于启发式Q学习的可信路由算法HQTR。将FANET中的路由选择问题映射为有限马尔科夫决策过程，针对路由层面的黑洞攻击与泛洪攻击，引入数据包转发率与路由请求发送速率，通过模糊推理计算节点的信任值，同时考虑节点的邻居关系，提出一种模糊动态信任奖励机制。结合单跳链路状况设计启发式函数，采用改进的ε-贪婪策略来平衡利用-探索过程，引导当前节点选择最优可信下一跳节点。仿真结果表明，相对AOMDV、TEAOMDV与ESRQ算法，HQTR算法能够有效应对黑洞攻击与RREQ泛洪攻击，降低节点高速运动与网络规模变化所造成的影响，提高数据包投递率与吞吐量，减少路由开销与平均端到端时延。

关键词: 无人机自组织网络, 路由攻击, 信任模型, Q学习, 启发式函数

Abstract: The Flying Ad hoc Network(FANET) is the key technology for realizing UAV autonomous clusters.It completes cooperative communication through UAV nodes.However, owing to the high mobility of nodes and the openness of the network structure, FANET topology changes frequently and is vulnerable to malicious attacks.Therefore, a trusted routing algorithm HQTR based on heuristic Q-learning is proposed.The routing problem in the FANET is mapped to a finite Markov decision process.To mitigate black hole and flooding attacks at the routing level, a packet forwarding rate and a routing request sending rate are introduced.The trust value of the node is calculated via fuzzy reasoning, and considering the neighbor relationship of the node, a fuzzy dynamic trust reward mechanism is proposed.Combined with the single hop link condition, a heuristic function is designed, and an improved greedy strategy is used to balance the utilization-exploration process to facilitate the current node in selecting the best trusted next hop node.Simulation results show that compared with AOMDV, TEAOMDV, and ESRQ algorithms, the HQTR algorithm can effectively address black hole and RREQ flooding attacks, reduce the effects of high-speed node movements and network scale changes, improve the packet delivery rate and throughput, and reduce the routing overhead and average end-to-end delay.

Key words: Flying Ad hoc Network(FANET), routing attack, trusted model, Q-learning, heuristic function

中图分类号:

TP393

赵蓓英, 姬伟峰, 翁江, 吴玄, 李映岐. 基于启发式Q学习的FANET可信路由算法[J]. 计算机工程, 2022, 48(5): 162-169.

ZHAO Beiying, JI Weifeng, WENG Jiang, WU Xuan, LI Yingqi. Trusted Routing Algorithm Based on Heuristic Q-Learning for FANET[J]. Computer Engineering, 2022, 48(5): 162-169.

https://www.ecice06.com/CN/Y2022/V48/I5/162

图/表 9

20220805180908

20220805180912

20220805180916

20220805180919

20220805180924

20220805180928

20220805180932

20220805180936

20220805180939

参考文献

[1] CHRIKI A, TOUATI H, SNOUSSI H, et al.FANET:communication, mobility models and security issues[J].Computer Networks, 2019, 163:106877.
[2] 游静, 董超, 吴启晖.大规模无人机自组网分层体系架构研究综述[J].计算机科学, 2020, 47(9):226-231. YOU J, DONG C, WU Q H.Survey of layered architecture in large-scale FANETs[J].Computer Science, 2020, 47(9):226-231.(in Chinese)
[3] SRIVASTAVA A, PRAKASH J.Future FANET with application and enabling techniques:anatomization and sustainability issues[J].Computer Science Review, 2021, 39:100359.
[4] JEAN-AIMÉ M, MOHAMED-SLIM B M, NICOLAS L.Survey on UAANET routing protocols and network security challenges[EB/OL].[2021-06-05].https://hal-enac.archives-ouvertes.fr/hal-01465993/document.
[5] NAZIB R A, MOH S.Reinforcement learning-based routing protocols for vehicular ad hoc networks:a comparative survey[J].IEEE Access, 2021, 9:27552-27587.
[6] CHETTIBI S, CHIKHI S.A survey of reinforcement learning based routing protocols for mobile ad-hoc networks[M].Berlin, Germany:Springer, 2011.
[7] JUNG W S, YIM J, KO Y B.QGeo:Q-learning-based geographic ad hoc routing protocol for unmanned robotic networks[J].IEEE Communications Letters, 2017, 21(10):2258-2261.
[8] LIU J M, WANG Q, HE C T, et al.QMR:Q-learning based multi-objective optimization routing protocol for flying ad hoc networks[J].Computer Communications, 2020, 150:304-316.
[9] SINGH K, VERMA A K.TBCS:a trust based clustering scheme for secure communication in flying ad-hoc networks[J].Wireless Personal Communications, 2020, 114(4):3173-3196.
[10] WU C, OHZAHATA S, KATO T.Flexible, portable, and practicable solution for routing in VANETs:a fuzzy constraint Q-learning approach[J].IEEE Transactions on Vehicular Technology, 2013, 62(9):4251-4263.
[11] ALSHEHRI A, BADAWY A H A, HUANG H.FQ-AGO:fuzzy logic Q-learning based asymmetric link aware and geographic opportunistic routing scheme for MANETs[J].Electronics, 2020, 9(4):576.
[12] BIANCHI R A C, RIBEIRO C H C, COSTA A H R.Accelerating autonomous learning by using heuristic selection of actions[J].Journal of Heuristics, 2008, 14(2):135-168.
[13] BIANCHI R A C, MARTINS M F, RIBEIRO C H C, et al.Heuristically-accelerated multiagent reinforcement learning[J].IEEE Transactions on Cybernetics, 2014, 44(2):252-265.
[14] YANG X Y, ZHANG W L, LU H M, et al.V2V routing in VANET based on heuristic Q-learning[J].International Journal of Computers Communications & Control, 2020, 15(5):12-23.
[15] KHANNA N, SACHDEVA M.Study of trust-based mechanism and its component model in MANET:current research state, issues, and future recommendation[J].International Journal of Communication Systems, 2019, 32(12):e4012.
[16] XIA H, YU J, TIAN C L, et al.Light-weight trust-enhanced on-demand multi-path routing in mobile ad hoc networks[J].Journal of Network and Computer Applications, 2016, 62:112-127.
[17] LIU G S, WANG X, LI X H, et al.ESRQ:an efficient secure routing method in wireless sensor networks based on Q-learning[C]//Proceedings of the 17th IEEE International Conference on Trust, Security and Privacy in Computing and Communications.Washington D.C., USA:IEEE Press, 2018:149-155.
[18] DAI C H, XIAO X Y, DING Y Z, et al.Learning based security for VANET with blockchain[C]//Proceedings of IEEE International Conference on Communication Systems.Washington D.C., USA:IEEE Press, 2018:210-215.
[19] ZHANG D J, YU F R, YANG R Z, et al.A deep reinforcement learning-based trust management scheme for software-defined vehicular networks[C]//Proceedings of the 8th ACM Symposium on Design and Analysis of Intelligent Vehicular Networks and Applications.New York, USA:ACM Press, 2018:1-7.
[20] ZHANG D J, YU F R, YANG R Z.A machine learning approach for software-defined vehicular ad hoc networks with trust management[C]//Proceedings of IEEE Global Communications Conference.Washington D.C., USA:IEEE Press, 2018:1-6.
[21] MAAKAR S K, SINGH Y, SINGH R.Flying ad hoc network:a newest research area for ad hoc networks[C]//Proceedings of the 2nd International Conference on Intelligent Communication and Computational Techniques.Washington D.C., USA:IEEE Press, 2019:298-302.
[22] YU Y J, QIN Y, GONG H C.A fuzzy Q-learning algorithm for storage optimization in islanding microgrid[J].Journal of Electrical Engineering & Technology, 2021, 16(5):2343-2353.
[23] SINGH K, VERMA A K.A fuzzy-based trust model for flying ad hoc networks(FANETs)[J].International Journal of Communication Systems, 2018, 31:23-47.

选择文件类型/文献管理软件名称

选择包含的内容