基于Q学习的无人机辅助WSN数据采集轨迹规划

doi:10.19678/j.issn.1000-3428.0057207

摘要/Abstract

摘要： 针对无人机辅助采集无线传感器网络数据时各节点数据产生速率随机和汇聚节点状态不一致的场景，提出基于Q学习的非连续无人机轨迹规划算法Q-TDUD，以提高无人机能量效率和数据采集效率。基于各节点在周期内数据产生速率的随机性建立汇聚节点的汇聚延时模型，应用强化学习中的Q学习算法将各汇聚节点的延迟时间和采集链路的上行传输速率归一化到奖励函数中，通过迭代计算得到最佳非连续无人机飞行轨迹。实验结果表明，与TSP-continues、TSP、NJS-continues和NJS算法相比，Q-TDUD算法能够缩短无人机的任务完成时间，提高无人机能效和数据采集效率。

关键词: 无线传感器网络, 数据采集, 无人机, 轨迹规划, Q学习算法

Abstract: In some scenarios where Unmanned Aerial Vehicle(UAV) assists in Wireless Sensor Network(WSN) data collection,the data generation rate of each node is random and the states of sink node are inconsistent.To address the problem,this paper proposes a Q-learning-based algorithm called Q-TDUD for discontinuous UAV trajectory planning,which can improve the energy efficiency of UAV and data collection efficiency.Based on the randomness of the data generation rate of each node in the cycle,the aggregation delay model of the sink node is established.The Q-learning algorithm in reinforcement learning is used to normalize the delay time of each sink node and the uplink transmission rate of the collection link into the reward function,and the optimal discontinuous flight trajectory of the UAV is obtained through iterative calculation.Experimental results show that,compared with TSP-continues,TSP,NJS-continues and NJS algorithms,the proposed Q-TDUD algorithm can reduce the task completion time of UAV,and improve the energy efficiency and data collection efficiency of UAV.

Key words: Wireless Sensor Network(WSN), data collection, Unmanned Aerial Vehicle(UAV), trajectory planning, Q-learning algorithm

中图分类号:

TP393

蒋宝庆, 陈宏滨. 基于Q学习的无人机辅助WSN数据采集轨迹规划[J]. 计算机工程, 2021, 47(4): 127-134,165.

JIANG Baoqing, CHEN Hongbin. Trajectory Planning for Unmanned Aerial Vehicle Assisted WSN Data Collection Based on Q-Learning[J]. Computer Engineering, 2021, 47(4): 127-134,165.

http://www.ecice06.com/CN/Y2021/V47/I4/127

图/表 9

20210425165734

20210425165737

20210425165740

20210425165743

20210425165747

20210425165751

20210425165754

20210425165757

20210425165759

参考文献

[1] AKYILDIZ I F,SU W,SANKARASUBRAMANIAM Y,et al.Wireless sensor networks:a survey[J].Computer Networks,2002,38(4):393-422.
[2] GAYATHRI V,ELYES B A B,IBNKAHLA M.Cognition in wireless sensor networks:a perspective[J].IEEE Sensors Journal,2011,11(3):582-592.
[3] WU Qingqing,LIU Liang,ZHANG Rui.Fundamental tradeoffs in communication and trajectory design for UAV enabled wireless network[J].IEEE Wireless Communications,2019,26(1):34-36.
[4] FRANCESCO M,DAS S K,ANASTASI G.Data collection in wireless sensor networks with mobile elements:a survey[J].ACM Transactions on Sensor Networks,2011,8(1):1-31.
[5] ZENG Yong,ZHANG Rui.Energy efficient UAV communication with trajectory optimization[J].IEEE Transactions on Wireless Communications,2017,16(6):3747-3760.
[6] YANG Dingcheng,WU Qingqing,ZENG Yong,et al.Energy trade off in ground to UAV communication via trajectory design[J].IEEE Transactions on Vehicular Technology,2018,67(7):6721-6726.
[7] ZHAN Cheng,ZENG Yong,ZHANG Rui.Energy efficient data collection in UAV enabled wireless sensor network[J].IEEE Wireless Communications Letters,2018,7(3):328-331.
[8] XU Jie,ZENG Yong,ZHANG Rui.UAV-enabled wireless power transfer:trajectory design and energy optimization[J].IEEE Transactions on Wireless Communications,2018,17(8):5029-5106.
[9] ZENG Yong,XU Xiaoli,ZHANG Rui.Trajectory design for completion time minimization in UAV enabled multicasting[J].IEEE Transactions on Wireless Communications,2018,17(4):2223-2246.
[10] YANG Tianyu,HU Yulin,YUAN Xiaopeng,et al.Genetic algorithm based UAV trajectory design in wireless power transfer system[C]//Proceedings of IEEE Wireless Commu-nications and Networking Conference.Washington D.C.,USA:IEEE Press,2019:1-6.
[11] ZAPPONE A,RENZO M D,DEBBAH M.Wireless networks design in the era of deep learning:model-based,Al-based,or both?[J].IEEE Transactions on Communications,2019,67(10):7731-7736.
[12] HU Jingzhi,ZHANG Hongliang,SONG Lingyang.Reinforcement learning for decentralized trajectory design in cellular UAV networks with sense and send protocol[J].IEEE Internet of Things Journal,2019,6(4):6177-6189.
[13] YIN Sixing,ZHAO Shuo,ZHAO Yifei,et al.Intelligent trajectory design in UAV-aided communications with reinforcement learning[J].IEEE Transactions on Vehicular Technology,2019,68(4):8227-8231.
[14] DAI Kui,QIU Guangyu,HU Shouren.A discrete-Markov-model-based neural network method for reliability design[J].Computer Engineering & Science,1999,21(3):10-14.(in Chinese)戴葵,仇广煜,胡守仁.一种基于离散马尔科夫模型的神经网络可靠性设计方法[J].计算机工程与科学,1999,21(3):10-14.
[15] LIU Jianwei,GAO Feng,LUO Xionglin.Survey of deep reinforcement learning based on value function and policy gradient[J].Chinese Journal of Computers,2019,42(6):1406-1438.(in Chinese)刘建伟,高峰,罗雄麟.基于值函数和策略梯度的深度强化学习综述[J].计算机学报,2019,42(6):1406-1438.
[16] PETRIU E M,GEORGANAS N D,PETRIU D C,et al.Sensor-based information appliances[J].IEEE Instrumenta-tion and Measurement Magazine,2000,3(4):31-35.
[17] AKYLDIZ I F,KASIMOGLU I H.Wireless sensor and actuator networks:research challenges[J].Ad Hoc Networks Journal Elsevier,2004,2(4):351-367.
[18] MELODIA T,POMPILI D,GUNGOR V.A distributed coordination framework for wireless sensor and actuator networks[C]//Proceedings of the 6th ACM International Symposium on Mobile Ad Hoc Networking and Computing.New York,USA:ACM Press,2005:341-352.
[19] OZAKI K,WATANABE K,ITAYA S.A fault tolerant model of wireless sensor actuator systems[C]//Proceedings of the 20th International Conference on Advanced Infor-mation Networking and Applications.Washington D.C.,USA:IEEE Press,2006:303-307.
[20] LI Shuanghu,WANG Tiehong.New validity index for determining the number of clusters in K-means clustering[J].Journal of the Hebei Academy of Sciences,2003,20(4):199-202.(in Chinese)李双虎,王铁洪.K-means聚类分析算法中一个新的确定聚类个数有效性的指标[J].河北省科学院学报,2003,20(4):199-202.
[21] CHEN QI,CHEN Hongbin.Research on optimization of data collection efficiency assisted by mobile collectors in wireless sensor networks[J].Application Research of Computers,2019,37(11):1-5.(in Chinese)陈琪,陈宏滨.无线传感器网络中移动节点辅助的数据采集效率优化研究[J].计算机应用研究,2019,37(11):1-5.
[22] ZHAO Miao,YANG Yuanyuan.Bounded relay hop mobile data gathering in wireless sensor networks[J].IEEE Transactions on Computers,2012,61(2):265-277.
[23] ZHANG Rubo,YANG Guangming,GU Guochang.Q-learning and its application in local path planning of intelligent robots[J].Journal of Computer Research & Development,1999,36(12):1430-1436.(in Chinese)张汝波,杨广铭,顾国昌.Q-学习及其在智能机器人局部路径规划中的应用研究[J].计算机研究与发展,1999,36(12):1430-1436.
[24] WANG Chengliang,YAN Junhui.Path planning for UAV to collect sensor data in large-scale WSNs[J].Transaction of Beijing Institute of Technology,2015,35(10):58-63.(in Chinese)汪成亮,严君辉.大规模无线传感网络数据收集的无人机路径规划[J].北京理工大学学报,2015,35(10):58-63.
[25] XIA Liangsheng,YAN Weisheng.Study on mobile robot motion planning based on grid method[J].Computer Simulation,2012,29(12):239-243.(in Chinese)夏梁盛,严卫生.基于栅格法的移动机器人运动规划研究[J].计算机仿真,2012,29(12):239-243.
[26] CHEN Hongbin,LI Xueyan,ZHAO Feng.A reinforcement learning based sleep scheduling algorithm for desired area coverage in solar powered wireless sensor networks[J].IEEE Sensors Journal,2016,16(8):2763-2774.
[27] LI Yong,DUAN Zhengcheng.A new ant system for TSPs[J].Computer Engineering and Applications,2003,39(17):106-109.(in Chinese)李勇,段正澄.动态蚁群算法求解TSP问题[J].计算机工程与应用,2003,39(17):106-109.
[28] KANG H L,LEE B,KIM K.Path planning algorithm using the particle swarm optimization and the improved Dijkstra algorithm[C]//Proceedings of IEEE Pacific Asia Workshop on Computational Intelligence and Industrial Application.Washington D.C.,USA:IEEE Press,2008:1002-1004.

选择文件类型/文献管理软件名称

选择包含的内容