SAG-MEC网络下支持WPT的无人机动态任务卸载与资源分配

doi:10.19678/j.issn.1000-3428.0070030

摘要/Abstract

摘要： 针对偏远地区蜂窝网络覆盖不足且物联网（IoT）设备能量和计算能力低，从而无法满足大量延迟敏感型任务卸载和计算需求的问题，考虑将空天地一体化网络(Space-Air-Ground Integrated Network, SAGIN)和移动边缘技术(mobile edge computing，MEC)相结合，并提出了一种支持无线供电(Wireless Power Transmission,WPT)技术的无人机辅助IoT设备的动态任务卸载和资源分配策略，其中无人机负责收集IoT设备产生的计算密集型任务，然后采用部分卸载模式将这些任务根据当前状态进行本地计算或动态卸载给基站和LEO卫星进一步处理。由于动态的异构网络和长期排队延迟与短期决策的耦合性，因此在排队延迟的约束下提出了一种基于Lyapunov优化的双延迟深度确定性策略梯度（Twin Delayed Deep Deterministic Policy Gradient,TD3PG）算法，该算法通过优化无人机动态关联、任务分配、计算资源分配和带宽分配来协调无人机学习最优卸载策略和资源分配。仿真结果表明，所提出的动态方案与其他对比方案相比能够有效降低无人机网络的能耗、网络积压总和及平均排队延迟，其次在不同学习率组合下，TD3PG算法相对于DDPG算法和DDQN算法的奖励分别提高了13.6%、24%和20.4%、17.9%。

Abstract: Addressing the problem of insufficient cellular network coverage and low energy and computing power of Internet of Things (IoT) devices in remote areas, which cannot meet the large number of delay-sensitive task offloading and computing, Considering the combination of Space-Air-Ground Integrated Network (SAGIN) and mobile edge computing (MEC), It also proposes a strategy for dynamic task offloading and resource allocation of UAV-assisted IoT devices that support Wireless Power Transmission (WPT) technology, in which UAVs are responsible for collecting compute-intensive tasks generated by IoT devices. These tasks are locally calculated or dynamically unloaded to the base station and LEO satellite for further processing by using the partial unloading mode according to the current state. Due to the dynamic heterogeneous network and the coupling of long-term queuing delay and short-term decision making, Therefore, under the constraint of queuing delay, a Twin Delayed Deep Deterministic Policy Gradient (TD3PG) algorithm based on Lyapunov optimization is proposed. The algorithm coordinates UAVs learning optimal unloading strategy and resource allocation by optimizing UAVs dynamic association, task allocation, computing resource allocation and bandwidth allocation. The simulation results show that compared with other schemes, the proposed dynamic scheme can effectively reduce the energy consumption, network backlog sum and average queue delay of UAV network. Secondly, under different learning rate combinations, the reward of TD3PG algorithm is increased by 13.6%, 24% and 20.4% and 17.9%, respectively, compared with DDPG algorithm and DDQN algorithm.

王怡, 覃团发, 韦睿, 黄金宝. SAG-MEC网络下支持WPT的无人机动态任务卸载与资源分配[J]. 计算机工程, doi: 10.19678/j.issn.1000-3428.0070030.

Wang Yi, QIN Tuanfa, Wei Rui, Huang JinBao. Dynamic task unloading and resource allocation of UAVs supported by WPT in SAG-MEC network[J]. Computer Engineering, doi: 10.19678/j.issn.1000-3428.0070030.

参考文献

[1] Bedi G, Venayagamoorthy G K, Singh R, et al. Review of Internet of Things (IoT) in electric power and energy systems[J]. IEEE Internet of Things Journal, 2018,5(2): 847-870.
[2] Wang H C, Wang J L, Ding G R, et al. Robust spectrum sharing in air-ground integrated networks: Opportunities and challenges[J]. IEEE Wireless Communications, 2020, 27(3): 148-155.
[3] SAHOO B P S, PUTHAL D, SHARMA P K. Toward advanced UAV communications: Properties, research challenges, and future potential[J]. IEEE Internet of Things Magazine, 2022, 5(1): 154-159.
[4] Li Z D, Wang, Y, Liu M, et al. Energy efficient resource allocation for UAV-Assisted Space-Air-Ground Internet of remote things networks[J]. IEEE Access, 2019, 7：145348–14536.
[5] Ai B, Molisch A F, Rupp M, et al. 5G key technologies for smart railways[J].Proceedings of the IEEE, 2020, 108(6):856-893.
[6] Jia Z Y, Sheng M, Li J D, et al. LEO Satellite-Assisted UAV: joint trajectory and data collection for internet of remote things in 6G aerial access networks[J].IEEE Internet of Things Journal, 2020, 8(12):9814–9826.
[7] 刘亮,毛武平,李汶蔚等. 空天地一体化边缘计算网络中基于博弈论的任务卸载策略 [J/OL]. 计算机工程 ,2024,1-11. DOI:10.19678/j.issn.1000-3428.0069161 Liu L，Mao W P, Li W W, et al. Task Offloading Strategy Based on Game Theory in the Space-Air-Ground Integrated Edge Computing Networks[J/OL]. Computer Engineering.2024,1-11. DOI:10.19678/j.issn.1000-3428.0069161
[8] Tang Q Q, Li B. Overview of mobile edge computing in space-air-ground integrated network[J]. Radio Communications Technology, 2021, 47(1): 25–35. doi: 10. 3969/j.issn.1003-3114.2021.01.004.
[9] Cui H X, Zhang J, Geng Y H, et al. Space-air-ground integrated network (SAGIN) for 6G: Requirements, architecture and challenges[J].China Communications, 2022,19(2):90-108. doi: 10.23919/JCC.2022.02.008.
[10] Ji B F, Wang Y A, Song K, et al. A survey of computational intelligence for 6G: Key technologies,applications and trends[J]. IEEE Transactions on Industrial Informatics, 2021, 17(10): 7145–7154.
[11] Liu J J, Shi Y P, Fadlullah Z M, et al. Space-air-ground integrated network: A survey[J]. IEEE Communications Surveys & Tutorials, 2018, 20(4): 2714–2741.
[12] Mao B , Tang F , Kawamoto Y ,et al.Optimizing Computation Offloading in Satellite-UAV-Served 6G IoT: A Deep Learning Approach[J].IEEE Network: The Magazine of Computer Communications, 2021, 35 (4):102-108.
[13] 李晓青,贺占权,周卫彤.支持 MEC 的天地一体化网络下任务卸载和资源分配[J].计算机应用与软件, 2023, 40(2):130-137.Li X Q, He Z Q, Zhou W T. Task offloading and resource allocation for the MEC enabled integrated satellite -terrestrial network[J].Computer applications and software,2023, 40(2):130-137.
[14] Liu J, Zhao X, Qin P, et al. Joint dynamic task offloading and resource scheduling for WPT enabled space-air-ground power Internet of Things[J] IEEE Transactions on Network Science and Engineering. 2022, 9(2): 660–677.
[15] Tang Q Q, Fei Z S, Li B, et al.Computation offloading in LEO satellite networks with hybrid cloud and edge computing[J]. IEEE Internet of Things Journal, 2021, 8(11):9164-9176.
[16] Zhang S B, Liu A J, Han C, et al. Multiagent reinforcement learning-based orbital edge offloading in SAGIN supporting internet of remote things[J]. IEEE Internet of Things Journal, 2023,10(23):20472-20483.
[17] Chen Y L, Ai B, Niu Y, et al. Energy-constrained computation offloading in Space-Air-Ground Integrated Networks using distributionally robust optimization[J]. IEEE Transactions on Vehicular Technology,2021, 70(11):12113-12125
[18] Zhou C H, Wu W, He H L, et al. Deep reinforcement learning for Delay-Oriented IoT task scheduling in SAGIN[J]. IEEE Transactions on Wireless Communications, 2021, 20(2):911-925.
[19] Zhou C H, Wu W, He H L, et al. Delay-Aware IoT task scheduling in Space-Air-Ground Integrated Network[C]//GLOBECOM 2019 - 2019 IEEE Global Communications Conference.IEEE, 2019:1-9.
[20] 李斌,刘文帅,费泽松.面向空天地异构网络的边缘计算部分任务卸载策略[J].电子与信息学报, 2022, 44(9):8. Li B, Liu W S, Fei Z S.Partial Computation Offloading for Mobile Edge Computing in Space-Air-Ground Integrated Network[J]. Journal of Electronics and Information Technology.2022, 44(9):8.
[21] Liu Y D, Li B, Yang Y X, et al.Joint delay and completion rate optimization for dependent task offloading in Space-Air-Ground Integrated Network[C]//2023 9th International Conference on Computer and Communications (ICCC), 2023:327-331.
[22] Zheng K C, Jiang G D, Liu X Y, et al. DRL-based offloading for computation delay minimization in wireless powered multi-access edge computing[J]. IEEE Trans. Commun. 2023, 71(3), 1755–1770.
[23] Wang Y H, Li B, He J H, et al. Joint latency-oriented, energy consumption and carbon emission for a Space–Air–Ground Integrated Network with newly designed power technology[J]. Electronics. 2023,12(17):3537.
[24] Qin P, Fu Y, Xie Y B, et al. Multi-Agent learning-based optimal task offloading and UAV trajectory planning for AGIN-power IoT[J]. IEEE Transactions on Communications, 2023, 71(7):4005-4017.
[25] Little J D C. A proof for the queuing formula: L= λ W[J]. Operations research, 1961, 9(3): 383-387
[26] Zhang Z B, Li X H, An J P, et al. Model-Free attitude control of spacecraft based on PID-Guide TD3 algorithm[J].International Journal of Aerospace Engineering, 2020,4: 8874619.
[27] Kumar A S, Zhao L, Fernando X. Task offloading and resource allocation in Vehicular networks: A lyapunov-based deep reinforcement learning approach[J].IEEE Transactions on Vehicular Technology, 2023.7(10):13360-13373.
[28] Ma G F, Wang X W, Hu M J, et al. DRL-based computation offloading with queue stability for vehicular-cloud-assisted mobile edge computing systems[J]. IEEE Transactions on Intelligent Vehicles, 2023, 8(4):2797-2809.
[29] Wu Q Q. Zeng Y, Zhang R. Joint trajectory and communication design for Multi-UAV enabled wireless networks[J]. IEEE Transactions on Wireless Communications, 2018,17(3):219-221.
[30] Deng D, Li J X, Jhaveri R H, et al. Reinforcement-learning-based optimization on energy efficiency in UAV networks for IoT[J]. IEEE Internet of Things Journal, 2023,10(3): 2767-2775.

选择文件类型/文献管理软件名称

选择包含的内容