Adaptive Cooperative Task Offloading Decision for Multiple Unmanned Aerial Vehicles

doi:10.19678/j.issn.1000-3428.0069899

Abstract

Abstract:

This study investigates adaptive cooperative task offloading and allocation in a multiple Unmanned Aerial Vehicles (UAVs) collaborative mobile edge computing network. To enhance collaboration among UAVs in a time-varying environment and improve the efficiency of task execution, this study constructs a UAV task queuing model in a time-varying environment and establishes a UAVs task offloading decision model based on the Markov Decision Process (MDP). Moreover, this study proposes a Cooperative-based Deep Deterministic Policy Gradient (CODDPG) algorithm to address the optimization problem of multiple UAVs offloading. The CODDPG algorithm, which integrates CommNet with the traditional Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, facilitates the sharing of environmental observations among all UAVs. This approach effectively extends the UAVs' perception of the environment and enhances their collaborative decision capability. It also addresses the issue of local optima in the MADDPG algorithm caused by its sole dependence on local information during agent training, thereby minimizing total computation delay. Experimental results demonstrate that the CODDPG algorithm not only significantly reduces task computation delay effectively but also converges faster than the traditional MADDPG algorithm.

Key words: mobile edge computing, multiple Unmanned Aerial Vehicles (UAVs) collaboration, deep reinforcement learning, offloading optimization, computation delay minimization

摘要：

研究多无人机(UAV)协同移动边缘计算网络中自适应合作任务卸载与分配问题。为了提高时变环境下无人机之间的协同性, 进而提升任务的执行效率, 构建时变环境下无人机任务队列模型, 并建立基于马尔可夫决策过程(MDP)的无人机任务卸载决策模型。提出一种基于合作的深度确定性策略梯度(CODDPG)算法, 以解决多无人机卸载决策优化问题。CODDPG算法结合神经网络CommNet与传统的多智能体深度确定性策略梯度(MADDPG)算法, 实现了无人机的环境观测值共享, 有效拓展了无人机的环境感知范围并增强了它们之间的协同决策能力, 并且解决了MADDPG算法中智能体的训练仅依赖局部信息而陷入局部最优解问题, 从而最小化总计算时延。经过实验证明, CODDPG算法不仅有效降低了任务计算时延, 而且与传统的MADDPG算法相比收敛速度更快。

关键词: 移动边缘计算, 多无人机协同, 深度强化学习, 卸载优化, 计算时延最小化

LIU Yi, LUO Chun, ZHONG Weifeng, YU Yi, OU Zhiqing. Adaptive Cooperative Task Offloading Decision for Multiple Unmanned Aerial Vehicles[J]. Computer Engineering, 2026, 52(4): 339-348.

刘义, 罗淳, 钟伟锋, 余意, 欧智清. 多无人机自适应合作任务卸载决策[J]. 计算机工程, 2026, 52(4): 339-348.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0069899

https://www.ecice06.com/EN/Y2026/V52/I4/339

Figures/Tables 9

Fig.1 System model

Fig.2 CODDPG algorithm training framework

Fig.3 CommNet training framework

Fig.4 Convergence comparison

Fig.5 Impact of task quantity on computation delay

Fig.6 Impact of UAV quantity on computation delay

Fig.7 Impact of UAV transmitting power on computation delay

Fig.8 Impact of UAV transmitting power on energy consumption

References 27

1	PAN W B , LIU Y , YANG C . Adaptive task offloading of rechargeable UAV edge computing network based on double decision value iteration. Computer Communications, 2022, 193, 136- 145. doi: 10.1016/j.comcom.2022.06.026
2	ZHANG L , ANSARI N . Optimizing the operation cost for UAV-aided mobile edge computing. IEEE Transactions on Vehicular Technology, 2021, 70 (6): 6085- 6093. doi: 10.1109/TVT.2021.3076980
3	WANG J R , LIU K Y , PAN J P . Online UAV-mounted edge server dispatching for mobile-to-mobile edge computing. IEEE Internet of Things Journal, 2020, 7 (2): 1375- 1386. doi: 10.1109/JIOT.2019.2954798
4	徐渊博, 任静, 王亮, 等. 面向无人机辅助边缘计算的卸载任务准入控制机制. 计算机工程, 2025, 51 (2): 54- 64. doi: 10.19678/j.issn.1000-3428.0069749
	XU Y B , REN J , WANG L , et al. Admission control mechanism for offloading tasks in UAV-assisted edge computing. Computer Engineering, 2025, 51 (2): 54- 64. doi: 10.19678/j.issn.1000-3428.0069749
5	陈卓, 操民涛, 周致圆, 等. 移动边缘计算中基于图到序列深度强化学习的复杂任务部署策略. 通信学报, 2024, 45 (3): 244- 257.
	CHEN Z , CAO M T , ZHOU Z Y , et al. Graph-to-sequence deep reinforcement learning based complex task deployment strategy in MEC. Journal on Communications, 2024, 45 (3): 244- 257.
6	彭世明, 林士飏, 贾硕, 等. 基于负载预测的多目标优化任务卸载策略. 计算机工程, 2024, 50 (1): 206- 215. doi: 10.19678/j.issn.1000-3428.0066766
	PENG S M , LIN S Y , JIA S , et al. Multi-objective optimization task offloading strategy based on load prediction. Computer Engineering, 2024, 50 (1): 206- 215. doi: 10.19678/j.issn.1000-3428.0066766
7	HU Q Y , CAI Y L , YU G D , et al. Joint offloading and trajectory design for UAV-enabled mobile edge computing systems. IEEE Internet of Things Journal, 2019, 6 (2): 1879- 1892. doi: 10.1109/JIOT.2018.2878876
8	YANG Z H , PAN C H , WANG K Z , et al. Energy efficient resource allocation in UAV-enabled mobile edge computing networks. IEEE Transactions on Wireless Communications, 2019, 18 (9): 4576- 4589. doi: 10.1109/TWC.2019.2927313
9	CHENG N , XU W C , SHI W S , et al. Air—ground integrated mobile edge networks: architecture, challenges, and opportunities. IEEE Communications Magazine, 2018, 56 (8): 26- 32. doi: 10.1109/MCOM.2018.1701092
10	XIONG J Y , GUO H Z , LIU J J . Task offloading in UAV-aided edge computing: bit allocation and trajectory optimization. IEEE Communications Letters, 2019, 23 (3): 538- 541. doi: 10.1109/LCOMM.2019.2891662
11	TUN Y K , PARK Y M , TRAN N H , et al. Energy-efficient resource management in UAV-assisted mobile edge computing. IEEE Communications Letters, 2021, 25 (1): 249- 253. doi: 10.1109/LCOMM.2020.3026033
12	ZHANG T K , XU Y , LOO J , et al. Joint computation and communication design for UAV-assisted mobile edge computing in IoT. IEEE Transactions on Industrial Informatics, 2020, 16 (8): 5505- 5516. doi: 10.1109/TII.2019.2948406
13	YU Z , GONG Y M , GONG S M , et al. Joint task offloading and resource allocation in UAV-enabled mobile edge computing. IEEE Internet of Things Journal, 2020, 7 (4): 3147- 3159. doi: 10.1109/JIOT.2020.2965898
14	LUO Y Z , DING W R , ZHANG B C . Optimization of task scheduling and dynamic service strategy for multi-UAV-enabled mobile-edge computing system. IEEE Transactions on Cognitive Communications and Networking, 2021, 7 (3): 970- 984. doi: 10.1109/TCCN.2021.3051947
15	APOSTOLOPOULOS P A , FRAGKOS G , TSIROPOULOU E E , et al. Data offloading in UAV-assisted multi-access edge computing systems under resource uncertainty. IEEE Transactions on Mobile Computing, 2023, 22 (1): 175- 190. doi: 10.1109/TMC.2021.3069911
16	DENG X H , ZHAO J H , KUANG Z F , et al. Computation efficiency maximization in multi-UAV-enabled mobile edge computing systems based on 3D deployment optimization. IEEE Transactions on Emerging Topics in Computing, 2023, 11 (3): 778- 790. doi: 10.1109/TETC.2023.3268346
17	LOWE R, WU Y, TAMAR A, et al. Multi-agent actor—critic for mixed cooperative-competitive environments[EB/OL]. [2024-04-11]. http://arxiv.org/abs/1706.02275.
18	WANG L , WANG K Z , PAN C H , et al. Multi-agent deep reinforcement learning-based trajectory planning for multi-UAV assisted mobile edge computing. IEEE Transactions on Cognitive Communications and Networking, 2021, 7 (1): 73- 84. doi: 10.1109/TCCN.2020.3027695
19	CHEN J X , YANG P , REN S Q , et al. Enhancing AIoT device association with task offloading in aerial MEC networks. IEEE Internet of Things Journal, 2024, 11 (1): 174- 187. doi: 10.1109/JIOT.2023.3300011
20	GONG Y K , YAO H P , WANG J J , et al. Multi-agent driven resource allocation and interference management for deep edge networks. IEEE Transactions on Vehicular Technology, 2022, 71 (2): 2018- 2030. doi: 10.1109/TVT.2021.3134467
21	PENG H X , SHEN X M . Multi-agent reinforcement learning based resource management in MEC- and UAV-assisted vehicular networks. IEEE Journal on Selected Areas in Communications, 2021, 39 (1): 131- 141. doi: 10.1109/JSAC.2020.3036962
22	KIRAN B R , SOBH I , TALPAERT V , et al. Deep reinforcement learning for autonomous driving: a survey. IEEE Transactions on Intelligent Transportation Systems, 2022, 23 (6): 4909- 4926. doi: 10.1109/TITS.2021.3054625
23	SUTTON R S , BARTO A G . Reinforcement learning: an introduction. Cambridge, USA: MIT press, 2018.
24	DEGRIS T, WHITE M, SUTTON R S. Off-policy actor—critic[EB/OL]. [2024-04-11]. https://arxiv.org/abs/1205.4839.
25	MALEKI M R , MILI M R , JAVAN M R , et al. Multi-agent reinforcement learning trajectory design and two-stage resource management in CoMP UAV VLC networks. IEEE Transactions on Communications, 2022, 70 (11): 7464- 7476. doi: 10.1109/TCOMM.2022.3211083
26	CHENG Z P , MIN M H , LIWANG M H , et al. Multiagent DDPG-based joint task partitioning and power control in fog computing networks. IEEE Internet of Things Journal, 2022, 9 (1): 104- 116. doi: 10.1109/JIOT.2021.3091508
27	SUKHBAATAR S, ARTHUR S, ROB F. Learning multiagent communication with backpropagation[EB/OL]. [2024-04-11]. https://arxiv.org/abs/1605.07736.

[1]	LI Bin, GUO Yi. Deep Reinforcement Learning Algorithms for Heterogeneous Multiple Knapsack Problems [J]. Computer Engineering, 2026, 52(4): 140-162.
[2]	WANG Xingjie, WANG Kan, FEI Rong, WANG Huaijun, GUO Yinbo, LAN Dapeng, ZHU Xiaojie. Computing Power Allocation Strategy Based on Diffusion Model in Satellite Edge Networks [J]. Computer Engineering, 2026, 52(1): 346-355.
[3]	ZHAO Jihong, ZANG Ruoyu, LIU Zhen. Collaborative Computation Offloading Method for Satellite Vehicle-Mounted Mobile Edge Computing Networks [J]. Computer Engineering, 2025, 51(9): 49-58.
[4]	CHEN Yanru, LIU Keliang, RAN Maoliang. Real-time Optimization of Instant Meal Delivery Based on Deep Reinforcement Learning [J]. Computer Engineering, 2025, 51(9): 328-339.
[5]	QIN Minhao, SUN Weiwei. Control Strategy for Intersections with Distorted Traffic Signals Based on Hidden State Prediction [J]. Computer Engineering, 2025, 51(9): 1-13.
[6]	CUI Mengmeng, SHI Jingyan, XIANG Haolong. Dynamic Vehicle Edge Task Offloading Method Based on Air-Ground Collaboration [J]. Computer Engineering, 2025, 51(9): 25-37.
[7]	QI Mingkai, WANG Di, ZHANG Liye. Online 3D Bin Packing Model Based on Hierarchical Reinforcement Learning [J]. Computer Engineering, 2025, 51(6): 136-145.
[8]	WU Xiaofeng, YUAN Peiyan. Dynamic Placement Strategy for Edge Servers Under Improved Snake Optimization Algorithm [J]. Computer Engineering, 2025, 51(6): 255-265.
[9]	LÜ Chaofeng, XU Pengfei, LUO Di, LIU Jinping. SD-IoT Controller Placement Based on Multi-Agent Deep Reinforcement Learning [J]. Computer Engineering, 2025, 51(5): 83-92.
[10]	WU Kaifeng, LIU Lei, LIU Chen, LIANG Chengqing. Unmanned Aerial Vehicle Formation Control Based on MADDPG with Integrated Curriculum Learning [J]. Computer Engineering, 2025, 51(5): 73-82.
[11]	LIN Shaofu, CHEN Yingying, LI Shuopeng. Method of Joint Optimization for Multi-UAV Energy Transfer and Edge Computing Based on Deep Reinforcement Learning [J]. Computer Engineering, 2025, 51(3): 144-154.
[12]	LI Siyuan, ZHONG Xingyu, LI Kaiyin, XU Qingzhen. Strategy Teaching Research Based on Multilayer Graph Relationship and Reinforcement Learning [J]. Computer Engineering, 2025, 51(3): 122-130.
[13]	ZENG Jianzhou, LI Zeping, ZHANG Suqin. Multi-agent Cooperative Caching Strategy Based on TD3 Algorithm [J]. Computer Engineering, 2025, 51(2): 365-374.
[14]	LIU Liang, MAO Wuping, LI Wenwei, TAN Siyuan, JING Tengxiang. Task Offloading Strategy Based on Game Theory in the Space-Air-Ground Integrated Edge Computing Networks [J]. Computer Engineering, 2025, 51(2): 238-249.
[15]	WANG Cong, LIU Shuai, ZUO Mingmin. Task Offloading Strategy for Internet of Vehicles Based on Implicit Quantile Network [J]. Computer Engineering, 2025, 51(12): 244-254.

Please choose a citation manager

Content to export