摘要：

在车辆边缘计算系统中, 受限于自身的计算能力, 单个车辆难以处理计算密集型任务, 并且由于车联网(IoV)环境的高动态性, 车辆难以获取全局信息及其他车辆的卸载行为来做出任务卸载决策。为了解决车辆边缘计算系统中计算资源不足、环境状态时变以及车辆观察范围有限等问题, 联合车辆与基础设施(V2I)和车辆与车辆(V2V)卸载方式并考虑任务划分问题, 提出一种基于多智能体深度确定性策略梯度的在线算法。首先, 综合考虑车辆位置、连接时间和可用计算资源, 选择服务性能值较高的车辆作为候选服务车辆; 其次, 以最小化系统的平均任务卸载时延为目标提出一个优化问题, 并将其建模为一个马尔可夫决策过程, 通过对模型进行集中训练, 车辆可以获取其他车辆的信息以调整自身策略, 在在线执行阶段, 车辆根据局部观察迅速做出卸载决策。将该算法与基准算法进行对比, 实验结果表明, 相较于深度确定性策略梯度算法和任务均分法, 所提任务卸载算法的平均任务卸载时延分别降低75%和66%, 且收敛速度更快, 证明了该算法的有效性。

关键词: 车辆边缘计算, 任务卸载, 任务划分, 多智能体深度强化学习, V2I与V2V联合卸载

Abstract:

In vehicle edge computing systems, individual vehicles frequently encounter difficulties in executing computation-intensive tasks due to their constrained processing capacities. Furthermore, the highly dynamic nature of the Internet of Vehicles (IoV) environment exacerbates the challenge, as vehicles struggle to gather comprehensive global information about their surroundings and the task offloading behaviors of neighboring vehicles. This complexity hampers effective decision-making in task offloading. To mitigate these challenges—limited computational resources, fluctuating environmental conditions, and restricted observational capabilities—this study introduces an online algorithm based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) framework. The proposed approach synergistically integrates both Vehicle-to-Infrastructure (V2I) and Vehicle-to-Vehicle (V2V) offloading mechanisms while also incorporating task division to optimize overall system performance. First, by considering vehicle location, connection duration, and available computational resources, the vehicle with the highest service performance value is selected as the candidate service vehicle. Second, an optimization problem is formulated to minimize the system's average task offloading delay, and this problem is modeled as a Markov decision process. Through centralized training, vehicles are able to obtain information from other vehicles, enabling them to adjust their own policies accordingly. In the online execution phase, vehicles can make rapid task offloading decisions based on local observations. Finally, the proposed algorithm is compared against benchmark algorithms. The experimental results demonstrate that, compared to the deep deterministic policy gradient method and the equal task division method, the proposed task offloading algorithm reduces the average task offloading delay by 75% and 66%, respectively, and exhibits a faster convergence rate, validating the algorithm's effectiveness.

Key words: vehicle edge computing, task offloading, task division, multi-agent deep reinforcement learning, V2I and V2V joint offloading