摘要：

通过协同管理地面网络、卫星网络和近地无人机网络等, 空天地一体化算力融合网络有望实现全域连接和普适智能, 为我国数字经济发展提供有力支撑。而低轨道地球(LEO)卫星具备泛在连接和边缘计算能力, 为实现空天地一体化的高效计算体系提供了基础。通过将移动边缘计算(MEC)沉降至LEO卫星网络, 形成面向业务的"端-边-云"三级计算架构, 可将时延敏感型业务从终端卸载到LEO卫星侧, 以提升该业务的任务完成率。然而, 为LEO卫星边缘网络制定高效计算卸载和算力分配决策是一个亟待解决的问题。针对高动态性的卫星网络环境和离散-连续的混合动作空间, 提出一种基于生成扩散模型的混合近端策略优化(H-PPO)方法。首先, 对具有时变特性的无线信道进行建模, 并构建不同卸载决策下的服务时延、通信和计算模型。其次, 在卸载决策、剩余计算资源和功率控制的多约束条件下, 构建最大化平均任务完成率的长期优化问题。然后, 建立具有参数化动作的马尔可夫决策过程模型, 将生成扩散模型引入为离散动作策略, 增强传统深度强化学习(DRL)方法的采样效率和探索能力, 并利用所提算法联合优化计算卸载、算力分配和功率控制变量。仿真结果表明, 所提算法具有较好收敛性能, 并在任务完成率方面优于其他3种对比方法。

关键词: 空天地一体化网络, 低轨道地球卫星, 移动边缘计算, 深度强化学习, 生成扩散模型, 计算卸载, 资源分配

Abstract:

Via coordinated management of ground networks, satellite networks, and near-earth unmanned aerial vehicle networks, computing-empowered space-air-ground integrated networks can achieve global connectivity and universal intelligence, providing strong support for the development of China's digital economy. Low Earth Orbit (LEO) satellites have the ability of ubiquitous connectivity and edge computing, which provides the basis for an efficient computing system for space-air-ground integration. By synchronizing the Mobile Edge Computing (MEC) to LEO satellite networks to form a service-oriented end-edge-cloud three-level computing architecture, latency-sensitive tasks can be offloaded from terminals to LEO satellites, which improves the task completion rate. However, methods to make efficient offloading decisions and compute power allocation in LEO satellite edge networks must be developed urgently. Aiming at high dynamics in the satellite network environment and the discrete-continuous hybrid action space, this study proposes a Hybrid Proximal Policy Optimization (H-PPO) method based on the generative diffusion model. First, a wireless channel with time-varying characteristics is modeled and service latency, communication, and computation models under different offloading decisions are constructed. Second, under the multiple constraints of offloading decisions, remaining computing resources, and power control, a long-term optimization problem for maximizing the average task completion rate is constructed. Subsequently, the Markov decision process with parameterized actions is established and the generative diffusion model is introduced as the discrete action policy to improve the sampling efficiency and exploration ability of traditional Deep Reinforcement Learning (DRL) methods. Finally, the proposed method is used to jointly optimize the computing offloading, computing power allocation, and power control. The simulation results show that the proposed method has a better convergence performance and is superior to the three comparison methods in terms of task completion rate.

Key words: space-air-ground integrated networks, Low Earth Orbit (LEO) satellite, Mobile Edge Computing (MEC), Deep Reinforcement Learning (DRL), generative diffusion model, computing offloading, resource allocation