基于隐式分位数网络的车联网任务卸载策略

doi:10.19678/j.issn.1000-3428.0069929

摘要/Abstract

摘要：

随着物联网(IoT)和无线技术的迅猛发展, 车辆面临着前所未有的计算资源需求挑战。为了应对这些挑战, 研究车辆边缘计算(VEC)场景中的任务卸载问题, 提出一种基于隐式分位数网络(IQN)的动态任务卸载策略。首先, 对VEC系统进行建模, 将任务卸载决策问题构建为一个马尔可夫决策过程(MDP); 然后, 引入一种融合时间优先和噪声增强策略的双分位数强化学习算法, 以实现更加精准的任务卸载。该算法利用IQN对值函数的完整概率分布进行估计, 进而实现对回报分布的连续参数化估计, 有效提升预测和决策的准确性。同时, 算法整合了时序优先经验回放机制和噪声网络, 前者优先重放对学习更有价值的经验, 后者通过引入随机性增强了探索效率。实验结果表明, 与传统算法相比, 该算法能够显著降低整体任务的完成时延, 同时提升任务卸载决策的准确性和系统资源的利用率。研究表明, 通过引入IQN和双分位数强化学习算法, 可以在动态且复杂的车联网(IoV)环境中实现高效的任务卸载。

关键词: 车联网, 边缘计算, 任务卸载, 深度强化学习, 隐式分位数网络

Abstract:

With the rapid development of the Internet of Things (IoT) and wireless technology, vehicles are facing unprecedented challenges in terms of computing resource demands. To address these challenges, task offloading in Vehicle Edge Computing (VEC) scenarios is studied, and a dynamic task offloading strategy based on an Implicit Quantile Network (IQN) is proposed. First, the VEC system is modeled, and the task offloading decision problem is constructed as a Markov Decision Process (MDP); Subsequently, a binary reinforcement learning algorithm that integrates time-first and noise enhancement strategies is introduced to achieve more accurate task offloading. This algorithm utilizes an IQN to estimate the complete probability distribution of the value function, thereby achieving a continuous parameterized estimation of the return distribution and effectively improving the prediction and decision-making accuracy. Simultaneously, the algorithm integrates a temporal-priority experience replay mechanism and a noise network. The former prioritizes replaying experiences that are more valuable for learning, whereas the latter enhances exploration efficiency by introducing randomness. The experimental results show that, compared with traditional algorithms, this algorithm can significantly reduce the overall task completion delay while improving the accuracy of task offloading decisions and the utilization of system resources. Research has shown that by introducing IQN and binary reinforcement learning algorithms, efficient task offloading can be achieved in dynamic and complex Internet of Vehicles (IoV) environments.

Key words: Internet of Vehicles (IoV), edge computing, task offloading, Deep Reinforcement Learning (DRL), Implicit Quantile Network (IQN)

王聪, 刘帅, 左明敏. 基于隐式分位数网络的车联网任务卸载策略[J]. 计算机工程, 2025, 51(12): 244-254.

WANG Cong, LIU Shuai, ZUO Mingmin. Task Offloading Strategy for Internet of Vehicles Based on Implicit Quantile Network[J]. Computer Engineering, 2025, 51(12): 244-254.

https://www.ecice06.com/CN/Y2025/V51/I12/244

图/表 11

图1 VEC架构

Fig.1 VEC architecture

图2 系统模型

Fig.2 System model

图3 TPNE-DQRLD算法架构

Fig.3 TPNE-DQRLD algorithm architecture

图4 算法的收敛性能对比

Fig.4 Comparison of convergence performance of algorithms

图5 TPER消融实验结果

Fig.5 Results of TPER ablation experiment

图6 车辆数量对时延的影响

Fig.6 The impact of vehicle quantity on latency

图7 3种算法的收敛性能对比

Fig.7 Comparison of convergence performance of three algorithms

图8 3种算法的时延比较

Fig.8 Time delay comparison of three algorithms

参考文献 25

1	刘雷, 陈晨, 冯杰, 等. 车载边缘计算卸载技术研究综述. 电子学报, 2021, 49 (5): 861- 871.
	LIU L , CHEN C , FENG J , et al. A survey of computation offloading in vehicular edge computing networks. Acta Electronica Sinica, 2021, 49 (5): 861- 871.
2	李萌, 司鹏搏, 孙恩昌, 等. 基于车联网和移动边缘计算的时延可容忍数据传输. 北京工业大学学报, 2018, 44 (4): 529- 537.
	LI M , SI P B , SUN E C , et al. Delay-tolerant data transmission based on Internet of Vehicles and mobile edge computing. Journal of Beijing University of Technology, 2018, 44 (4): 529- 537.
3	李子姝, 谢人超, 孙礼, 等. 移动边缘计算综述. 电信科学, 2018, 34 (1): 87- 101.
	LI Z S , XIE R C , SUN L , et al. A survey of mobile edge computing. Telecommunications Science, 2018, 34 (1): 87- 101.
4	曹敦, 张应宝, 邹电, 等. V2X多节点协同分布式卸载策略. 通信学报, 2022, 43 (2): 185- 195.
	CAO D , ZHANG Y B , ZOU D , et al. Multi-node cooperative distributed offloading strategy for V2X. Journal of Communications, 2022, 43 (2): 185- 195.
5	LEE J, NA W. A survey on vehicular edge computing architectures[C]//Proceedings of the 13th International Conference on Information and Communication Technology Convergence (ICTC). Washington D.C., USA: IEEE Press, 2022: 2198-2200.
6	戚可寒. 面向车辆边缘计算的任务卸载关键技术研究[D]. 杭州: 杭州电子科技大学, 2023.
	QI K H. Research on key technologies of task unloading for vehicle edge computing[D]. Hangzhou: Hangzhou Dianzi University, 2023. (in Chinese)
7	YUAN S, FAN Y F, CAI Y. A survey on computation offloading for vehicular edge computing[C]//Proceedings of the 7th International Conference on Information Technology: IoT and Smart City. New York, USA: ACM Press, 2019: 107-112.
8	AHMED M , RAZA S , MIRZA M A , et al. A survey on vehicular task offloading: classification, issues, and challenges. Journal of King Saud University - Computer and Information Sciences, 2022, 34 (7): 4135- 4162. doi: 10.1016/j.jksuci.2022.05.016
9	DAI F , LIU G Z , MO Q , et al. Task offloading for vehicular edge computing with edge-cloud cooperation. World Wide Web, 2022, 25 (5): 1999- 2017. doi: 10.1007/s11280-022-01011-8
10	ZHANG J , GUO H Z , LIU J J , et al. Task offloading in vehicular edge computing networks: a load-balancing solution. IEEE Transactions on Vehicular Technology, 2020, 69 (2): 2092- 2104. doi: 10.1109/TVT.2019.2959410
11	YAO J J , ANSARI N . Task allocation in fog-aided mobile IoT by Lyapunov online reinforcement learning. IEEE Transactions on Green Communications and Networking, 2020, 4 (2): 556- 565. doi: 10.1109/TGCN.2019.2956626
12	TANG H J , WU H M , QU G J , et al. Double deep Q-network based dynamic framing offloading in vehicular edge computing. IEEE Transactions on Network Science and Engineering, 2023, 10 (3): 1297- 1310. doi: 10.1109/TNSE.2022.3172794
13	LI S Y , HU X H , DU Y W . Deep reinforcement learning and game theory for computation offloading in dynamic edge computing markets. IEEE Access, 2021, 9, 121456- 121466. doi: 10.1109/ACCESS.2021.3109132
14	ZHAN W H , LUO C B , WANG J , et al. Deep-reinforcement-learning-based offloading scheduling for vehicular edge computing. IEEE Internet of Things Journal, 2020, 7 (6): 5449- 5465. doi: 10.1109/JIOT.2020.2978830
15	CUI Y P, DU L J, HE P, et al. Multi-vehicle intelligent collaborative computing strategy for Internet of vehicles[C]//Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC). Washington D.C., USA: IEEE Press, 2022: 1647-1652.
16	LIANG S N, WAN H B, QIN T F, et al. Multi-user computation offloading for mobile edge computing: a deep reinforcement learning and game theory approach[C]//Proceedings of the 20th IEEE International Conference on Communication Technology (ICCT). Washington D.C., USA: IEEE Press, 2020: 1534-1539.
17	ZHU A Q, GUO S T, MA M F, et al. Computation offloading for workflow in mobile edge computing based on deep Q-learning[C]//Proceedings of the 28th Wireless and Optical Communications Conference (WOCC). Washington D.C., USA: IEEE Press, 2019: 1-5.
18	WU Z Y , YAN D F . Deep reinforcement learning-based computation offloading for 5G vehicle-aware multi-access edge computing network. China Communications, 2021, 18 (11): 26- 41. doi: 10.23919/JCC.2021.11.003
19	李强, 仪晋辉, 杜婷婷, 等. 移动边缘计算中基于A3C的依赖任务卸载与资源分配. 计算机工程, 2023, 49 (6): 42- 52. doi: 10.19678/j.issn.1000-3428.0066095
	LI Q , YI J H , DU T T , et al. Dependent task offloading and resource allocation based on A3C in mobile edge computing. Computer Engineering, 2023, 49 (6): 42- 52. doi: 10.19678/j.issn.1000-3428.0066095
20	DAI P L, HU K W, WU X, et al. Asynchronous deep reinforcement learning for data-driven task offloading in MEC-empowered vehicular networks[C]//Proceedings of the IEEE INFOCOM Conference on Computer Communications. Washington D.C., USA: IEEE Press, 2021: 1-10.
21	WANG C L, PENG J, JIANG F, et al. An adaptive deep Q-learning service migration decision framework for connected vehicles[C]//Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC). Washington D.C., USA: IEEE Press, 2020: 944-949.
22	YUAN X M , CHEN J H , YANG J Y , et al. FedSTN: graph representation driven federated learning for edge computing enabled urban traffic flow prediction. IEEE Transactions on Intelligent Transportation Systems, 2023, 24 (8): 8738- 8748. doi: 10.1109/TITS.2022.3157056
23	YUAN X M , CHEN J H , ZHANG N , et al. Low-cost federated broad learning for privacy-preserved knowledge sharing in the RIS-aided Internet of vehicles. Engineering, 2024, 33, 178- 189. doi: 10.1016/j.eng.2023.04.015
24	班玉琦, 段利国, 温昊宇, 等. 面向移动感知的计算卸载及资源分配策略研究. 计算机工程, 2023, 49 (8): 163- 173. doi: 10.19678/j.issn.1000-3428.0066522
	BAN Y Q , DUAN L G , WEN H Y , et al. Research on mobility-aware computation offloading and resource allocation strategy. Computer Engineering, 2023, 49 (8): 163- 173. doi: 10.19678/j.issn.1000-3428.0066522
25	DABNEY W, OSTROVSKI G, SILVER D, et al. Implicit quantile networks for distributional reinforcement learning[C]//Proceedings of International Conference on Machine Learning. New York, USA: ACM Press, 2018: 1096-1105.

[1]	崔萌萌, 施静燕, 项昊龙. 基于空地协同的动态车载边缘任务卸载方法[J]. 计算机工程, 2025, 51(9): 25-37.
[2]	毕昌兵, 田有亮. 车联网中基于身份签名的匿名可追溯消息认证方案[J]. 计算机工程, 2025, 51(9): 158-165.
[3]	陈彦如, 刘珂良, 冉茂亮. 基于深度强化学习的外卖即时配送实时优化[J]. 计算机工程, 2025, 51(9): 328-339.
[4]	秦敏浩, 孙未未. 基于隐状态预测的失真交通信号灯路口控制策略[J]. 计算机工程, 2025, 51(9): 1-13.
[5]	赵季红, 臧若雨, 刘振. 面向卫星车载MEC网络的协同计算卸载方法[J]. 计算机工程, 2025, 51(9): 49-58.
[6]	黄业恒, 覃团发, 苏振朗, 王素红. 6G网络下超密集无线体域网高效卸载策略[J]. 计算机工程, 2025, 51(9): 201-212.
[7]	张伟航, 钟永彦, 向元柱, 丁士旵. 边云辅助下的可撤销属性加密方案[J]. 计算机工程, 2025, 51(7): 244-253.
[8]	杨莉斌, 詹成, 李婷婷, 廖婧睿. 多天线无人机视频通信中的节能资源分配策略[J]. 计算机工程, 2025, 51(6): 266-274.
[9]	亓明凯, 王迪, 张立晔. 基于分层强化学习的在线三维装箱模型[J]. 计算机工程, 2025, 51(6): 136-145.
[10]	武小丰, 袁培燕. 一种改进蛇优化算法的边缘服务器动态放置策略[J]. 计算机工程, 2025, 51(6): 255-265.
[11]	吕超峰, 徐鹏飞, 罗迪, 刘金平. 基于多智能体深度强化学习的SD-IoT控制器部署[J]. 计算机工程, 2025, 51(5): 83-92.
[12]	吴凯峰, 刘磊, 刘晨, 梁成庆. 基于融合课程思想MADDPG的无人机编队控制[J]. 计算机工程, 2025, 51(5): 73-82.
[13]	杜剑波, 董伟哲, 金蓉, 王军选, 康嘉文, 刘雷, 策力木格. 基于MEC的空天地一体化网络任务分割与资源分配[J]. 计算机工程, 2025, 51(5): 43-51.
[14]	李俊吉, 张佳琦, 高改梅, 杨莉. 基于信誉机制的车联网共识算法[J]. 计算机工程, 2025, 51(4): 217-226.
[15]	林绍福, 陈盈盈, 李硕朋. 基于深度强化学习的多无人机能量传输与边缘计算联合优化方法[J]. 计算机工程, 2025, 51(3): 144-154.

选择文件类型/文献管理软件名称

选择包含的内容