Edge-Cloud Collaborative Task Offloading Mechanism Based on DDQN in Vehicular Networks

doi:10.19678/j.issn.1000-3428.0063739

Abstract

Abstract: Computation offloading is a promising scheme to alleviate the shortage of vehicle resources facing the explosive growth trend of data computation.Compared with studying cloud computing or edge computing separately, integrating with each other can realize the complementary advantages and improve the overall quality of service.In vehicular networks, a primary challenge is to make offloading decisions, which can adapt to the dynamic environment.During this process, the urgency of tasks cannot be ignored.This paper constructs a collaborative edge-cloud task offloading architecture based on Software Defined Network (SDN), where the metrics of task priority is given.The task offloading problem is then formulated as a Markov Decision Process (MDP), which aims to maximize the utility composed of delay and cost.To solve task offloading decisions, this paper puts forward a task offloading decision algorithm based on Double Deep Q Network(DDQN)and a priority-based resource allocation scheme successively.On this basis, this paper designs a method of computing offloading ratio, which aims to minimize the task processing delay while ensuring that the part of tasks can be uploaded completely within the communication time.Simulation results show that the performance of delay and utility of the proposed algorithm is more than doubled compared to other fixed offloading algorithms such as All Local, All Offloading and Allocating Resources Evenly.Under the condition of moderate numbers of vehicles, the success rate of tasks can be maintained at 100%.

Key words: vehicular networks, edge-cloud collaboration, task offloading, Deep Reinforcement Learning(DRL), priority, resource allocation

摘要： 面对车载终端数据计算量的爆炸式增长，计算卸载是缓解车辆资源不足的有效手段。相比于单独研究云计算或边缘计算，让两者相互协作可以实现优势互补，提高系统的整体服务质量。在车联网中，制定适应环境动态性的卸载决策存在较大困难，其中任务的紧急程度也是一个不容忽视的因素。构建一个基于软件定义网络的边云协作任务卸载架构，并设计任务优先级的度量标准，将动态环境中的任务卸载决策问题建模为马尔可夫决策过程，从而最大化由时延和成本构成的任务平均效用。为了求解任务卸载决策，提出基于双深度Q网络的任务卸载决策算法以及基于优先级的资源分配方案，并设计一种卸载比例计算方法，以保障卸载的任务量能够在通信时间内上传完成的同时最小化任务处理时延。实验结果表明，相比于全部本地、全部卸载和平均分配资源3种固定的卸载算法，该算法时延和效用性能提高了2倍以上，在车辆数目适中的情况下，任务的完成比例可以稳定保持在100%。

关键词: 车联网, 边云协作, 任务卸载, 深度强化学习, 优先级, 资源分配

CLC Number:

TP391

YU Jing, LU Lingyun, LI Xiang. Edge-Cloud Collaborative Task Offloading Mechanism Based on DDQN in Vehicular Networks[J]. Computer Engineering, 2022, 48(12): 156-164.

于晶, 鲁凌云, 李翔. 车联网中基于DDQN的边云协作任务卸载机制[J]. 计算机工程, 2022, 48(12): 156-164.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0063739

http://www.ecice06.com/EN/Y2022/V48/I12/156

Figures/Tables 7

References

[1] DU J, JIANG C X, WANG J, et al.Machine learning for 6G wireless networks:carrying forward enhanced bandwidth, massive access, and ultrareliable/low-latency service[J].IEEE Vehicular Technology Magazine, 2020, 15(4):122-134.
[2] VAN LE D, THAM C K.Quality of service aware computation offloading in an ad-hoc mobile cloud[J].IEEE Transactions on Vehicular Technology, 2018, 67(9):8890-8904.
[3] HUANG W, HUANG Y M, HE S W, et al.Cloud and edge multicast beamforming for cache-enabled ultra-dense networks[J].IEEE Transactions on Vehicular Technology, 2020, 69(3):3481-3485.
[4] ZHANG J, LETAIEF K B.Mobile edge intelligence and computing for the Internet of vehicles[J].Proceedings of the IEEE, 2020, 108(2):246-261.
[5] TRAN T X, HAJISAMI A, PANDEY P, et al.Collaborative mobile edge computing in 5G networks:new paradigms, scenarios, and challenges[J].IEEE Communications Magazine, 2017, 55(4):54-61.
[6] 李波, 侯鹏, 牛力, 等.基于软件定义网络的云边协同架构研究综述[J].计算机工程与科学, 2021, 43(2):242-257. LI B, HOU P, NIU L, et al.Survey of cloud-edge collaborative architecture research based on software defined network[J].Computer Engineering & Science, 2021, 43(2):242-257.(in Chinese)
[7] ZHAN W H, LUO C B, WANG J, et al.Deep reinforcement learning-based computation offloading in vehicular edge computing[C]//Proceedings of IEEE Global Communications Conference.Washington D.C., USA:IEEE Press, 2019:1-6.
[8] SUN J N, GU Q, ZHENG T, et al.Joint optimization of computation offloading and task scheduling in vehicular edge computing networks[J].IEEE Access, 2020, 8:10466-10477.
[9] DAI Y Y, XU D, MAHARJAN S, et al.Joint load balancing and offloading in vehicular edge computing and networks[J].IEEE Internet of Things Journal, 2019, 6(3):4377-4387.
[10] ZHAO J H, LI Q P, GONG Y, et al.Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks[J].IEEE Transactions on Vehicular Technology, 2019, 68(8):7944-7956.
[11] MENG X L, WANG W, WANG Y T, et al.Closed-form delay-optimal computation offloading in mobile edge computing systems[J].IEEE Transactions on Wireless Communications, 2019, 18(10):4653-4667.
[12] ZHOU Z Y, FENG J H, CHANG Z, et al.Energy-efficient edge computing service provisioning for vehicular networks:a consensus ADMM approach[J].IEEE Transactions on Vehicular Technology, 2019, 68(5):5087-5099.
[13] ZHAN W H, LUO C B, WANG J, et al.Deep-reinforcement-learning-based offloading scheduling for vehicular edge computing[J].IEEE Internet of Things Journal, 2020, 7(6):5449-5465.
[14] GUO H Z, ZHANG J, LIU J J.WiFi-enhanced vehicular edge computing networks:collaborative task offloading[J].IEEE Vehicular Technology Magazine, 2019, 14(1):45-53.
[15] NI Z C, CHEN H L, LI Z, et al.MSCET:a multi-scenario offloading schedule for biomedical data processing and analysis in cloud-edge-terminal collaborative vehicular networks[EB/OL].[2021-12-02].https://ieeexplore.ieee.org/document/9628037.
[16] CHEN Q L, KUANG Z F, ZHAO L.Multiuser computation offloading and resource allocation for cloud-edge heterogeneous network[J].IEEE Internet of Things Journal, 2022, 9(5):3799-3811.
[17] 赵海涛, 张唐伟, 陈跃, 等.基于DQN的车载边缘网络任务分发卸载算法[J].通信学报, 2020, 41(10):172-178. ZHAO H T, ZHANG T W, CHEN Y, et al.Task distribution offloading algorithm of vehicle edge network based on DQN[J].Journal on Communications, 2020, 41(10):172-178.(in Chinese)
[18] PENG H X, SHEN X M.Deep reinforcement learning based resource management for multi-access edge computing in vehicular networks[J].IEEE Transactions on Network Science and Engineering, 2020, 7(4):2416-2428.
[19] KE H C, WANG J, DENG L Y, et al.Deep reinforcement learning-based adaptive computation offloading for MEC in heterogeneous vehicular networks[J].IEEE Transactions on Vehicular Technology, 2020, 69(7):7916-7929.
[20] ZHAO L, ZHAO W L, HAWBANI A, et al.Novel online sequential learning-based adaptive routing for edge software-defined vehicular networks[J].IEEE Transactions on Wireless Communications, 2021, 20(5):2991-3004.
[21] HE Y, LIANG C C, ZHANG Z, et al.Resource allocation in software-defined and information-centric vehicular networks with mobile edge computing[C]//Proceedings of the 86th Vehicular Technology Conference.Washington D.C., USA:IEEE Press, 2018:1-5.
[22] HOU X W, REN Z Y, WANG J J, et al.Reliable computation offloading for edge-computing-enabled software-defined IoV[J].IEEE Internet of Things Journal, 2020, 7(8):7097-7111.
[23] YE Q, ZHUANG W H, ZHANG S, et al.Dynamic radio resource slicing for a two-tier heterogeneous wireless network[J].IEEE Transactions on Vehicular Technology, 2018, 67(10):9896-9910.
[24] VAN HASSELT H, GUEZ A, SILVER D.Deep reinforcement learning with double Q-learning[J].Intelligent Control and Automation, 2016, 30(1):129-144.

Please choose a citation manager

Content to export