基于DDPG的边缘计算任务卸载和服务缓存算法

doi:10.19678/j.issn.1000-3428.0061371

摘要/Abstract

摘要： 当计算任务被转移到移动边缘计算（MEC）服务器上时，通过服务缓存能够降低获取和初始化服务应用程序的实时时延和带宽成本。此外，体验质量是驱动卸载决策的关键因素，有效利用有限的计算资源能够提升用户满意度。考虑一个边缘服务器帮助移动用户执行一系列计算任务的场景，建立混合整数非线性规划问题，提出一种基于深度确定性策略梯度（DDPG）的算法来联合优化服务缓存位置、计算卸载决策和资源分配，从而提高用户对服务的体验质量，最大化用户使用计算资源所节约的成本。仿真结果表明，该算法在提高用户体验质量和节约成本方面较使用无缓存策略、随机选择策略和无缓存随机选择策略的算法性能更优。

关键词: 移动边缘计算, 深度强化学习, 任务卸载, 服务缓存, 资源分配

Abstract: When computing tasks are transferred to the Mobile Edge Computing(MEC) servers, the service cache effectively reduces the real-time delay and bandwidth cost of acquiring and initializing the service application.In addition, Quality of Experience(QoE) is a key factor driving offload decisions, and effective utilization of limited computing resources can keep users satisfied.This paper considers a scenario where a single edge server is used to help mobile users perform a series of computing tasks.On this basis, a Mixed Integer Nonlinear Programming(MINLP) is established, and a Deep Deterministic Policy Gradient(DDPG) algorithm is proposed to jointly optimize the service cache location, the offload decision and the resource allocation, so as to improve the user's QoE of services and maximize the cost saved by users using computing resources.Simulation results show that the proposed method achieves higher QoE and lower cost than the algorithms using non-cache strategy, random-choice strategy and non-cache random-choice strategy.

Key words: Mobile Edge Computing (MEC), deep reinforcement learning, task offloading, service caching, resource allocation

中图分类号:

TP391

陈清林, 邝祝芳. 基于DDPG的边缘计算任务卸载和服务缓存算法[J]. 计算机工程, 2021, 47(10): 26-33.

CHEN Qinglin, KUANG Zhufang. Task Offloading and Service Caching Algorithm Based on DDPG in Edge Computing[J]. Computer Engineering, 2021, 47(10): 26-33.

https://www.ecice06.com/CN/Y2021/V47/I10/26

图/表 9

20211015185932

20211015185935

20211015185938

20211015185942

20211015185945

20211015185948

20211015185952

20211015185955

20211015185959

参考文献

[1] HASABELNABY M A, SELMY H, DESSOUKY M I.Joint optimal transceiver placement and resource allocation schemes for redirected cooperative hybrid FSO/mmW 5G fronthaul networks[J].Journal of Optical Communications and Networking, 2018, 10(12):975.
[2] SABELLA D, VAILLANT A, KUURE P, et al.Mobile-edge computing architecture:the role of MEC in the Internet of things[J].IEEE Consumer Electronics Magazine, 2016, 5(4):84-91.
[3] 施巍松, 张星洲, 王一帆, 等.边缘计算:现状与展望[J].计算机研究与发展, 2019, 56(1):69-89. SHI W S, ZHANG X Z, WANG Y F, et al.Edge computing:state-of-the-art and future directions[J].Journal of Computer Research and Development, 2019, 56(1):69-89.(in Chinese)
[4] 谢人超, 廉晓飞, 贾庆民, 等.移动边缘计算卸载技术综述[J].通信学报, 2018, 39(11):142-159. XIE R C, LIAN X F, JIA Q M, et al.Survey on computation offloading in mobile edge computing[J].Journal on Communications, 2018, 39(11):142-159.(in Chinese)
[5] 张鹏, 王博思.基于塑料光纤的智能交通移动边缘计算通信系统设计[J].半导体光电, 2019, 40(6):886-890. ZHANG P, WANG B S.Design of a POF-based communication system for edge computing in intelligent transportation system[J].Semiconductor Optoelectronics, 2019, 40(6):886-890.(in Chinese)
[6] MAO Y, YOU C, ZHANG J, et al.A survey on mobile edge computing:the communication perspective[J].IEEE Communications Surveys & Tutorials, 2017, 19(4):2322-2358.
[7] WANG C, YU F R, LIANG C, et al.Joint computation offloading and interference management in wireless cellular networks with mobile edge computing[J].IEEE Transactions on Vehicular Technology, 2017, 66(8):7432-7445.
[8] ZHANG G, YAN C, SHEN Z, et al.Distributed energy management for multiuser mobile-edge computing systems with energy harvesting devices and QoS constraints[J].IEEE Internet of Things Journal, 2019, 6(3):4035-4048.
[9] LIU Y, PENG M, SHOU G, et al.Towards edge intelligence:multi-access edge computing for 5G and Internet of things[J].IEEE Internet of Things Journal, 2020, 7(8):6722-6747.
[10] 朱丹, 王晓冬.移动网络边缘计算与缓存技术研究[J].铁路计算机应用, 2017, 26(8):56-59. ZHU D, WANG X D.Research on edge computing and caching technology of mobile network[J].Application of Railway Computer, 2017, 26(8):56-59.(in Chinese)
[11] 周知, 于帅, 陈旭.边缘智能:边缘计算与人工智能融合的新范式[J].大数据, 2019, 5(2):53-63. ZHOU Z, YU S, CHEN X.Edge intelligence:a new paradigm of edge computing and artificial intelligence[J].Big Data, 2019, 5(2):53-63.(in Chinese)
[12] YAO J, HAN T, ANSARI N.On mobile edge caching[J].IEEE Communications Surveys & Tutorials, 2019, 21(3):2525-2553.
[13] LYU X, REN C, NI W, et al.Distributed online learning of cooperative caching in edge cloud[J].IEEE Transactions on Mobile Computing, 2021, 20(8):2550-2562.
[14] MAO Y, ZHANG J, LETAIEF K B.Joint task offloading scheduling and transmit power allocation for mobile-edge computing systems[C]//Proceedings of 2017 IEEE Wireless Communications and Networking Conference.Washington D.C., USA:IEEE Press, 2017:1-6.
[15] JANATIAN N, STUPIA I, VANDENDORPE L.Optimal resource allocation in ultra-low power fog-computing SWIPT-based networks[C]//Proceedings of 2018 IEEE Wireless Communications and Networking Conference.Washington D.C., USA:IEEE Press, 2018:1-6.
[16] NING Z, DONG P, KONG X, et al.A cooperative partial computation offloading scheme for mobile edge computing enabled Internet of things[J].IEEE Internet of Things Journal, 2019, 6(3):4804-4814.
[17] QIN M, CHEN L, ZHAO N, et al.Power-constrained edge computing with maximum processing capacity for IoT networks[J].IEEE Internet of Things Journal, 2019, 6(3):4330-4343.
[18] YANG P, LI L, LIANG W, et al.Latency optimization for multi-user NOMA-MEC offloading using reinforcement learning[C]//Proceedings of the 28th Wireless and Optical Communications Conference.Beijing, China:[s.n.], 2019:1-5.
[19] ZHANG Y, ZHOU X, TENG Y, et al.Resource allocation for multi-user MEC system:machine learning approaches[C]//Proceedings of 2018 International Conference on Computational Science and Computational Intelligence.Washington D.C., USA:IEEE Press, 2018:794-799.
[20] ZHOU W, LIN L, LIU J, et al.Joint offloading decision and resource allocation for multiuser NOMA-MEC systems[J].IEEE Access, 2019, 7:181100-181116.
[21] SHENG M, DAI Y, LIU J, et al.Delay-aware computation offloading in NOMA MEC under differentiated uploading delay[J].IEEE Transactions on Wireless Communications, 2020, 19(4):2813-2826.
[22] 杨天, 杨军.MEC中卸载决策与资源分配的深度强化学习方法[J].计算机工程, 2021, 47(8):37-44. YANG T, YANG J.Deep reinforcement learning method of offloading decision and resource allocation in MEC[J].Computer Engineering, 2021, 47(8):37-44.(in Chinese)
[23] ALFAKIH T, HASSAN M M, GUMAEI A, et al.Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA[J].IEEE Access, 2020, 8:54074-54084.
[24] KUANG Z F, LI L F, GAO J, et al.Partial offloading scheduling and power allocation for mobile edge computing systems[J].IEEE Internet of Things Journal, 2019, 6(4):6774-6785.
[25] 余翔, 石雪琴, 刘一勋.移动边缘计算中卸载策略与功率的联合优化[J].计算机工程, 2020, 46(6):20-25. YU X, SHI X Q, LIU Y X.Joint optimization of offloading strategy and power in mobile-edge computing[J].Computer Engineering, 2020, 46(6):20-25.(in Chinese)
[26] SHUAI Y, LANGAR R, FU X, et al.Computation offloading with data caching enhancement for mobile edge computing[J].IEEE Transactions on Vehicular Technology, 2018, 67(11):11098-11112.
[27] ZHOU Y, YU F R, CHEN J, et al.Resource allocation for information-centric virtualized heterogeneous networks with in-network caching and mobile edge computing[J].IEEE Transactions on Vehicular Technology, 2017, 66(12):11339-11351.
[28] PHAM X Q, NGUYEN T D, NGUYEN V D, et al.Joint service caching and task offloading in multi-access edge computing:a QoE-based utility optimization approach[J].IEEE Communications Letters, 2021, 25(3):965-969.

选择文件类型/文献管理软件名称

选择包含的内容