毫米波网络中基于Q-Learning的阻塞感知功率分配

doi:10.19678/j.issn.1000-3428.0056421

摘要/Abstract

摘要： 毫米波通信可在5G无线通信系统超密集网络场景中提供显著的系统容量增益，但毫米波通信场景中干扰复杂多变，并且小区边缘用户动态链路的高阻塞率会引起中断问题。为此，基于Q-Learning算法，提出一种考虑毫米波链路高间歇性概率的功率分配方案。基于泊松簇过程对随机部署的基站用户系统进行建模，分析链路阻断对有用信号和干扰信号带来的不同影响，并将利己利他策略引入Q-Learning算法的状态和回报函数设计中，通过机器学习策略得到功率分配最优解。仿真结果表明，与未考虑链路阻塞概率的CDP-Q方案相比，该方案由于根据链路动态链接状况进行最优功率分配，显著提升了系统总容量。

关键词: 毫米波通信, 链路阻塞, Q-Learning算法, 功率分配, 泊松簇过程, 利己利他策略

Abstract: millimeter-Wave(mm-Wave) communication is expected to provide significant capacity gains in ultra-dense network scenarios of the 5G wireless communication system.To address the complex interference in the mm-Wave communication scenario and the interruption caused by the high block rate of the dynamic links of the cell edge users,this paper proposes a power allocation strategy scheme based on Q-Learning algorithm considering the high intermission rate of mm-Wave communication.Poisson Cluster Process(PCP) is used in the modelling of randomly deployed base station user systems,and the different influences of link block on the useful signals and interference signals are analyzed.Then the egoistic and altruistic strategy is introduced in the design of state and reward function of the Q-Learning algorithm,and the machine learning strategy is used to get the optimal solution to power allocation.Simulation results show that,compared with the CDP-Q scheme that does not consider the link block rate,the proposed algorithm significantly improves the total capacity of the system due to the optimal power allocation based on the dynamic status of links.

Key words: millimeter-Wave(mm-Wave) communication, link block, Q-Learning algorithm, power allocation, Poisson Cluster Process(PCP), egoistic and altruistic strategy

中图分类号:

TN929.5

施钊, 孙长印, 江帆. 毫米波网络中基于Q-Learning的阻塞感知功率分配[J]. 计算机工程, 2020, 46(12): 185-192.

SHI Zhao, SUN Changyin, JIANG Fan. Block-aware Power Allocation Based on Q-Learning in Millimeter-Wave Network[J]. Computer Engineering, 2020, 46(12): 185-192.

https://www.ecice06.com/CN/Y2020/V46/I12/185

图/表 6

20201216141806

20201216141810

20201216141813

20201216141816

20201216141819

20201216141822

参考文献

[1] CHEN Liang,YANG Qi.Analysis of spectrum occupation and perspectives of 5G network[J].Study on Optical Communications,2016,42(6):68-69.(in Chinese)陈亮,杨奇.5G网络中无线频谱资源分配的进展分析[J].光通信研究,2016,42(6):68-69.
[2] BOCCARDI F,HEATH R W,LOZANO A,et al.Five disruptive technology directions for 5G[J].IEEE Communications Magazine,2014,52(2):74-78.
[3] RANGAN S,RAPPAPORT T S,ERKIP E.Millimeter-wave cellular wireless networks:potentials and challenges[J].Proceedings of the IEEE,2014,102(3):366-373.
[4] GHOSH A,THOMAS T A,CUDAK M C,et al.Millimeter-wave enhanced local area systems:a high-data-rate approach for future wireless networks[J].IEEE Journal on Selected Areas in Communications,2014,32(6):1152-1163.
[5] AGRAWAL S K,SHARMA K.5G millimeter Wave(mmWave) communications[C]//Proceedings of the 3rd International Conference on Computing for Sustainable Global Development.Washington D.C.,USA:IEEE Press,2016:3630-3634.
[6] BALDEMAIR R,IRNICH T,BALACHANDRAN K,et al.Ultra-dense networks in millimeter-wave frequencies[J].IEEE Communications Magazine,2015,53(1):202-205.
[7] BAI T,HEATH R W.Coverage in dense millimeter wave cellular networks[C]//Proceedings of Asilomar Conference on Signals,Systems and Computers.Washington D.C.,USA:IEEE Press,2013:2062-2066.
[8] AMIRI R,MEHRPOUYAN H.Self-organizing mm-wave networks:a power allocation scheme based on machine learning[C]//Proceedings of the 11th Global Symposium on Millimeter Waves.Boulder,USA:GSMM Press,2018:1-4.
[9] CHEN Ming,HUA Ye,GU Xinyu,et al.A self-organizing resource allocation strategy based on Q-learning approach in ultra-dense networks[C]//Proceedings of IEEE International Conference on Network Infrastructure and Digital Content.Washington D.C.,USA:IEEE Press,2016:155-160.
[10] SAAD H,MOHAMED A,ELBATT T.Distributed cooperative Q-learning for power allocation in cognitive femtocell networks[C]//Proceedings of IEEE Vehicular Technology Conference.Washington D.C.,USA:IEEE Press,2012:1-5.
[11] WEN Bin,GAO Zhibin,HUANG Lianfen,et al.A Q-learning-based downlink resource scheduling method for capacity optimization in LTE femtocells[C]//Proceedings of the 9th International Conference on Computer Science and Education.Washington D.C.,USA:IEEE Press,2014:625-628.
[12] KWON S,WIDMER J.Relay selection for mmWave communications[C]//Proceedings of the 28th Annual IEEE International Symposium on Personal,Indoor,and Mobile Radio Communications.Washington D.C.,USA:IEEE Press,2017:1-6.
[13] HE Zhifeng,MAO Shiwen.A decomposition principle for link and relay selection in dual-hop 60 GHz networks[C]//Proceedings of the 35th Annual IEEE International Conference on Computer Communications.Washington D.C.,USA:IEEE Press,2016:1-9.
[14] TATINO C,MALANCHINI I,AZIZ D,et al.Beam based stochastic model of the coverage probability in 5G millimeter wave systems[C]//Proceedings of the 15th International Symposium on Modeling and Optimization in Mobile,Ad Hoc,and Wireless Networks.Washington D.C.,USA:IEEE Press,2017:1-6.
[15] SUN Changyin,LEI Yuan,JIANG Fan,et al.Beamforming for virtual cell with balanced interference[J].Journal of Xi'an University of Posts and Telecommunications,2017,22(1):18-21.(in Chinese)孙长印,雷远,江帆,等.虚拟小区干扰均衡的波束形成算法[J].西安邮电大学学报,2017,22(1):18-21.
[16] WATKINS C J C H,DAYAN P.Technical note:Q-learning[J].Journal of Machine Learning,1992,8(3):279-292.
[17] GAO Zhenhai,SUN Tianjun,HE Lei.Causal reasoning decision-making for vehicle longitudinal automatic driving[J].Journal of Jilin University(Engineering and Technology Edition),2019,49(5):1392-1404.(in Chinese)高振海,孙天骏,何磊.汽车纵向自动驾驶的因果推理型决策[J].吉林大学学报(工学版),2019,49(5):1392-1404.
[18] WANG Xiaolei,CHEN Yunjie,WANG Chen,et al.Scheduling method of virtual network function based on Q-learning[J].Computer Engineering,2019,45(2):66-67.(in Chinese)王晓雷,陈云杰,王琛,等.基于Q-learning的虚拟网络功能调度方法[J].计算机工程,2019,45(2):66-67.
[19] SHANG Sihan.Research of interference management techniques in ultra dense networks[D].Chengdu:University of Electronic Science and Technology of China,2018.(in Chinese)尚思翰.超密集网络中的干扰管理技术研究[D].成都:电子科技大学,2018.
[20] ASL Z D,DERHAMI V,YAZDIAN-DEHKORDI M.A new approach on multi-agent multi-objective reinforcement learning based on agents' preferences[C]//Proceedings of Artificial Intelligence and Signal Processing Conference.Washington D.C.,USA:IEEE Press,2017:75-77.
[21] CHUN Y J,HASNA M O.Analysis of heterogeneous cellular networks interference with biased cell association using Poisson cluster processes[C]//Proceedings of International Conference on Information and Communication Technology Convergence.Washington D.C.,USA:IEEE Press,2014:319-322.
[22] MA Zhonggui,LIU Liyu,YAN Wenbo,et al.Deployment model of three-layer heterogeneous cellular networks based on Poisson clustered process[J].Chinese Journal of Engineering,2017,39(2):309-311.(in Chinese)马忠贵,刘立宇,闫文博,等.基于泊松簇过程的三层异构蜂窝网络部署模型[J].工程科学学报,2017,39(2):309-311.
[23] ALONZO M,BUZZI S.Cell-free and user-centric massive MIMO at millimeter wave frequencies[C]//Proceedings of the 28th Annual IEEE International Symposium on Personal,Indoor,and Mobile Radio Communications.Washington D.C.,USA:IEEE Press,2017:1-2.
[24] GAPEYENKO M,PETROV V,MOLTCHANOV D,et al.On the degree of multi-connectivity in 5G millimeter-wave cellular urban deployments[J].IEEE Transactions on Vehicular Technology,2019,68(2):1973-1974.

选择文件类型/文献管理软件名称

选择包含的内容