Research on Power Control Based on Improved HSARSA(λ) Algorithm

doi:10.19678/j.issn.1000-3428.0054857

Abstract

Abstract: In a two-layer heterogeneous network,when the pico base station performs load balancing on the network by using the cell range extension technology,the communication of the edge users of the pico cell is greatly interfered by the MBS base station.Therefore,this paper proposes an improved HSARSA(λ) algorithm based on heuristic functions.The RP-ABS sub-frame technology with reduced power reduces the interference of the MBS on the pico edge users while ensuring the communication performance of the macro base station,and uses the HSARSA (λ) algorithm improved by the heuristic function to interact with the environment.The sub-frame density and power level are configured for interference coordination.Simulation results show that compared with the original SARSA and Q-Learning algorithms,the pico edge user throughput is increased by 12% and 40% respectively,and the system user throughput is increased by 10.3% and 20.2% respectively.The communication performance of the edge users of the pico cell is significantly improved.

Key words: heterogeneous network, Cell Range Extension(CRE), load balancing, RP-ABS technology, SARSA learning algorithm

摘要： 在MBS-PBS两层异构网络中，微微基站采用小区范围扩展技术对网络进行负载均衡时，pico小区边缘用户的通信受到MBS基站较大干扰。为此，提出一种基于启发函数的改进HSARSA（λ）算法。采用缩减功率的RP-ABS子帧技术，在保证宏基站自身通信性能的同时减小MBS基站对pico边缘用户的干扰，并运用基于启发函数的改进HSARSA（λ）算法与环境进行交互，以配置RP-ABS子帧密度与功率大小，达到干扰协调的目的。仿真结果表明，改进HSARSA算法与原始SARSA和Q-Learning等算法相比，pico边缘用户吞吐量分别提升12%和40%，系统用户吞吐量分别提升10.3%和20.2%，有效提高了pico边缘用户的通信性能。

关键词: 异构网络, 小区范围扩展, 负载均衡, RP-ABS技术, SARSA学习算法

CLC Number:

TP391

GU Jing, HOU Yongping, ZHANG Yuxuan, ZHANG Xin. Research on Power Control Based on Improved HSARSA(λ) Algorithm[J]. Computer Engineering, 2019, 45(11): 112-120.

谷静, 侯永平, 张雨轩, 张新. 基于改进HSARSA(λ)算法的功率控制研究[J]. 计算机工程, 2019, 45(11): 112-120.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0054857

http://www.ecice06.com/EN/Y2019/V45/I11/112

Figures/Tables 11

References

[1] 戴佳浩.多跳D2D蜂窝网络的资源分配及有限区域的物理层安全研究[D].西安:西安电子科技大学,2017.
[2] KIZILIRMAK R C,NARMANLIOGLU O,UYSAL M.Centralized light access network(C-LiAN):a novel paradigm for next generation indoor VLC networks[J].IEEE Access,2017,99(5):19703-19710.
[3] 马礼,张涛,马东超,等.基于Markov模型的接入网络选择算法[J].计算机工程,2019,45(5):105-109,115.
[4] WANG Xueting,ZHU Qi.Fairness based macro base station power control algorithm in heterogeneous cellular networks[C]//Proceedings of International Conference on Computational Intelligence and Communication Networks.Washington D.C.,USA:IEEE Press,2016:82-86.
[5] 全震.双层异构无线网络中资源管理关键技术研究[D].兰州:兰州交通大学,2017.
[6] MONEMI M,RASTI M,HOSSAIN E.On joint power and admission control in underlay cellular cognitive radio networks[J].IEEE Transactions on Wireless Communications,2015,14(1):265-278.
[7] YAO Kailing,WU Qijui,XU Yuhua,et al.Distributed ABS-slot access in dense heterogeneous networks? a potential game approach with generalized interference model[J].IEEE Access,2017,99(5):94-104.
[8] LI Peng,CHEN Jianxin,WANG Jian,et al.Systems pharmacology strategies for drug discovery and combination with applications to CVD[J].Journal of Ethnopharmacology,2014,151(1):93-107.
[9] VASUDEVAN S,PUPAL A,et al.Dynamic eICIC——a proactive strategy for improving spectral efficiencies of heterogeneous LTE cellular networks by leveraging user mobility and traffic dynamics[J].IEEE Transactions on Wireless Communications,2013,12(10):4956-4969.
[10] SHIM V A,RANJIT C S N,TIAN B,et al.A simplified cerebellar model with priority-based delayed eligibility trace learning for motor control[J].IEEE Transactions on Autonomous Mental Development,2015,7(1):26-38.
[11] 宋佳佳.部分观测马尔科夫决策过程中基于记忆的强化学习问题研究[D].天津:天津工业大学,2017.
[12] WILLIAMS R J,PENG Jing.Function optimization using connectionist reinforcement learning algorithms[J].Connection Science,2012,3(3):241-268.
[13] ZHANG Rubo,TANG Pingpeng,SU Yumin,et al.An adaptive obstacle avoidance algorithm for unmanned surface vehicle in complicated marine environments[J].IEEE/CAA Journal of Automatica Sinica,2014,1(4):385-396.
[14] SUTTON R S,BARTO A G.Reinforcement learning:an introduction[J].IEEE Transactions on Neural Networks,1998,9(5):1054-1054.
[15] 郑宇.分层强化学习算法及其应用研究[D].北京:北京交通大学,2009.
[16] 柴旭清,孙丽娜.基于量子粒子群和SARSA算法的蜂窝网络信道分配[J].计算机测量与控制,2015,23(10):3555-3557.
[17] 黄明和.基于Sarsa学习的TD-SCDMA/WLAN异构网络切换算法研究[D].哈尔滨:哈尔滨工业大学,2014.
[18] 赵天呈.基于ABS的干扰协调方案研究[D].南京:南京邮电大学,2017.
[19] HU Haonan,ZHANG Baoling,HONG Qi,et al.Coverage analysis of reduced power subframes applied in heterogeneous networks with subframe misalignment interference[J].IEEE Wireless Communications Letters,2018(99):1.
[20] 余林浩.异构网络时域干扰协调与资源分配的研究[D].合肥:中国科学技术大学,2015.
[21] GAO Liqi,TIAN Hui,TIAN Peng,et al.A distributed dynamic ABS ratio setting scheme for macro-femto heterogeneous networks[C]//Proceedings of IEEE International Conference on Communications Workshops.Washington D.C.,USA:IEEE Press,2013:1221-1225.
[22] 周昊.LTE-A异构网络中自适应ABS配置与功率控制算法的研究[D].北京:北京邮电大学,2015.

Please choose a citation manager

Content to export