基于MADDPG的多阵面相控阵雷达引导搜索资源优化算法

doi:10.19678/j.issn.1000-3428.0069838

摘要/Abstract

摘要：

针对传统单阵面雷达搜索资源优化算法在复杂多阵面场景下的参数求解困难问题, 提出一种基于多智能体深度确定性策略梯度(MADDPG)的多阵面雷达搜索资源优化算法。考虑多阵面相控阵雷达场景约束, 结合机载雷达实际搜索任务需求, 建立基于最大目标平均积累期望发现概率的多阵面雷达搜索资源优化模型。分别设计多智能体局部及全局观测空间和带折扣因子的复合奖励函数, 基于执行者-评论者(Actor-Critic)算法结构, 通过各智能体策略网络在线更新各雷达阵面搜索资源分配系数实现上述模型参数的优化求解。仿真结果表明, 该算法能够根据空域-目标覆盖情况及各目标威胁权系数迅速作出精确的自主决策, 在多阵面相控阵雷达搜索资源优化场景下的表现显著优于传统算法。

关键词: 多阵面相控阵雷达, 雷达搜索资源优化, 多智能体深度强化学习, 深度确定性策略梯度, 集群目标雷达引导搜索

Abstract:

An optimization algorithm for multi-antenna phased array radar search parameters based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is proposed to address the problem that search parameters of the traditional single-antenna phased array radar optimization model are difficult to solve for in complex multi-antenna phased array scenarios. First, considering the constraints of a multi-antenna phased array scenario and the actual search task requirements of airborne radar, an optimization model is established for multi-antenna radar search parameters based on the maximum accumulated discovery probability of cluster targets. Second, the local and global observation spaces of multi-agents and composite reward functions with discount factors are designed to update the allocation coefficients of search resources online for each antenna phased array and each agent strategy network based on the Actor-Critic structure. Finally, the simulation results show that the trained multi-agents of the proposed algorithm can quickly make accurate autonomous decisions based on the target airspace set-covering model and target guidance information. The performance of the proposed algorithm is significantly better than that of the traditional algorithm in the multi-antenna phased array scenario.

Key words: multi-antenna phased array radar, radar search resource optimization, multi-agent deep reinforcement learning, Deep Deterministic Policy Gradient (DDPG), radar-guided search for cluster target

王腾, 黄俊松, 王乐庭, 张才坤, 李枭扬. 基于MADDPG的多阵面相控阵雷达引导搜索资源优化算法[J]. 计算机工程, 2024, 50(11): 38-48.

WANG Teng, HUANG Junsong, WANG Leting, ZHANG Caikun, LI Xiaoyang. Multi-Antenna Phased Array Radar-Guided Search Resource Optimization Algorithm Based on MADDPG[J]. Computer Engineering, 2024, 50(11): 38-48.

https://www.ecice06.com/CN/Y2024/V50/I11/38

图/表 13

图1 多阵面相控阵雷达搜索资源优化过程

Fig.1 Optimization process for search resources of the multi-antenna phased array radar

图2 基于MADDPG的多阵面相控阵雷达智能体搜索资源优化算法架构

Fig.2 Architecture of agent search resource optimization algorithm of the multi-antenna phased array radar based on MADDPG

图3 多阵面相控阵雷达搜索资源优化多智能体环境示意图

Fig.3 Schematic diagram of multi-agent environment for search resource optimization of the multi-antenna phased array radar

图4 三维搜索空域及二维搜索波位示意图

Fig.4 Schematic diagram of 3D search airspace and 2D search beam position

图5 各算法奖励变化

Fig.5 Change of rewards of various algorithms

图6 各算法策略在不同搜索资源占比下的优化效果

Fig.6 Optimization effects of various algorithm strategies under different proportions of search resources

图7 雷达阵面数为动态时的雷达搜索资源优化场景

Fig.7 Optimization scenarios of radar search resources when the number of radar arrays is dynamic

参考文献 27

1	LU D J, WANG X, WU X T, et al. Adaptive allocation strategy for cooperatively jamming netted radar system based on improved cuckoo search algorithm. Defence Technology, 2023, 24, 285- 297. doi: 10.1016/j.dt.2022.04.013
2	LI Z J, XIE J W, LIU W J, et al. Pareto-based bi-objective optimization for robust power allocation in hybrid MIMO phased-array radar system under air defense maneuvering tracking. Expert Systems with Applications, 2024, 235, 121248. doi: 10.1016/j.eswa.2023.121248
3	HAO G Q, FENG D J, WANG J J, et al. Phased array radar resource consumption method based on phase-switched screen. Electronics, 2023, 12(18): 3750. doi: 10.3390/electronics12183750
4	袁志, 王益民. 基于VxWorks的多阵面雷达控制系统的设计和实现. 现代雷达, 2014, 36(8): 14-16, 21.
	YUAN Z, WANG Y M. Design and realization of radar control system with multi-antenna array based on VxWorks. Modern Radar, 2014, 36(8): 14-16, 21.
5	JIAN J W, HUANG B, WANG W Q. Physical-layer security with frequency diverse array for DF multi-antenna relaying SWIPT system. International Journal of Electronics Letters, 2023, 11(3): 291- 299. doi: 10.1080/21681724.2022.2087911
6	TRUJILLO-ROMERO C J, MERIDA J D, RAMÍREZ-GUZMÁN T J, et al. Thermal evaluation of multi-antenna systems proposed to treat bone tumors: finite element analysis. Sensors, 2022, 22(19): 7604. doi: 10.3390/s22197604
7	QIAN C X, ZHANG X B, LI L, et al. H3E: Learning air combat with a three-level hierarchical framework embedding expert knowledge. Expert Systems with Applications, 2024, 245, 123084. doi: 10.1016/j.eswa.2023.123084
8	ZHANG H W, LIU W J, YANG X. Resource saving based dwell time allocation and detection threshold optimization in an asynchronous distributed phased array radar network. Chinese Journal of Aeronautics, 2023, 36(11): 311- 327. doi: 10.1016/j.cja.2023.06.017
9	SHI C G, DONG J, SALOUS S, et al. Collaborative trajectory planning and resource allocation for multi-target tracking in airborne radar networks under spectral coexistence. Remote Sensing, 2023, 15(13): 3386. doi: 10.3390/rs15133386
10	ZHANG W W, SHI C G, ZHOU J J, et al. Joint aperture and transmit resource allocation strategy for multi-target localization in phased array radar network[EB/OL]. [2024-04-17]. https://www.semanticscholar.org/paper/Joint-Aperture-and-Transmit-Resource-Allocation-for-Zhang-Shi/c5c26ab296581093509ef99dba233074e4335ba5.
11	张华睿, 杨宏文, 郁文贤. 相控阵雷达最优搜索参数设计研究. 兵工学报, 2012, 33(9): 1062- 1065.
	ZHANG H R, YANG H W, YU W X. Design of optimal search operation parameters for phased array radar. Acta Armamentarii, 2012, 33(9): 1062- 1065.
12	吴其华, 刘进, 艾小锋, 等. 反导任务下一种相控阵雷达搜索参数优化方法. 现代防御技术, 2016, 44(2): 165- 170.
	WU Q H, LIU J, AI X F, et al. Method of search parameters optimization of phased array radar for antimissile mission. Modern Defence Technology, 2016, 44(2): 165- 170.
13	GUPTA S, SINGH N. Toward intelligent resource management in dynamic fog computing-based internet of things environment with deep reinforcement learning: a survey. International Journal of Communication Systems, 2023, 36(4): 1- 10.
14	WANG X, WANG S, LIANG X X, et al. Deep reinforcement learning: a survey. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(4): 5064- 5078. doi: 10.1109/TNNLS.2022.3207346
15	FENG C, FU X J, WANG Z Y, et al. An optimization method for collaborative radar antijamming based on multi-agent reinforcement learning. Remote Sensing, 2023, 15(11): 2893. doi: 10.3390/rs15112893
16	ZHENG Z X, LI W, ZOU K. Airborne radar anti-jamming waveform design based on deep reinforcement learning. Sensors, 2022, 22(22): 8689. doi: 10.3390/s22228689
17	AHMED A M, AHMAD A A, FORTUNATI S, et al. A reinforcement learning based approach for multitarget detection in massive MIMO radar. IEEE Transactions on Aerospace and Electronic Systems, 2021, 57(5): 2622- 2636. doi: 10.1109/TAES.2021.3061809
18	赵锋, 毕莉, 周颖, 等. 基于搜索数据率的弹道导弹跟踪制导雷达最优搜索时序. 电子学报, 2009, 37(12): 2626- 2632.
	ZHAO F, BI L, ZHOU Y, et al. Optimal search order of tracking and guiding radars in ballistic missile defense based on search data rate. Acta Electronica Sinica, 2009, 37(12): 2626- 2632.
19	BEN ABDELAZIZ F, MIR H. An optimization model and tabu search heuristic for scheduling of tasks on a radar sensor. IEEE Sensors Journal, 2016, 16(17): 6694- 6702. doi: 10.1109/JSEN.2016.2587730
20	陈建勇, 王健, 单志超. 离散时间探测随机恒速目标的最优搜索算法. 系统工程与电子技术, 2013, 35(8): 1627- 1630.
	CHEN J Y, WANG J, SHAN Z C. Optimal search algorithm for constant velocity random moving target detecting in discrete time. Systems Engineering and Electronics, 2013, 35(8): 1627- 1630.
21	XIAO L Y, XIE Y J, GAO S D, et al. Generalized radar range equation applied to the whole field region. Sensors, 2022, 22(12): 4608. doi: 10.3390/s22124608
22	张永杰, 李少洪, 朱海冰. 多功能相控阵雷达优化搜索建模与仿真研究. 系统仿真学报, 2008, 20(16): 4248- 4251.
	ZHANG Y J, LI S H, ZHU H B. Research on modeling and simulation of optimal search of multifunction phased array radar. Journal of System Simulation, 2008, 20(16): 4248- 4251.
23	LOWE R, WU Y, TAMAR A, et al. Multi-agent actor-critic for mixed cooperative-competitive environments[EB/OL]. [2024-04-17]. http://arxiv.org/abs/1706.02275v4.
24	SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[EB/OL]. [2024-04-17]. https://arxiv.org/abs/1707.06347.
25	HAARNOJA T, ZHOU A, HARTIKAINEN K, et al. Soft actor-critic algorithms and applications[EB/OL]. [2024-04-17]. https://arxiv.org/abs/1812.05905.
26	FUJIMOTO S, VAN HOOF H, MEGER D. Addressing function approximation error in actor-critic methods[EB/OL]. [2024-04-17]. http://arxiv.org/abs/1802.09477v3.
27	LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[J]. Journal of Machine Learning Research, 17(1): 834-839.

[1]	赵寅甫, 冯正勇. 基于深度强化学习的机械臂控制快速训练方法[J]. 计算机工程, 2022, 48(8): 113-120.
[2]	吴思凡, 杜煜, 徐世杰, 杨硕, 杜晨. 基于深度确定性策略梯度的智能车汇流模型[J]. 计算机工程, 2020, 46(1): 87-92.

选择文件类型/文献管理软件名称

选择包含的内容