摘要： 航天伺服系统因其特殊的工作环境，在使用永磁同步电机驱动具有高阶非线性运动特性的负载时，其传感器仅能测得电机角度反馈而无法测得负载位置反馈。在上述工作场景中，不准确的负载位置反馈让传统的基于闭环反馈的控制算法，如PID三环控制，表现出了跟踪精度低、指令适应能力不足的问题。为了解决上述问题，使用双延迟深度确定性策略梯度算法训练强化学习智能体，其对位置环中与负载位置近似的电机位置反馈进行调优，以克服半闭环带来的精度损失，增强控制器在多项任务下的控制性能。同时，将智能体的策略模型轻量化部署至TMS320C6713B DSP上并验证其运行的实时性。实验结果表明，基于深度强化学习的调优方案相较于对比控制方案在负载位置特性方面有2.07%的提升，在负载速度特性方面有59%的提升；在负载频率特性试验方面普遍优于对比控制方案，并且能够部署在算力有限的边缘控制器上实现实时控制。

Abstract: The aerospace servo system, due to its unique operating environment, faces challenges when driving loads with high-order nonlinear motion characteristics using permanent magnet synchronous motors. In the above scenario, inaccurate load position feedbacks cause traditional closed-loop feedback based control algorithms, such as PID three-loop control, to exhibit problems of low tracking accuracy and insufficient command adaptability. To address these issues, a dual-delay deep deterministic policy gradient algorithm is employed to train a reinforcement learning agent. This agent fine-tunes the motor position feedback approximating the load position in the position loop to overcome the accuracy loss caused by the semi-closed-loop and enhance the controller's performance across multiple tasks. Simultaneously, the policy model of the agent is lightweighted and deployed on TMS320C6713B DSP to verify its real-time operation. Experimental results show that the proposed optimization based on deep reinforcement learning has a 2.07% improvement in load position testing and a 59% improvement in speed testing compared to the comparative control algorithms; In terms of load frequency characteristic testing, it generally outperforms the comparative control scheme and can be deployed on edge controllers with limited computing power to achieve real-time control.