摘要：

交通信号灯控制对缓解交通拥堵、提升城市通勤效率有着重要作用。近年来，以实时交通数据为输入的基于深度强化学习的信号灯控制算法已取得突破性进展。然而，现实场景中的交通数据通常伴随着数据失真。传统方法在修复失真数据后使用强化学习算法控制信号灯，但一方面信号灯相位的动态性给失真修复引入了额外不确定性，另一方面失真修复难以与深度强化学习框架相结合来提升性能。为此，提出基于隐状态预测的失真交通信号灯路口控制模型HCRL。HCRL模型由编码子模型、控制子模型和编码预测子模型组成，通过引入信号灯路口的隐状态表示机制，更好地适应深度强化学习框架，有效表达信号灯路口的控制状态，并使用特殊的迁移训练方法避免数据失真对控制子模型的干扰。使用两个真实数据集验证了数据失真对智能信号灯控制算法的影响。实验结果表明，HCRL模型在所有失真场景和失真率下均优于基于失真修复的信号灯控制模型，并在与其他基线模型的对比中表现出了对数据失真更强的鲁棒性。

关键词: 交通信号灯控制, 智能交通, 深度强化学习, 隐状态, 数据失真

Abstract:

Traffic signal control plays an important role in alleviating traffic congestion and improving urban commuting efficiency. In recent years, breakthroughs have been made in traffic signal control algorithms based on deep reinforcement learning using real-time traffic data as input. However, traffic data in real-world scenarios often involve data distortion. Traditional solutions use reinforcement learning algorithms to control signal lights after repairing distorted data. However, on the one hand, the dynamic phases of traffic signal introduces additional uncertainty to distortion repair, and on the other hand, distortion repair is difficult to combine with deep reinforcement learning frameworks to improve performance. To address these issues, a distorted traffic signal control model based on hidden state prediction, HCRL, is proposed. The HCRL model comprises encoding, control, and encoding prediction sub-models. By introducing a hidden state representation mechanism for signalized intersections, the HCRL model can adapt better to deep reinforcement learning frameworks and effectively express the control state of signalized intersections. In addition, the HCRL model uses a special transfer training method to avoid data distortion interference in the control sub-model. Two real datasets are used to verify the impact of data distortion on the intelligent signal light control algorithms. The experimental results show that the HCRL model outperforms the distortion-completion-based traffic signal control models in all distortion scenarios and distortion rates; further, it demonstrates strong robustness against data distortion when compared with other baseline models.

Key words: traffic signal control, intelligent transportation, deep reinforcement learning, hidden state, data distortion