作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2011, Vol. 37 ›› Issue (17): 152-154. doi: 10.3969/j.issn.1000-3428.2011.17.051

• 人工智能及识别技术 • 上一篇    下一篇

基于CMAC强化学习的交叉口信号控制

温凯歌,杨照辉   

  1. (长安大学电子与控制工程学院,西安 710064)
  • 收稿日期:2011-02-23 出版日期:2011-09-05 发布日期:2011-09-05
  • 作者简介:温凯歌(1976-),男,讲师、博士,主研方向:智能交通控制;杨照辉,讲师、硕士
  • 基金资助:
    中央高校基本科研业务费专项基金资助项目(CHD2009JC060)

Intersection Signal Control Based on Reinforcement Learning with CMAC

WEN Kai-ge, YANG zhao-hui   

  1. (School of Electronic and Control Engineering, Chang’an University, Xi’an 710064, China)
  • Received:2011-02-23 Online:2011-09-05 Published:2011-09-05

摘要: 采用神经网络值函数逼近的强化学习方法处理交叉口的信号控制。根据交通流及交叉口信号特征,建立强化学习的状态空间、动作空间和回报空间,以最小化车辆在交叉口的延误为控制目标,对信号进行优化控制。引入小脑模型关节控制器神经网络对强化学习(RL)的Q值进行逼近。在变化的交通条件下,使用典型交叉口对提出的RL模型进行验证,同传统的定时控制和全感应控制进行对比分析。仿真结果表明,RL控制器具有较强的学习能力,可以适应交通流的动态变化,稳定性好、自适应性强,对于环境变化具有较强的适应能力。

关键词: 交通控制, 强化学习, 小脑模型关节控制器, 非均匀量化, 信号交叉口

Abstract: The intersection signal control is disposed with the Reinforcement Learning(RL) method based on the neural network function approximate. Considering the stochastic characteristic of the traffic system, an adaptive RL control scheme, based on Cerebellar Model Articulation Controller(CMAC), is introduced in the traffic signal control systems. Besides, CMAC is introduced to approximate the RL agent Q value. The model is tested on a typical isolated traffic intersection comprised of five four-legged signalized intersections, and compared to full-actuated control and pre-timed control. Analysis of simulation results using this approach shows significant improvement over traditional full-actuated control, especially for the case of accident and over-saturated traffic demand.

Key words: traffic control, reinforcement learning, Cerebellar Model Articulation Controller(CMAC), non-uniform quantization, signal intersection

中图分类号: