基于RDC-Q学习算法的移动机器人路径规划

doi:10.3969/j.issn.1000-3428.2014.06.045

摘要/Abstract

摘要： 传统Q算法对于机器人回报函数的定义较为宽泛，导致机器人的学习效率不高。为解决该问题，给出一种回报详细分类Q(RDC-Q)学习算法。综合机器人各个传感器的返回值，依据机器人距离障碍物的远近把机器人的状态划分为20个奖励状态和 15个惩罚状态，对机器人每个时刻所获得的回报值按其状态的安全等级分类，使机器人趋向于安全等级更高的状态，从而帮助机器人更快更好地学习。通过在一个障碍物密集的环境中进行仿真实验，证明该算法收敛速度相对传统回报Q算法有明显提高。

关键词: 路径规划, 移动机器人, 强化学习, Q学习算法, 回报函数, 学习效率

Abstract: Reward function is always simple in traditional Q-learning algorithm, which makes a low learning efficiency. To solve this problem, a Reward Detailed Classification Q(RDC-Q) learning algorithm is proposed. It synthesizes all sensors’ value of the robot and divides the robot’s states into 20 damp states and 15 reward states according to the distance between the robot and obstacles. The reward value that a robot gets at each time step is classified by the robot’s security level, which makes the robot go towards the states in higher security levels, so the robot can learn quicker and better. By simulating the new algorithm in an environment of dense barriers, it is proved that convergence speed of the new method is obviously improved than traditional reward Q methods.

Key words: path planning, mobile robot, reinforcement learning, Q-learning algorithm, reward function, learning efficiency

中图分类号:

TP301.6

王子强，武继刚. 基于RDC-Q学习算法的移动机器人路径规划[J]. 计算机工程, doi: 10.3969/j.issn.1000-3428.2014.06.045.

WANG Zi-qiang, WU Ji-gang. Mobile Robot Path Planning Based on RDC-Q Learning Algorithm[J]. Computer Engineering, doi: 10.3969/j.issn.1000-3428.2014.06.045.

参考文献

参考文献 [1] Chen Chunlin, Yang Pei, Zhou Xianzhong. A Quantum Inspired Q-learning Algorithm for Indoor Robot Navigation[C]//Proc. of IEEE International Conference on Networking, Sensing and Control. [S. l.]: IEEE Press, 2008: 1599-1603. [2] Niu Lianqiang, Li Ling. Application of Reinforcement Learn- ing in Autonomous Navigation for Virtual Vehicles[C]//Proc. of the 9th International Conference on Hybrid Intelligent Systems. [S. l.]: IEEE Press, 2009: 30-32. [3] 高阳, 陈世福, 陆鑫. 强化学习研究综述[J]. 自动化学报, 2004, 30(1): 89-100. [4] Adiprawita W, Ahmad A S, Sembiring J, et al. Simplified Q-learning for Holonomic Mobile Robot Navigation[C]//Proc. of the 2nd International Conference on Instrumentation, Communications, Information Technology, and Biomedical Engineering. [S. l.]: IEEE Press, 2011: 64-68. [5] Gordon S W, Reyes N H, Barczak A. A Hybrid Fuzzy Q- learning Algorithm for Robot Navigation[C]//Proc. of International Joint Conference on Neural Networks. [S. l.]: IEEE Press, 2011: 2625-2631. [6] Sepideh V, Reza G, Ataollah E. A Fuzzy Q-learning Approach to Navigation of an Autonomous Robot[C]//Proc. of the 16th International Symposium on Artificial Intelligence and Signal Processing. [S. l.]: IEEE Press, 2012: 520-525. [7] 梁泉. 未知环境中基于强化学习的移动机器人路径规划[J]. 机电工程, 2012, 29(4): 477-481. [8] Yang Guosheng, Chen Erkui, An Chengwan. Mobile Robot Navigation Using Neural Q-learning[C]//Proc. of International Conference on Machine Learning and Cybernetics. [S. l.]: IEEE Press, 2004: 48-52. [9] 陈自立, 徐娅萍, 顾立彬. 基于模糊Q学习算法的AGV路径规划研究[J]. 制造业自动化, 2012, 34(6): 4-6. [10] Watkins C, Dayan P. Q-learning[J]. Machine Learning, 1992, 8(3/4): 279-292. [11] Costa E D S, Gouvea M M. Autonomous Navigation in Dynamic Environments with Reinforcement Learning and Heuristic[C]//Proc. of the 9th International Conference on Machine Learning and Applications. Washington D. C., USA: [s. n.], 2010: 12-14. [12] Martinez M T, Rodriguez R. Navigation of Autonomous Vehicles in Unknown Environments Using Reinforcement Learning[C]//Proc. of IEEE Intelligent Vehicles Symposium. [S. l.]: IEEE Press, 2007: 872-876. 编辑顾逸斐

[1]	杨奔, 王炜晔, 赵婉婷, 谢瑾奎. 基于DQN的动态深度多分支搜索自动配载算法[J]. 计算机工程, 2020, 46(8): 313-320.
[2]	陈建平, 周鑫, 傅启明, 高振, 付保川, 吴宏杰. 基于二阶时序差分误差的双网络DQN算法[J]. 计算机工程, 2020, 46(5): 78-85,93.
[3]	陈恒, 韩雨婷, 李冠宇, 王京徽. 基于多级关系路径语义组合的关系推理算法[J]. 计算机工程, 2020, 46(5): 109-114.
[4]	谷静, 邓逸飞, 张新. 基于启发式强化学习的动态CRE偏置选择算法[J]. 计算机工程, 2020, 46(5): 200-206.
[5]	唐素勤, 孙亚茹, 李志欣, 张灿龙. 基于强化学习的壮语词性标注[J]. 计算机工程, 2020, 46(4): 309-315.
[6]	刘庆周, 吴锋. 多智能体路径规划研究进展[J]. 计算机工程, 2020, 46(4): 1-10.
[7]	王毅然, 经小川, 贾福凯, 孙宇健, 佟轶. 基于多智能体协同强化学习的多目标追踪方法[J]. 计算机工程, 2020, 46(11): 90-96.
[8]	王若愚, 陈勇全. 基于强化学习的旅行商问题解构造方法[J]. 计算机工程, 2020, 46(11): 293-300.
[9]	徐西建,王子磊,奚宏生. 基于深度强化学习的流媒体边缘云会话调度策略[J]. 计算机工程, 2019, 45(5): 237-242,248.
[10]	裴以建,杨超杰,杨亮亮. 基于改进RRT*的移动机器人路径规划算法[J]. 计算机工程, 2019, 45(5): 285-290,297.
[11]	胡章芳,孙林,张毅,鲍合章. 一种基于改进QPSO的机器人路径规划算法[J]. 计算机工程, 2019, 45(4): 281-287.
[12]	李军锋,李逃昌,彭继慎. 农业机器人视觉导航路径识别方法研究[J]. 计算机工程, 2018, 44(9): 38-44,58.
[13]	阳钧,鲍泓,梁军,马楠. 一种基于高精度地图的路径跟踪方法[J]. 计算机工程, 2018, 44(7): 8-13.
[14]	乔良,鲍泓,玄祖兴,梁军,潘峰. 基于强化学习的无人驾驶匝道汇入模型[J]. 计算机工程, 2018, 44(7): 20-24,31.
[15]	李飞琦,鲍泓,潘峰,潘卫国,吴桃. 智能车导航中的路口轨迹生成策略[J]. 计算机工程, 2018, 44(7): 25-31.