基于强化学习算法的软硬结合象鼻抓取器设计

doi:10.19678/j.issn.1000-3428.0252363

摘要/Abstract

摘要： 采用基于近端策略优化（PPO）强化学习算法的运动控制策略，设计并实现一种新型的模块化仿生象鼻抓取器。该抓取器融合了刚性与柔性的设计原则，构建模块化的结构，从而显著增强系统的灵活性和可扩展性。在硬件设计层面，应用刚柔结合的策略，刚性部分保障了结构的强度与稳定性，而柔性部分则致力于适应各种形状和硬度的物体，以实现精准抓取。运用PPO算法对仿生象鼻抓取器进行运动控制，从而模拟象鼻的复杂运动行为，并将训练完成的模型应用于实际抓取任务。在物理仿真环境下，设置多种抓取任务，并通过PPO算法的持续迭代优化进行训练，使得抓取器逐渐学会在各种环境条件下自适应地调整运动轨迹，以实现精确抓取不同的物体。实验结果表明，该模块化仿生象鼻抓取器在多样化抓取任务中具有出色表现，其抓取成功率超过90%，且抓取动作流畅自如。该研究成果不仅验证了PPO算法在解决复杂机器人抓取问题时的有效性，同时也为模块化机器人系统的设计及应用开辟了新途径，未来有望在智能制造、医疗辅助、灾害救援等领域发挥重要作用。

关键词: 深度强化学习, 近端策略优化算法, 仿生象鼻, 模块化机器人, 软体机器人

Abstract: A motion control strategy based on a Proximal Policy Optimization (PPO) reinforcement learning algorithm is employed to design and implement a novel modular bionic elephant trunk gripper. The gripper integrates rigid and flexible design principles with a modular structure, thereby significantly enhancing the flexibility and scalability of the system. A rigid-flexible hybrid strategy is used for hardware design. The rigid components ensure structural strength and stability, whereas the flexible components enable the adaptive gripping of objects of various shapes and hardnesses, enabling precise grasping. The PPO algorithm is used for motion control of the bionic elephant trunk gripper to simulate the complex movement behaviors of an elephant trunk, and the trained model is deployed in actual grasping tasks. In a physics simulation environment, multiple grasping tasks are set up, and the system is trained through a continuous iterative optimization of the PPO algorithm. This enables the gripper to gradually learn to adaptively adjust its motion trajectories under various environmental conditions to achieve precise grasping of different objects. Experimental results demonstrate that the modular bionic elephant trunk gripper achieves outstanding performance across diverse grasping tasks with a grasping success rate exceeding 90% and exhibits smooth natural grasping motions. This research not only validates the effectiveness of the PPO algorithm in addressing complex robotic grasping problems but also opens new avenues for the design and application of modular robotic systems, with promising future applications in intelligent manufacturing, medical assistance, and disaster rescue.

Key words: Deep Reinforcement Learning (DRL), Proximal Policy Optimization (PPO) algorithm, bionic elephant trunk, modular robot, soft robot

中图分类号:

TP181

朱家辰, 杨晔, 胡喜友, 王佳明. 基于强化学习算法的软硬结合象鼻抓取器设计[J]. 计算机工程, 2026, 52(11): 444-455.

ZHU Jiachen, YANG Ye, HU Xiyou, WANG Jiaming. Design of Software━Hardware Integrated Elephant Trunk Gripper Based on Reinforcement Learning Algorithm[J]. Computer Engineering, 2026, 52(11): 444-455.

https://www.ecice06.com/CN/Y2026/V52/I11/444

参考文献

[1] 廖明蕾. 煤矿钻机机械手结构设计及分析[J]. 现代信息科技, 2022, 6(17): 56-59.LIAO M L. Design and analysis of coal mine drilling rig manipulator structure[J]. Modern Informationn Technology, 2022, 6(17): 56-59. (in Chinese)
[2] 于涵, 李一染, 毕书博, 等. 一种基于随动视觉的排爆机器人自主抓取系统[J]. 计算机工程, 2021, 47(1): 298-304, 311.YU H, LI Y R, BI S B, et al. An autonomous grasping system for EOD robot based on follow-up vision[J]. Computer Engineering, 2021, 47(1): 298-304, 311. (in Chinese)
[3] 禹鑫燚, 黄睿, 欧林林. 一种用于易损物体的机器人抓取姿态预测方法[J]. 小型微型计算机系统, 2024, 45(9): 2149-2155.YU X Y, HUANG R, OU L L. Robot grasping pose prediction method for fragile objects[J]. Journal of Chinese Computer Systems, 2024, 45(9): 2149-2155. (in Chinese)
[4] ZHU Y L, FENG K, HUA C, et al. Model analysis and experimental investigation of soft pneumatic manipulator for fruit grasping[J]. Sensors, 2022, 22(12): 4532.
[5] HUANG W K, XIAO J L, XU Z P. A variable structure pneumatic soft robot[J]. Scientific Reports, 2020, 10: 18778.
[6] 刘思远, 张京涛, 王杰, 等. 一种流体驱动式模块化软体仿生象鼻关节变刚度结构设计及仿真分析[J]. 液压与气动, 2022, 46(5): 152-158.LIU S Y, ZHANG J T, WANG J, et al. Design and simulation analysis of a fluid-driven modular software bionic elephant trunk joint with variable stiffness[J]. Chinese Hydraulics & Pneumatics, 2022, 46(5): 152-158. (in Chinese)
[7] KADALAGERE S S, WANG N, WU H, et al. Review on human-like robot manipulation using dexterous hands[J]. Cognitive Computation and Systems, 2023, 5(1): 14-29.
[8] MAŃKOWSKI T, TOMCZYŃSKI J, WALAS K, et al. PUT-hand—hybrid industrial and biomimetic gripper for elastic object manipulation[J]. Electronics, 2020, 9(7): 1147.
[9] REN T, LI Y J, LIU Q Y, et al. Novel bionic soft robotic hand with dexterous deformation and reliable grasping[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 7502110.
[10] QIN G D, WU H P, JI A H. Variable-curvature elephant trunk robot in nuclear industry[J]. Fusion Engineering and Design, 2023, 192: 113642.
[11] NIE S L, HUO L F, JI H, et al. Deformation characteristics of three-dimensional spiral soft actuator driven by water hydraulics for underwater manipulator[J]. Soft Robotics, 2024, 11(3): 410-422.
[12] 王捍天. 基于PID控制的机器人轨迹跟踪性能研究与比较[J]. 电子元器件与信息技术, 2019, 3(6): 44-47.WANG H T. Research and comparison of robot trajectory tracking performance based on PID control[J]. Electronic Component and Information Technology, 2019, 3(6): 44-47. (in Chinese)
[13] NAZEER M S, LASCHI C, FALOTICO E. Imitation and reinforcement learning to control soft robots: a perspective[J]. IOP Conference Series: Materials Science and Engineering, 2023, 1292(1): 012010.
[14] YAO J P, CAO Q L, JU Y W, et al. Adaptive actuation of magnetic soft robots using deep reinforcement learning[J]. Advanced Intelligent Systems, 2023, 5(2): 2200339.
[15] LAN Y X, REN J K, TANG T, et al. Efficient reinforcement learning with least-squares soft Bellman residual for robotic grasping[J]. Robotics and Autonomous Systems, 2023, 164: 104385.
[16] 李丽霞, 陈艳. 基于D-DQN强化学习算法的双足机器人智能控制研究[J]. 计算机测量与控制, 2024, 32(3): 181-187.LI L X, CHEN Y. Research on intelligent control of biped robot based on D-DQN reinforcement learning algorithm[J]. Computer Measurement & Control, 2024, 32(3): 181-187. (in Chinese)
[17] 何浩东, 符浩, 王强, 等. 基于深度强化学习的多机器人路径跟随与编队[J]. 计算机应用, 2024, 44(8): 2626-2633.HE H D, FU H, WANG Q, et al. Multi-robot path following and formation based on deep reinforcement learning[J]. Journal of Computer Applications, 2024, 44(8): 2626-2633. (in Chinese)
[18] KHAN A, ZHANG J L, AHMAD S, et al. DQN-based proactive trajectory planning of UAVs in multi-access edge computing[J]. Computers, Materials & Continua, 2023, 74(3): 4685-4702.
[19] 李勇, 张朝兴, 柴燎宁. 基于人工势场DDPG算法的移动机械臂协同避障轨迹规划[J]. 计算机集成制造系统, 2024, 30(12): 4282-4291.LI Y, ZHANG C X, CHAI L N. Collaborative obstacle avoidance trajectory planning for mobile robotic arms based on artificial potential field DDPG algorithm[J]. Computer Integrated Manufacturing Systems, 2024, 30(12): 4282-4291. (in Chinese)
[20] YIN H Q, WANG C, YAN C, et al. Deep reinforcement learning with multicritic TD3 for decentralized multirobot path planning[J]. IEEE Transactions on Cognitive and Developmental Systems, 2024, 16(4): 1233-1247.
[21] ZHANG Q L, HU S Y, DUAN J G, et al. A SAC-Bi-RRT two-layer real-time motion planning approach for robot assembly tasks in unstructured environments[J]. Actuators, 2025, 14(2): 59.
[22] ZHANG S B, XIA Q X, CHEN M X, et al. Multi-objective optimal trajectory planning for robotic arms using deep reinforcement learning[J]. Sensors, 2023, 23(13): 5974.
[23] CHAVDAROV I, NAYDENOV B, NIKOLOV V, et al. Modular design, communication and control systems of a 3D-printed humanoid robotic hand[J]. Journal of Communications Software and Systems, 2024, 20(2): 146-156.
[24] WANG N F, CHEN B C, GE X D, et al. Modular crawling robots using soft pneumatic actuators[J]. Frontiers of Mechanical Engineering, 2021, 16(1): 163-175.
[25] MOHAMMED N K M, PILLAI R R, PATEL M S. Design and development of modular self-reconfigurable mobile robot[EB/OL].[2025-01-05]. https://pubs.aip.org/aip/acp/article/2853/1/020281/3290376/Design-and-development-of-modular-self.
[26] SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[EB/OL].[2025-01-05]. https://arxiv.org/abs/1707.06347.

选择文件类型/文献管理软件名称

选择包含的内容