摘要：

为满足冰壶智能训练的需求, 结合计算机视觉与深度强化学习(RL)技术, 提出一种新的现场冰壶决策方法AI-Curling。AI-Curling包含冰壶检测(SR-Yolo)以及策略生成(GSP-MCTS) 2个部分。SR-Yolo模块负责感知关键时刻冰壶状态, 提取实景冰壶的位置与种类信息。为提高大场景下的小目标检测精度, 防止不恰当下采样造成的特征损失, 引入浅层细化骨干网络(SRNet), 通过在网络初级阶段增加层级, 捕获更丰富的特征信息。此外, 在多尺度融合网络中, 引入自适应特征优化融合(AFOF)模块, 以增加各层网络有效样本, 避免小尺度目标淹没在复杂背景和噪声中。GSP-MCTS模块通过蒙特卡洛树搜索(MCTS)算法结合策略价值网络的方式, 实现冰壶比赛决策分析。该模块通过引入核函数处理动作空间连续性和执行不确定性, 并在策略价值网络中嵌入全局策略感知模块(GSP), 增强了网络空间感知能力。在实验中, SR-Yolo在常规冰壶数据集Curling上平均精度均值(mAP@0.5)为0.974, 在遮挡较多的复杂冰壶数据集Curling_hard上mAP@0.5为0.723。同时, GSP-MCTS与最新实景冰壶模型Curling MCTS对战获得62%的胜率。实验结果表明, GSP-MCTS具有更好的性能。

关键词: 强化学习, 深度学习, 冰壶检测, 小目标检测, 蒙特卡洛树搜索

Abstract:

In response to the need for intelligent curling training, a new on-site curling decision-making method that combines computer vision and deep Reinforcement Learning (RL) technologies, Artificial Intelligence (AI)-Curling, is proposed. AI-Curling comprises two components: SR-Yolo for curling detection and Global Strategy Perception-Monte Carlo Tree Search (GSP-MCTS) for strategy generation. The former is responsible for sensing the state of the curling stones at critical moments and extracting information on the location and type of stones in real scenes. To improve the detection accuracy of small targets in large scenes and prevent feature loss due to inappropriate downsampling, a Shallow Refinement Backbone Network (SRNet) is introduced to capture richer feature information by adding layers during the initial stages of the network. An Adaptive Feature Optimization Fusion (AFOF) module is further introduced into the multiscale fusion network to increase the number of effective samples in each layer, thereby preventing small-scale targets from being submerged in complex backgrounds and noise. In the strategy generation module, curling match decision analysis is implemented using a combination of the MCTS algorithm and policy value network. A GSP module is embedded into the policy value network to enhance network spatial perception by introducing a kernel function to deal with action space continuity and execution uncertainty. In the experiments, SR-Yolo achieved 0.974 mAP@0.5 on the standard Curling dataset and 0.723 mAP@0.5 on the more complex obstructed Curling_hard dataset. In addition, GSP-MCTS achieved a 62% winning percentage compared with the latest real-scene curling model Curling MCTS, indicating that GSP-MCTS has superior performance.

Key words: Reinforcement Learning (RL), deep learning, curling detection, small object detection, Monte Carlo Tree Search (MCTS)