作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2020, Vol. 46 ›› Issue (3): 237-245,253. doi: 10.19678/j.issn.1000-3428.0054222

• 图形图像处理 • 上一篇    下一篇

基于改进YOLOv3的手势实时识别方法

张强, 张勇, 刘芝国, 周文军, 刘佳慧   

  1. 合肥工业大学 计算机与信息学院, 合肥 230601
  • 收稿日期:2019-03-14 修回日期:2019-04-24 发布日期:2019-05-31
  • 作者简介:张强(1993-),男,硕士研究生,主研方向为手势识别、智能信息处理;张勇,副教授;刘芝国、周文军、刘佳慧,硕士研究生。
  • 基金资助:
    国家自然科学基金(61801162);国家大学生创新训练项目(201710359020)。

Real-time Hand Gesture Recognition Method Based on Improved YOLOv3

ZHANG Qiang, ZHANG Yong, LIU Zhiguo, ZHOU Wenjun, LIU Jiahui   

  1. School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601, China
  • Received:2019-03-14 Revised:2019-04-24 Published:2019-05-31

摘要: 针对基于人工建模方式的手势识别方法准确率低、速度慢的问题,提出一种基于改进YOLOv3的静态手势实时识别方法。采用卷积神经网络YOLOv3模型,将通过Kinect设备采集的IR、Registration of RGB、RGB和Depth图像代替常用的RGB图像作为数据集,并融合四类图像的识别结果以提高识别准确率。采用k-means聚类算法对YOLOv3中的初始候选框参数进行优化,从而加快识别速度。在此基础上,利用迁移学习的方法对基础特征提取器进行改进,以缩短模型的训练时间。实验结果表明,该方法对流式视频静态手势的平均识别准确率为99.8%,识别速度高达52 FPS,模型训练时间为12 h,与Faster R-CNN、SSD、YOLOv2等深度学习方法相比,其识别精度更高,识别速度更快。

关键词: 手势识别, YOLOv3模型, Kinect设备, 聚类算法, 迁移学习

Abstract: The hand gesture recognition method based on artificial modeling has many problems such as low accuracy and slow speed.Therefore,this paper proposes a static hand gesture recognition method based on improved YOLOv3.By using the convolutional neural network YOLOv3 model,the commonly used RGB images are replaced by the IR,Registration of RGB,RGB and Depth images collected by Kinect equipment as dataset.The recognition results of these 4 types of images are fused to improve the recognition accuracy.The k-means clustering algorithm is used to optimize the initial candidate frame parameters in YOLOv3,so as to improve the recognition speed.On this basis,the transfer learning is used to improve the basic feature extractor to shorten the training time of the model.Experimental results show that for the recognition of static hand gestures in stream videos,the mean Average Precision(mAP) of the proposed method is 99.8% and the recognition speed is up to 52 FPS.The training time of the proposed model is 12 hours,and its recognition accuracy and speed is better than other deep learning methods such as Faster R-CNN,SSD and YOLOv2.

Key words: hand gesture recognition, YOLOv3 model, Kinect equipment, clustering algorithm, transfer learning

中图分类号: