计算机工程 ›› 2020, Vol. 46 ›› Issue (9): 101-109.doi: 10.19678/j.issn.1000-3428.0056808

• 人工智能与模式识别 • 上一篇    下一篇

时域注意力Dense-TCNs在多模手势识别中的应用

张毅, 赵杰煜, 王翀, 郑烨   

  1. 宁波大学 信息科学与工程学院, 浙江 宁波 315211
  • 收稿日期:2019-12-05 修回日期:2020-03-26 发布日期:2020-04-02
  • 作者简介:张毅(1994-),男,硕士研究生,主研方向为计算机视觉;赵杰煜,教授;王翀,副教授;郑烨,硕士研究生。
  • 基金项目:
    国家自然科学基金(61603202,61571247);浙江省自然科学基金重点项目(LZ16F03001,LY17F030002)。

Application of Time Domain Attention Dense-TCNs in Multimodal Gesture Recognition

ZHANG Yi, ZHAO Jieyu, WANG Chong, ZHENG Ye   

  1. College of Information Science and Engineering, Ningbo University, Ningbo, Zhejiang 315211, China
  • Received:2019-12-05 Revised:2020-03-26 Published:2020-04-02

摘要: 为增强时间卷积网络(TCNs)在时间特征提取方面的能力,提出一种基于三维密集卷积网络与改进TCNs的多模态手势识别方法。通过时空特征表示方法将手势视频分析任务分为空间分析和时间分析两部分。在空间分析中采用三维DenseNets学习短期的时空特征,在时间分析中使用TCNs提取时间特征。在此基础上引入注意力机制,使用时域维度的压缩-激励网络调整每个TCNs层特征在时间维度上的权值比重。分别在VIVA和NVGesture两个动态手势数据集上对该方法进行评价,实验结果表明,该方法在VIVA数据集上的正确率为91.54%,在NVGesture数据集上的正确率为86.37%,且与最新的MTUT方法水平相近。

关键词: 手势识别, 三维密集卷积网络, 时间卷积网络, 短时时空特征, 注意力机制

Abstract: In order to enhance the temporal feature extraction ability of Temporal Convolutional Networks(TCNs),this paper proposes a multimodal gesture recognition method based on 3D Dense convolutional Networks(3D-DenseNets) and improved TCNs.3D-DenseNets are used in spatial analysis to effectively learn short-term temporal and spatial features,and TCNs are used to extract temporal features in temporal analysis.On this basis,the attention mechanism is introduced,and the time-domain compression-stimulation network is used to adjust the weight ratio of each TCN layer feature in the time dimension.The method is evaluated on two dynamic gesture data sets,VIVA and NVGesture.Experimental results show that the proposed method achieves an accuracy rate of 91.54% on VIVA and 86.37% on the benchmark of NVGesture,reaching a level similar to that of the latest MTUT method.

Key words: gesture recognition, 3D Dense convolutional Networks(3D-DenseNets), Temporal Convolutional Networks(TCNs), short-term temporal and spatial features, attention mechanism

中图分类号: