Abstract: To tackle the problem of redundant information in action video and the sparse distribution of feature channels in action information, a 3D residual network based on action-time-perception is proposed. The action- perception module (AM) calculates the temporal differences of feature level. The motion features can be obtained by utilizing these differences to excite the action-sensitive channel. The temporal attention module (TM) works out the attention weight matrix along the time dimension, based on which, the local time features are determined. The fusion features of action information are acquired by combining the results of the AM module and the TM module, and then the fusion feature is joined into the 3D convolution network, which construct an action-time-perception module (ATM) based 3DCNN action recognition network. Experimental results show that on the public datasets UCF101 and HMDB51, the accuracy of the action recognition of the 3DResNeXt-101 network based on the ATM module is improved by 1.6% and 2.8%, respectively, indicating that the method proposed in this paper is feasible and effective.

摘要： 针对动作视频中存在冗余信息及动作信息的特征通道分布稀疏问题，提出了一种基于运动-时间感知的3D残差网络。运动感知模块(Action- Perception Module,AM)计算特征级别的时间差来激励运动敏感通道以此获取运动特征；时间注意力模块(Temporal-Attention Module,TM)沿着时间维度计算注意力权重矩阵获取局部时间特征。将AM模块和TM模块计算结果相加，得到动作信息的融合特征，再加入到3D残差网络中，以此构造基于运动-时间感知模块(Action-Time-Perception Module,ATM)的3D残差网络。实验结果表明，在公共数据集UCF101和HMDB51上，基于ATM模块的3DResNeXt-101网络相对于3DResNeXt-101网络的动作识别的准确率分别提升了1.6 %和2.8 %，说明本文所提出的方法是可行、有效的。