摘要：

传统基于RGB视频的人体行为识别方法在面对背景复杂、光照影响以及外貌变化时存在诸多挑战。相比之下, 利用人体骨架信息进行行为识别的方法受到的影响较小。然而, 目前主流的基于骨架的行为识别方法难以兼顾精度与复杂度。为了在保持高识别精度的同时解决模型参数量大、计算复杂度高的问题, 提出一种由3个新编码块组成的轻量化网络结构。首先, 在用于空间建模的自注意图卷积模块和用于时间建模的多尺度时间卷积模块中添加高效的多尺度注意力模块, 提高模型对时间和空间特征信息的识别和利用, 丰富骨架数据特征; 其次, 利用多特征融合自适应模块来增强特征融合与泛化能力; 最后, 使用迭代特征融合增强模块进一步加强对复杂特征关系的理解。实验结果表明, 在大规模数据集NTU-RGB+D60上, 所提方法在交叉主题评估(CS)和交叉视角评估(CV)下的准确率分别为91.1%和95.4%, 在数据集NTU-RGB+D120上, 该方法在CS和交叉设置评估(SS)下的准确率分别为87.3%和88.8%, 参数量为0.72×10⁶, 浮点计算量为0.6×10⁹。对比实验表明, 所提算法的参数量、浮点计算量以及识别精度均优于近年来的一些主流算法, 其有效地平衡了这些指标间的关系, 为人体行为的精确识别提供了轻量级网络模型。

关键词: 人体骨架, 行为识别, 轻量级, 图卷积, 特征融合

Abstract:

Traditional methods for human behavior recognition based on RGB videos face numerous challenges when dealing with complex backgrounds, lighting effects, and variations in appearance. By contrast, methods that leverage human skeletal information for behavior recognition are less affected by these factors. However, the current mainstream skeleton-based behavior recognition methods struggle to balance accuracy and complexity. To maintain high recognition accuracy while addressing issues such as large model parameter size and high computational complexity, a lightweight network structure comprising three novel encoding blocks is proposed. First, efficient multiscale attention modules are incorporated into the self-attention graph convolutional module for spatial modeling and the multiscale temporal convolutional module for temporal modeling, enhancing the ability of the model to recognize and utilize temporal and spatial feature information, thereby enriching skeletal data features. Second, a multifeature fusion adaptive module is employed to strengthen the feature fusion and generalization capabilities. Finally, an iterative feature fusion enhancement module is utilized to further improve the understanding of complex feature relationships. Experimental results demonstrate that, on the large-scale NTU-RGB+D60 dataset, the proposed method achieves accuracy rates of 91.1% and 95.4% under Cross-Subject (CS) and Cross-View (CV) evaluations, respectively. On the NTU-RGB+D120 dataset, it attains accuracy rates of 87.3% and 88.8% under CS and Cross-Setup (SS) evaluations, respectively, with a parameter count of 0.72×10⁶ and a floating-point operation count of 0.6×10⁹. Comparative experiments indicate that the proposed algorithm outperforms several mainstream algorithms in recent years in terms of parameter size, floating-point operation count, and recognition accuracy, effectively balancing the relationships among these metrics and providing a lightweight network model for precise human behavior recognition.

Key words: human skeleton, behavior recognition, lightweight, graph convolution, feature fusion