基于MPCNN模型的sEMG快速迁移学习的手势识别应用研究

doi:10.19678/j.issn.1000-3428.0068879

摘要/Abstract

摘要：

为解决个体间差异性的问题并提高手势识别技术的普适性, 提出基于多并行卷积神经网络(MPCNN)的迁移学习策略, 旨在实现基于表面肌电信号的高效手势识别。MPCNN通过并行架构和优化的迁移学习机制, 对比以往的卷积神经网络(CNN)迁移框架以更有效地处理不同个体间的生理差异, 从而提高模型对新用户的适应性和识别准确率。此外, MPCNN通过减少模型训练时间和提高泛化能力, 增强系统的实用性。通过多组实验, 包括倍数交叉验证、消融实验和健壮性测试来证实所提策略在多个方面的有效性。实验结果表明, 与传统CNN模型相比, 提出的MPCNN迁移学习策略显著提升手势识别准确率, 在Ninapro DB7数据集上的识别率达到了94.95%, 对比CNN迁移学习框架提高了4.38百分点, 同时训练时间减少了超过50%, 验证了MPCNN迁移模型在减轻训练负担、增强泛化能力和提高抗干扰性方面的优点。基于实验模型对人机交互能力进行了验证, 验证了其在肌电控制应用前景。

关键词: 迁移学习, 表面肌电信号, 手势识别, 深度学习, 卷积神经网络, 肌电控制

Abstract:

To address the challenge of inter-individual variability and improve the universality of gesture recognition technology, this study proposes a migration learning strategy based on Multi Parallel Conventional Neural Network (MPCNN), which aims to achieve efficient gesture recognition based on surface Electromyogram (sEMG) signals through a parallel architecture and an optimized migration learning mechanism. With a parallel architecture and optimized migration learning mechanism, MPCNN can deal with physiological differences between individuals more efficiently than previous CNN migration frameworks, which improves the model's adaptability to new users and recognition accuracy. In addition, MPCNN significantly enhances the utility of the system by reducing the model training time and improving the generalization ability. Through multiple sets of experiments, including multiplicative cross-validation, ablation experiments, and robustness tests, this study validates the effectiveness of the proposed strategy in several respects. The experimental results demonstrate that MPCNN significantly improves the accuracy of gesture recognition compared to traditional CNN models, and the proposed MPCNN migration learning strategy achieves a recognition rate of 94.95% in Ninapro DB7 compared to previous CNN migration learning frameworks, with an improvement of 4.38 percentage points, with the training time reduced by more than 50%. These experiments validate the advantages of the MPCNN migration model in reducing the training burden, enhancing the generalization ability, and improving anti-interference. The human-computer interaction capability is validated based on an experimental model, which verifies its promising potential for myoelectric control applications.

Key words: transfer learning, surface Electromyographic(sEMG) signal, gesture recognition, deep learning, Convolutional Neural Network(CNN), myoelectric control

易鹏, 杨晔, 严仕嘉. 基于MPCNN模型的sEMG快速迁移学习的手势识别应用研究[J]. 计算机工程, 2025, 51(1): 304-311.

YI Peng, YANG Ye, YAN Shijia. Research of Fast Transfer Learning of sEMG Based on MPCNN Model for Gesture Recognition Applications[J]. Computer Engineering, 2025, 51(1): 304-311.

https://www.ecice06.com/CN/Y2025/V51/I1/304

图/表 11

图1 迁移学习流程

Fig.1 Procedure of transfer learning

图2 滑动窗口的划分方法

Fig.2 Partition method of sliding window

图3 MPCNN模型的构架

Fig.3 Architecture of MPCNN model

图4 DB7数据集的40种手势示意图

Fig.4 Schematic diagram of 40 gestures of DB7 dataset

图5 不同滑动窗口下的识别率与训练次数曲线

Fig.5 Recognition rate and training times curve under different sliding windows

图6 不同模型下的时间消耗

Fig.6 Time consumptions under different models

图7 消融实验结果

Fig.7 Results of ablation experiment

图8 不同训练数据量的平均准确率

Fig.8 Average accuracy of different training data volumes

图9 抗干扰性能实验结果

Fig.9 Experimental results of the anti-interference performance

图10 基于MPCNN模型的迁移学习人机交互实验

Fig.10 Transfer learning human computer interaction experiment based on MPCNN model

参考文献 27

1	XIONG D Z , ZHANG D H , ZHAO X G , et al. Deep learning for EMG-based human-machine interaction: a review. CAA Journal of Automatica Sinica, 2021, 8 (3): 512- 533.
2	BI L Z , FELEKE A , GUAN C T . A review on EMG-based motor intention prediction of continuous human upper limb motion for human-robot collaboration. Biomedical Signal Processing and Control, 2019, 51, 113- 127. doi: 10.1016/j.bspc.2019.02.011
3	SUN X H , DING J Y , DONG Y X , et al. A survey of technologies facilitating home and community-based stroke rehabilitation. International Journal of Human-Computer Interaction, 2023, 39 (5): 1016- 1042. doi: 10.1080/10447318.2022.2050545
4	李自由, 赵新刚, 张弼, 等. 基于表面肌电的意图识别方法在非理想条件下的研究进展. 自动化学报, 2021, 47 (5): 955- 969.
	LI Z Y , ZHAO X G , ZHANG B , et al. Review of sEMG-based motion intent recognition methods in non-ideal conditions. Acta Automatica Sinica, 2021, 47 (5): 955- 969.
5	陶建华, 龚江涛, 高楠, 等. 面向虚实融合的人机交互. 中国图象图形学报, 2023, 28 (6): 1513- 1542.
	TAO J H , GONG J T , GAO N , et al. Human-computer interaction for virtual-real fusion. Journal of Image and Graphics, 2023, 28 (6): 1513- 1542.
6	宋爱国, 胡旭晖, 祝佳航. 智能肌电控制假手研究进展. 南京信息工程大学学报, 2019, 11 (2): 127- 137.
	SONG A G , HU X H , ZHU J H . Research progress on intelligent myoelectric control prosthesis. Journal of Nanjing University of Information Science & Technology (Natural Science Edition), 2019, 11 (2): 127- 137.
7	丁其川, 熊安斌, 赵新刚, 等. 基于表面肌电的运动意图识别方法研究及应用综述. 自动化学报, 2016, 42 (1): 13- 25.
	DING Q C , XIONG A B , ZHAO X G , et al. A review on researches and applications of sEMG-based motion intent recognition methods. Acta Automatica Sinica, 2016, 42 (1): 13- 25.
8	孙志军, 薛磊, 许阳明, 等. 深度学习研究综述. 计算机应用研究, 2012, 29 (8): 2806- 2810. doi: 10.3969/j.issn.1001-3695.2012.08.002
	SUN Z J , XUE L , XU Y M , et al. Overview of deep learning. Application Research of Computers, 2012, 29 (8): 2806- 2810. doi: 10.3969/j.issn.1001-3695.2012.08.002
9	庄福振, 罗平, 何清, 等. 迁移学习研究进展. 软件学报, 2014, 26 (1): 26- 39.
	ZHUANG F Z , LUO P , HE Q , et al. Survey on transfer learning research. Journal of Software, 2014, 26 (1): 26- 39.
10	YOUSEFI J , HAMILTON-WRIGHT A . Characterizing EMG data using machine-learning tools. Computers in Biology and Medicine, 2014, 51, 1- 13. doi: 10.1016/j.compbiomed.2014.04.018
11	ERICSSON L, GOUK H, HOSPEDALES T M. How well do self-supervised models transfer?[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE Press, 2021: 5414-5423.
12	李自由, 赵新刚, 张弼, 等. 基于表面肌电的意图识别方法在非理想条件下的研究进展. 自动化学报, 2021, 47 (5): 955- 969.
	LI Z Y , ZHAO X G , ZHANG B , et al. Review of sEMG-based motion intent recognition methods in non-ideal conditions. Acta Automatica Sinica, 2021, 47 (5): 955- 969.
13	ZHUANG F Z , QI Z Y , DUAN K Y , et al. A comprehensive survey on transfer learning. Proceedings of the IEEE, 2021, 109 (1): 43- 76. doi: 10.1109/JPROC.2020.3004555
14	DAY O , KHOSHGOFTAAR T M . A survey on heterogeneous transfer learning. Journal of Big Data, 2017, 4 (1): 29. doi: 10.1186/s40537-017-0089-0
15	WANG M , DENG W H . Deep visual domain adaptation: a survey. Neurocomputing, 2018, 312, 135- 153. doi: 10.1016/j.neucom.2018.05.083
16	FARAHANI A , VOGHOEI S , RASHEED K , et al. A brief review of domain adaptation. Berlin, Germany: Springer, 2021.
17	PAN S J , YANG Q . A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22 (10): 1345- 1359. doi: 10.1109/TKDE.2009.191
18	赵凯琳, 靳小龙, 王元卓. 小样本学习研究综述. 软件学报, 2020, 32 (2): 349- 369.
	ZHAO K L , JIN X L , WANG Y Z , et al. Survey on few-shot learning. Journal of Software, 2020, 32 (2): 349- 369.
19	黎英, 宋佩华. 迁移学习在医学图像分类中的研究进展. 中国图象图形学报, 2022, 27 (3): 672- 686.
	LI Y , SONG P H . Review of transfer learning in medical image classification. Journal of Image and Graphics, 2022, 27 (3): 672- 686.
20	YOU K, LIU Y, WANG J, et al. Logme: practical assessment of pre-trained models for transfer learning[C]//Proceedings of IEEE/CVF International Conference on Machine Learning. Washington D. C., USA: IEEE Press, 2021: 12133-12143.
21	RUSU A A, RABINOWITZ N C, DESJARDINS G, et al. Progressive neural networks[EB/OL]. [2023-10-10]. https://arxiv.org/pdf/1606.04671.
22	KIM K T , GUAN C T , LEE S W . A subject-transfer framework based on single-trial EMG analysis using convolutional neural networks. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28 (1): 94- 103. doi: 10.1109/TNSRE.2019.2946625
23	CHEN X , LI Y , HU R , et al. Hand gesture recognition based on surface electromyography using convolutional neural network with transfer learning method. IEEE Journal of Biomedical and Health Informatics, 2021, 25 (4): 1292- 1304. doi: 10.1109/JBHI.2020.3009383
24	张应祥, 位少聪, 张茜茜, 等. 基于迁移学习的表面肌电信号手势识别. 南京理工大学学报(自然科学版), 2023, 47 (3): 304- 310.
	ZHANG Y X , WEI S C , ZHANG Q Q , et al. Surface electromyographic signal hand gesture recognition based on transfer learning. Journal of Nanjing University of Science and Technology, 2023, 47 (3): 304- 310.
25	KRASOULIS A , KYRANOU I , ERDEN M S , et al. Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements. Journal of Neuroengineering and Rehabilitation, 2017, 14 (1): 71.
26	孙旭晖, 刘银华. 基于CNN-LSTM的sEMG连续手势识别. 自动化与仪器仪表, 2023, (12): 1- 4.
	SUN X H , LIU Y H . CNN-LSTM based sEMG continuous gesture recognition. Automation and instrumentation, 2023, (12): 1- 4.
27	TSINGANOS P, CORNELIS J, CORNELIS B, et al. Transfer learning in sEMG-based gesture recognition[C]//Proceedings of the 12th International Conference on Information, Intelligence, Systems & Applications. Chania Crete, Greece: IEEE Press, 2021: 1-7.

[1]	王晓路, 汶建荣. 基于运动-时间感知的人体动作识别方法[J]. 计算机工程, 2025, 51(1): 216-224.
[2]	火久元, 苏泓瑞, 武泽宇, 王婷娟. 基于改进YOLOv8的道路交通小目标车辆检测算法[J]. 计算机工程, 2025, 51(1): 246-257.
[3]	郑雅洲, 刘万平, 黄东. 一种基于注意力机制的BERT-CNN-GRU检测方法[J]. 计算机工程, 2025, 51(1): 258-268.
[4]	王骞, 张俊华, 王泽彤, 李博. X2S-Net: 基于双平面X线片的脊柱三维重建[J]. 计算机工程, 2025, 51(1): 277-286.
[5]	刘兆伟, 方艳红, 郑明宇, 锁斌. 基于注意力机制与多任务的肺部疾病诊断方法[J]. 计算机工程, 2025, 51(1): 332-342.
[6]	周宇, 谢威, 邝得互, 江健民. 基于三元自注意力的视频快照压缩成像重建[J]. 计算机工程, 2025, 51(1): 20-30.
[7]	胡升龙, 陈彬, 张开华, 宋慧慧. 场景结构知识增强的协同显著性目标检测[J]. 计算机工程, 2025, 51(1): 31-41.
[8]	喻勇涛, 孙奥, 李昂, 朱琳琳. 基于孪生网络的分类器输出重复性优化方法[J]. 计算机工程, 2025, 51(1): 118-127.
[9]	张会影, 圣文顺. 基于标记适应的人脸年龄识别优化算法[J]. 计算机工程, 2025, 51(1): 174-181.
[10]	杨红菊, 吉昌. 学习驱动的图像压缩算法研究[J]. 计算机工程, 2025, 51(1): 190-197.
[11]	张天鹏, 韩晶, 吕学强. 基于多任务学习的超分辨率辅助小目标检测[J]. 计算机工程, 2024, 50(9): 304-312.
[12]	高煜宝, 文志诚. 基于注意力机制的双路解码器图像去噪方法[J]. 计算机工程, 2024, 50(9): 324-332.
[13]	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究[J]. 计算机工程, 2024, 50(9): 82-91.
[14]	张鲁, 田春伟, 宋焕生, 刘侍刚. 用于低剂量CT图像去噪的多级双树复小波网络[J]. 计算机工程, 2024, 50(9): 266-275.
[15]	朱凯, 李理, 张彤, 江晟, 别一鸣. 基于Transformer的多阶段运动模糊图像修复网络[J]. 计算机工程, 2024, 50(9): 276-285.

选择文件类型/文献管理软件名称

选择包含的内容