Real-Time Human Keypoint Detection Algorithm Based on Lightweight Network

doi:10.19678/j.issn.1000-3428.0057041

Abstract

Abstract: To improve the real-time performance and speed of human pose estimation on mobile devices,this paper proposes an improved human keypoint detection algorithm.The algorithm uses the depthwise separable convolution module and MobileNetV2,a light-weight backbone network,to accelerate feature extraction.Then the refine network is used to predict the keypoint of human body on multiple scales.The results at different scales are fused by using fusion network to obtain the final keypoint detection result.Experimental results show that compared with the traditional CPM algorithm,the proposed algorithm significantly reduces the amount of parameters and floating point operations while the PCKh@0.5 is decreased only by 0.1 percentage point. It has higher detection accuracy and better real-time performance.

Key words: depthwise separable convolution, multi-scale prediction, human keypoint detection, lightweight backbone network, fusion network

摘要： 为提升人体姿态估计在移动终端设备上的运行速度与实时性，提出一种改进的人体关键点检测算法。通过将MobileNetV2轻量级主干网络与深度可分离卷积模块相结合加速特征提取过程，使用精炼网络进行多尺度人体关键点预测，并利用融合网络整合多个尺度的预测结果得到最终人体关键点检测结果。实验结果表明，与传统CPM算法相比，该算法在网络模型参数量和浮点运算量明显减少的情况下PCKh@0.5仅下降0.1个百分点，具有较高的检测精度和较好的实时性。

关键词: 深度可分离卷积, 多尺度预测, 人体关键点检测, 轻量级主干网络, 融合网络

CLC Number:

TP183

HU Jianghao, WANG Hongyu, QIAO Wenchao, MA Jingxuan. Real-Time Human Keypoint Detection Algorithm Based on Lightweight Network[J]. Computer Engineering, 2021, 47(4): 218-225.

胡江颢, 王红雨, 乔文超, 马靖煊. 基于轻量级网络的实时人体关键点检测算法[J]. 计算机工程, 2021, 47(4): 218-225.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0057041

http://www.ecice06.com/EN/Y2021/V47/I4/218

Figures/Tables 12

References

[1] ZHENG Weicheng,LI Xuewei,LIU Hongzhe,et al.Fatigue driving detection algorithm based on deep learning[J].Computer Engineering,2020,46(7):21-29.(in Chinese)郑伟成,李学伟,刘宏哲,等.基于深度学习的疲劳驾驶检测算法[J].计算机工程,2020,46(7):21-29.
[2] XIA Hansheng,SHEN Huan,HU Wei.Detecting distraction of drivers using human pose keypoints[J].Computer Technology and Development,2019,29(7):1-5.(in Chinese)夏瀚笙,沈峘,胡委.基于人体关键点的分心驾驶行为识别[J].计算机技术与发展,2019,29(7):1-5.
[3] TANG Hui,WANG Qing,CHEN Hong,et al.Somatosensory interaction method based on deep learning[J].Computer and Modernization,2019(2):7-14.(in Chinese)唐晖,王庆,陈洪,等.基于深度学习的体感交互方法[J].计算机与现代化,2019(2):7-14.
[4] WANG Tao,WANG Hongzhang,XIA Yi,et al.Human gait recognition based on convolutional neural network and attention model[J].Chinese Journal of Sensors and Actuators,2019,32(7):1027-1033.(in Chinese)汪涛,汪泓章,夏懿,等.基于卷积神经网络与注意力模型的人体步态识别[J].传感技术学报,2019,32(7):1027-1033.
[5] CAO Z,SIMON T,WEI S E,et al.OpenPose:realtime multi-person 2D pose estimation using part affinity fields[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:7291-7299.
[6] CHEN Y,SHEN C,WEI X S,et al.Adversarial PoseNet:a structure-aware convolutional network for human pose estimation[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:1212-1221.
[7] FANG H S,XIE S,TAI Y W,et al.RMPE:regional multi-person pose estimation[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:2334-2343.
[8] PFISTER T,CHARLES J,ZISSERMAN A.Flowing ConvNets for human pose estimation in videos[C]//Proceedings of 2015 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2015:1913-1921.
[9] XIAO Bin,WU Haiping,WEI Yichen.Simple baselines for human pose estimation and tracking[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2018:472-487.
[10] NEWELL A,YANG K,DENG J.Stacked hourglass networks for human pose estimation[C]//Proceedings of 2016 European Conference on Computer Vision.Berlin,Germany:Springer,2016:483-499.
[11] WEI S,RAMAKRISHNA V,KANADE T,et al.Convolu-tional pose machines[C]//Proceedings of 2016 IEEE Con-ference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:483-499.
[12] CHEN Yilan,WANG Zhicheng,PENG Yuxiang,et al.Cascaded pyramid network for multi-person pose estimation[EB/OL].[2019-11-10].https://arxiv.org/pdf/1711.07319.pdf.
[13] SUN Ke,XIAO Bin,LIU Dong,et al.Deep high-resolution representation learning for human pose estimation[EB/OL].[2019-11-10].https://arxiv.org/pdf/1902.09212.pdf.
[14] ZOPH B,VASUDEVAN V,SHLENS J,et al.Learning transferable architectures for scalable image recognition[EB/OL].[2019-11-10].https://arxiv.org/pdf/1707.07012v3.pdf.
[15] LI Lingxia,WANG Yu,WU Jinjun,et al.Micro-motion hand gesture recognition method based on improved multiple dimensional convolution neural network[J].Computer Engineering,2018,44(9):243-249.(in Chinese)李玲霞,王羽,吴金君,等.基于改进型多维卷积神经网络的微动手势识别方法[J]. 计算机工程,2018,44(9):243-249.
[16] GUO Jichang,WU Jie,GUO Chunle,et al.Image super-resolution reconstruction based on residual connection convolutional neural network[J].Journal of Jilin University(Engineering and Technology Edition),2019,49(5):1726-1734.(in Chinese)郭继昌,吴洁,郭春乐.基于残差连接卷积神经网络的图像超分辨率重构[J].吉林大学学报(工学版),2019,49(5):1726-1734.
[17] SANDLER M,HOWARD A,ZHU M,et al.Inverted residuals and linear bottlenecks:mobile networks for classification,detection and segmentation[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:4510-4520.
[18] LIN T Y,DOLLAR P,GIRSHICK R,et al.Feature pyramid networks for object detection[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:936-944.
[19] ZHANG X,ZHOU X,LIN M,et al.ShuffleNet:an extremely efficient convolutional neural network for mobile devices[EB/OL].[2019-11-10].https://arxiv.org/pdf/1707.01083v2.pdf.
[20] HOWARD A G,ZHU M L,CHEN B,et al.MobileNets:efficient convolutional neural networks for mobile vision applications[EB/OL].[2019-11-10].https://arxiv.org/pdf/1704.04861.pdf.
[21] SU Zhihui,YE Ming,ZHANG Guohui,et al.Cascade feature aggregation for human pose estimation[EB/OL].[2019-11-10].https://arxiv.org/ftp/arxiv/papers/1902/1902.07837.pdf.

Please choose a citation manager

Content to export