Human Pose Estimation Based on Improved Pyramid Feature Network

doi:10.19678/j.issn.1000-3428.0058544

Abstract

Abstract: To accurately acquire multi-scale features and keypoint coordinates in human pose estimation, a human pose tracking model based on Improved Pyramid Feature Network(IPFN) is established. On the basis of the original feature pyramid, a new detector is used to expand the receptive field for the new feature pyramid. Then gaussian heatmaps are generated by using multi-scale convolutions to search and locate the keypoints. The heatmaps are converted into coordinates by the coordinate transformation layer to realize end-to-end learning. Experimental results show that compared with traditional FPN, the IPFN model improves PCKh by 2.05 percentage point on the MPⅡ data set and AP by 3.2 percentage point on the COCO data set. The proposed model also improves PCKh by 3.95 percentage point for the ankle part, 2.80 percentage point for the knee part, 2.52 percentage point for the wrist part, and 2.05 percentage point for the elbow part.

Key words: human pose estimation, feature pyramid, coordinate transformation, multi-scale convolution, Gaussian heatmap

摘要： 为在人体姿态估计过程中有效获取多尺度特征和关键点坐标，建立一种基于改进特征金字塔网络（IPFN）的人体姿态跟踪模型。在原特征金字塔上采用新的检测器扩大感受野得到新特征金字塔，通过引入多尺度卷积生成高斯热点图，同时搜索和定位关键点，使坐标转换层将高斯热点图转为坐标，实现端到端训练过程。实验结果表明，相比FPN， IPFN模型在MPⅡ数据集的PCKh和COCO数据集的AP上分别提高了2.05和3.20个百分比；在踝、膝、腕和肘4个难检测部位上的PCKh分别提高了3.95、2.80、2.52和2.05个百分点。

关键词: 人体姿态估计, 特征金字塔, 坐标变换, 多尺度卷积, 高斯热点图

CLC Number:

TP391.41

WANG Liucheng, OUYANG Chengtian, LIANG Wen. Human Pose Estimation Based on Improved Pyramid Feature Network[J]. Computer Engineering, 2021, 47(8): 251-259,270.

王柳程, 欧阳城添, 梁文. 基于改进特征金字塔网络的人体姿态估计[J]. 计算机工程, 2021, 47(8): 251-259,270.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0058544

http://www.ecice06.com/EN/Y2021/V47/I8/251

Figures/Tables 13

References

[1] FISCHLER M A, ELSCHLAGER R A.The representation and matching of pictorial structures[J]. IEEE Transactions on Computer, 1973, 22(1): 67-92.
[2] FELZENSZWALB P F, HUTTENLOCHER D P.Pictorial structures for object recognition[J]. International Journal of Computer Vision, 2005, 61(1): 55-79.
[3] DALAL N, TRIGGS B.Histograms of oriented gradients for human detection[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2005:886-893.
[4] TOSHEV A, SZEGEDY C.DeepPose:human pose estimation via deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2014:1653-1660.
[5] LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:2117-2125.
[6] CHEN Y L, WANG Z C, PENG Y X, et al. Cascaded pyramid network for multi-person pose estimation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:7103-7112.
[7] XIAO B, WU H P, WEI Y C.Simple baselines for human pose estimation and tracking[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:472-487.
[8] SUN X, XIAO B, WEI F Y, et al. Integral human pose regression[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:536-553.
[9] LUVIZON D C, TABIA H, PICARD D.Human pose regression by combining indirect part detection and contextual information[J]. Computers and Graphics, 2019, 85:15-22.
[10] 林志洁, 罗壮, 赵磊, 等. 特征金字塔多尺度全卷积目标检测算法[J]. 浙江大学学报, 2019, 53(3): 533-540. LUO Z H, LUO Z, ZHAO L, et al. Multi-scale convolution target detection algorithm with feature pyramid[J]. Journal of Zhejiang University, 2019, 53(3): 533-540.(in Chinese)
[11] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:770-778.
[12] NIBALI A, HE Z, MORGAN S, et al. Numerical coordinate regression with convolutional neural networks[EB/OL]. [2020-06-12]. http://export.arxiv.org/abs/1801.07372.
[13] JADERBERG M, SIMONYAN K, ZISSERMAN A, et al. Spatial transformer networks[C]//Proceedings of Advances in Neural Information Processing Systems.Cambridge, USA:MIT Press, 2015:2017-2025.
[14] CAO Z, SIMON T, WEI S-E, et al. Realtime multi-person 2D pose estimation using part affinity fields[C]//Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. Washington D. C., USA:IEEE Press, 2017:1302-1310.
[15] FANG H, XIE S, TAI S, et al. RMPE:regional multi-person pose estimation[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA:IEEE Press, 2017:2353-2362.
[16] LIFSHITZ I, FETAYA E, ULLMAN S. Human pose estimation using deep consensus voting[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany:Springer, 2016:246-260.
[17] GKIOXARI G, TOSHEV A, JAITLY N. Chained predictions using convolutional neural networks[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany:Springer, 2016:728-743.
[18] NEWELL A, YANG K, DENG J. Stacked hourglass networks for human pose estimation[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany:Springer, 2016:483-499.
[19] KOCABAS M, KARAGOZ S, AKBAS E.. Multiposenet:fast multi-person pose estimation using pose residual network[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany:Springer, 2018:417-433.
[20] NEWELL A, HUANG A, DENG J. Associative embedding:end-to-end learning for joint detection and grouping[C]//Proceedings of Advances in Neural Information Processing Systems. Cambridge, USA:MIT Press, 2017:2277-2287.
[21] PAPANDREOU G, ZHU T, CHEN L C, et al. Personlab:person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany:Springer, 2018:269-286.

Please choose a citation manager

Content to export