基于改进堆叠沙漏网络的动物姿态估计

doi:10.19678/j.issn.1000-3428.0063699

摘要/Abstract

摘要： 堆叠沙漏网络在动物姿态估计任务中得到成功应用，但其编码-解码过程容易丢失网络的浅层信息，影响了检测精度。针对该问题，提出基于改进堆叠沙漏网络的动物姿态估计模型。设计一种基于SE注意力机制的多尺度最大池化模块，完成多尺度信息的提取，解决池化后信息大量丢失的问题，提高网络获取全局信息的能力，同时提出多级特征融合方法，充分提取和融合特征信息。在此基础上，嵌入CBAM注意力机制，学习特征融合权重，提升网络对多通道信息的提取能力，抑制无效特征，使网络提取出更丰富、细腻的特征。在TigDog数据集和合成动物数据集上进行训练和测试，结果表明，该模型估计性能优于Syn、BDL、CyCADA和CC-SSL模型，其对于马和老虎的PCK@0.05指标较次优的CC-SSL模型分别提高4.6%和3.5%。消融实验结果也验证了整个网络体系结构的先进性和有效性。

关键词: 动物姿态估计, 堆叠沙漏网络, 多尺度信息提取, 注意力机制, 特征融合

Abstract: The Stacked Hourglass Network(SHN) has been successfully applied in animal pose estimation tasks, but it easily loses the shallow information of the network during the encode-decode process, which affects the detection accuracy.To address this problem, an animal pose estimation model based on improved SHN is proposed. A multi-scale maximum pooling module based on Squeeze-and-Excitation(SE) attention is designed to complete the extraction of multiscale information, address the large loss of information after pooling, and improve the ability of the network to obtain global information.Simultaneously, a multilevel feature fusion method is proposed to fully extract and fuse the feature information.Based on the Convolutional Block Attention Module(CBAM), an attention mechanism is embedded to learn feature fusion weights, improve the network's ability to extract multichannel information, suppress invalid features, and make the network extract richer and more detailed features.The proposed model is trained and tested on the TigDog and synthetic animal datasets.The results show that the proposed model outperforms the Syn, BDL, CyCADA, and CC-SSL models;its PCK@0.05 index for horses and tigers is 4.6% and 3.5% higher than that of the suboptimal CC-SSL model, respectively.The experimental ablation results also verify the advancement and effectiveness of the entire network architecture.

Key words: animal pose estimation, Stacked Hourglass Network(SHN), multi-scale information extraction, attention mechanism, feature fusion

中图分类号:

TP18

张雯雯, 徐杨, 白芮, 陈娜. 基于改进堆叠沙漏网络的动物姿态估计[J]. 计算机工程, 2023, 49(2): 263-270.

ZHANG Wenwen, XU Yang, BAI Rui, CHEN Na. Animal Pose Estimation Based on Improved Stacked Hourglass Network[J]. Computer Engineering, 2023, 49(2): 263-270.

http://www.ecice06.com/CN/Y2023/V49/I2/263

图/表 15

20230216182625

20230216182629

20230216182632

20230216182635

20230216182638

20230216182641

20230216182645

20230216182648

20230216182651

20230216182655

20230216182659

20230216182702

20230216182706

20230216182709

20230216182713

参考文献

[1] LI X H, LIU J F.A review of the research on two-dimensional human posture estimation[J].Modern Computer, 2019(22):33-37.
[2] 刘红, 马杰, 柴玉晶.基于改进沙漏网络的人体姿态估计模型[J].激光与光电子学进展, 2021, 58(20):385-393. LIU H, MA J, CHAI Y J.Human pose estimation model based on improved hourglass network[J].Laser & Optoelectronics Progress, 2021, 58(20):385-393.(in Chinese)
[3] DANTONE M, GALL J, LEISTNER C, et al.Human pose estimation using body parts dependent joint regressors[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2013:3041-3048.
[4] 罗梦诗, 徐杨, 叶星鑫.融入双注意力的高分辨率网络人体姿态估计[J].计算机工程, 2022, 48(2):314-320. LUO M S, XU Y, YE X X.Human pose estimation using high resolution network with dual attention[J].Computer Engineering, 2022, 48(2):314-320.(in Chinese)
[5] 陈永康, 宋亚男, 何嘉俊, 等.基于深度学习的动物姿态估计和状态评估研究[J].电子世界, 2019(5):47-48. CHEN Y K, SONG Y N, HE J J, et al.Research on animal posture estimation and state evaluation based on deep learning[J].Electronics World, 2019(5):47-48.(in Chinese)
[6] SAPP B, TASKAR B.MODEC:multimodal decomposable models for human pose estimation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2013:3674-3681.
[7] BIGGS B, RODDICK T, FITZGIBBON A, et al.Creatures great and SMAL:recovering the shape and motion of animals from video[C]//Proceedings of Asian Conference on Computer Vision.Berlin, Germany:Springer, 2019:3-19.
[8] HOFFMAN J, TZENG E, PARK T, et al.CyCADA:cycle-consistent adversarial domain adaptation[EB/OL].[2021-11-10].https://arxiv.org/abs/1711.03213.
[9] TREMBLAY J, PRAKASH A, ACUNA D, et al.Training deep networks with synthetic data:bridging the reality gap by domain randomization[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C., USA:IEEE Press, 2018:1-10.
[10] CHANG A X, FUNKHOUSER T, GUIBAS L, et al.ShapeNet:an information-rich 3D model repository[EB/OL].[2021-12-10].http://www.shapenet.orgAngel.
[11] CAO J K, TANG H Y, FANG H S, et al.Cross-domain adaptation for animal pose estimation[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:9497-9506.
[12] ZUFFI S, KANAZAWA A, BLACK M J.Lions and tigers and bears:capturing non-rigid, 3D, articulated shape from images[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:3955-3963.
[13] VAROL G, ROMERO J, MARTIN X, et al.Learning from synthetic humans[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:4627-4635.
[14] TOSHEV A, SZEGEDY C.DeepPose:human pose estimation via deep neural networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2013:1653-1660.
[15] CHEN Y L, WANG Z C, PENG Y X, et al.Cascaded pyramid network for multi-person pose estimation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:7103-7112.
[16] NEWELL A, YANG K Y, DENG J.Stacked hourglass networks for human pose estimation[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:483-499.
[17] HUA G G, LI L H, LIU S G.Multipath affinage stacked-hourglass networks for human pose estimation[J].Frontiers of Computer Science, 2020, 14(4):1-12.
[18] 王琦, 郑飂默, 王诗宇, 等.一种基于轻量级堆叠沙漏网络的机械臂姿态估计方法[J].小型微型计算机系统, 2022, 43(11):2370-2374. WANG Q, ZHENG L M, WANG S Y, et al.Robotic arm pose estimation method based on lightweight stacked hourglass network[J].Journal of Chinese Computer Systems, 2022, 43(11):2370-2374.(in Chinese)
[19] HU J, SHEN L, ALBANIE S, et al.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8):2011-2023.
[20] WOO S, PARK J, LEE J Y, et al.CBAM:convolutional block attention module[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:3-19.
[21] HE K M, ZHANG X Y, REN S Q, et al.Deep residual learning for image recognition[EB/OL].[2021-11-10].https://arxiv.org/abs/1512.03385.
[22] TIELEMAN T, HINTON G.RMSprop:divide the gradient by a running average of its recent magnitude[EB/OL].[2021-11-10].https://amara.org/videos/vrXNiLBHyW92/en/180511.
[23] MU J T, QIU W C, HAGER G, et al.Learning from synthetic animals[EB/OL].[2021-11-10].https://arxiv.org/abs/1912.08265.
[24] ZHU J Y, PARK T, ISOLA P, et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:2242-2251.
[25] LI Y S, YUAN L, VASCONCELOS N.Bidirectional learning for domain adaptation of semantic segmentation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:6929-6938.

选择文件类型/文献管理软件名称

选择包含的内容