遮挡与几何感知模型下的头部姿态估计方法

doi:10.19678/j.issn.1000-3428.0063878

计算机工程 ›› 2023, Vol. 49 ›› Issue (3): 296-303,311. doi: 10.19678/j.issn.1000-3428.0063878

遮挡与几何感知模型下的头部姿态估计方法

付齐^1,2,3, 谢凯^1,2,3, 文畅^3,4, 贺建飚⁵

1. 长江大学电子信息学院, 湖北荆州 434023;
2. 长江大学电工电子国家级实验教学示范中心, 湖北荆州 434023;
3. 长江大学西部研究院, 新疆克拉玛依 834000;
4. 长江大学计算机科学学院, 湖北荆州 434023;
5. 中南大学计算机学院, 长沙 410083

收稿日期:2022-01-30 修回日期:2022-04-10 发布日期:2022-05-02
作者简介:付齐(1999—),男,硕士研究生,主研方向为图形图像处理、人工智能;谢凯(通信作者),教授、博士;文畅,讲师;贺建飚,副教授。
基金资助:
新疆维吾尔自治区自然科学基金（2020D01A131）；湖北省自然科学基金（2021CFB119）。

Head Pose Estimation Method Under Occlusion and Geometric Perception Models

FU Qi^1,2,3, XIE Kai^1,2,3, WEN Chang^3,4, HE Jianbiao⁵

1. School of Electronic Information, Yangtze University, Jingzhou 434023, Hubei, China;
2. National Electrical and Electronic Experimental Teaching Demonstration Center, Yangtze University, Jingzhou 434023, Hubei, China;
3. Western Research Institute, Yangtze University, Karamay 834000, Xinjiang, China;
4. School of Computer Science, Yangtze University, Jingzhou 434023, Hubei, China;
5. College of Information Science and Engineering, Central South University, Changsha 410083, China

Received:2022-01-30 Revised:2022-04-10 Published:2022-05-02

摘要/Abstract

摘要： 头部姿态估计在人机交互、辅助驾驶等应用中起重要作用，但因受到光照变化、部分遮挡、不同面部外观差异等因素的影响，导致头部姿态估计的准确率不高。提出一种基于遮挡和几何感知模型的头部姿态估计方法，在预处理阶段采用多任务卷积神经网络进行人脸检测，减小背景环境的干扰，并进行图像增强操作以减小光照变化带来的影响。设计面部遮挡感知网络感知人脸的遮挡区域，从而提取信息量丰富的未遮挡面部特征。为充分利用面部的几何信息，采用堆叠胶囊自编码器对人脸各部分的姿态和位置进行编码，得到面部各部分间的几何关系。实验结果表明，该方法在AFLW2000数据集和BIWI数据集上的平均绝对误差分别为3.91和3.55，能有效提高头部姿态估计准确率，在复杂环境下的鲁棒性较好。

关键词: 遮挡感知, 几何感知, 头部姿态估计, 亮度均衡, 人脸检测

Abstract: Head posture estimation plays a crucial role in human-computer interaction and assisted driving.However, lighting variations, partial occlusion, and differences in face appearances degrade the accuracy of head pose estimation.Therefore, in this study, a head pose estimation method based on occlusion and geometric perception models is proposed.During the pre-processing phase, a Multi-Task Convolutional Neural Networks(MTCNN) is used for face detection to reduce the interference of the background environment, and then, an image enhancement operation is performed on the captured image to reduce the effect of illumination changes.A facial occlusion perception network is designed to perceive the occluded area of the face to extract unobstructed facial features with rich information.The posture and position of each part of the face are encoded using the Stacked Capsule Autoencoder(SCAE) to completely utilize the geometric information of the face, so that geometric relationship of each part of the face is obtained.The experimental results based on the AFLW2000 and BIWI datasets indicate that the Mean Absolute Error (MAE) values of the proposed method are 3.91 and 3.55, respectively, which can effectively help improve the accuracy of head pose estimation and its robustness in complex environments.

Key words: occlusion perception, geometric perception, head pose estimation, brightness equalization, face detection

中图分类号:

TP391.41

付齐, 谢凯, 文畅, 贺建飚. 遮挡与几何感知模型下的头部姿态估计方法[J]. 计算机工程, 2023, 49(3): 296-303,311.

FU Qi, XIE Kai, WEN Chang, HE Jianbiao. Head Pose Estimation Method Under Occlusion and Geometric Perception Models[J]. Computer Engineering, 2023, 49(3): 296-303,311.

http://www.ecice06.com/CN/Y2023/V49/I3/296

图/表 11

20230314190841

20230314190844

20230314190848

20230314190851

20230314190854

20230314190857

20230314190901

20230314190904

20230314190907

20230314190910

20230314190915

参考文献

[1] MURPHY-CHUTORIAN E, TRIVEDI M M.Head pose estimation in computer vision:a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(4):607-626.
[2] WANG K, ZHAO R, JI Q.Human computer interaction with head pose, eye gaze and body gestures[C]//Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition.Washington D.C., USA:IEEE Press, 2018:789-799.
[3] LI Y, LI J Y, JIANG X L, et al.A driving attention detection method based on head pose[C]//Proceedings of IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation.Washington D.C., USA:IEEE Press, 2019:483-490.
[4] BOSCH N, D'MELLO S K.Automatic detection of mind wandering from video in the lab and in the classroom[J].IEEE Transactions on Affective Computing, 2021, 12(4):974-988.
[5] WU G, ZHOU Z, GUO Z H.A robust method with dropblock for face anti-spoofing[C]//Proceedings of International Joint Conference on Neural Networks.Washington D.C., USA:IEEE Press, 2021:1-8.
[6] ZHAO Z Q, ZHENG Q L, ZHANG Y, et al.A head pose estimation method based on multi-feature fusion[C]//Proceedings of the 7th International Conference on Bioinformatics and Computational Biology.Washington D.C., USA:IEEE Press, 2019:150-155.
[7] LI D Q.A central profile-based 3D face pose estimation[J].Pattern Recognition, 2014, 47(2):525-534.
[8] 闵秋莎, 刘能, 陈雅婷, 等.基于面部特征点定位的头部姿态估计[J].计算机工程, 2018, 44(6):263-269. MIN Q S, LIU N, CHEN Y T, et al.Head pose estimation based on facial feature point localization[J].Computer Engineering, 2018, 44(6):263-269.(in Chinese)
[9] 刘振宇, 关彤.基于RGB-D图像的头部姿态检测[J].计算机科学, 2019, 46(S2):334-340. LIU Z Y, GUAN T.Head posture detection based on RGB-D image[J].Computer Science, 2019, 46(S2):334-340.(in Chinese)
[10] 戴惟嘉.基于LBP金字塔特征的头部姿态识别[J].信息化研究, 2016, 42(1):40-43. DAI W J.Head pose estimation based on pyramid LBP feature[J].Informatization Research, 2016, 42(1):40-43.(in Chinese)
[11] 齐永锋, 马中玉.基于深度残差网络的多损失头部姿态估计[J].计算机工程, 2020, 46(12):247-253. QI Y F, MA Z Y.Multi-loss head posture estimation based on deep residual network[J].Computer Engineering, 2020, 46(12):247-253.(in Chinese)
[12] 章惠, 张娜娜, 黄俊.优化LeNet-5网络的多角度头部姿态估计方法[J].计算机应用, 2021, 41(6):1667-1672. ZHANG H, ZHANG N N, HUANG J.Multi-angle head pose estimation method based on optimized LeNet-5 network[J].Journal of Computer Applications, 2021, 41(6):1667-1672.(in Chinese)
[13] YANG T Y, CHEN Y T, LIN Y, et al.FSA-net:learning fine-grained structure aggregation for head pose estimation from a single image[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:1087-1096.
[14] WU J, SHANG Z J, WANG K W, et al.Partially occluded head posture estimation for 2D images using pyramid HoG features[C]//Proceedings of IEEE International Conference on Multimedia & Expo Workshops.Washington D.C., USA:IEEE Press, 2019:507-512.
[15] WANG Y J.A deep coarse-to-fine network for head pose estimation from synthetic data[J].Pattern Recognition, 2019, 94(7):196-206.
[16] ZHANG W C, SHAN S G, CHEN X L, et al.Local gabor binary patterns based on kullback-leibler divergence for partially occluded face recognition[J].IEEE Signal Processing Letters, 2007, 14(11):875-878.
[17] XING J L, NIU Z H, HUANG J S, et al.Towards robust and accurate multi-view and partially-occluded face alignment[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):987-1001.
[18] ZHANG K P, ZHANG Z P, LI Z F, et al.Joint face detection and alignment using multitask cascaded convolutional networks[J].IEEE Signal Processing Letters, 2016, 23(10):1499-1503.
[19] LIU S G, ZHANG Y.Detail-preserving underexposed image enhancement via optimal weighted multi-exposure fusion[J].IEEE Transactions on Consumer Electronics, 2019, 65(3):303-311.
[20] ZHANG Y, LIU S G.Non-uniform illumination video enhancement based on zone system and fusion[C]//Proceedings of the 24th International Conference on Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:2711-2716.
[21] DRAGO F, MYSZKOWSKI K, ANNEN T, et al.Adaptive logarithmic mapping for displaying high contrast scenes[J].Computer Graphics Forum, 2003, 22(3):419-426.
[22] KOSIOREK A R, SABOUR S, TEH Y W, et al.Stacked capsule autoencoders[EB/OL].[2021-12-20].https://arxiv.org/abs/1906.06818.
[23] LEE J, LEE Y, KIM J, et al.Set transformer:a framework for attention-based permutation-invariant neural networks[EB/OL].[2021-12-20].https://arxiv.org/abs/1810.00825v3.
[24] ZHU X Y, LEI Z, LIU X M, et al.Face alignment across large poses:a 3D solution[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:146-155.
[25] FANELLI G, DANTONE M, GALL J, et al.Random forests for real time 3D face analysis[J].International Journal of Computer Vision, 2013, 101(3):437-458.
[26] SAGONAS C, TZIMIROPOULOS G, ZAFEIRIOU S, et al.300 faces in-the-wild challenge:the first facial landmark localization challenge[C]//Proceedings of IEEE International Conference on Computer Vision Workshops.Washington D.C., USA:IEEE Press, 2013:397-403.
[27] KAZEMI V, SULLIVAN J.One millisecond face alignment with an ensemble of regression trees[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2014:1867-1874.
[28] BULAT A, TZIMIROPOULOS G.How far are we from solving the 2D & 3D face alignment problem?(and a dataset of 230, 0003D facial landmarks)[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:1021-1030.
[29] RUIZ N, CHONG E, REHG J M.Fine-grained head pose estimation without keypoints[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C., USA:IEEE Press, 2018:2155-215509.
[30] ZHANG H, WANG M M, LIU Y, et al.FDN:feature decoupling network for head pose estimation[J].Proceedings of AAAI Conference on Artificial Intelligence, 2020, 34(7):12789-12796.
[31] DHINGRA N.LwPosr:lightweight efficient fine grained head pose estimation[C]//Proceedings of IEEE/CVF Winter Conference on Applications of Computer Vision.Washington D.C., USA:IEEE Press, 2022:1204-1214.
[32] DHINGRA N.HeadPosr:end-to-end trainable head pose estimation using transformer encoders[C]//Proceedings of the 16th IEEE International Conference on Automatic Face and Gesture Recognition.Washington D.C., USA:IEEE Press, 2021:1-8.
[33] HE K M, CHEN X L, XIE S N, et al.Masked autoencoders are scalable vision learners[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2022:15979-15988.

选择文件类型/文献管理软件名称

选择包含的内容

遮挡与几何感知模型下的头部姿态估计方法

Head Pose Estimation Method Under Occlusion and Geometric Perception Models

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李柯, 李邵梅, 吉立新, 刘硕. 基于自注意力胶囊网络的伪造人脸检测方法[J]. 计算机工程, 2022, 48(2): 194-200,206.
[2]	朱灵灵, 高超, 陈福才. 基于轻量级卷积神经网络的人脸检测算法[J]. 计算机工程, 2021, 47(7): 273-280.
[3]	陶施帆, 李玉峰, 黄煜峰, 蓝晓宇. 基于深度残差网络和注意力机制的人脸检测算法[J]. 计算机工程, 2021, 47(11): 276-282.
[4]	连泽宇, 田景文. 基于掩膜生成网络的遮挡人脸检测方法[J]. 计算机工程, 2021, 47(11): 292-297,304.
[5]	李祥兵, 陈炼. 基于改进Faster?RCNN的自然场景人脸检测[J]. 计算机工程, 2021, 47(1): 210-216.
[6]	郑伟成, 李学伟, 刘宏哲, 代松银. 基于深度学习的疲劳驾驶检测算法[J]. 计算机工程, 2020, 46(7): 21-29.
[7]	林凯瀚, 赵慧民, 吕巨建, 詹瑾, 刘晓勇, 陈荣军. 基于Mask R-CNN的人脸检测与分割方法[J]. 计算机工程, 2020, 46(6): 274-280.
[8]	闵秋莎,刘能,陈雅婷,王志锋. 基于面部特征点定位的头部姿态估计[J]. 计算机工程, 2018, 44(6): 263-269.
[9]	耿磊,袁菲,肖志涛,张芳,吴骏,李月龙. 基于面部行为分析的驾驶员疲劳检测方法[J]. 计算机工程, 2018, 44(1): 274-279.
[10]	李德强,李千目,孙康,朱虹. 基于二级级联支持向量机的人脸快速检测[J]. 计算机工程, 2017, 43(10): 203-208.
[11]	瞿绍军,李乔良. 基于人脸检测与细胞自动机的人物图像分割[J]. 计算机工程, 2016, 42(6): 247-254.
[12]	杨晋吉，李荣兵. 基于Adaboost与Clifford代数的人脸检测[J]. 计算机工程, 2013, 39(9): 214-217.
[13]	刘在英, 朱琳玲, 杨平. 基于肤色模型和中线定位的人脸检测算法[J]. 计算机工程, 2013, 39(1): 200-203.
[14]	何正风, 孙亚民. 基于神经网络和主元分析的人脸识别算法[J]. 计算机工程, 2012, 38(19): 175-178.
[15]	李明瑞, 傅明, 曹敦. 基于肤色检测的AdaBoost人脸检测算法改进[J]. 计算机工程, 2012, 38(19): 147-150.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

遮挡与几何感知模型下的头部姿态估计方法

Head Pose Estimation Method Under Occlusion and Geometric Perception Models

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献

相关文章 15

编辑推荐

Metrics

本文评价