基于深度学习的疲劳驾驶检测算法

doi:10.19678/j.issn.1000-3428.0055912

摘要/Abstract

摘要： 为实现复杂驾驶环境下驾驶人员疲劳状态识别与预警，提出基于深度学习的疲劳驾驶检测算法。利用基于shuffle-channel思想的MTCNN模型检测常规摄像头实时采集的驾驶人员人脸图像，使用PFLD深度学习模型进行人脸关键点检测以定位眼部、嘴部和头部位置，从中提取眨眼频率、嘴巴张开程度和点头频率等特征参数，并通过多特征融合策略获取驾驶人员疲劳状态，从而实现疲劳驾驶的有效预警。实验结果表明，该算法给出的疲劳驾驶预警结果均未出现误判情况，具有较高的检测准确率和较好的鲁棒性。

关键词: 疲劳驾驶检测, 疲劳特征提取, PERCLOS值, 人脸检测, 人脸关键点检测, 头部姿态估计

Abstract: To realize identification and warning of fatigue driving detection in complex driving environment,this paper proposes an algorithm for fatigue driving detection based on deep learning. The algorithm uses the MTCNN model based on the shuffle-channel concept to detect the facial images of drivers collected in real time by normal cameras.Then the PFLD deep learning model is used for facial keypoint detection to locate the eyes,the mouth and the head,so as to extract the feature parameters including the blinking rate,the extent to which the mouth opens,and the nodding frequency.Finally,based on the multi-feature fusion strategy,the fatigue state of the driver is obtained to implement effective alarming for fatigue driving.Experimental results show that false warning do not occur in fatigue driving warning generated by the proposed algorithm,which means the proposed algorithm has a high detection accuracy and robustness.

Key words: fatigue driving detection, fatigue feature extraction, PERCLOS value, face detection, face keypoint detection, head pose estimation

中图分类号:

TP391

郑伟成, 李学伟, 刘宏哲, 代松银. 基于深度学习的疲劳驾驶检测算法[J]. 计算机工程, 2020, 46(7): 21-29.

ZHENG Weicheng, LI Xuewei, LIU Hongzhe, DAI Songyin. Fatigue Driving Detection Algorithm Based on Deep Learning[J]. Computer Engineering, 2020, 46(7): 21-29.

http://www.ecice06.com/CN/Y2020/V46/I7/21

参考文献

[1] PAN Zhigeng,LIU Rongfei,ZHANG Mingmin.Research on fatigue driving detection algorithm based on fuzzy comprehensive evaluation[J].Journal of Software,2019,30(10):2954-2963.(in Chinese)潘志庚,刘荣飞,张明敏.基于模糊综合评价的疲劳驾驶检测算法研究[J].软件学报,2019,30(10):2954-2963.
[2] DINGES D F,GRACE R.PERCLOS:a valid psychophysiological measure of alertness as assessed by psychomotor vigilance:FHWA-MCRT-98-006[R].Washington D.C.,USA:Federal Highway Administration,1998.
[3] JIE Z,MAHMOUD M,STAFFORD-FRASER Q,et al.Analysis of yawning behavior in spontaneous expressions of drowsy drivers[C]//Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition.Washington D.C.,USA:IEEE Press,2018:571-576.
[4] MIN Qiusha,LIU Neng,CHEN Yating,et al.Head pose estimation based on facial feature point localization[J].Computer Engineering,2018,44(6):263-269.(in Chinese)闵秋莎,刘能,陈雅婷,等.基于面部特征点定位的头部姿态估计[J].计算机工程,2018,44(6):263-269.
[5] RUIZ N,CHONG E,REHG J M.Fine-grained head pose estimation without keypoints[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2018:2074-2083.
[6] HUANG Baisheng,JIANG Tao,ZHOU Weiguang.Performance analysis of HRRP ship target recognition based on deep learning[J].Modern Radar,2020,41(2):54-57.(in Chinese)黄柏圣,江涛,周伟光.基于深度学习的HRRP舰船目标识别性能分析.现代雷达,2020,41(2):54-57.
[7] LI Shiguo,ZHANG Ruiguo,SUN Jingming,et al.A study on the architecture of radar ATR based on deep learning[J].Modern Radar,2019,40(11):57-61.(in Chinese)李士国,张瑞国,孙晶明,等.基于深度学习的雷达自动目标识别架构研究.现代雷达,2019,40(11):57-61.
[8] REN S,HE K,GIRSHICK R,et al.Faster R-CNN:towards real-time object detection with region proposal networks[EB/OL].[2019-08-04].https://arxiv.org/abs/1506.01497.
[9] REDMON J,FARHADI A.YOLOv3:an incremental improvement[EB/OL].[2019-08-04].https://arxiv.org/abs/1804.02767.
[10] CHOI J,CHANG H J,FISCHER T,et al.Context-aware deep feature compression for high-speed visual tracking[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:479-488.
[11] LU Huchuan,LI Peixia,WANG Dong.Visual object tracking:a survey[J].Pattern Recognition and Artificial Intelligence,2018,31(1):61-76.(in Chinese)卢湖川,李佩霞,王栋.目标跟踪算法综述[J].模式识别与人工智能,2018,31(1):61-76.
[12] SCHROFF F,KALENICHENKO D,PHILBIN J.Facenet:a unified embedding for face recognition and clustering[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:815-823.
[13] ZOU Guofeng,FU Guixia,GAO Mingliang,et al.Pose varied face recognition based on self learning deep convolutional neural network[J].Journal of Chinese Computer Systems,2018,39(6):1156-1162.(in Chinese)邹国锋,傅桂霞,高明亮,等.基于自学习深度卷积神经网络的姿态变化人脸识别[J].小型微型计算机系统,2018,39(6):1156-1162.
[14] ANDERSON P,HE X,BUEHLER C,et al.Bottom-up and top-down attention for image captioning and casual question answering[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:6077-6086.
[15] ZHANG K P,ZHANG Z P,LI Z F,et al.Joint face detection and alignment using multitask cascaded convolutional networks[J].IEEE Signal Processing Letters,2016,23(10):1499-1503.
[16] GUO Xiaojie,LI Siyuan,YU Jinke,et al.PFLD:a practical facial landmark detector[EB/OL].[2019-08-04].https://arxiv.org/abs/1902.10859.
[17] ZHANG Xiangyu,ZHOU Xinyu,LIN Mengxiao,et al.ShuffleNet:an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:6848-6856.
[18] CHOLLET F.Xception:deep learning with depthwise separable convolutions[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:1251-1258.
[19] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2019-08-04].https://arxiv.org/abs/1409.1556.
[20] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.
[21] SANDLER M,HOWARD A,ZHU M,et al.MobileNetv2:inverted residuals and linear bottlenecks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:4510-4520.
[22] HU G S,YAN F,JOSEF K,et al.Efficient 3D morphable face model fitting[J].Pattern Recognition,67(C):366-379.
[23] FERRARIO V F,SFORZA C,SERRAO G,et al.Active range of motion of the head and cervical spine:a three-dimensional investigation in healthy young adults[J].Journal of Orthopaedic Research,2002,20(1):122-129.
[24] YANG S,LUO P,LOY C C,et al.WIDER FACE:a face detection benchmark[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:5525-5533.
[25] SAGONAS C,ANTONAKOS E,TZIMIROPOULOS G,et al.300 faces in-the-wild challenge:database and results[J].Image and Vision Computing,2016,47:3-18.
[26] KOESTINGER M,WOHLHART P,ROTH P M,et al.Annotated facial landmarks in the wild:a large-scale,real-world database for facial landmark localization[C]//Proceedings of 2011 IEEE International Conference on Computer Vision Workshops.Washington D.C.,USA:IEEE Press,2011:2144-2151.
[27] BELHUMEUR P N,JACOBS D W,KRIEGMAN D J,et al.Localizing parts of faces using a consensus of exemplars[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(12):2930-2940.
[28] ZHU X,RAMANAN D.Face detection,pose estimation,and landmark localization in the wild[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2012:2879-2886.
[29] LE V,BRANDT J,LIN Z,et al.Interactive facial feature localization[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2012:679-692.
[30] SAGONAS C,TZIMIROPOULOS G,ZAFEIRIOU S,et al.300 faces in-the-wild challenge:the first facial landmark localization challenge[C]//Proceedings of IEEE International Conference on Computer Vision Workshops.Washington D.C.,USA:IEEE Press,2013:397-403.
[31] ABTAHI S,OMIDYEGANEH M,SHIRMOHAMMADI S,et al.YawDD:a yawning detection dataset[C]//Proceedings of the 5th ACM Multimedia Systems Conference.New York,USA:ACM Press,2014:24-28.

选择文件类型/文献管理软件名称

选择包含的内容