融合人眼特征与深度学习的疲劳驾驶检测模型

doi:10.19678/j.issn.1000-3428.0059421

摘要/Abstract

摘要： 针对现有疲劳驾驶检测技术不能有效平衡准确性和实时性的问题，通过融合人眼特征与深度学习，构建一种新的疲劳驾驶检测模型。设计GP-VGG16网络进行眼部状态识别，通过将人工先验信息集成到轻量级深度网络中，提高眼部状态识别的准确性、稳定性和实时性。在此基础上，利用眼部特征-疲劳等级模型将疲劳状态划分为9个等级，定量估计驾驶员状态，同时基于少样本学习建立高效的自动标签生成网络，减少对大量无标签驾驶数据的语义标注。实验结果表明，该模型的准确率达到97.1%，运行速度达到39.96 frame/s，能够有效提高驾驶员疲劳状态识别的准确性与时效性。

关键词: 疲劳驾驶检测, 深度学习, 少样本学习, GP-VGG16网络, 积分投影

Abstract: To address the problem that the existing fatigue driving detection technology cannot balance the accuracy and real-time performance effectively, a fatigue driving detection model based on human eye features and deep learning is proposed. GP-VGG16 network is designed for eye state recognition. By integrating artificial prior information into lightweight deep network, the accuracy, stability and real-time performance of eye state recognition are effectively improved. On this basis, the eye feature-fatigue level model divides the fatigue state into nine grades, and quantitatively estimates the driver's state. At the same time, an efficient automatic label generation network is established based on few-shot learning to reduce the semantic annotation of a large number of unlabeled driving data. The experimental results show that the accuracy of this method is 97.1%, and the running speed is 39.96 frame/s. The method effectively improves the accuracy and real-time performance of driver fatigue state recognition.

Key words: fatigue driving detection, deep learning, few-shot learning, GP-VGG16 network, integral projection

中图分类号:

TP391

樊星, 刘占文, 林杉, 窦瑞娟. 融合人眼特征与深度学习的疲劳驾驶检测模型[J]. 计算机工程, 2021, 47(8): 243-250.

FAN Xing, LIU Zhanwen, LIN Shan, DOU Ruijuan. Fatigue Driving Detection Model Fusing with Human Eye Features and Deep Learning[J]. Computer Engineering, 2021, 47(8): 243-250.

http://www.ecice06.com/CN/Y2021/V47/I8/243

图/表 14

20210819201909

20210819201913

20210819201918

20210819201922

20210819201927

20210819201943

20210819201948

20210819201952

20210819201957

20210819202002

20210819202007

20210819202012

20210819202018

20210819202023

参考文献

[1] KAPLAN S, GUVENSAN M A, YAVUZ A G, et al. Driver behavior analysis for safe driving:a survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(6): 3017-3032.
[2] CHAI M, LI S W, SUN W C, et al. Drowsiness monitoring based on steering wheel status[J]. Transportation Research Part D:Transport and Environment, 2019, 66:95-103.
[3] MCDONALD A D, SCHWARZ C, LEE J D, et al. Real time detection of drowsiness related lane departures using steering wheel angle[C]//Proceedings of the 56th Annual Meeting of the Human Factors and Ergonomics Society.Boston, USA:[s.n.], 2012:1-5.
[4] 郭孜政, 牛琳博, 吴志敏, 等. 基于EEG的驾驶疲劳识别算法及其有效性验证[J]. 北京工业大学学报, 2017, 43(6): 929-934. GUO Z Z, NIU L B, WU Z M, et al. Driving fatigue recognition algorithm based on EEG and its validity verification[J]. Journal of Beijing University of Technology, 2017, 43(6): 929-934.(in Chinese)
[5] 徐礼胜, 张闻勖, 庞宇轩, 等. 基于短时心电信号的疲劳驾驶检测算法[J]. 东北大学学报:自然科学版, 2019, 40(7): 937-941. XU L S, ZHANG W X, PANG Y X, et al. Fatigue driving detection algorithm based on short time ECG signal[J]. Journal of Northeast University:Natural Science Edition, 2019, 40(7): 937-941.(in Chinese)
[6] LUO H, QIU T, LIU C, et al. Research on fatigue driving detection using forehead EEG based on adaptive multi-scale entropy[J]. Biomedical Signal Processing and Control, 2019, 51:50-58.
[7] GROMER M, SALB D, WALZER T, et al. ECG sensor for detection of driver's drowsiness[J]. Procedia Computer Science, 2019, 159:1938-1946.
[8] 王琳, 化成城, 姜鑫, 等. 基于颈腰部肌电及脑电信号的疲劳驾驶检测[J]. 东北大学学报(自然科学版), 2018, 39(1): 102-107. WANG L, HUA C C, JIANG X, et al. Fatigue driving detection based on cervical and lumbar EMG and EEG signals[J]. Journal of Northeast University:Natural Science Edition, 2018, 39(1): 102-107.(in Chinese)
[9] ARTANTO D, SULISTYANTO M P, PRANOWO I D, et al. Drowsiness detection system based on eye-closure using a low-cost EMG and ESP8266[C]//Proceedings of International Conference on Information Technology, Information Systems and Electrical Engineering.Washington D.C., USA:IEEE Press, 2017:1-5.
[10] POURSADEGHIYAN M, MAZLOUMI A, SARAJI G N, et al. Using image processing in the proposed drowsiness detection system design[J]. Iranian Journal of Public Health, 2018, 47(9): 1371-1378.
[11] KNAPIK M, CYGANEK B.Driver's fatigue recognition based on yawn detection in thermal images[J]. Neurocomputing, 2019, 338:274-292.
[12] 刘明周, 蒋倩男, 扈静.基于面部几何特征及手部运动特征的驾驶员疲劳检测[J]. 机械工程学报, 2019, 55(2): 32-40. LIU M Z, JIANG Q N, HU J.Driver fatigue detection based on facial geometric features and hand movement features[J]. Journal of Mechanical Engineering, 2019, 55(2): 32-40.(in Chinese)
[13] 王彦秋, 冯英伟.一种基于卷积神经网络的人脸识别改进算法[J]. 半导体光电, 2020, 41(4): 58-586. WANG Y Q, FENG Y W.An improved face recognition algorithm based on convolutional neural network[J]. Semiconductor Optoelectronics, 2020, 41(4): 58-586.(in Chinese)
[14] PARK S, PAN F, KANG S, et al. Driver drowsiness detection system based on feature representation learning using various deep networks[C]//Proceedings of ACCV'16.Berlin, Germany:Springer, 2016:154-164.
[15] ALEX K, SUTSKEVER I, HINTON G E, et al. ImageNet classification with deep convolutional neural networks[C]//Proceedings of NIPS'12.Berlin, Germany:Springer, 2012:1-5.
[16] PARKHI O M, VEDALDI A, ZISSERMAN A, et al. Deep face recognition[C]//Proceedings of BMVC'15.Swansea, UK:[s.n.], 2015:1-5.
[17] DONAHUE J, HENDRICKS L A, ROHRBACH M, et al. Long-term recurrent convolutional networks for visual recognition and description[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 677-691.
[18] 郑伟成, 李学伟, 刘宏哲, 代松银.基于深度学习的疲劳驾驶检测算法[J]. 计算机工程, 2020, 46(7): 21-29. ZHENG W C, LI X W, LIU H Z, et al. Fatigue driving detection algorithm based on deep learning[J]. Computer Engineering, 2020, 46(7): 21-29.(in Chinese)
[19] ZHANG K, ZHANG Z, LI Z, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.
[20] REDDY B, KIM Y H, YUN S, et al. Real-time driver drowsiness detection for embedded system using model compression of deep neural networks[C]//Proceedings of Computer Vision & Pattern Recognition Workshops.Washington D.C., USA:IEEE Press, 2017:1-5.
[21] MIAO X, ZHEN X, LIU X, et al. Direct shape regression networks for end-to-end face alignment[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018.
[22] LIAO S, JAIN A K, LI S Z.Partial face recognition:alignment-free approach[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2013, 35(5): 1193-1205.
[23] SIMONYAN K, ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[C]//Proceedings of ICLR'15.Washington D.C., USA:IEEE Press, 2015:1-14.
[24] HAN K, WANG Y H, TIAN Q, et al. GhostNet:more features from cheap operations[EB/OL]. (2019-11-27)[2020-05-10]. https://arxiv.org/abs/1911.11907.
[25] SHAHID A, WILKINSON K, MARCU S, et al. Karolinska Sleepiness Scale(KSS)[M]//SHAHID A, WILKINSON K, MARCU S, et al. STOP, THAT and One Hundred Other Sleep Scales.Berlin, Germany:Springer, 2011:1-5.
[26] SUNG F, YANG Y, ZHANG L, et al. Learning to compare:relation network for few-shot learning[EB/OL]. (2018-03-27)[2020-05-10]. https://arxiv.org/abs/1711.06025.
[27] MASSOZ Q, LANGOHR T, FRANCOIS C, et al. The ULg multimodality drowsiness database(called DROZY) and examples of use[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision.Washington D.C., USA:IEEE Press, 2016:1-5.
[28] WENG C H, LAI Y H, LAI S H.Driver drowsiness detection via a hierarchical temporal deep belief network[C]//Proceedings of the 13th Asian Conference on Computer Vision.Taipei, China:[s.n.], 2016:1-5.
[29] QIN X, TAN X, CHEN S.Tri-Subject kinship verification:understanding the core of a family[J]. IEEE Transactions on Multimedia, 2015, 17(10): 1855-1867.
[30] ZJU eyeblink database, bibinfonote[EB/OL]. (2017-09-26)[2020-05-10]. http://www.cs.zju.edu.cn/gpan/database/db_blink.html.
[31] WANG X S, XU C.Driver drowsiness detection based on non-intrusive metrics considering individual specifics[J]. Accident Analysis and Prevention, 2016, 95(Part B): 350-357.
[32] REN S, HE K, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.

选择文件类型/文献管理软件名称

选择包含的内容