基于YOLO的多模态加权融合行人检测算法

doi:10.19678/j.issn.1000-3428.0058745

摘要/Abstract

摘要： 在夜间光照不足、目标被遮挡导致信息缺失以及行人目标多尺度的情况下，可见光单模态行人检测算法的检测效果较差。为了提高行人检测器的鲁棒性，基于YOLO提出一种可见光与红外光融合的行人检测算法。使用Darknet53作为特征提取网络，分别提取2个模态的多尺度特征。对传统多模态行人检测算法所使用的concat融合方式进行改进，设计结合注意力机制的模态加权融合层，以加强对融合特征图的模态选择。在此基础上，使用多尺度的融合特征进行行人检测。实验结果表明，模态加权融合较concat融合有较大的精度提升，且该算法在夜间光照不足、目标遮挡和目标多尺度情况下检测效果良好，在KAIST数据集上的检测精度优于HalFusion和Fusion RPN+BDT等算法，检测速度也有较大提升。

关键词: 行人检测, 目标检测, 多模态算法, YOLO网络, 注意力机制

Abstract: The performance of single-modal pedestrian detection algorithms based on visible images is limited in the cases of insufficient light at night, lack of information caused by target occlusion, and multi-scale targets. In order to improve the robustness of pedestrian detectors, a YOLO-based pedestrian detection algorithm that combines visible light and infrared light is proposed. By taking Darknet53 as the feature extraction network, the multi-scale features of visible and infrared modalities are extracted. To improve the concat fusion method used by the existing multi-modal pedestrian detection algorithms, a modal weighted fusion layer combined with an attention mechanism is designed to strengthen the modal selection of the fusion feature map. On this basis, the multi-scale fusion features are used for pedestrian detection. Experimental results show that modal weighted fusion significantly improves the accuracy of concat fusion. The proposed algorithm displays excellent detection performance under the conditions of insufficient light at night, target occlusion and multi-scale targets, providing higher detection accuracy and speed than HalFusion, Fusion RPN+BDT and other algorithms on the KAIST dataset.

Key words: pedestrian detection, target detection, multi-modal algorithm, YOLO network, attention mechanism

中图分类号:

TP391

施政, 毛力, 孙俊. 基于YOLO的多模态加权融合行人检测算法[J]. 计算机工程, 2021, 47(8): 234-242.

SHI Zheng, MAO Li, SUN Jun. YOLO-Based Multi-Modal Weighted Fusion Pedestrian Detection Algorithm[J]. Computer Engineering, 2021, 47(8): 234-242.

https://www.ecice06.com/CN/Y2021/V47/I8/234

图/表 16

20210819201750

20210819201754

20210819201759

20210819201804

20210819201809

20210819201814

20210819201818

20210819201823

20210819201827

20210819201832

20210819201836

20210819201841

20210819201845

20210819201850

20210819201855

20210819201900

参考文献

[1] 张驰, 谭南林, 李国正, 等. 基于多级特征的红外图像行人检测算法[J]. 计算机工程, 2020, 46(4): 260-265. ZHANG C, TAN N L, LI G Z, et al. Pedestrian detection algorithm for infrared image based on multi-level features[J]. Computer Engineering, 2020, 46(4): 260-265.(in Chinese)
[2] NGUYEN D T, LI W, OGUNBONA P O.Human detection from images and videos:a survey[J]. Pattern Recognition, 2016, 51(C): 148-175.
[3] 高宗, 李少波, 陈济楠, 等. 基于YOLO网络的行人检测方法[J]. 计算机工程, 2018, 44(5): 215-219, 226. GAO Z, LI S B, CHEN J N, et al. Pedestrian detection method based on YOLO network[J]. Computer Engineering, 2018, 44(5): 215-219, 226.(in Chinese)
[4] DALAL N, TRIGGS B.Histograms of oriented radients for human detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2005:886-893.
[5] VIOLA P, JONES M J.Robust teal-time face detection[J]. Journal of Computer Vision, 2004, 57(2): 137-154.
[6] DOLLÁR P, WOJEK C, SCHIELE B, et al. Pedestrian detection:an evaluation of the state of the art[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 34:743-761.
[7] CHEN P H, LIN C J, SCHLKOPF B.A tutorial on ν-support vector machines[J]. Applied Stochastic Models in Business and Industry, 2005, 21(2): 111-136.
[8] FREUND Y, SCHAPIRE R E.Adecision-theoretic generalization of on-line learning and an application to boosting[J]. Journal of Computer and System Sciences, 1997, 55(1): 119-139.
[9] REN S, HE K, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.
[10] LIU W, ANGUELOV D, ERHAN D, et al. SSD:single shot multibox detector[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:21-37.
[11] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once:unified, real-time object detection[C]//Proceedings of CVPR.Washington D.C., USA:IEEE Press, 2015:779-788.
[12] LU R Q, MA H M.Semantic head enhanced pedestrian detection in a crowd[EB/OL]. [2020-05-10]. https://arxiv.org/pdf/1911.11985.pdf.
[13] ZHANG S, YANG X S, LIU Y X, et al. Asymmetric multi-stage CNNs for small-scale pedestrian detection[J]. Neurocomputing, 2020, 409:12-26.
[14] DAI J, ZHANG P P, LU H C, et al. Dynamic imposter based online instance matching for person search[J]. Pattern Recognition, 2020, 100:45-67.
[15] GE J, LUO Y, TEI G.Real-time pedestrian detection and tracking at nighttime for driver-assistance systems[J]. IEEE Transactions on Intelligent Transportation Systems, 2009, 10(2): 283-298.
[16] HWANG S, PARK J, KIM N, et al. Multispectral pedestrian detection:benchmark dataset and baseline[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2015:1037-1045.
[17] WAGNER J, FISCHER V, HERMAN M, et al. Multispectral pedestrian detection using deep fusion convolutional neural networks[C]//Proceedings of European Symposium on Artificial Neural Networks.Berlin, Germany:Springer, 2016:509-514.
[18] LIU J, ZHANG S, WANG S, et al. Multi-spectral deep neural networks for pedestrian detection[C]//Proceedings of British Machine Vision Conference.Berlin, Germany:Springer, 2016:1-13.
[19] KONIG D, ADAM M, JARVERS C, et al. Fully convolutional region proposal networks for multispectral person detection[C]//Proceedings of Computer Vision and Pattern Recognition Workshops.Washington D.C., USA:IEEE Press, 2017:243-250.
[20] YOLOv3:an incremental improvement[EB/OL]. [2020-05-10]. https://export.arxiv.org/pdf/1804.02767.
[21] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA:IEEE Press, 2016:770-778.
[22] LIN M, CHEN Q, YAN S, et al. Network in network[EB/OL]. [2020-05-10]. https://arxiv.org/pdf/1312.4400.pdf.
[23] WOO S, PARK J, LEE J Y, et al. CBAM:convolutional block attention module[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:3-19.
[24] 官大衍.可见光与长波红外图像融合的行人检测方法研究[D]. 杭州:浙江大学, 2019. GUAN D Y.Research on pedestrian detection methods via fusing visible and long-wave infrared images[D]. Hangzhou:Zhejiang University, 2019.(in Chinese)
[25] LIU S, ZHANG Y.Detail-preserving underexposed image enhancement via optimal weighted multi-exposure fusion[J]. IEEE Transactions on Consumer Electronics, 2019, 45:17-35.
[26] ZHANG Q, NIE Y, ZHANG L, et al. Underexposed video enhancement via perception-driven progressive fusion[J]. IEEE Transactions on Visualization & Computer Graphics, 2016, 22(6): 1773-1785.
[27] RIAD I.OTCBVS benchmark dataset collection[EB/OL]. [2020-05-10]. http://vcipl-okstate.org/pbvs/bench/.

选择文件类型/文献管理软件名称

选择包含的内容