基于条形池化和注意力机制的街道场景红外目标检测算法

doi:10.19678/j.issn.1000-3428.0065481

摘要/Abstract

摘要：

街道场景下的红外图像所含细节信息少、背景复杂，目前的目标检测模型存在检测精度低、检测速度慢的问题。为此，基于条形池化和注意力机制提出一种新的红外目标检测算法。使用包含条形池化和金字塔池化模块的混合池化模块改进快速空间池化金字塔模块，利用条形池化解决传统池化操作在进行目标检测时存在的特征丢失和污染问题，提高算法对长窄目标的特征提取能力，同时在孤立目标之间建立全局依赖关系，使模型收集更多的特征信息。在注意力模块中加入水平和垂直方向上的全局池化操作，以获取目标在特征图全局范围上的位置信息，将位置信息嵌入特征通道中，使算法更精准地定位目标，降低复杂背景对检测性能的影响。使用无批次归一化阻断批次归一化的估计偏移累积，解决算法性能退化问题，进一步提高算法的检测性能。在FLIR数据集上的实验结果表明，该算法的mAP(IoU值为0.5)和F1值分别达到80.7%和78.0%，相较YOLOv5分别提高了1.9和2.4个百分点。

关键词: 红外目标检测, 条形池化, 金字塔池化, 注意力机制, 无批次归一化

Abstract:

Infrared image in street scene contains less detail information and complex background, the existing target detection model exhibits low accuracy and sluggish processing speed. To address these issues, a new infrared target detection algorithm based on strip pooling and attention mechanism is proposed.The Mixed Pooling Module(MPM) includes strip pooling and the Pyramid Pooling Module(PPM) is used to improve the Spatial Pyramid Pooling Fast (SPPF) module. Strip pooling is applied to solve the feature loss and pollution issues existing in the traditional pooling operation during target detection, so as to improve the feature extraction ability for long and narrow targets, and the global dependency relationship is established between isolated targets, whereby this new method helps the model capture more enriched feature information. The global pooling operates in the horizontal direction, and vertical directions are handled by the attention module to obtain the position information of the target in the global range of the feature map, whereby the position information is embedded into the feature channel so that the algorithm can locate the target more accurately and reduce the impact of complex backgrounds on detection performance.Batch-Free Normalization(BFN) is used to address the performance degradation caused by the accumulation of the estimated offset in Batch Normalization(BN), which further improves the detection performance of the algorithm.The experimental results on FLIR dataset show that the improved algorithm has an mAP(IoU value is 0.5) of 80.7% and an F1 value of 78.0%, which are 1.9 and 2.4 percentage points higher than those of YOLOv5, respectively.

Key words: infrared target detection, strip pooling, pyramid pooling, attention mechanism, Batch-Free Normalization(BFN)

李强龙, 周新文, 位梦恩, 甘阳洲. 基于条形池化和注意力机制的街道场景红外目标检测算法[J]. 计算机工程, 2023, 49(8): 310-320.

Qianglong LI, Xinwen ZHOU, Meng'en WEI, Yangzhou GAN. Infrared Target Detection Algorithm Based on Strip Pooling and Attention Mechanism in Street Scene[J]. Computer Engineering, 2023, 49(8): 310-320.

https://www.ecice06.com/CN/Y2023/V49/I8/310

图/表 13

图1 不同的池化操作

Fig.1 Different pooling operations

图2 SPPF和SSPPF模块的整体结构

Fig.2 Overall structure of SPPF and SSPPF modules

图3 MPM模块结构

Fig.3 MPM module structure

图4 改进后的注意力模块结构

Fig.4 Improved attention module structure

图5 AC3模块结构

Fig.5 AC3 module structure

图6 Strip-YOLO模型结构

Fig.6 Strip-YOLO model structure

表1 FLIR数据集上不同模型的检测性能对比

Table 1 Comparison of detection performance of different models on FLIR dataset

检测模型	AP(IoU值为0.5)/%			P/%	R/%	mAP (IoU值为0.5)/%	F1值/%	时间/ms	参数量/10⁶	FLOPs/10⁹
检测模型	People	Bicycle	Car	P/%	R/%	mAP (IoU值为0.5)/%	F1值/%	时间/ms	参数量/10⁶	FLOPs/10⁹
Faster R-CNN	71.3	61.8	79.6	—	—	70.9	—	131.6	—	—
SSD	63.1	47.5	75.8	—	—	62.1	—	40.1	—	—
RefineDet	77.2	57.2	84.5	—	—	73.0	—	44.8	—	—
YOLOv3-tiny	71.9	51.9	85.3	79.5	61.9	69.7	69.6	9.1	8.9	13.3
YOLOv5s	82.6	63.4	90.4	80.6	71.2	78.8	75.6	10.8	7.1	16.0
Strip-YOLO	84.8	67.1	90.5	85.2	71.9	80.7	78.0	20.4	8.1	19.3

表2 不同改进方式对YOLOv5s检测性能的影响

Table 2 The effect of different improvement methods on the detection performance of YOLOv5s

SSPPF	AC3	BFN	AP(IoU值为0.5)/%			mAP (IoU值为0.5)/%	F1值/%	时间/ms	参数量/10⁶	FLOPs/10⁹
SSPPF	AC3	BFN	People	Bicycle	Car	mAP (IoU值为0.5)/%	F1值/%	时间/ms	参数量/10⁶	FLOPs/10⁹
			82.6	63.4	90.4	78.8	75.6	10.8	7.1	16.0
√			84.4	64.0	90.5	79.7	77.7	14.2	7.3	18.6
	√		83.3	63.6	90.4	79.1	77.2	16.8	7.9	16.6
		√	83.9	64.8	89.9	79.6	77.0	11.3	7.1	15.9
√	√	√	84.8	67.1	90.5	80.7	78.0	20.4	8.1	19.3

表3 不同改进方式对各个类别目标检测精确率和召回率的影响

Table 3 The effect of different improvement methods on the precision and recall of target detection in various categories %

SSPPF	AC3	BFN	P			平均P	R			平均R
SSPPF	AC3	BFN	People	Bicycle	Car	平均P	People	Bicycle	Car	平均R
			84.9	74.9	82.1	80.6	73.0	55.3	85.4	71.2
√			86.9	78.0	84.5	83.2	73.7	58.9	85.6	72.8
	√		85.8	78.9	85.6	83.4	73.6	56.6	85.0	71.8
		√	84.9	75.3	83.2	81.3	74.7	59.1	85.3	73.1
√	√	√	86.2	82.2	87.1	85.2	73.0	58.9	83.7	71.9

图7 不同改进方式对模型mAP的动态影响

Fig.7 Dynamic impact of different improvement methods on model mAP

图8 用于检测不同大小目标的特征图的热力图

Fig.8 The thermal maps of the feature maps for the detection of different size target

图9 不同改进方式在FLIR验证集上的loss值对比

Fig.9 Comparison of loss values between different improvement methods on FLIR validation set

图10 2种模型的部分检测结果

Fig.10 Partial detection results of two models

参考文献 28

1	秦鹏, 唐川明, 刘云峰, 等. 基于改进YOLOv3的红外目标检测方法. 计算机工程, 2022, 48(3): 211- 219. URL
	QIN P, TANG C M, LIU Y F, et al. Infrared target detection method based on improved YOLOv3. Computer Engineering, 2022, 48(3): 211- 219. URL
2	DAI X R, YUAN X, WEI X Y. TIRNet: object detection in thermal infrared images for autonomous driving. Applied Intelligence, 2021, 51(3): 1244- 1261. doi: 10.1007/s10489-020-01882-2
3	ZHANG H Z, LUO C B, WANG Q, et al. A novel infrared video surveillance system using deep learning based techniques. Multimedia Tools and Applications, 2018, 77(20): 26657- 26676. doi: 10.1007/s11042-018-5883-y
4	LI S S, LI Y J, LI Y, et al. YOLO-FIRI: improved YOLOv5 for infrared image object detection. IEEE Access, 2021, 9, 141861- 141875. doi: 10.1109/ACCESS.2021.3120870
5	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2014: 580-587.
6	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[EB/OL]. [2022-07-05]. https://arxiv.org/abs/1512.02325.
7	REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. [2022-07-05]. https://arxiv.org/abs/1804.02767.
8	张汝榛, 张建林, 祁小平, 等. 复杂场景下的红外目标检测. 光电工程, 2020, 47(10): 126- 135. URL
	ZHANG R Z, ZHANG J L, QI X P, et al. Infrared target detection and recognition in complex scene. Opto-Electronic Engineering, 2020, 47(10): 126- 135. URL
9	徐诚极, 王晓峰, 杨亚东. Attention-YOLO: 引入注意力机制的YOLO检测算法. 计算机工程与应用, 2019, 55(6): 13-23, 125 URL
	XU C J, WANG X F, YANG Y D. Attention-YOLO: YOLO detection algorithm that introduces attention mechanism. Computer Engineering and Applications, 2019, 55(6): 13-23, 125 URL
10	LOWE D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91- 110. doi: 10.1023/B:VISI.0000029664.99615.94
11	VIOLA P, JONES M. Rapid object detection using a boosted cascade of simple features[C]//Proceedings of 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2003: 1-9.
12	CHANG C C, LIN C J. LIBSVM. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): 1- 27.
13	FREUND Y, SCHAPIRE R. Experiments with a new boosting algorithm[EB/OL]. [2022-07-05]. https://cseweb.ucsd.edu/~yfreund/papers/boostingexperiments.pdf.
14	BREHAR R, NEDEVSCHI S. Pedestrian detection in infrared images using HOG, LBP, gradient magnitude and intensity feature channels[C]//Proceedings of the 17th IEEE International Conference on Intelligent Transportation Systems. Washington D. C., USA: IEEE Press, 2014: 1669-1674.
15	MURESAN M P, BREHAR R, NEDEVSCHI S. Vision algorithms and embedded solution for pedestrian detection with far infrared camera[C]//Proceedings of the 10th IEEE International Conference on Intelligent Computer Communication and Processing. Washington D. C., USA: IEEE Press, 2014: 133-136.
16	GHOSE D, DESAI S M, BHATTACHARYA S, et al. Pedestrian detection in thermal images using saliency maps[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 988-997.
17	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137- 1149. doi: 10.1109/TPAMI.2016.2577031
18	汪常建, 丁勇, 卢盼成. 融合改进FPN与关联网络的Faster R⁃CNN目标检测. 计算机工程, 2022, 48(2): 173- 179. URL
	WANG C J, DING Y, LU P C. Object detection using Faster R-CNN combining improved FPN and relation network. Computer Engineering, 2022, 48(2): 173- 179. URL
19	ZHANG S F, WEN L Y, BIAN X, et al. Single-shot refinement neural network for object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 4203-4212.
20	邹慧海, 侯进. 改进SSD算法的道路小目标检测研究. 计算机工程, 2022, 48(5): 281- 288. URL
	ZOU H H, HOU J. Research on road small target detection with improved SSD algorithm. Computer Engineering, 2022, 48(5): 281- 288. URL
21	HOU Q B, ZHANG L, CHENG M M, et al. Strip pooling: rethinking spatial pooling for scene parsing[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 4002-4011.
22	CAO Y, XU J R, LIN S, et al. GCNet: non-local networks meet squeeze-excitation networks and beyond[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2020: 1971-1980.
23	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 13708-13717.
24	ELFWING S, UCHIBE E, DOYA K. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Networks, 2018, 107, 3- 11. doi: 10.1016/j.neunet.2017.12.012
25	舒朗, 张智杰, 雷波. 一种针对红外目标检测的Dense-Yolov5算法研究. 光学与光电技术, 2021, 19(1): 69- 75. URL
	SHU L, ZHANG Z J, LEI B. Research on Dense-Yolov5 algorithm for infrared target detection. Optics & Optoelectronic Technology, 2021, 19(1): 69- 75. URL
26	HUANG L, ZHOU Y, WANG T, et al. Delving into the estimation shift of batch normalization in a network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2022: 753-762.
27	ZHOU T, YU Z, CAO Y, et al. Study on an infrared multi-target detection method based on the pseudo-two-stage model. Infrared Physics & Technology, 2021, 118, 103883.
28	ZHOU B L, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 2921-2929.

[1]	王炼红, 林飞鹏, 李潇瑶, 谌桂枝, 周莉. 融入课程知识图谱的KMAKT预测[J]. 计算机工程, 2024, 50(7): 23-31.
[2]	钱来, 赵卫伟. 基于对比学习和注意力机制的文本分类方法[J]. 计算机工程, 2024, 50(7): 104-111.
[3]	刘建敏, 林晖, 汪晓丁. 基于图注意力机制的无地图场景轨迹预测方法[J]. 计算机工程, 2024, 50(7): 144-153.
[4]	屠乃威, 焦猛, 阎馨. 复杂环境下输电线路鸟巢目标图像检测模型[J]. 计算机工程, 2024, 50(7): 216-226.
[5]	肖慈, 徐杨, 张永丹, 冯明文, 黄易仟. 结合注意力和低光增强的夜间语义分割[J]. 计算机工程, 2024, 50(7): 271-281.
[6]	张锡英, 孙守东, 于海浩, 边继龙. 基于空间传播的多视图三维重建[J]. 计算机工程, 2024, 50(7): 293-302.
[7]	贵向泉, 刘世清, 李立, 秦庆松, 李唐艳. 基于改进YOLOv8的景区行人检测算法[J]. 计算机工程, 2024, 50(7): 342-351.
[8]	程腾腾, 姚春龙, 于晓强, 李旭, 王庆丰. 基于多头注意力机制融合常识知识的共情对话生成[J]. 计算机工程, 2024, 50(6): 94-101.
[9]	更藏措毛, 黄鹤鸣, 杨毅杰. 融合多尺度特征与上下文信息的语音增强方法[J]. 计算机工程, 2024, 50(6): 138-147.
[10]	贺姗, 蔺素珍, 王彦博, 李大威. 基于特征融合的多波段图像描述生成方法[J]. 计算机工程, 2024, 50(6): 236-244.
[11]	杨硕, 王一丁. 基于改进薄板样条运动模型的人脸动画算法[J]. 计算机工程, 2024, 50(6): 255-265.
[12]	陈晓玉, 沈晨, 沈阅, 孔德明. 基于改进SwiftNet的堆场图像实时分割网络[J]. 计算机工程, 2024, 50(6): 296-303.
[13]	游奔, 李晓红, 姚锦, 冯绍杰. 基于多粒度图与注意力机制的半监督短文本分类[J]. 计算机工程, 2024, 50(5): 83-90.
[14]	代巍, 王丰羽, 冀常鹏. 基于情感增强与双图卷积网络的方面级情感分析[J]. 计算机工程, 2024, 50(5): 120-127.
[15]	刘仕兵, 周诗涵. 高铁接触网绝缘子检测算法研究[J]. 计算机工程, 2024, 50(5): 200-208.

选择文件类型/文献管理软件名称

选择包含的内容