Combining Lightweight and Multiscale Fusion for Traffic Sign Detection Algorithm

doi:10.19678/j.issn.1000-3428.0068269

Abstract

Abstract:

Traffic sign detection is critical for autonomous driving is of essence, as timely and accurate identification of these signs enhances driving safety and reduces the risk of traffic accidents. However, in complex environments, small-sized and obstructed traffic signs are often missed or incorrectly identified. To address this problem, a novel traffic-sign detection network architecture, Multiple You Only Look Once (M-YOLO), is proposed based on the YOLOv8 framework. M-YOLO integrates lightweight design and multiscale fusion to achieve precise detection tasks. A variant, Multiple YOLO small (M-YOLOs), is developed by adjusting network depth, and a lighter version, Multiple YOLO nano (M-YOLOn), is created to meet detection requirements in various environments. To enhance the detection of small traffic signs and mitigate feature loss, the network's feature learning capability is improved by incorporating a small target detection layer. This layer preserves more feature information, and a multiscale feature pyramid fusion network, Multiple Path Aggregation Network (MPANet), is proposed to reduce the dimensionality of shallow feature maps while using skip connections to fuse additional image feature information. Furthermore, a Bi-level Routing and Spatial Attention (BRSA) module is introduced to combine sparse and spatial attention, extracting both global and local positional information while reducing interference from complex backgrounds. To optimize the model, two lightweight and efficient modules, BBot and C2fGhost, are developed to reduce parameters and enhance computational speed. Experimental results demonstrated that M-YOLO reduced the number of parameters by approximately one-third compared to YOLOv8. On the TT100K and German Traffic Sign Detection Benchmark (GTSDB) datasets, the detection accuracy of M-YOLOs improved by 9.7 and 2.1 percentage points, respectively, and M-YOLOn achieved improvements of 14.5 and 2.6 percentage points. This lightweight approach led to enhanced detection performance, effectively addressing the issue of information loss during feature extraction from flat feature maps and significantly reducing unnecessary computations in the model. The effectiveness of the M-YOLO architecture is validated on a dataset collected from rea-world scenarios, demonstrating its practical value in traffic sign detection tasks.

Key words: Convolutional Neural Network(CNN), lightweight models, object detection, attention module, multi-scale fusion

摘要：

交通标志检测在自动驾驶领域具有重要的应用价值, 及时准确地检测交通目标对提高驾驶安全性和预防交通事故具有重要意义。针对交通标志尺寸小, 易受遮挡, 在复杂环境下容易出现漏检、错检等问题, 在YOLOv8的结构基础上提出一种结合轻量化与多尺度融合的交通标志检测网络架构M-YOLO, 构建M-YOLOs模型来应对高精度需求的检测任务, 并调整网络深度得到更轻量化的M-YOLOn模型来解决不同环境下的检测需求。首先针对交通标志目标尺寸小、图像特征流失的问题, 通过增加小目标检测层, 保留更多的特征信息, 提高网络对于小目标的特征学习能力。提出高效多尺度特征金字塔融合网络MPANet, 将浅层特征图进行降维与跳跃连接, 从而融合更多的图像特征信息。然后提出融合稀疏注意力和空间注意力的BRSA注意力模块, 有效提取全局和局部的位置信息, 减少复杂背景下对于关键信息的干扰。最后设计两种轻量高效的BBot模块和C2fGhost模块, 以提高模型运算速度并减少参数量。实验结果表明, M-YOLO相较于YOLOv8, 参数量降低约1/3。在TT100K数据集和GTSDB数据集上, M-YOLOs检测精度分别提升了9.7和2.1个百分点, M-YOLOn检测精度分别提升了14.5和2.6个百分点, 在轻量化的同时具备更高的检测效果。M-YOLO架构解决了浅层特征图在特征提取过程中信息丢失的问题, 并显著降低模型特征提取过程中冗余的计算开销, 在实景采集的数据集上证实效果有效, 表明在交通标志检测任务中具有应用价值。

关键词: 卷积神经网络, 轻量化模型, 目标检测, 注意力模块, 多尺度融合

LAN Hong, WANG Huizhao. Combining Lightweight and Multiscale Fusion for Traffic Sign Detection Algorithm[J]. Computer Engineering, 2024, 50(10): 381-392.

兰红, 王惠钊. 结合轻量化与多尺度融合的交通标志检测算法[J]. 计算机工程, 2024, 50(10): 381-392.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068269

https://www.ecice06.com/EN/Y2024/V50/I10/381

Figures/Tables 16

Fig.1 M-YOLO network structure

Fig.2 MPANet schematic diagram

Fig.3 ResNet Bottleneck and BRSA Bottleneck Structure Comparison

Fig.4 Structure of BRSA attention module

Fig.5 Structure of Conv and GhostConv

Fig.6 Structure of C2fGhost

Fig.7 Comparison of detection performance for various categories in TT100K dataset

Fig.8 TT100K dataset detection results

Fig.9 GTSDB dataset detection results

Fig.10 Real time sampling detection results

References 37

1	董文轩, 梁宏涛, 刘国柱, 等. 深度卷积应用于目标检测算法综述. 计算机科学与探索, 2022, 16(5): 1025- 1042.
	DONG W X, LIANG H T, LIU G Z, et al. Review of deep convolution applied to target detection algorithms. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 1025- 1042.
2	刘洪江, 王懋, 刘丽华, 等. 基于深度学习的小目标检测综述. 计算机工程与科学, 2021, 43(8): 1429- 1442.
	LIU H J, WANG M, LIU L H, et al. A survey of small object detection based on deep learning. Computer Engineering & Science, 2021, 43(8): 1429- 1442.
3	HINTON G E, SALAKHUTDINOV R R. Reducing thedimensionality of data with neural networks. Science, 2006, 313(5786): 504- 507. doi: 10.1126/science.1127647
4	GIRSHICK R, DONAHUE J, DARRELL T, et al. Richfeature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2014: 580-587.
5	GIRSHICK R. Fast R-CNN[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2015: 1440-1448.
6	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137- 1149. doi: 10.1109/TPAMI.2016.2577031
7	CAI Z W, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 6154-6162.
8	CAO J L, CHOLAKKAL H, ANWER R M, et al. D2Det: towards high quality object detection and instance segmentation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 11482-11491.
9	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[M]. Berlin, Germany: Springer, 2016.
10	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 779-788.
11	刘人玮, 李天昀, 张浩庭, 等. 基于YOLOX的跳频信号检测. 电讯技术, 2023, 63(7): 933- 940.
	LIU R W, LI T Y, ZHANG H T, et al. Frequency hopping signal detection based on YOLOX. Telecommunication Engineering, 2023, 63(7): 933- 940.
12	REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. [2023-07-10]. https://pjreddie.com/media/files/papers/YOLOv3.pdf.
13	LI C Y, LI L, JIANG H L, et al. YOLOv6: a single-stage object detection framework for industrial applications[EB/OL]. [2023-07-10]. https://arxiv.org/abs/2209.02976.
14	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[EB/OL]. [2023-07-10]. https://arxiv.org/abs/2207.02696.
15	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2023-07-10]. https://arxiv.org/abs/1409.1556.
16	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE Press, 2016: 236-248.
17	HAN K, WANG Y H, TIAN Q, et al. GhostNet: more features from cheap operations[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE Press, 2020: 1577-1586.
18	ZHANG X Y, ZHOU X Y, LIN M X, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Press, 2018: 6848-6856.
19	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 6000-6010.
20	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. [2023-07-10]. https://arxiv.org/abs/2010.11929.
21	PAN X R, GE C J, LU R, et al. On the integration of self-attention and convolution[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE Press, 2022: 815-825.
22	JAIN A, MISHRA A, SHUKLA A, et al. A novel genetically optimized convolutional neural network for traffic sign recognition: a new benchmark on Belgium and Chinese traffic sign datasets. Neural Processing Letters, 2019, 50(3): 3019- 3043. doi: 10.1007/s11063-019-09991-x
23	HOUBEN S, STALLKAMP J, SALMEN J, et al. Detection of traffic signs in real-world images: the German traffic sign detection benchmark[C]//Proceedings of 2013 International Joint Conference on Neural Networks. Dallas, USA: IEEE Press, 2013: 761-769.
24	TAN M X, PANG R M, LE Q V. EfficientDet: scalable and efficient object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE Press, 2020: 10781-10790.
25	YANG G Y, LEI J, ZHU Z K. AFPN: asymptotic feature pyramid network for object detection[EB/OL]. [2023-07-10]. https://arxiv.org/abs/2306.15988.
26	JIANG Y Q, TAN Z Y, WANG Y. GiraffeDet: a heavy-neck paradigm for object detection[EB/OL]. [2023-07-10]. https://arxiv.org/abs/2202.04256.
27	SRINIVAS A, LIN T Y, PARMAR N, et al. Bottleneck transformers for visual recognition[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE Press, 2021: 16519-16529.
28	ZHU L, WANG X, KE Z, et al. BiFormer: vision transformer with bi-level routing attention[EB/OL]. [2023-07-10]. https://arxiv.org/abs/2303.08810.
29	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module. Berlin, Germany: Springer, 2018.
30	刘紫燕, 袁磊, 朱明成, 等. 融合SPP和改进FPN的YOLOv3交通标志检测. 计算机工程与应用, 2021, 57(7): 164- 170.
	LIU Z Y, YUAN L, ZHU M C, et al. YOLOv3 traffic sign detection based on SPP and improved FPN. Computer Engineering and Applications, 2021, 57(7): 164- 170.
31	王卜, 何扬. 基于改进YOLOv3的交通标志检测. 四川大学学报(自然科学版), 2022, 59(1): 57- 67.
	WANG B, HE Y. Traffic sign detection based on improved YOLOv3. Journal of Sichuan University(Natural Science Edition), 2022, 59(1): 57- 67.
32	ZHU Z, LIANG D, ZHANG S H, et al. Traffic-sign detection and classification in the wild[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE Press, 2016: 2110-2118.
33	陈梦涛, 余粟. 基于改进YOLOV4模型的交通标志识别研究. 微电子学与计算机, 2022, 39(1): 17- 25.
	CHEN M T, YU S. Research on traffic sign recognitionbased on improved YOLOV4 model. Microelectronics & Computer, 2022, 39(1): 17- 25.
34	熊恩杰, 张荣芬, 刘宇红, 等. 面向交通标志的Ghost-YOLOv8检测算法. 计算机工程与应用, 2023, 59(20): 200- 207.
	XIONG E J, ZHANG R F, LIU Y H, et al. Ghost-YOLOv8 detection algorithm for traffic signs. Computer Engineering and Applications, 2023, 59(20): 200- 207.
35	REN K, HUANG L, FAN C Q, et al. Real-time traffic sign detection network using DS-DetNet and lite fusion FPN. Journal of Real-Time Image Processing, 2021, 18(6): 2181- 2191.
36	LIU F, QIAN Y R, LI H, et al. CAFFNet: channel attention and feature fusion network for multi-target traffic sign detection. International Journal of Pattern Recognition and Artificial Intelligence, 2021, 35(7): 2152008.
37	WEI H Y, ZHANG Q Q, QIAN Y R, et al. MTSDet: multi-scale traffic sign detection with attention and path aggregation. Applied Intelligence, 2023, 53(1): 238- 250.

[1]	ZHANG Lu, TIAN Chunwei, SONG Huansheng, LIU Shigang. Multi-Level Dual-Tree Complex Wavelet Network for Low-Dose CT Image Denoising [J]. Computer Engineering, 2024, 50(9): 266-275.
[2]	WANG Zhihao, QIAN Yuntao. Super-Resolution Reconstruction of Spatiotemporal Fusion for Dual-Stream Remote Sensing Images Based on Swin Transformer [J]. Computer Engineering, 2024, 50(9): 33-45.
[3]	GAO Yubao, WEN Zhicheng. Dual Decoder Image Denoising Method Based on Attention Mechanism [J]. Computer Engineering, 2024, 50(9): 324-332.
[4]	ZENG Yuqi, LIU Bo, ZHONG Baichang, ZHONG Jin. Student Classroom Behavior Detection Algorithm Based on Improved YOLOv8 in Smart Education [J]. Computer Engineering, 2024, 50(9): 344-355.
[5]	Li MIN, Bingjie DONG, Dong AN. Semantic Segmentation Algorithm Based on Multi-Attention Mechanism and Cross-Feature Fusion [J]. Computer Engineering, 2024, 50(8): 282-289.
[6]	Lei WANG, Shipeng DANG, Feng PAN. Model for Predicting Concealed Accessory Pathway Based on Convolutional Neural Network [J]. Computer Engineering, 2024, 50(8): 40-49.
[7]	Xiangquan GUI, Shiqing LIU, Li LI, Qingsong QIN, Tangyan LI. Pedestrian Detection Algorithm for Scenic Spots Based on Improved YOLOv8 [J]. Computer Engineering, 2024, 50(7): 342-351.
[8]	Lili GENG, Baoning NIU. Convolutional Neural Network Pruning Based on Channel Similarity Entropy [J]. Computer Engineering, 2024, 50(7): 133-143.
[9]	Ruiting NUI, Tianfeng YAN, Rui GAO, Yingzhi WANG. Deep Learning TCNN-MobileNet-Based Modulation Recognition Under Low Signal-to-Noise Radio [J]. Computer Engineering, 2024, 50(7): 204-215.
[10]	HU Qing. Multi-Scale Fusion and Dual Output U-Net Network for Person Re-Identification [J]. Computer Engineering, 2024, 50(6): 102-109.
[11]	LI Zhijin, TANG Jiahui, YAN Jinfeng. Lightweight Recognition Method Based on Edge Computing [J]. Computer Engineering, 2024, 50(6): 287-295.
[12]	SUN Bangyong, MA Ming, YU Tao. Multi-scale Camouflaged Object Detection Method Based on Regional Feature Enhancement [J]. Computer Engineering, 2024, 50(5): 209-219.
[13]	ZHOU Qinyuan, DENG Yueping, ZHANG Lei, ZHANG Chen, LU Rirong, HU Xianzhe. Dynamic Visual SLAM System Integrating Optical Flow and Multi-View Geometry [J]. Computer Engineering, 2024, 50(5): 250-259.
[14]	HU Shuai, LI Hualing, HAO Dechen. Improved Multistage Edge-Enhanced Medical Image Segmentation Network of U-Net [J]. Computer Engineering, 2024, 50(4): 286-293.
[15]	Lei ZHANG, Guochen SHEN, Dongxiu OU. Filtering Method for Feature Maps from Convolutional Neural Network for Thermal Imaging Data [J]. Computer Engineering, 2024, 50(4): 31-40.

Please choose a citation manager

Content to export