基于深度学习的道路小目标检测综述

doi:10.19678/j.issn.1000-3428.0065984

摘要/Abstract

摘要：

复杂道路场景下的小目标检测能够提高车辆对于周边环境的感知能力，是计算机视觉和智慧交通领域的重要研究方向。随着深度学习技术的发展，将深度学习方法与道路小目标检测相结合能够有效提高检测精度，使车辆快速对周边环境做出反应。从经典及最新的道路小目标检测的研究成果出发，给出小目标的两种定义方式，分析造成道路小目标检测困难的原因，阐述数据增强、多尺度策略、生成超分辨率细节信息、加强上下文信息联系、改进损失函数等5类基于深度学习的提高道路小目标检测精度的优化方法，总结归纳各类方法的核心思想及目前国内外最新的研究进展。介绍常用于道路小目标检测的大型和公共数据集，提供相应的用于评估小目标检测性能的指标，对比分析各类方法在不同数据集上的性能检测结果，指出道路小目标检测研究目前仍存在的问题，并结合这些问题从多个角度对其未来研究方向进行展望。

关键词: 小目标检测, 深度学习, 数据增强, 特征融合, 检测精度

Abstract:

Small target detection in complex road scenes can improve the vehicle's perception of the surrounding environment. Thus, it is an important research direction in the field of computer vision and intelligent transportation. With the development of deep learning technology, a combination of deep learning and small target detection on roads can effectively improve detection accuracy, allowing the vehicle to quickly respond to the surrounding environment. Starting with the latest classic research results in small target detection, this research provides two definitions for small targets and analyzes the reasons for the difficulty encountered in small target detection on roads. Subsequently, five types of optimization methods based on deep learning are expounded upon to improve detection accuracy of small targets on roads. The optimization methods include enhanced data, multi-scale strategy, generated Super-Resolution(SR) detail information, strengthened contextual information connection and improved loss function. The core ideas of various methods and the latest research progress at home and abroad are summarized. Large and public datasets commonly used in road small target detection are introduced along with corresponding indicators to evaluate the performance of small target detection. In comparing and analyzing the performance detection results of various methods on different datasets, this research presents the current research on road small target and associated problems, looking forward to future research directions from multiple perspectives.

Key words: small target detection, deep learning, data enhancement, feature fusion, detection accuracy

曹健, 陈怡梅, 李海生, 蔡强. 基于深度学习的道路小目标检测综述[J]. 计算机工程, 2023, 49(10): 1-12.

Jian CAO, Yimei CHEN, Haisheng LI, Qiang CAI. Survey of Small Target Detection on Roads Based on Deep Learning[J]. Computer Engineering, 2023, 49(10): 1-12.

http://www.ecice06.com/CN/Y2023/V49/I10/1

图/表 14

图1 MixUp、CutOut、CutMix数据增强方法的示例图

Fig.1 Example diagrams of MixUp, CutOut, and CutMix data enhancement methods

图2 Mosaic数据增强方法的示例图

Fig.2 Example diagrams of Mosaic data enhancement method

图3 图像金字塔结构

Fig.3 Structure of image pyramid

图4 SSD算法的多尺度检测

Fig.4 Multi-scale detection of SSD algorithm

图5 特征金字塔结构

Fig.5 Structure of feature pyramid

图6 PANet结构

Fig.6 Structure of the PANet

图7 基于GAN的细节信息生成流程

Fig.7 Procedure of detail information generation based on GAN

图8 特征级超分辨率图像生成模型结构

Fig.8 Structure of feature-level super-resolution image generation model

图9 关系上下文结构

Fig.9 Structure of relational contexts

参考文献 63

1	金沙沙, 龙伟, 胡灵犀, 等. 多目标检测与跟踪算法在智能交通监控系统中的研究进展. 控制与决策, 2023, 38 (4): 890- 901.
	JIN S S , LONG W , HU L X , et al. Research progress of detection and multi-object tracking algorithm in intelligent traffic monitoring system. Control and Decision, 2023, 38 (4): 890- 901.
2	TIAN D X , LIN C M , ZHOU J S , et al. SA-YOLOv3:an efficient and accurate object detector using self-attention mechanism for autonomous driving. IEEE Transactions on Intelligent Transportation Systems, 2022, 23 (5): 4099- 4110. doi: 10.1109/TITS.2020.3041278
3	张旭欣, 金婕. 基于量化神经网络的端到端车牌检测与识别系统. 传感器与微系统, 2020, 39 (12): 103- 105.
	ZHANG X X , JIN J . End-to-end license plate detection and recognition system based on quantized neural network. Transducer and Microsystem Technologies, 2020, 39 (12): 103- 105.
4	王子晔, 苗夺谦, 赵才荣, 等. 基于多粒度特征的行人跟踪检测结合算法. 计算机研究与发展, 2020, 57 (5): 996- 1002. URL
	WANG Z Y , MIAO D Q , ZHAO C R , et al. A pedestrian tracking algorithm based on multi-granularity feature. Journal of Computer Research and Development, 2020, 57 (5): 996- 1002. URL
5	李松江, 耿兰兰, 王鹏. 基于改进Yolov4的车辆目标检测. 计算机工程, 2023, 49 (4): 272- 280. URL
	LI S J , GENG L L , WANG P . Vehicle target detection based on improved Yolov4. Computer Engineering, 2023, 49 (4): 272- 280. URL
6	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2014: 740-755.
7	KISANTAL M, WOJNA Z, MURAWSKI J, et al. Augmentation for small object detection[C]//Proceedings of the 9th International Conference on Advances in Computing and Information Technology. [S. l. ]: AIRCC Publishing Corporation, 2019: 1-8.
8	WANG S H. An augmentation small object detection method based on NAS-FPN[C]//Proceedings of the 7th International Conference on Information Science and Control Engineering. Washington D.C., USA: IEEE Press, 2021: 213-218.
9	LI W J, TAN X F, WANG Z J. Small object detection of table tennis based on deep learning network[C]//Proceedings of International Conference on Computer Science and Management Technology. Washington D.C., USA: IEEE Press, 2021: 149-152.
10	刘素行, 吴媛, 张军军. 基于YOLOv3的交通场景目标检测方法. 国外电子测量技术, 2021, 40 (2): 116- 120.
	LIU S X , WU Y , ZHANG J J . Traffic scene object detection method based on YOLOv3. Foreign Electronic Measurement Technology, 2021, 40 (2): 116- 120.
11	CHEN Y K, LI Y W, KONG T, et al. Scale-aware automatic augmentation for object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2021: 9558-9567.
12	TANG Y , LI B P , LIU M , et al. AutoPedestrian: an automatic data augmentation and loss function search scheme for pedestrian detection. IEEE Transactions on Image Processing, 2021, 30, 8483- 8496. doi: 10.1109/TIP.2021.3115672
13	ZHANG H Y, CISSE M, DAUPHIN Y N, et al. MixUp: beyond empirical risk minimization[EB/OL]. [2022-09-12]. https://arxiv.org/abs/1710.09412.
14	HAN B, SIM J, ADAM H. BranchOut: regularization for online ensemble tracking with convolutional neural networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2017: 521-530.
15	YUN S, HAN D, CHUN S, et al. CutMix: regularization strategy to train strong classifiers with localizable features[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2020: 6022-6031.
16	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. [2022-09-12]. https://arxiv.org/abs/2004.10934.
17	JOCHER G, STOKEN A, BOROVEC J. Ultralytics/Yolov5: v4.0—nn. SiLU activations weights & biases logging PyTorch hub integration[EB/OL]. [202309-12]. https://zenodo.org/record/4418161.
18	LI M X, GE H Y, WANG H X. IMG-CenterNet: an optimized algorithm based on CenterNet for pedestrian detection[C]//Proceedings of the 6th Information Technology and Mechatronics Engineering Conference. Washington D.C., USA: IEEE Press, 2022: 203-208.
19	DUAN K W, BAI S, XIE L X, et al. CenterNet: keypoint triplets for object detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2020: 6568-6577.
20	姜靓, 詹永照. 基于高斯金字塔与差分法的多目标检测和跟踪算法. 微电子学与计算机, 2011, 28 (11): 129-132, 136.
	JIANG L , ZHAN Y Z . Multiple objects detection and tracking based on background subtraction and Gaussian pyramid. Microelectronics & Computer, 2011, 28 (11): 129-132, 136.
21	SINGH B, DAVIS L S. An analysis of scale invariance in object detection—SNIP[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 3578-3587.
22	FU C Y, LIU W, RANGA A, et al. DSSD: deconvolutional single shot detector[EB/OL]. [2022-09-12]. https://arxiv.org/abs/1701.06659.
23	LI X, LUO X G, HAO H J, et al. Pedestrian detection method based on multi-scale fusion inception-SSD Model[C]//Proceedings of the 9th Joint International Information Technology and Artificial Intelligence Conference. Washington D.C., USA: IEEE Press, 2021: 1549-1553.
24	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2017: 936-944.
25	王程, 刘元盛, 刘圣杰. 基于改进YOLOv4的小目标行人检测算法. 计算机工程, 2023, 49 (2): 296-302, 313. URL
	WANG C , LIU Y S , LIU S J . Small-target pedestrian-detection algorithm based on improved YOLOv4. Computer Engineering, 2023, 49 (2): 296-302, 313. URL
26	王飞, 王林, 张儒良, 等. 基于融合FPN和Faster R-CNN的行人检测算法. 数据采集与处理, 2019, 34 (3): 530- 537.
	WANG F , WANG L , ZHANG R L , et al. Pedestrian detection algorithm based on fusion FPN and faster R-CNN. Journal of Data Acquisition and Processing, 2019, 34 (3): 530- 537.
27	REN S Q , HE K M , GIRSHICK R , et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149.
28	LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 8759-8768.
29	GAN Y, XU W, SU J B. SFPN: semantic feature pyramid network for object detection[C]//Proceedings of the 25th International Conference on Pattern Recognition. Washington D.C., USA: IEEE Press, 2021: 795-802.
30	LIU Z M, GAO G Y, SUN L, et al. IPG-Net: image pyramid guidance network for small object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Washington D.C., USA: IEEE Press, 2020: 4422-4430.
31	ZHAO G M, GE W F, YU Y Z. GraphFPN: graph feature pyramid network for object detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2022: 2743-2752.
32	GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al. Generative adversarial networks. Communications of the ACM, 2020, 63 (11): 139- 144.
33	LI J N, LIANG X D, WEI Y C, et al. Perceptual generative adversarial networks for small object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2017: 1951-1959.
34	BAI Y C , ZHANG Y Q , DING M L , et al. SOD-MTGAN: small object detection via multi-task generative adversarial network. Berlin, Germany: Springer International Publishing, 2018.
35	NOH J, BAE W, LEE W, et al. Better to follow, follow to be better: towards precise supervision of feature super-resolution for small object detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2020: 9724-9733.
36	PANG Y W , CAO J L , WANG J , et al. JCS-Net: joint classification and super-resolution network for small-scale pedestrian detection in surveillance images. IEEE Transactions on Information Forensics and Security, 2019, 14 (12): 3322- 3331.
37	HSU W Y , CHEN P C . Pedestrian detection using stationary wavelet dilated residual super-resolution. IEEE Transactions on Instrumentation and Measurement, 2022, 71, 1- 11.
38	CHEN X L, GUPTA A. Spatial memory for context reasoning in object detection[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2017: 4106-4116.
39	HU H, GU J Y, ZHANG Z, et al. Relation networks for object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 3588-3597.
40	LIU Z X, ZHANG C Y, LUO Y, et al. Improving small-scale pedestrian detection using informed context[C]//Proceedings of IEEE Visual Communications and Image Processing. Washington D.C., USA: IEEE Press, 2020: 1-4.
41	CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2014: 1724-1734.
42	LIANG X , ZHANG J , ZHUO L , et al. Small object detection in unmanned aerial vehicle images using feature fusion and scaling-based single shot detector with spatial context analysis. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30 (6): 1758- 1770.
43	LIM J S, ASTRID M, YOON H J, et al. Small object detection using context and attention[C]//Proceedings of International Conference on Artificial Intelligence in Information and Communication. Washington D.C., USA: IEEE Press, 2021: 181-186.
44	NIE J , PANG Y W , ZHAO S J , et al. Efficient selective context network for accurate object detection. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31 (9): 3456- 3468.
45	郑秋梅, 王璐璐, 王风华. 基于改进卷积神经网络的交通场景小目标检测. 计算机工程, 2020, 46 (6): 26- 33. URL
	ZHENG Q M , WANG L L , WANG F H . Small object detection in traffic scene based on improved convolutional neural network. Computer Engineering, 2020, 46 (6): 26- 33. URL
46	REN K K, ZHAO M H. Road target detection method based on improved tiny-YOLOv3[C]//Proceedings of the 5th International Conference on Electronics Technology. Washington D.C., USA: IEEE Press, 2022: 1102-1106.
47	KIM J U , KIM S T , LEE H J , et al. CUA loss: class uncertainty-aware gradient modulation for robust object detection. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31 (9): 3529- 3543.
48	SHUANG K , LYU Z H , LOO J , et al. Scale-balanced loss for object detection. Pattern Recognition, 2021, 117, 107997.
49	LI Z X, ZHOU F Q. FSSD: feature fusion single shot multibox detector[EB/OL]. [2022-09-12]. https://arxiv.org/abs/1712.00960.
50	ZHANG S F, WEN L Y, BIAN X, et al. Single-shot refinement neural network for object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 4203-4212.
51	窦允冲, 侯进, 曾雷鸣, 等. 基于反馈机制与空洞卷积的道路小目标检测网络. 计算机工程, 2023, 49 (1): 287- 294. URL
	DOU Y C , HOU J , ZENG L M , et al. Road small target detection network based on feedback mechanism and hole convolution. Computer Engineering, 2023, 49 (1): 287- 294. URL
52	郭磊, 王邱龙, 薛伟, 等. 基于改进YOLOv5的小目标检测算法. 电子科技大学学报, 2022, 51 (2): 251- 258.
	GUO L , WANG Q L , XUE W , et al. A small object detection algorithm based on improved YOLOv5. Journal of University of Electronic Science and Technology of China, 2022, 51 (2): 251- 258.
53	REZATOFIGHI H, TSOI N, GWAK J, et al. Generalized intersection over union: a metric and a loss for bounding box regression[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 658-666.
54	YU J H, JIANG Y N, WANG Z Y, et al. UnitBox: an advanced object detection network[C]//Proceedings of the 24th ACM International Conference on Multimedia. New York, USA: ACM Press, 2016: 516-520.
55	ZHENG Z H, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]//Proceedings of AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2020: 12993-13000.
56	GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving? The KITTI vision benchmark suite[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2012: 3354-3361.
57	YU F, CHEN H F, WANG X, et al. BDD100K: a diverse driving dataset for heterogeneous multitask learning[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 2633-2642.
58	HUANG X Y, CHENG X J, GENG Q C, et al. The ApolloScape dataset for autonomous driving[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Washington D.C., USA: IEEE Press, 2018: 1067-10676.
59	CHE Z P, LI G Y, LI T, et al. D²-City: a large-scale Dashcam video dataset of diverse traffic scenarios[EB/OL]. [2022-09-12]. https://arxiv.org/abs/1904.01975.
60	SUN P, KRETZSCHMAR H, DOTIWALLA X, et al. Scalability in perception for autonomous driving: Waymo open dataset[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 2443-2451.
61	PHAM Q H, SEVESTRE P, PAHWA R S, et al. A*3D dataset: towards autonomous driving in challenging environments[C]//Proceedings of IEEE International Conference on Robotics and Automation. Washington D.C., USA: IEEE Press, 2020: 2267-2273.
62	CAESAR H, BANKITI V, LANG A H, et al. nuScenes: a multimodal dataset for autonomous driving[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 11618-11628.
63	SHETTY S. Application of convolutional neural network for image classification on PASCAL VOC challenge 2012 dataset[EB/OL]. [2022-09-12]. https://arxiv.org/ftp/arxiv/papers/1607/1607.03785.pdf.

[1]	范文卓, 吴涛, 许俊平, 李庆庆, 张建林, 李美惠, 魏宇星. 基于多分辨率特征融合的任意尺度图像超分辨率重建[J]. 计算机工程, 2023, 49(9): 217-225.
[2]	李嘉新, 侯进, 盛博莹, 周宇航. 基于改进YOLOv5的遥感小目标检测网络[J]. 计算机工程, 2023, 49(9): 256-264.
[3]	池亚平, 岳梓岩, 林雨衡. 基于Transformer的SM4算法工作模式识别[J]. 计算机工程, 2023, 49(9): 109-117.
[4]	林中霖, 时金桥, 王美琪, 王学宾, 王雨燕. 基于应用行为划分的Android恶意应用检测技术[J]. 计算机工程, 2023, 49(9): 125-136.
[5]	韩璐, 霍纬纲, 张永会, 刘涛. 基于多尺度特征融合与双注意力机制的多元时间序列预测[J]. 计算机工程, 2023, 49(9): 99-108.
[6]	江雨燕, 陶承凤, 李平. 数据增强和自适应自步学习的深度子空间聚类算法[J]. 计算机工程, 2023, 49(8): 96-103, 110.
[7]	宋志娜, 李莎, 杨建明, 徐川. 基于特征与区域定位增强的遥感舰船目标检测[J]. 计算机工程, 2023, 49(8): 257-264.
[8]	张欣怡, 张飞, 郝斌, 高鹭, 任晓颖. 基于改进YOLOv5的口罩佩戴检测算法[J]. 计算机工程, 2023, 49(8): 265-274.
[9]	李泽水, 冀俊忠, 杨翠翠. 基于边权重信息深度网络嵌入的PPIN功能模块检测[J]. 计算机工程, 2023, 49(8): 69-76.
[10]	杨长沛, 廖列法. 基于门控空洞卷积特征融合的中文命名实体识别[J]. 计算机工程, 2023, 49(8): 85-95.
[11]	王可铮, 徐玉芬, 周尚波. 结合对比感知损失和融合注意力的图像去雾模型[J]. 计算机工程, 2023, 49(8): 207-214.
[12]	杨祖赫, 黎智辉, 唐云祁, 晏于文, 宋华青. 结合语义与图像信息的行人属性识别算法[J]. 计算机工程, 2023, 49(8): 215-222, 231.
[13]	刘俊豪, 王美林, 谢兴, 宋烨兴, 许莉花. 基于改进YOLOv5的皮革瑕疵检测算法[J]. 计算机工程, 2023, 49(8): 240-249.
[14]	陈露萌, 曹彦彦, 黄民, 谢鑫钢. 基于改进YOLOv5的火焰检测方法[J]. 计算机工程, 2023, 49(8): 291-301, 309.
[15]	闫兴亚, 匡娅茜, 白光睿, 李月. 基于深度学习的学生课堂行为识别方法[J]. 计算机工程, 2023, 49(7): 251-258.

选择文件类型/文献管理软件名称

选择包含的内容