基于语义对齐和层次优化的非机动车车牌识别定位方法

doi:10.19678/j.issn.1000-3428.0068590

摘要/Abstract

摘要：

对非机动车违规行为依法追究责任是提高城市交通安全的有效手段。由于非机动车车牌具有尺寸小、分布密集、易遮挡等特点, 导致应用传统的深度学习方法会出现特征信息大量丢失的现象。为此, 提出一种基于语义对齐和层次优化的非机动车车牌识别定位方法。首先设计底层信息融合的语义对齐模块, 在上采样过程中利用底层目标信息引导高层语义向下融合, 以解决高底层语义冲突带来的小目标特征丢失问题; 然后构建CSP结构的层次优化模块替代深层ELAN模块, 使用堆叠少量卷积核模块提取目标信息以减少网络层数, 避免特征信息在深层丢失; 最后, 为减少训练过程中的匹配误差, 使用K-Means++算法聚类得到适合非机动车车牌的初始锚框, 提高小目标识别定位准确率。实验结果表明, 所提方法在自制非机动车车牌数据集上的识别定位准确率为90.95%, 与YOLOv7、YOLOv8等代表性方法相比至少提升3.58%, 为非机动车车牌识别定位提供了一种有效的方法。

关键词: 小目标检测, 非机动车车牌, 语义对齐, 层次优化, K-Means++算法

Abstract:

Holding non-motorized vehicles accountable for legal violations effectively enhances urban traffic safety. Non-motorized vehicle license plates are characterized by small size, dense distribution, and ease of being obscured, which leads to significant feature information loss during the detection process in traditional deep learning-based methods. A non-motorized vehicle license plate recognition and localization method based on semantic alignment and hierarchical optimization is proposed. In this method, a semantic alignment module is designed for the underlying information fusion. During the upsampling process, low-level target information is used to guide the fusion of high-level semantics downwards, addressing the loss of small target features caused by conflicts between high- and low-level semantics. Subsequently, a hierarchical optimization module is constructed within the CSP structure to replace the deep ELAN module. This module uses a stack of a few convolutional kernel modules to extract the target information, reducing the number of network layers and preventing the loss of feature information at deeper levels. In the final stage, the K-Means++ algorithm is employed to cluster and obtain the initial anchor boxes suitable for non-motorized license plates to reduce the matching error during the training process. This approach aims to improve the accuracy of small-object recognition and localization. The experimental results demonstrate that the proposed method achieves a recognition and localization accuracy of 90.95% on a non-motorized vehicle license plate dataset. Compared with representative methods such as YOLOv7 and YOLOv8, it improves the accuracy by at least 3.58%. The proposed approach is effective for non-motorized vehicle license plate recognition and localization.

Key words: small object detection, non-motorized license plate, semantic alignment, hierarchical optimization, K-Means++ algorithm

谭若琦, 董明刚, 赵唯肖, 武天昊. 基于语义对齐和层次优化的非机动车车牌识别定位方法[J]. 计算机工程, 2024, 50(11): 142-151.

TAN Ruoqi, DONG Minggang, ZHAO Weixiao, WU Tianhao. Non-Motorized License Plate Recognition and Localization Method Based on Semantic Alignment and Hierarchical Optimization[J]. Computer Engineering, 2024, 50(11): 142-151.

https://www.ecice06.com/CN/Y2024/V50/I11/142

图/表 16

图1 PlateNet架构

Fig.1 PlateNet architecture

图2 SAM模块

Fig.2 SAM module

图3 HOM模块

Fig.3 HOM module

图4 非机动车车牌数据集目标分布情况

Fig.4 Target distribution in non-motorized license plate dataset

图5 不同目标检测方法的识别定位能力对比

Fig.5 Comparison of recognition and localization abilities of different object detection methods

图6 不同方法的检测能力对比

Fig.6 Comparison of detection capabilities of different methods

图7 可视化效果对比

Fig.7 Comparison of visualization effects

参考文献 37

1	仲颖. 消除隐患: 如何加强非机动车管理. 检察风云, 2023,(1): 36- 37.
	ZHONG Y. Eliminating hidden dangers: how to strengthen the management of non-motor vehicles. Prosecutorial View, 2023,(1): 36- 37.
2	AL-SHEMARRY M S, LI Y, ABDULLA S. An efficient texture descriptor for the detection of license plates from vehicle images in difficult conditions. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(2): 553- 564. doi: 10.1109/TITS.2019.2897990
3	凌翔, 黄榜, 黄良俊, 等. 基于改进二维离散小波变换的多车牌定位. 重庆交通大学学报(自然科学版), 2020, 39(2): 16- 21.
	LING X, HUANG B, HUANG L J, et al. Multi-license plate location based on improved two-dimensional discrete wavelet transform. Journal of Chongqing Jiaotong University (Natural Science), 2020, 39(2): 16- 21.
4	谭鑫平, 高志辉, 韩航迪, 等. 基于改进YOLOv5的荧光图像细胞智能检测研究. 半导体光电, 2023, 44(5): 709- 716.
	TAN X P, GAO Z H, HAN H D, et al. Intelligent detection of cells in fluorescence images based on improved YOLOv5. Semiconductor Optoelectronics, 2023, 44(5): 709- 716.
5	WU D L, JIANG S, ZHAO E L, et al. Detection of camellia oleifera fruit in complex scenes by using YOLOv7 and data augmentation. Applied Sciences, 2022, 12(22): 11318. doi: 10.3390/app122211318
6	李松江, 耿兰兰, 王鹏. 基于改进Yolov4的车辆目标检测. 计算机工程, 2023, 49(4): 272- 280. doi: 10.19678/j.issn.1000-3428.0062943
	LI S J, GENG L L, WANG P. Vehicle target detection based on improved Yolov4. Computer Engineering, 2023, 49(4): 272- 280. doi: 10.19678/j.issn.1000-3428.0062943
7	李嘉豪, 闵卫东, 陈炯缙, 等. 一种复杂场景下高精度交通标志检测模型. 计算机工程, 2023, 49(11): 311- 320. doi: 10.19678/j.issn.1000-3428.0066372
	LI J H, MIN W D, CHEN J J, et al. A high precision traffic sign detection model in complex scenes. Computer Engineering, 2023, 49(11): 311- 320. doi: 10.19678/j.issn.1000-3428.0066372
8	SHI H L, ZHAO D N. License plate recognition system based on improved YOLOv5 and GRU. IEEE Access, 2023, 11, 10429- 10439.
9	LI S Y, MA N, WU Z X, et al. License plate detection and recognition based on light-Yolov7[EB/OL]. [2023-09-05]. https://link.springer.com/chapter/10.1007/978-981-99-6187-0_8.
10	XIA T, ZHANG R Z, ZHANG Y J, et al. Application of YOLOv7 and Transformer structures to small object (license plate) detection in complex traffic scenes[C]//Proceedings of the 4th International Conference on Machine Learning, Big Data and Business Intelligence. Washington D. C., USA: IEEE Press, 2022: 128-131.
11	AHMED S U, MAISHA F B F, HOSSAM-E-HAIDER M. Bangla license plate detection and recognition system with YOLOv7 and improved custom OCR engine[C]//Proceedings of the 4th International Conference on Emerging Research in Electronics, Computer Science and Technology. Washington D. C., USA: IEEE Press, 2022: 1-7.
12	庄建军, 叶振兴. 基于改进YOLOv5m的电动车骑行者头盔与车牌检测方法. 南京信息工程大学学报, 2024, 16(1): 1- 10.
	ZHUANG J J, YE Z X. Helmet and license plate detection for electric bike rider based on improved YOLOv5m. Journal of Nanjing University of Information Science & Technology (Natural Science Edition), 2024, 16(1): 1- 10.
13	WEI C, TAN Z, QING Q, et al. Fast helmet and license plate detection based on lightweight YOLOv5. Sensors (Basel, Switzerland), 2023, 23(9): 4335.
14	MAHMOOD Z, KHAN K, KHAN U, et al. Towards automatic license plate detection. Sensors (Basel, Switzerland), 2022, 22(3): 1245.
15	SLIMANI I, ZAARANE A, AL OKAISHI W, et al. An automated license plate detection and recognition system based on wavelet decomposition and CNN. Array, 2020, 8, 100040.
16	GONG Y X, DENG L J, TAO S, et al. Unified Chinese license plate detection and recognition with high efficiency. Journal of Visual Communication and Image Representation, 2022, 86, 103541.
17	SILVA S M, JUNG C R. Real-time license plate detection and recognition using deep convolutional neural networks. Journal of Visual Communication and Image Representation, 2020, 71, 102773.
18	HU X, LI H, LI X R, et al. MobileNet-SSD MicroScope using adaptive error correction algorithm: real-time detection of license plates on mobile devices. IET Intelligent Transport Systems, 2020, 14(2): 110- 118.
19	CHEN S L, TIAN S, MA J W, et al. End-to-end trainable network for degraded license plate detection via vehicle-plate relation mining. Neurocomputing, 2021, 446, 1- 10.
20	LI Z J, CHEN S L, LIU Q, et al. Anchor-free location refinement network for small license plate detection[EB/OL]. [2023-09-05]. https://link.springer.com/content/pdf/10.1007/978-3-031-18916-6_41.pdf?pdf=inline%20link.
21	FENG J H, WANG X L, LV H. Non-motor vehicle illegal behavior discrimination and license plate detection based on real-time video. Journal of Physics: Conference Series, 2020, 1544(1): 012105.
22	梁誉耀. 基于深度学习的电瓶车头盔检测及车牌识别算法研究[D]. 长沙: 湖南大学, 2022.
	LIANG Y Y. Research on helmet detection and license plate recognition algorithm for electric bicycle based on deep learning[D]. Changsha: Hunan University, 2022. (in Chinese)
23	吴静. 基于深度学习的电动车车牌识别研究[D]. 柳州: 广西科技大学, 2022.
	WU J. Research on electric vehicle license plate recognition based on deep learning[D]. Liuzhou: Guangxi University of Science and Technology, 2022. (in Chinese)
24	LI X, ZHANG J, YANG Y, et al. SFNet: faster, accurate, and domain agnostic semantic segmentation via semantic flow[EB/OL]. [2023-09-05]. https://arxiv.org/pdf/2207.04415v1.
25	JADERBERG M, SIMONYAN K, ZISSERMAN A. Spatial transformer networks[EB/OL]. [2023-09-05]. https://arxiv.org/abs/1506.02025.
26	HUANG L, LI W, SHEN L, et al. YOLOCS: object detection based on dense channel compression for feature spatial solidification[EB/OL]. [2023-09-05]. https://arxiv.org/abs/2305.04170.
27	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[EB/OL]. [2023-09-05]. https://arxiv.org/abs/1405.0312.
28	EVERINGHAM M, ALI ESLAMI S M, VAN GOOL L, et al. The pascal visual object classes challenge: a retrospective. International Journal of Computer Vision, 2015, 111(1): 98- 136.
29	ZHANG S S, BENENSON R, SCHIELE B. CityPersons: a diverse dataset for pedestrian detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 3213-3221.
30	CORDTS M, OMRAN M, RAMOS S, et al. The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 3213-3223.
31	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[EB/OL]. [2023-09-05]. https://arxiv.org/abs/1512.02325.
32	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 2980-2988.
33	DUAN K W, BAI S, XIE L X, et al. CenterNet: keypoint triplets for object detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 6569-6578.
34	TIAN Z, SHEN C H, CHEN H, et al. FCOS: fully convolutional one-stage object detection[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 9627-9636.
35	REIS D, KUPEC J, HONG J, et al. Real-time flying object detection with YOLOv8[EB/OL]. [2023-09-05]. https://arxiv.org/abs/2305.09972.
36	TANG S Y, ZHANG S, FANG Y N. HIC-YOLOv5: improved YOLOv5 for small object detection[EB/OL]. [2023-09-05]. https://arxiv.org/abs/2309.16393.
37	YUAN X, CHENG G, YAN K B, et al. Small object detection via coarse-to-fine proposal generation and imitation learning[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2023: 6317-6327.

[1]	张天鹏, 韩晶, 吕学强. 基于多任务学习的超分辨率辅助小目标检测[J]. 计算机工程, 2024, 50(9): 304-312.
[2]	马明旭, 马宏, 宋华伟. 基于YOLO-Pose的城市街景小目标行人姿态估计算法[J]. 计算机工程, 2024, 50(4): 177-186.
[3]	蒋心璐, 陈天恩, 王聪, 赵春江. 大田环境下的农业害虫图像小目标检测算法[J]. 计算机工程, 2024, 50(1): 232-241.
[4]	圣文顺, 余熊峰, 林佳燕, 陈欣. 融合注意力与特征金字塔的小尺度目标检测算法[J]. 计算机工程, 2024, 50(1): 242-250.
[5]	李嘉新, 侯进, 盛博莹, 周宇航. 基于改进YOLOv5的遥感小目标检测网络[J]. 计算机工程, 2023, 49(9): 256-264.
[6]	吴珊, 周凤. 基于改进SSD算法的小目标检测[J]. 计算机工程, 2023, 49(7): 179-188.
[7]	谌雨章, 黄逸姿, 张钧涵. 基于多速率空洞卷积的多尺度水下小目标检测[J]. 计算机工程, 2023, 49(6): 257-264.
[8]	胡清翔, 饶文碧, 熊盛武. 面向无人机遥感场景的轻量级小目标检测算法[J]. 计算机工程, 2023, 49(12): 169-177.
[9]	曹健, 陈怡梅, 李海生, 蔡强. 基于深度学习的道路小目标检测综述[J]. 计算机工程, 2023, 49(10): 1-12.
[10]	窦允冲, 侯进, 曾雷鸣, 陈子锐. 基于反馈机制与空洞卷积的道路小目标检测网络[J]. 计算机工程, 2023, 49(1): 287-294.
[11]	戚玲珑, 高建瓴. 基于改进YOLOv7的小目标检测[J]. 计算机工程, 2023, 49(1): 41-48.
[12]	邹慧海, 侯进. 改进SSD算法的道路小目标检测研究[J]. 计算机工程, 2022, 48(5): 281-288.
[13]	奚琦, 张正道, 彭力. 基于改进密集网络与二次回归的小目标检测算法[J]. 计算机工程, 2021, 47(4): 241-247,255.
[14]	黄凤琪, 陈明, 冯国富. 基于可变形卷积的改进YOLO目标检测算法[J]. 计算机工程, 2021, 47(10): 269-275,282.
[15]	包壮壮, 赵学军, 王明芳, 董玉浩, 庞梦洋, 黄林, 贺刚. 脱离预训练的多尺度目标检测网络模型[J]. 计算机工程, 2020, 46(6): 248-255.

选择文件类型/文献管理软件名称

选择包含的内容