Road Crack Detection Based on Position Information and Attention Mechanism

doi:10.19678/j.issn.1000-3428.0067758

Abstract

Abstract:

Road cracks are the main cause of highway safety problems. Traditional crack detection is typically based on manual detection, which faces problems such as low efficiency and insecurity. In addition, the existing deep learning detection model causes incomplete crack detection when facing interference factors, such as shadow occlusion and complex backgrounds. To address these problems, a road crack detection model based on location information and an attention mechanism, known as PA-TransUNet, is proposed. First, the hybrid encoder receives the input image, extracts the crack feature information, and introduces the position information of the query, key, and value to improve the ability of the self-attention mechanism in the encoder Transformer to capture the crack shape and compensate for the loss of feature information. Subsequently, the crack features are input into the decoder for upsampling, and an Attention Gating-based Decoding Module(AGDM) is designed to strengthen the learning of crack regions by suppressing non-crack regions and improving the accuracy and integrity of crack detection. The experimental results demonstrate that the F1 values of the PA-TransUNet model on the CrackForest Dataset(CFD) and Cracktree200 public datasets reach 87.44% and 82.58%, respectively. In addition, to further test the crack detection ability of the PA-TransUNet model in practical engineering, an F1 value of 88.68% is achieved on the self-made Unmanned Aerial Vehicle Cracks(UAV Cracks) dataset, which shows that it can better meet the needs of crack detection in practical engineering.

Key words: image processing, road crack detection, semantic segmentation, position information, attention mechanism

摘要：

路面裂缝是造成公路安全问题的主要因素。传统的裂缝检测通常以人工检测为主, 存在效率低、不安全等问题, 此外现有深度学习检测模型在面临阴影遮挡、背景复杂等干扰因素时会造成裂缝检测不完整。针对上述问题, 提出一种基于位置信息和注意力机制的路面裂缝检测模型(PA-TransUNet)。首先, 通过混合编码器接收输入图像, 提取裂缝特征信息, 引入查询项、键、值的位置信息, 提升编码器Transformer中自注意力机制捕获裂缝形状和补偿特征信息丢失的能力。然后, 输入裂缝特征到解码器进行上采样, 设计一种基于注意力门控的解码模块(AGDM), AGDM通过抑制非裂缝区域来加强对裂缝区域的学习, 提高裂缝检测的准确性和完整性。实验结果表明, PA-TransUNet模型在路面裂缝检测数据集(CFD)和Cracktree200这2个公开数据集上的F1值分别达到87.44%和82.58%。此外, 为了进一步检验PA-TransUNet模型在实际工程中的裂缝检测能力, 又在自制无人机裂缝(UAV Cracks)数据集上取得了88.68%的F1值, 由此可见其能较好地满足实际工程中的裂缝检测需求。

关键词: 图像处理, 路面裂缝检测, 语义分割, 位置信息, 注意力机制

Anzheng WANG, Jianwu DANG, Biao YUE, Jingyu YANG. Road Crack Detection Based on Position Information and Attention Mechanism[J]. Computer Engineering, 2024, 50(4): 303-312.

王安政, 党建武, 岳彪, 杨景玉. 基于位置信息和注意力机制的路面裂缝检测[J]. 计算机工程, 2024, 50(4): 303-312.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0067758

http://www.ecice06.com/EN/Y2024/V50/I4/303

Figures/Tables 13

Fig.1 PA-TransUNet model structure

Fig.2 Self-attention mechanism

Fig.3 Self-attention mechanism with location information

Fig.4 AGDM structure

Fig.5 AG structure

Fig.6 Visual effect comparison on CFD dataset

Fig.7 Visual effect comparison on Cracktree200 dataset

Fig.8 Comparison of PA-TransUNet detection results

Fig.9 Visual effect comparison on UAV Cracks dataset

References 27

1	XIANG X Z, ZHANG Y Q, EL SADDIK A. Pavement crack detection network based on pyramid structure and attention mechanism. IET Image Processing, 2020, 14(8): 1580- 1586. doi: 10.1049/iet-ipr.2019.0973
2	KANG D H, CHA Y J. Efficient attention-based deep encoder and decoder for automatic crack segmentation. Structural Health Monitoring, 2022, 21(5): 2190- 2205. doi: 10.1177/14759217211053776
3	YAN K, ZHANG Z H. Automated asphalt highway pavement crack detection based on deformable single shot multi-box detector under a complex environment. IEEE Access, 2021, 9, 150925- 150938. doi: 10.1109/ACCESS.2021.3125703
4	IBRAGIMOV E, LEE H J, LEE J J, et al. Automated pavement distress detection using region based convolutional neural networks. International Journal of Pavement Engineering, 2022, 23(6): 1981- 1992. doi: 10.1080/10298436.2020.1833204
5	TRAN V P, TRAN T S, LEE H J, et al. One stage detector(RetinaNet)-based crack detection for asphalt pavements considering pavement distresses and surface objects. Journal of Civil Structural Health Monitoring, 2021, 11(1): 205- 222. doi: 10.1007/s13349-020-00447-8
6	LUO H, LI J M, CAI L M, et al. STrans-YOLOX: fusing Swin Transformer and YOLOX for automatic pavement crack detection. Applied Sciences, 2023, 13(3): 1999. doi: 10.3390/app13031999
7	OLIVEIRA H, CORREIA P L. Automatic road crack segmentation using entropy and image dynamic thresholding[C]//Proceedings of the 17th European Signal Processing Conference. Washington D. C., USA: IEEE Press, 2009: 622-626.
8	任亮, 徐志刚, 赵祥模, 等. 基于Prim最小生成树的路面裂缝连接算法. 计算机工程, 2015, 41(1): 31-36, 43. URL
	REN L, XU Z G, ZHAO X M, et al. Pavement crack connection algorithm based on Prim minimum spanning tree. Computer Engineering, 2015, 41(1): 31-36, 43. URL
9	SALMAN M, MATHAVAN S, KAMAL K, et al. Pavement crack detection using the Gabor filter[C]//Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems. Washington D. C., USA: IEEE Press, 2013: 2039-2044.
10	MEDINA R, LLAMAS J, ZALAMA E, et al. Enhanced automatic detection of road surface cracks by combining 2D/3D image processing techniques[C]//Proceedings of 2014 IEEE International Conference on Image Processing. Washington D. C., USA: IEEE Press, 2014: 778-782.
11	ZOU Q, ZHANG Z, LI Q, et al. DeepCrack: learning hierarchical convolutional features for crack detection. IEEE Transactions on Image Processing, 2018, 28(3): 1498- 1512.
12	FAN Z, WU Y, LU J, et al. Automatic pavement crack detection based on structured prediction with the convolutional neural network[J]. [2023-05-11]. https://arxiv.org/abs/1802.02208.
13	LAU S L H, CHONG E K P, YANG X, et al. Automated pavement crack segmentation using U-Net-based convolutional neural network. IEEE Access, 2020, 8, 114892- 114899. doi: 10.1109/ACCESS.2020.3003638
14	GHOSH S, SINGH S, MAITY A, et al. CrackWeb: a modified U-Net based segmentation architecture for crack detection[C]//Proceedings of the 3rd International Conference on Advances in Mechanical Engineering and Its Interdisciplinary Areas. [S. l. ]: IOP Publishing, 2021: 012002.
15	于海洋, 景鹏, 张文涛, 等. 基于残差与注意力机制的道路裂缝检测U-Net改进模型. 计算机工程, 2023, 49(6): 265- 273. URL
	YU H Y, JING P, ZHANG W T, et al. Improved U-Net model for road crack detection based on residual and attention mechanism. Computer Engineering, 2023, 49(6): 265- 273. URL
16	张伯树, 张志华, 张洋. 改进的HRNet应用于路面裂缝分割与检测. 测绘通报, 2022,(3): 83- 89. URL
	ZHANG B S, ZHANG Z H, ZHANG Y. Improved HRNet applied to segmentation and detection of pavement cracks. Bulletin of Surveying and Mapping, 2022,(3): 83- 89. URL
17	CHEN J N, LU Y Y, YU Q H, et al. TransUNet: Transformers make strong encoders for medical image segmentation[EB/OL]. [2023-05-11]. https://arxiv.org/abs/2102.04306
18	YANG Y M, MEHRKANOON S. AA-TransUNet: attention augmented TransUNet for nowcasting tasks[C]//Proceedings of International Joint Conference on Neural Networks. Washington D. C., USA: IEEE Press, 2022: 1-8.
19	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2017: 6000-6010.
20	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: transformers for image recognition at scale[EB/OL]. [2023-05-11]. https://arxiv.org/abs/2010.11929.
21	OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: learning where to look for the pancreas[EB/OL]. [2023-05-11]. https://arxiv.org/abs/1804.03999.
22	SHI Y, CUI L M, QI Z Q, et al. Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12): 3434- 3445. doi: 10.1109/TITS.2016.2552248
23	ZOU Q, CAO Y, LI Q Q, et al. Cracktree: automatic crack detection from pavement images. Pattern Recognition Letters, 2012, 33(3): 227- 238. doi: 10.1016/j.patrec.2011.11.004
24	CAO H B, GAO Y X, CAI W W, et al. Segmentation detection method for complex road cracks collected by UAV based on HC-UNet++. Drones, 2023, 7(3): 189.
25	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481- 2495. doi: 10.1109/TPAMI.2016.2644615
26	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer. Berlin, Germany: Springer, 2015: 234-241.
27	LIU Y H, YAO J, LU X H, et al. DeepCrack: a deep hierarchical feature learning architecture for crack segmentation. Neurocomputing, 2019, 338, 139- 153.

[1]	Minghu WANG, Zhikui SHI, Jia SU, Xinsheng ZHANG. Sequence Recommendation Method Based on RoBERTa and Graph-Enhanced Transformer [J]. Computer Engineering, 2024, 50(4): 121-131.
[2]	LI Jingcan, XIAO Cuilin, QIN Xiaoting, XIE Xia. Text-Relation-Extraction Algorithm Based on Large-Language Model and Semantic Enhancement [J]. Computer Engineering, 2024, 50(4): 87-94.
[3]	Haipeng WU, Yurong QIAN, Hongyong LENG. Multimodal Relation Extraction Based on Bidirectional Attention Mechanism [J]. Computer Engineering, 2024, 50(4): 160-167.
[4]	Yu AN, Haibo GE, Wenhao HE, Sai MA, Mengyang CHENG. Siamese Network Tracking Algorithm Based on Compensated Attention Mechanism [J]. Computer Engineering, 2024, 50(4): 187-196.
[5]	Yudan YANG, Junhua ZHANG, Yunfeng LIU. Segmentation of Spine Computed Tomography Images Based on Three-Dimensional Recurrent Residual Convolution [J]. Computer Engineering, 2024, 50(4): 237-246.
[6]	ZHANG Chi, WANG Zhong, JIANG Tianhao, XIE Kangmin. Speech Enhancement Network Based on Parallel Multi-Attention [J]. Computer Engineering, 2024, 50(4): 68-77.
[7]	Jida ZHAO, Guoyong ZHEN, Chengqun CHU. Unmanned Aerial Vehicle Image Target Detection Algorithm Based on YOLOv8 [J]. Computer Engineering, 2024, 50(4): 113-120.
[8]	Mingxu MA, Hong MA, Huawei SONG. Pose Estimation Algorithm for Small Target Pedestrians in Urban Street View Based on YOLO-Pose [J]. Computer Engineering, 2024, 50(4): 177-186.
[9]	Jiayuan ZHAO, Yuru ZHANG, Xiaodong SU, Hongyan XU, Shizhou LI, Yurong ZHANG. Implicit Modeling Network of Human Keypoints Based on Attention Mechanism [J]. Computer Engineering, 2024, 50(3): 317-325.
[10]	Bohan WANG, Xiaoyan JIANG, Liuyi FAN. Semantic Segmentation Improvement Method Based on Deep Supervision for the Construction of Latent Space [J]. Computer Engineering, 2024, 50(3): 191-199.
[11]	Xinlin XIE, Dongxu YIN, Taoyuan ZHANG, Gang XIE. Multiscale Fusion Crowd Counting Algorithm Based on Attention Mechanism [J]. Computer Engineering, 2024, 50(3): 290-297.
[12]	Wentao YUAN, Wentao WEI, Demin GAO. Research on Multiview Convolutional Gesture Recognition with Fusion Attention Mechanism [J]. Computer Engineering, 2024, 50(3): 208-215.
[13]	Fangxin XU, Rong FAN, Xiaolu MA. Improved YOLOv7 Algorithm for Crowded Pedestrian Detection [J]. Computer Engineering, 2024, 50(3): 250-258.
[14]	Bochao ZHAO, Jiajun MA, Lei CUI, Wenpeng LUAN, Jing ZHU. Anomaly Detection for Photovoltaic Based on Improved VMD-XGBoost-BiLSTM Combination Model [J]. Computer Engineering, 2024, 50(3): 306-316.
[15]	Haochen XU, Manhua LIU. Facial Landmark Detection Based on Hierarchical Self-Attention Network [J]. Computer Engineering, 2024, 50(2): 239-246.

Please choose a citation manager

Content to export