Localization Method of Rebar Tying Nodes Based on Binocular Vision

doi:10.19678/j.issn.1000-3428.0069933

Abstract

Abstract:

Aiming at the problems at an actual rebar binding construction site, such as multilayered rebar mesh, complex operating environments and light, and dense components, and to realize the accurate positioning of rebar binding nodes, a joint localization method for rebar binding nodes based on binocular stereoscopic matching and binding state recognition is proposed starting from the actual needs of multilayer rebar skeleton plane binding. This method is based on joint target recognition with binocular vision. First, the feature extraction network of AnyNet is improved by introducing a Hourglass feature extraction network and an Efficient Channel Attention Network (ECANet) to improve matching accuracy in the rebar mesh region. As the multilayer rebar mesh has a complex structure and interlayer relationship, the target lashing work layer is obtained by depth filtering. Second, a lashing node localization model based on rebar skeleton extraction is proposed according to the characteristics of the target lashing work. Additionally, the coordinates of the rebar lashing nodes are obtained by extracting the rebar skeleton and fitting the equation of the rebar skeleton. Finally, the state of lashing nodes is identified by the light-weighted YOLOv5 to output the coordinates of the points to be tied. The experimental results show that the three Pixel Error (3PE) of the benchmark network AnyNet is 8.16% and that of the proposed algorithm is only 3.72%, which effectively improves the matching accuracy of the algorithm. The proposed algorithm can filter out the interference of deep-seated rebar, and the average error of the spatial localization of rebar tying nodes is 5.03 mm, which can satisfy the demand of rebar tying in a complex background.

Key words: rebar tying, deep learning, binocular vision, stereo matching, target detection

摘要：

针对实际钢筋绑扎施工现场存在钢筋网多层次, 作业环境和光线复杂, 以及构件密集等问题, 从多层钢筋骨架平面绑扎实际需求出发, 为实现钢筋绑扎节点的精确定位, 以双目视觉联合目标识别思想为基础, 提出一种基于双目立体匹配和绑扎状态识别的钢筋绑扎节点联合定位方法。首先, 通过引入Hourglass特征提取网络和有效通道注意力机制(ECANet)对AnyNet的特征提取网络进行改进, 提高钢筋网区域的匹配精度。多层钢筋网具有复杂的结构和层间关系, 通过深度过滤得到目标绑扎工作层。其次, 根据目标绑扎工作的特征, 提出一种基于钢筋骨架提取的绑扎节点定位模型, 通过提取钢筋骨架并拟合钢筋骨架方程获取钢筋绑扎节点坐标。最后, 通过轻量化的YOLOv5对绑扎节点状态进行识别, 输出待绑扎点坐标。实验结果表明, 基准网络AnyNet的3像素误差(3PE)为8.16%, 而所提算法的3PE仅为3.72%, 有效提高了算法的匹配精度; 所提算法可过滤掉深层次钢筋的干扰, 且钢筋绑扎节点空间定位的平均误差为5.03 mm, 能够满足复杂背景下的钢筋绑扎工作需求。

关键词: 钢筋绑扎, 深度学习, 双目视觉, 立体匹配, 目标检测

CHENG Bin, ZHAO Binbing, LEI Hua, HE Bo. Localization Method of Rebar Tying Nodes Based on Binocular Vision[J]. Computer Engineering, 2026, 52(4): 433-445.

成彬, 赵彬兵, 雷华, 何博. 基于双目视觉的钢筋绑扎节点定位方法[J]. 计算机工程, 2026, 52(4): 433-445.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0069933

https://www.ecice06.com/EN/Y2026/V52/I4/433

Figures/Tables 26

Fig.1 Binocular vision-based framework for accurate positioning of multilayer rebar binding nodes

Fig.2 Joint localization model of rebar-tied nodes based on binocular stereo matching and tie state recognition

Fig.3 Structure of ECANet

Fig.4 Extraction of working layer of target rebar binding

Fig.5 Rebar skeleton extraction

Fig.6 Pixel neighborhood map

Fig.7 Separation of transverse and longitudinal rebars

Fig.8 Straight line fitting of rebar skeleton

Fig.9 Presentation of results of binding node localization

Fig.10 MobileNetV3 schematic

Fig.11 Example of binocular dataset for rebar mesh scenario in this paper

Fig.12 Image acquisition experiment platform

Fig.13 Calibration results

Fig.14 Performance demonstration of stereo matching model in this paper

Fig.15 Comparison of visualized results of stereo matching model proposed in this paper with other methods (single-layer rebar mesh)

Fig.16 Comparison of visualized results of stereo matching model proposed in this paper with other methods (double-layer rebar mesh)

Fig.17 Error analysis of coordinates of rebar-tied nodes obtained by two methods

Fig.18 Partial dataset of rebar node tie status

Fig.19 Effect of detecting state of rebar node tying

Fig.20 Flow chart for accurate positioning of rebar binding nodes

References 26

1	任书文. 钢筋骨架焊接成型工艺优化实验及模拟研究[D]. 沈阳: 沈阳建筑大学, 2021.
	REN S W. Optimization and numerical simulation of welding and forming process of steel frame[D]. Shenyang: Shenyang Jianzhu University, 2021. (in Chinese)
2	CARDNO C A. Robotic rebar-tying system uses artificial intelligence. Civil Engineering Magazine, 2018, 88(1): 38- 39. doi: 10.1061/ciegag.0001260
3	GHAREEB G. Investigation of the potentials and constrains of employing robots in construction in Egypt. The Egyptian International Journal of Engineering Sciences and Technology, 2021, 36(1): 7- 24. doi: 10.21608/eijest.2021.79422.1069
4	王培, 李杨, 崔根, 等. 多旋翼无人机: 新设计、新应用及新发展. 人工智能, 2021(4): 78- 91. doi: 10.16453/j.cnki.ISSN2096-5036.2021.04.008
	WANG P, LI Y, CUI G, et al. Multi-rotor UAVs: new designs, applications and developments. AI-View, 2021(4): 78- 91. doi: 10.16453/j.cnki.ISSN2096-5036.2021.04.008
5	成彬, 邓磊. 融合深度学习与特征投影曲线的钢筋绑扎节点检测方法. 计算机辅助设计与图形学学报, 2024, 36(8): 1272- 1285. doi: 10.3724/SP.J.1089.2024.19964
	CHENG B, DENG L. Rebar binding crosspoints detection method based on deep learning and feature projection curve. Journal of Computer-Aided Design & Computer Graphics, 2024, 36(8): 1272- 1285. doi: 10.3724/SP.J.1089.2024.19964
6	JIN J H, ZHANG W M, LI F X, et al. Robotic binding of rebar based on active perception and planning. Automation in Construction, 2021, 132, 103939. doi: 10.1016/j.autcon.2021.103939
7	黄及远, 李敏, 谢兵兵, 等. 双目视觉关键技术研究综述. 制造业自动化, 2023, 45(5): 166- 171. doi: 10.3969/j.issn.1009-0134.2023.05.035
	HUANG J Y, LI M, XIE B B, et al. Survey of the key technologies of binocular vision. Manufacturing Automation, 2023, 45(5): 166- 171. doi: 10.3969/j.issn.1009-0134.2023.05.035
8	KENDALL A, MARTIROSYAN H, DASGUPTA S, et al. End-to-end learning of geometry and context for deep stereo regression[C]//Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy. Washington D. C., USA: IEEE Press, 2017: 66-75.
9	CHANG J R, CHEN Y S. Pyramid stereo matching network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA. Washington D. C., USA: IEEE Press, 2018: 5410-5418.
10	GUO X Y, YANG K, YANG W K, et al. Group-wise correlation stereo network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA. Washington D. C., USA: IEEE Press, 2019: 3268-3277.
11	CHENG X, ZHONG Y, HARANDI M, et al. Hierarchical neural architecture search for deep stereo matching. Advances in Neural Information Processing Systems, 2020, 33, 22158- 22169. doi: 10.48550/arXiv.2010.13501
12	董振武, 王春媛, 余嘉昕, 等. 基于四梯度多特征和四权重滤波的立体匹配. 计算机工程, 2025, 51(4): 271- 280. doi: 10.19678/j.issn.1000-3428.0068750
	DONG Z W, WANG C Y, YU J X, et al. Stereo matching based on quad-gradient multi-feature and quad-weight filtering. Computer Engineering, 2025, 51(4): 271- 280. doi: 10.19678/j.issn.1000-3428.0068750
13	WANG Y, LAI Z H, HUANG G, et al. Anytime stereo image depth estimation on mobile devices[C]//Proceedings of the International Conference on Robotics and Automation (ICRA). Montreal, QC, Canada. Washington D. C., USA: IEEE Press, 2019: 5893-5900.
14	CHEN S Z, ERGU D J, MA B, et al. Improvement of AnyNet-based end-to-end phased binocular stereo matching network. Procedia Computer Science, 2022, 199, 1450- 1457. doi: 10.1016/j.procs.2022.01.184
15	WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA. Washington D. C., USA: IEEE Press, 2020: 11531-11539.
16	HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Republic of Korea. Washington D. C., USA: IEEE Press, 2019: 1314-1324.
17	李良伟. 面向深度学习的遥感图像目标检测研究[D]. 西安: 西安电子科技大学, 2021.
	LI L W. Research on remote sensing image object detection based on deep learning[D]. Xian: Xidian University, 2021. (in Chinese)
18	YU F, KOLTUN V, FUNKHOUSER T. Dilated residual networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA. Washington D. C., USA: IEEE Press, 2017: 636-644.
19	崔陈, 甘文洋, 朱大奇. 基于改进YOLOv5s的鱼雷检测算法. 兵器装备工程学报, 2024, 45(1): 35-41, 79. doi: 10.11809/bqzbgcxb2024.01.005
	CUI C, GAN W Y, ZHU D Q. Torpedo detection algorithm based on improved YOLOv5s. Journal of Ordnance Equipment Engineering, 2024, 45(1): 35-41, 79. doi: 10.11809/bqzbgcxb2024.01.005
20	王雷雄, 王波, 马富齐, 等. 基于双目立体匹配和场景元素识别的变电人员近电安全距离检测方法研究. 电网技术, 2023, 47(3): 1010- 1021. doi: 10.13335/j.1000-3673.pst.2022.1763
	WANG L X, WANG B, MA F Q, et al. Research on the detection method of the approach distance between substation workers and the live equipment based on binocular stereo matching and scene element recognition. Power System Technology, 2023, 47(3): 1010- 1021. doi: 10.13335/j.1000-3673.pst.2022.1763
21	张铫, 夏雨风, 王梓琦, 等. 基于骨架邻近像素匹配的光条中心提取方法. 东北大学学报(自然科学版), 2023, 44(11): 1529- 1536. doi: 10.12068/j.issn.1005-3026.2023.11.002
	ZHANG Y, XIA Y F, WANG Z Q, et al. Extraction method of light stripe center based on the matching of adjacent pixels of skeleton. Journal of Northeastern University (Natural Science), 2023, 44(11): 1529- 1536. doi: 10.12068/j.issn.1005-3026.2023.11.002
22	GROMPONE VON GIOI R, JAKUBOWICZ J, MOREL J M, et al. LSD: a fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 32(4): 722- 732.
23	武历展, 王夏黎, 张倩, 等. 基于优化YOLOv5s的跌倒人物目标检测方法. 图学学报, 2022, 43(5): 791- 802. doi: 10.11996/JG.j.2095-302X.2022050791
	WU L Z, WANG X L, ZHANG Q, et al. An object detection method of falling person based on optimized YOLOv5s. Journal of Graphics, 2022, 43(5): 791- 802. doi: 10.11996/JG.j.2095-302X.2022050791
24	蔡管鸿, 李国平, 王国中, 等. 基于改进YOLOv5s的轻量化交通灯检测算法. 上海大学学报(自然科学版), 2024, 30(1): 94- 105. doi: 10.12066/j.issn.1007-2861.2411
	CAI G H, LI G P, WANG G Z, et al. Lightweight traffic-light detection algorithm based on improved YOLOv5s. Journal of Shanghai University (Natural Science Edition), 2024, 30(1): 94- 105. doi: 10.12066/j.issn.1007-2861.2411
25	鲁一铭. 基于MobileNetV3-YOLOv5网络的绝缘子缺陷检测[D]. 石家庄: 石家庄铁道大学, 2023.
	LU Y M. Insulator defect detection based on MobileNetV3-YOLOv5 network[D]. Shijiazhuang: Shijiazhuang Tiedao University, 2023. (in Chinese)
26	董国梁, 张雷, 辛山. 基于深度学习的钢筋绑扎机器人目标识别定位. 电子测量技术, 2022, 45(11): 35- 44. doi: 10.19651/j.cnki.emt.2208810
	DONG G L, ZHANG L, XIN S. Target recognition and localization of steel bar binding robot based on deep learning. Electronic Measurement Technology, 2022, 45(11): 35- 44. doi: 10.19651/j.cnki.emt.2208810

[1]	LI Jiao, FAN Haodong, HONG Xudong, XU Zhenyi, FAN Xu, HUANG Jun. Multi-Label Image Classification Based on Label Visual Prototype Learning [J]. Computer Engineering, 2026, 52(4): 229-238.
[2]	TANG Weibo, FANG Qiang, LI Peigen, AI Longjin, XIONG Jinhong, XIA Haiting. RSD-YOLO-Based Small Target Detection in UAV Aerial Images [J]. Computer Engineering, 2026, 52(4): 214-228.
[3]	LI Haoxuan, ZHANG Zhiyuan, LIU Rui, XU Peihua, TIAN Xin. Climate Downscaling of Image Super-Resolution Based on Implicit Neural Representation [J]. Computer Engineering, 2026, 52(4): 376-385.
[4]	WANG Wen, YANG Kuiwu, TONG Songsong, WEI Jianghong, XUE Yan, ZHOU Rongkui. Research on Watermarking Attack of Deep Neural Network Models [J]. Computer Engineering, 2026, 52(4): 22-38.
[5]	CUI Shaoguo, XU Song, WANG Mingyang, ZHOU Yue. Research Progress on Deep Learning Knowledge Tracing for Intelligent Education [J]. Computer Engineering, 2026, 52(4): 39-61.
[6]	YANG Lu, LIU Junjie, YU Xiang. Target Detection Algorithm for Remote Sensing Images with Multi-Scale Information Enhancement [J]. Computer Engineering, 2026, 52(4): 200-213.
[7]	ZHANG Zhi, YIN Yukai, SUN Yiling, MENG Wenjing, PENG Chang. Research on Android Malware Detection Model Based on Multi-modal Feature Fusion [J]. Computer Engineering, 2026, 52(3): 243-254.
[8]	CAO Jiwei, LUO Fei, DING Weichao. BS-YOLO: A Small Object Detection Algorithm Based on BSAM Attention Mechanism and SCConv [J]. Computer Engineering, 2026, 52(3): 119-127.
[9]	TANG Ke, WEI Feiming, LI Dongying, YU Wenxian. Lightweight Target Detection Algorithm for UAV Images Based on Improved YOLOv8 [J]. Computer Engineering, 2026, 52(3): 97-106.
[10]	ZHANG Yonghong, SUN Shulin, GONG Meng, WANG Junfei, MA Guangyi. Remote Sensing Cloud Image Prediction Method Based on Multi-scale Motion Memory Model [J]. Computer Engineering, 2026, 52(3): 128-140.
[11]	LIU Xiaoyu, LIAO Zhifang, TAN Sui, YU Zhiwu. Bridge Dynamic Strain Prediction Based on Stacked GRU Neural Network [J]. Computer Engineering, 2026, 52(3): 441-450.
[12]	QIN Yingxin, ZHANG Kejia, PAN Haiwei, JU Yahao. Adversarial Attacks in Computer Vision: A Survey [J]. Computer Engineering, 2026, 52(2): 46-68.
[13]	ZHANG Xinjia, WANG Fang. UAV Image Small Object Detection Algorithm Based on Multi-layer Feature Fusion and Attention Mechanism [J]. Computer Engineering, 2026, 52(2): 148-157.
[14]	XU Xiaoyang, WEI Wei, GAO Chongyang. Infrared Ship Target Detection Based on Improved YOLOv7-tiny [J]. Computer Engineering, 2026, 52(2): 209-220.
[15]	ZHAO Xudong, WU Hongyue, MENG Ke, XU Xiaolong, DOU Wanchun. Research Progress and Prospects of Service Recommendation Methods (Invited) [J]. Computer Engineering, 2026, 52(1): 61-75.

Please choose a citation manager

Content to export