Stereo Matching Based on Quad Gradient Multi-Feature and Quad-Weight Filtering

doi:10.19678/j.issn.1000-3428.0068750

Abstract

Abstract:

The accuracy of stereo matching directly determines the precision of subsequent 3D scene information recovery, and enhancing the accuracy of disparity maps has attracted considerable attention among researchers. Traditional stereo matching methods inadequately represent the local structure of images, particularly within regions of similar structures, at the junctions of the foreground and background, and in areas containing erroneous cost points. To address these issues, this paper proposes a stereo matching method based on a quad-gradient, multi-feature cost, and quad-weight filtering. This method constructs a multi-feature space composed of image intensity and gradients in four directions. It employs quadratic encoding to calculate the cost of the multi-feature census transform of images and then combines it with the multi-feature Absolute Difference (AD) cost to enhance the accuracy of the local structural representation. A filter kernel constructed with four weights, namely, spatial proximity, pixel intensity similarity, regional similarity, and cost similarity, is used for cost aggregation to mitigate the aggregation weight of abnormal costs. The initial disparity is calculated using the Winner-Take-All (WTA) algorithm and is preliminarily corrected through a left-right consistency check, followed by disparity optimization using an adaptive window and a disparity threshold. Results of experiments on the Middlebury V3 stereo platform indicate that the algorithm significantly outperforms existing traditional stereo matching algorithms. It yields a weighted average bad4.0 value (percentage of ″bad″ pixels having an error greater than 4.0 pixels) of 14.7% in non-occluded regions and 20.6% in all regions.

Key words: stereo matching, Absolute Difference (AD)-census transformation, quad-gradient multi-feature, bilateral filtering, Winner-Take-All (WTA) algorithm, cost aggregation

摘要：

立体匹配的精度直接决定了后续三维场景信息恢复的准确度, 提高视差图的精度一直是研究的热点问题。传统的立体匹配方法对图像的局部结构表达不够精准, 对相似结构区域内的点或前景和背景交界处及含有错误代价点的处理效果不理想。针对以上问题, 提出一种基于四梯度多特征代价和四权重滤波的立体匹配方法。由图像灰度和四方向梯度组成多特征空间, 通过二次编码计算多特征Census变换代价, 再结合多特征绝对误差(AD)代价形成多特征代价, 提升局部结构的表达准确度; 以空间邻近度、像素值相似度、区域相似度和代价相似度四权重构建的滤波核进行代价聚合, 降低异常代价的聚合权重; 以胜者为王(WTA)算法计算初始视差, 以左右一致性检测进行视差初步修正, 结合自适应窗口和视差阈值进行视差优化。在Middlebury V3立体平台上的实验结果表明, 该方法在非遮挡区域和全部区域加权平均的bad4.0分别为14.7%和20.6%, 效能显著优于现有的传统立体匹配算法。

关键词: 立体匹配, 绝对误差(AD)-Census变换, 四梯度多特征, 双边滤波, 胜者为王(WTA)算法, 代价聚合

DONG Zhenwu, WANG Chunyuan, YU Jiaxin, ZHOU Bin, WU Yang. Stereo Matching Based on Quad Gradient Multi-Feature and Quad-Weight Filtering[J]. Computer Engineering, 2025, 51(4): 271-280.

董振武, 王春媛, 余嘉昕, 周斌, 吴扬. 基于四梯度多特征和四权重滤波的立体匹配[J]. 计算机工程, 2025, 51(4): 271-280.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068750

https://www.ecice06.com/EN/Y2025/V51/I4/271

Figures/Tables 17

Fig.1 The realization procedure of proposed method

Fig.2 Conversion of image information to multiple feature spaces

Fig.3 First encoding matrix of MCensus transformation

Fig.4 Sencond encoding matrix of MCensus transformation

Fig.5 Adaptive window and region similarity weights

Fig.6 Submatrix selection

Fig.7 Comparison of different matching cost calculation methods

Fig.8 Comparison of different cost aggregation algorithms

Fig.9 Comparison of non occluded areas bad4.0

Fig.10 Comparison of bad4.0 across all regions

Fig.11 Comparison of final parallax images of different algorithms

Fig.12 Application of this algorithm in remote sensing images

References 26

1	王厚博. 面向自动驾驶的立体匹配算法[D]. 哈尔滨: 东北林业大学, 2022.
	WANG H B. Stereo matching algorithm for autonomous driving[D]. Harbin: Northeast Forestry University, 2022. (in Chinese)
2	刘明国. 面向机器人导航的立体视觉及目标检测技术研究[D]. 南京: 南京理工大学, 2017.
	LIU M G. Research on stereo vision and target detection technology for robot navigation[D]. Nanjing: Nanjing University of Science and Technology, 2017. (in Chinese)
3	周施. 三维重建中立体匹配算法研究[D]. 沈阳: 东北大学, 2020.
	ZHOU S. Research on stereo matching algorithm in 3D reconstruction[D]. Shenyang: Northeastern University, 2020. (in Chinese)
4	韩昊, 李参海, 丘晓枫. 基于神经网络密集匹配的资源三号DSM提取. 遥感信息, 2022, 37 (3): 101- 108.
	HAN H , LI C H , QIU X F . ZY-3 DSM extraction based on neural network dense matching. Remote Sensing Information, 2022, 37 (3): 101- 108.
5	陈华, 王立军, 刘刚. 立体匹配算法研究综述. 高技术通讯, 2020, 30 (2): 157- 165.
	CHEN H , WANG L J , LIU G . A survey of stereo matching algorithms. Chinese High Technology Letters, 2020, 30 (2): 157- 165.
6	谢小鹏, 欧永东, 王银安, 等. 基于融合代价和分段优化的立体匹配算法. 激光与光电子学进展, 2021, 58 (12): 1215004.
	XIE X P , OU Y D , WANG Y A , et al. Stereo matching algorithm based on fusion cost and segmentation optimization. Laser and Optoelectronics Progress, 2021, 58 (12): 1215004.
7	LU C H , UCHIYAMA H , THOMAS D , et al. Sparse cost volume for efficient stereo matching. Remote Sensing, 2018, 10 (11): 1844. doi: 10.3390/rs10111844
8	陈映光, 周佩, 朱江平, 等. 基于改进Census变换和自适应支持域的立体匹配. 激光与光电子学进展, 2021, 58 (14): 1433002.
	CHEN Y G , ZHOU P , ZHU J P , et al. Stereo matching based on improved Census transformation and adaptive support region. Laser and Optoelectronics Progress, 2021, 58 (14): 1433002.
9	OKUTOMI M, KANADE T. A multiple-baseline stereo[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 63-69.
10	范海瑞, 杨帆, 潘旭冉, 等. 一种改进Census变换与梯度融合的立体匹配算法. 光学学报, 2018, 38 (2): 267- 277.
	FAN H R , YANG F , PAN X R , et al. Stereo matching algorithm for improved Census transform and gradient fusion. Acta Optica Sinica, 2018, 38 (2): 267- 277.
11	HOSNI A , BLEYER M , GELAUTZ M . Secrets of adaptive support weight techniques for local stereo matching. Computer Vision and Image Understanding, 2013, 117 (6): 620- 632. doi: 10.1016/j.cviu.2013.01.007
12	MEI X, SUN X, ZHOU M C, et al. On building an accurate stereo matching system on graphics hardware[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2011: 467-474.
13	LIU H , WANG R , XIA Y P , et al. Improved cost computation and adaptive shape guided filter for local stereo matching of low texture stereo images. Applied Sciences, 2020, 10 (5): 1869. doi: 10.3390/app10051869
14	董磊. 基于自适应窗口的PatchMatch立体匹配算法研究及其应用[D]. 武汉: 武汉轻工大学, 2022.
	DONG L. Research and application of PatchMatch stereo matching algorithm based on adaptive window[D]. Wuhan: Wuhan Polytechnic University, 2022. (in Chinese)
15	文斌, 朱晗. 基于自适应权重的立体匹配优化算法. 计算机工程, 2021, 47 (4): 268- 276. doi: 10.19678/j.issn.1000-3428.0056991
	WEN B , ZHU H . Stereo matching optimization algorithm based on adaptive weights. Computer Engineering, 2021, 47 (4): 268- 276. doi: 10.19678/j.issn.1000-3428.0056991
16	YOON K J , KWEON I S . Adaptive support-weight approach for correspondence search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28 (4): 650- 656. doi: 10.1109/TPAMI.2006.70
17	ZHANG K , LU J B , LAFRUIT G . Cross-based local stereo matching using orthogonal integral images. IEEE Transactions on Circuits and Systems for Video Technology, 19 (7): 1073- 1079. doi: 10.1109/TCSVT.2009.2020478
18	HAMZAH R A, KADMIN A F, ABD GANI S F. A modelling of stereo matching algorithm for machine vision application[C]//Proceedings of the 4th International Conference on Mechanical, Manufacturing and Plant Engineering. Washington D. C., USA: IEEE Press, 2018: 499-508.
19	YANG Q X . Hardware-efficient bilateral filtering for stereo matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36 (5): 1026- 1032. doi: 10.1109/TPAMI.2013.186
20	卢昂, 储珺, 冷璐. 基于高低频特征增强的图像去雾. 计算机工程, 2023, 49 (8): 174- 181. doi: 10.19678/j.issn.1000-3428.0064826
	LU A , CHU J , LENG L . Image dehazing based on high-frequency and low-frequency feature enhancement. Computer Engineering, 2023, 49 (8): 174- 181. doi: 10.19678/j.issn.1000-3428.0064826
21	SCHARSTEIN D, HIRSCHMÜLLER H. Middlebury stereo evaluation-version 3[EB/OL]. [2023-09-30]. https://vision.middlebury.edu/stereo/.
22	HIRSCHMÜLLER H , INNOCENT P R , GARIBALDI J . Real-time correlation-based stereo vision with reduced border errors. International Journal of Computer Vision, 2002, 47 (1): 229- 246.
23	HAMZAH R A , IBRAHIM H , ABU HASSAN A H . Stereo matching algorithm based on per pixel difference adjustment, iterative guided filter and graph segmentation. Journal of Visual Communication and Image Representation, 2017, 42, 145- 160.
24	HAMZAH R A, IBRAHIM H, ABU HASSAN A H. Stereo matching algorithm for 3D surface reconstruction based on triangulation principle[C]//Proceedings of the 1st International Conference on Information Technology, Information Systems and Electrical Engineering. Washington D. C., USA: IEEE Press, 2016: 119-124.
25	KONG L Y , ZHU J P , YING S C . Stereo matching based on guidance image and adaptive support region. Acta Optica Sinica, 2020, 40 (9): 0915001.
26	LIU J, JI S P. A novel recurrent encoder-decoder structure for large-scale multi-view stereo reconstruction from an open aerial dataset[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 6050-6059.

[1]	Zhengjia WANG, Feifei HU, Chengjuan ZHANG, Zhuo LEI, Tao HE. Adaptive Window Stereo Matching Algorithm with Lightweight Transformer [J]. Computer Engineering, 2024, 50(2): 256-265.
[2]	Zhihao LIU, Fanyun MENG, Jinhe WANG, Nan ZHANG. Stereo Matching Algorithm Based on Atrous Convolution and Attention Module [J]. Computer Engineering, 2023, 49(8): 223-231.
[3]	Hua HOU, Hongyang GUO, Chaona DAI, Junhui LI. Stereo Matching Algorithm Combining Multiple Attention and Iterative Optimization [J]. Computer Engineering, 2023, 49(7): 161-168.
[4]	YU Jiaxin, WANG Chunyuan, HAN Hua, GAO Yan. Stereo Matching Algorithm Based on Fusion Cost and Optimized Guided Filtering [J]. Computer Engineering, 2023, 49(3): 257-262,270.
[5]	WANG Chunzhi, NIU Hongxia. Sand-Dust Degraded Image Enhancement Algorithm Based on Histogram Equalization and MSRCR [J]. Computer Engineering, 2022, 48(9): 223-229.
[6]	LIU Jianguo, JI Guo, YAN Fuwu, SHEN Jianhong, SUN Yunfei. Stereo Matching Network Based on Disparity Optimization [J]. Computer Engineering, 2022, 48(3): 220-228.
[7]	ZHANG Xiying, WANG Houbo, BIAN Jilong. Stereo Matching Network with Multi-Cost Fusion [J]. Computer Engineering, 2022, 48(2): 186-193.
[8]	LIU Zhenguo, LI Zhao, SONG Tengteng, HE Yizhi. Stereo Matching Network Combining Deformable Convolution and Bilateral Grid [J]. Computer Engineering, 2022, 48(12): 241-247,254.
[9]	ZHAO Chenyuan, LI Wenxin, ZHANG Qingxi. An Improved Real-Time Semi-Global Stereo Matching Algorithm and Its Hardware Implementation [J]. Computer Engineering, 2021, 47(9): 162-170.
[10]	HUANG Bin, HU Likun, ZHANG Yu. Improved Census Stereo Matching Algorithm Based on Adaptive Weight [J]. Computer Engineering, 2021, 47(5): 189-196.
[11]	WEN Bin, ZHU Han. Stereo Matching Optimization Algorithm Based on Adaptive Weights [J]. Computer Engineering, 2021, 47(4): 268-276.
[12]	WANG Jinhe, CHE Zhilong, ZHANG Nan, MENG Fanyun, SU Cuili, TAN Hao. Stereo Matching Method Based on Multi-Scale and Multi-Level Features Fusion [J]. Computer Engineering, 2021, 47(3): 243-248.
[13]	WANG Jinhe, SU Cuili, MENG Fanyun, CHE Zhilong, TAN Hao, ZHANG Nan. Stereo Matching Network Based on Asymmetric Spatial Pyramid Pooling [J]. Computer Engineering, 2020, 46(7): 228-234,242.
[14]	ZHANG Yifei, LI Xinfu, TIAN Xuedong. SAD Stereo Matching Algorithm Combining Edge Features [J]. Computer Engineering, 2020, 46(4): 236-240,246.
[15]	YANG Gang, JIN Tao, WANG Dawei, CAO Jingjin, ZHANG Na, YAN Biwu, LI Tao, CHENG Yuan. Improved Cost Aggregation Algorithm for Fast Stereo Matching [J]. Computer Engineering, 2019, 45(7): 268-272,281.

Please choose a citation manager

Content to export