基于四梯度多特征和四权重滤波的立体匹配

doi:10.19678/j.issn.1000-3428.0068750

摘要/Abstract

摘要：

立体匹配的精度直接决定了后续三维场景信息恢复的准确度, 提高视差图的精度一直是研究的热点问题。传统的立体匹配方法对图像的局部结构表达不够精准, 对相似结构区域内的点或前景和背景交界处及含有错误代价点的处理效果不理想。针对以上问题, 提出一种基于四梯度多特征代价和四权重滤波的立体匹配方法。由图像灰度和四方向梯度组成多特征空间, 通过二次编码计算多特征Census变换代价, 再结合多特征绝对误差(AD)代价形成多特征代价, 提升局部结构的表达准确度; 以空间邻近度、像素值相似度、区域相似度和代价相似度四权重构建的滤波核进行代价聚合, 降低异常代价的聚合权重; 以胜者为王(WTA)算法计算初始视差, 以左右一致性检测进行视差初步修正, 结合自适应窗口和视差阈值进行视差优化。在Middlebury V3立体平台上的实验结果表明, 该方法在非遮挡区域和全部区域加权平均的bad4.0分别为14.7%和20.6%, 效能显著优于现有的传统立体匹配算法。

关键词: 立体匹配, 绝对误差(AD)-Census变换, 四梯度多特征, 双边滤波, 胜者为王(WTA)算法, 代价聚合

Abstract:

The accuracy of stereo matching directly determines the precision of subsequent 3D scene information recovery, and enhancing the accuracy of disparity maps has attracted considerable attention among researchers. Traditional stereo matching methods inadequately represent the local structure of images, particularly within regions of similar structures, at the junctions of the foreground and background, and in areas containing erroneous cost points. To address these issues, this paper proposes a stereo matching method based on a quad-gradient, multi-feature cost, and quad-weight filtering. This method constructs a multi-feature space composed of image intensity and gradients in four directions. It employs quadratic encoding to calculate the cost of the multi-feature census transform of images and then combines it with the multi-feature Absolute Difference (AD) cost to enhance the accuracy of the local structural representation. A filter kernel constructed with four weights, namely, spatial proximity, pixel intensity similarity, regional similarity, and cost similarity, is used for cost aggregation to mitigate the aggregation weight of abnormal costs. The initial disparity is calculated using the Winner-Take-All (WTA) algorithm and is preliminarily corrected through a left-right consistency check, followed by disparity optimization using an adaptive window and a disparity threshold. Results of experiments on the Middlebury V3 stereo platform indicate that the algorithm significantly outperforms existing traditional stereo matching algorithms. It yields a weighted average bad4.0 value (percentage of ″bad″ pixels having an error greater than 4.0 pixels) of 14.7% in non-occluded regions and 20.6% in all regions.

Key words: stereo matching, Absolute Difference (AD)-census transformation, quad-gradient multi-feature, bilateral filtering, Winner-Take-All (WTA) algorithm, cost aggregation

董振武, 王春媛, 余嘉昕, 周斌, 吴扬. 基于四梯度多特征和四权重滤波的立体匹配[J]. 计算机工程, 2025, 51(4): 271-280.

DONG Zhenwu, WANG Chunyuan, YU Jiaxin, ZHOU Bin, WU Yang. Stereo Matching Based on Quad Gradient Multi-Feature and Quad-Weight Filtering[J]. Computer Engineering, 2025, 51(4): 271-280.

https://www.ecice06.com/CN/Y2025/V51/I4/271

图/表 17

图1 提出方法的实现流程

Fig.1 The realization procedure of proposed method

图2 图像信息到多特征空间的转换

Fig.2 Conversion of image information to multiple feature spaces

图3 MCensus变换一次编码矩阵

Fig.3 First encoding matrix of MCensus transformation

图4 MCensus变换二次编码矩阵

Fig.4 Sencond encoding matrix of MCensus transformation

图5 自适应窗口与区域相似度权重

Fig.5 Adaptive window and region similarity weights

图6 子矩阵选取

Fig.6 Submatrix selection

图7 不同匹配代价计算方法对比

Fig.7 Comparison of different matching cost calculation methods

图8 不同代价聚合算法对比

Fig.8 Comparison of different cost aggregation algorithms

图9 非遮挡区域bad4.0对比

Fig.9 Comparison of non occluded areas bad4.0

图10 全部区域bad4.0对比

Fig.10 Comparison of bad4.0 across all regions

图11 不同算法最终视差图对比

Fig.11 Comparison of final parallax images of different algorithms

图12 本文算法在遥感图像中的应用

Fig.12 Application of this algorithm in remote sensing images

参考文献 26

1	王厚博. 面向自动驾驶的立体匹配算法[D]. 哈尔滨: 东北林业大学, 2022.
	WANG H B. Stereo matching algorithm for autonomous driving[D]. Harbin: Northeast Forestry University, 2022. (in Chinese)
2	刘明国. 面向机器人导航的立体视觉及目标检测技术研究[D]. 南京: 南京理工大学, 2017.
	LIU M G. Research on stereo vision and target detection technology for robot navigation[D]. Nanjing: Nanjing University of Science and Technology, 2017. (in Chinese)
3	周施. 三维重建中立体匹配算法研究[D]. 沈阳: 东北大学, 2020.
	ZHOU S. Research on stereo matching algorithm in 3D reconstruction[D]. Shenyang: Northeastern University, 2020. (in Chinese)
4	韩昊, 李参海, 丘晓枫. 基于神经网络密集匹配的资源三号DSM提取. 遥感信息, 2022, 37 (3): 101- 108.
	HAN H , LI C H , QIU X F . ZY-3 DSM extraction based on neural network dense matching. Remote Sensing Information, 2022, 37 (3): 101- 108.
5	陈华, 王立军, 刘刚. 立体匹配算法研究综述. 高技术通讯, 2020, 30 (2): 157- 165.
	CHEN H , WANG L J , LIU G . A survey of stereo matching algorithms. Chinese High Technology Letters, 2020, 30 (2): 157- 165.
6	谢小鹏, 欧永东, 王银安, 等. 基于融合代价和分段优化的立体匹配算法. 激光与光电子学进展, 2021, 58 (12): 1215004.
	XIE X P , OU Y D , WANG Y A , et al. Stereo matching algorithm based on fusion cost and segmentation optimization. Laser and Optoelectronics Progress, 2021, 58 (12): 1215004.
7	LU C H , UCHIYAMA H , THOMAS D , et al. Sparse cost volume for efficient stereo matching. Remote Sensing, 2018, 10 (11): 1844. doi: 10.3390/rs10111844
8	陈映光, 周佩, 朱江平, 等. 基于改进Census变换和自适应支持域的立体匹配. 激光与光电子学进展, 2021, 58 (14): 1433002.
	CHEN Y G , ZHOU P , ZHU J P , et al. Stereo matching based on improved Census transformation and adaptive support region. Laser and Optoelectronics Progress, 2021, 58 (14): 1433002.
9	OKUTOMI M, KANADE T. A multiple-baseline stereo[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 63-69.
10	范海瑞, 杨帆, 潘旭冉, 等. 一种改进Census变换与梯度融合的立体匹配算法. 光学学报, 2018, 38 (2): 267- 277.
	FAN H R , YANG F , PAN X R , et al. Stereo matching algorithm for improved Census transform and gradient fusion. Acta Optica Sinica, 2018, 38 (2): 267- 277.
11	HOSNI A , BLEYER M , GELAUTZ M . Secrets of adaptive support weight techniques for local stereo matching. Computer Vision and Image Understanding, 2013, 117 (6): 620- 632. doi: 10.1016/j.cviu.2013.01.007
12	MEI X, SUN X, ZHOU M C, et al. On building an accurate stereo matching system on graphics hardware[C]//Proceedings of IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2011: 467-474.
13	LIU H , WANG R , XIA Y P , et al. Improved cost computation and adaptive shape guided filter for local stereo matching of low texture stereo images. Applied Sciences, 2020, 10 (5): 1869. doi: 10.3390/app10051869
14	董磊. 基于自适应窗口的PatchMatch立体匹配算法研究及其应用[D]. 武汉: 武汉轻工大学, 2022.
	DONG L. Research and application of PatchMatch stereo matching algorithm based on adaptive window[D]. Wuhan: Wuhan Polytechnic University, 2022. (in Chinese)
15	文斌, 朱晗. 基于自适应权重的立体匹配优化算法. 计算机工程, 2021, 47 (4): 268- 276. doi: 10.19678/j.issn.1000-3428.0056991
	WEN B , ZHU H . Stereo matching optimization algorithm based on adaptive weights. Computer Engineering, 2021, 47 (4): 268- 276. doi: 10.19678/j.issn.1000-3428.0056991
16	YOON K J , KWEON I S . Adaptive support-weight approach for correspondence search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28 (4): 650- 656. doi: 10.1109/TPAMI.2006.70
17	ZHANG K , LU J B , LAFRUIT G . Cross-based local stereo matching using orthogonal integral images. IEEE Transactions on Circuits and Systems for Video Technology, 19 (7): 1073- 1079. doi: 10.1109/TCSVT.2009.2020478
18	HAMZAH R A, KADMIN A F, ABD GANI S F. A modelling of stereo matching algorithm for machine vision application[C]//Proceedings of the 4th International Conference on Mechanical, Manufacturing and Plant Engineering. Washington D. C., USA: IEEE Press, 2018: 499-508.
19	YANG Q X . Hardware-efficient bilateral filtering for stereo matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36 (5): 1026- 1032. doi: 10.1109/TPAMI.2013.186
20	卢昂, 储珺, 冷璐. 基于高低频特征增强的图像去雾. 计算机工程, 2023, 49 (8): 174- 181. doi: 10.19678/j.issn.1000-3428.0064826
	LU A , CHU J , LENG L . Image dehazing based on high-frequency and low-frequency feature enhancement. Computer Engineering, 2023, 49 (8): 174- 181. doi: 10.19678/j.issn.1000-3428.0064826
21	SCHARSTEIN D, HIRSCHMÜLLER H. Middlebury stereo evaluation-version 3[EB/OL]. [2023-09-30]. https://vision.middlebury.edu/stereo/.
22	HIRSCHMÜLLER H , INNOCENT P R , GARIBALDI J . Real-time correlation-based stereo vision with reduced border errors. International Journal of Computer Vision, 2002, 47 (1): 229- 246.
23	HAMZAH R A , IBRAHIM H , ABU HASSAN A H . Stereo matching algorithm based on per pixel difference adjustment, iterative guided filter and graph segmentation. Journal of Visual Communication and Image Representation, 2017, 42, 145- 160.
24	HAMZAH R A, IBRAHIM H, ABU HASSAN A H. Stereo matching algorithm for 3D surface reconstruction based on triangulation principle[C]//Proceedings of the 1st International Conference on Information Technology, Information Systems and Electrical Engineering. Washington D. C., USA: IEEE Press, 2016: 119-124.
25	KONG L Y , ZHU J P , YING S C . Stereo matching based on guidance image and adaptive support region. Acta Optica Sinica, 2020, 40 (9): 0915001.
26	LIU J, JI S P. A novel recurrent encoder-decoder structure for large-scale multi-view stereo reconstruction from an open aerial dataset[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 6050-6059.

[1]	王正家, 胡飞飞, 张成娟, 雷卓, 何涛. 引入轻量级Transformer的自适应窗口立体匹配算法[J]. 计算机工程, 2024, 50(2): 256-265.
[2]	刘志浩, 孟凡云, 王金鹤, 张楠. 基于空洞卷积与注意力模块的立体匹配算法[J]. 计算机工程, 2023, 49(8): 223-231.
[3]	侯华, 郭宏洋, 代超娜, 李峻辉. 结合多重注意力与迭代优化的立体匹配算法[J]. 计算机工程, 2023, 49(7): 161-168.
[4]	余嘉昕, 王春媛, 韩华, 高燕. 基于融合代价和优化引导滤波的立体匹配算法[J]. 计算机工程, 2023, 49(3): 257-262,270.
[5]	王春智, 牛宏侠. 基于直方图均衡化与MSRCR的沙尘降质图像增强算法[J]. 计算机工程, 2022, 48(9): 223-229.
[6]	刘建国, 纪郭, 颜伏伍, 沈建宏, 孙云飞. 基于视差优化的立体匹配网络[J]. 计算机工程, 2022, 48(3): 220-228.
[7]	张锡英, 王厚博, 边继龙. 多成本融合的立体匹配网络[J]. 计算机工程, 2022, 48(2): 186-193.
[8]	刘振国, 李钊, 宋滕滕, 何益智. 结合可变形卷积与双边网格的立体匹配网络[J]. 计算机工程, 2022, 48(12): 241-247,254.
[9]	尹晨阳, 职恒辉, 李慧斌. 基于深度学习的双目立体匹配方法综述[J]. 计算机工程, 2022, 48(10): 1-12.
[10]	赵晨园, 李文新, 张庆熙. 一种改进的实时半全局立体匹配算法及硬件实现[J]. 计算机工程, 2021, 47(9): 162-170.
[11]	黄彬, 胡立坤, 张宇. 基于自适应权重的改进Census立体匹配算法[J]. 计算机工程, 2021, 47(5): 189-196.
[12]	文斌, 朱晗. 基于自适应权重的立体匹配优化算法[J]. 计算机工程, 2021, 47(4): 268-276.
[13]	王金鹤, 车志龙, 张楠, 孟凡云, 苏翠丽, 谭浩. 基于多尺度和多层级特征融合的立体匹配方法[J]. 计算机工程, 2021, 47(3): 243-248.
[14]	王金鹤, 苏翠丽, 孟凡云, 车志龙, 谭浩, 张楠. 基于非对称空间金字塔池化的立体匹配网络[J]. 计算机工程, 2020, 46(7): 228-234,242.
[15]	张一飞, 李新福, 田学东. 融合边缘特征的SAD立体匹配算法[J]. 计算机工程, 2020, 46(4): 236-240,246.

选择文件类型/文献管理软件名称

选择包含的内容