An Improved Real-Time Semi-Global Stereo Matching Algorithm and Its Hardware Implementation

doi:10.19678/j.issn.1000-3428.0059891

Abstract

Abstract: When applied to the real-time stereo matching systems based on Field Programmable Gate Array(FPGA), the Census Transform(CT) algorithm has a high false matching rate in specific areas.In order to improve the matching accuracy, a real-time Semi-Global stereo Matching(SGM) algorithm with highly parallel pipeline structure is proposed.The algorithm takes the combination of the improved Tanimoto distance and the weighted 4-direction Absolute Differences of Gradient(ADG) as the initial matching cost.In the cost aggregation stage, a 4-path SMG algorithm is used.In the disparity computation stage, the winner-takes-all strategy is chosen.Then in the parallax correction stage, the threshold detection algorithm is used to replace the traditional left-right check algorithm. Experimental results show that the proposed algorithm can effectively improve the discrimination between weak texture and edge regions, and reduce the dependence on center point and resource consumption.The average mismatch rate of the algorithm on Middlebury platform is 7.52%, and its matching rate on Xilinx zynq-7000 platform is 98 frame/s.

Key words: Field Programmable Gate Array(FPGA), stereo matching, Census Transform(CT), Tanimoto distance, real-time Semi-Global stereo Matching(SGM)

摘要： 在基于现场可编程门阵列的实时立体匹配系统中，Census变换算法针对特定区域的误匹配率较高。为提高匹配精度，提出一种具有高并行性流水线结构的实时半全局立体匹配算法并进行硬件实现。将改进的Tanimoto距离和带权重4方向的梯度绝对值差进行组合，作为新的初始匹配代价。在代价聚合阶段采用4路径并行结构的SGM算法，在视差选择阶段采用赢家通吃策略，在视差校正阶段采用阈值检测算法代替传统左右一致性检验算法。实验结果表明，该算法能够有效提高弱纹理和边缘区域的区分度，减少对中心点的依赖，降低资源占用，其在Middleburry平台上的平均误匹配率仅为7.52%，在Xilinx Zynq-7000平台上的匹配速率达到98 frame/s。

关键词: 现场可编程门阵列, 立体匹配, Census变换, Tanimoto距离, 实时半全局匹配

CLC Number:

TP391.41

ZHAO Chenyuan, LI Wenxin, ZHANG Qingxi. An Improved Real-Time Semi-Global Stereo Matching Algorithm and Its Hardware Implementation[J]. Computer Engineering, 2021, 47(9): 162-170.

赵晨园, 李文新, 张庆熙. 一种改进的实时半全局立体匹配算法及硬件实现[J]. 计算机工程, 2021, 47(9): 162-170.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0059891

http://www.ecice06.com/EN/Y2021/V47/I9/162

Figures/Tables 13

References

[1] CAO H, WANG X, et al.A novel recognition algorithm in 3D point clouds based for on local spherical harmonics[C]//Proceedings of 2019 IEEE International Conference on Mechatronics and Automation.Washington D.C., USA:IEEE Press, 2019:1041-1046.
[2] 周柚, 杨森, 李大琳, 等.基于现场可编程门电路的人脸检测识别加速平台[J].吉林大学学报(工学版), 2019, 49(6):2051-2057. ZHOU Y, YANG S, YAN D L, et al.Acceleration platform for face detection and recognition based on field-programmable gate array[J].Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6):2051-2057.(in Chinese)
[3] OUSSAMA Z, MOHAMMED R, AOUATIF A, et al.A hierarchical stereo matching algorithm based on adaptive support region aggregation method[J].Pattern Recognition Letters, 2018, 112:205-211.
[4] MA N, MEN Y, MEN C, et al.Segmentation-based stereo matching using combinatorial similarity measurement and adaptive support region[J].International Journal for Light and Electron Optics, 2017, 137:124-134.
[5] CHAI Y, YANG F.Semi-global stereo matching algorithm based on minimum spanning tree[C]//Proceedings of 2018 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference, Washington D.C., USA:IEEE Press, 2018:2181-2185.
[6] HERNANDEZ-JUAREZ D, CHACÓN A, ESPINOSA A, et al.Embedded real-time stereo estimation via semi-global matching on the GPU[J].Procedia Computer Science, 2016, 80(C):143-153.
[7] ZBONTAR J, LECUN Y.Stereo matching by training a convolutional neural network to compare image patches[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2015:1-28.
[8] SCHARSTEIN D, SZELISKI R.A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J].International Journal of Computer Vision, 2002, 47(1/2/3):7-42.
[9] HUMENBERGER M, ZINNER C, WEBER M, et al.A fast stereo matching algorithm suitable for embedded real-time system[J].Computer Vision and Image Understanding, 2010, 114(11):1180-1202.
[10] ZABIH R, WOODFILL J.Non-parametric local transforms for computing visual correspondence[C]//Proceedings of the 3rd European Conference on Computer Vision.Berlin, Germany:Springer, 1994:151-158.
[11] 黄彬, 胡立坤, 张宇.基于自适应权重的改进Census立体匹配算法[J].计算机工程, 2021, 47(5):189-196. HUANG B, HU L K, ZHANG Y.Improved Census stereo matching algorithm based on adaptive weight[J].Computer Engineering, 2021, 47(5):189-196.(in Chinese)
[12] 祝世平, 闫利那, 李政.基于改进Census变换和动态规划的立体匹配算法[J].光学学报, 2016, 36(4):216-224. ZHU S P, YAN L N, LI Z, Stereo matching algorithm based on improved Census transform and dynamic programming[J].Acta Optica Sinica, 2016, 36(4):216-224.(in Chinese)
[13] 卢迪, 林雪.多种相似性测度结合的局部立体匹配算法[J].机器人, 2016, 38(1):1-7. LU D, LIN X.A local stereo matching algorithm based on the combination of multiple similarity measures[J].Robot, 2016, 38(1):1-7.(in Chinese)
[14] MEI X, SUN X, ZHOU M C, et al.On building an accurate stereo matching system on graphics hardware[C]//Proceedings of 2011 IEEE International Conference on Computer Vision Workshops.Washington D.C., USA:IEEE Press, 2011:467-474.
[15] 李大华, 沈洪宇, 于晓, 等.一种改进Census变换的双目匹配测距方法[J].激光与光电子学进展, 2019, 56(11):197-202. LI D H, SHEN H Y, YU X, et al.Binocular ranging method using stereo matching based on improved Census transform[J].Laser & Optoelectronics Progress, 2019, 56(11):197-202.(in Chinese)
[16] 王旭初, 刘辉煌, 牛彦敏.融合多尺度局部特征与深度特征的双目立体匹配[J].光学学报, 2020, 40(2).119-131. WANG X C, LIU H H, NIU Y B.Binocular stereo matching by combining multiscale local and deep feature[J].Acta Optica Sinica, 2020, 40(2):119-131.(in Chinese)
[17] WANG W, YAN J, XU N, et al.Real-time high-quality stereo vision system in FPGA[J].IEEE Transactions on Circuits and Systems for Video Technology, 2015, 25(10):1696-1708.
[18] 孔令寅, 朱江平, 应三丛.基于引导图像和自适应支持域的立体匹配[J].光学学报, 2020, 40(9):86-98. KONG L Y, ZHU J P, YING S C.Stereo matching based on guidance image and adaptive support region[J].Acta Optica Sinica, 2020, 40(9):86-98.(in Chinese)
[19] LI Y, LI Z, YANG C, et al.High throughput hardware architecture for accurate semi-global matching[J].Integration, the VLSI Journal, 2019, 65:417-427.
[20] HIRSCHMULLER H.Stereo processing by semiglobal matching and mutual information[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2):328-341.
[21] RAHNAMA O, CAVALLARI T, GOLODETZ S, et al.R3SGM:real-time raster-respecting semi-global matching for power-constrained systems[C]//Proceedings of International Conference on Field Programmable Technology.Washington D.C., USA:IEEE Press, 2018:105-112.
[22] BANZ C, HESSELBARTH S, FLATT H, et al.Real-time stereo vision system using semi-global matching disparity estimation:architecture and FPGA-implementation[C]//Proceedings of International Conference on Embedded Computer Systems.Washington D.C., USA:IEEE Press, 2010:93-101.
[23] ALBA A, ARCE-SANTANA E, AGUILAR-PONCE R M, et al.Phase-correlation guided area matching for realtime vision and video encoding[J].Journal of Real Time Image Processing, 2014, 9(4):621-633.
[24] GEHRIG S K, EBERLI F, MEYER T.A real-time low-power stereo vision engine using semi-global matching[C]//Proceedings of International Conference on Colour Vision Society.Braga, The Portuguese Republic:[s.n.], 2009:134-143.

Please choose a citation manager

Content to export