多成本融合的立体匹配网络

doi:10.19678/j.issn.1000-3428.0062144

摘要/Abstract

摘要： 立体匹配网络中的特征提取是提高双目视觉立体匹配精确度的关键步骤。为充分提取图像特征信息，结合密集空洞卷积、空间金字塔池化和堆叠沙漏的特点，构建一种多成本融合的立体匹配网络DCNet。引入密集空洞卷积和空间金字塔池化方法提取多尺度特征信息，同时使用轻量化注意力模块优化多尺度特征信息，构建多特征融合的匹配代价卷。在此基础上，利用3D卷积神经网络和堆叠沙漏网络聚合匹配代价信息，并通过回归的方式生成视差图。实验结果表明，该网络在KITTI2015数据集上的误匹配率为2.12%，相比PSMNet、DisNetC、PDSNet等网络，在特征提取部分能够获得更丰富的特征信息，且提升特征匹配的效果。

关键词: 立体匹配, 密集神经网络, 深度卷积神经网络, 深度学习, 注意力机制

Abstract: In stereo matching networks, the feature extraction process is key for improving the accuracy of binocular stereo matching.To extract image feature information, this study combines the characteristics of dense atrous convolution, spatial pyramid pooling, and stacked hourglass to propose stereo matching network DCNet with multi-cost fusion.The network introduces dense atrous convolution and spatial pyramid pooling to extract multi-scale feature information.The lightweight attention module is used to optimize the multi-scale feature information as well as construct the matching cost volume of multi feature fusion.The matching cost information is aggregated by the 3D convolution neural network and stacked hourglass network, whereby the parallax map is generated by regression.The experimental results on KITTI2015 data set show that the false matching rate of this network is 2.12%.Compared with PSMNet, DISNetC, and PDSNet, the proposed method provides richer feature information in feature extraction and improves the effect of feature matching.

Key words: stereo matching, dense neural network, deep Convolutional Neural Network(CNN), Deep Learning(DL), attention mechanism

中图分类号:

TP391.4

张锡英, 王厚博, 边继龙. 多成本融合的立体匹配网络[J]. 计算机工程, 2022, 48(2): 186-193.

ZHANG Xiying, WANG Houbo, BIAN Jilong. Stereo Matching Network with Multi-Cost Fusion[J]. Computer Engineering, 2022, 48(2): 186-193.

https://www.ecice06.com/CN/Y2022/V48/I2/186

图/表 15

20220301124047

20220301124051

20220301124055

20220301124058

20220301124102

20220301124106

20220301124110

20220301124113

20220301124117

20220301124121

20220301124124

20220301124128

20220301124132

20220301124136

20220301124140

参考文献

[1] 宋宇鹏, 边继龙, 安翔, 等.基于注意力机制的DenseNet模型的树种识别应用[J].实验室研究与探索, 2020, 39(7):127-132, 178. SONG Y P, BIAN J L, AN X, et al.Application of tree species recognition based on attention mechanism DenseNet model[J].Laboratory Research and Exploration, 2020, 39(7):127-132, 178.(in Chinese)
[2] XU H F, ZHANG J Y.AANet:adaptive aggregation network for efficient stereo matching[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:1-10.
[3] 齐永锋, 马中玉.基于深度残差网络的多损失头部姿态估计[J].计算机工程, 2020, 46(12):247-253. QI Y F, MA Z Y.Multi-loss head pose estimation based on deep residual network[J].Computer Engineering, 2020, 46(12):247-253.(in Chinese)
[4] LIU Z Z, CHEN T D.Distance measurement system based on binocular stereo vision[C]//Proceedings of International Joint Conference on Artificial Intelligence.Washington D.C., USA:IEEE Press, 2009:1-10.
[5] SCHARSTEIN D, SZELISKI R.A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J].International Journal of Computer Vision, 2002, 47(1):7-42.
[6] SONG X, ZHAO X, FANG L J, et al.Edgestereo:an effective multi-task learning network for stereo matching and edge detection[J].International Journal of Computer Vision, 2020, 128(5):910-930.
[7] MAYER N, ILG E, HAUSSER P, et al.A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1-10.
[8] KENDALL A, MARTIROSYAN H, DASGUPTA S, et al.End-to-end learning of geometry and context for deep stereo regression[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:1-10.
[9] GU X D, FAN Z W, ZHU S Y, et al.Cascade cost volume for high-resolution multi-view stereo and stereo matching[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:1-11.
[10] CHEN R, HAN S F, XU J, et al.Visibility-aware point-based multi-view stereo network[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10):3695-3708.
[11] NEWELL A, YANG K Y, JIA D.Stacked hourglass networks for human pose estimation[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:483-499.
[12] CHANG J R, CHEN Y S.Pyramid stereo matching network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:1-10.
[13] HE K M, ZHANG X Y, REN S Q, et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:1-10.
[14] HE K M, ZHANG X Y, REN S Q, et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(9):1904-1916.
[15] GUO X Y, YANG K, YANG W K, et al.Group-wise correlation stereo network[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:1-10.
[16] YANG M K, YU K, ZHANG C, et al.DenseASPP for semantic segmentation in street scenes[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:1-11.
[17] WOO S, PARK J, LEE J Y, et al.CBAM:convolutional block attention module[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:3-19.
[18] 贾瑞明, 李阳, 李彤, 等.多层级特征融合结构的单目图像深度估计网络[J].计算机工程, 2020, 46(12):207-214. JIA R M, LI Y, LI T, et al.Monocular image depth estimation network based on multi-level feature fusion structure[J].Computer Engineering, 2020, 46(12):207-214.(in Chinese)
[19] HUANG G, LIU Z, LAURENS V, et al.Densely connected convolutional networks[C]//Proceedings of IEEE Computer Society.Washington D.C., USA:IEEE Press, 2016:1-12.
[20] CHEN L C, PAPANDREOU G, KOKKINOS I, et al.DeepLab:semantic image segmentation with deep convolutional Nets, atrous convolution, and fully connected CRFs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
[21] GIRSHICK R.Fast R-CNN[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2015:1-12.
[22] 杨蕙同, 雷亮, 林永春.基于多尺度注意力特征融合的双目深度估计算法[J/OL].激光与光电子学进展:1-10[2021-06-20].http://kns.cnki net/kcms/detail/31.1690.TN.20210802.1731.080.html. YANG H T, LEI L, LIN Y C.Binocular depth estimation algorithm based on multi-scale attention feature fusion[J/OL].Laser & Optoelectronics Progress:HO[2021-06-20].http://kns.cnki net/kcms/detail/31.1690.TN.20210802.1731.080.html.(in Chinese)

选择文件类型/文献管理软件名称

选择包含的内容