基于多尺度和多层级特征融合的立体匹配方法

doi:10.19678/j.issn.1000-3428.0056715

计算机工程 ›› 2021, Vol. 47 ›› Issue (3): 243-248. doi: 10.19678/j.issn.1000-3428.0056715

基于多尺度和多层级特征融合的立体匹配方法

王金鹤, 车志龙, 张楠, 孟凡云, 苏翠丽, 谭浩

青岛理工大学信息与控制工程学院, 山东青岛 266000

收稿日期:2019-11-26 修回日期:2020-03-08 发布日期:2020-06-18
作者简介:王金鹤(1963-),男,教授、博士,主研方向为模式识别与智能系统;车志龙,硕士研究生;张楠(通信作者),讲师、硕士;孟凡云,讲师、博士;苏翠丽、谭浩,硕士研究生。
基金资助:
国家自然科学基金（31271077）；山东省重点研发项目（2019GGX104089）；山东省高等学校科技计划项目（J17KA061）。

Stereo Matching Method Based on Multi-Scale and Multi-Level Features Fusion

WANG Jinhe, CHE Zhilong, ZHANG Nan, MENG Fanyun, SU Cuili, TAN Hao

School of Information and Control Engineering, Qingdao University of Technology, Qingdao, Shandong 266000, China

Received:2019-11-26 Revised:2020-03-08 Published:2020-06-18

摘要/Abstract

摘要： 基于卷积神经网络的立体匹配方法未充分利用图像中各个层级的特征图信息，造成对图像在不适定区域的特征提取能力较差。提出一种融合多尺度与多层级特征的立体匹配方法。通过在双塔结构卷积神经网络模型的前端设计一个池化金字塔层，提取图像的多尺度低层结构特征。在该网络模型的后端融合最后三层网络的高级语义特征来提取图像特征，并对图像特征进行相似性度量后输出视差图。在KITTI 2015数据集上的实验结果表明，与LUO和Anita方法相比，该方法的像素误差精度分别由14.65%、8.30%降至8.02%，且可得到细节信息更好的视差图。

关键词: 立体匹配, 卷积神经网络, 特征图信息, 多层级特征融合, 视差

Abstract: The stereo matching method based on Convolutional Neural Network(CNN) does not make full use of the feature map information of each level in the image,resulting in the poor feature extraction performance in the ill posed region in the image.This paper proposes a stereo matching method based on multi-scale and multi-level features.A pooled pyramid layer is designed at the front end of the CNN model with double tower structure to extract the multi-scale low-level structural features of the image.In the back end of the network model,the high-level semantic features of the last three layers of network are fused to extract image features,and the disparity map is output after similarity measurement of image features.The experimental results on KITTI 2015 dataset show that compared with the LUO and Anita methods,the proposed method reduces the pixel error accuracy from 14.65% and 8.30% to 8.02%,and can obtain a disparity map with better detail information.

Key words: stereo matching, Convolutional Neural Network(CNN), feature map information, multi-level features fusion, disparity

中图分类号:

TP391.41

王金鹤, 车志龙, 张楠, 孟凡云, 苏翠丽, 谭浩. 基于多尺度和多层级特征融合的立体匹配方法[J]. 计算机工程, 2021, 47(3): 243-248.

WANG Jinhe, CHE Zhilong, ZHANG Nan, MENG Fanyun, SU Cuili, TAN Hao. Stereo Matching Method Based on Multi-Scale and Multi-Level Features Fusion[J]. Computer Engineering, 2021, 47(3): 243-248.

https://www.ecice06.com/CN/Y2021/V47/I3/243

图/表 8

20210322173003

20210322173006

20210322173009

20210322173013

20210322173016

20210322173020

20210322173024

20210322173028

参考文献

[1] ZBONTAR J,LECUN Y.Computing the stereo matching cost with a convolutional neural network[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1592-1599.
[2] CHEN Tuo.Research on stereo matching technology based on convolutional neural network[D].Hangzhou:Zhejiang University,2017.(in Chinese)陈拓.基于卷积神经网络的立体匹配技术研究[D].杭州:浙江大学,2017.
[3] WANG Jieqiong.Study on stereo matching of binocular stereo vision[D].Kunming:Kunming University of Science and Technology,2016.(in Chinese)王杰琼.双目立体视觉匹配方法研究[D].昆明:昆明理工大学,2016.
[4] LUO W J,SCHWING A G,URTASUN R.Efficient deep learning for stereo matching[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:5695-5703.
[5] MAYER N,ⅡG E,HAUSSER P,et al.A large dataset to train convolutional networks for disparity,optical flow,and scene flow estimation[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:4040-4048.
[6] HAN N X,LEUNG T,JIA Y,et al.MatchNet:unifying feature and metric learning for patch-based matching[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:3279-3286.
[7] ZAGORUYKO S,KOMODAKIS N.Learning to compare image patches via vonvolutional neural networks[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:4353-4361.
[8] SIMO-SERRA E,TRULLS E,FERRAZ L,et al.Discriminative learning of deep convolutional feature point descriptors[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:118-126.
[9] PARK H,LEE K M.Look wider to match image patches with convolutional neural networks[J].IEEE Signal Processing Letters,2017,99:1-3.
[10] KENDALL A,MARTIROSYAN H,DASGUPTA S,et al.End-to-end learning of geometry and context for deep stereo regression[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:66-75.
[11] PANG J H,SUN W X,RRN J,et al.Cascade residual learning:a two-stage convolutional neural network for stereo matching[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:887-895.
[12] LIANG Zhengfa,FENG Yiliu,GUO Yulan,et al.Learning for disparity estimation through feature constancy[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:2811-2820.
[13] SHAKED A,WOLF L.Improved stereo matching with constant highway networks and reflective confidence learning[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:4641-4650.
[14] CHANG Jiaren,CHEN Yongsheng.Pyramid stereo matching network[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:5410-5418.
[15] MA Wei,LI Tong,GONG Chaofan,et al.Stereo matching with CNN and constraints from segmentation[J].Journal of Beijing University of Technology,2019,45(5):413-420.(in Chinese)马伟,李曈,龚超凡,等.结合CNN与分割约束的立体匹配算法[J].北京工业大学学报,2019,45(5):413-420.
[16] YU Lidong,WANG Yucheng,WU Yuwei,et al.Deep stereo matching with explicit cost aggregation sub-architecture[EB/OL].[2019-10-15].https://arxiv.org/pdf/1801.04065.pdf.
[17] SONG Xiao,ZHAO Xu,HU Hanwen,et al.EdgeStereo:a context integrated residual pyramid network for stereo matching[EB/OL].[2019-10-15].https://arxiv.org/pdf/1803.05196.pdf.
[18] YANG Guorun,ZHAO Hengshuang,SHI Jiaping,et al.SegStereo:exploiting semantic information for disparity estimation[C]//Proceedings of 2018 European Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2018:636-651.
[19] JIE Zequn,WANG Pengfei,LING Yonggen,et al.Left-right comparative recurrent model for stereo matching[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:3838-3846.
[20] ANITA R.Learing similarity for 3D reconstruction of intraoperative environment with convolutional neural networks[D].London,UK:University College London,2017.

选择文件类型/文献管理软件名称

选择包含的内容

基于多尺度和多层级特征融合的立体匹配方法

Stereo Matching Method Based on Multi-Scale and Multi-Level Features Fusion

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	王志浩, 钱沄涛. 基于Swin Transformer的双流遥感图像时空融合超分辨率重建[J]. 计算机工程, 2024, 50(9): 33-45.
[2]	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究[J]. 计算机工程, 2024, 50(9): 82-91.
[3]	张鲁, 田春伟, 宋焕生, 刘侍刚. 用于低剂量CT图像去噪的多级双树复小波网络[J]. 计算机工程, 2024, 50(9): 266-275.
[4]	高煜宝, 文志诚. 基于注意力机制的双路解码器图像去噪方法[J]. 计算机工程, 2024, 50(9): 324-332.
[5]	王蕾, 党时鹏, 潘丰. 基于卷积神经网络的隐匿性旁路预测模型[J]. 计算机工程, 2024, 50(8): 40-49.
[6]	耿丽丽, 牛保宁. 基于通道相似度熵的卷积神经网络裁剪[J]. 计算机工程, 2024, 50(7): 133-143.
[7]	张洋, 刘畅, 李少青. 基于可控制性度量的图神经网络门级硬件木马检测方法[J]. 计算机工程, 2024, 50(7): 164-173.
[8]	牛瑞婷, 严天峰, 高锐, 王映植. 低信噪比下基于深度学习TCNN-MobileNet的调制识别[J]. 计算机工程, 2024, 50(7): 204-215.
[9]	张溢文, 蔡满春, 陈咏豪, 朱懿, 姚利峰. 融合空间特征的多尺度深度伪造检测方法[J]. 计算机工程, 2024, 50(7): 240-250.
[10]	逯焕宇, 张永宏, 马光义, 谢东林, 田伟. 基于半监督对抗学习的遥感图像水体提取[J]. 计算机工程, 2024, 50(7): 251-263.
[11]	于洋, 孙芳芳, 吕华, 李扬, 王晓民. 基于多尺度时空注意力网络的微表情检测方法[J]. 计算机工程, 2024, 50(6): 228-235.
[12]	张雷, 沈国琛, 欧冬秀. 用于热成像数据的卷积神经网络特征图筛选方法[J]. 计算机工程, 2024, 50(4): 31-40.
[13]	张雷, 沈国琛, 欧冬秀. 用于热成像数据的卷积神经网络特征图筛选方法[J]. 计算机工程, 2024, 50(4): 31-40.
[14]	李政学, 李枝名, 彭德中, 陈杰. 基于特征对比学习和图卷积的社交网络用户分类[J]. 计算机工程, 2024, 50(4): 258-266.
[15]	姜百浩, 刘静, 仇大伟, 姜良. 深度学习在脊柱图像分割中的应用综述[J]. 计算机工程, 2024, 50(3): 1-15.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于多尺度和多层级特征融合的立体匹配方法

Stereo Matching Method Based on Multi-Scale and Multi-Level Features Fusion

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献

相关文章 15

编辑推荐

Metrics

本文评价