基于多尺度和多层级特征融合的立体匹配方法

doi:10.19678/j.issn.1000-3428.0056715

计算机工程 ›› 2021, Vol. 47 ›› Issue (3): 243-248. doi: 10.19678/j.issn.1000-3428.0056715

基于多尺度和多层级特征融合的立体匹配方法

王金鹤, 车志龙, 张楠, 孟凡云, 苏翠丽, 谭浩

青岛理工大学信息与控制工程学院, 山东青岛 266000

收稿日期:2019-11-26 修回日期:2020-03-08 发布日期:2020-06-18
作者简介:王金鹤(1963-),男,教授、博士,主研方向为模式识别与智能系统;车志龙,硕士研究生;张楠(通信作者),讲师、硕士;孟凡云,讲师、博士;苏翠丽、谭浩,硕士研究生。
基金资助:
国家自然科学基金（31271077）；山东省重点研发项目（2019GGX104089）；山东省高等学校科技计划项目（J17KA061）。

Stereo Matching Method Based on Multi-Scale and Multi-Level Features Fusion

WANG Jinhe, CHE Zhilong, ZHANG Nan, MENG Fanyun, SU Cuili, TAN Hao

School of Information and Control Engineering, Qingdao University of Technology, Qingdao, Shandong 266000, China

Received:2019-11-26 Revised:2020-03-08 Published:2020-06-18

摘要/Abstract

摘要： 基于卷积神经网络的立体匹配方法未充分利用图像中各个层级的特征图信息，造成对图像在不适定区域的特征提取能力较差。提出一种融合多尺度与多层级特征的立体匹配方法。通过在双塔结构卷积神经网络模型的前端设计一个池化金字塔层，提取图像的多尺度低层结构特征。在该网络模型的后端融合最后三层网络的高级语义特征来提取图像特征，并对图像特征进行相似性度量后输出视差图。在KITTI 2015数据集上的实验结果表明，与LUO和Anita方法相比，该方法的像素误差精度分别由14.65%、8.30%降至8.02%，且可得到细节信息更好的视差图。

关键词: 立体匹配, 卷积神经网络, 特征图信息, 多层级特征融合, 视差

Abstract: The stereo matching method based on Convolutional Neural Network(CNN) does not make full use of the feature map information of each level in the image,resulting in the poor feature extraction performance in the ill posed region in the image.This paper proposes a stereo matching method based on multi-scale and multi-level features.A pooled pyramid layer is designed at the front end of the CNN model with double tower structure to extract the multi-scale low-level structural features of the image.In the back end of the network model,the high-level semantic features of the last three layers of network are fused to extract image features,and the disparity map is output after similarity measurement of image features.The experimental results on KITTI 2015 dataset show that compared with the LUO and Anita methods,the proposed method reduces the pixel error accuracy from 14.65% and 8.30% to 8.02%,and can obtain a disparity map with better detail information.

Key words: stereo matching, Convolutional Neural Network(CNN), feature map information, multi-level features fusion, disparity

中图分类号:

TP391.41

王金鹤, 车志龙, 张楠, 孟凡云, 苏翠丽, 谭浩. 基于多尺度和多层级特征融合的立体匹配方法[J]. 计算机工程, 2021, 47(3): 243-248.

WANG Jinhe, CHE Zhilong, ZHANG Nan, MENG Fanyun, SU Cuili, TAN Hao. Stereo Matching Method Based on Multi-Scale and Multi-Level Features Fusion[J]. Computer Engineering, 2021, 47(3): 243-248.

http://www.ecice06.com/CN/Y2021/V47/I3/243

图/表 8

20210322173003

20210322173006

20210322173009

20210322173013

20210322173016

20210322173020

20210322173024

20210322173028

参考文献

[1] ZBONTAR J,LECUN Y.Computing the stereo matching cost with a convolutional neural network[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1592-1599.
[2] CHEN Tuo.Research on stereo matching technology based on convolutional neural network[D].Hangzhou:Zhejiang University,2017.(in Chinese)陈拓.基于卷积神经网络的立体匹配技术研究[D].杭州:浙江大学,2017.
[3] WANG Jieqiong.Study on stereo matching of binocular stereo vision[D].Kunming:Kunming University of Science and Technology,2016.(in Chinese)王杰琼.双目立体视觉匹配方法研究[D].昆明:昆明理工大学,2016.
[4] LUO W J,SCHWING A G,URTASUN R.Efficient deep learning for stereo matching[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:5695-5703.
[5] MAYER N,ⅡG E,HAUSSER P,et al.A large dataset to train convolutional networks for disparity,optical flow,and scene flow estimation[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:4040-4048.
[6] HAN N X,LEUNG T,JIA Y,et al.MatchNet:unifying feature and metric learning for patch-based matching[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:3279-3286.
[7] ZAGORUYKO S,KOMODAKIS N.Learning to compare image patches via vonvolutional neural networks[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:4353-4361.
[8] SIMO-SERRA E,TRULLS E,FERRAZ L,et al.Discriminative learning of deep convolutional feature point descriptors[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:118-126.
[9] PARK H,LEE K M.Look wider to match image patches with convolutional neural networks[J].IEEE Signal Processing Letters,2017,99:1-3.
[10] KENDALL A,MARTIROSYAN H,DASGUPTA S,et al.End-to-end learning of geometry and context for deep stereo regression[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:66-75.
[11] PANG J H,SUN W X,RRN J,et al.Cascade residual learning:a two-stage convolutional neural network for stereo matching[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:887-895.
[12] LIANG Zhengfa,FENG Yiliu,GUO Yulan,et al.Learning for disparity estimation through feature constancy[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:2811-2820.
[13] SHAKED A,WOLF L.Improved stereo matching with constant highway networks and reflective confidence learning[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:4641-4650.
[14] CHANG Jiaren,CHEN Yongsheng.Pyramid stereo matching network[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:5410-5418.
[15] MA Wei,LI Tong,GONG Chaofan,et al.Stereo matching with CNN and constraints from segmentation[J].Journal of Beijing University of Technology,2019,45(5):413-420.(in Chinese)马伟,李曈,龚超凡,等.结合CNN与分割约束的立体匹配算法[J].北京工业大学学报,2019,45(5):413-420.
[16] YU Lidong,WANG Yucheng,WU Yuwei,et al.Deep stereo matching with explicit cost aggregation sub-architecture[EB/OL].[2019-10-15].https://arxiv.org/pdf/1801.04065.pdf.
[17] SONG Xiao,ZHAO Xu,HU Hanwen,et al.EdgeStereo:a context integrated residual pyramid network for stereo matching[EB/OL].[2019-10-15].https://arxiv.org/pdf/1803.05196.pdf.
[18] YANG Guorun,ZHAO Hengshuang,SHI Jiaping,et al.SegStereo:exploiting semantic information for disparity estimation[C]//Proceedings of 2018 European Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2018:636-651.
[19] JIE Zequn,WANG Pengfei,LING Yonggen,et al.Left-right comparative recurrent model for stereo matching[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:3838-3846.
[20] ANITA R.Learing similarity for 3D reconstruction of intraoperative environment with convolutional neural networks[D].London,UK:University College London,2017.

选择文件类型/文献管理软件名称

选择包含的内容

基于多尺度和多层级特征融合的立体匹配方法

Stereo Matching Method Based on Multi-Scale and Multi-Level Features Fusion

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	刘志浩, 孟凡云, 王金鹤, 张楠. 基于空洞卷积与注意力模块的立体匹配算法[J]. 计算机工程, 2023, 49(8): 223-231.
[2]	曹坪, 杨怀志, 薄一军, 尤嘉, 张淳杰, 李丹勇. 面向低质量裂缝图像的多知识蒸馏分类[J]. 计算机工程, 2023, 49(7): 204-213.
[3]	白明昌. 基于折叠路径聚合的属性网络节点嵌入方法[J]. 计算机工程, 2023, 49(7): 76-84.
[4]	侯华, 郭宏洋, 代超娜, 李峻辉. 结合多重注意力与迭代优化的立体匹配算法[J]. 计算机工程, 2023, 49(7): 161-168.
[5]	代祖华, 刘园园, 狄世龙. 语义增强的图神经网络方面级文本情感分析[J]. 计算机工程, 2023, 49(6): 71-80.
[6]	沈学利, 田桂源, 姜彦吉, 马琳琳. 基于双阶段Conv-Transformer的时频域语音增强算法[J]. 计算机工程, 2023, 49(6): 123-130.
[7]	丁子轩, 俞雷, 张娟, 李想, 王新宇. 基于深度残差自适应注意力网络的图像超分辨率重建[J]. 计算机工程, 2023, 49(5): 231-238.
[8]	陈治旭, 靳雁霞, 芦烨, 杨晶, 刘亚变, 史志儒. 基于子图卷积神经网络的多精度服装建模方法[J]. 计算机工程, 2023, 49(4): 174-181.
[9]	徐康, 李霏, 姬东鸿. 结合依存图卷积与文本片段搜索的方面情感三元组抽取[J]. 计算机工程, 2023, 49(4): 61-67.
[10]	衡红军, 苗菁. 语义与句法信息加强的二元标记实体关系联合抽取[J]. 计算机工程, 2023, 49(4): 77-84.
[11]	钟宝荣, 吴夏灵. 基于高分辨率网络的轻量型人体姿态估计研究[J]. 计算机工程, 2023, 49(4): 226-232,239.
[12]	杨晶晶, 谢海燕, 薛妮妮, 张傲明. 基于双通道残差网络的水下图像去噪研究[J]. 计算机工程, 2023, 49(4): 188-198.
[13]	刘晶晶, 黄浩. 引入非局部模块卷积神经网络的基频提取模型[J]. 计算机工程, 2023, 49(3): 128-133,160.
[14]	邹长龙, 安敬民, 李冠宇. 基于邻域聚合与CNN的知识图谱实体类型补全[J]. 计算机工程, 2023, 49(3): 134-141.
[15]	翟社平, 张宇航, 柏晓夏. 融合实体邻域信息的知识图谱嵌入负采样方法[J]. 计算机工程, 2023, 49(3): 95-104.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于多尺度和多层级特征融合的立体匹配方法

Stereo Matching Method Based on Multi-Scale and Multi-Level Features Fusion

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献

相关文章 15

编辑推荐

Metrics

本文评价