基于场景对象注意与深度图融合的深度估计

doi:10.19678/j.issn.1000-3428.0064268

摘要/Abstract

摘要： 现有单目深度估计算法主要从单幅图像中获取立体信息，存在相邻深度边缘细节模糊、明显的对象缺失问题。提出一种基于场景对象注意机制与加权深度图融合的单目深度估计算法。通过特征矩阵相乘的方式计算特征图任意两个位置之间的相似特征向量，以快速捕获长距离依赖关系，增强用于估计相似深度区域的上下文信息，从而解决自然场景中对象深度信息不完整的问题。基于多尺度特征图融合的优点，设计加权深度图融合模块，为具有不同深度信息的多视觉粒度的深度图赋予不同的权值并进行融合，融合后的深度图包含深度信息和丰富的场景对象信息，有效地解决细节模糊问题。在KITTI数据集上的实验结果表明，该算法对目标图像预估时σ<1.25的准确率为0.879，绝对相对误差、平方相对误差和对数均方根误差分别为0.110、0.765和0.185，预测得到的深度图具有更加完整的场景对象轮廓和精确的深度信息。

关键词: 场景对象注意, 加权深度图融合, 上下文信息, 深度估计, 三维重建

Abstract: The existing monocular depth estimation algorithm mainly obtains stereo information from a single image.This approach leads to blurred details of adjacent depth edges and apparent missing objects.A monocular depth estimation algorithm based on scene object attention mechanism and weighted depth map fusion is proposed.The similarity feature vector between any two positions of feature map is calculated by multiplying the feature matrix to rapidly capture the long-distance dependency relationship.The dependency between any two positions in the image can enhance the context information used to estimate the similar depth area, thus, solving the incomplete object depth information in the natural scene.Based on the advantages of multi-scale feature map fusion, weighted depth map fusion module is designed.The multi-vision granularity depth map with different depth information data is assigned different weights for fusion.The fused depth map contains depth information and rich-scene object information for effectively solving the problem of fuzzy details.The experimental results on the KITTI dataset show that an accuracy rate of the proposed algorithm for target image prediction is 0.879 at σ<1.25, and the absolute relative error, square relative error, and logarithmic root mean square error are 0.110, 0.765, and 0.185, respectively.The predicted depth map has a complete scene object contour and accurate depth information.

Key words: scene object attention, weighted depth map fusion, context information, depth estimation, three-dimensional reconstruction

中图分类号:

TP391

温静, 杨洁. 基于场景对象注意与深度图融合的深度估计[J]. 计算机工程, 2023, 49(2): 222-230.

WEN Jing, YANG Jie. Depth Estimation Based on Scene Object Attention and Depth Map Fusion[J]. Computer Engineering, 2023, 49(2): 222-230.

http://www.ecice06.com/CN/Y2023/V49/I2/222

图/表 13

20230216181635

20230216181639

20230216181642

20230216181646

20230216181649

20230216181652

20230216181656

20230216181659

20230216181703

20230216181707

20230216181710

20230216181714

20230216181717

参考文献

[1] EIGEN D, PUHRSCH C, FERGUS R.Depth map prediction from a single image using a multi-scale deep network[EB/OL].[2022-02-18].http://de.arxiv.org/pdf/1406.2283.
[2] LIU F Y, SHEN C H, LIN G S.Deep convolutional neural fields for depth estimation from a single image[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2015:5162-5170.
[3] LI B, SHEN C H, DAI Y C, et al.Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2015:1119-1127.
[4] LAINA I, RUPPRECHT C, BELAGIANNIS V, et al.Deeper depth prediction with fully convolutional residual networks[C]//Proceedings of the 4th International Conference on 3D Vision.Washington D.C., USA:IEEE Press, 2016:239-248.
[5] GARG R, VIJAY KUMAR B G, CARNEIRO G, et al.Unsupervised CNN for single view depth estimation:geometry to the rescue[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:740-756.
[6] GODARD C, AODHA O M, FIRMAN M, et al.Digging into self-supervised monocular depth estimation[C]//Proceedings of International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:3827-3837.
[7] GUIZILINI V, AMBRUS R, PILLAI S, et al.3D packing for self-supervised monocular depth estimation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:2482-2491.
[8] 王亚群, 戴华林, 王丽, 等.基于密集卷积网络的单目图像深度估计方法[J].计算机工程, 2021, 47(11):262-267, 291. WANG Y Q, DAI H L, WANG L, et al.Method for estimating monocular image depth based on dense convolutional network[J].Computer Engineering, 2021, 47(11):262-267, 291.(in Chinese)
[9] VIJAYANARASIMHAN S, RICCO S, SCHMID C, et al.SfM-Net:learning of structure and motion from video[EB/OL].[2022-02-18].https://arxiv.org/pdf/1704.07804.pdf.
[10] ALBARQOUNI S, KONRAD U, WANG L C, et al.Single-view X-ray depth recovery:toward a novel concept for image-guided interventions[J].International Journal of Computer Assisted Radiology and Surgery, 2016, 11(6):873-880.
[11] HU J, SHEN L, ALBANIE S, et al.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8):2011-2023.
[12] WOO S, PARK J, LEE J Y, et al.CBAM:convolutional block attention module[EB/OL].[2022-02-18].https://arxiv.org/pdf/1807.06521.pdf.
[13] YANG Z H, WANG P, XU W, et al.Unsupervised learning of geometry with edge-aware depth-normal consistency[EB/OL].[2022-02-18].https://arxiv.org/pdf/1711.03665.pdf.
[14] JOHNSTON A, CARNEIRO G.Self-supervised monocular trained depth estimation using self-attention and discrete disparity volume[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:4755-4764.
[15] LIU S T, HUANG D, WANG Y H.Learning spatial fusion for single-shot object detection[EB/OL].[2022-02-18].https://arxiv.org/abs/1911.09516v1.
[16] HE K M, ZHANG X Y, REN S Q, et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 37(9):1904-1916.
[17] ZHAO H S, SHI J P, QI X J, et al.Pyramid scene parsing network[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:6230-6239.
[18] LIU W, RABINOVICH A, BERG A C.ParseNet:looking wider to see better[EB/OL].[2022-02-18].https://arxiv.org/pdf/1506.04579.pdf.
[19] WU J W, ZHOU W J, LUO T, et al.Multiscale multilevel context and multimodal fusion for RGB-D salient object detection[J].Signal Processing, 2021, 178:107766.
[20] 王子, 吴振宇.SFM算法在无人机航拍影像中的应用研究[J].科技创新与生产力, 2021(8):104-107. WANG Z, WU Z Y.Application of SFM algorithm in UAV aerial photography[J].Sci-Tech Innovation and Productivity, 2021(8):104-107.(in Chinese)
[21] ZHAN H Y, GARG R, WEERASEKERA C S, et al.Unsupervised learning of monocular depth estimation and visual odometry with deep feature reconstruction[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:340-349.
[22] 龙霄潇, 程新景, 朱昊, 等.三维视觉前沿进展[J].中国图象图形学报, 2021, 26(6):1389-1428. LONG X X, CHENG X J, ZHU H, et al.Recent progress in 3D vision[J].Journal of Image and Graphics, 2021, 26(6):1389-1428.(in Chinese)
[23] GEIGER A, LENZ P, STILLER C, et al.Vision meets robotics:the KITTI dataset[J].The International Journal of Robotics Research, 2013, 32(11):1231-1237.
[24] ZHOU T H, BROWN M, SNAVELY N, et al.Unsupervised learning of depth and ego-motion from video[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:6612-6619.
[25] 朱照飞, 刘伟.基于改进的SURF特征点的双目测距[J].电子测量技术, 2018, 41(12):133-138. ZHU Z F, LIU W.Binocular distance measurement based on improved SURF feature points[J].Electronic Measurement Technology, 2018, 41(12):133-138.(in Chinese)
[26] MAHJOURIAN R, WICKE M, ANGELOVA A.Unsupervised learning of depth and ego-motion from monocular video using 3D geometric constraints[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:5667-5675.
[27] YIN Z C, SHI J P.GeoNet:unsupervised learning of dense depth, optical flow and camera pose[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:1983-1992.
[28] ZOU Y L, LUO Z L, HUANG J B.DF-Net:unsupervised joint learning of depth and flow using cross-task consistency[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:38-55.
[29] CASSER V, PIRK S, MAHJOURIAN R, et al.Depth prediction without the sensors:leveraging structure for unsupervised learning from monocular videos[C]//Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.]:AAAI Press, 2019:8001-8008.

选择文件类型/文献管理软件名称

选择包含的内容