Lightweight Depth Estimation Based on Multi-scale Fusion and Cross-dimensional Interaction

doi:10.19678/j.issn.1000-3428.0252094

Abstract

Abstract: In recent years, self-supervised monocular depth estimation methods have shown significant potential in fields such as autonomous driving, virtual reality, and robotics. However, the existing lightweight approaches often struggle with feature representation because of the limitations imposed by smaller model sizes, leading to issues such as missing scene objects and blurred edge details. To address these challenges, this study proposes a novel lightweight self-supervised monocular depth estimation method. Specifically, this study introduces a Joint Attention-guided Multiscale spatial Feature Fusion (JAMFF) skip connection strategy that effectively bridges the gap in detail and semantics between the encoder and decoder. Additionally, this study employs a Lightweight Residual Cross-dimensional information Interaction module (LRCI) at all encoder—decoder stages. On the one hand, LRCI ensures that the relationships between spatial and channel are fully considered to protect and enhance depth features for small and weak objects or parts of them; on the other hand, the design of cross-dimensional information interaction is a crucial aspect in ensuring the lightweight nature of the model. The experimental results on the KITTI dataset show that the proposed method achieves an accuracy of 0.889, with Absolute Relative Error (ARE), Square Relative Error (SRE), Root Mean Square Error (RMSE), and Root Mean Squared Log Error (RMSLE) of 0.106, 0.789, 4.581, and 0.182, respectively. Additionally, the model parameter size is only 3.3×10⁶, which generates more accurate depth maps while maintaining model lightweighting. The test results on other datasets with complex scenarios and harsh conditions indicate that the algorithm has strong generalization ability.

Key words: depth estimation, lightweight, joint attention, multi-scale feature fusion, cross-dimensional interaction

摘要： 近年来，深度估计方法的轻量化设计使其可部署于资源受限设备，成为当下研究热点。然而现有轻量化单目深度估计模型受规模限制，特征表征能力存在瓶颈，易出现场景对象缺失、边缘细节模糊等问题。为此，提出一种轻量级自监督单目深度估计方法。设计一种基于联合注意力引导的多尺度空间特征融合机制(JAMFF)的跳跃连接方式，缩小编解码器之间的细节和语义差距。同时，将轻量级残差跨维度信息交互模块(LRCI)应用在所有编解码器阶段，一方面通过提取不同维度特征间的相关性提升微小目标深度区分度，另一方面通过跨维度信息交互设计保持模型轻量化。在KITTI数据集上的实验结果表明，所提方法在δ<1.25情况下准确率达到了 0.889，绝对相对误差（ARE）、平方相对误差(SRE)、均方根误差(RMSE)和对数均方根误差(RMSLE)分别为 0.106、0.789、4.581和0.182，同时模型参数量只有3.3×10⁶，在保持模型轻量化的同时生成更加准确的深度图。在其他大量具有复杂场景和恶劣条件的数据集上的测试结果表明，该算法也具备较强的泛化性。

关键词: 深度估计, 轻量级, 联合注意力, 多尺度特征融合, 跨维度交互

CLC Number:

TP391

WEN Jing, MA Haojiang, LI Minzhi. Lightweight Depth Estimation Based on Multi-scale Fusion and Cross-dimensional Interaction[J]. Computer Engineering, 2026, 52(11): 204-218.

温静, 马皓江, 李敏智. 基于多尺度融合与跨维度交互的轻量级深度估计[J]. 计算机工程, 2026, 52(11): 204-218.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0252094

https://www.ecice06.com/EN/Y2026/V52/I11/204

References

[1] GARG R, BG V K, CARNEIRO G, et al. Unsupervised CNN for single view depth estimation: geometry to the rescue[C]//Proceedings of the 14th European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 740-756.
[2] ZHOU T H, BROWN M, SNAVELY N, et al. Unsupervised learning of depth and ego-motion from video[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE Press, 2017: 6612-6619.
[3] GODARD C, MAC A O, BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE Press, 2017: 6602-6611.
[4] GODARD C, MAC A O, FIRMAN M, et al. Digging into self-supervised monocular depth estimation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Republic of Korea: IEEE Press, 2019: 3827-3837.
[5] LIU L N, SONG X B, WANG M M, et al. Self-supervised monocular depth estimation for all day images using domain separation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada: IEEE Press, 2022: 12717-12726.
[6] 温静, 杨洁. 基于场景对象注意与深度图融合的深度估计[J]. 计算机工程, 2023, 49(2): 222-230. WEN J, YANG J. Depth estimation based on scene object attention and depth map fusion[J]. Computer Engineering, 2023, 49(2): 222-230. (in Chinese)
[7] 宋霄罡, 胡浩越, 宁靖宇, 等. 联合语义分割的自监督单目深度估计方法[J]. 计算机研究与发展, 2024, 61(5): 1336-1347. SONG X G, HU H Y, NING J Y, et al. Self-supervised monocular depth estimation method for joint semantic segmentation[J]. Journal of Computer Research and Development, 2024, 61(5): 1336-1347. (in Chinese)
[8] 周大可, 田径, 杨欣. 结合局部平面参数预测的无监督单目图像深度估计[J]. 中国图象图形学报, 2021, 26(1): 165-175. ZHOU D K, TIAN J, YANG X. Unsupervised monocular image depth estimation based on the prediction of local plane parameters[J]. Journal of Image and Graphics, 2021, 26(1): 165-175. (in Chinese)
[9] YIN Z C, SHI J P. GeoNet: unsupervised learning of dense depth, optical flow and camera pose[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, USA: IEEE Press, 2018: 1983-1992.
[10] 王亚群, 戴华林, 王丽, 等. 基于密集卷积网络的单目图像深度估计方法[J]. 计算机工程, 2021, 47(11): 262-267, 291. WANG Y Q, DAI H L, WANG L, et al. Method for estimating monocular image depth based on dense convolutional network[J]. Computer Engineering, 2021, 47(11): 262-267, 291. (in Chinese)
[11] YAN J X, ZHAO H, BU P H, et al. Channel-wise attention-based network for self-supervised monocular depth estimation[C]//Proceedings of the International Conference on 3D Vision (3DV). London, UK: IEEE Press, 2022: 464-473.
[12] LÜ X Y, LIU L, WANG M M, et al. HR-Depth: high resolution self-supervised monocular depth estimation[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2021: 2294-2301.
[13] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: transformers for image recognition at scale[EB/OL]. [2025-01-02]. https://arxiv.org/abs/2010.11929.
[14] LEE Y, KIM J, WILLETTE J, et al. MPViT: multi-path vision transformer for dense prediction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE Press, 2022: 7277-7286.
[15] DONG X S, GARRATT M A, ANAVATTI S G, et al. Lightweight monocular depth estimation with an edge guided network[C]//Proceedings of the 17th International Conference on Control, Automation, Robotics and Vision (ICARCV). Singapore: IEEE Press, 2023: 204-210.
[16] ZHOU Z K, FAN X N, SHI P F, et al. R-MSFM: recurrent multi-scale feature modulation for monocular depth estimating[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada: IEEE Press, 2022: 12757-12766.
[17] ZHANG N, NEX F, VOSSELMAN G, et al. Lite-mono: a lightweight CNN and transformer architecture for self-supervised monocular depth estimation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada: IEEE Press, 2023: 18537-18546.
[18] SONG L N, SHI D X, XIA J Q, et al. Spatial-aware dynamic lightweight self-supervised monocular depth estimation[J]. IEEE Robotics and Automation Letters, 2024, 9(1): 883-890.
[19] ZHOU Z W, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++: a nested U-Net architecture for medical image segmentation[M]//STOYANOV D, TAYLOR Z, CARNEIRO G, et al. Deep learning in medical image analysis and multimodal learning for clinical decision support. Berlin, Germany: Springer, 2018: 3-11.
[20] LI Y, YAN B B, HOU J X, et al. UNet based on dynamic convolution decomposition and triplet attention[J]. Scientific Reports, 2024, 14(1): 271.
[21] ZHANG J W, ZHANG Y C, JIN Y Z, et al. MDU-Net: multi-scale densely connected U-Net for biomedical image segmentation[J]. Health Information Science and Systems, 2023, 11(1): 13.
[22] WANG H N, CAO P, LIU X L, et al. Narrowing the semantic gaps in U-Net with learnable skip connections: The case of medical image segmentation[EB/OL]. [2025-01-02]. https://arxiv.org/abs/2312.15182.
[23] PENG Y P, SONKA M, CHEN D Z. U-Net v2: rethinking the skip connections of U-Net for medical image segmentation[EB/OL]. [2025-01-02]. https://arxiv.org/abs/2311.17791.
[24] MISRA D, NALAMADA T, ARASANIPALAI A U, et al. Rotate to attend: convolutional triplet attention module[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE Press, 2021: 3138-3147.
[25] GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics: the KITTI dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237.
[26] SAXENA A, SUN M, NG A Y. Make3D: learning 3D scene structure from a single still image[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(5): 824-840.
[27] YANG G R, SONG X, HUANG C Q, et al. DrivingStereo: a large-scale dataset for stereo matching in autonomous driving scenarios[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA: IEEE Press, 2020: 899-908.
[28] SAKARIDIS C, DAI D X, VAN GOOL L. Semantic foggy scene understanding with synthetic data[J]. International Journal of Computer Vision, 2018, 126(9): 973-992.
[29] CAESAR H, BANKITI V, LANG A H, et al. nuScenes: a multimodal dataset for autonomous driving[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE Press, 2020: 11618-11628.
[30] DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE Press, 2009: 248-255.
[31] LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization[EB/OL]. [2025-01-02]. https://arxiv.org/abs/1711.05101.
[32] EIGEN D, FERGUS R. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture[C]//Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE Press, 2016: 2650-2658.
[33] BAE J, MOON S, IM S. Deep digging into the generalization of self-supervised monocular depth estimation[EB/OL]. [2025-01-02]. https://arxiv.org/abs/2205.11083.
[34] KLINGNER M, TERMÖHLEN J A, MIKOLAJCZYK J, et al. Self-supervised monocular depth estimation: solving the dynamic object problem by semantic guidance[C]//Proceedings of the 16th European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 582-600.
[35] JOHNSTON A, CARNEIRO G. Self-supervised monocular trained depth estimation using self-attention and discrete disparity volume[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE Press, 2020: 4755-4764.
[36] CORDTS M, OMRAN M, RAMOS S, et al. The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the 34th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, USA: IEEE Press, 2016: 3213-3223.
[37] ZHOU H, GREENWOOD D, TAYLOR S. Self-supervised monocular depth estimation with internal feature fusion[EB/OL]. [2025-01-02]. https://arxiv.org/abs/2110.09482.
[38] WANG K, ZHANG Z Y, YAN Z Q, et al. Regularizing nighttime weirdness: efficient self-supervised monocular depth estimation in the dark[C]//Proceedings of the 18th IEEE International Conference on Computer Vision. Piscataway, USA: IEEE Press, 2021: 16055-16064.

Please choose a citation manager

Content to export