HDMapFusion：用于自动驾驶的多模态融合高清地图生成

doi:10.19678/j.issn.1000-3428.0070569

摘要/Abstract

摘要： 高清(HD)环境语义地图的生成是自动驾驶系统中不可或缺的关键技术。针对相机与激光雷达在感知任务中存在的模态差异问题，本文提出了一种创新的多模态融合范式HDMapFusion。与传统的直接融合原始传感器数据方法不同，本方法通过将相机和激光雷达特征统一转化为鸟瞰图(BEV)表示，实现了多模态信息的物理可解释性融合。在nuScenes基准数据集上的实验结果表明，HDMapFusion在HD地图生成精度方面显著优于现有基准模型，其中IoU得分提升了23.0%，充分验证了该方法的有效性和优越性。

Abstract: The generation of high-definition (HD) environmental semantic maps plays a crucial and irreplaceable role in autonomous driving systems. To address the modality discrepancy between cameras and LiDAR in perception tasks, this paper proposes an innovative multi-modal fusion paradigm, HDMapFusion. Unlike traditional methods that directly fuse raw sensor data, this approach achieves physically interpretable fusion of multi-modal information by unifying camera and LiDAR features into a bird's-eye view (BEV) representation. Experimental results on the nuScenes benchmark dataset demonstrate that HDMapFusion significantly outperforms existing baseline models in HD map generation accuracy, with a 23.0% improvement in IoU score, fully validating the effectiveness and superiority of the proposed method.

刘洋宏, 付杨悠然, 董性平. HDMapFusion：用于自动驾驶的多模态融合高清地图生成[J]. 计算机工程, doi: 10.19678/j.issn.1000-3428.0070569.

LIU Yanghong, FUYANG Youran, DONG Xingping. HDMapFusion: HD Map Generation with Multi-Modality Fusion for Autonomous Driving[J]. Computer Engineering, doi: 10.19678/j.issn.1000-3428.0070569.

参考文献

[1] Liao, B., Chen, S., Wang, X., Cheng, T., Zhang, Q., Liu, W., Huang, C.: Maptr: Structured modeling and learning for online vectorized hd map construction[C]//Proceedings of International Conference on Learning Representations (ICLR), 2023.
[2] 赵南南, 高翡晨. 基于改进 YOLOv8 的交通场景实例分割算法[J]. 计算机工程, 2025, 51(1): 198-207. ZHAO Nannan, GAO Feichen. Improved YOLOv8-based Algorithm for Instance Segmentation in Traffic Scenes[J]. Computer Engineering, 2025, 51(1): 198-207.
[3] 秦严严. 交通流分析理论[M]. 人民交通出版社, 2023. Qin Yanyan. Theory of Traffic Flow Analysis[M]. China Communications Press, 2023.
[4] Li, Q., Wang, Y., Wang, Y., Zhao, H.: Hdmapnet: An online hd map construction and evaluation framework[C]//Proceedings of International Conference on Robotics and Automation (ICRA), 2021: 4628-4634.
[5] 阳钧, 鲍泓, 梁军, 马楠. 一种基于高精度地图的路径跟踪方法[J]. 计算机工程, 2018, 44(7): 8-13. YANG Jun, BAO Hong, LIANG Jun, MA Nan. A Path Tracking Method Based on High Precision Map[J]. Computer Engineering, 2018, 44(7): 8-13.
[6] 刘宏纬, 邵东恒, 杨剑, 魏宪, 李科, 游雄. 基于鸟瞰图融合的多级旋转等变目标检测网络[J]. 计算机工程, 2024, 50(11): 246-257. LIU Hongwei, SHAO Dongheng, YANG Jian, WEI Xian, LI Ke, YOU Xiong. Multi-Level Rotational Equivariant Object Detection Network Based on BEV Fusion[J]. Computer Engineering, 2024, 50(11): 246-257.
[7] Liu Z, Tang H, Amini A, et al. Bevfusion: Multi-task multi-sensor fusion with unified bird's-eye view representation[C]//Proceedings of IEEE international conference on robotics and automation (ICRA). IEEE, 2023: 2774-2781.
[8] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Advances in neural information processing systems[J]. Advances in neural information processing systems, 2017, 30.
[9] Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: nuscenes: A multimodal dataset for autonomous driving[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 11618-11628.
[10] Dong H, Gu W, Zhang X, et al. Superfusion: Multilevel lidar-camera fusion for long-range hd map generation[C]//Proceedings of IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024: 9056-9062
[11] Sun, L., Yang, K., Hu, X., Hu, W., Wang, K.: Real-time fusion network for rgb-d semantic segmentation incorporating unexpected obstacle detection for roaddriving images. IEEE Robotics and Automation Letters 5, 2020: 5558-5565.
[12] Hu S, Chen L, Wu P, et al. St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning[C]//Proceedings of European Conference on Computer Vision. 2022: 533-549
[13] Rhinehart, N., McAllister, R., Kitani, K., Levine, S.: Precog: Prediction conditioned on goals in visual multi-agent settings[C]//Proceedings of IEEE/CVF International Conference on Computer Vision (ICCV), 2019: 2821-2830.
[14] Xu, H., Yang, C., Li, Z.: Od-slam: Real-time localization and mapping in dynamic environment through multi-sensor fusion[C]//Proceedings of International Conference on Advanced Robotics and Mechatronics (ICARM), 2020: 172-177.
[15] Chen, J., Li, X., Xie, J., Li, J., Qian, J., Yang, J.: Cbi-gnn: Crossscale bilateral graph neural network for 3d object detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(12), 23124-23135.
[16] Philion J, Fidler S. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d[C]//Proceedings of European Conference on Computer Vision. 2020: 194-210.
[17] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2009: 248-255.
[18] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 770-778
[19] Lang A H, Vora S, Caesar H, et al. Pointpillars: Fast encoders for object detection from point clouds[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019: 12697-12705.
[20] Qi, C., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 77-85.
[21] Man, Y., Gui, L., Wang, Y.X.: CroMA: Cross-modality adaptation for monocular BEV perception[C]//Proceedings of International Conference on Learning Representations (ICLR) (2023)
[22] Radford, A., Narasimhan, K.: Improving language understanding by generative pre-training[C]//Proceedings of International Conference on Learning Representations (ICLR), 2018.
[23] Deng, L., Yang, M., Li, H., Li, T., Hu, B., Wang, C.: Restricted deformable convolution-based road scene semantic segmentation using surround view cameras[J]. IEEE Transactions on Intelligent Transportation Systems 21,2018: 4350-4362.
[24] Pan, B., Sun, J., Leung, H.Y.T., Andonian, A., Zhou, B.: Cross-view semantic segmentation for sensing surroundings[J]. IEEE Robotics and Automation Letters 5, 2019: 4867-4873.
[25] 吴永庆, 姜正宇. 基于解耦合动态时空卷积循环网络的交通流预测 [J]. 计算机工程 , doi: 10.19678/j.issn.1000-3428.0070319. WU Yongqing, JIANG Zhengyu. Traffic Flow Prediction Based on Decoupled Dynamic Spatial-Temporal Convolutional Recurrent Network[J]. Computer Engineering, doi: 10.19678/j.issn.1000-3428.0070319.
[26] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization[C]//Proceedings of International Conference on Learning Representations (ICLR), 2017.
[27] Lin T Y, Maire M, Belongie S, et al. Microsoft coco: Common objects in context[C]//Proceedings of European Conference on Computer Vision. 2014: 740-755.

选择文件类型/文献管理软件名称

选择包含的内容