基于多方向特征融合的室外三维目标检测方法

doi:10.19678/j.issn.1000-3428.0066214

计算机工程 ›› 2023, Vol. 49 ›› Issue (11): 238-246. doi: 10.19678/j.issn.1000-3428.0066214

基于多方向特征融合的室外三维目标检测方法

雷嘉铭¹^,², 俞辉²^,³^,*, 夏羽⁴, 郭杰龙²^,³, 魏宪²^,³

1. 福州大学先进制造学院, 福建泉州 362000
2. 中国科学院海西研究院泉州装备制造研究中心, 福建泉州 362000
3. 中国科学院福建物质结构研究所, 福州 350108
4. 上海宇航系统工程研究所, 上海 200000

收稿日期:2022-11-09 出版日期:2023-11-15 发布日期:2023-11-08
通讯作者: 俞辉
作者简介:
雷嘉铭（1995—），男，硕士研究生，主研方向为三维目标检测、机器视觉
夏羽，工程师、硕士
郭杰龙，工程师、硕士
魏宪，研究员、博士
基金资助:
福建省科技计划项目(2021T3003); 泉州市科技项目(2021C065L)

Outdoor 3D Object Detection Method Based on Multi-Direction Features Fusion

Jiaming LEI¹^,², Hui YU²^,³^,*, Yu XIA⁴, Jielong GUO²^,³, Xian WEI²^,³

1. School of Advanced Manufacturing, Fuzhou University, Quanzhou 362000, Fujian, China
2. Quanzhou Institute of Equipment Manufacturing Haixi Institutes, Chinese Academy of Sciences, Quanzhou 362000, Fujian, China
3. Fujian Institute of Research on the Structure of Matter, Chinese Academy of Sciences, Fuzhou 350108, China
4. Shanghai Institute of Aerospace System Engineering, Shanghai 200000, China

Received:2022-11-09 Online:2023-11-15 Published:2023-11-08
Contact: Hui YU

摘要/Abstract

摘要：

三维目标检测方法是自动驾驶环境感知系统的重要技术之一，但是现有三维目标检测方法大多存在目标位置识别精度不足、目标朝向预测偏差较大的问题。为此，提出一种基于多方向特征融合的三维目标检测方法。对场景点云进行数据编码，建模距离、夹角等信息并转换成伪图像。利用所提多方向特征融合骨干网络进行特征提取和融合，基于多方向融合特征，利用以中心为基准的检测器进行潜在目标的回归和预测。该方法通过点间距离、夹角建模提供点间的相互关系，丰富特征信息，利用多方向特征融合增强骨干网络的特征提取能力，从而提高位置及朝向的预测精度。实验结果表明，该方法在KITTI和nuScenes数据集上的平均精度均值分别为64.28%和50.17%，相比次优方法分别提升了0.36和1.30个百分点，且在朝向预测准确率对比实验中均取得了最好的平均朝向相似度和平均朝向误差。此外，泛化性实验结果验证了所提多方向特征融合骨干网络能提高网络检测能力并大幅减少参数量，从而提升检测方法的应用表现。

关键词: 机器视觉, 激光雷达, 三维目标检测, 自动驾驶, 点云

Abstract:

The 3D object detection method is one of the significant technologies in the environmental perception of autonomous driving. However, most existing 3D object detection methods have the problem of inaccurate position recognition and large orientation deviation. To address these issues, a 3D object detection method based on multi-direction features fusion is proposed. First, to perform data encoding for a point cloud scenario, modeling distance and angle and transforming into pseudo image. Second, a multi-direction feature-fusion backbone is proposed for features extraction and fusion. Finally, based on the fused features, a center-based detector regresses and predicts potential objects. Distance-angle modeling can supply the relationship between points and enrich features. The multi-direction feature-fusion backbone enhances the ability of features extraction and improves the accuracy of position and orientation estimation. The experimental results show that the mean Average Precision(mAP)of this method on the KITTI and nuScenes datasets was 64.28% and 50.17%, respectively, which is an improvement of 0.36 and 1.30 percentage points, respectively, compared to those of the suboptimal method. In addition, the best Average Orientation Similarity(AOS)and mean Average Orientation Error(mAOE)were achieved in the orientation prediction accuracy comparison experiments. The generalization experimental results verified that the proposed multi-direction feature-fusion backbone network can improve network detection ability and significantly reduce the number of parameters, thereby improving the application performance of the detection method.

Key words: machine vision, lidar, 3D object detection, autonomous driving, point cloud

雷嘉铭, 俞辉, 夏羽, 郭杰龙, 魏宪. 基于多方向特征融合的室外三维目标检测方法[J]. 计算机工程, 2023, 49(11): 238-246.

Jiaming LEI, Hui YU, Yu XIA, Jielong GUO, Xian WEI. Outdoor 3D Object Detection Method Based on Multi-Direction Features Fusion[J]. Computer Engineering, 2023, 49(11): 238-246.

http://www.ecice06.com/CN/Y2023/V49/I11/238

图/表 17

图1 三维目标检测方法整体框架

Fig.1 Overall framework of 3D object detection method

图2 点间距离、夹角建模的点云数据编码

Fig.2 Point cloud data encoding for modeling distance and angle between points

图3 空间提升层

Fig.3 Space lifting layer

图4 多方向卷积层

Fig.4 Multi-direction convolution layer

图5 多方向特征融合骨干网络

Fig.5 Backbone network for multi-direction features fusion

图6 可视化结果

Fig.6 Visualization results

参考文献 29

1	刘丹, 马世霞. 融合超像素3D与Appearance特征的可行驶区域检测. 计算机工程, 2017, 43(7): 293- 297. URL
	LIU D, MA S X. Travelable area detection fusing superpixel 3D and Apperance feature. Computer Engineering, 2017, 43(7): 293- 297. URL
2	张银, 任国全, 程子阳, 等. 三维激光雷达在无人车环境感知中的应用研究. 激光与光电子学进展, 2019, 56(13): 1- 11. URL
	ZHANG Y, REN G Q, CHENG Z Y, et al. Application research of there-dimensional lidar in unmanned vehicle environment perception. Laser & Optoelectronics Progress, 2019, 56(13): 1- 11. URL
3	LI H, FU K, YAN M L, et al. Vehicle detection in remote sensing images using denoizing-based convolutional neural networks. Remote Sensing Letters, 2017, 8(3): 262- 270. doi: 10.1080/2150704X.2016.1258127
4	李朝, 兰海, 魏宪. 基于注意力的毫米波-激光雷达融合目标检测. 计算机应用, 2021, 41(7): 2137- 2144. URL
	LI C, LAN H, WEI X. Attention-based object detection with millimeter wave radar-lidar fusion. Journal of Computer Applications, 2021, 41(7): 2137- 2144. URL
5	王海, 刘明亮, 蔡英凤, 等. 基于激光雷达与毫米波雷达融合的车辆目标检测算法. 江苏大学学报(自然科学版), 2021, 42(4): 389- 394. URL
	WANG H, LIU M L, CAI Y F, et al. Vehicle target detection algorithm based on fusion of lidar and millimeter wave radar. Journal of Jiangsu University (Natural Science Edition), 2021, 42(4): 389- 394. URL
6	赵亮, 胡杰, 刘汉, 等. 基于语义分割的深度学习激光点云三维目标检测. 中国激光, 2021, 48(17): 171- 183. URL
	ZHAO L, HU J, LIU H, et al. Deep learning based on semantic segmentation for three-dimensional object detection from point clouds. Chinese Journal of Lasers, 2021, 48(17): 171- 183. URL
7	ZHOU Y, TUZEL O. VoxelNet: end-to-end learning for point cloud based 3D object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 4490-4499.
8	YAN Y, MAO Y X, LI B. SECOND: sparsely embedded convolutional detection. Sensors, 2018, 18(10): 3337.
9	LANG A H, VORA S, CAESAR H, et al. PointPillars: fast encoders for object detection from point clouds[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 12689-12697.
10	张冬冬, 郭杰, 陈阳. 基于深度学习的三维目标检测方法研究综述. 机电工程技术, 2022, 51(4): 5- 11. URL
	ZHANG D D, GUO J, CHEN Y. Review on deep learning to 3D object detection methods. Mechanical & Electrical Engineering Technology, 2022, 51(4): 5- 11. URL
11	CHARLES R Q, HAO S, MO K C, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 77-85.
12	QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Washington D. C., USA: IEEE Press, 2017: 5105-5114.
13	GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics: the KITTI dataset. International Journal of Robotics Research, 2013, 32(11): 1231- 1237.
14	GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving?The KITTI vision benchmark suite[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2012: 3354-3361.
15	YIN T W, ZHOU X Y, KRÄHENBÜHL P. Center-based 3D object detection and tracking[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 11779-11788.
16	CAESAR H, BANKITI V, LANG A H, et al. nuScenes: a multimodal dataset for autonomous driving[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 11618-11628.
17	SUN P, KRETZSCHMAR H, DOTIWALLA X, et al. Scalability in perception for autonomous driving: waymo open dataset[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 2443-2451.
18	ZHU X G, MA Y X, WANG T, et al. SSN: shape signature networks for multi-class object detection from point clouds[EB/OL]. [2022-10-05]. https://arxiv.org/abs/2004.02774.
19	何卓逊. 图像识别中数据增强方法的理解与改进[D]. 上海: 上海交通大学, 2020.
	HE Z X. Understanding and improvement of data enhancement method in image recognition[D]. Shanghai: Shanghai Jiao Tong University, 2020. (in Chinese)
20	杨川. 基于点云数据的3D目标检测技术研究[D]. 成都: 电子科技大学, 2021.
	YANG C. Research on 3D object detection technology based on point cloud data[D]. Chengdu: University of Electronic Science and Technology of China, 2021. (in Chinese)
21	IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning. New York, USA: ACM Press, 2015: 448-456.
22	COHEN T S, WELLING M. Group equivariant convolutional networks[EB/OL]. [2022-10-05]. https://arxiv.org/abs/1602.07576.pdf.
23	李俊英. 深度卷积神经网络的旋转等变性研究[D]. 杭州: 浙江大学, 2019.
	LI J Y. Research on rotational isomorphism of deep convolution neural networks[D]. Hangzhou: Zhejiang University, 2019. (in Chinese)
24	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 936-944.
25	LIU Z, ZHAO X, HUANG T T, et al. TANet: robust 3D object detection from point clouds with triple attention. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11677- 11684.
26	SHI W J, RAJKUMAR R. Point-GNN: graph neural network for 3D object detection in a point cloud[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 1708-1716.
27	YANG Z T, SUN Y N, LIU S, et al. 3DSSD: point-based 3D single stage object detector[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2020: 11037-11045.
28	ZHANG X S, WAN F, LIU C, et al. Learning to match anchors for visual object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6): 3096- 3109.
29	SHI S S, WANG Z, SHI J P, et al. From points to parts: 3D object detection from point cloud with part-aware and part-aggregation network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(8): 2647- 2664.

[1]	费蓉, 马梦阳, 张晓, 黑新宏, 徐庆征, 邱原. 基于轨迹预测与冲突检测的自动驾驶碰撞检测模型[J]. 计算机工程, 2023, 49(7): 10-20.
[2]	席建锐, 唐红梅, 梁春阳, 刘鑫. 基于改进隐函数的点云物体重建[J]. 计算机工程, 2023, 49(7): 214-222.
[3]	孙超, 朱勇杰, 余林波, 苗隆鑫, 曹勉, 叶力, 郭乃宇. 基于点面特征融合的RGB-D SLAM算法[J]. 计算机工程, 2023, 49(6): 201-207.
[4]	李雪松, 张锲石, 宋呈群, 康宇航, 程俊. 自动驾驶场景下的轨迹预测技术综述[J]. 计算机工程, 2023, 49(5): 1-11.
[5]	苏鸣方, 胡立坤, 黄润辉. 基于上下文注意力的室外点云语义分割方法[J]. 计算机工程, 2023, 49(3): 248-256.
[6]	张文丽, 程兰, 任密蜂, 续欣莹, 阎高伟, 张喆. 基于AGConv局部特征描述符的点云配准[J]. 计算机工程, 2023, 49(11): 231-237.
[7]	刘洋, 陈俊, 胡诗佳, 赖佳华. 改进特征匹配的ORB-SLAM稠密建图算法[J]. 计算机工程, 2023, 49(10): 247-254.
[8]	高庆吉, 李天昊, 邢志伟, 刘佩佩. 基于区块特征融合的点云语义分割方法[J]. 计算机工程, 2022, 48(9): 37-44,54.
[9]	许嘉麟, 姚双, 张蕊华, 许浩, 沈洋. 基于亲疏度矩阵的点云置换不变特征提取方法[J]. 计算机工程, 2022, 48(6): 243-250.
[10]	梁智滨, 赵文义, 李灵巧, 杨辉华. 基于刀具刃口显微图像的附着物去除网络[J]. 计算机工程, 2022, 48(4): 247-254,261.
[11]	王志江, 秦品乐, 柴锐, 武峰, 程一彤, 史玥. 基于深度学习的牙齿嵌塞自动判别方法[J]. 计算机工程, 2022, 48(4): 307-313.
[12]	宋佳艳, 苏圣超. 基于改进蚁群优化算法的自动驾驶多车协同运动规划[J]. 计算机工程, 2022, 48(11): 299-305,313.
[13]	张毅, 林云汉, 刘双元. 一种面向低采样率的点云数据处理网络[J]. 计算机工程, 2022, 48(11): 240-246.
[14]	韩磊, 高永彬, 史志才. 基于稀疏Transformer的雷达点云三维目标检测[J]. 计算机工程, 2022, 48(11): 104-110,144.
[15]	张永亮, 陆阳, 朱芜强, 卫星, 魏臻. 基于多尺度特征提取与特征融合的交通标志检测[J]. 计算机工程, 2022, 48(10): 270-278,287.

选择文件类型/文献管理软件名称

选择包含的内容

基于多方向特征融合的室外三维目标检测方法

Outdoor 3D Object Detection Method Based on Multi-Direction Features Fusion

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 17

参考文献 29

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于多方向特征融合的室外三维目标检测方法

Outdoor 3D Object Detection Method Based on Multi-Direction Features Fusion

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 17

参考文献 29

相关文章 15

编辑推荐

Metrics

本文评价