Abstract: To achieve assisted driving and vehicle-road coordination, high-precision real-time detection and segmentation of traffic scenes are crucial. However, instance segmentation in traffic scenarios has its challenges, including complex environments, object stacking, and low object resolution that may cause false detections, missing detections, and missing masks. Moreover, the widely used two-stage models in high-precision instance segmentation studies often come with a large number of parameters, making real-time requirements challenging to achieve. Proposing an Instance Segmentation Algorithm (DE-YOLO) based on Improved YOLOv8. To decrease the effect of complex backgrounds in images, efficient multi-scale attention is introduced, and cross-dimensional interaction ensures an even spatial feature distribution within each feature group. In the backbone network, deformable convolution using DCNv2 is combined with the C2f convolutional layer to surpass the limitations of traditional convolutions and increase flexibility. This is done to reduce harmful gradient effects and improve the overall accuracy of the detector. The dynamic non-monotonic Wise-IoU (WIoU) focusing mechanism is used instead of the traditional CIoU loss function to evaluate the quality, optimize detection frame positioning, and improve segmentation accuracy. Meanwhile, Mixup data enhancement processing is enabled to enrich the training features of the dataset and improve the model's learning ability. The experimental results demonstrate that DE-YOLO improves the average accuracy (mAPmask) by 2.0 percentage points and 3.2 percentage points by APmask@0.5 compared to the benchmark model YOLOv8n-seg in the cityscapes dataset of urban landscapes. Furthermore, DE-YOLO maintains excellent detection speed and small parameter quantity while improving the accuracy, with the model requiring 2.2-31.3 percentage points fewer parameters than similar models.

摘要： 实现辅助驾驶、车路协同均需要对交通场景进行高精度的实时检测分割，但在实例分割过程中，由于环境复杂、目标堆叠、对象分辨率低等因素，存在着错检、漏检及掩膜缺失等问题，且针对高精度实例分割研究中多采用二阶段模型，通常因参数量过大无法满足实时性需求。提出一种基于改进型YOLOv8 的实例分割算法（DE-YOLO）。为减少图像中复杂背景的干扰，引入高效多尺度注意力机制，跨维交互使各特征组内空间语义特征分布平均。在主干网络部分，使用可变形卷积DCNv2结合C2f卷积层，突破原始卷积限制，增加可变性。为减小有害梯度并整体提升检测器精度，采用动态非单调聚焦机制Wise-IoU（WIoU）替代CIoU损失函数进行质量评估，优化检测框定位，提升分割精度。同时，开启Mixup数据增强处理，充实数据集丰富训练特征，提升模型学习能力。实验结果表明，DE-YOLO在城市景观数据集Cityscapes中的掩膜平均精度（mAPmask）较基准模型YOLOv8n-seg提高了2.0个百分点，APmask@0.5提升了3.2个百分点，在精度提升的同时，保持了优良的检测速度和小参数量，模型参数量相较同类模型低2.2-31.3个百分点。