作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程

• •    

结合注意力和低光增强的夜间语义分割

  • 发布日期:2023-12-05

Nighttime Semantic Segmentation with Attention and Low-Light Enhancement

  • Published:2023-12-05

摘要: 随着深度学习和计算能力的提升,对白天拍摄的自然场景图像进行语义分割的性能有了显著的提高。然而,在夜间图像语义分割任务中,由于曝光不平衡和缺乏标记数据等挑战,由白天数据训练的模型往往无法取得良好的表现。为了解决这些问题,本文提出了一种新的无监督夜间图像语义分割网络AI-USeg。首先,使用一个轻量级的照明网络SCI对夜间图像进行增强,以减少光照变化对后续语义分割网络的影响。其次,引入了领域自适应方法,将模型从包含大量有标签的Cityscapes数据自适应到Dark Zurich-D,解决缺乏标记数据的问题。此外,AI-USeg在基于FCN实现的判别器中引入SENet,通过在输出空间进行对抗学习来适应夜间低光照环境下的图像特征,以提升夜间图像语义分割任务的性能。为了验证AI-USeg的效果,实验使用Cityscapes和Dark Zurich-train中的2416个昼夜图像对进行无监督训练,在Dark Zurich-test和Nighttime Driving-test上的mIoU分别达到了47.88和51.49,相较于MGCDA分别提高了5.38和2.09,对夜间图像的特征适应性更强,具有更高的鲁棒性,为夜间场景的图像分割任务提供了一种有效的解决方案。

Abstract: With the improvement of deep learning and computing power, the performance of semantic segmentation of natural scene images captured during the day has significantly increased. However, due to challenges such as exposure imbalance and lack of labeled data, models trained on daytime data often fail to perform well in nighttime image semantic segmentation tasks. To address these issues, we proposed a new unsupervised nighttime image semantic segmentation network called AI-USeg. Firstly, a lightweight illumination network, SCI, is used to enhance nighttime images to reduce the impact of lighting changes on subsequent semantic segmentation networks. Secondly, domain adaptation methods are introduced to adapt the model from a large amount of labeled Cityscapes data to Dark Zurich-D, which lacks labeled data. Additionally, SENet is introduced into the discriminator implemented based on FCN, and adversarial learning is carried out in the output space to adapt to the image features under low-light conditions, thus improving the performance of nighttime image semantic segmentation tasks. Experimentally, using 2416 day-night images from Cityscapes and Dark Zurich-train for unsupervised training, our proposed approach achieved real-world mIoU scores of 47.88 and 51.49 on Dark Zurich-test and Nighttime Driving-test respectively, which improved by 5.38 and 2.09 compared to MGCDA. AI-USeg has stronger feature adaptation capabilities and higher robustness for nighttime scene image segmentation tasks, providing an effective solution..