面向语义分割模型的外接多尺度投票网络

doi:10.19678/j.issn.1000-3428.0062955

计算机工程 ›› 2022, Vol. 48 ›› Issue (10): 279-287. doi: 10.19678/j.issn.1000-3428.0062955

面向语义分割模型的外接多尺度投票网络

朱杰¹, 龚声蓉^1,2, 周立凡², 徐少杰¹, 钟珊²

1. 东北石油大学计算机与信息技术学院, 黑龙江大庆 163318;
2. 常熟理工学院计算机科学与工程学院, 江苏苏州 215000

收稿日期:2021-10-13 修回日期:2021-12-02 发布日期:2021-12-06
作者简介:朱杰(1996—),男,硕士研究生,主研方向为语义分割;龚声蓉(通信作者),教授、博士、博士生导师;周立凡,副教授、博士;徐少杰,硕士研究生;钟珊,副教授、博士。
基金资助:
国家自然科学基金（61972059，42071438）；江苏省自然科学基金（BK20191474）。

External Multi-Scale Voting Network for Semantic Segmentation Model

ZHU Jie¹, GONG Shengrong^1,2, ZHOU Lifan², XU Shaojie¹, ZHONG Shan²

1. School of Computer & Information Technology, Northeast Petroleum University, Daqing, Heilongjiang 163318, China;
2. School of Computer Science and Engineering, Changshu Institute of Technology, Suzhou, Jiangsu 215000, China

Received:2021-10-13 Revised:2021-12-02 Published:2021-12-06

摘要/Abstract

摘要： 针对语义分割领域中多尺度共享网络训练复杂度高，以及网络在小目标、长条状目标、目标边缘处拟合效果不佳的问题，提出一种新型外接多尺度投票网络。通过投票网络融合各尺度分割结果，降低网络训练复杂度，并将共享网络中的分割网络与各尺度注意力头剥离开，仅训练各尺度注意力头，以便于网络收敛。在投票网络的结构设计中，使用多类别投票方法扩大投票空间，通过融入混合池化模块聚合近程与远程权值，扩大网络感受野，缓解权值图中长条状目标拟合间断与缺失的问题。在此基础上引入类内、类间投票注意力模块获取权值及类间关系，并采用不规则卷积，改善投票权值图的边缘拟合效果。在Cityscapes数据集上的实验结果表明，相比FCN、PSPNet、DeepLabv3+分割网络，该网络的平均交并比分别提升了0.92、0.88、0.80个百分点，与共享网络相比，其训练复杂度更低，精度更高。

关键词: 语义分割, 多尺度投票网络, 平均交并比, 不规则卷积, 目标边缘

Abstract: To address the high training complexity of multi-scale shared networks in semantic segmentation and the unsatisfactory fitting effect of multi-scale attention maps for small targets, long targets, and target edges, a new external multi-scale voting network is proposed.The segmentation results of each scale are combined via the voting network to reduce the training complexity.Moreover, the segmentation network in the shared network is separated from the attention heads of each scale, and only the attention heads of each scale are trained to facilitate the convergence of the network.The multi-category voting method is used to expand the voting space.By integrating the hybrid pooling module to aggregate short and long-range weights, the receptive field of the network is expanded, and the discontinuity and missed fitting of long bar-shaped targets in the weight map are alleviated.Subsequently, intra-class and inter-class voting attention modules are introduced to obtain the weights and inter-class relations, and irregular convolution is used to improve the edge-fitting effect of the voting weight map.Experimental results on the Cityscapes dataset show that the mean Intersection over Union (mIoU) of the proposed network is higher than those of FCN, PSPNet, and DeepLabv3+ segmentation networks by 0.92, 0.88, and 0.80 percentage points, respectively.Compared with a shared network, the proposed multiscale voting network offers higher accuracy and requires less complex training.

Key words: semantic segmentaton, multi-scale voting network, mean Intersection over Union(mIoU), irregular convolution, target edge

中图分类号:

TP18

朱杰, 龚声蓉, 周立凡, 徐少杰, 钟珊. 面向语义分割模型的外接多尺度投票网络[J]. 计算机工程, 2022, 48(10): 279-287.

ZHU Jie, GONG Shengrong, ZHOU Lifan, XU Shaojie, ZHONG Shan. External Multi-Scale Voting Network for Semantic Segmentation Model[J]. Computer Engineering, 2022, 48(10): 279-287.

https://www.ecice06.com/CN/Y2022/V48/I10/279

图/表 14

参考文献

[1] SHELHAMER E, LONG J, DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of IEEE Transactions on Pattern Analysis and Machine Intelligence.Washington D.C., USA:IEEE Press, 2015:640-651.
[2] RONNEBERGER O, FISCHER P, BROX T.U-net:convolutional networks for biomedical image segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin, Germany:Springer, 2015:234-241.
[3] 易盟, 隋立春.基于改进全卷积神经网络的航拍图像语义分类方法[J].计算机工程, 2017, 43(10):216-221. YI M, SUI LI C.Aerial image semantic classification method based on improved full convolution neural network[J].Computer Engineering, 2017, 43(10):216-221.(in Chinese)
[4] CHEN L C, ZHU Y K, PAPANDREOU G, et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:801-818.
[5] BADRINARAYANAN V, KENDALL A, CIPOLLA R.SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.
[6] HUANG Z L, WANG X G, WEI Y C, et al.CCNet:criss-cross attention for semantic segmentation[C]//Proceedings of IEEE International Conference on Transactions on Pattern Analysis and Machine Intelligence.Washington D.C., USA:IEEE Press, 2019:603-612.
[7] IOFFE S, SZEGEDY C.Batch normalization:accelerating deep network training by reducing internal covariate shift[C]//Proceedings of International Conference on Machine Learning.Washington D.C., USA:IEEE Press, 2015:448-456.
[8] CHEN L C, PAPANDREOU G, KOKKINOS I, et al.DeepLab:semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
[9] FU J, LIU J, TIAN H J, et al.Dual attention network for scene segmentation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:3141-3149.
[10] HOU Q B, ZHANG L, CHENG M M, et al.Strip pooling:rethinking spatial pooling for scene parsing[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:4002-4011.
[11] ZHANG Z L, ZHANG X Y, PENG C, et al.ExFuse:enhancing feature fusion for semantic segmentation[C]//Proceedings of European Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2018:269-284.
[12] FAN M Y, LAI S Q, HUANG J S, et al.Rethinking BiSeNet for real-time semantic segmentation[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2021:9716-9725.
[13] CHEN L C, YANG Y, WANG J, et al.Attention to scale:scale-aware semantic image segmentation[C]//Proceedings of 2016 Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:3640-3649.
[14] YANG S Q, PENG G.Attention to refine through multi scales for semantic segmentation[C]//Proceedings of Conference on Multimedia Information.Washington D.C., USA:IEEE Press, 2018:232-241.
[15] TAO A, SAPRA K, CATANZARO B.Hierarchical multi-scale attention for semantic segmentation[EB/OL].[2021-09-11].https://arxiv.org/abs/2005.10821.
[16] DAI J F, HE K M, SUN J.BoxSup:exploiting bounding boxes to supervise convolutional networks for semantic segmentation[C]//Proceedings of 2015 IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2015:1635-1643.
[17] PAPANDREOU G, KOKKINOS I, SAVALLE P A.Untangling local and global deformations in deep convolutional networks for image classification and sliding window detection[EB/OL].[2021-09-11].https://arxiv.org/abs/1412.0296.
[18] CORDTS M, OMRAN M, RAMOS S, et al.The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:3213-3223.
[19] MOU L, ZHAO Y T, FU H Z, et al.CS2-Net:deep learning segmentation of curvilinear structures in medical imaging[J].Medical Image Analysis, 2021, 67:74-79.
[20] CHEN K, WANG J Q, PANG J M, et al.MMDetection:open MMLab detection toolbox and benchmark[EB/OL].[2021-09-11].https://arxiv.org/abs/1906.07155.
[21] PASZKE A, GROSS S, MASSA F, et al.PyTorch:an imperative style, high-performance deep learning library[EB/OL].[2021-09-11].https://arxiv.org/abs/1912.01703.
[22] HE K M, ZHANG X Y, REN S Q, et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:770-778.
[23] LIN T Y, GOYAL P, GIRSHICK R, et al.Focal loss for dense object detection[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2017:2999-3007.

选择文件类型/文献管理软件名称

选择包含的内容

面向语义分割模型的外接多尺度投票网络

External Multi-Scale Voting Network for Semantic Segmentation Model

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李仲, 冒睿瑞, 王晓龙, 王根一, 安国成. 基于改进PIDNet的水位线检测算法[J]. 计算机工程, 2024, 50(8): 102-112.
[2]	闵莉, 董冰洁, 安冬. 基于多注意力机制与跨特征融合的语义分割算法[J]. 计算机工程, 2024, 50(8): 282-289.
[3]	逯焕宇, 张永宏, 马光义, 谢东林, 田伟. 基于半监督对抗学习的遥感图像水体提取[J]. 计算机工程, 2024, 50(7): 251-263.
[4]	肖慈, 徐杨, 张永丹, 冯明文, 黄易仟. 结合注意力和低光增强的夜间语义分割[J]. 计算机工程, 2024, 50(7): 271-281.
[5]	陈晓玉, 沈晨, 沈阅, 孔德明. 基于改进SwiftNet的堆场图像实时分割网络[J]. 计算机工程, 2024, 50(6): 296-303.
[6]	王安政, 党建武, 岳彪, 杨景玉. 基于位置信息和注意力机制的路面裂缝检测[J]. 计算机工程, 2024, 50(4): 303-312.
[7]	王柏涵, 姜晓燕, 范柳伊. 基于深度监督隐空间构建的语义分割改进方法[J]. 计算机工程, 2024, 50(3): 191-199.
[8]	徐威, 付晓薇, 李曦, 汪尧坤. 融合多层感知注意力的电极微观图像分割方法[J]. 计算机工程, 2024, 50(1): 329-338.
[9]	苏晓东, 李世洲, 赵佳圆, 亮洪宇, 张玉荣, 徐红岩. 基于多级叠加和注意力机制的图像语义分割[J]. 计算机工程, 2023, 49(9): 265-271, 278.
[10]	徐春波, 闫娟, 杨慧斌, 王博, 吴晗. 基于目标检测和语义分割的视觉SLAM算法[J]. 计算机工程, 2023, 49(8): 199-206, 214.
[11]	白俊卿, 韩柏迅, 张丰侠. 基于深度学习的无人机图像语义分割算法研究[J]. 计算机工程, 2023, 49(4): 233-239.
[12]	苏鸣方, 胡立坤, 黄润辉. 基于上下文注意力的室外点云语义分割方法[J]. 计算机工程, 2023, 49(3): 248-256.
[13]	马素刚, 陈期梅, 侯志强, 杨小宝, 张子贤. 基于密集连接与特征增强的语义分割算法[J]. 计算机工程, 2023, 49(3): 263-270.
[14]	范润泽, 刘宇红, 张荣芬, 李景玉. 基于多尺度注意力机制的道路场景语义分割模型[J]. 计算机工程, 2023, 49(2): 288-295.
[15]	李嘉豪, 闵卫东, 陈炯缙, 朱梦, 展国伟. 一种复杂场景下高精度交通标志检测模型[J]. 计算机工程, 2023, 49(11): 311-320.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

面向语义分割模型的外接多尺度投票网络

External Multi-Scale Voting Network for Semantic Segmentation Model

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献

相关文章 15

编辑推荐

Metrics

本文评价