Improved CycleGAN Algorithm for Semi-Supervised Building Extraction

doi:10.19678/j.issn.1000-3428.0069081

Abstract

Abstract:

Currently, building extraction requires a large amount of labeled data for training, and it takes a long time to collect and label the data. In order to achieve building extraction based on semi-supervised learning on a small sample remote sensing image datasets, this study constructs four sets of building extraction datasets and proposes a building extraction method based on a Cycle-consistency Generative Adversarial Network (CycleGAN). First, this study introduces a Global Attention Mechanism (GAM) module into the generator to enhance the distinction between the details of the buildings and image backgrounds. Second, it adds a spectral normalization layer to the discriminator to enhance the stability of the training, which solves the problem of gradient vanishing during training. Finally, it improves adversarial and cycle consistency losses to improve the quality of generated images, avoids excessive smoothing of generated images, and introduces Identity loss to limit the generator from modifying the colors of the input images involuntarily to ensure consistency of the color composition of the input and output images. The experimental results show that on the first small-sample dataset, the Structure Similarity Index Measure (SSIM) and accuracy increase by at least 3 and 8.1 percentage points, respectively, compared with those of the UNIT, MUNIT, U-GAT-IT, SPatchGAN, and QS-Attn models. On a dataset with an expanded data scale, the improved algorithm is used to compare fully supervised and semi-supervised experiments, and the effectiveness of the improved algorithm in realizing the semi-supervised extraction of buildings on a small-sample remote sensing image dataset is verified.

Key words: building extraction, Cycle-consistency Generative Adversarial Network (CycleGAN), spectral normalization, Global Attention Mechanism (GAM), semi-supervised

摘要：

建筑物提取需要大量的标注数据进行训练, 收集和标注数据需要耗费大量时间。为了在小样本遥感图像数据集上基于半监督学习实现建筑物提取的目的, 构建4组建筑物提取数据集, 提出了一种基于循环一致性生成对抗网络(CycleGAN)的建筑物提取算法。首先, 在生成器中引入全局注意力机制(GAM)以增强对建筑物和图像背景细节特征的区分; 其次, 在判别器中加入谱归一化层以增强训练稳定性, 解决了训练过程中梯度消失问题; 最后, 改进对抗损失和循环一致性损失以提高生成图像的质量, 避免生成图像的过度平滑化, 并引入Identity损失以限制生成器不会自主修改输入图像的颜色, 保证输入图像与输出图像颜色组成的一致性。实验结果表明, 在第1组小样本数据集上, 与UNIT、MUNIT、U-GAT-IT、SPatchGAN、QS-Attn模型进行半监督实验对比, 结构相似性(SSIM)值和准确率分别至少提高了3、8.1百分点, 在扩充数据规模的数据集上, 使用改进后的算法进行全监督和半监督实验对比, 验证了改进后的算法在小样本遥感图像数据集上实现建筑物半监督提取的有效性。

关键词: 建筑物提取, 循环一致性生成对抗网络, 谱归一化, 全局注意力机制, 半监督

LU Peng, ZHONG Chuang. Improved CycleGAN Algorithm for Semi-Supervised Building Extraction[J]. Computer Engineering, 2025, 51(3): 241-251.

卢鹏, 仲闯. 改进CycleGAN的半监督建筑物提取算法[J]. 计算机工程, 2025, 51(3): 241-251.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0069081

https://www.ecice06.com/EN/Y2025/V51/I3/241

Figures/Tables 15

Fig.1 GAN structure

Fig.2 CycleGAN structure

Fig.3 Procedure of SG-CycleGAN algorithm

Fig.4 SG-CycleGAN network structure combining spectral normalization and dual attention mechanism

Fig.5 The structure of generator

Fig.6 Discriminator structure

Fig.7 Datasets of unpaired and paired images

Fig.8 The building detection results of the first set of datasets

Fig.9 Experimental results of different models

Fig.10 Experimental results of four datasets

Fig.11 Experimental results on datasets of different scales

Fig.12 Quantitative results variation trends of SG-CycleGAN on datasets of different scales

References 30

1	马长辉, 黄登山. 纹理与几何特征信息在高空间分辨率遥感影像分类中的应用. 测绘地理信息, 2019, 44 (6): 66-70, 92.
	MA C H , HUANG D S . Application of texture features and geometric feature information in high spatial resolution remote sensing image classification. Journal of Geomatics, 2019, 44 (6): 66-70, 92.
2	ORHEI C, VERT S, VASIU R. A novel edge detection operator for identifying buildings in augmented reality applications[EB/OL]. [2023-11-18]. https://arxiv.org/abs/2106.01055.
3	成行. 面向复杂场景的高分辨率遥感影像建筑物语义分割方法[D]. 南京: 南京邮电大学, 2022.
	CHENG H. Semantic segmentation of buildings from high-resolution remote sensing images of complex scenes[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2022. (in Chinese)
4	LAI X D , YANG J R , LI Y X , et al. A building extraction approach based on the fusion of LiDAR point cloud and elevation map texture features. Remote Sensing, 2019, 11 (14): 1636. doi: 10.3390/rs11141636
5	CHEN L F , CUI X L , LI Z H , et al. A new deep learning algorithm for SAR scene classification based on spatial statistical modeling and features re-calibration. Sensors, 2019, 19 (11): 2479. doi: 10.3390/s19112479
6	CHEN L F , TAN S Y , PAN Z H , et al. A new framework for automatic airports extraction from SAR images using multi-level dual attention mechanism. Remote Sensing, 2020, 12 (3): 560. doi: 10.3390/rs12030560
7	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2015: 234-241.
8	ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2017: 15-21.
9	PENG H X , XUE C , SHAO Y Y , et al. Semantic segmentation of litchi branches using DeepLabV3+ model. IEEE Access, 2020, 8, 164546- 164555. doi: 10.1109/ACCESS.2020.3021739
10	金澍, 关沫, 边玉婵, 等. 基于改进U-Net的遥感影像建筑物提取方法. 激光与光电子学进展, 2023, 60 (4): 0401002.
	JIN S , GUAN M , BIAN Y C , et al. Building extraction from remote sensing images based on improved U-Net. Laser & Optoelectronics Progress, 2023, 60 (4): 0401002.
11	武花, 张新长, 孙颖, 等. 融合多特征改进型PSPNet模型应用于复杂场景下的建筑物提取. 测绘通报, 2021 (6): 21- 27.
	WU H , ZHANG X C , SUN Y , et al. Building extraction in complex scenes based on the fusion of multi-feature improved PSPNet model. Bulletin of Surveying and Mapping, 2021 (6): 21- 27.
12	于明洋, 张文焯, 陈肖娴, 等. 基于DeepLabv3+的高分辨率遥感影像建筑物自动提取. 测绘工程, 2022, 31 (4): 1-10, 17.
	YU M Y , ZHANG W Z , CHEN X X , et al. Automatic extraction of buildings from high-resolution remote sensing images based on DeepLabv3+. Engineering of Surveying and Mapping, 2022, 31 (4): 1-10, 17.
13	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2016: 779-788.
14	HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision (ICCV). Washington D. C., USA: IEEE Press, 2017: 61-67.
15	GIRSHICK R. Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision (ICCV). Washington D. C., USA: IEEE Press, 2015: 1440-1448.
16	丁飞, 石颉, 吴宏杰. 改进YOLOv4的轻量级遥感图像建筑物检测模型. 计算机工程与应用, 2023, 59 (10): 213- 220. doi: 10.3778/j.issn.1002-8331.2202-0287
	DING F , SHI J , WU H J . Lightweight building detection model based on YOLOv4 optimization for remote sensing images. Computer Engineering and Applications, 2023, 59 (10): 213- 220. doi: 10.3778/j.issn.1002-8331.2202-0287
17	胡舒, 王树根, 王越, 等. 基于Mask R-CNN的高分遥感影像建筑物目标检测. 测绘地理信息, 2023, 48 (3): 50- 54.
	HU S , WANG S G , WANG Y , et al. Building object detection in high-resolution remote sensing image based on Mask R-CNN. Journal of Geomatics, 2023, 48 (3): 50- 54.
18	李东子, 范大昭, 苏亚龙. 结合Faster R-CNN模型的遥感影像建筑物检测. 测绘科学技术学报, 2018, 35 (4): 389- 394.
	LI D Z , FAN D Z , SU Y L . Building detection in remote sensing image based on faster R-CNN. Journal of Geomatics Science and Technology, 2018, 35 (4): 389- 394.
19	成浩维, 资文杰, 彭双, 等. 基于半监督学习的三维Mesh建筑物立面提取与语义分割方法. 郑州大学学报(理学版), 2023, 55 (4): 8- 15.
	CHENG H W , ZI W J , PENG S , et al. Semi-supervised learning based 3D Mesh building facade extraction and semantic segmentation method. Journal of Zhengzhou University (Natural Science Edition), 2023, 55 (4): 8- 15.
20	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2014: 2672-2680.
21	刘华超, 张俊然, 刘云飞. 引入特征损失对CycleGAN的影响研究. 计算机工程与应用, 2020, 56 (22): 217- 223.
	LIU H C , ZHANG J R , LIU Y F . Influence of identity loss on CycleGAN. Computer Engineering and Applications, 2020, 56 (22): 217- 223.
22	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision (ICCV). Washington D. C., USA: IEEE Press, 2017: 2242-2251.
23	卢鹏, 张娜, 邹国良, 等. 基于双重注意力机制的CycleGAN海岸线自动提取方法. 激光与光电子学进展, 2022, 59 (12): 1210005.
	LU P , ZHANG N , ZOU G L , et al. CycleGAN coastline automatic extraction method based on dual attention mechanism. Laser & Optoelectronics Progress, 2022, 59 (12): 1210005.
24	LIU Y C, SHAO Z R, HOFFMANN N. Global attention mechanism: retain information to enhance channel-spatial interactions[EB/OL]. [2023-11-18]. https://arxiv.org/abs/2112.05561?context=cs.
25	MIYATO T, KATAOKA T, KOYAMA M, et al. Spectral normalization for generative adversarial networks[EB/OL]. [2023-11-18]. https://arxiv.org/pdf/1802.05957.
26	LIU M Y, BREUEL T, KAUTZ J. Unsupervised image-to-image translation networks[EB/OL]. [2023-11-18]. https://arxiv.org/pdf/1703.00848.
27	HUANG X, LIU M Y, BELONGIE S, et al. Multimodal unsupervised image-to-image translation[EB/OL]. [2023-11-18]. https://arxiv.org/pdf/1804.04732.
28	KIM J, KIM M, KANG H, et al. U-GAT-IT: unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation[EB/OL]. [2023-11-18]. https://arxiv.org/abs/1907.10830v2.
29	SHAO X N, ZHANG W D. SPatchGAN: a statistical feature based discriminator for unsupervised image-to-image translation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D. C., USA: IEEE Press, 2021: 6546-6555.
30	HU X Q, ZHOU X Y, HUANG Q S, et al. QS-Attn: query-selected attention for contrastive learning in I2I translation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2022: 18291-18300.

[1]	GAO Rui, AN Guocheng, ZOU Danping, PEI Ling. Semi-Supervised Vehicle Detection Algorithm Based on Improved YOLOv5 [J]. Computer Engineering, 2025, 51(3): 300-309.
[2]	ZHANG Xinbo, ZHANG Xueying, HUANG Lixia, CHEN Guijun. Classification Algorithm and Application Based on Semi-Supervised Deep Auto-Encoder Network [J]. Computer Engineering, 2025, 51(1): 71-80.
[3]	GUO Min, ZHANG Xihan, LI Yang. Integrated Attentional Teacher Mutual Consistency Semi-Supervised Medical Image Segmentation [J]. Computer Engineering, 2024, 50(9): 313-323.
[4]	Huanyu LU, Yonghong ZHANG, Guangyi MA, Donglin XIE, Wei TIAN. Semi-Supervised Adversarial Learning-Based Water Body Extraction from Remote Sensing Images [J]. Computer Engineering, 2024, 50(7): 251-263.
[5]	GU Yonggen, GAO Lingxuan, WU Xiaohong, TAO Jie. Research on Data Sharing of Federated Semi-Supervised Learning with Non-IID [J]. Computer Engineering, 2024, 50(6): 188-196.
[6]	YOU Ben, LI Xiaohong, YAO Jin, FENG Shaojie. Semi-supervised Classification for Short Text Based on Multi-grained Graphs and Attention Mechanism [J]. Computer Engineering, 2024, 50(5): 83-90.
[7]	CHEN Zhonglei, YI Peng, CHEN Xiang, HU Tao. Real-time Anomaly Detection Framework via System Calls Based on Integrated Learning [J]. Computer Engineering, 2023, 49(6): 162-169,179.
[8]	Nannan GU. Self-Paced Semi-Supervised Dimensionality Reduction for Data with Noisy Labels [J]. Computer Engineering, 2023, 49(11): 131-142.
[9]	SHE Zhaoyang, YAN Xin, XU Guangyi, CHEN Wei, DENG Zhongying. Adverse Drug Reaction Detection Combined with Data Augmentation and Semi-supervised Learning [J]. Computer Engineering, 2022, 48(6): 314-320.
[10]	WANG Shufen, ZHANG Zhe, MA Shiyao, CHEN Yuqiang, WU Yi. A Robust Semi-Supervised Federated Learning System [J]. Computer Engineering, 2022, 48(6): 107-114,123.
[11]	ZHANG Lei, WANG Xiaolong, LIU Chang. SAR Building Segmentation Algorithm Combining Saliency and MRF [J]. Computer Engineering, 2022, 48(4): 284-291,298.
[12]	HU Bin, WANG Xiaojun, ZHANG Lei. A Semi-Supervised Adversarial Robust Model-Agnostic Meta-Learning Method [J]. Computer Engineering, 2022, 48(12): 112-118.
[13]	GAO Wei, WU Shun. Scratch Repairing of Old Photos Based on Multi-Scale Attention Semi-Supervised Learning [J]. Computer Engineering, 2022, 48(10): 245-251,261.
[14]	KANG Lulu, FAN Xingrong, WANG Qianzhu, YANG Xiaoya, MING Rui. Mouse Trajectory Recognition Based on Feature Group Hierarchy and Semi-Supervised Learning [J]. Computer Engineering, 2021, 47(4): 277-284.
[15]	XUE Zihan, PAN Di, HE Li. Optimization of LGC Semi-Supervised Learning Method Combined with Improved Density Peaks Clustering [J]. Computer Engineering, 2021, 47(2): 77-83,89.

Please choose a citation manager

Content to export