基于注意力机制的CycleGAN服装局部风格迁移研究

doi:10.19678/j.issn.1000-3428.0059665

计算机工程 ›› 2021, Vol. 47 ›› Issue (11): 305-312. doi: 10.19678/j.issn.1000-3428.0059665

基于注意力机制的CycleGAN服装局部风格迁移研究

陈佳^1,2, 董学良¹, 梁金星¹, 何儒汉¹

1. 武汉纺织大学数学与计算机学院, 武汉 430000;
2. 湖北省服装信息化工程技术研究中心, 武汉 430000

收稿日期:2020-10-09 修回日期:2020-11-13 发布日期:2020-11-24
作者简介:陈佳(1982-),女,副教授、博士,主研方向为图像处理、模式识别;董学良,硕士研究生;梁金星(通信作者),讲师、博士;何儒汉,教授、博士。
基金资助:
湖北省自然科学基金计划一般面上项目（2020CFB801）；湖北省高等学校优秀中青年科技创新团队计划（T201807）。

Research on the Local Style Transfer of Clothing Images by CycleGANBased on Attention Mechanism

CHEN Jia^1,2, DONG Xueliang¹, LIANG Jinxing¹, HE Ruhan¹

1. School of Mathematics and Computer Science, Wuhan Textile University, Wuhan 430000, China;
2. Engineering Research Center of Hubei Province for Clothing Information, Wuhan 430000, China

Received:2020-10-09 Revised:2020-11-13 Published:2020-11-24

摘要/Abstract

摘要： 针对复杂背景下服装图像局部区域风格迁移难以控制及迁移后容易产生边界伪影的问题，提出一种基于注意力机制的CycleGAN服装局部风格迁移方法。通过VGG16网络分别提取服装图像的内容特征与风格特征，将其输入基于注意力机制的CycleGAN生成器中，应用注意力机制在复杂背景下的各个服装区域分配概率分布信息，获得注意力分布更多的区域及相关度更高的区域，并采用改进的损失函数校正边界伪影，对该区域进行风格迁移得到所需的风格迁移服装图像。实验结果表明，与CNN、FCN、BeautyGAN图像局部风格迁移方法相比，该方法不仅可以突出服装图像局部风格迁移效果，而且增强了图像细节，有利于提高输出图像的真实性和艺术性。

关键词: 图像风格迁移, 边界伪影, 注意力机制, 循环生成对抗网络, 损失函数

Abstract: For the clothing images with complex backgrounds,it is difficult to control the style transfer in local areas of the images,and boundary artifacts are easily generated in this process.To address the problem,a CycleGAN-based method using attention mechanism is proposed for local style transfer of clothing images.The method employs VGG16 to extract the content features and style features from the clothing images separately,and the features are subsequently input into the CycleGAN generator based on the attention mechanism.The attention mechanism is used to distribute the probability distribution information to each clothing area of the complex background to obtain the area with more attention distribution and the area with higher correlation.Then an improved loss function is used to correct boundary artifacts.Finally,style transfer is performed on the area to obtain the desired style transfer clothing image.The experimental results show that compared with the local style transfer methods based on CNN,FCN and BeautyGAN,the proposed method can not only highlight the effect of local style transfer of clothing images,but also enhance the image details,which is conducive to improving the authenticity and artistry of the output images.

Key words: image style transfer, boundary artifacts, attention mechanism, Cycle Generative Adversarial Network(CycleGAN), loss function

中图分类号:

TP391.41

陈佳, 董学良, 梁金星, 何儒汉. 基于注意力机制的CycleGAN服装局部风格迁移研究[J]. 计算机工程, 2021, 47(11): 305-312.

CHEN Jia, DONG Xueliang, LIANG Jinxing, HE Ruhan. Research on the Local Style Transfer of Clothing Images by CycleGANBased on Attention Mechanism[J]. Computer Engineering, 2021, 47(11): 305-312.

http://www.ecice06.com/CN/Y2021/V47/I11/305

图/表 11

20211113145926

20211113145929

20211113145932

20211113145935

20211113145938

20211113145941

20211113145943

20211113145947

20211113145950

20211113145952

20211113145955

参考文献

[1] GATYS L A,ECKER A S,BETHGE M.Image style transfer using convolutional neural networks[C]//Proceedings of Computer Vision and Pattern Recognition Conference.Washington D.C.,USA:IEEE Press,2016:234-245.
[2] EFROS A A,FREEMAN W T.Image quilting for texture synthesis and transfer[C]//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques.New York,USA:ACM press,2001:341-346.
[3] HUANG X,BELONGIE S.Arbitrary style transfer in real-time with adaptive instance normalization[C]//Proceedings of International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:2380-7504.
[4] HERTZMANN A,JACOBS C E,OLIVER N,et al.Image analogies[C]//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques.New York,USA:ACM Press,2001:327-340.
[5] 钱小燕,肖亮,吴慧中.快速风格迁移[J].计算机工程,2006,32(21):15-17. QIAN X Y,XIAO L,WU H Z.Fast style transfer[J].Computer Engineering,2006,32(21):15-17.(in Chinese)
[6] LECUN Y,BENGIO Y,HINTON G.Deep learning[J].Nature,2015,521(7553):436-445.
[7] GATYS L A,ECKER A S,BETHGE M.A neural algorithm of artistic style[J].Journal of Vision,2015,22(11):54-65.
[8] RADFORD A,METZ L,CHINTALA S.Unsupervised representation learning with deep convolutional generative adversarial networks[J].Computer Ence,2015,22(11):524-535.
[9] ZHU J Y,PARK T,ISOLA P,et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:334-354.
[10] LUAN F,PARIS S,SHECHTMAN E,et al.Deep photo style transfer[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:1234-1254.
[11] WANGJ,MA Y,ZHANG L,et al.Deep learning for smart manufacturing:methods and applications[J].Journal of Manufacturing Systems,2018,34(2):144-156.
[12] GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial networks[J].Advances in Neural Information Processing Systems,2014,3:2672-2680.
[13] 李婷婷,胡玉龙,魏枫林.基于GAN改进的人脸表情识别算法及应用[J].吉林大学学报(理学版),2020,243(3):163-168. LI T T,HU Y L,WEI F L.Improved facial expression recognition algorithm based on GaN and its application[J].Journal of Jilin University(Science Edition),2020,243(3):163-168.(in Chinese)
[14] WEN L,ZHANG,Y H LIU,C.et al.Ranksrgan:generative adversarial networks with ranker forimage[EB/OL].[2020-09-01].https://arxiv.org/abs/1908.06382
[15] 张德园,常云翔,张利国,等.SAT CNN:基于卷积神经网络的遥感图像分类算法[J].小型微型计算机系统,2018,39(4):859-864. ZHANG D Y,CHANG Y X,ZHANG L G,et al.SAT-CNN:convolutional neural network framework for remote sensing image classification[J].Journal of Chinese Computer Systems,2018,39(4):859-864.(in Chinese)
[16] 缪永伟,李高怡,鲍陈,等.基于卷积神经网络的图像局部风格迁移[J].计算机科学,2019,46(9):259-264. MIAO Y W,LI G Y,BAO C,et al.Image localized style transfer based on convolutional neural network[J].Computer Science,2019,46(9):259-264.(in Chinese)
[17] 刘哲良,朱玮,袁梓洋.结合全卷积网络与CycleGAN的图像实例风格迁移[J].中国图象图形学报,2019,24(8):1283-1291. LIU Z L,ZHU W,YUAN Z Y.Image instance style transfer combined with fully convolutional network and CycleGAN[J].Journal of Image and Graphics,2019,24(8):1283-1291.(in Chinese)
[18] LI C,WAND M.Combining markov random fields and convolutional neural networks for image synthesis[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:2479-2486.
[19] CHAMPANDARD A J.Semantic style transfer and turning two-bit doodles into fine artworks[EB/OL].[2020-09-01].https://arxiv.org/abs/1603.01768.
[20] TING T L,RUI H Q,CHAO D,et al. Beautygan:instance level facial makeup transfer with deep generative adversarial network[C]//Proceedings of Multi-media Conference.New York,USA:ACM Press,2018:645-653.
[21] GU S,BAO J,YANG H,et al.Mask-guided portrait editing with conditional gans[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:3436-3445.
[22] 林泓,任硕,杨益,等.融合自注意力机制和相对鉴别的无监督图像翻译[J].自动化学报,2019,17(3):1-10. LIN H,REN S,YANG Y,etal.Unsupervised image-to-image translation with self-attention and relativistic discriminator adversarial networks[J].ACTA Automatica Sinica 2019,17(3):1-10.(in Chinese)
[23] CHOI M,KIM H,HAN B,et al.Channel attention is all you need for video frame interpolation[C]//Proceedings of the Conference on Artificial Intelligence.Washington D.C.,USA:IEEE Press,2020:10663-10671.
[24] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.

选择文件类型/文献管理软件名称

选择包含的内容

基于注意力机制的CycleGAN服装局部风格迁移研究

Research on the Local Style Transfer of Clothing Images by CycleGANBased on Attention Mechanism

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	丰芳宇, 罗晓曙, 蒙志明, 王广宇. 基于抗混叠残差注意力网络的人脸表情识别[J]. 计算机工程, 2023, 49(8): 190-198.
[2]	王书朋, 何引弟. 融合特征注意力机制的非均匀光照图像增强算法[J]. 计算机工程, 2023, 49(8): 232-239.
[3]	刘昊鑫, 董超, 勾智楠, 高凯. 融合混合表征的小样本关系抽取方法[J]. 计算机工程, 2023, 49(8): 63-68.
[4]	杨长沛, 廖列法. 基于门控空洞卷积特征融合的中文命名实体识别[J]. 计算机工程, 2023, 49(8): 85-95.
[5]	刘俊豪, 王美林, 谢兴, 宋烨兴, 许莉花. 基于改进YOLOv5的皮革瑕疵检测算法[J]. 计算机工程, 2023, 49(8): 240-249.
[6]	马娜, 温廷新, 贾旭, 李晓会. 复杂光照条件下自适应的车脸重识别模型[J]. 计算机工程, 2023, 49(8): 275-282, 290.
[7]	陈露萌, 曹彦彦, 黄民, 谢鑫钢. 基于改进YOLOv5的火焰检测方法[J]. 计算机工程, 2023, 49(8): 291-301, 309.
[8]	李强龙, 周新文, 位梦恩, 甘阳洲. 基于条形池化和注意力机制的街道场景红外目标检测算法[J]. 计算机工程, 2023, 49(8): 310-320.
[9]	张家熔, 苑津莎, 许珈宁, 罗志宏. 基于多元信息嵌入与协同神经网络的力学实体识别算法[J]. 计算机工程, 2023, 49(7): 125-134.
[10]	白明昌. 基于折叠路径聚合的属性网络节点嵌入方法[J]. 计算机工程, 2023, 49(7): 76-84.
[11]	费蓉, 马梦阳, 张晓, 黑新宏, 徐庆征, 邱原. 基于轨迹预测与冲突检测的自动驾驶碰撞检测模型[J]. 计算机工程, 2023, 49(7): 10-20.
[12]	刘豪, 吴红兰, 房宇轩. 结合全局上下文信息的高效人体姿态估计[J]. 计算机工程, 2023, 49(7): 102-109.
[13]	吴珊, 周凤. 基于改进SSD算法的小目标检测[J]. 计算机工程, 2023, 49(7): 179-188.
[14]	齐咏生, 杜晓旭, 朱俊峰, 高胜利, 刘利强. 基于增强型轻量深度网络的牧区牲畜高效检测[J]. 计算机工程, 2023, 49(7): 278-287.
[15]	陈明, 刘蓉, 张晔. 基于多重注意力机制的中文医疗实体识别[J]. 计算机工程, 2023, 49(6): 314-320.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于注意力机制的CycleGAN服装局部风格迁移研究

Research on the Local Style Transfer of Clothing Images by CycleGANBased on Attention Mechanism

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献

相关文章 15

编辑推荐

Metrics

本文评价