基于注意力机制的CycleGAN服装局部风格迁移研究

doi:10.19678/j.issn.1000-3428.0059665

计算机工程 ›› 2021, Vol. 47 ›› Issue (11): 305-312. doi: 10.19678/j.issn.1000-3428.0059665

基于注意力机制的CycleGAN服装局部风格迁移研究

陈佳^1,2, 董学良¹, 梁金星¹, 何儒汉¹

1. 武汉纺织大学数学与计算机学院, 武汉 430000;
2. 湖北省服装信息化工程技术研究中心, 武汉 430000

收稿日期:2020-10-09 修回日期:2020-11-13 发布日期:2020-11-24
作者简介:陈佳(1982-),女,副教授、博士,主研方向为图像处理、模式识别;董学良,硕士研究生;梁金星(通信作者),讲师、博士;何儒汉,教授、博士。
基金资助:
湖北省自然科学基金计划一般面上项目（2020CFB801）；湖北省高等学校优秀中青年科技创新团队计划（T201807）。

Research on the Local Style Transfer of Clothing Images by CycleGANBased on Attention Mechanism

CHEN Jia^1,2, DONG Xueliang¹, LIANG Jinxing¹, HE Ruhan¹

1. School of Mathematics and Computer Science, Wuhan Textile University, Wuhan 430000, China;
2. Engineering Research Center of Hubei Province for Clothing Information, Wuhan 430000, China

Received:2020-10-09 Revised:2020-11-13 Published:2020-11-24

摘要/Abstract

摘要： 针对复杂背景下服装图像局部区域风格迁移难以控制及迁移后容易产生边界伪影的问题，提出一种基于注意力机制的CycleGAN服装局部风格迁移方法。通过VGG16网络分别提取服装图像的内容特征与风格特征，将其输入基于注意力机制的CycleGAN生成器中，应用注意力机制在复杂背景下的各个服装区域分配概率分布信息，获得注意力分布更多的区域及相关度更高的区域，并采用改进的损失函数校正边界伪影，对该区域进行风格迁移得到所需的风格迁移服装图像。实验结果表明，与CNN、FCN、BeautyGAN图像局部风格迁移方法相比，该方法不仅可以突出服装图像局部风格迁移效果，而且增强了图像细节，有利于提高输出图像的真实性和艺术性。

关键词: 图像风格迁移, 边界伪影, 注意力机制, 循环生成对抗网络, 损失函数

Abstract: For the clothing images with complex backgrounds,it is difficult to control the style transfer in local areas of the images,and boundary artifacts are easily generated in this process.To address the problem,a CycleGAN-based method using attention mechanism is proposed for local style transfer of clothing images.The method employs VGG16 to extract the content features and style features from the clothing images separately,and the features are subsequently input into the CycleGAN generator based on the attention mechanism.The attention mechanism is used to distribute the probability distribution information to each clothing area of the complex background to obtain the area with more attention distribution and the area with higher correlation.Then an improved loss function is used to correct boundary artifacts.Finally,style transfer is performed on the area to obtain the desired style transfer clothing image.The experimental results show that compared with the local style transfer methods based on CNN,FCN and BeautyGAN,the proposed method can not only highlight the effect of local style transfer of clothing images,but also enhance the image details,which is conducive to improving the authenticity and artistry of the output images.

Key words: image style transfer, boundary artifacts, attention mechanism, Cycle Generative Adversarial Network(CycleGAN), loss function

中图分类号:

TP391.41

陈佳, 董学良, 梁金星, 何儒汉. 基于注意力机制的CycleGAN服装局部风格迁移研究[J]. 计算机工程, 2021, 47(11): 305-312.

CHEN Jia, DONG Xueliang, LIANG Jinxing, HE Ruhan. Research on the Local Style Transfer of Clothing Images by CycleGANBased on Attention Mechanism[J]. Computer Engineering, 2021, 47(11): 305-312.

https://www.ecice06.com/CN/Y2021/V47/I11/305

图/表 11

20211113145926

20211113145929

20211113145932

20211113145935

20211113145938

20211113145941

20211113145943

20211113145947

20211113145950

20211113145952

20211113145955

参考文献

[1] GATYS L A,ECKER A S,BETHGE M.Image style transfer using convolutional neural networks[C]//Proceedings of Computer Vision and Pattern Recognition Conference.Washington D.C.,USA:IEEE Press,2016:234-245.
[2] EFROS A A,FREEMAN W T.Image quilting for texture synthesis and transfer[C]//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques.New York,USA:ACM press,2001:341-346.
[3] HUANG X,BELONGIE S.Arbitrary style transfer in real-time with adaptive instance normalization[C]//Proceedings of International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:2380-7504.
[4] HERTZMANN A,JACOBS C E,OLIVER N,et al.Image analogies[C]//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques.New York,USA:ACM Press,2001:327-340.
[5] 钱小燕,肖亮,吴慧中.快速风格迁移[J].计算机工程,2006,32(21):15-17. QIAN X Y,XIAO L,WU H Z.Fast style transfer[J].Computer Engineering,2006,32(21):15-17.(in Chinese)
[6] LECUN Y,BENGIO Y,HINTON G.Deep learning[J].Nature,2015,521(7553):436-445.
[7] GATYS L A,ECKER A S,BETHGE M.A neural algorithm of artistic style[J].Journal of Vision,2015,22(11):54-65.
[8] RADFORD A,METZ L,CHINTALA S.Unsupervised representation learning with deep convolutional generative adversarial networks[J].Computer Ence,2015,22(11):524-535.
[9] ZHU J Y,PARK T,ISOLA P,et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:334-354.
[10] LUAN F,PARIS S,SHECHTMAN E,et al.Deep photo style transfer[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:1234-1254.
[11] WANGJ,MA Y,ZHANG L,et al.Deep learning for smart manufacturing:methods and applications[J].Journal of Manufacturing Systems,2018,34(2):144-156.
[12] GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial networks[J].Advances in Neural Information Processing Systems,2014,3:2672-2680.
[13] 李婷婷,胡玉龙,魏枫林.基于GAN改进的人脸表情识别算法及应用[J].吉林大学学报(理学版),2020,243(3):163-168. LI T T,HU Y L,WEI F L.Improved facial expression recognition algorithm based on GaN and its application[J].Journal of Jilin University(Science Edition),2020,243(3):163-168.(in Chinese)
[14] WEN L,ZHANG,Y H LIU,C.et al.Ranksrgan:generative adversarial networks with ranker forimage[EB/OL].[2020-09-01].https://arxiv.org/abs/1908.06382
[15] 张德园,常云翔,张利国,等.SAT CNN:基于卷积神经网络的遥感图像分类算法[J].小型微型计算机系统,2018,39(4):859-864. ZHANG D Y,CHANG Y X,ZHANG L G,et al.SAT-CNN:convolutional neural network framework for remote sensing image classification[J].Journal of Chinese Computer Systems,2018,39(4):859-864.(in Chinese)
[16] 缪永伟,李高怡,鲍陈,等.基于卷积神经网络的图像局部风格迁移[J].计算机科学,2019,46(9):259-264. MIAO Y W,LI G Y,BAO C,et al.Image localized style transfer based on convolutional neural network[J].Computer Science,2019,46(9):259-264.(in Chinese)
[17] 刘哲良,朱玮,袁梓洋.结合全卷积网络与CycleGAN的图像实例风格迁移[J].中国图象图形学报,2019,24(8):1283-1291. LIU Z L,ZHU W,YUAN Z Y.Image instance style transfer combined with fully convolutional network and CycleGAN[J].Journal of Image and Graphics,2019,24(8):1283-1291.(in Chinese)
[18] LI C,WAND M.Combining markov random fields and convolutional neural networks for image synthesis[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:2479-2486.
[19] CHAMPANDARD A J.Semantic style transfer and turning two-bit doodles into fine artworks[EB/OL].[2020-09-01].https://arxiv.org/abs/1603.01768.
[20] TING T L,RUI H Q,CHAO D,et al. Beautygan:instance level facial makeup transfer with deep generative adversarial network[C]//Proceedings of Multi-media Conference.New York,USA:ACM Press,2018:645-653.
[21] GU S,BAO J,YANG H,et al.Mask-guided portrait editing with conditional gans[C]//Proceedings of the Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:3436-3445.
[22] 林泓,任硕,杨益,等.融合自注意力机制和相对鉴别的无监督图像翻译[J].自动化学报,2019,17(3):1-10. LIN H,REN S,YANG Y,etal.Unsupervised image-to-image translation with self-attention and relativistic discriminator adversarial networks[J].ACTA Automatica Sinica 2019,17(3):1-10.(in Chinese)
[23] CHOI M,KIM H,HAN B,et al.Channel attention is all you need for video frame interpolation[C]//Proceedings of the Conference on Artificial Intelligence.Washington D.C.,USA:IEEE Press,2020:10663-10671.
[24] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.

选择文件类型/文献管理软件名称

选择包含的内容

基于注意力机制的CycleGAN服装局部风格迁移研究

Research on the Local Style Transfer of Clothing Images by CycleGANBased on Attention Mechanism

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究[J]. 计算机工程, 2024, 50(9): 82-91.
[2]	党小超, 刘涧, 董晓辉, 祝忠彦, 李芬芳. 面向不平衡数据的机械设备故障命名实体识别[J]. 计算机工程, 2024, 50(9): 104-112.
[3]	林畅, 郭伟, 任哲聪, 金海波. 基于Transformer的目标跟踪与分割统一算法[J]. 计算机工程, 2024, 50(9): 130-141.
[4]	李泽霖, 吕兆峰, 陈富强, 李克. 基于多跳信息融合的实体对齐模型[J]. 计算机工程, 2024, 50(9): 142-152.
[5]	王汝英, 马嘉骏, 董建强, 刘万龙, 张海涛, 尹凯, 赵博超. 基于MTS-BiGRU-DMHSA的工业负荷预测方法[J]. 计算机工程, 2024, 50(9): 169-178.
[6]	朱凯, 李理, 张彤, 江晟, 别一鸣. 基于Transformer的多阶段运动模糊图像修复网络[J]. 计算机工程, 2024, 50(9): 276-285.
[7]	张天鹏, 韩晶, 吕学强. 基于多任务学习的超分辨率辅助小目标检测[J]. 计算机工程, 2024, 50(9): 304-312.
[8]	郭敏, 张熙涵, 李阳. 融合注意力的教师互一致性半监督医学图像分割[J]. 计算机工程, 2024, 50(9): 313-323.
[9]	曾钰琦, 刘博, 钟柏昌, 钟瑾. 智慧教育下基于改进YOLOv8的学生课堂行为检测算法[J]. 计算机工程, 2024, 50(9): 344-355.
[10]	饶日昕, 王怡文, 曾砺志, 童心恬, 赵海涛. 面向废旧电缆检测的轻量化网络模型[J]. 计算机工程, 2024, 50(8): 22-30.
[11]	李华昱, 张智康, 闫阳, 岳阳. 基于知识图谱增强的领域多模态实体识别[J]. 计算机工程, 2024, 50(8): 31-39.
[12]	王蕾, 党时鹏, 潘丰. 基于卷积神经网络的隐匿性旁路预测模型[J]. 计算机工程, 2024, 50(8): 40-49.
[13]	陈瀚, 赵春蕾, 蒋昊达, 王春东. 基于融合模型与语义网络的App用户意图识别研究[J]. 计算机工程, 2024, 50(8): 50-63.
[14]	高爽, 史轶伦, 徐巧枝, 于磊. 基于对比学习的非对称编解码结构的心脏MRI分割研究[J]. 计算机工程, 2024, 50(8): 290-300.
[15]	王夙喆, 张雪英, 陈晓玉, 李凤莲, 吴泽林. 基于有效注意力和GAN结合的脑卒中EEG增强算法[J]. 计算机工程, 2024, 50(8): 336-344.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于注意力机制的CycleGAN服装局部风格迁移研究

Research on the Local Style Transfer of Clothing Images by CycleGANBased on Attention Mechanism

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献

相关文章 15

编辑推荐

Metrics

本文评价