混洗门控注意力与通道对齐策略协同优化的轻量级真实图像风格迁移

doi:10.19678/j.issn.1000-3428.0069830

计算机工程 ›› 2026, Vol. 52 ›› Issue (2): 197-208. doi: 10.19678/j.issn.1000-3428.0069830

• 计算机视觉与图形图像处理 • 上一篇下一篇

混洗门控注意力与通道对齐策略协同优化的轻量级真实图像风格迁移

刘惠临, 方琼, 王燕思, 张顺香*(), 苏树智

安徽理工大学计算机科学与工程学院, 安徽淮南 232001

收稿日期:2024-05-09 修回日期:2024-08-23 出版日期:2026-02-15 发布日期:2024-10-30
通讯作者: 张顺香
作者简介:
刘惠临(CCF会员)，高级实验师、博士，主研方向为计算机视觉、图像处理、多模态数据融合
方琼，硕士研究生
王燕思，硕士研究生
张顺香(通信作者)，教授、博士、博士生导师
苏树智，副教授、博士
基金资助:
国家自然科学基金(52374155); 国家自然科学基金(62102003); 安徽省自然科学基金(2308085MF218); 安徽省智能感知与健康养老工程研究中心2022年度开放课题(2022OPB01)

Lightweight Photorealistic Image Style Transfer with Collaborative Optimization of Shuffle Gate Attention and Channel Alignment Whitening and Coloring Transform

LIU Huilin, FANG Qiong, WANG Yansi, ZHANG Shunxiang*(), SU Shuzhi

School of Computer Science and Engineering, Anhui University of Science and Technology, Huainan 232001, Anhui, China

Received:2024-05-09 Revised:2024-08-23 Online:2026-02-15 Published:2024-10-30
Contact: ZHANG Shunxiang

摘要/Abstract

摘要：

现有的真实图像风格迁移算法在追求提升图像的真实感和风格化强度的同时, 通常未充分考虑算法模型尺寸和计算效率问题, 因此很难适用于低算力设备。为解决这一问题, 提出一种轻量级真实图像风格迁移算法。使用ShuffleNet V2轻量级网络替代VGG19作为特征提取器, 并引入块式训练和跳跃连接技术, 旨在大幅度减少参数量, 提高图像的风格迁移速度。同时, 为了更好地平衡迁移图像的内容和风格, 设计混洗门控通道注意力机制(SGCAM)和通道对齐策略(CAWCT)。SGCAM将通道混洗和门控机制巧妙结合, 不仅增强了生成图像的真实感, 还进一步保持了算法轻量化的优势。CAWCT通过引入二值化操作对白化后的内容特征和风格特征进行相似性匹配, 显著提升了生成图像的风格化强度。实验结果表明, 所提算法的参数量仅为PhotoWCT²的14.8%, 迁移一张1 000×750像素的图像只需4.22 s, 比PhotoWCT²少0.79 s, 同时生成图像的质量和风格化强度均得到明显提升, 结构相似性(SSIM)和峰值信噪比(PSNR)指标分别提高0.031 dB和0.066 dB, 内容损失(Content loss)、Gram损失(Gram loss)和风格损失(Style loss)指标分别降低0.227、0.138×10^-5和0.116。

关键词: 风格迁移, 轻量级, ShuffleNet V2, 通道注意力, 相似性匹配

Abstract:

Existing photorealistic image style transfer algorithms typically do not fully consider the issues of algorithm model size and computational efficiency while pursuing improvements in image realism and stylization intensity. Therefore, applying these methods to low computing power devices is difficult. To address this issue, this study proposes a lightweight real image style transfer algorithm. The VGG19 is replaced with the ShuffleNet V2 lightweight network as the feature extractor, with block-wise training and skip-connection techniques introduced to significantly reduce the number of parameters and improve the speed of image style transfer. To better balance the content and style of the transferred images, the study also proposes a Shuffle Gated Channel Attention Mechanism (SGCAM) and Channel Alignment Whitening and Coloring Transform (CAWCT). SGCAM combines channel shuffling with gating mechanisms efficiently, which not only enhances the realism of generated images but also further maintains the advantage of the lightweight algorithm. CAWCT significantly boosts the stylization intensity of the generated images by introducing binary operations to match the whitened content features and style features for similarity. Experimental results show that the parameter size of the proposed algorithm is only 14.8% of that of PhotoWCT². It takes only 4.22 s to transfer an image with a resolution of 1 000×750 pixel, which is 0.79 s faster than that achieved by PhotoWCT². Simultaneously, the quality and stylization strength of the generated images are significantly improved. In performance evaluations, the Structural Similarity Index Measure (SSIM) and Peak Signal-to-Noise Ratio (PSNR) indicators increase by 0.031 dB and 0.066 dB, respectively, while the Content loss, Gram loss, and Style loss metrics decrease by 0.227, 0.138×10^-5, and 0.116, respectively.

Key words: style transfer, lightweight, ShuffleNet V2, channel attention, similarity match

刘惠临, 方琼, 王燕思, 张顺香, 苏树智. 混洗门控注意力与通道对齐策略协同优化的轻量级真实图像风格迁移[J]. 计算机工程, 2026, 52(2): 197-208.

LIU Huilin, FANG Qiong, WANG Yansi, ZHANG Shunxiang, SU Shuzhi. Lightweight Photorealistic Image Style Transfer with Collaborative Optimization of Shuffle Gate Attention and Channel Alignment Whitening and Coloring Transform[J]. Computer Engineering, 2026, 52(2): 197-208.

https://www.ecice06.com/CN/Y2026/V52/I2/197

图/表 14

图1 本文算法整体结构

Fig.1 The overall structure of the proposed algorithm

图2 Encblk3和Encblk4结构单元

Fig.2 Encblk3 and Encblk4 structural unit

图3 SGCAM结构

Fig.3 SGCAM structure

图4 CAWCT结构

Fig.4 CAWCT structure

图5 本文算法与其他算法效果比较

Fig.5 Comparison of the effectiveness of the proposed algorithm with other algorithms

图6 不同分辨率下真实图像风格迁移算法生成图像对比

Fig.6 Comparison of images generated by photorealistic image style transfer algorithms at different resolutions

图7 本文算法与艺术图像风格迁移算法效果比较

Fig.7 Comparison of the effect of the proposed algorithm with artistic image style transfer algorithms

图8 SGCAM的消融实验结果

Fig.8 Ablation experimental results of SGCAM

图9 整体消融实验结果

Fig.9 Overall ablation experimental results

参考文献 25

1	GATYS L A, ECKER A S, BETHGE M. Image style transfer using convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE Press, 2016: 2414-2423.
2	DENG Y Y, TANG F, DONG W M, et al. StyTr2: image style transfer with transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE Press, 2022: 11316-11326.
3	毛琳, 王萌, 杨大伟. 内容特征一致性风格迁移网络. 计算机辅助设计与图形学学报, 2022, 34(6): 892- 900.
	MAO L, WANG M, YANG D W. Content consistency preserving style transfer network. Journal of Computer-Aided Design & Computer Graphics, 2022, 34(6): 892- 900.
4	CHIANG P-Z, TSAI M S, TSENG H Y, et al. Stylizing 3D scene via implicit representation and HyperNetwork[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE Press, 2022: 215-224.
5	WU J Y, HOU L F, LI Z J, et al. Preserving structural consistency in arbitrary artist and artwork style transfer. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(3): 2830- 2838. doi: 10.1609/aaai.v37i3.25384
6	LUAN F J, PARIS S, SHECHTMAN E, et al. Deep photo style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE Press, 2017: 6997-7005.
7	LI Y J, LIU M Y, LI X T, et al. A closed-form solution to photorealistic image stylization[C]//Proceedings of ECCV 2018. Berlin, Germany: Springer, 2018: 468-483.
8	张颖涛, 张杰, 张睿, 等. 全局信息引导的真实图像风格迁移. 计算机科学, 2022, 49(7): 100- 105.
	ZHANG Y T, ZHANG J, ZHANG R, et al. Photorealistic style transfer guided by global information. Computer Science, 2022, 49(7): 100- 105.
9	GUNAWAN A, KIM S Y, SIM H, et al. Modernizing old photos using multiple references via photorealistic style transfer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada: IEEE Press, 2023: 12460-12469.
10	WU Z J, ZHU Z, DU J P, et al. CCPL: contrastive coherence preserving loss for versatile style transfer[C]//Proceedings of ECCV 2022. Berlin, Germany: Springer, 2022: 189-206.
11	CHIU T Y, GURARI D. Line search-based feature transformation for fast, stable, and tunable content-style control in photorealistic style transfer[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE Press, 2023: 249-258.
12	YOO J, UH Y, CHUN S, et al. Photorealistic style transfer via wavelet transforms[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea: IEEE Press, 2019: 9035-9044.
13	AN J, XIONG H Y, HUAN J, et al. Ultrafast photorealistic style transfer via neural architecture search[C]//Proceedings of the AAAI Conference on Artificial Intelligence. [S. l. ]: AAAI Press, 2020: 10443-10450.
14	CHIU T Y, GURARI D. PhotoWCT²: compact autoencoder for photorealistic style transfer resulting from blockwise training and skip connections of high-frequency residuals[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE Press, 2022: 2978-2987.
15	AN J, LI T, HUANG H Z, et al. Is bigger always better? An empirical study on efficient architectures for style transfer and beyond[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE Press, 2023: 4073-4083.
16	吕文锐, 普园媛, 赵征鹏, 等. 基于字形约束和注意力的艺术字体风格迁移. 计算机工程, 2024, 50(12): 306- 317. doi: 10.19678/j.issn.1000-3428.0068380
	LV W R, PU Y Y, ZHAO Z P, et al. Artistic font style transfer based on glyph constraints and attention. Computer Engineering, 2024, 50(12): 306- 317. doi: 10.19678/j.issn.1000-3428.0068380
17	PARK D Y, LEE K H. Arbitrary style transfer with style-attentional networks[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA: IEEE Press, 2019: 5873-5881.
18	LIU S H, LIN T W, HE D L, et al. AdaAttN: revisit attention mechanism in arbitrary neural style transfer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada: IEEE Press, 2021: 6629-6638.
19	ZHU M R, HE X, WANG N N, et al. All-to-key attention for arbitrary style transfer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Paris, France: IEEE Press, 2023: 23052-23062.
20	HONG K, JEON S, LEE J, et al. AesPA-Net: aesthetic pattern-aware style transfer networks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Paris, France: IEEE Press, 2023: 22701-22710.
21	GE B, HU Z S, XIA C X, et al. Arbitrary style transfer method with attentional feature distribution matching. Multimedia Systems, 2024, 30(2): 96. doi: 10.1007/s00530-024-01300-4
22	MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[C]//Proceedings of ECCV 2018. Berlin, Germany: Springer, 2018: 122-138.
23	HU J, SHEN L, SUN G. Squeeze-and-Excitation networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, USA: IEEE Press, 2018: 7132-7141.
24	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]//Proceedings of ECCV 2014. Berlin, Germany: Springer, 2014: 740-755.
25	MA Y N, ZHAO C Q, BASU A, et al. RAST: restorable arbitrary style transfer via multi-restoration[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE Press, 2023: 331-340.

[1]	许晓阳, 魏伟, 高重阳. 基于改进YOLOv7-tiny的红外船舶目标检测[J]. 计算机工程, 2026, 52(2): 209-220.
[2]	陈诗航, 孙玉宝. 基于模态仿射融合的语音控制说话人脸视频对抗生成[J]. 计算机工程, 2026, 52(2): 393-403.
[3]	刘春霞, 孟吉星, 潘理虎, 龚大立. 融合RGB与IR图像的遥感小目标检测方法[J]. 计算机工程, 2025, 51(7): 326-338.
[4]	张佳承, 韦锦, 陈义时. 改进YOLOv8的实时轻量化鲁棒绿篱检测算法[J]. 计算机工程, 2025, 51(7): 362-374.
[5]	丁帅, 况立群, 曹亚明, 韩慧妍, 熊风光. 时空特征融合的高精度轻量级骨架行为识别[J]. 计算机工程, 2025, 51(11): 283-293.
[6]	胡涌涛, 黄洪琼. 结合特征融合和通道注意力的多分支换装行人重识别[J]. 计算机工程, 2025, 51(1): 225-234.
[7]	蔡俊民, 梁正友, 孙宇, 陈子奥. 基于可变形三维图卷积的轻量级点云分类研究[J]. 计算机工程, 2024, 50(9): 255-265.
[8]	杨郅树, 梁佳楠, 曹永军, 钟震宇, 何永伦. 基于局部分离与多尺度融合的图像超分辨率重建[J]. 计算机工程, 2024, 50(7): 314-323.
[9]	李致金, 汤佳辉, 闫金凤. 基于边缘计算的轻量化识别方法[J]. 计算机工程, 2024, 50(6): 287-295.
[10]	杨硕, 王一丁. 基于改进薄板样条运动模型的人脸动画算法[J]. 计算机工程, 2024, 50(6): 255-265.
[11]	李振鲁, 黄威, 孙锴. 复杂环境下的轻量化道路目标识别算法研究[J]. 计算机工程, 2024, 50(4): 219-227.
[12]	刘彦红, 杨秋翔, 胡帅. 基于特征差异的多尺度特征融合去雾网络研究[J]. 计算机工程, 2024, 50(4): 247-257.
[13]	冯妍舟, 刘建霞, 王海翼, 冯国昊, 白宇. 基于多级残差信息蒸馏的真实图像去噪方法[J]. 计算机工程, 2024, 50(3): 216-223.
[14]	吕文锐, 普园媛, 赵征鹏, 张衡, 阳秋霞. 基于字形约束和注意力的艺术字体风格迁移[J]. 计算机工程, 2024, 50(12): 306-317.
[15]	赵健, 崔骞, 石佳, 刘岳. 基于文本和声学特征的双模态融合抑郁倾向识别算法[J]. 计算机工程, 2024, 50(11): 49-58.

选择文件类型/文献管理软件名称

选择包含的内容

混洗门控注意力与通道对齐策略协同优化的轻量级真实图像风格迁移

Lightweight Photorealistic Image Style Transfer with Collaborative Optimization of Shuffle Gate Attention and Channel Alignment Whitening and Coloring Transform

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 25

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

混洗门控注意力与通道对齐策略协同优化的轻量级真实图像风格迁移

Lightweight Photorealistic Image Style Transfer with Collaborative Optimization of Shuffle Gate Attention and Channel Alignment Whitening and Coloring Transform

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 25

相关文章 15

编辑推荐

Metrics

本文评价