基于注意力残差网络的人脸超分辨率重建

doi:10.19678/j.issn.1000-3428.0064830

摘要/Abstract

摘要： 为解决通道内部特征信息交互性不足、特征利用和表示不够充分导致的人脸面部细节信息恢复不理想的问题，提出一种基于编码器-解码器的注意力残差网络，并设计基于注意力的残差模块，其主要由基准残差模块、沙漏模块与内部特征拆分注意力模块组成，通过内部特征拆分注意力模块加强通道内部之间的交互性，使网络能够提取到更详细的特征信息，恢复出更多人脸面部细节，同时在残差模块中利用一个预激活模块，解决批量归一化层在超分辨率网络中存在的伪影问题。在特征提取单元末端运用多阶特征融合模块充分融合多个阶段的特征，缓解特征在网络传输过程中的丢失现象，提高特征利用率。实验结果表明，该方法可以恢复出更多人脸面部细节，在Helen人脸数据集上，重建人脸图像的PSNR值为27.74 dB，相比SISN和DICNet方法，分别提高了1.47 dB、1.12 dB。在CelebA人脸数据集上，重建人脸图像的PSNR值为27.40 dB，相比SISN和DICNet方法，分别提高了1.26 dB、0.39 dB。

关键词: 人脸超分辨率, 注意力机制, 残差网络, 特征融合, 编码器, 解码器

Abstract: Insufficient interactivity of feature information inside the channel and insufficient feature utilization and representation leads to less than ideal recovery of facial information.To address the problem，this paper proposes an attentional residual network based on encoder-decoder.In particular，it proposes a new Residual Attention Block（RAB），which mainly consists of a baseline residual block，an hourglass block，and an internal-feature split attention block used to strengthen the interactivity between channel interiors.This enables the network to extract more detailed feature information and recover more facial details.In addition，in the proposed residual block，a pre-activation block is used to solve the artifact problem that occurs in the Batch Normalization（BN） layer in super-resolution networks.Finally，a multi-stage feature fusion block is used at the end of the feature extraction unit to fully fuse the features obtained at different stages，which alleviates the feature loss during network transmission and improves the feature utilization.Experimental results show that the proposed method can recover more facial details.On the Helen face dataset，the PSNR value of face image reconstructed by this algorithm is 27.74 dB，which is 1.47 dB and 1.12 dB higher than that of SISN and DICNet methods，respectively.Similarly，on the CelebA face dataset，the PSNR value of face image reconstructed by the proposed algorithm is 27.40 dB，which is 1.26 dB and 0.39 dB higher than that of SISN and DICNet methods，respectively.

Key words: face super-resolution, attention mechanism, residual network, feature fusion, encoder, decoder

中图分类号:

TP391.41

王同官, 赖惠成, 蔡玉玺, 高古学, 汪烈军. 基于注意力残差网络的人脸超分辨率重建[J]. 计算机工程, 2023, 49(6): 234-241.

WANG Tongguan, LAI Huicheng, CAI Yuxi, GAO Guxue, WANG Liejun. Face Super-Resolution Reconstruction Based on Attention Residual Network[J]. Computer Engineering, 2023, 49(6): 234-241.

https://www.ecice06.com/CN/Y2023/V49/I6/234

图/表 16

20230615170336

20230615170340

20230615170343

20230615170346

20230615170349

20230615170353

20230615170356

20230615170400

20230615170404

20230615170408

20230615170411

20230615170414

20230615170418

20230615170421

20230615170425

20230615170431

参考文献

[1] BAKER S,KANADE T.Hallucinating faces[C]//Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition.Washington D.C.,USA:IEEE Press,2002:83-88.
[2] CHEN Y,TAI Y,LIU X M,et al.FSRNet:end-to-end learning face super-resolution with facial priors[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:2492-2501.
[3] MA C,JIANG Z Y,RAO Y M,et al.Deep face super-resolution with iterative collaboration between attentive recovery and landmark estimation[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2020:5568-5577.
[4] YU X Y,ZHANG L W,XIE W.Semantic-driven face hallucination based on residual network[J].IEEE Transactions on Biometrics,Behavior,and Identity Science,2021,3(2):214-228.
[5] 袁健,李佳慧.融合先验信息的残差空间注意力人脸超分辨率重建模型[J/OL].小型微型计算机系统:1-8[2022-02-26].https://kns.cnki.net/kcms/detail/21.1106.TP.20220225.1355.010.html.YUAN J,LI J H.Residual spatial attention face super resolution algorithm based on prior information-fusion[J/OL].Journal of Chinese Computer Systems:1-8[2022-02-26].https://kns.cnki.net/kcms/detail/21.1106.TP.20220225.1355.010.html.(in Chinese)
[6] LU T,WANG J M,JIANG J J,et al.Global-local fusion network for face super-resolution[J].Neurocomputing,2020,387:309-320.
[7] WANG Y,LU T,ZHANG Y,et al.TANet:a new paradigm for global face super-resolution via transformer-CNN aggregation network[EB/OL].[2022-04-20].https://arxiv.org/abs/2109.08174.
[8] JIANG K,WANG Z Y,YI P,et al.Dual-path deep fusion network for face image hallucination[J].IEEE Transactions on Neural Networks and Learning Systems,2022,33(1):378-391.
[9] CHEN C F,GONG D H,WANG H,et al.Learning spatial attention for face super-resolution[J].IEEE Transactions on Image Processing,2021,30(1):1219-1231.
[10] HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:770-778.
[11] KIM J,LEE J K,LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1646-1654.
[12] ZHANG Y L,TIAN Y P,KONG Y,et al.Residual dense network for image super-resolution[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:2472-2481.
[13] LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2017:1132-1140.
[14] LIU J,TANG J,WU G S.Residual feature distillation network for lightweight image super-resolution[C]//Proceedings of ECCVʼ20.Berlin,Germany:Springer,2020:41-55.
[15] HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:7132-7141.
[16] WOO S,PARK J,LEE J Y,et al.CBAM:convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision.Berlin,Germany:Springer,2018:3-19.
[17] ZHANG H,WU C,ZHANG Z,et al.ResNeSt:split-attention networks[EB/OL].[2022-02-26].https://arxiv.org/abs/2004.08955.
[18] 王诗言,曾茜,周田,等.基于注意力机制与特征融合的图像超分辨率重建[J].计算机工程,2021, 47(3):269-275,283.WANG S Y,ZENG X,ZHOU T,et al.Image super-resolution reconstruction based on attention mechanism and feature fusion[J].Computer Engineering,2021,47(3):269-275,283.(in Chinese)
[19] 陈乔松,蒲柳,张羽,等.结合整体注意力与分形稠密特征的图像超分辨率重建[J].计算机工程,2022,48(11):207-214,223.CHEN Q S,PU L,ZHANG Y,et al.Image super-resolution reconstruction combining holistic attention and fractal density feature[J].Computer Engineering,2022,48(11):207-214,223.(in Chinese)
[20] 鲁甜,刘蓉,刘明,等.基于特征图注意力机制的图像超分辨率重建[J].计算机工程,2021,47(3):261-268.LU T,LIU R,LIU M,et al.Image super-resolution reconstruction based on attention mechanism of feature map[J].Computer Engineering,2021,47(3):261-268.(in Chinese)
[21] LI J C,FANG F M,MEI K F,et al.Multi-scale residual network for image super-resolution[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2018:527-542.
[22] HE K M,ZHANG X Y,REN S Q,et al.Identity mappings in deep residual networks[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2016:630-645.
[23] NEWELL A,YANG K Y,DENG J.Stacked hourglass networks for human pose estimation[C]//Proceedings of the 15th European Conference on Computer Vision.Berlin,Germany:Springer,2016:483-499.
[24] LU T,WANG Y Z,ZHANG Y D,et al.Face hallucination via split-attention in split-attention network[C]//Proceedings of the 29th ACM International Conference on Multimedia.New York,USA:ACM Press,2021:5501-5509.
[25] LIU Z W,LUO P,WANG X G,et al.Deep learning face attributes in the wild[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2016:3730-3738.
[26] ZHANG K P,ZHANG Z P,LI Z F,et al.Joint face detection and alignment using multitask cascaded convolutional networks[J].IEEE Signal Processing Letters,2016,23(10):1499-1503.
[27] LE V,BRANDT J,LIN Z,et al.Interactive facial feature localization[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2012:679-692.

选择文件类型/文献管理软件名称

选择包含的内容