结合双注意力和结构相似度量的图像超分辨率重建网络

黄友文; 唐欣; 周斌

doi:10.37188/CJLCD.2021-0178

您当前的位置：

首页 >

文章列表页 >

结合双注意力和结构相似度量的图像超分辨率重建网络

图像处理 | 更新时间：2022-03-25

- 结合双注意力和结构相似度量的图像超分辨率重建网络
- Image super-resolution reconstruction network with dual attention and structural similarity measure
- 液晶与显示 2022年37卷第3期页码：367-375
- 作者机构：
  
  江西理工大学信息工程学院，江西赣州 341000
- 作者简介：
  
  [ "黄友文(1982-)，江西赣州人，博士，副教授，2009年于同济大学获得博士学位，研究方向为智能信息处理、嵌入式应用和图像处理。E-mail：ywhuang@jxust.edu.cn" ]
  [ "唐欣(1996-)女，湖北黄石人，硕士研究生，2018年于武汉工程大学邮电与信息工程学院获得学士学位，研究方向为图像处理。E-mail：tangxin.2108@163.com" ]
- 基金信息：
  
  江西省教育厅科技项目(GJJ180443)
- DOI：10.37188/CJLCD.2021-0178
  中图分类号： TP391.4
- 收稿日期：2021-07-06，
  
  修回日期：2021-09-27，
  
  纸质出版日期：2022-03
- 稿件说明：
移动端阅览
黄友文, 唐欣, 周斌. 结合双注意力和结构相似度量的图像超分辨率重建网络[J]. 液晶与显示, 2022,37(3):367-375.

You-wen HUANG, Xin TANG, Bin ZHOU. Image super-resolution reconstruction network with dual attention and structural similarity measure[J]. Chinese journal of liquid crystals and displays, 2022, 37(3): 367-375.
黄友文, 唐欣, 周斌. 结合双注意力和结构相似度量的图像超分辨率重建网络[J]. 液晶与显示, 2022,37(3):367-375. DOI： 10.37188/CJLCD.2021-0178.

You-wen HUANG, Xin TANG, Bin ZHOU. Image super-resolution reconstruction network with dual attention and structural similarity measure[J]. Chinese journal of liquid crystals and displays, 2022, 37(3): 367-375. DOI： 10.37188/CJLCD.2021-0178.

摘要

针对低分辨率图像到高分辨率图像的映射函数解空间极大，导致超分辨率重建模型难以产生细致纹理的问题，本文提出一种结合双注意力和结构相似度量的图像超分辨率重建网络。以改进的U-Net网络模型作为基本结构，引入针对低级别视觉任务的数据增强方法，增加样本多样性。编码器部分由卷积层和自适应参数线性整流激活函数(Dynamic ReLU)组成。同时引入了一种残差双注意力模块(Residual Dual Attention Module，RDAM)与像素重组(PixelShuffle)模块共同构成解码器，通过上采样操作，逐级放大图像。为了使生成图像更加符合人眼视觉特性，采用了一种结合结构相似度量准则的损失函数，增强网络约束。实验结果表明：重建图像的质量对比SRCNN，在Set5、Set14、BSD100和Urban100标准测试集上的平均PSNR提升约1.64 dB，SSIM提升约0.047。本文方法能够使重建的图像纹理细节更丰富，有效地减少了映射函数可能的解空间。

Abstract

Aiming at the problem that the solution space of mapping function from low resolution image to high resolution image is extremely large

which makes it difficult for super-resolution reconstruction models to generate detailed textures

this paper proposes a image super resolution that combines dual attention and structural similarity measure. With the improved U-Net network model as the basic structure

the data augmentation methods for low-level vision tasks are introduced to increase sample diversity. The encoder is composed of a convolution layer and an adaptive parameter linear rectifier function (Dynamic ReLU). At the same time

a residual dual attention module(RDAM) is introduced

which forms a decoder together with the PixelShuffle module. The image is enlarged gradually through the up-sampling operation. In order to make the generated image more in line with the human visual characteristics

a loss function combined with structural similarity measurement criteria is proposed to enhance the network constraints. The experimental results show that the average PSNR of the quality of the reconstructed image on the Set5

Set14

BSD100 and Urban100 standard test sets is improved by about 1.64 dB

and the SSIM is improved by about 0.047 compared with SRCNN.The proposed method can make the reconstructed image texture more detailed and reduce the possible solution space of the mapping function effectively.

关键词

Keywords

references

陈宗航 , 胡海龙 , 姚剑敏 , 等 . 基于改进生成对抗网络的单帧图像超分辨率重建 . 液晶与显示 , 2021 . 36 ( 5 ): 705 - 712 . DOI: 10.37188/CJLCD.2020-0250 http://doi.org/10.37188/CJLCD.2020-0250 .

Z H CHEN , H L HU , J M YAO , 等 . Single frame image super-resolution reconstruction based on improved generative adversarial network . Chinese Journal of Liquid Crystals and Displays , 2021 . 36 ( 5 ): 705 - 712 . DOI: 10.37188/CJLCD.2020-0250 http://doi.org/10.37188/CJLCD.2020-0250 .

C DONG , C C LOY , K M HE , 等 . Image super-resolution using deep convolutional networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2016 . 38 ( 2 ): 295 - 307 . DOI: 10.1109/TPAMI.2015.2439281 http://doi.org/10.1109/TPAMI.2015.2439281 .

KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas: IEEE, 2016: 1646-1654.

KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR ). Las Vegas: IEEE, 2016: 1637-1645.

LAI W S, HUANG J B, AHUJA N, et al . Deep Laplacian pyramid networks for fast and accurate super-resolution [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR ). Honolulu: IEEE, 2017: 5835-5843.

LIM B, SON S, KIM H, et al . Enhanced deep residual networks for single image super-resolution [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops . Honolulu: IEEE, 2017: 1132-1140.

ZHANG Y L, TIAN Y P, KONG Y, et al . Residual dense network for image super-resolution [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 2472-2481.

ZHANG Y L, LI K P, LI K, et al . Image super-resolution using very deep residual channel attention networks [C]// Proceedings of 15 th European Conference on Computer Vision . Munich: Springer, 2018: 294-310.

LEDIG C, THEIS L, HUSZÁR F, et al . Photo-realistic single image super-resolution using a generative adversarial network [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR ). Honolulu: IEEE, 2017: 105-114.

WANG X T, YU K, WU S X, et al . ESRGAN: enhanced super-resolution generative adversarial networks [C]// Proceedings of European Conference on Computer Vision . Munich: Springer, 2019: 63-79.

XIAO M Q, ZHENG S X, LIU C, et al . Invertible image rescaling [C]//16 th European Conference on Computer Vision . Glasgow: Springer, 2020: 126-144.

RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation [C]//18 th International Conference on Medical Image Computing and Computer-Assisted Intervention . Munich: Springer, 2015: 234-241.

YUN S, HAN D, CHUN S, et al . CutMix: regularization strategy to train strong classifiers with localizable features [C]//2019 IEEE/CVF International Conference on Computer Vision (I CCV ). Seoul: IEEE, 2019: 6022-6031.

YOO J, AHN N, SOHN K A. Rethinking data augmentation for image super-resolution: a comprehensive analysis and a new strategy [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 8372-8381.

GUO Y, CHEN J, WANG J D, et al . Closed-loop matters: dual regression networks for single image super-resolution [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 5406-5415.

HE K M, ZHANG X Y, REN S Q, et al . Deep residual learning for image recognition [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas: IEEE, 2016: 770-778.

WOO S, PARK J, LEE J Y, et al . CBAM: convolutional block attention module [C]// Proceedings of 15 th European Conference on Computer Vision . Munich: Springer, 2018: 3-19.

CHEN Y P, DAI X Y, LIU M C, et al . Dynamic ReLU [EB/OL]. [2020-05-12] . https://arxiv.org/abs/2003.10027 https://arxiv.org/abs/2003.10027 .

S ANWAR , S KHAN , N BARNES . A deep journey into super-resolution: a survey . ACM Computing Surveys , 2020 . 53 ( 3 ): 60 DOI: 10.48550/arXiv.1904.07523 http://doi.org/10.48550/arXiv.1904.07523 .

AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network [C]// Proceedings of 15 th European Conference on Computer Vision . Munich: Springer, 2018: 256-272.

HARIS M, SHAKHNAROVICH G, UKITA N. Deep back-projection networks for super-resolution [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018: 1664-1673.

LI Z, YANG J L, LIU Z, et al . Feedback network for image super-resolution [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach: IEEE, 2019: 3862-3871.

浏览量

470

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于笔划三维深度特征的签名识别

基于YOLO模型的宫颈细胞簇团智能识别方法

基于深度卷积神经网络的宫颈细胞病理智能辅助诊断方法

基于DSP系统的超分辨率图像重建技术研究