Blind super-resolution model based on degradation-aware

CAI Jian-feng; JIANG Nian-de

doi:10.37188/CJLCD.2022-0385

您当前的位置：

首页 >

文章列表页 >

Blind super-resolution model based on degradation-aware

Image Processing | 更新时间：2023-09-27

- Blind super-resolution model based on degradation-aware
- Chinese Journal of Liquid Crystals and Displays Vol. 38, Issue 9, Pages: 1224-1233(2023)
- 作者机构：
  
  东华理工大学信息工程学院，江西南昌330013
- 作者简介：
- 基金信息：
- DOI：10.37188/CJLCD.2022-0385
  CLC： TP391.4
- Received：18 November 2022，
  
  Revised：08 December 2022，
  
  Published：05 September 2023
- 稿件说明：
移动端阅览
CAI Jian-feng, JIANG Nian-de. Blind super-resolution model based on degradation-aware[J]. Chinese journal of liquid crystals and displays, 2023, 38(9): 1224-1233.
DOI：

CAI Jian-feng, JIANG Nian-de. Blind super-resolution model based on degradation-aware[J]. Chinese journal of liquid crystals and displays, 2023, 38(9): 1224-1233. DOI： 10.37188/CJLCD.2022-0385.

摘要

现有的超分辨率方法假设了从高分辨率图像到低分辨率图像的预定义退化过程，这对于具有复杂退化类型的真实世界图像来说很难成立。针对该问题，本文提出了基于退化感知的盲超分辨率模型。该模型用随机模糊核生成低分辨率图像，用对比学习退化表征。模型生成器由包含多个退化感知块的残差组构成。退化感知块用退化表征和图像特征做交叉注意力计算空间权重图，此外模型还收集残差组输出的层级特征，计算层间注意力来复用层级特征，使得模型更加关注高频细节，模型特征提取能力进一步提高。通过消融实验验证了各模块的有效性。在多个国际公开测试集上，放大倍数为4的平均PSNR提升1.45 dB，SSIM提升0.058。实验结果表明，该模型在盲超分辨率任务上取得了显著的性能，且具有良好的视觉效果。

Abstract

Existing super-resolution methods assume a predefined degradation process from high-resolution images to low-resolution images， which is difficult to hold for real-world images with complex degradation types. For this problem， a blind super-resolution model based on degradation-aware is proposed. The model generates low-resolution images with random blur kernels and learns degenerate representations with contrasts. The model generator consists of residual groups containing multiple degradation-aware blocks. Degraded perceptual blocks use degraded representations and image features to do cross-attention to calculate spatial weight maps. In addition， the model collects layer-level features from the output of residual groups and calculates inter-layer attention to reuse layer-level features. This enables the model to pay more attention to high-frequency details， and the model feature extraction capability is further improved. The effectiveness of each module is verified by ablation experiments. On multiple international public test sets， the average PSNR with a magnification of 4 is increased by 1.45 dB， and the SSIM is increased by 0.058. Experimental results show that the model achieves significant performance on blind super-resolution tasks with good visual results.

关键词

Keywords

references

毕勇，潘鸣奇，张硕，等 . 三维点云数据超分辨率技术［J］. 中国光学， 2022 ， 15 （ 2 ）： 210 - 223 ． doi: 10.37188/CO.2021-0176 http://dx.doi.org/10.37188/CO.2021-0176

BI Y ， PAN M Q ， ZHANG S ， et al . Overview of 3D point cloud super-resolution technology ［J］. Chinese Optics ， 2022 ， 15 （ 2 ）： 210 . （in Chinese） . doi: 10.37188/CO.2021-0176 http://dx.doi.org/10.37188/CO.2021-0176

DONG C ， LOY C C ， HE K ， et al . Image super-resolution using deep convolutional networks ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence ， 2016 ， 38 （ 2 ）： 295 - 307 . doi: 10.1109/tpami.2015.2439281 http://dx.doi.org/10.1109/tpami.2015.2439281

KIM J ， LEE J K ， LEE K M . Deeply-recursive convolutional network for image super-resolution ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . Las Vegas， NV， USA ： IEEE ， 2016 ： 1637 - 1645 . doi: 10.1109/cvpr.2016.181 http://dx.doi.org/10.1109/cvpr.2016.181

LIM B ， SON S ， KIM H ， et al . Enhanced deep residual networks for single image super-resolution ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops . Honolulu， HI， USA ： IEEE ， 2017 ： 1132 - 1140 . doi: 10.1109/cvprw.2017.151 http://dx.doi.org/10.1109/cvprw.2017.151

DONG C ， LOY C C ， HE K M ， et al . Learning a deep convolutional network for image super-resolution ［C］. 13th European Conference on Computer Vision . Zurich ： Springer ， 2014 ： 184 - 199 . doi: 10.1007/978-3-319-10593-2_13 http://dx.doi.org/10.1007/978-3-319-10593-2_13

HUI Z ， WANG X M ， GAO X B . Fast and accurate single image super-resolution via information distillation network ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City， UT， USA ： IEEE ， 2018 ： 723 - 731 . doi: 10.1109/cvpr.2018.00082 http://dx.doi.org/10.1109/cvpr.2018.00082

QIU Y J ， WANG R X ， TAO D P ， et al . Embedded block residual network： a recursive restoration model for single-image super-resolution ［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision . Seoul， Korea （South）： IEEE ， 2019 ： 4179 - 4188 . doi: 10.1109/iccv.2019.00428 http://dx.doi.org/10.1109/iccv.2019.00428

LIU J ， ZHANG W J ， TANG Y T ， et al . Residual feature aggregation network for image super-resolution ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle， WA， USA ： IEEE ， 2020 ： 2356 - 2365 . doi: 10.1109/cvpr42600.2020.00243 http://dx.doi.org/10.1109/cvpr42600.2020.00243

BELL-KLIGLER S ， SHOCHER A ， IRANI M . Blind super-resolution kernel estimation using an internal-GAN ［C］// Proceedings of the 33rd International Conference on Neural Information Processing Systems . Vancouver ： Curran Associates Inc. ， 2019 ： 26 .

ZHU F ， NING Y ， CHEN X C ， et al . On removing potential redundant constraints for SVOR learning ［J］. Applied Soft Computing ， 2021 ， 102 ： 106941 . doi: 10.1016/j.asoc.2020.106941 http://dx.doi.org/10.1016/j.asoc.2020.106941

EFRAT N ， GLASNER D ， APARTSIN A ， et al . Accurate blur models vs . image priors in single image super-resolution ［C］// Proceedings of the 2013 IEEE International Conference on Computer Vision . Sydney， NSW， Australia ： IEEE ， 2013 ： 2832 - 2839 . doi: 10.1109/iccv.2013.352 http://dx.doi.org/10.1109/iccv.2013.352

YANG C Y ， MA C ， YANG M H . Single-image super-resolution： a benchmark ［C］. 13th European Conference on Computer Vision . Zurich ： Springer ， 2014 ： 372 - 386 . doi: 10.1109/cvpr.2019.00170 http://dx.doi.org/10.1109/cvpr.2019.00170

ZHANG K ， ZUO W M ， ZHANG L . Learning a single convolutional super-resolution network for multiple degradations ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City， UT， USA ： IEEE ， 2018 ： 3262 - 3271 . doi: 10.1109/cvpr.2018.00344 http://dx.doi.org/10.1109/cvpr.2018.00344

SHOCHER A ， COHEN N ， IRANI M . Zero-shot super-resolution using deep internal learning ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City， UT， USA ： IEEE ， 2018 ： 3118 - 3126 . doi: 10.1109/cvpr.2018.00329 http://dx.doi.org/10.1109/cvpr.2018.00329

ZHANG K ， VAN GOOL L ， TIMOFTE R . Deep unfolding network for image super-resolution ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle， WA， USA ： IEEE ， 2020 ： 3214 - 3223 . doi: 10.1109/cvpr42600.2020.00328 http://dx.doi.org/10.1109/cvpr42600.2020.00328

ZHANG K ， ZUO W ， ZHANG L . Deep plug-and-play super-resolution for arbitrary blur kernels ［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach ， IEEE ， 2019 ： 1671 - 1681 . doi: 10.1109/cvpr.2019.00177 http://dx.doi.org/10.1109/cvpr.2019.00177

GU J J ， LU H N ， ZUO W M ， et al . Blind super-resolution with iterative kernel correction ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach， CA， USA ： IEEE ， 2019 ： 1604 - 1613 . doi: 10.1109/cvpr.2019.00170 http://dx.doi.org/10.1109/cvpr.2019.00170

WANG L G ， WANG Y Q ， DONG X Q ， et al . Unsupervised degradation representation learning for blind super-resolution ［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville， TN， USA ： IEEE ， 2021 ： 10576 - 10585 . doi: 10.1109/cvpr46437.2021.01044 http://dx.doi.org/10.1109/cvpr46437.2021.01044

ELAD M ， FEUER A . Restoration of a single superresolution image from several blurred， noisy， and undersampled measured images ［J］. IEEE Transactions on Image Processing ， 1997 ， 6 （ 12 ）： 1646 - 1658 . doi: 10.1109/83.650118 http://dx.doi.org/10.1109/83.650118

HE K M ， FAN H Q ， WU Y X ， et al . Momentum contrast for unsupervised visual representation learning ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle， WA， USA ： IEEE ， 2020 ： 9726 - 9735 . doi: 10.1109/cvpr42600.2020.00975 http://dx.doi.org/10.1109/cvpr42600.2020.00975

CHEN Y P ， DAI X Y ， CHEN D D ， et al . Mobile-former： bridging mobileNet and transformer ［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans， LA， USA ： IEEE ， 2022 ： 5260 - 5269 . doi: 10.1109/cvpr52688.2022.00520 http://dx.doi.org/10.1109/cvpr52688.2022.00520

董本志，于明聪，赵鹏 . 基于小波域的图像超分辨率重建方法［J］. 液晶与显示， 2021 ， 36 （ 2 ）： 317 - 326 ． doi: 10.37188/CJLCD.2020-0101 http://dx.doi.org/10.37188/CJLCD.2020-0101

DONG B Z ， YU M C ， ZHAO P . Image super-resolution reconstruction based on wavelet domain ［J］. Chinese Journal of Liquid Crystals and Displays ， 2021 ， 36 （ 2 ）： 317 - 326 . （in Chinese） . doi: 10.37188/CJLCD.2020-0101 http://dx.doi.org/10.37188/CJLCD.2020-0101

WANG Z W ， LIU D ， YANG J C ， et al . Deep networks for image super-resolution with sparse prior ［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision . Santiago， Chile ： IEEE ， 2015 ： 370 - 378 . doi: 10.1109/iccv.2015.50 http://dx.doi.org/10.1109/iccv.2015.50

LAI W S ， HUANG J B ， AHUJA N ， et al . Deep laplacian pyramid networks for fast and accurate super-resolution ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu， HI， USA ： IEEE ， 2017 ： 5835 - 5843 . doi: 10.1109/cvpr.2017.618 http://dx.doi.org/10.1109/cvpr.2017.618

SAJJADI M S M ， SCHÖLKOPF B ， HIRSCH M . EnhanceNet： single image super-resolution through automated texture synthesis ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision . Venice， Italy ： IEEE ， 2017 ： 4501 - 4510 . doi: 10.1109/iccv.2017.481 http://dx.doi.org/10.1109/iccv.2017.481

ZHANG Y L ， LI K P ， LI K ， et al . Image super-resolution using very deep residual channel attention networks ［C］// Proceedings of the 15th European Conference on Computer Vision . Munich ： Springer ， 2018 ： 294 - 310 . doi: 10.1007/978-3-030-01234-2_18 http://dx.doi.org/10.1007/978-3-030-01234-2_18

NIU B ， WEN W L ， REN W Q ， et al . Single image super-resolution via a holistic attention network ［C］. 16th European Conference on Computer Vision . Glasgow ： Springer ， 2020 ： 191 - 207 . doi: 10.1007/978-3-030-58610-2_12 http://dx.doi.org/10.1007/978-3-030-58610-2_12

Views

358

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Depth estimation of light field based on correction occlusion perception

Single image super-resolution reconstruction based on split-attention networks

Hybrid distortion image correction method based on improved U-Net networks

Related Author

WU Jin

XIANG Sen

DENG Huiping

NI Jing

LI Yongxin

MENG Xin

WANG Gang

LIU Lanxi

Related Institution

School of Information Science and Engineering， Wuhan University of Science and Technology

School of Electronic and Information Engineering， Liaoning Technical University

Bohai Shipbuilding Vocational College

College of Information， Shanghai Ocean University

East China Sea Standard Metrology Center， State Oceanic Administration

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰