基于小波域的图像超分辨率重建方法

董本志; 于明聪; 赵鹏

doi:10.37188/CJLCD.2020-0101

您当前的位置：

首页 >

文章列表页 >

基于小波域的图像超分辨率重建方法

图像处理 | 更新时间：2021-02-07

- 基于小波域的图像超分辨率重建方法
- Image super-resolution reconstruction based on wavelet domain
- 液晶与显示 2021年36卷第2期页码：317-326
- 作者机构：
  
  东北林业大学信息与计算机工程学院, 黑龙江哈尔滨 150040
- 作者简介：
  
  [ "董本志(1975-), 男, 黑龙江绥化人, 博士, 副教授, 2010年于东北林业大学获得博士学位, 主要从事模式识别与智能控制、数控及数控仿真技术方面的研究。E-mail:nefu_dbz@163.com" ]
  [ "于明聪(1995-), 男, 辽宁大连人, 硕士研究生, 2018年于东北林业大学获得学士学位, 主要从事模式识别与智能控制方面的研究, E-mail:782968624@qq.com" ]
- 基金信息：
  
  国家自然科学基金(31370565);国家自然科学基金(31770768)
- DOI：10.37188/CJLCD.2020-0101
  中图分类号： TP394.41
- 收稿日期：2020-06-28，
  
  修回日期：2020-09-07，
  
  录用日期：2020-9-7，
  
  纸质出版日期：2021-02
- 稿件说明：
移动端阅览
董本志, 于明聪, 赵鹏. 基于小波域的图像超分辨率重建方法[J]. 液晶与显示, 2021,36(2):317-326.

Ben-zhi DONG, Ming-cong YU, Peng ZHAO. Image super-resolution reconstruction based on wavelet domain[J]. Chinese journal of liquid crystals and displays, 2021, 36(2): 317-326.
董本志, 于明聪, 赵鹏. 基于小波域的图像超分辨率重建方法[J]. 液晶与显示, 2021,36(2):317-326. DOI： 10.37188/CJLCD.2020-0101.

Ben-zhi DONG, Ming-cong YU, Peng ZHAO. Image super-resolution reconstruction based on wavelet domain[J]. Chinese journal of liquid crystals and displays, 2021, 36(2): 317-326. DOI： 10.37188/CJLCD.2020-0101.

摘要

传统的基于CNN的方法在对低分辨率图像进行重构处理过程中，并未将图像中的低频结构信息和高频细节信息进行区别处理，且网络的层与层之间缺乏信息交流，从而造成高分辨率重建图像结果中出现信息缺失。为获取更多图像各层次特征的结构与细节信息，本文构建了基于小波域的残差密集网络（WRDSR）。该网络在二维离散小波变换形成的小波域内，利用密集连接和残差连接对图像不同频率的信息进行充分提取后，将融合后的特征输入到亚像素卷积层生成高分辨率图像的小波子带图像，最后通过二维离散小波逆变换生成高分辨率图像。与Bicubic、SRCNN、VDSR、LapSRN、DWSR、SDSR等算法相比，WRDSR在评价指标PSNR/SSIM上平均提高了2.824 dB/0.059 5、0.747 dB/0.016 8、0.016 dB/0.002 4、0.025 dB/0.004 3、0.21 dB/0.004 7和0.20 dB/0.005 7，在更高效地利用原始图像信息的同时，解决了信息缺失的问题，使得重建图像的纹理更清晰，细节更丰富，视觉效果更佳。

Abstract

The traditional CNN-based method can not distinguish between low-frequency structural information and high-frequency detailed information in the process of reconstructing low-resolution images. And there is a lack of information communication between layers of the network

which leads to the problem of missing information in the high-resolution reconstruction image. In order to obtain more information about the structure and details of each level of image features a residual dense network is constructed based on the wavelet domain (WRDSR). In the wavelet domain formed by the two-dimensional discrete wavelet transform

the network uses dense connections and residual connections to fully extract the information of different frequencies of the image

and then generates high-resolution image wavelets by inputting the fused features into sub-pixel convolution. Finally

the final high-resolution image is generated by two-dimensional discrete wavelet inversion. Compared with Bicubic

A+

SRCNN

VDSR

LapSRN

DWSR

SDSR etc.

WRDSR improves 2.824 dB/0.059 5

0.896 dB/0.018 2

0.747 dB/0.016 8

0.016 dB/0.002 4

0.025 dB/0.004 3

0.21 dB/0.004 7 and 0.20 dB/0.0057 on average on PSNR/SSIM

respectively. While making more efficient use of the original image information

WRDSR solves the drawback of missing information

making the reconstructed image texture clearer

richer in details and better in visual effect.

关键词

Keywords

references

S C PARK , M K PARK , M G KANG . Super-resolution image reconstruction:a technical overview . IEEE Signal Processing Magazine , 2003 . 20 ( 3 ): 21 - 36 . DOI: 10.1109/MSP.2003.1203207 http://doi.org/10.1109/MSP.2003.1203207 .

J L HARRIS . Diffraction and resolving power . Journal of the Optical Society of America , 1964 . 54 ( 7 ): 931 - 936 . DOI: 10.1364/JOSA.54.000931 http://doi.org/10.1364/JOSA.54.000931 .

刘涛 , 钱锋 , 张葆 . 遥感图像的MAP超分辨重建 . 液晶与显示 , 2018 . 33 ( 10 ): 884 - 892 . http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1462189&Fpath=&index=-1&l=zh http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1462189&Fpath=&index=-1&l=zh .

T LIU , F QIAN , B ZHANG . MAP super-resolution reconstruction of remote sensing image . Chinese Journal of Liquid Crystals and Displays , 2018 . 33 ( 10 ): 884 - 892 . http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1462189&Fpath=&index=-1&l=zh http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1462189&Fpath=&index=-1&l=zh .

吴磊 , 吕国强 , 赵晨 , 等 . 基于多尺度残差网络的CT图像超分辨率重建 . 液晶与显示 , 2019 . 34 ( 10 ): 1006 - 1012 . http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1384623&Fpath=&index=-1&l=zh http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1384623&Fpath=&index=-1&l=zh .

L WU , G Q LYU , C ZHAO , 等 . CT image super-resolution reconstruction based on multi-scale residual network . Chinese Journal of Liquid Crystals and Displays , 2019 . 34 ( 10 ): 1006 - 1012 . http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1384623&Fpath=&index=-1&l=zh http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1384623&Fpath=&index=-1&l=zh .

何阳 , 黄玮 , 王新华 , 等 . 稀疏阈值的超分辨率图像重建 . 中国光学 , 2016 . 9 ( 5 ): 532 - 539 .

Y HE , W HUANG , X H WANG , 等 . Super-resolution image reconstruction based on sparse threshold . Chinese Optics , 2016 . 9 ( 5 ): 532 - 539 .

邓建青 , 刘晶红 , 刘铁军 . 基于DSP系统的超分辨率图像重建技术研究 . 液晶与显示 , 2012 . 27 ( 1 ): 114 - 120 . http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1695821&Fpath=&index=-1&l=zh http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1695821&Fpath=&index=-1&l=zh .

J Q DENG , J H LIU , T J LIU . Super-resolution image reconstruction technology based on DSP system . Chinese Journal of Liquid Crystals and Displays , 2012 . 27 ( 1 ): 114 - 120 . http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1695821&Fpath=&index=-1&l=zh http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1695821&Fpath=&index=-1&l=zh .

刘志贺 , 吴长锋 . 超分辨率成像荧光探针材料应用进展 . 中国光学 , 2018 . 11 ( 3 ): 344 - 362 .

Z H LIU , C F WU . Advances in application of materials of super-resolution imaging fluorescent probe . Chinese Optics , 2018 . 11 ( 3 ): 344 - 362 .

TIMOFTE R, DE SMET V, VAN GOOL L. A+: adjusted anchored neighborhood regression for fast super-resolution[C]// Proceeding of the 12 th Asian Conference on Computer Vision . Singapore: Springer, 2015: 111-126.

C DONG , C C LOY , K M HE , 等 . Image super-resolution using deep convolutional networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2016 . 38 ( 2 ): 295 - 307 . DOI: 10.1109/TPAMI.2015.2439281 http://doi.org/10.1109/TPAMI.2015.2439281 .

DONG C, LOY C C, TANG X O. Accelerating the super-resolution convolutional neural network[C]// Proceedings of the 14 th European Conference on Computer Vision . Amsterdam, The Netherlands: Springer, 2016: 391-407.

SHI W Z, CABALLERO J, HUSZAR F, et al . Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: IEEE, 2016: 1874-1883.

HE K M, ZHANG X Y, REN S Q, et al . Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: IEEE, 2016: 770-778.

KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: IEEE, 2016: 1646-1654.

KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: IEEE, 2016: 1637-1645.

TAI Y, YANG J, LIU X M. Image super-resolution via deep recursive residual network[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 2790-2798.

LAI W S, HUANG J B, AHUJA N, et al . Deep Laplacian pyramid networks for fast and accurate super-resolution[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 5835-5843.

LEDIG C, THEIS L, HUSZÁR F, et al . Photo-realistic single image super-resolution using a generative adversarial network[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 105-114.

HUANG G, LIU Z, VAN DER MAATEN L, et al . Densely connected convolutional networks[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 2261-2269.

LI S M, FAN R, LEI G Q, et al . A two-channel convolutional neural network for image super-resolution[J]. Neurocomputing , 201, 2758:267-277.

X F DU , X B QU , Y F HE , 等 . Single image super-resolution based on multi-scale competitive convolutional neural network . Sensors , 2018 . 18 ( 3 ): 789 DOI: 10.3390/s18030789 http://doi.org/10.3390/s18030789 .

BAE W, YOO J, YE J C. Beyond deep residual learning for image restoration: persistent homology-guided manifold simplification[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops . Honolulu, HI, USA: IEEE, 2017: 1141-1149.

GUO T T, MOUSAVI H S, VU T H, et al . Deep wavelet prediction for image super-resolution[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops . Honolulu, HI, USA: IEEE, 2017: 1100-1109.

LIU P J, ZHANG H Z, ZHANG K, et al . Multi-level wavelet-CNN for image restoration[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Salt Lake City, UT, USA: IEEE, 2018.

RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// Proceedings of the 18 th International Conference on Medical Image Computing and Computer-Assisted Intervention . Munich, Germany: Springer, 2015: 234-241.

YU X, PORIKLI F. Ultra-resolving face images by discriminative generative networks[C]// Proceedings of the 14 th European Conference on Computer Vision . Amsterdam, The Netherlands: Springer, 2016: 318-333.

S MALLAT . Wavelets for a vision . Proceedings of the IEEE , 1996 . 84 ( 4 ): 604 - 614 . DOI: 10.1109/5.488702 http://doi.org/10.1109/5.488702 .

H B HUANG , R HE , Z A SUN , 等 . Wavelet domain generative adversarial network for multi-scale face hallucination . International Journal of Computer Vision , 2019 . 127 ( 6/7 ): 763 - 784 . DOI: 10.1007/s11263-019-01154-8 http://doi.org/10.1007/s11263-019-01154-8 .

TIMOFTE R, GU S H, WU J Q, et al . NTIRE 2018 challenge on single image super-resolution: methods and results[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Salt Lake City, UT, USA: IEEE, 2018.

BEVILACQUA M, ROUMY A, GUILLEMOT C, et al . Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]// Proceedings of the 23 rd British Machine Vision Conference . Surrey, United Kingdom: BMVA Press, 2012: 1-10.

ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations[C]// Proceedings of the 7 th International Conference on Curves and Surfaces . Avignon, France: Springer, 2010: 711-730.

MARTIN D, FOWLKES C, TAL D, et al . A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]// Proceedings of the Eighth IEEE International Conference on Computer Vision . Vancouver, BC, Canada: IEEE, 2001.

HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston, MA, USA: IEEE, 2015: 5197-5206.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

金字塔方差池化网络的图像超分辨率重建

基于Retinex-Net网络模型的渐晕图像校正

基于改进条件生成对抗网络的水下图像增强方法