基于深度残差生成对抗网络的运动图像去模糊

魏丙财; 张立晔; 孟晓亮; 王康涛

doi:10.37188/CJLCD.2021-0120

您当前的位置：

首页 >

文章列表页 >

基于深度残差生成对抗网络的运动图像去模糊

图像处理 | 更新时间：2021-12-10

- 基于深度残差生成对抗网络的运动图像去模糊
- Motion image deblurring based on depth residual generative adversarial network
- 液晶与显示 2021年36卷第12期页码：1693-1701
- 作者机构：
  
  山东理工大学计算机科学与技术学院, 山东淄博 255000
- 作者简介：
  
  [ "魏丙财(1997-), 男, 山东济南人, 硕士研究生, 2020年于曲阜师范大学获得学士学位, 主要从事图像复原及深度学习方面的研究。E-mail: 1394594109@qq.com" ]
  [ "张立晔(1986-), 男, 山东济南人, 博士, 讲师, 2018年于哈尔滨工业大学获得博士学位, 主要从事机器学习与图像处理方面的研究。E-mail: zhangliye@sdut.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(62001272);山东省自然科学基金(ZR2019BF022)
- DOI：10.37188/CJLCD.2021-0120
  中图分类号： TP391.4
- 收稿日期：2021-04-08，
  
  修回日期：2021-06-17，
  
  纸质出版日期：2021-12
- 稿件说明：
移动端阅览
魏丙财, 张立晔, 孟晓亮, 等. 基于深度残差生成对抗网络的运动图像去模糊[J]. 液晶与显示, 2021,36(12):1693-1701.

Bing-cai WEI, Li-ye ZHANG, Xiao-liang MENG, et al. Motion image deblurring based on depth residual generative adversarial network[J]. Chinese journal of liquid crystals and displays, 2021, 36(12): 1693-1701.
魏丙财, 张立晔, 孟晓亮, 等. 基于深度残差生成对抗网络的运动图像去模糊[J]. 液晶与显示, 2021,36(12):1693-1701. DOI： 10.37188/CJLCD.2021-0120.

Bing-cai WEI, Li-ye ZHANG, Xiao-liang MENG, et al. Motion image deblurring based on depth residual generative adversarial network[J]. Chinese journal of liquid crystals and displays, 2021, 36(12): 1693-1701. DOI： 10.37188/CJLCD.2021-0120.

摘要

针对图像拍摄过程中由于运动、抖动、电子干扰等产生的运动图像模糊问题，提出一种基于深度残差生成对抗网络的运动图像去模糊算法。对图像模糊模型与盲去模糊过程进行了研究，介绍了生成对抗网络，改进了残差块的结构。改进的残差块包含3个卷积层，两个ReLU激活函数，一个Dropout层以及一个跳跃连接块，提升了复原图像的质量。改进了PatchGAN的结构，在只增加少量参数与网络复杂性的情况下，将最底层感受野变为原先的两倍以上。利用GOPRO数据集和Lai数据集进行测试，测试结果表明，本文提出的基于深度残差生成对抗网络的去模糊算法复原图像可达到较高的客观评价指标，可以恢复出较高质量的清晰图像。在GOPRO数据集上，相比于其他同类方法，本文提出的算法具有较好的复原能力，可达到更高的峰值信噪比（28.31 dB）和较高的结构相似度（0.831 7）；而在Lai数据集上，可以恢复出较高质量的图像。

Abstract

A motion image deblurring algorithm based on a deep residual generative adversarial network is proposed for the motion image blurring problem arising from motion

jitter and electronic interference during image capture. Firstly

this paper investigates the image blurring model and the blind deblurring process. Secondly

the generative adversarial network is introduced

and the structure of the residual block is improved. The improved residual block contains three convolutional layers

two ReLU activation functions

a Dropout layer

and a skip connection block

which improves the quality of the recovered image. Thirdly

the structure of PatchGAN is improved

and the receptive field of the lowest layer is more than twice of the original one with only a few additional paramters and network complexity. The tests are conducted using the GOPRO dataset and Lai dataset. The test results show that the deblurring algorithm based on deep residual generation adversarial network proposed in this paper can achieve high objective evaluation indexes and can recover clear images of high quality. On the GOPRO dataset

compared with other similar methods

the algorithm proposed in this paper has better recovery ability and can achieve higher peak signal-to-noise ratio (28.31 dB) and higher structural similarity (0.831 7). On the Lai dataset

the higher quality images can be recovered.

关键词

Keywords

references

王海峰 , 李萍 , 王博 , 等 . 灰狼算法优化BP神经网络的图像去模糊复原 . 液晶与显示 , 2019 . 34 ( 10 ): 992 - 999 . DOI: 10.3788/YJYXS20193410.0992 http://doi.org/10.3788/YJYXS20193410.0992 .

H F WANG , P LI , B WANG , 等 . Image deblurring restoration of BP neural network based on grey wolf algorithm . Chinese Journal of Liquid Crystals and Displays , 2019 . 34 ( 10 ): 992 - 999 . DOI: 10.3788/YJYXS20193410.0992 http://doi.org/10.3788/YJYXS20193410.0992 .

PAN J S, SUN D Q, PFISTER H, et al . Blind image deblurring using dark channel prior[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: IEEE, 2016: 1628-1636.

LEVIN A, WEISS Y, DURAND F, et al . Understanding and evaluating blind deconvolution algorithms[C]// Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition . Miami, FL, USA: IEEE, 2009: 1964-1971.

L XU , J S J REN , C LIU , 等 . Deep convolutional neural network for image deconvolution . Advances in Neural Information Processing Systems , 2014 . 2 1790 - 1798 . http://ieeexplore.ieee.org/document/6843274/ http://ieeexplore.ieee.org/document/6843274/ .

SU S C, DELBRACIO M, WANG J, et al . Deep video deblurring for hand-held cameras[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 237-246.

XU L, JIA J Y. Two-phase kernel estimation for robust motion deblurring[C]// Proceedings of the 11 th European Conference on Computer Vision . Berlin, Heidelberg: Springer, 2010: 157-170.

YAN Y Y, REN W Q, GUO Y F, et al . Image deblurring via extreme channels prior[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 4003-4011.

NAH S, KIM T H, LEE K M. Deep multi-scale convolutional neural network for dynamic scene deblurring[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 3883-3891.

TAO X, GAO H Y, SHEN X Y, et al . Scale-recurrent network for deep image deblurring[C]// Proceedings of 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, USA: IEEE, 2018: 8174-8182.

I GOODFELLOW , J POUGET-ABADIE , M MIRZA , 等 . Generative adversarial networks . Advances in Neural Information Processing Systems , 2014 . 27 2672 - 2680 . DOI: 10.1145/3422622 http://doi.org/10.1145/3422622 .

MAO X D, LI Q, XIE H R, et al . Least squares generative adversarial networks[C]// Proceedings of 2017 IEEE International Conference on Computer Vision . Venice, Italy: IEEE, 2017: 2794-2802.

JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution[C]// Proceedings of the 14 th European Conference on Computer Vision . Amsterdam, The Netherlands: Springer, 2016: 694-711.

KUPYN O, BUDZAN V, MYKHAILYCH M, et al . DeblurGAN: blind motion deblurring using conditional adversarial networks[C]// Proceedings of 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, USA: IEEE, 2018: 8183-8192.

ZHANG K H, LUO W H, ZHONG Y R, et al . Deblurring by realistic blurring[C]// Proceedings of 2020 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Seattle, WA, USA: IEEE, 2020: 2734-2743.

Y W TAI , X G CHEN , S KIM , 等 . Nonlinear camera response functions and image deblurring: theoretical analysis and practice . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 . 35 ( 10 ): 2498 - 2512 . DOI: 10.1109/TPAMI.2013.40 http://doi.org/10.1109/TPAMI.2013.40 .

HIRSCH M, SCHULER C J, HARMELING S, et al . Fast removal of non-uniform camera shake[C]// Proceedings of 2011 International Conference on Computer Vision . Barcelona, Spain: IEEE, 2011: 463-470.

李俊山 , 杨亚威 , 张姣 , 等 . 退化图像复原方法研究进展 . 液晶与显示 , 2018 . 33 ( 8 ): 676 - 689 . DOI: 10.3788/YJYXS20183308.0676 http://doi.org/10.3788/YJYXS20183308.0676 .

J S LI , Y W YANG , J ZHANG , 等 . Progress of degraded image restoration methods . Chinese Journal of Liquid Crystals and Displays , 2018 . 33 ( 8 ): 676 - 689 . DOI: 10.3788/YJYXS20183308.0676 http://doi.org/10.3788/YJYXS20183308.0676 .

ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[J]. arXiv preprint arXiv : 1701.07875, 2017.

GULRAJANI I, AHMED F, ARJOVSKY M, et al . Improved training of wasserstein GANs[C]// Proceedings of the 31 st International Conference on Neural Information Processing Systems . Red Hook, NY, USA: ACM, 2017: 5769-5779.

HE K M, ZHANG X Y, REN S Q, et al . Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, USA: IEEE, 2016: 770-778.

ISOLA P, ZHU J Y, ZHOU T H, et al . Image-to-image translation with conditional adversarial networks[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 5967-5976.

KIM T H, AHN B, LEE K M. Dynamic scene deblurring[C]// Proceedings of 2013 IEEE International Conference on Computer Vision . Sydney, NSW, Australia: IEEE, 2013: 3160-3167.

WIESCHOLLEK P, HIRSCH M, SCHÖLKOPF B, et al . Learning blind motion deblurring[C]// Proceedings of 2017 IEEE International Conference on Computer Vision . Venice, Italy: IEEE, 2017: 231-240.

LAI W S, HUANG J B, HU Z, et al . A comparative study for single image blind deblurring[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . LAS Vegas, USA: IEEE, 2016: 1701-1709.

KIM T H, LEE K M, SCHÖLKOPF B, et al . Online video deblurring via dynamic temporal blending network[C]// Proceedings of 2017 IEEE International Conference on Computer Vision . Venice, Italy: IEEE, 2017: 4058-4067.

SUN J, CAO W F, XU Z B, et al . Learning a convolutional neural network for non-uniform motion blur removal[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston, MA, USA: IEEE, 2015: 769-777.

KRISHNAN D, TAY T, FERGUS R. Blind deconvolution using a normalized sparsity measure[C]// Proceedings of the CVPR 2011. Colorado Springs, USA: IEEE, 2011: 233-240.

WHYTE O, SIVIC J, ZISSERMAN A, et al . Non-uniform deblurring for shaken images[C]// Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . San Francisco, California, USA: IEEE, 2010: 491-498.

XU L, ZHENG S C, JIA J Y. Unnatural L 0 sparse representation for natural image deblurring[C]// Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition . Portland, Oregon, USA: IEEE, 2013: 1107-1114.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

金字塔方差池化网络的图像超分辨率重建

基于卷积注意力机制的无透镜对抗编码成像

基于拆分注意力网络的单图像超分辨率重建

基于卷积稀疏编码与生成对抗网络的图像超分辨率重建