Motion image deblurring based on depth residual generative adversarial network

Bing-cai WEI; Li-ye ZHANG; Xiao-liang MENG; Kang-tao WANG

doi:10.37188/CJLCD.2021-0120

您当前的位置：

首页 >

文章列表页 >

Motion image deblurring based on depth residual generative adversarial network

Image Processing | 更新时间：2021-12-10

- Motion image deblurring based on depth residual generative adversarial network
- Chinese Journal of Liquid Crystals and Displays Vol. 36, Issue 12, Pages: 1693-1701(2021)
- 作者机构：
  
  山东理工大学计算机科学与技术学院, 山东淄博 255000
- 作者简介：
- 基金信息：
  
  Supported by National Natural Science Foundation of China(62001272);Shandong Provincial Natural Science Foundation of China(ZR2019BF022)
- DOI：10.37188/CJLCD.2021-0120
  CLC： TP391.4
- Received：08 April 2021，
  
  Revised：17 June 2021，
  
  Published：2021-12
- 稿件说明：
移动端阅览
Bing-cai WEI, Li-ye ZHANG, Xiao-liang MENG, et al. Motion image deblurring based on depth residual generative adversarial network[J]. Chinese journal of liquid crystals and displays, 2021, 36(12): 1693-1701.
DOI：

Bing-cai WEI, Li-ye ZHANG, Xiao-liang MENG, et al. Motion image deblurring based on depth residual generative adversarial network[J]. Chinese journal of liquid crystals and displays, 2021, 36(12): 1693-1701. DOI： 10.37188/CJLCD.2021-0120.

摘要

针对图像拍摄过程中由于运动、抖动、电子干扰等产生的运动图像模糊问题，提出一种基于深度残差生成对抗网络的运动图像去模糊算法。对图像模糊模型与盲去模糊过程进行了研究，介绍了生成对抗网络，改进了残差块的结构。改进的残差块包含3个卷积层，两个ReLU激活函数，一个Dropout层以及一个跳跃连接块，提升了复原图像的质量。改进了PatchGAN的结构，在只增加少量参数与网络复杂性的情况下，将最底层感受野变为原先的两倍以上。利用GOPRO数据集和Lai数据集进行测试，测试结果表明，本文提出的基于深度残差生成对抗网络的去模糊算法复原图像可达到较高的客观评价指标，可以恢复出较高质量的清晰图像。在GOPRO数据集上，相比于其他同类方法，本文提出的算法具有较好的复原能力，可达到更高的峰值信噪比（28.31 dB）和较高的结构相似度（0.831 7）；而在Lai数据集上，可以恢复出较高质量的图像。

Abstract

A motion image deblurring algorithm based on a deep residual generative adversarial network is proposed for the motion image blurring problem arising from motion

jitter and electronic interference during image capture. Firstly

this paper investigates the image blurring model and the blind deblurring process. Secondly

the generative adversarial network is introduced

and the structure of the residual block is improved. The improved residual block contains three convolutional layers

two ReLU activation functions

a Dropout layer

and a skip connection block

which improves the quality of the recovered image. Thirdly

the structure of PatchGAN is improved

and the receptive field of the lowest layer is more than twice of the original one with only a few additional paramters and network complexity. The tests are conducted using the GOPRO dataset and Lai dataset. The test results show that the deblurring algorithm based on deep residual generation adversarial network proposed in this paper can achieve high objective evaluation indexes and can recover clear images of high quality. On the GOPRO dataset

compared with other similar methods

the algorithm proposed in this paper has better recovery ability and can achieve higher peak signal-to-noise ratio (28.31 dB) and higher structural similarity (0.831 7). On the Lai dataset

the higher quality images can be recovered.

关键词

Keywords

references

王海峰 , 李萍 , 王博 , 等 . 灰狼算法优化BP神经网络的图像去模糊复原 . 液晶与显示 , 2019 . 34 ( 10 ): 992 - 999 . DOI: 10.3788/YJYXS20193410.0992 http://doi.org/10.3788/YJYXS20193410.0992 .

H F WANG , P LI , B WANG , 等 . Image deblurring restoration of BP neural network based on grey wolf algorithm . Chinese Journal of Liquid Crystals and Displays , 2019 . 34 ( 10 ): 992 - 999 . DOI: 10.3788/YJYXS20193410.0992 http://doi.org/10.3788/YJYXS20193410.0992 .

PAN J S, SUN D Q, PFISTER H, et al . Blind image deblurring using dark channel prior[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: IEEE, 2016: 1628-1636.

LEVIN A, WEISS Y, DURAND F, et al . Understanding and evaluating blind deconvolution algorithms[C]// Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition . Miami, FL, USA: IEEE, 2009: 1964-1971.

L XU , J S J REN , C LIU , 等 . Deep convolutional neural network for image deconvolution . Advances in Neural Information Processing Systems , 2014 . 2 1790 - 1798 . http://ieeexplore.ieee.org/document/6843274/ http://ieeexplore.ieee.org/document/6843274/ .

SU S C, DELBRACIO M, WANG J, et al . Deep video deblurring for hand-held cameras[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 237-246.

XU L, JIA J Y. Two-phase kernel estimation for robust motion deblurring[C]// Proceedings of the 11 th European Conference on Computer Vision . Berlin, Heidelberg: Springer, 2010: 157-170.

YAN Y Y, REN W Q, GUO Y F, et al . Image deblurring via extreme channels prior[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 4003-4011.

NAH S, KIM T H, LEE K M. Deep multi-scale convolutional neural network for dynamic scene deblurring[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 3883-3891.

TAO X, GAO H Y, SHEN X Y, et al . Scale-recurrent network for deep image deblurring[C]// Proceedings of 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, USA: IEEE, 2018: 8174-8182.

I GOODFELLOW , J POUGET-ABADIE , M MIRZA , 等 . Generative adversarial networks . Advances in Neural Information Processing Systems , 2014 . 27 2672 - 2680 . DOI: 10.1145/3422622 http://doi.org/10.1145/3422622 .

MAO X D, LI Q, XIE H R, et al . Least squares generative adversarial networks[C]// Proceedings of 2017 IEEE International Conference on Computer Vision . Venice, Italy: IEEE, 2017: 2794-2802.

JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution[C]// Proceedings of the 14 th European Conference on Computer Vision . Amsterdam, The Netherlands: Springer, 2016: 694-711.

KUPYN O, BUDZAN V, MYKHAILYCH M, et al . DeblurGAN: blind motion deblurring using conditional adversarial networks[C]// Proceedings of 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, USA: IEEE, 2018: 8183-8192.

ZHANG K H, LUO W H, ZHONG Y R, et al . Deblurring by realistic blurring[C]// Proceedings of 2020 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Seattle, WA, USA: IEEE, 2020: 2734-2743.

Y W TAI , X G CHEN , S KIM , 等 . Nonlinear camera response functions and image deblurring: theoretical analysis and practice . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 . 35 ( 10 ): 2498 - 2512 . DOI: 10.1109/TPAMI.2013.40 http://doi.org/10.1109/TPAMI.2013.40 .

HIRSCH M, SCHULER C J, HARMELING S, et al . Fast removal of non-uniform camera shake[C]// Proceedings of 2011 International Conference on Computer Vision . Barcelona, Spain: IEEE, 2011: 463-470.

李俊山 , 杨亚威 , 张姣 , 等 . 退化图像复原方法研究进展 . 液晶与显示 , 2018 . 33 ( 8 ): 676 - 689 . DOI: 10.3788/YJYXS20183308.0676 http://doi.org/10.3788/YJYXS20183308.0676 .

J S LI , Y W YANG , J ZHANG , 等 . Progress of degraded image restoration methods . Chinese Journal of Liquid Crystals and Displays , 2018 . 33 ( 8 ): 676 - 689 . DOI: 10.3788/YJYXS20183308.0676 http://doi.org/10.3788/YJYXS20183308.0676 .

ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[J]. arXiv preprint arXiv : 1701.07875, 2017.

GULRAJANI I, AHMED F, ARJOVSKY M, et al . Improved training of wasserstein GANs[C]// Proceedings of the 31 st International Conference on Neural Information Processing Systems . Red Hook, NY, USA: ACM, 2017: 5769-5779.

HE K M, ZHANG X Y, REN S Q, et al . Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, USA: IEEE, 2016: 770-778.

ISOLA P, ZHU J Y, ZHOU T H, et al . Image-to-image translation with conditional adversarial networks[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 5967-5976.

KIM T H, AHN B, LEE K M. Dynamic scene deblurring[C]// Proceedings of 2013 IEEE International Conference on Computer Vision . Sydney, NSW, Australia: IEEE, 2013: 3160-3167.

WIESCHOLLEK P, HIRSCH M, SCHÖLKOPF B, et al . Learning blind motion deblurring[C]// Proceedings of 2017 IEEE International Conference on Computer Vision . Venice, Italy: IEEE, 2017: 231-240.

LAI W S, HUANG J B, HU Z, et al . A comparative study for single image blind deblurring[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . LAS Vegas, USA: IEEE, 2016: 1701-1709.

KIM T H, LEE K M, SCHÖLKOPF B, et al . Online video deblurring via dynamic temporal blending network[C]// Proceedings of 2017 IEEE International Conference on Computer Vision . Venice, Italy: IEEE, 2017: 4058-4067.

SUN J, CAO W F, XU Z B, et al . Learning a convolutional neural network for non-uniform motion blur removal[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston, MA, USA: IEEE, 2015: 769-777.

KRISHNAN D, TAY T, FERGUS R. Blind deconvolution using a normalized sparsity measure[C]// Proceedings of the CVPR 2011. Colorado Springs, USA: IEEE, 2011: 233-240.

WHYTE O, SIVIC J, ZISSERMAN A, et al . Non-uniform deblurring for shaken images[C]// Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . San Francisco, California, USA: IEEE, 2010: 491-498.

XU L, ZHENG S C, JIA J Y. Unnatural L 0 sparse representation for natural image deblurring[C]// Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition . Portland, Oregon, USA: IEEE, 2013: 1107-1114.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Image super-resolution reconstruction using pyramid variance pooling network

Lens-free adversarial encoded imaging method based on convolutional attention mechanism

Single image super-resolution reconstruction based on split-attention networks

Super-resolution image reconstruction based on convolutional sparse coding and generative adversarial networks

Related Author

Bing-cai WEI

Li-ye ZHANG

Xiao-liang Meng

Kang-tao Wang

CUI Yun

MENG Xin

LI Yongxin

PENG Yanfei

Related Institution

College of computer science and technology， Shandong University of technology

School of Electronic and Information Engineering， Liaoning Technical University

Optica Information Science and Technology Department， Jiangnan University

Bohai Shipbuilding Vocational College

School of Advanced Manufacturing， Fuzhou University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰