Single image deraining based on the concatenation residual network

Qing-jiang CHEN; Tian-tian WU

doi:10.37188/CJLCD.2020-0173

您当前的位置：

首页 >

文章列表页 >

Single image deraining based on the concatenation residual network

Image Processing | 更新时间：2021-04-02

- Single image deraining based on the concatenation residual network
- Chinese Journal of Liquid Crystals and Displays Vol. 36, Issue 4, Pages: 605-614(2021)
- 作者机构：
  
  西安建筑科技大学理学院, 陕西西安 710055
- 作者简介：
- 基金信息：
- DOI：10.37188/CJLCD.2020-0173
  CLC： TP391.4
- Received：07 July 2020，
  
  Revised：22 October 2020，
  
  Published：2021-04
- 稿件说明：
移动端阅览
Qing-jiang CHEN, Tian-tian WU. Single image deraining based on the concatenation residual network[J]. Chinese journal of liquid crystals and displays, 2021, 36(4): 605-614.
DOI：

Qing-jiang CHEN, Tian-tian WU. Single image deraining based on the concatenation residual network[J]. Chinese journal of liquid crystals and displays, 2021, 36(4): 605-614. DOI： 10.37188/CJLCD.2020-0173.

摘要

本文通过改进的残差网络，学习有雨图像和无雨图像之间的映射关系来实现图像去雨，提出了一种基于联结残差网络的单幅图像去雨算法。首先，利用改进的残差块简化网络模块，减少网络参数，提升网络训练速度；其次，设计的联结结构不仅实现了多特征提取，有效减少了参数，而且增加了特征图的输出，有利于保留更多的图像细节信息；最后，利用单尺度卷积实现图像细节重建，提高去雨图像的视觉效果。实验结果表明：本文算法在合成雨天图像数据集上，其结构相似度和峰值信噪比的平均值分别高于0.95和33 dB，而真实雨天图像数据集的盲图像质量评价值较低。本文算法不仅能有效去除图像中的雨，雨纹残留较少，而且能更多地保留图像的纹理和边缘细节，视觉效果清晰自然。

Abstract

In this paper

the mapping relation between the rainy image and the clear image is learned through the improved residual network to realize the image rain removal

and a single image deraining algorithm based on the concatenation residual network is proposed. Firstly

the improved residual block is used to simplify the network module

reduce the network parameters and improve the network training speed. Secondly

the designed concatenation structure not only realizes multi-feature extraction

effectively reduces parameters

but also increases the output of feature map

which is beneficial to retain more details of the image. Finally

the single-scale convolution is used to reconstruct the image details

and further improve the visual effect of the de-rained image. Experimental results indicate that the mean values of structural similarity and peak signal-to-noise ratio on the synthetic rainy image sets are both higher than 0.95 and 33 dB

and the blind image quality evaluation values of real rainy image sets are relatively low. The method can not only remove the rain in the image effectively

the rain streak residue is less

but also keep more texture and edge details of the image

and the visual effect is clear and natural.

关键词

Keywords

references

GARG K, NAYAR S K. Detection and removal of rain from videos[C]// Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Washington, DC, USA: IEEE, 2004: 528-535.

SHEN M M, XUE P. A fast algorithm for rain detection and removal from videos[C]// Proceedings of 2011 IEEE International Conference on Multimedia and Expo . Barcelona, Spain: IEEE, 2011: 1-6.

J H KIM , J Y SIM , C S KIM . Video deraining and desnowing using temporal correlation and low-rank matrix completion . IEEE Transactions on Image Processing , 2015 . 24 ( 9 ): 2658 - 2670 . DOI: 10.1109/TIP.2015.2428933 http://doi.org/10.1109/TIP.2015.2428933 .

V SANTHASEELAN , V K ASARI . Utilizing local phase information to remove rain from video . International Journal of Computer Vision , 2015 . 112 ( 1 ): 71 - 89 . DOI: 10.1007/s11263-014-0759-8 http://doi.org/10.1007/s11263-014-0759-8 .

REN W H, TIAN J D, HAN Z, et al . Video desnowing and deraining based on matrix decomposition[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 2838-2847.

ZHENG X H, LIAO Y H, GUO W, et al . Single-image-based rain and snow removal using multi-guided filter[C]// Proceedings of 20 th International Conference International Conference on Neural Information Processing . Daegu, Korea: Springer, 2013: 258-265.

LUO Y, XU Y, JI H. Removing rain from a single image via discriminative sparse coding[C]// Proceedings of 2015 IEEE International Conference on Computer Vision ( ICCV ). Santiago, Chile: IEEE, 2015: 3397-3405.

LI Y, TAN R T, GUO X J, et al . Rain streak removal using layer priors[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: IEEE, 2016: 2736-2744.

L J DENG , T Z HUANG , X L ZHAO , 等 . A directional global sparse model for single image rain removal . Applied Mathematical Modelling , 2018 . 59 662 - 679 . DOI: 10.1016/j.apm.2018.03.001 http://doi.org/10.1016/j.apm.2018.03.001 .

B H CHEN , S C HUANG , S Y KUO . Error-optimized sparse representation for single image rain removal . IEEE Transactions on Industrial Electronics , 2017 . 64 ( 8 ): 6573 - 6581 . DOI: 10.1109/TIE.2017.2682036 http://doi.org/10.1109/TIE.2017.2682036 .

K DOEDERER , Z ILIEVA , J KELLER . Impact of a severe rain event on C-and N-DBP precursor removal using IEX . Water Science & Technology: Water Supply , 2018 . 18 ( 6 ): 2092 - 2099 . http://smartsearch.nstl.gov.cn/paper_detail.html?id=24f6bbe141a2a6780ded54084c3d77a7 http://smartsearch.nstl.gov.cn/paper_detail.html?id=24f6bbe141a2a6780ded54084c3d77a7 .

L W KANG , C W LIN , Y H FU . Automatic single-image-based rain streaks removal via image decomposition . IEEE Transactions on Image Processing , 2012 . 21 ( 4 ): 1742 - 1755 . DOI: 10.1109/TIP.2011.2179057 http://doi.org/10.1109/TIP.2011.2179057 .

X Y FU , J B HUANG , X H DING , 等 . Clearing the skies: a deep network architecture for single-image rain removal . IEEE Transactions on Image Processing , 2017 . 26 ( 6 ): 2944 - 2956 . DOI: 10.1109/TIP.2017.2691802 http://doi.org/10.1109/TIP.2017.2691802 .

FU X Y, HUANG J B, ZENG D L, et al . Removing rain from single images via a deep detail network[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 3855-3863.

YANG W H, TAN R T, FENG J S, et al . Deep joint rain detection and removal from a single image[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 1357-1366.

QIAN R, TAN R T, YANG W H, et al . Attentive generative adversarial network for raindrop removal from a single image[C]// Proceedings of 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT: IEEE, 2018: 2482-2491.

ZHANG H, PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]// Proceedings of 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 695-704.

H Y XIA , R B ZHUGE , H S LI , 等 . Single image rain removal via a simplified residual dense network . IEEE Access , 2018 . 6 66522 - 66535 . DOI: 10.1109/ACCESS.2018.2879330 http://doi.org/10.1109/ACCESS.2018.2879330 .

Y LUO , J ZHU , J LING , 等 . Fast removal of rain streaks from a single image via a shape prior . IEEE Access , 2018 . 6 60069 - 60078 . DOI: 10.1109/ACCESS.2018.2875171 http://doi.org/10.1109/ACCESS.2018.2875171 .

C H YEH , C Y LIN , K MUCHTAR , 等 . Rain streak removal based on non-negative matrix factorization . Multimedia Tools and Applications , 2018 . 77 ( 15 ): 20001 - 20020 . DOI: 10.1007/s11042-017-5430-2 http://doi.org/10.1007/s11042-017-5430-2 .

T Y CHEN , C Z FU . Single-image-based rain detection and removal via CNN . Journal of Physics: Conference Series , 2018 . 1004 ( 1 ): 012007 http://kns.cnki.net/KCMS/detail/detail.aspx?dbcode=IPFD&filename=CDLK201802001008 http://kns.cnki.net/KCMS/detail/detail.aspx?dbcode=IPFD&filename=CDLK201802001008 .

Y C GUO , C LI , Q LIU . R 2 N: a novel deep learning architecture for rain removal from single image . Computers, Materials & Continua , 2019 . 58 ( 3 ): 829 - 843 . http://www.researchgate.net/publication/331615963_R2N_A_Novel_Deep_Learning_Architecture_for_Rain_Removal_from_Single_Image http://www.researchgate.net/publication/331615963_R2N_A_Novel_Deep_Learning_Architecture_for_Rain_Removal_from_Single_Image .

陈清江 , 张雪 , 柴昱洲 . 基于卷积神经网络的图像去雾算法 . 液晶与显示 , 2019 . 34 ( 2 ): 220 - 227 . http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20193402.0220 http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20193402.0220 .

Q J CHEN , X ZHANG , Y Z CHAI . Image defogging algorithms based on multiscale convolution neural network . Chinese Journal of Liquid Crystals and Displays , 2019 . 34 ( 2 ): 220 - 227 . http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20193402.0220 http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20193402.0220 .

S T WU , S H ZHONG , Y LIU . Deep residual learning for image steganalysis . Multimedia Tools and Applications , 2018 . 77 ( 9 ): 10437 - 10453 . DOI: 10.1007/s11042-017-4440-4 http://doi.org/10.1007/s11042-017-4440-4 .

DONG C, LOY C C, HE K M, et al . Learning a deep convolutional network for image super-resolution[C]// Proceedings of the 13 th European Conference European Conference on Computer Vision . Zurich, Switzerland: Springer, 2014: 184-199.

A MITTAL , R SOUNDARARAJAN , A C BOVIK . Making a "completely blind" image quality analyzer . IEEE Signal Processing Letters , 2013 . 20 ( 3 ): 209 - 212 . DOI: 10.1109/LSP.2012.2227726 http://doi.org/10.1109/LSP.2012.2227726 .

OU F Z, WANG Y G, ZHU G P. A novel blind image quality assessment method based on refined natural scene statistics[C]// Proceedings of 2019 IEEE International Conference on Image Processing ( ICIP ). Taipei, China: IEEE, 2019: 1004-1008.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Lightweight image super-resolution combining residual learning and layer attention

Design of heterogeneous FPGA hardware accelerator based on CNN

Remote sensing image change detection based on CNN-Transformer structure

Related Author

Qing-jiang CHEN

Tian-tian WU

WU Difan

ZHANG Xuande

WU Di-fan

ZHANG Xuan-de

JI Haolin

XU Wei

Related Institution

College of Science， Xi’an University of Architecture and Technology

School of Electronic Information and Artificial Intelligence， Shaanxi University of Science & Technology， Xi′an

Key Laboratory of Space-based Dynamic & Rapid Optical Imaging Technology， Chinese Academy of Sciences

Changchun Institute of Optics， Fine Mechanics and Physics， Chinese Academy of Sciences

University of Chinese Academy of Sciences

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰