基于联结残差网络的单幅图像去雨

陈清江; 吴田田

doi:10.37188/CJLCD.2020-0173

您当前的位置：

首页 >

文章列表页 >

基于联结残差网络的单幅图像去雨

图像处理 | 更新时间：2021-04-02

- 基于联结残差网络的单幅图像去雨
- Single image deraining based on the concatenation residual network
- 液晶与显示 2021年36卷第4期页码：605-614
- 作者机构：
  
  西安建筑科技大学理学院, 陕西西安 710055
- 作者简介：
  
  [ "陈清江(1966-), 男, 河南信阳人, 博士, 教授, 2006年于西安交通大学获得博士学位, 主要从事小波分析、图像处理与信号处理方面的研究。E-mail:qjchen66xytu@126.com" ]
  [ "吴田田(1995-), 女, 陕西榆林人, 硕士研究生, 2018年于西安建筑科技大学获得学士学位, 主要从事小波分析、图像处理与信号处理方面的研究。E-mail: ttianwu@xauat.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(61403298)
- DOI：10.37188/CJLCD.2020-0173
  中图分类号： TP391.4
- 收稿日期：2020-07-07，
  
  修回日期：2020-10-22，
  
  纸质出版日期：2021-04
- 稿件说明：
移动端阅览
陈清江, 吴田田. 基于联结残差网络的单幅图像去雨[J]. 液晶与显示, 2021,36(4):605-614.

Qing-jiang CHEN, Tian-tian WU. Single image deraining based on the concatenation residual network[J]. Chinese journal of liquid crystals and displays, 2021, 36(4): 605-614.
陈清江, 吴田田. 基于联结残差网络的单幅图像去雨[J]. 液晶与显示, 2021,36(4):605-614. DOI： 10.37188/CJLCD.2020-0173.

Qing-jiang CHEN, Tian-tian WU. Single image deraining based on the concatenation residual network[J]. Chinese journal of liquid crystals and displays, 2021, 36(4): 605-614. DOI： 10.37188/CJLCD.2020-0173.

摘要

本文通过改进的残差网络，学习有雨图像和无雨图像之间的映射关系来实现图像去雨，提出了一种基于联结残差网络的单幅图像去雨算法。首先，利用改进的残差块简化网络模块，减少网络参数，提升网络训练速度；其次，设计的联结结构不仅实现了多特征提取，有效减少了参数，而且增加了特征图的输出，有利于保留更多的图像细节信息；最后，利用单尺度卷积实现图像细节重建，提高去雨图像的视觉效果。实验结果表明：本文算法在合成雨天图像数据集上，其结构相似度和峰值信噪比的平均值分别高于0.95和33 dB，而真实雨天图像数据集的盲图像质量评价值较低。本文算法不仅能有效去除图像中的雨，雨纹残留较少，而且能更多地保留图像的纹理和边缘细节，视觉效果清晰自然。

Abstract

In this paper

the mapping relation between the rainy image and the clear image is learned through the improved residual network to realize the image rain removal

and a single image deraining algorithm based on the concatenation residual network is proposed. Firstly

the improved residual block is used to simplify the network module

reduce the network parameters and improve the network training speed. Secondly

the designed concatenation structure not only realizes multi-feature extraction

effectively reduces parameters

but also increases the output of feature map

which is beneficial to retain more details of the image. Finally

the single-scale convolution is used to reconstruct the image details

and further improve the visual effect of the de-rained image. Experimental results indicate that the mean values of structural similarity and peak signal-to-noise ratio on the synthetic rainy image sets are both higher than 0.95 and 33 dB

and the blind image quality evaluation values of real rainy image sets are relatively low. The method can not only remove the rain in the image effectively

the rain streak residue is less

but also keep more texture and edge details of the image

and the visual effect is clear and natural.

关键词

Keywords

references

GARG K, NAYAR S K. Detection and removal of rain from videos[C]// Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Washington, DC, USA: IEEE, 2004: 528-535.

SHEN M M, XUE P. A fast algorithm for rain detection and removal from videos[C]// Proceedings of 2011 IEEE International Conference on Multimedia and Expo . Barcelona, Spain: IEEE, 2011: 1-6.

J H KIM , J Y SIM , C S KIM . Video deraining and desnowing using temporal correlation and low-rank matrix completion . IEEE Transactions on Image Processing , 2015 . 24 ( 9 ): 2658 - 2670 . DOI: 10.1109/TIP.2015.2428933 http://doi.org/10.1109/TIP.2015.2428933 .

V SANTHASEELAN , V K ASARI . Utilizing local phase information to remove rain from video . International Journal of Computer Vision , 2015 . 112 ( 1 ): 71 - 89 . DOI: 10.1007/s11263-014-0759-8 http://doi.org/10.1007/s11263-014-0759-8 .

REN W H, TIAN J D, HAN Z, et al . Video desnowing and deraining based on matrix decomposition[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 2838-2847.

ZHENG X H, LIAO Y H, GUO W, et al . Single-image-based rain and snow removal using multi-guided filter[C]// Proceedings of 20 th International Conference International Conference on Neural Information Processing . Daegu, Korea: Springer, 2013: 258-265.

LUO Y, XU Y, JI H. Removing rain from a single image via discriminative sparse coding[C]// Proceedings of 2015 IEEE International Conference on Computer Vision ( ICCV ). Santiago, Chile: IEEE, 2015: 3397-3405.

LI Y, TAN R T, GUO X J, et al . Rain streak removal using layer priors[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: IEEE, 2016: 2736-2744.

L J DENG , T Z HUANG , X L ZHAO , 等 . A directional global sparse model for single image rain removal . Applied Mathematical Modelling , 2018 . 59 662 - 679 . DOI: 10.1016/j.apm.2018.03.001 http://doi.org/10.1016/j.apm.2018.03.001 .

B H CHEN , S C HUANG , S Y KUO . Error-optimized sparse representation for single image rain removal . IEEE Transactions on Industrial Electronics , 2017 . 64 ( 8 ): 6573 - 6581 . DOI: 10.1109/TIE.2017.2682036 http://doi.org/10.1109/TIE.2017.2682036 .

K DOEDERER , Z ILIEVA , J KELLER . Impact of a severe rain event on C-and N-DBP precursor removal using IEX . Water Science & Technology: Water Supply , 2018 . 18 ( 6 ): 2092 - 2099 . http://smartsearch.nstl.gov.cn/paper_detail.html?id=24f6bbe141a2a6780ded54084c3d77a7 http://smartsearch.nstl.gov.cn/paper_detail.html?id=24f6bbe141a2a6780ded54084c3d77a7 .

L W KANG , C W LIN , Y H FU . Automatic single-image-based rain streaks removal via image decomposition . IEEE Transactions on Image Processing , 2012 . 21 ( 4 ): 1742 - 1755 . DOI: 10.1109/TIP.2011.2179057 http://doi.org/10.1109/TIP.2011.2179057 .

X Y FU , J B HUANG , X H DING , 等 . Clearing the skies: a deep network architecture for single-image rain removal . IEEE Transactions on Image Processing , 2017 . 26 ( 6 ): 2944 - 2956 . DOI: 10.1109/TIP.2017.2691802 http://doi.org/10.1109/TIP.2017.2691802 .

FU X Y, HUANG J B, ZENG D L, et al . Removing rain from single images via a deep detail network[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 3855-3863.

YANG W H, TAN R T, FENG J S, et al . Deep joint rain detection and removal from a single image[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 1357-1366.

QIAN R, TAN R T, YANG W H, et al . Attentive generative adversarial network for raindrop removal from a single image[C]// Proceedings of 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT: IEEE, 2018: 2482-2491.

ZHANG H, PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]// Proceedings of 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 695-704.

H Y XIA , R B ZHUGE , H S LI , 等 . Single image rain removal via a simplified residual dense network . IEEE Access , 2018 . 6 66522 - 66535 . DOI: 10.1109/ACCESS.2018.2879330 http://doi.org/10.1109/ACCESS.2018.2879330 .

Y LUO , J ZHU , J LING , 等 . Fast removal of rain streaks from a single image via a shape prior . IEEE Access , 2018 . 6 60069 - 60078 . DOI: 10.1109/ACCESS.2018.2875171 http://doi.org/10.1109/ACCESS.2018.2875171 .

C H YEH , C Y LIN , K MUCHTAR , 等 . Rain streak removal based on non-negative matrix factorization . Multimedia Tools and Applications , 2018 . 77 ( 15 ): 20001 - 20020 . DOI: 10.1007/s11042-017-5430-2 http://doi.org/10.1007/s11042-017-5430-2 .

T Y CHEN , C Z FU . Single-image-based rain detection and removal via CNN . Journal of Physics: Conference Series , 2018 . 1004 ( 1 ): 012007 http://kns.cnki.net/KCMS/detail/detail.aspx?dbcode=IPFD&filename=CDLK201802001008 http://kns.cnki.net/KCMS/detail/detail.aspx?dbcode=IPFD&filename=CDLK201802001008 .

Y C GUO , C LI , Q LIU . R 2 N: a novel deep learning architecture for rain removal from single image . Computers, Materials & Continua , 2019 . 58 ( 3 ): 829 - 843 . http://www.researchgate.net/publication/331615963_R2N_A_Novel_Deep_Learning_Architecture_for_Rain_Removal_from_Single_Image http://www.researchgate.net/publication/331615963_R2N_A_Novel_Deep_Learning_Architecture_for_Rain_Removal_from_Single_Image .

陈清江 , 张雪 , 柴昱洲 . 基于卷积神经网络的图像去雾算法 . 液晶与显示 , 2019 . 34 ( 2 ): 220 - 227 . http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20193402.0220 http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20193402.0220 .

Q J CHEN , X ZHANG , Y Z CHAI . Image defogging algorithms based on multiscale convolution neural network . Chinese Journal of Liquid Crystals and Displays , 2019 . 34 ( 2 ): 220 - 227 . http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20193402.0220 http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20193402.0220 .

S T WU , S H ZHONG , Y LIU . Deep residual learning for image steganalysis . Multimedia Tools and Applications , 2018 . 77 ( 9 ): 10437 - 10453 . DOI: 10.1007/s11042-017-4440-4 http://doi.org/10.1007/s11042-017-4440-4 .

DONG C, LOY C C, HE K M, et al . Learning a deep convolutional network for image super-resolution[C]// Proceedings of the 13 th European Conference European Conference on Computer Vision . Zurich, Switzerland: Springer, 2014: 184-199.

A MITTAL , R SOUNDARARAJAN , A C BOVIK . Making a "completely blind" image quality analyzer . IEEE Signal Processing Letters , 2013 . 20 ( 3 ): 209 - 212 . DOI: 10.1109/LSP.2012.2227726 http://doi.org/10.1109/LSP.2012.2227726 .

OU F Z, WANG Y G, ZHU G P. A novel blind image quality assessment method based on refined natural scene statistics[C]// Proceedings of 2019 IEEE International Conference on Image Processing ( ICIP ). Taipei, China: IEEE, 2019: 1004-1008.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

残差学习与层注意力相结合的轻量级图像超分辨

基于CNN的异构FPGA硬件加速器设计

基于CNN-Transformer结构的遥感影像变化检测