Technical research of composite residual network in low illumination image enhancement

Xing-rui WANG; Yan PIAO; Yu-mo WANG

doi:10.37188/CJLCD.2021-0228

您当前的位置：

首页 >

文章列表页 >

Technical research of composite residual network in low illumination image enhancement

Image Processing | 更新时间：2022-06-14

- Technical research of composite residual network in low illumination image enhancement
- Chinese Journal of Liquid Crystals and Displays Vol. 37, Issue 4, Pages: 508-518(2022)
- 作者机构：
  
  长春理工大学电子信息工程学院，吉林长春 130022
- 作者简介：
- 基金信息：
  
  Project by the Ministry of Science and Technology of China(2015FR076)
- DOI：10.37188/CJLCD.2021-0228
  CLC： TP391.4
- Received：30 August 2021，
  
  Revised：21 October 2021，
  
  Published：05 April 2022
- 稿件说明：
移动端阅览
Xing-rui WANG, Yan PIAO, Yu-mo WANG. Technical research of composite residual network in low illumination image enhancement[J]. Chinese journal of liquid crystals and displays, 2022, 37(4): 508-518.
DOI：

Xing-rui WANG, Yan PIAO, Yu-mo WANG. Technical research of composite residual network in low illumination image enhancement[J]. Chinese journal of liquid crystals and displays, 2022, 37(4): 508-518. DOI： 10.37188/CJLCD.2021-0228.

摘要

成像设备在暗光照环境下会出现对比度不高、图像细节信息丢失、颜色失真等问题，这会对视频监控、智能交通、人脸识别等应用场景产生巨大干扰。为了解决这一问题，本文提出了一种融合了注意力机制的的复合残差网络来实现对低照度图像的增强。该算法首先通过色彩空间上的转换（RGB-HSV）将亮度分量

放入构造的神经网络中，然后神经网络通过融合了注意力机制的多分支结构进行图像浅层特征的提取，接着经过复合残差网络提取深层特征，再经过图像重建得到增强后的

分量，最后通过分量融合实现图像增强。实验结果表明，对比目前国内外主流低照度图像增强算法，所提算法在主观视觉上对图像亮度与对比度有显著提升，在PSNR、SSIM指标上与传统算法的对比结果分别提升了约20%和15%，与深度学习算法的对比结果分别提升约9%和3%，不论是在人工合成的低照度图像还是真实、自然低照度图像中均有良好表现，基本满足图像增强的颜色自然、对比度和鲁棒性高等要求。

Abstract

Imaging equipment in a dark environment has problems such as low contrast， loss of image detail information， and color distortion， which can cause huge interference in application scenarios such as video surveillance， intelligent transportation， and face recognition. In order to solve this situation， this paper proposes a composite residual network that incorporates the attention mechanism to enhance low-illuminance images. The algorithm firstly puts the brightness component

into the constructed neural network through color space conversion （RGB-HSV）. The neural network extracts the shallow features of the image through a multi-branch structure that incorporates the attention mechanism， passes through the composite residual network extracts deep features， and then reconstructs the image to obtain the enhanced

component. Finally， the low-illuminance image enhancement is achieved through component fusion. The experimental results show that compared with the current mainstream low illumination image enhancement algorithms at home and abroad， the proposed algorithm significantly improves the image brightness and contrast in subjective vision. Compared with the traditional algorithm， the PSNR and SSIM are improved about 20% and 15%， respectively. And compared with the deep learning algorithm， the PSNR and SSIM are improved about 9% and 3%， respectively. It performs well in artificially synthesized low-light images or real and natural low-light images， basically meet the requirements of natural color， contrast and robustness for image enhancement.

关键词

Keywords

references

ZUIDERVELD K . Contrast limited adaptive histogram equalization ［M］// HECKBERT P S . Graphics Gems. Amsterdam ： Elsevier ， 1994 ： 474 - 485 . doi: 10.1016/b978-0-12-336156-1.50061-6 http://dx.doi.org/10.1016/b978-0-12-336156-1.50061-6

IBRAHIM H ， KONG N S P . Brightness preserving dynamic histogram equalization for image contrast enhancement ［J］. IEEE Transactions on Consumer Electronics ， 2007 ， 53 （ 4 ）： 1752 - 1758 . doi: 10.1109/tce.2007.4429280 http://dx.doi.org/10.1109/tce.2007.4429280

LAND E H . The Retinex theory of color vision ［J］. Scientific American ， 1977 ， 237 （ 6 ）： 108 - 128 . doi: 10.1038/scientificamerican1277-108 http://dx.doi.org/10.1038/scientificamerican1277-108

JOBSON D J ， RAHMAN Z ， WOODELL G A . Properties and performance of a center/surround Retinex ［J］. IEEE Transactions on Image Processing ， 1997 ， 6 （ 3 ）： 451 - 462 . doi: 10.1109/83.557356 http://dx.doi.org/10.1109/83.557356

RAHMAN Z ， JOBSON D J ， WOODELL G A . Multi-scale Retinex for color image enhancement ［C］// Proceedings of 3rd IEEE International Conference on Image Processing . Lausanne ： IEEE ， 1996 ： 1003 - 1006 . doi: 10.1117/12.258224 http://dx.doi.org/10.1117/12.258224

黄慧，董林鹭，刘小芳，等 . 改进Retinex的低光照图像增强［J］. 光学精密工程， 2020 ， 28 （ 8 ）： 1835 - 1849 . doi: 10.3788/OPE.20202808.1835 http://dx.doi.org/10.3788/OPE.20202808.1835

HUANG H ， DONG L L ， LIU X F ， et al . Improved Retinex low light image enhancement method ［J］. Optics and Precision Engineering ， 2020 ， 28 （ 8 ）： 1835 - 1849 . （in Chinese） . doi: 10.3788/OPE.20202808.1835 http://dx.doi.org/10.3788/OPE.20202808.1835

冯维，吴贵铭，赵大兴，等 . 多图像融合Retinex用于弱光图像增强［J］. 光学精密工程， 2020 ， 28 （ 3 ）： 736 - 744 . doi: 10.3788/OPE.20202803.0736 http://dx.doi.org/10.3788/OPE.20202803.0736

FENG W ， WU G M ， ZHAO D X ， et al . Multi images fusion Retinex for low light image enhancement ［J］. Optics and Precision Engineering ， 2020 ， 28 （ 3 ）： 736 - 744 . （in Chinese） . doi: 10.3788/OPE.20202803.0736 http://dx.doi.org/10.3788/OPE.20202803.0736

王卫星，赵恒 . 结合改进Retinex及自适应分数阶微分的雾霾公路交通图像增强［J］. 光学精密工程， 2020 ， 28 （ 8 ）： 1820 - 1834 .

WANG W X ， ZHAO H . Haze traffic image enhancement based on improved Retinex and adaptive fractional differential ［J］. Optics and Precision Engineering ， 2020 ， 28 （ 8 ）： 1820 - 1834 . （in Chinese）

CHENG Y ， YAN J ， WANG Z . Enhancement of weakly illuminated images by deep fusion networks ［C］// 2019 IEEE International Conference on Image Processing . Taipei， China ： IEEE ， 2019 ： 924 - 928 . doi: 10.1109/icip.2019.8803041 http://dx.doi.org/10.1109/icip.2019.8803041

GUO X J ， LI Y ， LING H B . LIME： low-light image enhancement via illumination map estimation ［J］. IEEE Transactions on Image Processing ， 2017 ， 26 （ 2 ）： 982 - 993 . doi: 10.1109/tip.2016.2639450 http://dx.doi.org/10.1109/tip.2016.2639450

LI C Y ， GUO J C ， PORIKLI F ， et al . LightenNet： a convolutional neural network for weakly illuminated image enhancement ［J］. Pattern Recognition Letters ， 2018 ， 104 ： 15 - 22 . doi: 10.1016/j.patrec.2018.01.010 http://dx.doi.org/10.1016/j.patrec.2018.01.010

MORAN S ， MARZA P ， MCDONAGH S ， et al . DeepLPF： deep local parametric filters for image enhancement ［C］// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR） . Seattle ： IEEE ， 2020 ： 12823 - 12832 . doi: 10.1109/cvpr42600.2020.01284 http://dx.doi.org/10.1109/cvpr42600.2020.01284

JIANG Y F ， GONG X Y ， LIU D ， et al . EnlightenGAN： deep light enhancement without paired supervision ［J］. IEEE Transactions on Image Processing ， 2021 ， 30 ： 2340 - 2349 . doi: 10.1109/tip.2021.3051462 http://dx.doi.org/10.1109/tip.2021.3051462

陈清江，屈梅 . 基于双残差卷积网络的低照度图像增强［J］. 液晶与显示， 2021 ， 36 （ 2 ）： 305 - 316 . doi: 10.37188/CJLCD.2020-0168 http://dx.doi.org/10.37188/CJLCD.2020-0168

CHEN Q J ， QU M . Low-light image enhancement based on dual-residual convolutional network ［J］. Chinese Journal of Liquid Crystals and Displays ， 2021 ， 36 （ 2 ）： 305 - 316 . （in Chinese） . doi: 10.37188/CJLCD.2020-0168 http://dx.doi.org/10.37188/CJLCD.2020-0168

黄辉先，陈凡浩 . 基于注意力机制和Retinex的低照度图像增强方法［J］. 激光与光电子学进展， 2020 ， 57 （ 20 ）： 201004 . doi: 10.3788/LOP57.201004 http://dx.doi.org/10.3788/LOP57.201004

HUANG H X ， CHEN F H . Low-illumination image enhancement method based on attention mechanism and Retinex ［J］. Laser & Optoelectronics Progress ， 2020 ， 57 （ 20 ）： 201004 . （in Chinese） . doi: 10.3788/LOP57.201004 http://dx.doi.org/10.3788/LOP57.201004

WOO S ， PARK J ， LEE J Y ， et al . CBAM： convolutional block attention module ［C］// Proceedings of the 15th European Conference on Computer Vision . Munich ： Springer ， 2018 ： 3 - 19 . doi: 10.1007/978-3-030-01234-2_1 http://dx.doi.org/10.1007/978-3-030-01234-2_1

SZEGEDY C ， LIU W ， JIA Y Q ， et al . Going deeper with convolutions ［C］// 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston ： IEEE ， 2015 ： 1 - 9 . doi: 10.1109/cvpr.2015.7298594 http://dx.doi.org/10.1109/cvpr.2015.7298594

WEI C ， WANG W J ， YANG W H ， et al . Deep retinex decomposition for low-light enhancement ［J］. arXiv ， 2018 ： 1808 .04560.

MA K D ， WU Q B ， WANG Z ， et al . Group MAD competition？ A new methodology to compare objective image quality models ［C］// 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . Las Vegas ： IEEE ， 2016 ： 1664 - 1673 . doi: 10.1109/cvpr.2016.184 http://dx.doi.org/10.1109/cvpr.2016.184

CHENG H D ， SHI X J . A simple and effective histogram equalization approach to image enhancement ［J］. Digital Signal Processing ， 2004 ， 14 （ 2 ）： 158 - 170 . doi: 10.1016/j.dsp.2003.07.002 http://dx.doi.org/10.1016/j.dsp.2003.07.002

Views

145

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Correction of vignetting images based on Retinex-Net network model

Fast and robust low-light image enhancement based on iterative propagation network

Improved neural network nonuniformity correction algorithm based on side window filter

FRKDNet： feature refine semantic segmentation network based on knowledge distillation

Related Author

Yu-mo WANG

HUANG Dandan

WANG Fei

LIU Zhi

GAO Han

WANG Huiji

XIAO Zhibo

JIANG Zhilong

Related Institution

Space Optoelectronics Technology National and Local Joint Engineering Research Center， Changchun University of Technology

School of Electronic Information Engineering， Changchun University of Technology

School of Science， Jiangnan University

Changchun Institute of Optics， Fine Mechanics and Physics， Chinese Academy of Sciences

University of Chinese Academy of Sciences

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰