Lightweight underwater image enhancement network based on GAN

LIU Hao-xuan; LIN Shan-ling; LIN Zhi-xian; GUO Tai-liang; LIN Jian-pu

doi:10.37188/CJLCD.2022-0212

您当前的位置：

首页 >

文章列表页 >

Lightweight underwater image enhancement network based on GAN

Image Processing | 更新时间：2023-03-15

- Lightweight underwater image enhancement network based on GAN
- Chinese Journal of Liquid Crystals and Displays Vol. 38, Issue 3, Pages: 378-386(2023)
- 作者机构：
  
  1.福州大学先进制造学院，福建泉州 3622001
  2.福州大学物理与信息工程学院，福建福州 350116
  3.中国福建光电信息科学与技术创新实验室，福建福州 350116
- 作者简介：
- 基金信息：
  
  National Key R&D Program of China(2021YFB3600603);Projects supported by Natural Science Foundation of Fujian Province(2020J01468);Educational Research Project for Young and Middle-aged Teachers of Fujian Provincial Department of Education(JAT210030)
- DOI：10.37188/CJLCD.2022-0212
  CLC：
扫描看全文
LIU Hao-xuan, LIN Shan-ling, LIN Zhi-xian, et al. Lightweight underwater image enhancement network based on GAN. [J]. Chinese Journal of Liquid Crystals and Displays 38(3):378-386(2023)
DOI：

LIU Hao-xuan, LIN Shan-ling, LIN Zhi-xian, et al. Lightweight underwater image enhancement network based on GAN. [J]. Chinese Journal of Liquid Crystals and Displays 38(3):378-386(2023) DOI： 10.37188/CJLCD.2022-0212.

摘要

由于光在水下存在吸收与散射，导致水下图像存在颜色失真和细节损失，严重影响了后续水下目标的检测和识别。本文提出了一种轻量级全卷积层的生成式对抗神经网络模型（DUnet-GAN）来增强水下图像。针对水下图像的特点，提出了多任务目标函数，使得模型从感知图像的整体内容、颜色、局部纹理和风格信息等方面来增强图像的质量。此外，与现有的一些重要的模型做了对比，进行了定量的评估。结果表明，在EUVP数据集中本文所提模型峰值信噪比在26 dB以上，结构相似度为0.8，参数量为11 MB，仅为其他达到同等性能模型参数量的5%且比26 MB参数量的FUNIE-GAN指标更好。同时UIQM为2.85，仅次于Cycle-GAN模型，且主观增强效果显著。更重要的是，增强后的图像为水下目标检测等模型提供了更好的性能，也满足了水下机器人等设备对模型的轻量化要求。

Abstract

Due to the absorption and scattering of underwater light， the underwater image suffers from distortion and loss of details， which seriously affects the detection and recognition of subsequent underwater target. In this paper， a lightweight fully convolutional layer generative adversarial neural network DUnet-GAN is proposed to enhance underwater image. According to the characteristics of underwater image， this paper proposes a multi-task objective function， which enables the model to enhance the image quality by perceiving the overall content， color， local texture and style information of the image. In addition， we compare DUnet-GAN with some important existing models and make a quantitative evaluation. The results show that in EUVP dataset， the PSNR of the proposed model is above 26 dB， the SSIM is 0.8， and the number of parameters is 11 MB， which is only 5% of the number of parameters of other models with the same performance and better than the FunIE-GAN with 26 MB parameters. Meanwhile， UIQM is 2.85， second only to Cycle-GAN model， and the enhancement effect is significant subjectively. More importantly， the enhanced image provides better performance for underwater target detection and other models， and also meets the lightweight requirements of models for equipment such as underwater robots.

关键词

生成式对抗神经网络图像增强轻量级生成器目标检测

Keywords

generative adversarial networksimage enhancementlightweightgeneratorobject detection

references

BERMAN D， LEVY D， AVIDAN S， et al. Underwater single image color restoration using haze-lines and a new quantitative dataset ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（8）： 2822-2837. doi: 10.1109/TPAMI.2020.2977624http://dx.doi.org/10.1109/TPAMI.2020.2977624

ZHANG R， ISOLA P， EFROS A A. Colorful image colorization ［C］//Proceedings of the 14th European Conference on Computer Vision. Amsterdam： Springer， 2016： 649-666. doi: 10.1007/978-3-319-46487-9_40http://dx.doi.org/10.1007/978-3-319-46487-9_40

CHENG Z Z， YANG Q X， SHENG B. Deep colorization ［C］//Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago： IEEE， 2015： 415-423. doi: 10.1109/iccv.2015.55http://dx.doi.org/10.1109/iccv.2015.55

CAI B L， XU X M， JIA K， et al. DehazeNet： an end-to-end system for single image haze removal ［J］. IEEE Transactions on Image Processing， 2016， 25（11）： 5187-5198. doi: 10.1109/tip.2016.2598681http://dx.doi.org/10.1109/tip.2016.2598681

ZHU J Y， PARK T， ISOLA P， et al. Unpaired image-to-image translation using cycle-consistent adversarial networks ［C］//Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice： IEEE， 2017： 2223-2232. doi: 10.1109/iccv.2017.244http://dx.doi.org/10.1109/iccv.2017.244

YI Z L， ZHANG H， TAN P， et al. DualGAN： unsupervised dual learning for image-to-image translation ［C］//Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice： IEEE， 2017： 2849-2857. doi: 10.1109/iccv.2017.310http://dx.doi.org/10.1109/iccv.2017.310

LI J， SKINNER K， EUSTICE R M， et al. WaterGAN： Unsupervised generative network to enable real-time color correction of monocular underwater images ［J］. IEEE Robotics and Automation Letters， 2018， 3（1）： 387-394.

ISOLA P， ZHU J Y， ZHOU T H， et al. Image-to-image translation with conditional adversarial networks ［C］//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu： IEEE， 2017： 1125-1134. doi: 10.1109/cvpr.2017.632http://dx.doi.org/10.1109/cvpr.2017.632

FABBRI C， ISLAM M， SATTAR J. Enhancing underwater imagery using generative adversarial networks ［C］//Proceedings of 2018 IEEE International Conference on Robotics and Automation （ICRA）. Brisbane： IEEE， 2018： 7159-7165. doi: 10.1109/icra.2018.8460552http://dx.doi.org/10.1109/icra.2018.8460552

MIRZA M， OSINDERO S. Conditional generative adversarial nets ［J/OL］. arXiv， 2014： 1411.1784.

ISLAM M J， XIA Y Y， SATTAR J. Fast underwater image enhancement for improved visual perception ［J］. IEEE Robotics and Automation Letters， 2020， 5（2）： 3227-3234. doi: 10.1109/lra.2020.2974710http://dx.doi.org/10.1109/lra.2020.2974710

LI C Y， GUO J C， GUO C L. Emerging from water： underwater image color correction based on weakly supervised color transfer ［J］. IEEE Signal Processing Letters， 2018， 25（3）： 323-327. doi: 10.1109/lsp.2018.2792050http://dx.doi.org/10.1109/lsp.2018.2792050

YU X L， QU Y Y， HONG M. Underwater-GAN： underwater image restoration via conditional generative adversarial network ［C］//Proceedings of 2018 International Conference on Pattern Recognition. Beijing： Springer， 2019： 66-75. doi: 10.1007/978-3-030-05792-3_7http://dx.doi.org/10.1007/978-3-030-05792-3_7

LIU P， WANG G Y， QI H， et al. Underwater image enhancement with a deep residual framework ［J］. IEEE Access， 2019， 7： 94614-94629. doi: 10.1109/access.2019.2928976http://dx.doi.org/10.1109/access.2019.2928976

YANG H H， HUANG K C， CHEN W T. LAFFNet： a lightweight adaptive feature fusion network for underwater image enhancement ［C］//Proceedings of the 2021 IEEE International Conference on Robotics and Automation （ICRA）. Xi'an： IEEE， 2021： 685-692. doi: 10.1109/icra48506.2021.9561263http://dx.doi.org/10.1109/icra48506.2021.9561263

FU X Y， CAO X Y. Underwater image enhancement with global-local networks and compressed-histogram equalization ［J］. Signal Processing： Image Communication， 2020， 86： 115892. doi: 10.1016/j.image.2020.115892http://dx.doi.org/10.1016/j.image.2020.115892

ISLAM M J， ENAN S S， LUO P G， et al. Underwater image super-resolution using deep residual multipliers ［C］//Proceedings of the 2020 IEEE International Conference on Robotics and Automation （ICRA）. Paris： IEEE， 2020： 900-906. doi: 10.1109/icra40945.2020.9197213http://dx.doi.org/10.1109/icra40945.2020.9197213

ZHANG W D， DONG L L， ZHANG T， et al. Enhancing underwater image via color correction and Bi-interval contrast enhancement ［J］. Signal Processing： Image Communication， 2021， 90： 116030. doi: 10.1016/j.image.2020.116030http://dx.doi.org/10.1016/j.image.2020.116030

LUO W L， DUAN S Q， ZHENG J W. Underwater image restoration and enhancement based on a fusion algorithm with color balance， contrast optimization， and histogram stretching ［J］. IEEE Access， 2021， 9： 31792-31804. doi: 10.1109/access.2021.3060947http://dx.doi.org/10.1109/access.2021.3060947

RONNEBERGER O， FISCHER P， BROX T. U-Net： convolutional networks for biomedical image segmentation ［C］//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich： Springer， 2015： 234-241. doi: 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28

王建林，付雪松，黄展超，等.改进YOLOv2卷积神经网络的多类型合作目标检测［J］.光学精密工程，2020，28（1）：251-260. doi: 10.3788/ope.20202801.0251http://dx.doi.org/10.3788/ope.20202801.0251

WANG J L， FU X S， HANG Z C， et al. Multi-type cooperative targets detection using improved YOLOv2 convolutional neural network ［J］. Optics and Precision Engineering， 2020， 28（1）： 251-260. （in Chinese）. doi: 10.3788/ope.20202801.0251http://dx.doi.org/10.3788/ope.20202801.0251

BOCHKOVSKIY A， WANG C Y， LIAO H Y M. YOLOv4： Optimal speed and accuracy of object detection ［J/OL］. arXiv， 2020： 2004.10934.

JOHNSON J， ALAHI A， FEI-FEI L. Perceptual losses for real-time style transfer and super-resolution ［C］//Proceedings of the 14th European Conference on Computer Vision. Amsterdam： Springer， 2016： 694-711. doi: 10.1007/978-3-319-46475-6_43http://dx.doi.org/10.1007/978-3-319-46475-6_43

IGNATOV A, KOBYSHEV N, TIMOFTE R, et al. Dslr-quality photos on mobile devices with deep convolutional networks [C]//Proceedings of the IEEE international conference on computer vision. Venice:IEEE, 2017: 3277-3285. doi: 10.1109/iccv.2017.355http://dx.doi.org/10.1109/iccv.2017.355

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Object detection algorithm based on adaptive focal CRIoU loss

Mask wearing detection based on improved YOLOv7

Improved lightweight helmet wear detection algorithm

Lightweight and high-precision object detection algorithm based on YOLO framework

Related Author

No data

Related Institution

College of Software Engineering， Liaoning Technical University

Air Ammunition Research Institute Co. Ltd.， NORINCO Group， Haerbin

Shanghai Institute of Aerospace System Engineer

Quanzhou Institute of Equipment Manufacturing， Haixi Institutes， Chinese Academy of Sciences

School of Automation， Qingdao University

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰