HDR image processing algorithm for portrait based on multi feature fusion

WU Chunlin; ZHANG Yongai; LIN Zhixian; GUO Tailiang; LIN Pengfei; LIN Jianpu

doi:10.37188/CJLCD.2023-0255

您当前的位置：

首页 >

文章列表页 >

HDR image processing algorithm for portrait based on multi feature fusion

Image Enhancement | 更新时间：2024-08-16

- HDR image processing algorithm for portrait based on multi feature fusion
- Chinese Journal of Liquid Crystals and Displays Vol. 39, Issue 8, Pages: 1024-1036(2024)
- 作者机构：
  
  1.福州大学先进制造学院，福建泉州 362200
  2.福州大学物理与信息工程学院，福建福州 350108
  3.东京大学信息科学技术学院，日本东京 113-8657
- 作者简介：
- 基金信息：
  
  National Key R&D Program of China(2021YFB3600603);Projects supported by Natural Science Foundation of Fujian Province(2020J01468);Educational Research Project for Young and Middle-aged Teachers of Fujian Provincial Department of Education(JAT210030)
- DOI：10.37188/CJLCD.2023-0255
  CLC： TP391
- Received：19 July 2023，
  
  Revised：12 October 2023，
  
  Published：05 August 2024
- 稿件说明：
移动端阅览
WU Chunlin, ZHANG Yongai, LIN Zhixian, et al. HDR image processing algorithm for portrait based on multi feature fusion[J]. Chinese journal of liquid crystals and displays, 2024, 39(8): 1024-1036.
DOI：

WU Chunlin, ZHANG Yongai, LIN Zhixian, et al. HDR image processing algorithm for portrait based on multi feature fusion[J]. Chinese journal of liquid crystals and displays, 2024, 39(8): 1024-1036. DOI： 10.37188/CJLCD.2023-0255.

摘要

基于深度学习的高动态范围（High Dynamic Range，HDR）图像处理算法在处理含有人像的图片时存在皮肤偏色问题。针对此问题，本文提出了一种基于多特征融合的人像HDR图像处理算法U²HDRnet。该算法由皮肤特征提取模块、三边特征提取模块、色彩重建模块3部分构成。首先，皮肤特征提取模块分离出皮肤区域的颜色和位置信息；其次，三边特征提取模块分别提取图片的局部特征、全局特征和语义特征，并与皮肤特征融合；最后，色彩重建模块对网格做空间和颜色深度上的插值。此外，本文引入改进的自注意力与卷积融合模块以提升HDR的处理效果；同时本文还制作了人像HDR数据集PortraitHDR，填补了该领域内数据集的空白。实验结果显示，U²HDRnet的PSNR达31.42 dB，SSIM达0.985，均优于目前常见的HDR算法，在获得高质量人像HDR图像的同时避免了皮肤的失真。

Abstract

Deep learning based high dynamic range （HDR） image processing algorithms has the problem of skin color deviation when processing images containing human figures. In response to this issue， this article proposes a portrait HDR image processing algorithm based on multi feature fusion-U²HDRnet. This algorithm consists of three parts： skin feature extraction module， trilateral feature extraction module and color reconstruction module. Firstly， the skin feature extraction module separates the color and position information of the skin region. Secondly， the trilateral feature extraction module extracts local features， global features and semantic features of the image， and fuses them with skin features. Finally， the color reconstruction module interpolates the grid in terms of space and color depth. In addition， this article adds an improved fusion module of self attention and convolution to improve the processing performance of HDR. At the same time， this article also produces the PortraitHDR dataset for portraits， filling the gap in the dataset in this field. The test results show that the PSNR of U²HDRnet reaches 31.42 dB， and the SSIM reaches 0.985， both of which are superior to the commonly used HDR algorithms. They obtain high-quality portrait HDR images while avoiding skin distortion.

关键词

Keywords

references

吴玲风，李娜，胡骏保 . 基于重映射和曝光融合的HDR成像［J］. 液晶与显示， 2021 ， 36 （ 12 ）： 1712 - 1719 . doi: 10.37188/CJLCD.2021-0112 http://dx.doi.org/10.37188/CJLCD.2021-0112

WU L F ， LI N ， HU J B . HDR imaging using remapping and exposure fusion ［J］. Chinese Journal of Liquid Crystals and Displays ， 2021 ， 36 （ 12 ）： 1712 - 1719 . （in Chinese） . doi: 10.37188/CJLCD.2021-0112 http://dx.doi.org/10.37188/CJLCD.2021-0112

芦碧波，皇甫珍珍，郭凯，等 . 基于双边滤波的多尺度分层色调映射算法［J］. 液晶与显示， 2018 ， 33 （ 9 ）： 816 - 822 . doi: 10.3788/yjyxs20183309.0816 http://dx.doi.org/10.3788/yjyxs20183309.0816

LU B B ， HUANGFU Z Z ， GUO K ， et al . Bilateral filter based multiscale layer tone mapping algorithm ［J］. Chinese Journal of Liquid Crystals and Displays ， 2018 ， 33 （ 9 ）： 816 - 822 . （in Chinese） . doi: 10.3788/yjyxs20183309.0816 http://dx.doi.org/10.3788/yjyxs20183309.0816

XIA X D ， ZHANG M ， XUE T F ， et al . Joint bilateral learning for real-time universal photorealistic style transfer ［C］. 16th European Conference on Computer Vision . Glasgow ： Springer ， 2020 ： 327 - 342 . doi: 10.1007/978-3-030-58598-3_20 http://dx.doi.org/10.1007/978-3-030-58598-3_20

LEE M J ， RHEE C H ， LEE C H . HSVNet： reconstructing HDR image from a single exposure LDR image with CNN ［J］. Applied Sciences ， 2022 ， 12 （ 5 ）： 2370 . doi: 10.3390/app12052370 http://dx.doi.org/10.3390/app12052370

KHAN Z ， KHANNA M ， RAMAN S . FHDR： HDR image reconstruction from a single LDR image using feedback network ［C］. IEEE Global Conference on Signal and Information Processing （GlobalSIP） . Ottawa ： IEEE ， 2019 ： 1 - 5 . doi: 10.1109/globalsip45357.2019.8969167 http://dx.doi.org/10.1109/globalsip45357.2019.8969167

LIU Y L ， LAI W S ， CHEN Y S ， et al . Single-image HDR reconstruction by learning to reverse the camera pipelin ［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle ： IEEE ， 2020 ： 1648 - 1657 . doi: 10.1109/cvpr42600.2020.00172 http://dx.doi.org/10.1109/cvpr42600.2020.00172

LIBA O ， MOVSHOVITZ-ATTIAS Y ， CAI L Q ， et al . Sky optimization： semantically aware image processing of skies in low-light photography ［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Seattle ： IEEE ， 2020 ： 2230 - 2238 . doi: 10.1109/cvprw50498.2020.00271 http://dx.doi.org/10.1109/cvprw50498.2020.00271

MNIH V ， HEESS N ， GRAVES A ， et al . Recurrent models of visual attention ［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems . Montreal ： MIT Press ， 2014 ： 2204 - 2212 .

ZHANG Y L ， LI K P ， LI K ， et al . Image super-resolution using very deep residual channel attention networks ［C］// Proceedings of the 15th European Conference on Computer Vision （ECCV） . Munich ： Springer ， 2018 ： 294 - 310 . doi: 10.1007/978-3-030-01234-2_18 http://dx.doi.org/10.1007/978-3-030-01234-2_18

ZHAO H S ， JIA J Y ， KOLTUN V . Exploring self-attention for image recognition ［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle ： IEEE ， 2020 ： 10073 - 10082 . doi: 10.1109/cvpr42600.2020.01009 http://dx.doi.org/10.1109/cvpr42600.2020.01009

FU J ， LIU J ， TIAN H J ， et al . Dual attention network for scene segmentation ［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach ： IEEE ， 2019 ： 3141 - 3149 . doi: 10.1109/cvpr.2019.00326 http://dx.doi.org/10.1109/cvpr.2019.00326

HASSANI A ， WALTON S ， LI J C ， et al . Neighborhood attention transformer ［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Vancouver ： IEEE ， 2023 ： 6185 - 6194 . doi: 10.1109/cvpr52729.2023.00599 http://dx.doi.org/10.1109/cvpr52729.2023.00599

PAN X R ， GE C J ， LU R ， et al . On the integration of self-attention and convolution ［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans ： IEEE ， 2022 ： 805 - 815 . doi: 10.1109/cvpr52688.2022.00089 http://dx.doi.org/10.1109/cvpr52688.2022.00089

GHARBI M ， CHEN J W ， BARRON J T ， et al . Deep bilateral learning for real-time image enhancement ［J］. ACM Transactions on Graphics ， 2017 ， 36 （ 4 ）： 118 . doi: 10.1145/3072959.3073592 http://dx.doi.org/10.1145/3072959.3073592

HASHEMIFARD K ， FLOREZ-REVUELTA F . From garment to skin： the visuAAL skin segmentation dataset ［C］. ICIAP 2022 —Workshops on Image Analysis and Processing . Lecce ： Springer ， 2022 ： 59 - 70 . doi: 10.1007/978-3-031-13321-3_6 http://dx.doi.org/10.1007/978-3-031-13321-3_6

QIN X B ， ZHANG Z C ， HUANG C Y ， et al . U 2 -Net： going deeper with nested U-structure for salient object detection ［J］. Pattern Recognition ， 2020 ， 106 ： 107404 . doi: 10.1016/j.patcog.2020.107404 http://dx.doi.org/10.1016/j.patcog.2020.107404

BADRINARAYANAN V ， KENDALL A ， CIPOLLA R . SegNet： a deep convolutional encoder-decoder architecture for image segmentation ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence ， 2017 ， 39 （ 12 ）： 2481 - 2495 . doi: 10.1109/tpami.2016.2644615 http://dx.doi.org/10.1109/tpami.2016.2644615

SHEN Z R ， BELLO I ， VEMULAPALLI R ， et al . Global self-attention networks for image recognition ［J/OL］. arXiv ， 2020 ： 2010 . 03019 .

LONG J ， SHELHAMER E ， DARRELL T . Fully convolutional networks for semantic segmentation ［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Boston ： IEEE ， 2015 ： 3431 - 3440 . doi: 10.1109/cvpr.2015.7298965 http://dx.doi.org/10.1109/cvpr.2015.7298965

RONNEBERGER O ， FISCHER P ， BROX T . U-Net： convolutional networks for biomedical image segmentation ［C］. 18th International Conference on Medical Image Computing and Computer-Assisted Intervention . Munich ： Springer ， 2015 ： 234 - 241 . doi: 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28

ZHOU Z W ， SIDDIQUEE M M R ， TAJBAKHSH N ， et al . UNet++： a nested U-Net architecture for medical image segmentation ［C］. 4th International Workshop on Deep Learning in Medical Image Analysis and 8th International Workshop on Multimodal Learning for Clinical Decision Support . Granada ： Springer ， 2018 ： 3 - 11 . doi: 10.1007/978-3-030-00889-5_1 http://dx.doi.org/10.1007/978-3-030-00889-5_1

HUANG H M ， LIN L F ， TONG R F ， et al . UNet 3+： a full-scale connected UNet for medical image segmentation ［C］. 2020 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP） . Barcelona ： IEEE ， 2020 ： 1055 - 1059 . doi: 10.1109/icassp40776.2020.9053405 http://dx.doi.org/10.1109/icassp40776.2020.9053405

LAI W S ， HUANG J B ， AHUJA N ， et al . Fast and accurate image super-resolution with deep laplacian pyramid networks ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence ， 2019 ， 41 （ 11 ）： 2599 - 2613 . doi: 10.1109/tpami.2018.2865304 http://dx.doi.org/10.1109/tpami.2018.2865304

SIMONYAN K ， ZISSERMAN A . Very deep convolutional networks for large-scale image recognition ［C］// Proceedings of the 3rd International Conference on Learning Representations （ICLR） . San Diego， Curran Associates Inc. ， 2015 ： 1 - 14 .

Views

164

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Remote sensing scene classification model based on improved ShuffleNetV2 network

Lens-free adversarial encoded imaging method based on convolutional attention mechanism

Attention and cross-scale fusion for vehicle and pedestrian detection

Infrared dim-small target detection under complex background based on attention mechanism

Related Author

WU Chun-lin

ZHANG Yong-ai

LIN Zhi-xian

GUO Tai-liang

LIN Peng-fei

LIN Jian-pu

XU Huiwen

ZHAO Weichao

Related Institution

Digital Center， Changchun Institute of Optics， Fine Mechanics and Physics， Chinese Academy of Sciences

Optica Information Science and Technology Department， Jiangnan University

College of Software， Liaoning Technical University

College of Mining， Liaoning Technical University

Changchun Institute of Optics， Fine Mechanics and Physics， Chinese Academy of Sciences

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰