浏览全部资源
扫码关注微信
1.中国科学院 长春光学精密机械与物理研究所, 吉林 长春 130033
2.中国科学院大学, 北京 100049
Received:01 September 2022,
Revised:23 September 2022,
Published:05 July 2023
移动端阅览
WANG Xian-tao, ZHAO Jin-yu. Fusion of NSCT infrared and visible images based on improved FT saliency detection[J]. Chinese journal of liquid crystals and displays, 2023, 38(7): 933-944.
WANG Xian-tao, ZHAO Jin-yu. Fusion of NSCT infrared and visible images based on improved FT saliency detection[J]. Chinese journal of liquid crystals and displays, 2023, 38(7): 933-944. DOI: 10.37188/CJLCD.2022-0286.
为了克服传统算法存在对比度低、细节和纹理缺失严重以及基于显著性检测算法存在对噪声敏感、适应性和抗干扰能力不强的问题,提出了一种基于改进的频率调谐(FT)显著性检测的非下采样轮廓波变换(NSCT)红外与可见光融合方法。首先,采用改进的显著图提取算法瞄准红外图像,用于从背景中区分目标。其次,使用NSCT对红外图像和可见光图像进行高、低频子带的分解。利用计算得到的红外显著权重图对低频子带系数进行指导融合,可以很好地保留目标和背景之间的对比度;对高频部分采用局部加权能量的规则进行抉择,再通过加权最小二乘(WLS)优化可以获得更多的细节信息和减小噪声影响。最后,对融合后的高频和低频子带系数进行NSCT逆变换得到最终的融合图像。通过4组图像的实验对比结果可知,在主观上,本文方法相较于其他方法具有目标突出、细节提取丰富、边缘伪影现象消除明显、视觉效果更好的优点。本文方法在4个客观评价指标——平均梯度(AG)、信息熵(IE)、空间频率(SF)、互信息(MI)上都处于最好的状态,与5种对比方法相比,AG、IE、SF、MI的平均值分别提高了8.19%、5.34%、8.54%、68.18%,说明了所提出方法的可靠性和有效性。
In order to overcome the problems of low contrast, serious lack of details and textures in traditional algorithms, and noise sensitivity, weak adaptability and anti-interference ability of saliency-based detection algorithms, a non-subsampled contourlet transform (NSCT) infrared and visible light fusion method based on improved frequency tuning (FT) saliency detection is proposed. Firstly, an improved saliency map extraction algorithm is used to target the infrared image to distinguish the target from the background. Secondly, the infrared image and visible light image are decomposed into high and low frequency subbands using NSCT. The coefficients are used to guide the fusion, which can well preserve the contrast between the target and the background. For the high-frequency part, the rule of local weighted energy is used to make a decision, the weighted least squares (WLS) optimization can obtain more detailed information and reduce the influence of noise, and the fused high-frequency and low-frequency subbands coefficients are processed. Finally, the fused image is obtained by inverse NSCT transform. The experimental comparison results of four sets of images show that the method in this paper has better visual effect than other methods in terms of prominent target, rich details extraction, and obvious elimination of edge artifacts, subjectively. It is in the best state on the average gradient (AG), information entropy (IE), spatial frequency (SF), and mutual information (MI) of four objective evaluation indicators. Compared with the average of the five comparison methods, the AG, IE, SF, and MI are increased by 8.19%, 5.34%, 8.54%, and 68.18%, respectively. Thus, the proposed method is reliable and effective.
程博阳 , 李婷 , 王喻林 . 基于视觉显著性加权与梯度奇异值最大的红外与可见光图像融合 [J]. 中国光学(中英文) , 2022 , 15 ( 4 ): 675 - 688 . doi: 10.37188/co.2022-0124 http://dx.doi.org/10.37188/co.2022-0124
CHENG B Y , LI T , WANG Y L . Fusion of infrared and visible light images based on visual saliency weighting and maximum gradient singular value [J]. China Optical , 2022 , 15 ( 4 ): 675 - 688 . (in Chinese) . doi: 10.37188/co.2022-0124 http://dx.doi.org/10.37188/co.2022-0124
JIN X , JIANG Q , YAO S W , et al . A survey of infrared and visual image fusion methods [J]. Infrared Physics & Technology , 2017 , 85 : 478 - 501 . doi: 10.1016/j.infrared.2017.07.010 http://dx.doi.org/10.1016/j.infrared.2017.07.010
MA J Y , MA Y , LI C . Infrared and visible image fusion methods and applications: a survey [J]. Information Fusion , 2019 , 45 : 153 - 178 . doi: 10.1016/j.inffus.2018.02.004 http://dx.doi.org/10.1016/j.inffus.2018.02.004
杨艳春 , 高晓宇 , 党建武 , 等 . 基于WEMD和生成对抗网络重建的红外与可见光图像融合 [J]. 光学 精密工程 , 2022 , 30 ( 3 ): 320 - 330 . doi: 10.37188/OPE.20223003.0320 http://dx.doi.org/10.37188/OPE.20223003.0320
YANG Y C , GAO X Y , DANG J W , et al . Infrared and visible image fusion based on WEMD and generative adversarial network reconstruction [J]. Optics and Precision Engineering , 2022 , 30 ( 3 ): 320 - 330 . (in Chinese) . doi: 10.37188/OPE.20223003.0320 http://dx.doi.org/10.37188/OPE.20223003.0320
郭玲 , 杨斌 . 基于视觉显著性的红外与可见光图像融合 [J]. 计算机科学 , 2015 , 42 ( S1 ): 211 - 214,235 .
GUO L , YANG B . Fusion of infrared and visible images based on visual saliency [J]. Computer Science , 2015 , 42 ( S1 ): 211 - 214,235 . (in Chinese)
丁贵鹏 , 陶钢 , 李英超 , 等 . 基于非下采样轮廓波变换与引导滤波器的红外及可见光图像融合 [J]. 兵工学报 , 2021 , 42 ( 9 ): 1911 - 1922 . doi: 10.3969/j.issn.1000-1093.2021.09.012 http://dx.doi.org/10.3969/j.issn.1000-1093.2021.09.012
DING G P , TAO G , LI Y C , et al . Infrared and visible images fusion based on non-subsampled contourlet transform and guided filter [J]. Acta Armamentarii , 2021 , 42 ( 9 ): 1911 - 1922 . (in Chinese) . doi: 10.3969/j.issn.1000-1093.2021.09.012 http://dx.doi.org/10.3969/j.issn.1000-1093.2021.09.012
林剑萍 , 廖一鹏 . 结合分数阶显著性检测及量子烟花算法的NSST域图像融合 [J]. 光学 精密工程 , 2021 , 29 ( 6 ): 1406 - 1419 . doi: 10.37188/OPE.20212906.1406 http://dx.doi.org/10.37188/OPE.20212906.1406
LIN J P , LIAO Y P . A novel image fusion method with fractional saliency detection and QFWA in NSST [J]. Optics and Precision Engineering , 2021 , 29 ( 6 ): 1406 - 1419 . (in Chinese) . doi: 10.37188/OPE.20212906.1406 http://dx.doi.org/10.37188/OPE.20212906.1406
LI S T , YANG B , HU J W . Performance comparison of different multi-resolution transforms for image fusion [J]. Information Fusion , 2011 , 12 ( 2 ): 74 - 84 . doi: 10.1016/j.inffus.2010.03.002 http://dx.doi.org/10.1016/j.inffus.2010.03.002
LI H F , QIU H M , YU Z T , et al . Infrared and visible image fusion scheme based on NSCT and low-level visual features [J]. Infrared Physics & Technology , 2016 , 76 : 174 - 184 . doi: 10.1016/j.infrared.2016.02.005 http://dx.doi.org/10.1016/j.infrared.2016.02.005
赵立昌 , 张宝辉 , 吴杰 , 等 . 基于灰度能量差异性的红外与可见光图像融合 [J]. 红外技术 , 2020 , 42 ( 8 ): 775 - 782 . doi: 10.3724/sp.j.7102614861 http://dx.doi.org/10.3724/sp.j.7102614861
ZHAO L C , ZHANG B H , WU J , et al . Fusion of infrared and visible images based on gray energy difference [J]. Infrared Technology , 2020 , 42 ( 8 ): 775 - 782 . (in Chinese) . doi: 10.3724/sp.j.7102614861 http://dx.doi.org/10.3724/sp.j.7102614861
林子慧 , 魏宇星 , 张建林 , 等 . 基于显著性图的红外与可见光图像融合 [J]. 红外技术 , 2019 , 41 ( 7 ): 640 - 645 .
LIN Z H , WEI Y X , ZHANG J L , et al . Image fusion of infrared and visible images based on saliency map [J]. Infrared Technology , 2019 , 41 ( 7 ): 640 - 645 . (in Chinese)
叶坤涛 , 李文 , 舒蕾蕾 , 等 . 结合改进显著性检测与NSST的红外与可见光图像融合方法 [J]. 红外技术 , 2021 , 43 ( 12 ): 1212 - 1221 .
YE K T , LI W , SHU L L , et al . Infrared and visible image fusion method based on improved saliency detection and non-subsampled shearlet transform [J]. Infrared Technology , 2021 , 43 ( 12 ): 1212 - 1221 . (in Chinese)
ZHANG X Y , MA Y , FAN F , et al . Infrared and visible image fusion via saliency analysis and local edge-preserving multi-scale decomposition [J]. Journal of the Optical Society of America A , 2017 , 34 ( 8 ): 1400 - 1410 . doi: 10.1364/josaa.34.001400 http://dx.doi.org/10.1364/josaa.34.001400
DUAN C W , LIU Y L , XING C D , et al . Infrared and visible image fusion using truncated Huber penalty function smoothing and visual saliency based threshold optimization [J]. Electronics , 2021 , 11 ( 1 ): 33 . doi: 10.3390/electronics11010033 http://dx.doi.org/10.3390/electronics11010033
BAVIRISETTI D P , DHULI R . Two-scale image fusion of visible and infrared images using saliency detection [J]. Infrared Physics & Technology , 2016 , 76 : 52 - 64 . doi: 10.1016/j.infrared.2016.01.009 http://dx.doi.org/10.1016/j.infrared.2016.01.009
GAN W , WU X H , WU W , et al . Infrared and visible image fusion with the use of multi-scale edge-preserving decomposition and guided image filter [J]. Infrared Physics & Technology , 2015 , 72 : 37 - 51 . doi: 10.1016/j.infrared.2015.07.003 http://dx.doi.org/10.1016/j.infrared.2015.07.003
MA J L , ZHOU Z Q , WANG B , et al . Infrared and visible image fusion based on visual saliency map and weighted least square optimization [J]. Infrared Physics & Technology , 2017 , 82 : 8 - 17 . doi: 10.1016/j.infrared.2017.02.005 http://dx.doi.org/10.1016/j.infrared.2017.02.005
CHAI P F , LUO X Q , ZHANG Z C . Image fusion using quaternion wavelet transform and multiple features [J]. IEEE Access , 2017 , 5 : 6724 - 6734 . doi: 10.1109/access.2017.2685178 http://dx.doi.org/10.1109/access.2017.2685178
GANASALA P , KUMAR V . CT and MR image fusion scheme in nonsubsampled contourlet transform domain [J]. Journal of Digital Imaging , 2014 , 27 ( 3 ): 407 - 418 . doi: 10.1007/s10278-013-9664-x http://dx.doi.org/10.1007/s10278-013-9664-x
王贤涛 , 赵金宇 . 基于多判断和加权最小二乘优化的NSCT红外和可见图像融合 [J]. 液晶与显示 , 2023 , 38 ( 2 ): 204 - 215 . doi: 10.37188/CJLCD.2022-0179 http://dx.doi.org/10.37188/CJLCD.2022-0179
WANG X T , ZHAO J Y . Fusion of NSCT infrared and visible images based on multi-judgment and WLS optimization [J]. Chinese Journal of Liquid Crystals and Displays , 2023 , 38 ( 2 ): 204 - 215 . (in Chinese) . doi: 10.37188/CJLCD.2022-0179 http://dx.doi.org/10.37188/CJLCD.2022-0179
0
Views
163
下载量
0
CSCD
Publicity Resources
Related Articles
Related Author
Related Institution