融合多尺度特征及注意力机制的医学图像检索

周林鹏; 姚剑敏; 严群; 林志贤

doi:10.37188/CJLCD.2020-0248

您当前的位置：

首页 >

文章列表页 >

融合多尺度特征及注意力机制的医学图像检索

图像处理 | 更新时间：2021-08-05

- 融合多尺度特征及注意力机制的医学图像检索
- Medicalimage retrieval with multiscale features and attention mechanisms
- 液晶与显示 2021年36卷第8期页码：1174-1185
- 作者机构：
  
  1.福州大学物理与信息工程学院, 福建福州 350108
  2.晋江市博感电子科技有限公司, 福建晋江 362200
- 作者简介：
  
  [ "周林鹏(1996-), 男, 江西赣州人, 硕士研究生, 2018年于江西理工大学获得学士学位, 主要从事深度学习、图像处理方面的研究。E-mail: 961031645@qq.com" ]
  [ "姚剑敏(1978-), 男, 福建莆田人, 博士, 副研究员, 2005年于中科院长春光学精密机械与物理研究所获得博士学位, 主要从事人工智能、图像处理、信息显示技术等方面的研究。E-mail: yaojm@fzu.edu.cn" ]
- 基金信息：
  
  国家重点研发计划课题(2016YFB0401503);广东省科技重大专项(2016B090906001);福建省科技重大专项(2014HZ0003-1);广东省光信息材料与技术重点实验室开放基金(2017B030301007)
- DOI：10.37188/CJLCD.2020-0248
  中图分类号： TP391.4;TH691.9
- 收稿日期：2020-09-24，
  
  修回日期：2021-01-18，
  
  纸质出版日期：2021-08
- 稿件说明：
移动端阅览
周林鹏, 姚剑敏, 严群, 等. 融合多尺度特征及注意力机制的医学图像检索[J]. 液晶与显示, 2021,36(8):1174-1185.

Lin-peng ZHOU, Jian-min YAO, Qun YAN, et al. Medicalimage retrieval with multiscale features and attention mechanisms[J]. Chinese journal of liquid crystals and displays, 2021, 36(8): 1174-1185.
周林鹏, 姚剑敏, 严群, 等. 融合多尺度特征及注意力机制的医学图像检索[J]. 液晶与显示, 2021,36(8):1174-1185. DOI： 10.37188/CJLCD.2020-0248.

Lin-peng ZHOU, Jian-min YAO, Qun YAN, et al. Medicalimage retrieval with multiscale features and attention mechanisms[J]. Chinese journal of liquid crystals and displays, 2021, 36(8): 1174-1185. DOI： 10.37188/CJLCD.2020-0248.

摘要

针对目前医学图像普遍存在病理区域尺寸分布较分散、细节特征不明显以及同类组织影像间的视觉差异较大等问题，本文在CBMIR系统的基础上，提出了一种融合多尺度特征及注意力机制的医学图像检索方法。该方法通过融合多尺度特征并设置可学习权重系数来自适应平衡浅层图像纹理特征和深层图像语义特征的关系，提高网络对不同尺度上的病理特征提取能力。同时，引入注意力模块，对网络输出的特征图进行通道加权求和，提高关键特征通道的特征表达能力，使网络更能关注到图像中的具有辨识性的病理特征区域。最后，在损失函数设计时，使用多重损失进一步优化样本特征在特征空间的分布。最终在Mura数据集上的mAP@100、mAP@20两个指标上分别达到了0.95、0.98的检索精度，基本符合实际场景对模型的检索精度要求。

Abstract

In order to solve the common problems of current medical images

such as relatively scattered size distribution of pathological areas

the ambiguous detail features

and the big visual difference of similar tissue images

this paper proposes a medical retrieval method integrating multi-scale features and attention mechanism based on the CBMIR system. This method adaptively balances the relationship between shallow image texture features and deep image semantic features by fusing multi-scale features and setting learnable weight coefficients

thereby the network's ability to extract pathological features at different scales is improved. At the same time

this method introduces the attention module and perform channel weighted summation on the feature maps output by the network to improve the feature expression ability of key feature channels

so that the network can pay more attention to the recognizable pathological feature areas in the image accurately. Finally

the multiple losses are used to further optimize the distribution of sample features in the feature space when the loss function is designed. As a result

the retrieval accuracy of 0.95 and 0.98 is achieved on the mAP@100 and mAP@20 indicators on the Mura dataset

which basically meets the retrieval accuracy requirements of the actual scene on the model.

关键词

Keywords

references

H MVLLER , N MICHOUX , D BANDON , 等 . A review of content-based image retrieval systems in medical applications-clinical benefits and future directions . International Journal of Medical Informatics , 2004 . 73 ( l ): 1 - 23 . http://www.researchgate.net/publication/220080490_A_review_of_content-based_image_retrieval_systems_in_medical_applications_-_clinical_benefits_and_future_directions http://www.researchgate.net/publication/220080490_A_review_of_content-based_image_retrieval_systems_in_medical_applications_-_clinical_benefits_and_future_directions .

周进凡. 基于大数据的肺部X光图像的分析与研究方法[D]. 贵阳: 贵州大学, 2019.

ZHOU J F. Analysis and research methods of lung X-ray images based on big data[D]. Guiyang: Guizhou University, 2019. (in Chinese)

陈静. 基于深度学习的肺结节检测系统[D]. 上海: 上海交通大学, 2018.

CHEN J. Lung nodule detection with deep learning[D]. Shanghai: Shanghai Jiao Tong University, 2018. (in Chinese)

彭朋 , 蒋涛 , 李敏 . PACS系统与临床信息系统互联对医学影像教学的意义 . 中国病案 , 2016 . 17 ( 1 ): 82 - 84 . https://www.cnki.com.cn/Article/CJFDTOTAL-ZGBN201601032.htm https://www.cnki.com.cn/Article/CJFDTOTAL-ZGBN201601032.htm .

P PENG , T JIANG , M LI . The significance of PACS system and clinical information system interconnection to medical imaging teaching . Chinese Medical Record , 2016 . 17 ( 1 ): 82 - 84 . https://www.cnki.com.cn/Article/CJFDTOTAL-ZGBN201601032.htm https://www.cnki.com.cn/Article/CJFDTOTAL-ZGBN201601032.htm .

G W JIJI , P S J D RAJ . Content-based image retrieval in dermatology using intelligent technique . IET Image Processing , 2015 . 9 ( 4 ): 306 - 317 . DOI: 10.1049/iet-ipr.2013.0501 http://doi.org/10.1049/iet-ipr.2013.0501 .

MIZOTIN M, BENOIS-PINEAU J, ALLARD M, et al . Feature-based brain MRI retrieval for Alzheimer disease diagnosis[C]// Proceedings of the 2012 19 th IEEE International Conference on Image Processing . Orlando: IEEE, 2012: 1241-1244.

M RAHMAN , S K ANTANI , G R THOMA . A learning-based similarity fusion and filtering approach for biomedical image retrieval using SVM classification and relevance feedback . IEEE Transactions on Information Technology in Biomedicine , 2011 . 15 ( 4 ): 640 - 646 . DOI: 10.1109/TITB.2011.2151258 http://doi.org/10.1109/TITB.2011.2151258 .

陈晓冬 , 盛婧 , 杨晋 , 等 . 多参数Gabor预处理融合多尺度局部水平集的超声图像分割 . 中国光学 , 2020 . 13 ( 5 ): 1075 - 1084 . DOI: 10.37188/CO.2020-0025 http://doi.org/10.37188/CO.2020-0025 .

X D CHEN , J SHENG , J YANG , 等 . Ultrasound image segmentation based on a multi-parameter Gabor filter and multiscale local level set method . Chinese Optics , 2020 . 13 ( 5 ): 1075 - 1084 . DOI: 10.37188/CO.2020-0025 http://doi.org/10.37188/CO.2020-0025 .

A QAYYUM , S M ANWAR , M AWAIS , 等 . Medical image retrieval using deep convolutional neural network . Neurocomputing , 2017 . 266 8 - 20 . DOI: 10.1016/j.neucom.2017.05.025 http://doi.org/10.1016/j.neucom.2017.05.025 .

吕晓琪 , 吴凉 , 谷宇 , 等 . 基于三维卷积神经网络的低剂量CT肺结节检测 . 光学精密工程 , 2018 . 26 ( 5 ): 1211 - 1218 . https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201805023.htm https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201805023.htm .

X Q LÜ , L WU , Y GU , 等 . Detection of low dose CT pulmonary nodules based on 3D convolution neural network . Optics and Precision Engineering , 2018 . 26 ( 5 ): 1211 - 1218 . https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201805023.htm https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201805023.htm .

熊舒羽. 基于深度学习技术的医学图像检索方法研究[D]. 重庆: 重庆理工大学, 2019.

XIONG S Y. Research on medical image retrieval using deep learning technology[D]. Chongqing: Chongqing University of Technology, 2019. (in Chinese)

彭晏飞 , 梅金业 , 王恺欣 , 等 . 基于区域注意力机制的遥感图像检索 . 激光与光电子学进展 , 2020 . 57 ( 10 ): 101017 https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202010019.htm https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202010019.htm .

Y F PENG , J Y MEI , K X WANG , 等 . Remote sensing image retrieval based on regional attention mechanism . Laser & Optoelectronics Progress , 2020 . 57 ( 10 ): 101017 https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202010019.htm https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202010019.htm .

周国华 , 蒋晖 , 顾晓清 , 等 . 多视角判别度量学习的乳腺影像检索方法 . 液晶与显示 , 2020 . 35 ( 6 ): 619 - 630 . DOI: 10.3788/YJYXS20203506.0619 http://doi.org/10.3788/YJYXS20203506.0619 .

G H ZHOU , H JIANG , X Q GU , 等 . Multi-view metric learning with Fisher discriminant analysis and its applications for breast image retrieval . Chinese Journal of Liquid Crystals and Displays , 2020 . 35 ( 6 ): 619 - 630 . DOI: 10.3788/YJYXS20203506.0619 http://doi.org/10.3788/YJYXS20203506.0619 .

Y GU , J YANG . Multi-level magnification correlation hashing for scalable histopathological image retrieval . Neurocomputing , 2019 . 351 134 - 145 . DOI: 10.1016/j.neucom.2019.03.050 http://doi.org/10.1016/j.neucom.2019.03.050 .

HE K M, ZHANG X Y, REN S Q, et al . Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas: IEEE, 2016.

A BUADES , B COLL , J M MOREL . Non-local means denoising . Image Processing on Line , 2011 . 1 208 - 212 . DOI: 10.5201/ipol.2011.bcm_nlm http://doi.org/10.5201/ipol.2011.bcm_nlm .

WANG X L, GIRSHICK R, GUPTA A, et al . Non-local neural networks[C]// Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018.

TANG H, XU D, SEBE N, et al . Attention-guided generative adversarial networks for unsupervised image-to-image translation[C]// Proceedings of 2019 International Joint Conference on Neural Networks . Budapest: IEEE, 2019: 1-8.

TANG H, LIU H, XU D, et al . AttentionGAN: unpaired image-to-image translation using attention-guided generative adversarial networks[J]. arXiv : 1911.11897, 2019.

WEN Y D, ZHANG K P, LI Z F, et al . A discriminative feature learning approach for deep face recognition[C]// Proceedings of the 14 th European Conference on Computer Vision . Amsterdam: Springer, 2016.

浏览量

228

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于头部和整体信息联合的多尺度行人检测算法

基于类特征注意力机制融合的语义分割算法