Medicalimage retrieval with multiscale features and attention mechanisms

Lin-peng ZHOU; Jian-min YAO; Qun YAN; Zhi-xian LIN

doi:10.37188/CJLCD.2020-0248

您当前的位置：

首页 >

文章列表页 >

Medicalimage retrieval with multiscale features and attention mechanisms

Image Processing | 更新时间：2021-08-05

- Medicalimage retrieval with multiscale features and attention mechanisms
- Chinese Journal of Liquid Crystals and Displays Vol. 36, Issue 8, Pages: 1174-1185(2021)
- 作者机构：
  
  1.福州大学物理与信息工程学院, 福建福州 350108
  2.晋江市博感电子科技有限公司, 福建晋江 362200
- 作者简介：
- 基金信息：
  
  National Key Research and Development Program of China(2016YFB0401503);Open Fund of Guangdong Provincial Key Laboratory of Optical Information Materials and Technology(2017B030301007)
- DOI：10.37188/CJLCD.2020-0248
  CLC： TP391.4;TH691.9
- Received：24 September 2020，
  
  Revised：18 January 2021，
  
  Published：2021-08
- 稿件说明：
移动端阅览
Lin-peng ZHOU, Jian-min YAO, Qun YAN, et al. Medicalimage retrieval with multiscale features and attention mechanisms[J]. Chinese journal of liquid crystals and displays, 2021, 36(8): 1174-1185.
DOI：

Lin-peng ZHOU, Jian-min YAO, Qun YAN, et al. Medicalimage retrieval with multiscale features and attention mechanisms[J]. Chinese journal of liquid crystals and displays, 2021, 36(8): 1174-1185. DOI： 10.37188/CJLCD.2020-0248.

摘要

针对目前医学图像普遍存在病理区域尺寸分布较分散、细节特征不明显以及同类组织影像间的视觉差异较大等问题，本文在CBMIR系统的基础上，提出了一种融合多尺度特征及注意力机制的医学图像检索方法。该方法通过融合多尺度特征并设置可学习权重系数来自适应平衡浅层图像纹理特征和深层图像语义特征的关系，提高网络对不同尺度上的病理特征提取能力。同时，引入注意力模块，对网络输出的特征图进行通道加权求和，提高关键特征通道的特征表达能力，使网络更能关注到图像中的具有辨识性的病理特征区域。最后，在损失函数设计时，使用多重损失进一步优化样本特征在特征空间的分布。最终在Mura数据集上的mAP@100、mAP@20两个指标上分别达到了0.95、0.98的检索精度，基本符合实际场景对模型的检索精度要求。

Abstract

In order to solve the common problems of current medical images

such as relatively scattered size distribution of pathological areas

the ambiguous detail features

and the big visual difference of similar tissue images

this paper proposes a medical retrieval method integrating multi-scale features and attention mechanism based on the CBMIR system. This method adaptively balances the relationship between shallow image texture features and deep image semantic features by fusing multi-scale features and setting learnable weight coefficients

thereby the network's ability to extract pathological features at different scales is improved. At the same time

this method introduces the attention module and perform channel weighted summation on the feature maps output by the network to improve the feature expression ability of key feature channels

so that the network can pay more attention to the recognizable pathological feature areas in the image accurately. Finally

the multiple losses are used to further optimize the distribution of sample features in the feature space when the loss function is designed. As a result

the retrieval accuracy of 0.95 and 0.98 is achieved on the mAP@100 and mAP@20 indicators on the Mura dataset

which basically meets the retrieval accuracy requirements of the actual scene on the model.

关键词

Keywords

references

H MVLLER , N MICHOUX , D BANDON , 等 . A review of content-based image retrieval systems in medical applications-clinical benefits and future directions . International Journal of Medical Informatics , 2004 . 73 ( l ): 1 - 23 . http://www.researchgate.net/publication/220080490_A_review_of_content-based_image_retrieval_systems_in_medical_applications_-_clinical_benefits_and_future_directions http://www.researchgate.net/publication/220080490_A_review_of_content-based_image_retrieval_systems_in_medical_applications_-_clinical_benefits_and_future_directions .

周进凡. 基于大数据的肺部X光图像的分析与研究方法[D]. 贵阳: 贵州大学, 2019.

ZHOU J F. Analysis and research methods of lung X-ray images based on big data[D]. Guiyang: Guizhou University, 2019. (in Chinese)

陈静. 基于深度学习的肺结节检测系统[D]. 上海: 上海交通大学, 2018.

CHEN J. Lung nodule detection with deep learning[D]. Shanghai: Shanghai Jiao Tong University, 2018. (in Chinese)

彭朋 , 蒋涛 , 李敏 . PACS系统与临床信息系统互联对医学影像教学的意义 . 中国病案 , 2016 . 17 ( 1 ): 82 - 84 . https://www.cnki.com.cn/Article/CJFDTOTAL-ZGBN201601032.htm https://www.cnki.com.cn/Article/CJFDTOTAL-ZGBN201601032.htm .

P PENG , T JIANG , M LI . The significance of PACS system and clinical information system interconnection to medical imaging teaching . Chinese Medical Record , 2016 . 17 ( 1 ): 82 - 84 . https://www.cnki.com.cn/Article/CJFDTOTAL-ZGBN201601032.htm https://www.cnki.com.cn/Article/CJFDTOTAL-ZGBN201601032.htm .

G W JIJI , P S J D RAJ . Content-based image retrieval in dermatology using intelligent technique . IET Image Processing , 2015 . 9 ( 4 ): 306 - 317 . DOI: 10.1049/iet-ipr.2013.0501 http://doi.org/10.1049/iet-ipr.2013.0501 .

MIZOTIN M, BENOIS-PINEAU J, ALLARD M, et al . Feature-based brain MRI retrieval for Alzheimer disease diagnosis[C]// Proceedings of the 2012 19 th IEEE International Conference on Image Processing . Orlando: IEEE, 2012: 1241-1244.

M RAHMAN , S K ANTANI , G R THOMA . A learning-based similarity fusion and filtering approach for biomedical image retrieval using SVM classification and relevance feedback . IEEE Transactions on Information Technology in Biomedicine , 2011 . 15 ( 4 ): 640 - 646 . DOI: 10.1109/TITB.2011.2151258 http://doi.org/10.1109/TITB.2011.2151258 .

陈晓冬 , 盛婧 , 杨晋 , 等 . 多参数Gabor预处理融合多尺度局部水平集的超声图像分割 . 中国光学 , 2020 . 13 ( 5 ): 1075 - 1084 . DOI: 10.37188/CO.2020-0025 http://doi.org/10.37188/CO.2020-0025 .

X D CHEN , J SHENG , J YANG , 等 . Ultrasound image segmentation based on a multi-parameter Gabor filter and multiscale local level set method . Chinese Optics , 2020 . 13 ( 5 ): 1075 - 1084 . DOI: 10.37188/CO.2020-0025 http://doi.org/10.37188/CO.2020-0025 .

A QAYYUM , S M ANWAR , M AWAIS , 等 . Medical image retrieval using deep convolutional neural network . Neurocomputing , 2017 . 266 8 - 20 . DOI: 10.1016/j.neucom.2017.05.025 http://doi.org/10.1016/j.neucom.2017.05.025 .

吕晓琪 , 吴凉 , 谷宇 , 等 . 基于三维卷积神经网络的低剂量CT肺结节检测 . 光学精密工程 , 2018 . 26 ( 5 ): 1211 - 1218 . https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201805023.htm https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201805023.htm .

X Q LÜ , L WU , Y GU , 等 . Detection of low dose CT pulmonary nodules based on 3D convolution neural network . Optics and Precision Engineering , 2018 . 26 ( 5 ): 1211 - 1218 . https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201805023.htm https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201805023.htm .

熊舒羽. 基于深度学习技术的医学图像检索方法研究[D]. 重庆: 重庆理工大学, 2019.

XIONG S Y. Research on medical image retrieval using deep learning technology[D]. Chongqing: Chongqing University of Technology, 2019. (in Chinese)

彭晏飞 , 梅金业 , 王恺欣 , 等 . 基于区域注意力机制的遥感图像检索 . 激光与光电子学进展 , 2020 . 57 ( 10 ): 101017 https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202010019.htm https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202010019.htm .

Y F PENG , J Y MEI , K X WANG , 等 . Remote sensing image retrieval based on regional attention mechanism . Laser & Optoelectronics Progress , 2020 . 57 ( 10 ): 101017 https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202010019.htm https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202010019.htm .

周国华 , 蒋晖 , 顾晓清 , 等 . 多视角判别度量学习的乳腺影像检索方法 . 液晶与显示 , 2020 . 35 ( 6 ): 619 - 630 . DOI: 10.3788/YJYXS20203506.0619 http://doi.org/10.3788/YJYXS20203506.0619 .

G H ZHOU , H JIANG , X Q GU , 等 . Multi-view metric learning with Fisher discriminant analysis and its applications for breast image retrieval . Chinese Journal of Liquid Crystals and Displays , 2020 . 35 ( 6 ): 619 - 630 . DOI: 10.3788/YJYXS20203506.0619 http://doi.org/10.3788/YJYXS20203506.0619 .

Y GU , J YANG . Multi-level magnification correlation hashing for scalable histopathological image retrieval . Neurocomputing , 2019 . 351 134 - 145 . DOI: 10.1016/j.neucom.2019.03.050 http://doi.org/10.1016/j.neucom.2019.03.050 .

HE K M, ZHANG X Y, REN S Q, et al . Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas: IEEE, 2016.

A BUADES , B COLL , J M MOREL . Non-local means denoising . Image Processing on Line , 2011 . 1 208 - 212 . DOI: 10.5201/ipol.2011.bcm_nlm http://doi.org/10.5201/ipol.2011.bcm_nlm .

WANG X L, GIRSHICK R, GUPTA A, et al . Non-local neural networks[C]// Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition . Salt Lake City: IEEE, 2018.

TANG H, XU D, SEBE N, et al . Attention-guided generative adversarial networks for unsupervised image-to-image translation[C]// Proceedings of 2019 International Joint Conference on Neural Networks . Budapest: IEEE, 2019: 1-8.

TANG H, LIU H, XU D, et al . AttentionGAN: unpaired image-to-image translation using attention-guided generative adversarial networks[J]. arXiv : 1911.11897, 2019.

WEN Y D, ZHANG K P, LI Z F, et al . A discriminative feature learning approach for deep face recognition[C]// Proceedings of the 14 th European Conference on Computer Vision . Amsterdam: Springer, 2016.

Views

228

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multi-scale pedestrian detection algorithm based on joint head and overall information

Semantic segmentation algorithm based on class feature attention mechanism fusion

Related Author

Lin-peng ZHOU

Jian-min YAO

Qun YAN

Zhi-xian Lin

MA Ximing

LI Ning

WU Di

WANG Jianfei

Related Institution

College of Physics and Information Engineering， Fuzhou University

Jinjiang RichSense Electronic Technology Co.， Ltd.， Jinjiang

Changchun Institute of Optics， Fine Mechanics and Physics， Chinese Academy of Sciences

Military Representative Office Stationed in a Region by Air Force Equipment Department

College of Big Data and Information Engineering， Guizhou University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰