Remote sensing images matching based on ResNet and RF-Net

Ming-zhe LIAO; Jin WU; Lei ZHU

doi:10.37188/YJYXS20203509.0972

您当前的位置：

首页 >

文章列表页 >

Remote sensing images matching based on ResNet and RF-Net

Image Processing | 更新时间：2020-09-08

- Remote sensing images matching based on ResNet and RF-Net
- Chinese Journal of Liquid Crystals and Displays Vol. 35, Issue 9, Pages: 972-980(2020)
- 作者机构：
  
  1.武汉科技大学信息科学与工程学院, 湖北武汉 430081
  2.冶金自动化与检测技术教育部工程中心, 湖北武汉 430000
- 作者简介：
- 基金信息：
  
  the Youth Fund Project of National Natural Science Foundation of China(61502358);the Youth Fund Project of National Natural Science Foundation of China(61702384)
- DOI：10.37188/YJYXS20203509.0972
  CLC： TP751.1
- Received：16 February 2020，
  
  Revised：05 May 2020，
  
  Accepted：05 May 2020，
  
  Published：05 September 2020
- 稿件说明：
移动端阅览
Ming-zhe LIAO, Jin WU, Lei ZHU. Remote sensing images matching based on ResNet and RF-Net[J]. Chinese journal of liquid crystals and displays, 2020, 35(9): 972-980.
DOI：

Ming-zhe LIAO, Jin WU, Lei ZHU. Remote sensing images matching based on ResNet and RF-Net[J]. Chinese journal of liquid crystals and displays, 2020, 35(9): 972-980. DOI： 10.37188/YJYXS20203509.0972.

摘要

针对Receptive Fields Network (RF-Net)中存在网络较浅、缺乏深层语义信息的问题，提出了一种基于Residual Network (ResNet)和RF-Net的改进网络用于遥感影像匹配。首先，通过对真实遥感影像进行裁剪、光照变换和仿射变换处理，得到图像对并计算同一序列中不同图像间的单应性矩阵，构建了一个遥感影像数据集。然后，提出了一种双通道的网络结构用于关键点检测，该双通道网络由Receptive Fields Detection (RF-Det)和ResNet构成，前者提取含有细节信息的浅层特征图，后者提取含有语义信息的深层特征图。此外，采用特征描述子提取网络L2-Net，得到128维特征向量用以描述关键点。最后，分别采用最近邻、带阈值的最近邻和最近邻距离比的策略对特征描述子进行匹配。实验结果表明，该网络在仅含光照变换、仅含仿射变换和同时包含这两种变换的遥感影像数据集上的匹配得分，比RF-Net分别提高了0.002，0.117，0.104，在关键点检测和匹配精度方面具有更优的性能。

Abstract

Aiming at the problems of shallow network and lack of high-level semantic information in Receptive Fields Network(RF-Net)

a new network based on Residual Network(ResNet) and RF-Net is proposed for the matching of remote sensing images. Firstly

the new remote sensing image datasets are setup

which consist of images and homography matrices. The images are obtained by cropping

illumination changing and affine transforming of the original remote sensing images. The matrices are obtained by calculating the homography between different images of one sequence. Secondly

a dual-channel network structure is proposed for the keypoints detection. Specifically

the dual-channel network composed by Receptive Fields Detection(RF-Det) and ResNet

respectively extracts receptive feature maps with detail information

and the deep layer feature maps with semantic information. The L2-Net is introduced at descriptor extraction module to obtain the 128-dimension feature vectors as description of keypoints. Lastly

the strategies of the nearest neighbor

nearest neighbor with a threshold and nearest neighbor distance ration are used to match the descriptors. Experiments on remote sensing images show that the matching scores of the proposed network are 0.002

0.117

0.104

higher than RF-Net

when testing on different datasets. It outperforms on keypoints detection and matching accuracy.

关键词

Keywords

references

B ZITOVÁ , J FLUSSER . Image registration methods:a survey . Image and Vision Computing , 2003 . 21 ( 11 ): 977 - 1000 . DOI: 10.1016/S0262-8856(03)00137-9 http://doi.org/10.1016/S0262-8856(03)00137-9 .

余先川 , 吕中华 , 胡丹 . 遥感图像配准技术综述 . 光学精密工程 , 2013 . 21 ( 11 ): 2960 - 2972 . http://www.cqvip.com/qk/92835a/201311/47762782.html http://www.cqvip.com/qk/92835a/201311/47762782.html .

X C YU , Z H LV , D HU . Review of remote sensing image registration techniques . Optics and Precision Engineering , 2013 . 21 ( 11 ): 2960 - 2972 . http://www.cqvip.com/qk/92835a/201311/47762782.html http://www.cqvip.com/qk/92835a/201311/47762782.html .

贾迪 , 朱宁丹 , 杨宁华 , 等 . 图像匹配方法研究综述 . 中国图象图形学报 , 2019 . 24 ( 5 ): 677 - 699 . http://www.cnki.com.cn/Article/CJFDTotal-ZGTB201905002.htm http://www.cnki.com.cn/Article/CJFDTotal-ZGTB201905002.htm .

D JIA , N D ZHU , N H YANG , 等 . Image matching methods . Journal of Image and Graphics , 2019 . 24 ( 5 ): 677 - 699 . http://www.cnki.com.cn/Article/CJFDTotal-ZGTB201905002.htm http://www.cnki.com.cn/Article/CJFDTotal-ZGTB201905002.htm .

ONO Y, TRULLS E, FUA P, et al . LF-Net: learning local features from images[C]// Proceedings of the 32 nd International Conference on Neural Information Processing Systems . Montreal, Canada: ACM, 2018: 6237-6247.

YI K M, TRULLS E, LEPETIT V, et al . LIFT: Learned invariant feature transform[C]// Proceedings of the 14 th European Conference on Computer Vision . Amsterdam, the Netherlands: Springer, 2016: 467-483.

D G LOWE . Distinctive image features from scale-invariant keypoints . International Journal of Computer Vision , 2004 . 60 ( 2 ): 91 - 110 .

DETONE D, MALISIEWICZ T, RABINOVICH A. SuperPoint: self-supervised interest point detection and description[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Salt Lake City, UT, USA: IEEE, 2018.

SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]// Proceedings of 3 rd International Conference on Learning Representations .San Diego, CA, USA: ICLR, 2015: 1-13.

CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]// Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . San Diego, CA, USA: IEEE, 2005: 539-546.

SHEN X L, WANG C, LI X, et al . RF-Net: an end-to-end image matching network based on receptive field[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach, CA, USA: IEEE, 2019: 8124-8132.

MISHCHUK A, MISHKIN D, RADENOVIC F, et al . Working hard to know your neighbor's margins: Local descriptor learning loss[C]// Proceedings of the 31 st Conference on Neural Information Processing Systems . Long Beach, CA, USA: NeurIPS, 2017: 4826-4837.

HE K M, ZHANG X Y, REN S Q, et al . Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, Nevada, USA: IEEE, 2016: 770-778.

IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]// Proceedings of the 32 nd International Conference on Machine Learning . Lille, France: ACM, 2015: 448-456.

NAIR V, HINTON G E. Rectified linear units improve restricted boltzmann machines[C]// Proceedings of the 27 th International Conference on Machine Learning . Haifa, Israel: ACM, 2010: 807-814.

TIAN Y R, FAN B, WU F C. L2-Net: Deep learning of discriminative patch descriptor in euclidean space[C]// Proceedings of the 30 nd IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, Hawaii, USA: IEEE, 2017: 6128-6136.

LIN T Y, DOLLÁR P, GIRSHICK R, et al . Feature pyramid networks for object detection[C]// Proceedings of the 30 nd IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, Hawaii, USA: IEEE, 2017: 936-944.

K MIKOLAJCZYK , C SCHMID . A performance evaluation of local descriptors . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2005 . 27 ( 10 ): 1615 - 1630 . DOI: 10.1109/TPAMI.2005.188 http://doi.org/10.1109/TPAMI.2005.188 .

KINGMA D, BA J. Adam: a method for stochastic optimization[C]// Proceedings of the 3 rd International Conference on Learning Representations .San Diego, CA, USA: ICLR, 2015: 1-15.

Views

298

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

SIFT fast image matching algorithm with local adaptive threshold

Anti-Viewpoint Changing Image Matching Algorithm Based on SURF

Related Author

WANG Yin

JIANG Zheng

LIU Bin

SU Ke-xin

HAN Guang-liang

SUN Hai-jiang

WANG Yin

JIANG Zheng

Related Institution

College of Information Science and Engineering， Wuhan University of Science and Technology

Engineering Research Center for Metallurgical Automation and Measurement Technology of Ministry of Education， Wuhan University of Science and Technology

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China

Graduate University of Chinese Academy of Sciences, Beijing 100049, China

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰