Survey of remote sensing image registration based on deep learning

LI Xinghua; AI Wenhao; FENG Ruitao; LUO Shaojie

doi:10.11834/jrs.20235012

Review | Views : 0 下载量: 6862 CSCD: 3

R-PDF
PDF
Export
Share
Collection
Album

Survey of remote sensing image registration based on deep learning
Vol. 27, Issue 2, Pages: 267-284(2023)
Received：31 May 2021，

Published：07 February 2023
DOI： 10.11834/jrs.20235012
稿件说明：

移动端阅览

李星华，艾文浩，冯蕊涛，罗少杰.2023.遥感影像深度学习配准方法综述.遥感学报，27（2）： 267-284 DOI： 10.11834/jrs.20235012.

Li X H，Ai W H，Feng R T and Luo S J. 2023. Survey of remote sensing image registration based on deep learning. National Remote Sensing Bulletin， 27（2）：267-284 DOI： 10.11834/jrs.20235012.

摘要

遥感影像配准是指通过几何变换使两景或多景影像空间位置对齐的过程，是影像融合、变化检测、农业监测等应用的重要预处理步骤。近年来，深度学习引起了人们的广泛关注，并在遥感影像配准中成功应用。本文在简要介绍传统遥感影像配准方法的基础上，重点分析了深度学习在基于区域的配准方法、基于特征的配准方法两方面取得的重要进展，分享了用于遥感影像配准的公开数据集，并总结了深度学习在遥感影像配准中的机遇与挑战。

Abstract

Remote sensing image registration is the process of spatial alignment of two or more images through geometric transformation. It is an important preprocessing operation for image fusion

change detection

agricultural monitoring and other remote sensing applications. Considering that remote sensing images have the characteristics of large-scale changes

complex ground covers and imaging modalities

although a large number of registration methods have been developed

there is still a lack of methods that can be widely used in different scenarios. Therefore

research on registration algorithms with high efficiency

high robustness

high precision and wide applicability is of great significance. In recent years

deep learning

which has achieved great success in the field of natural image and medical image registration

has provided a new method for remote sensing image registration. First

we introduced two kinds of traditional registration methods and analyzed the advantages and disadvantages of area-based and feature-based registration methods in detail from the aspects of registration accuracy

efficiency and algorithm robustness. Generally

there are two main problems in traditional methods: poor applicability and insufficient utilization of the deep semantic information of the image. Second

we focused on the important progress of deep learning in area-based registration methods and feature-based registration methods. According to the specific application purpose of deep learning

we made a more detailed division of the above two methods and summarized the advantages and disadvantages of the existing research. In addition

considering the importance of datasets for deep learning

we sorted and shared some public datasets for remote sensing image registration. Due to the great progress of earth observation technology

an increasing number of remote sensing images are being applied. Image registration is the key step of remote sensing image preprocessing and the basic research content of quantitative remote sensing analysis. In recent years

research on remote sensing image registration algorithms based on deep learning has shown an increasing trend

but it is still in the early stage

and the framework is not mature. It mainly includes but is not limited to the following shortcomings: (1) lack of open source standard datasets; (2) difficult to apply to large-scale remote sensing images; (3) insufficient utilization of geospatial information and spectral information of remote sensing images; and (4) long training time and the large computing overhead. From the perspective of data and methods

we looked forward to the application of deep learning in the field of remote sensing image registration and put forward four main research directions: (1) remote sensing image registration datasets; (2) registration methods based on hybrid models; (3) registration methods based on different neural networks; and (4) training strategies based on small samples.

关键词

Keywords

references

Balntas V , Lenc K , Vedaldi A and Mikolajczyk K . 2017 . HPatches: a benchmark and evaluation of handcrafted and learned local descriptors // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Hawaii : IEEE: 3852 - 3861 [ DOI: 10.1109/CVPR.2017.410 http://dx.doi.org/10.1109/CVPR.2017.410 ]

Barroso-Laguna A , Riba E , Ponsa D and Mikolajczyk K . 2019 . Key.Net: keypoint detection by handcrafted and learned CNN filters. arXiv: 1904 . 00889 v 3

Blendowski M and Heinrich M P . 2019 . Combining MRF-based deformable registration and deep binary 3D-CNN descriptors for large lung motion estimation in COPD patients . International Journal of Computer Assisted Radiology and Surgery , 14 ( 1 ): 43 - 52 [ DOI: 10.1007/s11548-018-1888-2 http://dx.doi.org/10.1007/s11548-018-1888-2 ]

Chen H M , Varshney P K and Arora M K . 2003 . Performance of mutual information similarity measure for registration of multitemporal remote sensing images . IEEE Transactions on Geoscience and Remote Sensing , 41 ( 11 ): 2445 - 2454 [ DOI: 10.1109/TGRS.2003.817664 http://dx.doi.org/10.1109/TGRS.2003.817664 ]

Cheng X , Zhang L and Zheng Y F . 2018 . Deep similarity learning for multimodal medical images . Computer Methods in Biomechanics and Biomedical Engineering : Imaging and Visualization , 6 ( 3 ): 248 - 252 [ DOI: 10.1080/21681163.2015.1135299 http://dx.doi.org/10.1080/21681163.2015.1135299 ]

Chopra S , Hadsell R and LeCun Y . 2005 . Learning a similarity metric discriminatively, with application to face verification // 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . San Diego : IEEE: 539 - 546 [ DOI: 10.1109/CVPR.2005.202 http://dx.doi.org/10.1109/CVPR.2005.202 ]

de Vos B D , Berendsen F F , Viergever M A , Sokooti H , Staring M and Išgum I . 2019 . A deep learning framework for unsupervised affine and deformable image registration . Medical Image Analysis , 52 : 128 - 143 [ DOI: 10.1016/j.media.2018.11.010 http://dx.doi.org/10.1016/j.media.2018.11.010 ]

DeTone D , Malisiewicz T and Rabinovich A . 2018 . SuperPoint: self-supervised interest point detection and description. arXiv: 1712 . 07629 v 4

Dong Y Y , Jiao W L , Long T F , Liu L F , He G J , Gong C J and Guo Y T . 2019 . Local deep descriptor for remote sensing image feature matching . Remote Sensing , 11 ( 4 ): 430 [ DOI: 10.3390/rs11040430 http://dx.doi.org/10.3390/rs11040430 ]

Fan R B , Hou B C , Liu J B , Yang J H and Hong Z L . 2021 . Registration of multiresolution remote sensing images based on L2-Siamese model . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 14 : 237 - 248 [ DOI: 10.1109/JSTARS.2020.3038922 http://dx.doi.org/10.1109/JSTARS.2020.3038922 ]

Feng R T , Du Q Y , Li X H and Shen H F . 2019 . Robust registration for remote sensing images by combining and localizing feature- and area-based methods . ISPRS Journal of Photogrammetry and Remote Sensing , 151 : 15 - 26 [ DOI: 10.1016/j.isprsjprs.2019.03.002 http://dx.doi.org/10.1016/j.isprsjprs.2019.03.002 ]

Feng R T . 2020 . The Registration Algorithm for Remote Sensing Images Covering the Complex Terrain by Multi-Model Union . Wuhan : Wuhan University : 1 - 122

冯蕊涛 . 2020 . 复杂地形条件下的遥感影像多模型联合配准方法 . 武汉 : 武汉大学 : 1 - 122

Feng R T , Shen H F , Bai J J and Li X H . 2021 . Advances and opportunities in remote sensing image geometric registration: a systematic review of state-of-the-art approaches and future research directions . IEEE Geoscience and Remote Sensing Magazine , 9 ( 4 ): 120 - 142 [ DOI: 10.1109/MGRS.2021.3081763 http://dx.doi.org/10.1109/MGRS.2021.3081763 ]

Feng R T , Du Q Y , Luo H , Shen H F , Li X H and Liu B . 2021 . A registration algorithm based on optical flow modification for multi-temporal remote sensing images covering the complex-terrain region . National Remote Sensing Bulletin , 25 ( 2 ): 630 - 640

冯蕊涛 , 杜清运 , 罗恒 , 沈焕锋 , 李星华 , 刘波 . 2021 . 基于光流校正的复杂地形区多时相遥感影像配准 . 遥感学报 , 25 ( 2 ): 630 - 640 [ DOI: 10.11834/jrs.20209280 http://dx.doi.org/10.11834/jrs.20209280 ]

Fischler M A and Bolles R C . 1981 . Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography . Communications of the ACM , 24 ( 6 ): 381 - 395 [ DOI: 10.1145/358669.358692 http://dx.doi.org/10.1145/358669.358692 ]

Girard N , Charpiat G and Tarabalka Y . 2019 . Aligning and updating cadaster maps with aerial images by multi-task, multi-resolution deep learning // 14th Asian Conference on Computer Vision . Perth : Springer: 675 - 690 [ DOI: 10.1007/978-3-030-20873-8_43 http://dx.doi.org/10.1007/978-3-030-20873-8_43 ]

Gong M G , Zhao S M , Jiao L C , Tian D Y and Wang S . 2014 . A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information . IEEE Transactions on Geoscience and Remote Sensing , 52 ( 7 ): 4328 - 4338 [ DOI: 10.1109/TGRS.2013.2281391 http://dx.doi.org/10.1109/TGRS.2013.2281391 ]

Goodfellow I J , Pouget-Abadie J , Mirza M , Xu B , Warde-Faley D , Ozair S , Courville A and Bengio Y . 2014 . Generative adversarial networks. arXiv: 1406 . 2661 v 1

Harris C and Stephens M . 1988 . A combined corner and edge detector // Proceedings of the Alvey Vision Conference . Manchester : AVC: 147 - 151 [ DOI: 10.5244/C.2.23 http://dx.doi.org/10.5244/C.2.23 ]

Haskins G , Kruecker J , Kruger U , Xu S , Pinto P A , Wood B J and Yan P K . 2019 . Learning deep similarity metric for 3D MR–TRUS image registration . International Journal of Computer Assisted Radiology and Surgery , 14 ( 3 ): 417 - 425 [ DOI: 10.1007/s11548-018-1875-7 http://dx.doi.org/10.1007/s11548-018-1875-7 ]

He H Q , Chen M , Chen T and Li D J . 2018 . Matching of remote sensing images with complex background variations via Siamese convolutional neural network . Remote Sensing , 10 ( 2 ): 355 [ DOI: 10.3390/rs10020355 http://dx.doi.org/10.3390/rs10020355 ]

He H Q , Chen M , Chen T , Li D J and Cheng P G . 2019 . Learning to match multitemporal optical satellite images using multi-support-patches Siamese networks . Remote Sensing Letters , 10 ( 6 ): 516 - 525 [ DOI: 10.1080/2150704x.2019.1577572 http://dx.doi.org/10.1080/2150704x.2019.1577572 ]

Hinton G E and Salakhutdinov R R . 2006 . Reducing the dimensionality of data with neural networks . Science , 313 ( 5786 ): 504 - 507 [ DOI: 10.1126/science.1127647 http://dx.doi.org/10.1126/science.1127647 ]

Holland P W and Welsch R E . 1977 . Robust regression using iteratively reweighted least-squares . Communications in Statistics-Theory and Methods , 6 ( 9 ): 813 - 827 [ DOI: 10.1080/03610927708827533 http://dx.doi.org/10.1080/03610927708827533 ]

Hughes L H , Schmitt M , Mou L C , Wang Y Y and Zhu X X . 2018 . Identifying corresponding patches in SAR and optical images with a Pseudo-Siamese CNN . IEEE Geoscience and Remote Sensing Letters , 15 ( 5 ): 784 - 788 [ DOI: 10.1109/LGRS.2018.2799232 http://dx.doi.org/10.1109/LGRS.2018.2799232 ]

Hui T W , Tang X O and Loy C C . 2018 . LiteFlowNet: a lightweight convolutional neural network for optical flow estimation // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE: 8981 - 8989 [ DOI: 10.1109/CVPR.2018.00936 http://dx.doi.org/10.1109/CVPR.2018.00936 ]

Hui T W and Loy C C . 2020 . LiteFlowNet3: resolving correspondence ambiguity for more accurate optical flow estimation // 16th European Conference on Computer Vision . Glasgow : Springer: 169 - 184 [ DOI: 10.1007/978-3-030-58565-5_11 http://dx.doi.org/10.1007/978-3-030-58565-5_11 ]

Hui T W , Tang X O and Loy C C . 2021 . A lightweight optical flow CNN—revisiting data fidelity and regularization . IEEE Transactions on Pattern Analysis and Machine Intelligence , 43 ( 8 ): 2555 - 2569 [ DOI: 10.1109/TPAMI.2020.2976928 http://dx.doi.org/10.1109/TPAMI.2020.2976928 ]

Jiang X Y , Ma J Y , Xiao G B , Shao Z F and Guo X J . 2021a . A review of multimodal image matching: methods and applications . Information Fusion , 73 : 22 - 71 [ DOI: 10.1016/j.inffus.2021.02.012 http://dx.doi.org/10.1016/j.inffus.2021.02.012 ]

Jiang X Y , Ma J Y , Fan A X , Xu H P , Lin G , Lu T and Tian X . 2021b . Robust feature matching for remote sensing image registration via linear adaptive filtering . IEEE Transactions on Geoscience and Remote Sensing , 59 ( 2 ): 1577 - 1591 [ DOI: 10.1109/TGRS.2020.3001089 http://dx.doi.org/10.1109/TGRS.2020.3001089 ]

Johnson K , Cole-Rhodes A , Zavorin I and Le Moigne J . 2001 . Mutual information as a similarity measure for remote sensing image registration // Proceedings Volume 4383, Geo-Spatial Image and Data Exploitation II . Orlando : SPIE: 51 - 61 [ DOI: 10.1117/12.428251 http://dx.doi.org/10.1117/12.428251 ]

Kim D G , Nam W J and Lee S W . 2019 . A robust matching network for gradually estimating geometric transformation on remote sensing imagery // 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC) . Bari : IEEE: 3889 - 3894 [ DOI: 10.1109/SMC.2019.8913881 http://dx.doi.org/10.1109/SMC.2019.8913881 ]

Kuppala K , Banda S and Barige T R . 2020 . An overview of deep learning methods for image registration with focus on feature-based approaches . International Journal of Image and Data Fusion , 11 ( 2 ): 113 - 135 [ DOI: 10.1080/19479832.2019.1707720 http://dx.doi.org/10.1080/19479832.2019.1707720 ]

Lan C Z , Lu W J , Yu J M and Xu Q . 2021 . Deep learning algorithm for feature matching of cross modality remote sensing images . Acta Geodaetica et Cartographica Sinica , 50 ( 2 ): 189 - 202

蓝朝桢 , 卢万杰 , 于君明 , 徐青 . 2021 . 异源遥感影像特征匹配的深度学习算法 . 测绘学报 , 50 ( 2 ): 189 - 202 [ DOI: 10.11947/j.AGCS.2021.20200048 http://dx.doi.org/10.11947/j.AGCS.2021.20200048 ]

LeCun Y , Bottou L , Bengio Y and Haffner P . 1998 . Gradient-based learning applied to document recognition . Proceedings of the IEEE , 86 ( 11 ): 2278 - 2324 [ DOI: 10.1109/5.726791 http://dx.doi.org/10.1109/5.726791 ]

LeCun Y , Bengio Y and Hinton G . 2015 . Deep learning . Nature , 521 ( 7553 ): 436 - 444 [ DOI: 10.1038/nature14539 http://dx.doi.org/10.1038/nature14539 ]

Lee W , Sim D and Oh S J . 2021 . A CNN-based high-accuracy registration for remote sensing images . Remote Sensing , 13 ( 8 ): 1482 [ DOI: 10.3390/rs13081482 http://dx.doi.org/10.3390/rs13081482 ]

Lee W J and Oh S J . 2021 . Remote sensing image registration using equivariance features // 2021 International Conference on Information Networking (ICOIN) . Bangkok : IEEE: 776 - 781 [ DOI: 10.1109/ICOIN50884.2021.9333861 http://dx.doi.org/10.1109/ICOIN50884.2021.9333861 ]

Li J Y , Hu Q W and Ai M Y . 2020a . RIFT: multi-modal image matching based on radiation-variation insensitive feature transform . IEEE Transactions on Image Processing , 29 : 3296 - 3310 [ DOI: 10.1109/TIP.2019.2959244 http://dx.doi.org/10.1109/TIP.2019.2959244 ]

Li J Y , Zhao P C , Hu Q W and Ai M Y . 2020b . Robust point cloud registration based on topological graph and Cauchy weighted l q -norm . ISPRS Journal of Photogrammetry and Remote Sensing , 160 : 244 - 259 [ DOI: 10.1016/j.isprsjprs.2019.12.008 http://dx.doi.org/10.1016/j.isprsjprs.2019.12.008 ]

Li L , Ji S , Yu Y and Zhang Y S . 2020 . A multi-feature-based registration method adapted to multi-source remote sensing images . Journal of Geomatics Science and Technology , 37 ( 1 ): 74 - 78

李力 , 纪松 , 于英 , 张永生 . 2020 . 一种基于组合特征的异源遥感影像配准方法 . 测绘科学技术学报 , 37 ( 1 ): 74 - 78 [ DOI: 10.3969/j.issn.1673-6338.2020.01.014 http://dx.doi.org/10.3969/j.issn.1673-6338.2020.01.014 ]

Li X H , Feng R T , Guan X B , Shen H F and Zhang L P . 2019 . Remote sensing image mosaicking: achievements and challenges . IEEE Geoscience and Remote Sensing Magazine , 7 ( 4 ): 8 - 22 [ DOI: 10.1109/MGRS.2019.2921780 http://dx.doi.org/10.1109/MGRS.2019.2921780 ]

Li X H , Du Z S , Huang Y Y and Tan Z Y . 2021 . A deep translation (GAN) based change detection network for optical and SAR remote sensing images . ISPRS Journal of Photogrammetry and Remote Sensing , 179 : 14 - 34 [ DOI: 10.1016/j.isprsjprs.2021.07.007 http://dx.doi.org/10.1016/j.isprsjprs.2021.07.007 ]

Liang J Y , Liu X P , Huang K N , Li X , Wang D G and Wang X W . 2014 . Automatic registration of multisensor images using an integrated spatial and mutual information (SMI) metric . IEEE Transactions on Geoscience and Remote Sensing , 52 ( 1 ): 603 - 615 [ DOI: 10.1109/TGRS.2013.2242895 http://dx.doi.org/10.1109/TGRS.2013.2242895 ]

Liang Y , Sheng Y H , Zhang K and Yang L . 2014 . Linear feature matching method based on local affine invariant and Epipolar constraint for close-range images . Geomatics and Information Science of Wuhan University , 39 ( 2 ): 229 - 233

梁艳 , 盛业华 , 张卡 , 杨林 . 2014 . 利用局部仿射不变及核线约束的近景影像直线特征匹配 . 武汉大学学报(信息科学版) , 39 ( 2 ): 229 - 233 [ DOI: 10.13203/j.whugis20120611 http://dx.doi.org/10.13203/j.whugis20120611 ]

Liao R , Miao S , de Tournemire P , Grbic S , Kamen A , Mansi T and Comaniciu D . 2016 . An artificial agent for robust image registration. arXiv: 1611 . 10336 v 1

Liu P P , Irwin K , Lyu M R and Xu J . 2019a . DDFlow: learning optical flow with unlabeled data distillation . Proceedings of the AAAI Conference on Artificial Intelligence , 33 ( 1 ): 8770 - 8777 [ DOI: 10.1609/aaai.v33i01.33018770 http://dx.doi.org/10.1609/aaai.v33i01.33018770 ]

Liu P P , Lyu M , King I and Xu J . 2019b . SelFlow: self-supervised learning of optical flow // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Long Beach : IEEE: 4571 - 4580 [ DOI: 10.1109/CVPR.2019.00470 http://dx.doi.org/10.1109/CVPR.2019.00470 ]

Lowe D G . 2004 . Distinctive image features from scale-invariant keypoints . International Journal of Computer Vision , 60 ( 2 ): 91 - 110 [ DOI: 10.1023/b:visi.0000029664.99615.94 http://dx.doi.org/10.1023/b:visi.0000029664.99615.94 ]

Lu J Y , Jia H G , Li T , Li Z Q , Ma J Y and Zhu R F . 2021 . An instance segmentation based framework for large-sized high-resolution remote sensing images registration . Remote Sensing , 13 ( 9 ): 1657 [ DOI: 10.3390/rs13091657 http://dx.doi.org/10.3390/rs13091657 ]

Luo Y , Cao W M , He Z Q , Zou W L and He Z H . 2021 . Deformable adversarial registration network with multiple loss constraints . Computerized Medical Imaging and Graphics , 91 : 101931 [ DOI: 10.1016/j.compmedimag.2021.101931 http://dx.doi.org/10.1016/j.compmedimag.2021.101931 ]

Ma D A , Tang P , Zhao L J and Zhang Z . 2021 . Review of data augmentation for image in deep learning . Journal of Image and Graphics , 26 ( 3 ): 487 - 502

马岽奡 , 唐娉 , 赵理君 , 张正 . 2021 . 深度学习图像数据增广方法研究综述 . 中国图象图形学报 , 26 ( 3 ): 487 - 502 [ DOI: 10.11834/jig.200089 http://dx.doi.org/10.11834/jig.200089 ]

Ma F L . 2018 . Research on Color-To-Gray Conversion based on Normalized Cross Correlation . Lanzhou : Lanzhou University : 1 - 61

马方龙 . 2018 . 基于归一化积相关匹配的彩色图像灰度化研究 . 兰州 : 兰州大学 : 1 - 61

Ma J Y , Jiang X Y , Jiang J J , Zhao J and Guo X J . 2019c . LMR: learning a two-class classifier for mismatch removal . IEEE Transactions on Image Processing , 28 ( 8 ): 4045 - 4059 [ DOI: 10.1109/TIP.2019.2906490 http://dx.doi.org/10.1109/TIP.2019.2906490 ]

Ma J Y , Jiang X Y , Fan A X , Jiang J J and Yan J C . 2021 . Image matching from handcrafted to deep features: a survey . International Journal of Computer Vision , 129 ( 1 ): 23 - 79 [ DOI: 10.1007/s11263-020-01359-2 http://dx.doi.org/10.1007/s11263-020-01359-2 ]

Ma L , Liu Y , Zhang X L , Ye Y X , Yin G F and Johnson B A . 2019a . Deep learning in remote sensing applications: a meta-analysis and review . ISPRS Journal of Photogrammetry and Remote Sensing , 152 : 166 - 177 [ DOI: 10.1016/j.isprsjprs.2019.04.015 http://dx.doi.org/10.1016/j.isprsjprs.2019.04.015 ]

Ma W P , Zhang J , Wu Y , Jiao L C , Zhu H and Zhao W . 2019b . A novel two-step registration method for remote sensing images based on deep and local features . IEEE Transactions on Geoscience and Remote Sensing , 57 ( 7 ): 4834 - 4843 [ DOI: 10.1109/TGRS.2019.2893310 http://dx.doi.org/10.1109/TGRS.2019.2893310 ]

Ni H , Feng Z , Guan Y , Jia X Y , Chen W , Jiang T , Zhong Q Y , Yuan J , Ren M , Li X N , Gong H , Luo Q M and Li A N . 2021 . DeepMapi: a fully automatic registration method for mesoscopic optical brain images using convolutional neural networks . Neuroinformatics , 19 ( 2 ): 267 - 284 [ DOI: 10.1007/s12021-020-09483-7 http://dx.doi.org/10.1007/s12021-020-09483-7 ]

Niu R G , Sun X , Tian Y , Diao W H , Chen K Q and Fu K . 2020 . Hybrid multiple attention network for semantic segmentation in aerial images. arXiv: 2001 . 02870 v 3

Park J H , Nam W J and Lee S W . 2020 . A two-stream symmetric network with bidirectional ensemble for aerial image matching . Remote Sensing , 12 ( 3 ): 465 [ DOI: 10.3390/rs12030465 http://dx.doi.org/10.3390/rs12030465 ]

Rahaghi A I , Lemmin U , Sage D and Barry D A . 2019 . Achieving high-resolution thermal imagery in low-contrast lake surface waters by aerial remote sensing and image registration . Remote Sensing of Environment , 221 : 773 - 783 [ DOI: 10.1016/j.rse.2018.12.018 http://dx.doi.org/10.1016/j.rse.2018.12.018 ]

Ranjan A and Black M J . 2017 . Optical flow estimation using a spatial pyramid network // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Hawaii : IEEE: 2720 - 2729 [ DOI: 10.1109/CVPR.2017.291 http://dx.doi.org/10.1109/CVPR.2017.291 ]

Rocco I , Arandjelovic R and Sivic J . 2017 . Convolutional neural network architecture for geometric matching // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Hawaii : IEEE: 39 - 48 [ DOI: 10.1109/CVPR.2017.12 http://dx.doi.org/10.1109/CVPR.2017.12 ]

Savinov N , Seki A , Ladicky L , Sattler T and Pollefeys M . 2017 . Quad-networks: unsupervised learning to rank for interest point detection. arXiv: 1611 . 07571 v 2

Schmidhuber J . 2015 . Deep learning in neural networks: an overview . Neural Networks , 61 : 85 - 117 [ DOI: 10.1016/j.neunet.2014.09.003 http://dx.doi.org/10.1016/j.neunet.2014.09.003 ]

Schmitt M , Hughes L H and Zhu X X . 2018 . The SEN1-2 dataset for deep learning in SAR-optical data fusion . ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-1 : 141 - 146 [ DOI: 10.5194/isprs-annals-IV-1-141-2018 http://dx.doi.org/10.5194/isprs-annals-IV-1-141-2018 ]

Seo S , Choi J S , Lee J , Kim H H , Seo D , Jeong J and Kim M . 2020 . UPSNet: unsupervised pan-sharpening network with registration learning between panchromatic and multi-spectral images . IEEE Access , 8 : 201199 - 201217 [ DOI: 10.1109/ACCESS.2020.3035802 http://dx.doi.org/10.1109/ACCESS.2020.3035802 ]

Shabanov A , Gladilin S and Shvets E . 2020 . Optical-to-SAR image registration using a combination of CNN descriptors and cross-correlation coefficient // Twelfth International Conference on Machine Vision (ICMV) . Amsterdam : SPIE: 440 - 449 [ DOI: 10.1117/12.2558414 http://dx.doi.org/10.1117/12.2558414 ]

Shelhamer E , Long J and Darrell T . 2017 . Fully convolutional networks for semantic segmentation . IEEE Transactions on Pattern Analysis and Machine Intelligence , 39 ( 4 ): 640 - 651 [ DOI: 10.1109/TPAMI.2016.2572683 http://dx.doi.org/10.1109/TPAMI.2016.2572683 ]

Simonovsky M , Gutiérrez-Becker B , Mateus D , Navab N and Komodakis N . 2016 . A deep metric for multimodal registration // 19th International Conference on Medical Image Computing and Computer-Assisted Intervention . Athens : Springer: 10 - 18 [ DOI: 10.1007/978-3-319-46726-9_2 http://dx.doi.org/10.1007/978-3-319-46726-9_2 ]

Simonyan K and Zisserman A . 2015 . Very deep convolutional networks for large-scale image recognition. arXiv: 1409 . 1556 v 6

Smith S M and Brady J M . 1997 . SUSAN-a new approach to low level image processing . International Journal of Computer Vision , 23 ( 1 ): 45 - 78 [ DOI: 10.1023/A:1007963824710 http://dx.doi.org/10.1023/A:1007963824710 ]

Suri S and Reinartz P . 2010 . Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas . IEEE Transactions on Geoscience and Remote Sensing , 48 ( 2 ): 939 - 949 [ DOI: 10.1109/TGRS.2009.2034842 http://dx.doi.org/10.1109/TGRS.2009.2034842 ]

Vaswani A , Shazeer N , Parmar N , Uszkoreit J , Jones L , Gomez A N , Kaiser L and Polosukhin I . 2017 . Attention is all you need. arXiv: 1706 . 03762 v 5

Verdie Y , Yi K M , Fua P and Lepetit V . 2015 . TILDE: a temporally invariant learned detector // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Boston : IEEE: 5279 - 5288 [ DOI: 10.1109/CVPR.2015.7299165 http://dx.doi.org/10.1109/CVPR.2015.7299165 ]

Wang S , Quan D , Liang X F , Ning M D , Guo Y H and Jiao L C . 2018 . A deep learning framework for remote sensing image registration . ISPRS Journal of Photogrammetry and Remote Sensing , 145 : 148 - 164 [ DOI: 10.1016/j.isprsjprs.2017.12.012 http://dx.doi.org/10.1016/j.isprsjprs.2017.12.012 ]

Wang Y Y and Zhu X X . 2018 . The SARptical dataset for joint analysis of SAR and optical image in dense urban area. arXiv: 1801 . 07532

Wang Z H and Wu F C . 2009 . Mean-standard deviation descriptor and line matching . Pattern Recognition and Artificial Intelligence , 22 ( 1 ): 32 - 39

王志衡 , 吴福朝 . 2009 . 均值-标准差描述子与直线匹配 . 模式识别与人工智能 , 22 ( 1 ): 32 - 39 [ DOI: 10.3969/j.issn.1003-6059.2009.01.006 http://dx.doi.org/10.3969/j.issn.1003-6059.2009.01.006 ]

Xiang Y M , Tao R S , Wang F , You H J and Han B . 2020 . Automatic registration of optical and SAR images via improved phase congruency model . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13 : 5847 - 5861 [ DOI: 10.1109/JSTARS.2020.3026162 http://dx.doi.org/10.1109/JSTARS.2020.3026162 ]

Yang J , Yang J H , Zhao F and Zhang W J . 2021 . An unsupervised multi-scale framework with attention-based network (MANet) for lung 4D-CT registration . Physics in Medicine and Biology , 66 ( 13 ): 135008 [ DOI: 10.1088/1361-6560/ac0afc http://dx.doi.org/10.1088/1361-6560/ac0afc ]

Yang Z Q , Dan T T and Yang Y . 2018 . Multi-temporal remote sensing image registration using deep convolutional features . IEEE Access , 6 : 38544 - 38555 [ DOI: 10.1109/ACCESS.2018.2853100 http://dx.doi.org/10.1109/ACCESS.2018.2853100 ]

Yao M Q and Hu J . 2020 . Robust multimodal medical image registration using deep recurrent reinforcement learning . Journal of Computer-Aided Design and Computer Graphics , 32 ( 8 ): 1236 - 1247

姚明青 , 胡靖 . 2020 . 基于深度强化学习的多模态医学图像配准 . 计算机辅助设计与图形学学报 , 32 ( 8 ): 1236 - 1247 [ DOI: 10.3724/SP.J.1089.2020.17847 http://dx.doi.org/10.3724/SP.J.1089.2020.17847 ]

Ye F M , Su Y F , Xiao H , Zhao X Q and Min W D . 2018 . Remote sensing image registration using convolutional neural network features . IEEE Geoscience and Remote Sensing Letters , 15 ( 2 ): 232 - 236 [ DOI: 10.1109/LGRS.2017.2781741 http://dx.doi.org/10.1109/LGRS.2017.2781741 ]

Ye Y X and Shan J . 2014 . A local descriptor based registration method for multispectral remote sensing images with non-linear intensity differences . ISPRS Journal of Photogrammetry and Remote Sensing , 90 : 83 - 95 [ DOI: 10.1016/j.isprsjprs.2014.01.009 http://dx.doi.org/10.1016/j.isprsjprs.2014.01.009 ]

Yi K M , Trulls E , Lepetit V and Fua P . 2016 . LIFT: learned invariant feature transform // 14th European Conference on Computer Vision (ECCV) . Amsterdam : Springer: 467 - 483 [ DOI: 10.1007/978-3-319-46466-4_28 http://dx.doi.org/10.1007/978-3-319-46466-4_28 ]

Yi K M , Trulls E , Ono Y , Lepetit V , Salzmann M and Fua P . 2018 . Learning to find good correspondences // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Salt Lake City : IEEE: 2666 - 2674 [ DOI: 10.1109/CVPR.2018.00282 http://dx.doi.org/10.1109/CVPR.2018.00282 ]

Yu X C , Lü Z H and Hu D . 2013 . Review of remote sensing image registration techniques . Optics and Precision Engineering , 21 ( 11 ): 2960 - 2972

余先川 , 吕中华 , 胡丹 . 2013 . 遥感图像配准技术综述 . 光学精密工程 , 21 ( 11 ): 2960 - 2972 [ DOI: 10.3788/OPE.20132111.2960 http://dx.doi.org/10.3788/OPE.20132111.2960 ]

Yuan Z , Guo H T , Lu J , Lu W and Lin Y Z . 2021 . High-resolution remote sensing image change detection technology based on UNet++ and attention mechanism . Journal of Geomatics Science and Technology , 38 ( 2 ): 155 - 159

袁洲 , 郭海涛 , 卢俊 , 路威 , 林雨准 . 2021 . 融合UNet++网络和注意力机制的高分辨率遥感影像变化检测算法 . 测绘科学技术学报 , 38 ( 2 ): 155 - 159 [ DOI: 10.3969/j.issn.1673-6338.2021.02.008 http://dx.doi.org/10.3969/j.issn.1673-6338.2021.02.008 ]

Zampieri A , Charpiat G , Girard N and Tarabalka Y . 2018 . Multimodal image alignment through a multiscale chain of neural networks with application to remote sensing // 15th European Conference on Computer Vision (ECCV) . Munich : Springer: 679 - 696 [ DOI: 10.1007/978-3-030-01270-0_40 http://dx.doi.org/10.1007/978-3-030-01270-0_40 ]

Zeng L , Du Y L , Lin H P , Wang J , Yin J J and Yang J . 2021 . A novel region-based image registration method for multisource remote sensing images via CNN . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 14 : 1821 - 1831 [ DOI: 10.1109/JSTARS.2020.3047656 http://dx.doi.org/10.1109/JSTARS.2020.3047656 ]

Zeng Y , Ning Z H , Liu P , Luo P L , Zhang Y and He G J . 2020 . A mosaic method for multi-temporal data registration by using convolutional neural networks for forestry remote sensing applications . Computing , 102 ( 3 ): 795 - 811 [ DOI: 10.1007/s00607-019-00716-5 http://dx.doi.org/10.1007/s00607-019-00716-5 ]

Zhang H , Ni W P , Yan W D , Xiang D L , Wu J Z , Yang X L and Bian H . 2019a . Registration of multimodal remote sensing image based on deep fully convolutional neural network . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 12 ( 8 ): 3028 - 3042 [ DOI: 10.1109/JSTARS.2019.2916560 http://dx.doi.org/10.1109/JSTARS.2019.2916560 ]

Zhang J , Ma W P , Wu Y and Jiao L C . 2019b . Multimodal remote sensing image registration based on image transfer and local features . IEEE Geoscience and Remote Sensing Letters , 16 ( 8 ): 1210 - 1214 [ DOI: 10.1109/LGRS.2019.2896341 http://dx.doi.org/10.1109/LGRS.2019.2896341 ]

Zhang L G and Rusinkiewicz S . 2018 . Learning to detect features in texture images // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Salt Lake City : IEEE: 6325 - 6333 [ DOI: 10.1109/CVPR.2018.00662 http://dx.doi.org/10.1109/CVPR.2018.00662 ]

Zhao L Y , Lü B Y , Li X R and Chen S H . 2015 . Multi-source remote sensing image registration based on scale-invariant feature transform and optimization of regional mutual information . Acta Physica Sinica , 64 ( 12 ): 124204

赵辽英 , 吕步云 , 厉小润 , 陈淑涵 . 2015 . 基于尺度不变特征变换和区域互信息优化的多源遥感图像配准 . 物理学报 , 64 ( 12 ): 124204 [ DOI: 10.7498/aps.64.124204 http://dx.doi.org/10.7498/aps.64.124204 ]

Zheng Z D , Wei Y C and Yang Y . 2020 . University-1652: a multi-view multi-source benchmark for drone-based geo-localization. arXiv: 2002 . 12186 v 2

Zhou Z H . 2018 . A brief introduction to weakly supervised learning . National Science Review , 5 ( 1 ): 44 - 53 [ DOI: 10.1093/nsr/nwx106 http://dx.doi.org/10.1093/nsr/nwx106 ]

Zhu H , Jiao L C , Ma W P , Liu F and Zhao W . 2019a . A novel neural network for remote sensing image matching . IEEE Transactions on Neural Networks and Learning Systems , 30 ( 9 ): 2853 - 2865 [ DOI: 10.1109/TNNLS.2018.2888757 http://dx.doi.org/10.1109/TNNLS.2018.2888757 ]

Zhu R J , Yu D W , Ji S P and Lu M . 2019b . Matching RGB and infrared remote sensing images with densely-connected convolutional neural networks . Remote Sensing , 11 ( 23 ): 2836 [ DOI: 10.3390/rs11232836 http://dx.doi.org/10.3390/rs11232836 ]

Alert me when the article has been cited

提交

Status and progress of deep learning-based pairwise point cloud rigid registration

Fine 3D mapping and vegetation spatial differentiation study based on UAV LiDAR point cloud in karst wetland

LAE-Transformer： An airborne LiDAR point cloud segmentation network incorporating local-aware enhancement

A multi-scale InSAR phase filtering method incorporating adaptive enhancement and dynamic screening mechanisms