遥感影像深度学习配准方法综述

李星华; 艾文浩; 冯蕊涛; 罗少杰

doi:10.11834/jrs.20235012

综述 | 浏览量 : 0 下载量: 871 CSCD: 0 更多指标

R-PDF
PDF
导出
分享
收藏
专辑

遥感影像深度学习配准方法综述
Survey of remote sensing image registration based on deep learning
2023年27卷第2期页码：267-284
纸质出版日期： 2023-02-07 ，
DOI： 10.11834/jrs.20235012

扫描看全文

李星华，艾文浩，冯蕊涛，罗少杰.2023.遥感影像深度学习配准方法综述.遥感学报，27（2）： 267-284

Li X H，Ai W H，Feng R T and Luo S J. 2023. Survey of remote sensing image registration based on deep learning. National Remote Sensing Bulletin， 27（2）：267-284
李星华，艾文浩，冯蕊涛，罗少杰.2023.遥感影像深度学习配准方法综述.遥感学报，27（2）： 267-284 DOI： 10.11834/jrs.20235012.

Li X H，Ai W H，Feng R T and Luo S J. 2023. Survey of remote sensing image registration based on deep learning. National Remote Sensing Bulletin， 27（2）：267-284 DOI： 10.11834/jrs.20235012.

摘要

遥感影像配准是指通过几何变换使两景或多景影像空间位置对齐的过程，是影像融合、变化检测、农业监测等应用的重要预处理步骤。近年来，深度学习引起了人们的广泛关注，并在遥感影像配准中成功应用。本文在简要介绍传统遥感影像配准方法的基础上，重点分析了深度学习在基于区域的配准方法、基于特征的配准方法两方面取得的重要进展，分享了用于遥感影像配准的公开数据集，并总结了深度学习在遥感影像配准中的机遇与挑战。

Abstract

Remote sensing image registration is the process of spatial alignment of two or more images through geometric transformation. It is an important preprocessing operation for image fusion

change detection

agricultural monitoring and other remote sensing applications. Considering that remote sensing images have the characteristics of large-scale changes

complex ground covers and imaging modalities

although a large number of registration methods have been developed

there is still a lack of methods that can be widely used in different scenarios. Therefore

research on registration algorithms with high efficiency

high robustness

high precision and wide applicability is of great significance. In recent years

deep learning

which has achieved great success in the field of natural image and medical image registration

has provided a new method for remote sensing image registration. First

we introduced two kinds of traditional registration methods and analyzed the advantages and disadvantages of area-based and feature-based registration methods in detail from the aspects of registration accuracy

efficiency and algorithm robustness. Generally

there are two main problems in traditional methods: poor applicability and insufficient utilization of the deep semantic information of the image. Second

we focused on the important progress of deep learning in area-based registration methods and feature-based registration methods. According to the specific application purpose of deep learning

we made a more detailed division of the above two methods and summarized the advantages and disadvantages of the existing research. In addition

considering the importance of datasets for deep learning

we sorted and shared some public datasets for remote sensing image registration. Due to the great progress of earth observation technology

an increasing number of remote sensing images are being applied. Image registration is the key step of remote sensing image preprocessing and the basic research content of quantitative remote sensing analysis. In recent years

research on remote sensing image registration algorithms based on deep learning has shown an increasing trend

but it is still in the early stage

and the framework is not mature. It mainly includes but is not limited to the following shortcomings: (1) lack of open source standard datasets; (2) difficult to apply to large-scale remote sensing images; (3) insufficient utilization of geospatial information and spectral information of remote sensing images; and (4) long training time and the large computing overhead. From the perspective of data and methods

we looked forward to the application of deep learning in the field of remote sensing image registration and put forward four main research directions: (1) remote sensing image registration datasets; (2) registration methods based on hybrid models; (3) registration methods based on different neural networks; and (4) training strategies based on small samples.

关键词

深度学习影像配准基于区域基于特征配准数据集

Keywords

deep learningimage registrationarea-basedfeature-basedregistration datasets

references

Balntas V, Lenc K, Vedaldi A and Mikolajczyk K. 2017. HPatches: a benchmark and evaluation of handcrafted and learned local descriptors//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hawaii: IEEE: 3852-3861 [DOI: 10.1109/CVPR.2017.410http://dx.doi.org/10.1109/CVPR.2017.410]

Barroso-Laguna A, Riba E, Ponsa D and Mikolajczyk K. 2019. Key.Net: keypoint detection by handcrafted and learned CNN filters. arXiv: 1904.00889v3

Blendowski M and Heinrich M P. 2019. Combining MRF-based deformable registration and deep binary 3D-CNN descriptors for large lung motion estimation in COPD patients. International Journal of Computer Assisted Radiology and Surgery, 14(1): 43-52 [DOI: 10.1007/s11548-018-1888-2http://dx.doi.org/10.1007/s11548-018-1888-2]

Chen H M, Varshney P K and Arora M K. 2003. Performance of mutual information similarity measure for registration of multitemporal remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 41(11): 2445-2454 [DOI: 10.1109/TGRS.2003.817664http://dx.doi.org/10.1109/TGRS.2003.817664]

Cheng X, Zhang L and Zheng Y F. 2018. Deep similarity learning for multimodal medical images. Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, 6(3): 248-252 [DOI: 10.1080/21681163.2015.1135299http://dx.doi.org/10.1080/21681163.2015.1135299]

Chopra S, Hadsell R and LeCun Y. 2005. Learning a similarity metric discriminatively, with application to face verification//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego: IEEE: 539-546 [DOI: 10.1109/CVPR.2005.202http://dx.doi.org/10.1109/CVPR.2005.202]

de Vos B D, Berendsen F F, Viergever M A, Sokooti H, Staring M and Išgum I. 2019. A deep learning framework for unsupervised affine and deformable image registration. Medical Image Analysis, 52: 128-143 [DOI: 10.1016/j.media.2018.11.010http://dx.doi.org/10.1016/j.media.2018.11.010]

DeTone D, Malisiewicz T and Rabinovich A. 2018. SuperPoint: self-supervised interest point detection and description. arXiv: 1712.07629v4

Dong Y Y, Jiao W L, Long T F, Liu L F, He G J, Gong C J and Guo Y T. 2019. Local deep descriptor for remote sensing image feature matching. Remote Sensing, 11(4): 430 [DOI: 10.3390/rs11040430http://dx.doi.org/10.3390/rs11040430]

Fan R B, Hou B C, Liu J B, Yang J H and Hong Z L. 2021. Registration of multiresolution remote sensing images based on L2-Siamese model. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14: 237-248 [DOI: 10.1109/JSTARS.2020.3038922http://dx.doi.org/10.1109/JSTARS.2020.3038922]

Feng R T, Du Q Y, Li X H and Shen H F. 2019. Robust registration for remote sensing images by combining and localizing feature- and area-based methods. ISPRS Journal of Photogrammetry and Remote Sensing, 151: 15-26 [DOI: 10.1016/j.isprsjprs.2019.03.002http://dx.doi.org/10.1016/j.isprsjprs.2019.03.002]

Feng R T. 2020. The Registration Algorithm for Remote Sensing Images Covering the Complex Terrain by Multi-Model Union. Wuhan: Wuhan University: 1-122

冯蕊涛. 2020. 复杂地形条件下的遥感影像多模型联合配准方法. 武汉: 武汉大学: 1-122

Feng R T, Shen H F, Bai J J and Li X H. 2021. Advances and opportunities in remote sensing image geometric registration: a systematic review of state-of-the-art approaches and future research directions. IEEE Geoscience and Remote Sensing Magazine, 9(4): 120-142 [DOI: 10.1109/MGRS.2021.3081763http://dx.doi.org/10.1109/MGRS.2021.3081763]

Feng R T, Du Q Y, Luo H, Shen H F, Li X H and Liu B. 2021. A registration algorithm based on optical flow modification for multi-temporal remote sensing images covering the complex-terrain region. National Remote Sensing Bulletin, 25(2): 630-640

冯蕊涛, 杜清运, 罗恒, 沈焕锋, 李星华, 刘波. 2021. 基于光流校正的复杂地形区多时相遥感影像配准. 遥感学报, 25(2): 630-640 [DOI: 10.11834/jrs.20209280http://dx.doi.org/10.11834/jrs.20209280]

Fischler M A and Bolles R C. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6): 381-395 [DOI: 10.1145/358669.358692http://dx.doi.org/10.1145/358669.358692]

Girard N, Charpiat G and Tarabalka Y. 2019. Aligning and updating cadaster maps with aerial images by multi-task, multi-resolution deep learning//14th Asian Conference on Computer Vision. Perth: Springer: 675-690 [DOI: 10.1007/978-3-030-20873-8_43http://dx.doi.org/10.1007/978-3-030-20873-8_43]

Gong M G, Zhao S M, Jiao L C, Tian D Y and Wang S. 2014. A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information. IEEE Transactions on Geoscience and Remote Sensing, 52(7): 4328-4338 [DOI: 10.1109/TGRS.2013.2281391http://dx.doi.org/10.1109/TGRS.2013.2281391]

Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Faley D, Ozair S, Courville A and Bengio Y. 2014. Generative adversarial networks. arXiv: 1406.2661v1

Harris C and Stephens M. 1988. A combined corner and edge detector//Proceedings of the Alvey Vision Conference. Manchester: AVC: 147-151 [DOI: 10.5244/C.2.23http://dx.doi.org/10.5244/C.2.23]

Haskins G, Kruecker J, Kruger U, Xu S, Pinto P A, Wood B J and Yan P K. 2019. Learning deep similarity metric for 3D MR–TRUS image registration. International Journal of Computer Assisted Radiology and Surgery, 14(3): 417-425 [DOI: 10.1007/s11548-018-1875-7http://dx.doi.org/10.1007/s11548-018-1875-7]

He H Q, Chen M, Chen T and Li D J. 2018. Matching of remote sensing images with complex background variations via Siamese convolutional neural network. Remote Sensing, 10(2): 355 [DOI: 10.3390/rs10020355http://dx.doi.org/10.3390/rs10020355]

He H Q, Chen M, Chen T, Li D J and Cheng P G. 2019. Learning to match multitemporal optical satellite images using multi-support-patches Siamese networks. Remote Sensing Letters, 10(6): 516-525 [DOI: 10.1080/2150704x.2019.1577572http://dx.doi.org/10.1080/2150704x.2019.1577572]

Hinton G E and Salakhutdinov R R. 2006. Reducing the dimensionality of data with neural networks. Science, 313(5786): 504-507 [DOI: 10.1126/science.1127647http://dx.doi.org/10.1126/science.1127647]

Holland P W and Welsch R E. 1977. Robust regression using iteratively reweighted least-squares. Communications in Statistics-Theory and Methods, 6(9): 813-827 [DOI: 10.1080/03610927708827533http://dx.doi.org/10.1080/03610927708827533]

Hughes L H, Schmitt M, Mou L C, Wang Y Y and Zhu X X. 2018. Identifying corresponding patches in SAR and optical images with a Pseudo-Siamese CNN. IEEE Geoscience and Remote Sensing Letters, 15(5): 784-788 [DOI: 10.1109/LGRS.2018.2799232http://dx.doi.org/10.1109/LGRS.2018.2799232]

Hui T W, Tang X O and Loy C C. 2018. LiteFlowNet: a lightweight convolutional neural network for optical flow estimation//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 8981-8989 [DOI: 10.1109/CVPR.2018.00936http://dx.doi.org/10.1109/CVPR.2018.00936]

Hui T W and Loy C C. 2020. LiteFlowNet3: resolving correspondence ambiguity for more accurate optical flow estimation//16th European Conference on Computer Vision. Glasgow: Springer: 169-184 [DOI: 10.1007/978-3-030-58565-5_11http://dx.doi.org/10.1007/978-3-030-58565-5_11]

Hui T W, Tang X O and Loy C C. 2021. A lightweight optical flow CNN—revisiting data fidelity and regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(8): 2555-2569 [DOI: 10.1109/TPAMI.2020.2976928http://dx.doi.org/10.1109/TPAMI.2020.2976928]

Jiang X Y, Ma J Y, Xiao G B, Shao Z F and Guo X J. 2021a. A review of multimodal image matching: methods and applications. Information Fusion, 73: 22-71 [DOI: 10.1016/j.inffus.2021.02.012http://dx.doi.org/10.1016/j.inffus.2021.02.012]

Jiang X Y, Ma J Y, Fan A X, Xu H P, Lin G, Lu T and Tian X. 2021b. Robust feature matching for remote sensing image registration via linear adaptive filtering. IEEE Transactions on Geoscience and Remote Sensing, 59(2): 1577-1591 [DOI: 10.1109/TGRS.2020.3001089http://dx.doi.org/10.1109/TGRS.2020.3001089]

Johnson K, Cole-Rhodes A, Zavorin I and Le Moigne J. 2001. Mutual information as a similarity measure for remote sensing image registration//Proceedings Volume 4383, Geo-Spatial Image and Data Exploitation II. Orlando: SPIE: 51-61 [DOI: 10.1117/12.428251http://dx.doi.org/10.1117/12.428251]

Kim D G, Nam W J and Lee S W. 2019. A robust matching network for gradually estimating geometric transformation on remote sensing imagery//2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). Bari: IEEE: 3889-3894 [DOI: 10.1109/SMC.2019.8913881http://dx.doi.org/10.1109/SMC.2019.8913881]

Kuppala K, Banda S and Barige T R. 2020. An overview of deep learning methods for image registration with focus on feature-based approaches. International Journal of Image and Data Fusion, 11(2): 113-135 [DOI: 10.1080/19479832.2019.1707720http://dx.doi.org/10.1080/19479832.2019.1707720]

Lan C Z, Lu W J, Yu J M and Xu Q. 2021. Deep learning algorithm for feature matching of cross modality remote sensing images. Acta Geodaetica et Cartographica Sinica, 50(2): 189-202

蓝朝桢, 卢万杰, 于君明, 徐青. 2021. 异源遥感影像特征匹配的深度学习算法. 测绘学报, 50(2): 189-202 [DOI: 10.11947/j.AGCS.2021.20200048http://dx.doi.org/10.11947/j.AGCS.2021.20200048]

LeCun Y, Bottou L, Bengio Y and Haffner P. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278-2324 [DOI: 10.1109/5.726791http://dx.doi.org/10.1109/5.726791]

LeCun Y, Bengio Y and Hinton G. 2015. Deep learning. Nature, 521(7553): 436-444 [DOI: 10.1038/nature14539http://dx.doi.org/10.1038/nature14539]

Lee W, Sim D and Oh S J. 2021. A CNN-based high-accuracy registration for remote sensing images. Remote Sensing, 13(8): 1482 [DOI: 10.3390/rs13081482http://dx.doi.org/10.3390/rs13081482]

Lee W J and Oh S J. 2021. Remote sensing image registration using equivariance features//2021 International Conference on Information Networking (ICOIN). Bangkok: IEEE: 776-781 [DOI: 10.1109/ICOIN50884.2021.9333861http://dx.doi.org/10.1109/ICOIN50884.2021.9333861]

Li J Y, Hu Q W and Ai M Y. 2020a. RIFT: multi-modal image matching based on radiation-variation insensitive feature transform. IEEE Transactions on Image Processing, 29: 3296-3310 [DOI: 10.1109/TIP.2019.2959244http://dx.doi.org/10.1109/TIP.2019.2959244]

Li J Y, Zhao P C, Hu Q W and Ai M Y. 2020b. Robust point cloud registration based on topological graph and Cauchy weighted lq-norm. ISPRS Journal of Photogrammetry and Remote Sensing, 160: 244-259 [DOI: 10.1016/j.isprsjprs.2019.12.008http://dx.doi.org/10.1016/j.isprsjprs.2019.12.008]

Li L, Ji S, Yu Y and Zhang Y S. 2020. A multi-feature-based registration method adapted to multi-source remote sensing images. Journal of Geomatics Science and Technology, 37(1): 74-78

李力, 纪松, 于英, 张永生. 2020. 一种基于组合特征的异源遥感影像配准方法. 测绘科学技术学报, 37(1): 74-78 [DOI: 10.3969/j.issn.1673-6338.2020.01.014http://dx.doi.org/10.3969/j.issn.1673-6338.2020.01.014]

Li X H, Feng R T, Guan X B, Shen H F and Zhang L P. 2019. Remote sensing image mosaicking: achievements and challenges. IEEE Geoscience and Remote Sensing Magazine, 7(4): 8-22 [DOI: 10.1109/MGRS.2019.2921780http://dx.doi.org/10.1109/MGRS.2019.2921780]

Li X H, Du Z S, Huang Y Y and Tan Z Y. 2021. A deep translation (GAN) based change detection network for optical and SAR remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 179: 14-34 [DOI: 10.1016/j.isprsjprs.2021.07.007http://dx.doi.org/10.1016/j.isprsjprs.2021.07.007]

Liang J Y, Liu X P, Huang K N, Li X, Wang D G and Wang X W. 2014. Automatic registration of multisensor images using an integrated spatial and mutual information (SMI) metric. IEEE Transactions on Geoscience and Remote Sensing, 52(1): 603-615 [DOI: 10.1109/TGRS.2013.2242895http://dx.doi.org/10.1109/TGRS.2013.2242895]

Liang Y, Sheng Y H, Zhang K and Yang L. 2014. Linear feature matching method based on local affine invariant and Epipolar constraint for close-range images. Geomatics and Information Science of Wuhan University, 39(2): 229-233

梁艳, 盛业华, 张卡, 杨林. 2014. 利用局部仿射不变及核线约束的近景影像直线特征匹配. 武汉大学学报(信息科学版), 39(2): 229-233 [DOI: 10.13203/j.whugis20120611http://dx.doi.org/10.13203/j.whugis20120611]

Liao R, Miao S, de Tournemire P, Grbic S, Kamen A, Mansi T and Comaniciu D. 2016. An artificial agent for robust image registration. arXiv: 1611.10336v1

Liu P P, Irwin K, Lyu M R and Xu J. 2019a. DDFlow: learning optical flow with unlabeled data distillation. Proceedings of the AAAI Conference on Artificial Intelligence, 33(1): 8770-8777 [DOI: 10.1609/aaai.v33i01.33018770http://dx.doi.org/10.1609/aaai.v33i01.33018770]

Liu P P, Lyu M, King I and Xu J. 2019b. SelFlow: self-supervised learning of optical flow//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE: 4571-4580 [DOI: 10.1109/CVPR.2019.00470http://dx.doi.org/10.1109/CVPR.2019.00470]

Lowe D G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2): 91-110 [DOI: 10.1023/b:visi.0000029664.99615.94http://dx.doi.org/10.1023/b:visi.0000029664.99615.94]

Lu J Y, Jia H G, Li T, Li Z Q, Ma J Y and Zhu R F. 2021. An instance segmentation based framework for large-sized high-resolution remote sensing images registration. Remote Sensing, 13(9): 1657 [DOI: 10.3390/rs13091657http://dx.doi.org/10.3390/rs13091657]

Luo Y, Cao W M, He Z Q, Zou W L and He Z H. 2021. Deformable adversarial registration network with multiple loss constraints. Computerized Medical Imaging and Graphics, 91: 101931 [DOI: 10.1016/j.compmedimag.2021.101931http://dx.doi.org/10.1016/j.compmedimag.2021.101931]

Ma D A, Tang P, Zhao L J and Zhang Z. 2021. Review of data augmentation for image in deep learning. Journal of Image and Graphics, 26(3): 487-502

马岽奡, 唐娉, 赵理君, 张正. 2021. 深度学习图像数据增广方法研究综述. 中国图象图形学报, 26(3): 487-502 [DOI: 10.11834/jig.200089http://dx.doi.org/10.11834/jig.200089]

Ma F L. 2018. Research on Color-To-Gray Conversion based on Normalized Cross Correlation. Lanzhou: Lanzhou University: 1-61

马方龙. 2018. 基于归一化积相关匹配的彩色图像灰度化研究. 兰州: 兰州大学: 1-61

Ma J Y, Jiang X Y, Jiang J J, Zhao J and Guo X J. 2019c. LMR: learning a two-class classifier for mismatch removal. IEEE Transactions on Image Processing, 28(8): 4045-4059 [DOI: 10.1109/TIP.2019.2906490http://dx.doi.org/10.1109/TIP.2019.2906490]

Ma J Y, Jiang X Y, Fan A X, Jiang J J and Yan J C. 2021. Image matching from handcrafted to deep features: a survey. International Journal of Computer Vision, 129(1): 23-79 [DOI: 10.1007/s11263-020-01359-2http://dx.doi.org/10.1007/s11263-020-01359-2]

Ma L, Liu Y, Zhang X L, Ye Y X, Yin G F and Johnson B A. 2019a. Deep learning in remote sensing applications: a meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, 152: 166-177 [DOI: 10.1016/j.isprsjprs.2019.04.015http://dx.doi.org/10.1016/j.isprsjprs.2019.04.015]

Ma W P, Zhang J, Wu Y, Jiao L C, Zhu H and Zhao W. 2019b. A novel two-step registration method for remote sensing images based on deep and local features. IEEE Transactions on Geoscience and Remote Sensing, 57(7): 4834-4843 [DOI: 10.1109/TGRS.2019.2893310http://dx.doi.org/10.1109/TGRS.2019.2893310]

Ni H, Feng Z, Guan Y, Jia X Y, Chen W, Jiang T, Zhong Q Y, Yuan J, Ren M, Li X N, Gong H, Luo Q M and Li A N. 2021. DeepMapi: a fully automatic registration method for mesoscopic optical brain images using convolutional neural networks. Neuroinformatics, 19(2): 267-284 [DOI: 10.1007/s12021-020-09483-7http://dx.doi.org/10.1007/s12021-020-09483-7]

Niu R G, Sun X, Tian Y, Diao W H, Chen K Q and Fu K. 2020. Hybrid multiple attention network for semantic segmentation in aerial images. arXiv: 2001.02870v3

Park J H, Nam W J and Lee S W. 2020. A two-stream symmetric network with bidirectional ensemble for aerial image matching. Remote Sensing, 12(3): 465 [DOI: 10.3390/rs12030465http://dx.doi.org/10.3390/rs12030465]

Rahaghi A I, Lemmin U, Sage D and Barry D A. 2019. Achieving high-resolution thermal imagery in low-contrast lake surface waters by aerial remote sensing and image registration. Remote Sensing of Environment, 221: 773-783 [DOI: 10.1016/j.rse.2018.12.018http://dx.doi.org/10.1016/j.rse.2018.12.018]

Ranjan A and Black M J. 2017. Optical flow estimation using a spatial pyramid network//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hawaii: IEEE: 2720-2729 [DOI: 10.1109/CVPR.2017.291http://dx.doi.org/10.1109/CVPR.2017.291]

Rocco I, Arandjelovic R and Sivic J. 2017. Convolutional neural network architecture for geometric matching//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hawaii: IEEE: 39-48 [DOI: 10.1109/CVPR.2017.12http://dx.doi.org/10.1109/CVPR.2017.12]

Savinov N, Seki A, Ladicky L, Sattler T and Pollefeys M. 2017. Quad-networks: unsupervised learning to rank for interest point detection. arXiv: 1611.07571v2

Schmidhuber J. 2015. Deep learning in neural networks: an overview. Neural Networks, 61: 85-117 [DOI: 10.1016/j.neunet.2014.09.003http://dx.doi.org/10.1016/j.neunet.2014.09.003]

Schmitt M, Hughes L H and Zhu X X. 2018. The SEN1-2 dataset for deep learning in SAR-optical data fusion. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-1: 141-146 [DOI: 10.5194/isprs-annals-IV-1-141-2018http://dx.doi.org/10.5194/isprs-annals-IV-1-141-2018]

Seo S, Choi J S, Lee J, Kim H H, Seo D, Jeong J and Kim M. 2020. UPSNet: unsupervised pan-sharpening network with registration learning between panchromatic and multi-spectral images. IEEE Access, 8: 201199-201217 [DOI: 10.1109/ACCESS.2020.3035802http://dx.doi.org/10.1109/ACCESS.2020.3035802]

Shabanov A, Gladilin S and Shvets E. 2020. Optical-to-SAR image registration using a combination of CNN descriptors and cross-correlation coefficient//Twelfth International Conference on Machine Vision (ICMV). Amsterdam: SPIE: 440-449 [DOI: 10.1117/12.2558414http://dx.doi.org/10.1117/12.2558414]

Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640-651 [DOI: 10.1109/TPAMI.2016.2572683http://dx.doi.org/10.1109/TPAMI.2016.2572683]

Simonovsky M, Gutiérrez-Becker B, Mateus D, Navab N and Komodakis N. 2016. A deep metric for multimodal registration//19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens: Springer: 10-18 [DOI: 10.1007/978-3-319-46726-9_2http://dx.doi.org/10.1007/978-3-319-46726-9_2]

Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition. arXiv: 1409.1556v6

Smith S M and Brady J M. 1997. SUSAN-a new approach to low level image processing. International Journal of Computer Vision, 23(1): 45-78 [DOI: 10.1023/A:1007963824710http://dx.doi.org/10.1023/A:1007963824710]

Suri S and Reinartz P. 2010. Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas. IEEE Transactions on Geoscience and Remote Sensing, 48(2): 939-949 [DOI: 10.1109/TGRS.2009.2034842http://dx.doi.org/10.1109/TGRS.2009.2034842]

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser L and Polosukhin I. 2017. Attention is all you need. arXiv: 1706.03762v5

Verdie Y, Yi K M, Fua P and Lepetit V. 2015. TILDE: a temporally invariant learned detector//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE: 5279-5288 [DOI: 10.1109/CVPR.2015.7299165http://dx.doi.org/10.1109/CVPR.2015.7299165]

Wang S, Quan D, Liang X F, Ning M D, Guo Y H and Jiao L C. 2018. A deep learning framework for remote sensing image registration. ISPRS Journal of Photogrammetry and Remote Sensing, 145: 148-164 [DOI: 10.1016/j.isprsjprs.2017.12.012http://dx.doi.org/10.1016/j.isprsjprs.2017.12.012]

Wang Y Y and Zhu X X. 2018. The SARptical dataset for joint analysis of SAR and optical image in dense urban area. arXiv: 1801.07532

Wang Z H and Wu F C. 2009. Mean-standard deviation descriptor and line matching. Pattern Recognition and Artificial Intelligence, 22(1): 32-39

王志衡, 吴福朝. 2009. 均值-标准差描述子与直线匹配. 模式识别与人工智能, 22(1): 32-39 [DOI: 10.3969/j.issn.1003-6059.2009.01.006http://dx.doi.org/10.3969/j.issn.1003-6059.2009.01.006]

Xiang Y M, Tao R S, Wang F, You H J and Han B. 2020. Automatic registration of optical and SAR images via improved phase congruency model. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 5847-5861 [DOI: 10.1109/JSTARS.2020.3026162http://dx.doi.org/10.1109/JSTARS.2020.3026162]

Yang J, Yang J H, Zhao F and Zhang W J. 2021. An unsupervised multi-scale framework with attention-based network (MANet) for lung 4D-CT registration. Physics in Medicine and Biology, 66(13): 135008 [DOI: 10.1088/1361-6560/ac0afchttp://dx.doi.org/10.1088/1361-6560/ac0afc]

Yang Z Q, Dan T T and Yang Y. 2018. Multi-temporal remote sensing image registration using deep convolutional features. IEEE Access, 6: 38544-38555 [DOI: 10.1109/ACCESS.2018.2853100http://dx.doi.org/10.1109/ACCESS.2018.2853100]

Yao M Q and Hu J. 2020. Robust multimodal medical image registration using deep recurrent reinforcement learning. Journal of Computer-Aided Design and Computer Graphics, 32(8): 1236-1247

姚明青, 胡靖. 2020. 基于深度强化学习的多模态医学图像配准. 计算机辅助设计与图形学学报, 32(8): 1236-1247 [DOI: 10.3724/SP.J.1089.2020.17847http://dx.doi.org/10.3724/SP.J.1089.2020.17847]

Ye F M, Su Y F, Xiao H, Zhao X Q and Min W D. 2018. Remote sensing image registration using convolutional neural network features. IEEE Geoscience and Remote Sensing Letters, 15(2): 232-236 [DOI: 10.1109/LGRS.2017.2781741http://dx.doi.org/10.1109/LGRS.2017.2781741]

Ye Y X and Shan J. 2014. A local descriptor based registration method for multispectral remote sensing images with non-linear intensity differences. ISPRS Journal of Photogrammetry and Remote Sensing, 90: 83-95 [DOI: 10.1016/j.isprsjprs.2014.01.009http://dx.doi.org/10.1016/j.isprsjprs.2014.01.009]

Yi K M, Trulls E, Lepetit V and Fua P. 2016. LIFT: learned invariant feature transform//14th European Conference on Computer Vision (ECCV). Amsterdam: Springer: 467-483 [DOI: 10.1007/978-3-319-46466-4_28http://dx.doi.org/10.1007/978-3-319-46466-4_28]

Yi K M, Trulls E, Ono Y, Lepetit V, Salzmann M and Fua P. 2018. Learning to find good correspondences//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE: 2666-2674 [DOI: 10.1109/CVPR.2018.00282http://dx.doi.org/10.1109/CVPR.2018.00282]

Yu X C, Lü Z H and Hu D. 2013. Review of remote sensing image registration techniques. Optics and Precision Engineering, 21(11): 2960-2972

余先川, 吕中华, 胡丹. 2013. 遥感图像配准技术综述. 光学精密工程, 21(11): 2960-2972 [DOI: 10.3788/OPE.20132111.2960http://dx.doi.org/10.3788/OPE.20132111.2960]

Yuan Z, Guo H T, Lu J, Lu W and Lin Y Z. 2021. High-resolution remote sensing image change detection technology based on UNet++ and attention mechanism. Journal of Geomatics Science and Technology, 38(2): 155-159

袁洲, 郭海涛, 卢俊, 路威, 林雨准. 2021. 融合UNet++网络和注意力机制的高分辨率遥感影像变化检测算法. 测绘科学技术学报, 38(2): 155-159 [DOI: 10.3969/j.issn.1673-6338.2021.02.008http://dx.doi.org/10.3969/j.issn.1673-6338.2021.02.008]

Zampieri A, Charpiat G, Girard N and Tarabalka Y. 2018. Multimodal image alignment through a multiscale chain of neural networks with application to remote sensing//15th European Conference on Computer Vision (ECCV). Munich: Springer: 679-696 [DOI: 10.1007/978-3-030-01270-0_40http://dx.doi.org/10.1007/978-3-030-01270-0_40]

Zeng L, Du Y L, Lin H P, Wang J, Yin J J and Yang J. 2021. A novel region-based image registration method for multisource remote sensing images via CNN. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14: 1821-1831 [DOI: 10.1109/JSTARS.2020.3047656http://dx.doi.org/10.1109/JSTARS.2020.3047656]

Zeng Y, Ning Z H, Liu P, Luo P L, Zhang Y and He G J. 2020. A mosaic method for multi-temporal data registration by using convolutional neural networks for forestry remote sensing applications. Computing, 102(3): 795-811 [DOI: 10.1007/s00607-019-00716-5http://dx.doi.org/10.1007/s00607-019-00716-5]

Zhang H, Ni W P, Yan W D, Xiang D L, Wu J Z, Yang X L and Bian H. 2019a. Registration of multimodal remote sensing image based on deep fully convolutional neural network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(8): 3028-3042 [DOI: 10.1109/JSTARS.2019.2916560http://dx.doi.org/10.1109/JSTARS.2019.2916560]

Zhang J, Ma W P, Wu Y and Jiao L C. 2019b. Multimodal remote sensing image registration based on image transfer and local features. IEEE Geoscience and Remote Sensing Letters, 16(8): 1210-1214 [DOI: 10.1109/LGRS.2019.2896341http://dx.doi.org/10.1109/LGRS.2019.2896341]

Zhang L G and Rusinkiewicz S. 2018. Learning to detect features in texture images//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE: 6325-6333 [DOI: 10.1109/CVPR.2018.00662http://dx.doi.org/10.1109/CVPR.2018.00662]

Zhao L Y, Lü B Y, Li X R and Chen S H. 2015. Multi-source remote sensing image registration based on scale-invariant feature transform and optimization of regional mutual information. Acta Physica Sinica, 64(12): 124204

赵辽英, 吕步云, 厉小润, 陈淑涵. 2015. 基于尺度不变特征变换和区域互信息优化的多源遥感图像配准. 物理学报, 64(12): 124204 [DOI: 10.7498/aps.64.124204http://dx.doi.org/10.7498/aps.64.124204]

Zheng Z D, Wei Y C and Yang Y. 2020. University-1652: a multi-view multi-source benchmark for drone-based geo-localization. arXiv: 2002.12186v2

Zhou Z H. 2018. A brief introduction to weakly supervised learning. National Science Review, 5(1): 44-53 [DOI: 10.1093/nsr/nwx106http://dx.doi.org/10.1093/nsr/nwx106]

Zhu H, Jiao L C, Ma W P, Liu F and Zhao W. 2019a. A novel neural network for remote sensing image matching. IEEE Transactions on Neural Networks and Learning Systems, 30(9): 2853-2865 [DOI: 10.1109/TNNLS.2018.2888757http://dx.doi.org/10.1109/TNNLS.2018.2888757]

Zhu R J, Yu D W, Ji S P and Lu M. 2019b. Matching RGB and infrared remote sensing images with densely-connected convolutional neural networks. Remote Sensing, 11(23): 2836 [DOI: 10.3390/rs11232836http://dx.doi.org/10.3390/rs11232836]

文章被引用时，请邮件提醒。

提交

生成式知识迁移的SAR舰船检测

MtSCCD：面向深度学习的土地利用场景分类与变化检测数据集

光学信号Token引导的异源遥感变化检测网络

模态间匹配学习的高光谱和激光雷达联合分类

考虑光谱信息和超像素分割的高光谱解混网络