基于深度学习的像素级全色图像锐化研究综述

杨勇; 苏昭; 黄淑英; 万伟国; 涂伟; 卢航远

doi:10.11834/jrs.20211325

综述 | 浏览量 : 0 下载量: 650 CSCD: 1 更多指标

R-PDF
PDF
导出
分享
收藏
专辑

基于深度学习的像素级全色图像锐化研究综述
Deep-learning approaches for pixel-level pansharpening
2022年26卷第12期页码：2411-2432
纸质出版日期： 2022-12-07 ，
DOI： 10.11834/jrs.20211325

扫描看全文

杨勇，苏昭，黄淑英，万伟国，涂伟，卢航远.2022.基于深度学习的像素级全色图像锐化研究综述.遥感学报，26（12）： 2411-2432

Yang Y， Su Z， Huang S Y， Wan W G， Tu W and Lu H Y. 2022. Survey of deep-learning approaches for pixel-level pansharpening. National Remote Sensing Bulletin， 26（12）：2411-2432
杨勇，苏昭，黄淑英，万伟国，涂伟，卢航远.2022.基于深度学习的像素级全色图像锐化研究综述.遥感学报，26（12）： 2411-2432 DOI： 10.11834/jrs.20211325.

Yang Y， Su Z， Huang S Y， Wan W G， Tu W and Lu H Y. 2022. Survey of deep-learning approaches for pixel-level pansharpening. National Remote Sensing Bulletin， 26（12）：2411-2432 DOI： 10.11834/jrs.20211325.

摘要

全色图像锐化是遥感数据处理领域的一个基础性问题，在地物分类、目标识别等方面具有重要的研究意义和应用价值。近年来，深度学习在自然语言处理、计算机视觉等领域取得了巨大进展，也推动了像素级全色图像锐化技术的发展。本文提出从经典方式和协同方式两个方面对深度学习在全色图像锐化中的研究进行系统的综述，并在此基础上进行前景展望。首先，给出全色图像锐化常用的数据集和全色图像锐化的质量评价指标；接着，从经典方式与协同方式两个方面对基于深度学习的全色图像锐化最新研究成果进行分门别类的介绍，并进行算法性能的对比、分析和归纳；然后，对全色图像锐化的3个主要应用领域如地物分类、目标识别和地表变化检测进行分析；最后，本文探讨了基于深度学习的全色图像锐化的5个未来研究方向。

Abstract

Pansharpening is a fundamental problem in the field of remote sensing data processing. It has important research significance and application value in ground object classification and ground surface change detection. In recent years

Deep Learning (DL) has made great progress in natural language processing

computer vision

etc. and has promoted the development of pixel-level pansharpening technology. This work presents a systematic review of the research of DL in pansharpening from two aspects (classical and collaborative approaches) and makes a prospect on this basis. First

the common datasets of pansharpening and the objective evaluation indexes of pansharpening

including reference and non-reference quality evaluation indexes

are provided. Second

the latest research results of DL-based pansharpening are introduced in two different categories from the classical and collaborative methods

and the performance of their algorithms is compared

analyzed

and summarized. The classical methods mainly include AE-based pansharpening

CNN-based pansharpening

DRN-based pansharpening

and GAN-based pansharpening methods. Meanwhile

the collaborative methods mainly include DL+CS-based pansharpening

DL+MRA-based pansharpening

DL+MB-based pansharpening

DL+injection model-based pansharpening

CNN+DRN-based pansharpening

and RNN+CNN-based pansharpening methods. In the comparative analysis of the classical and collaborative methods

the common point is that the DL technology can automatically learn the advantages of complex data features and extract the feature information of the MS or PAN image (i.e.

the information that needs to be retained in the HRMS fusion image). The difference is that the structure of the classical mode is more concise

while that of the collaborative mode is more complex because it is the combination of multiple methods or frameworks. In addition

most early DL-based pansharpening methods utilized the powerful data fitting ability of the DL model and seldom paid attention to the field of pansharpening problems. With the gradual deepening of research

such as using DL methods combined with traditional pansharpening methods

this designed fusion model considers spectral and spatial distortions. Accordingly

the DL methods can further enhance the pansharpening effect. Thirdly

the three main application fields of pansharpening are analyzed

such as object classification

target recognition

and surface change detection. Finally

this work discusses the future research direction of DL-based pansharpening in combination with remote sensing knowledge to fully tap the potential of DL to obtain fused images with richer details and more natural spectra. For example

for the evaluation of pansharpening application

the performance of pansharpening in a certain application is related not only to the high quality of fusion image but also to the knowledge of a specific application field. Accordingly

the application-oriented pansharpening evaluation algorithms will be the focus of future study. Furthermore

DL-based pansharpening needs to train a large number of network parameters

resulting in a longer training time for the pansharpening model. The lightweight depth model has a smaller network capacity

lower time complexity

and lower hardware requirements. Therefore

constructing a lightweight pansharpening model is a promising future direction.

关键词

全色图像锐化深度学习经典方式协同方式质量评价遥感图像融合

Keywords

pansharpeningdeep learningclassical modecollaborative modequality evaluationremote sensing image fusion

references

Aguilar M A, Saldaña M M and Aguilar F. 2013. GeoEye-1 and worldview-2 pan-sharpened imagery for object-based classification in urban environments. International Journal of Remote Sensing, 34(7):2583-2606 [DOI: 10.1080/01431161.2012.747018http://dx.doi.org/10.1080/01431161.2012.747018]

Aiazzi B, Baronti S and Selva M. 2007. Improving component substitution pansharpening through multivariate regression of ms+pan data. IEEE Transactions on Geoscience and Remote Sensing, 45(10):3230-3239 [DOI: 10.1109/tgrs.2007.901007http://dx.doi.org/10.1109/tgrs.2007.901007]

Alparone L, Aiazzi B, Baronti S and Garzelli A. 2015. Remote sensing image fusion. London, UK: CRC Press

Alparone L, Wald L, Chanussot J, Thomas C, Gamba P and Bruce L M. 2007. Comparison of pansharpening algorithms: outcome of the 2006 grs-s data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45(10):3012-3021 [DOI: 10.1109/TGRS.2007.904923http://dx.doi.org/10.1109/TGRS.2007.904923]

Azarang A and Ghassemian H. 2017. A new pansharpening method using multi resolution analysis framework and deep neural networks. // Proceedings of the International Conference on Pattern Recognition and Image Analysis. Shahrekord, Iran, 1-6 [DOI: 10.1109/pria.2017.7983017http://dx.doi.org/10.1109/pria.2017.7983017]

Azarang A, Manoochehri H E and Kehtarnavaz N. 2019. Convolutional autoencoder-based multispectral image fusion. IEEE Access, 7:35673-35683 [DOI: 10.1109/ACCESS.2019.2905511http://dx.doi.org/10.1109/ACCESS.2019.2905511]

Benzenati T, Kessentini Y, Kallel A and Hallabia H. 2020. Generalized laplacian pyramid pan-sharpening gain injection prediction based on cnn. IEEE Geoscience and Remote Sensing Letters, 17(4):651-655 [DOI: 10.1109/LGRS.2019.2928181http://dx.doi.org/10.1109/LGRS.2019.2928181]

Bovolo F, Bruzzone L, Capobianco L, Garzelli A, Marchesi S and Nencini F. 2010. Analysis of the effects of pansharpening in change detection on VHR images. IEEE Geoscience and Remote Sensing Letters, 7(1): 53-57 [DOI: 10.1109/LGRS.2009.2029248http://dx.doi.org/10.1109/LGRS.2009.2029248]

Bowman S R, Angeli G, Potts C and Manning C D. 2015. A large annotated corpus for learning　natural language　inference//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Lisbon, Portugal: 632-642 [DOI: 10.18653/v1/D15-1075http://dx.doi.org/10.18653/v1/D15-1075]

Cai W T, Xu Y, Wu Z B, Liu H Y, Qian L and Wei Z H. 2018. Pan-sharpening based on multilevel coupled deep network. // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain: 7046-7049 [DOI: 10.1109/IGARSS.2018.8518121http://dx.doi.org/10.1109/IGARSS.2018.8518121]

Carper W J, Lillesand T M and Kiefer P W. 1990. The use of intensity-hue-saturation transformations for merging spot panchromatic and multispectral image data. Photogrammetric Engineering and Remote Sensing, 56(4):459–467

Chen C, Fu J Q, Gai Y Y, Li J, Chen L, Mantravadi V S and Tan A H. 2018. Damaged bridges over water: using high-spatial-resolution remote-sensing images for recognition, detection, and assessment. IEEE Geoscience and Remote Sensing Magazine, 6(3):69-85 [DOI: 10.1109/MGRS.2018.2852804http://dx.doi.org/10.1109/MGRS.2018.2852804]

Chen C, Fu J Q, Lu N, Chu YL, Hu J C, Guo B Y and Zhao X. 2019. Knowledge-based identification and damage detection of bridges spanning water via high-spatial-resolution optical remotely sensed imagery. Journal of the Indian Society of Remote Sensing, 47(12):1999-2008 [DOI: 10.1007/s12524-019-01036-zhttp://dx.doi.org/10.1007/s12524-019-01036-z]

Chen F J, Zhu F, Wu Q X, Hao Y M, Wang E D and Cui Y G. 2021. A survey about image generation with generative adversarial nets. Chinese Journal of Computers, 44(2): 347-369

陈佛计,朱枫,吴清潇,郝颖明,王恩德,崔芸阁. 2021.生成对抗网络及其在图像生成中的应用研究综述.计算机学报, 44(2):347-369 [DOI: 10.11897/SP.J.1016.2021.00347http://dx.doi.org/10.11897/SP.J.1016.2021.00347]

Chen J F, Pan Y and Chen Y. 2020. Remote sensing image fusion based on Bayesian GAN. ArXiv Preprint ArXiv: 2009.09465

Chen M M, Guo Q, Liu M L and Li A. 2021. Pan-sharpening by residual network with dense convolution for remote sensing images. Journal of Remote Sensing，25（6）：1270-1283

陈毛毛,郭擎, 刘明亮, 李安. 2021. 密集卷积残差网络的遥感图像融合. 遥感学报, 25(6): 1270-1283 [DOI：10.11834/jrs.20219411http://dx.doi.org/10.11834/jrs.20219411]

Da Cunha A L, Zhou J P and Do M N. 2006. The nonsubsampled contourlet transform: theory, design, and applications. IEEE Transactions on Image Processing, 15(10):3089-3101 [DOI: 10.1109/TIP.2006.877507http://dx.doi.org/10.1109/TIP.2006.877507]

Deng L J, Vivone G, Jin C and Chanussot J. 2020. Detail injection-based deep convolutional neural networks for pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 59(8):6995-7010 [DOI: 10.1109/TGRS.2020.3031366http://dx.doi.org/10.1109/TGRS.2020.3031366]

Dong C, Loy C C, He K M and Tang X O. 2016. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2):295-307 [DOI: 10.1109/TPAMI.2015.2439281http://dx.doi.org/10.1109/TPAMI.2015.2439281]

Eghbalian S and Ghassemian H. 2018. Multi spectral image fusion by deep convolutional neural network and new spectral loss function. International Journal of Remote Sensing, 39(12):3983-4002 [DOI: 10.1080/01431161.2018.1452074http://dx.doi.org/10.1080/01431161.2018.1452074]

Garzelli A, Capobianco L and Nencini F. 2009. On the effects of pan-sharpening to target detection//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Cape Town, South Africa: II-136‒II-139 [DOI: 10.1109/IGARSS.2009.5418022http://dx.doi.org/10.1109/IGARSS.2009.5418022]

Garzelli A, Nencini F and Capobianco L. 2008. Optimal mmse pansharpening of very high resolution multispectral images. IEEE Transactions on Geoscience and Remote Sensing, 46(1):228-236 [DOI: 10.1109/TGRS.2007.907604http://dx.doi.org/10.1109/TGRS.2007.907604]

Gianinetto M, Rusmini M, Candiani G, Via G D, Frassy F, Maianti P, Marchesi A, Nodari FR and Dini L. 2017. Hierarchical classification of complex landscape with VHR pan-sharpened satellite data and obia techniques. European Journal of Remote Sensing, 47(1):229-250 [DOI: 10.5721/EuJRS20144715http://dx.doi.org/10.5721/EuJRS20144715]

Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A C and Bengio Y. 2014. Generative adversarial nets//Proceedings of the Advances in Neural Information Processing Systems (NIPS). Montreal, Quebec, Canada: 2672-2680

Guo Y C, Ye F and Gong H. 2019. Learning an efficient convolution neural network for pansharpening. Algorithms, 12(1):16:1-16:14 [DOI: 10.3390/a12010016http://dx.doi.org/10.3390/a12010016]

He G Q, Xing S Y, Xia Z Q, Huang Q Q and Fan J P. 2018. Panchromatic and multi-spectral image fusion for new satellites based on multi-channel deep model. Machine Vision and Applications, 29(6):933-946 [DOI: 10.1007/s00138-018-0964-5http://dx.doi.org/10.1007/s00138-018-0964-5]

He K M and Sun J. 2015. Convolutional neural networks at constrained time cost//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: 5353-5360 [DOI: 10.1109/CVPR.2015.7299173http://dx.doi.org/10.1109/CVPR.2015.7299173]

He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition. //Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: 770-778 [DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]

He L, Rao Y Z, Li J, Chanussot J, Plaza A, Zhu J W and Li B. 2019. Pansharpening via detail injection based convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(4):1188-1204 [DOI: 10.1109/JSTARS.2019.2898574http://dx.doi.org/10.1109/JSTARS.2019.2898574]

He L, Xi D H, Li J and Zhu J W. 2020. A spectral-aware convolutional neural network for pansharpening. Applied Sciences, 10(17):5809:1-5809:18 [DOI: 10.3390/app10175809http://dx.doi.org/10.3390/app10175809]

Hinton G E and Salakhutdinov R R. 2006. Reducing the dimensionality of data with neural networks. Science, 313:504-507 [DOI: 10.1126/science.1127647http://dx.doi.org/10.1126/science.1127647]

Hinton G E, Osindero S and Teh Y W. 2006. A fast learning algorithm for deep belief nets. Neural Computation, 18(7):1527-1554 [DOI: 10.1162/neco.2006.18.7.1527http://dx.doi.org/10.1162/neco.2006.18.7.1527]

Hochreiter S and Schmidhuber J. 1997. Long short-term memory. Neural Computation, 9(8):1735-1780 [DOI: 10.1162/neco.1997.9.8.1735http://dx.doi.org/10.1162/neco.1997.9.8.1735]

Hu J, He Z and Wu J M. 2019. Deep self-learning network for adaptive pansharpening. Remote Sensing, 11(20):2395:1-2395:22 [DOI: 10.3390/rs11202395http://dx.doi.org/10.3390/rs11202395]

Huang G, Liu Z, Maaten L V D, and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: 2261-2269 [DOI: 10.1109/CVPR.2017.243http://dx.doi.org/10.1109/CVPR.2017.243]

Huang W, Fei X, Yin J R and Liu Y. 2018. A multi-direction subbands and deep neural networks based pan-sharpening method//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain: 5139-5142 [DOI: 10.1109/IGARSS.2018.8517817http://dx.doi.org/10.1109/IGARSS.2018.8517817]

Huang W, Xiao L, Wei Z H, Liu H Y and Tang S Z. 2015. A new pan-sharpening method with deep neural networks. IEEE Geoscience and Remote Sensing Letters, 12(5):1037-1041 [DOI: 10.1109/LGRS.2014.2376034http://dx.doi.org/10.1109/LGRS.2014.2376034]

Javan F D, Samadzadegan F, Mehravar S, Toosi A, Khatami R and Stein A. 2021. A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 171:101-117 [DOI: 10.1016/j.isprsjprs.2020.11.001http://dx.doi.org/10.1016/j.isprsjprs.2020.11.001]

Jiang M H, Li J, Yuan Q Q, Shen H F, Liu X X and Xu M M. 2019. Differential information residual convolutional neural network for pansharpening. // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan: 4865-4868 [DOI: 10.1109/IGARSS.2019.8900270http://dx.doi.org/10.1109/IGARSS.2019.8900270]

Kang X D, Li S T and Benediktsson J A. 2014. Pansharpening with matting model. IEEE Transactions on Geoscience and Remote Sensing, 52(8):5088-5099 [DOI: 10.1109/TGRS.2013.2286827http://dx.doi.org/10.1109/TGRS.2013.2286827]

Khan M M, Chanussot J, Condat L and Montanvert A. 2008. Indusion: fusion of multispectral and panchromatic images using the induction scaling technique. IEEE Geoscience and Remote Sensing Letters, 5(1):98-102 [DOI: 10.1109/LGRS.2007.909934http://dx.doi.org/10.1109/LGRS.2007.909934]

Kim J, Lee J K and Lee K M. 2016. Accurate image super-resolution using very deep convolutional networks. // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: 1646-1654 [DOI: 10.1109/CVPR.2016.182http://dx.doi.org/10.1109/CVPR.2016.182]

Laben C A and Brower B V. 2000. Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. U.S., No.6011875

Lee J, Seo S and Kim M. 2021. Sipsa-net: shift-invariant pan sharpening with moving object alignment for satellite imagery//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Online: 10161-10169 [DOI: 10.1109/CVPR46437.2021.01003http://dx.doi.org/10.1109/CVPR46437.2021.01003]

Li H, Liu F, Yang S Y and Zhang K. 2016. Remote sensing image fusion based on deep support value learning networks. Chinese Journal of Computers, 39(8): 1583-1596

李红, 刘芳, 杨淑媛, 张凯.2016.基于深度支撑值学习网络的遥感图像融合. 计算机学报, 39(8): 1583-1596 [DOI: 10.11897/SP.J.1016.2016.01583http://dx.doi.org/10.11897/SP.J.1016.2016.01583]

Li J, Li Y F, He L, Chen J and Plaza A. 2020a. Spatio-temporal fusion for remote sensing data: an overview and new benchmark. Science China Information Sciences, 63(4) :7-23 [DOI: 10.1007/s11432-019-2785-yhttp://dx.doi.org/10.1007/s11432-019-2785-y]

Li N, Huang N and Xiao L. 2017. Pan-sharpening via residual deep learning//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Fort Worth, TX, USA: 5133-5136 [DOI: 10.1109/IGARSS.2017.8128158http://dx.doi.org/10.1109/IGARSS.2017.8128158]

Li S T，Li C Y and Kang X D. 2021. Development status and future prospects of multi-source remote sensing image fusion. Journal of Remote Sensing, 25(1)：148-166

李树涛, 李聪妤, 康旭东. 2021. 多源遥感图像融合发展现状与未来展望. 遥感学报, 25(1):148-166 [DOI：10.11834/jrs.20210259http://dx.doi.org/10.11834/jrs.20210259]

Li S. 2018. Detection Method of Urban Land Cover Change Based on Combining Domain Knowledge and Deep Learning. Wuhan: Wuhan University

李胜. 2018. 联合领域知识和深度学习的城市地表覆盖变化检测方法. 武汉：武汉大学

Li X, Pan Y, Gao A, Li L X, Mei S H and Yue S G. 2018. Pansharpening based on joint gaussian guided upsampling //Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain: 7184-7187 [DOI: 10.1109/IGARSS.2018.8518150http://dx.doi.org/10.1109/IGARSS.2018.8518150]

Li X, Xu F, Lyu X, Tong Y, Chen Z Q, Li S Y and Liu D F. 2020b. A remote-sensing image pan-sharpening method based on multi-scale channel attention residual network. IEEE Access, 8:27163-27177 [DOI: 10.1109/ACCESS.2020.2971502http://dx.doi.org/10.1109/ACCESS.2020.2971502]

Li Z Q and Cheng C Q. 2019. A cnn-based pan-sharpening method for integrating panchromatic and multispectral images using landsat 8. Remote Sensing, 11(22): 2606:1-2606:17 [DOI: 10.3390/rs11222606http://dx.doi.org/10.3390/rs11222606]

Liu J M and Liang S L. 2016. Pan-sharpening using a guided filter. International Journal of Remote Sensing, 37(8):1777-1800 [DOI: 10.1080/01431161.2016.1163749http://dx.doi.org/10.1080/01431161.2016.1163749]

Liu L, Wang J, Zhang E L, Li B, Zhu X, Zhang Y Q and Peng J Y. 2020. Shallow–deep convolutional network and spectral-discrimination-based detail injection for multispectral imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13:1772-1783 [DOI: 10.1109/JSTARS.2020.2981695http://dx.doi.org/10.1109/JSTARS.2020.2981695]

Liu P F, Xiao L and Li T. 2018a. A variational pan-sharpening method based on spatial fractional-order geometry and spectral-spatial low-rank priors. IEEE Transactions on Geoscience and Remote Sensing, 56(3):1788-1802 [DOI: 10.1109/TGRS.2017.2768386http://dx.doi.org/10.1109/TGRS.2017.2768386]

Liu X Y, Wang Y H and Liu Q J. 2018b. Psgan: a generative adversarial network for remote sensing image pan-sharpening//Proceedings of the IEEE International Conference on Image Processing. Athens, Greece: 873-877 [DOI: 10.1109/ICIP.2018.8451049http://dx.doi.org/10.1109/ICIP.2018.8451049]

Liu Y, Chen X, Wang Z F, Wang Z J, Ward R K and Wang X S. 2018c. Deep learning for pixel level image fusion: recent advances and future prospects. Information Fusion, 42:158-173 [DOI: 10.1016/j.inffus.2017.10.007http://dx.doi.org/10.1016/j.inffus.2017.10.007]

Lu X C, Zhang J P and Zhang Y. 2017. An improved non-subsampled contourlet transform-based hybrid pan-sharpening algorithm//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Fort Worth, TX, USA: 3393-3396 [DOI: 10.1109/IGARSS.2017.8127726http://dx.doi.org/10.1109/IGARSS.2017.8127726]

Luo D Q, Huang Y Q, Wang S Z, Fang Y P and Jin W B. 2016. Spectral signature analysis and band selection of Macheng rhododendron based on Landsat5 TM. Hubei Agricultural Sciences, 55(19):4991-4994

罗代清, 黄勇奇, 王书珍, 方元平, 金卫斌 . 2016. 基于 Landsat5 TM 的麻城杜鹃花光谱分析与波段选择 . 湖北农业科学, 55(19): 4991-4994 [DOI: 10.14088/j.cnki.issn0439-8114.2016.19.023http://dx.doi.org/10.14088/j.cnki.issn0439-8114.2016.19.023]

Ma J Y, Yu W, Chen C, Liang P W, Guo X J and Jiang J J. 2020. Pan-gan: an unsupervised pan-sharpening method for remote sensing image fusion. Information Fusion, 62:110-120 [DOI: 10.1016/j.inffus.2020.04.006http://dx.doi.org/10.1016/j.inffus.2020.04.006]

Ma L, Liu Y, Zhang X L, Ye Y X, Yin G F and Johnson B A. 2019. Deep learning in remote sensing applications: a meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, 152:166-177 [DOI: 10.1016/j.isprsjprs.2019.04.015http://dx.doi.org/10.1016/j.isprsjprs.2019.04.015]

Masi G, Cozzolino D, Verdoliva L and Scarpa G. 2016. Pansharpening by convolutional neural networks. Remote Sensing, 8(7):594:1-594:22 [DOI: 10.3390/rs8070594http://dx.doi.org/10.3390/rs8070594]

Masi G, Cozzolino D, Verdoliva L and Scarpa G. 2017. CNN-based pansharpening of multi-resolution remote-sensing images// Proceedings of the Joint Urban Remote Sensing Event. Dubai, United Arab Emirates: 1-4 [DOI: 10.1109/JURSE.2017.7924534http://dx.doi.org/10.1109/JURSE.2017.7924534]

Mohammadzadeh A, Tavakoli A and Valadan Z M. 2006. Road extraction based on fuzzy logic and mathematical morphology from pan-sharpened ikonos images. The photogrammetric record, 21(113):44-60 [DOI: 10.1111/j.1477-9730.2006.00353.xhttp://dx.doi.org/10.1111/j.1477-9730.2006.00353.x]

Otazu X, González-Audícana M, Fors O and Nunez J. 2005. Introduction of sensor spectral response into image fusion methods: application to wavelet-based methods. IEEE Transactions on Geoscience and Remote Sensing, 43(10):2376-2385 [DOI: 10.1109/TGRS.2005.856106http://dx.doi.org/10.1109/TGRS.2005.856106]

Ozcelik F, Alganci U, Sertel E and Unal G. 2020. Rethinking cnn-based pansharpening: guided colorization of panchromatic images via gans. IEEE Transactions on Geoscience and Remote Sensing, 59(4):3486-3501 [DOI: 10.1109/TGRS.2020.3010441http://dx.doi.org/10.1109/TGRS.2020.3010441]

Pálsson F, Sveinsson J R, Benediktsson J A and Aanaes H. 2012. Classification of pansharpened Urban satellite images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 5(1):281-297 [DOI: 10.1109/JSTARS.2011.2176467http://dx.doi.org/10.1109/JSTARS.2011.2176467]

Palsson F, Sveinsson J R, Ulfarsson M O and Benediktsson J A. 2015. Model based pansharpening method based on tv and mtf deblurring//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Milan, Italy: 33-36 [DOI: 10.1109/IGARSS.2015.7325690http://dx.doi.org/10.1109/IGARSS.2015.7325690]

Rao Y Z, He L and Zhu J W. 2017. A residual convolutional neural network for pan-shaprening//2017 International Workshop on Remote Sensing with Intelligent Processing. 1-4 [DOI: 10.1109/RSIP.2017.7958807http://dx.doi.org/10.1109/RSIP.2017.7958807]

Ripley B D. 1996. Pattern Recognition and Neural Networks. Cambridge: Cambridge University Press

Scarpa G, Vitale S and Cozzolino D. 2018. Target-adaptive cnn-based pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 56(9):5443-5457 [DOI: 10.1109/TGRS.2018.2817393http://dx.doi.org/10.1109/TGRS.2018.2817393]

Shah V P, Younan N H and King R L. 2008. An efficient pan-sharpening method via a combined adaptive pca approach and contourlets. IEEE Transactions on Geoscience and Remote Sensing, 46(5):1323-1335 [DOI: 10.1109/TGRS.2008.916211http://dx.doi.org/10.1109/TGRS.2008.916211]

Shahdoosti H R and Javaheri N. 2017. Pansharpening of clustered ms and pan images considering mixed pixels. IEEE Geoscience and Remote Sensing Letters, 14(6):826-830 [DOI: 10.1109/LGRS.2017.2682122http://dx.doi.org/10.1109/LGRS.2017.2682122]

Shao Z F and Cai J J. 2018. Remote sensing image fusion with deep convolutional neural network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(5):1656-1669 [DOI: 10.1109/JSTARS.2018.2805923http://dx.doi.org/10.1109/JSTARS.2018.2805923]

Shao Z M, Lu Z X, Ran M S, Fang L Y, Zhou J L and Zhang Y. 2020. Residual encoder-decoder conditional generative adversarial network for pansharpening. IEEE Geoscience and Remote Sensing Letters, 17(9):1573-1577 [DOI: 10.1109/LGRS.2019.2949745http://dx.doi.org/10.1109/LGRS.2019.2949745]

Shi X J, Chen Z R, Wang H, Yeung D Y, Wong W K and Woo W C. 2015. Convolutional lstm network: a machine learning approach for precipitation nowcasting//Proceedings of the International Conference on Neural Information Processing Systems (NIPS). Montreal, Quebec, Canada: 802-810

Souza C, Firestone L, Silva L M and Roberts D. 2003. Mapping forest degradation in the eastern amazon from spot 4 through spectral mixture models. Remote Sensing of Environment, 87(4):494-506 [DOI: 10.1016/j.rse.2002.08.002http://dx.doi.org/10.1016/j.rse.2002.08.002]

Tsagkatakis G, Aidini A, Fotiadou K, Giannopoulos M, Pentari A and Tsakalides P. 2019. Survey of deep-learning approaches for remote sensing observation enhancement. Sensors, 19(18):3929:1-3929:39 [DOI: 10.3390/s19183929http://dx.doi.org/10.3390/s19183929]

Tu T M, Su S C, Shyu H C and Huang P S. 2001. A new look at IHS-like image fusion methods. Information Fusion, 2(3):177-186 [DOI: 10.1016/S1566-2535(01)00036-7http://dx.doi.org/10.1016/S1566-2535(01)00036-7]

Vinothini D S and Bama B S. 2019. Residual dense network for pan-sharpening satellite data. IEEE Sensors Journal, 19(24):12279-12285 [DOI: 10.1109/JSEN.2019.2939844http://dx.doi.org/10.1109/JSEN.2019.2939844]

Vitale S, Ferraioli G and Scarpa G. 2018. A CNN-based model for pansharpening of worldview-3 images//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain: 5108-5111 [DOI: 10.1109/IGARSS.2018.8519202http://dx.doi.org/10.1109/IGARSS.2018.8519202]

Vitale S. 2019. A cnn-based pansharpening method with perceptual loss//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan: 3105-3108 [DOI: 10.1109/IGARSS.2019.8900390http://dx.doi.org/10.1109/IGARSS.2019.8900390]

Vivone G, Alparone L, Chanussot J, Mura M D, Garzelli A, Licciardi G A, Restaino R and Wald L. 2015. A critical comparison among pansharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing, 53(5):2565-2586 [DOI: 10.1109/TGRS.2014.2361734http://dx.doi.org/10.1109/TGRS.2014.2361734]

Wald L, Ranchin T and Mangolini M. 1997. Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images. Photogrammetric Engineering & Remote Sensing, 63(6):691-699

Wald L. 2000. Quality of high resolution synthesised images: Is there a simple criterion? // Proceedings of the Fusion of Earth data: merging point measurements, raster maps and remotely sensed images. Sophia Antipolis, France: 99-103

Wang D, Bai Y P and Li Y. 2020. Multispectral pan-sharpening via dual-channel convolutional network with convolutional lstm based hierarchical spatial-spectral feature fusion. ArXiv Preprint ArXiv: 2007.10060

Wang F，Guo Q and Ge X Q. 2021. Pan-sharpening by deep recursive residual network. Journal of Remote Sensing, 25(6)：1244-1256

王芬, 郭擎, 葛小青. 深度递归残差网络的遥感图像空谱融合. 2021. 遥感学报, 25(6): 1244-1256 [DOI：10.11834/jrs.20219250http://dx.doi.org/10.11834/jrs.20219250]

Wei Y C and Yuan Q Q. 2017. Deep residual learning for remote sensed imagery pansharpening//Proceedings of the International Workshop on Remote Sensing with Intelligent Processing. Shanghai, China: 1-4 [DOI: 10.1109/RSIP.2017.7958794http://dx.doi.org/10.1109/RSIP.2017.7958794]

Wei Y C, Yuan Q Q, Shen H F and Zhang L P. 2017. Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geoscience and Remote Sensing Letters, 14(10):1795-1799 [DOI: 10.1109/LGRS.2017.2736020http://dx.doi.org/10.1109/LGRS.2017.2736020]

Xiang Z K, Xiao L, Liu P F and Zhang Y F. 2019. A multi-scale densely deep learning method for pansharpening//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan: 2786-2789 [DOI: 10.1109/IGARSS.2019.8898095http://dx.doi.org/10.1109/IGARSS.2019.8898095]

Xing Y H, Wang M, Yang S Y and Jiao L C. 2018a. Pan-sharpening via deep metric learning. ISPRS Journal of Photogrammetry and Remote Sensing, 145:165-183 [DOI: 10.1016/j.isprsjprs.2018.01.016http://dx.doi.org/10.1016/j.isprsjprs.2018.01.016]

Xing Y H, Wang M, Yang S Y and Zhang K. 2018b. Pansharpening with multiscale geometric support tensor machine. IEEE Transactions on Geoscience and Remote Sensing, 56(5):2503-2517 [DOI: 10.1109/TGRS.2017.2742002http://dx.doi.org/10.1109/TGRS.2017.2742002]

Yang J F, Fu X Y, Hu Y W, Huang Y, Ding X H and Paisley J. 2017. Pannet: a deep network architecture for pan-sharpening// Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: 1753-1761 [DOI: 10.1109/ICCV.2017.193http://dx.doi.org/10.1109/ICCV.2017.193]

Yang S Y, Zhang K and Wang M. 2018a. Learning low-rank decomposition for pansharpening with spatial-spectral offsets. IEEE Transactions on Neural Networks & Learning Systems, 29(8):3647-3657 [DOI: 10.1109/TNNLS.2017.2736011http://dx.doi.org/10.1109/TNNLS.2017.2736011]

Yang Y, Li L Y, Huang S Y, Zhang Y M and Lu H Y. 2020. Remote sensing image fusion with convolutional sparse representation based on adaptive dictionary learning. Journal of Signal Processing, 36(1):125-138

杨勇, 李露奕, 黄淑英, 张迎梅, 卢航远. 2020. 自适应字典学习的卷积稀疏表示遥感图像融合. 信号处理, 36(1):125-138 [DOI: 10.16798/j.issn.1003-0530.2020.01.016http://dx.doi.org/10.16798/j.issn.1003-0530.2020.01.016]

Yang Y, Tu W, Huang S Y and Lu H Y. 2020a. Pcdrn: progressive cascade deep residual network for pansharpening. Remote Sensing, 12(4):676:1-676:20 [DOI:10.3390/rs12040676http://dx.doi.org/10.3390/rs12040676]

Yang Y, Wu L, Huang S Y, Sun J C, Wan W G and Wu J H. 2018b. Compensation details-based injection model for remote sensing image fusion. IEEE Geoscience and Remote Sensing Letters, 15(5):734-738 [DOI: 10.1109/LGRS.2018.2810219http://dx.doi.org/10.1109/LGRS.2018.2810219]

Yang Y, Wu L, Huang S Y, Tang Y J and Wan W G. 2018c. Pansharpening for multiband images with adaptive spectral–intensity modulation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(9):3196-3208 [DOI: 10.1109/JSTARS.2018.2849011http://dx.doi.org/10.1109/JSTARS.2018.2849011]

Yang Y, Wu L, Huang S Y, Wan W G, Tu W and Lu H Y. 2020b. Multiband remote sensing image pansharpening based on dual-injection model. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13:1888-1904 [DOI: 10.1109/JSTARS.2020.2981975http://dx.doi.org/10.1109/JSTARS.2020.2981975]

Yao W, Zeng Z G, Lian C and Tang H M. 2018. Pixel-wise regression using u-net and its application on pansharpening. Neurocomputing, 312:364-371 [DOI: 10.1016/j.neucom.2018.05.103http://dx.doi.org/10.1016/j.neucom.2018.05.103]

Ye F J, Li X F and Zhang X L. 2018. Fusioncnn: a remote sensing image fusion algorithm based on deep convolutional neural networks. Multimedia Tools and Applications, 78(11):14683-14703 [DOI: 10.1007/s11042-018-6850-3http://dx.doi.org/10.1007/s11042-018-6850-3]

Ye F, Guo Y C and Zhuang P X. 2019. Pan-sharpening via a gradient-based deep network prior. Signal Processing: Image Communication, 74:322-331 [DOI: 10.1016/j.image.2019.03.004http://dx.doi.org/10.1016/j.image.2019.03.004]

Yuan Q Q, Wei Y C, Meng X C, Shen H F and Zhang L P. 2018. A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3):978-989 [DOI: 10.1109/JSTARS.2018.2794888http://dx.doi.org/10.1109/JSTARS.2018.2794888]

Zeiler M D and Fergus R. 2014. Visualizing and understanding convolutional networks//Proceedings of the Conference on European Conference on Computer Vision (ECCV). Zurich, Switzerland: 818-833 [DOI:10.1007/978-3-319-10590-1_53http://dx.doi.org/10.1007/978-3-319-10590-1_53]

Zhang H Q, Liu X Y, Yang S and Li Y. 2017. Retrieval of remote sensing images based on semisupervised deep learning. Journal of Remote Sensing, 21(3): 406-414

张洪群,刘雪莹,杨森,李宇. 2017. 深度学习的半监督遥感图像检索. 遥感学报, 21(3):406-414 [DOI: 10.11834/jrs.20176105http://dx.doi.org/10.11834/jrs.20176105]

Zhang L B, Zhang J, Lyu X R and Ma J. 2019a. A new pansharpening method using objectness based saliency analysis and saliency guided deep residual network//Proceedings of the IEEE International Conference on Image Processing. Taipei, China: 4529-4533 [DOI: 10.1109/ICIP.2019.8803477http://dx.doi.org/10.1109/ICIP.2019.8803477]

Zhang L P, Li W S, Zhang C and Lei D J. 2020. A generative adversarial network with structural enhancement and spectral supplement for pan-sharpening. Neural Computing and Applications, 32(24):18347-18359 [DOI: 10.1007/s00521-020-04973-whttp://dx.doi.org/10.1007/s00521-020-04973-w]

Zhang Y J, Liu C, Sun M W and Ou Y J. 2019b. Pan-sharpening using an efficient bidirectional pyramid network. IEEE Transactions on Geoscience and Remote Sensing, 57(8):5549-5563 [DOI: 10.1109/TGRS.2019.2900419http://dx.doi.org/10.1109/TGRS.2019.2900419]

Zhang Y L, Tian Y P, Kong Y, Zhong B E and Fu Y. 2018. Residual dense network for image super-resolution//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, UT, USA: 2472-2481 [DOI: 10.1109/CVPR.2018.00262http://dx.doi.org/10.1109/CVPR.2018.00262]

Zhang Y T, Li X H, Zhou JL. 2019c. Sftgan: a generative adversarial network for pan-sharpening equipped with spatial feature transform layers. Journal of Applied Remote Sensing, 13(2): 026507:1-026507:16 [DOI: 10.1117/1.JRS.13.026507http://dx.doi.org/10.1117/1.JRS.13.026507]

Zhong J Y, Yang B, Huang G Y, Zhong Fei and Chen Z Z. 2016. Remote sensing image fusion with convolutional neural network. Sensing and Imaging, 17(1):6:1-6:16 [DOI: 10.1007/s11220-016-0135-6http://dx.doi.org/10.1007/s11220-016-0135-6]

Zhou M, Yan K Y, Huang J, Yang Z H, Fu X Y and Zhao F. 2022. Mutual information-driven pan-sharpening//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, Louisiana, USA: 1788-1798 [DOI: 10.1109/CVPR52688.2022.00184http://dx.doi.org/10.1109/CVPR52688.2022.00184]

文章被引用时，请邮件提醒。

提交

深度递归残差网络的遥感图像空谱融合

生成式知识迁移的SAR舰船检测

MtSCCD：面向深度学习的土地利用场景分类与变化检测数据集

光学信号Token引导的异源遥感变化检测网络