Deep-learning approaches for pixel-level pansharpening

YANG Yong; SU Zhao; HUANG Shuying; WAN Weiguo; TU Wei; LU Hangyuan

doi:10.11834/jrs.20211325

Review | Views : 0 下载量: 3824 CSCD: 6

R-PDF
PDF
Export
Share
Collection
Album

Deep-learning approaches for pixel-level pansharpening
Vol. 26, Issue 12, Pages: 2411-2432(2022)
Received：21 May 2021，

Published：07 December 2022
DOI： 10.11834/jrs.20211325
稿件说明：

移动端阅览

杨勇，苏昭，黄淑英，万伟国，涂伟，卢航远.2022.基于深度学习的像素级全色图像锐化研究综述.遥感学报，26（12）： 2411-2432 DOI： 10.11834/jrs.20211325.

Yang Y， Su Z， Huang S Y， Wan W G， Tu W and Lu H Y. 2022. Survey of deep-learning approaches for pixel-level pansharpening. National Remote Sensing Bulletin， 26（12）：2411-2432 DOI： 10.11834/jrs.20211325.

摘要

全色图像锐化是遥感数据处理领域的一个基础性问题，在地物分类、目标识别等方面具有重要的研究意义和应用价值。近年来，深度学习在自然语言处理、计算机视觉等领域取得了巨大进展，也推动了像素级全色图像锐化技术的发展。本文提出从经典方式和协同方式两个方面对深度学习在全色图像锐化中的研究进行系统的综述，并在此基础上进行前景展望。首先，给出全色图像锐化常用的数据集和全色图像锐化的质量评价指标；接着，从经典方式与协同方式两个方面对基于深度学习的全色图像锐化最新研究成果进行分门别类的介绍，并进行算法性能的对比、分析和归纳；然后，对全色图像锐化的3个主要应用领域如地物分类、目标识别和地表变化检测进行分析；最后，本文探讨了基于深度学习的全色图像锐化的5个未来研究方向。

Abstract

Pansharpening is a fundamental problem in the field of remote sensing data processing. It has important research significance and application value in ground object classification and ground surface change detection. In recent years

Deep Learning (DL) has made great progress in natural language processing

computer vision

etc. and has promoted the development of pixel-level pansharpening technology. This work presents a systematic review of the research of DL in pansharpening from two aspects (classical and collaborative approaches) and makes a prospect on this basis. First

the common datasets of pansharpening and the objective evaluation indexes of pansharpening

including reference and non-reference quality evaluation indexes

are provided. Second

the latest research results of DL-based pansharpening are introduced in two different categories from the classical and collaborative methods

and the performance of their algorithms is compared

analyzed

and summarized. The classical methods mainly include AE-based pansharpening

CNN-based pansharpening

DRN-based pansharpening

and GAN-based pansharpening methods. Meanwhile

the collaborative methods mainly include DL+CS-based pansharpening

DL+MRA-based pansharpening

DL+MB-based pansharpening

DL+injection model-based pansharpening

CNN+DRN-based pansharpening

and RNN+CNN-based pansharpening methods. In the comparative analysis of the classical and collaborative methods

the common point is that the DL technology can automatically learn the advantages of complex data features and extract the feature information of the MS or PAN image (i.e.

the information that needs to be retained in the HRMS fusion image). The difference is that the structure of the classical mode is more concise

while that of the collaborative mode is more complex because it is the combination of multiple methods or frameworks. In addition

most early DL-based pansharpening methods utilized the powerful data fitting ability of the DL model and seldom paid attention to the field of pansharpening problems. With the gradual deepening of research

such as using DL methods combined with traditional pansharpening methods

this designed fusion model considers spectral and spatial distortions. Accordingly

the DL methods can further enhance the pansharpening effect. Thirdly

the three main application fields of pansharpening are analyzed

such as object classification

target recognition

and surface change detection. Finally

this work discusses the future research direction of DL-based pansharpening in combination with remote sensing knowledge to fully tap the potential of DL to obtain fused images with richer details and more natural spectra. For example

for the evaluation of pansharpening application

the performance of pansharpening in a certain application is related not only to the high quality of fusion image but also to the knowledge of a specific application field. Accordingly

the application-oriented pansharpening evaluation algorithms will be the focus of future study. Furthermore

DL-based pansharpening needs to train a large number of network parameters

resulting in a longer training time for the pansharpening model. The lightweight depth model has a smaller network capacity

lower time complexity

and lower hardware requirements. Therefore

constructing a lightweight pansharpening model is a promising future direction.

关键词

Keywords

references

Aguilar M A , Saldaña M M and Aguilar F . 2013 . GeoEye-1 and worldview-2 pan-sharpened imagery for object-based classification in urban environments . International Journal of Remote Sensing , 34 ( 7 ): 2583 - 2606 [ DOI: 10.1080/01431161.2012.747018 http://dx.doi.org/10.1080/01431161.2012.747018 ]

Aiazzi B , Baronti S and Selva M . 2007 . Improving component substitution pansharpening through multivariate regression of ms+pan data . IEEE Transactions on Geoscience and Remote Sensing , 45 ( 10 ): 3230 - 3239 [ DOI: 10.1109/tgrs.2007.901007 http://dx.doi.org/10.1109/tgrs.2007.901007 ]

Alparone L , Aiazzi B , Baronti S and Garzelli A . 2015 . Remote sensing image fusion . London, UK : CRC Press

Alparone L , Wald L , Chanussot J , Thomas C , Gamba P and Bruce L M . 2007 . Comparison of pansharpening algorithms: outcome of the 2006 grs-s data-fusion contest . IEEE Transactions on Geoscience and Remote Sensing , 45 ( 10 ): 3012 - 3021 [ DOI: 10.1109/TGRS.2007.904923 http://dx.doi.org/10.1109/TGRS.2007.904923 ]

Azarang A and Ghassemian H . 2017 . A new pansharpening method using multi resolution analysis framework and deep neural networks . // Proceedings of the International Conference on Pattern Recognition and Image Analysis . Shahrekord, Iran , 1 - 6 [ DOI: 10.1109/pria.2017.7983017 http://dx.doi.org/10.1109/pria.2017.7983017 ]

Azarang A , Manoochehri H E and Kehtarnavaz N . 2019 . Convolutional autoencoder-based multispectral image fusion . IEEE Access , 7 : 35673 - 35683 [ DOI: 10.1109/ACCESS.2019.2905511 http://dx.doi.org/10.1109/ACCESS.2019.2905511 ]

Benzenati T , Kessentini Y , Kallel A and Hallabia H . 2020 . Generalized laplacian pyramid pan-sharpening gain injection prediction based on cnn . IEEE Geoscience and Remote Sensing Letters , 17 ( 4 ): 651 - 655 [ DOI: 10.1109/LGRS.2019.2928181 http://dx.doi.org/10.1109/LGRS.2019.2928181 ]

Bovolo F , Bruzzone L , Capobianco L , Garzelli A , Marchesi S and Nencini F . 2010 . Analysis of the effects of pansharpening in change detection on VHR images . IEEE Geoscience and Remote Sensing Letters , 7 ( 1 ): 53 - 57 [ DOI: 10.1109/LGRS.2009.2029248 http://dx.doi.org/10.1109/LGRS.2009.2029248 ]

Bowman S R , Angeli G , Potts C and Manning C D . 2015 . A large annotated corpus for learning　natural language　inference // Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Lisbon, Portugal : 632 - 642 [ DOI: 10.18653/v1/D15-1075 http://dx.doi.org/10.18653/v1/D15-1075 ]

Cai W T , Xu Y , Wu Z B , Liu H Y , Qian L and Wei Z H . 2018 . Pan-sharpening based on multilevel coupled deep network . // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Valencia, Spain: 7046 - 7049 [ DOI: 10.1109/IGARSS.2018.8518121 http://dx.doi.org/10.1109/IGARSS.2018.8518121 ]

Carper W J , Lillesand T M and Kiefer P W . 1990 . The use of intensity-hue-saturation transformations for merging spot panchromatic and multispectral image data . Photogrammetric Engineering and Remote Sensing , 56 ( 4 ): 459 – 467

Chen C , Fu J Q , Gai Y Y , Li J , Chen L , Mantravadi V S and Tan A H . 2018 . Damaged bridges over water: using high-spatial-resolution remote-sensing images for recognition, detection, and assessment . IEEE Geoscience and Remote Sensing Magazine , 6 ( 3 ): 69 - 85 [ DOI: 10.1109/MGRS.2018.2852804 http://dx.doi.org/10.1109/MGRS.2018.2852804 ]

Chen C , Fu J Q , Lu N , Chu YL , Hu J C , Guo B Y and Zhao X . 2019 . Knowledge-based identification and damage detection of bridges spanning water via high-spatial-resolution optical remotely sensed imagery . Journal of the Indian Society of Remote Sensing , 47 ( 12 ): 1999 - 2008 [ DOI: 10.1007/s12524-019-01036-z http://dx.doi.org/10.1007/s12524-019-01036-z ]

Chen F J , Zhu F , Wu Q X , Hao Y M , Wang E D and Cui Y G . 2021 . A survey about image generation with generative adversarial nets . Chinese Journal of Computers , 44 ( 2 ): 347 - 369

陈佛计 , 朱枫 , 吴清潇 , 郝颖明 , 王恩德 , 崔芸阁 . 2021 . 生成对抗网络及其在图像生成中的应用研究综述 . 计算机学报 , 44 ( 2 ): 347 - 369 [ DOI: 10.11897/SP.J.1016.2021.00347 http://dx.doi.org/10.11897/SP.J.1016.2021.00347 ]

Chen J F , Pan Y and Chen Y . 2020 . Remote sensing image fusion based on Bayesian GAN. ArXiv Preprint ArXiv: 2009 . 09465

Chen M M , Guo Q , Liu M L and Li A . 2021 . Pan-sharpening by residual network with dense convolution for remote sensing images . Journal of Remote Sensing ， 25 （ 6 ）： 1270 - 1283

陈毛毛 , 郭擎 , 刘明亮 , 李安 . 2021 . 密集卷积残差网络的遥感图像融合 . 遥感学报 , 25 ( 6 ): 1270 - 1283 [ DOI： 10.11834/jrs.20219411 http://dx.doi.org/10.11834/jrs.20219411 ]

Da Cunha A L , Zhou J P and Do M N . 2006 . The nonsubsampled contourlet transform: theory, design, and applications . IEEE Transactions on Image Processing , 15 ( 10 ): 3089 - 3101 [ DOI: 10.1109/TIP.2006.877507 http://dx.doi.org/10.1109/TIP.2006.877507 ]

Deng L J , Vivone G , Jin C and Chanussot J . 2020 . Detail injection-based deep convolutional neural networks for pansharpening . IEEE Transactions on Geoscience and Remote Sensing , 59 ( 8 ): 6995 - 7010 [ DOI: 10.1109/TGRS.2020.3031366 http://dx.doi.org/10.1109/TGRS.2020.3031366 ]

Dong C , Loy C C , He K M and Tang X O . 2016 . Image super-resolution using deep convolutional networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 38 ( 2 ): 295 - 307 [ DOI: 10.1109/TPAMI.2015.2439281 http://dx.doi.org/10.1109/TPAMI.2015.2439281 ]

Eghbalian S and Ghassemian H . 2018 . Multi spectral image fusion by deep convolutional neural network and new spectral loss function . International Journal of Remote Sensing , 39 ( 12 ): 3983 - 4002 [ DOI: 10.1080/01431161.2018.1452074 http://dx.doi.org/10.1080/01431161.2018.1452074 ]

Garzelli A , Capobianco L and Nencini F . 2009 . On the effects of pan-sharpening to target detection // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Cape Town, South Africa : II-136‒II-139 [ DOI: 10.1109/IGARSS.2009.5418022 http://dx.doi.org/10.1109/IGARSS.2009.5418022 ]

Garzelli A , Nencini F and Capobianco L . 2008 . Optimal mmse pansharpening of very high resolution multispectral images . IEEE Transactions on Geoscience and Remote Sensing , 46 ( 1 ): 228 - 236 [ DOI: 10.1109/TGRS.2007.907604 http://dx.doi.org/10.1109/TGRS.2007.907604 ]

Gianinetto M , Rusmini M , Candiani G , Via G D , Frassy F , Maianti P , Marchesi A , Nodari FR and Dini L . 2017 . Hierarchical classification of complex landscape with VHR pan-sharpened satellite data and obia techniques . European Journal of Remote Sensing , 47 ( 1 ): 229 - 250 [ DOI: 10.5721/EuJRS20144715 http://dx.doi.org/10.5721/EuJRS20144715 ]

Goodfellow I J , Pouget-Abadie J , Mirza M , Xu B , Warde-Farley D , Ozair S , Courville A C and Bengio Y . 2014 . Generative adversarial nets // Proceedings of the Advances in Neural Information Processing Systems (NIPS) . Montreal, Quebec, Canada: 2672 - 2680

Guo Y C , Ye F and Gong H . 2019 . Learning an efficient convolution neural network for pansharpening . Algorithms , 12 ( 1 ): 16 : 1 - 16 : 14 [ DOI: 10.3390/a12010016 http://dx.doi.org/10.3390/a12010016 ]

He G Q , Xing S Y , Xia Z Q , Huang Q Q and Fan J P . 2018 . Panchromatic and multi-spectral image fusion for new satellites based on multi-channel deep model . Machine Vision and Applications , 29 ( 6 ): 933 - 946 [ DOI: 10.1007/s00138-018-0964-5 http://dx.doi.org/10.1007/s00138-018-0964-5 ]

He K M and Sun J . 2015 . Convolutional neural networks at constrained time cost // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Boston, MA, USA : 5353 - 5360 [ DOI: 10.1109/CVPR.2015.7299173 http://dx.doi.org/10.1109/CVPR.2015.7299173 ]

He K M , Zhang X Y , Ren S Q and Sun J . 2016 . Deep residual learning for image recognition . // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas, NV, USA : 770 - 778 [ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]

He L , Rao Y Z , Li J , Chanussot J , Plaza A , Zhu J W and Li B . 2019 . Pansharpening via detail injection based convolutional neural networks . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 12 ( 4 ): 1188 - 1204 [ DOI: 10.1109/JSTARS.2019.2898574 http://dx.doi.org/10.1109/JSTARS.2019.2898574 ]

He L , Xi D H , Li J and Zhu J W . 2020 . A spectral-aware convolutional neural network for pansharpening . Applied Sciences , 10 ( 17 ): 5809 : 1 - 5809 : 18 [ DOI: 10.3390/app10175809 http://dx.doi.org/10.3390/app10175809 ]

Hinton G E and Salakhutdinov R R . 2006 . Reducing the dimensionality of data with neural networks . Science , 313 : 504 - 507 [ DOI: 10.1126/science.1127647 http://dx.doi.org/10.1126/science.1127647 ]

Hinton G E , Osindero S and Teh Y W . 2006 . A fast learning algorithm for deep belief nets . Neural Computation , 18 ( 7 ): 1527 - 1554 [ DOI: 10.1162/neco.2006.18.7.1527 http://dx.doi.org/10.1162/neco.2006.18.7.1527 ]

Hochreiter S and Schmidhuber J . 1997 . Long short-term memory . Neural Computation , 9 ( 8 ): 1735 - 1780 [ DOI: 10.1162/neco.1997.9.8.1735 http://dx.doi.org/10.1162/neco.1997.9.8.1735 ]

Hu J , He Z and Wu J M . 2019 . Deep self-learning network for adaptive pansharpening . Remote Sensing , 11 ( 20 ): 2395 : 1 - 2395 : 22 [ DOI: 10.3390/rs11202395 http://dx.doi.org/10.3390/rs11202395 ]

Huang G , Liu Z , Maaten L V D , and Weinberger K Q . 2017 . Densely connected convolutional networks // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Honolulu, HI, USA : 2261 - 2269 [ DOI: 10.1109/CVPR.2017.243 http://dx.doi.org/10.1109/CVPR.2017.243 ]

Huang W , Fei X , Yin J R and Liu Y . 2018 . A multi-direction subbands and deep neural networks based pan-sharpening method // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Valencia, Spain: 5139 - 5142 [ DOI: 10.1109/IGARSS.2018.8517817 http://dx.doi.org/10.1109/IGARSS.2018.8517817 ]

Huang W , Xiao L , Wei Z H , Liu H Y and Tang S Z . 2015 . A new pan-sharpening method with deep neural networks . IEEE Geoscience and Remote Sensing Letters , 12 ( 5 ): 1037 - 1041 [ DOI: 10.1109/LGRS.2014.2376034 http://dx.doi.org/10.1109/LGRS.2014.2376034 ]

Javan F D , Samadzadegan F , Mehravar S , Toosi A , Khatami R and Stein A . 2021 . A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery . ISPRS Journal of Photogrammetry and Remote Sensing , 171 : 101 - 117 [ DOI: 10.1016/j.isprsjprs.2020.11.001 http://dx.doi.org/10.1016/j.isprsjprs.2020.11.001 ]

Jiang M H , Li J , Yuan Q Q , Shen H F , Liu X X and Xu M M . 2019 . Differential information residual convolutional neural network for pansharpening . // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Yokohama, Japan: 4865 - 4868 [ DOI: 10.1109/IGARSS.2019.8900270 http://dx.doi.org/10.1109/IGARSS.2019.8900270 ]

Kang X D , Li S T and Benediktsson J A . 2014 . Pansharpening with matting model . IEEE Transactions on Geoscience and Remote Sensing , 52 ( 8 ): 5088 - 5099 [ DOI: 10.1109/TGRS.2013.2286827 http://dx.doi.org/10.1109/TGRS.2013.2286827 ]

Khan M M , Chanussot J , Condat L and Montanvert A . 2008 . Indusion: fusion of multispectral and panchromatic images using the induction scaling technique . IEEE Geoscience and Remote Sensing Letters , 5 ( 1 ): 98 - 102 [ DOI: 10.1109/LGRS.2007.909934 http://dx.doi.org/10.1109/LGRS.2007.909934 ]

Kim J , Lee J K and Lee K M . 2016 . Accurate image super-resolution using very deep convolutional networks . // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas, NV, USA : 1646 - 1654 [ DOI: 10.1109/CVPR.2016.182 http://dx.doi.org/10.1109/CVPR.2016.182 ]

Laben C A and Brower B V . 2000 . Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening . U . S ., No. 6011875

Lee J , Seo S and Kim M . 2021 . Sipsa-net: shift-invariant pan sharpening with moving object alignment for satellite imagery // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Online : 10161 - 10169 [ DOI: 10.1109/CVPR46437.2021.01003 http://dx.doi.org/10.1109/CVPR46437.2021.01003 ]

Li H , Liu F , Yang S Y and Zhang K . 2016 . Remote sensing image fusion based on deep support value learning networks . Chinese Journal of Computers , 39 ( 8 ): 1583 - 1596

李红 , 刘芳 , 杨淑媛 , 张凯 . 2016 . 基于深度支撑值学习网络的遥感图像融合 . 计算机学报 , 39 ( 8 ): 1583 - 1596 [ DOI: 10.11897/SP.J.1016.2016.01583 http://dx.doi.org/10.11897/SP.J.1016.2016.01583 ]

Li J , Li Y F , He L , Chen J and Plaza A . 2020a . Spatio-temporal fusion for remote sensing data: an overview and new benchmark . Science China Information Sciences , 63 ( 4 ) : 7 - 23 [ DOI: 10.1007/s11432-019-2785-y http://dx.doi.org/10.1007/s11432-019-2785-y ]

Li N , Huang N and Xiao L . 2017 . Pan-sharpening via residual deep learning // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Fort Worth, TX, USA: 5133 - 5136 [ DOI: 10.1109/IGARSS.2017.8128158 http://dx.doi.org/10.1109/IGARSS.2017.8128158 ]

Li S T ， Li C Y and Kang X D . 2021 . Development status and future prospects of multi-source remote sensing image fusion . Journal of Remote Sensing , 25 ( 1 )： 148 - 166

李树涛 , 李聪妤 , 康旭东 . 2021 . 多源遥感图像融合发展现状与未来展望 . 遥感学报 , 25 ( 1 ): 148 - 166 [ DOI： 10.11834/jrs.20210259 http://dx.doi.org/10.11834/jrs.20210259 ]

Li S . 2018 . Detection Method of Urban Land Cover Change Based on Combining Domain Knowledge and Deep Learning . Wuhan : Wuhan University

李胜 . 2018 . 联合领域知识和深度学习的城市地表覆盖变化检测方法 . 武汉：武汉大学

Li X , Pan Y , Gao A , Li L X , Mei S H and Yue S G . 2018 . Pansharpening based on joint gaussian guided upsampling // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Valencia, Spain: 7184 - 7187 [ DOI: 10.1109/IGARSS.2018.8518150 http://dx.doi.org/10.1109/IGARSS.2018.8518150 ]

Li X , Xu F , Lyu X , Tong Y , Chen Z Q , Li S Y and Liu D F . 2020b . A remote-sensing image pan-sharpening method based on multi-scale channel attention residual network . IEEE Access , 8 : 27163 - 27177 [ DOI: 10.1109/ACCESS.2020.2971502 http://dx.doi.org/10.1109/ACCESS.2020.2971502 ]

Li Z Q and Cheng C Q . 2019 . A cnn-based pan-sharpening method for integrating panchromatic and multispectral images using landsat 8 . Remote Sensing , 11 ( 22 ): 2606 : 1 - 2606 : 17 [ DOI: 10.3390/rs11222606 http://dx.doi.org/10.3390/rs11222606 ]

Liu J M and Liang S L . 2016 . Pan-sharpening using a guided filter . International Journal of Remote Sensing , 37 ( 8 ): 1777 - 1800 [ DOI: 10.1080/01431161.2016.1163749 http://dx.doi.org/10.1080/01431161.2016.1163749 ]

Liu L , Wang J , Zhang E L , Li B , Zhu X , Zhang Y Q and Peng J Y . 2020 . Shallow–deep convolutional network and spectral-discrimination-based detail injection for multispectral imagery pan-sharpening . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13 : 1772 - 1783 [ DOI: 10.1109/JSTARS.2020.2981695 http://dx.doi.org/10.1109/JSTARS.2020.2981695 ]

Liu P F , Xiao L and Li T . 2018a . A variational pan-sharpening method based on spatial fractional-order geometry and spectral-spatial low-rank priors . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 3 ): 1788 - 1802 [ DOI: 10.1109/TGRS.2017.2768386 http://dx.doi.org/10.1109/TGRS.2017.2768386 ]

Liu X Y , Wang Y H and Liu Q J . 2018b . Psgan: a generative adversarial network for remote sensing image pan-sharpening // Proceedings of the IEEE International Conference on Image Processing . Athens, Greece : 873 - 877 [ DOI: 10.1109/ICIP.2018.8451049 http://dx.doi.org/10.1109/ICIP.2018.8451049 ]

Liu Y , Chen X , Wang Z F , Wang Z J , Ward R K and Wang X S . 2018c . Deep learning for pixel level image fusion: recent advances and future prospects . Information Fusion , 42 : 158 - 173 [ DOI: 10.1016/j.inffus.2017.10.007 http://dx.doi.org/10.1016/j.inffus.2017.10.007 ]

Lu X C , Zhang J P and Zhang Y . 2017 . An improved non-subsampled contourlet transform-based hybrid pan-sharpening algorithm // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Fort Worth, TX, USA: 3393 - 3396 [ DOI: 10.1109/IGARSS.2017.8127726 http://dx.doi.org/10.1109/IGARSS.2017.8127726 ]

Luo D Q , Huang Y Q , Wang S Z , Fang Y P and Jin W B . 2016 . Spectral signature analysis and band selection of Macheng rhododendron based on Landsat5 TM . Hubei Agricultural Sciences , 55 ( 19 ): 4991 - 4994

罗代清 , 黄勇奇 , 王书珍 , 方元平 , 金卫斌 . 2016 . 基于 Landsat5 TM 的麻城杜鹃花光谱分析与波段选择 . 湖北农业科学 , 55 ( 19 ): 4991 - 4994 [ DOI: 10.14088/j.cnki.issn0439-8114.2016.19.023 http://dx.doi.org/10.14088/j.cnki.issn0439-8114.2016.19.023 ]

Ma J Y , Yu W , Chen C , Liang P W , Guo X J and Jiang J J . 2020 . Pan-gan: an unsupervised pan-sharpening method for remote sensing image fusion . Information Fusion , 62 : 110 - 120 [ DOI: 10.1016/j.inffus.2020.04.006 http://dx.doi.org/10.1016/j.inffus.2020.04.006 ]

Ma L , Liu Y , Zhang X L , Ye Y X , Yin G F and Johnson B A . 2019 . Deep learning in remote sensing applications: a meta-analysis and review . ISPRS Journal of Photogrammetry and Remote Sensing , 152 : 166 - 177 [ DOI: 10.1016/j.isprsjprs.2019.04.015 http://dx.doi.org/10.1016/j.isprsjprs.2019.04.015 ]

Masi G , Cozzolino D , Verdoliva L and Scarpa G . 2016 . Pansharpening by convolutional neural networks . Remote Sensing , 8 ( 7 ): 594 : 1 - 594 : 22 [ DOI: 10.3390/rs8070594 http://dx.doi.org/10.3390/rs8070594 ]

Masi G , Cozzolino D , Verdoliva L and Scarpa G . 2017 . CNN-based pansharpening of multi-resolution remote-sensing images // Proceedings of the Joint Urban Remote Sensing Event . Dubai, United Arab Emirates: 1 - 4 [ DOI: 10.1109/JURSE.2017.7924534 http://dx.doi.org/10.1109/JURSE.2017.7924534 ]

Mohammadzadeh A , Tavakoli A and Valadan Z M . 2006 . Road extraction based on fuzzy logic and mathematical morphology from pan-sharpened ikonos images . The photogrammetric record , 21 ( 113 ): 44 - 60 [ DOI: 10.1111/j.1477-9730.2006.00353.x http://dx.doi.org/10.1111/j.1477-9730.2006.00353.x ]

Otazu X , González-Audícana M , Fors O and Nunez J . 2005 . Introduction of sensor spectral response into image fusion methods: application to wavelet-based methods . IEEE Transactions on Geoscience and Remote Sensing , 43 ( 10 ): 2376 - 2385 [ DOI: 10.1109/TGRS.2005.856106 http://dx.doi.org/10.1109/TGRS.2005.856106 ]

Ozcelik F , Alganci U , Sertel E and Unal G . 2020 . Rethinking cnn-based pansharpening: guided colorization of panchromatic images via gans . IEEE Transactions on Geoscience and Remote Sensing , 59 ( 4 ): 3486 - 3501 [ DOI: 10.1109/TGRS.2020.3010441 http://dx.doi.org/10.1109/TGRS.2020.3010441 ]

Pálsson F , Sveinsson J R , Benediktsson J A and Aanaes H . 2012 . Classification of pansharpened Urban satellite images . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 5 ( 1 ): 281 - 297 [ DOI: 10.1109/JSTARS.2011.2176467 http://dx.doi.org/10.1109/JSTARS.2011.2176467 ]

Palsson F , Sveinsson J R , Ulfarsson M O and Benediktsson J A . 2015 . Model based pansharpening method based on tv and mtf deblurring // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Milan, Italy : 33 - 36 [ DOI: 10.1109/IGARSS.2015.7325690 http://dx.doi.org/10.1109/IGARSS.2015.7325690 ]

Rao Y Z , He L and Zhu J W . 2017 . A residual convolutional neural network for pan-shaprening// 2017 International Workshop on Remote Sensing with Intelligent Processing . 1 - 4 [ DOI: 10.1109/RSIP.2017.7958807 http://dx.doi.org/10.1109/RSIP.2017.7958807 ]

Ripley B D . 1996 . Pattern Recognition and Neural Networks . Cambridge : Cambridge University Press

Scarpa G , Vitale S and Cozzolino D . 2018 . Target-adaptive cnn-based pansharpening . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 9 ): 5443 - 5457 [ DOI: 10.1109/TGRS.2018.2817393 http://dx.doi.org/10.1109/TGRS.2018.2817393 ]

Shah V P , Younan N H and King R L . 2008 . An efficient pan-sharpening method via a combined adaptive pca approach and contourlets . IEEE Transactions on Geoscience and Remote Sensing , 46 ( 5 ): 1323 - 1335 [ DOI: 10.1109/TGRS.2008.916211 http://dx.doi.org/10.1109/TGRS.2008.916211 ]

Shahdoosti H R and Javaheri N . 2017 . Pansharpening of clustered ms and pan images considering mixed pixels . IEEE Geoscience and Remote Sensing Letters , 14 ( 6 ): 826 - 830 [ DOI: 10.1109/LGRS.2017.2682122 http://dx.doi.org/10.1109/LGRS.2017.2682122 ]

Shao Z F and Cai J J . 2018 . Remote sensing image fusion with deep convolutional neural network . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 11 ( 5 ): 1656 - 1669 [ DOI: 10.1109/JSTARS.2018.2805923 http://dx.doi.org/10.1109/JSTARS.2018.2805923 ]

Shao Z M , Lu Z X , Ran M S , Fang L Y , Zhou J L and Zhang Y . 2020 . Residual encoder-decoder conditional generative adversarial network for pansharpening . IEEE Geoscience and Remote Sensing Letters , 17 ( 9 ): 1573 - 1577 [ DOI: 10.1109/LGRS.2019.2949745 http://dx.doi.org/10.1109/LGRS.2019.2949745 ]

Shi X J , Chen Z R , Wang H , Yeung D Y , Wong W K and Woo W C . 2015 . Convolutional lstm network: a machine learning approach for precipitation nowcasting // Proceedings of the International Conference on Neural Information Processing Systems (NIPS) . Montreal, Quebec, Canada : 802 - 810

Souza C , Firestone L , Silva L M and Roberts D . 2003 . Mapping forest degradation in the eastern amazon from spot 4 through spectral mixture models . Remote Sensing of Environment , 87 ( 4 ): 494 - 506 [ DOI: 10.1016/j.rse.2002.08.002 http://dx.doi.org/10.1016/j.rse.2002.08.002 ]

Tsagkatakis G , Aidini A , Fotiadou K , Giannopoulos M , Pentari A and Tsakalides P . 2019 . Survey of deep-learning approaches for remote sensing observation enhancement . Sensors , 19 ( 18 ): 3929 : 1 - 3929 : 39 [ DOI: 10.3390/s19183929 http://dx.doi.org/10.3390/s19183929 ]

Tu T M , Su S C , Shyu H C and Huang P S . 2001 . A new look at IHS-like image fusion methods . Information Fusion , 2 ( 3 ): 177 - 186 [ DOI: 10.1016/S1566-2535(01)00036-7 http://dx.doi.org/10.1016/S1566-2535(01)00036-7 ]

Vinothini D S and Bama B S . 2019 . Residual dense network for pan-sharpening satellite data . IEEE Sensors Journal , 19 ( 24 ): 12279 - 12285 [ DOI: 10.1109/JSEN.2019.2939844 http://dx.doi.org/10.1109/JSEN.2019.2939844 ]

Vitale S , Ferraioli G and Scarpa G . 2018 . A CNN-based model for pansharpening of worldview-3 images // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Valencia, Spain: 5108 - 5111 [ DOI: 10.1109/IGARSS.2018.8519202 http://dx.doi.org/10.1109/IGARSS.2018.8519202 ]

Vitale S . 2019 . A cnn-based pansharpening method with perceptual loss // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Yokohama, Japan: 3105 - 3108 [ DOI: 10.1109/IGARSS.2019.8900390 http://dx.doi.org/10.1109/IGARSS.2019.8900390 ]

Vivone G , Alparone L , Chanussot J , Mura M D , Garzelli A , Licciardi G A , Restaino R and Wald L . 2015 . A critical comparison among pansharpening algorithms . IEEE Transactions on Geoscience and Remote Sensing , 53 ( 5 ): 2565 - 2586 [ DOI: 10.1109/TGRS.2014.2361734 http://dx.doi.org/10.1109/TGRS.2014.2361734 ]

Wald L , Ranchin T and Mangolini M . 1997 . Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images . Photogrammetric Engineering & Remote Sensing , 63 ( 6 ): 691 - 699

Wald L . 2000 . Quality of high resolution synthesised images: Is there a simple criterion? // Proceedings of the Fusion of Earth data: merging point measurements, raster maps and remotely sensed images . Sophia Antipolis, France: 99 - 103

Wang D , Bai Y P and Li Y . 2020 . Multispectral pan-sharpening via dual-channel convolutional network with convolutional lstm based hierarchical spatial-spectral feature fusion. ArXiv Preprint ArXiv: 2007 . 10060

Wang F ， Guo Q and Ge X Q . 2021 . Pan-sharpening by deep recursive residual network . Journal of Remote Sensing , 25 ( 6 )： 1244 - 1256

王芬 , 郭擎 , 葛小青 . 深度递归残差网络的遥感图像空谱融合 . 2021 . 遥感学报 , 25 ( 6 ): 1244 - 1256 [ DOI： 10.11834/jrs.20219250 http://dx.doi.org/10.11834/jrs.20219250 ]

Wei Y C and Yuan Q Q . 2017 . Deep residual learning for remote sensed imagery pansharpening // Proceedings of the International Workshop on Remote Sensing with Intelligent Processing . Shanghai, China : 1 - 4 [ DOI: 10.1109/RSIP.2017.7958794 http://dx.doi.org/10.1109/RSIP.2017.7958794 ]

Wei Y C , Yuan Q Q , Shen H F and Zhang L P . 2017 . Boosting the accuracy of multispectral image pansharpening by learning a deep residual network . IEEE Geoscience and Remote Sensing Letters , 14 ( 10 ): 1795 - 1799 [ DOI: 10.1109/LGRS.2017.2736020 http://dx.doi.org/10.1109/LGRS.2017.2736020 ]

Xiang Z K , Xiao L , Liu P F and Zhang Y F . 2019 . A multi-scale densely deep learning method for pansharpening // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium . Yokohama, Japan: 2786 - 2789 [ DOI: 10.1109/IGARSS.2019.8898095 http://dx.doi.org/10.1109/IGARSS.2019.8898095 ]

Xing Y H , Wang M , Yang S Y and Jiao L C . 2018a . Pan-sharpening via deep metric learning . ISPRS Journal of Photogrammetry and Remote Sensing , 145 : 165 - 183 [ DOI: 10.1016/j.isprsjprs.2018.01.016 http://dx.doi.org/10.1016/j.isprsjprs.2018.01.016 ]

Xing Y H , Wang M , Yang S Y and Zhang K . 2018b . Pansharpening with multiscale geometric support tensor machine . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 5 ): 2503 - 2517 [ DOI: 10.1109/TGRS.2017.2742002 http://dx.doi.org/10.1109/TGRS.2017.2742002 ]

Yang J F , Fu X Y , Hu Y W , Huang Y , Ding X H and Paisley J . 2017 . Pannet: a deep network architecture for pan-sharpening // Proceedings of the IEEE International Conference on Computer Vision (ICCV) . Venice, Italy : 1753 - 1761 [ DOI: 10.1109/ICCV.2017.193 http://dx.doi.org/10.1109/ICCV.2017.193 ]

Yang S Y , Zhang K and Wang M . 2018a . Learning low-rank decomposition for pansharpening with spatial-spectral offsets . IEEE Transactions on Neural Networks & Learning Systems , 29 ( 8 ): 3647 - 3657 [ DOI: 10.1109/TNNLS.2017.2736011 http://dx.doi.org/10.1109/TNNLS.2017.2736011 ]

Yang Y , Li L Y , Huang S Y , Zhang Y M and Lu H Y . 2020 . Remote sensing image fusion with convolutional sparse representation based on adaptive dictionary learning . Journal of Signal Processing , 36 ( 1 ): 125 - 138

杨勇 , 李露奕 , 黄淑英 , 张迎梅 , 卢航远 . 2020 . 自适应字典学习的卷积稀疏表示遥感图像融合 . 信号处理 , 36 ( 1 ): 125 - 138 [ DOI: 10.16798/j.issn.1003-0530.2020.01.016 http://dx.doi.org/10.16798/j.issn.1003-0530.2020.01.016 ]

Yang Y , Tu W , Huang S Y and Lu H Y . 2020a . Pcdrn: progressive cascade deep residual network for pansharpening . Remote Sensing , 12 ( 4 ): 676 : 1 - 676 : 20 [ DOI: 10.3390/rs12040676 http://dx.doi.org/10.3390/rs12040676 ]

Yang Y , Wu L , Huang S Y , Sun J C , Wan W G and Wu J H . 2018b . Compensation details-based injection model for remote sensing image fusion . IEEE Geoscience and Remote Sensing Letters , 15 ( 5 ): 734 - 738 [ DOI: 10.1109/LGRS.2018.2810219 http://dx.doi.org/10.1109/LGRS.2018.2810219 ]

Yang Y , Wu L , Huang S Y , Tang Y J and Wan W G . 2018c . Pansharpening for multiband images with adaptive spectral–intensity modulation . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 11 ( 9 ): 3196 - 3208 [ DOI: 10.1109/JSTARS.2018.2849011 http://dx.doi.org/10.1109/JSTARS.2018.2849011 ]

Yang Y , Wu L , Huang S Y , Wan W G , Tu W and Lu H Y . 2020b . Multiband remote sensing image pansharpening based on dual-injection model . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13 : 1888 - 1904 [ DOI: 10.1109/JSTARS.2020.2981975 http://dx.doi.org/10.1109/JSTARS.2020.2981975 ]

Yao W , Zeng Z G , Lian C and Tang H M . 2018 . Pixel-wise regression using u-net and its application on pansharpening . Neurocomputing , 312 : 364 - 371 [ DOI: 10.1016/j.neucom.2018.05.103 http://dx.doi.org/10.1016/j.neucom.2018.05.103 ]

Ye F J , Li X F and Zhang X L . 2018 . Fusioncnn: a remote sensing image fusion algorithm based on deep convolutional neural networks . Multimedia Tools and Applications , 78 ( 11 ): 14683 - 14703 [ DOI: 10.1007/s11042-018-6850-3 http://dx.doi.org/10.1007/s11042-018-6850-3 ]

Ye F , Guo Y C and Zhuang P X . 2019 . Pan-sharpening via a gradient-based deep network prior . Signal Processing : Image Communication , 74 : 322 - 331 [ DOI: 10.1016/j.image.2019.03.004 http://dx.doi.org/10.1016/j.image.2019.03.004 ]

Yuan Q Q , Wei Y C , Meng X C , Shen H F and Zhang L P . 2018 . A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 11 ( 3 ): 978 - 989 [ DOI: 10.1109/JSTARS.2018.2794888 http://dx.doi.org/10.1109/JSTARS.2018.2794888 ]

Zeiler M D and Fergus R . 2014 . Visualizing and understanding convolutional networks // Proceedings of the Conference on European Conference on Computer Vision (ECCV) . Zurich, Switzerland : 818 - 833 [ DOI: 10.1007/978-3-319-10590-1_53 http://dx.doi.org/10.1007/978-3-319-10590-1_53 ]

Zhang H Q , Liu X Y , Yang S and Li Y . 2017 . Retrieval of remote sensing images based on semisupervised deep learning . Journal of Remote Sensing , 21 ( 3 ): 406 - 414

张洪群 , 刘雪莹 , 杨森 , 李宇 . 2017 . 深度学习的半监督遥感图像检索 . 遥感学报 , 21 ( 3 ): 406 - 414 [ DOI: 10.11834/jrs.20176105 http://dx.doi.org/10.11834/jrs.20176105 ]

Zhang L B , Zhang J , Lyu X R and Ma J . 2019a . A new pansharpening method using objectness based saliency analysis and saliency guided deep residual network // Proceedings of the IEEE International Conference on Image Processing . Taipei, China : 4529 - 4533 [ DOI: 10.1109/ICIP.2019.8803477 http://dx.doi.org/10.1109/ICIP.2019.8803477 ]

Zhang L P , Li W S , Zhang C and Lei D J . 2020 . A generative adversarial network with structural enhancement and spectral supplement for pan-sharpening . Neural Computing and Applications , 32 ( 24 ): 18347 - 18359 [ DOI: 10.1007/s00521-020-04973-w http://dx.doi.org/10.1007/s00521-020-04973-w ]

Zhang Y J , Liu C , Sun M W and Ou Y J . 2019b . Pan-sharpening using an efficient bidirectional pyramid network . IEEE Transactions on Geoscience and Remote Sensing , 57 ( 8 ): 5549 - 5563 [ DOI: 10.1109/TGRS.2019.2900419 http://dx.doi.org/10.1109/TGRS.2019.2900419 ]

Zhang Y L , Tian Y P , Kong Y , Zhong B E and Fu Y . 2018 . Residual dense network for image super-resolution // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Salt Lake City, UT, USA : 2472 - 2481 [ DOI: 10.1109/CVPR.2018.00262 http://dx.doi.org/10.1109/CVPR.2018.00262 ]

Zhang Y T , Li X H , Zhou JL . 2019c . Sftgan: a generative adversarial network for pan-sharpening equipped with spatial feature transform layers . Journal of Applied Remote Sensing , 13 ( 2 ): 026507 : 1 - 026507 : 16 [ DOI: 10.1117/1.JRS.13.026507 http://dx.doi.org/10.1117/1.JRS.13.026507 ]

Zhong J Y , Yang B , Huang G Y , Zhong Fei and Chen Z Z . 2016 . Remote sensing image fusion with convolutional neural network . Sensing and Imaging , 17 ( 1 ): 6 : 1 - 6 : 16 [ DOI: 10.1007/s11220-016-0135-6 http://dx.doi.org/10.1007/s11220-016-0135-6 ]

Zhou M , Yan K Y , Huang J , Yang Z H , Fu X Y and Zhao F . 2022 . Mutual information-driven pan-sharpening // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . New Orleans, Louisiana, USA : 1788 - 1798 [ DOI: 10.1109/CVPR52688.2022.00184 http://dx.doi.org/10.1109/CVPR52688.2022.00184 ]

Alert me when the article has been cited

提交

Pan-sharpening by deep recursive residual network

Fine 3D mapping and vegetation spatial differentiation study based on UAV LiDAR point cloud in karst wetland

LAE-Transformer： An airborne LiDAR point cloud segmentation network incorporating local-aware enhancement

A multi-scale InSAR phase filtering method incorporating adaptive enhancement and dynamic screening mechanisms