Deep-learning approaches for pixel-level pansharpening
- Vol. 26, Issue 12, Pages: 2411-2432(2022)
Published： 07 December 2022
DOI： 10.11834/jrs.20211325
扫 描 看 全 文
浏览全部资源
扫码关注微信
- About Journal
- Journal Online
- Special Source
摘要
全色图像锐化是遥感数据处理领域的一个基础性问题，在地物分类、目标识别等方面具有重要的研究意义和应用价值。近年来， 深度学习在自然语言处理、计算机视觉等领域取得了巨大进展， 也推动了像素级全色图像锐化技术的发展。本文提出从经典方式和协同方式两个方面对深度学习在全色图像锐化中的研究进行系统的综述，并在此基础上进行前景展望。首先，给出全色图像锐化常用的数据集和全色图像锐化的质量评价指标；接着，从经典方式与协同方式两个方面对基于深度学习的全色图像锐化最新研究成果进行分门别类的介绍，并进行算法性能的对比、分析和归纳；然后，对全色图像锐化的3个主要应用领域如地物分类、目标识别和地表变化检测进行分析；最后，本文探讨了基于深度学习的全色图像锐化的5个未来研究方向。
Abstract
Pansharpening is a fundamental problem in the field of remote sensing data processing. It has important research significance and application value in ground object classification and ground surface change detection. In recent years
Deep Learning (DL) has made great progress in natural language processing
computer vision
etc. and has promoted the development of pixel-level pansharpening technology. This work presents a systematic review of the research of DL in pansharpening from two aspects (classical and collaborative approaches) and makes a prospect on this basis. First
the common datasets of pansharpening and the objective evaluation indexes of pansharpening
including reference and non-reference quality evaluation indexes
are provided. Second
the latest research results of DL-based pansharpening are introduced in two different categories from the classical and collaborative methods
and the performance of their algorithms is compared
analyzed
and summarized. The classical methods mainly include AE-based pansharpening
CNN-based pansharpening
DRN-based pansharpening
and GAN-based pansharpening methods. Meanwhile
the collaborative methods mainly include DL+CS-based pansharpening
DL+MRA-based pansharpening
DL+MB-based pansharpening
DL+injection model-based pansharpening
CNN+DRN-based pansharpening
and RNN+CNN-based pansharpening methods. In the comparative analysis of the classical and collaborative methods
the common point is that the DL technology can automatically learn the advantages of complex data features and extract the feature information of the MS or PAN image (i.e.
the information that needs to be retained in the HRMS fusion image). The difference is that the structure of the classical mode is more concise
while that of the collaborative mode is more complex because it is the combination of multiple methods or frameworks. In addition
most early DL-based pansharpening methods utilized the powerful data fitting ability of the DL model and seldom paid attention to the field of pansharpening problems. With the gradual deepening of research
such as using DL methods combined with traditional pansharpening methods
this designed fusion model considers spectral and spatial distortions. Accordingly
the DL methods can further enhance the pansharpening effect. Thirdly
the three main application fields of pansharpening are analyzed
such as object classification
target recognition
and surface change detection. Finally
this work discusses the future research direction of DL-based pansharpening in combination with remote sensing knowledge to fully tap the potential of DL to obtain fused images with richer details and more natural spectra. For example
for the evaluation of pansharpening application
the performance of pansharpening in a certain application is related not only to the high quality of fusion image but also to the knowledge of a specific application field. Accordingly
the application-oriented pansharpening evaluation algorithms will be the focus of future study. Furthermore
DL-based pansharpening needs to train a large number of network parameters
resulting in a longer training time for the pansharpening model. The lightweight depth model has a smaller network capacity
lower time complexity
and lower hardware requirements. Therefore
constructing a lightweight pansharpening model is a promising future direction.
关键词
全色图像锐化深度学习经典方式协同方式质量评价遥感图像融合
Keywords
pansharpeningdeep learningclassical modecollaborative modequality evaluationremote sensing image fusion
references
Aguilar M A, Saldaña M M and Aguilar F. 2013. GeoEye-1 and worldview-2 pan-sharpened imagery for object-based classification in urban environments. International Journal of Remote Sensing, 34(7):2583-2606 [DOI: 10.1080/01431161.2012.747018http://dx.doi.org/10.1080/01431161.2012.747018]
Aiazzi B, Baronti S and Selva M. 2007. Improving component substitution pansharpening through multivariate regression of ms+pan data. IEEE Transactions on Geoscience and Remote Sensing, 45(10):3230-3239 [DOI: 10.1109/tgrs.2007.901007http://dx.doi.org/10.1109/tgrs.2007.901007]
Alparone L, Aiazzi B, Baronti S and Garzelli A. 2015. Remote sensing image fusion. London, UK: CRC Press
Alparone L, Wald L, Chanussot J, Thomas C, Gamba P and Bruce L M. 2007. Comparison of pansharpening algorithms: outcome of the 2006 grs-s data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45(10):3012-3021 [DOI: 10.1109/TGRS.2007.904923http://dx.doi.org/10.1109/TGRS.2007.904923]
Azarang A and Ghassemian H. 2017. A new pansharpening method using multi resolution analysis framework and deep neural networks. // Proceedings of the International Conference on Pattern Recognition and Image Analysis. Shahrekord, Iran, 1-6 [DOI: 10.1109/pria.2017.7983017http://dx.doi.org/10.1109/pria.2017.7983017]
Azarang A, Manoochehri H E and Kehtarnavaz N. 2019. Convolutional autoencoder-based multispectral image fusion. IEEE Access, 7:35673-35683 [DOI: 10.1109/ACCESS.2019.2905511http://dx.doi.org/10.1109/ACCESS.2019.2905511]
Benzenati T, Kessentini Y, Kallel A and Hallabia H. 2020. Generalized laplacian pyramid pan-sharpening gain injection prediction based on cnn. IEEE Geoscience and Remote Sensing Letters, 17(4):651-655 [DOI: 10.1109/LGRS.2019.2928181http://dx.doi.org/10.1109/LGRS.2019.2928181]
Bovolo F, Bruzzone L, Capobianco L, Garzelli A, Marchesi S and Nencini F. 2010. Analysis of the effects of pansharpening in change detection on VHR images. IEEE Geoscience and Remote Sensing Letters, 7(1): 53-57 [DOI: 10.1109/LGRS.2009.2029248http://dx.doi.org/10.1109/LGRS.2009.2029248]
Bowman S R, Angeli G, Potts C and Manning C D. 2015. A large annotated corpus for learning natural language inference//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Lisbon, Portugal: 632-642 [DOI: 10.18653/v1/D15-1075http://dx.doi.org/10.18653/v1/D15-1075]
Cai W T, Xu Y, Wu Z B, Liu H Y, Qian L and Wei Z H. 2018. Pan-sharpening based on multilevel coupled deep network. // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain: 7046-7049 [DOI: 10.1109/IGARSS.2018.8518121http://dx.doi.org/10.1109/IGARSS.2018.8518121]
Carper W J, Lillesand T M and Kiefer P W. 1990. The use of intensity-hue-saturation transformations for merging spot panchromatic and multispectral image data. Photogrammetric Engineering and Remote Sensing, 56(4):459–467
Chen C, Fu J Q, Gai Y Y, Li J, Chen L, Mantravadi V S and Tan A H. 2018. Damaged bridges over water: using high-spatial-resolution remote-sensing images for recognition, detection, and assessment. IEEE Geoscience and Remote Sensing Magazine, 6(3):69-85 [DOI: 10.1109/MGRS.2018.2852804http://dx.doi.org/10.1109/MGRS.2018.2852804]
Chen C, Fu J Q, Lu N, Chu YL, Hu J C, Guo B Y and Zhao X. 2019. Knowledge-based identification and damage detection of bridges spanning water via high-spatial-resolution optical remotely sensed imagery. Journal of the Indian Society of Remote Sensing, 47(12):1999-2008 [DOI: 10.1007/s12524-019-01036-zhttp://dx.doi.org/10.1007/s12524-019-01036-z]
Chen F J, Zhu F, Wu Q X, Hao Y M, Wang E D and Cui Y G. 2021. A survey about image generation with generative adversarial nets. Chinese Journal of Computers, 44(2): 347-369
陈佛计,朱枫,吴清潇,郝颖明,王恩德,崔芸阁. 2021.生成对抗网络及其在图像生成中的应用研究综述.计算机学报, 44(2):347-369 [DOI: 10.11897/SP.J.1016.2021.00347http://dx.doi.org/10.11897/SP.J.1016.2021.00347]
Chen J F, Pan Y and Chen Y. 2020. Remote sensing image fusion based on Bayesian GAN. ArXiv Preprint ArXiv: 2009.09465
Chen M M, Guo Q, Liu M L and Li A. 2021. Pan-sharpening by residual network with dense convolution for remote sensing images. Journal of Remote Sensing，25（6）：1270-1283
陈毛毛,郭擎, 刘明亮, 李安. 2021. 密集卷积残差网络的遥感图像融合. 遥感学报, 25(6): 1270-1283 [DOI：10.11834/jrs.20219411http://dx.doi.org/10.11834/jrs.20219411]
Da Cunha A L, Zhou J P and Do M N. 2006. The nonsubsampled contourlet transform: theory, design, and applications. IEEE Transactions on Image Processing, 15(10):3089-3101 [DOI: 10.1109/TIP.2006.877507http://dx.doi.org/10.1109/TIP.2006.877507]
Deng L J, Vivone G, Jin C and Chanussot J. 2020. Detail injection-based deep convolutional neural networks for pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 59(8):6995-7010 [DOI: 10.1109/TGRS.2020.3031366http://dx.doi.org/10.1109/TGRS.2020.3031366]
Dong C, Loy C C, He K M and Tang X O. 2016. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2):295-307 [DOI: 10.1109/TPAMI.2015.2439281http://dx.doi.org/10.1109/TPAMI.2015.2439281]
Eghbalian S and Ghassemian H. 2018. Multi spectral image fusion by deep convolutional neural network and new spectral loss function. International Journal of Remote Sensing, 39(12):3983-4002 [DOI: 10.1080/01431161.2018.1452074http://dx.doi.org/10.1080/01431161.2018.1452074]
Garzelli A, Capobianco L and Nencini F. 2009. On the effects of pan-sharpening to target detection//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Cape Town, South Africa: II-136‒II-139 [DOI: 10.1109/IGARSS.2009.5418022http://dx.doi.org/10.1109/IGARSS.2009.5418022]
Garzelli A, Nencini F and Capobianco L. 2008. Optimal mmse pansharpening of very high resolution multispectral images. IEEE Transactions on Geoscience and Remote Sensing, 46(1):228-236 [DOI: 10.1109/TGRS.2007.907604http://dx.doi.org/10.1109/TGRS.2007.907604]
Gianinetto M, Rusmini M, Candiani G, Via G D, Frassy F, Maianti P, Marchesi A, Nodari FR and Dini L. 2017. Hierarchical classification of complex landscape with VHR pan-sharpened satellite data and obia techniques. European Journal of Remote Sensing, 47(1):229-250 [DOI: 10.5721/EuJRS20144715http://dx.doi.org/10.5721/EuJRS20144715]
Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A C and Bengio Y. 2014. Generative adversarial nets//Proceedings of the Advances in Neural Information Processing Systems (NIPS). Montreal, Quebec, Canada: 2672-2680
Guo Y C, Ye F and Gong H. 2019. Learning an efficient convolution neural network for pansharpening. Algorithms, 12(1):16:1-16:14 [DOI: 10.3390/a12010016http://dx.doi.org/10.3390/a12010016]
He G Q, Xing S Y, Xia Z Q, Huang Q Q and Fan J P. 2018. Panchromatic and multi-spectral image fusion for new satellites based on multi-channel deep model. Machine Vision and Applications, 29(6):933-946 [DOI: 10.1007/s00138-018-0964-5http://dx.doi.org/10.1007/s00138-018-0964-5]
He K M and Sun J. 2015. Convolutional neural networks at constrained time cost//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: 5353-5360 [DOI: 10.1109/CVPR.2015.7299173http://dx.doi.org/10.1109/CVPR.2015.7299173]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition. //Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: 770-778 [DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
He L, Rao Y Z, Li J, Chanussot J, Plaza A, Zhu J W and Li B. 2019. Pansharpening via detail injection based convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(4):1188-1204 [DOI: 10.1109/JSTARS.2019.2898574http://dx.doi.org/10.1109/JSTARS.2019.2898574]
He L, Xi D H, Li J and Zhu J W. 2020. A spectral-aware convolutional neural network for pansharpening. Applied Sciences, 10(17):5809:1-5809:18 [DOI: 10.3390/app10175809http://dx.doi.org/10.3390/app10175809]
Hinton G E and Salakhutdinov R R. 2006. Reducing the dimensionality of data with neural networks. Science, 313:504-507 [DOI: 10.1126/science.1127647http://dx.doi.org/10.1126/science.1127647]
Hinton G E, Osindero S and Teh Y W. 2006. A fast learning algorithm for deep belief nets. Neural Computation, 18(7):1527-1554 [DOI: 10.1162/neco.2006.18.7.1527http://dx.doi.org/10.1162/neco.2006.18.7.1527]
Hochreiter S and Schmidhuber J. 1997. Long short-term memory. Neural Computation, 9(8):1735-1780 [DOI: 10.1162/neco.1997.9.8.1735http://dx.doi.org/10.1162/neco.1997.9.8.1735]
Hu J, He Z and Wu J M. 2019. Deep self-learning network for adaptive pansharpening. Remote Sensing, 11(20):2395:1-2395:22 [DOI: 10.3390/rs11202395http://dx.doi.org/10.3390/rs11202395]
Huang G, Liu Z, Maaten L V D, and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: 2261-2269 [DOI: 10.1109/CVPR.2017.243http://dx.doi.org/10.1109/CVPR.2017.243]
Huang W, Fei X, Yin J R and Liu Y. 2018. A multi-direction subbands and deep neural networks based pan-sharpening method//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain: 5139-5142 [DOI: 10.1109/IGARSS.2018.8517817http://dx.doi.org/10.1109/IGARSS.2018.8517817]
Huang W, Xiao L, Wei Z H, Liu H Y and Tang S Z. 2015. A new pan-sharpening method with deep neural networks. IEEE Geoscience and Remote Sensing Letters, 12(5):1037-1041 [DOI: 10.1109/LGRS.2014.2376034http://dx.doi.org/10.1109/LGRS.2014.2376034]
Javan F D, Samadzadegan F, Mehravar S, Toosi A, Khatami R and Stein A. 2021. A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 171:101-117 [DOI: 10.1016/j.isprsjprs.2020.11.001http://dx.doi.org/10.1016/j.isprsjprs.2020.11.001]
Jiang M H, Li J, Yuan Q Q, Shen H F, Liu X X and Xu M M. 2019. Differential information residual convolutional neural network for pansharpening. // Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan: 4865-4868 [DOI: 10.1109/IGARSS.2019.8900270http://dx.doi.org/10.1109/IGARSS.2019.8900270]
Kang X D, Li S T and Benediktsson J A. 2014. Pansharpening with matting model. IEEE Transactions on Geoscience and Remote Sensing, 52(8):5088-5099 [DOI: 10.1109/TGRS.2013.2286827http://dx.doi.org/10.1109/TGRS.2013.2286827]
Khan M M, Chanussot J, Condat L and Montanvert A. 2008. Indusion: fusion of multispectral and panchromatic images using the induction scaling technique. IEEE Geoscience and Remote Sensing Letters, 5(1):98-102 [DOI: 10.1109/LGRS.2007.909934http://dx.doi.org/10.1109/LGRS.2007.909934]
Kim J, Lee J K and Lee K M. 2016. Accurate image super-resolution using very deep convolutional networks. // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: 1646-1654 [DOI: 10.1109/CVPR.2016.182http://dx.doi.org/10.1109/CVPR.2016.182]
Laben C A and Brower B V. 2000. Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. U.S., No.6011875
Lee J, Seo S and Kim M. 2021. Sipsa-net: shift-invariant pan sharpening with moving object alignment for satellite imagery//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Online: 10161-10169 [DOI: 10.1109/CVPR46437.2021.01003http://dx.doi.org/10.1109/CVPR46437.2021.01003]
Li H, Liu F, Yang S Y and Zhang K. 2016. Remote sensing image fusion based on deep support value learning networks. Chinese Journal of Computers, 39(8): 1583-1596
李红, 刘芳, 杨淑媛, 张凯.2016.基于深度支撑值学习网络的遥感图像融合. 计算机学报, 39(8): 1583-1596 [DOI: 10.11897/SP.J.1016.2016.01583http://dx.doi.org/10.11897/SP.J.1016.2016.01583]
Li J, Li Y F, He L, Chen J and Plaza A. 2020a. Spatio-temporal fusion for remote sensing data: an overview and new benchmark. Science China Information Sciences, 63(4) :7-23 [DOI: 10.1007/s11432-019-2785-yhttp://dx.doi.org/10.1007/s11432-019-2785-y]
Li N, Huang N and Xiao L. 2017. Pan-sharpening via residual deep learning//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Fort Worth, TX, USA: 5133-5136 [DOI: 10.1109/IGARSS.2017.8128158http://dx.doi.org/10.1109/IGARSS.2017.8128158]
Li S T，Li C Y and Kang X D. 2021. Development status and future prospects of multi-source remote sensing image fusion. Journal of Remote Sensing, 25(1)：148-166
李树涛, 李聪妤, 康旭东. 2021. 多源遥感图像融合发展现状与未来展望. 遥感学报, 25(1):148-166 [DOI：10.11834/jrs.20210259http://dx.doi.org/10.11834/jrs.20210259]
Li S. 2018. Detection Method of Urban Land Cover Change Based on Combining Domain Knowledge and Deep Learning. Wuhan: Wuhan University
李胜. 2018. 联合领域知识和深度学习的城市地表覆盖变化检测方法. 武汉：武汉大学
Li X, Pan Y, Gao A, Li L X, Mei S H and Yue S G. 2018. Pansharpening based on joint gaussian guided upsampling //Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain: 7184-7187 [DOI: 10.1109/IGARSS.2018.8518150http://dx.doi.org/10.1109/IGARSS.2018.8518150]
Li X, Xu F, Lyu X, Tong Y, Chen Z Q, Li S Y and Liu D F. 2020b. A remote-sensing image pan-sharpening method based on multi-scale channel attention residual network. IEEE Access, 8:27163-27177 [DOI: 10.1109/ACCESS.2020.2971502http://dx.doi.org/10.1109/ACCESS.2020.2971502]
Li Z Q and Cheng C Q. 2019. A cnn-based pan-sharpening method for integrating panchromatic and multispectral images using landsat 8. Remote Sensing, 11(22): 2606:1-2606:17 [DOI: 10.3390/rs11222606http://dx.doi.org/10.3390/rs11222606]
Liu J M and Liang S L. 2016. Pan-sharpening using a guided filter. International Journal of Remote Sensing, 37(8):1777-1800 [DOI: 10.1080/01431161.2016.1163749http://dx.doi.org/10.1080/01431161.2016.1163749]
Liu L, Wang J, Zhang E L, Li B, Zhu X, Zhang Y Q and Peng J Y. 2020. Shallow–deep convolutional network and spectral-discrimination-based detail injection for multispectral imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13:1772-1783 [DOI: 10.1109/JSTARS.2020.2981695http://dx.doi.org/10.1109/JSTARS.2020.2981695]
Liu P F, Xiao L and Li T. 2018a. A variational pan-sharpening method based on spatial fractional-order geometry and spectral-spatial low-rank priors. IEEE Transactions on Geoscience and Remote Sensing, 56(3):1788-1802 [DOI: 10.1109/TGRS.2017.2768386http://dx.doi.org/10.1109/TGRS.2017.2768386]
Liu X Y, Wang Y H and Liu Q J. 2018b. Psgan: a generative adversarial network for remote sensing image pan-sharpening//Proceedings of the IEEE International Conference on Image Processing. Athens, Greece: 873-877 [DOI: 10.1109/ICIP.2018.8451049http://dx.doi.org/10.1109/ICIP.2018.8451049]
Liu Y, Chen X, Wang Z F, Wang Z J, Ward R K and Wang X S. 2018c. Deep learning for pixel level image fusion: recent advances and future prospects. Information Fusion, 42:158-173 [DOI: 10.1016/j.inffus.2017.10.007http://dx.doi.org/10.1016/j.inffus.2017.10.007]
Lu X C, Zhang J P and Zhang Y. 2017. An improved non-subsampled contourlet transform-based hybrid pan-sharpening algorithm//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Fort Worth, TX, USA: 3393-3396 [DOI: 10.1109/IGARSS.2017.8127726http://dx.doi.org/10.1109/IGARSS.2017.8127726]
Luo D Q, Huang Y Q, Wang S Z, Fang Y P and Jin W B. 2016. Spectral signature analysis and band selection of Macheng rhododendron based on Landsat5 TM. Hubei Agricultural Sciences, 55(19):4991-4994
罗代清, 黄勇奇, 王书珍, 方元平, 金卫斌 . 2016. 基于 Landsat5 TM 的麻城杜鹃花光谱分析与波段选择 . 湖北农业科学, 55(19): 4991-4994 [DOI: 10.14088/j.cnki.issn0439-8114.2016.19.023http://dx.doi.org/10.14088/j.cnki.issn0439-8114.2016.19.023]
Ma J Y, Yu W, Chen C, Liang P W, Guo X J and Jiang J J. 2020. Pan-gan: an unsupervised pan-sharpening method for remote sensing image fusion. Information Fusion, 62:110-120 [DOI: 10.1016/j.inffus.2020.04.006http://dx.doi.org/10.1016/j.inffus.2020.04.006]
Ma L, Liu Y, Zhang X L, Ye Y X, Yin G F and Johnson B A. 2019. Deep learning in remote sensing applications: a meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, 152:166-177 [DOI: 10.1016/j.isprsjprs.2019.04.015http://dx.doi.org/10.1016/j.isprsjprs.2019.04.015]
Masi G, Cozzolino D, Verdoliva L and Scarpa G. 2016. Pansharpening by convolutional neural networks. Remote Sensing, 8(7):594:1-594:22 [DOI: 10.3390/rs8070594http://dx.doi.org/10.3390/rs8070594]
Masi G, Cozzolino D, Verdoliva L and Scarpa G. 2017. CNN-based pansharpening of multi-resolution remote-sensing images// Proceedings of the Joint Urban Remote Sensing Event. Dubai, United Arab Emirates: 1-4 [DOI: 10.1109/JURSE.2017.7924534http://dx.doi.org/10.1109/JURSE.2017.7924534]
Mohammadzadeh A, Tavakoli A and Valadan Z M. 2006. Road extraction based on fuzzy logic and mathematical morphology from pan-sharpened ikonos images. The photogrammetric record, 21(113):44-60 [DOI: 10.1111/j.1477-9730.2006.00353.xhttp://dx.doi.org/10.1111/j.1477-9730.2006.00353.x]
Otazu X, González-Audícana M, Fors O and Nunez J. 2005. Introduction of sensor spectral response into image fusion methods: application to wavelet-based methods. IEEE Transactions on Geoscience and Remote Sensing, 43(10):2376-2385 [DOI: 10.1109/TGRS.2005.856106http://dx.doi.org/10.1109/TGRS.2005.856106]
Ozcelik F, Alganci U, Sertel E and Unal G. 2020. Rethinking cnn-based pansharpening: guided colorization of panchromatic images via gans. IEEE Transactions on Geoscience and Remote Sensing, 59(4):3486-3501 [DOI: 10.1109/TGRS.2020.3010441http://dx.doi.org/10.1109/TGRS.2020.3010441]
Pálsson F, Sveinsson J R, Benediktsson J A and Aanaes H. 2012. Classification of pansharpened Urban satellite images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 5(1):281-297 [DOI: 10.1109/JSTARS.2011.2176467http://dx.doi.org/10.1109/JSTARS.2011.2176467]
Palsson F, Sveinsson J R, Ulfarsson M O and Benediktsson J A. 2015. Model based pansharpening method based on tv and mtf deblurring//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Milan, Italy: 33-36 [DOI: 10.1109/IGARSS.2015.7325690http://dx.doi.org/10.1109/IGARSS.2015.7325690]
Rao Y Z, He L and Zhu J W. 2017. A residual convolutional neural network for pan-shaprening//2017 International Workshop on Remote Sensing with Intelligent Processing. 1-4 [DOI: 10.1109/RSIP.2017.7958807http://dx.doi.org/10.1109/RSIP.2017.7958807]
Ripley B D. 1996. Pattern Recognition and Neural Networks. Cambridge: Cambridge University Press
Scarpa G, Vitale S and Cozzolino D. 2018. Target-adaptive cnn-based pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 56(9):5443-5457 [DOI: 10.1109/TGRS.2018.2817393http://dx.doi.org/10.1109/TGRS.2018.2817393]
Shah V P, Younan N H and King R L. 2008. An efficient pan-sharpening method via a combined adaptive pca approach and contourlets. IEEE Transactions on Geoscience and Remote Sensing, 46(5):1323-1335 [DOI: 10.1109/TGRS.2008.916211http://dx.doi.org/10.1109/TGRS.2008.916211]
Shahdoosti H R and Javaheri N. 2017. Pansharpening of clustered ms and pan images considering mixed pixels. IEEE Geoscience and Remote Sensing Letters, 14(6):826-830 [DOI: 10.1109/LGRS.2017.2682122http://dx.doi.org/10.1109/LGRS.2017.2682122]
Shao Z F and Cai J J. 2018. Remote sensing image fusion with deep convolutional neural network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(5):1656-1669 [DOI: 10.1109/JSTARS.2018.2805923http://dx.doi.org/10.1109/JSTARS.2018.2805923]
Shao Z M, Lu Z X, Ran M S, Fang L Y, Zhou J L and Zhang Y. 2020. Residual encoder-decoder conditional generative adversarial network for pansharpening. IEEE Geoscience and Remote Sensing Letters, 17(9):1573-1577 [DOI: 10.1109/LGRS.2019.2949745http://dx.doi.org/10.1109/LGRS.2019.2949745]
Shi X J, Chen Z R, Wang H, Yeung D Y, Wong W K and Woo W C. 2015. Convolutional lstm network: a machine learning approach for precipitation nowcasting//Proceedings of the International Conference on Neural Information Processing Systems (NIPS). Montreal, Quebec, Canada: 802-810
Souza C, Firestone L, Silva L M and Roberts D. 2003. Mapping forest degradation in the eastern amazon from spot 4 through spectral mixture models. Remote Sensing of Environment, 87(4):494-506 [DOI: 10.1016/j.rse.2002.08.002http://dx.doi.org/10.1016/j.rse.2002.08.002]
Tsagkatakis G, Aidini A, Fotiadou K, Giannopoulos M, Pentari A and Tsakalides P. 2019. Survey of deep-learning approaches for remote sensing observation enhancement. Sensors, 19(18):3929:1-3929:39 [DOI: 10.3390/s19183929http://dx.doi.org/10.3390/s19183929]
Tu T M, Su S C, Shyu H C and Huang P S. 2001. A new look at IHS-like image fusion methods. Information Fusion, 2(3):177-186 [DOI: 10.1016/S1566-2535(01)00036-7http://dx.doi.org/10.1016/S1566-2535(01)00036-7]
Vinothini D S and Bama B S. 2019. Residual dense network for pan-sharpening satellite data. IEEE Sensors Journal, 19(24):12279-12285 [DOI: 10.1109/JSEN.2019.2939844http://dx.doi.org/10.1109/JSEN.2019.2939844]
Vitale S, Ferraioli G and Scarpa G. 2018. A CNN-based model for pansharpening of worldview-3 images//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain: 5108-5111 [DOI: 10.1109/IGARSS.2018.8519202http://dx.doi.org/10.1109/IGARSS.2018.8519202]
Vitale S. 2019. A cnn-based pansharpening method with perceptual loss//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan: 3105-3108 [DOI: 10.1109/IGARSS.2019.8900390http://dx.doi.org/10.1109/IGARSS.2019.8900390]
Vivone G, Alparone L, Chanussot J, Mura M D, Garzelli A, Licciardi G A, Restaino R and Wald L. 2015. A critical comparison among pansharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing, 53(5):2565-2586 [DOI: 10.1109/TGRS.2014.2361734http://dx.doi.org/10.1109/TGRS.2014.2361734]
Wald L, Ranchin T and Mangolini M. 1997. Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images. Photogrammetric Engineering & Remote Sensing, 63(6):691-699
Wald L. 2000. Quality of high resolution synthesised images: Is there a simple criterion? // Proceedings of the Fusion of Earth data: merging point measurements, raster maps and remotely sensed images. Sophia Antipolis, France: 99-103
Wang D, Bai Y P and Li Y. 2020. Multispectral pan-sharpening via dual-channel convolutional network with convolutional lstm based hierarchical spatial-spectral feature fusion. ArXiv Preprint ArXiv: 2007.10060
Wang F，Guo Q and Ge X Q. 2021. Pan-sharpening by deep recursive residual network. Journal of Remote Sensing, 25(6)：1244-1256
王芬, 郭擎, 葛小青. 深度递归残差网络的遥感图像空谱融合. 2021. 遥感学报, 25(6): 1244-1256 [DOI：10.11834/jrs.20219250http://dx.doi.org/10.11834/jrs.20219250]
Wei Y C and Yuan Q Q. 2017. Deep residual learning for remote sensed imagery pansharpening//Proceedings of the International Workshop on Remote Sensing with Intelligent Processing. Shanghai, China: 1-4 [DOI: 10.1109/RSIP.2017.7958794http://dx.doi.org/10.1109/RSIP.2017.7958794]
Wei Y C, Yuan Q Q, Shen H F and Zhang L P. 2017. Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geoscience and Remote Sensing Letters, 14(10):1795-1799 [DOI: 10.1109/LGRS.2017.2736020http://dx.doi.org/10.1109/LGRS.2017.2736020]
Xiang Z K, Xiao L, Liu P F and Zhang Y F. 2019. A multi-scale densely deep learning method for pansharpening//Proceedings of the IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan: 2786-2789 [DOI: 10.1109/IGARSS.2019.8898095http://dx.doi.org/10.1109/IGARSS.2019.8898095]
Xing Y H, Wang M, Yang S Y and Jiao L C. 2018a. Pan-sharpening via deep metric learning. ISPRS Journal of Photogrammetry and Remote Sensing, 145:165-183 [DOI: 10.1016/j.isprsjprs.2018.01.016http://dx.doi.org/10.1016/j.isprsjprs.2018.01.016]
Xing Y H, Wang M, Yang S Y and Zhang K. 2018b. Pansharpening with multiscale geometric support tensor machine. IEEE Transactions on Geoscience and Remote Sensing, 56(5):2503-2517 [DOI: 10.1109/TGRS.2017.2742002http://dx.doi.org/10.1109/TGRS.2017.2742002]
Yang J F, Fu X Y, Hu Y W, Huang Y, Ding X H and Paisley J. 2017. Pannet: a deep network architecture for pan-sharpening// Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: 1753-1761 [DOI: 10.1109/ICCV.2017.193http://dx.doi.org/10.1109/ICCV.2017.193]
Yang S Y, Zhang K and Wang M. 2018a. Learning low-rank decomposition for pansharpening with spatial-spectral offsets. IEEE Transactions on Neural Networks & Learning Systems, 29(8):3647-3657 [DOI: 10.1109/TNNLS.2017.2736011http://dx.doi.org/10.1109/TNNLS.2017.2736011]
Yang Y, Li L Y, Huang S Y, Zhang Y M and Lu H Y. 2020. Remote sensing image fusion with convolutional sparse representation based on adaptive dictionary learning. Journal of Signal Processing, 36(1):125-138
杨勇, 李露奕, 黄淑英, 张迎梅, 卢航远. 2020. 自适应字典学习的卷积稀疏表示遥感图像融合. 信号处理, 36(1):125-138 [DOI: 10.16798/j.issn.1003-0530.2020.01.016http://dx.doi.org/10.16798/j.issn.1003-0530.2020.01.016]
Yang Y, Tu W, Huang S Y and Lu H Y. 2020a. Pcdrn: progressive cascade deep residual network for pansharpening. Remote Sensing, 12(4):676:1-676:20 [DOI:10.3390/rs12040676http://dx.doi.org/10.3390/rs12040676]
Yang Y, Wu L, Huang S Y, Sun J C, Wan W G and Wu J H. 2018b. Compensation details-based injection model for remote sensing image fusion. IEEE Geoscience and Remote Sensing Letters, 15(5):734-738 [DOI: 10.1109/LGRS.2018.2810219http://dx.doi.org/10.1109/LGRS.2018.2810219]
Yang Y, Wu L, Huang S Y, Tang Y J and Wan W G. 2018c. Pansharpening for multiband images with adaptive spectral–intensity modulation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(9):3196-3208 [DOI: 10.1109/JSTARS.2018.2849011http://dx.doi.org/10.1109/JSTARS.2018.2849011]
Yang Y, Wu L, Huang S Y, Wan W G, Tu W and Lu H Y. 2020b. Multiband remote sensing image pansharpening based on dual-injection model. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13:1888-1904 [DOI: 10.1109/JSTARS.2020.2981975http://dx.doi.org/10.1109/JSTARS.2020.2981975]
Yao W, Zeng Z G, Lian C and Tang H M. 2018. Pixel-wise regression using u-net and its application on pansharpening. Neurocomputing, 312:364-371 [DOI: 10.1016/j.neucom.2018.05.103http://dx.doi.org/10.1016/j.neucom.2018.05.103]
Ye F J, Li X F and Zhang X L. 2018. Fusioncnn: a remote sensing image fusion algorithm based on deep convolutional neural networks. Multimedia Tools and Applications, 78(11):14683-14703 [DOI: 10.1007/s11042-018-6850-3http://dx.doi.org/10.1007/s11042-018-6850-3]
Ye F, Guo Y C and Zhuang P X. 2019. Pan-sharpening via a gradient-based deep network prior. Signal Processing: Image Communication, 74:322-331 [DOI: 10.1016/j.image.2019.03.004http://dx.doi.org/10.1016/j.image.2019.03.004]
Yuan Q Q, Wei Y C, Meng X C, Shen H F and Zhang L P. 2018. A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3):978-989 [DOI: 10.1109/JSTARS.2018.2794888http://dx.doi.org/10.1109/JSTARS.2018.2794888]
Zeiler M D and Fergus R. 2014. Visualizing and understanding convolutional networks//Proceedings of the Conference on European Conference on Computer Vision (ECCV). Zurich, Switzerland: 818-833 [DOI:10.1007/978-3-319-10590-1_53http://dx.doi.org/10.1007/978-3-319-10590-1_53]
Zhang H Q, Liu X Y, Yang S and Li Y. 2017. Retrieval of remote sensing images based on semisupervised deep learning. Journal of Remote Sensing, 21(3): 406-414
张洪群,刘雪莹,杨森,李宇. 2017. 深度学习的半监督遥感图像检索. 遥感学报, 21(3):406-414 [DOI: 10.11834/jrs.20176105http://dx.doi.org/10.11834/jrs.20176105]
Zhang L B, Zhang J, Lyu X R and Ma J. 2019a. A new pansharpening method using objectness based saliency analysis and saliency guided deep residual network//Proceedings of the IEEE International Conference on Image Processing. Taipei, China: 4529-4533 [DOI: 10.1109/ICIP.2019.8803477http://dx.doi.org/10.1109/ICIP.2019.8803477]
Zhang L P, Li W S, Zhang C and Lei D J. 2020. A generative adversarial network with structural enhancement and spectral supplement for pan-sharpening. Neural Computing and Applications, 32(24):18347-18359 [DOI: 10.1007/s00521-020-04973-whttp://dx.doi.org/10.1007/s00521-020-04973-w]
Zhang Y J, Liu C, Sun M W and Ou Y J. 2019b. Pan-sharpening using an efficient bidirectional pyramid network. IEEE Transactions on Geoscience and Remote Sensing, 57(8):5549-5563 [DOI: 10.1109/TGRS.2019.2900419http://dx.doi.org/10.1109/TGRS.2019.2900419]
Zhang Y L, Tian Y P, Kong Y, Zhong B E and Fu Y. 2018. Residual dense network for image super-resolution//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, UT, USA: 2472-2481 [DOI: 10.1109/CVPR.2018.00262http://dx.doi.org/10.1109/CVPR.2018.00262]
Zhang Y T, Li X H, Zhou JL. 2019c. Sftgan: a generative adversarial network for pan-sharpening equipped with spatial feature transform layers. Journal of Applied Remote Sensing, 13(2): 026507:1-026507:16 [DOI: 10.1117/1.JRS.13.026507http://dx.doi.org/10.1117/1.JRS.13.026507]
Zhong J Y, Yang B, Huang G Y, Zhong Fei and Chen Z Z. 2016. Remote sensing image fusion with convolutional neural network. Sensing and Imaging, 17(1):6:1-6:16 [DOI: 10.1007/s11220-016-0135-6http://dx.doi.org/10.1007/s11220-016-0135-6]
Zhou M, Yan K Y, Huang J, Yang Z H, Fu X Y and Zhao F. 2022. Mutual information-driven pan-sharpening//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, Louisiana, USA: 1788-1798 [DOI: 10.1109/CVPR52688.2022.00184http://dx.doi.org/10.1109/CVPR52688.2022.00184]
Alert me when the article has been cited
提交
相关作者
暂无数据
相关机构
School of Computer Science and Technology, Tiangong University
Aerospace Information Research Institute, Chinese Academy of Sciences
School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences
Beijing Forestry University
Engineering Research Center for Forestry-oriented Intelligent Information Processing, National Forestry and Grassland Administration
选择翻译语言
- Postal code：100190
- Tel：010-58887052 Email：nrsb@aircas.ac.cn
- Technical support is provided by Beijing Founder electronics co., LTD 京ICP备20021838号-9
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
0
Bulk Citation
Export to
TOP