Früher|Nächster

Progress and bibliometric analysis of remote sensing data fusion methods (1992—2018)

  • role： First author第一作者
  • Affiliation：

    The Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100101, China

  • Email：zhanglf@radi.ac.cn
  • Introduction：张立福，1967年生，男，研究员，研究方向为高光谱数据处理技术。E-mail：zhanglf@radi.ac.cn
ZHANG Lifu1,  
  • Affiliation：

    The Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100101, China

    University of Chinese Academy of Sciences, Beijing 100049, China

PENG Mingyuan12,  
  • role： Corresponding author通信作者
  • Affiliation：

    The Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100101, China

  • Email：sunxj@radi.ac.cn
  • Introduction：孙雪剑，1987年生，博士，助理研究员，研究方向为高光谱数据融合。E-mail：sunxj@radi.ac.cn
SUN Xuejian1*,  
  • Affiliation：

    The Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100101, China

CEN Yi1,  
  • Affiliation：

    The Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100101, China

TONG Qingxi1

Resümee

Remote sensing applications have been promoting the development of satellite sensors and their performances. The satellite sensors are now possessing higher and higher data resolutions, and it in turn greatly facilitating remote sensing applications. The improvement of spatial resolution makes it possible to precisely record textures and spatial features of land covers. The improvement of spectral resolution makes it possible to precisely classify and retrieve parameters. The improvement of temporal resolution makes it easier to record the temporal changes of land covers during different phases. However, due to the limitation of sensors, satellite data can not possess high spatial resolution, high temporal resolution and high spectral resolution at the same time, which greatly hinders further remote sensing applications. Remote sensing data fusion is one of the effective solutions to deal with the problem of limitation in sensors’ resolutions, which is to integrate data from different sources and with different resolutions using algorithm methods to get richer information than one single image data. In recent years, the remote sensing data fusion methods have been greatly developed and related articles boost in growing numbers and importance. Thus, this article aims to systematically introduce the remote sensing data fusion and its current progress. Former reviews of remote sensing data fusion divides fusion methods into different categories. Based on processing levels, the remote sensing data fusion methods can be divided into pixel-level fusion, feature-level fusion and decision-level fusion. And based on the data sources, the remote sensing data fusion methods can be divided into homogeneous data fusion, heterogeneous data fusion, fusion for remote sensing observation and station data, and fusion for remote sensing observation and non-observed data. And this article adopts another category system which makes it more systematic and comprehensive in mathematical principles. In order to systematically indicate the current progress and developed history of remote sensing data fusion methods, this article first did bibliometric analysis on the remote sensing data fusion articles from data sources from Web of Science (WOS) and Chinese National Knowledge Infrastructure (CNKI). The data was preprocessed and has removed duplicates. Analysis including the aspects of yearly publishments, countries and organizations, journals and key words to see the history and trends have been done by Histcite, Bibliometric, Citespace and Endnote. Then the remote sensing fusion methods are systematically introduced. According to different resolution improvements, this article firstly divided the fusion methods into three categories: fusion methods emphasizing improvements of spatial resolution development, spectral development and temporal development, and based on that we further divide them into subcategories according to different mathematical principles. And the basic principles, some of the important methods, the advantages and flaws of these methods are systematically introduced in this section. Also, we did detailed instructions on fusion result assessment methods including methods with reference image and without reference image. Under these two categories we further introduced different assessment metrics and its applicable scenarios. And finally, this article made summaries of these fusion methods and described its features and directions. And finally makes predictions of future trends in remote sensing data fusion methods.

Schlüsselwort

remote sensing data;data fusion;image fusion;fusion quality assessment;document analysis

References

  1. 1.
    Adams J B, Smith M O and Johnson P E. 1986. Spectral mixture modeling: a new analysis of rock and soil types at the Viking Lander 1 sites. Journal of Geophysical Research: Solid Earth, 91(B8): 8098–8112
  2. 2.
    Aiazzi B, Baronti S and Selva M. 2007. Improving component substitution pansharpening through multivariate regression of MS+Pan data. IEEE Transactions on Geoscience and Remote Sensing, 45(10): 3230–3239
  3. 3.
    Alparone L, Aiazzi B, Baronti S, Garzelli A, Nencini F and Selva M. 2008. Multispectral and panchromatic data fusion assessment without reference. Photogrammetric Engineering and Remote Sensing, 74(2): 193–200
  4. 4.
    Alparone L, Wald L, Chanussot J, Thomas C, Gamba P and Bruce L M. 2007. Comparison of pansharpening algorithms: outcome of the 2006 GRS-S data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45(10): 3012–3021
  5. 5.
    Amorós-López J, Gómez-Chova L, Alonso L, Guanter L, Zurita-Milla R, Moreno J and Camps-Valls G. 2013. Multitemporal fusion of Landsat/TM and ENVISAT/MERIS for crop monitoring. International Journal of Applied Earth Observation and Geoinformation, 23: 132–141
  6. 6.
    Azarang A and Ghassemian H. 2017. A new pansharpening method using multi resolution analysis framework and deep neural networks//Proceedings of the 3rd International Conference on Pattern Recognition and Image Analysis. Shahrekord, Iran: IEEE: 1-6 [DOI: 10.1109/PRIA.2017.7983017]
  7. 7.
    Bendoumi M A, He M Y and Mei S H. 2014. Hyperspectral image resolution enhancement using high-resolution multispectral image based on spectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 52(10): 6574–6583
  8. 8.
    Chang C I. 1999. Spectral information divergence for hyperspectral image analysis//Proceedings of 1999 International Geoscience and Remote Sensing Symposium. Hamburg, Germany: IEEE [DOI: 10.1109/IGARSS.1999.773549]
  9. 9.
    Chavez P C Jr, Sides S C and Anderson J A. 1991. Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic. Photogrammetric Engineering and Remote Sensing, 57(3): 295–303
  10. 10.
    Chen Z, Pu H Y, Wang B and Jiang G M. 2014. Fusion of hyperspectral and multispectral images: a novel framework based on generalization of pan-sharpening methods. IEEE Geoscience and Remote Sensing Letters, 11(8): 1418–1422
  11. 11.
    Chen Z X, Ren J Q, Tang H J, Shi Y, Leng P, Liu J, Wang L M, Wu W B, Yao Y M and Hasiyuya. 2016. Progress and perspectives on agricultural remote sensing research and applications in China. Journal of Remote Sensing, 20(5): 748–767
  12. 12.
    Choi M. 2006. A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter. IEEE Transactions on Geoscience and Remote Sensing, 44(6): 1672–1682
  13. 13.
    Do M N and Vetterli M. 2005. The contourlet transform: an efficient directional multiresolution image representation. IEEE Transactions on Image Processing, 14(12): 2091–2106
  14. 14.
    Eismann M T and Hardie R C. 2003. Resolution enhancement of hyperspectral imagery using coincident panchromatic imagery and a stochastic mixing model//Proceedings of IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data. Greenbelt, MD, USA: IEEE: 282-289 [DOI: 10.1109/WARSD.2003.1295206]
  15. 15.
    Eismann M T and Hardie R C. 2005. Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions. IEEE Transactions on Geoscience and Remote Sensing, 43(3): 455–465
  16. 16.
    Elad M, Figueiredo M A T and Ma Y. 2010. On the role of sparse and redundant representations in image processing. Proceedings of the IEEE, 98(6): 972–982
  17. 17.
    Fu D J, Chen B Z, Wang J, Zhu X L and Hilker T. 2013. An improved image fusion approach based on enhanced spatial and temporal the adaptive reflectance fusion model. Remote Sensing, 5(12): 6346–6360
  18. 18.
    Gao F, Masek J, Schwaller M and Hall F. 2006. On the blending of the Landsat and MODIS surface reflectance: predicting daily Landsat surface reflectance. IEEE Transactions on Geoscience and Remote Sensing, 44(8): 2207–2218
  19. 19.
    Ge Z R, Wang B and Zhang L M. 2007. Remote sensing image fusion based on Bayesian linear estimation. Science in China Series F: Information Sciences, 50(2): 227–240
  20. 20.
    Ghassemian H. 2016. A review of remote sensing image fusion methods. Information Fusion, 32: 75–89
  21. 21.
    Gross H N and Schott J R. 1998. Application of spectral mixture analysis and image fusion techniques for image sharpening. Remote Sensing of Environment, 63(2): 85–94
  22. 22.
    Han C, Zhang H Y, Gao C X, Jiang C, Sang N and Zhang L P. 2016. A remote sensing image fusion method based on the analysis sparse model. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(1): 439–453
  23. 23.
    Hardie R C, Eismann M T and Wilson G L. 2004. MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor. IEEE Transactions on Image Processing, 13(9): 1174–1184
  24. 24.
    Hilker T, Wulder M A, Coops N C, Linke J, McDermid G, Masek J G, Gao F and White J C. 2009. A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS. Remote Sensing of Environment, 113(8): 1613–1627
  25. 25.
    Huang B and Song H H. 2012. Spatiotemporal reflectance fusion via sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 50(10): 3707–3716
  26. 26.
    Huang B, Song H H, Cui H B, Peng J G and Xu Z B. 2014. Spatial and spectral image fusion using sparse matrix factorization. IEEE Transactions on Geoscience and Remote Sensing, 52(3): 1693–1704
  27. 27.
    Huang B, Wang J, Song H H, Fu D J and Wong K. 2013. Generating high spatiotemporal resolution land surface temperature for urban heat island monitoring. IEEE Geoscience and Remote Sensing Letters, 10(5): 1011–1015
  28. 28.
    Huang W, Xiao L, Wei Z H, Liu H Y and Tang S Z. 2015. A new pan-sharpening method with deep neural networks. IEEE Geoscience and Remote Sensing Letters, 12(5): 1037–1041
  29. 29.
    Huang Z, Yu X C, Wang G A and Wang Z N. 2008. Application of several non-negative matrix factorization-based methods in remote sensing image fusion//Proceedings of the Fifth International Conference on Fuzzy Systems and Knowledge Discovery. Shandong, China: IEEE [DOI: 10.1109/FSKD.2008.439]
  30. 30.
    Huynh-Thu Q and Ghanbari M. 2008. Scope of validity of PSNR in image/video quality assessment. Electronics Letters, 44(13): 800–801
  31. 31.
    Jagalingam P and Hegde A V. 2015. A review of quality metrics for fused image. Aquatic Procedia, 4: 133–142
  32. 32.
    Li D, Hao M F, Zhang J Q, Hu B and Lu Q Y. 2012. A universal hypercomplex color image quality index//Proceedings of 2012 IEEE International Instrumentation and Measurement Technology Conference. Graz, Austria: IEEE: 985-990 [DOI: 10.1109/I2MTC.2012.6229639]
  33. 33.
    Li S T and Yang B. 2011. A new pan-sharpening method using a compressed sensing technique. IEEE Transactions on Geoscience and Remote Sensing, 49(2): 738–746
  34. 34.
    Liao C H, Wang J F, Pritchard I, Liu J G and Shang J L. 2017. A spatio-temporal data fusion model for generating NDVI time series in heterogeneous regions. Remote Sensing, 9(11): 1125
  35. 35.
    Liao L M, Song J L, Wang J D, Xiao Z Q and Wang J. 2016. Bayesian method for building frequent Landsat-like NDVI datasets by integrating MODIS and Landsat NDVI. Remote Sensing, 8(6): 452
  36. 36.
    Liu J G. 2000. Smoothing filter-based intensity modulation: a spectral preserve image fusion technique for improving spatial details. International Journal of Remote Sensing, 21(18): 3461–3472
  37. 37.
    Liu X, Deng C W, Wang S G, Huang G B, Zhao B J and Lauren P. 2016. Fast and accurate spatiotemporal fusion based upon extreme learning machine. IEEE Geoscience and Remote Sensing Letters, 13(12): 2039–2043
  38. 38.
    Lu M, Chen J, Tang H J, Rao Y H, Yang P and Wu W B. 2016. Land cover change detection by integrating object-based data blending model of Landsat and MODIS. Remote Sensing of Environment, 184: 374–386
  39. 39.
    Masi G, Cozzolino D, Verdoliva L and Scarpa G. 2016. Pansharpening by convolutional neural networks. Remote Sensing, 8(7): 594
  40. 40.
    Moosavi V, Talebi A, Mokhtari M H, Shamsi S R F and Niazi Y. 2015. A wavelet-artificial intelligence fusion approach (WAIFA) for blending Landsat and MODIS surface temperature. Remote Sensing of Environment, 169: 243–254
  41. 41.
    Nencini F, Garzelli A, Baronti S and Alparone L. 2007. Remote sensing image fusion using the curvelet transform. Information Fusion, 8(2): 143–156
  42. 42.
    Nunez J, Otazu X, Fors O, Prades A, Palà V and Arbiol R. 1999. Multiresolution-based image fusion with additive wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing, 37(3): 1204–1211
  43. 43.
    Palsson F, Sveinsson J R and Ulfarsson M O. 2017. Multispectral and hyperspectral image fusion using a 3-D-convolutional neural network. IEEE Geoscience and Remote Sensing Letters, 14(5): 639–643
  44. 44.
    Pohl C and Van Genderen J L. 1998. Review article Multisensor image fusion in remote sensing: concepts, methods and applications. International Journal of Remote Sensing, 19(5): 823–854
  45. 45.
    Schmitt M and Zhu X X. 2016. Data fusion and remote sensing: an ever-growing relationship. IEEE Geoscience and Remote Sensing Magazine, 4(4): 6–23
  46. 46.
    Shettigara V K. 1992. A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set. Photogrammetric Engineering and Remote Sensing, 58(5): 561–567
  47. 47.
    Song H H, Liu Q S, Wang G J, Hang R L and Huang B. 2018. Spatiotemporal satellite image fusion using deep convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3): 821–829
  48. 48.
    Sun X J, Zhang L F, Yang H, Wu T X, Cen Y and Guo Y. 2015. Enhancement of spectral resolution for remotely sensed multispectral image. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(5): 2198–2211
  49. 49.
    Tan Z Y, Yue P, Di L P and Tang J M. 2018. Deriving high spatiotemporal remote sensing images using deep convolutional network. Remote Sensing, 10(7): 1066
  50. 50.
    Tang H J, Wu W B, Yang P, Zhou Q B and Chen Z X. 2010. Recent progresses in monitoring crop spatial patterns by using remote sensing technologies. Scientia Agricultura Sinica, 43(14): 2879–2888
  51. 51.
    Thomas C and Wald L. 2006. Analysis of changes in quality assessment with scale//Proceedings of the 9th International Conference on Information Fusion. Florence, Italy: IEEE [DOI: 10.1109/ICIF.2006.301595]
  52. 52.
    Tong Q X, Xue Y Q and Zhang L F. 2014. Progress in hyperspectral remote sensing science and technology in China over the past three decades. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(1): 70–91
  53. 53.
    Tu T M, Lee Y C, Chang C P and Huang P S. 2005. Adjustable intensity-hue-saturation and Brovey transform fusion technique for IKONOS/QuickBird imagery. Optical Engineering, 44(11): 116201
  54. 54.
    Tu T M, Su S C, Shyu H C and Huang P S. 2001. A new look at IHS-like image fusion methods. Information Fusion, 2(3): 177–186
  55. 55.
    Wald L, Ranchin T, Mangolini M. 1997. Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images. Photogrammetric Engineering and Remote Sensing, 63(6): 691–699
  56. 56.
    Wang J and Huang B. 2017. A rigorously-weighted spatiotemporal fusion model with uncertainty analysis. Remote Sensing, 9(10): 990
  57. 57.
    Wang J Y, Wang Y M and Li C L. 2010. Noise model of hyperspectral imaging system and influence on radiation sensitivity. Journal of Remote Sensing, 14(4): 607–620
  58. 58.
    Wang Z, Bovik A C, Sheikh H R and Simoncelli E P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600–612
  59. 59.
    Wei J B, Wang L Z, Liu P, Chen X D, Li W and Zomaya A Y. 2017a. Spatiotemporal fusion of MODIS and Landsat-7 reflectance images via compressed sensing. IEEE Transactions on Geoscience and Remote Sensing, 55(12): 7126–7139
  60. 60.
    Wei J B, Wang L Z, Liu P and Song W J. 2017b. Spatiotemporal fusion of remote sensing images with structural sparsity and semi-coupled dictionary learning. Remote Sensing, 9(1): 21
  61. 61.
    Wei Q, Bioucas-Dias J, Dobigeon N and Tourneret J Y. 2015. Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 53(7): 3658–3668
  62. 62.
    Wei Q, Dobigeon N and Tourneret J Y. 2014a. Bayesian fusion of hyperspectral and multispectral images//Proceedings of 2014 IEEE International Conference on Acoustics, Speech and Signal Processing. Florence, Italy: IEEE: 3176-3180 [DOI: 10.1109/ICASSP.2014.6854186]
  63. 63.
    Wei Q, Dobigeon N and Tourneret J Y. 2014b. Bayesian fusion of multispectral and hyperspectral images with unknown sensor spectral response//Proceedings of 2014 IEEE International Conference on Image Processing. Paris, France: IEEE: 698-702 [DOI: 10.1109/ICIP.2014.7025140]
  64. 64.
    Wei Y C, Yuan Q Q, Shen H F and Zhang L P. 2017. Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geoscience and Remote Sensing Letters, 14(10): 1795–1799
  65. 65.
    Weng Q H, Fu P and Gao F. 2014. Generating daily land surface temperature at Landsat resolution by fusing Landsat and MODIS data. Remote Sensing of Environment, 145: 55–67
  66. 66.
    Wu B, Huang B and Zhang L P. 2015. An error-bound-regularized sparse coding for spatiotemporal reflectance fusion. IEEE Transactions on Geoscience and Remote Sensing, 53(12): 6791–6803
  67. 67.
    Wu M Q, Huang W J, Niu Z and Wang C Y. 2015. Generating daily synthetic Landsat imagery by combining Landsat and MODIS data. Sensors, 15(9): 24002–24025
  68. 68.
    Wu M Q, Niu Z, Wang C Y, Wu C Y and Wang L. 2012. Use of MODIS and Landsat time series data to generate high-resolution temporal synthetic Landsat data using a spatial and temporal reflectance fusion model. Journal of Applied Remote Sensing, 6(1): 063507
  69. 69.
    Wu P H, Shen H F, Zhang L P and Göttsche F M. 2015. Integrated fusion of multi-scale polar-orbiting and geostationary satellite observations for the mapping of high spatial and temporal resolution land surface temperature. Remote Sensing of Environment, 156: 169–181
  70. 70.
    Xue J, Leung Y and Fung T. 2017. A Bayesian data fusion approach to spatio-temporal fusion of remotely sensed images. Remote Sensing, 9(12): 1310
  71. 71.
    Yokoya N, Yairi T and Iwasaki A. 2012. Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Transactions on Geoscience and Remote Sensing, 50(2): 528–537
  72. 72.
    Yuan Q Q, Wei Y C, Meng X C, Shen H F and Zhang L P. 2018. A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3): 978–989
  73. 73.
    Yusuf Y, Sumantyo J T S and Kuze H. 2013. Spectral information analysis of image fusion data for remote sensing applications. Geocarto International, 28(4): 291–310
  74. 74.
    Zhang L P and Shen H F. 2016. Progress and future of remote sensing data fusion. Journal of Remote Sensing, 20(5): 1050–1061
  75. 75.
    Zhang W, Li A N, Jin H A, Bian J H, Zhang Z J, Lei G B, Qin Z H and Huang C Q. 2013. An enhanced spatial and temporal data fusion model for fusing Landsat and MODIS surface reflectance to generate high temporal Landsat-like data. Remote Sensing, 5(10): 5346–5368
  76. 76.
    Zhang Y F. 2012. Wavelet-based Bayesian fusion of multispectral and hyperspectral images using Gaussian scale mixture model. International Journal of Image and Data Fusion, 3(1): 23–37
  77. 77.
    Zhang Y F, De Backer S and Scheunders P. 2009. Noise-resistant wavelet-based Bayesian fusion of multispectral and hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 47(11): 3834–3843
  78. 78.
    Zhu X L, Cai F Y, Tian J Q and Williams T K A. 2018. Spatiotemporal fusion of multisource remote sensing data: literature survey, taxonomy, principles, applications, and future directions. Remote Sensing, 10(4): 527
  79. 79.
    Zhu X L, Chen J, Gao F, Chen X H and Masek J G. 2010. An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions. Remote Sensing of Environment, 114(11): 2610–2623
  80. 80.
    Zhukov B, Oertel D, Lanzl F and Reinhackel G. 1999. Unmixing-based multisensor multiresolution image fusion. IEEE Transactions on Geoscience and Remote Sensing, 37(3): 1212–1226
  81. 81.
    Zurita-Milla R, Clevers J G P W and Schaepman M E. 2008. Unmixing-based Landsat TM and MERIS FR data fusion. IEEE Geoscience and Remote Sensing Letters, 5(3): 453–457

Lesen Sie die ganze Passage

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website