سابِق|تال

Development of deep learning-based hyperspectral remote sensing image unmixing

  • role： First author第一作者
  • Affiliation：

    College of Geomatic, Xi'an University of Science and Technology, Xi'an 710054, China

    Key Laboratory of Computational Optical Imaging Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China

  • Email：suych3@xust.edu.cn
  • Introduction：苏远超，研究方向为高光谱遥感数据信息提取、深度学习。E-mail：suych3@xust.edu.cn
SU Yuanchao12,  
  • Affiliation：

    College of Geomatic, Xi'an University of Science and Technology, Xi'an 710054, China

XU Ruoqing1,  
  • role： Corresponding author通信作者
  • Affiliation：

    Key Laboratory of Computational Optical Imaging Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China

  • Email：gaolr@aircas.ac.cn
  • Introduction：高连如，研究方向为高光谱遥感图像处理与信息提取。E-mail：gaolr@aircas.ac.cn
GAO Lianru2*,  
  • Affiliation：

    Key Laboratory of Computational Optical Imaging Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China

    University of Chinese Academy of Sciences, Beijing 100049, China

HAN Zhu23,  
  • Affiliation：

    Key Laboratory of Computational Optical Imaging Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China

SUN Xu2

ملخص

Hyperspectral remote sensing is an advanced technique for earth observation that combines physical imagery and spectral analysis technology. Therefore, hyperspectral remote sensing can obtain fine spectral and rich spatial information from imaged scenes, merging the spatial and spectral information into data cubes. These data cubes exhibit narrow spectral bands and a high spectral resolution, allowing different land cover objects to be distinguished. Hyperspectral remote sensing images, with their high spectral resolution and cube characteristics, have gradually become among the most essential supporting data in remote sensing engineering applications. However, due to spatial resolution limitations, the mixed pixel problem has hindered the development of hyperspectral remote sensing in fine-scale object information extraction. At present, hyperspectral unmixing is one of the most effective analytical techniques for dealing with mixed pixel problems, aiming to break through spatial resolution limitations by analyzing the components within pixels. Hyperspectral unmixing refers to any process that separates pixel spectra from a hyperspectral image into a collection of pure constituent spectra, called endmembers, and a set of corresponding abundance fractions. At each pixel, the endmembers are generally assumed to represent the pure materials in the scene, while the abundances represent the percentage of each endmember. For the fine-scale interpretation of object information, many unmixing methods have been developed for hyperspectral remote sensing images in the remote sensing field over the past 30 years, mitigating the impact of mixed pixel problems on quantitative remote sensing analysis. Currently, with the development of deep learning, an increasing number of deep learning theories and tools are used to deal with mixed pixel problems. Many new methods using deep learning for unmixing have been developed, and unmixing technology research has gradually entered a new stage of development with deep learning. Deep-learning-based methods make better use of hidden information, have a relatively lower dependence on prior knowledge, and have a stronger adaptability to complex scenes than traditional unmixing methods. Although deep learning-based unmixing methods have developed rapidly in recent years and are diverse, the analysis and summary of the work on such methods have not kept up with the pace of technological development. A timely summary of the latest research progress on developing a specific field of research has a significant role in promoting the technology. Thus, this paper sorts out the existing deep learning-based unmixing methods, classifying them according to the adopted spectral mixing models, the deep network training modes, and whether spectral variability is considered. Furthermore, this paper introduces these deep learning-based approaches and summarizes their characteristics, making the use of these methods in special works convenient for users or readers. Finally, the development of deep learning methods is summarized, referring to the current technical status, characteristics, and development prospects. In addition, some existing deep learning unmixing methods were tested in this study and organized to facilitate the research and application of unmixing technology. The development of deep learning will continue to promote the progress of unmixing techniques. In recent years, deep learning-based unmixing methods have developed rapidly and have been gradually used in vegetation distribution investigation and agricultural yield estimation, implying their good development prospect and application value. his paper can provide valuable references for researching unmixing technology in the future.

مفهوم

hyperspectral remote sensing;unmixing;deep learning;machine learning;deep neural network;remote sensing image processing;remote sensing intelligent interpretation;subpixel interpretation

References

  1. 1.
    Bioucas-Dias J M, Plaza A, Camps-Valls G, Scheunders P, Nasrabadi N and Chanussot J. 2013. Hyperspectral remote sensing data analysis and future challenges. IEEE Geoscience and Remote Sensing Magazine, 1(2): 6-36
  2. 2.
    Bioucas-Dias J M, Plaza A, Dobigeon N, Parente M, Du Q, Gader P and Chanussot J. 2012. Hyperspectral unmixing overview: geometrical, statistical, and sparse regression-based approaches. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 5(2): 354-379
  3. 3.
    Borsoi R A, Imbiriba T and Bermudez J C M. 2020. Deep generative endmember modeling: an application to unsupervised spectral unmixing. IEEE Transactions on Computational Imaging, 6: 374-384
  4. 4.
    Chen J, Gamba P and Li J. 2023a. MAHUM: a multitasks autoencoder hyperspectral unmixing model. IEEE Transactions on Geoscience and Remote Sensing, 61: 5519516
  5. 5.
    Chen J, Zhao M, Wang X H, Richard C and Rahardja S. 2023b. Integration of physics-based and data-driven models for hyperspectral image unmixing: a summary of current methods. IEEE Signal Processing Magazine, 40(2): 61-74
  6. 6.
    Cui C Y, Wang X Y, Wang S Y, Zhang L P and Zhong Y F. 2023. Unrolling nonnegative matrix factorization with group sparsity for blind hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 61: 5516712
  7. 7.
    Dobigeon N, Tourneret J Y, Richard C, Bermudez J C M, McLaughlin S and Hero A O. 2014. Nonlinear unmixing of hyperspectral images: models and algorithms. IEEE Signal Processing Magazine, 31(1): 82-94
  8. 8.
    Dou Z Y, Gao K, Zhang X D, Wang H and Wang J W. 2020. Hyperspectral unmixing using orthogonal sparse prior-based autoencoder with hyper-Laplacian loss and data-driven outlier detection. IEEE Transactions on Geoscience and Remote Sensing, 58(9): 6550-6564
  9. 9.
    Fang H, Li A H, Xu H X and Wang T. 2018. Sparsity-constrained deep nonnegative matrix factorization for hyperspectral unmixing. IEEE Geoscience and Remote Sensing Letters, 15(7): 1105-1109
  10. 10.
    Fang Y, Wang Y X, Xu L L, Zhuo R M, Wong A and Clausi D A. 2022. BCUN: Bayesian fully convolutional neural network for hyperspectral spectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 60: 5523714
  11. 11.
    Feng X R, Li H C, Li J, Du Q, Plaza A and Emery W J. 2018. Hyperspectral unmixing using sparsity-constrained deep nonnegative matrix factorization with total variation. IEEE Transactions on Geoscience and Remote Sensing, 56(10): 6245-6257
  12. 12.
    Feng X R, Li H C, Liu S and Zhang H. 2022. Correntropy-based autoencoder-like NMF with total variation for hyperspectral unmixing. IEEE Geoscience and Remote Sensing Letters, 19: 5500505
  13. 13.
    Gao L R, Han Z, Hong D F, Zhang B and Chanussot J. 2022. CyCU-Net: cycle-consistency unmixing network by learning cascaded autoencoders. IEEE Transactions on Geoscience and Remote Sensing, 60: 5503914
  14. 14.
    Ghosh P, Roy S K, Koirala B, Rasti B and Scheunders P. 2022. Hyperspectral unmixing using transformer network. IEEE Transactions on Geoscience and Remote Sensing, 60: 5535116
  15. 15.
    Guo R, Wang W and Qi H R. 2015. Hyperspectral image unmixing using autoencoder cascade//2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS). Tokyo: IEEE: 1-4
  16. 16.
    Han Z, Gao L R, Zhang B, Sun X and Li Q T. 2020. Nonlinear hyperspectral unmixing algorithm for GF-5 satellite based on deep autoencoder networks. Journal of Remote Sensing (Chinese), 24(4): 388-400
  17. 17.
    Han Z, Hong D F, Gao L R, Yao J, Zhang B and Chanussot J. 2022a. Multimodal hyperspectral unmixing: insights from attention networks. IEEE Transactions on Geoscience and Remote Sensing, 60: 5524913
  18. 18.
    Han Z, Hong D F, Gao L R, Zhang B, Huang M and Chanussot J. 2022b. AutoNAS: automatic neural architecture search for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 60: 5532214
  19. 19.
    Hapke B. 1981. Bidirectional reflectance spectroscopy: 1. Theory. Journal of Geophysical Research: Solid Earth, 86(B4): 3039-3054
  20. 20.
    He M Y, Chang W J, Mei S H. 2013. Advance in Feature Mining from Hyperspectral Remote Sensing Data. Spacecraft Recovery and remote sensing, 34(1): 1-12.
  21. 21.
    Heylen R, Parente M and Gader P. 2014. A review of nonlinear hyperspectral unmixing methods. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6): 1844-1868
  22. 22.
    Hong D F, Gao L R, Yao J, Yokoya N, Chanussot J, Heiden U and Zhang B. 2022. Endmember-guided unmixing network (EGU-Net): a general deep learning framework for self-supervised hyperspectral unmixing. IEEE Transactions on Neural Networks and Learning Systems, 33(11): 6518-6531
  23. 23.
    Horn R A and Johnson C R. 2012. Matrix Analysis. 2nd ed. Cambridge: Cambridge University Press
  24. 24.
    Iordache M D, Bioucas-Dias J M and Plaza A. 2011. Sparse unmixing of hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing, 49(6): 2014-2039
  25. 25.
    Jin Q W, Ma Y, Fan F, Huang J, Mei X G and Ma J Y. 2023. Adversarial autoencoder network for hyperspectral unmixing. IEEE Transactions on Neural Networks and Learning Systems, 34(8): 4555-4569
  26. 26.
    Jin Q W, Ma Y, Mei X G and Ma J Y. 2022. TANet: an unsupervised two-stream autoencoder network for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 60: 5506215
  27. 27.
    Keshava N and Mustard J F. 2002. Spectral unmixing. IEEE Signal Processing Magazine, 19(1): 44-57
  28. 28.
    Khajehrayeni F and Ghassemian H. 2020. Hyperspectral unmixing using deep convolutional autoencoders in a supervised scenario. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 567-576
  29. 29.
    Khajehrayeni F and Ghassemian H. 2021. A linear hyperspectral unmixing method by means of autoencoder networks. International Journal of Remote Sensing, 42(7): 2517-2531
  30. 30.
    Koirala B, Khodadadzadeh M, Contreras C, Zahiri Z, Gloaguen R and Scheunders P. 2019. A supervised method for nonlinear hyperspectral unmixing. Remote Sensing, 11(20): 2458
  31. 31.
    Koirala B, Zahiri Z, Lamberti A and Scheunders P. 2021. Robust supervised method for nonlinear spectral unmixing accounting for endmember variability. IEEE Transactions on Geoscience and Remote Sensing, 59(9): 7434-7448
  32. 32.
    Kong F Q, Chen M Y, Li Y S and Li D. 2022. A global spectral–spatial feature learning network for semisupervised hyperspectral unmixing. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15: 3190-3203
  33. 33.
    Kong F Q, Zheng Y H, Li D, Li Y S and Chen M Y. 2023. Window transformer convolutional autoencoder for hyperspectral sparse unmixing. IEEE Geoscience and Remote Sensing Letters, 20: 5508305
  34. 34.
    Li H C, Feng X R, Zhai D H, Du Q and Plaza A. 2022. Self-supervised robust deep matrix factorization for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 60: 5513214
  35. 35.
    Li M L, Zhu F, Guo A J X and Chen J. 2019. A graph regularized multilinear mixing model for nonlinear hyperspectral unmixing. Remote Sensing, 11(19): 2188
  36. 36.
    Min A Y, Guo Z Y, Li H and Peng J T. 2022. JMnet: joint metric neural network for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 60: 5505412
  37. 37.
    Nobbs J H. 1985. Kubelka—Munk theory and the prediction of reflectance. Review of Progress in Coloration and Related Topics, 15(1): 66-75
  38. 38.
    Ozkan S, Kaya B and Akar G B. 2019a. EndNet: sparse AutoEncoder network for endmember extraction and hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 57(1): 482-496
  39. 39.
    Ozkan S and Akar G B. 2019b. Improved deep spectral convolution network for hyperspectral unmixing with multinomial mixture kernel and endmember uncertainty. arXiv preprint arXiv: 1808.01104
  40. 40.
    Palsson B, Sigurdsson J, Sveinsson J R and Ulfarsson M O. 2018. Hyperspectral unmixing using a neural network autoencoder. IEEE Access, 6: 25646-25656
  41. 41.
    Palsson B, Sveinsson J R and Ulfarsson M O. 2019. Spectral-spatial hyperspectral unmixing using multitask learning. IEEE Access, 7: 148861-148872
  42. 42.
    Palsson B, Ulfarsson M O and Sveinsson J R. 2021. Convolutional autoencoder for spectral–spatial hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 59(1): 535-549
  43. 43.
    Palsson F, Sigurdsson J, Sveinsson J R and Ulfarsson M O. 2017. Neural network hyperspectral unmixing with spectral information divergence objective//2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Fort Worth: IEEE: 755-758
  44. 44.
    Qi L, Chen Z W, Gao F, Dong J Y, Gao X B and Du Q. 2023. Multiview spatial-spectral two-stream network for hyperspectral image unmixing. IEEE Transactions on Geoscience and Remote Sensing, 61: 5502016
  45. 45.
    Qi L, Gao F, Dong J Y, Gao X B and Du Q. 2022. SSCU-Net: spatial-spectral collaborative unmixing network for hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 60: 5407515
  46. 46.
    Qian Y T, Xiong F C, Qian Q P and Zhou J. 2020. Spectral mixture model inspired network architectures for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 58(10): 7418-7434
  47. 47.
    Qu Y and Qi H R. 2019. uDAS: an untied denoising autoencoder with sparsity for spectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 57(3): 1698-1712
  48. 48.
    Rasti B and Koirala B. 2022a. SUnCNN: sparse unmixing using unsupervised convolutional neural network. IEEE Geoscience and Remote Sensing Letters, 19: 5508205
  49. 49.
    Rasti B, Koirala B and Scheunders P. 2022b. HapkeCNN: blind nonlinear unmixing for intimate mixtures using hapke model and convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 60: 5536315
  50. 50.
    Rasti B, Koirala B, Scheunders P and Chanussot J. 2022c. MiSiCNet: minimum simplex convolutional network for deep hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 60: 5522815
  51. 51.
    Rasti B, Koirala B, Scheunders P and Ghamisi P. 2022d. UnDIP: hyperspectral unmixing using deep image prior. IEEE Transactions on Geoscience and Remote Sensing, 60: 5504615
  52. 52.
    Shahid K T and Schizas I D. 2022. Unsupervised hyperspectral unmixing via nonlinear autoencoders. IEEE Transactions on Geoscience and Remote Sensing, 60: 5506513
  53. 53.
    Shao Y T, Liu Q C and Xiao L. 2023. IVIU-Net: implicit variable iterative unrolling network for hyperspectral sparse unmixing. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16: 1756-1770
  54. 54.
    Shi S K, Zhao M, Zhang L J, Altmann Y and Chen J. 2022. Probabilistic generative model for hyperspectral unmixing accounting for endmember variability. IEEE Transactions on Geoscience and Remote Sensing, 60: 5516915
  55. 55.
    Somers B, Cools K, Delalieux S, Stuckens J, Van der Zande D, Verstraeten W W and Coppin P. 2009. Nonlinear hyperspectral mixture analysis for tree cover estimates in orchards. Remote Sensing of Environment, 113(6): 1183-1193
  56. 56.
    Su Y C, Li J, Plaza A, Marinoni A, Gamba P and Chakravortty S. 2019. DAEN: deep autoencoder networks for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 57(7): 4309-4321
  57. 57.
    Su Y C, Marinoni A, Li J, Plaza J and Gamba P. 2018. Stacked nonnegative sparse autoencoders for robust hyperspectral unmixing. IEEE Geoscience and Remote Sensing Letters, 15(9): 1427-1431
  58. 58.
    Su Y C, Xu X, Li J, Qi H R, Gamba P and Plaza A. 2021. Deep autoencoders with multitask learning for bilinear hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 59(10): 8615-8629
  59. 59.
    Tang M F, Qu Y and Qi H R. 2020. Hyperspectral nonlinear unmixing via generative adversarial network//IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa: IEEE: 2404-2407
  60. 60.
    Tong Q X, Zhang B and Zhang L F. 2016. Current progress of hyperspectral remote sensing in China. Journal of Remote Sensing, 20(5): 689-707
  61. 61.
    Wan L L, Chen T, Plaza A and Cai H J. 2021. Hyperspectral unmixing based on spectral and sparse deep convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14: 11669-11682
  62. 62.
    Wang L G, Wang L F, Wang Q M and Bruzzone L. 2022. RSCNet: a residual self-calibrated network for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 5529917
  63. 63.
    Wang M, Zhao M, Chen J and Rahardja S. 2019. Nonlinear unmixing of hyperspectral data via deep autoencoder networks. IEEE Geoscience and Remote Sensing Letters, 16(9): 1467-1471
  64. 64.
    Wang Z C, Zhuang L N, Gao L R, Marinoni A, Zhang B and Ng M K. 2020. Hyperspectral nonlinear unmixing by using plug-and-play prior for abundance maps. Remote Sensing, 12(24): 4117
  65. 65.
    Xiong F C, Zhou J, Tao S Y, Lu J F and Qian Y T. 2022. SNMF-Net: learning a deep alternating neural network for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 60: 5510816
  66. 66.
    Xu X, Song X Y, Li T, Shi Z W and Pan B. 2022. Deep autoencoder for hyperspectral unmixing via global-local smoothing. IEEE Transactions on Geoscience and Remote Sensing, 60: 5524216
  67. 67.
    Yang B and Wang B. 2018. Band-wise nonlinear unmixing for hyperspectral imagery using an extended multilinear mixing model. IEEE Transactions on Geoscience and Remote Sensing, 56(11): 6747-6762
  68. 68.
    Zhang B. 2016. Advancement of hyperspectral image processing and information extraction. Journal of Remote Sensing, 20(5): 1062-1090
  69. 69.
    Zhang B and Sun X. 2015. Hyperspectral images unmixing[M]. Beijing: Science Press
  70. 70.
    Zhang L P, Du B and Zhang L F. 2014. Hyperspectral Image Processing[M]. Beijing: Science Press
  71. 71.
    Zhang X R, Sun Y J, Zhang J Y, Wu P and Jiao L C. 2018. Hyperspectral unmixing via deep convolutional neural networks. IEEE Geoscience and Remote Sensing Letters, 15(11): 1755-1759
  72. 72.
    Zhao M, Wang M, Chen J and Rahardja S. 2021a. Hyperspectral unmixing via deep autoencoder networks for a generalized linear-mixture/nonlinear-fluctuation model. arXiv preprint arXiv: 1904.13017
  73. 73.
    Zhao M, Yan L B and Chen J. 2021b. LSTM-DNN based autoencoder network for nonlinear hyperspectral image unmixing. IEEE Journal of Selected Topics in Signal Processing, 15(2): 295-309
  74. 74.
    Zhao M, Shi S K, Chen J and Dobigeon N. 2022a. A 3-D-CNN framework for hyperspectral unmixing with spectral variability. IEEE Transactions on Geoscience and Remote Sensing, 60: 5521914
  75. 75.
    Zhao M, Wang M, Chen J and Rahardja S. 2022b. Hyperspectral unmixing for additive nonlinear models with a 3-D-CNN autoencoder network. IEEE Transactions on Geoscience and Remote Sensing, 60: 5509415
  76. 76.
    Zhao M, Wang X H, Chen J and Chen W. 2022c. A plug-and-play priors framework for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 60: 5501213
  77. 77.
    Zhou C and Rodrigues M R D. 2022. ADMM-based hyperspectral unmixing networks for abundance and endmember estimation. IEEE Transactions on Geoscience and Remote Sensing, 60: 5520018
  78. 78.
    Zhu Q Q, Deng W H, Zheng Z, Zhong Y F, Guan Q F, Lin W H, Zhang L P and Li D R. 2022. A spectral-spatial-dependent global learning framework for insufficient and imbalanced hyperspectral image classification. IEEE Transactions on Cybernetics, 52(11): 11709-11723
  79. 79.
    Zhu Z Q, Su Y C, Li P F, Bai J Y, Liu Y and Liu F. 2023. Spectral-spatial hyperspectral unmixing using deep double-constraints convolutional network. Journal of Signal Processing, 39(1): 128-142
  80. 80.
    Zhuang L N. 2015. Hyperspectral Mixture Analysis Incorporating Endmember Variability［D］.Beijing：Chinese Academy of Sciences

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website