предыдущий|следующий

Deep fusion of hyperspectral images and multi-source remote sensing data for classification with convolutional neural network

  • role： First author第一作者
  • Affiliation：

    China Remote Sensing Satellite Ground Station, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100094, China

    School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100049, China

  • Email：zhaowd@radi.ac.cn
  • Introduction：赵伍迪， 1992年生， 女， 博士研究生， 研究方向为高光谱数据分类。E-mail： zhaowd@radi.ac.cn
ZHAO Wudi12,  
  • role： Corresponding author通信作者
  • Affiliation：

    China Remote Sensing Satellite Ground Station, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100094, China

  • Email：lishanshan@aircas.ac.cn
  • Introduction：李山山， 1979年生， 副研究员， 研究方向为高光谱图像处理。E-mail： lishanshan@aircas.ac.cn
LI Shanshan1*,  
  • Affiliation：

    China Remote Sensing Satellite Ground Station, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100094, China

LI An1,  
  • Affiliation：

    Key Laboratory of Digital Earth Science, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100094, China

Zhang Bing3,  
  • Affiliation：

    China Remote Sensing Satellite Ground Station, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100094, China

Chen Jun1

реферат

Hyperspectral images (HSIs) have abundant spectral characteristics. However, their spatial resolution is relatively low. Some remote sensing data have complementary advantages with HSIs, such as the LiDAR, which can provide elevation information, and the high spatial resolution data, which have precise spatial information. The combination of HSIs with the multi-source remote sensing data for fusion classification can make up the deficiency of tits relatively low spatial resolution. In recent years, deep learning-based methods have been investigated for hyperspectral remote sensing classification and have made breakthroughs. Meanwhile, the feature extraction process of the deep network is an independent process. Therefore, it may not obtain the most beneficial features for classification accurately and may influence the classification accuracy. At the same time, the techniques may not perform well when using limited training samples in HSIs because of massive parameters and complex network structure.Aiming at this problem, the frequently used traditional features of remote sensing data for classification are discussed in this paper. A new deep learning-based feature level data fusion classification framework that integrates traditional textural features into Convolutional Neural Network (CNN) approach (T-F-CNN) is proposed for the accurate fusion classification of HSIs and multi-source remote sensing data. The proposed method can be implemented in three steps. First, the traditional features are extracted from the HSIs or the multi-source remote sensing data. Second, CNN are built. The original HSIs, the original multi-source remote sensing data, and the traditional features, which are obtained in the first step, are inputted into the CNN of the deep feature extraction. Finally, the deep features obtained in the second step are concatenated in a concatenate layer of CNN, and SoftMax is used to generate classification maps at the end of the framework.Result The proposed classification scheme is tested on two data sets, namely, Houston and Thetford Mines Area data sets. The proposed T-F-CNN is compared with the pixel-level methods, such as Support Vector Machines (SVM) with the Radial Basis Function (RBF) (T-P-SVM), the CNN fusion method (P-CNN), and the CNN with traditional features (T-P-CNN); and the feature-level methods, such as CNN fusion method (F-CNN) and CNN method combined with original traditional features (T’-F-CNN). On both data sets, the proposed method shows a higher classification accuracy than other methods. Meanwhile, when the training samples reach a minimum number, the proposed method could provide the highest overall classification accuracy.The results obtained by the proposed method on the two real hyperspectral data sets demonstrate that the classification accuracy can be improved. Furthermore, the proposed T-F-CNN method outperforms some traditional deep learning methods and exhibits higher computing efficiency than a few advanced deep learning techniques.

ключеви́че слова́

Convolutional Neural Network (CNN);hyperspectral image;Light Detection and Ranging (LiDAR);high spatial resolution remote sensing data;data fusion;Gray Level Co-occurrence Matrix (GLCM);classification

References

  1. 1.
    Cao Q, Ma A L, Zhong Y F, Zhao J, Zhao B, and Zhang L P. 2019. Urban classification by multi-feature fusion of hyperspectral image and LiDAR data. Journal of Remote Sensing, 23(5):892-903.
  2. 2.
    Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10): 6232-6251
  3. 3.
    Chen Y S, Li C Y, Ghamisi P, Jia X P and Gu Y F. 2017. Deep fusion of remote sensing data for accurate classification. IEEE Geoscience and Remote Sensing Letters, 14(8): 1253-1257
  4. 4.
    Cui B G, Wu Y N, Zhong Y, Zhong L W and Lu Y. 2019. Hyperspectral image rolling guidance recursive filtering and classification. Journal of Remote Sensing, 23(3): 431-442
  5. 5.
    Dai Q L, Luo B, Zhen C, and Wang L G. 2020. Regional multiscale Markov random field for remote sensing image classification. Journal of Remote Sensing, 24(3):245-253
  6. 6.
    Dian Y Y, Li Z Y and Pang Y. 2015. Spectral and texture features combined for forest tree species classification with airborne hyperspectral imagery. Journal of the Indian Society of Remote Sensing, 43(1): 101-107
  7. 7.
    Fauvel M, Benediktsson J A, Chanussot J and Sveinsson J R. 2008. Spectral and Spatial Classification of Hyperspectral Data Using SVMs and Morphological Profiles. IEEE Transactions Geoscience and Remote Sensing, 46(11):3804-3814
  8. 8.
    Feng M B, Liu X and Zhao D. 2014. A fusion method of hyperspectral and multispectral images based on projection and wavelet transformation. Acta Geodaetica et Cartographica Sinica, 43(2): 158-163
  9. 9.
    Ghamisi P, Benediktsson J, Cavallaro G and Plaza A. 2014. Automatic Framework for Spectral-Spatial Classification based on Supervised Feature Extraction and Morphological Attribute Profiles. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6):2147-2160
  10. 10.
    Gao Q S, Lim S and Jia X P. 2018. Hyperspectral image classification using convolutional neural networks and multiple feature learning. Remote Sensing, 10(2): 299
  11. 11.
    Hu W, Huang Y Y, Wei L, Zhang F and Li H C. 2015. Deep convolutional neural networks for hyperspectral image classification. Journal of Sensors, 2015: 258619
  12. 12.
    Imani M and Ghassemian H. 2016. GLCM, Gabor, and morphology profiles fusion for hyperspectral image classification//Proceedings of the 2016 24th Iranian Conference on Electrical Engineering. Shiraz, Iran: IEEE, 460-465
  13. 13.
    Kaufman J R, Eismann M T and Celenk M. 2015. Assessment of spatial-spectral feature-level fusion for hyperspectral target detection. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(6): 2534-2544
  14. 14.
    Kumar B and Dikshit O. 2015. Spectral–Spatial Classification of Hyperspectral Imagery Based on Moment Invariants. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(6):2457-2463
  15. 15.
    Liang L, Yang M H and Li Y F. 2010. Hyperspectral remote sensing image classification based on ICA and SVM algorithm. Spectroscopy and Spectral Analysis, 30(10): 2724-2728
  16. 16.
    Li J, Xi B, Li Y, Du Q and Wang K. 2018. Hyperspectral Classification Based on Texture Feature Enhancement and Deep Belief Networks. Remote Sensing, 10(3):396-416
  17. 17.
    Mahmood Z, Akhter M A, Thoonen G and Scheunders P. 2013. Contextual subpixel mapping of hyperspectral images making use of a high resolution color image. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 6(2): 779-791
  18. 18.
    Man Q X, Dong P L and Guo H D. 2015. Pixel- and feature-level fusion of hyperspectral and lidar data for urban land-use classification. International Journal of Remote Sensing, 36(6): 1618-1644
  19. 19.
    梅安新, 彭望琭, 秦其明, 刘慧平. 2001. 遥感导论. 北京: 高等教育出版社
  20. 20.
    Merentitis A, Debes C and Heremans R. 2014. Ensemble learning in hyperspectral image classification: toward selecting a favorable bias-variance tradeoff. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(4): 1089-1102
  21. 21.
    Mirzapour F and Ghassemian H. 2015. Improving hyperspectral image classification by combining spectral, texture, and shape features. International Journal of Remote Sensing, 36(4): 1070-1096
  22. 22.
    Pohl C and van Genderen J L. 1998. Review article multisensor image fusion in remote sensing: concepts, methods and applications. International Journal of Remote Sensing, 19(5): 823-854
  23. 23.
    Ruan Q Q. 2001. Digital Image Processing. Beijing: Publishing House of Electronics Industry
  24. 24.
    Shen L L and Jia S. 2011. Three-dimensional Gabor wavelets for pixel-based hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 49(12): 5039-5046
  25. 25.
    Shen X, Cao L, Xu T and She G H. 2015. Classification of Pinus massoniana and secondary deciduous tree species in northern subtropical region based on high resolution and hyperspectral remotely sensed data. Chinese Journal of Plant Ecology, 39(12): 1125-1135
  26. 26.
    Tong Q X, Zhang B and Zheng L F. 2006. Hyperspectral Remote Sensing. Beijing: Higher Education Press
  27. 27.
    Wang J N, Zheng L F and Tong Q X. 1996. The spectral absorption identification model and mineral mapping by imaging spectrometer data. Remote Sensing of Environment China, 11(1): 20-31
  28. 28.
    Wei X P, Yu X C, Zhang P Q, Zhi L, and Yang F. 2020. CNN with local binary patterns for hyperspectral images classification. Journal of Remote Sensing, 24(8):1000-1009
  29. 29.
    Wu F Y, Wang X, Ding J W, Du P J, and Tan K. 2020. Improved cascade forest deep learning model for hyperspectral imagery classification. Journal of Remote Sensing, 24(4):439-453
  30. 30.
    Yang M D, Huang K S, Yang Y F, Lu L Y, Feng Z Y and Tsai H P. 2016. Hyperspectral image classification using fast and adaptive bidimensional empirical mode decomposition with minimum noise fraction. IEEE Geoscience and Remote Sensing Letters, 13(12): 1950-1954
  31. 31.
    Yue J, Mao S J and Li M. 2016. A deep learning framework for hyperspectral image classification using spatial pyramid pooling. Remote Sensing Letters, 7(9): 875-884
  32. 32.
    Zhang B, Li S S, Jia X P, Gao L R and Peng M. 2011. Adaptive markov random field approach for classification of hyperspectral imagery. IEEE Geoscience and Remote Sensing Letters, 8(5): 973-977
  33. 33.
    Zhao W D, Li S S, Li A, Zhang B and Li Y. 2019. Hyperspectral images classification with convolutional neural network and textural feature using limited training samples. Remote Sensing Letters, 10(5): 449-458
  34. 34.
    Zubko V, Kaufman Y J, Burg R I and Martins J V. 2007. Principal component analysis of remote sensing of aerosols over oceans. IEEE Transactions on Geoscience and Remote Sensing, 45(3): 730-745

Читать полностью

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website