سابِق|تال

Deep capsule network combined with spatial-spectral information for hyperspectral image classification

  • role： First author第一作者
  • Affiliation：

    Strategic Support Force Information Engineering University, Zhengzhou 450001, China

  • Email：gokling1219@163.com
  • Introduction：高奎亮，1996年生，男，硕士研究生， 研究方向为高光谱影像处理与分析。E-mail： gokling1219@163.com
GAO Kuiliang1,  
  • Affiliation：

    Strategic Support Force Information Engineering University, Zhengzhou 450001, China

YU Xuchu1,  
  • Affiliation：

    Institute of Kunming Physics, Kunming 650223, China

SONG Zhihang2,  
  • Affiliation：

    Institute of Kunming Physics, Kunming 650223, China

ZHANG Jin2,  
  • Affiliation：

    Strategic Support Force Information Engineering University, Zhengzhou 450001, China

LIU Bing1,  
  • Affiliation：

    Strategic Support Force Information Engineering University, Zhengzhou 450001, China

SUN Yifan1

ملخص

Owing to vectorized capsule neurons and dynamic routing algorithm, capsule network possesses stronger feature representation capability than traditional convolutional neural networks. In the field of remote sensing, hyperspectral image (HSI) classification methods based on capsule network have obtained better classification results than traditional deep learning models. Aiming at the problems in the capsule classification models, such as shallow network layers and insufficient utilization of spatial–spectral information, this paper constructs a new deep capsule network for HSI classification. The designed network utilizes convolutional capsule layer, residual connection, and three dimensional convolutional capsule layer to further improve classification accuracy.The proposed method includes a convolutional layer, four capsule residual blocks, a class capsule layer, and a reconstruction network. First, the proposed method takes HSI data cubes as the input directly to retain the spatial–spectral details in the HSI. Then, the deep features in the input data are extracted layer by layer with the capsule residual blocks. Finally, the three dimensional convolutional capsule layer is introduced to make full use of the spatial–spectral information, so as to improve the classification accuracy.Three public HSI data sets including University of Pavia, Indian Pines and Salinas, as well as a large-scale hyperspectral data set Chikusei, are selected for the experiments. The results demonstrate that the proposed method outperforms the existing deep learning classification methods. Compared with the shallow capsule network model and the deep three dimensional convolutional network with residual structure, the proposed method improves the overall classification accuracy by 1%—3%, 2%—5%, 1%—2%, and 2%—4% on four different data sets. In addition, hyperparameters such as learning rate, capsule neurons dimension, network structure, and spatial neighborhood have been analyzed in detail. The effectiveness of three dimensional convolutional capsule layer has also been proven by conducting ablation studies.Compared with the existing HSI classification model based on capsule network, the proposed method has the following advantages. (1) The high-dimensional cubes are used for input data without any dimension reduction process, thus enabling the model to make full use of spatial–spectral details. (2) A deep network is constructed utilizing convolutional capsule layers and residual connection, thus allowing the model to extract more robust and abstract deep features. (3) The introduction of three dimensional convolutional capsule layer can make full use of spatial–spectral information in HSI to further improve the classification accuracy.

مفهوم

hyperspectral image classification;deep capsule network;three dimensional convolutional capsule;three dimensional convolutional routing;deep learning

References

  1. 1.
    Benediktsson J A, Palmason J A and Sveinsson J R. 2005. Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 43(3): 480-491
  2. 2.
    Chen Y S, Lin Z H, Zhao X, Wang G and Gu Y F. 2014. Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6): 2094-2107
  3. 3.
    Chen Y S, Zhao X and Jia X P. 2015. Spectral–spatial classification of hyperspectral data based on deep belief network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(6): 2381-2392
  4. 4.
    Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10): 6232-6251.
  5. 5.
    Dong X F, Gan F P, Li N, Yan B K, Zhang L, Zhao J Q, Yu J C, Liu R Y and Ma Y N. 2020. Fine mineral identification of GF-5 hyperspectral image. Journal of Remote Sensing, 24(4): 454-464
  6. 6.
    Gao K L, Zhang P Q, Yu X C, Tan X and Liu B. 2019. Classification method of hyperspectral image based on Network In Network structure. Journal of Geomatics Science and Technology, 36(5): 500-504, 510
  7. 7.
    Gao K L, Liu B, Yu X C, Qin J C, Zhang P Q and Tan X. 2020. Deep relation network for hyperspectral image few-shot classification. Remote Sensing, 12(6): 923
  8. 8.
    He L, Li J, Liu C Y and Li S T. 2018. Recent advances on spectral-spatial hyperspectral image classification: an overview and new guidelines. IEEE Transactions on Geoscience and Remote Sensing, 56(3): 1579-1597
  9. 9.
    Hu W, Huang Y Y, Li W, Zhang F and Li H C. 2015. Deep convolutional neural networks for hyperspectral image classification. Journal of Sensors, 2015: 258619
  10. 10.
    Jiang X F, Liu W B, Zhang Y, Liu J R, Li S Y and Lin J Z. 2020. Spectral–spatial hyperspectral image classification using dual-channel capsule networks. IEEE Geoscience and Remote Sensing Letters
  11. 11.
    Kang X D, Zhuo B B and Duan P H. 2019. Semi-supervised deep learning for hyperspectral image classification. Remote Sensing Letters, 10(4): 353-362
  12. 12.
    Lee H and Kwon H. 2017. Going deeper with contextual CNN for hyperspectral image classification. IEEE Transactions on Image Processing, 26(10): 4843-4855
  13. 13.
    Li J, Marpu P R, Plaza A, Bioucas-Dias J M and Benediktsson J A. 2013. Generalized composite kernel framework for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 51(9): 4816-4829
  14. 14.
    Li Y, Zhang H K and Shen Q. 2017. Spectral–spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sensing, 9(1): 67
  15. 15.
    Liu B, Yu X C, Zhang P Q, Yu A Z, Fu Q Y and Wei X P. 2018a. Supervised deep feature extraction for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 56(4): 1909-1921
  16. 16.
    Liu B, Yu X C, Yu A Z and Wan G. 2018b. Deep convolutional recurrent neural network with transfer learning for hyperspectral image classification. Journal of Applied Remote Sensing, 12(2): 026028
  17. 17.
    Liu B, Yu X C, Zhang P Q and Tan X. 2019. Deep 3D convolutional network combined with spatial-spectral features for hyperspectral image classification. Acta Geodaetica et Cartographica Sinica, 48(1): 53-63
  18. 18.
    Paoletti M E, Haut J M, Fernandez-Beltran R, Plaza J, Plaza A, Li J and Pla F. 2019. Capsule networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(4): 2145-2160
  19. 19.
    Rajasegaran J, Jayasundara V, Jayasekara S, Jayasekara H, Seneviratne S and Rodrigo R. 2019. DeepCaps: going deeper with capsule networks//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA: IEEE: 10717-10725
  20. 20.
    Sabour S, Frosst N and Hinton G E. 2017. Dynamic routing between capsules. arXiv: 1710.09829
  21. 21.
    Tarabalka Y, Fauvel M, Chanussot J and Benediktsson J A. 2010. SVM- and MRF-based method for accurate classification of hyperspectral images. IEEE Geoscience and Remote Sensing Letters, 7(4): 736-740
  22. 22.
    Xi E, Bing S and Jin Y. 2017. Capsule network performance on complex data. arXiv: 1712.03480
  23. 23.
    Xu Q, Xiao Y, Wang D Y and Luo B. 2020. CSA-MSO3DCNN: multiscale octave 3D CNN with channel and spatial attention for hyperspectral image classification. Remote Sensing, 12(1): 188
  24. 24.
    Yin J H, Li S, Zhu H M and Luo X Y. 2019. Hyperspectral image classification using CapsNet with well-initialized shallow layers. IEEE Geoscience and Remote Sensing Letters, 16(7): 1095-1099
  25. 25.
    Zhang M M, Li W and Du Q. 2018. Diverse region-based CNN for hyperspectral image classification. IEEE Transactions on Image Processing, 27(6): 2623-2634
  26. 26.
    Zhang L, Zhang J Y, Wei W and Zhang Y N. 2019. Learning discriminative compact representation for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 57(10): 8276-8289
  27. 27.
    Zhi L, Yu X C, Liu B and Wei X P. 2019. A dense convolutional neural network for hyperspectral image classification. Remote Sensing Letters, 10(1): 59-66
  28. 28.
    Zhu K Q, Chen Y S, Ghamisi P, Jia X P and Benediktsson J A. 2019. Deep convolutional capsule network for hyperspectral image spectral and spectral-spatial classification. Remote Sensing, 11(3): 223

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website