Deep capsule network combined with spatial-spectral information for hyperspectral image classification

Kuiliang GAO; Xuchu YU; Zhihang SONG; Jin ZHANG; Bing LIU; Yifan SUN

doi:10.11834/jrs.20210309

Technologies and Methodologies | Views : 0 下载量: 278 CSCD: 2 更多指标

PDF
Export
Share
Collection
Album

Deep capsule network combined with spatial-spectral information for hyperspectral image classification
Vol. 25, Issue 6, Pages: 1257-1269(2021)
Published： 07 June 2021 ，
DOI： 10.11834/jrs.20210309

扫描看全文

高奎亮，余旭初，宋治杭，张晋，刘冰，孙一帆.2021.联合空谱信息的高光谱影像深度胶囊网络分类.遥感学报，25（6）： 1257-1269

Gao K L，Yu X C，Song Z H，Zhang J，Liu B and Sun Y F. 2021. Deep capsule network combined with spatial-spectral information for hyperspectral image classification. National Remote Sensing Bulletin， 25（6）：1257-1269
高奎亮，余旭初，宋治杭，张晋，刘冰，孙一帆.2021.联合空谱信息的高光谱影像深度胶囊网络分类.遥感学报，25（6）： 1257-1269 DOI： 10.11834/jrs.20210309.

Gao K L，Yu X C，Song Z H，Zhang J，Liu B and Sun Y F. 2021. Deep capsule network combined with spatial-spectral information for hyperspectral image classification. National Remote Sensing Bulletin， 25（6）：1257-1269 DOI： 10.11834/jrs.20210309.

摘要

向量化的胶囊神经元和动态路由式的信息传递机制赋予了胶囊网络更强的特征表示能力。在遥感领域，基于胶囊网络的高光谱影像分类方法已经获得了较传统深度学习模型更为优异的分类结果。针对现有胶囊分类模型中存在的网络浅层、空谱联合信息利用不足等问题，本文利用卷积胶囊层、残差连接、三维卷积胶囊层构建了一种用于高光谱影像分类的新型深度胶囊网络。具体地，本文方法直接以高维数据立方体作为网络输入，并利用胶囊残差块逐层提取数据中的深层抽象特征。为了更加充分地利用影像中的空谱联合特征，在深层次的胶囊残差块中引入三维卷积胶囊层，以进一步提高分类精度。为了验证本文方法的有效性，选择University of Pavia、Indian Pines和Salinas等3个常用高光谱数据集和一个大规模机载高光谱数据集Chikusei进行实验。结果表明，与现有深度学习模型相比，本文方法能够获得更为优异的分类效果，在4个数据集上分别获得了99.43%、98.85%、97.14%和97.43%的总体分类精度。

Abstract

Owing to vectorized capsule neurons and dynamic routing algorithm

capsule network possesses stronger feature representation capability than traditional convolutional neural networks. In the field of remote sensing

hyperspectral image (HSI) classification methods based on capsule network have obtained better classification results than traditional deep learning models. Aiming at the problems in the capsule classification models

such as shallow network layers and insufficient utilization of spatial–spectral information

this paper constructs a new deep capsule network for HSI classification. The designed network utilizes convolutional capsule layer

residual connection

and three dimensional convolutional capsule layer to further improve classification accuracy.

The proposed method includes a convolutional layer

four capsule residual blocks

a class capsule layer

and a reconstruction network. First

the proposed method takes HSI data cubes as the input directly to retain the spatial–spectral details in the HSI. Then

the deep features in the input data are extracted layer by layer with the capsule residual blocks. Finally

the three dimensional convolutional capsule layer is introduced to make full use of the spatial–spectral information

so as to improve the classification accuracy.

Three public HSI data sets including University of Pavia

Indian Pines and Salinas

as well as a large-scale hyperspectral data set Chikusei

are selected for the experiments. The results demonstrate that the proposed method outperforms the existing deep learning classification methods. Compared with the shallow capsule network model and the deep three dimensional convolutional network with residual structure

the proposed method improves the overall classification accuracy by 1%—3%

2%—5%

1%—2%

and 2%—4% on four different data sets. In addition

hyperparameters such as learning rate

capsule neurons dimension

network structure

and spatial neighborhood have been analyzed in detail. The effectiveness of three dimensional convolutional capsule layer has also been proven by conducting ablation studies.

Compared with the existing HSI classification model based on capsule network

the proposed method has the following advantages. (1) The high-dimensional cubes are used for input data without any dimension reduction process

thus enabling the model to make full use of spatial–spectral details. (2) A deep network is constructed utilizing convolutional capsule layers and residual connection

thus allowing the model to extract more robust and abstract deep features. (3) The introduction of three dimensional convolutional capsule layer can make full use of spatial–spectral information in HSI to further improve the classification accuracy.

关键词

高光谱影像分类深度胶囊网络三维卷积胶囊三维卷积路由深度学习

Keywords

hyperspectral image classificationdeep capsule networkthree dimensional convolutional capsulethree dimensional convolutional routingdeep learning

references

Benediktsson J A, Palmason J A and Sveinsson J R. 2005. Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 43(3): 480-491[DOI: 10.1109/tgrs.2004.842478http://dx.doi.org/10.1109/tgrs.2004.842478]

Chen Y S, Lin Z H, Zhao X, Wang G and Gu Y F. 2014. Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6): 2094-2107 [DOI: 10.1109/JSTARS.2014.2329330http://dx.doi.org/10.1109/JSTARS.2014.2329330]

Chen Y S, Zhao X and Jia X P. 2015. Spectral–spatial classification of hyperspectral data based on deep belief network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(6): 2381-2392 [DOI: 10.1109/JSTARS.2015.2388577http://dx.doi.org/10.1109/JSTARS.2015.2388577]

Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10): 6232-6251[DOI: 10.1109/TGRS.2016.2584107http://dx.doi.org/10.1109/TGRS.2016.2584107].

Dong X F, Gan F P, Li N, Yan B K, Zhang L, Zhao J Q, Yu J C, Liu R Y and Ma Y N. 2020. Fine mineral identification of GF-5 hyperspectral image. Journal of Remote Sensing, 24(4): 454-464

董新丰, 甘甫平, 李娜, 闫柏琨, 张磊, 赵佳琪, 于峻川, 刘镕源, 马燕妮. 2020. 高分五号高光谱影像矿物精细识别. 遥感学报, 24(4): 454-464 [DOI: 10.11834/jrs.20209194http://dx.doi.org/10.11834/jrs.20209194]

Gao K L, Zhang P Q, Yu X C, Tan X and Liu B. 2019. Classification method of hyperspectral image based on Network In Network structure. Journal of Geomatics Science and Technology, 36(5): 500-504, 510

高奎亮, 张鹏强, 余旭初, 谭熊, 刘冰. 2019. 基于Network In Network网络结构的高光谱影像分类方法. 测绘科学技术学报, 36(5): 500-504, 510

Gao K L, Liu B, Yu X C, Qin J C, Zhang P Q and Tan X. 2020. Deep relation network for hyperspectral image few-shot classification. Remote Sensing, 12(6): 923 [DOI: 10.3390/rs12060923http://dx.doi.org/10.3390/rs12060923]

He L, Li J, Liu C Y and Li S T. 2018. Recent advances on spectral-spatial hyperspectral image classification: an overview and new guidelines. IEEE Transactions on Geoscience and Remote Sensing, 56(3): 1579-1597 [DOI: 10.1109/TGRS.2017.2765364http://dx.doi.org/10.1109/TGRS.2017.2765364]

Hu W, Huang Y Y, Li W, Zhang F and Li H C. 2015. Deep convolutional neural networks for hyperspectral image classification. Journal of Sensors, 2015: 258619 [DOI: 10.1155/2015/258619http://dx.doi.org/10.1155/2015/258619]

Jiang X F, Liu W B, Zhang Y, Liu J R, Li S Y and Lin J Z. 2020. Spectral–spatial hyperspectral image classification using dual-channel capsule networks. IEEE Geoscience and Remote Sensing Letters [DOI: 10.1109/LGRS.2020.2991405http://dx.doi.org/10.1109/LGRS.2020.2991405]

Kang X D, Zhuo B B and Duan P H. 2019. Semi-supervised deep learning for hyperspectral image classification. Remote Sensing Letters, 10(4): 353-362

Lee H and Kwon H. 2017. Going deeper with contextual CNN for hyperspectral image classification. IEEE Transactions on Image Processing, 26(10): 4843-4855[DOI: 10.1109/tip.2017.2725580http://dx.doi.org/10.1109/tip.2017.2725580]

Li J, Marpu P R, Plaza A, Bioucas-Dias J M and Benediktsson J A. 2013. Generalized composite kernel framework for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 51(9): 4816-4829[DOI: 10.1109/TGRS.2012.2230268http://dx.doi.org/10.1109/TGRS.2012.2230268]

Li Y, Zhang H K and Shen Q. 2017. Spectral–spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sensing, 9(1): 67 [DOI: 10.3390/rs9010067http://dx.doi.org/10.3390/rs9010067]

Liu B, Yu X C, Zhang P Q, Yu A Z, Fu Q Y and Wei X P. 2018a. Supervised deep feature extraction for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 56(4): 1909-1921[DOI: 10.1109/TGRS.2017.2769673http://dx.doi.org/10.1109/TGRS.2017.2769673]

Liu B, Yu X C, Yu A Z and Wan G. 2018b. Deep convolutional recurrent neural network with transfer learning for hyperspectral image classification. Journal of Applied Remote Sensing, 12(2): 026028[DOI: 10.1117/1.JRS.12.026028http://dx.doi.org/10.1117/1.JRS.12.026028]

Liu B, Yu X C, Zhang P Q and Tan X. 2019. Deep 3D convolutional network combined with spatial-spectral features for hyperspectral image classification. Acta Geodaetica et Cartographica Sinica, 48(1): 53-63

刘冰, 余旭初, 张鹏强. 2019. 联合空-谱信息的高光谱影像深度三维卷积网络分类. 测绘学报. 48(1): 53-63

Paoletti M E, Haut J M, Fernandez-Beltran R, Plaza J, Plaza A, Li J and Pla F. 2019. Capsule networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(4): 2145-2160[DOI: 10.1109/TGRS.2018.2871782http://dx.doi.org/10.1109/TGRS.2018.2871782]

Rajasegaran J, Jayasundara V, Jayasekara S, Jayasekara H, Seneviratne S and Rodrigo R. 2019. DeepCaps: going deeper with capsule networks//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA: IEEE: 10717-10725 [DOI: 10.1109/CVPR.2019.01098http://dx.doi.org/10.1109/CVPR.2019.01098]

Sabour S, Frosst N and Hinton G E. 2017. Dynamic routing between capsules. arXiv: 1710.09829

Tarabalka Y, Fauvel M, Chanussot J and Benediktsson J A. 2010. SVM- and MRF-based method for accurate classification of hyperspectral images. IEEE Geoscience and Remote Sensing Letters, 7(4): 736-740 [DOI: 10.1109/LGRS.2010.2047711http://dx.doi.org/10.1109/LGRS.2010.2047711]

Xi E, Bing S and Jin Y. 2017. Capsule network performance on complex data. arXiv: 1712.03480

Xu Q, Xiao Y, Wang D Y and Luo B. 2020. CSA-MSO3DCNN: multiscale octave 3D CNN with channel and spatial attention for hyperspectral image classification. Remote Sensing, 12(1): 188 [DOI: 10.3390/rs12010188http://dx.doi.org/10.3390/rs12010188]

Yin J H, Li S, Zhu H M and Luo X Y. 2019. Hyperspectral image classification using CapsNet with well-initialized shallow layers. IEEE Geoscience and Remote Sensing Letters, 16(7): 1095-1099[DOI: 10.1109/lgrs.2019.2891076http://dx.doi.org/10.1109/lgrs.2019.2891076]

Zhang M M, Li W and Du Q. 2018. Diverse region-based CNN for hyperspectral image classification. IEEE Transactions on Image Processing, 27(6): 2623-2634[DOI: 10.1109/TIP.2018.2809606http://dx.doi.org/10.1109/TIP.2018.2809606]

Zhang L, Zhang J Y, Wei W and Zhang Y N. 2019. Learning discriminative compact representation for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 57(10): 8276-8289 [DOI: 10.1109/TGRS.2019.2919938http://dx.doi.org/10.1109/TGRS.2019.2919938]

Zhi L, Yu X C, Liu B and Wei X P. 2019. A dense convolutional neural network for hyperspectral image classification. Remote Sensing Letters, 10(1): 59-66[DOI: 10.1080/2150704X.2018.1526424http://dx.doi.org/10.1080/2150704X.2018.1526424]

Zhu K Q, Chen Y S, Ghamisi P, Jia X P and Benediktsson J A. 2019. Deep convolutional capsule network for hyperspectral image spectral and spectral-spatial classification. Remote Sensing, 11(3): 223 [DOI: 10.3390/rs11030223http://dx.doi.org/10.3390/rs11030223]

Alert me when the article has been cited

提交

SAR ship detection through generative knowledge transfer

MtSCCD： Land-use scene classification and change-detection dataset for deep learning

Optical-signal token guided change detection network for heterogeneous remote sensing image

Joint classification of hyperspectral and LiDAR data based on inter-modality match learning

Hyperspectral image classification based on three branch network with grouped spatial-spectral attention