سابِق|تال

Hyperspectral image classification based on three branch network with grouped spatial-spectral attention

  • role： First author第一作者
  • Affiliation：

    Hubei Key Laboratory of Applied Mathematics, Faculty of Mathematics and Statistics, Hubei University, Wuhan 430062, China

  • Email：suhan@stu.hubu.edu.cn
  • Introduction：苏涵，研究方向为图像处理。E-mail： suhan@stu.hubu.edu.cn
SU Han1,  
  • Affiliation：

    Hubei Key Laboratory of Applied Mathematics, Faculty of Mathematics and Statistics, Hubei University, Wuhan 430062, China

CHEN Na1,  
  • Affiliation：

    Hubei Key Laboratory of Applied Mathematics, Faculty of Mathematics and Statistics, Hubei University, Wuhan 430062, China

    Industrial Research Institute of Remote Sensing, Ningbo University, Shiye Intelligent Technology Co., Ltd, Ningbo 315211, China

PENG Jiangtao13,  
  • role： Corresponding author通信作者
  • Affiliation：

    Department of Geography and Spatial Information Techniques, Ningbo University, Ningbo 315211, China

    Industrial Research Institute of Remote Sensing, Ningbo University, Shiye Intelligent Technology Co., Ltd, Ningbo 315211, China

  • Email：sunweiwei@nbu.edu.cn
  • Introduction：孙伟伟，研究方向为多源遥感协同智能解译与海岸带健康监测。E-mail： sunweiwei@nbu.edu.cn
SUN Weiwei23*

ملخص

Hyperspectral images have abundant spatial and spectral information. Numerous hyperspectral classification algorithms focus on the extraction and maximization of spatial and spectral information. Deep feature extraction networks generally extract spectral-spatial features using single-branch serial or double-branch parallel structures. However, single-branch structures may lead to mutual interference between features of spectral and spatial dimensions, and double-branch parallel structures tend to ignore the correlation between spatial and spectral features. This paper proposes a three-branch grouped spatial-spectral attention network (TGSSAN) to consider the differences and correlations between spatial and spectral features. TGSSAN can extract independent spectral-spatial features while preserving their correlation.This paper proposes the TGSSAN, which has three parallel branches (i.e., spectral, spatial, and spectral-spatial branches). These branches can separately extract spectral, spatial, and spatial-spectral features. Different attention blocks are designed in three branches to enhance the discriminative capability of features. In particular, a grouped spatial-spectral attention mechanism is proposed in the spectral-spatial branch to obtain spatial and spectral attention simultaneously. Finally, three branch features are fused for classification.In the experiment, the proposed TGSSAN algorithm is compared with some advanced deep learning algorithms, such as SSRN, FDSSC, DBMA, DBDA, HResNetAM, and A2S2KResNet. The performance of different algorithms is evaluated on five hyperspectral data sets. Experimental results show that the proposed algorithm achieved superior classification performance on IP, PU, SA, HU, and HHK datasets. In particular, the proposed algorithm achieves higher classification accuracy despite limited training samples compared with the existing advanced algorithms.The TGSSAN method proposed in this paper improves the shortcomings of the single-branch serial and double-branch parallel structures for continuous extraction of spectral-spatial features, which can effectively extract image spectral-spatial feature information. The three attention blocks designed in this paper namely, spectral, grouped spatial-spectral, and spatial attention modules, can effectively enhance the feature discrimination capability and further improve the classification performance.

مفهوم

hyperspectral image classification;attention mechanism;three-branch network;deep network

References

  1. 1.
    Chen G Y and Qian S E. 2007. Dimensionality reduction of hyperspectral imagery. Journal of Applied Remote Sensing, 1(1): 013509
  2. 2.
    Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10): 6232-6251
  3. 3.
    Chen Y S, Zhu L, Ghamisi P, Jia X P, Li G Y and Tang L. 2017. Hyperspectral images classification with Gabor filtering and convolutional neural network. IEEE Geoscience and Remote Sensing Letters, 14(12): 2355-2359
  4. 4.
    Fu J, Liu J, Tian H J, Li Y, Bao Y J, Fang Z W and Lu H Q. 2019. Dual attention network for scene segmentation//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 3141-3149
  5. 5.
    Ghamisi P, Yokoya N, Li J, Liao W Z, Liu S C, Plaza J, Rasti B and Plaza A. 2017. Advances in hyperspectral image and signal processing: a comprehensive overview of the state of the art. IEEE Geoscience and Remote Sensing Magazine, 5(4): 37-78
  6. 6.
    Govender M, Chetty K and Bulcock H. 2007. A review of hyperspectral remote sensing and its application in vegetation and water resource studies. Water SA, 33(2): 145-152
  7. 7.
    Guo M H, Xu T X, Liu J J, Liu Z N, Jiang P T, Mu T J, Zhang S H, Martin R R, Cheng M M and Hu S M. 2022. Attention mechanisms in computer vision: a survey. Computational Visual Media, 8(3): 331-368
  8. 8.
    Hu J, Shen L and Sun G. 2018. Squeeze-and-excitation networks//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 7132-7141
  9. 9.
    Jia X P, Kuo B C and Crawford M M. 2013. Feature mining for hyperspectral image classification. Proceedings of the IEEE, 101(3): 676-697
  10. 10.
    Kumar B, Dikshit O, Gupta A and Singh M K. 2020. Feature extraction for hyperspectral image classification: a review. International Journal of Remote Sensing, 41(16): 6248-6287
  11. 11.
    Li J J, Ma Y L, Song R, Xi B B, Hong D F and Du Q. 2022. A triplet semisupervised deep network for fusion classification of hyperspectral and LiDAR data. IEEE Transactions on Geoscience and Remote Sensing, 60: 5540513
  12. 12.
    Li R, Zheng S Y, Duan C X, Yang Y and Wang X Q. 2020. Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sensing, 12(3): 582
  13. 13.
    Liu H, Li W, Xia X G, Zhang M M, Gao C Z and Tao R. 2023. Central attention network for hyperspectral imagery classification. IEEE Transactions on Neural Networks and Learning Systems, 34(11): 8989-9003
  14. 14.
    Liu Y S, Zhou S B, Han W, Liu W X, Qiu Z F and Li C. 2019. Convolutional neural network for hyperspectral data analysis and effective wavelengths selection. Analytica Chimica Acta, 1086: 46-54
  15. 15.
    Ma W P, Yang Q F, Wu Y, Zhao W and Zhang X R. 2019. Double-branch multi-attention mechanism network for hyperspectral image classification. Remote Sensing, 11(11): 1307
  16. 16.
    Mei S H, Ji J Y, Bi Q Q, Hou J H, Du Q and Li W. 2016. Integrating spectral and spatial information into deep convolutional neural networks for hyperspectral classification//2016 IEEE International Geoscience and Remote Sensing Symposium. Beijing: IEEE: 5067-5070
  17. 17.
    Melgani F and Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8): 1778-1790
  18. 18.
    Pal M. 2012. Multinomial logistic regression-based feature selection for hyperspectral data. International Journal of Applied Earth Observation and Geoinformation, 14(1): 214-220
  19. 19.
    Peng J T, Sun W W, Wei T H and Fan W Q. 2020. A modified correlation alignment algorithm for the domain adaptation of GF-5 hyperspectral image. Journal of Remote Sensing, 24(4): 417-426
  20. 20.
    Qian Y T, Ye M C and Zhou J. 2013. Hyperspectral image classification based on structured sparse logistic regression and three-dimensional wavelet texture features. IEEE Transactions on Geoscience and Remote Sensing, 51(4): 2276-2291
  21. 21.
    Roy S K, Manna S, Song T C and Bruzzone L. 2021. Attention-based adaptive spectral-spatial kernel resnet for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(9): 7831-7843
  22. 22.
    Sisodia P S, Tiwari V and Kumar A. 2014. Analysis of supervised maximum likelihood classification for remote sensing image//International Conference on Recent Advances and Innovations in Engineering. Jaipur: IEEE: 1-4
  23. 23.
    Sun W W, Zhang D F, Yang G and Li W Y. 2018. Band selection for hyperspectral imagery based on weighted probabilistic archetypal analysis. Journal of Remote Sensing, 22(1): 110-118
  24. 24.
    Tang J L, Alelyani S and Liu H. 2014. Feature selection for classification: a review//Aggarwal C C. Data Classification: Algorithms and Applications. New York: Chapman and Hall/CRC: 37-64
  25. 25.
    Tang X, Meng F B, Zhang X R, Cheung Y M, Ma J J, Liu F and Jiao L C. 2021. Hyperspectral image classification based on 3-D octave convolution with spatial–spectral attention network. IEEE Transactions on Geoscience and Remote Sensing, 59(3): 2430-2447
  26. 26.
    Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł and Polosukhin I. 2017. Attention is all you need//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: Curran Associates Inc.: 6000-6010
  27. 27.
    Vincent P, Larochelle H, Lajoie I, Bengio Y and Manzagol P A. 2010. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research, 11: 3371-3408
  28. 28.
    Wan S, Yeh M L and Ma H L. 2021. An innovative intelligent system with integrated CNN and SVM: considering various crops through hyperspectral image data. ISPRS International Journal of Geo-Information, 10(4): 242
  29. 29.
    Wang Q L, Wu B G, Zhu P F, Li P H, Zuo W M and Hu Q H. 2020. ECA-Net: efficient channel attention for deep convolutional neural networks//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE: 11531-11539
  30. 30.
    Wang W J, Dou S G, Jiang Z M and Sun L J. 2018. A fast dense spectral-spatial convolution network framework for hyperspectral images classification. Remote Sensing, 10(7): 1068
  31. 31.
    Woo S, Park J, Lee J Y and Kweon I S. 2018. CBAM: convolutional block attention module//15th European Conference on Computer Vision. Munich: Springer: 3-19
  32. 32.
    Xue Z X, Yu X C, Liu B, Tan X and Wei X P. 2021. HResNetAM: hierarchical residual network with attention mechanism for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14: 3566-3580
  33. 33.
    Yang K, Sun H, Zou C B and Lu X Q. 2022. Cross-attention spectral-spatial network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60: 5518714
  34. 34.
    Zhang B, Li S S, Jia X P, Gao L R and Peng M. 2011. Adaptive Markov random field approach for classification of hyperspectral imagery. IEEE Geoscience and Remote Sensing Letters, 8(5): 973-977
  35. 35.
    Zhang M M, Li W, Tao R, Li H C and Du Q. 2022a. Information fusion for classification of hyperspectral and LiDAR Data using IP-CNN. IEEE Transactions on Geoscience and Remote Sensing, 60: 5506812
  36. 36.
    Zhang Q L and Yang Y B. 2021. SA-Net: Shuffle attention for deep convolutional neural networks. IEEE International Conference on Acoustics, Speech and Signal Processing, 2235-2239
  37. 37.
    Zhang X, Sun Y L, Shang K, Zhang L F and Wang S D. 2016. Crop classification based on feature band set construction and object-oriented approach using hyperspectral images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(9): 4117-4128
  38. 38.
    Zhang X M, Sun G Y, Jia X P, Wu L X, Zhang A Z, Ren J C, Fu H and Yao Y J. 2022b. Spectral-spatial self-attention networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60: 5512115
  39. 39.
    Zhao H S. 2022. Research on Dimensionality Reduction and Classification of Hyperspectral Remote Sensing Images under Weakly Supervised Learning Framework. Changchun: Jilin University (赵海士. 2022. 弱监督学习框架下高光谱遥感影像降维与分类方法研究. 长春: 吉林大学) [DOI: 10.27162/d.cnki.gjlin.2022.000591]
  40. 40.
    Zhong Z L, Li J, Luo Z M and Chapman M. 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858
  41. 41.
    Zhou P C, Han J W, Cheng G and Zhang B C. 2019. Learning compact and discriminative stacked autoencoder for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(7): 4823-4833
  42. 42.
    Zhu C Q and Yang X M. 1998. Study of remote sensing image texture analysis and classification using wavelet. International Journal of Remote Sensing, 19(16): 3197-3203
  43. 43.
    Zhu Z X, Jia S, He S, Sun Y W, Ji Z and Shen L L. 2015. Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework. Information Sciences, 298: 274-287

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website