سابِق|تال

Hyperspectral image classification based on spectral deformable convolution neural network

  • role： First author第一作者
  • Affiliation：

    School of Earth Sciences and Engineering, Hohai University, Nanjing 211100, China

    Jiangsu Province Engineering Research Center of Water Resources and Environment Assessment Using Remote Sensing, Hohai University, Nanjing 211100, China

  • Email：zhaohui.xue@hhu.edu.cn
  • Introduction：薛朝辉，研究方向为遥感影像智能处理与地学应用。E-mail： zhaohui.xue@hhu.edu.cn
XUE Zhaohui,  
  • Affiliation：

    School of Earth Sciences and Engineering, Hohai University, Nanjing 211100, China

    Jiangsu Province Engineering Research Center of Water Resources and Environment Assessment Using Remote Sensing, Hohai University, Nanjing 211100, China

LI Bo

ملخص

Currently, convolutional neural network is a research hot spot in hyperspectral image classification. Some advanced models such as dilated convolution and deformable convolution have been developed successfully. However, the existing deformable convolution modules only shift in spatial domain and ignore the spectral difference information. Therefore, this paper proposed a novel deformable convolution, which extends spatial deformable to the spectral domain. In addition, a spectral deformable convolutional neural network (SDCNN) is proposed for hyperspectral image classification. In view of the existing problems in the application of spatial deformable convolution in HSI classification, this paper extends the deformable convolution to the spectral dimension, and proposes the spectral deformable convolution. Given that different feature categories may have different classification effects, more appropriate classification bands can be selected for different class by the learnt shifts in spectral domain. Therefore, the spectral feature extraction can focus on more effective band, thereby promoting more discriminative features. In this way, only one direction of spectral dimension needs to be learned, which reduces computational complexity, that is, only half of the occupying time than that of spatial deformable convolution. First, a full connection layer is used to learn the offset of the spectral deformable convolution, and a linear difference is used to perform feature correction in spectral domain. Second, a multilayer 1×1 convolution is used for spectral feature aggregation. Finally, a 3D convolution layer is used to extract spectral–spatial features. Experiments were conducted on three international popular datasets including Indian Pines, University of Pavia, and University of Houston. The experimental results demonstrate that the SDCNN is superior to other deep learning methods. SDCNN yields the highest classification accuracy with an overall accuracy of 98.86% (Indian pines, 10%/class), 99.81% (University of Pavia, 5%/class), and 97.41% (University of Houston, 50/class), which can well verify the effectiveness of the proposed model. First, through comprehensive experiments, SDCNN yields the highest accuracy compared with other existing models on three international popular datasets, which can prove the validity of the SDCNN method. Second, the effectiveness of the spectral deformable convolution module is proven by comparing the traditional dilated convolution module and the spatial deformable convolution module. SDCNN generalizes better with limited training samples, especially when the number of labeled samples is small, The stability of classification performance of SDCNN method is proved;. Finally, In the experiment of separating samples, SDCNN method achieves the best classification effect compared with other methods, and has obvious accuracy advantages compared with spatial deformable convolution method, which proves that SDCNN method has less dependence on spatial features and stronger feature extraction ability.

مفهوم

deep learning;hyperspectral image classification;convolutional neural network;Deformable convolution

References

  1. 1.
    Belgiu M and Drăguţ L. 2016. Random forest in remote sensing: a review of applications and future directions. ISPRS Journal of Photogrammetry and Remote Sensing, 114: 24-31
  2. 2.
    Camps-Valls G, Gomez-Chova L, Munoz-Mari J, Vila-Frances J and Calpe-Maravilla J. 2006. Composite kernels for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 3(1): 93-97
  3. 3.
    Chen X, Li Z, Jiang Jet al. 2021. Adaptive Effective Receptive Field Convolution for Semantic Segmentation of VHR Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing, 59(4): 3532-3546
  4. 4.
    Chen Y, Nasrabadi N M and Tran T D. 2011. Hyperspectral image classification using dictionary-based sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 49(10): 3973-3985
  5. 5.
    Chen Y S, Lin Z H, Zhao X, Wang G and Gu Y F. 2014. Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6): 2094-2107
  6. 6.
    Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10): 6232-6251
  7. 7.
    Dai J F, Qi H Z, Xiong Y W, Li Y, Zhang G D, Hu H and Wei Y C. 2017. Deformable convolutional networks//2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE: 764-773
  8. 8.
    Du P J, Xia J S, Xue Z H, Tan K, Su H J and Bao R. 2016. Review of hyperspectral remote sensing image classification. Journal of Remote Sensing, 20(2): 236-256
  9. 9.
    Fauvel M, Tarabalka Y, Benediktsson J A, Chanussot J and Tilton J C. 2013. Advances in spectral-spatial classification of hyperspectral images. Proceedings of the IEEE, 101(3):652-675
  10. 10.
    Feng J, Wu X D, Shang R H, Sui C, Li J, Jiao L C and Zhang X R. 2021. Attention multibranch convolutional neural network for hyperspectral image classification based on adaptive region search. IEEE Transactions on Geoscience and Remote Sensing, 59(6): 5054-5070
  11. 11.
    Hinton G E and Salakhutdinov R R. 2006. Reducing the dimensionality of data with Neural Networks. Science, 313(5786): 504-507
  12. 12.
    Huang H and Qu H P. 2014. Hyperspectral remote sensing image classification based on SSDE. Optics and Precision Engineering, 22(2):434-442
  13. 13.
    Hu W, Huang Y Y, Wei L, Zhang F and Li H C. 2015. Deep convolutional neural networks for hyperspectral image classification. Journal of Sensors, 2015: 258619
  14. 14.
    Li C M, Qiu Z L, Cao X Y, Chen Z H, Gao H M and Hua Z J. 2021. Hybrid dilated convolution with multi-scale residual fusion network for hyperspectral image classification. Micromachines, 12(5): 545
  15. 15.
    Li J, Bioucas-Dias J M and Plaza A. 2010. Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning. IEEE Transactions on Geoscience and Remote Sensing, 48(11): 4085-4098
  16. 16.
    Li W, Du Q and Xiong M M. 2015. Kernel collaborative representation with Tikhonov regularization for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 12(1): 48-52
  17. 17.
    Li Z K, Wang T N, Li W, Du Q, Wang C Y, Liu C W and Shi X B. 2020. Deep multilayer fusion dense network for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 1258-1270
  18. 18.
    Liu G C, Lin Z C, Yan S C, Sun J, Yu Y and Ma Y. 2013. Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1): 171-184
  19. 19.
    Lowe A, Harrison N and French A P. 2017. Hyperspectral image analysis techniques for the detection and classification of the early onset of plant disease and stress. Plant Methods, 13(1):1-12
  20. 20.
    Lu B, Dao P D, Liu J G, He Y H and Shang J L. 2020. Recent advances of hyperspectral imaging technology and applications in agriculture. Remote Sensing, 12(16):1-44
  21. 21.
    Melgani F and Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8): 1778-1790
  22. 22.
    Mou L C, Ghamisi P and Zhu X X. 2017. Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3639-3655
  23. 23.
    Mou L C and Zhu X X. 2020. Learning to pay attention on spectral domain: a spectral attention module-based convolutional network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(1): 110-122
  24. 24.
    Paoletti M E, Haut J M, Plaza J and Plaza A. 2019. Deep learning classifiers for hyperspectral imaging: a review. ISPRS Journal of Photogrammetry and Remote Sensing, 158: 279-317
  25. 25.
    Pooja K, Nidamanuri R R and Mishra D. 2019. Multi-scale dilated residual convolutional neural network for hyperspectral image classification//2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS). Amsterdam: IEEE: 1-5
  26. 26.
    Tong Q X, Zhang B and Zhang L F. 2016. Current progress of hyperspectral remote sensing in China. Journal of Remote Sensing, 20(5): 689-707
  27. 27.
    van der Meer F D, van der Werff H M A, van Ruitenbeek F J A, Hecker C A, Bakker W H, Noomen M F, Van Der meijde M, Carranza E J M, Smeth J B D and Woldai T. 2012. Multi- and hyperspectral geologic remote sensing: a review. International Journal of Applied Earth Observation and Geoinformation, 14(1): 112-128
  28. 28.
    Wang P Q, Chen P F, Yuan Y, Liu D, Huang Z H, Hou X D and Cottrell G. 2018. Understanding convolution for semantic segmentation//2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Lake Tahoe, NV: IEEE: 1451-1460
  29. 29.
    Wei X P, Yu X C, Zhang P Q, Zhi L and Yang F. 2020. CNN with local binary patterns for hyperspectral images classification. Journal of Remote Sensing, 24(8): 1000-1009
  30. 30.
    Xu Y H, Du B and Zhang L P. 2020. Beyond the patchwise classification: spectral-spatial fully convolutional networks for hyperspectral image classification. IEEE Transactions on Big Data, 6(3): 492-506
  31. 31.
    Xu Y H, Zhang L P, Du B and Zhang F. 2018. Spectral-spatial unified networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 56(10): 5893-5909
  32. 32.
    Yu C Y, Han R, Song M P, Liu C Y and Chang C I. 2020. A simplified 2D-3D CNN architecture for hyperspectral image classification based on spatial–spectral fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 2485-2501
  33. 33.
    Zhang L P, Zhang L F and Du B. 2016. Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art. IEEE Geoscience and Remote Sensing, 4(2): 22-40
  34. 34.
    Zhang M M, Li W and Du Q. 2018. Diverse region-based CNN for hyperspectral image classification. IEEE Transactions on Image Processing, 27(6): 2623-2634
  35. 35.
    Zhao F, Zhang J J, Meng Z and Liu H Q. 2021. Densely connected pyramidal dilated convolutional network for hyperspectral image classification. Remote Sensing, 13(17): 3396
  36. 36.
    Zhao W Z and Du S H. 2016. Spectral-spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach. IEEE Transactions on Geoscience and Remote Sensing, 54(8): 4544-4554
  37. 37.
    Zhong Z L, Li J, Luo Z M and Chapman M. 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858
  38. 38.
    Zhong Z L, Li J, Ma L F, Jiang H and Zhao H. 2017. Deep residual networks for hyperspectral image classification//2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Fort Worth, TX: IEEE: 1824-1827
  39. 39.
    Zhu J, Fang L Y and Ghamisi P. 2018. Deformable convolutional neural networks for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 15(8): 1254-1258

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website