Anterior|Siguiente

Hyperspectral image classification of the deep neural network based on 3D convolution and dense connection

  • role： First author第一作者
  • Affiliation：

    College of Information Science and Technology, Qingdao University of Science and Technology, Qingdao 266061, China

  • Email：116638307@qq.com
  • Introduction：宋廷强，研究方向为数据、人工智能、遥感的理论和应用。E-mail： 116638307@qq.com
SONG Tingqiang1,  
  • role： Corresponding author通信作者
  • Affiliation：

    College of Information Science and Technology, Qingdao University of Science and Technology, Qingdao 266061, China

  • Email：594880373@qq.com
  • Introduction：宗达，研究方向为图像处理、人工智能、高光谱应用。E-mail： 594880373@qq.com
ZONG Da1*,  
  • Affiliation：

    College of Information Science and Technology, Qingdao University of Science and Technology, Qingdao 266061, China

LIU Tongxin1,  
  • Affiliation：

    Zhuhai Orbita Aerospace Science&Technology Co., Ltd. Artificial Intelligence Research Institute, Zhuhai 519080, China

FAN Haisheng2,  
  • Affiliation：

    Zhuhai Orbita Aerospace Science&Technology Co., Ltd. Artificial Intelligence Research Institute, Zhuhai 519080, China

HUANG Tengjie2,  
  • Affiliation：

    Zhuhai Orbita Aerospace Science&Technology Co., Ltd. Artificial Intelligence Research Institute, Zhuhai 519080, China

JIANG Xiaoxu2,  
  • Affiliation：

    College of Information Science and Technology, Qingdao University of Science and Technology, Qingdao 266061, China

WANG Haoyu1

resumen

With the progress of deep learning, researchers are increasingly paying attention to its application in hyperspectral image classification. Many experiments are conducted to achieve a trade-off between accuracy and efficiency to improve the feature extraction performance of neural networks toward small training sample sets.This work has proposed a high-speed and high-precision neural network structure based on spatial spectral information. A cascaded neural network for spectral spatial information extraction is constructed by combining the idea of DenseNet and adopting dilated convolutions instead of 3D convolutions as the main calculation method. The whole network structure is divided into four components: spectral information extraction, spectral compression, fusion of spatial and spectral information, and voting solution.Three convolutional layers are built in the spectral information extraction component. In each layer, 1×1×7 convolution kernels are used to extract spectral information and maintain the independence of spatial information. The number of kernels is set to 60. In light of the DenseNet idea, the network outputs of the first and second layers are dimensionally split in spectrum and inputted into the third layer. The outputs of the first, second, and third layers are also dimensionally split and inputted into the spectral compression component.In the spectral compression component, a 1×1×7 convolution kernel is used with a step size set to three. The spectral dimension is compressed, and the number of parameters of the deeper network is lessened by reducing the size of the feature map.In the spatial and spectral information fusion component, the goal is to fuse spatial information for the first time with 3×3 receptive fields and integrate the spectral information of the data. Separable convolutions are adopted instead of traditional 3D convolutions, and the 3×3×K convolution kernel is decomposed into a 3×3×1 convolution and a 1×1×K convolution. The value of K is equal to the spectral dimension of the input feature map. Then, 40 9×9×1 feature maps are outputted.Voting means that if the output of most pixels is the same value, then the average value of all values will also be pulled near this certain value. In the voting solution component using parameter-free global average pooling, the 9×9×1 feature maps are voted to obtain 1×1×1 output values. These 40 output values are spliced into the fully connected layer, and the classification results our outputted through Softmax.A series of experiments were carried out on the Indian Pains and Pavia University and Kennedy Space Center datasets. In the IP data set, the average accuracy reaches 95.0%, the overall accuracy 97.4%, and Kappa 0.97 by training with 5% data sets. In the UP data set, OA, AA, and Kappa reach 97.6%, 97.1%, and 0.97, respectively, by training with a 0.5% data set. The overall accuracy in the KSC data set can reach 99.2%. The network has been proven to strong feature extraction and classification ability.This method effectively improves the classification accuracy of hyperspectral images in the case of small sample sets and studies the effect of training and input data sizes on the classification accuracy. The classification accuracy of the network is improved with the increase in the training or input data. However, redundant information generated by a large amount of training data and excessive input data does not help improve the classification performance.

palabra clave

hyperspectral image classification;deep learning;lightweight network;dense connection;separable convolution

References

  1. 1.
    Archibald R and Fann G. 2007. Feature selection and classification of hyperspectral images with support vector machines. IEEE Geoscience and Remote Sensing Letters, 4(4): 674-677
  2. 2.
    Ball J E and Wei P. 2018. Deep learning hyperspectral image classification using multiple class-based denoising autoencoders, mixed pixel training augmentation, and morphological operations//IGARSS 2018—2018 IEEE International Geoscience and Remote Sensing Symposium. Valencia: IEEE: 6903-6906
  3. 3.
    Chang C I, Zhao X L, Althouse M L G and Pan J J. 1998. Least squares subspace projection approach to mixed pixel classification for hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 36(3): 898-912
  4. 4.
    Cui B G, Ma X D and Xie X Y, 2017. Hyperspectral image de-noising and classification with small training samples. Journal of Remote Sensing, 21(5): 728-738
  5. 5.
    Fu W, Li S T and Fang L Y. 2015. Spectral-spatial hyperspectral image classification via superpixel merging and sparse representation//2015 IEEE International Geoscience and Remote Sensing Symposium. Milan: IEEE: 4971-4974
  6. 6.
    Gao Q S, Lim S and Jia X P. 2018. Hyperspectral image classification using convolutional neural networks and multiple feature learning. Remote Sensing, 10(2): 299
  7. 7.
    He M Y, Li B and Chen H H. 2017. Multi-scale 3D deep convolutional neural network for hyperspectral image classification//2017 IEEE International Conference on Image Processing (ICIP). Beijing: IEEE: 3904-3908
  8. 8.
    Huang G, Liu Z, Laurens V and Weinberger K. Q. 2017. Densely connected convolutional networks//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1063-6919
  9. 9.
    Howard A G, Zhu M L, Chen B, Kalenichenko D, Wang W J, Weyand T, Andreetto M and Adam H. 2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. ArXiv Preprint ArXiv: 1704.04861
  10. 10.
    Ioffe S and Szegedy C. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. ArXiv Preprint ArXiv: 1502.03167
  11. 11.
    Jiao L C, Liang M M, Chen H, Yang S Y, Liu H Y and Cao X H. 2017. Deep fully convolutional network-based spatial distribution prediction for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(10): 5585-5599
  12. 12.
    Joelsson S R , Benediktsson J A and Sveinsson J R . 2005. Random forest classifiers for hyperspectral data//Proceedings of IEEE International Geoscience and Remote Sensing Symposium, 2005(IGARSS '05)
  13. 13.
    Kirsch M, Lorenz S, Zimmermann R, Tusa L, Möckel R, Hödl P, Booysen R, Khodadadzadeh M and Gloaguen R. 2018. Integration of terrestrial and drone-borne hyperspectral and photogrammetric sensing methods for exploration mapping and mining monitoring. Remote Sensing, 10(9): 1366-1366
  14. 14.
    Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada: Curran Associates Inc.: 1097-1105
  15. 15.
    Li F, Lu H C and Zhang P P. 2019a. An innovative multi-kernel learning algorithm for hyperspectral classification. Computers and Electrical Engineering, 79: 106456-106464
  16. 16.
    Li R, Zheng S Y, Duan C X, Yang Y and Wang X Q. 2020. Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sensing, 12(3): 582
  17. 17.
    Li W, Chen C, Su H J and Du Q. 2015. Local binary patterns and extreme learning machine for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 53(7): 3681-3693
  18. 18.
    Li X, Chen S, Hu X L and Yang J. 2019b. Understanding the disharmony between dropout and batch normalization by variance shift//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA: IEEE: 2677-2685
  19. 19.
    Liu X F, Sun Q Q, Meng Y, Fu M and Bourennane S. 2018. Hyperspectral image classification based on parameter-optimized 3D-CNNs combined with transfer learning and virtual samples. Remote Sensing, 10(9): 1425-1425
  20. 20.
    Makantasis K, Karantzalos K, Doulamis A and Doulamis N. 2015. Deep supervised learning for hyperspectral data classification through convolutional neural networks//2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Milan: IEEE: 4959-4962
  21. 21.
    Mei S H, Ji J Y, Hou J H, Li X and Du Q. 2017. Learning sensor-specific spatial-spectral features of hyperspectral images via convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 55(8): 4520-4533
  22. 22.
    Melgani F and Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8): 1778-1790
  23. 23.
    Paoletti M E, Haut J M, Fernandez-Beltran R, Plaza J, Plaza A J and Pla F. 2019. Deep pyramidal residual networks for spectral-spatial hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(2): 740-754
  24. 24.
    Prey L, von Bloh M and Schmidhalter U. 2018. Evaluating RGB imaging and multispectral active and hyperspectral passive sensing for assessing early plant vigor in winter wheat. Sensors, 18(9): 2931
  25. 25.
    Rao M , Tang P and Zhang Z . 2020. A developed siamese CNN with 3D adaptive spatial-spectral pyramid pooling for hyperspectral image classification. Remote Sensing, 12(12):1964
  26. 26.
    Redmon J and Farhadi A. 2018. YOLOv3: An Incremental Improvement. ArXiv E-prints
  27. 27.
    Santurkar S, Tsipras D, Ilyas A, Madry A. 2018. How does batch normalization help optimization? //NeurIPS 2018
  28. 28.
    Simonyan K and Zisserman A. 2014. Very Deep Convolutional Networks for Large-scale Image Recognition//2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  29. 29.
    Srivastava N, Hinton G, Krizhevsky A, Sutskever I and Salakhutdinov R. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1): 1929-1958
  30. 30.
    Sun W W, Zhang D F, Yang G and Li W Y. 2018. Band selection for hyperspectral imagery based on weighted probabilistic archetypal analysis. Journal of Remote Sensing, 22(1): 110-118
  31. 31.
    Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA: IEEE: 1-9
  32. 32.
    Wan S, Gong C, Zhong P, Du B, Zhang L F and Yang J. 2020. Multiscale dynamic graph convolutional network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(5): 3162-3177
  33. 33.
    Wang L Q, Zhao X and Qin Z C. 2021. Hyperspectral image classification with spectral-spatial consistency regularization. Journal of China Academy of Electronics and Information Technology,16(8): 789-796
  34. 34.
    Zhang K, Hei B Q, Zhou Z and Li S Y. 2018. CNN with coefficient of variation-based dimensionality reduction for hyperspectral remote sensing images classification. Journal of Remote Sensing, 22(1): 87-96
  35. 35.
    Zheng Q, Huang W J, Cui X M, Dong Y Y, Shi Y, Ma H Q and Liu L Y. 2018. Identification of wheat yellow rust using optimal three-band spectral indices in different growth stages. Sensors, 19(1): 35
  36. 36.
    Zhong Z L, Li J, Luo Z M and Chapman M. 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858
  37. 37.
    Zhou Y J and Tian Q J. 2008. Image quality evaluation of EO-1 hyperion sensor. Geo-information Science, 10(5):678-683

Leer el texto completo

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website