Anterior|Siguiente

CNN-GCN-based dual-stream network for scene classification of remote sensing images

  • role： First author第一作者
  • Affiliation：

    Key Laboratory of Optoelectronic Technology and Systems of the Education Ministry of China,Chongqing University, Chongqing 400044, China

  • Email：dpf19970528@gmail.com
  • Introduction：邓培芳， 1997年生， 女， 硕士研究生， 研究方向为深度学习与遥感场景分类。E-mail： dpf19970528@gmail.com
DENG Peifang1,  
  • Affiliation：

    Key Laboratory of Optoelectronic Technology and Systems of the Education Ministry of China,Chongqing University, Chongqing 400044, China

XU Kejie1,  
  • role： Corresponding author通信作者
  • Affiliation：

    Key Laboratory of Optoelectronic Technology and Systems of the Education Ministry of China,Chongqing University, Chongqing 400044, China

    State Key Laboratory of Coal Mine Disaster Dynamics and Control,Chongqing University, Chongqing 400044, China

  • Email：hhuang@cqu.edu.cn
  • Introduction：黄鸿， 1980年生， 男， 教授， 研究方向为无人机遥感、遥感信息处理、流形学习、稀疏表示等。E-mail： hhuang@cqu.edu.cn
HUANG Hong12*

resumen

Scene classification is an important research topic, which aims at assigning a semantic label to a given image. High-Spatial-Resolution (HSR) images contain abundant information of ground objects, such as geometric structure and spatial layout. Complex HSR images are difficult to interpret effectively. Extracting discriminative features is the key step to improve classification accuracy. Various methods for constructing discriminative representations, including handcrafted feature-based methods and deep learning-based methods, have been proposed. The former methods focus on designing different handcrafted features via professional knowledge and describing a scene through single feature or multifeature fusion. However, for a complex scene, handcrafted features show limited discriminative and generalization capabilities. Deep learning-based methods, due to the powerful capability of feature extraction, have made incredible progress in the field of scene classification. Compared with the former methods, Convolutional Neural Networks (CNNs) can automatically extract deep features from massive HSR images. Nevertheless, CNNs merely focus on global information, which makes it fail to explore the context relationship of HSR images. Recently, Graph Convolutional Networks (GCNs) have become an important branch of deep learning, and they have been adopted to model spatial relations hidden in HSR images via graph structure. In this paper, a novel architecture termed CNN-GCN-based Dual-Stream Network (CGDSN) is proposed for scene classification. The CGDSN method contains two modules: CNN and GCN streams. For the CNN stream, the pretrained DenseNet-121 is employed as the backbone to extract the global features of HSR images. In the GCN stream, VGGNet-16 that is pretrained well on ImageNet is introduced to generate feature maps of the last convolutional layer. Then, an average pooling is organized for downsampling before the construction of an adjacency matrix. Given that every image is represented by a graph, a GCN model is developed to demonstrate context relationships. Two graph convolutional layers of the GCN stream are followed by a global average pooling layer and a Fully Connected (FC) layer to form the context features of HSR images. Lastly, to fuse global and context features adequately, a weighted concatenation layer is constructed to integrate them, and an FC layer is introduced to predict scene categories. The AID, RSSCN7, and NWPU-RESISC45 data sets are chosen to verify the effectiveness of the CGDSN method. Experimental results illustrate that the proposed CGDSN algorithm outperforms some state-of-the-art methods in terms of Overall Accuracies (OAs). On the AID data set, the OAs reach 95.62% and 97.14% under the training ratios of 20% and 50%, respectively. On the RSSCN7 data set, the classification result obtained by the CGDSN method is 95.46% with 50% training samples. For the NWPU-RESISC45 data set, the classification accuracies achieved via the CGDSN method are 91.86% and 94.12% under the training ratios of 10% and 20%, respectively. The proposed CGDSN method can extract discriminative features and achieve competitive accuracies for scene classification.

palabra clave

high-resolution image;remote sensing scene classification;graph neural network;convolutional neural network;Feature fusion

References

  1. 1.
    Anwer R M, Khan F S, van de Weijer J, Molinier M and Laaksonen J. 2018. Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification. ISPRS Journal of Photogrammetry and Remote Sensing, 138: 74-85
  2. 2.
    Chaib S, Liu H, Gu Y F and Yao H X. 2017. Deep feature fusion for VHR remote sensing scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(8): 4775-4784
  3. 3.
    Cheng G, Han J W and Lu X Q. 2017. Remote sensing image scene classification: benchmark and state of the art. Proceedings of the IEEE, 105(10): 1865-1883
  4. 4.
    Cheng G, Yang C Y, Yao X W, Guo L and Han J W. 2018. When deep learning meets metric learning: remote sensing image scene classification via learning discriminative CNNs. IEEE Transactions on Geoscience and Remote Sensing, 56(5): 2811-2821
  5. 5.
    Gao Y, Shi J, Li J and Wang R Y. 2021. Remote sensing scene classification based on high-order graph convolutional network. European Journal of Remote Sensing, 54(Sup1): 141-155
  6. 6.
    He N J, Fang L Y, Li S T, Plaza J and Plaza A. 2020. Skip-connected covariance network for remote sensing scene classification. IEEE Transactions on Neural Networks and Learning Systems, 31(5): 1461-1474
  7. 7.
    Hong D F, Gao L R, Yao J, Zhang B, Plaza A and Chanussot J. 2021. Graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(7): 5966-5978
  8. 8.
    Huang G, Liu Z, Van Der Maaten L and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE: 2261-2269
  9. 9.
    Huang H and Xu K J. 2019. Combing triple-part features of convolutional neural networks for scene classification in remote sensing. Remote Sensing, 11(14): 1687
  10. 10.
    Huang H, Xu K J and Shi G Y. 2020. Scene classification of high-resolution remote sensing image by multi-scale and multi-feature fusion. Acta Electronica Sinica, 48(9): 1824-1833
  11. 11.
    Kang J, Fernandez-Beltran R, Hong D F, Chanussot J and Plaza A. 2021. Graph relation network: modeling relations between scenes for multilabel remote-sensing image classification and retrieval. IEEE Transactions on Geoscience and Remote Sensing, 59(5): 4355-4369
  12. 12.
    Kipf T N and Welling M. 2017. Semi-supervised classification with graph convolutional networks//Proceedings of 2017 International Conference on Learning Representations. Toulon, France: [s. n.]
  13. 13.
    Liang J L, Deng Y F and Zeng D. 2020. A deep neural network combined CNN and GCN for remote sensing scene classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 4325-4338
  14. 14.
    Liu B D, Meng J, Xie W Y, Shao S, Li Y and Wang Y J. 2019. Weighted spatial pyramid matching collaborative representation for remote-sensing-image scene classification. Remote Sensing, 11(5): 518
  15. 15.
    Liu B D, Xie W Y, Meng J, Li Y and Wang Y J. 2018. Hybrid collaborative representation for remote-sensing image scene classification. Remote Sensing, 10(12): 1934
  16. 16.
    Lu X Q, Sun H and Zheng X T. 2019. A feature aggregation convolutional neural network for remote sensing scene classification. IEEE Transactions on Geoscience and Remote Sensing, 57(10): 7894-7906
  17. 17.
    Qian X L, Li J, Cheng G, Yao X W, Zhao S N, Chen Y B and Jiang L Y. 2018. Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification. Journal of Remote Sensing, 22(5): 758-776
  18. 18.
    Shi C P, Wang T and Wang L G. 2020. Branch feature fusion convolution network for remote sensing scene classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 5194-5210
  19. 19.
    Sun W W, Yang G, Chen C, Chang M H, Huang K, Meng X Z and Liu L Y. 2020. Development status and literature analysis of China’s earth observation remote sensing satellites. Journal of Remote Sensing, 24(5): 479-510
  20. 20.
    Wang Y B, Zhang L Q, Tong X H, Nie F P, Huang H Y and Mei J. 2018. LRAGE: learning latent relationships with adaptive graph embedding for aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 621-634
  21. 21.
    Wu H, Liu B Z, Su W H, Zhang W C and Sun J G. 2016. Deep filter banks for land-use scene classification. IEEE Geoscience and Remote Sensing Letters, 13(12): 1895-1899
  22. 22.
    Xia G S, Hu J W, Hu F, Shi B G, Bai X, Zhong Y F, Zhang L P and Lu X Q. 2017. AID: A benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3965-3981
  23. 23.
    Xu K J, Huang H, Deng P F and Shi G Y. 2020a. Two-stream feature aggregation deep neural network for scene classification of remote sensing images. Information Sciences, 539: 250-268
  24. 24.
    Xu K J, Huang H, Li Y and Shi G Y. 2020b. Multilayer feature fusion network for scene classification in remote sensing. IEEE Geoscience and Remote Sensing Letters, 17(11): 1894-1898
  25. 25.
    Yang Z, Mu X D, Wang S Y and Ma C H. 2018. Scene classification of remote sensing images based on multiscale features fusion. Optics and Precision Engineering, 26(12): 3099-3107
  26. 26.
    Yu D H, Zhang B M, Zhao C, Guo H T and Lu J. 2020. Scene classification of remote sensing image using ensemble convolutional neural network. Journal of Remote Sensing, 24(6): 717-727
  27. 27.
    Zhang B. 2018. Remotely sensed big data era and intelligent information extraction. Geomatics and Information Science of Wuhan University, 43(12): 1861-1871
  28. 28.
    Zhang F, Du B and Zhang L P. 2015. Saliency-guided unsupervised feature learning for scene classification. IEEE Transactions on Geoscience and Remote Sensing, 53(4): 2175-2184
  29. 29.
    Zhang L F, Peng M Y, Sun X J, Cen Y and Tong Q X. 2019. Progress and bibliometric analysis of remote sensing data fusion methods (1992-2018). Journal of Remote Sensing, 23(4): 603-619
  30. 30.
    Zhang L F, Zhang Q, Du B, Huang X, Tang Y Y and Tao D C. 2018. Simultaneous spectral-spatial feature selection and extraction for hyperspectral images. IEEE Transactions on Cybernetics, 48(1): 16-28
  31. 31.
    Zhang T, Zheng E R, Shen J G and Gao A T. 2020. Remote sensing image scene classification based on deep multi-branch feature fusion network. Acta Photonica Sinica, 49(5): 0510002
  32. 32.
    Zhao F A, Mu X D, Yang Z and Yi Z X. 2019. Hierarchical feature coding model for high-resolution satellite scene classification. Journal of Applied Remote Sensing, 13(1): 016520
  33. 33.
    Zhu Q Q, Zhong Y F, Zhang L P and Li D R. 2018. Adaptive deep sparse semantic modeling framework for high spatial resolution image scene classification. IEEE Transactions on Geoscience and Remote Sensing, 56(10): 6180-6195
  34. 34.
    Zou Q, Ni L H, Zhang T and Wang Q. 2015. Deep learning based feature selection for remote sensing scene classification. IEEE Geoscience and Remote Sensing Letters, 12(11): 2321-2325

Leer el texto completo

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website