收稿：2021-01-07，
纸质出版：2021-11-07
移动端阅览
高分辨率遥感影像具有复杂的几何结构和空间布局，传统的卷积神经网络的方法仅能提取场景图像中的全局特征，忽略了上下文的关系，导致特征的表达能力受限，制约了分类精度提高。针对此问题，本文提出一个面向高分辨率遥感影像场景分类的CNN-GCN双流网络，该算法包含CNN流和GCN流两个模块。CNN流基于预训练DenseNet-121网络提取高分影像的全局特征；而GCN流采用由预训练VGGNet-16网络得到的卷积特征图构建邻接图，再通过GCN模型提取高分影像的上下文特征。最后，通过加权级联的方式有效地融合全局特征和上下文特征并利用线性分类器实现分类。本文选取AID、RSSCN7和NWPU-RESISC45共3个具有挑战性的数据集进行实验，得到的最高分类精度分别是97.14%、95.46%和94.12%，结果表明本文算法能够有效地表征场景并取得具有竞争力的分类结果。
Scene classification is an important research topic
which aims at assigning a semantic label to a given image. High-Spatial-Resolution (HSR) images contain abundant information of ground objects
such as geometric structure and spatial layout. Complex HSR images are difficult to interpret effectively. Extracting discriminative features is the key step to improve classification accuracy. Various methods for constructing discriminative representations
including handcrafted feature-based methods and deep learning-based methods
have been proposed. The former methods focus on designing different handcrafted features via professional knowledge and describing a scene through single feature or multifeature fusion. However
for a complex scene
handcrafted features show limited discriminative and generalization capabilities. Deep learning-based methods
due to the powerful capability of feature extraction
have made incredible progress in the field of scene classification. Compared with the former methods
Convolutional Neural Networks (CNNs) can automatically extract deep features from massive HSR images. Nevertheless
CNNs merely focus on global information
which makes it fail to explore the context relationship of HSR images. Recently
Graph Convolutional Networks (GCNs) have become an important branch of deep learning
and they have been adopted to model spatial relations hidden in HSR images via graph structure. In this paper
a novel architecture termed CNN-GCN-based Dual-Stream Network (CGDSN) is proposed for scene classification. The CGDSN method contains two modules: CNN and GCN streams. For the CNN stream
the pretrained DenseNet-121 is employed as the backbone to extract the global features of HSR images. In the GCN stream
VGGNet-16 that is pretrained well on ImageNet is introduced to generate feature maps of the last convolutional layer. Then
an average pooling is organized for downsampling before the construction of an adjacency matrix. Given that every image is represented by a graph
a GCN model is developed to demonstrate context relationships. Two graph convolutional layers of the GCN stream are followed by a global average pooling layer and a Fully Connected (FC) layer to form the context features of HSR images. Lastly
to fuse global and context features adequately
a weighted concatenation layer is constructed to integrate them
and an FC layer is introduced to predict scene categories. The AID
RSSCN7
and NWPU-RESISC45 data sets are chosen to verify the effectiveness of the CGDSN method. Experimental results illustrate that the proposed CGDSN algorithm outperforms some state-of-the-art methods in terms of Overall Accuracies (OAs). On the AID data set
the OAs reach 95.62% and 97.14% under the training ratios of 20% and 50%
respectively. On the RSSCN7 data set
the classification result obtained by the CGDSN method is 95.46% with 50% training samples. For the NWPU-RESISC45 data set
the classification accuracies achieved via the CGDSN method are 91.86% and 94.12% under the training ratios of 10% and 20%
respectively. The proposed CGDSN method can extract discriminative features and achieve competitive accuracies for scene classification.
Anwer R M , Khan F S , van de Weijer J , Molinier M and Laaksonen J . 2018 . Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification . ISPRS Journal of Photogrammetry and Remote Sensing , 138 : 74 - 85 [ DOI: 10.1016/j.isprsjprs.2018.01.023 http://dx.doi.org/10.1016/j.isprsjprs.2018.01.023 ]
Chaib S , Liu H , Gu Y F and Yao H X . 2017 . Deep feature fusion for VHR remote sensing scene classification . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 8 ): 4775 - 4784 [ DOI: 10.1109/TGRS.2017.2700322 http://dx.doi.org/10.1109/TGRS.2017.2700322 ]
Cheng G , Han J W and Lu X Q . 2017 . Remote sensing image scene classification: benchmark and state of the art . Proceedings of the IEEE , 105 ( 10 ): 1865 - 1883 [ DOI: 10.1109/JPROC.2017.2675998 http://dx.doi.org/10.1109/JPROC.2017.2675998 ]
Cheng G , Yang C Y , Yao X W , Guo L and Han J W . 2018 . When deep learning meets metric learning: remote sensing image scene classification via learning discriminative CNNs . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 5 ): 2811 - 2821 [ DOI: 10.1109/TGRS.2017.2783902 http://dx.doi.org/10.1109/TGRS.2017.2783902 ]
Gao Y , Shi J , Li J and Wang R Y . 2021 . Remote sensing scene classification based on high-order graph convolutional network . European Journal of Remote Sensing , 54 ( Sup1 ): 141 - 155 [ DOI: 10.1080/22797254.2020.1868273 http://dx.doi.org/10.1080/22797254.2020.1868273 ]
He N J , Fang L Y , Li S T , Plaza J and Plaza A . 2020 . Skip-connected covariance network for remote sensing scene classification . IEEE Transactions on Neural Networks and Learning Systems , 31 ( 5 ): 1461 - 1474 [ DOI: 10.1109/TNNLS.2019.2920374 http://dx.doi.org/10.1109/TNNLS.2019.2920374 ]
Hong D F , Gao L R , Yao J , Zhang B , Plaza A and Chanussot J . 2021 . Graph convolutional networks for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 59 ( 7 ): 5966 - 5978 [ DOI: 10.1109/TGRS.2020.3015157 http://dx.doi.org/10.1109/TGRS.2020.3015157 ]
Huang G , Liu Z , Van Der Maaten L and Weinberger K Q . 2017 . Densely connected convolutional networks // Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA : IEEE : 2261 - 2269 [ DOI: 10.1109/CVPR.2017.243 http://dx.doi.org/10.1109/CVPR.2017.243 ]
Huang H and Xu K J . 2019 . Combing triple-part features of convolutional neural networks for scene classification in remote sensing . Remote Sensing , 11 ( 14 ): 1687 [ DOI: 10.3390/rs11141687 http://dx.doi.org/10.3390/rs11141687 ]
Huang H , Xu K J and Shi G Y . 2020 . Scene classification of high-resolution remote sensing image by multi-scale and multi-feature fusion . Acta Electronica Sinica , 48 ( 9 ): 1824 - 1833
黄鸿 , 徐科杰 , 石光耀 . 2020 . 联合多尺度多特征的高分遥感图像场景分类 . 电子学报 , 48 ( 9 ): 1824- 1833 [ DOI: 10.3969/j.issn.0372-2112.2020.09.021 http://dx.doi.org/10.3969/j.issn.0372-2112.2020.09.021 ]
Kang J , Fernandez-Beltran R , Hong D F , Chanussot J and Plaza A . 2021 . Graph relation network: modeling relations between scenes for multilabel remote-sensing image classification and retrieval . IEEE Transactions on Geoscience and Remote Sensing , 59 ( 5 ): 4355 - 4369 [ DOI: 10.1109/TGRS.2020.3016020 http://dx.doi.org/10.1109/TGRS.2020.3016020 ]
Kipf T N and Welling M . 2017 . Semi-supervised classification with graph convolutional networks // Proceedings of 2017 International Conference on Learning Representations . Toulon, France : [s. n.]
Liang J L , Deng Y F and Zeng D . 2020 . A deep neural network combined CNN and GCN for remote sensing scene classification . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13 : 4325 - 4338 [ DOI: 10.1109/JSTARS.2020.3011333 http://dx.doi.org/10.1109/JSTARS.2020.3011333 ]
Liu B D , Meng J , Xie W Y , Shao S , Li Y and Wang Y J . 2019 . Weighted spatial pyramid matching collaborative representation for remote-sensing-image scene classification . Remote Sensing , 11 ( 5 ): 518 [ DOI: 10.3390/rs11050518 http://dx.doi.org/10.3390/rs11050518 ]
Liu B D , Xie W Y , Meng J , Li Y and Wang Y J . 2018 . Hybrid collaborative representation for remote-sensing image scene classification . Remote Sensing , 10 ( 12 ): 1934 [ DOI: 10.3390/rs10121934 http://dx.doi.org/10.3390/rs10121934 ]
Lu X Q , Sun H and Zheng X T . 2019 . A feature aggregation convolutional neural network for remote sensing scene classification . IEEE Transactions on Geoscience and Remote Sensing , 57 ( 10 ): 7894 - 7906 [ DOI: 10.1109/TGRS.2019.2917161 http://dx.doi.org/10.1109/TGRS.2019.2917161 ]
Qian X L , Li J , Cheng G , Yao X W , Zhao S N , Chen Y B and Jiang L Y . 2018 . Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification . Journal of Remote Sensing , 22 ( 5 ): 758 - 776
钱晓亮 , 李佳 , 程塨 , 姚西文 , 赵素娜 , 陈宜滨 , 姜利英 . 2018 . 特征提取策略对高分辨率遥感图像场景分类性能影响的评估 . 遥感学报 , 22 ( 5 ): 758- 776 [ DOI: 10.11834/jrs.20188015 http://dx.doi.org/10.11834/jrs.20188015 ]
Shi C P , Wang T and Wang L G . 2020 . Branch feature fusion convolution network for remote sensing scene classification . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13 : 5194 - 5210 [ DOI: 10.1109/JSTARS.2020.3018307 http://dx.doi.org/10.1109/JSTARS.2020.3018307 ]
Sun W W , Yang G , Chen C , Chang M H , Huang K , Meng X Z and Liu L Y . 2020 . Development status and literature analysis of China’s earth observation remote sensing satellites . Journal of Remote Sensing , 24 ( 5 ): 479 - 510
孙伟伟 , 杨刚 , 陈超 , 常明会 , 黄可 , 孟祥珍 , 刘良云 . 2020 . 中国地球观测遥感卫星发展现状及文献分析 . 遥感学报 , 24 ( 5 ): 479- 510 [ DOI: 10.11834/jrs.20209464 http://dx.doi.org/10.11834/jrs.20209464 ]
Wang Y B , Zhang L Q , Tong X H , Nie F P , Huang H Y and Mei J . 2018 . LRAGE: learning latent relationships with adaptive graph embedding for aerial scene classification . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 2 ): 621 - 634 [ DOI: 10.1109/LGRS.2017.2752217 http://dx.doi.org/10.1109/LGRS.2017.2752217 ]
Wu H , Liu B Z , Su W H , Zhang W C and Sun J G . 2016 . Deep filter banks for land-use scene classification . IEEE Geoscience and Remote Sensing Letters , 13 ( 12 ): 1895 - 1899 [ DOI: 10.1109/LGRS.2016.2616440 http://dx.doi.org/10.1109/LGRS.2016.2616440 ]
Xia G S , Hu J W , Hu F , Shi B G , Bai X , Zhong Y F , Zhang L P and Lu X Q . 2017 . AID: A benchmark data set for performance evaluation of aerial scene classification . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 7 ): 3965 - 3981 [ DOI: 10.1109/TGRS.2017.2685945 http://dx.doi.org/10.1109/TGRS.2017.2685945 ]
Xu K J , Huang H , Deng P F and Shi G Y . 2020a . Two-stream feature aggregation deep neural network for scene classification of remote sensing images . Information Sciences , 539 : 250 - 268 [ DOI: 10.1016/j.ins.2020.06.011 http://dx.doi.org/10.1016/j.ins.2020.06.011 ]
Xu K J , Huang H , Li Y and Shi G Y . 2020b . Multilayer feature fusion network for scene classification in remote sensing . IEEE Geoscience and Remote Sensing Letters , 17 ( 11 ): 1894 - 1898 [ DOI: 10.1109/LGRS.2019.2960026 http://dx.doi.org/10.1109/LGRS.2019.2960026 ]
Yang Z , Mu X D , Wang S Y and Ma C H . 2018 . Scene classification of remote sensing images based on multiscale features fusion . Optics and Precision Engineering , 26 ( 12 ): 3099 - 3107
杨州 , 慕晓冬 , 王舒洋 , 马晨晖 . 2018 . 基于多尺度特征融合的遥感图像场景分类 . 光学精密工程 , 26 ( 12 ): 3099- 3107 [ DOI: 10.3788/OPE.20182612.3099 http://dx.doi.org/10.3788/OPE.20182612.3099 ]
Yu D H , Zhang B M , Zhao C , Guo H T and Lu J . 2020 . Scene classification of remote sensing image using ensemble convolutional neural network . Journal of Remote Sensing , 24 ( 6 ): 717 - 727
余东行 , 张保明 , 赵传 , 郭海涛 , 卢俊 . 2020 . 联合卷积神经网络与集成学习的遥感影像场景分类 . 遥感学报 , 24 ( 6 ): 717- 727 [ DOI: 10.11834/jrs.20208273 http://dx.doi.org/10.11834/jrs.20208273 ]
Zhang B . 2018 . Remotely sensed big data era and intelligent information extraction . Geomatics and Information Science of Wuhan University , 43 ( 12 ): 1861 - 1871
张兵 . 2018 . 遥感大数据时代与智能信息提取 . 武汉大学学报(信息科学版) , 43 ( 12 ): 1861- 1871 [ DOI: 10.13203/j.whugis20180172 http://dx.doi.org/10.13203/j.whugis20180172 ]
Zhang F , Du B and Zhang L P . 2015 . Saliency-guided unsupervised feature learning for scene classification . IEEE Transactions on Geoscience and Remote Sensing , 53 ( 4 ): 2175 - 2184 [ DOI: 10.1109/TGRS.2014.2357078 http://dx.doi.org/10.1109/TGRS.2014.2357078 ]
Zhang L F , Peng M Y , Sun X J , Cen Y and Tong Q X . 2019 . Progress and bibliometric analysis of remote sensing data fusion methods ( 1992-2018 ). Journal of Remote Sensing , 23 ( 4 ): 603 - 619
张立福 , 彭明媛 , 孙雪剑 , 岑奕 , 童庆禧 . 2019 . 遥感数据融合研究进展与文献定量分析(1992—2018) . 遥感学报 , 23 ( 4 ): 603- 619 [ DOI: 10.11834/jrs.20199073 http://dx.doi.org/10.11834/jrs.20199073 ]
Zhang L F , Zhang Q , Du B , Huang X , Tang Y Y and Tao D C . 2018 . Simultaneous spectral-spatial feature selection and extraction for hyperspectral images . IEEE Transactions on Cybernetics , 48 ( 1 ): 16 - 28 [ DOI: 10.1109/TCYB.2016.2605044 http://dx.doi.org/10.1109/TCYB.2016.2605044 ]
Zhang T , Zheng E R , Shen J G and Gao A T . 2020 . Remote sensing image scene classification based on deep multi-branch feature fusion network . Acta Photonica Sinica , 49 ( 5 ): 0510002
张桐 , 郑恩让 , 沈钧戈 , 高安同 . 2020 . 基于深度多分支特征融合网络的光学遥感场景分类 . 光子学报 , 49 ( 5 ): 0510002 [ DOI: 10.3788/gzxb20204905.0510002 http://dx.doi.org/10.3788/gzxb20204905.0510002 ]
Zhao F A , Mu X D , Yang Z and Yi Z X . 2019 . Hierarchical feature coding model for high-resolution satellite scene classification . Journal of Applied Remote Sensing , 13 ( 1 ): 016520 [ DOI: 10.1117/1.JRS.13.016520 http://dx.doi.org/10.1117/1.JRS.13.016520 ]
Zhu Q Q , Zhong Y F , Zhang L P and Li D R . 2018 . Adaptive deep sparse semantic modeling framework for high spatial resolution image scene classification . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 10 ): 6180 - 6195 [ DOI: 10.1109/TGRS.2018.2833293 http://dx.doi.org/10.1109/TGRS.2018.2833293 ]
Zou Q , Ni L H , Zhang T and Wang Q . 2015 . Deep learning based feature selection for remote sensing scene classification . IEEE Geoscience and Remote Sensing Letters , 12 ( 11 ): 2321 - 2325 [ DOI: 10.1109/LGRS.2015.2475299 http://dx.doi.org/10.1109/LGRS.2015.2475299 ]
相关作者
相关机构
京公网安备11010802024621
