多图卷积网络的遥感图像小样本分类

陈杰虎; 汪西莉

doi:10.11834/jrs.20210522

遥感智能解译 | 浏览量 : 0 下载量: 194 CSCD: 0

R-PDF
PDF
导出
分享
收藏
专辑

多图卷积网络的遥感图像小样本分类
Multi-graph convolutional network for a remote sensing image few shot classification
2022年26卷第10期页码：2029-2042
DOI： 10.11834/jrs.20210522

扫描看全文

引用
陈杰虎，汪西莉.2022.多图卷积网络的遥感图像小样本分类.遥感学报，26（10）： 2029-2042

Chen J H and Wang X L. 2022. Multi-graph convolutional network for a remote sensing image few shot classification. National Remote Sensing Bulletin， 26（10）：2029-2042
陈杰虎，汪西莉.2022.多图卷积网络的遥感图像小样本分类.遥感学报，26（10）： 2029-2042 DOI： 10.11834/jrs.20210522.

Chen J H and Wang X L. 2022. Multi-graph convolutional network for a remote sensing image few shot classification. National Remote Sensing Bulletin， 26（10）：2029-2042 DOI： 10.11834/jrs.20210522.

摘要

小样本学习旨在利用非常少的监督信息识别出新的类别，由于忽视了样本之间的关联信息，现有的小样本分类方法用于遥感图像小样本分类时往往不能获得令人满意的精度。为此，本文利用图来建模图像在特征空间的相似关系，使用图卷积运算平滑同类别图像的特征，增强不同类别图像特征的区分度，提升分类精度。所提方法在现有图卷积运算的基础上，使用多阶次的邻接矩阵线性加权的方法代替传统的一阶邻接矩阵，通过图谱分析得出这种改进方法能够让不同阶次邻接矩阵的频率响应函数在高频部分正负相抵，有效抑制图信号的高频分量，更显著的提升同类别节点特征的聚集程度；同时，在训练过程引入了微调的方法，使用新类别中的标记数据对最后一层图卷积网络进行少量次数的训练，能够进一步提高精度，增强模型的迁移能力。实验使用AID、OPTIMAL31以及RSI-CB256这3个常用的遥感数据集对方法的有效性进行了测试，结果表明提出的方法在同数据集小样本分类任务和跨数据集小样本分类任务中，在分类精度方面均优于原型网络等比较方法。

Abstract

Few shot classification is a hot topic in the field of deep learning, which aims at identifying novel concepts with little supervisory information. In the remote sensing scene few shot classification, existing methods are often unable to achieve satisfactory accuracy because the relationships among samples are ignored. To improve the RS few shot classification accuracy, the multi-graph convolutional network (Multi-GCN) is proposed in this paper. In the proposed method, a graph convolutional network is introduced into the metric network to smooth the samples’ features, which can model relationships among samples, make images from the same class get more similar feature representation and improve the classification accuracy. The proposed Multi-GCN is mainly composed of three parts: (1) Feature extraction network, which is composed of 4-layer convolutional neural networks and is used to extract images’ features; (2) Graph convolutional network, which is used to model the relationships among samples in the feature space, and update the node features by multi-graph convolution; (3) Metric prediction part, which is used to calculate the prototype of each class, and predict labels of the unlabeled samples according to the distance between samples and prototypes. Based on spectral domain analysis, the proposed multi-graph convolution can effectively suppress the high-frequency components of the graph signals and significantly enhance the clustering coefficients of features from the same class. In addition, the fine-tuning method is introduced in the training process, and labeled samples in the new class are used to train the last layer of the graph convolutional network for a small number of steps, which can further improve the classification accuracy and enhance the transfer ability of the model. To validate the effectiveness of the proposed method, in the experiments, Multi-GCN is compared with ProtoNet, GNN-FSL, and two single graph based methods ProtoGCN and ProtoIGCN on two different tasks, i.e. the same dataset few shot classification task and cross datasets few shot classification task. From the experimental results, we can obtain that in the same dataset few shot classification task, the proposed method is significantly superior to ProtoNet. When the number of labeled samples of each class is 1, the accuracy of the proposed method is higher than ProtoNet by more than 5%; Compared with GNN-FSL, the accuracy of the proposed method is about 10% higher on average. In the cross datasets few shot classification task, the classification accuracy of the proposed method is 10%—15% higher than that of GNN-FSL; And compared with ProtoNet, it is about 10% higher in the case of 1-shot and about 2%—5% higher in the case of 5-shot. In most cases, Multi-GCN is also better than single graph methods, such as ProtoGCN and ProtoIGCN. The classification accuracy can be further improved by using fine-tuning training methods. From the above experimental results, we can conclude that the proposed method can achieve a higher classification accuracy compared with ProtoNet, GNN-FSL, and the methods based on a single graph convolutional network. The conclusions of multi-graph convolutional operation and spectrum analysis in this work can also be extended to other graph-based semi-supervised learning.

关键词

小样本学习遥感场景分类度量学习多图卷积网络图谱分析

Keywords

few shot learningremote sensing scene classificationmetric learningmulti-graph convolutiongraph spectral analysis

references

Chen W Y, Liu Y C, Kira Z, Wang Y C and Huang J B. 2019. A closer look at few-shot classification//Proceedings of the 7th International Conference on Learning Representations. New Orleans, United State: ICLR: 1-17

Cheng G, Xie X X, Han J W, Guo L and Xia G S. 2020. Remote sensing image scene classification meets deep learning: challenges, methods, benchmarks, and opportunities. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 3735-3756 [DOI: 10.1109/jstars.2020.3005403http://dx.doi.org/10.1109/jstars.2020.3005403]

Deng P F, Xu K J and Huang H. 2021. CNN-GCN-based dual-stream network for scene classification of remote sensing images. Journal of Remote Sensing, 25(11):2270-2282

邓培芳 ,徐科杰, 黄鸿. 2021. 基于CNN-GCN双流网络的高分辨率遥感影像场景分类. 遥感学报, 25(11): 2270-2282 [DOI: 10.11834/jrs.20210587http://dx.doi.org/10.11834/jrs.20210587]

Garcia V and Bruna J. 2018. Few-shot learning with graph neural networks//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada: ICLR: 1-13

Hoang N T and Maehara T. 2019. Revisiting graph neural networks: all we have is low-pass filters. arXiv: 1905.09550

Hong D F, Gao L R, Yao J, Zhang B, Plaza A and Chanussot J. 2021. Graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(7): 5966-5978 [DOI: 10.1109/TGRS.2020.3015157http://dx.doi.org/10.1109/TGRS.2020.3015157]

Kipf T N and Welling M. 2017. Semi-supervised classification with graph convolutional networks//Proceedings of the 5th International Conference on Learning Representations. Toulon, France: ICLR: 1-14

Koch G, Zemel R and Salakhutdinov R. 2015. Siamese neural networks for one-shot image recognition//Proceedings of the 32nd International Conference on Machine Learning. Lille, France: ICML: 1-9

Li Q M, Wu X M, Liu H, Zhang X T and Guan Z C. 2019a. Label efficient semi-supervised learning via graph filtering//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA: IEEE: 9574-9583 [DOI: 10.1109/CVPR.2019.00981http://dx.doi.org/10.1109/CVPR.2019.00981]

Li W B, Wang L, Xu J L, Huo J, Gao Y and Luo J B. 2019b. Revisiting local descriptor based image-to-class measure for few-shot learning//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA: IEEE: 7253-7260 [DOI: 10.1109/CVPR.2019.00743http://dx.doi.org/10.1109/CVPR.2019.00743]

Liu K, Zhou Z, Li S Y, Liu Y F, Wan X, Liu Z W, Tan H and Zhang W F. 2020. Scene classification dataset using the Tiangong-1 hyperspectral remote sensing imagery and its applications. Journal of Remote Sensing, 24(9): 1077-1087

刘康, 周壮, 李盛阳, 刘云飞, 万雪, 刘志文, 谭洪, 张万峰. 2020. 天宫一号高光谱遥感场景分类数据集及应用. 遥感学报, 24(9): 1077-1087 [DOI: 10.11834/jrs.20209323http://dx.doi.org/10.11834/jrs.20209323]

Liu X R, Jin W, Ma Y, Li Y X, Wang Y Q, Yan M and Tang J L. 2021. Elastic graph neural networks//Proceedings of the 38th International Conference on Machine Learning. Vienna, Austria: ICML: 6837-6849

Lu J, Gong P H, Ye J P and Zhang C S. 2020. Learning from very few samples: a survey. arXiv: 2009.02653

Mou L C, Lu X Q, Li X L and Zhu X X. 2020. Nonlocal graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 58(12): 8246-8257 [DOI: 10.1109/TGRS.2020.2973363http://dx.doi.org/10.1109/TGRS.2020.2973363]

Ouyang S B, Chen W T, Li X J, Dong Y S and Wang L Z. 2022. Geomorphological scene classification dataset of high-resolution remote sensing imagery in vegetation-covered areas. Journal of Remote Sensing, 26(4):606-619

欧阳淑冰, 陈伟涛, 李显巨, 董玉森，王力哲. 2022. 植被覆盖区高精度遥感地貌场景分类数据集. 遥感学报, 26(4):606-619 [DOI：10.11834/jrs.20221385http://dx.doi.org/10.11834/jrs.20221385]

Scott G J, Hagan K C, Marcum R A, Hurt J A, Anderson D T and Davis C H. 2018. Enhanced fusion of deep neural networks for classification of benchmark high-resolution image data sets. IEEE Geoscience and Remote Sensing Letters, 15(9): 1451-1455 [DOI: 10.1109/LGRS.2018.2839092http://dx.doi.org/10.1109/LGRS.2018.2839092]

Shuman D I, Narang S K, Frossard P, Ortega A and Vandergheynst P. 2013. The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 30(3): 83-98 [DOI: 10.1109/MSP.2012.2235192http://dx.doi.org/10.1109/MSP.2012.2235192]

Snell J, Swersky K and Zemel R. 2017. Prototypical networks for few-shot learning//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, California: Curran Associates Inc.: 4080-4090.

Sung F, Yang Y X, Zhang L, Xiang T, Torr P H S and Hospedales T M. 2018. Learning to compare: relation network for few-shot learning//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE: 1199-1208 [DOI: 10.1109/CVPR.2018.00131http://dx.doi.org/10.1109/CVPR.2018.00131]

Vinyals O, Blundell C, Lillicrap T, Kavukcuoglu K and Wierstra D. 2016. Matching networks for one shot learning//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: Curran Associates Inc.: 3637-3645

Wan S, Gong C, Zhong P, Pan S R, Li G Y and Yang J. 2021. Hyperspectral image classification with context-aware dynamic graph convolutional network. IEEE Transactions on Geoscience and Remote Sensing, 59(1): 597-612 [DOI: 10.1109/TGRS.2020.2994205http://dx.doi.org/10.1109/TGRS.2020.2994205]

Wang Q, Liu S T, Chanussot J and Li X L. 2019. Scene classification with recurrent attention of VHR remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 57(2): 1155-1167 [DOI: 10.1109/TGRS.2018.2864987http://dx.doi.org/10.1109/TGRS.2018.2864987]

Wu F, Zhang T Y, de Souza A H, Fifty C, Yu T and Weinberger K Q. 2019. Simplifying graph convolutional networks//Proceedings of the 36th International Conference on Machine Learning. Long Beach, United States: ICML: 6861-6871

Yang S Q, Jin Z, Yu Y L and Lin H. 2019. Analysis of the spatial-temporal characteristics of LUCC from 1975 to 2015 in the Dongzhi loess tableland. Quaternary Sciences, 39(6): 1433-1442

杨思齐, 金钊, 余云龙, Henry Lin. 2019. 1975—2015年董志塬土地利用/覆被变化及人类活动影响. 第四纪研究, 39(6): 1433-1442 [DOI: 10.11928/j.issn.1001-7410.2019.06.11http://dx.doi.org/10.11928/j.issn.1001-7410.2019.06.11]

Yu D H, Zhang B M, Zhao C, Guo H T and Lu J. 2020. Scene classification of remote sensing image using ensemble convolutional neural network. Journal of Remote Sensing, 24(6): 717-727

余东行, 张保明, 赵传, 郭海涛, 卢俊. 2020. 联合卷积神经网络与集成学习的遥感影像场景分类. 遥感学报, 24(6): 717-727 [DOI: 10.11834/jrs.20208273http://dx.doi.org/10.11834/jrs.20208273]

Zhou W X, Newsam S, Li C M and Shao Z F. 2018. PatternNet: a benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS Journal of Photogrammetry and Remote Sensing, 145: 197-209 [DOI: 10.1016/j.isprsjprs.2018.01.004http://dx.doi.org/10.1016/j.isprsjprs.2018.01.004]

文章被引用时，请邮件提醒。

提交

植被覆盖区高精度遥感地貌场景分类数据集

基于CNN-GCN双流网络的高分辨率遥感影像场景分类