Scene classification of remote sensing image using ensemble convolutional neural network

Donghang YU; Baoming ZHANG; Chuan ZHAO; Haitao GUO; Jun LU

doi:10.11834/jrs.20208273

Technology and Methodology | Views : 0 下载量: 2581 CSCD: 28

PDF
Export
Share
Collection
Album

Scene classification of remote sensing image using ensemble convolutional neural network
Vol. 24, Issue 6, Pages: 717-727(2020)
Received：16 July 2018，

Published：07 June 2020
DOI： 10.11834/jrs.20208273
稿件说明：

移动端阅览

余东行,张保明,赵传,郭海涛,卢俊.2020.联合卷积神经网络与集成学习的遥感影像场景分类.遥感学报,24(6): 717-727 DOI： 10.11834/jrs.20208273.

YU Donghang,ZHANG Baoming,ZHAO Chuan,GUO Haitao,LU Jun. 2020. Scene classification of remote sensing image using ensemble convolutional neural network. Journal of Remote Sensing(Chinese)，24(6): 717-727 DOI： 10.11834/jrs.20208273.

摘要

针对人工设计的中、低层特征难以实现复杂场景影像的高精度分类以及卷积神经网络依赖大量训练数据等问题，结合迁移学习与集成学习，提出了一种联合卷积神经网络与集成学习的遥感影像场景分类算法。首先基于迁移学习的思想，利用在自然影像数据集上训练好的多个深层卷积神经网络模型作为特征提取器，提取图像多个高度抽象的语义特征；然后构建由Logistic回归和支持向量机组成的Stacking集成模型，对同一图像的多个特征分别训练Logistic模型，将预测概率结果融合构建概率特征；最后利用支持向量机对概率特征训练和预测，得到场景影像的分类结果。利用UCMerced_LandUse和NWPU-RESISC 45两种不同规模的遥感影像数据集进行试验，即使在只有10%的数据作为训练样本情况下，本文方法能够分别达到90.74%和87.21%的分类精度。

Abstract

Scene classification and recognition of remote sensing image is an important task for image interpretation. High-resolution remote sensing images have rich spatial texture features and semantic information

and their scene categories are diversified. As a result

images in the same category have a huge difference and some images in different categories become similar. which makes images difficult to be classified and recognized correctly. Therefore

choosing effective features and classification algorithms can improve classification performance. In this case

high-precision classification can only be achieved by selecting effective features and classifiers.

Traditional scene classification algorithms adopt low-level or mid-level handcrafted features. These features have poor ability to represent high-level semantic information of images

which makes it difficult to achieve satisfactory results on massive complex scene images difficult. Deep learning

especially convolutional neural networks

has made great progress in computer vision. Compared with the methods using handcrafted features

deep learning is currently the most effective way for image classification. The application of a convolutional neural network to remote sensing image classification has achieved higher precision than methods using traditional handcrafted features. However

training a deep convolutional neural network that has too many parameters needs many labeled images

and the process of training is complicated and time-consuming. Generally

a deep convolutional neural network would not perform well with only a few images.

A method for image classification using an ensemble convolutional neural network is proposed to improve the performance of convolutional neural networks. The method is composed of three main phases

namely

preprocessing

feature extraction

and ensemble learning. Firstly

the preprocessing stage includes geometry normalization

image intensity normalization

and image augmentation. Secondly

the feature extraction phase considers several deep convolutional neural networks

which have been well pre-trained on ImageNet

and are chosen to remove the last classification layer in the network and to extract different deep features of the same image. Thirdly

a stacking model is constructed in the ensemble learning phase. The stacking model consists of base and meta classifiers. The base classifier is composed of several logistic regression modes that are used to train different features extracted by deep convolutional neural networks. The meta classifier is a support vector machine. Finally

the probability distribution predicted by the base classifier is used to construct a new dataset that would be trained by the meta classifier.

Experiments were conducted on two datasets named UCMerced_LandUse and NWPU-RESISC45 to verify the effectiveness of the proposed method. Compared with state-of-the-art methods

the proposed method performed better in overall accuracies. The proposed method could greatly improve performance and achieve overall accuracies of 90.74% and 87.21% on the two datasets

respectively

even with only 10% data used for training.

With transfer learning

the features extracted by the deep convolutional neural networks are highly abstract and semantic

which have better ability in classification than other handcrafted features. Through feature fusion and model transferring

the advantages of different features and classification methods could be synthetically utilized. In this way

high classification accuracy could be achieved even with very little training data.

关键词

Keywords

references

Chang L , Deng X M , Zhou M Q , Wu Z K , Yuan Y , Yang S and Wang H A . 2016 . Convolutional neural networks in image understanding . Acta Automatica Sinica , 42 ( 9 ): 1300 - 1312

常亮 , 邓小明 , 周明全 , 武仲科 , 袁野 , 杨硕 , 王宏安 . 2016 . 图像理解中的卷积神经网络 . 自动化学报 , 42 ( 9 ): 1300-1312 [ DOI: 10.16383/j.aas.2016.c150800 http://dx.doi.org/10.16383/j.aas.2016.c150800 ]

Cheng G , Han J W and Lu X Q . 2017 . Remote sensing image scene classification: benchmark and state of the art . Proceedings of the IEEE , 105 ( 10 ): 1865 - 1883 [ DOI: 10.1109/JPROC.2017.2675998 http://dx.doi.org/10.1109/JPROC.2017.2675998 ]

Chollet F . 2015 . Keras [EB/OL][2018-05-01] . https://github.com/fchollet/keras https://github.com/fchollet/keras

Chollet F . 2017 . Xception: deep learning with depthwise separable convolutions // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA : IEEE : 1800 - 1807 [DOI: 10.1109/CVPR.2017.195]

Dalal N and Triggs B . 2005 . Histograms of oriented gradients for human detection // Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . San Diego : IEEE : 886 - 893 [DOI: 10.1109/CVPR.2005.177]

He K M , Zhang X Y , Ren S Q , Sun J and Research M . 2016 . Deep residual learning for image recognition // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA : IEEE : 770 - 778 [DOI: 10.1109/CVPR.2016.90]

He X F , Zou Z R , Tao C and Zhang J X . 2016 . Combined saliency with multi-convolutional neural network for high resolution remote sensing scene classification . Acta Geodaetica et Cartographica Sinica , 45 ( 9 ): 1073 - 1080

何小飞 , 邹峥嵘 , 陶超 , 张佳兴 . 2016 . 联合显著性和多层卷积神经网络的高分影像场景分类 . 测绘学报 , 45 ( 9 ): 1073-1080 [ DOI: 10.11947/J.AGCS.2016.20150612 http://dx.doi.org/10.11947/J.AGCS.2016.20150612 ]

Hu F , Xia G S , Hu J W and Zhang L P . 2015 . Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery . Remote Sensing , 7 ( 11 ): 14680-1470 7 [ DOI: 10.3390/Rs71114680 http://dx.doi.org/10.3390/Rs71114680 ]

Huang L H , Chen C , Li W and Du Q . 2016 . Remote sensing image scene classification using multi-scale completed local binary patterns and fisher vectors . Remote Sensing , 8 ( 6 ): 483 [ DOI: 10.3390/Rs8060483 http://dx.doi.org/10.3390/Rs8060483 ]

Krizhevsky A , Sutskever I and Hinton G E . 2012 . ImageNet classification with deep convolutional neural networks // Proceedings of the 25th International Conference on Neural Information Processing Systems . Lake Tahoe, Nevada : Curran Associates Inc .: 1907 - 1105

Nogueira K , Penatti O A B and dos Santos J A . 2016 . Towards better exploiting convolutional neural networks for remote sensing scene classification . Pattern Recognition , 61 : 539 - 556 [ DOI: 10.1016/j.patcog.2016.07.001 http://dx.doi.org/10.1016/j.patcog.2016.07.001 ]

Pan S J and Yang Q . 2010 . A survey on transfer learning . IEEE Transactions on Knowledge and Data Engineering , 22 ( 10 ): 1345 - 1359 [ DOI: 10.1109/Tkde.2009.191 http://dx.doi.org/10.1109/Tkde.2009.191 ]

Sheng G F , Yang W , Xu T and Sun H . 2012 . High-resolution satellite scene classification using a sparse coding based multiple feature combination . International Journal of Remote Sensing , 33 ( 8 ): 2395 - 2412 [ DOI: 10.1080/01431161.2011.608740 http://dx.doi.org/10.1080/01431161.2011.608740 ]

Simonyan K and Zisserman A . 2015 . Very Deep convolutional networks for large-scale image recognition // Proceedings of 2015 IEEE International Conference on Learning Representations . San Diego, CA, USA : IEEE : 1 - 14

Szegedy C , Vanhoucke V , Ioffe S , Shlens J and Wojna Z . 2015 . Rethinking the inception architecture for computer vision // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA : IEEE : 2818 - 2826 [DOI: 10.1109/CVPR.2016.308]

Wang J , Luo C , Huang H Q , Zhao H Z and Wang S WQ . 2017 . Transferring pre-trained deep cnns for remote scene classification with general features learned from linear PCA network . Remote Sensing , 9 ( 3 ): 225 [ DOI: 10.3390/Rs9030225 http://dx.doi.org/10.3390/Rs9030225 ]

Weiss K , Khoshgoftaar T M and Wang D D . 2016 . A survey of transfer learning . Journal of Big Data , 3 : 9 [ DOI: 10.1186/S40537-016-0043-6 http://dx.doi.org/10.1186/S40537-016-0043-6 ]

Wu H , Liu B Z , Su W C , Zhang W C and Sun J G . 2016 . Hierarchical coding vectors for scene level land-use classification . Remote Sensing , 8 : 436 [ DOI: 10.3390/Rs8050436 http://dx.doi.org/10.3390/Rs8050436 ]

Xu S H , Mu X D , Zhao P and Ma J . 2016 . Scene classification of remote sensing image based on multi-scale feature and deep neural network . Acta Geodaetica et Cartographica Sinica , 45 ( 7 ): 834 - 840

许凤晖 , 慕晓冬 , 赵鹏 , 马骥 . 2016 . 利用多尺度特征与深度网络对遥感影像进行场景分类 . 测绘学报 , 45 ( 7 ): 834-840 [ DOI: 10.11947/J.AGCS.2016.20150623 http://dx.doi.org/10.11947/J.AGCS.2016.20150623 ]

Yan L , Zhu R X , Liu Y and Mu N . 2017 . Scene classification of remote sensing images by optimizing visual vocabulary concerning scene label information . Journal of Remote Sensing , 21 ( 2 ): 280 - 290

闫利 , 朱睿希 , 刘异 , 莫楠 . 2017 . 顾及遥感影像场景类别信息的视觉单词优化分类 . 遥感学报 , 21 ( 2 ): 280-290 [ DOI: 10.11834/Jrs.201761971 http://dx.doi.org/10.11834/Jrs.201761971 ]

Yang Y and Newsam S . 2010 . Bag-of-visual-words and spatial extensions for land-use classification // Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems . San Jose, California : ACM : 270 - 279 [DOI: 10.1145/1869790.1869829]

Yin J H , Li H and Jia X P . 2015 . Crater detection based on GIST features . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 8 ( 1 ): 23 - 29 [ DOI: 10.1109/Jstars.2014.2375066 http://dx.doi.org/10.1109/Jstars.2014.2375066 ]

Zhang F , Du B and Zhang L P . 2016 . Scene Classification via a gradient boosting random convolutional network framework . IEEE Transactions on Geoscience and Remote Sensing , 54 ( 3 ): 1793 - 1802 [ DOI: 10.1109/Tgrs.2015.248868 http://dx.doi.org/10.1109/Tgrs.2015.248868 ]

Zheng Z , Fang F , Liu Y Y , Gong X , Guo M Q and Luo Z W . 2018 . Joint multi-scale convolution neural network for scene classification of high resolution remote sensing imagery . Acta Geodaetica et Cartographica Sinica , 47 ( 5 ): 620 - 630

郑卓 , 方芳 , 刘袁缘 , 龚希 , 郭明强 , 罗忠文 . 2018 . 高分辨率遥感影像场景的多尺度神经网络分类法 . 测绘学报 , 47 ( 5 ): 620-630 [ DOI: 10.11947/J.AGCS.2018.20170191 http://dx.doi.org/10.11947/J.AGCS.2018.20170191 ]

Zhou F Y , Jin L P and Dong J . 2017 . Review of convolutional neural network . Chinese Journal of Computers , 40 ( 6 ): 1229 - 1251

周飞燕 , 金林鹏 , 董军 . 2017 . 卷积神经网络研究综述 . 计算机学报 , 40 ( 6 ): 1229-1251 [ DOI: 10.11897/SP.J.1016.2017.01229 http://dx.doi.org/10.11897/SP.J.1016.2017.01229 ]

Zhou W X , Newsam S , Li C M and Shao Z F . 2017 . Learning low dimensional convolutional neural networks for high-resolution remote sensing image retrieval . Remote Sensing , 9 ( 5 ): 489 [ DOI: 10.3390/Rs9050489 http://dx.doi.org/10.3390/Rs9050489 ]

Zhou Z H . 2012 . Ensemble Methods: Foundations and Algorithms . New York : CRC Press : 83 - 84

Zhou Z H . 2016 . Machine Learning. Beijing: Tsinghua University Press .

周志华 . 机器学习. 北京 : 清华大学出版社 , 2016

Alert me when the article has been cited

提交

ADC-CPANet：A remote sensing image classification method based on local-global feature fusion

MtSCCD： Land-use scene classification and change-detection dataset for deep learning

Hyperspectral Image Cross-Domain Few-Shot Classification Based on Disentangled Confidence Prototype Network

Multimodal object detection method using adaptive dusion of infrared and visible features