Applicability of weak samples to deep learning crop classification

Qing XU; Jinshui ZHANG; Feng ZHANG; Shuang GE; Zhi YANG; Yaming DUAN

doi:10.11834/jrs.20221127

Agricultural Remote Sensing | Views : 0 下载量: 898 CSCD: 9

R-PDF
PDF
Export
Share
Collection
Album

Applicability of weak samples to deep learning crop classification
Vol. 26, Issue 7, Pages: 1395-1409(2022)
Received：11 March 2021，

Published：07 July 2022
DOI： 10.11834/jrs.20221127
稿件说明：

移动端阅览

许晴，张锦水，张凤，盖爽，杨志，段雅鸣.2022.深度学习农作物分类的弱样本适用性.遥感学报，26（7）： 1395-1409 DOI： 10.11834/jrs.20221127.

Xu Q，Zhang J S，Zhang F，Ge S， Yang Z and Duan Y M. 2022. Applicability of weak samples to deep learning crop classification. National Remote Sensing Bulletin， 26（7）：1395-1409 DOI： 10.11834/jrs.20221127.

摘要

基于大数据驱动的深度学习挖掘图像数据的规律和层次已成为遥感影像解译的研究热点。海量标签样本是训练深度学习模型的前提条件，但成本昂贵的人工标记样本限制了深度学习技术在遥感领域的应用。本文提出了一种基于弱样本的深度学习模型农作物分类策略：以GF-1影像为数据源，将传统分类器SVM分类结果视为弱样本，训练深度卷积网络模型DCNN（Deep Convolutional Neural Networks），获取辽宁省水稻和玉米的空间分布，分析弱样本的适用性。结果显示：测试集总体精度达到0.90，水稻和玉米F1分数分别为0.81和0.90；在不同地形地貌、复杂种植结构的农业景观下均表现出良好的分类效果；与SVM结果的空间一致性为0.90；当弱样本最大面积误差比例小于0.36时，弱样本仍适用于DCNN作物分类，结果的总体精度保持在0.86以上。综上，该策略一定程度上消除了深度学习模型对大量人工标记样本高度依赖的局限性，为实现大尺度农作物遥感分类提供了一种新途径。

Abstract

Driven by big data

deep learning has been widely and successfully applied in many fields

such as computer vision and speech recognition. With the increase in network depth

deep learning models can determine the rules and layers of images and obtain high classification accuracy

so they have become a research hotspot in remote sensing image interpretation. As data-driven algorithms

deep learning models need a large number of labeled samples for training to ensure that the trained model can learn accurate and comprehensive sample features

and they exhibit good classification performance. However

although the development and maturity of remote sensing technology provide abundant remote sensing image sources for deep learning models

the application of deep learning technology in remote sensing is limited by expensive manually labeled samples

. In practical crop classification applications

the quantity and quality of existing ground truth samples are often insufficient to train a classification model with high performance.

This study proposes a crop classification strategy for the deep learning model based on weak samples to verify the applicability of the deep learning model with weak samples.

GF-1 was used as the data source

and the SVM classifier was used to classify three types of rice

corn

and other ground objects in Liaoning Province at the county level. The results were used as the training label samples of the deep learning model. This process included sub-county SVM classification

manual post-classification processing

cropland masking

and other operations. This human-computer interaction was chosen to ensure the accuracy of the results. In this study

samples with non-100% accuracy were labeled as weak samples. Then

a Deep Convolutional Neural Network (DCNN) model was used to train the weak samples and obtain the spatial distribution of rice and corn in Liaoning Province.

Results showed that OA reached 0.90

and the F1 scores of rice and corn were 0.81 and 0.90

respectively. The spatial consistency with the SVM results was 0.90. The model showed good robustness under the different topography and landform types of the agricultural landscape with a median OA that was greater than 0.93. It overcame the influence of topography in the study area to a certain extent through subregion analysis. In the agricultural landscape with a complex planting structure

the proposed method still maintained a certain accuracy in crop classification. Subsequently

noise experiments were designed to analyze the influence of SVM label noise on model classification. The corn distribution in the original SVM training label was expanded from 1 to 40 times to obtain new labels

which were then used to train the DCNN model and predict the testing data. When the model was within five times the sample noise

that is

the sample maximum error area ratio was not more than 0.36

the model was robust to a certain extent

and the results could be maintained within a reliable accuracy range (OA remained to be greater than 0.86).

In conclusion

this study verified that crop classification results obtained with the deep learning model whose training labels are based on traditional classification methods can achieve high recognition accuracy good robustness under different topography and landform types of agricultural landscapes and the feasibility of using traditional classification results as weak samples. The experiment on increasing noise in the weak samples showed that weak samples can be used to train DCNN as long as their identification accuracy is guaranteed

that is

the maximum error area ratio of samples is not more than 0.36. This approach further reduces the threshold of obtaining labeled samples via deep learning models. It makes up for the limitation of the deep learning model

which is highly dependent on a large number of manually labeled samples

and provides a new approach for large-area remote sensing crop classification.

关键词

Keywords

references

Audebert N , Le Saux B and Lefèvre S . 2018 . Beyond RGB: very high resolution urban remote sensing with multimodal deep networks . ISPRS Journal of Photogrammetry and Remote Sensing , 140 : 30 - 32 [ DOI: 10.1016/j.isprsjprs.2017.11.011 http://dx.doi.org/10.1016/j.isprsjprs.2017.11.011 ]

Ball J E , Anderson D T and Chan C S . 2017 . Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community . Journal of Applied Remote Sensing , 11 ( 4 ): 042609 [ DOI: 10.1117/1.JRS.11.042609 http://dx.doi.org/10.1117/1.JRS.11.042609 ]

Belgiu M and Drăguţ L . 2016 . Random forest in remote sensing: a review of applications and future directions . ISPRS Journal of Photogrammetry and Remote Sensing , 114 : 24 - 31 [ DOI: 10.1016/j.isprsjprs.2016.01.011 http://dx.doi.org/10.1016/j.isprsjprs.2016.01.011 ]

Cai Y P , Guan K Y , Peng J , Wang S W , Seifert C , Wardlow B and Li Z . 2018 . A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach . Remote Sensing of Environment , 210 : 35 - 47 [ DOI: 10.1016/j.rse.2018.02.045 http://dx.doi.org/10.1016/j.rse.2018.02.045 ]

Duan Y M . 2020 . Research on Cloud/Shadow Detection Method of Medium Resolution Remote Sensing Image Based on Deep Learning and Data Enhancement . Beijing : Beijing Normal University

段雅鸣 . 2020 . 基于深度学习和数据增强的中分辨率遥感影像云/阴影检测方法研究 . 北京 : 北京师范大学

Hao P Y , Di L P , Zhang C and Guo L Y . 2020 . Transfer learning for crop classification with cropland data layer data (CDL) as training samples . Science of the Total Environment , 733 : 138869 [ DOI: 10.1016/j.scitotenv.2020.138869 http://dx.doi.org/10.1016/j.scitotenv.2020.138869 ]

He K M , Zhang X Y , Ren S Q and Sun J . 2016 . Deep residual learning for image recognition // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE: 770 - 778 [ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]

Hinton G E , Osindero S and Teh Y W . 2006 . A fast learning algorithm for deep belief nets . Neural Computation , 18 ( 7 ): 1527 - 1554 [ DOI: 10.1162/neco.2006.18.7.1527 http://dx.doi.org/10.1162/neco.2006.18.7.1527 ]

Holschneider M , Kronland-Martinet R , Morlet J and Tchamitchian P . 1990 . A real-time algorithm for signal analysis with the help of the wavelet transform // Proceedings of Time-Frequency Methods and Phase Space Proceedings of the International Conference . Marseille : Springer: 286 - 297 [ DOI: 10.1007/978-3-642-75988-8_28 http://dx.doi.org/10.1007/978-3-642-75988-8_28 ]

Hu F , Xia G S , Hu J W and Zhang L P . 2015 . Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery . Remote Sensing , 7 ( 11 ): 14680 - 14707 [ DOI: 10.3390/rs71114680 http://dx.doi.org/10.3390/rs71114680 ]

Jia K and Li Q Z . 2013 . Review of features selection in crop classification using remote sensing data . Resources Science , 35 ( 12 ): 2507 - 2516

贾坤 , 李强子 . 2013 . 农作物遥感分类特征变量选择研究现状与展望 . 资源科学 , 35 ( 12 ): 2507 - 2516

Jia K , Wu B F and Li Q Z . 2013 . Crop classification using HJ satellite multispectral data in the North China plain . Journal of Applied Remote Sensing , 7 ( 1 ): 073576 [ DOI: 10.1117/1.JRS.7.073576 http://dx.doi.org/10.1117/1.JRS.7.073576 ]

Kaiser P , Wegner J D , Lucchi A , Jaggi M , Hofmann T and Schindler K . 2017 . Learning aerial image segmentation from online maps . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 11 ): 6054 - 6068 [ DOI: 10.1109/TGRS.2017.2719738 http://dx.doi.org/10.1109/TGRS.2017.2719738 ]

Krizhevsky A , Sutskever I and Hinton G E . 2017 . ImageNet classification with deep convolutional neural networks . Communications of the ACM , 60 ( 6 ): 84 - 90 [ DOI: 10.1145/3065386 http://dx.doi.org/10.1145/3065386 ]

Kussul N , Lavreniuk M , Skakun S and Shelestov A . 2017 . Deep learning classification of land cover and crop types using remote sensing data . IEEE Geoscience and Remote Sensing Letters , 14 ( 5 ): 778 - 782 [ DOI: 10.1109/LGRS.2017.2681128 http://dx.doi.org/10.1109/LGRS.2017.2681128 ]

Li Y , Zhang H K and Shen Q . 2017 . Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network . Remote Sensing , 9 ( 1 ): 67 [ DOI: 10.3390/rs9010067 http://dx.doi.org/10.3390/rs9010067 ]

Liu D W , Han L and Han X Y . 2016 . High spatial resolution remote sensing image classification based on deep learning . Acta Optica Sinica , 36 ( 4 ): 0428001

刘大伟 , 韩玲 , 韩晓勇 . 2016 . 基于深度学习的高分辨率遥感影像分类研究 . 光学学报 , 36 ( 4 ): 0428001 [ DOI: 10.3788/AOS201636.0428001 http://dx.doi.org/10.3788/AOS201636.0428001 ]

Lu D and Weng Q . 2007 . A survey of image classification methods and techniques for improving classification performance . International Journal of Remote Sensing , 28 ( 5 ): 823 - 870 [ DOI: 10.1080/01431160600746456 http://dx.doi.org/10.1080/01431160600746456 ]

Murthy C S , Raju P V and Badrinath K V S . 2003 . Classification of wheat crop with multi-temporal images: performance of maximum likelihood and artificial neural networks . International Journal of Remote Sensing , 24 ( 23 ): 4871 - 4890 [ DOI: 10.1080/0143116031000070490 http://dx.doi.org/10.1080/0143116031000070490 ]

Qu J Y , Sun X and Gao X . 2016 . Remote sensing image target recognition based on CNN . Foreign Electronic Measurement Technology , 35 ( 8 ): 45 - 50

曲景影 , 孙显 , 高鑫 . 2016 . 基于CNN模型的高分辨率遥感图像目标识别 . 国外电子测量技术 , 35 ( 8 ): 45 - 50 [ DOI: 10.3969/j.issn.1002-8978.2016.08.011 http://dx.doi.org/10.3969/j.issn.1002-8978.2016.08.011 ]

Rodriguez-Galiano V F , Ghimire B , Rogan J , Chica-Olmo M and Rigol-Sanchez J P . 2012 . An assessment of the effectiveness of a random forest classifier for land-cover classification . ISPRS Journal of Photogrammetry and Remote Sensing , 67 : 93 - 104 [ DOI: 10.1016/j.isprsjprs.2011.11.002 http://dx.doi.org/10.1016/j.isprsjprs.2011.11.002 ]

Roy D P , Wulder M A , Loveland T R , Woodcock C E , Allen R G , Anderson M C , Helder D , Irons J R , Johnson D M , Kennedy R , Scambos T A , Schaaf C B , Schott J R , Sheng Y , Vermote E F , Belward A S , Bindschadler R , Cohen W B , Gao F , Hipple J D , Hostert P , Huntington J , Justice C O , Kilic A , Kovalskyy V , Lee Z P , Lymburner L , Masek J G , McCorkel J , Shuai Y , Trezza R , Vogelmann J , Wynne R H and Zhu Z . 2014 . Landsat-8: science and product vision for terrestrial global change research . Remote Sensing of Environment , 145 : 154 - 172 [ DOI: 10.1016/j.rse.2014.02.001 http://dx.doi.org/10.1016/j.rse.2014.02.001 ]

Rudiyanto , Minasny B , Shah R M , Soh N C , Arif C and Setiawan B I . 2019 . Automated near-real-time mapping and monitoring of rice extent, cropping patterns, and growth stages in Southeast Asia using sentinel-1 time series on a Google Earth Engine platform . Remote Sensing , 11 ( 14 ): 1666 [ DOI: 10.3390/rs11141666 http://dx.doi.org/10.3390/rs11141666 ]

Shao Y , Fan X T , Liu H , Xiao J H , Ross S , Brisco B , Brown R and Staples G . 2001 . Rice monitoring and production estimation using multitemporal RADARSAT . Remote Sensing of Environment , 76 ( 3 ): 310 - 325 [ DOI: s10.1016/S0034-4257(00)00212-1 http://dx.doi.org/s10.1016/S0034-4257(00)00212-1 ]

Song D J , Zhang C M , Yang X X , Li F , Han Y J , Gao S and Dong H Y . 2020 . Extracting winter wheat spatial distribution information from GF-2 image . Journal of Remote Sensing , 24 ( 5 ): 596 - 608

宋德娟 , 张承明 , 杨晓霞 , 李峰 , 韩颖娟 , 高帅 , 董海燕 . 2020 . 高分二号遥感影像提取冬小麦空间分布 . 遥感学报 , 24 ( 5 ): 596 - 608 [ DOI: 10.11834/jrs.20208285 http://dx.doi.org/10.11834/jrs.20208285 ]

Wagner S . 2014 . Combination of convolutional feature extraction and support vector machines for radar ATR // Proceedings of the 17 th International Conference on Information Fusion . Salamanca : IEEE: 1 - 6 .

Wang N , Li Q Z , Du X , Zhang Y , Zhao L C and Wang H Y . 2017 . Identification of main crops based on the univariate feature selection in Subei . Journal of Remote Sensing , 21 ( 4 ): 519 - 530

王娜 , 李强子 , 杜鑫 , 张源 , 赵龙才 , 王红岩 . 2017 . 单变量特征选择的苏北地区主要农作物遥感识别 . 遥感学报 , 21 ( 4 ): 519 - 30 [ DOI: 10.11834/jrs.20176373 http://dx.doi.org/10.11834/jrs.20176373 ]

Waske B and Benediktsson J A . 2007 . Fusion of support vector machines for classification of multisensor data . IEEE Transactions on Geoscience and Remote Sensing , 45 ( 12 ): 3858 - 3866 [ DOI: 10.1109/TGRS.2007.898446 http://dx.doi.org/10.1109/TGRS.2007.898446 ]

Wu J G . 2000 . Landscape Ecology: Pattern, Process, Scale and Hierarchy . Beijing : Higher Education Press

邬建国 . 2000 . 景观生态学-格局、过程、尺度与等级 . 北京 : 高等教育出版社

Yang C H , Everitt J H and Murden D . 2011 . Evaluating high resolution SPOT 5 satellite imagery for crop identification . Computers and Electronics in Agriculture , 75 ( 2 ): 347 - 354 [ DOI: 10.1016/j.compag.2010.12.012 http://dx.doi.org/10.1016/j.compag.2010.12.012 ]

Yang H W and Tong X H . 2012 . Application status of middle and high resolution remote sensing images in agriculture . Transactions of the Chinese Society of Agricultural Engineering , 28 ( 24 ): 138 - 149

杨红卫 , 童小华 . 2012 . 中高分辨率遥感影像在农业中的应用现状 . 农业工程学报 , 28 ( 24 ): 138 - 149

Zhang D J , Pan Y Z , Zhang J S , Hu T G , Zhao J H , Li N and Chen Q . 2020 . A generalized approach based on convolutional neural networks for large area cropland mapping at very high resolution . Remote Sensing of Environment , 247 : 111912 [ DOI: 10.1016/j.rse.2020.111912 http://dx.doi.org/10.1016/j.rse.2020.111912 ]

Zhang J K , Cheng Y P , Zhang F W , Yue D P , Guo X X , Dong H , Wang J P and Tang H C . 2012 . Crops planting information extraction based on multi-temporal remote sensing images . Transactions of the Chinese Society of Agricultural Engineering , 28 ( 2 ): 134 - 141

张健康 , 程彦培 , 张发旺 , 岳德鹏 , 郭晓晓 , 董华 , 王计平 , 唐宏才 . 2012 . 基于多时相遥感影像的作物种植信息提取 . 农业工程学报 , 28 ( 2 ): 134 - 141 [ DOI: 10.3969/j.issn.1002-6819.2012.02.024 http://dx.doi.org/10.3969/j.issn.1002-6819.2012.02.024 ]

Zhao H S , Shi J P , Qi X J , Wang X G and Jia J Y . 2017 . Pyramid scene parsing network // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Honolulu : IEEE: 6230 - 6239 [ DOI: 10.1109/CVPR.2017.660 http://dx.doi.org/10.1109/CVPR.2017.660 ]

Zhong L H , Hu L N and Zhou H . 2019 . Deep learning based multi-temporal crop classification . Remote Sensing of Environment , 221 : 430 - 443 [ DOI: 10.1016/j.rse.2018.11.032 http://dx.doi.org/10.1016/j.rse.2018.11.032 ]

Zhuang F Z , Luo P , He Q and Shi Z Z . 2015 . Survey on transfer learning research . Journal of Software , 26 ( 1 ): 26 - 39

庄福振 , 罗平 , 何清 , 史忠植 . 2015 . 迁移学习研究进展 . 软件学报 , 26 ( 1 ): 26 - 39 [ DOI: 10.13328/j.cnki.jos.004631 http://dx.doi.org/10.13328/j.cnki.jos.004631 ]

Alert me when the article has been cited

提交

Fine 3D mapping and vegetation spatial differentiation study based on UAV LiDAR point cloud in karst wetland

LAE-Transformer： An airborne LiDAR point cloud segmentation network incorporating local-aware enhancement

A multi-scale InSAR phase filtering method incorporating adaptive enhancement and dynamic screening mechanisms

Multimodal object detection method using adaptive dusion of infrared and visible features