Spatial generalization ability analysis of deep learning crop classification models

Shuang GE; JinShui ZHANG; Shuang ZHU

doi:10.11834/jrs.20221408

Views : 0 下载量: 111 CSCD: 0

R-PDF
PDF
Export
Share
Collection
Album

Spatial generalization ability analysis of deep learning crop classification models
Pages: 1-25(2022)
DOI： 10.11834/jrs.20221408

扫描看全文

Quote
盖爽，张锦水，朱爽.XXXX.深度学习作物分类模型空间泛化能力分析.遥感学报，XX（XX）： 1-25

GE Shuang，ZHANG JinShui，ZHU Shuang. XXXX. Spatial generalization ability analysis of deep learning crop classification models. National Remote Sensing Bulletin， XX（XX）：1-25
盖爽，张锦水，朱爽.XXXX.深度学习作物分类模型空间泛化能力分析.遥感学报，XX（XX）： 1-25 DOI： 10.11834/jrs.20221408.

GE Shuang，ZHANG JinShui，ZHU Shuang. XXXX. Spatial generalization ability analysis of deep learning crop classification models. National Remote Sensing Bulletin， XX（XX）：1-25 DOI： 10.11834/jrs.20221408.

摘要

大数据驱动训练的深度学习模型是当今农作物分类的最新方法，有越来越多的新的分类模型涌现。然而，当前研究仍然主要关注模型方法的创新性，其应用方式往往针对特定时间、特定地区，而作物分类模型的泛化能力分析经常被忽略。提高遥感分类模型在大尺度空间范围的有效迁移能力是遥感技术支撑地球系统科学研究和社会应用的关键。本研究通过设计实验分析了模型架构、作物物候特征、农区地块尺度、数据类型等因素对作物分类模型泛化能力的影响。结果表明当训练区和测试区地块大小发生明显变化时，MultiResUNet相对于SegNet，　DeepLab V3+和U-Net具有更好的泛化性能。然而，单纯依靠MultiResUNet的泛化能力依然无法完全克服地块空间形态的变化对模型迁移的不利影响，为获得更高精度的华北玉米分布信息，需要优先使用与华北地区农业景观更相似的东北作物分布数据产品进行深度学习模型训练。在另一方面，我们发现相对于TOA（top of atmosphere）数据，采用SR（surface reflectance）数据更有利于模型在跨洲际尺度进行空间迁移。因此，在大尺度作物制图研究中，应优先考虑使用SR数据。本研究从一定程度上验证了影响农作物分类模型迁移性能的内在因素，为大尺度作物制图提供了一定的实验基础。

Abstract

Timely and accurate global crop mapping is of great significance for global food security assessment. However, existing crop classification models are often targeted at specific regions, and their performance in other regions has not been fully evaluated. In order to realize the effective transfer of the model in large-scale regions, this study determined the critical period of crop growth in different regions, and filtered the remote sensing data during these critical periods so that the same crop in different regions showed similar characteristics on these remote sensing images. This helped the model achieve a better transfer effect. In this study, the MultiResUNet,　SegNet,　DeepLab V3+ and U-Net　models were trained using data from Northeast China, and the optimal F1 value for summer corn recognition in the study areas in North China can reach more than 0.97. In addition, this research analyzed the factors that affected the generalization ability of the model. The issues that this article focused on include: (1) Using existing crop distribution data products as the ground truth samples for model training to solve the problem of lack of training samples for the deep learning model. We compared and analyzed the applicability of the models trained using the US Cropland Data Layer (CDL) and Northeast crop distribution data products in North China. (2) We compared the generalization performance of depth models with different architectures. (3) We compared and analyzed the influence of different data type on the generalization ability of the model. (4) We have comparatively analyzed the impact of crop phenology changes on the generalization ability of the model. The results show that MultiResUNet has better generalization performance than other networks when the size of plot in training area and test area varies significantly However, the generalization ability of MultiResUNet alone still cannot completely overcome the adverse effect of the change of plot spatial morphology on model migration. In order to obtain more accurate information of maize distribution in North China, it is necessary to use the crop distribution data products in Northeast China that are more similar to the agricultural landscape in North China for deep learning model training. On the other hand, compared with TOA data, we found that SR data is more conducive to the spatial migration of the model at the trans-continental scale. Therefore, SR data should be given priority in large-scale crop mapping. This research provides a useful discussion for large-scale crop mapping research using only local samples.

关键词

模型泛化深度学习SegNetDeepLab V3+U-NetMultiResUNet作物制图

Keywords

Model generalizationdeep learningSegNetDeepLab V3+U-NetMultiResUNetcrop mapping

references

Azzari G. and Lobell D. B. 2017. Landsat-based classification in the cloud: An opportunity for a paradigm shift in land cover monitoring. Remote Sensing of Environment, 202: 64-74

Boryan C., Yang Z., Mueller R. and Craig M. 2011. Monitoring US agriculture: the US Department of Agriculture, National Agricultural Statistics Service, Cropland Data Layer Program. Geocarto International, 26(5): 341-358

Cai Y., Guan K., Peng J., Wang S., Seifert C., Wardlow B. and Li, Z. 2018. A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach. Remote Sensing of Environment, 210: 35-47

Chen L.-C., Papandreou G., Schroff F. and Adam, H. 2017. Rethinking Atrous Convolution for Semantic Image Segmentation.

Chen L.-C., Zhu Y., Papandreou G., Schroff F. and Adam, H. 2018. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation.

Chen L. C., Papandreou G., Kokkinos I., Murphy K. and Yuille A. L. 2018. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4): 834-848

De Wit, A. J. W. Clevers, J. G. P. W. 2004. Efficiency and accuracy of per-field classification for operational crop mapping. International Journal of Remote Sensing, 25(20): 4091-4112

Diao C. 2020. Remote sensing phenological monitoring framework to characterize corn and soybean physiological growing stages. Remote Sensing of Environment, 248: 111960

Foody G. M. 2020. Explaining the unsuitability of the kappa coefficient in the assessment and comparison of the accuracy of thematic maps obtained by image classification. Remote Sensing of Environment, 239: 111630

Ge S., Zhang J., Pan Y., Yang Z. and Zhu, S. 2021. Transferable deep learning model based on the phenological matching principle for mapping crop extent. International Journal of Applied Earth Observation and Geoinformation, 102: 102451

Hamdi Z. M., Brandmeier M. and Straub C. 2019. Forest Damage Assessment Using Deep Learning on High Resolution Remote Sensing Data. Remote Sensing, 11(17)

Hamrouni Y., Paillassa E., Chéret V., Monteil C. and Sheeren, D. 2021. From local to global: A transfer learning-based approach for mapping poplar plantations at national scale using Sentinel-2. ISPRS Journal of Photogrammetry and Remote Sensing, 171: 76-100

Hao P., Di L., Zhang C. and Guo, L. 2020. Transfer Learning for Crop classification with Cropland Data Layer data (CDL) as training samples. Science of The Total Environment, 733: 138869

He K., Zhang X., Ren S. and Sun, J. (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

Ibtehaz N. and Rahman M. S. 2020. MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Networks, 121: 74-87

Kattenborn T., Eichel J., Wiser S., Burrows L., Fassnacht F. E. and Schmidtlein S. 2020. Convolutional Neural Networks accurately predict cover fractions of plant species and communities in Unmanned Aerial Vehicle imagery. Remote Sensing in Ecology and Conservation, 6(4): 472-486

Kattenborn T., Leitloff J., Schiefer F. and Hinz S. 2021. Review on Convolutional Neural Networks (CNN) in vegetation remote sensing. ISPRS Journal of Photogrammetry and Remote Sensing, 173: 24-49

Kingma D. P. and Ba J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

Kluger D. M., Wang S. and Lobell D. B. 2021. Two shifts for crop mapping: Leveraging aggregate crop statistics to improve satellite-based maps in new regions. Remote Sensing of Environment, 262: 112488

Konduri V. S., Kumar J., Hargrove W. W., Hoffman F. M. and Ganguly A. R. 2020. Mapping crops within the growing season across the United States. Remote Sensing of Environment, 251: 112048

LeCun Y., Bengio Y. and Hinton G. 2015. Deep learning. Nature, 521(7553): 436-444

Li Z., Shen H., Cheng Q., Liu Y., You S. and He, Z. 2019. Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors. ISPRS Journal of Photogrammetry and Remote Sensing, 150: 197-212

Liu T., Abd-Elrahman A., Jon M. and Wilhelm, V. 2018. Comparing Fully Convolutional Networks, ForestRandom, Support Vector Machine, and Patch-based Deep Convolutional Neural Networks for Object-based Wetland Mapping using Images from small Unmanned Aircraft System. GIScience & Remote Sensing, 55

Neupane B., Horanont T. and Hung N. D. 2019. Deep learning based banana plant detection and counting using high-resolution red-green-blue (RGB) images collected from unmanned aerial vehicle (UAV). PLOS ONE, 14(10): e0223906

Nezami S., Khoramshahi E., Nevalainen O., Pölönen I. and Honkavaara, E. 2020. Tree Species Classification of Drone Hyperspectral and RGB Imagery with Deep Learning Convolutional Neural Networks. Remote Sensing, 12(7)

Panboonyuen T., Jitkajornwanich K., Lawawirojwong S., Srestasathiern P. and Vateekul, P. 2017. Road Segmentation of Remotely-Sensed Images Using Deep Convolutional Neural Networks with Landscape Metrics and Conditional Random Fields. Remote Sensing, 9(7)

Qi X., Zhu P., Wang Y., Zhang L., Peng J., Wu M., Chen J., Zhao X., Zang N. and Mathiopoulos, P. T. 2020. MLRSNet: A multi-label high spatial resolution remote sensing dataset for semantic scene understanding. ISPRS Journal of Photogrammetry and Remote Sensing, 169: 337-350

Ronneberger O., Fischer, P. Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Cham, Springer International Publishing.

Rußwurm, M. Korner, M. (2017). Temporal vegetation modelling using long short-term memory networks for crop identification from medium-resolution multi-spectral satellite images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.

Schiefer F., Kattenborn T., Frick A., Frey J., Schall P., Koch B. and Schmidtlein S. 2020. Mapping forest tree species in high resolution UAV-based RGB-imagery by means of convolutional neural networks. ISPRS Journal of Photogrammetry and Remote Sensing, 170: 205-215

Song C., Woodcock C. E., Seto K. C., Lenney M. P. and Macomber S. A. 2001. Classification and Change Detection Using Landsat TM Data: When and How to Correct Atmospheric Effects? Remote Sensing of Environment, 75(2): 230-244

Song X.-P., Potapov P. V., Krylov A., King L., Di Bella C. M., Hudson A., Khan A., Adusei B., Stehman S. V. and Hansen, M. C. 2017. National-scale soybean mapping and area estimation in the United States using medium resolution satellite imagery and field survey. Remote Sensing of Environment, 190: 383-395

Szegedy C., Wei L., Yangqing J., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V. and Rabinovich, A. (2015). Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

Wang S., Azzari G. and Lobell D. B. 2019. Crop type mapping without field-level labels: Random forest transfer and unsupervised clustering techniques. Remote Sensing of Environment, 222: 303-317

Weinstein B. G., Marconi S., Bohlman S. A., Zare A. and White E. P. 2020. Cross-site learning in deep learning RGB tree crown detection. Ecological Informatics, 56: 101061

Wieland M., Li, Y. Martinis, S. 2019. Multi-sensor cloud and cloud shadow segmentation with a convolutional neural network. Remote Sensing of Environment, 230: 111-203

Xu J., Zhu Y., Zhong R., Lin Z., Xu J., Jiang H., Huang J., Li H. and Lin, T. 2020. DeepCropMapping: A multi-temporal deep learning approach with improved spatial generalizability for dynamic corn and soybean mapping. Remote Sensing of Environment, 247: 111946

Yang R., Ahmed Z. U., Schulthess U. C., Kamal M. and Rai R. 2020. Detecting functional field units from satellite images in smallholder farming systems using a deep learning based computer vision approach: A case study from Bangladesh. Remote Sensing Applications: Society and Environment, 20: 100413

You, N. Dong, J. 2020. Examining earliest identifiable timing of crops using all available Sentinel 1/2 imagery and Google Earth Engine. ISPRS Journal of Photogrammetry and Remote Sensing, 161: 109-123

You N., Dong J., Huang J., Du G., Zhang G., He Y., Yang T., Di Y. and Xiao, X. 2021. The 10-m crop type maps in Northeast China during 2017–2019. Scientific Data, 8(1): 41

Zhang D., Pan Y., Zhang J., Hu T., Zhao J., Li N. and Chen, Q. 2020. A generalized approach based on convolutional neural networks for large area cropland mapping at very high resolution. Remote Sensing of Environment, 247: 111912

Zhang L., Liu Z., Liu D., Xiong Q., Yang N., Ren T., Zhang C., Zhang X. and Li, S. 2019. Crop Mapping Based on Historical Samples and New Training Samples Generation in Heilongjiang Province, China. Sustainability, 11(18)

Zhang M., Lin H., Wang G., Sun H. and Fu, J. 2018. Mapping Paddy Rice Using a Convolutional Neural Network (CNN) with Landsat 8 Datasets in the Dongting Lake Area, China. Remote Sensing, 10(11)

Zhong L., Gong P. and Biging G. S. 2014. Efficient corn and soybean mapping with temporal extendability: A multi-year experiment using Landsat imagery. Remote Sensing of Environment, 140: 1-13

Zhong L., Hu, L. Zhou, H. 2019. Deep learning based multi-temporal crop classification. Remote Sensing of Environment, 221: 430-443

Zhong Y., Hu X., Luo C., Wang X., Zhao J. and Zhang, L. 2020. WHU-Hi: UAV-borne hyperspectral with high spatial resolution (H2) benchmark datasets and classifier for precise crop identification based on deep convolutional neural network with CRF. Remote Sensing of Environment, 250: 112012

Zhu X. X., Tuia D., Mou L., Xia G. S., Zhang L., Xu F. and Fraundorfer, F. 2017. Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources. IEEE Geoscience and Remote Sensing Magazine, 5(4): 8-36

傅帅, 陈舒慧, 张小泉 and 徐士琦 2017. 江苏省1960—2014年降水及气温变化特征. 环保科技, 23(02): 26-33

姜伊兰, 陈保旺, 黄玉芳, 崔佳琪 and 郭宇龙 2021. 基于Google Earth Engine和NDVI时序差异指数的作物种植区提取. 地球信息科学学报, 23(05): 938-947

李前景, 刘珺, 米晓飞, 杨健 and 余涛 2021. 面向对象与卷积神经网络模型的GF-6 WFV影像作物分类遥感学报, 25(02): 549-558

李元华 and 车少静 2005. 河北省温度和降水变化对农业的影响. 中国农业气象,(04): 22-26

齐晓宁, 王洋, 王其存, 刘兆永 and 鲍强 2002. 吉林玉米带的地位与发展前景. 地理科学,(03): 379-384

王协, 章孝灿 and 苏程 2020. 基于多尺度学习与深度卷积神经网络的遥感图像土地利用分类. 浙江大学学报(理学版), 47(06): 715-723

徐春燕 and 冯学智 2007. TM图像大气校正及其对地物光谱响应特征的影响分析. 南京大学学报(自然科学版),(03): 309-317

杨蜀秦, 宋志双, 尹瀚平, 张智韬 and 宁纪锋 2021. 基于深度语义分割的无人机多光谱遥感作物分类方法. 农业机械学报, 52(03): 185-192

赵红伟, 陈仲新, 姜浩 and 刘佳 2020. 基于Sentinel-1A影像和一维CNN的中国南方生长季早期作物种类识别. 农业工程学报, 36(03): 169-177

Alert me when the article has been cited

提交

Hyperspectral unmixing based on adversarial autoencoder network