سابِق|تال

Spatial generalization ability analysis of deep learning crop classification models

  • role： First author第一作者
  • Affiliation：

    State Key Laboratory of Remote Sensing Science, Beijing Normal University, Beijing 100875, China

    Jiangsu Provincial Key Laboratory of Geographic Information Science and Technology, International Institute for Earth System Science, Nanjing University, Nanjing 210023, China

    Beijing Engineering Research Center for Global Land Remote Sensing Products, Faculty of Geographical Science, Beijing Normal University, Beijing 100875, China

    Institute of Remote Sensing Science and Engineering, Faculty of Geographical Science, Beijing Normal University, Beijing 100875, China

  • Email：201921051175@mail.bnu.edu.cn
  • Introduction：盖爽， 研究方向为农业遥感、作物分类模型泛化。E-mail： 201921051175@mail.bnu.edu.cn
GE Shuang1234,  
  • role： Corresponding author通信作者
  • Affiliation：

    State Key Laboratory of Remote Sensing Science, Beijing Normal University, Beijing 100875, China

    Beijing Engineering Research Center for Global Land Remote Sensing Products, Faculty of Geographical Science, Beijing Normal University, Beijing 100875, China

    Institute of Remote Sensing Science and Engineering, Faculty of Geographical Science, Beijing Normal University, Beijing 100875, China

  • Email：zhangjs@bnu.edu.cn
  • Introduction：张锦水，研究方向为农业遥感。E-mail： zhangjs@bnu.edu.cn
ZHANG JinShui134*,  
  • Affiliation：

    Beijing Polytechnic College, Beijing 100042, China

ZHU Shuang5

ملخص

Timely and accurate global crop mapping is important for global food security assessment. However, existing crop classification models are often targeted at specific regions, and their performance in other regions has not been fully evaluated. This study determined the critical period of crop growth in different regions to realize the effective transfer of the model in large-scale regions, and the remote sensing data during these critical periods were filtered such that the same crop in different regions showed similar characteristics on these remote sensing images. This way helps the model achieve a better transfer effect. In this study, the MultiResUNet, SegNet, DeepLab V3+, and U-Net models were trained using data from Northeast China, and the optimal F1 value for summer corn recognition in the study areas in North China can reach more than 0.97. This research also analyzed the factors that affected the generalization ability of the model. The issues addressed in this article include (1) using existing crop distribution data products as the ground truth samples for model training to solve the problem of lack of training samples for the deep learning model. We compared and analyzed the applicability of the models trained using the US Cropland Data Layer and Northeast crop distribution data products in North China. (2) We compared the generalization performance of depth models with different architectures. (3) We compared and analyzed the influence of different data types on the generalization ability of the model. (4) We comparatively analyzed the impact of crop phenology changes on the generalization ability of the model. Results show that MultiResUNet has better generalization performance than other networks when the plot size in the training and test areas varies significantly However, the generalization ability of MultiResUNet alone still cannot completely overcome the adverse effect of the change in plot spatial morphology on model migration. The crop distribution data products in Northeast China, which are more similar to the agricultural landscape in North China, need to be used for deep learning model training to obtain more accurate information of maize distribution in North China. Compared with TOA data, we found that SR data are more conducive to the spatial migration of the model at the transcontinental scale. Therefore, SR data should be given priority in large-scale crop mapping. This research provides a useful discussion for large-scale crop mapping research using only local samples.

مفهوم

Model generalization;deep learning;SegNet;DeepLab V3+;U-Net;MultiResUNet;crop mapping

References

  1. 1.
    Azzari G, Lobell D B. 2017. Landsat-based classification in the cloud: an opportunity for a paradigm shift in land cover monitoring. Remote Sensing of Environment, 202: 64-74
  2. 2.
    Boryan C, Yang Z W, Mueller R and Craig M. 2011. Monitoring US agriculture: the US department of agriculture, national agricultural statistics service, cropland data layer program. Geocarto International, 26(5): 341-358
  3. 3.
    Cai Y P, Guan K Y, Peng J, Wang S W, Seifert C, Wardlow B and Li Z. 2018. A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach. Remote Sensing of Environment, 210: 35-47
  4. 4.
    Chen L C, Papandreou G, Kokkinos I, Murphy K and Yuille A L. 2018a. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4): 834-848
  5. 5.
    Chen L C, Papandreou G, Schroff F and Adam H. 2017. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587
  6. 6.
    Chen L C, Zhu Y K, Papandreou G, Schroff F and Adam H. 2018b. Encoder-decoder with atrous separable convolution for semantic image segmentation//Proceedings of the 15th European Conference on Computer Vision. Munich: Springer: 833-851
  7. 7.
    De Wit A J W and Clevers J G P W. 2004. Efficiency and accuracy of per-field classification for operational crop mapping. International Journal of Remote Sensing, 25(20): 4091-4112
  8. 8.
    Diao C Y. 2020. Remote sensing phenological monitoring framework to characterize corn and soybean physiological growing stages. Remote Sensing of Environment, 248: 111960
  9. 9.
    Foody G M. 2020. Explaining the unsuitability of the kappa coefficient in the assessment and comparison of the accuracy of thematic maps obtained by image classification. Remote Sensing of Environment, 239: 111630
  10. 10.
    Fu S, Chen S H, Zhang X Q and Xu S Q. 2017. Variations of precipitation and air temperature during 1960-2014 in Jiangsu province. Environmental Protection and Technology, 23(2): 26-33
  11. 11.
    Ge S, Zhang J S, Pan Y Z, Yang Z and Zhu S. 2021. Transferable deep learning model based on the phenological matching principle for mapping crop extent. International Journal of Applied Earth Observation and Geoinformation, 102: 102451
  12. 12.
    Hamdi Z M, Brandmeier M and Straub C. 2019. Forest damage assessment using deep learning on high resolution remote sensing data. Remote Sensing, 11(17): 1976
  13. 13.
    Hamrouni Y, Paillassa E, Chéret V, Monteil C and Sheeren D. 2021. From local to global: a transfer learning-based approach for mapping poplar plantations at national scale using Sentinel-2. ISPRS Journal of Photogrammetry and Remote Sensing, 171: 76-100
  14. 14.
    Hao P Y, Di L P, Zhang C and Guo L Y. 2020. Transfer Learning for Crop classification with Cropland Data Layer data (CDL) as training samples. Science of the Total Environment, 733: 138869
  15. 15.
    He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE: 770-778
  16. 16.
    Ibtehaz N and Rahman M S. 2020. MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Networks, 121: 74-87
  17. 17.
    Jiang Y L, Chen B W, Huang Y F, Cui J Q and Guo Y L. 2021. Crop planting area extraction based on google earth engine and NDVI time series difference index. Journal of Geo-Information Science, 23(5): 938-947
  18. 18.
    Kattenborn T, Eichel J, Wiser S, Burrows L, Fassnacht F E and Schmidtlein S. 2020. Convolutional Neural Networks accurately predict cover fractions of plant species and communities in Unmanned Aerial Vehicle imagery. Remote Sensing in Ecology and Conservation, 6(4): 472-486
  19. 19.
    Kattenborn T, Leitloff J, Schiefer F and Hinz S. 2021. Review on Convolutional Neural Networks (CNN) in vegetation remote sensing. ISPRS Journal of Photogrammetry and Remote Sensing, 173: 24-49
  20. 20.
    Kingma D P and Ba J. 2014. Adam: a method for stochastic optimization//3rd International Conference on Learning Representations. San Diego: ICLR
  21. 21.
    Kluger D M, Wang S and Lobell D B. 2021. Two shifts for crop mapping: leveraging aggregate crop statistics to improve satellite-based maps in new regions. Remote Sensing of Environment, 262: 112488
  22. 22.
    Konduri V S, Kumar J, Hargrove W W, Hoffman F M and Ganguly A R. 2020. Mapping crops within the growing season across the United States. Remote Sensing of Environment, 251: 112048
  23. 23.
    LeCun Y, Bengio Y and Hinton G. 2015. Deep learning. Nature, 521(7553): 436-444
  24. 24.
    Li Q J, Liu J, Mi X F, Yang J and Yu T. 2021. Object-oriented crop classification for GF-6 WFV remote sensing images based on Convolutional Neural Network. Journal of Remote Sensing, 25(2): 549-558
  25. 25.
    Li Y H and Che S J. 2005. Impacts of changes of temperature and precipitation on agriculture in Hebei regions. Chinese Journal of Agrometeorology, 26(4): 224-228
  26. 26.
    Li Z W, Shen H F, Cheng Q, Liu Y H, You S C and He Z Y. 2019. Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors. ISPRS Journal of Photogrammetry and Remote Sensing, 150: 197-212
  27. 27.
    Liu T, Abd-Elrahman A, Morton J and Wilhelm V L. 2018. Comparing fully convolutional networks, random forest, support vector machine, and patch-based deep convolutional neural networks for object-based wetland mapping using images from small unmanned aircraft system. GIScience and Remote Sensing, 55(2): 243-264
  28. 28.
    Neupane B, Horanont T and Hung N D. 2019. Deep learning based banana plant detection and counting using high-resolution red-green-blue (RGB) images collected from unmanned aerial vehicle (UAV). PLoS One, 14(10): e0223906
  29. 29.
    Nezami S, Khoramshahi E, Nevalainen O, Pölönen I and Honkavaara E. 2020. Tree species classification of drone hyperspectral and RGB imagery with deep learning convolutional neural networks. Remote Sensing, 12(7): 1070
  30. 30.
    Panboonyuen T, Jitkajornwanich K, Lawawirojwong S, Srestasathiern P and Vateekul P. 2017. Road segmentation of remotely-sensed images using deep convolutional neural networks with landscape metrics and conditional random fields. Remote Sensing, 9(7): 680
  31. 31.
    Qi X M, Zhu P P, Wang Y B, Zhang L Q, Peng J H, Wu M F, Chen J L, Zhao X D, Zang N and Mathiopoulos P T. 2020. MLRSNet: a multi-label high spatial resolution remote sensing dataset for semantic scene understanding. ISPRS Journal of Photogrammetry and Remote Sensing, 169: 337-350
  32. 32.
    Qi X N, Wang Y, Wang Q C, Liu Z Y and Bao Q. 2002. The situation and developing prospect of corn zone in Jinlin province. Scientia Geographica Sinica, 22(3): 379-384
  33. 33.
    Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer: 234-241
  34. 34.
    Rußwurm M and Körner M. 2017. Temporal vegetation modelling using long short-term memory networks for crop identification from medium-resolution multi-spectral satellite images. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu: IEEE: 1496-1504
  35. 35.
    Schiefer F, Kattenborn T, Frick A, Frey J, Schall P, Koch B and Schmidtlein S. 2020. Mapping forest tree species in high resolution UAV-based RGB-imagery by means of convolutional neural networks. ISPRS Journal of Photogrammetry and Remote Sensing, 170: 205-215
  36. 36.
    Song C H, Woodcock C E, Seto K C, Lenney M P and Macomber S A. 2001. Classification and change detection using landsat TM data: when and how to correct atmospheric effects?. Remote Sensing of Environment, 75(2): 230-244
  37. 37.
    Song X P, Potapov P V, Krylov A, King L A, Di Bella C M, Hudson A, Khan A, Adusei B, Stehman S V and Hansen M C. 2017. National-scale soybean mapping and area estimation in the United States using medium resolution satellite imagery and field survey. Remote Sensing of Environment, 190: 383-395
  38. 38.
    Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE: 1-9
  39. 39.
    Wang S, Azzari G and Lobell D B. 2019. Crop type mapping without field-level labels: random forest transfer and unsupervised clustering techniques. Remote Sensing of Environment, 222: 303-317
  40. 40.
    Wang X, Zhang X C and Su C. 2020. Land use classification of remote sensing images based on multi-scale learning and deep convolution neural. Journal of Zhejiang University (Science Edition), 47(6): 715-723
  41. 41.
    Weinstein B G, Marconi S, Bohlman S A, Zare A and White E P. 2020. Cross-site learning in deep learning RGB tree crown detection. Ecological Informatics, 56: 101061
  42. 42.
    Wieland M, Li Y and Martinis S. 2019. Multi-sensor cloud and cloud shadow segmentation with a convolutional neural network. Remote Sensing of Environment, 230: 111203
  43. 43.
    Xu C Y and Feng X Z. 2007. Atmospheric correction on TM image and its influence analysis on spectral response characteristics. Journal of Nanjing University (Natural Sciences), 43(3): 309-317
  44. 44.
    Xu J F, Zhu Y, Zhong R H, Lin Z X, Xu J L, Jiang H, Huang J F, Li H F and Lin T. 2020. DeepCropMapping: a multi-temporal deep learning approach with improved spatial generalizability for dynamic corn and soybean mapping. Remote Sensing of Environment, 247: 111946
  45. 45.
    Yang R Y, Ahmed Z U, Schulthess U C, Kamal M and Rai R. 2020. Detecting functional field units from satellite images in smallholder farming systems using a deep learning based computer vision approach: a case study from Bangladesh. Remote Sensing Applications: Society and Environment, 20: 100413
  46. 46.
    Yang S Q, Song Z S, Yin H P, Zhang Z T and Ning J F. 2021. Crop classification method of UVA multispectral remote sensing based on deep semantic segmentation. Transactions of the Chinese Society for Agricultural Machinery, 52(3): 185-192
  47. 47.
    You N S and Dong J W. 2020. Examining earliest identifiable timing of crops using all available Sentinel 1/2 imagery and Google Earth Engine. ISPRS Journal of Photogrammetry and Remote Sensing, 161: 109-123
  48. 48.
    You N S, Dong J W, Huang J X, Du G M, Zhang G L, He Y L, Yang T, Di Y Y and Xiao X M. 2021. The 10-m crop type maps in Northeast China during 2017—2019. Scientific Data, 8(1): 41
  49. 49.
    Zhang D J, Pan Y Z, Zhang J S, Hu T G, Zhao J H, Li N and Chen Q. 2020. A generalized approach based on convolutional neural networks for large area cropland mapping at very high resolution. Remote Sensing of Environment, 247: 111912
  50. 50.
    Zhang L, Liu Z, Liu D Y, Xiong Q, Yang N, Ren T W, Zhang C, Zhang X D and Li S M. 2019. Crop mapping based on historical samples and new training samples generation in Heilongjiang province, China. Sustainability, 11(18): 5052
  51. 51.
    Zhang M, Lin H, Wang G X, Sun H and Fu J. 2018. Mapping paddy rice using a convolutional neural network (CNN) with landsat 8 datasets in the Dongting lake area, China. Remote Sensing, 10(11): 1840
  52. 52.
    Zhao H W, Chen Z X, Jiang H and Liu J. 2020. Early growing stage crop species identification in southern China based on sentinel-1A time series imagery and one-dimensional CNN. Transactions of the Chinese Society of Agricultural Engineering, 36(3): 169-177
  53. 53.
    Zhong L H, Gong P and Biging G S. 2014. Efficient corn and soybean mapping with temporal extendability: a multi-year experiment using Landsat imagery. Remote Sensing of Environment, 140: 1-13
  54. 54.
    Zhong L H, Hu L N and Zhou H. 2019. Deep learning based multi-temporal crop classification. Remote Sensing of Environment, 221: 430-443
  55. 55.
    Zhong Y F, Hu X, Luo C, Wang X Y, Zhao J and Zhang L P. 2020. WHU-Hi: UAV-borne hyperspectral with high spatial resolution (H2) benchmark datasets and classifier for precise crop identification based on deep convolutional neural network with CRF. Remote Sensing of Environment, 250: 112012
  56. 56.
    Zhu X X, Tuia D, Mou L C, Xia G S, Zhang L P, Xu F and Fraundorfer F. 2017. Deep learning in remote sensing: a comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4): 8-36

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website