Précédent|Prochain

A review for sample datasets of remote sensing imagery

  • role： First author第一作者
  • Affiliation：

    College of Land Science and Technology, China Agricultural University, Beijing 100083, China

    Key Laboratory of Remote Sensing for Agri-Hazards, Ministry of Agriculture, Beijing 100083, China

    State Key Laboratory of Resources and Environmental Information System, Beijing 100101, China

  • Email：fengql@cau.edu.cn
  • Introduction：冯权泷，研究方向为机器学习、深度学习及在农业遥感中的应用。E-mail：fengql@cau.edu.cn
FENG Quanlong123,  
  • Affiliation：

    College of Land Science and Technology, China Agricultural University, Beijing 100083, China

CHEN Boan1,  
  • Affiliation：

    Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China

LI Guoqing4,  
  • role： Corresponding author通信作者
  • Affiliation：

    College of Land Science and Technology, China Agricultural University, Beijing 100083, China

    Key Laboratory of Remote Sensing for Agri-Hazards, Ministry of Agriculture, Beijing 100083, China

  • Email：yxc@cau.edu.cn
  • Introduction：姚晓闯，研究方向为空间数据组织、管理与分析等。E-mail：yxc@cau.edu.cn
YAO Xiaochuang12*,  
  • Affiliation：

    College of Land Science and Technology, China Agricultural University, Beijing 100083, China

    Key Laboratory of Remote Sensing for Agri-Hazards, Ministry of Agriculture, Beijing 100083, China

GAO Bingbo12,  
  • Affiliation：

    Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China

ZHANG Lianchong4

résumé

With the rapid development of artificial intelligence technology such as machine learning and deep learning in remote sensing, data-driven models have become a new research paradigm for automatic information retrieval from remote sensing imagery, calling for higher requirements for the quantity, quality, and diversity of sample datasets. Before the era of deep learning, because classical machine learning methods (e.‍g., support vector machine and random forest) do not require huge numbers of samples for model training, the previously published sample datasets usually have a relatively small size (i.e., less than 100). In recent years, with the rapid development of technologies such as big data, parallel computing, and deep learning, many scholars and research institutions have issued a series of sample datasets, laying a solid foundation for a wide range of research and applications such as scene understanding, semantic segmentation, and object detection from remote sensing images. However, comprehensive review of the recently published sample datasets for remote sensing image analysis under the context of big data and deep learning remains lacking. Therefore, the objective of this study is to summarize and analyze these datasets to provide a valuable data reference for relevant researchers.On the basis of literature retrieval and analysis, this paper summarized a total of 124 widely used, open access, and influential remote sensing image sample datasets that were published between 2001 and 2020.We reviewed and summarized the development of recently published sample datasets for remote sensing imagery based on metadata analysis from the following aspects, such as data sources, application fields, keywords, and data size. Afterward, we analyzed these sample datasets from the perspective of spatial, spectral, and temporal resolutions. We listed the commonly used deep learning models (e.g., convolutional neural networks, recurrent neural networks, and generative adversarial networks) in the remote sensing field to show how these sample datasets could be used. We also divided the remote sensing image sample datasets into eight categories based on the following application fields: scene recognition, land cover/land use classification, thematic information extraction, change detection, ground-object detection, semantic segmentation, quantitative remote sensing, and other applications. The typical datasets and related research progress were carefully reviewed for each application field. In addition, because deep learning models are data-hungry, how to train a model with good generalization capability under limited labeled data has become a significant issue, especially for remote sensing applications given that obtaining sufficient labeled samples is time-consuming. To address this issue, we discussed several methods that could increase the model’s generalization capability, including sample transfer between spatio-temporal domains, few-shot learning, and zero-shot learning, active learning, and semi-supervised learning for sample discovery, as well as sample generation through generative adversarial networks.By means of multi-dimensional analysis, we give a comprehensive overview of remote sensing image sample datasets. To the best of our knowledge, this paper is the first review of remote sensing image sample datasets for deep learning, potentially providing data reference for researchers in related fields.

mots-clés

remote sensing imagery;sample datasets;machine learning;deep learning

References

  1. 1.
    Bai J H, Xiao Q and Liu Q H. 2020. Ground observation data set from HL_RS_TS. Digital Journal of Global Change Data Repository.
  2. 2.
    Bao J F. 2014. Research on Remote Sensing Image Processing with Machine Learning and Its Applications. Shanghai: Fudan University.
  3. 3.
    Basu S, Ganguly S, Mukhopadhyay S, DiBiano R, Karki M and Nemani R. 2015. DeepSat: a learning framework for satellite imagery//Proceedings of the 23rd SIGSPATIAL International Conference on Advances in Geographic Information Systems. Seattle, Washington: ACM
  4. 4.
    Benedek C and Szirányi T Á. 2009. Change detection in optical aerial images by a multilayer conditional mixed Markov model. IEEE Transactions on Geoscience and Remote Sensing, 47(10): 3416-3430
  5. 5.
    Bourdis N, Marraud D and Sahbi H. 2011. Constrained optical flow for aerial image change detection//2011 IEEE International Geoscience and Remote Sensing Symposium. Vancouver, BC, Canada: IEEE
  6. 6.
    Cen Y, Zhang L F, Zhang X, Wang Y M, Qi W C, Tang S L and Zhang P. 2020. Aerial hyperspectral remote sensing classification dataset of Xiongan New Area (Matiwan Village). Journal of Remote Sensing, 24(11): 1299-1306
  7. 7.
    Chen Q, Wang L, Wu Y F, Wu G M, Guo Z L and Waslander S L. 2019. TEMPORARY REMOVAL: aerial Imagery for roof segmentation: a large-scale dataset towards automatic mapping of buildings. ISPRS Journal of Photogrammetry and Remote Sensing, 147: 42-55
  8. 8.
    Cheng G and Han J W. 2016. A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 117: 11-28
  9. 9.
    Chiu M T, Xu X Q, Wei Y C, Huang Z L, Schwing A G, Brunner R, Khachatrian H, Karapetyan H, Dozier I, Rose G, Wilson D, Tudor A, Hovakimyan N, Huang T S and Shi H H. 2020. Agriculture-vision: a large aerial image database for agricultural pattern analysis//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, WA, USA: IEEE
  10. 10.
    Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H and Bengio Y. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078
  11. 11.
    Cosmi Q and Globe D. 2018. SpaceNet on Amazon Web Services (AWS). “Datasets”: the SpaceNet catalog. Last modified October 1st [EB/OL].
  12. 12.
    Dai D X and Yang W. 2011. Satellite image classification via two-layer sparse coding with biased image representation. IEEE Geoscience and Remote Sensing Letters, 8(1): 173-176
  13. 13.
    Daudt R C, Le Saux B, Boulch A and Gousseau Y. 2018. Urban change detection for multispectral earth observation using convolutional neural networks//Proceedings of 2018 IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain: IEEE
  14. 14.
    Debes C, Merentitis A, Heremans R, HahnJ, Frangiadakis N, van Kasteren T, Liao W Z, Bellens R, Pižurica A, Gautama S, Gautama S, Philips W, Prasad S, Du Q and Pacifici F. 2014. Hyperspectral and LiDAR data fusion: outcome of the 2013 GRSS data fusion contest. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6): 2405-2418
  15. 15.
    Dierckx W, Sterckx S, Benhadj I, Livens S, Duhoux G, van Achteren T, Francois M, Mellab K and Saint G. 2014. PROBA-V mission for global vegetation monitoring: standard products and image quality. International Journal of Remote Sensing, 35(7): 2589-2614
  16. 16.
    Fujita A, Sakurada K, Imaizumi T, Ito R, Hikosaka S and Nakamura R. 2017. Damage detection from aerial images via convolutional neural networks//2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA). Nagoya, Japan: IEEE
  17. 17.
    GIC. Hyperspectral Remote Sensing Scenes. 2020. Available online: [2021-10-01]
  18. 18.
    Hackel T, Savinov N, Ladicky L, Wegner J D, Schindler K and Pollefeys M. 2017. Semantic3d.net: a new large-scale point cloud classification benchmark. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-1/W1: 91-98
  19. 19.
    Han S and Han Q H. 2020. Review of semi-supervised learning research. Computer Engineering and Applications, 56(6): 19-27
  20. 20.
    Helber P, Bischke B, Dengel A and Borth D. 2019. EuroSAT: a novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7): 2217-2226
  21. 21.
    Hochreiter S and Schmidhuber J. 1997. Long short-term memory. Neural Computation, 9(8): 1735-1780
  22. 22.
    Hughes M J and Hayes D J. 2014. Automated detection of cloud and cloud shadow in single-date Landsat imagery using neural networks and spatial post-processing. Remote Sensing, 6(6): 4907-4926
  23. 23.
    Ji S P and Wei S Q. 2019. Building extraction via convolutional neural networks from an open remote sensing building dataset. Acta Geodaetica et Cartographica Sinica, 48(4): 448-459
  24. 24.
    Johnson B and Xie Z X. 2013. Classifying a high resolution image of an urban area using super-object information. ISPRS Journal of Photogrammetry and Remote Sensing, 83: 40-49
  25. 25.
    Kaiser P, Wegner J D, Lucchi A, Jaggi M, Hofmann T and Schindler K. 2017. Learning aerial image segmentation from online maps. IEEE Transactions on Geoscience and Remote Sensing, 55(11): 6054-6068
  26. 26.
    Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada: Curran Associates Inc.: 1097-1105
  27. 27.
    Lebedev M A, Vizilter Y V, Vygolov O V, Knyaz V A and Rubis A Y. 2018. Change detection in remote sensing images using conditional adversarial networks. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2: 565-571
  28. 28.
    Li K, Wan G, Cheng G, Meng L Q and Han J W. 2020. Object detection in optical remote sensing images: a survey and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing, 159: 296-307
  29. 29.
    Li W B and He R. 2020. Aircraft target detection of remote sensing images based on deep neural network. Computer Engineering, 46(7): 268-276
  30. 30.
    Li X, Li X W, Li Z Y, Wang J, Ma M G, Liu Q, Xiao Q, Hu Z Y, Che T, Wang J M, Liu Q H, Chen E X, Yan G J, Liu S M, Wang W Z, Zhang L X, Wang J D, Niu Z, Jin R, Ran Y H and Wang L X. 2012. Progresses on the watershed allied telemetry experimental research (WATER). Remote Sensing Technology and Application, 27(5): 637-649
  31. 31.
    Li X W. 2005. Retrospect, prospect and innovation in quantitative remote sensing. Journal of Henan University (Natural Science), 35(4): 49-56
  32. 32.
    Long Y, Gong Y P, Xiao Z F and Liu Q. 2017. Accurate object localization in remote sensing images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 55(5): 2486-2498
  33. 33.
    Lu L, Li X, Veroustraete F, Kang E and Wang J. 2009. Analysing the forcing mechanisms for net primary productivity changes in the Heihe River Basin, North‐West China. International Journal of Remote Sensing, 30(3): 793-816
  34. 34.
    Lu X Q, Wang B Q, Zheng X T and Li X L. 2017. Exploring models and data for remote sensing image caption generation. IEEE Transactions on Geoscience and Remote Sensing, 56(4): 2183-2195
  35. 35.
    Ma A L, Zhong Y F and Zhang L P. 2015. Adaptive multiobjective memetic fuzzy clustering algorithm for remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 53(8): 4202-4217
  36. 36.
    Maggiori E, Tarabalka Y, Charpiat G and Alliez P. 2017. Can semantic labeling methods generalize to any city? The inria aerial image labeling benchmark//2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS): Fort Worth, TX, USA: IEEE
  37. 37.
    Mnih V. 2013. Machine Learning for Aerial Image Labeling. Toronto: University of Toronto
  38. 38.
    Mou L C, Bruzzone L and Zhu X X. 2019. Learning spectral-spatial-temporal features via a recurrent convolutional neural network for change detection in multispectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 57(2): 924-935
  39. 39.
    Mueller M, Smith N and Ghanem B. 2016. A benchmark and simulator for UAV tracking//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer
  40. 40.
    Mundhenk T N, Konjevod G, Sakla W A and Boakye K. 2016. A large contextual dataset for classification, detection and counting of cars with deep learning//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer
  41. 41.
    Papadomanolaki M, Vakalopoulou M and Karantzalos K. 2017. Patch-based deep learning architectures for sparse annotated very high resolution datasets//2017 Joint Urban Remote Sensing Event (JURSE). Dubai, United Arab Emirates: IEEE
  42. 42.
    Petrişor A I. 2015. Assessment of the green infrastructure of Bucharest using Corine and urban atlas data. Urbanism Arhitectură Construcţii, 6(2): 19-24
  43. 43.
    Qiu Y B and Wang X X. 2020. Daily fractional snow cover dataset over High Asia during 2002 to 2018. National Cryosphere and Technology Data Center
  44. 44.
    Rabbi J, Ray N, Schubert M, Chowdhury S and Chao D. 2020. Small-object detection in remote sensing images with end-to-end edge-enhanced GAN and object detector network. Remote Sensing, 12(9): 1432
  45. 45.
    Robicquet A, Sadeghian A, Alexandre A and Savarese S. 2016. Learning social etiquette:Human trajectory understanding in crowded scenes//14th European conference on computer vision. Amsterdam, Netherlands: Springer: 549-565
  46. 46.
    Schmitt M, Hughes L H and Zhu X X. 2018. The SEN1-2 dataset for deep learning in SAR-optical data fusion. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-1: 141-146
  47. 47.
    Settles B. 2009. Active Learning Literature Survey. Madison: University of Wisconsin: 52.
  48. 48.
    Sharma A, Liu X W and Yang X J. 2018. Land cover classification from multi-temporal, multi-spectral remotely sensed imagery using patch-based recurrent neural networks. Neural Networks, 105: 346-355
  49. 49.
    Shermeyer J, Hossler T, Van Etten A, Hogan D, Lewis R and Kim D. 2020. RarePlanes: synthetic data takes flight//2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa, HI, USA: IEEE
  50. 50.
    Singh A. 1989. Review article digital change detection techniques using remotely-sensed data. International Journal of Remote Sensing, 10(6): 989-1003
  51. 51.
    Sumbul G, Charfuelan M, Demir B and Markl V. 2019. Bigearthnet: a large-scale benchmark archive for remote sensing image understanding//2019 IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan: IEEE
  52. 52.
    Tong X Y, Xia G S, Lu Q K, Shen H F, Li S Y, You S C and Zhang L P. 2020. Land-cover classification with high-resolution remote sensing images using transferable deep models. Remote Sensing of Environment, 237: 111322
  53. 53.
    Wang J S, Jiang N, Zhang G M, Li Y and Lv H. 2015. Incremental classification algorithm of hyperspectral remote sensing images based on spectral-spatial information. Acta Geodaetica et Cartographica Sinica, 44(9): 1003-1013
  54. 54.
    Wang Q, Yuan Z H, Du Q and Li X L. 2019a. GETNET: a general end-to-end 2-D CNN framework for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing, 57(1): 3-13
  55. 55.
    Wang Y Q, Yao Q M, Kwok J T and Ni L M. 2020. Generalizing from a few examples: a survey on few-shot learning. ACM Computing Surveys, 53(3): 63
  56. 56.
    Wang Y Y, Wang C, Zhang H, Dong Y B and Wei S S. 2019b. A SAR dataset of ship detection for deep learning under complex backgrounds. Remote Sensing, 11(7): 765
  57. 57.
    Weber E and Kané H. 2020. Building disaster damage assessment in satellite imagery with multi-temporal fusion. arXiv: 2004.05525
  58. 58.
    Wei D S and Zhou X G. 2019. Automatic sampling of remote sensing image change detection samples based on prior information of vector data. Journal of Remote Sensing, 23(3): 464-475.
  59. 59.
    Wu C, Zhang L F and Zhang L P. 2016. A scene change detection framework for multi-temporal very high resolution remote sensing images. Signal Processing, 124: 184-197
  60. 60.
    Wu C, Zhang L P and Du B. 2017. Kernel slow feature analysis for scene change detection. IEEE Transactions on Geoscience and Remote Sensing, 55(4): 2367-2384
  61. 61.
    Xia G S, Bai X, Ding D, Zhu Z, Belongie S, Luo J B, Datcu M, Pelillo M and Zhang L P. 2018. DOTA: a large-scale dataset for object detection in aerial images//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE
  62. 62.
    Xia G S, Hu J W, Hu F, Shi B G, Bai X, Zhong Y F, Zhang L P and Lu X Q. 2017. AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3965-3981
  63. 63.
    Yan P Y. 2019. Research on Remote Sensing Image Scene Classification Based on Generative Adversarial Networks. Wuhan: Huazhong University of Science and Technology.
  64. 64.
    Yang Y and Newsam S. 2010. Bag-of-visual-words and spatial extensions for land-use classification//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. San Jose, California: ACM
  65. 65.
    Yang Y M. 2010. Estimation of evapotranspiration in Heihe River Basin based on remote sensing. Gansu: Lanzhou University
  66. 66.
    Yin J B, Gentine P, Zhou S, Sullivan S C, Wang R, Zhang Y and Guo S L. 2018. Large increase in global storm runoff extremes driven by climate and anthropogenic changes. Nature Communications, 9(1): 4389
  67. 67.
    Yokoya N and Iwasaki A. 2016. Airborne Hyperspectral Data over Chikusei. Space Application Laboratory, University of Tokyo
  68. 68.
    Yu D H, Zhang B M, Zhao C, Guo H T and Lu J. 2020. Scene classification of remote sensing image using ensemble convolutional neural network. Journal of Remote Sensing, 24(6): 717-727
  69. 69.
    Yu L X, Zhang S W, Bu K, Yang J C, Yan F Q and Chang L P. 2013. A Review on Snow Data Sets. Scientia Geographica Sinica, 33(7): 878-883
  70. 70.
    Yu X F, Zhuang D F and Wang Q K. 2014. Remote sensing based forest phenology data of Northeast China (2000-2010). Acta Geographica Sinica, 69(S1): 72-76
  71. 71.
    Yuan J Y, Gleason S S and Cheriyadat A M. 2013. Systematic benchmarking of aerial image segmentation. IEEE Geoscience and Remote Sensing Letters, 10(6): 1527-1531
  72. 72.
    Zeng Y L, Li J and Liu Q H. 2012. Global LAI ground validation dataset and product validation framework: a review. Advances in Earth Science, 27(2): 165-174
  73. 73.
    Zhang B. 2018. Remotely sensed big data era and intelligent information extraction. Geomatics and Information Science of Wuhan University, 43(12): 1861-1871
  74. 74.
    Zhang L N and Zhou Q. 2014. Proceedings of the Twenty-eighth Academic Information Exchange Meeting of Central South Branch of National Surveying and Mapping Science and Technology Information Network. Kaifeng: Science and Technology Information Network Branch of China Society of Surveying and Mapping
  75. 75.
    Zhang M, Hu X Y, Zhao L K, Lv Y, Luo M and Pang S Y. 2017. Learning dual multi-scale manifold ranking for semantic segmentation of high-resolution images. Remote Sensing, 9(5):500
  76. 76.
    Zhang X R, Liu L P, Fang S, Jiang W C and Wang J C. 2014. Research advances on the relationship between land use/cover change and environmental change. Ecology and Environmental Sciences, 23(12): 2013-2021
  77. 77.
    Zhang Y L, Yuan Y, Feng Y C and Lu X Q. 2019. Hierarchical and robust convolutional neural network for very high-resolution remote sensing object detection. IEEE Transactions on Geoscience and Remote Sensing, 57(8): 5535-5548
  78. 78.
    Zhu H G, Chen X G, Dai W Q, Fu K, Ye Q X and Jiao J B. 2015. Orientation robust object detection in aerial images using deep convolutional neural network//2015 IEEE International Conference on Image Processing (ICIP). Quebec City, QC, Canada: IEEE
  79. 79.
    Zhu P F, Wen L Y, Du D W, Bian X, Fan H, Hu Q H and Ling H B. 2020. Detection and tracking meet drones challenge. arXiv: 2001.06303
  80. 80.
    Zhu Q Q, Zhong Y F, Zhao B, Xia G S and Zhang L P. 2016. Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery. IEEE Geoscience and Remote Sensing Letters, 13(6): 747-751
  81. 81.
    Zou Q, Ni L H, Zhang T and Wang Q. 2015. Deep learning based feature selection for remote sensing scene classification. IEEE Geoscience and Remote Sensing Letters, 12(11): 2321-2325

Lire l'article complet

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website