سابِق|تال

A comprehensive review of optical remote-sensing image object detection datasets

  • role： First author第一作者
  • Affiliation：

    School of Automation, Northwestern Polytechnic University, Xi'an 710021, China

  • Email：NPUyuan@163.com
  • Introduction：袁一钦，研究方向为深度学习，遥感图像处理。E-mail：NPUyuan@163.com
YUAN Yiqin,  
  • Affiliation：

    School of Automation, Northwestern Polytechnic University, Xi'an 710021, China

LI Lang,  
  • role： Corresponding author通信作者
  • Affiliation：

    School of Automation, Northwestern Polytechnic University, Xi'an 710021, China

  • Email：yaoxiwen@nwpu.edu.cn
  • Introduction：姚西文，研究方向为资源受限下的遥感智能解译。E-mail：yaoxiwen@nwpu.edu.cn
YAO Xiwen*,  
  • Affiliation：

    School of Automation, Northwestern Polytechnic University, Xi'an 710021, China

LI Lingjun,  
  • Affiliation：

    School of Automation, Northwestern Polytechnic University, Xi'an 710021, China

FENG Xiaoxu,  
  • Affiliation：

    School of Automation, Northwestern Polytechnic University, Xi'an 710021, China

CHENG Gong,  
  • Affiliation：

    School of Automation, Northwestern Polytechnic University, Xi'an 710021, China

HAN Junwei

ملخص

With the introduction of artificial-intelligence technologies such as deep learning into the field of optical remote-sensing detection, various algorithms have emerged. The use of these algorithms has gradually formed a new paradigm of data-driven optical remote-sensing image object detection. Consequently, high-quality remote-sensing data has become a prerequisite and a necessary resource for researching these paradigm algorithms. highlighting the increasing importance of remote-sensing data. To date, numerous optical remote-sensing image object detection datasets have been published by major research institutions domestically and internationally. These datasets have laid the foundation for the development of deep learning-based remote-sensing image detection tasks. However, no comprehensive summarization and analysis of the published optical remote-sensing image detection datasets have been conducted by scholars. Therefore, this paper aimed to provide a comprehensive review of the published datasets and an overview of algorithm applications. We also aimed to provide a reference for subsequent research in related fields.This paper presents an overview and synthesis of the optical remote-sensing image object detection datasets published between 2008 and 2023. The synthesis is based on an extensive and comprehensive survey of literature in the field. By reviewing and analyzing these datasets, we enable a comprehensive understanding of the progress and trends in optical remote-sensing image object detection dataset research.This paper categorizes the optical remote-sensing image object detection datasets published from 2008 to 2023 based on the annotation method. A comprehensive description of 11 representative datasets is provided, and all dataset information are summarized in tabular form. The analysis considers the information in the datasets themselves and also the spatial and spectral resolution of the images in the datasets. Other basic information including the number of categories, number of images, number of instances, and image-width information are also considered. This analysis effectively demonstrates the trend toward high quality, large scale, and multi-category development of object-detection datasets for optical remote-sensing images. Additionally, we provide an overview of the development and application of algorithms related to published datasets from different perspectives (e.g., horizontal bounding box object detection and rotated bounding box object detection), as well as a subdivision of detection directions (e.g., small object detection and fine-grained detection). Our findings confirm the influential role of remote-sensing data in driving algorithmic advances.In summary, we offer a comprehensive review of optical remote-sensing image object detection datasets from various perspectives. To our best knowledge, this comprehensive review is the first one on such datasets in the field. The work serves as a valuable reference for subsequent research on deep learning-based optical remote-sensing image object detection, providing insights into data availability and research directions. This study is expected to contribute to the advancement of this field by offering a solid foundation for further investigation and innovation.

مفهوم

deep learning;optical remote sensing imagery;data source;object detection;development of datasets

References

  1. 1.
    Bakirman T and Sertel E. 2022. HRPlanes: high resolution airplane dataset for deep learning. arXiv preprint arXiv:2204.10959
  2. 2.
    Benedek C, Descombes X and Zerubia J. 2012. Building development monitoring in multitemporal remotely sensed image pairs with stochastic birth-death dynamics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(1): 33-50
  3. 3.
    Cha K, Seo J and Lee T. 2023. A billion-scale foundation model for remote sensing images. arXiv:2304.05215
  4. 4.
    Chen K Y, Wu M, Liu J M and Zhang C. 2020. FGSD: a dataset for fine-grained ship detection in high resolution satellite images. arXiv:2003.06832
  5. 5.
    Cheng G and Han J W. 2016a. A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 117: 11-28
  6. 6.
    Cheng G, Han J W, Zhou P C and Guo L. 2014. Multi-class geospatial object detection and geographic image classification based on collection of part detectors. ISPRS Journal of Photogrammetry and Remote Sensing, 98: 119-132
  7. 7.
    Cheng G, Wang J B, Li k, Xie X X, Lang C B, Yao Y Q and Han J W. 2022. Anchor-free oriented proposal generator for object detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 5625411
  8. 8.
    Cheng G, Yuan X, Yao X W, Yan K B, Zeng Q H, Xie X X and Han J W. 2023. Towards large-scale small object detection: survey and benchmarks. arXiv preprint arXiv:2207.14096
  9. 9.
    Cheng G, Zhou P C and Han J W. 2016b. Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 54(12): 7405-7415
  10. 10.
    Devaki P, Vineetha P N, Reddy C H, Bharathi P, Karimulla S and Kumar S U. 2023. Fine-grained feature enhancement for object detection in remote sensing images. International Research Journal of Modernization in Engineering Technology and Science, 5(3): 2112-2118
  11. 11.
    Ding J, Xue N, Long Y, Xia G S and Lu Q K. 2019. Learning roi transformer for oriented object detection in aerial images//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 2844-2853
  12. 12.
    Ding J, Xue N, Xia G S, Bai X, Yang W, Yang M Y, Belongie S, Luo J B, Datcu M, Pelillo M and Zhang L P. 2022. Object detection in aerial images: a large-scale benchmark and challenges. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11): 7778-7796
  13. 13.
    Fu S S, He Y F, Du X F and Zhu Y. 2023. Anchor-free object detection in remote sensing images using a variable receptive field network. EURASIP Journal on Advances in Signal Processing, 2023(1): 53
  14. 14.
    Girshick R, Donahue J, Darrell T and Malik J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 580-587
  15. 15.
    Han J M, Ding J, Li J and Xia G S. 2022a. Align deep features for oriented object detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 5602511
  16. 16.
    Han J M, Ding J, Xue N and Xia G S. 2021. ReDet: a rotation-equivariant detector for aerial object detection//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE: 2785-2794
  17. 17.
    Han Y Q, Yang X Y, Pu T and Peng Z M. 2022b. Fine-grained recognition for oriented ship against complex scenes in optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 60: 5612318
  18. 18.
    Haroon M, Shahzad M and Fraz M M. 2020. Multisized object detection using spaceborne optical imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 3032-3046
  19. 19.
    He K M, Chen X L, Xie S N, Li Y H, Dollár P and Girshick R. 2022. Masked autoencoders are scalable vision learners//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE: 15979-15988
  20. 20.
    Heitz G and Koller D. 2008. Learning spatial context: using stuff to find things//10th European Conference on Computer Vision. Marseille, France: Springer: 30-43
  21. 21.
    Hsieh M R, Lin Y L and Hsu W H. 2017. Drone-based object counting by spatially regularized regional proposal network//Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice: IEEE: 4165-4173
  22. 22.
    Lam D, Kuzma R, McGee K, Dooley S, Laielli M, Klaric M, Bulatov Y and McCord B. 2018. xView: objects in context in overhead imagery. arXiv preprint arXiv:1802.07856
  23. 23.
    Li K, Wan G, Cheng G, Meng L Q and Han J W. 2020a. Object detection in optical remote sensing images: a survey and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing, 159: 296-307
  24. 24.
    Li W T, Chen Y J, Hu K X and Zhu J K. 2022a. Oriented RepPoints for aerial object detection//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE: 1819-1828
  25. 25.
    Li X, Deng J Y and Fang Y. 2022b. Few-shot object detection on remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 60: 5601614
  26. 26.
    Li Y Y, Pei X, Huang Q, Jiao L C, Shang R H and Marturi N. 2020b. Anchor-free single stage detector in remote sensing images based on multiscale dense path aggregation feature pyramid network. IEEE Access, 8: 63121-63133
  27. 27.
    Lin T Y, Goyal P, Girshick R, He K M and Dollár P. 2017. Focal loss for dense object detection//Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice: IEEE: 2999-3007
  28. 28.
    Lin Y T, Feng P M, Guan J, Wang W W and Chambers J. 2021. IENet: interacting embranchment one stage anchor free detector for orientation aerial object detection. arXiv:1912.00969
  29. 29.
    Liu F, Chen D L, Guan Z Q Y, Zhou X C, Zhu J L and Zhou J. 2023. RemoteCLIP: a vision language foundation model for remote sensing. arXiv:2306.11029
  30. 30.
    Liu J M, Li S J, Zhou C S, Cao X Y, Gao Y and Wang B. 2022. SRAF-Net: a scene-relevant anchor-free object detection network in remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 60: 5405914
  31. 31.
    Liu K and Mattyus G. 2015. Fast multiclass vehicle detection on aerial images. IEEE Geoscience and Remote Sensing Letters, 12(9): 1938-1942
  32. 32.
    Liu Z K, Yuan L, Weng L B and Yang Y P. 2017. A high resolution optical satellite image dataset for ship recognition and some new baselines//Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods. Porto: SciTePress: 324-331
  33. 33.
    Long Y, Gong Y P, Xiao Z F and Liu Q. 2017. Accurate object localization in remote sensing images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 55(5): 2486-2498
  34. 34.
    Long Y, Xia G S, Li S Y, Yang W, Yang M Y, Zhu X X, Zhang L P and Li D R. 2021. On creating benchmark dataset for aerial image interpretation: reviews, guidances, and million-aid. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14: 4205-4230
  35. 35.
    Ma J Q, Shao W Y, Ye H, Wang L, Wang H, Zheng Y B and Xue X Y. 2018. Arbitrary-oriented scene text detection via rotation proposals. IEEE Transactions on Multimedia, 20(11): 3111-3122
  36. 36.
    Mundhenk T N, Konjevod G, Sakla W A and Boakye K. 2016. A large contextual dataset for classification, detection and counting of cars with deep learning//14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer: 785-800
  37. 37.
    Nie G T and Huang H. 2021. A survey of object detection in optical remote sensing images. Acta Automatica Sinica, 47(8): 1749-1768
  38. 38.
    Nogueira K, Cesar C, Gama P H T, Machado G L S and dos Santos J A. 2019. A tool for bridge detection in major infrastructure works using satellite images//2019 XV Workshop de Visão Computacional (WVC). São Bernardo do Campo: IEEE: 72-77
  39. 39.
    Razakarivony S and Jurie F. 2016. Vehicle detection in aerial imagery: a small target detection benchmark. Journal of Visual Communication and Image Representation, 34: 187-203
  40. 40.
    Redmon J and Farhadi A. 2018. YOLOv3: an incremental improvement. arXiv:1804.02767
  41. 41.
    Ren S Q, He K M, Girshick R and Sun J. 2015. Faster R-CNN: towards real-time object detection with region proposal networks//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal: MIT Press: 91-99
  42. 42.
    Shermeyer J, Hossler T, Van Etten A, Hogan D, Lewis R and Kim D. 2021. RarePlanes: synthetic data takes flight//Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa: IEEE: 207-217
  43. 43.
    Shi T J, Gong J N, Jiang S K, Zhi X Y, Bao G Z, Sun Y and Zhang W. 2023. Complex optical remote-sensing aircraft detection dataset and benchmark. IEEE Transactions on Geoscience and Remote Sensing, 61: 5612309
  44. 44.
    Shivappriya S N, Priyadarsini M J P, Stateczny A, Puttamadappa C and Parameshachari B D. 2021. Cascade object detection and remote sensing object detection method based on trainable activation function. Remote Sensing, 13(2): 200
  45. 45.
    Song J J, Miao L J, Ming Q, Zhou Z Q and Dong Y P. 2023. Fine-grained object detection in remote sensing images via adaptive label assignment and refined-balanced feature pyramid network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16: 71-82
  46. 46.
    Sun X, Wang P J, Lu W X, Zhu Z C, Lu X N, He Q B, Li J X, Rong X E, Yang Z J, Chang H , He Q L, Yang G, Wang R P, Lu J W and Fu K. 2023. RingMo: a remote sensing foundation model with masked image modeling. IEEE Transactions on Geoscience and Remote Sensing, 61: 5612822
  47. 47.
    Sun X, Wang P J, Yan Z Y, Xu F, Wang R P, Diao W H, Chen J, Li J H, Feng Y C, Xu T, Weinmann M, Hinz S, Wang C and Fu K. 2022. FAIR1M: a benchmark dataset for fine-grained object recognition in high-resolution remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 184: 116-130
  48. 48.
    Tanner F, Colder B, Pullen C, Heagy D, Eppolito M, Carlan V, Oertel C and Sallee P. 2009. Overhead imagery research data set—An annotated data library and tools to aid in the development of computer vision algorithms//2009 IEEE Applied Imagery Pattern Recognition Workshop (AIPR 2009). Washington: IEEE: 1-8
  49. 49.
    Tian Z, Shen C H, Chen H and He T. 2019. FCOS: fully convolutional one-stage object detection//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE: 9626-9635
  50. 50.
    Uijlings J R R, Van De Sande K E A, Gevers T and Smeulders A W M. 2013. Selective search for object recognition. International Journal of Computer Vision, 104(2): 154-171
  51. 51.
    Wang J, Yang L and Li F. 2021a. Predicting arbitrary-oriented objects as points in remote sensing images. Remote Sensing, 13(18): 3731
  52. 52.
    Wang J W, Yang W, Guo H W, Zhang R X and Xia G S. 2021b. Tiny object detection in aerial images//2020 25th International Conference on Pattern Recognition (ICPR). Milan: IEEE: 3791-3798
  53. 53.
    Wang J W, Yang W, Li H C, Zhang H J and Xia G S. 2021c. Learning center probability map for detecting objects in aerial images. IEEE Transactions on Geoscience and Remote Sensing, 59(5): 4307-4323
  54. 54.
    Wang K, Wang Z, Li Z, Su A, Teng X C, Liu M H and Yu Q F. 2023. Oriented object detection in optical remote sensing images using deep learning: a survey. arXiv preprint arXiv:2302.10473
  55. 55.
    Wei H R, Zhang Y, Chang Z H, Li H, Wang H Q and Sun X. 2020. Oriented objects as pairs of middle lines. ISPRS Journal of Photogrammetry and Remote Sensing, 169: 268-279
  56. 56.
    Xia G S, Bai X, Ding J, Zhu Z, Belongie S, Luo J B, Datcu M, Pelillo M and Zhang L P. 2018. DOTA: a large-scale dataset for object detection in aerial images//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 3974-3983
  57. 57.
    Xiao Z F, Liu Q, Tang G F and Zhai X F. 2015. Elliptic Fourier transformation-based histograms of oriented gradients for rotationally invariant object detection in remote-sensing images. International Journal of Remote Sensing, 36(2): 618-644
  58. 58.
    Xie X X, Cheng G, Wang J B, Yao X W and Han J W. 2021. Oriented R-CNN for object detection//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE: 3500-3509
  59. 59.
    Xu C, Ding J, Wang J W, Yang W, Yu H, Yu L and Xia G S. 2023. Dynamic coarse-to-fine learning for oriented tiny object detection//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE: 7318-7328
  60. 60.
    Xu C, Wang J W, Yang W, Yu H, Yu L and Xia G S. 2022a. Detecting tiny objects in aerial images: a normalized Wasserstein distance and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing, 190: 79-93
  61. 61.
    Xu C, Wang J W, Yang W, Yu H, Yu L and Xia G S. 2022b. RFLA: gaussian receptive field based label assignment for tiny object detection//17th European Conference on Computer Vision. Tel Aviv: Springer: 526-543
  62. 62.
    Xu C, Wang J W, Yang W and Yu L. 2021. Dot distance for tiny object detection in aerial images//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville: IEEE: 1192-1201
  63. 63.
    Yang M Y, Liao W T, Li X B and Rosenhahn B. 2018. Deep learning for vehicle detection in aerial images//2018 25th IEEE International Conference on Image Processing (ICIP). Athens: IEEE: 3079-3083
  64. 64.
    Yang X, Yan J C, Feng Z M and He T. 2021a. R3Det: refined single-stage detector with feature refinement for rotating object//Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press: 3163-3171
  65. 65.
    Yang X, Yan J C, Ming Q, Wang W T, Zhang X P and Tian Q. 2021b. Rethinking rotated object detection with gaussian wasserstein distance loss//Proceedings of the 38th International Conference on Machine Learning. Virtual Event: [s.n.]: 11830-11841
  66. 66.
    Yang X, Zhou Y, Zhang G F, Yang J R, Wang W T, Yan J C, Zhang X P and Tian Q. 2023. The KFIoU loss for rotated object detection. arXiv:2201.12558
  67. 67.
    Yao Y Q, Cheng G, Xie X X and Han J W. 2021. Optical remote sensing image object detection based on multi-resolution feature fusion. National Remote Sensing Bulletin, 25(5): 1124-1137
  68. 68.
    Ye Y X, Ren X Y, Zhu B, Tang T F, Tan X, Gui Y and Yao Q. 2022. An adaptive attention fusion mechanism convolutional network for object detection in remote sensing images. Remote Sensing, 14(3): 516
  69. 69.
    Yu W Q, Cheng G, Wang M J, Yao Y Q, Xie X X, Yao X W and Han J W. 2022. MAR20: a benchmark for military aircraft recognition in remote sensing images. National Remote Sensing Bulletin: 1-11
  70. 70.
    Yu X H, Gong Y Q, Jiang N, Ye Q X and Han Z J. 2020. Scale match for tiny person detection//Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision. Snowmass: IEEE: 1246-1254
  71. 71.
    Zhang Y L, Yuan Y, Feng Y C and Lu X Q. 2019. Hierarchical and robust convolutional neural network for very high-resolution remote sensing object detection. IEEE Transactions on Geoscience and Remote Sensing, 57(8): 5535-5548
  72. 72.
    Zhong Y F, Han X B and Zhang L P. 2018. Multi-class geospatial object detection based on a position-sensitive balancing framework for high spatial resolution remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 138: 281-294
  73. 73.
    Zhou P C, Cheng G, Yao X W and Han J W. 2021. Machine learning paradigms in high-resolution remote sensing image interpretation. National Remote Sensing Bulletin, 25(1): 182-197
  74. 74.
    Zhu H G, Chen X G, Dai W Q, Fu K, Ye Q X and Jiao J B. 2015. Orientation robust object detection in aerial images using deep convolutional neural network//2015 IEEE International Conference on Image Processing (ICIP). Quebec City: IEEE: 3735-3739
  75. 75.
    Zhu P F, Wen L Y, Du D W, Bian X, Fan H, Hu Q H and Ling H B. 2022. Detection and tracking meet drones challenge. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11): 7380-7399
  76. 76.
    Zhuang S, Wang P, Jiang B R, Wang G and Wang C. 2019. A single shot framework with multi-scale feature fusion for geospatial object detection. Remote Sensing, 11(5): 594
  77. 77.
    Zou Z X and Shi Z W. 2018. Random access memories: s new paradigm for target detection in high resolution aerial remote sensing images. IEEE Transactions on Image Processing, 27(3): 1100-1111

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website