سابِق|تال

Survey on object detection in tilting box for remote sensing images

  • role： First author第一作者
  • Affiliation：

    Information Engineering University, Institute of Geospatial Information, Zhengzhou 450001, China

  • Email：zhang295498@126.com
  • Introduction：张磊，研究方向为深度学习与计算机视觉、遥感图像处理。E-mail：zhang295498@126.com
ZHANG Lei1,  
  • Affiliation：

    Information Engineering University, Institute of Geospatial Information, Zhengzhou 450001, China

ZHANG Yongsheng1,  
  • role： Corresponding author通信作者
  • Affiliation：

    Information Engineering University, Institute of Geospatial Information, Zhengzhou 450001, China

  • Email：yuying5559104@163.com
  • Introduction：于英，研究方向为无人机摄影测量、计算机视觉。E-mail：yuying5559104@163.com
YU Ying1*,  
  • Affiliation：

    Information Engineering University, Institute of Geospatial Information, Zhengzhou 450001, China

    JiMei University, Xiamen 361000, China

MA Yongzheng12,  
  • Affiliation：

    Information Engineering University, Institute of Geospatial Information, Zhengzhou 450001, China

    Marine Map Information Center, Tianjin 300450, China

JIANG Huaigang13

ملخص

Object detection is a fundamental aspect of remote sensing image processing. With the development of remote sensing image acquisition and the breakthrough in deep learning, object detection in aerial imagery based on deep learning has attracted considerable interest. Although significant progress has been made, there are still numerous obstacles due to the large-scale and highly complex backgrounds of optical remote sensing images. In addition, approaches based on horizontal proposals for common object detection frequently suffer from the mismatch issue when detecting densely arranged and arbitrarily oriented objects in aerial imagery. Therefore, numerous domestic and international researchers have proposed tilting box object detection algorithms based on deep learning that enhances the object detection effect of remote sensing images. This paper systematically organizes and summarizes them for researchers in related fields to comprehensively understand the theory, process, and existing problems of deep learning-based remote sensing image tilting box object detection.In this paper, we first analyze the limitations of Horizontal Bounding Box (HBB) object detection algorithms applied to remote sensing images, namely, the introduction of background noise, inappropriate post-processing operation, Non-Maximum Suppression (NMS), and the inability to accurately determine the orientation of objects, which can be remedied by the tilting bounding box object detection method.Following this, we list the classical HBB object detection algorithms based on deep learning and briefly describe their underlying principles. Then, the development of the tilting bounding box object detection algorithm and the process of improving the two-stage tilting bounding box object detection algorithm is described from three perspectives: the feature extraction network, anchor boxes, and the proposed region design. Finally, the one-stage detection algorithm’s loss function has been studied infrequently, so the two algorithms are merely introduced.In the fourth section, the detection performance of existing tilt box object detection algorithms is demonstrated on two publicly available and challenging aerial datasets (i.‍e., DOTA and HRSC2016). The comparison results of the three tables indicate that a particular object feature enhancement module must be designed to account for the uniqueness of the objects in remote sensing images and that the RSE problem in the algorithm for detecting tilting bounding boxes requires additional consideration. Although the one-stage detection algorithm is marginally less accurate than the two-stage algorithm, it has clear advantages in terms of efficiency and therefore has some research value.The paper concludes with a six-point summary of the tilt box target detection algorithm’s existing problems and an outlook on its future development trend.

مفهوم

remote sensing image;deep learning;convolutional neural network;object detection in tilting bounding box

References

  1. 1.
    Azimi S M, Vig E, Bahmanyar R, Körner M and Reinartz P. 2018. Towards multi-class object detection in unconstrained remote sensing imagery//14th Asian Conference on Computer Vision. Perth: Springer: 150-165
  2. 2.
    Dai J F, Li Y, He K M and Sun J. 2016. R-FCN: object detection via region-based fully convolutional networks//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona: ACM: 379-387
  3. 3.
    Dalal N and Triggs B. 2005. Histograms of oriented gradients for human detection//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). San Diego: IEEE: 886-893
  4. 4.
    Ding J, Xue N, Long Y, Xia G S and Lu Q K. 2019. Learning RoI transformer for oriented object detection in aerial images//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 2844-2853
  5. 5.
    Duan K W, Bai S, Xie L X, Qi H G, Huang Q M and Tian Q. 2019. CenterNet: keypoint triplets for object detection//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE: 6569-6578
  6. 6.
    Felzenszwalb P F, Girshick R B and McAllester D. 2010. Cascade object detection with deformable part models//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco: IEEE: 2241-2248
  7. 7.
    Fu K, Chen Z, Zhang Y and Sun X. 2019. Enhanced feature representation in detection for optical remote sensing images. Remote Sensing, 11(18): 2095
  8. 8.
    Fu K, Chang Z H, Zhang Y, Xu G L, Zhang K S and Sun X. 2020. Rotation-aware and multi-scale convolutional neural network for object detection in remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 161: 294-308
  9. 9.
    Girshick R. 2015. Fast R-CNN//Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE: 1440-1448
  10. 10.
    Girshick R, Donahue J, Darrell T and Malik J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 580-587
  11. 11.
    He K M, Gkioxari G, Dollár P and Girshick R. 2017. Mask R-CNN//Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE: 2980-2988
  12. 12.
    Jiang Y Y, Zhu X Y, Wang X B, Yang S L, Li W, Wang H, Fu P and Luo Z B. 2018. Rotational region cnn for arbitrarily-oriented scene text detection//2018 24th International Conference on Pattern Recognition (ICPR). IEEE: 3610-3615
  13. 13.
    Law H and Deng J. 2018. CornerNet: detecting objects as paired keypoints//Proceedings of the 15th European Conference on Computer Vision. Munich: Springer: 765-781
  14. 14.
    LeCun Y, Bengio Y and Hinton G. 2015. Deep learning. Nature, 521(7553): 436-444
  15. 15.
    Li C Z, Xu C Y, Cui Z, Wang D, Zhang T and Yang J. 2019. Feature—attentioned object detection in remote sensing imagery//2019 IEEE International Conference on Image Processing (ICIP). Taipei, China: IEEE: 3886-3890
  16. 16.
    Li D Z, Fan D Z and Su Y L. 2018. Building Detection in remote sensing image based on faster R-CNN. Journal of Geomatics Science and Technology, 35(4): 389-394
  17. 17.
    Li X Y, Long S P and Zhu J. 2020. Survey of few-shot learning based on deep neural network. Application Research of Computers, 37(8): 2241-2247.
  18. 18.
    Li Y Y, Huang Q, Pei X, Jiao L C and Shang R H. 2020. RADet: refine feature pyramid network and multi-layer attention network for arbitrary-oriented object detection of remote sensing images. Remote Sensing, 12(3): 389
  19. 19.
    Li Z M, Peng C, Yu G, Zhang X Y, Deng Y D and Sun J. 2017. Light-head R-CNN: in defense of two-stage object detector. arXiv Preprint arXiv: 1711.07264
  20. 20.
    Liao M H, Shi B G and Bai X. 2018a. Textboxes++: a single-shot oriented scene text detector. IEEE Transactions on Image Processing, 27(8): 3676-3690
  21. 21.
    Liao M H, Shi B G, Bai X, Wang X G and Liu W Y. 2017. TextBoxes: a fast text detector with a single deep neural network//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. San Francisco: ACM: 4161-4167
  22. 22.
    Liao M H, Zhu Z, Shi B G, Xia G S and Bai X. 2018b. Rotation-sensitive regression for oriented scene text detection//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 5909-5918
  23. 23.
    Lin T Y, Dollár P, Girshick R, He K M, Hariharan B and Belongie S. 2017a. Feature pyramid networks for object detection//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 936-944
  24. 24.
    Lin T Y, Goyal P, Girshick R, He K M and Dollár P. 2017b. Focal loss for dense object detection//Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE: 2999-3007
  25. 25.
    Lin Y T, Feng P M, Guan J, Wang W W and Chambers J. 2019. IENet: interacting embranchment one stage anchor free detector for orientation aerial object detection. arXiv Preprint arXiv: 1912.00969
  26. 26.
    Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y and Berg A C. 2016. SSD: single shot MultiBox detector//14th European Conference on Computer Vision. Amsterdam: Springer: 21-37
  27. 27.
    Liu Z K, Yuan L, Weng L B and Yang Y P. 2017. A high resolution optical satellite image dataset for ship recognition and some new baselines//Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods. Porto: ICPRAM: 324-331
  28. 28.
    Ma J Q, Shao W Y, Ye H, Wang L, Wang H, Zheng Y B and Xue X Y. 2018. Arbitrary-oriented scene text detection via rotation proposals. IEEE Transactions on Multimedia, 20(11): 3111-3122
  29. 29.
    Qian W, Yang X, Peng S L, Yan J C and Guo Y. 2021. Learning modulated loss for rotated object detection//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI: 2458-2466.
  30. 30.
    Qiu H Q, Li H L, Wu Q B, Meng F M, Ngan K N and Shi H C. 2019. A2RMNet: adaptively aspect ratio multi-scale network for object detection in remote sensing images. Remote Sensing, 11(13): 1594
  31. 31.
    Redmon J, Divvala S, Girshick R and Farhadi A. 2016. You only look once: unified, real-time object detection//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 779-788
  32. 32.
    Redmon J and Farhadi A. 2017. YOLO9000: better, faster, stronger//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 6517-6525
  33. 33.
    Redmon J and Farhadi A. 2018. YOLOv3: an incremental improvement. arXiv Preprint arXiv: 1804.02767
  34. 34.
    Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137-1149
  35. 35.
    Shi B, Bai X and Belongie S. 2017. Detecting oriented text in natural images by linking segments//Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. IEEE: 2550-2558
  36. 36.
    Sun P, Chen G, Luke G and Shang Y. 2018. Salience biased loss for object detection in aerial images. arXiv Preprint arXiv: 1810.08103
  37. 37.
    Van De Sande K E A, Uijlings J R R, Gevers T and Smeulders A W M. 2011. Segmentation as selective search for object recognition//2011 International Conference on Computer Vision. Barcelona: IEEE: 1879-1886
  38. 38.
    Wang C A. 2019. Detection and Fine-Grained Recognition of Inshore Ships on Optical Remote Sensing Images. Wuhan: Huazhong University of Science and Technology: 1-2
  39. 39.
    Wang J W, Ding J, Guo H W, Cheng W S, Pan T and Yang W. 2019. Mask OBB: a semantic attention-based mask oriented bounding box representation for multi-category object detection in aerial images. Remote Sensing, 11(24): 2930
  40. 40.
    Wang P J, Sun X, Diao W H and Fu K. 2020. FMSSD: feature-merged single-shot detection for multiscale objects in large-scale remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 58(5): 3377-3390
  41. 41.
    Wang Y S, Zhang Y, Zhang Y, Zhao L J, Sun X and Guo Z. 2019. SARD: towards scale-aware rotated object detection in aerial imagery. IEEE Access, 7: 173855-173865
  42. 42.
    Wei H R, Zhang Y, Chang Z H, Li H, Wang H Q and Sun X. 2020. Oriented objects as pairs of middle lines. ISPRS Journal of Photogrammetry and Remote Sensing, 169: 268-279
  43. 43.
    Wen J W. 2018. Research on Object Detection with Deep Learning. Guangzhou: Guangdong University of Technology: 1-2
  44. 44.
    Xia G S, Bai X, Ding J, Zhu Z, Belongie S, Luo J B, Datcu M, Pelillo M and Zhang L P. 2018. DOTA: a large-scale dataset for object detection in aerial images//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 3974-3983
  45. 45.
    Xiao Z F, Qian L J, Shao W P, Tan X W and Wang K. 2020. Axis learning for orientated objects detection in aerial images. Remote Sensing, 12(6): 908
  46. 46.
    Xu Y C, Fu M T, Wang Q M, Wang Y K, Chen K, Xia G S and Bai X. 2021. Gliding vertex on the horizontal bounding box for multi-oriented object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(4): 1452-1459.
  47. 47.
    Xu Y, Wang Y, Zhou W, Wang Y, Yang Z, Bai X. 2019. Textfield: Learning a deep direction field for irregular scene text detection. IEEE Transactions on Image Processing, 28(11): 5566-5579
  48. 48.
    Yao Y Q, Cheng G, Xie X X and Han J W. 2021. Optical remote sensing image object detection based on multi-resolution feature fusion. National Remote Sensing Bulletin, 25(5): 1124-1137
  49. 49.
    Yang X, Fu K, Sun H, Yang J R, Guo Z, Yan M L, Zhang T F and Xian S. 2018. R2CNN++: multi-dimensional attention based rotation invariant detector with robust anchor strategy. arXiv Preprint arXiv: 1811.07126
  50. 50.
    Yang X, Yan J C, Feng Z M and He T. 2021. R3Det: refined single-stage detector with feature refinement for rotating object//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI: 3163-3171
  51. 51.
    Yang X, Yang J R, Yan J C, Zhang Y, Zhang T F, Guo Z, Sun X and Fu K. 2019. SCRDet: towards more robust detection for small, cluttered and rotated objects//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South): IEEE: 8231-8240
  52. 52.
    Yu Y,Ai H, He X J, Yu S H, Zhong X, Zhu R F. 2020. Attention-based feature pyramid networks for ship detection of optical remote sensing image. National Remote Sensing Bulletin. 24(2): 107-115
  53. 53.
    Yi J R, Wu P X, Liu B, Huang Q Y, Qu H and Metaxas D. 2021. Oriented object detection in aerial images with box boundary-aware vectors//2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa: IEEE: 2149-2158
  54. 54.
    Zhang F. 2019. Research and Application of FMCG Detection Based on Slant Object Detection Technology. Changsha: Hunan University: 1-2
  55. 55.
    Zhang G J, Lu S J and Zhang W. 2019. CAD-Net: a context-aware detection network for objects in remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 57(12): 10015-10024
  56. 56.
    Zhang L, Zhang Y S, Yu Y, Geng Y L and Wang H. 2019. Research on data augmentation for object detection of remote sensing image. Journal of Geomatics Science and Technology, 36(5): 505-510
  57. 57.
    Zhang X H, Yao L B, Lv Y F, Han P and Li J W. 2020. Center based model for arbitrary-oriented ship detection in remote sensing images. Acta Photonica Sinica, 49(4): 210-218
  58. 58.
    Zhang Z H, Guo W W, Zhu S N and Yu W X. 2018. Toward arbitrary-oriented ship detection with rotated region proposal and discrimination networks. IEEE Geoscience and Remote Sensing Letters, 15(11): 1745-1749
  59. 59.
    Zhao Y Q, Rao Y, Dong S P and Zhang J Y. 2020. Survey on deep learning object detection. Journal of Image and Graphics, 25(4): 629-654
  60. 60.
    Zhou B Y and Yang P. 2020. Natural text detector for natural scene based on neural network. Computer and Digital Engineering, 48(1): 163-166
  61. 61.
    Zhou L, Wei H R, Li H, Zhang Y, Sun X and Zhao W Z. 2020. Objects detection for remote sensing images based on polar coordinates. arXiv Preprint arXiv: 2001.02988
  62. 62.
    Zhou X Y, Yao C, Wen H, Wang Y Z, Zhou S C, He W R and Liang J J. 2017. EAST: an efficient and accurate scene text detector//Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 2642-2651
  63. 63.
    Zhou X Y, Zhuo J C and Krähenbühl P. 2019. Bottom-up object detection by grouping extreme and center points//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 850-859
  64. 64.
    Zhu Y and Du J. 2021. Textmountain: Accurate scene text detection via instance segmentation. Pattern Recognition, 110: 107336
  65. 65.
    Zhu Y, Fang G S, Zheng B B and Han F. 2020. Research on detection method of refined rotated boxes in remote sensing. Acta Automatica Sinica, 1-11[2020-10-10].
  66. 66.
    Zhu Y X, Du J and Wu X Q. 2020. Adaptive period embedding for representing oriented objects in aerial images. IEEE Transactions on Geoscience and Remote Sensing, 58(10): 7247-7257 .
  67. 67.
    Zitnick C L and Dollár P. 2014. Edge boxes: locating object proposals from edges//13th European Conference on Computer Vision. Zurich: Springer: 391-405

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website