سابِق|تال

Joint dual attention mechanism and bidirectional feature pyramid for remote sensing small targets detection

  • role： First author第一作者
  • Affiliation：

    National Energy Group Jianbi Power Plant, Zhenjiang 212006, China

  • Email：12030141@ceic.com
  • Introduction：李科文，研究方向为基于无人机遥感的新能源科技创新。E-mail： 12030141@ceic.com
LI Kewen1,  
  • Affiliation：

    Yellow River Engineering Consulting Co., Ltd., Zhengzhou 450003, China

ZHU Guanglei2,  
  • Affiliation：

    National Energy Group Jianbi Power Plant, Zhenjiang 212006, China

WANG Hui1,  
  • Affiliation：

    National Energy Group Jianbi Power Plant, Zhenjiang 212006, China

ZHU Rui1,  
  • Affiliation：

    School of Earth Sciences and Engineering, Hohai University, Nanjing 211100, China

    Jiangsu Province Engineering Research Center of Water Resources and Environment Assessment Using Remote Sensing, Hohai University, Nanjing 211100, China

DI Xiyao34,  
  • Affiliation：

    School of Earth Sciences and Engineering, Hohai University, Nanjing 211100, China

    Jiangsu Province Engineering Research Center of Water Resources and Environment Assessment Using Remote Sensing, Hohai University, Nanjing 211100, China

ZHANG Tianjian34,  
  • role： Corresponding author通信作者
  • Affiliation：

    School of Earth Sciences and Engineering, Hohai University, Nanjing 211100, China

    Jiangsu Province Engineering Research Center of Water Resources and Environment Assessment Using Remote Sensing, Hohai University, Nanjing 211100, China

  • Email：zhaohui.xue@hhu.edu.cn
  • Introduction：薛朝辉，研究方向为深度学习等先进机器学习模型在高光谱遥感影像小样本分类、多源遥感数据迁移学习、地表参数遥感估算等方面的理论与应用研究。E-mail： zhaohui.xue@hhu.edu.cn
XUE Zhaohui34*

ملخص

Given imaging characteristics and limitations in spatial resolution, feature extraction for small targets is difficult, which increases the hardship of small-target detection. Existing deep learning target detection network architectures are mostly based on natural images and insufficient for the research on and exploration of small targets in remote sensing images. To overcome these issues, this study proposes a remote sensing image small-target detection algorithm that combines a dual attention mechanism and a bidirectional feature pyramid.This work provides innovative contributions. (1) To solve the problems of small-target occupation in remote sensing images and the usually huge parameter size, we introduce the LKG bottleneck and generalized intersection over union loss function into YOLOv3 and propose the LKGNet-YOLO network. (2) To resolve the noise disturbance issue and the drawback of feature fusion, we introduce the DA-LKGNet bottleneck and bidirectional feature pyramid network into LKGNet-YOLO and propose the DA-LKGNet-YOLO network.The remote sensing image dataset (UA) released by the University of Chinese Academy of Sciences in 2014 and the AI-TOD dataset released by Wuhan University in 2021 is used for experiments to validate the effectiveness of the proposed method in remote sensing image small-target detection. Experimental results demonstrate that the proposed method achieves a mean average precision (mAP) of 96.21% at a threshold of 0.5 on the UA dataset. On the AI-TOD dataset, the mAP0.5—0.95 is 9.51% at thresholds ranging from 0.5 to 0.95. Compared with YOLOv3, RFBNet, SSD, FSSD, RetinaNet, and RefineDet, the proposed method has an mAP accuracy that is 3.45%—7.52% and 1.36%—4.84% higher, indicating a considerable performance improvement. Meanwhile, its detection level on small-scale targets is better than that of Faster-RCNN and YOLOv7 algorithms. Compared with the original YOLOv3, our method reduces the number of floating-point operations per second by 48% and the number of parameters by 42%.A small-target detection method for remote sensing images is proposed in this study. Its contribution lies in the utilization of the YOLOv3 model with LKGNet as the backbone network, along with the design of a dual attention mechanism and a bidirectional feature pyramid network, resulting in a lightweight DA-LKGNet-YOLO model. This model can effectively extract small-sized objects in complex remote sensing images. The experimental results confirm the effectiveness of the proposed method.

مفهوم

remote sensing image;Small Target Detection;deep learning;YOLOv3;Attention Mechanism;feature pyramid

References

  1. 1.
    Dai W C, Jin L X, Li G N and Zheng Z Q. 2018. Real-time airplane detection algorithm in remote-sensing images based on improved YOLOv3. Opto-Electronic Engineering, 45(12): 180350
  2. 2.
    Deng L, Yang M, Li T, He Y and Wang C. RFBNet: Deep multimodal Networks with residual fusion blocks for RGB-D semantic segmentation, 2019, p.arXiv: 1907. 00135
  3. 3.
    Dong R C, Xu D Z, Zhao J, Jiao L C and An J G. 2019. Sig-NMS-based faster R-CNN combining transfer learning for small target detection in VHR optical remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 57(11): 8534-8545
  4. 4.
    Fu J, Liu J, Tian H J, Li Y, Bao Y J, Fang Z W and Lu H Q. 2019. Dual attention network for scene segmentation//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 3141-3149
  5. 5.
    Girshick R. 2015. Fast R-CNN//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 1440-1448
  6. 6.
    Girshick R, Donahue J, Darrell T and Malik J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 580-587
  7. 7.
    Han K, Wang Y H, Tian Q, Guo J Y, Xu C J and Xu C. 2020. GhostNet: more features from cheap operations//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE: 1577-1586
  8. 8.
    He K, Zhang X, Ren S and Sun J. Spatial pyramid pooling in deep convolutional networks for visual rewgnition, 2014, p. arXiv: 1406.4729
  9. 9.
    Li K, Wan G, Cheng G, Meng L Q and Han J W. 2020. Object detection in optical remote sensing images: a survey and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing, 159: 296-307
  10. 10.
    Li M J, Guo W W, Zhang Z H, Yu W X and Zhang T. 2018. Rotated region based fully convolutional network for ship detection//Proceedings of 2018 IEEE International Geoscience and Remote Sensing Symposium. Valencia: IEEE: 673-676
  11. 11.
    Li Z X, Yang L and Zhou F Q. 2017. FSSD: feature fusion single shot multibox detector[EB/OL].
  12. 12.
    Lin T Y, Dollár P, Girshick R, He K M, Hariharan B and Belongie S. 2017. Feature pyramid networks for object detection//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 936-944
  13. 13.
    Lin T Y, Goyal P, Girshick R, He K M and Dollár P. 2020. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2): 318-327
  14. 14.
    Liu S, Qi L, Qin H F, Shi J P and Jia J Y. 2018a. Path aggregation network for instance segmentation//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 8759-8768
  15. 15.
    Liu S T, Huang D and Wang Y H. 2018b. Receptive field block net for accurate and fast object detection//Proceedings of the 15th European Conference. Munich: Springer: 404-419
  16. 16.
    Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y and Berg A C. 2016. SSD: single shot MultiBox detector//Proceedings of the 14th European Conference. Amsterdam: Springer: 21-37
  17. 17.
    Nie G T and Huang H. 2021. A survey of object detection in optical remote sensing images. Acta Automatica Sinica, 47(8): 1749-1768
  18. 18.
    Qin D D, Wan L, He P E, Zhang Y, Guo Y and Chen J. 2022. Multiscale object detection in remote sensing image by combining data fusion and feature selection. National Remote Sensing Bulletin, 26(8): 1662-1673
  19. 19.
    Redmon J, Divvala S, Girshick R and Farhadi A. 2016. You only look once: unified, real-time object detection//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 779-788
  20. 20.
    Redmon J and Farhadi A. 2017. YOLO9000: better, faster, stronger//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 6517-6525
  21. 21.
    Redmon J and Farhadi A. 2018. YOLOv3: an incremental improvement//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. [s.l.]: IEEE: 89-95
  22. 22.
    Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137-1149
  23. 23.
    Sha M M, Li Y and Li A. 2022. Multiscale aircraft detection in optical remote sensing imagery based on advanced Faster R-CNN. National Remote Sensing Bulletin, 26(8): 1624-1635
  24. 24.
    Tan M X, Pang R M and Le Q V. 2020. EfficientDet: scalable and efficient object detection//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE: 10778-10787
  25. 25.
    Wang C Y, Bochkovskiy A and Liao H Y M. 2023. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors//Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE: 7464-7475
  26. 26.
    Wang C Y, Liao H Y M, Wu Y H, Chen P Y, Hsieh J W and Yeh I H. 2020. CSPNet: a new backbone that can enhance learning capability of CNN//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE: 1571-1580
  27. 27.
    Wang J W, Yang W, Guo H W, Zhang R X and Xia G S. 2021a. Tiny object detection in aerial images//Proceedings of the 25th International Conference on Pattern Recognition. Milan: IEEE: 3791-3798
  28. 28.
    Wang N, Li B, Wei X X, Wang Y H and Yan H Q. 2021b. Ship detection in spaceborne infrared image based on lightweight CNN and multisource feature cascade decision. IEEE Transactions on Geoscience and Remote Sensing, 59(5): 4324-4339
  29. 29.
    Zhang S F, Wen L Y, Bian X, Lei Z and Li S Z. 2018a. Single-shot refinement neural network for object detection//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 4203-4212
  30. 30.
    Zhang W, Wang S H, Thachan S, Chen J Z and Qian Y T. 2018b. Deconv R-CNN for small object detection on remote sensing images//Proceedings of 2018 IEEE International Geoscience and Remote Sensing Symposium. Valencia: IEEE: 2483-2486
  31. 31.
    Zhang Y, Wu X J, Zhao X L, Zhuo L and Zhang J. 2022. Scene constrained object detection method in high-resolution remote sensing images by relation-aware global attention. Journal of Electronics and Information Technology, 44(8): 2924-2931
  32. 32.
    Zhang Y, Zhu G Y, Shi T J, Zhang K and Yan J H. 2022. Small object detection in remote sensing images based on feature fusion and attention. Acta Optica Sinica, 42(24): 2415001
  33. 33.
    Zhou Q H and Wang Z. 2022. A remote sensing target detection algorithm based on multi-scale feature enhancement CNNs. Electronics Optics and Control, 29(11): 74-81
  34. 34.
    Zhu H G, Chen X G, Dai W Q, Fu K, Ye Q X and Jiao J B. 2015. Orientation robust object detection in aerial images using deep convolutional neural network//Proceedings of 2015 IEEE International Conference on Image Processing. Quebec City: IEEE: 3735-3739

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website