Anterior|Siguiente

Aircraft type recognition method by integrating target segmentation and key points detection

  • role： First author第一作者
  • Affiliation：

    Chinese Academy of Surveying and Mapping, Beijing 100830, China

    Faculty of Geomatics, Lanzhou Jiaotong University, Lanzhou 730070, China

  • Email：0219760@stu.lzjtu.edu.cn
  • Introduction：刘思婷， 研究方向为遥感影像智能解译。E-mail：0219760@stu.lzjtu.edu.cn
LIU Siting12,  
  • role： Corresponding author通信作者
  • Affiliation：

    Chinese Academy of Surveying and Mapping, Beijing 100830, China

  • Email：wangqd@casm.ac.cn
  • Introduction：王庆栋， 研究方向为基于点云深度学习的室内外语义三维重建。E-mail：wangqd@casm.ac.cn
WANG Qingdong1*,  
  • Affiliation：

    Chinese Academy of Surveying and Mapping, Beijing 100830, China

ZHANG Li1,  
  • Affiliation：

    Chinese Academy of Surveying and Mapping, Beijing 100830, China

HAN Xiaoxia1,  
  • Affiliation：

    Chinese Academy of Surveying and Mapping, Beijing 100830, China

WANG Baoqian1,  
  • Affiliation：

    Shenzhen Investigation & Research Institute Co., Ltd, Shenzhen 518026, China

LIU Yuxian3

resumen

Aircraft detection via deep learning is a popular field in remote sensing image analysis. However, given the limited perspectives of satellite imagery and high similarities in image appearance, aircraft type recognition remains a challenging task. The existing deep learning methods cannot be satisfactorily applied to fine-grained aircraft type recognition tasks, which require refined labels for datasets. With the aim of effectively recognizing aircraft types in remote sensing images, we propose an integrated target segmentation and key point detection method for aircraft type recognition.The proposed method combines an organic multitask deep neural network with a conditional random field and template matching algorithm to achieve high-precision recognition of aircraft types by pretraining, fine-tuning, and postprocessing. First, we performed target aircraft position and mask and keypoint recognition by deploying multitask learning and transfer learning technology. Second, to facilitate high-precision template matching in the later stage, we utilized an aircraft target mask refinement algorithm and a keypoint-based mask attitude adjustment algorithm to achieve boundary refinement of the recognition target and aircraft target mask attitude adjustment. Finally, on the basis of the aircraft type template library constructed in this study, we matched the refined aircraft mask information with the template library to identify the aircraft type.The proposed algorithm was applied to the MTARSI dataset and remote sensing images for verification. The results showed that the recognition accuracy of the 11 types of images was 89%. Aircraft with simple structures and unique shapes, such as B-2 and B-1, exhibited high recognition accuracy, whereas aircraft with complex structures and high similarity with other shapes, such as E-3 reconnaissance aircraft, exhibited low recognition accuracy. Subsequently, the algorithm was compared with traditional algorithms and end-to-end deep learning methods. Eleven types of aircraft were studied. The results showed that the accuracy of our method was 15.4% and 20.7% better than those of the other two methods.The use of target segmentation and keypoint information has achieved good results in model recognition on high-resolution remote sensing images. However, limitations remain in terms of the breadth of identifiable aircraft types; therefore, further research is needed to address this research gap.

palabra clave

object detection;segmentation;key points detection;conditional random field;aircraft type recognition

References

  1. 1.
    Ai J Q, Tian R T, Luo Q W, Jin J and Tang B. 2019. Multi-scale rotation-invariant haar-like feature integrated CNN-based ship detection algorithm of multiple-target environment in SAR imagery. IEEE Transactions on Geoscience and Remote Sensing, 57(12): 10070-10087
  2. 2.
    Cao Z, Simon T, Wei S E and Sheikh Y. 2017. Realtime multi-person 2D pose estimation using part affinity fields//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 1302-1310
  3. 3.
    Chen S Z, Wang H P, Xu F and Jin Y Q. 2016. Target classification using the deep convolutional networks for SAR images. IEEE Transactions on Geoscience and Remote Sensing, 54(8): 4806-4817
  4. 4.
    Chen Y L, Wang Z C, Peng Y X, Zhang Z Q, Yu G and Sun J. 2018. Cascaded pyramid network for multi-person pose estimation//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 7103-7112
  5. 5.
    Diao W H, Sun X, Dou F Z, Yan M L, Wang H Q and Fu K. 2015. Object recognition in remote sensing images using sparse deep belief networks. Remote Sensing Letters, 6(10): 745-754
  6. 6.
    Fang H S, Xie S Q, Tai Y W and Lu C W. 2017. RMPE: regional multi-person pose estimation//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE: 2353-2362
  7. 7.
    Fu K, Dai W, Zhang Y, Wang Z R, Yan M L and Sun X. 2019. MultiCAM: multiple class activation mapping for aircraft recognition in remote sensing images. Remote Sensing, 11(5): 544
  8. 8.
    Guo Q, Wang H P and Xu F. 2021. Scattering enhanced attention pyramid network for aircraft detection in SAR images. IEEE Transactions on Geoscience and Remote Sensing, 59(9): 7570-7587
  9. 9.
    He K M, Gkioxari G, Dollár P and Girshick R. 2020. Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2): 386-397
  10. 10.
    He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 770-778
  11. 11.
    Lafferty J D, McCallum A and Pereira F C N. 2001. Conditional random fields: probabilistic models for segmenting and labeling sequence data//Proceedings of the Eighteenth International Conference on Machine Learning. Williamstown: Morgan Kaufmann Publishers Inc.: 282-289
  12. 12.
    Li H. 2019. Statistical Learning Methods. 2nd ed. Beijing: Tsinghua University Press: 5
  13. 13.
    Li K, Wang R S and Wang C. 2006. A method of tree classifier for the recognition of airplane types. Computer Engineering and Science, 28(11): 136-139
  14. 14.
    Long Y, Gong Y P, Xiao Z F and Liu Q. 2017. Accurate object localization in remote sensing images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 55(5): 2486-2498
  15. 15.
    Mahi H, Isabaten H and Serief C. 2014. Zernike moments and SVM for shape classification in very high resolution satellite images. The International Arab Journal of Information Technology, 11(1): 43-51
  16. 16.
    Papandreou G, Zhu T, Kanazawa N, Toshev A, Tompson J, Bregler C and Murphy K. 2017. Towards accurate multi-person pose estimation in the wild//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 3711-3719
  17. 17.
    Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137-1149
  18. 18.
    Sun X, Wang P J, Yan Z Y, Xu F, Wang R P, Diao W H, Chen J, Li J H, Feng Y C, Xu T, Weinmann M, Hinz S, Wang C and Fu K. 2021. FAIR1M: a benchmark dataset for fine-grained object recognition in high-resolution remote sensing imagery. arXiv: 2103. 05569
  19. 19.
    Sun Y Y. 2018. Multi-scaled FCN-CRF and Reinforcement Learning for High Resolution SAR Image Semantic Segmentation. Xi’an: Xidian University
  20. 20.
    Wahi A, Palamsamy C and Sundaramurthy S. 2012. Rotated object recognition-based on hu moment invariants using artificial neural system//Proceedings of 2012 World Congress on Information and Communication Technologies. Trivandrum: IEEE: 45-49
  21. 21.
    Wang B, Zhou Y, Zhang H N and Zhao K. 2021. Aircraft detection method based on SSD framework for remote sensing images. Fire Control and Command Control, 46(1): 14-19
  22. 22.
    Wang W, Li Y L, Liang F and Zhang L F. 2013. A ground aircraft target recognition method based on region segmentation. Computer Engineering, 39(6): 214-218
  23. 23.
    Wu Q C, Sun H, Sun X, Zhang D B, Fu K and Wang H Q. 2015. Aircraft recognition in high-resolution optical satellite remote sensing images. IEEE Geoscience and Remote Sensing Letters, 12(1): 112-116
  24. 24.
    Wu Z Z, Wan S H, Wang X F, Tan M, Zou L, Li X L and Chen Y. 2020. A benchmark data set for aircraft type recognition from remote sensing images. Applied Soft Computing, 89: 106132
  25. 25.
    Xia F T, Wang P, Chen X J and Yuille A L. 2017. Joint multi-person pose estimation and semantic part segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 6080-6089
  26. 26.
    Yan L. 2018. Airplane Target Recognition in Remote Sensing Images Based on Convolutional Neural Networks. Hefei: University of Science and Technology of China
  27. 27.
    Yang Y M, Liao Y R, Cheng L F, Zhang K, Wang H N and Chen S M. 2021. Remote sensing image aircraft target detection based on GIoU-YOLO V3//Proceedings of the 2021 6th International Conference on Intelligent Computing and Signal Processing. Xi’an: IEEE: 474-478
  28. 28.
    Yao Q L, Hu X and Lei H. 2019. Aircraft detection in remote sensing imagery with multi-scale feature fusion convolutional neural networks. Acta Geodaetica et Cartographica Sinica, 48(10): 1266-1274
  29. 29.
    Zhang K, Liu H, Yang X, Li S Y and Wang X T. 2020. Identification algorithm based on key-point detection network for vital parts of infrared aerial target. Journal of Northwestern Polytechnical University, 38(6): 1154-1162
  30. 30.
    Zhang M C, Wu X Q and Wang P W. 2006. Aircraft recognition via extraction of closed contour and partial feature matching. Computer Simulation, 23(11): 193-197
  31. 31.
    Zhang R Q, Newsam S, Shao Z F, Huang X, Wang J M and Li D R. 2021. Multi-scale adversarial network for vehicle detection in UAV imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 180: 283-295
  32. 32.
    Zhang W C, Sun X, Fu K, Wang C Y and Wang H Q. 2014. Object detection in high-resolution remote sensing images using rotation invariant parts based model. IEEE Geoscience and Remote Sensing Letters, 11(1): 74-78
  33. 33.
    Zhang Y H, Sun H, Zuo J W, Wang H Q, Xu G L and Sun X. 2018. Aircraft type recognition in remote sensing images based on feature learning with conditional generative adversarial networks. Remote Sensing, 10(7): 1123
  34. 34.
    Zhu D, Wang B and Zhang L M. 2015. Airport detection based on near parallelity of line segments and GBVS saliency. Journal of Infrared and Millimeter Waves, 34(3): 375-384
  35. 35.
    Zuo J W, Xu G L, Fu K, Sun X and Sun H. 2018. Aircraft type recognition based on segmentation with deep convolutional neural networks. IEEE Geoscience and Remote Sensing Letters, 15(2): 282-286

Leer el texto completo

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website