سابِق|تال

Hybrid-Gird： An explainable method for fine-grained classification of remote sensing images

  • role： First author第一作者
  • Affiliation：

    School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China

  • Email：zhukaiwen@bupt.edu.cn
  • Introduction：朱凯雯，研究方向为遥感图像智能解译、可解释人工智能。E-mail： zhukaiwen@bupt.edu.cn
ZHU Kaiwen1,  
  • role： Corresponding author通信作者
  • Affiliation：

    School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China

  • Email：youyanan@bupt.edu.cn
  • Introduction：尤亚楠，研究方向为成像探测与智能感知，人工智能遥感图像理解。E-mail：youyanan@bupt.edu.cn
YOU Yanan1*,  
  • Affiliation：

    School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China

CAO Jingyi1,  
  • Affiliation：

    Beijing Institute of Remote Sensing Information, Beijing 100192, China

MENG Gang2,  
  • Affiliation：

    School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China

QIAO Yuanyuan1,  
  • Affiliation：

    School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China

YANG Jie1

ملخص

At present, the deep neural network for targets fine-grained classification based on remote sensing images has been widely applied in military and civilian fields. The “black box” problem of deep learning network models still makes it difficult for people to understand the decision-making basis of the network in fine-grained classification tasks. This not only limits the possibility of deep neural networks optimizing and improving through feedback guidance, but also makes them unable to be fully trusted by humans and applied in important fields such as military and medical fields. Thus, how to carry out explainability research on its internal decision-making mechanism is the key problem for the current fine-grained algorithm to further improve the credibility of decision-making basis. This article summarized the commonly used explainable methods in existing fine-grained classification networks for remote sensing image targets. On this basis, a mathematical model for the problem of target fine-grained classification is established. It also models the fine-grained classification task of remote sensing image targets based on game competition theory, analyzes the applicability of explainable methods such as IG, SmoothGrad, and Grad CAM on the fine-grained classification network of remote sensing image targets, and proposes a scale adaptive method for analyzing the explainability of essential features of fine-grained classification of targets, that is, Hybrid Grid. The fusion algorithm of pixel level and local feature relationships is used to improve the accurate description ability of essential features of targets that support network decision-making. The typical explaining methods involved in this article were evaluated using indicators such as Average Drop, Coherence, Complexity, ADCC, and Deletion and precision loss, confirming the accuracy of the proposed method. The experimental results show that the Hybrid-Grid proposed in this paper achieves a score of 78.87 on the quantitative evaluation index of ADCC for target fine-grained classification networks, which is significantly improved compared to Score-CAM. Compared with the explanation results of SmoothGrad and Grad-CAM, our method performed the best in deletion and accuracy loss experiments, resulting in a loss of 16.92%, 1.61%, and 17.21% in the Top-1 accuracy, Top-5 accuracy, and F1 score of EFM Net, respectively. This proves that the Hybrid Grid accurately explains the essential feature of the target that contributes the most to fine-grained classification network decision-making. Based on several typical fine-grained classification networks, the feasibility and universality of the explaining method proposed in this paper have been confirmed. The explaining method proposed in this article can more accurately reveal the decision feature basis of the current target fine-grained classification network. Future work will further enhance the visualization and interpretation accuracy of the essential features of the target, assisting in improving the classification performance and decision-making credibility of fine-grained classification networks.

مفهوم

remote sensing image;explainability analysis method;target fine-grained classification network;explainable artificial intelligence;cooperative game theory;essential fearture

References

  1. 1.
    Bach S, Binder A, Montavon G, Klauschen F, Müller K R and Samek W. 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS One, 10(7): e0130140
  2. 2.
    Cheng X, Chu C, Zheng Y, Ren J and Zhang Q S. 2021. A game-theoretic taxonomy of visual concepts in DNNs. arXiv preprint arXiv: 2106.10938
  3. 3.
    Deng H, Zou N, Du M, Chen W, Feng G, Yang Z, Li Z and Zhang Q. 2023. Understanding and unifying fourteen attribution methods with taylor interactions. arXiv preprint arXiv:2303.01506.
  4. 4.
    Du R, Xie J, Ma Z, Chang D, Song Y Z and Guo J. 2021. Progressive learning of category-consistent multi-granularity features for fine-grained visual classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12): 9521-9535.
  5. 5.
    Fu K, Dai W, Zhang Y, Wang Z R, Yan M L and Sun X. 2019. MultiCAM: multiple class activation mapping for aircraft recognition in remote sensing images. Remote Sensing, 11(5): 544
  6. 6.
    Gong J Y, Huan L X and Zheng X W. 2022. Deep learning interpretability analysis methods in image interpretation. Acta Geodaetica et Cartographica Sinica, 51(6): 873-884
  7. 7.
    Han Y Q, Yang X Y, Pu T and Peng Z M. 2022. Fine-grained recognition for oriented ship against complex scenes in optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 60: 5612318
  8. 8.
    He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of the IEEE conference on computer vision and pattern recognition: 770-778
  9. 9.
    Liang W, Li J H, Diao W H, Sun X, Fu K and Wu Y R. 2020. FGATR-Net: automatic network architecture design for fine-grained aircraft type recognition in remote sensing images. Remote Sensing, 12(24): 4187
  10. 10.
    Lundberg S M and Lee S I. 2017. A unified approach to interpreting model predictions//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: Curran Associates, Inc.: 4768-4777
  11. 11.
    Nie Y, Bian C J and Li L G. 2022. Adap-EMD: adaptive EMD for aircraft fine-grained classification in remote sensing. IEEE Geoscience and Remote Sensing Letters, 19: 6509005
  12. 12.
    Petsiuk V, Das A and Saenko K. 2018. RISE: randomized input sampling for explanation of black-box models//Proceedings of the British Machine Vision Conference 2018. Newcastle: BMVA Press: 151
  13. 13.
    Poppi S, Cornia M, Baraldi L and Cucchiara R. 2021. Revisiting the evaluation of class activation mapping for explainability: A novel metric and experimental analysis//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition: 2299-2304
  14. 14.
    Ribeiro M T, Singh S and Guestrin C. 2016. Why should I trust you?": explaining the predictions of any classifier//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco: ACM: 1135-1144
  15. 15.
    Chen S, Wang H, Xu F and Jin Y Q. 2016. Target classification using the deep convolutional networks for sar images. IEEE transactions on geoscience and remote sensing, 54(8): 4806-4817
  16. 16.
    Selvaraju R R, Cogswell M, Das A, Vedantam R, Parikh D and Batra D. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization//Proceedings of the IEEE international conference on computer vision: 618-626
  17. 17.
    Selvaraju R R, Das A, Vedantam R, Cogswell M, Parikh D and Batra D. 2016. Grad-CAM: Why did you say that?. arXiv preprint arXiv: 1611.07450
  18. 18.
    Simonyan K, Vedaldi A and Zisserman A. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv: 1312.6034
  19. 19.
    Smilkov D, Thorat N, Kim B, Viégas F and Wattenberg M. 2017. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv: 1706.03825
  20. 20.
    Springenberg J T, Dosovitskiy A, Brox T and Riedmiller M. 2014. Striving for simplicity: The all convolutional net. arXiv preprint arXiv: 1412.6806
  21. 21.
    Springenberg J T, Dosovitskiy A, Brox T and Riedmiller M. Striving for simplicity: The all convolutional net. arXiv preprint arXiv: 1412.6806, 2014.
  22. 22.
    Sundararajan M, Taly A, Yan Q. Axiomatic attribution for deep networks//International conference on machine learning. PMLR, 2017: 3319-3328.
  23. 23.
    Wang H F, Wang Z F, Du M N, Yang F, Zhang Z J, Ding S R, Mardziel P and Hu X.2020. Score-CAM: Score-weighted visual explanations for convolutional neural networks//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 2020: 24-25 [D0I:arXiv:1910.01279v2]
  24. 24.
    Xiong W, Xiong Z Y and Cui Y Q. 2022. An explainable attention network for fine-grained ship classification using remote-sensing images. IEEE Transactions on Geoscience and Remote Sensing, 60: 5620314
  25. 25.
    Yang P B, Sang J T, Zhang B, Feng Y G and Yu J. 2023. Survey on interpretability of deep models for image classification. Journal of Software, 34(1): 230-254
  26. 26.
    Yi Y H, You Y N, Li C and Zhou W L. 2023. EFM-net: An essential feature mining network for target fine-grained classification in optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing.
  27. 27.
    Zeiler M D and Fergus R. 2014. Visualizing and understanding convolutional networks//Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13. Springer International Publishing, 2014: 818-833

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website