سابِق|تال

A hierarchical geometry-to-semantic fusion GNN framework for earth surface anomalies detection

  • role： First author第一作者
  • Affiliation：

    School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China

  • Email：gaozhinus@gmail.com
  • Introduction：高智，研究方向为计算机视觉、机器学习和遥感影像智能解译。E-mail： gaozhinus@gmail.com
GAO Zhi1,  
  • Affiliation：

    School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China

HU Aohan1,  
  • role： Corresponding author通信作者
  • Affiliation：

    School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China

  • Email：cbachen1997@whu.edu.cn
  • Introduction：陈泊安，研究方向为遥感影像智能处理、图神经网络。E-mail： cbachen1997@whu.edu.cn
CHEN Boan1*,  
  • Affiliation：

    Beijing Institute of Remote Sensing Information, Beijing 100192, China

LU Yao2,  
  • Affiliation：

    State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210093, China

GE Jiachen3

ملخص

The increasing occurrence of Earth Surface Anomalies (ESAs) highlights the importance of the timely and accurate detection of such events. Therefore, researchers have utilized satellite imagery for large-scale detection and developed advanced deep learning methods. However, the performance of the methods is hindered by inadequate labeled data and the complexity of semantic information in satellite imagery. In this paper, we aim to address the aforementioned problems and improve the performance of Earth surface anomaly detection. This paper reviews and summarizes developments in Earth’s surface anomaly detection and the problems that hinder the performance of existing methods. Then, we proposed a hierarchical geometry-to-semantic fusion Graph Neural Network (GNN) framework, which utilizes a single image for the detection of Earth’s surface anomalies and reduces the demand for data and time required for preprocessing and inference. Specifically, our method employs two branches for the extraction of geoentities and construction of graphs at different levels. Then, a hierarchical graph attention network was utilized to update node features and extract graph embeddings for each level. An attention-based feature fusion module then combined them to yield the graph-level feature vector of the input image, which was finally processed through a multilayer perceptron for ESA detection. Given that existing ESA datasets mainly focus on single-class detection or post-hoc analysis, which is insufficient for our research needs, we proposed ESAD, which is a composite dataset, to bridge the gap between large-scale multiclass datasets. Specifically, ESAD is composed of three publicly datasets: xBD, Multi3Net and Sichuan Landslide, and Debrisflow. The proposed method was effective and accurate for detecting ESA, outperforming many baseline methods and balancing between accuracy and efficiency. Thus, it is suitable for ESA detection, saving valuable time and resources for downstream tasks. In conclusion, we proposed a hierarchical geometry-to-semantic fusion GNN framework for ESA detection. It leverages GNN to learn high-order semantic information from satellite imagery. To address insufficiency in benchmark datasets, we created the ESAD dataset based on existing related datasets. Our method achieved a good balance between accuracy and efficiency and is suitable for ESA detection with high timeliness requirements. In future work, we will further explore more models and extend our method to on-orbit real-time ESA detection task.

مفهوم

remote sensing;Earth surface anomalies;hierarchical fusion;graph neural network;fast detection;attention mechanism;emergency response;satellite imagery

References

  1. 1.
    Achanta R, Shaji A, Smith K, Lucchi A, Fua P and Süsstrunk S. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11): 2274-2282
  2. 2.
    Bergmann P, Fauser M, Sattlegger D and Steger C. 2019. MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection //Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 9592-9600.
  3. 3.
    Feng Q L, Chen B A, Li G Q, Yao X C, Gao B B and Zhang L C. 2022. A review for sample datasets of remote sensing imagery. National Remote Sensing Bulletin, 26(4): 589-605
  4. 4.
    Gao Y, Shi J, Li J and Wang R Y. 2021. Remote sensing scene classification based on high-order graph convolutional network. European Journal of Remote Sensing, 54(S1): 141-155
  5. 5.
    Gueguen L and Hamid R. 2015. Large-scale damage detection using satellite imagery. Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 1321-1328.
  6. 6.
    Gupta R, Goodman B, Patel N, Hosfelt R, Sajeev S, Heim E, Doshi J, Lucas K, Choset H and Gaston M. 2019. Creating xBD: a dataset for assessing building damage from satellite imagery//Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition Workshops: 10-17
  7. 7.
    Ji M, Liu L F and Buchroithner M. 2018. Identifying collapsed buildings using post-earthquake satellite imagery and convolutional neural networks: a case study of the 2010 Haiti earthquake. Remote Sensing, 10(11): 1689
  8. 8.
    Jia S, Jiang S G, Zhang S Y, Xu M and Jia X P. 2024. Graph-in-graph convolutional network for hyperspectral image classification. IEEE Transactions on Neural Networks and Learning Systems, 35(1): 1157-1171
  9. 9.
    Jiang J J, Ma J Y and Liu X M. 2022. Multilayer spectral–spatial graphs for label noisy robust hyperspectral image classification. IEEE Transactions on Neural Networks and Learning Systems, 33(2): 839-852
  10. 10.
    Jiang X, Li G, Liu Y, Zhang X P and He Y. 2020. Change detection in heterogeneous optical and SAR remote sensing images via deep homogeneous feature fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 1551-1566
  11. 11.
    Kuzin D, Isupova O, Simmons B D and Reece S. 2021. Disaster mapping from satellites: damage detection with crowdsourced point labels. arXiv:2111.03693
  12. 12.
    Kyrkou C and Theocharides T. 2020. EmergencyNet: efficient aerial image classification for drone-based emergency monitoring using atrous convolutional feature fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 1687-1699
  13. 13.
    Lai J B, Kang X D, Lu X K and Li S T. 2022. A review of land observation satellite remote sensing application technology with new generation artificial intelligence. National Remote Sensing Bulletin, 26(8): 1530-1546
  14. 14.
    Lee J, Xu J Z, Sohn K, Lu W H, Berthelot D, Gur I, Khaitan P, Huang K W, Koupparis K and Kowatsch B. 2020. Assessing post-disaster damage from satellite imagery using semi-supervised learning techniques. arXiv:2011.14004
  15. 15.
    Li M, Zhang X W, Fan X N and Zhang Z. 2015. Fly vision-inspired anomaly detection algorithm for remote sensing data under clutter background. Journal of Remote Sensing (in Chinese), 19(5): 780-790
  16. 16.
    Li Z Y, Kuang G Y, Zou H X and Wu H. 2003. Research of anomaly detection approaches based on feature fusion in hyperspectral imagery. Journal of Remote Sensing (in Chinese), 7(4): 304-308
  17. 17.
    Liang J L, Deng Y F and Zeng D. 2020. A deep neural network combined CNN and GCN for remote sensing scene classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 4325-4338 [DOI: .]
  18. 18.
    Liu Q C, Xiao L, Yang J X and Wei Z H. 2021. CNN-enhanced graph convolutional network with pixel- and superpixel-level feature fusion for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(10): 8657-8671
  19. 19.
    Liu Q L, Deng M, Wang J Q, Peng S L, Mei X M and Zhao L. 2011. Spatio-temporal outliers detection within the space-time framework. Journal of Remote Sensing (in Chinese), 15(3): 457-474
  20. 20.
    Mateo-Garcia G, Oprea S, Smith L, Veitch-Michaelis J, Schumann G, Gal Y, Baydin A G and Backes D. 2020. Flood detection on low cost orbital hardware. arXiv:1910.03019
  21. 21.
    Niu C Y, Gao O Y, Lu W J, Liu W and Lai T. 2022. Reg-SA–UNet++: a lightweight landslide detection network based on single-temporal images captured postlandslide. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15: 9746-9759
  22. 22.
    Pati P, Jaume G, Foncubierta-Rodríguez A, Feroce F, Anniciello A M, Scognamiglio G, Brancati N, Fiche M, Dubruc E, Riccio D, Di Bonito M, De Pietro G, Botti G, Thiran J P, Frucci M, Goksel O and Gabrani M. 2022. Hierarchical graph representations in digital pathology. Medical Image Analysis, 75: 102264 [DOI: .]
  23. 23.
    Rudner T G J, Rußwurm M, Fil J, Pelich R, Bischke B, Kopačková V and Biliński P. 2019. Multi3Net: segmenting flooded buildings via fusion of multiresolution, multisensor, and multitemporal satellite imagery//Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Honolulu: AAAI Press: 702-709
  24. 24.
    Rui X, Cao Y, Yuan X, Kang Y and Song W G. 2021. DisasterGAN: generative adversarial networks for remote sensing disaster image generation. Remote Sensing, 13(21): 4284
  25. 25.
    Saha S, Shahzad M, Ebel P and Zhu X X. 2022. Supervised change detection using prechange optical-SAR and postchange SAR data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15: 8170-8178
  26. 26.
    Tilon S, Nex F, Kerle N and Vosselman G. 2020. Post-disaster building damage detection from earth observation imagery using unsupervised and transferable anomaly detecting generative adversarial networks. Remote Sensing, 12(24): 4193
  27. 27.
    Veličković P, Cucurull G, Casanova A, Romero A, Liò P and Bengio Y. 2018. Graph attention networks. arXiv:1710.10903
  28. 28.
    Wang Q. 2022. Research framework of remote sensing monitoring and real-time diagnosis of earth surface anomalies. Acta Geodaetica et Cartographica Sinica, 51(7): 1141-1152
  29. 29.
    Wang S X, Zhou Y, Wei C J, Wang W M, Zhang Z K, Zhao Q, Yan F and Wang LT. 2009. Remote sensing early warning, monitoring and evaluation of major natural disasters. Journal of Remote Sensing(in Chinese),13(S1): 364-372
  30. 30.
    Weber E and Kané H. 2020. Building disaster damage assessment in satellite imagery with multi-temporal fusion. arXiv:2004.05525
  31. 31.
    Weber E, Papadopoulos D P, Lapedriza A, Ofli F, Imran M and Torralba A. 2023. Incidents1M: a large-scale dataset of images with natural disasters, damage, and incidents. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4): 4768-4781
  32. 32.
    Wu Z H, Pan S R, Chen F W, Long G D, Zhang C Q and Yu P S. 2021. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1): 4-24
  33. 33.
    Xi B B, Li J J, Li Y S, Song R, Xiao Y C, Du Q and Chanussot J. 2022. Semisupervised cross-scale graph prototypical network for hyperspectral image classification. IEEE Transactions on Neural Networks and Learning Systems, 34(11): 9337-9351
  34. 34.
    Xie W Y, Zhong J P and Li Y S. 2024. A background memory model for hyperspectral anomaly detection. National Remote Sensing Bulletin, 28(3): 717-729
  35. 35.
    Xu J Z, Lu W H, Li Z B, Khaitan P and Zaytseva V. 2019. Building damage detection in satellite imagery using convolutional neural networks. arXiv:1910.06444
  36. 36.
    Yu B, She J, Liu G X, Ma D Y, Zhang R, Zhou Z W and Zhang B. 2022. Coal fire identification and state assessment by integrating multitemporal thermal infrared and InSAR remote sensing data: a case study of Midong District, Urumqi, China. ISPRS Journal of Photogrammetry and Remote Sensing, 190: 144-164
  37. 37.
    Zeng C, Cao Z Y, Su F H, Zeng Z and Yu C X. 2022. A dataset of high-precision aerial imagery and interpretation of landslide and debris flow disaster in Sichuan and surrounding areas between 2008 and 2020. China Scientific Data, 7(2): 195-205
  38. 38.
    Zhang Z, Bu J J, Ester M, Zhang J F, Yao C W, Yu Z and Wang C. 2019. Hierarchical graph pooling with structure learning. arXiv:1911.05954

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website