多层级几何—语义融合的图神经网络地表异常检测框架

高智; 胡傲涵; 陈泊安; 路遥; 葛家辰

doi:10.11834/jrs.20243301

卫星信息智能应用 | 浏览量 : 0 下载量: 1396 CSCD: 0

R-PDF
PDF
导出
分享
收藏
专辑

多层级几何—语义融合的图神经网络地表异常检测框架
A hierarchical geometry-to-semantic fusion GNN framework for earth surface anomalies detection
2024年28卷第7期页码：1760-1770
收稿：2023-07-23，

纸质出版：2024-07-07
DOI： 10.11834/jrs.20243301
稿件说明：

移动端阅览

高智，胡傲涵，陈泊安，路遥，葛家辰.2024.多层级几何—语义融合的图神经网络地表异常检测框架.遥感学报，28（7）： 1760-1770 DOI： 10.11834/jrs.20243301.

Gao Z，Hu A H，Chen B A，Lu Y and Ge J C. 2024. A hierarchical geometry-to-semantic fusion GNN framework for earth surface anomalies detection. National Remote Sensing Bulletin， 28（7）：1760-1770 DOI： 10.11834/jrs.20243301.

摘要

近年来突发性地表异常ESA（Earth Surface Anomalies）事件频发且呈上升趋势，给人类的生命、财产安全带来了巨大威胁，如何及时准确地发现地表异常事件对后续救援与灾害响应具有重要意义。一些研究人员利用卫星影像开展大尺度地表异常发现与监测，并开发、运用了先进的深度学习方法。然而，由于标签数据不足和卫星影像中几何、语义信息十分复杂，现阶段的地表异常检测方法的表现往往不能达到很好的效果。针对上述问题，本文提出了一个多层级几何—语义融合的图神经网络GNN（Graph Neural Network）框架，以实现高精度地表异常快速发现。具体而言，本文提出的方法先利用两个不同的分支从输入的卫星影像中提取不同层级的地理实体并构建图表示，然后通过分配矩阵实现图的联合表达。此后，构建了分层图注意力网络GAT（Graph Attention Network），基于图节点信息传递、聚合与注意力机制从图中进一步挖掘卫星影像中复杂的几何、语义信息，用于准确地检测地表异常。考虑到现有大规模多类地表异常检测基准数据集的缺乏，我们基于现有公开可分发数据集构建了ESAD数据集，用于基于单张卫星影像的地表异常检测任务。大量实验表明，提出的方法在地表异常检测任务中取得了优异的性能，在准确率、召回率与推理时间方面优于许多基线方法，可精确、有效地开展地表异常检测任务。

Abstract

The increasing occurrence of Earth Surface Anomalies (ESAs) highlights the importance of the timely and accurate detection of such events. Therefore

researchers have utilized satellite imagery for large-scale detection and developed advanced deep learning methods. However

the performance of the methods is hindered by inadequate labeled data and the complexity of semantic information in satellite imagery. In this paper

we aim to address the aforementioned problems and improve the performance of Earth surface anomaly detection. This paper reviews and summarizes developments in Earth’s surface anomaly detection and the problems that hinder the performance of existing methods. Then

we proposed a hierarchical geometry-to-semantic fusion Graph Neural Network (GNN) framework

which utilizes a single image for the detection of Earth’s surface anomalies and reduces the demand for data and time required for preprocessing and inference. Specifically

our method employs two branches for the extraction of geoentities and construction of graphs at different levels. Then

a hierarchical graph attention network was utilized to update node features and extract graph embeddings for each level. An attention-based feature fusion module then combined them to yield the graph-level feature vector of the input image

which was finally processed through a multilayer perceptron for ESA detection. Given that existing ESA datasets mainly focus on single-class detection or post-hoc analysis

which is insufficient for our research needs

we proposed ESAD

which is a composite dataset

to bridge the gap between large-scale multiclass data

sets. Specifically

ESAD is composed of three publicly datasets: xBD

Multi

Net and Sichuan Landslide

and Debrisflow. The proposed method was effective and accurate for detecting ESA

outperforming many baseline methods and balancing between accuracy and efficiency. Thus

it is suitable for ESA detection

saving valuable time and resources for downstream tasks. In conclusion

we proposed a hierarchical geometry-to-semantic fusion GNN framework for ESA detection. It leverages GNN to learn high-order semantic information from satellite imagery. To address insufficiency in benchmark datasets

we created the ESAD dataset based on existing related datasets. Our method achieved a good balance between accuracy and efficiency and is suitable for ESA detection with high timeliness requirements. In future work

we will further explore more models and extend our method to on-orbit real-time ESA detection task.

关键词

Keywords

references

Achanta R , Shaji A , Smith K , Lucchi A , Fua P and Süsstrunk S . 2012 . SLIC superpixels compared to state-of-the-art superpixel methods . IEEE Transactions on Pattern Analysis and Machine Intelligence , 34 ( 11 ): 2274 - 2282 [ DOI: 10.1109/TPAMI.2012.120 http://dx.doi.org/10.1109/TPAMI.2012.120 ]

Bergmann P , Fauser M , Sattlegger D and Steger C . 2019 . MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 2019: 9592 - 9600 . [ DOI: 10.1109/CVPR.2019.00982 http://dx.doi.org/10.1109/CVPR.2019.00982 ]

Feng Q L , Chen B A , Li G Q , Yao X C , Gao B B and Zhang L C . 2022 . A review for sample datasets of remote sensing imagery . National Remote Sensing Bulletin , 26 ( 4 ): 589 - 605

冯权泷 , 陈泊安 , 李国庆 , 姚晓闯 , 高秉博 , 张连翀 . 2022 . 遥感影像样本数据集研究综述 . 遥感学报 , 26 ( 4 ): 589 - 605 [ DOI: 10.11834/jrs.20221162 http://dx.doi.org/10.11834/jrs.20221162 ]

Gao Y , Shi J , Li J and Wang R Y . 2021 . Remote sensing scene classification based on high-order graph convolutional network . European Journal of Remote Sensing , 54 ( S1 ): 141 - 155 [ DOI: 10.1080/22797254.2020.1868273 http://dx.doi.org/10.1080/22797254.2020.1868273 ]

Gueguen L and Hamid R . 2015 . Large-scale damage detection using satellite imagery . Proceedings of the IEEE conference on computer vision and pattern recognition . 2015 : 1321 - 1328 . [ DOI: 10.1109/CVPR.2015.7298737 http://dx.doi.org/10.1109/CVPR.2015.7298737 ]

Gupta R , Goodman B , Patel N , Hosfelt R , Sajeev S , Heim E , Doshi J , Lucas K , Choset H and Gaston M . 2019 . Creating xBD : a dataset for assessing building damage from satellite imagery// Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition Workshops : 10 - 17 [ DOI: 10.1184/R1/8135576.V1 http://dx.doi.org/10.1184/R1/8135576.V1 ]

Ji M , Liu L F and Buchroithner M . 2018 . Identifying collapsed buildings using post-earthquake satellite imagery and convolutional neural networks: a case study of the 2010 Haiti earthquake . Remote Sensing , 10 ( 11 ): 1689 [ DOI: 10.3390/rs10111689 http://dx.doi.org/10.3390/rs10111689 ]

Jia S , Jiang S G , Zhang S Y , Xu M and Jia X P . 2024 . Graph-in-graph convolutional network for hyperspectral image classification . IEEE Transactions on Neural Networks and Learning Systems , 35 ( 1 ): 1157 - 1171 [ DOI: 10.1109/TNNLS.2022.3182715 http://dx.doi.org/10.1109/TNNLS.2022.3182715 ]

Jiang J J , Ma J Y and Liu X M . 2022 . Multilayer spectral–spatial graphs for label noisy robust hyperspectral image classification . IEEE Transactions on Neural Networks and Learning Systems , 33 ( 2 ): 839 - 852 [ DOI: 10.1109/TNNLS.2020.3029523 http://dx.doi.org/10.1109/TNNLS.2020.3029523 ]

Jiang X , Li G , Liu Y , Zhang X P and He Y . 2020 . Change detection in heterogeneous optical and SAR remote sensing images via deep homogeneous feature fusion . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13 : 1551 - 1566 [ DOI: 10.1109/JSTARS.2020.2983993 http://dx.doi.org/10.1109/JSTARS.2020.2983993 ]

Kuzin D , Isupova O , Simmons B D and Reece S . 2021 . Disaster mapping from satellites: damage detection with crowdsourced point labels . arXiv : 2111 . 03693 [ DOI: 10.48550/arXiv.2111.03693 http://dx.doi.org/10.48550/arXiv.2111.03693 ]

Kyrkou C and Theocharides T . 2020 . EmergencyNet: efficient aerial image classification for drone-based emergency monitoring using atrous convolutional feature fusion . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13 : 1687 - 1699 [ DOI: 10.1109/JSTARS.2020.2969809 http://dx.doi.org/10.1109/JSTARS.2020.2969809 ]

Lai J B , Kang X D , Lu X K and Li S T . 2022 . A review of land observation satellite remote sensing application technology with new generation artificial intelligence . National Remote Sensing Bulletin , 26 ( 8 ): 1530 - 1546

赖积保 , 康旭东 , 鲁续坤 , 李树涛 . 2022 . 新一代人工智能驱动的陆地观测卫星遥感应用技术综述 . 遥感学报 , 26 ( 8 ): 1530 - 1546 [ DOI: 10.11834/jrs.20221555 http://dx.doi.org/10.11834/jrs.20221555 ]

Lee J , Xu J Z , Sohn K , Lu W H , Berthelot D , Gur I , Khaitan P , Huang K W , Koupparis K and Kowatsch B . 2020 . Assessing post-disaster damage from satellite imagery using semi-supervised learning techniques . arXiv : 2011 . 14004 [ DOI: 10.48550/arXiv.2011.14004 http://dx.doi.org/10.48550/arXiv.2011.14004 ]

Li M , Zhang X W , Fan X N and Zhang Z . 2015 . Fly vision-inspired anomaly detection algorithm for remote sensing data under clutter background . Journal of Remote Sensing (in Chinese) , 19 ( 5 ): 780 - 790

李敏 , 张学武 , 范新南 , 张卓 . 2015 . 复杂背景下的仿蝇视觉遥感异常检测 . 遥感学报 , 19 ( 5 ): 780 - 790 [ DOI: 10.11834/jrs.20154057 http://dx.doi.org/10.11834/jrs.20154057 ]

Li Z Y , Kuang G Y , Zou H X and Wu H . 2003 . Research of anomaly detection approaches based on feature fusion in hyperspectral imagery . Journal of Remote Sensing (in Chinese) , 7 ( 4 ): 304 - 308

李智勇 , 匡纲要 , 邹焕新 , 吴昊 . 2003 . 基于特征层融合的高光谱图像异常检测算法研究 . 遥感学报 , 7 ( 4 ): 304 - 308 [ DOI: 10.11834/jrs.20030412 http://dx.doi.org/10.11834/jrs.20030412 ]

Liang J L , Deng Y F and Zeng D . 2020 . A deep neural network combined CNN and GCN for remote sensing scene classification . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13 : 4325 - 4338 [ DOI: 10.1109/JSTARS.2020.3011333 http://dx.doi.org/10.1109/JSTARS.2020.3011333 . ]

Liu Q C , Xiao L , Yang J X and Wei Z H . 2021 . CNN-enhanced graph convolutional network with pixel- and superpixel-level feature fusion for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 59 ( 10 ): 8657 - 8671 [ DOI: 10.1109/TGRS.2020.3037361 http://dx.doi.org/10.1109/TGRS.2020.3037361 ]

Liu Q L , Deng M , Wang J Q , Peng S L , Mei X M and Zhao L . 2011 . Spatio-temporal outliers detection within the space-time framework . Journal of Remote Sensing (in Chinese) , 15 ( 3 ): 457 - 474

刘启亮 , 邓敏 , 王佳璆 , 彭思岭 , 梅小明 , 赵玲 . 2011 . 时空一体化框架下时空异常探测 . 遥感学报 , 15 ( 3 ): 457 - 474 [ DOI: 10.11834/jrs.20110013 http://dx.doi.org/10.11834/jrs.20110013 ]

Mateo-Garcia G , Oprea S , Smith L , Veitch-Michaelis J , Schumann G , Gal Y , Baydin A G and Backes D . 2020 . Flood detection on low cost orbital hardware . arXiv : 1910 . 03019 [ DOI: 10.48550/arXiv.1910.03019 http://dx.doi.org/10.48550/arXiv.1910.03019 ]

Niu C Y , Gao O Y , Lu W J , Liu W and Lai T . 2022 . Reg-SA–UNet++: a lightweight landslide detection network based on single-temporal images captured postlandslide . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 15 : 9746 - 9759 [ DOI: 10.1109/JSTARS.2022.3219897 http://dx.doi.org/10.1109/JSTARS.2022.3219897 ]

Pati P , Jaume G , Foncubierta-Rodríguez A , Feroce F , Anniciello A M , Scognamiglio G , Brancati N , Fiche M , Dubruc E , Riccio D , Di Bonito M , De Pietro G , Botti G , Thiran J P , Frucci M , Goksel O and Gabrani M . 2022 . Hierarchical graph representations in digital pathology . Medical Image Analysis , 75 : 102264 [ DOI: 10.1016/j.media.2021.102264 http://dx.doi.org/10.1016/j.media.2021.102264 . ]

Rudner T G J , Rußwurm M , Fil J , Pelich R , Bischke B , Kopačková V and Biliński P . 2019 . Multi 3 Net: segmenting flooded buildings via fusion of multiresolution, multisensor, and multitemporal satellite imagery // Proceedings of the 33rd AAAI Conference on Artificial Intelligence . Honolulu : AAAI Press: 702 - 709 [ DOI: 10.1609/aaai.v33i01.3301702 http://dx.doi.org/10.1609/aaai.v33i01.3301702 ]

Rui X , Cao Y , Yuan X , Kang Y and Song W G . 2021 . DisasterGAN: generative adversarial networks for remote sensing disaster image generation . Remote Sensing , 13 ( 21 ): 4284 [ DOI: 10.3390/rs13214284 http://dx.doi.org/10.3390/rs13214284 ]

Saha S , Shahzad M , Ebel P and Zhu X X . 2022 . Supervised change detection using prechange optical-SAR and postchange SAR data . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 15 : 8170 - 8178 [ DOI: 10.1109/JSTARS.2022.3206898 http://dx.doi.org/10.1109/JSTARS.2022.3206898 ]

Tilon S , Nex F , Kerle N and Vosselman G . 2020 . Post-disaster building damage detection from earth observation imagery using unsupervised and transferable anomaly detecting generative adversarial networks . Remote Sensing , 12 ( 24 ): 4193 [ DOI: 10.3390/rs12244193 http://dx.doi.org/10.3390/rs12244193 ]

Veličković P , Cucurull G , Casanova A , Romero A , Liò P and Bengio Y . 2018 . Graph attention networks . arXiv : 1710 . 10903 [ DOI: 10.48550/arXiv.1710.10903 http://dx.doi.org/10.48550/arXiv.1710.10903 ]

Wang Q . 2022 . Research framework of remote sensing monitoring and real-time diagnosis of earth surface anomalies . Acta Geodaetica et Cartographica Sinica , 51 ( 7 ): 1141 - 1152

王桥 . 2022 . 地表异常遥感探测与即时诊断方法研究框架 . 测绘学报 , 51 ( 7 ): 1141 - 1152 [ DOI: 10.11947/j.AGCS.2022.20220124 http://dx.doi.org/10.11947/j.AGCS.2022.20220124 ]

Wang S X , Zhou Y , Wei C J , Wang W M , Zhang Z K , Zhao Q , Yan F and Wang LT . 2009 . Remote sensing early warning, monitoring and evaluation of major natural disasters . Journal of Remote Sensing(in Chinese) , 13 ( S1 ): 364 - 372

王世新 , 周艺 , 魏成阶 , 王为民 , 张宗科 , 赵清 , 阎福礼 , 王丽涛 . 2009 . 重大自然灾害遥感预警、监测和评价 . 遥感学报 , 13 ( S1 ): 364 - 372 [ DOI: 10.11834/jrs.20090050 http://dx.doi.org/10.11834/jrs.20090050 ]

Weber E and Kané H . 2020 . Building disaster damage assessment in satellite imagery with multi-temporal fusion . arXiv : 2004 . 05525 [ DOI: 10.48550/arXiv.2004.05525 http://dx.doi.org/10.48550/arXiv.2004.05525 ]

Weber E , Papadopoulos D P , Lapedriza A , Ofli F , Imran M and Torralba A . 2023 . Incidents1M: a large-scale dataset of images with natural disasters, damage, and incidents . IEEE Transactions on Pattern Analysis and Machine Intelligence , 45 ( 4 ): 4768 - 4781 [ DOI: 10.1109/TPAMI.2022.3191996 http://dx.doi.org/10.1109/TPAMI.2022.3191996 ]

Wu Z H , Pan S R , Chen F W , Long G D , Zhang C Q and Yu P S . 2021 . A comprehensive survey on graph neural networks . IEEE Transactions on Neural Networks and Learning Systems , 32 ( 1 ): 4 - 24 [ DOI: 10.1109/TNNLS.2020.2978386 http://dx.doi.org/10.1109/TNNLS.2020.2978386 ]

Xi B B , Li J J , Li Y S , Song R , Xiao Y C , Du Q and Chanussot J . 2022 . Semisupervised cross-scale graph prototypical network for hyperspectral image classification . IEEE Transactions on Neural Networks and Learning Systems , 34 ( 11 ): 9337 - 9351 [ DOI: 10.1109/TNNLS.2022.3158280 http://dx.doi.org/10.1109/TNNLS.2022.3158280 ]

Xie W Y , Zhong J P and Li Y S . 2024 . A background memory model for hyperspectral anomaly detection . National Remote Sensing Bulletin , 28 ( 3 ): 717 - 729

谢卫莹 , 钟佳平 , 李云松 . 2024 . 面向高光谱异常检测的背景记忆模型 . 遥感学报 , 28 ( 3 ): 717 - 729 [ DOI: 10.11834/jrs.20221241 http://dx.doi.org/10.11834/jrs.20221241 ]

Xu J Z , Lu W H , Li Z B , Khaitan P and Zaytseva V . 2019 . Building damage detection in satellite imagery using convolutional neural networks . arXiv : 1910 . 06444 [ DOI: 10.48550/arXiv.1910.06444 http://dx.doi.org/10.48550/arXiv.1910.06444 ]

Yu B , She J , Liu G X , Ma D Y , Zhang R , Zhou Z W and Zhang B . 2022 . Coal fire identification and state assessment by integrating multitemporal thermal infrared and InSAR remote sensing data: a case study of Midong District, Urumqi, China . ISPRS Journal of Photogrammetry and Remote Sensing , 190 : 144 - 164 [ DOI: 10.1016/j.isprsjprs.2022.06.007 http://dx.doi.org/10.1016/j.isprsjprs.2022.06.007 ]

Zeng C , Cao Z Y , Su F H , Zeng Z and Yu C X . 2022 . A dataset of high-precision aerial imagery and interpretation of landslide and debris flow disaster in Sichuan and surrounding areas between 2008 and 2020 . China Scientific Data , 7 ( 2 ): 195 - 205

曾超 , 曹振宇 , 苏凤环 , 曾珍 , 余长锡 . 2022 . 四川及周边滑坡泥石流灾害高精度航空影像及解译数据集 ( 2008 - 2020 年). 中国科学数据 , 7 ( 2 ): 195 - 205 [ DOI: 10.11922/noda.2021.0005.zh http://dx.doi.org/10.11922/noda.2021.0005.zh ]

Zhang Z , Bu J J , Ester M , Zhang J F , Yao C W , Yu Z and Wang C . 2019 . Hierarchical graph pooling with structure learning . arXiv : 1911 . 05954 [ DOI: 10.48550/arXiv.1911.05954 http://dx.doi.org/10.48550/arXiv.1911.05954 ]

文章被引用时，请邮件提醒。

提交

基于多尺度监督对比学习的高光谱图像分类网络

面向高分辨率遥感影像变化检测的混合空间金字塔池化网络

基于多尺度注意力机制的PolSAR深度学习超分辨率模型

从光学到SAR：基于多级跨模态对齐的SAR图像舰船检测算法

注意力引导全尺度连接网络的高分辨率影像变化检测