光学信号Token引导的异源遥感变化检测网络
Optical-signal token guided change detection network for heterogeneous remote sensing image
- 2024年28卷第1期 页码：88-104
纸质出版日期： 2024-01-07
DOI： 10.11834/jrs.20233067
扫 描 看 全 文
浏览全部资源
扫码关注微信
摘要
基于遥感图像中的光学信号检测出一定时间内特定区域的变化状态的遥感图像变化检测方法，在国防安全、环境监测、城市建设等领域具有重要应用价值。由于多时相异源图像在成像机理、光谱范围、空间分辨率等方面存在差异，现阶段异源遥感图像变化检测仍存在精度不够高、漏检和误检等问题，本文提出一种基于Transformer网络的异源变化检测网络框架，该框架能够利用不同类别的异源遥感图像获得准确的变化检测结果。首先，所提出检测网络为多时相遥感图像自适应生成对应的光学信号Token（光信Token）；然后，以光信Token作为引导与对应图像块Token进行交互计算，从而对双时相序列特征进行变化分析，并且在交互学习过程中构建了差分放大模块以提高网络对特征间差分信息的提取精度；最后，利用多层感知机对输出的差分Token进行预测并分割出变化区域。采用Sardinia、Shuguang和Bastrop等3个不同类别的异源遥感图像数据集和Farmland同源高光谱图像数据集来验证本文提出的方法，结果证明在选取有限训练样本数据情况下，本文方法与现有主流变化检测方法相比，在多个客观指标以及主观视觉上都表现出先进性。
Abstract
Change Detection (CD) is a vital technique for identifying and analyzing changes over time in a specific area using optical signals from remote sensing images. This technique has been extensively utilized in various fields
including national defense security
environmental monitoring
and urban construction. However
some challenges in achieving accurate and reliable CD are still encountered due to inherent disparities in imaging mechanisms
spectral ranges
and spatial resolutions among heterogeneous images. These challenges lead to issues such as inadequate accuracy
missed detections
and false detections. Heterogeneous remote sensing images can be regarded as sequences of different optical signals from the channel perspective. For example
RGB and infrared images can be regarded as sequences of spectral signals from different ranges. Transformers employ a multi-head attention mechanism that can effectively handle and analyze sequence information to achieve accurate heterogeneous CD. Thus
the paper proposes an optical signal token guided CD network for heterogeneous remote sensing images.
This paper presents a novel heterogeneous CD network
primarily comprising the optical-signal token transformer (OT-Former) and the cross-temporal transformer (CT-Former). The proposed method demonstrates the capacity to effectively handle diverse remote sensing images of distinct categories and attain precise CD results. Specifically
OT-Former can encode diverse heterogeneous images in channel-wise for adaptively generating the optical-signal tokens. Meanwhile
CT-Former can use the optical-signal tokens as a guide to interact with the patch token for the learning of change rules. Moreover
a Difference Amplification Module (DAM) is embedded into the network to enhance the extraction of difference information. This module utilizes a 1×2 convolutional kernel to effectively fuse difference information. Finally
the differential token is predicted by multilayer perceptron to output the CD results.
Experiments were conducted on three heterogeneous datasets and one homogeneous dataset to evaluate the performance of the proposed method. Furthermore
the proposed method was compared with six typical CD methods and evaluated the performance using overall accuracy (OA)
Kappa coefficient
and F1-score
among other evaluation metrics
to validate the effectiveness of the proposed network in this study. A limited number of samples were utilized for training during the experiment. Under identical experimental conditions
the proposed method demonstrated exceptional performance in homogeneous and heterogeneous CD. The results show that the proposed approach surpasses existing state-of-the-art methods in terms of qualitative and visual performance. Additionally
ablation experiments and parameter analyses were conducted to validate the effectiveness of the proposed methods
including the OT-Former
CT-Former
and DAM modules
and to assess the impact of various parameters within the network.
Overall
the current study presents a novel heterogeneous CD network based on the transformer framework. Within this network
OT-Former is proposed to achieve the adaptive generation of optical-signal tokens from diverse remote sensing images. Moreover
the CT-Former utilizes these optical-signal tokens as a guide to facilitate interaction with patch tokens for the learning of change rules. Additionally
DAM modules were embedded into the network to effectively extract the difference information. An extremely limited number of samples were utilized only for training in the experiments. Remarkably
the proposed method outperformed the existing state-of-the-art methods
achieving a significantly advanced performance in heterogeneous CD.
关键词
遥感异源图像变化检测多模态分析深度学习Transformer
Keywords
remote sensingheterogeneous imageschange detectionmultimodal analysisdeep learningtransformer
references
Adão T, Hruška J, Pádua L, Bessa J, Peres E, Morais R and Sousa J J. 2017. Hyperspectral imaging: a review on UAV-based sensors, data processing and applications for agriculture and forestry. Remote Sensing, 9(11): 1110 [DOI: 10.3390/rs9111110http://dx.doi.org/10.3390/rs9111110]
Bandara W G C and Patel V M. 2022. A transformer-based Siamese network for change detection//IEEE International Geoscience and Remote Sensing Symposium. Kuala Lumpur: IEEE: 207-210 [DOI: 10.1109/IGARSS46834.2022.9883686http://dx.doi.org/10.1109/IGARSS46834.2022.9883686]
Bengio Y, Simard P and Frasconi P. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2): 157-166 [DOI: 10.1109/72.279181http://dx.doi.org/10.1109/72.279181]
Bovolo F and Bruzzone L. 2007. A theoretical framework for unsupervised change detection based on change vector analysis in the polar domain. IEEE Transactions on Geoscience and Remote Sensing, 45(1): 218-236 [DOI: 10.1109/TGRS.2006.885408http://dx.doi.org/10.1109/TGRS.2006.885408]
Bovolo F, Marchesi S and Bruzzone L. 2012. A framework for automatic and unsupervised detection of multiple changes in multitemporal images. IEEE Transactions on Geoscience and Remote Sensing, 50(6): 2196-2212 [DOI: 10.1109/TGRS.2011.2171493http://dx.doi.org/10.1109/TGRS.2011.2171493]
Canty M J and Nielsen A A. 2008. Automatic radiometric normalization of multitemporal satellite imagery with the iteratively re-weighted MAD transformation. Remote Sensing of Environment, 112(3): 1025-1036 [DOI: 10.1016/j.rse.2007.07.013http://dx.doi.org/10.1016/j.rse.2007.07.013]
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X H, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J and Houlsby N. 2020. An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint 2010.11929 [DOI: 10.48550/arXiv.2010.11929http://dx.doi.org/10.48550/arXiv.2010.11929]
Hasanlou M and Seydi S T. 2018. Hyperspectral change detection: an experimental comparative study. International Journal of Remote Sensing, 39(20): 7029-7083 [DOI: 10.1080/01431161.2018.1466079http://dx.doi.org/10.1080/01431161.2018.1466079]
Hong D F, Han Z, Yao J, Gao L R, Zhang B, Plaza A and Chanussot J. 2022. SpectralFormer: rethinking hyperspectral image classification with transformers. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-15 [DOI: 10.1109/TGRS.2021.3130716http://dx.doi.org/10.1109/TGRS.2021.3130716]
Hou Z F, Li W and Du Q. 2021. A patch tensor-based change detection method for hyperspectral images//2021 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Brussels: IEEE: 4328-4331 [DOI: 10.1109/IGARSS47720.2021.9554630http://dx.doi.org/10.1109/IGARSS47720.2021.9554630]
Liu S C, Du K C, Zheng Y J, Chen J, Du P J and Tong X H. 2023. Remote sensing change detection technology in the era of artificial intelligence: inheritance, development and challenges. National Remote Sensing Bulletin, 27(9): 1975-1987
柳思聪, 都科丞, 郑永杰, 陈晋, 杜培军, 童小华. 2023. 人工智能时代的遥感变化检测技术: 继承、发展与挑战. 遥感学报, 27(9): 1975-1987 [DOI: 10.11834/jrs.20222199http://dx.doi.org/10.11834/jrs.20222199]
Lyu H B, Lu H and Mou L C. 2016. Learning a transferable change rule from a recurrent neural network for land cover change detection. Remote Sensing, 8(6): 506 [DOI: 10.3390/rs8060506http://dx.doi.org/10.3390/rs8060506]
Ma W P, Yang H, Wu Y, Xiong Y T, Hu T, Jiao L C and Hou B. 2019. Change detection based on multi-grained cascade forest and multi-scale fusion for SAR images. Remote Sensing, 11(2): 142 [DOI: 10.3390/rs11020142http://dx.doi.org/10.3390/rs11020142]
Mandal M and Vipparthi S K. 2022. An empirical review of deep learning frameworks for change detection: model design, experimental frameworks, challenges and research needs. IEEE Transactions on Intelligent Transportation Systems, 23(7): 6101-6122 [DOI: 10.1109/TITS.2021.3077883http://dx.doi.org/10.1109/TITS.2021.3077883]
Pirrone D, Bovolo F and Bruzzone L. 2020. A novel framework based on polarimetric change vectors for unsupervised multiclass change detection in dual-pol intensity SAR images. IEEE Transactions on Geoscience and Remote Sensing, 58(7): 4780-4795 [DOI: 10.1109/TGRS.2020.2966865http://dx.doi.org/10.1109/TGRS.2020.2966865]
Qu J H, Xu Y S, Dong W Q, Li Y S and Du Q. 2022. Dual-branch difference amplification graph convolutional network for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-12 [DOI: 10.1109/TGRS.2021.3135567http://dx.doi.org/10.1109/TGRS.2021.3135567]
Sun W W, Yang G, Peng J T and Du Q. 2020. Lateral-slice sparse tensor robust principal component analysis for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(1): 107-111 [DOI: 10.1109/LGRS.2019.2915315http://dx.doi.org/10.1109/LGRS.2019.2915315]
Sun Y L, Lei L, Guan D D, Kuang G Y and Liu L. 2022a. Graph signal processing for heterogeneous change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-23 [DOI: 10.1109/TGRS.2022.3221489http://dx.doi.org/10.1109/TGRS.2022.3221489]
Sun Y L, Lei L, Guan D D, Li M and Kuang G Y. 2022b. Sparse-constrained adaptive structure consistency-based unsupervised image regression for heterogeneous remote-sensing change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-14 [DOI: 10.1109/TGRS.2021.3110998http://dx.doi.org/10.1109/TGRS.2021.3110998]
Wang Y H, Hong D F, Sha J J, Gao L R, Liu L, Zhang Y G and Rong X H. 2022a. Spectral-spatial-temporal transformers for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-14 [DOI: 10.1109/TGRS.2022.3203075http://dx.doi.org/10.1109/TGRS.2022.3203075]
Wang Y K, Chen X H, Gao L L, Huang W B, Sun F C and Wang Y H. 2022b. Multimodal token fusion for vision transformers//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 12176-12185 [DOI: 10.1109/CVPR52688.2022.01187http://dx.doi.org/10.1109/CVPR52688.2022.01187]
Wu C, Du B and Zhang L P. 2014. Slow feature analysis for change detection in multispectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 52(5): 2858-2874 [DOI: 10.1109/TGRS.2013.2266673http://dx.doi.org/10.1109/TGRS.2013.2266673]
Yang B, Mao Y, Chen J, Liu JQ, Chen J and Yan K. 2023. Review of remote sensing change detection in deep learning: bibliometric and analysis. National Remote Sensing Bulletin, 27(9): 1988-2005
杨彬, 毛银, 陈晋, 刘建强, 陈杰, 闫凯. 2023. 深度学习的遥感变化检测综述: 文献计量与分析. 遥感学报, 27(9): 1988-2005 [DOI: 10.11834/jrs.20222156http://dx.doi.org/10.11834/jrs.20222156]
Yang X F, Ye Y M, Li X T, Lau R Y K, Zhang X F and Huang X H. 2018. Hyperspectral image classification with deep learning models. IEEE Transactions on Geoscience and Remote Sensing, 56(9): 5408-5423 [DOI: 10.1109/TGRS.2018.2815613http://dx.doi.org/10.1109/TGRS.2018.2815613]
Zhan Y, Fu K, Yan M L, Sun X, Wang H Q and Qiu X S. 2017. Change detection based on deep Siamese convolutional network for optical aerial images. IEEE Geoscience and Remote Sensing Letters, 14(10): 1845-1849 [DOI: 10.1109/LGRS.2017.2738149http://dx.doi.org/10.1109/LGRS.2017.2738149]
文章被引用时，请邮件提醒。
提交
相关作者
暂无数据
相关机构
湖南大学 电气与信息工程学院
北京师范大学 地理科学学部
国家卫星海洋应用中心
中南大学 地球科学与信息物理学院
中国地质大学(北京) 土地科学技术学院
选择翻译语言
- 地址：北京市海淀区北四环西路19号 《遥感学报》编辑部 邮编：100190
- 联系电话：010-58887052 Email：nrsb@aircas.ac.cn
- 技术支持由北京北大方正电子有限公司提供 京ICP备20021838号-9
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
0
回到顶部