Früher|Nächster

Optical-signal token guided change detection network for heterogeneous remote sensing image

  • role： First author第一作者
  • Affiliation：

    Faculty of Printing, Packaging Engineering and Digital Media Technology, Xi’an University of Technology, Xi’an 710054, China

  • Email：2371759407@qq.com
  • Introduction：刘秦森，研究方向为人工智能与遥感图像变化检测方法。E-mail：2371759407@qq.com
LIU Qinsen1,  
  • role： Corresponding author通信作者
  • Affiliation：

    Faculty of Printing, Packaging Engineering and Digital Media Technology, Xi’an University of Technology, Xi’an 710054, China

    State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China

  • Email：sunbangyong@xaut.edu.cn
  • Introduction：孙帮勇，研究方向为光谱成像技术与光谱图像处理与复现。E-mail：sunbangyong@xaut.edu.cn
SUN Bangyong12*

Resümee

Change Detection (CD) is a vital technique for identifying and analyzing changes over time in a specific area using optical signals from remote sensing images. This technique has been extensively utilized in various fields, including national defense security, environmental monitoring, and urban construction. However, some challenges in achieving accurate and reliable CD are still encountered due to inherent disparities in imaging mechanisms, spectral ranges, and spatial resolutions among heterogeneous images. These challenges lead to issues such as inadequate accuracy, missed detections, and false detections. Heterogeneous remote sensing images can be regarded as sequences of different optical signals from the channel perspective. For example, RGB and infrared images can be regarded as sequences of spectral signals from different ranges. Transformers employ a multi-head attention mechanism that can effectively handle and analyze sequence information to achieve accurate heterogeneous CD. Thus, the paper proposes an optical signal token guided CD network for heterogeneous remote sensing images.This paper presents a novel heterogeneous CD network, primarily comprising the optical-signal token transformer (OT-Former) and the cross-temporal transformer (CT-Former). The proposed method demonstrates the capacity to effectively handle diverse remote sensing images of distinct categories and attain precise CD results. Specifically, OT-Former can encode diverse heterogeneous images in channel-wise for adaptively generating the optical-signal tokens. Meanwhile, CT-Former can use the optical-signal tokens as a guide to interact with the patch token for the learning of change rules. Moreover, a Difference Amplification Module (DAM) is embedded into the network to enhance the extraction of difference information. This module utilizes a 1×2 convolutional kernel to effectively fuse difference information. Finally, the differential token is predicted by multilayer perceptron to output the CD results.Experiments were conducted on three heterogeneous datasets and one homogeneous dataset to evaluate the performance of the proposed method. Furthermore, the proposed method was compared with six typical CD methods and evaluated the performance using overall accuracy (OA), Kappa coefficient, and F1-score, among other evaluation metrics, to validate the effectiveness of the proposed network in this study. A limited number of samples were utilized for training during the experiment. Under identical experimental conditions, the proposed method demonstrated exceptional performance in homogeneous and heterogeneous CD. The results show that the proposed approach surpasses existing state-of-the-art methods in terms of qualitative and visual performance. Additionally, ablation experiments and parameter analyses were conducted to validate the effectiveness of the proposed methods, including the OT-Former, CT-Former, and DAM modules, and to assess the impact of various parameters within the network.Overall, the current study presents a novel heterogeneous CD network based on the transformer framework. Within this network, OT-Former is proposed to achieve the adaptive generation of optical-signal tokens from diverse remote sensing images. Moreover, the CT-Former utilizes these optical-signal tokens as a guide to facilitate interaction with patch tokens for the learning of change rules. Additionally, DAM modules were embedded into the network to effectively extract the difference information. An extremely limited number of samples were utilized only for training in the experiments. Remarkably, the proposed method outperformed the existing state-of-the-art methods, achieving a significantly advanced performance in heterogeneous CD.

Schlüsselwort

remote sensing;heterogeneous images;change detection;multimodal analysis;deep learning;Transformer

References

  1. 1.
    Adão T, Hruška J, Pádua L, Bessa J, Peres E, Morais R and Sousa J J. 2017. Hyperspectral imaging: a review on UAV-based sensors, data processing and applications for agriculture and forestry. Remote Sensing, 9(11): 1110
  2. 2.
    Bandara W G C and Patel V M. 2022. A transformer-based Siamese network for change detection//IEEE International Geoscience and Remote Sensing Symposium. Kuala Lumpur: IEEE: 207-210
  3. 3.
    Bengio Y, Simard P and Frasconi P. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2): 157-166
  4. 4.
    Bovolo F and Bruzzone L. 2007. A theoretical framework for unsupervised change detection based on change vector analysis in the polar domain. IEEE Transactions on Geoscience and Remote Sensing, 45(1): 218-236
  5. 5.
    Bovolo F, Marchesi S and Bruzzone L. 2012. A framework for automatic and unsupervised detection of multiple changes in multitemporal images. IEEE Transactions on Geoscience and Remote Sensing, 50(6): 2196-2212
  6. 6.
    Canty M J and Nielsen A A. 2008. Automatic radiometric normalization of multitemporal satellite imagery with the iteratively re-weighted MAD transformation. Remote Sensing of Environment, 112(3): 1025-1036
  7. 7.
    Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X H, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J and Houlsby N. 2020. An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint 2010.11929
  8. 8.
    Hasanlou M and Seydi S T. 2018. Hyperspectral change detection: an experimental comparative study. International Journal of Remote Sensing, 39(20): 7029-7083
  9. 9.
    Hong D F, Han Z, Yao J, Gao L R, Zhang B, Plaza A and Chanussot J. 2022. SpectralFormer: rethinking hyperspectral image classification with transformers. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-15
  10. 10.
    Hou Z F, Li W and Du Q. 2021. A patch tensor-based change detection method for hyperspectral images//2021 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Brussels: IEEE: 4328-4331
  11. 11.
    Liu S C, Du K C, Zheng Y J, Chen J, Du P J and Tong X H. 2023. Remote sensing change detection technology in the era of artificial intelligence: inheritance, development and challenges. National Remote Sensing Bulletin, 27(9): 1975-1987
  12. 12.
    Lyu H B, Lu H and Mou L C. 2016. Learning a transferable change rule from a recurrent neural network for land cover change detection. Remote Sensing, 8(6): 506
  13. 13.
    Ma W P, Yang H, Wu Y, Xiong Y T, Hu T, Jiao L C and Hou B. 2019. Change detection based on multi-grained cascade forest and multi-scale fusion for SAR images. Remote Sensing, 11(2): 142
  14. 14.
    Mandal M and Vipparthi S K. 2022. An empirical review of deep learning frameworks for change detection: model design, experimental frameworks, challenges and research needs. IEEE Transactions on Intelligent Transportation Systems, 23(7): 6101-6122
  15. 15.
    Pirrone D, Bovolo F and Bruzzone L. 2020. A novel framework based on polarimetric change vectors for unsupervised multiclass change detection in dual-pol intensity SAR images. IEEE Transactions on Geoscience and Remote Sensing, 58(7): 4780-4795
  16. 16.
    Qu J H, Xu Y S, Dong W Q, Li Y S and Du Q. 2022. Dual-branch difference amplification graph convolutional network for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-12
  17. 17.
    Sun W W, Yang G, Peng J T and Du Q. 2020. Lateral-slice sparse tensor robust principal component analysis for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(1): 107-111
  18. 18.
    Sun Y L, Lei L, Guan D D, Kuang G Y and Liu L. 2022a. Graph signal processing for heterogeneous change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-23
  19. 19.
    Sun Y L, Lei L, Guan D D, Li M and Kuang G Y. 2022b. Sparse-constrained adaptive structure consistency-based unsupervised image regression for heterogeneous remote-sensing change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-14
  20. 20.
    Wang Y H, Hong D F, Sha J J, Gao L R, Liu L, Zhang Y G and Rong X H. 2022a. Spectral-spatial-temporal transformers for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-14
  21. 21.
    Wang Y K, Chen X H, Gao L L, Huang W B, Sun F C and Wang Y H. 2022b. Multimodal token fusion for vision transformers//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 12176-12185
  22. 22.
    Wu C, Du B and Zhang L P. 2014. Slow feature analysis for change detection in multispectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 52(5): 2858-2874
  23. 23.
    Yang B, Mao Y, Chen J, Liu JQ, Chen J and Yan K. 2023. Review of remote sensing change detection in deep learning: bibliometric and analysis. National Remote Sensing Bulletin, 27(9): 1988-2005
  24. 24.
    Yang X F, Ye Y M, Li X T, Lau R Y K, Zhang X F and Huang X H. 2018. Hyperspectral image classification with deep learning models. IEEE Transactions on Geoscience and Remote Sensing, 56(9): 5408-5423
  25. 25.
    Zhan Y, Fu K, Yan M L, Sun X, Wang H Q and Qiu X S. 2017. Change detection based on deep Siamese convolutional network for optical aerial images. IEEE Geoscience and Remote Sensing Letters, 14(10): 1845-1849

Lesen Sie die ganze Passage

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website