变化检测与深度学习
Theme Keywords： deep learningchange detectionremote sensingsemantic segmentation networksample imbalancesmall samplereviewposterior probabilitymulti-temporal analysismethods classification
- The Paper
Domain-adaptation algorithm for remotely sensing building changes through instance contrast learning
Abstract：Building-change detection automatically identifies changes in ground buildings in remote-sensing images acquired in the same geographic area at different times. Fully supervised change-detection algorithms require a large amount of labeled remote sensing data to make accurate predictions. Manually labeling a building change detection label is time-consuming and labor intensive because it requires a professional to compare and label two images pixel by pixel. Unsupervised domain adaptation technique is an effective means to alleviate this problem. Although the current-domain adaptation algorithm has achieved good results in building-change detection, the following problems persist: Problem 1: A class-based domain mixing strategy is applicable to a large number of categories. In building-change detection, only positive samples of the category “change” are available. Problem 2: In the current pixel-based contrast learning method, pseudo labels generated by a model must have samples with classification errors because the labels of target domains are unidentifiable. This requirement introduces large noise information during contrast training. Problem 3: The pseudo label generated by high-confidence threshold filtering does not leverage the low confidence prediction results of a teacher model. To solve the above problems, this paper proposes a case-level contrast-learning domain-adaptation algorithm for cross-domain building-change detection task.This paper proposes an instance contrast-learning domain adaptation for change detection (ICDA-CD) method for cross-domain building-change detection. The main contributions are as follows: (1) A region-level domain-mixing method is proposed, which combines data containing the buildings in a source domain and data containing buildings in a target domain on one sample simultaneously. (2) Case-level contrast learning method is proposed. In the encoder, the distance between the biphasic features of a changing building area is pulled apart. In the decoder, the distance between the features of each changing building area is narrowed. (3) A pseudo label quality estimation method is proposed. The pseudo-label quality of each pixel position is estimated by the value predicted by a teacher model, and then loss is weighted.Domain migration experiments were performed on the LEVIR-CD and S2Looking datasets, and comparison and ablation experiments were performed with advanced domain-adaptation algorithms. In the migration of the LEVIR-CD task to the S2Looking task, the proposed algorithm achieved the highest F1 and IOU of 43.91 and 28.31, respectively. In the migration of the S2Looking task to the LEVIR-CD task, the proposed algorithm achieved the highest F1 scores and IOU of 74.75 and 59.68, respectively.To solve the problem of unsupervised domain adaptive change detection algorithm across data domains, an ICDA-CD method was proposed. The accuracy of the cross-domain unsupervised domain adaptive change detection algorithm was effectively improved by using region-level domain mixing, case-level contrast learning, and pseudo label quality estimation-weighted loss.Keywords：remote sensing images;building change detection;contrast learning;domain adaptation;deep learning;pseudo label1201|2800|1
<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-07-31- Abstract：Compared with homogeneous image change detection (homo-CD), Change Detection (CD) of optical images and SAR images offers the advantage of utilizing complementary information from different types of data. This advantage has made it a research hotspot in the field of remote sensing image processing and holds promise for emergency disaster monitoring. However, the differences in imaging mechanisms between optical and SAR images prevent direct comparison of bitemporal images for CD. Existing methods for optical image and SAR image CD still face certain challenges. Methods aiming to unify the feature space of optical and SAR images often suffer from issues, such as low mapping precision and efficiency. In this study, we propose a Symmetric Change Detection Network (SCDN) that addresses the difference in imaging features between optical and SAR images by mapping them to a common feature space for comparison. The SCDN is initialized and optimized using similarity measurement, and it subsequently maps the optical and SAR images to a similar feature space for change information extraction.The proposed method consists of several steps. First, the similarity between multiple sets of features generated by the symmetrical network is measured, and the weights corresponding to the most similar features are used to initialize the network. This initialization guides the network to map optical and SAR image features. Subsequently, the SCDN maps the optical images and SAR images into the same feature space using similarity optimal learning, enabling direct comparison. Finally, change types are determined by clustering the multitemporal change vectors.To validate the proposed method, we conduct experiments using three sets of images, namely, Google Earth, Landsat-8, and Sentinel-1 images. Comparative analysis with five state-of-the-art methods reveals that the proposed method achieves an increase of at least 4.02% in the kappa coefficient while reducing the running time by at least 30.79%.In this study, we introduce SCDN, a CD method for optical and SAR images. Experimental results demonstrate its effectiveness in achieving relatively high precision and efficiency compared with existing methods.Keywords：remote sensing;optical image;SAR image;change detection;symmetric network;feature extraction;spatial mapping;similarity measure;change type769|1563|2<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-07-19
CBAM UNet+++： Attention mechanism to guide change detection studies of full-scale connected networks
Abstract：Existing change detection networks rely heavily on layer-by-layer convolution for feature extraction. However, the use of this method leads to a loss of information, and it lacks the ability to mine important change features. Therefore, knowing how to effectively suppress the influence of the background and identifying ways to increase the ability of the network to learn salient features and generate recognizable feature information are highly important for change detection tasks. Traditional skip connections lack the ability to obtain change information from a full-scale perspective and perform encoder feature extraction. Thus, a UNet+++ high-resolution remote sensing image change detection network called CBAM UNet+++ combined with a coupled attention mechanism (i.e., a convolutional block attention module [CBAM]) was designed in this research.CBAM UNet+++ is based on the semantic segmentation structure UNet+++. The unique full-scale concatenation operation of UNet+++ effectively fuses the semantic and spatial information from the full-scale perspective to avoid information loss. The basic convolutional unit can be replaced by a residual attention module (Residual Block_CBAM and ResBlock_CBAM) to suppress background effects and enhance the learning ability of the encoder to handle significant features. The residual attention module was validated on two remote sensing image change detection datasets—LEBEDV and LEVIR-CD—involving different high-resolution change regions.The proposed method has the highest accuracy on the LEBEDEV multifeature change dataset, with F1 and OA values of 88.9% and 97.3%, respectively, and the second highest accuracy on the LEVIR-CD building change dataset, with F1 and OA values of 86.7% and 96.8%, respectively. The proposed method can obtain deep semantics in a targeted manner, and its qualitative results are better than those of other benchmark networks.The CBAM UNet+++ method can accurately locate and detect change regions with better detection and accuracy than can the benchmark method. The accuracy results of the two selected datasets were slightly different, but they were not inconsistent. The accuracy of the CBAM UNet+++ model was disrupted by pseudochange information in the building dataset. Future work may focus on the usability of this network for change detection in heterogeneous dual-temporal images to further address the impact of early fusion on change detection accuracy.Keywords：remote sensing;change detection;UNet+++;attention mechanism;encoding and decoding3061|4905|8<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-05-15- Abstract：Change detection, a critical task in remote sensing and geospatial analysis, involves the identification of areas where alterations in land cover types have occurred over time using multi-temporal images. The accurate detection of such changes is essential for various applications, including environmental monitoring, urban development assessment, and natural disaster management. However, existing change detection methods are often susceptible to noise and the influence of specific land features, resulting in significant speckle phenomena and reduced detection accuracy. These limitations hinder the reliable identification of change patterns in land cover, impacting the effectiveness of downstream analyses and decision-making processes.To address these challenges, this paper proposes an unsupervised superpixel-level change detection method that combines canonical correlation analysis and histogram matching. This method aims to improve the accuracy and reliability of change detection by addressing the limitations associated with traditional approaches. The proposed method consists of several steps. First, the remote sensing images were preprocessed and superpixel-segmented. This step is aimed at improving the quality of the image and dividing it into homogeneous regions called superpixels. Superpixel segmentation helps to preserve spatial information and reduces the influence of noise on subsequent analysis. Next, the weight of each superpixel was calculated based on the superpixel scale and the unchanged probability. Superpixel weights are used to highlight the importance of different regions in the change detection process. After obtaining the weights, the method proceeds to extract change features at the superpixel level using multivariate change detection and histogram matching. Multivariate change detection involves analyzing the spectral information of the superpixels to identify changes in land cover types. Histogram matching, on the other hand, aims to align the histograms of the superpixels from different time periods, enabling more accurate comparison and detection of changes. Finally, a change detection result map was developed based on the weighted image, classical methods, and change features.Three hyperspectral test datasets and one multispectral test dataset were used for experimental verification.Experimental validation of the proposed method was conducted on three hyperspectral test datasets and one multispectral test dataset. The results demonstrate the superior performance of the proposed method, with the Overall Accuracy (OA) and Kappa index surpassing those of existing methods across all four test datasets. Specifically, the OA values consistently exceed 90% on all datasets, indicating the high accuracy and robustness of the proposed method. Moreover, comparative analysis reveals significant improvements in the OA when compared to other existing methods. The proposed method achieves an OA increase of 4.41%, 3.44%, 1.74%, and 0.19% on the four datasets, highlighting its efficacy in enhancing change detection accuracy and reliability. In conclusion, the proposed unsupervised superpixel-level change detection method, which integrates canonical correlation analysis and histogram matching, demonstrates remarkable performance in detecting changes in land cover types from multi-temporal remote sensing images.Keywords：remote sensing;super pixel;change detection;canonical correlation analysis;histogram specification2094|2134|2<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-05-15
- Abstract：Building change detection is essential to many applications, such as monitoring of urban areas, land use management, and illegal building detection. It has been seen as an effective means to detect building changes from remote-sensing images.This paper proposes an object-based Siamese neural network, labeled as Obj-SiamNet, to detect building changes from high-resolution remote-sensing images. We combine the advantages of object-based image analysis methods and Siamese neural networks to improve the geometric accuracies of detected boundaries. Moreover, we implement the Obj-SiamNet at multiple segmentation levels and automatically construct a set of fuzzy measures to fuse the obtained results at multi-levels. Furthermore, we use generative adversarial methods to generate target-like training samples from publicly available datasets and construct a relatively sufficient training dataset for the Obj-SiamNet model. Finally, we apply the proposed method into three high-resolution remote-sensing datasets, i.e., a GF-2 image-pair in Fuzhou City, and a GF2 image pair in Pucheng County, and a GF-2—GF-7 image pair in Quanzhou City. We also compare the proposed method with three other existing ones, namely, STANet, ChangeNet, and Siam-NestedUNet.Experimental results show that the proposed method performs better than the other three in terms of detection accuracy. (1) Compared with the detection results from single-scale segmentation, the detection results from multi-scale increases the recall rate by up to 32%, the F1-Score increases by up to 25%, and the Global Total Classification error (GTC) decreases by up to 7%. (2) When the number of available samples is limited, the adopted Generative Adversarial Network (GAN) is able to generate effective target-like samples for diverting samples. Compared with the detection without using GAN-generated samples, the proposed detection increases the recall rate by up to 16%, increases the F1-Score by up to 14%, and decreases GTC by 9%. (3) Compared with other change-detection methods, the proposed method improves the detection accuracies significantly, i.e., the F1-Score increases by up to 23%, and GTC decreases by up to 9%. Moreover, the boundaries of the detected changes by the proposed method have a high consistency with that of ground truth.We conclude that the proposed Obj-SiamNet method has a high potential for building change detection from high-resolution remote-sensing images.Keywords：change detection of remote sensing;Siamese Neural Network;object-based multi-scale analysis;fuzzy sets fusion;Generative Adversarial Network;Very High Resolution remote sensing images2933|2984|8<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-03-22
- Abstract：HyperSpectral Imagery (HSI) is a three-dimensional cube data that combines spatial imagery and spectral information, which introduces increased conveniences to the accurate interpretation of observation information of ground coverings. However, high-dimensional nonlinear data processing for the HSI Change Detection (HSI-CD) task encounters challenges. Therefore, an HSI-CD method based on Spectral-Frequency Domain Attribute Pattern Fusion (SFDAPF) is introduced to gradually quantify the spectral representation of pixel attribute patterns. Specifically, a Saliency Enhancement (SE) strategy for pixel attribute patterns based on Fourier transform theory is developed to improve the separability between pixel attribute patterns in the current work. The proposed SFDAPF method comprises four components as follows.First, a gradient correlation-based spectral absolute distance (GCASD) is designed in this paper. Therefore, the attribute patterns of pixel pairs in bitemporal HSI can be gradually quantified from the aspect of spectral information representation. Then, an SE strategy of attribute patterns of pixel pairs is proposed in accordance with Fourier transform theory, which improves the separability of attribute patterns of changing and non-changing pixel pairs in terms of global spatial information utilization. Next, the saliency level and GCASD per pixel are fused to obtain the comprehensive discrimination value of change detection. Finally, the binarization results of the bitemporal HSI-CD are obtained in accordance with the false alarm threshold.The proposed SFDAPF method is applied to two open-source bitemporal HSI datasets (i.e., River and Farmland datasets). Experimental results show that the proposed SFDAPF method can outperform the traditional and state-of-the-art HSI-CD methods. For the River dataset, compared with the traditional methods, the SFDAPF method in this paper introduces the local context information of the pixel in the calculation stage of the GCASD and adopts the global SE strategy, which is effective in reducing false alarms. Compared with the state-of-the-art methods, the SFDAPF method in this paper achieves the highest accuracy for most of the performance evaluation indicators. For the Farmland dataset, the AA, Kappa, F1, IoU, and OA indicators of the SFDAPF method in this paper have reached the highest accuracy, which is 0.01985, 0.05653, 0.01474, 0.02798, and 0.02187 higher than the second highest accuracy. In addition, the OAu (0.97500) and OAc (0.96766) indicators of the SFDAPF method did not achieve the highest accuracy. However, they were only 0.00673 and 0.01237 lower than the highest accuracy, which can be called slightly lower than the highest accuracy. Therefore, the experiments verified the effectiveness of the proposed SFDAPF method in the HSI-CD task.The proposed SFDAPF method generally considers the representation of spectral information and the utilization of neighborhood spatial information, thus promoting the overall accuracy of HSI-CD. However, the proposed SFDAPF method only considers the single-window eight-connected neighborhood in the spectral characterization stage and the magnitude features represented in the frequency domain. Therefore, future research work should further explore the contribution of dual-window spectral information representation and phase information of frequency domain representation to HSI-CD task.Keywords：hyperspectral image;change detection;image fusion;feature extraction;saliency analysis;fourier transform1361|3195|3<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-02-29
- Abstract：Change Detection (CD) is a vital technique for identifying and analyzing changes over time in a specific area using optical signals from remote sensing images. This technique has been extensively utilized in various fields, including national defense security, environmental monitoring, and urban construction. However, some challenges in achieving accurate and reliable CD are still encountered due to inherent disparities in imaging mechanisms, spectral ranges, and spatial resolutions among heterogeneous images. These challenges lead to issues such as inadequate accuracy, missed detections, and false detections. Heterogeneous remote sensing images can be regarded as sequences of different optical signals from the channel perspective. For example, RGB and infrared images can be regarded as sequences of spectral signals from different ranges. Transformers employ a multi-head attention mechanism that can effectively handle and analyze sequence information to achieve accurate heterogeneous CD. Thus, the paper proposes an optical signal token guided CD network for heterogeneous remote sensing images.This paper presents a novel heterogeneous CD network, primarily comprising the optical-signal token transformer (OT-Former) and the cross-temporal transformer (CT-Former). The proposed method demonstrates the capacity to effectively handle diverse remote sensing images of distinct categories and attain precise CD results. Specifically, OT-Former can encode diverse heterogeneous images in channel-wise for adaptively generating the optical-signal tokens. Meanwhile, CT-Former can use the optical-signal tokens as a guide to interact with the patch token for the learning of change rules. Moreover, a Difference Amplification Module (DAM) is embedded into the network to enhance the extraction of difference information. This module utilizes a 1×2 convolutional kernel to effectively fuse difference information. Finally, the differential token is predicted by multilayer perceptron to output the CD results.Experiments were conducted on three heterogeneous datasets and one homogeneous dataset to evaluate the performance of the proposed method. Furthermore, the proposed method was compared with six typical CD methods and evaluated the performance using overall accuracy (OA), Kappa coefficient, and F1-score, among other evaluation metrics, to validate the effectiveness of the proposed network in this study. A limited number of samples were utilized for training during the experiment. Under identical experimental conditions, the proposed method demonstrated exceptional performance in homogeneous and heterogeneous CD. The results show that the proposed approach surpasses existing state-of-the-art methods in terms of qualitative and visual performance. Additionally, ablation experiments and parameter analyses were conducted to validate the effectiveness of the proposed methods, including the OT-Former, CT-Former, and DAM modules, and to assess the impact of various parameters within the network.Overall, the current study presents a novel heterogeneous CD network based on the transformer framework. Within this network, OT-Former is proposed to achieve the adaptive generation of optical-signal tokens from diverse remote sensing images. Moreover, the CT-Former utilizes these optical-signal tokens as a guide to facilitate interaction with patch tokens for the learning of change rules. Additionally, DAM modules were embedded into the network to effectively extract the difference information. An extremely limited number of samples were utilized only for training in the experiments. Remarkably, the proposed method outperformed the existing state-of-the-art methods, achieving a significantly advanced performance in heterogeneous CD.Keywords：remote sensing;heterogeneous images;change detection;multimodal analysis;deep learning;Transformer735|1748|2<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-02-29
- Abstract：Deep learning has shown great potential in the change detection of multi-temporal remote sensing images in recent years. However, the annotated datasets, which are critically required in training change detection networks, is often limited in various change detection tasks in practical applications. As land cover change usually occupies only a small portion of an image, the number of changed samples is often very small, leading to a serious imbalance between changed and unchanged samples. Therefore, it is an urgent challenge to effectively training change detection networks with small and imbalanced change detection samples. Compared to the collection of change detection samples, it is much easier to obtain land cover classification samples at a single time. Based on the adequate land cover classification samples, a well-trained land cover segmentation network can provide important prior features for change detection.Therefore, this paper proposes a method named as siamese Nested-UNet for change detection in posterior probability space (SNU-PS), which aims to reduce the dependence on change detection samples by utilizing the posterior probability information of segmentation network. The method first trains a High-Resolution Network (HRNet) based on land cover classification samples to obtain the posterior probability of the bi-temporal image. Then, the posterior probability images are input into a siamese Nested-UNet for change detection(SNU) to obtain the change detection results. In order to simplify the network complexity and reduce the training difficulty, the training of semantic segmentation network and change detection network are carried out step by step without interactions in their training stages. As the posterior probability image already contains semantic information of land cover, the requirement of the change detection samples is reduced because the change detection network does not need to extract the features in the multi-spectral images.The change detection experiments based on the SpaceNet7 and HRSCD datasets show that SNU-PS can well utilize the semantic information provided by the land cover segmentation network and maintain stable change detection accuracy when it was trained with different change detection sample sizes. Compared with Post Classification Comparison (PCC), CVAPS (Change-vector analysis in posterior probability space), and different change detection networks (FC-EF, BIT, PCFN, and SNU), SNU-PS achieved higher accuracy and better stability, especially when the sample size is small. Unfortunately, all of the compared methods failed to identify the change type due to the extreme imbalanced samples of different change types.SNU-PS method makes full use of the low-cost classification samples to train the semantic segmentation network, which helps to reduce the reliance on the change detection samples because the change detection network in SNU-PS does not undertake the feature exploration of multi-spectral images. Moreover, the semantic segmentation network and the change detection network are integrated with independent training process in SNU-PS, thus the integration of two networks does not increase the training difficulty and semantic segmentation network can be flexibly replaced with better network if available. In conclusion, the proposed SNU-PS maintains good performance under small sample size, thus has a good applicability in various change detection tasks.Keywords：land cover;change detection;deep learning;small sample;sample imbalance;semantic segmentation network;Siamese network;posterior probability1526|2648|4<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2026-04-08
- Abstract：Remote sensing change detection can provide information on land surface change, which is important for studying man-nature interactions and facilitating sustainable development. With the advancement of remote sensing imaging technology and the rapid development of computer technology, extensive remote sensing images with various modes and spectral, spatial, and temporal resolutions have been collected, enabling the development of massive remote sensing change detection methods based on deep learning and their successful application in a wide range of fields.Unlike previous reviews, this work examines remote sensing change detection based on deep learning from the perspectives of bibliometric analysis, research scale, and critical problem exploration to provide reference materials for future remote sensing change detection research. The definition and importance of remote sensing change detection as well as the motivation for this review are briefly presented in the introduction. The literature structure and research hotspot information of existing research, such as the number of publications, distribution of journals and institutions, main researchers, common data sources, network model, and application field information, are clarified in the second section, which is combined with bibliometric analysis. In the third section, focus is on deep learning-based remote sensing change detection algorithms, which are categorized and presented on three scales: pixel, object, and scene. How to extract pixels, objects, and scenes from remote sensing images as well as how to perform network analysis are also explained. In the fourth section, the limitations of deep learning-based remote sensing change detection are covered, and the most recent research are presented to address these issues as well as future development possibilities. Next, a segment dedicated to the finale.The bibliometric analysis reveals deep learning-based change detection has progressed rapidly in the last three years, with fruitful research results and domestic institutional scholars dominating. High-resolution images and CNN are the most used data sources and network model, and extensive land use/coverage and building change detection are hot application fields. As for methods, different research scales respond to varied data features and network model structures. The object and scene technique have advantages, and they face similar issues, which are summarized below. First is the problem of detecting changes using multimodal remote sensing data. To address this, adversarial training, attention mechanisms, and feature deep fusion methods based on feature space transformation appear promising. Multimodal data fusion and other multimodal learning approaches are among the future’s emerging directions. Second, change detection under small sample and imbalanced sample settings is difficult. Semi-supervised schemes must be improved to address the problem of small sample size, and self-supervised methods are predicted to become a research hotspot. The oversampling technique and ensemble learning in deep learning models provide a new path for unbalanced samples. The third issue is obtaining diversified change information. Semantic change detection, which obtains extensive information on change types, and Transformer for time series change detection, which obtains long-term change information, are the future trends. Furthermore, deep learning-based change detection requires advances in gathering dynamic information such as time and seasonal pattern of change.This work systematically compiles and reviews the research status and progress of deep learning-based remote sensing image change detection. Multimodal heterogeneous change detection, semantic change detection, and time series change detection are future prospects as application needs and data diversity grow. In the areas of resources, the environment, and disaster relief, practical uses of existing knowledge are few. Continuously extending the in-depth study of new technologies and methods is required as is promoting wide, in-depth remote sensing change detection research and application.Keywords：remote sensing;change detection;deep learning;bibliometric;methods classification;challenges and prospects;review5787|13457|21<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2026-04-08
- Abstract：In the past decades, the effects of global climate change and the increase of human activities have remarkably increased the demand for remote sensing monitoring. Moreover, with the accumulation of remote sensing data from multiple platforms and multiple sensors, the quantity and quality of multitemporal images have substantially improved. Multitemporal remote sensing images Change Detection (CD) is a processing and analysis technology that aims to automatically detect, identify, and describe changes occurring in the same geographical area at different times. With the advancement of remote sensing and Artificial Intelligence (AI) technology, traditional data-driven and modal CD methods are evolving toward data-model-knowledge jointly driven direction to solve the land surface spatio-temporal CD problem in a variety of application fields in a more automatic, refined, and intelligent manner. This paper first summarizes existing problems in multitemporal remote sensing CD by analyzing the use of homogeneous and heterogenous data sources, developments from traditional to intelligent CD models, and challenges from theoretical to practical CD applications. Optical image CD is taken as an example, and the evolution of CD technology in the era of AI is examined, which can be summarized as three periods of data-driven CD, model-driven CD, and data-model-knowledge driven CD. Then, the characteristics and problems of each periods are discussed. Furthermore, for each of the three aspects (unsupervised, supervised, and weakly supervised), the characteristics and trends in the development of traditional to cutting-edge CD techniques are discussed. In the future, one can focus on breaking through key issues such as the physical interpretability, generalization, and transferability of the CD models as well as their successful implementation in cross-data, cross-scene, and cross-domain applications.Keywords：remote sensing;change detection;multi-temporal analysis;artificial intelligence;machine learning;deep learning8622|6739|25<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2026-04-08
