Your Location：Home>Browse special issues>Deep learning and intelligent interpretation of remote sensing
Deep learning and intelligent interpretation of remote sensing
Theme Keywords： remote sensing imageremote sensingdeep learningsemantic segmentationtarget detection
- The Paper
- Abstract：The automatic precise registration between optical and Synthetic Aperture Radar (SAR) imagery remains a major challenge in remote sensing due to fundamental differences in their imaging mechanisms. The inherent modality gap manifests as substantial radiometric discrepancies (speckle noise vs. photometric consistency) and geometric distortions (side-looking geometry vs. nadir projection), which introduce critical obstacles for conventional feature matching approaches. While existing deep learning-based methods have progressed in extracting deep features, most architectures inadequately address two crucial aspects: multiscale feature fusion across different imaging characteristics and cross-modal invariant feature representation. This insufficiency leads to compromised robustness in complex geographical scenarios. To address this concern, we propose a robust matching method based on deep feature reconstruction for optical and SAR images. Our method features a pseudo-Siamese network that integrates multiscale deep features and image reconstruction. First, a multiscale feature extraction architecture efficiently obtains multiscale deep features at the pixel level. This architecture allows the network to capture detailed information from various scales, which is crucial for understanding the complex patterns present in remote sensing images. Second, a pseudo-SAR translation branch for optical images is designed to reconstruct images from deep features, which enhances the ability of the network to learn robust common features. This branch mimics the characteristics of SAR images. This feature enables the network to find shared features between the two image types more effectively. Through this translation process, the network learns to focus on the essential elements that are common to optical and SAR images, which improves the matching accuracy. Finally, a joint loss function based on multilayer feature matching similarity and reconstructed image average error is constructed for robust matching. This loss function ensures that the network not only matches features accurately across different layers but also maintains a high degree of fidelity in reconstructing the original images. By combining the two aspects, the network can achieve a balance between feature similarity and image reconstruction quality, which leads to more reliable matching results. Experiments on two remote sensing image datasets with different resolutions and diverse terrain scenes (urban, suburban, desert, mountain, and water) show that our method outperforms several state-of-the-art matching methods in correct matching rate. The proposed method demonstrates superior performance in various environments, which indicates its versatility and effectiveness in real-world applications. The ability to handle different resolutions and terrain types is particularly important for practical remote sensing tasks, where conditions can vary widely.This research advances cross-modal image analysis in two aspects: (1) a new paradigm combining feature learning with cross-domain translation is provided, and (2) practical solutions for SAR-optical registration in challenging environments are proposed.Keywords：remote sensing;optical image;SAR image;image matching;deep learning839|3097|0
<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2025-10-13 - Abstract：The Sea Surface Temperature (SST) is an important indicator for studying ocean dynamics, ocean-atmosphere interaction, and climate change, which is closely related to multiple marine environmental factors, such as ocean currents, salinity, and nutrient distribution, collectively affecting the balance and evolution of marine ecosystems. Although the traditional SST acquisition methods are precise, they are limited by the number and coverage of sampling points, making it difficult to meet the requirements of large-scale and high-resolution ocean research. Satellite remote sensing data can cover global waters with high update frequency and is widely used in ocean research. However, during the collection process of satellite remote sensing data, SST data are often missing due to factors such as weather conditions, satellite scanning orbit range, and satellite sensor operation failures, which limits the use of data to some extent. Therefore, precise reconstruction of missing data in satellite remote sensing SST data to obtain high-quality and fully covered SST datasets is of great significance for ocean research. This study incorporates an Inception module into a deep interpolation convolutional autoencoder (DINCAE) and proposes the improved DINCAE (I-DINCAE) model used for data reconstruction of SST products with the FY-3C satellite in the South China Sea. The I-DINCAE is used to reconstruct the missing SST data in the South China Sea from 2014 to 2020, and the reconstruction accuracy of the DINCAE and I-DINCAE models is compared and analyzed. To further improve the accuracy of SST data, deep neural networks (DNNs) are used to calibrate satellite data in combination with the measured data, thereby optimizing the quality of the SST dataset. Finally, based on the corrected SST data, spatiotemporal variation analysis is conducted to reveal the characteristics of SST changes. At the same time, combined with many years of measured data, DNN model is used to calibrate the reconstructed temperature data of the new model. A dataset of 11,993 independent measured data points is used for testing. Results show that the RMSE, MAE, and R² of the reconstructed SST and measured SST are 1.27 ℃, 0.96 ℃, and 0.84, which decreased to 0.57 ℃, 0.43 ℃, and 0.92 after the DNN model correction, respectively. Based on the corrected SST data, the spatiotemporal distribution and variation characteristics of SST in the South China Sea at monthly and quarterly scales are analyzed from two dimensions of time and space. Results show that on the seasonal scale, the SST of the South China Sea has obvious variation characteristics. This shows that the SST reaches the highest value in the summer, and the SST decreases to the lowest value in the winter. On the monthly scale, the SST variation in the South China Sea presents a sine (cosine) wave form, with SST usually reaching a maximum value in June and a minimum value in January. This study not only reveals the uniqueness of the marine environment in the South China Sea but also provides an important basis for understanding the marine ecosystem and climate change in the South China Sea.Keywords：sea surface temperature;data reconstruction;deep learning;FY-3C;spatio-temporal variation797|3820|0<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2025-11-03
- Abstract：Bi-temporal remote sensing image change detection stands as a prevalent direction in the realm of intellectual interpretation and applied research of remote sensing imagery. It aims to acquire information regarding changes in land cover types or geophysical attributes within a monitored area over a specified time span, according to practical application requirements. Over the past few years, remote sensing image change detection techniques have undergone a rapid evolution and upgrading, fueled by the synergetic forces of the ever-growing remote sensing big data (especially the proliferation of very-high-resolution remote sensing images) and the revolutionary advancements in deep learning. In this paper, we delve into and analyze the existing popular algorithms for common change detection tasks using bi-temporal very-high-resolution remote sensing images, encompassing binary change detection that aims to identify the presence or absence of changes, semantic change detection that delves deeper into the semantic categories of the changed areas, building damage assessment that applies to natural disasters, and change captioning that focuses on generating meaningful descriptions of the detected changes using natural language. Finally, we present an outlook on the pivotal research trends in remote sensing image change detection and highlight lingering open issues under the current developmental trajectory, with the intention of offering some valuable insights and perspectives for future research endeavors in this field.Through this review, we raise two observations as follows. 1) Deep learning solutions for the common change detection tasks in remote sensing have been continuously evolving, with many innovative algorithms proposed; 2) The advanced capabilities of current vision and language foundation models have subtly permeated into this field. In fact, the remote sensing domain itself has witnessed a highly encouraging development trend of foundation models. On this basis, remote sensing change detection tends to embark on new development opportunities, and would face a reshaping of its technological landscape in the future.Though, researchers have to spend efforts in developing specialized solutions to (1) enhance the reliability of change detection in complex scenes. At present, even for the relatively mature fully supervised binary change detection, the detection results often show a complete loss of changed entities when faced with some complex scenarios. In semantic change detection, there may even be a phenomenon where the segmented changed semantics are not consistent with the binary change detection results. (2) reduce the dependence on bi-temporal image registration. This requires new algorithms to be developed to adaptively handle small misalignments and deformations in the spatial positions of corresponding objects in bi-temporal images and complete change detection in this case. (3) improve the practicality of multi-modal change detection. This may ask for a comprehensive framework that integrates multi-modal information extraction, feature registration and fusion, and change detection. Meanwhile, semi-supervised or weakly supervised learning methods are expected to become a research focus due to the annotation difficulty of heterogeneous remote sensing data. Besides, it is also believed that the synergy between image modality and language modality has broad prospects for future research in remote sensing change detection, owing to the increasing advancement of large language models as well as vision and remote sensing foundation models.Keywords：very-high-resolution (VHR) remote sensing images;bi-temporal images;deep learning;change detection;literature review3875|9915|2<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2025-09-01
- Abstract：High spatiotemporal resolution snow depth data are crucial for hydrological modeling and disaster forecasting. Currently, high temporal resolution snow depth data are typically derived from passive microwave measurements, which has not been able to meet the needs of regional hydrological and disaster studies due to the low spatial resolution based only on passive microwave data. On the basis of passive microwave brightness temperature data and combined with high-resolution optical remote sensing data, this paper aims to develop a high-precision downscale snow depth inversion algorithm to provide high spatial and temporal resolution snow depth data for regional-scale hydrology and climate research.This study proposes a downscaling snow depth retrieval algorithm based on multi-source remote sensing data such as passive microwave and optical, coupled with a deep learning model (Feature Tokenizer + Transformer, FT-Transformer) and a Snow Microwave Radiative Transfer (SMRT) model. Deep learning is used to map the complex nonlinear relationship between features such as AMSR 2 brightness temperature difference (TBD), Snow Cover Days (SCD), Snow Cover Fraction (SCF), and snow depth. Considering the influence of the physical properties of snow, the coupled SMRT is fitted with the Effective Snow Grain size (ESG) to characterize the spatiotemporal dynamic snow properties, and it is input into the deep learning model to achieve downscale inversion of snow depth.This algorithm was used to obtain downscaled snow depth data at 500 m spatial resolution in northern Xinjiang. Model training and validation were conducted using observed data from 39 stations in northern Xinjiang. The validation results revealed that SCD can effectively represent the snow accumulation process. Independent validation showed an 18% improvement in RMSE, indicating enhanced spatial generalization capability of the model. The inclusion of the ESG feature significantly improved the overall accuracy of the deep learning-based downscaled snow depth retrieval, resulting in an RMSE of 6.82 cm. This represents a 15% improvement compared with the model without the ESG feature. Additionally, the inclusion of the ESG feature greatly mitigated the underestimation of deep snow (>40 cm), leading to a 35% improvement in RMSE for such conditions. Furthermore, the time series analysis shows that ESG is consistent with the trend of measured snow depth, thereby constraining and stabilizing the output of the FT-Transformer model. Finally, compared with existing snow depth products such as AMSR2, ERA5-Land, and SDDsd, the downscaled snow depth data from this study exhibited superior validation accuracy, with an RMSE of 6.51 cm. The spatial distribution of snow depth was also refined, particularly capturing the complex snow depth heterogeneity in the mountainous regions of northern Xinjiang.This study explored the feasibility of combining the SMRT model with deep learning for downscaled snow depth retrieval. It obtained a downscaled snow depth product with high accuracy performance in northern Xinjiang, providing a reliable snow depth inversion method with high spatial and temporal resolution for hydrological modeling and disaster forecasting.Keywords：remote sensing;snow depth;Downscaling algorithm;deep learning;SMRT;AMSR 2;snow cover days1157|3761|0<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2025-07-01
- Abstract：With the recent development of deep learning, remote sensing technology, and other related disciplines, research on tropical cyclone intensity estimation has made rapid progress. The data sources used have gradually expanded from single-channel data to infrared, water vapor, microwave, and other multichannel data. Intensity estimation methods have also developed from subjective estimation methods of manual feature extraction to objective estimation methods that rely on automation models with deep learning. This paper systematically summarizes and comments on the current research progress of tropical cyclone intensity estimation based on deep learning and briefly sums up the data sources and best track datasets used in intensity estimation algorithms based on deep learning. For future research, on the one hand, tropical cyclone intensity estimation methods should adapt to the development status of remote sensing big data, new theories and methods must be introduced in the field of deep learning, and multisource data should be comprehensively used to improve accuracy and generalization ability; on the other hand, attention should be paid to the meteorological characteristics and mechanism of tropical cyclones to improve the existing methods. In the future, under the background of big data, deep learning methods are expected to have a breakthrough in the field of tropical cyclone intensity estimation.Keywords：tropical cyclone;deep learning;intensity estimation;convolutional network;multichannel data1051|3808|0<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2026-01-05
- Abstract：Deep Convolutional Neural Networks (DCNNs) have been increasingly applied in remote sensing crop recognition due to their “end-to-end” advantages and efficient extraction of shallow shape details and deep semantic features. However, deep learning models require a large number of labeled samples, which are time-consuming, labor-intensive, and costly to obtain, limiting the 2016—2020 period. The U-net model based on CDL training can be popularized and applied in the United States. First, the overall accuracy of time generalization in the three test areas in the United States from 2016 to 2020 is more than 80%, and the recognition accuracy of corn is higher than that of soybeans. Deep learning models have good transferability in space. Second, for autumn grain in Heihe City, the average recognition accuracy of corn for many years is 3% higher than that of soybean. This is because the corn planting plot is more regular and the planting scale is higher than that of soybean; the overall accuracy of autumn grain identification in a single year is between 69% and 79%. The year classification model is better than the single-year classification model, which may be because the representativeness of the training samples is enhanced with the increase of the number of labeled samples, and the difference in autumn grain planting between China and the United States can be compensated by the expansion of the number of samples. However, the model is migrated to the Heihe region of China. The accuracy of the models is lower than that of the continental United States, which is due to the inconsistency of remote sensing response characteristics due to differences in intercontinental climate and crop planting habits. These, in turn, reduce the generalization performance of the model. The DCNN model is better than random forest algorithm because of the training process driven by big data. The principle of transferring the basic trained crop classification model to map crop distribution timely and accurately has broad prospects for application across a large scale of time and space. The consistency of remote sensing features and phenology of the crops of the test area compared to the training data are fundamental factors that must be carefully considered, as these determine the success of high crop mapping performance. Therefore, it is essential to analyze the prerequisites when transferring the model to other places.Keywords：remote sensing;transfer learning;CDL;time-space generalization;soybeans;maize666|4398|3<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-10-30
- Abstract：Hyperspectral images have abundant spatial and spectral information. Numerous hyperspectral classification algorithms focus on the extraction and maximization of spatial and spectral information. Deep feature extraction networks generally extract spectral-spatial features using single-branch serial or double-branch parallel structures. However, single-branch structures may lead to mutual interference between features of spectral and spatial dimensions, and double-branch parallel structures tend to ignore the correlation between spatial and spectral features. This paper proposes a three-branch grouped spatial-spectral attention network (TGSSAN) to consider the differences and correlations between spatial and spectral features. TGSSAN can extract independent spectral-spatial features while preserving their correlation.This paper proposes the TGSSAN, which has three parallel branches (i.e., spectral, spatial, and spectral-spatial branches). These branches can separately extract spectral, spatial, and spatial-spectral features. Different attention blocks are designed in three branches to enhance the discriminative capability of features. In particular, a grouped spatial-spectral attention mechanism is proposed in the spectral-spatial branch to obtain spatial and spectral attention simultaneously. Finally, three branch features are fused for classification.In the experiment, the proposed TGSSAN algorithm is compared with some advanced deep learning algorithms, such as SSRN, FDSSC, DBMA, DBDA, HResNetAM, and A2S2KResNet. The performance of different algorithms is evaluated on five hyperspectral data sets. Experimental results show that the proposed algorithm achieved superior classification performance on IP, PU, SA, HU, and HHK datasets. In particular, the proposed algorithm achieves higher classification accuracy despite limited training samples compared with the existing advanced algorithms.The TGSSAN method proposed in this paper improves the shortcomings of the single-branch serial and double-branch parallel structures for continuous extraction of spectral-spatial features, which can effectively extract image spectral-spatial feature information. The three attention blocks designed in this paper namely, spectral, grouped spatial-spectral, and spatial attention modules, can effectively enhance the feature discrimination capability and further improve the classification performance.Keywords：hyperspectral image classification;attention mechanism;three-branch network;deep network769|2073|1<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-02-29
- Abstract：Hyperspectral Images (HSIs) contain abundant spectral information of ground objects through dozens or even hundreds of contiguous narrow bands with a high spatial resolution. However, HSIs are plagued by strong correlation between high-dimensional bands, which increases the difficulty in processing and applications of HSIs. Therefore, dimensionality reduction is one of the important steps of hyperspectral preprocessing. Band selection can effectively preserve the spectral significance of HSIs; thus, it is broadly used for dimensionality reduction. Unfortunately, the existing hyperspectral band selection methods typically consider inter-band relationships in a linear perspective while only partially focusing on multiscale information and demonstrating susceptibility to noise, resulting in poor performance of the subset of bands selected by existing methods. This paper proposes a multifeature deep subspace clustering for HSI band selection to overcome the above problems.MFDSC embeds the self-expression layer into the autoencoder to learn subspace self-expression coefficients, which considers the interaction of spatial and spectral information and explores the inter-band relationship with a nonlinear perspective. In addition, this paper couples the spatial-spectral attention and multifeature extraction modules with DSC to further reduce the interference of outliers and improve the learning capability of latent representation, thus enhancing the accuracy of self-expression coefficients. MFDSC starts with a spatial–spectral attention module to reweight the HSI to suppress useless information such as noise. Afterward, MFDSC uses convolution kernels of different sizes to extract features at different scales for encoding. Then, MFDSC learns the subspace coefficient matrix through the self-expression layer and reconstructs the original HSI using the decoder. Finally, the subspace coefficient matrix is partitioned using spectral clustering, and the bands closest to the cluster center in each class are calculated. These bands are the final results of band selection.In this paper, the proposed method is compared with the five state-of-the-art methods in a variety of experiments on three hyperspectral datasets (i.e., Indian Pines, PaviaU, and YRD datasets). The support vector machine, which adopts radial basis function as kernels, is employed as the classifier. Experimental results demonstrate that the proposed method can attain better performances than the comparison methods. Superior results are obtained when the number of bands reaches a certain number instead of using all bands. In addition, the computational efficiency of MFDSC is acceptable and significantly faster than that of DARecNet, which is a deep learning-based method.MFDSC considers the interference of noise and outliers on the self-expression performance of subspace clustering. Meanwhile, MFDSC nonlinearly learns the latent representation of data at different scales without deepening the network depth based on multiscale autoencoders. Thus, MFDSC can select a representative subset of bands and reduce the difficulty of subsequent applications.Keywords：hyperspectral remote sensing;dimensionality reduction;band selection;multi-feature;deep subspace clustering808|3038|1<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-02-29
- Abstract：Hyperspectral Anomaly Detection(HAD) is one of the most critical topic in hyperspectral remote sensing and has been extensively addressed in the literature over the past decade. Among them, Low-Rank Representation(LRR) models are widely used owing to their powerful separation ability for the background and targets. But their applications in practical situations still remain limited due to the extreme dependence on manual parameter selection and relatively poor generalization ability. To this end, this paper combines the LRR model with deep learning techniques to propose a new underlying network for HAD, called LRR-Net. This method efficiently solves the LRR model with the help of the Alternating Direction Method of Multipliers (ADMM) optimizer, and incorporates the solution as a priori knowledge into the deep network to guide the optimization of parameters, providing a theoretical basis for deep networks. In addition, LRR-Net converts a series of regularized parameters into learnable network parameters in an end-to-end manner, thus avoiding manual tuning of parameters. Experimental results obtained from publicly available datasets and our datasets demonstrate that the LRR-Net method outperforms many state-of-the-art model-based and deep-based algorithms of hyperspectral anomaly detection. Overall, deep learning networks are powerful in learning and are robust compared to traditional models in processing datasets with different complexity. However, despite the strong fitting ability of deep learning data, the necessary prior information is lacking, which often makes the algorithm fall into the local optima, which leads to the failure of deep learning to guarantee the stability of HAD results. The model-based algorithm can better make up for this defect, which can often get better results by improving the separability between the background and the target. Nonetheless, these LRR-based methods are unable to effectively suppress background noise due to their limited representation power, such as shadows, trees, and edges in complex scenes, with relatively large volatility in detection effects. The LRR-Net presented in this paper combines the advantages of the above two methods, and the experimental results of four typical scenarios show that the search of the optimal parameters in the neural network can effectively solve the HAD problem in an adaptive way, which is more physically meaningful.Keywords：hyperspectral remote sensing image;anomaly detection;deep unfolding;Low-Rank Representation (LRR);Alternating Direction Multiplier Method (ADMM)1275|3450|4<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-02-29
- Abstract：Currently, combining remote sensing imagery with deep learning is a growing trend in individual tree crown detection. RGB image is the most commonly used data type in detection. However, given that the color and texture of the tree crowns are generally close, distinguishing the crowns of different individuals by using only the color and texture information of RGB image in areas with high density of crowns is difficult. In this study, the elevation information is superimposed to improve the accuracy of individual tree crown detection by using RGB images. In the experiment, RGB image (color image) and DSM (digital surface model) were used as data sources, and band combination and double-source detection network model were used to combine RGB and DSM for individual tree crown detection. In the former method, band combination of RGB and DSM was conducted to generate GBD, RGD, and RBD images, and the three kinds of images were used for network training and testing. In the latter method, RGB and DSM were input into the double-source detection network model, and the detection results were obtained. FPN-Faster-R-CNN and Yolov3 were used for experiments in this study. Compared with RGB scheme as the control scheme (which uses only the color and texture information of ground objects for individual tree crown detection), the average accuracy of FPN-Faster-R-CNN in the GBD scheme, RBD scheme, and double-source detection network scheme increased by 3.36%, 2.45%, and 7.77%, respectively; it decreased by 0.17% in the RGD scheme. The average accuracy of Yolov3 in the GBD scheme, RBD scheme, and double-source detection network scheme increased by 0.72%, 0.14%, and 5.71%, respectively; it decreased by 0.98% in the RGD scheme. Under the two networks, the double-source detection network scheme achieved the best detection result in each scheme. Compared with the RGB scheme, the improvement in average accuracy of double-source detection network scheme showed a rising trend with the increase in forest density. Comparative analysis of the experimental results shows that proper combination and utilization of the color, texture, and elevation information of the ground objects is beneficial to improve the performance in the urban individual tree crown detection task based on deep learning.Keywords：remote sensing;individual tree crown detection;deep learning;urban;elevation;color image;UAV2483|4469|1<HTML><L-PDF><Enhanced-PDF><Meta-XML>Updated：2024-01-22
