Issue Ebookcover storyIssue CoverInteractive Contents
Latest Issue
Online FirstList of IssuesDocument Types
2026
2025
2024
Issue 12
Issue 11
Issue 10
Issue 9
Issue 7
Issue 6
Issue 5
Issue 4
Issue 3
Issue 2
Issue 1
2023
2022
2021
2020
2019
2018
2017
Abstract：Since the 1970s, China has made remarkable achievements in the field of remote-sensing satellite systems. It is moving toward the era of intelligent and networked giant remote-sensing satellite constellations. The realization of networking and intelligence of giant remote sensing satellite constellations is not a one-off process. The development of the national remote sensing satellite industry in China hinges on the current opportunity period of historical development, careful resolution of technical challenges, direction of future technological breakthroughs, and preparations for the era of remote-sensing satellite constellation. This paper reviews the development history of China’s remote-sensing satellite systems and analyzes the technical challenges faced by the development of a giant remote-sensing constellation. First, owing to the massive data processing problem in the remote-sensing constellation, this paper focuses on on-orbit data preprocessing and intelligent information extraction methods, analyzes the problems through on-board intelligent processing, and proposes an on-board processing mode supported by the ground. Second, complex problems in the management of tasks in a large constellation and the current status of on-board autonomous mission planning capabilities are analyzed, and a digital twin-based integrated intelligent mission management and distributed intelligent collaboration model is proposed.Overall, China is currently in the stage of limited intelligent processing, particularly on-board and ground-based processing, and is entering the stage at which it can perform simple mission control on board and ground-based control. Driven by advancements in intelligent processing capabilities on board, the development of remote-sensing constellation systems toward the ideal “organic giant constellation” can be divided into three stages, that is, giant remote-sensing constellation 1.0-3.0. The integrated design and technical laying out of the intelligent giant remote-sensing constellation system as a whole, combining space and ground elements, must be carried out as early as possible. Key technical research on highly efficient and reliable intersatellite communication technologies should be undertaken. Furthermore, the performance of algorithms interpreting remote sensing data based on random sample data, digital twin-based integrated intelligent mission planning technology for space and ground, on-orbit unsupervised continuous self-learning theories, methods for the intelligent processing of remote sensing satellite data, and event-driven rapid task allocation technologies for large-scale constellations should be evaluated and optimized. These efforts can promote the development of national remote-sensing satellites for a new era of intelligence.Keywords：giant remote sensing constellation;development history;Intelligent;networked;application technology;satellite constellation mission control;data processing;development plan7654|9919|13<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：In recent years, remote sensing intelligent interpretation technologies have advanced rapidly, but most established models are task oriented. Therefore, generalizing them to different tasks is difficult, and considerable amounts of resources are wasted. The foundation model is a straightforward approach that has recently attracted considerable interest in the field of remote sensing. Although many works have achieved remarkable results in some tasks for perception recognition and cognitive prediction by using remote sensing single-temporal or multitemporal data, a comprehensive review that provides a systematic overview of the remote sensing foundation model is lacking. Thus, this paper begins by summarizing developments in research on existing remote sensing foundation models from the perspectives of data, methods, and applications. Then, after analyzing the current situation’s limits, we proposed a novel general predictive foundation model. Finally, some essential research areas were highlighted, and past achievements were linked with the future possibilities of remote sensing foundation model.Existing remote sensing foundation models were categorized into three groups according to the data types used (single-temporal/multitemporal) and the tasks involved (perceptual recognition/cognitive prediction): the foundation model of perceptual recognition based on single-temporal data, the foundation model of perceptual recognition based on multitemporal data, and the foundation model of cognitive prediction based on multitemporal data. According to the different self-supervised learning methods adopted, we divided the existing foundation models of perceptual recognition based on single-temporal data into those based on contrastive learning and those based on generative learning. According to the number of tasks, the foundation model of perceptual recognition based on multitemporal data was divided into a single-task-oriented foundation model and a multitask-oriented foundation model. According to different model architectures, the cognitive prediction foundation models based on multitemporal data were divided into transformer-based and graph network-based foundation models. In accordance with the aforementioned categorization, we described the current state of each type of remote sensing foundation models and summarized their data, methods, and application restrictions.Based on the summary and analysis of the existing remote sensing foundation models, a novel general predictive foundation model assumption was proposed. The information pipeline for multidomain or temporal data input and multitime or spatial scale task output can be opened up by extracting stable and generalized time-series hyper-pixel features. This approach enabled the accurate cognitive prediction of the future state. Tens of millions of multiplatform, multitype, multimodal, and multitemporal data were included. By combining the benefits of the transformer model and the graph network, a new foundation model architecture was created, which increased the model’s capacity and enhanced generalization while predicting multitarget interactions in large remote sensing scenes over the long term. In terms of application, the general predictive foundation model can be applied to diverse cognitive prediction tasks with multiple spatial and time scales. Under this assumption, we proposed four exploratory directions: multidomain time series data representation, stable feature extraction, object-environment interaction modeling, and multitask interaction reasoning, aiming to provide a reference for researchers exploring remote sensing foundation models.In general, foundation models with generalization ability are crucial to development of remote sensing intelligent interpretation. We provided an overview of current advances in this field by collating the current state of research on remote sensing foundation models. By analyzing the limitations of current remote sensing foundation models in terms of data, methods, and applications, we proposed a novel general predictive foundation model assumption and further clarified four exploratory directions that urgently need breakthroughs under this idea. The follow-up work will make specific and important technological breakthroughs in multidomain time series data representation, stable feature extraction, object-environment interaction modeling, and multitask interaction reasoning. We explored a general remote sensing foundation model integrating perception recognition and cognitive prediction into a single architecture.Keywords：remote sensing intelligent interpretation;remote sensing foundation models;general prediction;multi temporal data;multi-task6715|11747|13<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：Moving object detection in spaceborne optical remote sensing images (particularly for satellite videos) is a critical technique for interpreting remote sensing data. It has numerous applications, including surveillance, environmental monitoring, and military intelligence. Moving object detection in satellite videos aims to locate and classify moving targets accurately, such as moving vehicles, ships, trains, and airplanes. The advancements in spaceborne optical remote sensing satellite technology and the emergence of deep learning techniques allow the traditional model-driven methods for moving object detection to evolve toward data-driven deep learning methods and achieve high reliability, efficiency, and performance. This study introduces the current status of optical remote sensing satellite systems and summarizes model-driven and data-driven approaches for optical remote sensing moving target detection. First, the current state of video satellites is presented. Spaceborne optical video satellites hold great potential for advancing our ability to detect and monitor dynamic phenomena across various domains, thereby forking the data foundation for moving object detection. Second, the development of model-driven methods for moving object detection is analyzed, including frame differencing-based, optical flow-based, and background subtraction-based methods. However, these methods rely heavily on handcrafted features and prior knowledge to identify moving objects. Thus, they often struggle with complex backgrounds, varying lighting conditions, and occlusions. By contrast, with their powerful feature learning capabilities, data-driven deep learning-based methods have brought significant progress in moving object detection in satellite videos. Third, we present the development of data-driven deep learning-based methods. We also summarize the supervised and unsupervised deep learning-based methods for moving object detection in satellite videos, and their trends are discussed. In deep learning-based methods, extracting spatiotemporal information is crucial for efficient and effective moving object detection in satellite videos. Furthermore, the methods of alleviating annotation costs and addressing massive data are underdeveloped and need further exploration. Then, we introduce the datasets, evaluation criteria, and state-of-the-art experimental results. We evaluate nine representative model-driven and data-driven methods (supervised and unsupervised) for moving object detection in satellite videos on the VISO car dataset. We conclude from the results that deep learning methods with superior performance can adapt effectively to diverse environmental conditions and target characteristics. The unsupervised deep learning methods are underdeveloped and need considerable focus. Moreover, the combination of model-driven and data-driven methods has shown great potential for moving object detection, which leads to a new development trend for the community. Lastly, on the basis of the summary above and the discussion, we conclude the future trends of moving object detection in satellite videos. On the one hand, the development of satellite constellations can open up new possibilities for collaborative and distributive multisatellite and multisensor systems. On the other hand, the potential to merge feature representation, detection, and tracking into an end-to-end network with multitask learning needs further exploration. Weakly supervised and unsupervised methods are worth further development and research to alleviate the burden of annotation cost. The ability to tackle massive data and the construction of an end-to-end detection and recognition network for high-resolution satellite videos are also worth exploring.Keywords：space-based optical remote sensing;moving object detection;satellite videos;model-driven;data-driven;neural network;spatio-temporal information;unsupervised learning4092|3401|4<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Reviews
Abstract：Objects in remote sensing images are detected by determining the positions and correct categories of objects. Given that this approach has broad application prospects and plays a vital role in many fields, the purpose of this paper is to study its issues, such as insufficient feature extraction, difficultly in locating objects, and confusing classification, when applied to small samples.The specific contributions of this paper are as follows: (1) A collaborative attention module is proposed, which includes designed background attenuating attention and spatial perception attention. The network focuses on key information related to object positioning on the basis of rich background and object feature information, and an RPN network generates improved regional suggestion boxes, reduces the probability of missing targets, and improves the positioning performance of the model for small-sample categories. (2) A contrastive learning branch is designed. Based on the design of the contrast loss function, feature learning is gradually transferred to classifier learning through a joint training strategy, and classification accuracy is improved. (3) A few-shot object detection model based on the fine-tuning transfer learning paradigm is designed, which is divided into basic training and fine-tuning stages. In the basic training stage, the model is trained to learn class-independent parameters with sufficient base class samples. In the fine-tuning stage, a small sample data set is used to enable the target detection model to adapt to specific objects and improve its detection performance. On the basis of the TFA, this article verifies the effectiveness of the proposed algorithm on the remote sensing data sets NWPU, VHR-10, and DIOR. To demonstrate the superiority of our method to other methods for few-shot object detection, we compared our method with TFA, FR, FSODM, DeFRCN, and MFDC. Results show that the proposed algorithm considerably improved the mean average accuracy on NWPU, VHR-10, and DIOR data sets, demonstrating the superiority of the proposed algorithm over the above algorithms. In summary, our method can achieve exceptional detection results from small remote sensing images, exhibiting effectiveness and superiority over other detection methods. We hope that our method can promote further research into few-shot object detection and contribute to the development of this field.Keywords：object detection;few-shot learning;remote sensing image;attention mechanism;contrastive learning2547|4266|4<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：Object detection in remote sensing images is essential research contents of visual image recognition tasks. However, in the remote sensing images of ships, a ship target is small and sparsely distributed, and using a traditional Artificial Neural Network (ANN) for object detection often wastes a considerable amount of computing resources. SpikinG Neural Networks (SNNs) can be applied due to its event-driven and low-power characteristics, greatly saving energy and computing resources. However, training an SNN is difficult because of the complex dynamics and nondifferentiable pulse operation of SNN neurons. Instead, converting a trained ANN into SNN can effectively circumvent training difficulties. For a converted deep SNN, many time steps are often required to maintain its performance. Unfortunately, this process requires a substantial amount of computing resources.This paper studies the reason why a large number of time steps are required to maintain an SNN model’s performance after conversion and proposes a novel conversion method: a layer-by-layer conversion method based on fine-tuning. During conversion, the network is converted layer by layer, and the subsequent unconverted network is fine-tuned, and thus the errors accumulate layer by layer during conversion is prevented. In addition, given the rationality of hardware deployment, we propose Poisson group coding, which uses multiple Poisson coding neurons to encode input images and sends them to the network after average pooling. Compared with Poisson coding, the output of Poisson group coding is less noisy and has less impact on model performance.The fine-tuning transformation method achieves a result (96.9%, 70.3%), similar to that of YOLOv3-tiny (97.9%, 79.6%) on the SAR ship detection datasets (AIR-SARShip), and 80% of the performance of the preconversion model can be achieved by using few time steps (20 and 80 steps). The method achieves good detection performance (49.2%) on the PASCAL VOC dataset. By contrast, the performance of the conventional conversion method is inferior to that of the fine-tuning conversion method in the same number of time steps and usually requires many time steps (more than 150 time steps), achieving improved detection performance. For Poisson group coding, the impact on model performance under the same time steps decreases with increasing neurons. Performance similar to the input simulation frequency can be achieved with few time steps.The layer-by-layer conversion method based on the proposed fine-tuning method effectively adapts the SNN network for the layer-by-layer conversion of error, preventing the accumulation of error in each layer and reducing SNN. This method can improve the performance of a converted SNN and reduce the time steps. Meanwhile, Poisson group coding provides an effective input coding method for the hardware deployment of an SNN.Keywords：Spiking Neural Network (SNN);object detection;ship remote sensing images;convert ANN to SNN;Poisson group coding1090|2576|1<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：In the intelligent processing of remote sensing images, third-generation brain-inspired Spiking Neural Network (SNN) surpasses its predecessor, the second-generation artificial neural network, showing remarkable advantages in terms of high energy efficiency, high precision, and high interpretability. These advantages stem from the SNN’s characteristic features, including high energy efficiency, elevated sparsity, and remarkable bio-plausibility. The integration of these features in the SNN presents an enthralling solution to challenges faced in remote sensing image processing and holds immense potential for advancing this field further.The proposed algorithm introduces a novel approach and initially establishes a target-detection neural network, which serves as the source artificial neural network for pretraining. This source network utilizes a dynamic clipping threshold activation function to optimize its performance. Subsequently, the algorithm transforms the source network into a brain-inspired SNN by leveraging the mapping relationship between activated neurons and spiking neurons. This conversion process effectively incorporates the clipping thresholds obtained during training. By seamlessly transitioning the source network into an SNN, the algorithm ensures the preservation and enhancement of key characteristics essential for remote sensing image processing.The integration of the third-generation brain-inspired SNN into remote sensing image processing holds tremendous potential primarily due to its high precision and low energy consumption. The proposed algorithm highlights the distinctive attributes of the SNN, such as its low delay, high bionics, and ability to inherit high precision observed in a source network. These characteristics are promising indicators of SNN’s capability to considerably enhance the intelligent processing of remote sensing images. By leveraging the SNN’s high precision and bio-plausible principles, the proposed algorithm lays a robust foundation for future advancements in the field of remote sensing. The SNN’s inclusion in the image processing pipeline causes a paradigm shift, challenging traditional assumptions and unlocking new possibilities. The SNN is highly accurate in identifying and classifying targets with remarkable precision during remote sensing target detection. In conclusion, the proposed algorithm exhibits low delay and high bionics while demonstrating the high precision of a source network, demonstrating its potential to considerably improve the intelligent processing of remote sensing images and offering high accuracy, precision, interpretability; low energy consumption; and high bio-plausibility.The efficacy of the proposed method was evaluated through extensive experiments performed on two widely recognized open remote sensing datasetsSAR-Ship-Detection-Datasets (SSDD) and RSOD. Experimental results highlighted the exceptional capabilities of the proposed method in transforming the source network into a brain-inspired spiking neural network, demonstrating negligible loss in performance. Furthermore, the transformed SNN exhibited remarkable accuracy in detecting and recognizing remote sensing targets within significantly reduced time steps. The performance achieved by the transformed SNN was comparable to that of the source artificial neural network while tremendously reducing power consumption by over two orders of magnitudes. This outcome highlights the immense potential of the proposed method in revolutionizing the field of remote sensing image processing by delivering high precision and interpretability which considerably reducing energy consumption.Keywords：SAR;remote sensing image;optical remote sensing;target detection;deep learning;spiking neural networks1692|3877|0<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：At present, the deep neural network for targets fine-grained classification based on remote sensing images has been widely applied in military and civilian fields. The “black box” problem of deep learning network models still makes it difficult for people to understand the decision-making basis of the network in fine-grained classification tasks. This not only limits the possibility of deep neural networks optimizing and improving through feedback guidance, but also makes them unable to be fully trusted by humans and applied in important fields such as military and medical fields. Thus, how to carry out explainability research on its internal decision-making mechanism is the key problem for the current fine-grained algorithm to further improve the credibility of decision-making basis. This article summarized the commonly used explainable methods in existing fine-grained classification networks for remote sensing image targets. On this basis, a mathematical model for the problem of target fine-grained classification is established. It also models the fine-grained classification task of remote sensing image targets based on game competition theory, analyzes the applicability of explainable methods such as IG, SmoothGrad, and Grad CAM on the fine-grained classification network of remote sensing image targets, and proposes a scale adaptive method for analyzing the explainability of essential features of fine-grained classification of targets, that is, Hybrid Grid. The fusion algorithm of pixel level and local feature relationships is used to improve the accurate description ability of essential features of targets that support network decision-making. The typical explaining methods involved in this article were evaluated using indicators such as Average Drop, Coherence, Complexity, ADCC, and Deletion and precision loss, confirming the accuracy of the proposed method. The experimental results show that the Hybrid-Grid proposed in this paper achieves a score of 78.87 on the quantitative evaluation index of ADCC for target fine-grained classification networks, which is significantly improved compared to Score-CAM. Compared with the explanation results of SmoothGrad and Grad-CAM, our method performed the best in deletion and accuracy loss experiments, resulting in a loss of 16.92%, 1.61%, and 17.21% in the Top-1 accuracy, Top-5 accuracy, and F1 score of EFM Net, respectively. This proves that the Hybrid Grid accurately explains the essential feature of the target that contributes the most to fine-grained classification network decision-making. Based on several typical fine-grained classification networks, the feasibility and universality of the explaining method proposed in this paper have been confirmed. The explaining method proposed in this article can more accurately reveal the decision feature basis of the current target fine-grained classification network. Future work will further enhance the visualization and interpretation accuracy of the essential features of the target, assisting in improving the classification performance and decision-making credibility of fine-grained classification networks.Keywords：remote sensing image;explainability analysis method;target fine-grained classification network;explainable artificial intelligence;cooperative game theory;essential fearture2590|2801|2<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：Super-resolution reconstruction technology plays an important role in the intelligent processing of satellite remote sensing images. Existing deep learning methods for super-resolution remote sensing image reconstruction can only handle super-resolution tasks with a single scale factor, lacking generalization at the multiscale level and failing to meet the requirements of real super-resolution remote sensing image reconstruction for continuous zooming at multiple magnification levels.To address the problem of arbitrary-scale super-resolution reconstruction and enhance the quality of reconstructed real remote sensing images, this paper proposes a super-resolution reconstruction method called the Meta-RDCAN, which utilizes meta-learning and residual dense channel attention network.The proposed method employs a meta-upscale module that incorporates three functions: weight prediction, location projection, and feature mapping. The module adaptively adjusts the internal parameters of a model according to different scale factors for arbitrary-scale super-resolution reconstruction. From the perspective of extracting detailed information of local land objects in a remote sensing image, a dense residual network with an attention mechanism is used as a feature extractor, enabling the reconstructed results to possess clear and distinguishable details.Extensive experiments are conducted on various datasets, including DIV2K, AID, UCMerced, WIDS, Set5, and real remote sensing image from Macao Science Popularization Satellite. The influence of variations in spatial resolution on the super-resolution reconstruction results is analyzed, and the effectiveness of the training scheme, which involves pretraining on a general dataset followed by fine-tuning on a remote sensing dataset, is validated using the loss curve.The test results with different scale factors demonstrate that the proposed model is suitable for arbitrary-scale super-resolution reconstruction tasks in remote sensing images, having scale factors of up to 4.0. Comparative experimental results show that the improved model with added channel attention achieves enhanced performance in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural SIMilarity (SSIM), compared with the baseline model. On real remote sensing data, the reconstruction results of the proposed model achieve a PSNR of over 40 dB and an SSIM of over 0.95. The comparison based on the no-reference quality indicator NIQE confirms that the perceived quality of the super-resolution reconstruction results of the improved model surpasses that of the baseline model.The proposed method for arbitrary-scale super-resolution reconstruction is effective for remote sensing images by utilizing meta-learning and dense residual channel attention. The main contributions of this paper include two aspects. First, for the arbitrary-scale super-resolution reconstruction of remote sensing image, the meta-learning approach is employed to adaptively adjust the internal parameters of a model. This approach enables continuous integer- and non-integer-scale super-resolution reconstruction of a single remote sensing image with a single model. Second, to address issues, such as missing details and unclear edges of geographic features, in reconstruction results, a channel-attention mechanism is applied to enhance the dense residual network and improve the quality of super-resolution reconstruction results.Keywords：super-resolution reconstruction;remote sensing image;arbitrary-scale;meta-learning;residual dense network;channel attention mechanism1793|3119|0<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：Super-resolution improves the spatial resolution of remote sensing images, providing detailed information for various satellite applications. However, existing methods often generate pseudo-detail and lose true detail in reconstructed images due to insufficient supervision images. To address this issue, a progressive super-resolution method based on multilevel supervision structure (MSSR) was proposed.First, the MSSR introduced ground truth images as guides, which reduced the loss of true detail and mitigated the appearance of pseudo-detail in an output image. The MSSR network consists of several basic super-resolution components (BSRCs) and multilevel supervision. The overall super-resolution scale factor of the MSSR network can be set flexibly. The BSRCs can be increased or decreased, similar to building blocks, and BSRCs decrease with decreasing overall super-resolution scale factor and increase with increasing overall super-resolution scale factor. The scale factor of each BSRC is determined by the overall scale factor and the number of BSRCs. Second, a scale-factor-adjustable and lightweight basic super-resolution component was designed to enable the construction of multilevel supervision networks with different number of BSRCs and different scale factors, such as building blocks. The BSRC consists of a multiscale feature extraction module, a global feature extraction module and an image reconstruction module. Given that the scale factor of each BSRC should have a degree of flexibility, the network structure of each BSRC is the same except for the image reconstruction module. This approach shortens the overall training time of the network. Finally, a method of dividing super-resolution overall scale factor was proposed, and the effects of different number and different scale factors of BSRCs on the performance of multilevel supervision network were explored. For the super-resolution process with a certain scale factor, we need a method to divide the overall scale factor into each BSRC. The number of supervision increases with the number of BSRCs, and the super-resolution ill-posedness is reduced. In addition, the total number of network layers and the number of computations increase. We determined the optimal number of BSRCs and their respective super-resolution scale factors by comparing the super-resolution effects of multiple BSRC combinations. Additionally, a new remote sensing dataset containing worldwide scenes was constructed for the super-resolution task in this paper.To adequately train and test the proposed super-resolution method and existing methods, we used our datasets and two existing super-resolution datasets: the UCMerced and AID datasets. We compared our method with the state-of-the-art methods: VDSR, SRGAN, RDN, RCAN, DRN, and TransENet. The experiment results on three datasets demonstrated that our MSSR network outperformed these methods.The progressive network and multilevel supervision structure can effectively suppress super-resolution ill-posedness and reduce pseudo-detail and detail loss in super-resolution results. By further analyzing the experimental results, we found that multilevel supervision has greater performance gains in super-resolution tasks with a larger scale factor than other methods. We speculated that the multilevel supervision network exhibit improved performance in super-resolution tasks at 4× magnification. In future research, we will explore multilevel supervision networks in super-resolution tasks at large magnifications (i.e., 5× and 8×).Keywords：remote sensing image;deep learning;multi-level supervision;super-resolution;multi-scale feature extration;progressive network;parameter sharing;transfer learning2341|2939|1<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Satellite Information Intelligent Processing Technology
Abstract：The increasing occurrence of Earth Surface Anomalies (ESAs) highlights the importance of the timely and accurate detection of such events. Therefore, researchers have utilized satellite imagery for large-scale detection and developed advanced deep learning methods. However, the performance of the methods is hindered by inadequate labeled data and the complexity of semantic information in satellite imagery. In this paper, we aim to address the aforementioned problems and improve the performance of Earth surface anomaly detection. This paper reviews and summarizes developments in Earth’s surface anomaly detection and the problems that hinder the performance of existing methods. Then, we proposed a hierarchical geometry-to-semantic fusion Graph Neural Network (GNN) framework, which utilizes a single image for the detection of Earth’s surface anomalies and reduces the demand for data and time required for preprocessing and inference. Specifically, our method employs two branches for the extraction of geoentities and construction of graphs at different levels. Then, a hierarchical graph attention network was utilized to update node features and extract graph embeddings for each level. An attention-based feature fusion module then combined them to yield the graph-level feature vector of the input image, which was finally processed through a multilayer perceptron for ESA detection. Given that existing ESA datasets mainly focus on single-class detection or post-hoc analysis, which is insufficient for our research needs, we proposed ESAD, which is a composite dataset, to bridge the gap between large-scale multiclass datasets. Specifically, ESAD is composed of three publicly datasets: xBD, Multi3Net and Sichuan Landslide, and Debrisflow. The proposed method was effective and accurate for detecting ESA, outperforming many baseline methods and balancing between accuracy and efficiency. Thus, it is suitable for ESA detection, saving valuable time and resources for downstream tasks. In conclusion, we proposed a hierarchical geometry-to-semantic fusion GNN framework for ESA detection. It leverages GNN to learn high-order semantic information from satellite imagery. To address insufficiency in benchmark datasets, we created the ESAD dataset based on existing related datasets. Our method achieved a good balance between accuracy and efficiency and is suitable for ESA detection with high timeliness requirements. In future work, we will further explore more models and extend our method to on-orbit real-time ESA detection task.Keywords：remote sensing;Earth surface anomalies;hierarchical fusion;graph neural network;fast detection;attention mechanism;emergency response;satellite imagery2179|2913|2<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31Domain-adaptation algorithm for remotely sensing building changes through instance contrast learning
Abstract：Building-change detection automatically identifies changes in ground buildings in remote-sensing images acquired in the same geographic area at different times. Fully supervised change-detection algorithms require a large amount of labeled remote sensing data to make accurate predictions. Manually labeling a building change detection label is time-consuming and labor intensive because it requires a professional to compare and label two images pixel by pixel. Unsupervised domain adaptation technique is an effective means to alleviate this problem. Although the current-domain adaptation algorithm has achieved good results in building-change detection, the following problems persist: Problem 1: A class-based domain mixing strategy is applicable to a large number of categories. In building-change detection, only positive samples of the category “change” are available. Problem 2: In the current pixel-based contrast learning method, pseudo labels generated by a model must have samples with classification errors because the labels of target domains are unidentifiable. This requirement introduces large noise information during contrast training. Problem 3: The pseudo label generated by high-confidence threshold filtering does not leverage the low confidence prediction results of a teacher model. To solve the above problems, this paper proposes a case-level contrast-learning domain-adaptation algorithm for cross-domain building-change detection task.This paper proposes an instance contrast-learning domain adaptation for change detection (ICDA-CD) method for cross-domain building-change detection. The main contributions are as follows: (1) A region-level domain-mixing method is proposed, which combines data containing the buildings in a source domain and data containing buildings in a target domain on one sample simultaneously. (2) Case-level contrast learning method is proposed. In the encoder, the distance between the biphasic features of a changing building area is pulled apart. In the decoder, the distance between the features of each changing building area is narrowed. (3) A pseudo label quality estimation method is proposed. The pseudo-label quality of each pixel position is estimated by the value predicted by a teacher model, and then loss is weighted.Domain migration experiments were performed on the LEVIR-CD and S2Looking datasets, and comparison and ablation experiments were performed with advanced domain-adaptation algorithms. In the migration of the LEVIR-CD task to the S2Looking task, the proposed algorithm achieved the highest F1 and IOU of 43.91 and 28.31, respectively. In the migration of the S2Looking task to the LEVIR-CD task, the proposed algorithm achieved the highest F1 scores and IOU of 74.75 and 59.68, respectively.To solve the problem of unsupervised domain adaptive change detection algorithm across data domains, an ICDA-CD method was proposed. The accuracy of the cross-domain unsupervised domain adaptive change detection algorithm was effectively improved by using region-level domain mixing, case-level contrast learning, and pseudo label quality estimation-weighted loss.Keywords：remote sensing images;building change detection;contrast learning;domain adaptation;deep learning;pseudo label1637|2965|1<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：In recent years, interest in Synthetic Aperture Radar (SAR) ship detection has considerably grown. Its distinctive strengths position it as a pivotal player in numerous fields of research. However, the inherent characteristics of SAR images have presented a range of challenges. For instance, in contrast to optical images, SAR images have counterintuitive feature representation. Additionally, owing to the constrained number of SAR image data, achieving satisfactory results with existing methods that depend on a substantial number of annotated SAR images might be challenging.How to effectively train a high-performance SAR ship detection network with a limited quantity of SAR images should be investigated. Given that single-modality SAR detection algorithms have inherent limitations, other effective modalities that can assist the SAR modality in completing tasks are needed. For instance, in SAR image target detection, optical images can serve as supplementary data sources. A knowledge-rich model can be developed by utilizing a large volume of optical data in training with SAR data. Hence, reasonable training approaches for effectively utilizing images from SAR and optical modalities should be explored.To address these challenges, a SAR ship detection algorithm called MCMA-Net, which is based on multilevel cross-modality alignment, is proposed in this paper. The MCMA-Net enriches SAR feature representation by incorporating valuable knowledge from optical modality. First, we propose a neighborhood–global attention-based feature interaction network (NGAN), which employs a neighborhood attention mechanism that enables the local interaction of low-level features and a global self-attention mechanism that captures global context from high-level features. When the ability of global context modeling is considered, the encoding ability of local features improves, NGAN enables the network to focus on corresponding information at different levels and can promote the subsequent multilevel modality alignment. Second, we propose a multilevel modality alignment module (MLMA), which aligns features in the different hidden spaces of the two modalities from three levels. MLMA facilitates the model to acquire modality-invariant features, bridging the modality gap and realizing optical knowledge transmission. Valuable information from the optical modality can compensate for certain deficiencies in SAR images. With the aid of these two modules, we have incorporated optical superiority information by leveraging SAR’s inherent advantages, achieving an enhancement in the performance of SAR detection tasks.Our algorithm is superior to current detection algorithms. Notably, whether on public SAR image datasets or our own SAR image dataset, the MCMA-Net consistently achieves optimal detection results, which indicates the model’s stable performance and robustness. The visualization results indicate that the MCMA-Net achieves excellent detection capabilities in complex scenarios. The ablation experiments demonstrate that compared with the baseline model, our algorithm achieved a 2.7% increase in mAP on the SSDD dataset. Various experimental results have consistently validated the rationality of the MCMA-Net.Keywords：remote sensing;SAR;target detection;cross-modality;feature alignment;attention mechanism4494|4939|6<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：Optical remote sensing is a widely used technology in aerospace reconnaissance and geological exploration. Visible light images captured by this technology provide a wealth of information and have important applications in intelligence gathering, object monitoring, and situational forecasting. Considerable progress in remote sensing image object perception has been achieved, particularly in ship and airplane detection. However, technical challenges, including with large object-scale variations and numerous small objects, in remote sensing image object perception remain. Existing work has mainly focused on improving boundary box representations, and single-object detection models fail to fully exploit spatial correlation information from surrounding or similar objects. To address the inherent inefficiency of existing remote sensing image object detection algorithms that detect different objects independently, this paper proposes a novel detection framework called group object detection. By detecting the state information of a group object, our framework alleviates problems, such as insufficient perception information and poor reliability of single-object perception, generating reliable multi-object detection results. This paper introduces a concept of group objects and proposes an automated annotation scheme for group objects. By analyzing existing labels on a public dataset, the proposed scheme obtains annotated information with group object labels without manual annotation. Based on the automated annotation of group targets, a group target detection algorithm is presented, which enhances single-object detection results by utilizing the spatial constraints of group objects. Experimental results on the DOTA dataset, a widely-used remote sensing object detection benchmark, demonstrate that the proposed group target detection algorithm outperforms state-of-the-art methods.Keywords：remote sensing image;image object;group object detection;automatic annotation;DOTA;object sensing;multi object2213|3641|2<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：Multi-object tracking determines the position of an object and estimates the trajectory of objects in remote sensing satellite videos. This method has attracted considerable interest, and its application to security monitoring, motion analysis, and intelligent transportation has been explored. Compared with surveillance videos, remote sensing satellite videos contain smaller objects and a larger background, and thus the foreground object is difficult to detect. In addition, remote sensing satellite videos are extremely large, requiring massive computation and storage. Multi-object tracking in remote sensing satellite videos have high real-time requirements. Based on the mentioned problems, a multi-object tracking method for remote sensing satellite videos is proposed in this paper, which adopts tracking-by-detection paradigm. First, the backbone added a transformer that capture the global context information in the detection stage, enabling the detector to distinguish between objects and background. Then, an attention mechanism was used to enhance objects’ features, enabling the proposed method to focus on the region of objects. Finally, an extra prediction branch was added to the network to generat a high-resolution feature map, which retained the details of small objects and was beneficial to small-object detection. Owing to the small objects and occlusion in remote sensing satellite videos, the confidence of hard positive samples was quite low. In the data association stage, an association strategy was adopted, which considered high and low confidence detection simultaneously and associated detected small objects with existing trajectories. To verify the effectiveness of the proposed method, ablation and comparison experiments were carried out on the remote sensing satellite videos dataset. The proposed method achieved 63.1% MOTA and 78.0% IDF1. The proposed method showed optimal performance, which reflected its suitability for multi-object tracking in remote sensing satellite videos. The proposed method ranked second in the multi-object tracking challenge of the 2021 Gaofen Challenge. The proposed method was dedicated to solving the difficulty of small-object tracking in remote sensing satellite videos, and some helpful methods for small-object tracking were used. Experimental results showed that the proposed method can improve the performance of multi-object tracking in remote sensing satellite videos.Keywords：remote sensing;multi-object tracking;small object detection;attention;neural network;trajectory tracking3015|6144|3<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Abstract：Large-scale regional observations by remote-sensing satellites play an important role in mapping, disaster relief, and other fields. The efficiency of single-satellite observation is low, and multisatellite collaborative observation is the main means for rapidly observing large areas. To date, multisatellite collaborative observation is mostly studied on optical remote-sensing satellites, but research on the collaborative observation of SAR satellites is limited. Moreover, SAR satellites have imaging mechanisms and modes distinct from those of optical satellites, and thus the optical satellite collaborative planning method cannot be fully applied to SAR satellites. For the optimization of the performance of SAR multisatellite collaborative observation, research into SAR multisatellite collaborative regional observation technologies is crucial.First, the coverage calculation of large-scale complex areas was analyzed, and a complex area coverage calculation method that combines Gaussian projection, grid division, and geometric operations and can realize the coverage calculation of any complex area, was proposed. Then, an accurate coverage analysis was performed on the SAR strip imaging mode, and a candidate area decomposition method combining angle restriction and two-dimensional decomposition was established. Optimization efficiency was improved by reducing the optimization space through angle restriction, and complex continuous optimization problems were discretized through two-dimensional decomposition. These approaches allowed the use of genetic algorithms for optimization. Finally, an improved genetic algorithm combining greedy algorithm initialization, elite retention strategy, and cubic fitness function was formulated for regional coverage optimization. Chromosome encoding, crossover, and mutation operations were designed for optimization, an optimal retention strategy was used to improve optimization speed and stability, and the cubic fitness function was used to improve the optimization effect.This study selected four on-orbit SAR satellites, namely, GF-3 01, GF-3 02, GF-3 03, and HISEA-1, and three regional targets, namely, Beijing, Tianjin, and Shanghai, for simulation experiments. The simulation time was 5 days, and the orbit data included real TLE data. The SGP4 orbit propagation model was used, and the beam parameters of the SAR satellite were reasonably simulated. Experimental results show that the coverage rates of the proposed method on three regional targets in Beijing, Tianjin, and Shanghai increased by 3.17%, 2.94%, and 9.02% compared with the coverage rates obtained with the greedy algorithm. In fine grids, the coverage result of the proposed method in the Shanghai area was 7.3% higher than that of the greedy algorithm.This study analyzes the SAR multisatellite collaborative complex area coverage planning technology, constructs a feasible SAR multisatellite collaborative complex area observation planning process, and proposes a complex area coverage planning method suitable for the SAR multisatellite strip imaging mode. This algorithm can provide a technical basis for the establishment of a SAR multisatellite collaborative regional planning system. However, the proposed method simplifies constraints at the imaging signal processing level of SAR satellites, and subsequent research will conduct in-depth research on the characteristics of SAR satellite imaging signal processing.Keywords：remote sensing;spaceborne SAR;multi-satellite collaboration;regional observation;coverage calculation;regional decomposition;genetic algorithm3152|2165|5<HTML><L-PDF> <Enhanced-PDF> <Meta-XML>Updated：2024-07-31
Satellite Information Intelligent Application
