Remote sensing image detection method based on brain-inspired spiking neural networks

DUAN Dexin; LU Yao; HUANG Liwei; LIU Peilin; WEN Fei

doi:10.11834/jrs.20243269

Satellite Information Intelligent Processing Technology | Views : 0 下载量: 2466 CSCD: 0

R-PDF
PDF
Export
Share
Collection
Album

Remote sensing image detection method based on brain-inspired spiking neural networks
Vol. 28, Issue 7, Pages: 1713-1721(2024)
Received：11 July 2023，

Published：07 July 2024
DOI： 10.11834/jrs.20243269
稿件说明：

移动端阅览

段德鑫，路遥，黄立威，刘佩林，文飞.2024.基于类脑脉冲神经网络的遥感图像检测算法.遥感学报，28（7）： 1713-1721 DOI： 10.11834/jrs.20243269.

Duan D X，Lu Y，Huang L W，Liu P L and Wen F. 2024. Remote sensing image detection method based on brain-inspired spiking neural networks. National Remote Sensing Bulletin， 28（7）：1713-1721 DOI： 10.11834/jrs.20243269.

摘要

与第二代人工神经网络（ANN）相比，第三代类脑脉冲神经网络（SNN）由于其高能效、高仿生、可解释等特点，在遥感影像智能处理的高能效、高精度、高可解译方面具有较大的潜在优势。针对现有脉冲神经网络算法延时较大的问题，本文提出一种基于类脑脉冲神经网络的遥感图像检测算法。该算法首先搭建了一个带有动态裁剪阈值激活函数的目标检测神经网络作为源网络进行预训练，随后借助训练过程中得到的裁剪阈值，通过激活神经元与脉冲神经元的映射关系将源网络转换为类脑脉冲神经网络，在继承源网络较高精度的同时还具备了低延迟、高仿生的特点。在SSDD（SAR-Ship-Detection-Datasets）和RSOD两个公开遥感数据集上的实验结果表明，该方法能够以极低的损失将源网络转换至类脑脉冲神经网络，并能在低时间步下对遥感目标实现较高的检测识别精度。同时该方法能够在继承ANN网络易于训练的特性与精度优势的同时，充分展现SNN的高稀疏度的带来的巨大能效优势。

Abstract

In the intelligent processing of remote sensing images

third-generation brain-inspired Spiking Neural Network (SNN) surpasses its predecessor

the second-generation artificial neural network

showing remarkable advantages in terms of high energy efficiency

high precision

and high interpretability. These advantages stem from the SNN’s characteristic features

including high energy efficiency

elevated sparsity

and remarkable bio-plausibility. The integration of these features in the SNN presents an enthralling solution to challenges faced in remote sensing image processing and holds immense potential for advancing this field further.

The proposed algorithm introduces a novel approach and initially establishes a target-detection neural network

which serves as the source artificial neural network for pretraining. This source network utilizes a dynamic clipping threshold activation function to optimize its performance. Subsequently

the algorithm transforms the source network into a brain-inspired SNN by leveraging the mapping relationship between activated neurons and spiking neurons. This conversion process effectively incorporates the clipping thresholds obtained during training. By seamlessly transitioning the source network into an SNN

the algorithm ensures the preservation and enhancement of key characteristics essential for remote sensing image processing.

The integration of the third-generation brain-inspired SNN into remote sensing image processing holds tremendous potential primarily due to its high precision and low energy consumption. The proposed algorithm highlights the distinctive attributes of the SNN

such as its low delay

high bionics

and ability to inherit high precision observed in a source network. These characteristics are promising indicators of SNN’s capability to considerably enhance the intelligent processing of remote sensing images. By leveraging the SNN’s high precision and bio-plausible principles

the proposed algorithm lays a robust foundation for future advancements in the field of remote sensing. The SNN’s inclusion in the image processing pipeline causes a paradigm shift

challenging traditional assumptions and unlocking new possibilities. The SNN is highly accurate in identifying and classifying targets with remarkable precision during remote sensing target detection. In conclusion

the proposed algorithm exhibits low delay and high bionics while demonstrating the high precision of a source network

demonstrating its potential to considerably improve the intelligent processing of remote sensing images and offering high accuracy

precision

interpretability; low energy consumption; and high bio-plausibility.

The efficacy of the proposed method was evaluated through extensive experiments performed on two widely recognized open remote sensing datasets

SAR-Ship-Detection-Datasets (SSDD) and RSOD. Experimental results highlighted the exceptional capabilities of the proposed method in transforming the source network into a brain-inspired spiking neural network

demonstrating negligible loss in performance. Furthermore

the transformed SNN exhibited remarkable accuracy in detecting and recognizing remote sensing targets within significantly reduced time steps. The performance achieved by the transformed SNN was comparable to that of the source artificial neural network while tremendously reducing power consumption by over two orders of magnitudes. This outcome highlights the immense potential of the proposed method in revolutionizing the field of remote sensing image processing by delivering high precision and interpretability which considerably reducing energy consumption.

关键词

Keywords

references

An R , Gong P , Wang H L , Feng X Z , Xiao P F , Chen Q , Zhang Q , Chen C Y and Yan P . 2010 . A modified PSO algorithm for remote sensing image template matching . Photogrammetric Engineering and Remote Sensing , 76 ( 4 ): 379 - 389 [ DOI: 10.14358/PERS.76.4.379 http://dx.doi.org/10.14358/PERS.76.4.379 ]

Carion N , Massa F , Synnaeve G , Usunier N , Kirillov A and Zagoruyko S . 2020 . End-to-end object detection with transformers // 16th European Conference on Computer Vision . Glasgow : Springer: 213 - 229 [ DOI: 10.1007/978-3-030-58452-8_13 http://dx.doi.org/10.1007/978-3-030-58452-8_13 ]

Che K W , Leng L , Zhang K X , Zhang J G , Meng M Q H , Cheng J , Guo Q H and Liao J X . 2022 . Differentiable hierarchical and surrogate gradient search for spiking neural networks // Proceedings of the 36th International Conference on Neural Information Processing Systems . New Orleans : Curran Associates Inc.: 24975 - 24990

Chen Y L , Qin R J , Zhang G X and Albanwan H . 2021 . Spatial temporal analysis of traffic patterns during the COVID-19 epidemic by vehicle detection using planet remote-sensing satellite images . Remote Sensing , 13 ( 2 ): 208 [ DOI: 10.3390/rs13020208 http://dx.doi.org/10.3390/rs13020208 ]

El Shair I M . 2003 . GIS and remote sensing in urban transportation planning : a case study of Birkenhead , Auckland//6th Annual International Conference. 28 - 31

Gao J F , Chen Y , Wei Y M and Li J N . 2021 . Detection of specific building in remote sensing images using a novel YOLO-S-CIOU model. Case: gas station identification . Sensors , 21 ( 4 ): 1375 [ DOI: 10.3390/s21041375 http://dx.doi.org/10.3390/s21041375 ]

Ghosh-Dastidar S and Adeli H . 2009 . Spiking neural networks . International Journal of Neural Systems , 19 ( 4 ): 295 - 308 [ DOI: 10.1142/S0129065709002002 http://dx.doi.org/10.1142/S0129065709002002 ]

Girshick R . 2015 . Fast R-CNN // 2015 IEEE International Conference on Computer Vision . Santiago : IEEE: 1440 - 1448 [ DOI: 10.1109/ICCV.2015.169 http://dx.doi.org/10.1109/ICCV.2015.169 ]

Horowitz M . 2014 . 1.1 computing’s energy problem (and what we can do about it)//2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC) . San Francisco : IEEE : 10 - 14 [ DOI: 10.1109/ISSCC.2014.6757323 http://dx.doi.org/10.1109/ISSCC.2014.6757323 ]

Kheradpisheh S R , Ganjtabesh M , Thorpe S J and Masquelier T . 2018 . STDP-based spiking deep convolutional neural networks for object recognition . Neural Networks , 99 : 56 - 67 [ DOI: 10.1016/j.neunet.2017.12.005 http://dx.doi.org/10.1016/j.neunet.2017.12.005 ]

Li D R , Ding L and Shao Z F . 2021 . Application-oriented real-time remote sensing service technology . National Remote Sensing Bulletin , 25 ( 1 ): 15 - 24

李德仁 , 丁霖 , 邵振峰 . 2021 . 面向实时应用的遥感服务技术 . 遥感学报 , 25 ( 1 ): 15 - 24 [ DOI: 10.11834/jrs.20210260 http://dx.doi.org/10.11834/jrs.20210260 ]

Liu S C , Du K C , Zheng Y J , Chen J , Du P J and Tong X H . 2023 . Remote sensing change detection technology in the era of artificial intelligence: inheritance, development and challenges . National Remote Sensing Bulletin , 27 ( 9 ): 1975 - 1987

柳思聪 , 都科丞 , 郑永杰 , 陈晋 , 杜培军 , 童小华 . 2023 . 人工智能时代的遥感变化检测技术: 继承、发展与挑战 . 遥感学报 , 27 ( 9 ): 1975 - 1987 [ DOI: 10.11834/jrs.20222199 http://dx.doi.org/10.11834/jrs.20222199 ]

Niu L Y , Wei Y and Liu Y . 2023 . Event-driven spiking neural network based on membrane potential modulation for remote sensing image classification . Engineering Applications of Artificial Intelligence , 123 : 106322 [ DOI: 10.1016/j.engappai.2023.106322 http://dx.doi.org/10.1016/j.engappai.2023.106322 ]

Pham M T , Courtrai L , Friguet C , Lefèvre S and Baussard A . 2020 . YOLO-Fine: one-stage detector of small objects under various backgrounds in remote sensing images . Remote Sensing , 12 ( 15 ): 2501 [ DOI: 10.3390/rs12152501 http://dx.doi.org/10.3390/rs12152501 ]

Redmon J , Divvala S , Girshick R and Farhadi A . 2016 . You only look once: unified, real-time object detection // 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE: 779 - 788 [ DOI: 10.1109/CVPR.2016.91 http://dx.doi.org/10.1109/CVPR.2016.91 ]

Redmon J and Farhadi A . 2018 . Yolov3: an incremental improvement. arXiv preprint arXiv:1804 . 02767

Rueckauer B , Lungu I A , Hu Y H , Pfeiffer M and Liu S C . 2017 . Conversion of continuous-valued deep networks to efficient event-driven networks for image classification . Frontiers in Neuroscience , 11 : 682 [ DOI: 10.3389/fnins.2017.00682 http://dx.doi.org/10.3389/fnins.2017.00682 ]

Sha M M , Li Y and Li A . 2022 . Multiscale aircraft detection in optical remote sensing imagery based on advanced Faster R-CNN . National Remote Sensing Bulletin , 26 ( 8 ): 1624 - 1635

沙苗苗 , 李宇 , 李安 . 2022 . 改进Faster R-CNN的遥感图像多尺度飞机目标检测 . 遥感学报 , 26 ( 8 ): 1624 - 1635 [ DOI: 10.11834/jrs.20219365 http://dx.doi.org/10.11834/jrs.20219365 ]

Tavanaei A , Ghodrati M , Kheradpisheh S R , Masquelier T and Maida A . 2019 . Deep learning in spiking neural networks . Neural Networks , 111 : 47 - 63 [ DOI: 10.1016/j.neunet.2018.12.002 http://dx.doi.org/10.1016/j.neunet.2018.12.002 ]

Xu Z Y , Shen Z F , Li Y , Li J L , Wang H Y , Li S , Jiao S H and Li L L . 2022 . Detection of water conservancy facilities in large-format image combining E-YOLO algorithm and NDWI constraint . National Remote Sensing Bulletin , 26 ( 10 ): 2083 - 2093

许泽宇 , 沈占锋 , 李杨 , 李均力 , 王浩宇 , 李硕 , 焦淑慧 , 李苓苓 . 2022 . 结合E-YOLO和水体指数约束的大面幅影像水利设施检测 . 遥感学报 , 26 ( 10 ): 2083 - 2093 [ DOI: 10.11834/jrs.20200124 http://dx.doi.org/10.11834/jrs.20200124 ]

Zhang S M , Wu R Z , Xu K Y , Wang J M and Sun W W . 2019 . R-CNN-based ship detection from high resolution remote sensing imagery . Remote Sensing , 11 ( 6 ): 631 [ DOI: 10.3390/rs11060631 http://dx.doi.org/10.3390/rs11060631 ]

Zhang Y P , Bi D P , Zhou Y , Zhang B and Fang M X . 2019 . Effect of cosinusoidal phase-modulated scatter-wave jamming to the jamming suppression of SAR dual-channel cancellation . Journal of Remote Sensing (in Chinese) , 23 ( 1 ): 99 - 107

张云鹏 , 毕大平 , 周阳 , 张博 , 房明星 . 2019 . 余弦调相散射波干扰对SAR双通道对消干扰抑制的影响 . 遥感学报 , 23 ( 1 ): 99 - 107 [ DOI: 10.11834/jrs.20197114 http://dx.doi.org/10.11834/jrs.20197114 ]

Zhu X X , Montazeri S , Ali M , Hua Y S , Wang Y Y , Mou L C , Shi Y L , Xu F and Bamler R . 2021 . Deep learning meets SAR: concepts, models, pitfalls, and perspectives . IEEE Geoscience and Remote Sensing Magazine , 9 ( 4 ): 143 - 172 [ DOI: 10.1109/MGRS.2020.3046356 http://dx.doi.org/10.1109/MGRS.2020.3046356 ]

Alert me when the article has been cited

提交

Multimodal object detection method using adaptive dusion of infrared and visible features

From optical to SAR： A SAR ship detection algorithm based on multi-level cross-modality alignment

Remote sensing image super-resolution guided by multi-level supervision paradigm

SAR ship detection through generative knowledge transfer

Joint classification of hyperspectral and LiDAR data based on inter-modality match learning