Anterior|Siguiente

Research on Spiking Neural Network fine-tuning method for object detection in remote sensing images

  • role： First author第一作者
  • Affiliation：

    National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi'an 710049, China

    National Engineering Research Center forVisual Information and Applications, Xi'an 710049, China

    Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an 710049, China

  • Email：guobailin@stu.xjtu.edu.cn
  • Introduction：郭柏麟，研究方向为脉冲神经网络与遥感目标检测。E-mail： guobailin@stu.xjtu.edu.cn
GUO Bailin134,  
  • Affiliation：

    Beijing Institute of Remote Sensing, Beijing 100192, China

HUANG Liwei2,  
  • Affiliation：

    Beijing Institute of Remote Sensing, Beijing 100192, China

LU Yao2,  
  • Affiliation：

    National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi'an 710049, China

    National Engineering Research Center forVisual Information and Applications, Xi'an 710049, China

    Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an 710049, China

ZHANG Xuetao134,  
  • role： Corresponding author通信作者
  • Affiliation：

    National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi'an 710049, China

    National Engineering Research Center forVisual Information and Applications, Xi'an 710049, China

    Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an 710049, China

  • Email：musayq@xjtu.edu.cn
  • Introduction：马永强，研究方向为类脑与认知计算模型设计，生物大脑认知机理探索，以及脑信号分析识别。E-mail： musayq@xjtu.edu.cn
MA Yongqiang134*

resumen

Object detection in remote sensing images is essential research contents of visual image recognition tasks. However, in the remote sensing images of ships, a ship target is small and sparsely distributed, and using a traditional Artificial Neural Network (ANN) for object detection often wastes a considerable amount of computing resources. SpikinG Neural Networks (SNNs) can be applied due to its event-driven and low-power characteristics, greatly saving energy and computing resources. However, training an SNN is difficult because of the complex dynamics and nondifferentiable pulse operation of SNN neurons. Instead, converting a trained ANN into SNN can effectively circumvent training difficulties. For a converted deep SNN, many time steps are often required to maintain its performance. Unfortunately, this process requires a substantial amount of computing resources.This paper studies the reason why a large number of time steps are required to maintain an SNN model’s performance after conversion and proposes a novel conversion method: a layer-by-layer conversion method based on fine-tuning. During conversion, the network is converted layer by layer, and the subsequent unconverted network is fine-tuned, and thus the errors accumulate layer by layer during conversion is prevented. In addition, given the rationality of hardware deployment, we propose Poisson group coding, which uses multiple Poisson coding neurons to encode input images and sends them to the network after average pooling. Compared with Poisson coding, the output of Poisson group coding is less noisy and has less impact on model performance.The fine-tuning transformation method achieves a result (96.9%, 70.3%), similar to that of YOLOv3-tiny (97.9%, 79.6%) on the SAR ship detection datasets (AIR-SARShip), and 80% of the performance of the preconversion model can be achieved by using few time steps (20 and 80 steps). The method achieves good detection performance (49.2%) on the PASCAL VOC dataset. By contrast, the performance of the conventional conversion method is inferior to that of the fine-tuning conversion method in the same number of time steps and usually requires many time steps (more than 150 time steps), achieving improved detection performance. For Poisson group coding, the impact on model performance under the same time steps decreases with increasing neurons. Performance similar to the input simulation frequency can be achieved with few time steps.The layer-by-layer conversion method based on the proposed fine-tuning method effectively adapts the SNN network for the layer-by-layer conversion of error, preventing the accumulation of error in each layer and reducing SNN. This method can improve the performance of a converted SNN and reduce the time steps. Meanwhile, Poisson group coding provides an effective input coding method for the hardware deployment of an SNN.

palabra clave

Spiking Neural Network (SNN);object detection;ship remote sensing images;convert ANN to SNN;Poisson group coding

References

  1. 1.
    Bochkovskiy A, Wang C Y and Liao H Y M. 2020. YOLOv4: optimal Speed and accuracy of object detection. arXiv:2004.10934
  2. 2.
    Cao Y Q, Chen Y and Khosla D. 2015. Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision, 113(1): 54-66
  3. 3.
    Cheng G and Han J W. 2016. A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 117: 11-28
  4. 4.
    Dalal N and Triggs B. 2005. Histograms of oriented gradients for human detection//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego: IEEE: 886-893
  5. 5.
    Diehl P U and Cook M. 2015. Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in Computational Neuroscience, 9: 99
  6. 6.
    Diehl P U, Neil D, Binas J, Cook M, Liu S C and Pfeiffer M. 2015. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing//2015 International Joint Conference on Neural Networks (IJCNN). Killarney: IEEE: 1-8
  7. 7.
    Fan X N, Yan W, Shi P F and Zhang X W. 2022. Remote sensing image target detection based on a multi-scale deep feature fusion network. National Remote Sensing Bulletin, 26(11): 2292-2303
  8. 8.
    Felzenszwalb P F, Girshick R B, McAllester D and Ramanan D. 2010. Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9): 1627-1645
  9. 9.
    Girshick R, Donahue J, Darrell T and Malik J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 580-587
  10. 10.
    Girshick R. 2015. Fast R-CNN//2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 1440-1448
  11. 11.
    He K M, Gkioxari G, Dollár P and Girshick R. 2017. Mask R-CNN//2017 IEEE International Conference on Computer Vision. Venice: IEEE: 2980-2988
  12. 12.
    Jiao L C, Zhang F, Liu F, Yang S Y, Li L L, Feng Z X and Qu R. 2019. A survey of deep learning-based object detection. IEEE Access, 7: 128837-128868
  13. 13.
    Kim S, Park S, Na B, Kim J and Yoon S. 2021. Towards fast and accurate object detection in bio-inspired spiking neural networks through Bayesian optimization. IEEE Access, 9: 2633-2643
  14. 14.
    Kim S, Park S, Na B and Yoon S. 2020. Spiking-YOLO: spiking neural network for energy-efficient object detection. Proceedings of the AAAI Conference on Artificial Intelligence, 34(7): 11270-11277
  15. 15.
    Lee J H, Delbruck T and Pfeiffer M. 2016. Training deep spiking neural networks using backpropagation. Frontiers in Neuroscience, 10: 508
  16. 16.
    Li C Y, Li L L, Jiang H L, Weng K H, Geng Y F, Li L, Ke Z D, Li Q Y, Cheng M, Nie W Q, Li Y D, Zhang B, Liang Y F, Zhou L Y, Xu X M, Chu X X, Wei X M and Wei X L. 2022. YOLOv6: a single-stage object detection framework for industrial applications. arXiv:2209.02976
  17. 17.
    Li J W, Qu C W and Shao J Q. 2017. Ship detection in SAR images based on an improved faster R-CNN//2017 SAR in Big Data Era: Models, Methods and Applications (BIGSARDATA). Beijing: IEEE: 1-6
  18. 18.
    Li K, Wan G, Cheng G, Meng L Q and Han J W. 2020. Object detection in optical remote sensing images: a survey and a new benchmark. ISPRS Journal of Photogrammetry and Remote Sensing, 159: 296-307
  19. 19.
    Li Y and Zheng Y. 2022. Efficient and accurate conversion of spiking neural network with burst spikes//Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence. Vienna: [s.n.]: 2485-2491
  20. 20.
    Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y and Berg A C. 2016. SSD: single shot MultiBox detector//14th European Conference on Computer Vision. Amsterdam: Springer: 21-37
  21. 21.
    Luo Y H, Xu M, Yuan C H, Cao X, Zhang L Q, Xu Y, Wang T J and Feng Q. 2021. SiamSNN: siamese spiking neural networks for energy-efficient object tracking//30th International Conference on Artificial Neural Networks. Bratislava: Springer: 182-194
  22. 22.
    Maass W. 1997. Networks of spiking neurons: the third generation of neural network models. Neural Networks, 10(9): 1659-1671
  23. 23.
    Menghani G. 2023. Efficient deep learning: a survey on making deep learning models smaller, faster, and better. ACM Computing Surveys, 55(12): 259
  24. 24.
    Patel K, Hunsberger E, Batir S and Eliasmith C. 2021. A spiking neural network for image segmentation. arXiv:2106.08921
  25. 25.
    Redmon J, Divvala S, Girshick R and Farhadi A. 2016. You only look once: unified, real-time object detection//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 779-788
  26. 26.
    Redmon J and Farhadi A. 2017. YOLO9000: better, faster, stronger//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 6517-6525
  27. 27.
    Redmon J and Farhadi A. 2018. YOLOv3: an incremental improvement. arXiv:1804.02767
  28. 28.
    Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137-1149
  29. 29.
    Sengupta A, Ye Y T, Wang R, Liu C A and Roy K. 2019. Going deeper in spiking neural networks: VGG and residual architectures. Frontiers in Neuroscience, 13: 425055
  30. 30.
    Sun X, Wang Z R, Sun Y R, Diao W H, Zhang Y and Fu K. 2019. AIR-SARShip-1.0: high-resolution SAR ship detection dataset. Journal of Radars, 8(6): 852-862
  31. 31.
    Viola P and Jones M J. 2004. Robust real-time face detection. International Journal of Computer Vision, 57(2): 137-154
  32. 32.
    Wu Y J, Deng L, Li G Q, Zhu J, Xie Y and Shi L P. 2019. Direct training for spiking neural networks: faster, larger, better. Proceedings of the AAAI Conference on Artificial Intelligence, 33(1): 1311-1318
  33. 33.
    Xiang S Y, Jiang S Q, Liu X S, Zhang T and Yu L C. 2022. Spiking VGG7: deep convolutional spiking neural network with direct training for object recognition. Electronics, 11(13): 2097
  34. 34.
    Yu Y, Ai H, He X J, Yu S H, Zhong X and Zhu R F. 2020. Attention-based feature pyramid networks for ship detection of optical remote sensing image. National Remote Sensing Bulletin, 24(2): 107-115
  35. 35.
    Zhang T, Yang X G, Lu X Q, Lu R T and Zhang S X. 2022. Ship detection in remote sensing image based on dense RFB and LSTM. National Remote Sensing Bulletin, 26(9): 1859-1871

Leer el texto completo

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website