سابِق|تال

Intelligent detection for moving targets in space-borne optical remote sensing： A review

  • role： First author第一作者
  • Affiliation：

    College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, China

  • Email：xiaochao12@nudt.edu.cn
  • Introduction：肖超， 研究方向为光电探测。E-mail： xiaochao12@nudt.edu.cn
XIAO Chao,  
  • role： Corresponding author通信作者
  • Affiliation：

    College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, China

  • Email：anwei@nudt.edu.cn
  • Introduction：安玮，研究方向为光电探测。E-mail： anwei@nudt.edu.cn
AN Wei*,  
  • Affiliation：

    College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, China

LI Zhaoxu,  
  • Affiliation：

    College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, China

LI Boyang,  
  • Affiliation：

    College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, China

YING Xinyi,  
  • Affiliation：

    College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, China

LIN Zaiping

ملخص

Moving object detection in spaceborne optical remote sensing images (particularly for satellite videos) is a critical technique for interpreting remote sensing data. It has numerous applications, including surveillance, environmental monitoring, and military intelligence. Moving object detection in satellite videos aims to locate and classify moving targets accurately, such as moving vehicles, ships, trains, and airplanes. The advancements in spaceborne optical remote sensing satellite technology and the emergence of deep learning techniques allow the traditional model-driven methods for moving object detection to evolve toward data-driven deep learning methods and achieve high reliability, efficiency, and performance. This study introduces the current status of optical remote sensing satellite systems and summarizes model-driven and data-driven approaches for optical remote sensing moving target detection. First, the current state of video satellites is presented. Spaceborne optical video satellites hold great potential for advancing our ability to detect and monitor dynamic phenomena across various domains, thereby forking the data foundation for moving object detection. Second, the development of model-driven methods for moving object detection is analyzed, including frame differencing-based, optical flow-based, and background subtraction-based methods. However, these methods rely heavily on handcrafted features and prior knowledge to identify moving objects. Thus, they often struggle with complex backgrounds, varying lighting conditions, and occlusions. By contrast, with their powerful feature learning capabilities, data-driven deep learning-based methods have brought significant progress in moving object detection in satellite videos. Third, we present the development of data-driven deep learning-based methods. We also summarize the supervised and unsupervised deep learning-based methods for moving object detection in satellite videos, and their trends are discussed. In deep learning-based methods, extracting spatiotemporal information is crucial for efficient and effective moving object detection in satellite videos. Furthermore, the methods of alleviating annotation costs and addressing massive data are underdeveloped and need further exploration. Then, we introduce the datasets, evaluation criteria, and state-of-the-art experimental results. We evaluate nine representative model-driven and data-driven methods (supervised and unsupervised) for moving object detection in satellite videos on the VISO car dataset. We conclude from the results that deep learning methods with superior performance can adapt effectively to diverse environmental conditions and target characteristics. The unsupervised deep learning methods are underdeveloped and need considerable focus. Moreover, the combination of model-driven and data-driven methods has shown great potential for moving object detection, which leads to a new development trend for the community. Lastly, on the basis of the summary above and the discussion, we conclude the future trends of moving object detection in satellite videos. On the one hand, the development of satellite constellations can open up new possibilities for collaborative and distributive multisatellite and multisensor systems. On the other hand, the potential to merge feature representation, detection, and tracking into an end-to-end network with multitask learning needs further exploration. Weakly supervised and unsupervised methods are worth further development and research to alleviate the burden of annotation cost. The ability to tackle massive data and the construction of an end-to-end detection and recognition network for high-resolution satellite videos are also worth exploring.

مفهوم

space-based optical remote sensing;moving object detection;satellite videos;model-driven;data-driven;neural network;spatio-temporal information;unsupervised learning

References

  1. 1.
    Abdi H and Williams L J. 2010. Principal component analysis. WIREs Computational Statistics, 2(4): 433-459
  2. 2.
    Ahmadi S A, Ghorbanian A and Mohammadzadeh A. 2019. Moving vehicle detection, tracking and traffic parameter estimation from a satellite video: a perspective on a smarter city. International Journal of Remote Sensing, 40(22): 8379-8394
  3. 3.
    Ahmadi S A and Mohammadzadeh A. 2017. A simple method for detecting and tracking vehicles and vessels from high resolution spaceborne videos//2017 Joint Urban Remote Sensing Event. Dubai: IEEE: 1-4
  4. 4.
    Ao W, Fu Y W, Hou X Y and Xu F. 2020. Needles in a haystack: tracking city-scale moving vehicles from continuously moving satellite. IEEE Transactions on Image Processing, 29: 1944-1957
  5. 5.
    Bao Z P, Tokmakov P, Jabri A, Wang Y X, Gaidon A and Hebert M. 2022. Discovering objects that can move//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE: 11779-11788
  6. 6.
    Barnich O and van Droogenbroeck M. 2011. ViBe: a universal background subtraction algorithm for video sequences. IEEE Transactions on Image Processing, 20(6): 1709-1724
  7. 7.
    Boyd S, Parikh N, Chu E, Peleato B and Eckstein J. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning, 3(1): 1-122
  8. 8.
    Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A and Zagoruyko S. 2020. End-to-end object detection with transformers//Proceedings of the 16th European Conference on Computer Vision. Glasgow: Springer: 213-229
  9. 9.
    Chen R X, Ferreira V G and Li X H. 2023. Detecting moving vehicles from satellite-based videos by tracklet feature classification. Remote Sensing, 15(1): 34
  10. 10.
    Cui Z G, Wang H, Li A H, Wang T and Li H. 2017. Moving object detection based on optical flow field analysis in dynamic scenes. Acta Physica Sinica, 66(8): 084203
  11. 11.
    Feng J, Liang Y P, Zhang X R, Zhang J P and Jiao L C. 2023. SDANet: semantic-embedded density adaptive network for moving vehicle detection in satellite videos. IEEE Transactions on Image Processing, 32: 1788-1801
  12. 12.
    Feng J, Zeng D N, Jia X P, Zhang X R, Li J, Liang Y P and Jiao L C. 2021. Cross-frame keypoint-based and spatial motion information-guided networks for moving vehicle detection and tracking in satellite videos. ISPRS Journal of Photogrammetry and Remote Sensing, 177: 116-130
  13. 13.
    Ghadimi E, Teixeira A, Shames I and Johansson M. 2015. Optimal parameter selection for the alternating direction method of multipliers (ADMM): quadratic problems. IEEE Transactions on Automatic Control, 60(3): 644-658
  14. 14.
    Girshick R, Donahue J, Darrell T and Malik J. 2016. Region-based convolutional networks for accurate object detection and segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1): 142-158
  15. 15.
    He Q B, Sun X, Yan Z Y, Li B B and Fu K. 2022. Multi-object tracking in satellite videos with graph-based multitask modeling. IEEE Transactions on Geoscience and Remote Sensing, 60: 5619513
  16. 16.
    Jin J. 2020. Research on Moving Targets Detection and Tracking in Complex Video Scenes. Lanzhou: Lanzhou Jiaotong University
  17. 17.
    Kang J Z, Wang G Z, He G J, Wang H H, Yin R Y, Jiang W and Zhang Z M. 2020. Moving vehicle detection for remote sensing satellite video. Journal of Remote Sensing (in Chinese), 24(9): 1099-1107
  18. 18.
    Kopsiaftis G and Karantzalos K. 2015. Vehicle detection and traffic density monitoring from very high resolution satellite video data//2015 IEEE International Geoscience and Remote Sensing Symposium. Milan: IEEE: 1881-1884
  19. 19.
    LaLonde R, Zhang D and Shah M. 2018. ClusterNet: detecting small objects in large scenes by exploiting spatio-temporal information//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 4003-4012
  20. 20.
    Law H and Deng J. 2018. CornerNet: detecting objects as paired keypoints//Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich: Springer: 765-781
  21. 21.
    Li C M, Bai H Y, Guo H W and Liang H J. 2018. Moving object detection and tracking based on improved optical flow method. Chinese Journal of Scientific Instrument, 39(5): 249-256
  22. 22.
    Li F, Zhang H, Liu S L, Guo J, Ni L M and Zhang L. 2022. DN-DETR: accelerate DETR training by introducing query DeNoising//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE: 13609-13617
  23. 23.
    Liu H B and Chang F L. 2016. Moving object detection by optical flow method based on adaptive weight coefficient. Optics and Precision Engineering, 24(2): 460-468
  24. 24.
    Liu T. 2014. Research on the development of video satellites abroad. Space International, (9): 50-56
  25. 25.
    Liu X, Zhao G Y, Yao J W and Qi C. 2015. Background subtraction based on low-rank and structured sparse decomposition. IEEE Transactions on Image Processing, 24(8): 2502-2514
  26. 26.
    Luo Y L, Liang Y P and Wang Y. 2018. Traffic flow parameter estimation from satellite video data based on optical flow. Computer Engineering and Applications, 54(10): 204-207, 255
  27. 27.
    Pi Z L, Jiao L C, Liu F, Liu X, Li L L, Hou B and Yang S Y. 2022. Very low-resolution moving vehicle detection in satellite videos. IEEE Transactions on Geoscience and Remote Sensing, 60: 5624517
  28. 28.
    Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137-1149
  29. 29.
    Shu M, Zhong Y F and Lv P Y. 2021. Small moving vehicle detection via local enhancement fusion for satellite video. International Journal of Remote Sensing, 42(19): 7189-7214
  30. 30.
    Song J S. 2014. Study on Moving Object Detection and Tracking in Sequence Images. Guangzhou: South China University of Technology
  31. 31.
    Stauffer C and Grimson W E L. 1999. Adaptive background mixture models for real-time tracking//1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Fort Collins: IEEE: 246-252
  32. 32.
    Sultana M, Mahmood A and Jung S K. 2021. Unsupervised moving object detection in complex scenes using adversarial regularizations. IEEE Transactions on Multimedia, 23: 2005-2018
  33. 33.
    Wang Y, Yin W T and Zeng J S. 2019. Global convergence of ADMM in nonconvex nonsmooth optimization. Journal of Scientific Computing, 78(1): 29-63
  34. 34.
    Wren C R, Azarbayejani A, Darrell T and Pentland A P. 1997. Pfinder: real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7): 780-785
  35. 35.
    Wu J Q, Jiang Y H, Shen X, Li B B and Pan S L. 2019. Satellite video motion detection supported by decision tree weak classification. Geomatics and Information Science of Wuhan University, 44(8): 1182-1190
  36. 36.
    Xiao C, Liu T, Ying X Y, Wang Y Q, Li M, Liu L, An W and Chen Z J. 2023. Incorporating deep background prior into model-based method for unsupervised moving vehicle detection in satellite videos. IEEE Transactions on Geoscience and Remote Sensing, 61: 5603114
  37. 37.
    Xiao C, Yin Q, Ying X Y, Li R J, Wu S L, Li M, Liu L, An W and Chen Z J. 2022. DSFNet: dynamic and static fusion network for moving object detection in satellite videos. IEEE Geoscience and Remote Sensing Letters, 19: 3510405
  38. 38.
    Xu W, Jin G and Wang J Q. 2017. Optical imaging technology of JL-1 lightweight high resolution multispectral remote sensing satellite. Optics and Precision Engineering, 25(8): 1969-1978
  39. 39.
    Yang T, Wang X W, Yao B W, Li J, Zhang Y N, He Z N and Duan W C. 2016. Small moving vehicle detection in a satellite video of an urban area. Sensors, 16(9): 1528
  40. 40.
    Yin Q, Hu Q Y, Liu H, Zhang F, Wang Y Q, Lin Z P, An W and Guo Y L. 2022a. Detecting and tracking small and dense moving objects in satellite videos: a benchmark. IEEE Transactions on Geoscience and Remote Sensing, 60: 5612518
  41. 41.
    Yin Q, Liu T, Lin Z P, An W and Guo Y L. 2022b. Moving object detection in satellite videos via spatial–temporal tensor model and weighted Schatten p-norm minimization. IEEE Geoscience and Remote Sensing Letters, 19: 8022405
  42. 42.
    Yu Y B, Zhang T, Guo L H and He X J. 2017. Moving objects detection on satellite video. Chinese Journal of Liquid Crystals and Displays, 32(2): 138-143
  43. 43.
    Yuan Y Q, He G J, Wang G Z, Jiang W and Kang J Z. 2018. A background subtraction and frame subtraction combined method for moving vehicle detection in satellite video data. Journal of University of Chinese Academy of Sciences, 35(1): 50-58
  44. 44.
    Yun K, Kim H, Bae K and Park J. 2021. Unsupervised moving object detection through background models for PTZ camera//2020 25th International Conference on Pattern Recognition. Milan: IEEE: 3201-3208
  45. 45.
    Zhang J P, Jia X P and Hu J K. 2019. Local region proposing for frame-based vehicle detection in satellite videos. Remote Sensing, 11(20): 2372
  46. 46.
    Zhang J P, Jia X P and Hu J K. 2020a. Error bounded foreground and background modeling for moving object detection in satellite videos. IEEE Transactions on Geoscience and Remote Sensing, 58(4): 2659-2669
  47. 47.
    Zhang J P, Jia X P, Hu J K and Chanussot J. 2020b. Online structured sparsity-based moving-object detection from satellite videos. IEEE Transactions on Geoscience and Remote Sensing, 58(9): 6420-6433
  48. 48.
    Zhang J P, Jia X P, Hu J K and Tan K. 2022. Moving vehicle detection for remote sensing video surveillance with nonstationary satellite platform. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9): 5185-5198
  49. 49.
    Zhang J P, Zhang J and Jia X P. 2021. Learning via watching: a weakly supervised moving object detector for satellite videos//2021 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Brussels: IEEE: 2333-2336
  50. 50.
    Zhang X Y. 2017. Study on Moving Objects Intelligent Sensing and Tracking Control for Video Satellite. Changsha: National University of Defense Technology
  51. 51.
    Zhou T Y and Tao D C. 2011. GoDec: randomized low-rank & sparse matrix decomposition in noisy case//Proceedings of the 28th International Conference on Machine Learning. Bellevue: Omnipress: 33-40
  52. 52.
    Zhou X W, Yang C and Yu W C. 2013. Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(3): 597-610
  53. 53.
    Zhou Y, Wang P and Fu D Y. 2015. System Innovation and Enlightenment of SkySat. Spacecraft Engineering, 24(5): 91-98
  54. 54.
    Zhuo T, Cheng Z Y, Zhang P, Wong Y and Kankanhalli M. 2020. Unsupervised online video object segmentation with motion property understanding. IEEE Transactions on Image Processing, 29: 237-249
  55. 55.
    Zivkovic Z and van der Heijden F. 2006. Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognition Letters, 27(7): 773-780

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website