Intelligent detection for moving targets in space-borne optical remote sensing: A review
- Vol. 28, Issue 7, Pages: 1681-1692(2024)
Received:10 July 2023,
Published:07 July 2024
DOI: 10.11834/jrs.20243277
移动端阅览


浏览全部资源
扫码关注微信
Received:10 July 2023,
Published:07 July 2024
移动端阅览
天基光学遥感动目标检测旨在对遥感卫星视频中具有连续运动特性的目标进行定位和分类,比如遥感视频卫星中的运动车辆、舰船和飞机。随着遥感视频卫星技术和深度学习技术的快速发展,基于模型驱动的传统遥感动目标检测方法正朝着基于数据驱动的深度学习方法进行演变,以完成高可靠、高时效、高性能的天基光学遥感图像动目标检测。本文介绍了光学遥感视频卫星的发展现状,并对基于模型驱动和基于数据驱动的光学遥感动目标检测方法进行了总结,梳理和分析了光学遥感动目标检测技术的发展历程。最后,在此基础上对光学遥感动目标检测的未来发展趋势进行了展望。
Moving object detection in spaceborne optical remote sensing images (particularly for satellite videos) is a critical technique for interpreting remote sensing data. It has numerous applications
including surveillance
environmental monitoring
and military intelligence. Moving object detection in satellite videos aims to locate and classify moving targets accurately
such as moving vehicles
ships
trains
and airplanes. The advancements in spaceborne optical remote sensing satellite technology and the emergence of deep learning techniques allow the traditional model-driven methods for moving object detection to evolve toward data-driven deep learning methods and achieve high reliability
efficiency
and performance. This study introduces the current status of optical remote sensing satellite systems and summarizes model-driven and data-driven approaches for optical remote sensing moving target detection. First
the current state of video satellites is presented. Spaceborne optical video satellites hold great potential for advancing our ability to detect and monitor dynamic phenomena across various domains
thereby forking the data foundation for moving object detection. Second
the development of model-driven methods for moving object detection is analyzed
including frame differencing-based
optical flow-based
and background subtraction-based methods. However
these methods rely heavily on handcrafted features and prior knowledge to identify moving objects. Thus
they often struggle with complex backgrounds
varying lighting conditions
and occlusions. By contrast
with their powerful feature learning capabilities
data-driven deep learning-based methods have brought significant progress in moving object detection in satellite videos. Third
we present the development of data-driven deep learning-based methods. We also summarize the supervised and unsupervised deep learning-based methods for moving object detection in satellite videos
and their trends are discussed. In deep learning-based methods
extracting spatiotemporal information is crucial for efficient and effective moving object detection in satellite videos. Furthermore
the methods of alleviating annotation costs and addressing massive data are underdeveloped and need further exploration. Then
we introduce the datasets
evaluation criteria
and state-of-the-art experimental results. We evaluate nine representative model-driven and data-driven methods (supervised and unsupervised) for moving object detection in satellite videos on the VISO car dataset. We conclude from the results that deep learning methods with superior performance can adapt effectively to diverse environmental conditions and target characteristics. The unsupervised deep learning methods are underdeveloped and need considerable focus. Moreover
the combination of model-driven and data-driven methods has shown great potential for moving object detection
which leads to a new development trend for the community. Lastly
on the basis of the summary above and the discussion
we conclude the future trends of moving object detection in satellite videos. On the one hand
the development of satellite constellations can open up new possibilities for collaborative and distributive multisatellite and multisensor systems. On the other hand
the potential to merge feature representation
detection
and tracking into an end-to-end network with multitask learning needs further exploration. Weakly supervised and unsupervised methods are worth further development and research to alleviate the burden of annotation cost. The ability to tackle massive data and the construction of an end-to-end detection and recognition network for high-resolution satellite videos are also worth exploring.
Abdi H and Williams L J . 2010 . Principal component analysis . WIREs Computational Statistics , 2 ( 4 ): 433 - 459 [ DOI: 10.1002/wics.101 http://dx.doi.org/10.1002/wics.101 ]
Ahmadi S A , Ghorbanian A and Mohammadzadeh A . 2019 . Moving vehicle detection, tracking and traffic parameter estimation from a satellite video: a perspective on a smarter city . International Journal of Remote Sensing , 40 ( 22 ): 8379 - 8394 [ DOI: 10.1080/01431161.2019.1610983 http://dx.doi.org/10.1080/01431161.2019.1610983 ]
Ahmadi S A and Mohammadzadeh A . 2017 . A simple method for detecting and tracking vehicles and vessels from high resolution spaceborne videos // 2017 Joint Urban Remote Sensing Event . Dubai : IEEE: 1 - 4 [ DOI: 10.1109/JURSE.2017.7924614 http://dx.doi.org/10.1109/JURSE.2017.7924614 ]
Ao W , Fu Y W , Hou X Y and Xu F . 2020 . Needles in a haystack: tracking city-scale moving vehicles from continuously moving satellite . IEEE Transactions on Image Processing , 29 : 1944 - 1957 [ DOI: 10.1109/TIP.2019.2944097 http://dx.doi.org/10.1109/TIP.2019.2944097 ]
Bao Z P , Tokmakov P , Jabri A , Wang Y X , Gaidon A and Hebert M . 2022 . Discovering objects that can move // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans : IEEE: 11779 - 11788 [ DOI: 10.1109/CVPR52688.2022.01149 http://dx.doi.org/10.1109/CVPR52688.2022.01149 ]
Barnich O and van Droogenbroeck M . 2011 . ViBe: a universal background subtraction algorithm for video sequences . IEEE Transactions on Image Processing , 20 ( 6 ): 1709 - 1724 [ DOI: 10.1109/TIP.2010.2101613 http://dx.doi.org/10.1109/TIP.2010.2101613 ]
Boyd S , Parikh N , Chu E , Peleato B and Eckstein J . 2011 . Distributed optimization and statistical learning via the alternating direction method of multipliers . Foundations and Trends® in Machine Learning , 3 ( 1 ): 1 - 122 [ DOI: 10.1561/2200000016 http://dx.doi.org/10.1561/2200000016 ]
Carion N , Massa F , Synnaeve G , Usunier N , Kirillov A and Zagoruyko S . 2020 . End-to-end object detection with transformers // Proceedings of the 16th European Conference on Computer Vision . Glasgow : Springer: 213 - 229 [ DOI: 10.1007/978-3-030-58452-8_13 http://dx.doi.org/10.1007/978-3-030-58452-8_13 ]
Chen R X , Ferreira V G and Li X H . 2023 . Detecting moving vehicles from satellite-based videos by tracklet feature classification . Remote Sensing , 15 ( 1 ): 34 [ DOI: 10.3390/rs15010034 http://dx.doi.org/10.3390/rs15010034 ]
Cui Z G , Wang H , Li A H , Wang T and Li H . 2017 . Moving object detection based on optical flow field analysis in dynamic scenes . Acta Physica Sinica , 66 ( 8 ): 084203
崔智高 , 王华 , 李艾华 , 王涛 , 李辉 . 2017 . 动态背景下基于光流场分析的运动目标检测算法 . 物理学报 , 66 ( 8 ): 084203 [ DOI: 10.7498/aps.66.084203 http://dx.doi.org/10.7498/aps.66.084203 ]
Feng J , Liang Y P , Zhang X R , Zhang J P and Jiao L C . 2023 . SDANet: semantic-embedded density adaptive network for moving vehicle detection in satellite videos . IEEE Transactions on Image Processing , 32 : 1788 - 1801 [ DOI: 10.1109/TIP.2023.3251026 http://dx.doi.org/10.1109/TIP.2023.3251026 ]
Feng J , Zeng D N , Jia X P , Zhang X R , Li J , Liang Y P and Jiao L C . 2021 . Cross-frame keypoint-based and spatial motion information-guided networks for moving vehicle detection and tracking in satellite videos . ISPRS Journal of Photogrammetry and Remote Sensing , 177 : 116 - 130 [ DOI: 10.1016/j.isprsjprs.2021.05.005 http://dx.doi.org/10.1016/j.isprsjprs.2021.05.005 ]
Ghadimi E , Teixeira A , Shames I and Johansson M . 2015 . Optimal parameter selection for the alternating direction method of multipliers (ADMM): quadratic problems . IEEE Transactions on Automatic Control , 60 ( 3 ): 644 - 658 [ DOI: 10.1109/TAC.2014.2354892 http://dx.doi.org/10.1109/TAC.2014.2354892 ]
Girshick R , Donahue J , Darrell T and Malik J . 2016 . Region-based convolutional networks for accurate object detection and segmentation . IEEE Transactions on Pattern Analysis and Machine Intelligence , 38 ( 1 ): 142 - 158 [ DOI: 10.1109/TPAMI.2015.2437384 http://dx.doi.org/10.1109/TPAMI.2015.2437384 ]
He Q B , Sun X , Yan Z Y , Li B B and Fu K . 2022 . Multi-object tracking in satellite videos with graph-based multitask modeling . IEEE Transactions on Geoscience and Remote Sensing , 60 : 5619513 [ DOI: 10.1109/TGRS.2022.3152250 http://dx.doi.org/10.1109/TGRS.2022.3152250 ]
Jin J . 2020 . Research on Moving Targets Detection and Tracking in Complex Video Scenes . Lanzhou : Lanzhou Jiaotong University
金静 . 2020 . 复杂视频场景下的运动目标检测与跟踪研究 . 兰州 : 兰州交通大学 [ DOI: 10.27205/d.cnki.gltec.2020.000008 http://dx.doi.org/10.27205/d.cnki.gltec.2020.000008 ]
Kang J Z , Wang G Z , He G J , Wang H H , Yin R Y , Jiang W and Zhang Z M . 2020 . Moving vehicle detection for remote sensing satellite video . Journal of Remote Sensing (in Chinese) , 24 ( 9 ): 1099 - 1107
康金忠 , 王桂周 , 何国金 , 王慧慧 , 尹然宇 , 江威 , 张兆明 . 2020 . 遥感视频卫星运动车辆目标快速检测 . 遥感学报 , 24 ( 9 ): 1099 - 1107 [ DOI: 10.11834/jrs.20208364 http://dx.doi.org/10.11834/jrs.20208364 ]
Kopsiaftis G and Karantzalos K . 2015 . Vehicle detection and traffic density monitoring from very high resolution satellite video data // 2015 IEEE International Geoscience and Remote Sensing Symposium . Milan : IEEE: 1881 - 1884 [ DOI: 10.1109/IGARSS.2015.7326160 http://dx.doi.org/10.1109/IGARSS.2015.7326160 ]
LaLonde R , Zhang D and Shah M . 2018 . ClusterNet: detecting small objects in large scenes by exploiting spatio-temporal information // Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE: 4003 - 4012 [ DOI: 10.1109/CVPR.2018.00421 http://dx.doi.org/10.1109/CVPR.2018.00421 ]
Law H and Deng J . 2018 . CornerNet: detecting objects as paired keypoints // Proceedings of the 15th European Conference on Computer Vision (ECCV) . Munich : Springer: 765 - 781 [ DOI: 10.1007/978-3-030-01264-9_45 http://dx.doi.org/10.1007/978-3-030-01264-9_45 ]
Li C M , Bai H Y , Guo H W and Liang H J . 2018 . Moving object detection and tracking based on improved optical flow method . Chinese Journal of Scientific Instrument , 39 ( 5 ): 249 - 256
李成美 , 白宏阳 , 郭宏伟 , 梁华驹 . 2018 . 一种改进光流法的运动目标检测及跟踪算法 . 仪器仪表学报 , 39 ( 5 ): 249 - 256 [ DOI: 10.19650/j.cnki.cjsi.J1803270 http://dx.doi.org/10.19650/j.cnki.cjsi.J1803270 ]
Li F , Zhang H , Liu S L , Guo J , Ni L M and Zhang L . 2022 . DN-DETR: accelerate DETR training by introducing query DeNoising // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans : IEEE: 13609 - 13617 [ DOI: 10.1109/CVPR52688.2022.01325 http://dx.doi.org/10.1109/CVPR52688.2022.01325 ]
Liu H B and Chang F L . 2016 . Moving object detection by optical flow method based on adaptive weight coefficient . Optics and Precision Engineering , 24 ( 2 ): 460 - 468
刘洪彬 , 常发亮 . 2016 . 权重系数自适应光流法运动目标检测 . 光学精密工程 , 24 ( 2 ): 460 - 468 [ DOI: 10.3788/OPE.20162402.0460 http://dx.doi.org/10.3788/OPE.20162402.0460 ]
Liu T . 2014 . Research on the development of video satellites abroad . Space International , ( 9 ): 50 - 56
刘韬 . 2014 . 国外视频卫星发展研究 . 国际太空 , ( 9 ): 50 - 56
Liu X , Zhao G Y , Yao J W and Qi C . 2015 . Background subtraction based on low-rank and structured sparse decomposition . IEEE Transactions on Image Processing , 24 ( 8 ): 2502 - 2514 [ DOI: 10.1109/TIP.2015.2419084 http://dx.doi.org/10.1109/TIP.2015.2419084 ]
Luo Y L , Liang Y P and Wang Y . 2018 . Traffic flow parameter estimation from satellite video data based on optical flow . Computer Engineering and Applications , 54 ( 10 ): 204 - 207 , 255
罗亦乐 , 梁艳平 , 王妍 . 2018 . 基于光流法的卫星视频交通流参数提取研究 . 计算机工程与应用 , 54 ( 10 ): 204- 207 , 255 [ DOI: 10.3778/j.issn.1002-8331.1612-0537 http://dx.doi.org/10.3778/j.issn.1002-8331.1612-0537 ]
Pi Z L , Jiao L C , Liu F , Liu X , Li L L , Hou B and Yang S Y . 2022 . Very low-resolution moving vehicle detection in satellite videos . IEEE Transactions on Geoscience and Remote Sensing , 60 : 5624517 [ DOI: 10.1109/TGRS.2022.3179502 http://dx.doi.org/10.1109/TGRS.2022.3179502 ]
Ren S Q , He K M , Girshick R and Sun J . 2017 . Faster R-CNN: towards real-time object detection with region proposal networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 39 ( 6 ): 1137 - 1149 [ DOI: 10.1109/TPAMI.2016.2577031 http://dx.doi.org/10.1109/TPAMI.2016.2577031 ]
Shu M , Zhong Y F and Lv P Y . 2021 . Small moving vehicle detection via local enhancement fusion for satellite video . International Journal of Remote Sensing , 42 ( 19 ): 7189 - 7214 [ DOI: 10.1080/01431161.2021.1944694 http://dx.doi.org/10.1080/01431161.2021.1944694 ]
Song J S . 2014 . Study on Moving Object Detection and Tracking in Sequence Images . Guangzhou : South China University of Technology
宋佳声 . 2014 . 视频序列图像中运动目标检测与跟踪算法研究 . 广州 : 华南理工大学
Stauffer C and Grimson W E L . 1999 . Adaptive background mixture models for real-time tracking // 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Fort Collins : IEEE: 246 - 252 [ DOI: 10.1109/CVPR.1999.784637 http://dx.doi.org/10.1109/CVPR.1999.784637 ]
Sultana M , Mahmood A and Jung S K . 2021 . Unsupervised moving object detection in complex scenes using adversarial regularizations . IEEE Transactions on Multimedia , 23 : 2005 - 2018 [ DOI: 10.1109/TMM.2020.3006419 http://dx.doi.org/10.1109/TMM.2020.3006419 ]
Wang Y , Yin W T and Zeng J S . 2019 . Global convergence of ADMM in nonconvex nonsmooth optimization . Journal of Scientific Computing , 78 ( 1 ): 29 - 63 [ DOI: 10.1007/s10915-018-0757-z http://dx.doi.org/10.1007/s10915-018-0757-z ]
Wren C R , Azarbayejani A , Darrell T and Pentland A P . 1997 . Pfinder: real-time tracking of the human body . IEEE Transactions on Pattern Analysis and Machine Intelligence , 19 ( 7 ): 780 - 785 [ DOI: 10.1109/34.598236 http://dx.doi.org/10.1109/34.598236 ]
Wu J Q , Jiang Y H , Shen X , Li B B and Pan S L . 2019 . Satellite video motion detection supported by decision tree weak classification . Geomatics and Information Science of Wuhan University , 44 ( 8 ): 1182 - 1190
吴佳奇 , 蒋永华 , 沈欣 , 李贝贝 , 潘申林 . 2019 . 决策树弱分类支持的卫星视频运动检测 . 武汉大学学报 (信息科学版) , 44 ( 8 ): 1182 - 1190 [ DOI: 10.13203/j.whugis20180094 http://dx.doi.org/10.13203/j.whugis20180094 ]
Xiao C , Liu T , Ying X Y , Wang Y Q , Li M , Liu L , An W and Chen Z J . 2023 . Incorporating deep background prior into model-based method for unsupervised moving vehicle detection in satellite videos . IEEE Transactions on Geoscience and Remote Sensing , 61 : 5603114 [ DOI: 10.1109/TGRS.2023.3243055 http://dx.doi.org/10.1109/TGRS.2023.3243055 ]
Xiao C , Yin Q , Ying X Y , Li R J , Wu S L , Li M , Liu L , An W and Chen Z J . 2022 . DSFNet: dynamic and static fusion network for moving object detection in satellite videos . IEEE Geoscience and Remote Sensing Letters , 19 : 3510405 [ DOI: 10.1109/LGRS.2021.3124222 http://dx.doi.org/10.1109/LGRS.2021.3124222 ]
Xu W , Jin G and Wang J Q . 2017 . Optical imaging technology of JL-1 lightweight high resolution multispectral remote sensing satellite . Optics and Precision Engineering , 25 ( 8 ): 1969 - 1978
徐伟 , 金光 , 王家骐 . 2017 . 吉林一号轻型高分辨率遥感卫星光学成像技术 . 光学精密工程 , 25 ( 8 ): 1969 - 1978 [ DOI: 10.3788/OPE.20172508.1969 http://dx.doi.org/10.3788/OPE.20172508.1969 ]
Yang T , Wang X W , Yao B W , Li J , Zhang Y N , He Z N and Duan W C . 2016 . Small moving vehicle detection in a satellite video of an urban area . Sensors , 16 ( 9 ): 1528 [ DOI: 10.3390/s16091528 http://dx.doi.org/10.3390/s16091528 ]
Yin Q , Hu Q Y , Liu H , Zhang F , Wang Y Q , Lin Z P , An W and Guo Y L . 2022a . Detecting and tracking small and dense moving objects in satellite videos: a benchmark . IEEE Transactions on Geoscience and Remote Sensing , 60 : 5612518 [ DOI: 10.1109/TGRS.2021.3130436 http://dx.doi.org/10.1109/TGRS.2021.3130436 ]
Yin Q , Liu T , Lin Z P , An W and Guo Y L . 2022b . Moving object detection in satellite videos via spatial–temporal tensor model and weighted Schatten p -norm minimization . IEEE Geoscience and Remote Sensing Letters , 19 : 8022405 [ DOI: 10.1109/LGRS.2021.3117054 http://dx.doi.org/10.1109/LGRS.2021.3117054 ]
Yu Y B , Zhang T , Guo L H and He X J . 2017 . Moving objects detection on satellite video . Chinese Journal of Liquid Crystals and Displays , 32 ( 2 ): 138 - 143
于渊博 , 张涛 , 郭立红 , 贺小军 . 2017 . 卫星视频运动目标检测算法 . 液晶与显示 , 32 ( 2 ): 138 - 143 [ DOI: 10.3788/YJYXS20173202.0138 http://dx.doi.org/10.3788/YJYXS20173202.0138 ]
Yuan Y Q , He G J , Wang G Z , Jiang W and Kang J Z . 2018 . A background subtraction and frame subtraction combined method for moving vehicle detection in satellite video data . Journal of University of Chinese Academy of Sciences , 35 ( 1 ): 50 - 58
袁益琴 , 何国金 , 王桂周 , 江威 , 康金忠 . 2018 . 背景差分与帧间差分相融合的遥感卫星视频运动车辆检测方法 . 中国科学院大学学报 , 35 ( 1 ): 50 - 58 [ DOI: 10.7523/j.issn.2095-6134.2018.01.007 http://dx.doi.org/10.7523/j.issn.2095-6134.2018.01.007 ]
Yun K , Kim H , Bae K and Park J . 2021 . Unsupervised moving object detection through background models for PTZ camera // 2020 25th International Conference on Pattern Recognition . Milan : IEEE: 3201 - 3208 [ DOI: 10.1109/ICPR48806.2021.9413085 http://dx.doi.org/10.1109/ICPR48806.2021.9413085 ]
Zhang J P , Jia X P and Hu J K . 2019 . Local region proposing for frame-based vehicle detection in satellite videos . Remote Sensing , 11 ( 20 ): 2372 [ DOI: 10.3390/rs1120237 http://dx.doi.org/10.3390/rs1120237 ]
Zhang J P , Jia X P and Hu J K . 2020a . Error bounded foreground and background modeling for moving object detection in satellite videos . IEEE Transactions on Geoscience and Remote Sensing , 58 ( 4 ): 2659 - 2669 [ DOI: 10.1109/TGRS.2019.2953181 http://dx.doi.org/10.1109/TGRS.2019.2953181 ]
Zhang J P , Jia X P , Hu J K and Chanussot J . 2020b . Online structured sparsity-based moving-object detection from satellite videos . IEEE Transactions on Geoscience and Remote Sensing , 58 ( 9 ): 6420 - 6433 [ DOI: 10.1109/TGRS.2020.2976855 http://dx.doi.org/10.1109/TGRS.2020.2976855 ]
Zhang J P , Jia X P , Hu J K and Tan K . 2022 . Moving vehicle detection for remote sensing video surveillance with nonstationary satellite platform . IEEE Transactions on Pattern Analysis and Machine Intelligence , 44 ( 9 ): 5185 - 5198 [ DOI: 10.1109/TPAMI.2021.3066696 http://dx.doi.org/10.1109/TPAMI.2021.3066696 ]
Zhang J P , Zhang J and Jia X P . 2021 . Learning via watching: a weakly supervised moving object detector for satellite videos // 2021 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) . Brussels : IEEE: 2333 - 2336 [ DOI: 10.1109/IGARSS47720.2021.9554146 http://dx.doi.org/10.1109/IGARSS47720.2021.9554146 ]
Zhang X Y . 2017 . Study on Moving Objects Intelligent Sensing and Tracking Control for Video Satellite . Changsha : National University of Defense Technology
张学阳 . 2017 . 视频卫星运动目标智能感知与跟踪控制研究 . 长沙 : 国防科技大学
Zhou T Y and Tao D C . 2011 . GoDec: randomized low-rank & sparse matrix decomposition in noisy case // Proceedings of the 28th International Conference on Machine Learning . Bellevue : Omnipress: 33 - 40
Zhou X W , Yang C and Yu W C . 2013 . Moving object detection by detecting contiguous outliers in the low-rank representation . IEEE Transactions on Pattern Analysis and Machine Intelligence , 35 ( 3 ): 597 - 610 [ DOI: 10.1109/TPAMI.2012.132 http://dx.doi.org/10.1109/TPAMI.2012.132 ]
Zhou Y , Wang P and Fu D Y . 2015 . System Innovation and Enlightenment of SkySat . Spacecraft Engineering , 24 ( 5 ): 91 - 98
周宇 , 王鹏 , 傅丹膺 . 2015 . SkySat卫星的系统创新设计及启示 . 航天器工程 , 24 ( 5 ): 91 - 98 [ DOI: 10.3969/j.issn.1673-8748.2015.05.014 http://dx.doi.org/10.3969/j.issn.1673-8748.2015.05.014 ]
Zhuo T , Cheng Z Y , Zhang P , Wong Y and Kankanhalli M . 2020 . Unsupervised online video object segmentation with motion property understanding . IEEE Transactions on Image Processing , 29 : 237 - 249 [ DOI: 10.1109/TIP.2019.2930152 http://dx.doi.org/10.1109/TIP.2019.2930152 ]
Zivkovic Z and van der Heijden F . 2006 . Efficient adaptive density estimation per image pixel for the task of background subtraction . Pattern Recognition Letters , 27 ( 7 ): 773 - 780 [ DOI: 10.1016/j.patrec.2005.11.005 http://dx.doi.org/10.1016/j.patrec.2005.11.005 ]
相关文章
相关作者
相关机构
京公网安备11010802024621