A Shadow Detection Method for High-Resolution Remote Sensing Images based on Shadow Direction Prior
- Pages: 1-16(2024)
Published Online： 11 March 2024
DOI： 10.11834/jrs.20243314
扫 描 看 全 文
浏览全部资源
扫码关注微信
- About Journal
- Journal Online
- Special Source
摘要
地物阴影的标注成本昂贵且难以全面覆盖高分辨率遥感影像蕴含的丰富信息，其训练样本的匮乏，严重限制了监督学习模型的性能。针对上述问题，本文提出了一种基于阴影方向先验的高分辨率遥感影像地物阴影检测方法，该方法探究了阴影方向先验对于遥感影像地物阴影高级语义特征表达的有效性，并基于阴影的方向性先验构建了遥感地物阴影检测辅助任务，实现自监督的遥感地物阴影检测方法。本文方法设计了一种方向变换无关性噪声处理机制和一种自监督阴影检测的数据增强策略，进一步提升深度神经网络对于地物阴影关键特征的学习能力。在AISD数据集上的实验结果表明，本文方法仅使用少量的标签即可显著提升地物阴影检测精度，并且地物阴影的边界更加平滑和规整，更接近于地面真实情况。
Abstract
Objective.Due to the high cost and difficulty of shadow annotation
the shadow detection performance of supervised learning models in high-resolution remote sensing images is severely limited
mainly due to the lack of training samples. To solve this problem
this study proposes a shadow detection method based on shadow direction prior
which has unique advantages in learning the inherent shadow information contained in unlabeled high-resolution remote sensing images. Through self-supervised learning methods
we can deeply explore the advanced semantic information of ground object shadows contained in remote sensing images
thereby improving the applicability and accuracy of the model. This innovative approach provides a new way to overcome the data limitation problem in shadow detection and brings new possibilities for improving the model's ability to understand complex high-resolution images.
Methods
2
.This paper proposes an innovative shadow detection method for high-resolution remote sensing images
which is based on prior knowledge of shadow direction and explores the effectiveness of shadow direction prior in representing high-level semantic features of object shadows in remote sensing images. The key of this method is to construct an auxiliary task through the directional prior of shadow
so that the deep neural network can better understand the shadow of objects in remote sensing images. Moreover
in order to further enhance the deep neural network's ability to learn key features of object shadows
this method designs a direction transformation-independent noise processing mechanism and data enhancement strategy for self-supervised shadow detection
which improves the generalization ability of the model.
Results
2
.This paper confirms the effectiveness of the proposed method through verification on the AISD data set. Experimental results show that compared with the traditional Unet benchmark model
our method can successfully learn prior knowledge in unlabeled high-resolution remote sensing images. By leveraging this prior knowledge
our method performs well in improving the accuracy of the Unet model and demonstrates competitive efficiency and performance in small sample cases. The results of this study highlight the superiority of the proposed method in addressing the challenge of shadow identification in high-resolution remote sensing images
especially in the absence of labeled samples. By fully utilizing prior knowledge
our method successfully extends the performance of the Unet model and provides a promising solution for processing unlabeled high-resolution remote sensing images in practical applications.
Conclusion
2
.The qualitative and quantitative results of this paper consistently show that using a shadow direction prior method can significantly improve the accuracy of shadow detection. The use of DTICT strategy helps to alleviate the interference caused by shadow-free samples and samples containing shadows in special directions during network training. The data enhancement strategy combined with contrast cropping and color transformation can provide the network with more diverse training samples
thereby enhancing the pre-trained model's ability to extract object shadow features. Ultimately
this method achieves competitive object shadow detection accuracy using a small number of labeled samples.
关键词
阴影检测自监督学习数据增强阴影方向先验高分辨率遥感影像
Keywords
Shadow detectionself-supervised learningdata augmentationshadow direction priorhigh-resolution remote sensing imagery
references
Belkin M, Hsu D, Ma S and Mandal S. 2019. Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences, 116(32): 15849-15854 [DOI:10.1073/pnas.1903070116http://dx.doi.org/10.1073/pnas.1903070116]
Chao T, Ziwei Y, Qing Z and Haifeng L I. 2021. Remote sensing image intelligent interpretation: from supervised learning to self-supervised learning. Acta Geodaetica et Cartographica Sinica, 50(8): 1122 [DOI:10.11947/j.AGCS.2021.20210089http://dx.doi.org/10.11947/j.AGCS.2021.20210089]
Han B, Yao Q, Yu X, Niu G, Xu M, Hu W, Tsang I and Sugiyama M. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31 [DOI:10.48550/arXiv.1804.06872http://dx.doi.org/10.48550/arXiv.1804.06872]
He K, Fan H, Wu Y, Xie S and Girshick R. 2020. Momentum contrast for unsupervised visual representation learning//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Virtual: IEEE: 9729-9738 [DOI: 10.1109/CVPR42600.2020.00975http://dx.doi.org/10.1109/CVPR42600.2020.00975]
He K, Zhang X, Ren S and Sun J. 2016. Deep residual learning for image recognition//Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas: IEEE: 770-778 [DOI: 10.48550/arXiv.1512.03385http://dx.doi.org/10.48550/arXiv.1512.03385]
Hou L, Vicente T F Y, Hoai M and Samaras D. 2019. Large scale shadow annotation and detection using lazy annotation and stacked CNNs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(4): 1337-1351 [DOI:10.1109/TPAMI.2019.2948011http://dx.doi.org/10.1109/TPAMI.2019.2948011]
Ji S, Wei S and Lu M. 2018. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set. IEEE Transactions on Geoscience and Remote Sensing, 57(1): 574-586 [DOI:10.1109/TGRS.2018.2858817http://dx.doi.org/10.1109/TGRS.2018.2858817]
Jing L and Tian Y. 2020. Self-supervised visual feature learning with deep neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 43(11): 4037-4058 [DOI:10.1109/TPAMI.2020.2992393http://dx.doi.org/10.1109/TPAMI.2020.2992393]
Jung H, Oh Y, Jeong S, Lee C and Jeon T. 2021. Contrastive self-supervised learning with smoothed representation for remote sensing. IEEE Geoscience and Remote Sensing Letters, 19: 1-5 [DOI:10.1109/LGRS.2021.3069799http://dx.doi.org/10.1109/LGRS.2021.3069799]
Khan S H, Bennamoun M, Sohel F and Togneri R. 2014. Automatic feature learning for robust shadow detection//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 1939-1946 [DOI: 10.1109/CVPR.2014.249http://dx.doi.org/10.1109/CVPR.2014.249]
Li J, Hu Q and Ai M. 2016. Joint model and observation cues for single-image shadow detection. Remote Sensing, 8(6): 484 [DOI:10.3390/rs8060484http://dx.doi.org/10.3390/rs8060484]
Lin Z, Sun J, Davis A and Snavely N. 2020. Visual chirality//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE: 12295-12303 [DOI: 10.1109/CVPR42600.2020.01231http://dx.doi.org/10.1109/CVPR42600.2020.01231]
Liu D, Zhang J, Wu Y and Zhang Y. 2021. A shadow detection algorithm based on multiscale spatial attention mechanism for aerial remote sensing images. IEEE Geoscience and Remote Sensing Letters, 19: 1-5 [DOI:10.1109/LGRS.2021.3100294http://dx.doi.org/10.1109/LGRS.2021.3100294]
Liu X, Zhang F, Hou Z, Mian L, Wang Z, Zhang J and Tang J. 2021. Self-supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering, 35(1): 857-876 [DOI:10.1109/TKDE.2021.3090866http://dx.doi.org/10.1109/TKDE.2021.3090866]
Luo S, Li H and Shen H. 2020a. Deeply supervised convolutional neural network for shadow detection based on a novel aerial shadow imagery dataset. ISPRS Journal of Photogrammetry and Remote Sensing, 167: 443-457 [DOI:10.1016/j.isprsjprs.2020.07.016http://dx.doi.org/10.1016/j.isprsjprs.2020.07.016]
Luo S, Li H and Shen H. 2020b. Deeply supervised convolutional neural network for shadow detection based on a novel aerial shadow imagery dataset. ISPRS Journal of Photogrammetry and Remote Sensing, 167: 443-457 [DOI:10.1016/j.isprsjprs.2020.07.016http://dx.doi.org/10.1016/j.isprsjprs.2020.07.016]
Luo S, Li H, Zhu R, Gong Y and Shen H. 2021. ESPFNet: An edge-aware spatial pyramid fusion network for salient shadow detection in aerial remote sensing images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14: 4633-4646
Makarau A, Richter R, Muller R and Reinartz P. 2011. Adaptive shadow detection using a blackbody radiator model. IEEE Transactions on Geoscience and Remote Sensing, 49(6): 2049-2059 [DOI:10.1109/TGRS.2010.2096515http://dx.doi.org/10.1109/TGRS.2010.2096515]
Peng X, Wang K, Zhu Z, Wang M and You Y. 2022. Crafting better contrastive views for siamese representation learning//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE: 16031-16040 [DOI: 10.48550/arXiv.2202.03278http://dx.doi.org/10.48550/arXiv.2202.03278]
Selvaraju R R, Cogswell M, Das A, Vedantam R, Parikh D and Batra D. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization//Proceedings of the IEEE international conference on computer vision. Venice: IEEE: 618-626 [DOI: 10.1109/ICCV.2017.74http://dx.doi.org/10.1109/ICCV.2017.74]
Shahtahmassebi A, Yang N, Wang K, Moore N and Shen Z. 2013. Review of shadow detection and de-shadowing methods in remote sensing. Chinese geographical science, 23: 403-420 [DOI:10.1007/s11769-013-0613-xhttp://dx.doi.org/10.1007/s11769-013-0613-x]
Shen L, Wee Chua T and Leman K. 2015. Shadow optimization from structured deep edge detection//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE: 2067-2074 [DOI: 10.1109/CVPR.2015.7298818http://dx.doi.org/10.1109/CVPR.2015.7298818]
Shi L, Fang J and Zhao Y. 2023. Automatic shadow detection in high-resolution multispectral remote sensing images. Computers and Electrical Engineering, 105: 108557
Stojnic V and Risojevic V. 2021. Self-supervised learning of remote sensing scene representations using contrastive multiview coding//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Virtual: IEEE: 1182-1191 [DOI: 10.48550/arXiv.2104.07070http://dx.doi.org/10.48550/arXiv.2104.07070]
Su N, Zhang Y, Tian S, Yan Y and Miao X. 2016. Shadow detection and removal for occluded object information recovery in urban high-resolution panchromatic satellite images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(6): 2568-2582 [DOI:10.1109/JSTARS.2016.2570234http://dx.doi.org/10.1109/JSTARS.2016.2570234]
Tao C, Qi J, Lu W, Wang H and Li H. 2020. Remote sensing image scene classification with self-supervised paradigm under limited labeled samples. IEEE Geoscience and Remote Sensing Letters, 19: 1-5 [DOI:10.1109/LGRS.2020.3038420http://dx.doi.org/10.1109/LGRS.2020.3038420]
Wang Q, Yan L, Yuan Q and Ma Z. 2017. An automatic shadow detection method for VHR remote sensing orthoimagery. Remote Sensing, 9(5): 469 [DOI:10.3390/rs9050469http://dx.doi.org/10.3390/rs9050469]
Xie Y, Feng D, Chen H, Liao Z, Zhu J, Li C and Baik S W. 2022. An omni-scale global–local aware network for shadow extraction in remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 193: 29-44 [DOI:10.1016/j.isprsjprs.2022.09.004http://dx.doi.org/10.1016/j.isprsjprs.2022.09.004]
Yang Y, Guo M and Zhu Q. 2021. CADNet: Top-Down Contextual Saliency Detection Network for High Spatial Resolution Remote Sensing Image Shadow Detection//2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS. Brussels: IEEE: 4075-4078 [DOI: 10.1109/IGARSS47720.2021.9553138http://dx.doi.org/10.1109/IGARSS47720.2021.9553138]
Zhu Q, Yang Y, Sun X and Guo M. 2022. CDANet: Contextual Detail-Aware Network for High-Spatial-Resolution Remote-Sensing Imagery Shadow Detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 1-15 [DOI:10.1109/TGRS.2022.3143886http://dx.doi.org/10.1109/TGRS.2022.3143886]
Duan G Y, Gong H L, Li X J, Chen B B. 2014. Shadow extraction based on characteristic components and object-oriented method for high-resolution images. Journal of Remote Sensing, 18(4): 760-770
段光耀, 宫辉力, 李小娟, 陈蓓蓓. 2014. 结合特征分量构建和面向对象方法提取高分辨率卫星影像阴影. 遥感学报, 18(4): 760-770 [DOI:10.11834 /jrs.20143243http://dx.doi.org/10.11834/jrs.20143243]
Li D R, Tong Q X, Li R X, Gong J Y, Zhang L P. 2012. Current issues in high-resolution Earth observation technology. Sci China Earth Sci, 42(6): 805-813
李德仁, 童庆禧, 李荣兴, 龚健雅, 张良培. 2012. 高分辨率对地观测的若干前沿科学问题. 中国科学: 地球科学, 42(6): 805-813 [DOI:10.1007/s11430-012-4445-9http://dx.doi.org/10.1007/s11430-012-4445-9]
Li Y, Gong P. 2005. Integrating Photogrammetry and Image Analysis for Shadow Detection. Journal of Remote Sensing, 9(4): 357-362
李艳, 宫鹏. 2005. 基于 DSM 阴影仿真和高度场光线跟踪的影像阴影检测. 遥感学报, 9(4): 357-362 [DOI:10.3321/j.issn:1007-4619.2005.04.004http://dx.doi.org/10.3321/j.issn:1007-4619.2005.04.004]
Liao C J, Zhao J, Xing J, Wen J G, Liu Q H. 2023. Research on classification of GF satellite data application products and construction of common product system. National Remote Sensing Bulletin, 27(3)
廖楚江, 赵坚, 邢进, 闻建光, 柳钦火. 2023. 高分卫星数据应用产品分级与共性产品体系 构建研究. Journal of Remote Sensing, 27(3) [DOI:10. 11834/jrs.20232593]
Ma C, Feng D J, Zhang L. 2012. Detecting the Building Shadow with the Two Algorithms of Threshold Segmentation and Mathematical Morphology. Surveying and Mapping, 35(4): 151-154
马川, 冯德俊, 张丽. 2012. 结合阈值分割和数学形态学的建筑物阴影检测. 测绘, 35(4): 151-154 [DOI:CNKI:SUN:SCCH.0.2012-04-003http://dx.doi.org/CNKI:SUN:SCCH.0.2012-04-003]
Mu X D, Bai K, You X, Zhu Y Q, Chen X B. 2021. Remote sensing image feature extraction and classification based on contrastive learning method. Optics and Precision Engineering, 29(9): 2222
慕晓冬, 白坤, 尤轩昂, 朱永清, 陈雪冰. 2021. 基于对比学习方法的遥感影像特征提取与分类. 光学 精密工程, 29(9): 2222 [DOI:10. 37188/OPE. 20212909. 2222http://dx.doi.org/10.37188/OPE.20212909.2222]
Tong X D. 2016. Development of China high-resolution earth observation system. Journal of Remote Sensing, 20(5): 775-780
童旭东. 2016. 中国高分辨率对地观测系统重大专项建设进展. 遥感学报, 20(5): 775-780 [DOI:10.11834/jrs.20166302http://dx.doi.org/10.11834/jrs.20166302]
Yu D F, Yin J P, Zhang G M. 2008. An Automatic Shadow Detection Method for Remote Sensing Images Based on Gray Histogram. Computer Engineering &Science, 30(12): 43-44
于东方, 殷建平, 张国敏. 2008. 一种基于灰度直方图的遥感影像阴影自动检测方法. 计算机工程与科学, 30(12): 43-44 [DOI:10.3969/j.issn.1007-130X.2008.12.011http://dx.doi.org/10.3969/j.issn.1007-130X.2008.12.011]
Zhou P C, Cheng G, Yao X W, Han J W. 2021. Machine learning paradigms in high-resolution remote sensing image interpretation. Journal of Remote Sensing, 25(1)
周培诚, 程塨, 姚西文, 韩军伟. 2021. 高分辨率遥感影像解译中的机器学习范式. 遥感学报, 25(1) [DOI:10.11834/jrs.20210164]
Alert me when the article has been cited
提交
相关文章
相关作者
暂无数据
相关机构
Fundamental Science on Communication Information Transmission and Fusion Technology Laboratory, Hangzhou Dianzi University
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University
National Engineering Research Center of Geographic Information System
Chinese Academy of Surveying and Mapping
National Quality Inspection and Testing Center for Surveying and Mapping Products State
选择翻译语言
- Postal code：100190
- Tel：010-58887052 Email：nrsb@aircas.ac.cn
- Technical support is provided by Beijing Founder electronics co., LTD 京ICP备20021838号-9
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
0
Bulk Citation
Export to
TOP