结合多光谱影像降维与深度学习的城市单木树冠检测

奚祥书; 夏凯; 杨垠晖; 杜晓晨; 冯海林

doi:10.11834/jrs.20220163

智能遥感处理与分析 | 浏览量 : 0 下载量: 199 CSCD: 1

R-PDF
PDF
导出
分享
收藏
专辑

结合多光谱影像降维与深度学习的城市单木树冠检测
Urban individual tree crown detection research using multispectral image dimensionality reduction with deep learning
2022年26卷第4期页码：711-721
DOI： 10.11834/jrs.20220163

扫描看全文

奚祥书, 夏凯, 杨垠晖, 等. 结合多光谱影像降维与深度学习的城市单木树冠检测[J]. 遥感学报, 2022,26(4):711-721.

XI Xiangshu, XIA Kai, YANG Yinhui, et al. Urban individual tree crown detection research using multispectral image dimensionality reduction with deep learning[J]. National Remote Sensing Bulletin, 2022,26(4):711-721.
奚祥书, 夏凯, 杨垠晖, 等. 结合多光谱影像降维与深度学习的城市单木树冠检测[J]. 遥感学报, 2022,26(4):711-721. DOI： 10.11834/jrs.20220163.

XI Xiangshu, XIA Kai, YANG Yinhui, et al. Urban individual tree crown detection research using multispectral image dimensionality reduction with deep learning[J]. National Remote Sensing Bulletin, 2022,26(4):711-721. DOI： 10.11834/jrs.20220163.

摘要

多光谱数据的降维处理对基于深度学习的单木树冠检测研究有重要意义，如何使用合适的降维方法以提高单木检测的精度却少有研究讨论。本文使用无人机搭载多光谱相机进行航拍作业，采集研究区内银杏树种多光谱影像。将原始多光谱影像通过特征波段选择、特征提取、波段组合的方法生成5种不同的数据集用于训练3种经典的深度学习网络FPN-Faster-R-CNN，YOLOv3，Faster R-CNN。其中由波段组合方法得到的近红外、红色、绿色波段组合在不同类型的目标检测网络中都有最好的检测结果，其中FPN-Faster-R-CNN网络对银杏树冠的检测精度最高为88.4%，由OIF指标得到的蓝色、红色、近红外波段组合信息量最高，但在所有网络中的平均检测精度最低，仅为79.3%。实验结果表明：在不同波段降维方法中，若降维后的影像中目标物体的色彩与背景差异较明显，且轮廓清晰，则深度学习网络对树冠的检测可获得较好的结果。而影像自身的信息量则对深度学习网络的树冠检测能力的提升作用有限。本研究中针对多光谱影像的降维方法分析，为基于深度学习的单木树冠检测研究提供了重要的实验参考。

Abstract

The dimensionality reduction processing of multispectral data is of considerable importance to deep learning-based single-tree crown detection research. However, how to use the appropriate dimensionality reduction method to improve the accuracy of single-tree detection is rarely discussed. In this work, an unmanned aerial vehicle equipped with a multispectral camera was used for aerial photography to collect multispectral images of ginkgo tree species in the research area. The original multispectral images were used to generate five different data sets through feature band selection, feature extraction, and band combination method for training three classical deep learning networks: FPN-Faster-R-CNN, YOLOv3, and Faster R-CNN. Based on the characteristics of the band selection method, the red, green, and near-infrared bands combined with different types of target detection in the network have the best results. The FPN-Faster-R-CNN network detection accuracy is up to 88.4% for ginkgo trees. The blue, red, and near-infrared band combination obtained by the OIF index has the highest amount of information but the lowest average network accuracy at 79.3%. Experimental results show the following. In the different dimensionality reduction methods, if the color and background of the target object in the image after dimensionality reduction are obviously different, and the contour is clear, the deep learning network can obtain better results in tree crown detection. However, the information content of the image itself has a limited effect on the ability of the deep learning network to detect tree crowns. In this study, the dimensionality reduction method of multispectral images is analyzed, providing an important experimental reference for the deep learning-based single-tree crown detection.

关键词

遥感单木树冠检测深度学习无人机多光谱影像降维

Keywords

remote sensingindividual tree crown detectiondeep learningunmanned aerial vehiclemultispectral imagesdimensionality reduction

references

Ampatzidis Y and Partel V. 2019. UAV-based high throughput phenotyping in citrus utilizing multispectral imaging and artificial intelligence. Remote Sensing, 11(4): 410 [DOI: 10.3390/rs11040410http://dx.doi.org/10.3390/rs11040410]

Bunting P and Lucas R. 2006. The delineation of tree crowns in Australian mixed species forests using hyperspectral Compact Airborne Spectrographic Imager (CASI) data. Remote Sensing of Environment, 101(2): 230-248 [DOI: 10.1016/j.rse.2005.12.015http://dx.doi.org/10.1016/j.rse.2005.12.015]

Chavez Jr P S, Berlin G L and Sowers L B. 1982. Statistical method for selecting Landsat MSS Ratios. Journal of Applied Photographic Engineering, 8(1): 23-30.

Cui B G, Wu Y N, Zhong Y, Zhong L W and Lu Y. 2019. Hyperspectral image rolling guidance recursive filtering and classification. Journal of Remote Sensing, 23(3): 431-442

崔宾阁, 吴亚男, 钟勇, 钟利伟, 路燕. 2019. 高光谱图像滚动引导递归滤波与地物分类. 遥感学报, 23(3): 431-442 [DOI: 10.11834/jrs.20197510http://dx.doi.org/10.11834/jrs.20197510]

Dai W X, Yang B S, Dong Z and Shaker A. 2018. A new method for 3D individual tree extraction using multispectral airborne LiDAR point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 144: 400-411 [DOI: 10.1016/j.isprsjprs.2018.08.010http://dx.doi.org/10.1016/j.isprsjprs.2018.08.010]

Dong X Y, Li J G, Chen H Y, Zhao L, Zhang L M and Xing S H. 2019. Extraction of individual tree information based on remote sensing images from an Unmanned Aerial Vehicle. Journal of Remote Sensing, 23(6): 1269-1280

董新宇, 李家国, 陈瀚阅, 赵磊, 张黎明, 邢世和. 2019. 无人机遥感影像林地单株立木信息提取. 遥感学报, 23(6): 1269-1280 [DOI: 10.11834/jrs.20198073http://dx.doi.org/10.11834/jrs.20198073]

Feng J J, Liu H L and Zhang X L. 2017. Single tree crown extraction based on gray gradient image segmentation. Journal of Beijing Forestry University, 39(3): 16-23

冯静静, 刘会玲, 张晓丽. 2017. 基于灰度梯度图像分割的单木树冠提取研究. 北京林业大学学报, 39(3): 16-23 [DOI: 10.13332/j.1000-1522.20160373http://dx.doi.org/10.13332/j.1000-1522.20160373]

Gougeon F A. 1995. A crown-following approach to the automatic delineation of individual tree crowns in high spatial resolution aerial images. Canadian Journal of Remote Sensing, 21(3): 274-284 [DOI: 10.1080/07038992.1995.10874622http://dx.doi.org/10.1080/07038992.1995.10874622]

Ke Y and Quackenbush L J. 2011. A review of methods for automatic individual tree-crown detection and delineation from passive remote sensing. International Journal of Remote Sensing, 32(17): 4725-4747 [DOI: 10.1080/01431161.2010.494184http://dx.doi.org/10.1080/01431161.2010.494184]

Kellenberger B, Marcos D and Tuia D. 2018. Detecting mammals in UAV images: best practices to address a substantially imbalanced dataset with deep learning. Remote Sensing of Environment, 216: 139-153 [DOI: 10.1016/j.rse.2018.06.028http://dx.doi.org/10.1016/j.rse.2018.06.028]

Korpela I, Anttila P and Pitkänen J. 2006. The performance of a local maxima method for detecting individual tree tops in aerial photographs. International Journal of Remote Sensing, 27(6): 1159-1175 [DOI: 10.1080/01431160500354070http://dx.doi.org/10.1080/01431160500354070]

Lee J H, Ko Y and McPherson E G. 2016. The feasibility of remotely sensed data to estimate urban tree dimensions and biomass. Urban Forestry and Urban Greening, 16: 208-220 [DOI: 10.1016/j.ufug.2016.02.010http://dx.doi.org/10.1016/j.ufug.2016.02.010]

Li D, Zhang J J and Zhao M X. 2019. Extraction of stand factors in UAV image based on FCM and watershed algorithm. Scientia Silvae Sinicae, 55(5): 181-187

李丹, 张俊杰, 赵梦溪. 2019. 基于FCM和分水岭算法的无人机影像中林分因子提取. 林业科学, 55(5): 181-187 [DOI: 10.11707/j.1001-7488.20190520http://dx.doi.org/10.11707/j.1001-7488.20190520]

Lin T Y, Dollár P, Girshick R, He K M, Hariharan B and Belongie S. 2017. Feature pyramid networks for object detection//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE: 936-944 [DOI: 10.1109/CVPR.2017.106http://dx.doi.org/10.1109/CVPR.2017.106]

Liu X S, Ge L, Wang B and Zhang L M. 2012. An unsupervised band selection algorithm for hyperspectral imagery based on maximal information. Journal of Infrared and Millimeter Waves, 31(2): 166-170, 176

刘雪松, 葛亮, 王斌, 张立明. 2012. 基于最大信息量的高光谱遥感图像无监督波段选择方法. 红外与毫米波学报, 31(2): 166-170, 176

Majeed Y, Zhang J, Zhang X, Fu L S, Karkee M, Zhang Q and Whiting M D. 2020. Deep learning based segmentation for automated training of apple trees on trellis wires. Computers and Electronics in Agriculture, 170: 105277 [DOI: 10.1016/j.compag.2020.105277http://dx.doi.org/10.1016/j.compag.2020.105277]

Mozgeris G, Juodkienė V, Jonikavičius D, Straigytė L, Gadal S and Ouerghemmi W. 2018. Ultra-light aircraft-based hyperspectral and colour-infrared imaging to identify deciduous tree species in an urban environment. Remote Sensing, 10(10): 1668 [DOI: 10.3390/rs10101668http://dx.doi.org/10.3390/rs10101668]

Osco L P, dos Santos de Arruda M, Marcato Junior J, da Silva N B, Ramos A P M, Moryia É A S, Imai N N, Pereira D R, Creste J E, Matsubara E T, Li J and Gonçalves W N. 2020. A convolutional neural network approach for counting and geolocating citrus-trees in UAV multispectral imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 160: 97-106 [DOI: 10.1016/j.isprsjprs.2019.12.010http://dx.doi.org/10.1016/j.isprsjprs.2019.12.010]

Pan S J and Yang Q. 2010. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10): 1345-1359 [DOI: 10.1109/TKDE.2009.191http://dx.doi.org/10.1109/TKDE.2009.191]

Puliti S, Solberg S and Granhus A. 2019. Use of UAV photogrammetric data for estimation of biophysical properties in forest stands under regeneration. Remote Sensing, 11(3): 233 [DOI: 10.3390/rs11030233http://dx.doi.org/10.3390/rs11030233]

Redmon J and Farhadi A. 2018. YOLOv3: An incremental improvement. [2019.12.1]. https://arxiv.org/abs/1804.02767https://arxiv.org/abs/1804.02767.

Ren S Q, He K M, Girshick R and Sun J. 2017. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137-1149 [DOI: 10.1109/TPAMI.2016.2577031http://dx.doi.org/10.1109/TPAMI.2016.2577031]

Roth S I B, Leiterer R, Volpi M, Celio E, Schaepman M E and Joerg P C. 2019. Automated detection of individual clove trees for yield quantification in northeastern Madagascar based on multi-spectral satellite data. Remote Sensing of Environment, 221: 144-156 [DOI: 10.1016/J.RSE.2018.11.009http://dx.doi.org/10.1016/J.RSE.2018.11.009]

Wang J M, Chen X X, Cao L, An F, Chen B Q, Xue L F and Yun T. 2019. Individual rubber tree segmentation based on ground-based LiDAR data and faster R-CNN of deep learning. Forests, 10(9): 793 [DOI: 10.3390/f10090793http://dx.doi.org/10.3390/f10090793]

Weinstein B G, Marconi S, Bohlman S, Zare A and White E. 2019. Individual tree-crown detection in RGB imagery using Semi-supervised deep learning neural networks. Remote Sensing, 11(11): 1309 [DOI: 10.3390/rs11111309http://dx.doi.org/10.3390/rs11111309]

Weinstein B G, Marconi S, Bohlman S A, Zare A and White E P. 2020. Cross-site learning in deep learning RGB tree crown detection. Ecological Informatics, 56: 101061 [DOI: 10.1016/j.ecoinf.2020.101061http://dx.doi.org/10.1016/j.ecoinf.2020.101061]

Wold S, Esbensen K and Geladi P. 1987. Principal component analysis. Chemometrics and Intelligent Laboratory Systems, 2(1/3): 37-52 [DOI: 10.1016/0169-7439(87)80084-9http://dx.doi.org/10.1016/0169-7439(87)80084-9]

Wu X W, Sahoo D and Hoi S C H. 2020. Recent advances in deep learning for object detection. Neurocomputing, 396: 39-64 [DOI: 10.1016/j.neucom.2020.01.085http://dx.doi.org/10.1016/j.neucom.2020.01.085]

Zhao Q Z, Liu W, Yin X J and Zhang T Y. 2016. Selection of optimum bands combination based on multispectral images of UAV. Transactions of the Chinese Society for Agricultural Machinery, 47(3): 242-248, 291

赵庆展, 刘伟, 尹小君, 张天毅. 2016. 基于无人机多光谱影像特征的最佳波段组合研究. 农业机械学报, 47(3): 242-248, 291 [DOI: 10.6041/j.issn.1000-1298.2016.03.034http://dx.doi.org/10.6041/j.issn.1000-1298.2016.03.034]

Zhen Z, Quackenbush L J and Zhang L J. 2016. Trends in automatic individual tree crown detection and delineation-evolution of LiDAR data. Remote Sensing, 8(4): 333 [DOI: 10.3390/rs8040333http://dx.doi.org/10.3390/rs8040333]

文章被引用时，请邮件提醒。

提交

基于深度学习的全极化SAR影像冰川边界识别

深度学习的遥感变化检测综述：文献计量与分析

人工智能时代的遥感变化检测技术：继承、发展与挑战

生成对抗网络的无监督高光谱解混