结合多光谱影像降维与深度学习的城市单木树冠检测

奚祥书; 夏凯; 杨垠晖; 杜晓晨; 冯海林

doi:10.11834/jrs.20220163

智能遥感处理与分析 | 浏览量 : 0 下载量: 2015 CSCD: 7

R-PDF
PDF
导出
分享
收藏
专辑

结合多光谱影像降维与深度学习的城市单木树冠检测
Urban individual tree crown detection research using multispectral image dimensionality reduction with deep learning
2022年26卷第4期页码：711-721
收稿：2020-05-19，

纸质出版：2022-04-07
DOI： 10.11834/jrs.20220163
稿件说明：

移动端阅览

奚祥书，夏凯，杨垠晖，杜晓晨，冯海林.2022.结合多光谱影像降维与深度学习的城市单木树冠检测.遥感学报，26（4）： 711-721 DOI： 10.11834/jrs.20220163.

Xi X S，Xia K，Yang Y H，Du X C and Feng H L. 2022. Urban individual tree crown detection research using multispectral image dimensionality reduction with deep learning. National Remote Sensing Bulletin， 26（4）：711-721 DOI： 10.11834/jrs.20220163.

摘要

多光谱数据的降维处理对基于深度学习的单木树冠检测研究有重要意义，如何使用合适的降维方法以提高单木检测的精度却少有研究讨论。本文使用无人机搭载多光谱相机进行航拍作业，采集研究区内银杏树种多光谱影像。将原始多光谱影像通过特征波段选择、特征提取、波段组合的方法生成5种不同的数据集用于训练3种经典的深度学习网络FPN-Faster-R-CNN，YOLOv3，Faster R-CNN。其中由波段组合方法得到的近红外、红色、绿色波段组合在不同类型的目标检测网络中都有最好的检测结果，其中FPN-Faster-R-CNN网络对银杏树冠的检测精度最高为88.4%，由OIF指标得到的蓝色、红色、近红外波段组合信息量最高，但在所有网络中的平均检测精度最低，仅为79.3%。实验结果表明：在不同波段降维方法中，若降维后的影像中目标物体的色彩与背景差异较明显，且轮廓清晰，则深度学习网络对树冠的检测可获得较好的结果。而影像自身的信息量则对深度学习网络的树冠检测能力的提升作用有限。本研究中针对多光谱影像的降维方法分析，为基于深度学习的单木树冠检测研究提供了重要的实验参考。

Abstract

The dimensionality reduction processing of multispectral data is of considerable importance to deep learning-based single-tree crown detection research. However

how to use the appropriate dimensionality reduction method to improve the accuracy of single-tree detection is rarely discussed. In this work

an unmanned aerial vehicle equipped with a multispectral camera was used for aerial photography to collect multispectral images of ginkgo tree species in the research area. The original multispectral images were used to generate five different data sets through feature band selection

feature extraction

and band combination method for training three classical deep learning networks: FPN-Faster-R-CNN

YOLOv3

and Faster R-CNN. Based on the characteristics of the band selection method

the red

green

and near-infrared bands combined with different types of target detection in the network have the best results. The FPN-Faster-R-CNN network detection accuracy is up to 88.4% for ginkgo trees. The blue

red

and near-infrared band combination obtained by the OIF index has the highest amount of information but the lowest average network accuracy at 79.3%. Experimental results show the following. In the different dimensionality reduction methods

if the color and background of the target object in the image after dimensionality reduction are obviously different

and the contour is clear

the deep learning network can obtain better results in tree crown detection. However

the information content of the image itself has a limited effect on the ability of the deep learning network to detect tree crowns. In this study

the dimensionality reduction method of multispectral images is analyzed

providing an important experimental reference for the deep learning-based single-tree crown detection.

关键词

Keywords

references

Ampatzidis Y and Partel V . 2019 . UAV-based high throughput phenotyping in citrus utilizing multispectral imaging and artificial intelligence . Remote Sensing , 11 ( 4 ): 410 [ DOI: 10.3390/rs11040410 http://dx.doi.org/10.3390/rs11040410 ]

Bunting P and Lucas R . 2006 . The delineation of tree crowns in Australian mixed species forests using hyperspectral Compact Airborne Spectrographic Imager (CASI) data . Remote Sensing of Environment , 101 ( 2 ): 230 - 248 [ DOI: 10.1016/j.rse.2005.12.015 http://dx.doi.org/10.1016/j.rse.2005.12.015 ]

Chavez Jr P S , Berlin G L and Sowers L B . 1982 . Statistical method for selecting Landsat MSS Ratios . Journal of Applied Photographic Engineering , 8 ( 1 ): 23 - 30 .

Cui B G , Wu Y N , Zhong Y , Zhong L W and Lu Y . 2019 . Hyperspectral image rolling guidance recursive filtering and classification . Journal of Remote Sensing , 23 ( 3 ): 431 - 442

崔宾阁 , 吴亚男 , 钟勇 , 钟利伟 , 路燕 . 2019 . 高光谱图像滚动引导递归滤波与地物分类 . 遥感学报 , 23 ( 3 ): 431 - 442 [ DOI: 10.11834/jrs.20197510 http://dx.doi.org/10.11834/jrs.20197510 ]

Dai W X , Yang B S , Dong Z and Shaker A . 2018 . A new method for 3D individual tree extraction using multispectral airborne LiDAR point clouds . ISPRS Journal of Photogrammetry and Remote Sensing , 144 : 400 - 411 [ DOI: 10.1016/j.isprsjprs.2018.08.010 http://dx.doi.org/10.1016/j.isprsjprs.2018.08.010 ]

Dong X Y , Li J G , Chen H Y , Zhao L , Zhang L M and Xing S H . 2019 . Extraction of individual tree information based on remote sensing images from an Unmanned Aerial Vehicle . Journal of Remote Sensing , 23 ( 6 ): 1269 - 1280

董新宇 , 李家国 , 陈瀚阅 , 赵磊 , 张黎明 , 邢世和 . 2019 . 无人机遥感影像林地单株立木信息提取 . 遥感学报 , 23 ( 6 ): 1269 - 1280 [ DOI: 10.11834/jrs.20198073 http://dx.doi.org/10.11834/jrs.20198073 ]

Feng J J , Liu H L and Zhang X L . 2017 . Single tree crown extraction based on gray gradient image segmentation . Journal of Beijing Forestry University , 39 ( 3 ): 16 - 23

冯静静 , 刘会玲 , 张晓丽 . 2017 . 基于灰度梯度图像分割的单木树冠提取研究 . 北京林业大学学报 , 39 ( 3 ): 16 - 23 [ DOI: 10.13332/j.1000-1522.20160373 http://dx.doi.org/10.13332/j.1000-1522.20160373 ]

Gougeon F A . 1995 . A crown-following approach to the automatic delineation of individual tree crowns in high spatial resolution aerial images . Canadian Journal of Remote Sensing , 21 ( 3 ): 274 - 284 [ DOI: 10.1080/07038992.1995.10874622 http://dx.doi.org/10.1080/07038992.1995.10874622 ]

Ke Y and Quackenbush L J . 2011 . A review of methods for automatic individual tree-crown detection and delineation from passive remote sensing . International Journal of Remote Sensing , 32 ( 17 ): 4725 - 4747 [ DOI: 10.1080/01431161.2010.494184 http://dx.doi.org/10.1080/01431161.2010.494184 ]

Kellenberger B , Marcos D and Tuia D . 2018 . Detecting mammals in UAV images: best practices to address a substantially imbalanced dataset with deep learning . Remote Sensing of Environment , 216 : 139 - 153 [ DOI: 10.1016/j.rse.2018.06.028 http://dx.doi.org/10.1016/j.rse.2018.06.028 ]

Korpela I , Anttila P and Pitkänen J . 2006 . The performance of a local maxima method for detecting individual tree tops in aerial photographs . International Journal of Remote Sensing , 27 ( 6 ): 1159 - 1175 [ DOI: 10.1080/01431160500354070 http://dx.doi.org/10.1080/01431160500354070 ]

Lee J H , Ko Y and McPherson E G . 2016 . The feasibility of remotely sensed data to estimate urban tree dimensions and biomass . Urban Forestry and Urban Greening , 16 : 208 - 220 [ DOI: 10.1016/j.ufug.2016.02.010 http://dx.doi.org/10.1016/j.ufug.2016.02.010 ]

Li D , Zhang J J and Zhao M X . 2019 . Extraction of stand factors in UAV image based on FCM and watershed algorithm . Scientia Silvae Sinicae , 55 ( 5 ): 181 - 187

李丹 , 张俊杰 , 赵梦溪 . 2019 . 基于FCM和分水岭算法的无人机影像中林分因子提取 . 林业科学 , 55 ( 5 ): 181 - 187 [ DOI: 10.11707/j.1001-7488.20190520 http://dx.doi.org/10.11707/j.1001-7488.20190520 ]

Lin T Y , Dollár P , Girshick R , He K M , Hariharan B and Belongie S . 2017 . Feature pyramid networks for object detection // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA : IEEE: 936 - 944 [ DOI: 10.1109/CVPR.2017.106 http://dx.doi.org/10.1109/CVPR.2017.106 ]

Liu X S , Ge L , Wang B and Zhang L M . 2012 . An unsupervised band selection algorithm for hyperspectral imagery based on maximal information . Journal of Infrared and Millimeter Waves , 31 ( 2 ): 166 - 170 , 176

刘雪松 , 葛亮 , 王斌 , 张立明 . 2012 . 基于最大信息量的高光谱遥感图像无监督波段选择方法 . 红外与毫米波学报 , 31 ( 2 ): 166 - 170 , 176

Majeed Y , Zhang J , Zhang X , Fu L S , Karkee M , Zhang Q and Whiting M D . 2020 . Deep learning based segmentation for automated training of apple trees on trellis wires . Computers and Electronics in Agriculture , 170 : 105277 [ DOI: 10.1016/j.compag.2020.105277 http://dx.doi.org/10.1016/j.compag.2020.105277 ]

Mozgeris G , Juodkienė V , Jonikavičius D , Straigytė L , Gadal S and Ouerghemmi W . 2018 . Ultra-light aircraft-based hyperspectral and colour-infrared imaging to identify deciduous tree species in an urban environment . Remote Sensing , 10 ( 10 ): 1668 [ DOI: 10.3390/rs10101668 http://dx.doi.org/10.3390/rs10101668 ]

Osco L P , dos Santos de Arruda M , Marcato Junior J , da Silva N B , Ramos A P M , Moryia É A S , Imai N N , Pereira D R , Creste J E , Matsubara E T , Li J and Gonçalves W N . 2020 . A convolutional neural network approach for counting and geolocating citrus-trees in UAV multispectral imagery . ISPRS Journal of Photogrammetry and Remote Sensing , 160 : 97 - 106 [ DOI: 10.1016/j.isprsjprs.2019.12.010 http://dx.doi.org/10.1016/j.isprsjprs.2019.12.010 ]

Pan S J and Yang Q . 2010 . A survey on transfer learning . IEEE Transactions on Knowledge and Data Engineering , 22 ( 10 ): 1345 - 1359 [ DOI: 10.1109/TKDE.2009.191 http://dx.doi.org/10.1109/TKDE.2009.191 ]

Puliti S , Solberg S and Granhus A . 2019 . Use of UAV photogrammetric data for estimation of biophysical properties in forest stands under regeneration . Remote Sensing , 11 ( 3 ): 233 [ DOI: 10.3390/rs11030233 http://dx.doi.org/10.3390/rs11030233 ]

Redmon J and Farhadi A . 2018 . YOLOv3: An incremental improvement. [2019.12 . 1 ]. https://arxiv.org/abs/1804.02767 https://arxiv.org/abs/1804.02767 .

Ren S Q , He K M , Girshick R and Sun J . 2017 . Faster R-CNN: towards real-time object detection with region proposal networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 39 ( 6 ): 1137 - 1149 [ DOI: 10.1109/TPAMI.2016.2577031 http://dx.doi.org/10.1109/TPAMI.2016.2577031 ]

Roth S I B , Leiterer R , Volpi M , Celio E , Schaepman M E and Joerg P C . 2019 . Automated detection of individual clove trees for yield quantification in northeastern Madagascar based on multi-spectral satellite data . Remote Sensing of Environment , 221 : 144 - 156 [ DOI: 10.1016/J.RSE.2018.11.009 http://dx.doi.org/10.1016/J.RSE.2018.11.009 ]

Wang J M , Chen X X , Cao L , An F , Chen B Q , Xue L F and Yun T . 2019 . Individual rubber tree segmentation based on ground-based LiDAR data and faster R-CNN of deep learning . Forests , 10 ( 9 ): 793 [ DOI: 10.3390/f10090793 http://dx.doi.org/10.3390/f10090793 ]

Weinstein B G , Marconi S , Bohlman S , Zare A and White E . 2019 . Individual tree-crown detection in RGB imagery using Semi-supervised deep learning neural networks . Remote Sensing , 11 ( 11 ): 1309 [ DOI: 10.3390/rs11111309 http://dx.doi.org/10.3390/rs11111309 ]

Weinstein B G , Marconi S , Bohlman S A , Zare A and White E P . 2020 . Cross-site learning in deep learning RGB tree crown detection . Ecological Informatics , 56 : 101061 [ DOI: 10.1016/j.ecoinf.2020.101061 http://dx.doi.org/10.1016/j.ecoinf.2020.101061 ]

Wold S , Esbensen K and Geladi P . 1987 . Principal component analysis . Chemometrics and Intelligent Laboratory Systems , 2 ( 1/3 ): 37 - 52 [ DOI: 10.1016/0169-7439(87)80084-9 http://dx.doi.org/10.1016/0169-7439(87)80084-9 ]

Wu X W , Sahoo D and Hoi S C H . 2020 . Recent advances in deep learning for object detection . Neurocomputing , 396 : 39 - 64 [ DOI: 10.1016/j.neucom.2020.01.085 http://dx.doi.org/10.1016/j.neucom.2020.01.085 ]

Zhao Q Z , Liu W , Yin X J and Zhang T Y . 2016 . Selection of optimum bands combination based on multispectral images of UAV . Transactions of the Chinese Society for Agricultural Machinery , 47 ( 3 ): 242 - 248 , 291

赵庆展 , 刘伟 , 尹小君 , 张天毅 . 2016 . 基于无人机多光谱影像特征的最佳波段组合研究 . 农业机械学报 , 47 ( 3 ): 242- 248 , 291 [ DOI: 10.6041/j.issn.1000-1298.2016.03.034 http://dx.doi.org/10.6041/j.issn.1000-1298.2016.03.034 ]

Zhen Z , Quackenbush L J and Zhang L J . 2016 . Trends in automatic individual tree crown detection and delineation-evolution of LiDAR data . Remote Sensing , 8 ( 4 ): 333 [ DOI: 10.3390/rs8040333 http://dx.doi.org/10.3390/rs8040333 ]

文章被引用时，请邮件提醒。

提交

结合RGB-DSM图像和深度学习的城市樟树树冠检测

基于深度特征重构增强的光学和SAR图像鲁棒匹配

跨视角图像地理定位数据集综述

遥感农作物早期分类研究现状与展望