Urban cinnamomum camphora crown detection research using RGB-DSM images and deep learning

WANG Hao; XIA Kai; YANG Yinhui; FENG Hailin

doi:10.11834/jrs.20221613

Artificial Intelligence (AI) for Remote Sensing | Views : 0 下载量: 2344 CSCD: 1

R-PDF
PDF
Export
Share
Collection
Album

Urban cinnamomum camphora crown detection research using RGB-DSM images and deep learning
Vol. 27, Issue 12, Pages: 2762-2773(2023)
Received：22 October 2021，

Published：07 December 2023
DOI： 10.11834/jrs.20221613
稿件说明：

移动端阅览

王昊，夏凯，杨垠晖，冯海林.2023.结合RGB-DSM图像和深度学习的城市樟树树冠检测.遥感学报，27（12）： 2762-2773 DOI： 10.11834/jrs.20221613.

Wang H， Xia K， Yang Y H and Feng H L. 2023. Urban cinnamomum camphora crown detection research using RGB-DSM images and deep learning. National Remote Sensing Bulletin， 27（12）：2762-2773 DOI： 10.11834/jrs.20221613.

摘要

当前，结合遥感图像和深度学习进行单木树冠检测已经成为一种趋势。RGB图像是检测中最常用的数据类型，但由于树冠的颜色和纹理一般比较接近，在树冠密度较高的区域，仅使用RGB图像的颜色和纹理信息难以区分不同个体的树冠。对此，本研究在RGB图像的基础上，叠加了高程信息，以提高单木树冠检测的精度。实验采用彩色图像RGB图像和数字表面模型DSM（Digital Surface Model）作为数据源，并分别利用波段组合和双源检测网络模型两种方法结合RGB和DSM进行单木树冠检测。在前一种方法中，对RGB和DSM进行波段组合，生成GBD、RGD和RBD 3类图像，并使用这3类图像分别进行网络的训练和测试。在后一种方法中，将RGB和DSM输入双源检测网络模型，并得到检测结果。本文使用FPN-Faster-R-CNN和Yolov3进行实验，相比于RGB方案（仅使用地物的颜色和纹理信息进行单木树冠检测，是对照方案），FPN-Faster-R-CNN在GBD方案、RBD方案和双源检测网络方案中的平均精度分别上升了3.36%、2.45%和7.77%，在RGD方案中的平均精度下降了0.17%，Yolov3在GBD方案、RBD方案和双源检测网络方案中的平均精度分别上升了0.72%、0.14%和5.71%，在RGD方案中的平均精度下降了0.98%。在两个网络下，双源检测网络方案都在各方案中取得了最佳的检测结果。并且相对于RGB方案，双源检测网络方案在平均精度上的提升幅度随着树冠密度的上升呈现出上升的趋势。对比分析实验结果可知，在基于深度学习的城市单木树冠检测任务中，妥善结合并利用地物的颜色、纹理信息和高程信息有利于提高任务性能。

Abstract

Currently

combining remote sensing imagery with deep learning is a growing trend in individual tree crown detection. RGB image is the most commonly used data type in detection. However

given that the color and texture of the tree crowns are generally close

distinguishing the crowns of different individuals by using only the color and texture information of RGB image in areas with high density of crowns is difficult. In this study

the elevation information is superimposed to improve the accuracy of individual tree crown detection by using RGB images. In the experiment

RGB image (color image) and DSM (digital surface model) were used as data sources

and band combination and double-source detection network model were used to combine RGB and DSM for individual tree crown detection. In the former method

band combination of RGB and DSM was conducted to generate GBD

RGD

and RBD images

and the three kinds of images were used for network training and testing. In the latter method

RGB and DSM were input into the double-source detection network model

and the detection results were obtained. FPN-Faster-R-CNN and Yolov3 were used for experiments in this study. Compared with RGB scheme as the control scheme (which uses only the color and texture information of ground objects for individual tree crown detection)

the average accuracy of FPN-Faster-R-CNN in the GBD scheme

RBD scheme

and double-source detection network scheme increased by 3.36%

2.45%

and 7.77%

respectively; it decreased by 0.17% in the RGD scheme. The average accuracy of Yolov3 in the GBD scheme

RBD scheme

and double-source detection network scheme increased by 0.72%

0.14%

and 5.71%

respectively; it decreased by 0.98% in the RGD scheme. Under the two networks

the double-source detection network scheme achieved the best detection result in each scheme. Compared with the RGB scheme

the improvement in average accuracy of double-source detection network scheme showed a rising trend with the increase in forest density. Comparative analysis of the experimental results shows that proper combination and utilization of the color

texture

and elevation information of the ground objects is beneficial to improve the performance in the urban individual tree crown detection task based on deep learning.

关键词

Keywords

references

Ampatzidis Y and Partel V . 2019 . UAV-based high throughput phenotyping in citrus utilizing multispectral imaging and artificial intelligence . Remote Sensing , 11 ( 4 ): 410 [ DOI: 10.3390/rs11040410 http://dx.doi.org/10.3390/rs11040410 ]

Apolo-Apolo O E , Martínez-Guanter J , Egea G , Raja P and Pérez-Ruiz M . 2020 . Deep learning techniques for estimation of the yield and size of citrus fruits using a UAV . European Journal of Agronomy , 115 : 126030 [ DOI: 10.1016/j.eja.2020.126030 http://dx.doi.org/10.1016/j.eja.2020.126030 ]

Dong T Y , Zhang X P , Ding Z F and Fan J . 2020 . Multi-layered tree crown extraction from LiDAR data using graph-based segmentation . Computers and Electronics in Agriculture , 170 : 105213 [ DOI: 10.1016/j.compag.2020.105213 http://dx.doi.org/10.1016/j.compag.2020.105213 ]

Dos Santos A A , Junior J M , Araújo M S , Di Martini D R , Tetila E C , Siqueira H L , Aoki C , Eltner A , Matsubara E T , Pistori H , Feitosa R Q , Liesenberg V and Gonçalves W N . 2019 . Assessment of CNN-based methods for individual tree detection on images captured by RGB cameras attached to UAVs . Sensors , 19 ( 16 ): 3595 [ DOI: 10.3390/s19163595 http://dx.doi.org/10.3390/s19163595 ]

Endreny T A . 2018 . Strategically growing the urban forest will improve our world . Nature Communications , 9 ( 1 ): 1160 [ DOI: 10.1038/s41467-018-03622-0 http://dx.doi.org/10.1038/s41467-018-03622-0 ]

Goldbergs G , Maier S W , Levick S R and Edwards A . 2018 . Efficiency of individual tree detection approaches based on light-weight and low-cost UAS imagery in Australian savannas . Remote Sensing , 10 ( 2 ): 161 [ DOI: 10.3390/rs10020161 http://dx.doi.org/10.3390/rs10020161 ]

Gong T Y and Niu H Q . 2020 . An implementation of ResNet on the classification of RGB-D images // 2nd BenchCouncil International Symposium on Benchmarking, Measuring, and Optimizing . Denver: 149 - 155 [ DOI: 10.1007/978-3-030-49556-5_15 http://dx.doi.org/10.1007/978-3-030-49556-5_15 ]

Guo Y R and Chen T . 2018 . Semantic segmentation of RGBD images based on deep depth regression . Pattern Recognition Letters , 109 : 55 - 64 [ DOI: 10.1016/j.patrec.2017.08.026 http://dx.doi.org/10.1016/j.patrec.2017.08.026 ]

Huang H Y , Li X and Chen C C . 2018 . Individual tree crown detection and delineation from very-high-resolution UAV images based on bias field and marker-controlled watershed segmentation algorithms . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 11 ( 7 ): 2253 - 2262 [ DOI: 10.1109/JSTARS.2018.2830410 http://dx.doi.org/10.1109/JSTARS.2018.2830410 ]

Lan Y B , Zhu Z H , Deng X L , Lian B Z , Huang J Y , Huang Z X and Hu J . 2019 . Monitoring and classification of citrus Huanglongbing based on UAV hyperspectral remote sensing . Transactions of the Chinese Society of Agricultural Engineering , 35 ( 3 ): 92 - 100

兰玉彬 , 朱梓豪 , 邓小玲 , 练碧桢 , 黄敬易 , 黄梓效 , 胡洁 . 2019 . 基于无人机高光谱遥感的柑橘黄龙病植株的监测与分类 . 农业工程学报 , 35 ( 3 ): 92 - 100 [ DOI: 10.11975/j.issn.1002-6819.2019.03.012 http://dx.doi.org/10.11975/j.issn.1002-6819.2019.03.012 ]

Lin T Y , Dollár P , Girshick R , He K M , Hariharan B and Belongie S . 2017 . Feature pyramid networks for object detection // Proceedings of the 2017 IEEE Conference on Computer Vision and pattern Recognition . Honolulu : IEEE: 936 - 944 [ DOI: 10.1109/cvpr.2017.106 http://dx.doi.org/10.1109/cvpr.2017.106 ]

Miyoshi G T , dos Santos Arruda M , Osco L P , Junior J M , Gonçalves D N , Imai N N , Tommaselli A M G , Honkavaara E and Gonçalves W N . 2020 . A novel deep learning method to identify single tree species in UAV-based hyperspectral images . Remote Sensing , 12 ( 8 ): 1294 [ DOI: 10.3390/rs12081294 http://dx.doi.org/10.3390/rs12081294 ]

Nevalainen O , Honkavaara E , Tuominen S , Viljanen N , Hakala T , Yu X W , Hyyppä J , Saari H , Pölönen I , Imai N N and Tommaselli A M G . 2017 . Individual tree detection and classification with UAV-based photogrammetric point clouds and hyperspectral imaging . Remote Sensing , 9 ( 3 ): 185 [ DOI: 10.3390/rs9030185 http://dx.doi.org/10.3390/rs9030185 ]

Redmon J and Farhadi A . 2018 . YOLOv3: an incremental improvement . arXiv preprint arXiv : 1804 . 02767 [ DOI: 10.48550/arXiv.1804.02767 http://dx.doi.org/10.48550/arXiv.1804.02767 ]

Sun Y , Zhou Y , Yuan M S , Liu W P , Luo Y Q and Zong S X . 2018 . UAV real-time monitoring for forest pest based on deep learning . Transactions of the Chinese Society of Agricultural Engineering , 34 ( 21 ): 74 - 81

孙钰 , 周焱 , 袁明帅 , 刘文萍 , 骆有庆 , 宗世祥 . 2018 . 基于深度学习的森林虫害无人机实时监测方法 . 农业工程学报 , 34 ( 21 ): 74 - 81 [ DOI: 10.11975/j.issn.1002-6819.2018.21.009 http://dx.doi.org/10.11975/j.issn.1002-6819.2018.21.009 ]

Torres D L , Feitosa R Q , Happ P N , La Rosa L E C , Junior J M , Martins J , Bressan P O , Gonçalves W N and Liesenberg V . 2020 . Applying fully convolutional architectures for semantic segmentation of a single tree species in urban environment on high resolution UAV optical imagery . Sensors , 20 ( 2 ): 563 [ DOI: 10.3390/s20020563 http://dx.doi.org/10.3390/s20020563 ]

Torresan C , Carotenuto F , Chiavetta U , Miglietta F , Zaldei A and Gioli B . 2020 . Individual tree crown segmentation in two-layered dense mixed forests from UAV LiDAR data . Drones , 4 ( 2 ): 10 [ DOI: 10.3390/drones4020010 http://dx.doi.org/10.3390/drones4020010 ]

Wang W H . 2020 . Detection of panoramic vision pedestrian based on deep learning . Image and Vision Computing , 103 : 103986 [ DOI: 10.1016/j.imavis.2020.103986 http://dx.doi.org/10.1016/j.imavis.2020.103986 ]

Wegner J D , Branson S , Hall D , Schindler K and Perona P . 2016 . Cataloging public objects using aerial and street-level images—urban trees // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE: 6014 - 6023 [ DOI: 10.1109/CVPR.2016.647 http://dx.doi.org/10.1109/CVPR.2016.647 ]

Weinstein B G , Marconi S , Bohlman S , Zare A and White E . 2019 . Individual tree-crown detection in RGB imagery using semi-supervised deep learning neural networks . Remote Sensing , 11 ( 11 ): 1309 [ DOI: 10.3390/rs11111309 http://dx.doi.org/10.3390/rs11111309 ]

Wu J T , Yang G J , Yang H , Zhu Y H , Li Z H , Lei L and Zhao C J . 2020 . Extracting apple tree crown information from remote imagery using deep learning . Computers and Electronics in Agriculture , 174 : 105504 [ DOI: 10.1016/j.compag.2020.105504 http://dx.doi.org/10.1016/j.compag.2020.105504 ]

Xiao Y Z , Tian Z Q , Yu J C , Zhang Y S , Liu S , Du S Y and Lan X G . 2020 . A review of object detection based on deep learning . Multimedia Tools and Applications , 79 ( 33/34 ): 23729 - 23791 [ DOI: 10.1007/s11042-020-08976-6 http://dx.doi.org/10.1007/s11042-020-08976-6 ]

Xie Y Q , Bao H , Shekhar S and Knight J . 2018 . A TIMBER framework for mining urban tree inventories using remote sensing datasets // 2018 IEEE International Conference on Data Mining (ICDM) . Singapore : IEEE: 1344 - 1349 [ DOI: 10.1109/ICDM.2018.00183 http://dx.doi.org/10.1109/ICDM.2018.00183 ]

Xu W B , Deng S S , Liang D and Cheng X J . 2021a . A crown morphology-based approach to individual tree detection in subtropical mixed broadleaf urban forests using UAV LiDAR data . Remote Sensing , 13 ( 7 ): 1278 [ DOI: 10.3390/rs13071278 http://dx.doi.org/10.3390/rs13071278 ]

Xu X J , Zhou Z S , Tang Y and Qu Y L . 2021b . Individual tree crown detection from high spatial resolution imagery using a revised local maximum filtering . Remote Sensing of Environment , 258 : 112397 [ DOI: 10.1016/j.rse.2021.112397 http://dx.doi.org/10.1016/j.rse.2021.112397 ]

Zhen Z , Quackenbush L J and Zhang L J . 2014 . Impact of tree-oriented growth order in marker-controlled region growing for individual tree crown delineation using airborne laser scanner (ALS) data . Remote Sensing , 6 ( 1 ): 555 - 579 [ DOI: 10.3390/rs6010555 http://dx.doi.org/10.3390/rs6010555 ]

Alert me when the article has been cited

提交

Urban individual tree crown detection research using multispectral image dimensionality reduction with deep learning

Robust matching of optical and SAR images based on deep feature reconstruction enhancement

Review and prospects of the development of LiDAR in ecology and geosciences

Early-season crop classification： Recent developments and prospects