Multiscale aircraft detection in optical remote sensing imagery based on advanced Faster R-CNN

Miaomiao SHA; Yu LI; An LI

doi:10.11834/jrs.20219365

Remote Sensing Intelligent Interpretation | Views : 0 下载量: 1320 CSCD: 18

R-PDF
PDF
Export
Share
Collection
Album

Multiscale aircraft detection in optical remote sensing imagery based on advanced Faster R-CNN
Vol. 26, Issue 8, Pages: 1624-1635(2022)
Received：12 October 2019，

Published：07 August 2022
DOI： 10.11834/jrs.20219365
稿件说明：

移动端阅览

沙苗苗，李宇，李安.2022.改进Faster R-CNN的遥感图像多尺度飞机目标检测.遥感学报，26（8）： 1624-1635 DOI： 10.11834/jrs.20219365.

Sha M M，Li Y and Li A. 2022. Multiscale aircraft detection in optical remote sensing imagery based on advanced Faster R-CNN. National Remote Sensing Bulletin， 26（8）：1624-1635 DOI： 10.11834/jrs.20219365.

摘要

为了提高遥感图像中多尺度飞机目标的检测精度，本文提出一种基于改进Faster R-CNN的遥感图像飞机目标检测方法。该方法借助多层级融合结构，将深层次的语义特征与浅层次的细节特征相结合，生成多种尺度的既具有精确的位置信息又具有深层次的语义特征的特征图；再借助Faster R-CNN的多尺度RPN（Region Proposal Network）机制，通过对RPN中候选区域尺度的修正，从而提高遥感图像中多尺度飞机目标的定位精度；最后利用Faster R-CNN的分类回归网络，得到飞机目标检测结果。在高分辨率遥感图像中进行了实验，对3种特征提取网络ZF、VGG-16以及ResNet-50进行改进，改进后的精度分别提高了11.34%、9.87%以及1.66%，并且生成的检测框更加贴合飞机目标。实验结果表明，本文方法适用于遥感图像多尺度飞机目标检测，在提高目标定位精度的同时降低了目标漏检现象。

Abstract

Aircraft detection from optical imagery is a significant application in remote sensing. Traditional methods based on corner points or shape of the aircraft can only generate shallow features with limited representative ability. These methods are insufficient for detecting aircraft in remote sensing imagery under complex and diverse circumstances. Current methods based on CNNs

especially Faster R-CNN

have improved the detection performance greatly with its magnificent feature extraction ability. However

detecting aircraft on a single-scale feature map is unsuitable for multiscale aircraft in remote sensing imagery. After several pooling operations on a single-scale feature map

the feature map loses its precise details and small target that corresponds to a smaller area in the feature map. Thus

aircraft detection may result in low target positioning accuracy and target missing.

An advanced Faster R-CNN is presented by constructing a multiscale feature extraction network using multistage fusion structure to detect aircraft with multiple scales. The promoted network produces features of higher resolution by upsampling deep feature maps. These features are then enhanced with shallow features at the same scale. After this modification

we end up with four feature maps F2

and F5

which have different scales. The structure combines the high-level semantic information with the low-level detailed information. Thus

the generated multiscale feature maps have high positioning accuracy and good distinguishability. In addition

because the original RPN anchors are extremely large to cover the range of aircraft sizes in remote sensing imagery

we select suitable RPN anchor parameters for aircraft detection

i.e.

anchor size of 32

for the larger-scale feature map F2

for the large-scale F3

128

is set for the F4

and 256

for the small-scale F5. With these settings

the RPN can generate proposals

which can cover the aircraft of multiple scales. Finally

these proposals are assigned to their corresponding feature map

and we use the classification and regression network to obtain our final detection results.

The experiment was carried out on RSOD dataset

in which only the aircraft dataset was used for training

validation

and testing. Comparison of detection performance with different anchor scales showed that anchor scales greatly affect detection accuracy

and our selection of anchor scales is suitable for the dataset. Three feature extraction networks (ZF

VGG-16

and ResNet-50) were modified based on Faster R-CNN using multistage fusion structure. The experiment showed that the modification can effectively improve the model’s ability of detecting multiscale aircraft. Compared with models without the modification

AP increased by 11.34%

9.87%

and 1.66% for the three networks. The qualitative and quantitative results also showed that this modification can generate adaptive detection box. The experiment results on Beijing Capital International Airport GF-2 imagery showed that this method performs well in different remote sensing imagery

in which most airplanes in the airport were detected successfully.

We can draw the following conclusions: (1) the proposed method is suitable for multiscale aircraft detection

and it can generate detection box consistent with the scale of multiscale aircraft targets while reducing missing targets; (2) correction of the RPN candidate region scale improves the accuracy of aircraft detection in remote sensing imagery; (3) the method has good generalization ability.

关键词

Keywords

references

Cai D , Chen Y M and Wei W . 2014 . Study on aircraft recognition in multi-spectral remote sensing image based on skeleton characteristics analysis . Bulletin of Surveying and Mapping , 2 : 50 - 54 , 71

蔡栋 , 陈焱明 , 魏巍 . 2014 . 基于骨架特征的多光谱遥感影像飞机目标识别方法研究 . 测绘通报 , ( 2 ): 50-54 , 71 [ DOI: 10.13474/j.cnki.11-2246.2014.0052 http://dx.doi.org/10.13474/j.cnki.11-2246.2014.0052 ]

Girshick R . 2015 . Fast R-CNN // 2015 IEEE International Conference on Computer Vision . Santiago, Chile : IEEE: 1440 - 1448 [ DOI: 10.1109/ICCV.2015.169 http://dx.doi.org/10.1109/ICCV.2015.169 ]

Girshick R , Donahue J , Darrell T and Malik J . 2013 . Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv:1311 . 2524

He K M , Zhang X Y , Ren S Q and Sun J . 2014 . Spatial pyramid pooling in deep convolutional networks for visual recognition // Computer Vision - ECCV 2014 . Switzerland : Springer , 8681 : 346 - 361 [ DOI: 10.1007/978-3-319-10578-9_23 http://dx.doi.org/10.1007/978-3-319-10578-9_23 ]

He K M , Zhang X Y , Ren S Q and Sun J . 2016 . Deep residual learning for image recognition // 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA : IEEE: 770 - 778 [ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]

Krizhevsky A , Sutskever I and Hinton G E . 2017 . ImageNet classification with deep convolutional neural networks . Communications of the ACM , 60 ( 6 ): 84 - 90 [ DOI: 10.1145/3065386 http://dx.doi.org/10.1145/3065386 ]

Li W , Xiang S M , Wang H B and Pan C H . 2011 . Robust airplane detection in satellite images // 2011 18th IEEE International Conference on Image Processing . Brussels, Belgium : IEEE: 2821 - 2824 [ DOI: 10.1109/ICIP.2011.6116259 http://dx.doi.org/10.1109/ICIP.2011.6116259 ]

Li Y B , Zhang S Y , Zhao J F and Tan W A . 2019 . Aircraft detection in remote sensing images based on deep convolutional neural network . IOP Conference Series : Earth and Environmental Science , 252 ( 5 ): 052122 [ DOI: 10.1088/1755-1315/252/5/052122 http://dx.doi.org/10.1088/1755-1315/252/5/052122 ]

Lin T Y , Dollár P , Girshick R , He K M , Hariharan B and Belongie S . 2017 . Feature pyramid networks for object detection // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Honolulu, HI, USA : IEEE: 936 - 944 [ DOI: 10.1109/CVPR.2017.106 http://dx.doi.org/10.1109/CVPR.2017.106 ]

Long Y , Gong Y P , Xiao Z F and Liu Q . 2017 . Accurate object localization in remote sensing images based on convolutional neural networks . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 5 ): 2486 - 2498 [ DOI: 10.1109/TGRS.2016.2645610 http://dx.doi.org/10.1109/TGRS.2016.2645610 ]

Qiu J B , Li S J and Wang W . 2011 . A new approach to detect aircrafts in remote sensing images based on corner and edge information fusion . Microelectronics and Computer , 28 ( 9 ): 214 - 216

仇建斌 , 李士进 , 王玮 . 2011 . 角点与边缘信息相结合的遥感图像飞机检测新方法 . 微电子学与计算机 , 28 ( 9 ): 214 - 216 [ DOI: 10.19304/j.cnki.issn1000-7180.2011.09.056 http://dx.doi.org/10.19304/j.cnki.issn1000-7180.2011.09.056 ]

Ren S Q , He K M , Girshick R and Sun J . 2017 . Faster R-CNN: towards real-time object detection with region proposal networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 39 ( 6 ): 1137 - 1149 [ DOI: 10.1109/TPAMI.2016.2577031 http://dx.doi.org/10.1109/TPAMI.2016.2577031 ]

Ren Y , Zhu C R and Xiao S P . 2018 . Deformable faster R-CNN with aggregating multi-layer features for partially occluded object detection in optical remote sensing images . Remote Sensing , 10 ( 9 ): 1470 [ DOI: 10.3390/rs10091470 http://dx.doi.org/10.3390/rs10091470 ]

Simonyan K and Zisserman A . 2015 . Very deep convolutional networks for large-scale image recognition. arXiv:1409 . 1556

Wang H Z , Gong Y C , Wang Y , Wang L F and Pan C H . 2017 . DeepPlane: a unified deep model for aircraft detection and recognition in remote sensing images . Journal of Applied Remote Sensing , 11 ( 4 ): 042606 [ DOI: 10.1117/1.JRS.11.042606 http://dx.doi.org/10.1117/1.JRS.11.042606 ]

Wang Y , Yang Y , Wang B S , Wang T , Bo X H and Wang C Y . 2019 . Building segmentation in high-resolution remote sensing image through deep neural network and conditional random fields . Journal of Remote Sensing , 23 ( 6 ): 1194 - 1208

王宇 , 杨艺 , 王宝山 , 王田 , 卜旭辉 , 王传云 . 2019 . 深度神经网络条件随机场高分辨率遥感图像建筑物分割 . 遥感学报 , 23 ( 6 ): 1194 - 1208 [ DOI: 10.11834/jrs.20198141 http://dx.doi.org/10.11834/jrs.20198141 ]

Zeiler M D and Fergus R . 2014 . Visualizing and understanding convolutional networks //Fleet D, Pajdla T, Schiele B and Tuytelaars T, eds. Computer Vision - ECCV 2014 . Switzerland : Springer: 818 - 833 [ DOI: 10.1007/978-3-319-10590-1_53 http://dx.doi.org/10.1007/978-3-319-10590-1_53 ]

Zhang H Q , Liu X Y , Yang S and Li Y . 2017 . Retrieval of remote sensing images based on semisupervised deep learning . Journal of Remote Sensing , 21 ( 3 ): 406 - 414

张洪群 , 刘雪莹 , 杨森 , 李宇 . 2017 . 深度学习的半监督遥感图像检索 . 遥感学报 , 21 ( 3 ): 406 - 414 [ DOI: 10.11834/jrs.20176105 http://dx.doi.org/10.11834/jrs.20176105 ]

Zhang K , Hei B Q , Zhou Z and Li S Y . 2018 . CNN with coefficient of variation-based dimensionality reduction for hyperspectral remote sensing images classification . Journal of Remote Sensing , 22 ( 1 ): 87 - 96

张康 , 黑保琴 , 周壮 , 李盛阳 . 2018 . 变异系数降维的CNN高光谱遥感图像分类 . 遥感学报 , 22 ( 1 ): 87 - 96 [ DOI: 10.11834/jrs.20187075 http://dx.doi.org/10.11834/jrs.20187075 ]

Zhao A , Fu K , Sun H , Sun X , Li F , Zhang D B and Wang H Q . 2017 . An effective method based on ACF for aircraft detection in remote sensing images . IEEE Geoscience and Remote Sensing Letters , 14 ( 5 ): 744 - 748 [ DOI: 10.1109/LGRS.2017.2677954 http://dx.doi.org/10.1109/LGRS.2017.2677954 ]

Alert me when the article has been cited

提交

Few-shot object detection in optical remote sensing images

Object detection in remote sensing images using densely connected recursive feature pyramids

Application of an improved CenterNet in remote sensing images object detection

Feature attention pyramid-based remote sensing image object detection method

Remote sensing image target detection based on a multi-scale deep feature fusion network