改进Faster R-CNN的遥感图像多尺度飞机目标检测
Multiscale aircraft detection in optical remote sensing imagery based on advanced Faster R-CNN
- 2022年26卷第8期 页码：1624-1635
收稿：2019-10-12，
纸质出版：2022-08-07
DOI： 10.11834/jrs.20219365
移动端阅览
摘要
为了提高遥感图像中多尺度飞机目标的检测精度，本文提出一种基于改进Faster R-CNN的遥感图像飞机目标检测方法。该方法借助多层级融合结构，将深层次的语义特征与浅层次的细节特征相结合，生成多种尺度的既具有精确的位置信息又具有深层次的语义特征的特征图；再借助Faster R-CNN的多尺度RPN（Region Proposal Network）机制，通过对RPN中候选区域尺度的修正，从而提高遥感图像中多尺度飞机目标的定位精度；最后利用Faster R-CNN的分类回归网络，得到飞机目标检测结果。在高分辨率遥感图像中进行了实验，对3种特征提取网络ZF、VGG-16以及ResNet-50进行改进，改进后的精度分别提高了11.34%、9.87%以及1.66%，并且生成的检测框更加贴合飞机目标。实验结果表明，本文方法适用于遥感图像多尺度飞机目标检测，在提高目标定位精度的同时降低了目标漏检现象。
Abstract
Aircraft detection from optical imagery is a significant application in remote sensing. Traditional methods based on corner points or shape of the aircraft can only generate shallow features with limited representative ability. These methods are insufficient for detecting aircraft in remote sensing imagery under complex and diverse circumstances. Current methods based on CNNs
especially Faster R-CNN
have improved the detection performance greatly with its magnificent feature extraction ability. However
detecting aircraft on a single-scale feature map is unsuitable for multiscale aircraft in remote sensing imagery. After several pooling operations on a single-scale feature map
the feature map loses its precise details and small target that corresponds to a smaller area in the feature map. Thus
aircraft detection may result in low target positioning accuracy and target missing.
An advanced Faster R-CNN is presented by constructing a multiscale feature extraction network using multistage fusion structure to detect aircraft with multiple scales. The promoted network produces features of higher resolution by upsampling deep feature maps. These features are then enhanced with shallow features at the same scale. After this modification
we end up with four feature maps F2
F3
F4
and F5
which have different scales. The structure combines the high-level semantic information with the low-level detailed information. Thus
the generated multiscale feature maps have high positioning accuracy and good distinguishability. In addition
because the original RPN anchors are extremely large to cover the range of aircraft sizes in remote sensing imagery
we select suitable RPN anchor parameters for aircraft detection
i.e.
anchor size of 32
2
for the larger-scale feature map F2
64
2
for the large-scale F3
128
2
is set for the F4
and 256
2
for the small-scale F5. With these settings
the RPN can generate proposals
which can cover the aircraft of multiple scales. Finally
these proposals are assigned to their corresponding feature map
and we use the classification and regression network to obtain our final detection results.
The experiment was carried out on RSOD dataset
in which only the aircraft dataset was used for training
validation
and testing. Comparison of detection performance with different anchor scales showed that anchor scales greatly affect detection accuracy
and our selection of anchor scales is suitable for the dataset. Three feature extraction networks (ZF
VGG-16
and ResNet-50) were modified based on Faster R-CNN using multistage fusion structure. The experiment showed that the modification can effectively improve the model’s ability of detecting multiscale aircraft. Compared with models without the modification
AP increased by 11.34%
9.87%
and 1.66% for the three networks. The qualitative and quantitative results also showed that this modification can generate adaptive detection box. The experiment results on Beijing Capital International Airport GF-2 imagery showed that this method performs well in different remote sensing imagery
in which most airplanes in the airport were detected successfully.
We can draw the following conclusions: (1) the proposed method is suitable for multiscale aircraft detection
and it can generate detection box consistent with the scale of multiscale aircraft targets while reducing missing targets; (2) correction of the RPN candidate region scale improves the accuracy of aircraft detection in remote sensing imagery; (3) the method has good generalization ability.
关键词
Keywords
references
Cai D , Chen Y M and Wei W . 2014 . Study on aircraft recognition in multi-spectral remote sensing image based on skeleton characteristics analysis . Bulletin of Surveying and Mapping , 2 : 50 - 54 , 71
蔡栋 , 陈焱明 , 魏巍 . 2014 . 基于骨架特征的多光谱遥感影像飞机目标识别方法研究 . 测绘通报 , ( 2 ): 50-54 , 71 [ DOI: 10.13474/j.cnki.11-2246.2014.0052 http://dx.doi.org/10.13474/j.cnki.11-2246.2014.0052 ]
Girshick R . 2015 . Fast R-CNN // 2015 IEEE International Conference on Computer Vision . Santiago, Chile : IEEE: 1440 - 1448 [ DOI: 10.1109/ICCV.2015.169 http://dx.doi.org/10.1109/ICCV.2015.169 ]
Girshick R , Donahue J , Darrell T and Malik J . 2013 . Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv:1311 . 2524
He K M , Zhang X Y , Ren S Q and Sun J . 2014 . Spatial pyramid pooling in deep convolutional networks for visual recognition // Computer Vision - ECCV 2014 . Switzerland : Springer , 8681 : 346 - 361 [ DOI: 10.1007/978-3-319-10578-9_23 http://dx.doi.org/10.1007/978-3-319-10578-9_23 ]
He K M , Zhang X Y , Ren S Q and Sun J . 2016 . Deep residual learning for image recognition // 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA : IEEE: 770 - 778 [ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]
Krizhevsky A , Sutskever I and Hinton G E . 2017 . ImageNet classification with deep convolutional neural networks . Communications of the ACM , 60 ( 6 ): 84 - 90 [ DOI: 10.1145/3065386 http://dx.doi.org/10.1145/3065386 ]
Li W , Xiang S M , Wang H B and Pan C H . 2011 . Robust airplane detection in satellite images // 2011 18th IEEE International Conference on Image Processing . Brussels, Belgium : IEEE: 2821 - 2824 [ DOI: 10.1109/ICIP.2011.6116259 http://dx.doi.org/10.1109/ICIP.2011.6116259 ]
Li Y B , Zhang S Y , Zhao J F and Tan W A . 2019 . Aircraft detection in remote sensing images based on deep convolutional neural network . IOP Conference Series : Earth and Environmental Science , 252 ( 5 ): 052122 [ DOI: 10.1088/1755-1315/252/5/052122 http://dx.doi.org/10.1088/1755-1315/252/5/052122 ]
Lin T Y , Dollár P , Girshick R , He K M , Hariharan B and Belongie S . 2017 . Feature pyramid networks for object detection // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Honolulu, HI, USA : IEEE: 936 - 944 [ DOI: 10.1109/CVPR.2017.106 http://dx.doi.org/10.1109/CVPR.2017.106 ]
Long Y , Gong Y P , Xiao Z F and Liu Q . 2017 . Accurate object localization in remote sensing images based on convolutional neural networks . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 5 ): 2486 - 2498 [ DOI: 10.1109/TGRS.2016.2645610 http://dx.doi.org/10.1109/TGRS.2016.2645610 ]
Qiu J B , Li S J and Wang W . 2011 . A new approach to detect aircrafts in remote sensing images based on corner and edge information fusion . Microelectronics and Computer , 28 ( 9 ): 214 - 216
仇建斌 , 李士进 , 王玮 . 2011 . 角点与边缘信息相结合的遥感图像飞机检测新方法 . 微电子学与计算机 , 28 ( 9 ): 214 - 216 [ DOI: 10.19304/j.cnki.issn1000-7180.2011.09.056 http://dx.doi.org/10.19304/j.cnki.issn1000-7180.2011.09.056 ]
Ren S Q , He K M , Girshick R and Sun J . 2017 . Faster R-CNN: towards real-time object detection with region proposal networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 39 ( 6 ): 1137 - 1149 [ DOI: 10.1109/TPAMI.2016.2577031 http://dx.doi.org/10.1109/TPAMI.2016.2577031 ]
Ren Y , Zhu C R and Xiao S P . 2018 . Deformable faster R-CNN with aggregating multi-layer features for partially occluded object detection in optical remote sensing images . Remote Sensing , 10 ( 9 ): 1470 [ DOI: 10.3390/rs10091470 http://dx.doi.org/10.3390/rs10091470 ]
Simonyan K and Zisserman A . 2015 . Very deep convolutional networks for large-scale image recognition. arXiv:1409 . 1556
Wang H Z , Gong Y C , Wang Y , Wang L F and Pan C H . 2017 . DeepPlane: a unified deep model for aircraft detection and recognition in remote sensing images . Journal of Applied Remote Sensing , 11 ( 4 ): 042606 [ DOI: 10.1117/1.JRS.11.042606 http://dx.doi.org/10.1117/1.JRS.11.042606 ]
Wang Y , Yang Y , Wang B S , Wang T , Bo X H and Wang C Y . 2019 . Building segmentation in high-resolution remote sensing image through deep neural network and conditional random fields . Journal of Remote Sensing , 23 ( 6 ): 1194 - 1208
王宇 , 杨艺 , 王宝山 , 王田 , 卜旭辉 , 王传云 . 2019 . 深度神经网络条件随机场高分辨率遥感图像建筑物分割 . 遥感学报 , 23 ( 6 ): 1194 - 1208 [ DOI: 10.11834/jrs.20198141 http://dx.doi.org/10.11834/jrs.20198141 ]
Zeiler M D and Fergus R . 2014 . Visualizing and understanding convolutional networks //Fleet D, Pajdla T, Schiele B and Tuytelaars T, eds. Computer Vision - ECCV 2014 . Switzerland : Springer: 818 - 833 [ DOI: 10.1007/978-3-319-10590-1_53 http://dx.doi.org/10.1007/978-3-319-10590-1_53 ]
Zhang H Q , Liu X Y , Yang S and Li Y . 2017 . Retrieval of remote sensing images based on semisupervised deep learning . Journal of Remote Sensing , 21 ( 3 ): 406 - 414
张洪群 , 刘雪莹 , 杨森 , 李宇 . 2017 . 深度学习的半监督遥感图像检索 . 遥感学报 , 21 ( 3 ): 406 - 414 [ DOI: 10.11834/jrs.20176105 http://dx.doi.org/10.11834/jrs.20176105 ]
Zhang K , Hei B Q , Zhou Z and Li S Y . 2018 . CNN with coefficient of variation-based dimensionality reduction for hyperspectral remote sensing images classification . Journal of Remote Sensing , 22 ( 1 ): 87 - 96
张康 , 黑保琴 , 周壮 , 李盛阳 . 2018 . 变异系数降维的CNN高光谱遥感图像分类 . 遥感学报 , 22 ( 1 ): 87 - 96 [ DOI: 10.11834/jrs.20187075 http://dx.doi.org/10.11834/jrs.20187075 ]
Zhao A , Fu K , Sun H , Sun X , Li F , Zhang D B and Wang H Q . 2017 . An effective method based on ACF for aircraft detection in remote sensing images . IEEE Geoscience and Remote Sensing Letters , 14 ( 5 ): 744 - 748 [ DOI: 10.1109/LGRS.2017.2677954 http://dx.doi.org/10.1109/LGRS.2017.2677954 ]
文章被引用时，请邮件提醒。
提交
相关作者
周莲 武汉大学 计算机学院
何楚 武汉大学 电子信息学院
汪鼎文 武汉大学 计算机学院
郭子琪 武汉大学 计算机学院
吕奕龙 火箭军工程大学 作战保障学院
李敏 火箭军工程大学 作战保障学院
吴肇青 火箭军工程大学 作战保障学院
何玉杰 火箭军工程大学 作战保障学院
相关机构
武汉大学 计算机学院
武汉大学 电子信息学院
火箭军工程大学 作战保障学院
电子信息系统复杂电磁环境效应国家重点实验室
国防科技大学 电子科学学院
选择翻译语言
- 地址：北京市海淀区北四环西路19号 《遥感学报》编辑部 邮编：100190
- 联系电话：010-58887052 Email：nrsb@aircas.ac.cn
- 技术支持由北京北大方正电子有限公司提供 京ICP备20021838号-9
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
0
批量引用
导出至
BibTeXEndnoteRefManNoteExpressNoteFirst
回到顶部