结合数据融合与特征选择的遥感影像尺度多样目标检测
Multiscale object detection in remote sensing image by combining data fusion and feature selection
- 2022年26卷第8期 页码：1662-1673
收稿：2021-04-29，
纸质出版：2022-08-07
DOI： 10.11834/jrs.20221249
移动端阅览
摘要
基于深度神经网络模型的遥感影像地物检测取得了巨大成功，很大程度上得益于大规模数据集的支撑。但是，从现有遥感影像数据集本身来看，不同类别地物的数量分布不一致，同类地物对象以不同尺寸大小呈现，是导致地物样本的尺度不均衡问题的直接因素。对此，本文采用数据集内影像加权融合与地物多尺度特征选择的策略来缓解该问题。首先，将数据集内两张影像的像素值进行加权并得到融合后的影像，从而使不同类别地物样本更加均衡且具有较高的背景多样性；其次，通过选择合适尺度的特征图预测相应尺度的目标类别，且允许同一尺度目标在相邻特征图上进行预测，这样使模型能根据目标尺度进行训练；最后，基于目标中心区域的特征图预测目标边界框，预测的边界框更符合目标本身的尺度。通过在两个遥感数据集上分别进行实验，表明训练的模型在对复杂背景下的类别不均衡目标的识别更加准确，能够适应遥感影像下不同尺度目标的识别。
Abstract
Remote sensing image object detection based on depth neural network model has achieved great success largely due to the support of large-scale data sets. However
from the perspective of the existing remote sensing image datasets
the number distribution of different types of ground objects is inconsistent
and the same type of ground objects is presented in different sizes. This factor leads to the scale imbalance of ground object samples.
To alleviate this problem
the strategy of weighted image fusion and multiscale feature selection is adopted. First
the pixel values of the two images in the data set are weighed to obtain the fused image. Therefore
the different types of feature samples are more balanced and have higher background diversity. Second
the target category of the corresponding scale is predicted by selecting the appropriate scale feature map
and the same scale target can be predicted on the adjacent feature map. Thus
the model can be trained according to the target scale. Finally
the bounding box of the target is predicted based on the feature map of the target center area
and the result is more consistent with the scale of the target itself.
The image fusion and multiscale feature selection network is experimentally compared in the paper on two datasets
RSOD and NWPUVHR10. The results show that the trained model is more accurate in the recognition of imbalanced objects in complex background and can adapt to the recognition of different scale objects in remote sensing images. In addition
the proposed method enhances the diversity of sample scenes and can be adapted to different scales of targets. Qualitative analysis shows that the proposed method has better robustness and performance advantages when applied to remote sensing images.
The proposed method can cope with remote sensing images with complex backgrounds
mitigate the effects of category imbalances and better fit the scenarios in which remote sensing images are used. Mitigation of category imbalances and Scale Selection enhance the performance of remote sensing image object detection.
关键词
Keywords
references
Cai Z W and Vasconcelos N . 2018 . Cascade R-CNN: Delving into high quality object detection // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT : IEEE: 6154 - 6162 [ DOI: 10.1109/CVPR.2018.00644 http://dx.doi.org/10.1109/CVPR.2018.00644 ]
Cao Y H , Chen K , Loy C C and Lin D . 2020 . Prime sample attention in object detection // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Seattle, WA : IEEE: 11580 - 11588 [ DOI: 10.1109/CVPR42600.2020.01160 http://dx.doi.org/10.1109/CVPR42600.2020.01160 ]
Chen J , Wan L , Zhu J R , Xu G and Deng M . 2020 . Multi-scale spatial and channel-wise attention for improving object detection in remote sensing imagery . IEEE Geoscience and Remote Sensing Letters , 17 ( 4 ): 681 - 685 [ DOI: 10.1109/LGRS.2019.2930462 http://dx.doi.org/10.1109/LGRS.2019.2930462 ]
Cheng G and Han J W . 2016a . A survey on object detection in optical remote sensing images . ISPRS Journal of Photogrammetry and Remote Sensing , 117 : 11 - 28 [ DOI: 10.1016/j.isprsjprs.2016.03.014 http://dx.doi.org/10.1016/j.isprsjprs.2016.03.014 ]
Cheng G , Han J W , Zhou P C and Guo L . 2014 . Multi-class geospatial object detection and geographic image classification based on collection of part detectors . ISPRS Journal of Photogrammetry and Remote Sensing , 98 : 119 - 132 [ DOI: 10.1016/j.isprsjprs.2014.10.002 http://dx.doi.org/10.1016/j.isprsjprs.2014.10.002 ]
Cheng G , Zhou P C and Han J W . 2016b . Learning Rotation-Invariant convolutional neural networks for object detection in VHR optical remote sensing images . IEEE Transactions on Geoscience and Remote Sensing , 54 ( 12 ): 7405 - 7415 [ DOI: 10.1109/TGRS.2016.2601622 http://dx.doi.org/10.1109/TGRS.2016.2601622 ]
Durieux L , Lagabrielle E and Nelson A . 2008 . A method for monitoring building construction in urban sprawl areas using object-based analysis of Spot 5 images and existing GIS data . ISPRS Journal of Photogrammetry and Remote Sensing , 63 ( 4 ): 399 - 408 [ DOI: 10.1016/j.isprsjprs.2008.01.005 http://dx.doi.org/10.1016/j.isprsjprs.2008.01.005 ]
Feng X , Qiu K , Cui W H , Chen Y X and Li X H . 2014 . Multiscale description and recognition of target shape in high-resolution remote sensing images . Journal of Remote Sensing , 18 ( 1 ): 90 - 104
冯霞 , 秦昆 , 崔卫红 , 陈一祥 , 李向辉 . 2014 . 高分辨率遥感影像目标形状特征多尺度描述与识别 . 遥感学报 , 18 ( 1 ): 90 - 104 [ DOI: 10.11834/jrs.20133056 http://dx.doi.org/10.11834/jrs.20133056 ]
Girshick R , Donahue J , Darrell T and Malik J . 2014 . Rich feature hierarchies for accurate object detection and semantic segmentation // 2014 IEEE Conference on Computer Vision and Pattern Recognition . Columbus, OH : IEEE: 580 - 587 [ DOI: 10.1109/CVPR.2014.81 http://dx.doi.org/10.1109/CVPR.2014.81 ]
Girshick R . 2015 . Fast R-CNN // 2015 IEEE International Conference on Computer Vision (ICCV) . Santiago : IEEE: 1440 - 1448 [ DOI: 10.1109/ICCV.2015.169 http://dx.doi.org/10.1109/ICCV.2015.169 ]
Jiang Y F , Yu H Y , Li C L , Liu P and Zhang H Y . 2020 . Oil storage tank detection and information extraction from high-resolution remote sensing imagery based on improved Faster R-CNN . Remote Sensing Information , 35 ( 4 ): 89 - 96
江一帆 , 于海洋 , 李朝亮 , 刘鹏 , 张慧勇 . 2020 . 基于改进faster R-CNN的高分遥感影像储油罐检测与信息提取 . 遥感信息 , 35 ( 4 ): 89 - 96 [ DOI: 10.3969/j.issn.1000-3177.2020.04.013 http://dx.doi.org/10.3969/j.issn.1000-3177.2020.04.013 ]
Kong T , Sun F C , Liu H P , Jiang Y N , Li L and Shi J B . 2020 . FoveaBox: Beyound anchor-based object detection . IEEE Transactions on Image Processing , 29 : 7389 - 7398 [ DOI: 10.1109/TIP.2020.3002345 http://dx.doi.org/10.1109/TIP.2020.3002345 ]
Li B Y , Liu Y and Wang X G . 2019 . Gradient harmonized single-stage detector . Proceedings of the AAAI Conference on Artificial Intelligence , 33 ( 1 ): 8577 - 8584 [ DOI: 10.1609/aaai.v33i01.33018577 http://dx.doi.org/10.1609/aaai.v33i01.33018577 ]
Li C Y , Liu J , Hong H L , Mao W J , Wang C J , Hu C D , Su X and Luo B . 2020 . Object detection based on OcSaFPN in aerial images with noise. arXiv:2012 . 09859
Lin T Y , Dollár P , Girshick R , He K M , Hariharan B and Belongie S . 2017b . Feature pyramid networks for object detection // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Honolulu, HI : IEEE: 936 - 944 [ DOI: 10.1109/CVPR.2017.106 http://dx.doi.org/10.1109/CVPR.2017.106 ]
Lin T Y , Goyal P , Girshick R , He K M and Dollár P . 2017a . Focal loss for dense object detection // 2017 IEEE International Conference on Computer Vision (ICCV) . Venice : IEEE: 2999 - 3007 [ DOI: 10.1109/ICCV.2017.324 http://dx.doi.org/10.1109/ICCV.2017.324 ]
Liu S , Qi L , Qin H F , Shi J P and Jia J Y . 2018 . Path aggregation network for instance segmentation // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT : IEEE: 8759 - 8768 [ DOI: 10.1109/CVPR.2018.00913 http://dx.doi.org/10.1109/CVPR.2018.00913 ]
Liu W , Anguelov D , Erhan D , Szegedy C , Reed S , Fu C Y and Berg A C . 2016 . SSD: single shot MultiBox detector // 14th European Conference on Computer Vision . Amsterdam : Springer: 21 - 37 [ DOI: 10.1007/978-3-319-46448-0_2 http://dx.doi.org/10.1007/978-3-319-46448-0_2 ]
Long Y , Gong Y P , Xiao Z F and Liu Q . 2017 . Accurate object localization in remote sensing images based on convolutional neural networks . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 5 ): 2486 - 2498 [ DOI: 10.1109/TGRS.2016.2645610 http://dx.doi.org/10.1109/TGRS.2016.2645610 ]
Pang J M , Chen K , Shi J P , Feng H J , Ouyang W L and Lin D H . 2019 . Libra R-CNN: towards balanced learning for object detection // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Long Beach, CA : IEEE: 821 - 830 [ DOI: 10.1109/CVPR.2019.00091 http://dx.doi.org/10.1109/CVPR.2019.00091 ]
Redmon J , Divvala S , Girshick R and Farhadi A . 2016 . You only look once : unified, real-time object detection// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas, NV : IEEE: 779 - 788 [ DOI: 10.1109/CVPR.2016.91 http://dx.doi.org/10.1109/CVPR.2016.91 ]
Redmon J and Farhadi A . 2017 . YOLO9000 : better, faster, stronger// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Honolulu, HI : IEEE: 6517 - 6525 [ DOI: 10.1109/CVPR.2017.690 http://dx.doi.org/10.1109/CVPR.2017.690 ]
Redmon J and Farhadi A . 2018 . YOLOv3: an incremental improvement. arXiv:1804 . 02767
Ren S Q , He K M , Girshick R and Sun J . 2017 . Faster R-CNN: towards real-time object detection with region proposal networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 39 ( 6 ): 1137 - 1149 [ DOI: 10.1109/TPAMI.2016.2577031 http://dx.doi.org/10.1109/TPAMI.2016.2577031 ]
Shrivastava A , Gupta A and Girshick R . 2016 . Training region-based object detectors with online hard example mining // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas, NV : IEEE: 761 - 769 [ DOI: 10.1109/CVPR.2016.89 http://dx.doi.org/10.1109/CVPR.2016.89 ]
Simonyan K and Zisserman A . 2015 . Very deep convolutional networks for Large-Scale image recognition. arXiv:1409 . 1556
Singh B and Davis L S . 2018 . An analysis of scale invariance in object detection - SNIP // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT : IEEE: 3578 - 3587 [ DOI: 10.1109/CVPR.2018.00377 http://dx.doi.org/10.1109/CVPR.2018.00377 ]
Wang B , Zhou Y , Zhang H N and Zhao K . 2021 . Aircraft detection method based on SSD framework for remote sensing images . Fire Control and Command Control , 46 ( 1 ): 14 - 19
王冰 , 周焰 , 张怀念 , 赵凯 . 2021 . 基于改进SSD框架的遥感影像飞机目标检测方法 . 火力与指挥控制 , 46 ( 1 ): 14 - 19 [ DOI: 10.3969/j.issn.1002-0640.2021.01.003 http://dx.doi.org/10.3969/j.issn.1002-0640.2021.01.003 ]
Xiao Z F , Liu Q , Tang G F and Zhai X F . 2015 . Elliptic Fourier transformation-based histograms of oriented gradients for rotationally invariant object detection in remote-sensing images . International Journal of Remote Sensing , 36 ( 2 ): 618 - 644 [ DOI: 10.1080/01431161.2014.999881 http://dx.doi.org/10.1080/01431161.2014.999881 ]
Yang Y , Li W L , Gao S Y , Bai H and Jiang W C . 2021 . Objects detection from high-resolution remote sensing imagery using training-optimized YOLOv3 network . Laser and Optoelectronics Progress , 58 ( 16 ): 147 - 153
杨耘 , 李龙威 , 高思岩 , 柏晗 , 江万成 . 2021 . 基于YOLOv3网络训练优化的高分辨率遥感影像目标检测 . 激光与光电子学进展 , 58 ( 16 ): 147 - 153 [ DOI: 10.3788/lop202158.1601002 http://dx.doi.org/10.3788/lop202158.1601002 ]
Yao H G , Wang C , Yu J , Bai X J and Li W . 2020 . Recognition of small-target ships in complex satellite images . Journal of Remote Sensing , 24 ( 2 ): 116 - 125
姚红革 , 王诚 , 喻钧 , 白小军 , 李蔚 . 2020 . 复杂卫星图像中的小目标船舶识别 . 遥感学报 , 24 ( 2 ): 116 - 125 [ DOI: 10.11834/jrs.20208238 http://dx.doi.org/10.11834/jrs.20208238 ]
Yu Y , Ai H , He X J , Yu S H , Zhong X and Zhu R F . 2020 . Attention-based feature pyramid networks for ship detection of optical remote sensing image . Journal of Remote Sensing , 24 ( 2 ): 107 - 115
于野 , 艾华 , 贺小军 , 于树海 , 钟兴 , 朱瑞飞 . 2020 . A-FPN算法及其在遥感图像船舶检测中的应用 . 遥感学报 , 24 ( 2 ): 107 - 115 [ DOI: 10.11834/jrs.20208264 http://dx.doi.org/10.11834/jrs.20208264 ]
Zhong P and Wang R S . 2007 . A multiple conditional random fields ensemble model for urban area detection in remote sensing optical images . IEEE Transactions on Geoscience and Remote Sensing , 45 ( 12 ): 3978 - 3988 [ DOI: 10.1109/TGRS.2007.907109 http://dx.doi.org/10.1109/TGRS.2007.907109 ]
文章被引用时，请邮件提醒。
提交
相关作者
喻智睿 西南交通大学 地球科学与工程学院
尹展鹏 西南交通大学 地球科学与工程学院
王俊宇 西南交通大学 地球科学与工程学院
周亮 西南交通大学 地球科学与工程学院
叶沅鑫 西南交通大学 地球科学与工程学院
姬云翔 江苏海洋大学 电子工程学院
康家银 江苏海洋大学 电子工程学院
马寒雁 江苏海洋大学 电子工程学院
相关机构
西南交通大学 地球科学与工程学院
江苏海洋大学 电子工程学院
西安电子科技大学 计算机科学与技术学院
西安电子科技大学 广州研究院
陕西省智能人机交互与可穿戴技术重点实验室
选择翻译语言
- 地址：北京市海淀区北四环西路19号 《遥感学报》编辑部 邮编：100190
- 联系电话：010-58887052 Email：nrsb@aircas.ac.cn
- 技术支持由北京北大方正电子有限公司提供 京ICP备20021838号-9
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
0
批量引用
导出至
BibTeXEndnoteRefManNoteExpressNoteFirst
回到顶部