Few-shot object detection in optical remote sensing images

ZHOU Lian; HE Chu; WANG Dingwen; GUO Ziqi

doi:10.11834/jrs.20243209

Satellite Information Intelligent Processing Technology | Views : 0 下载量: 1981 CSCD: 0

R-PDF
PDF
Export
Share
Collection
Album

Few-shot object detection in optical remote sensing images
Vol. 28, Issue 7, Pages: 1693-1701(2024)
Received：13 June 2023，

Published：07 July 2024
DOI： 10.11834/jrs.20243209
稿件说明：

移动端阅览

周莲，何楚，汪鼎文，郭子琪.2024.光学遥感图像的小样本目标检测.遥感学报，28（7）： 1693-1701 DOI： 10.11834/jrs.20243209.

Zhou L，He C，Wang D W and Guo Z Q. 2024. Few-shot object detection in optical remote sensing images. National Remote Sensing Bulletin， 28（7）：1693-1701 DOI： 10.11834/jrs.20243209.

摘要

对遥感图像进行目标检测，具有广阔的应用前景。针对小样本背景下遥感图像目标检测任务存在特征提取不足、定位困难和分类易混淆的问题，本文提出了一种基于协同注意力模块和对比学习分支的小样本目标检测算法。首先，对训练样本进行数据增强操作，以扩充数据集规模；其次，提出了一种协同注意力模块，包括设计的背景衰减注意力和空间感知注意力，利用遥感图像丰富的背景与目标特征信息，指导网络关注与目标定位相关的重点信息，从而便于RPN网络生成更好的区域建议框，减少遗漏目标的概率，提升模型对小样本类别的定位性能；然后，设计了一种对比学习分支。基于设计的对比损失函数，通过联合训练策略，在训练时从特征学习逐步过渡到分类器学习，提高了分类的准确率；最后，设计出一种基于微调的迁移学习范式的小样本目标检测模型，分为基础训练阶段和微调阶段，在基础训练阶段借助充足的基类样本训练模型学习类无关的参数，在微调阶段使用制作的小样本数据集帮助目标检测模型适应特定类别目标，提升其检测性能。此外，本文以两阶段微调方法TFA（Two-stage Fine-tuning Approach）为基准，通过在遥感数据集NWPU VHR-10和DIOR上验证本文提出算法的有效性，结果显示本文算法在NWPU VHR-10和DIOR数据集上与其他基准算法相比，平均精度均有大幅提升。

Abstract

Objects in remote sensing images are detected by determining the positions and correct categories of objects. Given that this approach has broad application prospects and plays a vital role in many fields

the purpose of this paper is to study its issues

such as insufficient feature extraction

difficultly in locating objects

and confusing classification

when applied to small samples.The specific contributions of this paper are as follows: (1) A collaborative attention module is proposed

which includes designed background attenuating attention and spatial perception attention. The network focuses on key information related to object positioning on the basis of rich background and object feature information

and an RPN network generates improved regional suggestion boxes

reduces the probability of missing targets

and improves the positioning performance of the model for small-sample categories. (2) A contrastive learning branch is designed. Based on the design of the contrast loss function

feature learning is gradually transferred to classifier learning through a joint training strategy

and classification accuracy is improved. (3) A few-shot object detection model based on the fine-tuning transfer learning paradigm is designed

which is divided into basic training and fine-tuning stages. In the basic training stage

the model is trained to learn class-independent parameters with sufficient base class samples. In the fine-tuning stage

a small sample data set is used to enable the target detection model to adapt to specific objects and improve its detection performance. On the basis of the TFA

this article verifies the effectiveness of the proposed algorithm on the remote sensing data sets NWPU

VHR-10

and DIOR. To demonstrate the superiority of our method to other methods for few-shot object detection

we compared our method with TFA

FSODM

DeFRCN

and MFDC. Results show that the proposed algorithm considerably improved the mean average accuracy on NWPU

VHR-10

and DIOR data sets

demonstrating the superiority of the proposed algorithm over the above algorithms. In summary

our method can achieve exceptional detection results from small remote sensing images

exhibiting effectiveness and superiority over other detection methods. We hope that our method can promote further research into few-shot object detection and contribute to the development of this field.

关键词

Keywords

references

Chen T , Kornblith S , Norouzi M and Hinton G . 2020 . A simple framework for contrastive learning of visual representations //Proceedings of the 37th International Conference on Machine Learning. [s.l. ] : ACM: 149 [DOI: 10.5555/3524938.3525087 ]

Chen W Y , Liu Y C , Kira Z , Wang Y C F and Huang J B . 2019 . A closer look at few-shot classification // 7th International Conference on Learning Representations . New Orleans : ICLR

Fan Q , Zhuo W , Tang C K and Tai Y W . 2020 . Few-shot object detection with attention-RPN and multi-relation detector // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE: 4012 - 4021 [ DOI: 10.1109/CVPR42600.2020.00407 http://dx.doi.org/10.1109/CVPR42600.2020.00407 ]

Kang B , Liu Z , Wang X . Few-shot object detection via feature reweighting [A ] . Proceedings of the IEEE/CVF International Conference on Computer Vision [C ] , 2019 : 8420 - 8429 .

Karlinsky L , Shtok J , Harary S , Schwartz E , Aides A , Feris R , Giryes R and Bronstein A M . 2019 . RepMet: representative-based metric learning for classification and few-shot object detection // IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE: 5192 - 5201 [ DOI: 10.1109/CVPR.2019.00534 http://dx.doi.org/10.1109/CVPR.2019.00534 ]

Liu X B , Liu P , Cai Z H , Qiao Y L , Wang L and Wang M . 2021 . Research progress of optical remote sensing image object detection based on deep learning . Acta Automatica Sinica , 47 ( 9 ): 2078 - 2089

刘小波 , 刘鹏 , 蔡之华 , 乔禹霖 , 王凌 , 汪敏 . 2021 . 基于深度学习的光学遥感图像目标检测研究进展 . 自动化学报 , 47 ( 9 ): 2078 - 2089 [ DOI: 10.16383/j.aas.c190455 http://dx.doi.org/10.16383/j.aas.c190455 ]

Li X , Deng J , Fang Y . Few-shot object detection on remote sensing images [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2021 , 60 : 1 - 14 .

Niemeyer J , Rottensteiner F and Soergel U . 2014 . Contextual classification of lidar data and building object detection in urban areas . ISPRS Journal of Photogrammetry and Remote Sensing , 87 : 152 - 165 [ DOI: 10.1016/j.isprsjprs.2013.11.001 http://dx.doi.org/10.1016/j.isprsjprs.2013.11.001 ]

Qiao L , Zhao Y , Li Z . Defrcn: Decoupled faster r-cnn for few-shot object detection [A ] . Proceedings of the IEEE/CVF International Conference on Computer Vision [C ] , 2021 : 8681 - 8690 .

Shi J R , Wang D , Shang F H and Zhang H Y . 2021 . Research advances on stochastic gradient descent algorithms . Acta Automatica Sinica , 47 ( 9 ): 2103 - 2119

史加荣 , 王丹 , 尚凡华 , 张鹤于 . 2021 . 随机梯度下降算法研究进展 . 自动化学报 , 47 ( 9 ): 2103 - 2119 [ DOI: 10.16383/j.aas.c190260 http://dx.doi.org/10.16383/j.aas.c190260 ]

Sun B , Li B H , Cai S C , Yuan Y and Zhang C . 2021 . FSCE: few-shot object detection via contrastive proposal encoding // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville : IEEE: 7348 - 7358 [ DOI: 10.1109/CVPR46437.2021.00727 http://dx.doi.org/10.1109/CVPR46437.2021.00727 ]

Wang X , Huang T E , Darrell T . Frustratingly simple few-shot object detection [J ] . arXiv preprint arXiv: 2003.06957 , 2020 .

Wu S , Pei W , Mei D . Multi-faceted Distillation of Base-Novel Commonality for FewShot Object Detection [A ] . Computer Vision-ECCV 2022: 17th European Conference , Tel Aviv, Israel, October 23-27, 2022 , Proceedings, Part IX[C] , 2022 : 578 - 594 .

Yang Z , Wang Y L , Chen X Y , Liu J Z and Qiao Y . 2020 . Context-transformer: tackling object confusion for few-shot detection // Proceedings of the 34th AAAI Conference on Artificial Intelligence . New York : AAAI: 12653 - 12660 [ DOI: 10.1609/aaai.v34i07.6957 http://dx.doi.org/10.1609/aaai.v34i07.6957 ]

Yao H G , Wang C , Yu J , Bai X J and Li W . 2020 . Recognition of small-target ships in complex satellite images . Journal of Remote Sensing (in Chinese) , 24 ( 2 ): 116 - 125

姚红革 , 王诚 , 喻钧 , 白小军 , 李蔚 . 2020 . 复杂卫星图像中的小目标船舶识别 . 遥感学报 , 24 ( 2 ): 116 - 125 [ DOI: 10.11834/jrs.20208238 http://dx.doi.org/10.11834/jrs.20208238 ]

Yao Q L , Hu X and Lei H . 2019 . Aircraft detection in remote sensing imagery with multi-scale feature fusion convolutional neural networks . Acta Geodaetica et Cartographica Sinica , 48 ( 10 ): 1266 - 1274

姚群力 , 胡显 , 雷宏 . 2019 . 基于多尺度融合特征卷积神经网络的遥感图像飞机目标检测 . 测绘学报 , 48 ( 10 ): 1266 - 1274 [ DOI: 10.11947/j.AGCS.2019.20180398 http://dx.doi.org/10.11947/j.AGCS.2019.20180398 ]

Yao Y Q , Cheng G , Xie X X and Han J W . 2021 . Optical remote sensing image object detection based on multi-resolution feature fusion . National Remote Sensing Bulletin , 25 ( 5 ): 1124 - 1137

姚艳清 , 程塨 , 谢星星 , 韩军伟 . 2021 . 多分辨率特征融合的光学遥感图像目标检测 . 遥感学报 , 25 ( 5 ): 1124 - 1137 [ DOI: 10.11834/jrs.20210505 http://dx.doi.org/10.11834/jrs.20210505 ]

Yu Y , Ai H , He X J , Yu S H , Zhong X and Zhu R F . 2020 . Attention-based feature pyramid networks for ship detection of optical remote sensing image . Journal of Remote Sensing (in Chinese) , 24 ( 2 ): 107 - 115

于野 , 艾华 , 贺小军 , 于树海 , 钟兴 , 朱瑞飞 . 2020 . A-FPN算法及其在遥感图像船舶检测中的应用 . 遥感学报 , 24 ( 2 ): 107 - 115 [ DOI: 10.11834/jrs.20208264 http://dx.doi.org/10.11834/jrs.20208264 ]

Zhang T F , Zhang Y , Sun X , Sun H , Yan M L , Yang X and Fu K . 2019 . Comparison network for one-shot conditional object detection . arXiv . 1904 .02317[DOI:10.48550 ] .

Zhang W L and Wang Y X . 2021 . Hallucination improves few-shot object detection // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville : IEEE: 13003 - 13012 [ DOI: 10.1109/CVPR46437.2021.01281 http://dx.doi.org/10.1109/CVPR46437.2021.01281 ]

Zhang W L , Wang Y X and Forsyth D A . 2020 . Cooperating RPN’s improve few-shot object detection . arXiv : 2011 . 10142 [ DOI: 10.48550 http://dx.doi.org/10.48550 ]

Zhang Z W , Hao J G , Huang J and Pan C Y . 2022 . Review of few-shot object detection . Computer Engineering and Applications , 58 ( 5 ): 1 - 11

张振伟 , 郝建国 , 黄健 , 潘崇煜 . 2022 . 小样本图像目标检测研究综述 . 计算机工程与应用 , 58 ( 5 ): 1 - 11 [ DOI: 10.3778/j.issn.1002-8331.2109-0405 http://dx.doi.org/10.3778/j.issn.1002-8331.2109-0405 ]

Zhou P C , Cheng G , Yao X W and Han J W . 2021 . Machine learning paradigms in high-resolution remote sensing image interpretation . National Remote Sensing Bulletin , 25 ( 1 ): 182 - 197

周培诚 , 程塨 , 姚西文 , 韩军伟 . 2021 . 高分辨率遥感影像解译中的机器学习范式 . 遥感学报 , 25 ( 1 ): 182 - 197 [ DOI: 10.11834/jrs.20210164 http://dx.doi.org/10.11834/jrs.20210164 ]

Alert me when the article has been cited

提交

Application of an improved CenterNet in remote sensing images object detection

Feature attention pyramid-based remote sensing image object detection method

Hyperspectral image classification network based on multiscale supervised contrastive learning

Object detection in remote sensing images using densely connected recursive feature pyramids

Remote sensing image target detection based on a multi-scale deep feature fusion network