Agricultural Field Segmentation using Spatial Attention Mechanism and Multi-task Learning Strategy

TIAN Fuyou; CAO Yupei; ZHAO Hang; WU Bingfang; ZENG Hongwei; LIU Yazhou; QIN Xingli; ZHANG Miao; ZHU Liang; ZHU Weiwei

doi:10.11834/jrs.20243191

Views : 0 下载量: 21 CSCD: 0 更多指标

R-PDF
PDF
Export
Share
Collection
Album

Agricultural Field Segmentation using Spatial Attention Mechanism and Multi-task Learning Strategy
Pages: 1-15(2024)
Published Online： 25 March 2024 ，
DOI： 10.11834/jrs.20243191

扫描看全文

田富有，曹玉佩，赵航，吴炳方，曾红伟，刘亚洲，覃星力，张淼，朱亮，朱伟伟.XXXX.基于空间注意力机制与多任务学习的耕地地块分割模型研究.遥感学报，XX（XX）： 1-15

TIAN Fuyou，CAO Yupei，ZHAO Hang，WU Bingfang，ZENG Hongwei，LIU Yazhou，QIN Xingli，ZHANG Miao，ZHU Liang，ZHU Weiwei. XXXX. Agricultural Field Segmentation using Spatial Attention Mechanism and Multi-task Learning Strategy. National Remote Sensing Bulletin， XX（XX）：1-15
田富有，曹玉佩，赵航，吴炳方，曾红伟，刘亚洲，覃星力，张淼，朱亮，朱伟伟.XXXX.基于空间注意力机制与多任务学习的耕地地块分割模型研究.遥感学报，XX（XX）： 1-15 DOI： 10.11834/jrs.20243191.

TIAN Fuyou，CAO Yupei，ZHAO Hang，WU Bingfang，ZENG Hongwei，LIU Yazhou，QIN Xingli，ZHANG Miao，ZHU Liang，ZHU Weiwei. XXXX. Agricultural Field Segmentation using Spatial Attention Mechanism and Multi-task Learning Strategy. National Remote Sensing Bulletin， XX（XX）：1-15 DOI： 10.11834/jrs.20243191.

摘要

地块作为农业耕作的最小单元，对其精准识别是国土资源监测、耕地利用监测的需要。现有的方法多使用手工勾绘的方式获取，耗时费力，成本高昂，并且无法实现实时、近实时更新。本文设计了一种基于空间注意力机制与多任务学习的地块分割模型—Field-Net。模型基于UNet架构，增加了空间注意力机制，并采用多任务学习的策略，在语义分割的基础上增加了边界、像素到地块边界的距离等任务。在山东省东营市利津县对模型的性能进行了测试，结果发现耕地地块识别的交并比达到了87.05%，总体精度为92.23%。Field-Net模型的性能优于几种高性能的深度学习框架，交并比较Link-Net模型高出0.26%，较DeepLab v3+高出7.59%。在空间泛化性能测试中，Field-Net模型的平均交并比Link-Net模型高出3.51%，空间泛化性能明显提升。通过消融试验发现，使用空间注意力机制的Field-Net较ResUNet模型F1-Score提高了1.01%，交并比提高了1.6%；多任务学习策略使得Field-Net模型的F1-Score提高了0.18%，交并比提高了0.21%；将模型权重特征进行可视化后发现空间注意力机制模块和多任务学习策略可以使模型学习到的特征更加聚集于地块边界和地块内部，使学习到的特征更具代表性。总体而言，Field-Net模型可以支撑地块级别国土资源和耕地非农化、非粮化利用监测，从而提高监测的效率和时效性。

Abstract

Objective Accurate identification of agricultural fields

the smallest units of agricultural farming

is crucial for monitoring land resources and arable land utilization. Manual mapping methods are time-consuming and expensive

incapable of real-time or near-real-time updates. To enhance agricultural field delineation efficiency

we introduce Field-Net

a field segmentation model leveraging spatial attention mechanisms and multi-task learning in this research.Method Field-Net is based on the UNet architecture

integrating spatial attention mechanisms and a multi-task learning approach. In addition to the segmentation task

we incorporate boundary identification and distance to the field boundary as two additional tasks

enabling the model to learn more representative features related to fields. The model's performance was evaluated using GF-1 and ZY-3 satellite images with a spatial resolution of 2 meters in Lijin County

Dongying City

Shandong Province. We labeled 3

480 tiles measuring 256x256 pixels in the YanWo district

with 3

000 used for training

360 for validation and 120 for spatial generalisation performance test.Results Initially

we analyzed the loss weights for the three tasks—mask

boundary

and pixel-to-boundary distance—in multi-task learning using a gradient test. We found that for multi-task learning

the loss weights should prioritize the mask segmentation task as the primary task and the others as secondary. Across the entire test set

Field-Net achieved an overall accuracy of 92.23% and an IOU of 87.05%. We compared Field-Net with four state-of-the-art architectures: DeepLabv3+

HRNet

LinkNet

and D-LinkNet. Field-Net outperformed them all in semantic segmentation tasks

with an IOU 0.26% higher than Link-Net

the most accurate among the four selected models

and 7.59% higher than DeepLab v3+. In the spatial generalisation performance test

the average IOU of Field-Net model is 3.51% higher than that of Link-Net model

and the spatial generalisation performance is significantly improved. Ablation tests demonstrated that the spatial attention mechanism and multi-task learning strategy improved F1-Score by 1.01% and IOU by 1.6% compared to the ResUNet model. The multi-task learning strategy led to a 0.18% F1-Score improvement for Field-Net and a 0.21% improvement in IOU.Conclusion While challenges remain in identifying contiguous fields due to unclear boundaries

future enhancements could incorporate multi-temporal and higher-resolution remote sensing images to improve field feature discrimination. Feature visualization analysis revealed that the spatial attention mechanism and multi-task learning strategy enabled the model to learn clustered features at field boundaries and within plots

enhancing feature representativeness. Overall

the Field-Net model supports field-level monitoring of cropland use

including non-agricultural applications

such as grain production

enhancing the efficiency and timeliness of land resource monitoring. In the process of generating the field dataset in China

the complex and fragmented cropland bring considerable challenges to this task. In the future

the problem of lack of samples for model training can be solved by accumulating field segmentation datasets from different regions by borrowing the paradigm of Image-Net

while a more general model for channels

regions

and sensors should be constructed subsequently. In the future

with the arrival of the "large model" era of deep learning

for the task of parcel segmentation

it is also necessary to construct a model to segment every field from the perspective of both the model and the dataset.

关键词

地块分割Field-Net模型空间注意力机制多任务学习高分卫星数据

Keywords

Filed segmentationField-Net modelSpatial attention mechanismMulti-task learningGF satellite data

references

Falk T., D. Mai, R. Bensch, Ö. Çiçek, A. Abdulkadir, Y. Marrakchi, A. Böhm, J. Deubner, Z. Jäckel, K. Seiwald. 2019. U-Net: deep learning for cell counting, detection, and morphometry. Nature methods, 16(1): 67-70. [DOI 10.1038/s41592-018-0261-2http://dx.doi.org/10.1038/s41592-018-0261-2]

Fu J., J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, H. Lu. 2019. Dual attention network for scene segmentation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. [DOI 10.48550/arXiv.1809.02983http://dx.doi.org/10.48550/arXiv.1809.02983]

He K., X. Zhang, S. Ren, J. Sun. 2016. Identity mappings in deep residual networks. European conference on computer vision, Springer. [DOI 10.48550/arXiv.1603.05027http://dx.doi.org/10.48550/arXiv.1603.05027]

Jeppesen J. H., R. H. Jacobsen, F. Inceoglu, T. S. Toftegaard. 2019. A cloud detection algorithm for satellite imagery based on deep learning. Remote sensing of environment, 229247-259. [DOI 10.1016/j.rse.2019.03.039http://dx.doi.org/10.1016/j.rse.2019.03.039]

Li Z., J. Dong. 2022. A Framework Integrating DeeplabV3+, Transfer Learning, Active Learning, and Incremental Learning for Mapping Building Footprints. Remote Sensing, 14(19): 4738. [DOI 10.3390/rs14194738http://dx.doi.org/10.3390/rs14194738]

Liu M., B. Fu, D. Fan, P. Zuo, S. Xie, H. He, L. Liu, L. Huang, E. Gao, M. Zhao. 2021. Study on transfer learning ability for classifying marsh vegetation with multi-sensor images using DeepLabV3+ and HRNet deep learning algorithms. International Journal of Applied Earth Observation and Geoinformation, 103102531. [DOI 10.1016/j.jag.2021.102531http://dx.doi.org/10.1016/j.jag.2021.102531]

Murugesan B., K. Sarveswaran, S. M. Shankaranarayana, K. Ram, J. Joseph, M. Sivaprakasam. 2019. Psi-Net: Shape and boundary aware joint multi-task deep network for medical image segmentation. 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE. [DOI 10.1109/EMBC.2019.8857339http://dx.doi.org/10.1109/EMBC.2019.8857339]

Persello C., V. A. Tolpekin, J. R. Bergado, R. A. de By. 2019. Delineation of agricultural fields in smallholder farms from satellite images using fully convolutional networks and combinatorial grouping. Remote Sens Environ, 231111253. [DOI 10.1016/j.rse.2019.111253http://dx.doi.org/10.1016/j.rse.2019.111253]

Ronneberger O., P. Fischer, T. Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention, Springer. [DOI 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28]

Saraiva M., É. Protas, M. Salgado, C. Souza. 2020. Automatic mapping of center pivot irrigation systems from satellite images using deep learning. Remote Sensing, 12(3): 558. [DOI 10.3390/rs12030558http://dx.doi.org/10.3390/rs12030558]

Sun J., F. Darbehani, M. Zaidi, B. Wang. 2020. Saunet: Shape attentive u-net for interpretable medical image segmentation. Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, PartIV 23, Springer. [DOI 10.1007/978-3-030-59719-1_77http://dx.doi.org/10.1007/978-3-030-59719-1_77]

Takikawa T., D. Acuna, V. Jampani, S. Fidler. 2019. Gated-scnn: Gated shape cnns for semantic segmentation. Proceedings of the IEEE/CVF international conference on computer vision, [DOI 10.48550/arXiv.1907.05740http://dx.doi.org/10.48550/arXiv.1907.05740]

Tian F., B. Wu, H. Zeng, M. Zhang, Y. Hu, Y. Xie, C. Wen, Z. Wang, X. Qin, W. Han. 2023. A shape-attention Pivot-Net for identifying central pivot irrigation systems from satellite images using a cloud computing platform: an application in the contiguous US. GIScience & Remote Sensing, 60(1): 2165256. [DOI 10.1080/15481603.2023.2165256http://dx.doi.org/10.1080/15481603.2023.2165256]

Vaswani A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdfhttps://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

Waldner F., F. I. Diakogiannis. 2020. Deep learning on edge: Extracting field boundaries from satellite images with a convolutional neural network. Remote Sensing of Environment, 245111741. [DOI 10.1016/j.rse.2020.111741http://dx.doi.org/10.1016/j.rse.2020.111741]

Wang F., M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, X. Tang. 2017. Residual attention network for image classification. Proceedings of the IEEE conference on computer vision and pattern recognition, [DOI 10.1109/CVPR.2017.683http://dx.doi.org/10.1109/CVPR.2017.683]

Woo S., J. Park, J.-Y. Lee, I. S. Kweon. 2018. Cbam: Convolutional block attention module. Proceedings of the European conference on computer vision (ECCV), [DOI 10.48550/arXiv.1807.06521http://dx.doi.org/10.48550/arXiv.1807.06521]

Wu B., M. Zhang, H. Zeng, F. Tian, A. B. Potgieter, X. Qin, N. Yan, S. Chang, Y. Zhao, Q. Dong, V. Boken, D. Plotnikov, H. Guo, F. Wu, H. Zhao, B. Deronde, L. Tits, E. Loupian. 2023. Challenges and opportunities in remote sensing-based crop monitoring: a review. Natl Sci Rev, 10(4): nwac290. [DOI 10.1093/nsr/nwac290http://dx.doi.org/10.1093/nsr/nwac290]

Xu L., P. Yang, J. Yu, F. Peng, J. Xu, S. Song, Y. Wu. 2023. Extraction of cropland field parcels with high resolution remote sensing using multi-task learning. European Journal of Remote Sensing, 56(1). [DOI 10.1080/22797254.2023.2181874http://dx.doi.org/10.1080/22797254.2023.2181874]

Yi Y., Z. Zhang, W. Zhang, C. Zhang, W. Li, T. Zhao. 2019. Semantic segmentation of urban buildings from VHR remote sensing imagery using a deep convolutional neural network. Remote sensing, 11(15): 1774. [DOI 10.3390/rs11151774http://dx.doi.org/10.3390/rs11151774]

Zhang Z., Q. Liu, Y. Wang. 2018. Road extraction by deep residual u-net. IEEE Geoscience and Remote Sensing Letters, 15(5): 749-753. [DOI 10.1109/LGRS.2018.2802944http://dx.doi.org/10.1109/LGRS.2018.2802944]

Zhou L., C. Zhang, M. Wu. 2018. D-LinkNet: LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. [DOI 10.1109/CVPRW.2018.00034http://dx.doi.org/10.1109/CVPRW.2018.00034]

Cheng R, Wei Y B, Lu M, and Wu W B. 2022. Cultivated Land Parcel Extraction Based on Ensemble Deep Learning Model. Agricultural Resources and Zoning in China, 43(7): 273-281

程锐, 魏妍冰, 陆苗, 吴文斌. 2022. 基于集成深度学习模型的耕地地块提取. 中国农业资源与区划, 007: 043. [DOI 10.7621/cjarrp.1005-9121.20220727http://dx.doi.org/10.7621/cjarrp.1005-9121.20220727]

Han Y X, and Meng J H 2019. A review of per - field crop classification using remote sensing. Agricultural Resources and Zoning in China, 31(2): 1-9

韩衍欣, 蒙继华. 2019. 面向地块的农作物遥感分类研究进展. 国土资源遥感, 31(2): 1-

9[DOI 10.6046/gtzyyg.2019.02.01http://dx.doi.org/10.6046/gtzyyg.2019.02.01]

Li B. 2020. Study on the Application of Remote Sensing Technology in Land Resources Survey--The Third National Land Survey as an Example. Metallurgy and Materials. 40(05):51+53.

李博. 2020. 遥感技术在国土资源调查中的应用研究——以第三次全国土地调查为例. 冶金与材料, 40(05):51+53. [DOI CNKI:SUN:NYYS.0.2019-17-072]

Lijin County Government . (2021). "Lizzie County Natural Resources." 利津县政府. (2021). "利津县自然资源." [2023-05-13] http://www.lijin.gov.cn/col/col70883/index.htmlhttp://www.lijin.gov.cn/col/col70883/index.html.

Lin M P, Li J and Shen H F.2023. PolSAR image deep learning super-resolution model based on multi-scale attention mechanism[J/OL]. National Remote Sensing Bulletin, 2023,1-10.

林镠鹏, 李杰, 沈焕锋. 2023. 基于多尺度注意力机制的PolSAR深度学习超分辨率模型. 遥感学报, 1-10. [DOI: 10.11834/jrs.20233002].

Song Q, Hu Q, Lu M, Yu Q Y, Yang P, Shi Y, Duan Y L and Wu WB, 2020. Prospect of crop mapping. Agricultural Resources and Zoning in China, 41(6): 57-65

宋茜, 胡琼, 陆苗, 余强毅, 杨鹏, 史云, 段玉林, 吴文斌. 2020. 农作物空间分布遥感制图发展方向探讨. 中国农业资源与区划, 41(6): 57-

65[DOI 10.7621/cjarrp.1005-9121.20200607http://dx.doi.org/10.7621/cjarrp.1005-9121.20200607]

Yang Y P, Wu Z F, Luo J C, Huang Q T, Zhang D Y, Wu T J, Sun Y W, Cao Z, Dong W and Liu W. 2021. Parcel-based crop distribution extraction using the spatiotemporal collaboration of remote sensing data. Journal of Agricultural Engineering, 37(7): 166-174

杨颖频, 吴志峰, 骆剑承, 黄启厅, 张冬韵, 吴田军, 孙营伟, 曹峥, 董文, 刘巍. 2021. 时空协同的地块尺度作物分布遥感提取. 农业工程学报, 37(7): 166-

174[DOI 10.11975/j.issn.1002-6819.2021.07.020http://dx.doi.org/10.11975/j.issn.1002-6819.2021.07.020]

Zhang H T, Gao H F, and Ren C. 2022. Deep Segmentation and Extraction of Cultivated Land in Hilly Areas Based on lmproved Unet, Space return and remote sensing, 43(4): 36-45.

张海天, 高懋芳, 任超. 2022. 基于改进Unet++的丘陵地区耕地地块深度分割与提取. 航天返回与遥感, 43(4): 36-

45[DOI 10.11975/j.issn.1002-6819.2021.07.020http://dx.doi.org/10.11975/j.issn.1002-6819.2021.07.020]

Zhang Q K, Meng J H and Ren C. 2022. Crop classification based on two-dimensional representation and CNN model from remote sensing. National Remote Sensing Bulletin, 26(7): 1437-1449.

张乾坤, 蒙继华, 任超. 2022. 构建地块二维表征及CNN模型的作物遥感分类. 遥感学报, 26(7): 1437-1449 [DOI 10.11834/jrs.20219432http://dx.doi.org/10.11834/jrs.20219432]

Zhou N, Yang P, Wei C S, Shen Z F, Yu JJ, Ma X Y, and Luo J C 2021. Accurate extraction method for cropland in mountainous areas based on field parcel. Transactions of the Chinese Society of Agricultural Engineering, 37(19): 260-266

周楠, 杨鹏, 魏春山, 沈占锋, 余娟娟, 马晓宇, 骆剑承. 2021. 地块尺度的山区耕地精准提取方法. 农业工程学报, 37(19): 260-

266[DOI 10.11975/j.issn.1002-6819.2021.19.030http://dx.doi.org/10.11975/j.issn.1002-6819.2021.19.030]

Zhu Y, Pan Y Z and Zhang D J 2022. An Agriculture Parcel Identification Method based on Convolutional Neural Network and Watershed Segmentation. Journal of Geo-Information Science, 24(12): 2389-2403

朱昱, 潘耀忠, 张杜娟. 2022. 基于深度卷积网络和分水岭分割的耕地地块识别方法. 地球信息科学学报, 24(12): 2389-2403 [DOI 10.12082/dqxxkx.2022.220447http://dx.doi.org/10.12082/dqxxkx.2022.220447]

Alert me when the article has been cited

提交

暂无数据