单样本对卷积神经网络遥感图像时空融合
Convolutional neural network based single image pair method for spatiotemporal fusion
- 2022年26卷第8期 页码：1614-1623
DOI： 10.11834/jrs.20219348
引用
扫 描 看 全 文
摘要
遥感图像时空融合是一种生成兼具高时空分辨率的合成遥感数据的技术。近年来，产生了一些基于卷积神经网络的时空融合方法。这些方法效果良好，但需要较多的图像样本对训练模型，限制了它们的应用。针对此问题，本文提出了一种单样本对卷积神经网络时空融合方法（SS-CNN）。该方法以高空间分辨率图像的波段平均图像提供的空间信息激励卷积神经网络建立高、低空间分辨率图像间的超分关系，进而利用该超分关系映射求解目标高空间分辨率图像。在实验中使用两个模拟数据集和一个真实数据集对该方法进行了测试，并与两种常用的时空融合方法做了比较。实验结果表明，SS-CNN在单样本对训练的情况下，可以较好地预测地物的物候变化和类型的变化，且在异质性高、地块破碎的区域表现良好。其不足之处在于会在地物边界上会造成轻微的模糊，将来需针对此问题做进一步改进。
Abstract
Spatiotemporal fusion is a feasible way to provide synthetic satellite images with high spatial and high temporal resolution simultaneously. In recent years, some efficient STF methods based on Convolutional Neural Networks (CNNs) have been developed. However, these methods require a significant number of training image pairs, where each pair generally consists of a high spatial resolution image and a low spatial resolution image. Such a requirement limits the applicability of STF methods to actual scenarios because image pairs for training are not widely available in many cases. To overcome this important limitation, we introduce a CNN-based single image pair method for STF of remotely sensed images. Our method, called SS-CNN, uses the spatial information provided by the average image (obtained across available spectral bands) of the high spatial resolution image to perform CNN-based Super-Resolution Mapping (SRM) between the low and high spatial resolution images. The proposed SS-CNN has been tested in experiments using two simulated and one real dataset and compared with two commonly used spatiotemporal fusion methods. The experimental results show that SS-CNN can predict the phenological changes and land cover changes well. Plus, its performance in heterogeneous areas is remarkable. The disadvantage is that it will slightly blur the boundary, which needs to be further improved in the future.
关键词
遥感时空融合遥感图像单样本对卷积神经网络
Keywords
remote sensingspatio-temporal fusionremote sensing imagessingle image pairConvolutional Neural Networks (CNN)
references
Boyte S P, Wylie B K, Rigge M B and Dahal D. 2018. Fusing MODIS with Landsat 8 data to downscale weekly normalized difference vegetation index estimates for central great basin rangelands, USA. GIScience and Remote Sensing, 55(3): 376-399 [DOI: 10.1080/15481603.2017.1382065http://dx.doi.org/10.1080/15481603.2017.1382065]
Gao F, Masek J, Schwaller M and Hall F. 2006. On the blending of the Landsat and MODIS surface reflectance: predicting daily Landsat surface reflectance. IEEE Transactions on Geoscience and Remote Sensing, 44(8): 2207-2218 [DOI: 10.1109/TGRS.2006.872081http://dx.doi.org/10.1109/TGRS.2006.872081]
Gevaert C M and García-Haro F J. 2015. A comparison of STARFM and an unmixing-based algorithm for Landsat and MODIS data fusion. Remote Sensing of Environment, 156: 34-44 [DOI: 10.1016/j.rse.2014.09.012http://dx.doi.org/10.1016/j.rse.2014.09.012]
Heimhuber V, Tulbure M G and Broich M. 2018. Addressing spatio-temporal resolution constraints in Landsat and MODIS-based mapping of large-scale floodplain inundation dynamics. Remote Sensing of Environment, 211: 307-320 [DOI: 10.1016/j.rse.2018.04.016http://dx.doi.org/10.1016/j.rse.2018.04.016]
Hilker T, Wulder M A, Coops N C, Linke J, McDermid G, Masek J G, Gao F and White J C. 2009. A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS. Remote Sensing of Environment, 113(8): 1613-1627 [DOI: 10.1016/j.rse.2009.03.007http://dx.doi.org/10.1016/j.rse.2009.03.007]
Huang B, and Song H H. 2012. Spatiotemporal reflectance fusion via sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 50(10): 3707-3716 [DOI: 10.1109/TGRS.2012.2186638http://dx.doi.org/10.1109/TGRS.2012.2186638]
Huang B, Zhao Y Q. 2017. Research status and prospect of spatiotemporal fusion of multi-source satellite remote sensing imagery. Acta Geodaetica et Cartographica Sinica, 46(10): 1492-1499
黄波, 赵涌泉. 2017. 多源卫星遥感影像时空融合研究的现状及展. 测绘学报, 46(10): 1492-1499 [DOI: 10.11947/j.AGCS.2017.20170376http://dx.doi.org/10.11947/j.AGCS.2017.20170376]
Ke Y H, Im J, Park S and Gong H L. 2016. Downscaling of MODIS one kilometer evapotranspiration using Landsat-8 data and machine learning approaches. Remote Sensing, 8(3): 215 [DOI: 10.3390/rs8030215http://dx.doi.org/10.3390/rs8030215]
Kingma D P and Ba J. 2015. Adam: a method for stochastic optimization//3rd International Conference on Learning Representations. San Diego: ICLR.
Knipper K R, Kustas W P, Anderson M C, Alfieri J G, Prueger J H, Hain C R, Gao F, Yang Y, McKee L G, Nieto H, Hipps L E, Alsina M M and Sanchez L. 2019. Evapotranspiration estimates derived using thermal-based satellite remote sensing and data fusion for irrigation management in California vineyards. Irrigation Science, 37(3): 431-449 [DOI: 10.1007/s00271-018-0591-yhttp://dx.doi.org/10.1007/s00271-018-0591-y]
Li D C, Li Y R, Yang W F, Ge Y Q, Han Q J, Ma L L, Chen Y H and Li X. 2018. An enhanced single-pair learning-based reflectance fusion algorithm with spatiotemporally extended training samples. Remote Sensing, 10(8): 1207 [DOI: 10.3390/rs10081207http://dx.doi.org/10.3390/rs10081207]
Liao C H, Wang J F, Dong T F, Shang J L, Liu J G and Song Y. 2019. Using spatio-temporal fusion of Landsat-8 and MODIS data to derive phenology, biomass and yield estimates for corn and soybean. Science of the Total Environment, 650: 1707-1721 [DOI: 10.1016/j.scitotenv.2018.09.308http://dx.doi.org/10.1016/j.scitotenv.2018.09.308]
Liu X, Deng X W, Chanussot J, Hong D F and Zhao B J. 2019. StfNet: a two-stream convolutional neural network for spatiotemporal image fusion. IEEE Transactions on Geoscience and Remote Sensing, 57(9): 6552-6564 [DOI: 10.1109/TGRS.2019.2907310http://dx.doi.org/10.1109/TGRS.2019.2907310]
Nduati E, Sofue Y, Matniyaz A, Park J G, Yang W and Kondoh A. 2019. Cropland mapping using fusion of multi-sensor data in a complex Urban/Peri-Urban area. Remote Sensing, 2(11): 207 [DOI: 10.3390/rs11020207http://dx.doi.org/10.3390/rs11020207]
Pan Y Q, Shen F and Wei X D. 2018. Fusion of landsat-8/OLI and GOCI data for hourly mapping of suspended particulate matter at high spatial resolution: a case study in the yangtze (Changjiang) estuary. Remote Sensing, 2(10): 158 [DOI: 10.3390/rs10020158http://dx.doi.org/10.3390/rs10020158]
Park B, Windham W R, Lawrence K C and Smith D P. 2007. Contaminant classification of poultry hyperspectral imagery using a spectral angle mapper algorithm. Biosystems Engineering, 96(3): 323-333 [DOI: 10.1016/j.biosystemseng.2006.11.012http://dx.doi.org/10.1016/j.biosystemseng.2006.11.012]
Renza D, Martinez E and Arquero A. 2013. A new approach to change detection in multispectral images by means of ERGAS index. IEEE Geoscience and Remote Sensing Letters, 10(1): 76-80 [DOI: 10.1109/LGRS.2012.2193372http://dx.doi.org/10.1109/LGRS.2012.2193372]
Song H H and Huang B. 2013. Spatiotemporal satellite image fusion through one-pair image learning. IEEE Transactions on Geoscience and Remote Sensing, 51(4): 1883-1896 [DOI: 10.1109/TGRS.2012.2213095http://dx.doi.org/10.1109/TGRS.2012.2213095]
Song H H, Liu Q S, Wang G J, Hang R L and Huang B. 2018. Spatiotemporal satellite image fusion using deep convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3): 821-829 [DOI: 10.1109/JSTARS.2018.2797894http://dx.doi.org/10.1109/JSTARS.2018.2797894]
Wang Z, Bovik A C, Sheikh H R and Simoncelli E P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600-612 [DOI: 10.1109/TIP.2003.819861http://dx.doi.org/10.1109/TIP.2003.819861]
Weng Q H, Peng F and Feng G. 2014. Generating daily land surface temperature at Landsat resolution by fusing Landsat and MODIS data. Remote Sensing of Environment, 145(8): 55-67 [DOI: 10.1016/j.rse.2014.02.003http://dx.doi.org/10.1016/j.rse.2014.02.003]
Wu B, Huang B and Zhang L P. 2015. An error-bound-regularized sparse coding for spatiotemporal reflectance fusion. IEEE Transactions on Geoscience and Remote Sensing, 53(12): 6791-6803 [DOI: 10.1109/TGRS.2015.2448100http://dx.doi.org/10.1109/TGRS.2015.2448100]
Wu M Q, Huang W J, Niu Z and Wang C Y. 2015. Generating daily synthetic Landsat imagery by combining Landsat and MODIS data. Sensors, 15(9): 24002-24025 [DOI: 10.3390/s150924002http://dx.doi.org/10.3390/s150924002]
Zhang M, Lin H, Wang G, Sun H and Cai Y T. 2019. Estimation of vegetation productivity using a Landsat 8 time series in a heavily Urbanized area, central China. Remote Sensing, 11(2): 133 [DOI: 10.3390/rs11020133http://dx.doi.org/10.3390/rs11020133]
Zhang W, Li A N, Jin H A, Bian J H, Zhang Z J, Lei G B, Qin Z H and Huang C Q. 2013. An enhanced spatial and temporal data fusion model for fusing Landsat and MODIS surface reflectance to generate high temporal Landsat-like data. Remote Sensing, 5(10): 5346-5368 [DOI: 10.3390/rs5105346http://dx.doi.org/10.3390/rs5105346]
Zhao C Y, Gao X B, Emery W J, Ying W and Li J. 2018. An integrated spatio-spectral-temporal sparse representation method for fusing remote-sensing images with different resolutions. IEEE Transactions on Geoscience and Remote Sensing, 56(6): 3358-3370 [DOI: 10.1109/TGRS.2018.2798663http://dx.doi.org/10.1109/TGRS.2018.2798663]
Zhu X L, Chen J, Gao F, Chen X H and Masek J G. 2010. An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions. Remote Sensing of Environment, 114(11): 2610-2623 [DOI: 10.1016/j.rse.2010.05.032http://dx.doi.org/10.1016/j.rse.2010.05.032]
Zhu X L, Helmer E H,, Gao F, Liu D S, Chen J and Lefsky M A. 2016. A flexible spatiotemporal method for fusing satellite images with different resolutions. Remote Sensing of Environment, 172: 165-177 [DOI: 10.1016/j.rse.2015.11.016http://dx.doi.org/10.1016/j.rse.2015.11.016]
文章被引用时，请邮件提醒。
提交
相关作者
暂无数据
相关机构
中南林业科技大学 林业遥感信息工程研究中心
中南林业科技大学 林业遥感大数据与生态安全湖南省重点实验室
中南林业科技大学 南方森林资源经营与监测国家林业与草原局重点实验室
中国林业科学研究院资源信息研究所
中国地质大学(武汉) 地理与信息工程学院
- 地址：北京市海淀区北四环西路19号 《遥感学报》编辑部 邮编：100190
- 联系电话：010-58887052 Email：nrsb@aircas.ac.cn
- 技术支持由北京北大方正电子有限公司提供 京ICP备20021838号-9
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
0
回到顶部