Précédent|Prochain

Convolutional neural network based single image pair method for spatiotemporal fusion

  • role： First author第一作者
  • Affiliation：

    Guangdong Provincial Key Laboratory of Urbanization and Geo-simulation, School of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, China

  • Email：liyf18213483@163.com
  • Introduction：李云飞，研究方向为多源遥感图像融合。E-mail： liyf18213483@163.com
LI Yunfei1,  
  • role： Corresponding author通信作者
  • Affiliation：

    Guangdong Provincial Key Laboratory of Urbanization and Geo-simulation, School of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, China

  • Email：lijun48@mail.sysu.edu.cn
  • Introduction：李军，研究方向为遥感图像处理等。E-mail： lijun48@mail.sysu.edu.cn
LI Jun1*,  
  • Affiliation：

    School of Automation Science and Engineering, South China University of Technology, Guangzhou 510640, China

HE Lin2

résumé

Spatiotemporal fusion is a feasible way to provide synthetic satellite images with high spatial and high temporal resolution simultaneously. In recent years, some efficient STF methods based on Convolutional Neural Networks (CNNs) have been developed. However, these methods require a significant number of training image pairs, where each pair generally consists of a high spatial resolution image and a low spatial resolution image. Such a requirement limits the applicability of STF methods to actual scenarios because image pairs for training are not widely available in many cases. To overcome this important limitation, we introduce a CNN-based single image pair method for STF of remotely sensed images. Our method, called SS-CNN, uses the spatial information provided by the average image (obtained across available spectral bands) of the high spatial resolution image to perform CNN-based Super-Resolution Mapping (SRM) between the low and high spatial resolution images. The proposed SS-CNN has been tested in experiments using two simulated and one real dataset and compared with two commonly used spatiotemporal fusion methods. The experimental results show that SS-CNN can predict the phenological changes and land cover changes well. Plus, its performance in heterogeneous areas is remarkable. The disadvantage is that it will slightly blur the boundary, which needs to be further improved in the future.

mots-clés

remote sensing;spatio-temporal fusion;remote sensing images;single image pair;Convolutional Neural Networks (CNN)

References

  1. 1.
    Boyte S P, Wylie B K, Rigge M B and Dahal D. 2018. Fusing MODIS with Landsat 8 data to downscale weekly normalized difference vegetation index estimates for central great basin rangelands, USA. GIScience and Remote Sensing, 55(3): 376-399
  2. 2.
    Gao F, Masek J, Schwaller M and Hall F. 2006. On the blending of the Landsat and MODIS surface reflectance: predicting daily Landsat surface reflectance. IEEE Transactions on Geoscience and Remote Sensing, 44(8): 2207-2218
  3. 3.
    Gevaert C M and García-Haro F J. 2015. A comparison of STARFM and an unmixing-based algorithm for Landsat and MODIS data fusion. Remote Sensing of Environment, 156: 34-44
  4. 4.
    Heimhuber V, Tulbure M G and Broich M. 2018. Addressing spatio-temporal resolution constraints in Landsat and MODIS-based mapping of large-scale floodplain inundation dynamics. Remote Sensing of Environment, 211: 307-320
  5. 5.
    Hilker T, Wulder M A, Coops N C, Linke J, McDermid G, Masek J G, Gao F and White J C. 2009. A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS. Remote Sensing of Environment, 113(8): 1613-1627
  6. 6.
    Huang B, and Song H H. 2012. Spatiotemporal reflectance fusion via sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 50(10): 3707-3716
  7. 7.
    Huang B, Zhao Y Q. 2017. Research status and prospect of spatiotemporal fusion of multi-source satellite remote sensing imagery. Acta Geodaetica et Cartographica Sinica, 46(10): 1492-1499
  8. 8.
    Ke Y H, Im J, Park S and Gong H L. 2016. Downscaling of MODIS one kilometer evapotranspiration using Landsat-8 data and machine learning approaches. Remote Sensing, 8(3): 215
  9. 9.
    Kingma D P and Ba J. 2015. Adam: a method for stochastic optimization//3rd International Conference on Learning Representations. San Diego: ICLR.
  10. 10.
    Knipper K R, Kustas W P, Anderson M C, Alfieri J G, Prueger J H, Hain C R, Gao F, Yang Y, McKee L G, Nieto H, Hipps L E, Alsina M M and Sanchez L. 2019. Evapotranspiration estimates derived using thermal-based satellite remote sensing and data fusion for irrigation management in California vineyards. Irrigation Science, 37(3): 431-449
  11. 11.
    Li D C, Li Y R, Yang W F, Ge Y Q, Han Q J, Ma L L, Chen Y H and Li X. 2018. An enhanced single-pair learning-based reflectance fusion algorithm with spatiotemporally extended training samples. Remote Sensing, 10(8): 1207
  12. 12.
    Liao C H, Wang J F, Dong T F, Shang J L, Liu J G and Song Y. 2019. Using spatio-temporal fusion of Landsat-8 and MODIS data to derive phenology, biomass and yield estimates for corn and soybean. Science of the Total Environment, 650: 1707-1721
  13. 13.
    Liu X, Deng X W, Chanussot J, Hong D F and Zhao B J. 2019. StfNet: a two-stream convolutional neural network for spatiotemporal image fusion. IEEE Transactions on Geoscience and Remote Sensing, 57(9): 6552-6564
  14. 14.
    Nduati E, Sofue Y, Matniyaz A, Park J G, Yang W and Kondoh A. 2019. Cropland mapping using fusion of multi-sensor data in a complex Urban/Peri-Urban area. Remote Sensing, 2(11): 207
  15. 15.
    Pan Y Q, Shen F and Wei X D. 2018. Fusion of landsat-8/OLI and GOCI data for hourly mapping of suspended particulate matter at high spatial resolution: a case study in the yangtze (Changjiang) estuary. Remote Sensing, 2(10): 158
  16. 16.
    Park B, Windham W R, Lawrence K C and Smith D P. 2007. Contaminant classification of poultry hyperspectral imagery using a spectral angle mapper algorithm. Biosystems Engineering, 96(3): 323-333
  17. 17.
    Renza D, Martinez E and Arquero A. 2013. A new approach to change detection in multispectral images by means of ERGAS index. IEEE Geoscience and Remote Sensing Letters, 10(1): 76-80
  18. 18.
    Song H H and Huang B. 2013. Spatiotemporal satellite image fusion through one-pair image learning. IEEE Transactions on Geoscience and Remote Sensing, 51(4): 1883-1896
  19. 19.
    Song H H, Liu Q S, Wang G J, Hang R L and Huang B. 2018. Spatiotemporal satellite image fusion using deep convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3): 821-829
  20. 20.
    Wang Z, Bovik A C, Sheikh H R and Simoncelli E P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600-612
  21. 21.
    Weng Q H, Peng F and Feng G. 2014. Generating daily land surface temperature at Landsat resolution by fusing Landsat and MODIS data. Remote Sensing of Environment, 145(8): 55-67
  22. 22.
    Wu B, Huang B and Zhang L P. 2015. An error-bound-regularized sparse coding for spatiotemporal reflectance fusion. IEEE Transactions on Geoscience and Remote Sensing, 53(12): 6791-6803
  23. 23.
    Wu M Q, Huang W J, Niu Z and Wang C Y. 2015. Generating daily synthetic Landsat imagery by combining Landsat and MODIS data. Sensors, 15(9): 24002-24025
  24. 24.
    Zhang M, Lin H, Wang G, Sun H and Cai Y T. 2019. Estimation of vegetation productivity using a Landsat 8 time series in a heavily Urbanized area, central China. Remote Sensing, 11(2): 133
  25. 25.
    Zhang W, Li A N, Jin H A, Bian J H, Zhang Z J, Lei G B, Qin Z H and Huang C Q. 2013. An enhanced spatial and temporal data fusion model for fusing Landsat and MODIS surface reflectance to generate high temporal Landsat-like data. Remote Sensing, 5(10): 5346-5368
  26. 26.
    Zhao C Y, Gao X B, Emery W J, Ying W and Li J. 2018. An integrated spatio-spectral-temporal sparse representation method for fusing remote-sensing images with different resolutions. IEEE Transactions on Geoscience and Remote Sensing, 56(6): 3358-3370
  27. 27.
    Zhu X L, Chen J, Gao F, Chen X H and Masek J G. 2010. An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions. Remote Sensing of Environment, 114(11): 2610-2623
  28. 28.
    Zhu X L, Helmer E H,, Gao F, Liu D S, Chen J and Lefsky M A. 2016. A flexible spatiotemporal method for fusing satellite images with different resolutions. Remote Sensing of Environment, 172: 165-177

Lire l'article complet

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website