سابِق|تال

Fast 3D reconstruction of satellite images via the Global Affine Model

  • role： First author第一作者
  • Affiliation：

    National Key Laboratory of Multi-modal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China

    School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China

    Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing 100190, China

  • Email：chenbao2019@ia.ac.cn
  • Introduction：陈豹，研究方向为三维计算机视觉。E-mail： chenbao2019@ia.ac.cn
CHEN Bao123,  
  • Affiliation：

    School of Computer Science and Engineering, University of Northeastern, Shenyang 110169, China

WANG Pinhe4,  
  • role： Corresponding author通信作者
  • Affiliation：

    National Key Laboratory of Multi-modal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China

    School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China

    Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing 100190, China

  • Email：qldong@nlpr.ia.ac.cn
  • Introduction：董秋雷，研究方向为模式分类、视觉定位与三维建模。E-mail： qldong@nlpr.ia.ac.cn
DONG Qiulei123*

ملخص

Three-dimensional (3D) scene reconstruction based on multiview satellite remote sensing images is a challenging task in the field of remote sensing. Most of the existing methods either have to perform bundle adjustment repeatedly or must calculate several parameters in the rational polynomial camera model, resulting in a relatively long reconstruction time. To solve the abovementioned problems, this study considers that the local small-sized patches in large-sized satellites could be approximately modeled by the affine imaging model and proposes a fast 3D reconstruction method of satellite images based on global affine model estimation.First, the input multiview satellite images are cropped into a set of small-sized patches with overlapping regions. For each pair of patches that have a sufficient number of point correspondences from two views, the corresponding 3D affine point cloud is calculated. Second, on the basis of the obtained local point clouds, a global affine camera motion estimation algorithm is presented for calculating the affine motion matrices of the cameras corresponding to all the patches in a unified coordinate system. Finally, the obtained affine camera motion matrices and a few ground control points are utilized to recover the Euclidean scene structure.3D reconstruction is conducted for the same group of remote sensing images and all remote sensing images to verify the effectiveness of the method. The proposed solution is compared with three state-of-the-art methods (i.e., COLMAP, S2P, and JHUAPL). Experimental results on two public datasets (i.e., MVS3DM and DFC2019) show that the proposed method outperforms the three comparison algorithms in most cases with respect to speed, accuracy, and completeness. To verify further the reconstruction accuracy of the method, this study selects 15 complex scene areas from two public datasets, including complex scenes with built-up areas, shadow areas, and complex object areas. For 15 complex scenarios, the proposed method outperforms the three methods with respect to accuracy and completeness in most cases.This study proposes a fast reconstruction method of satellite images based on the global affine model estimation algorithm. The method assumes that the local image tile in large-scale satellite remote sensing images conforms to the affine imaging model and introduces a global affine motion matrix estimation algorithm based on local point clouds. Consequently, the proposed solution can calculate the global affine motion matrix of each local image tile through only one bundle adjustment, considerably reducing the reconstruction running time. Experimental results show that the proposed method can quickly solve the global affine matrix corresponding to each image tile and realize fast 3D reconstruction of remote sensing images.

مفهوم

3D reconstruction;satellite images;affine imaging model;Euclidean structure update;global affine matrix

References

  1. 1.
    Agarwal S and Mierle K. 2012. Ceres solver: Tutorial and reference. Google Inc, 2(72): 8.
  2. 2.
    Bosch M, Kurtz Z, Hagstrom S and Brown M. 2016. A multiple view stereo benchmark for satellite imagery//Proceedings of the 2016 IEEE Applied Imagery Pattern Recognition Workshop (AIPR). Washington, USA: IEEE: 1-9
  3. 3.
    Bosch M, Leichtman A, Chilcott D, Goldberg H and Brown M. 2017. Metric evaluation pipeline for 3D modeling of urban scenes. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-1/W1: 239-246
  4. 4.
    Bosch M, Foster K, Christie G, Wang S A, Hager G D and Brown M. 2019. Semantic stereo for incidental satellite images//Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE: 1524-1532
  5. 5.
    D’Angelo P and Reinartz P. 2012. DSM based orientation of large stereo satellite image blocks. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 39(B1): 209-214
  6. 6.
    De Franchis C, Meinhardt-Llopis E, Michel J, Morel J M and Facciolo G. 2014. An automatic and modular stereo pipeline for pushbroom images. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, II-3: 49-56
  7. 7.
    Facciolo G, De Franchis C and Meinhardt E. 2015. MGM: A significantly more global matching for stereovision//Proceedings of the British Machine Vision Conference. Swansea: BMVA Press
  8. 8.
    Facciolo G, De Franchis C and Meinhardt-Llopis E. 2017. Automatic 3D reconstruction from multi-date satellite images//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu: IEEE: 1542-1551
  9. 9.
    Fischler M A and Bolles R C. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6): 381-395
  10. 10.
    Fraser C S, Dare P M and Yamakawa T. 2004. Digital surface modeling from spot 5 HRS imagery using the affine projective model//Proceedings of the XXth International Society for Photogrammetry and Remote Sensing Congress. 35: 385-388
  11. 11.
    Govindu V M. 2004. Lie-algebraic averaging for globally consistent motion estimation//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. Washington: IEEE: I
  12. 12.
    Hartley R I and Saxena T. 1997. The cubic rational polynomial camera model//Proceeding of the Image Understanding Workshop. New Orleans, USA: Citeceer: 649-653
  13. 13.
    Hartley R and Zisserman A. 2004. Multiple View Geometry in Computer Vision. Cambridge: Cambridge University Press.
  14. 14.
    Huber P J. 1981. Robust statistics//Lovric M, ed. International Encyclopedia of Statistical Science. Berlin: Springer
  15. 15.
    Kuschk G. 2013. Large scale urban reconstruction from remote sensing imagery. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XL-5-W1: 139-146
  16. 16.
    Li C, Xiong H, Tao S Y and Han Z K. 2014. Quantitative change detection of vegetation and three-dimensional terrain for geographical disaster areas from optical stereopair images. Journal of Remote Sensing (in Chinese), 18(6): 1258-1267
  17. 17.
    Lowe D G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2): 91-110
  18. 18.
    Ozcanli O C, Yi D, Mundy J L, Webb H, Hammoud R and Tom V. 2015. A comparison of stereo and multiview 3-D reconstruction using cross-sensor satellite imagery//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Boston, USA: IEEE
  19. 19.
    Rupnik E, Daakir M and Deseilligny M P. 2017. MicMac–a free, open-source solution for photogrammetry. Open Geospatial Data, Software and Standards, 2(1): 14
  20. 20.
    Schonberger J L and Frahm J M. 2016. Structure-from-motion revisited//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 4104-4113
  21. 21.
    Snavely N, Seitz S M and Szeliski R. 2008. Modeling the world from internet photo collections. International Journal of Computer Vision, 80(2): 189-210
  22. 22.
    Triggs B, McLauchlan P F, Hartley R I and Fitzgibbon A. 2000. Bundle adjustment—a modern synthesis//Proceedings of the International Workshop on Vision Algorithms. Corfu: Springer: 298-372
  23. 23.
    Xue J S, Yi H, Wu Z H and Chen X N. 2020. A hybrid multi-view 3D reconstruction method based on scene graph partition. Acta Automatica Sinica, 46(4): 782-795
  24. 24.
    Wang P H, Shi L M, Chen B, Hu Z Y, Dong Q L and Qiao J Z. 2022. Pursuing 3-D scene structures with optical satellite images from affine reconstruction to Euclidean reconstruction. IEEE Transactions on Geoscience and Remote Sensing, 60: 5632214
  25. 25.
    Wilson K and Snavely N. 2014. Robust global translations with 1DSfM//Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland: Springer: 61-75
  26. 26.
    Yu Y, Zhang Y S, Xue W and Wang T. 2019. A incremental structure from motion method of robustness enhancement and accuracy improvement. Acta Geodaetica et Cartographica Sinica, 48(2): 207-215
  27. 27.
    Zhang G, Jiang Y H, Li L T, Deng M J and Zhao R S. 2019. Research progress of high-resolution optical/SAR satellite geometric radiometric calibration. Acta Geodaetica et Cartographica Sinica, 48(12): 1604-1623
  28. 28.
    Zhang K, Snavely N and Sun J. 2019. Leveraging vision reconstruction pipelines for satellite imagery//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshops. Seoul, Korea: IEEE: 2139-2148

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website