Früher|Nächster

Progress in neural radiance field and its application in large-scale real-scene 3D visualization

  • role： First author第一作者
  • Affiliation：

    Jiangsu Provincial Key Laboratory of Geographic Information Science and Technology, Key Laboratory for Land Satellite Remote Sensing Applications of Ministry of Natural Resources, School of Geography and Ocean Science, Nanjing University, Nanjing 210023, China

    Henan Agricultural Remote Sensing Big Data Development and Innovation Laboratory, Henan Engineering Technology Research Center of Ecological protection and management of the Old Course of Yellow River, College of Surveying and Planning, Shangqiu Normal University, Shangqiu 476000, China

  • Email：giser.zq@smail.nju.edu.cn
  • Introduction：赵强，研究方向为虚拟地理环境与计算机视觉。E-mail： giser.zq@smail.nju.edu.cn
ZHAO Qiang12,  
  • role： Corresponding author通信作者
  • Affiliation：

    Jiangsu Provincial Key Laboratory of Geographic Information Science and Technology, Key Laboratory for Land Satellite Remote Sensing Applications of Ministry of Natural Resources, School of Geography and Ocean Science, Nanjing University, Nanjing 210023, China

  • Email：GISJF@nju.edu.cn
  • Introduction：佘江峰，研究方向为虚拟地理环境与计算机视觉。E-mail： GISJF@nju.edu.cn
SHE Jiangfeng1*,  
  • Affiliation：

    Jiangsu Provincial Key Laboratory of Geographic Information Science and Technology, Key Laboratory for Land Satellite Remote Sensing Applications of Ministry of Natural Resources, School of Geography and Ocean Science, Nanjing University, Nanjing 210023, China

WAN Qifeng1,  
  • Affiliation：

    Jiangsu Provincial Key Laboratory of Geographic Information Science and Technology, Key Laboratory for Land Satellite Remote Sensing Applications of Ministry of Natural Resources, School of Geography and Ocean Science, Nanjing University, Nanjing 210023, China

HE Lixia1,  
  • Affiliation：

    Jiangsu Provincial Key Laboratory of Geographic Information Science and Technology, Key Laboratory for Land Satellite Remote Sensing Applications of Ministry of Natural Resources, School of Geography and Ocean Science, Nanjing University, Nanjing 210023, China

LI Sirui1,  
  • Affiliation：

    Jiangsu Provincial Key Laboratory of Geographic Information Science and Technology, Key Laboratory for Land Satellite Remote Sensing Applications of Ministry of Natural Resources, School of Geography and Ocean Science, Nanjing University, Nanjing 210023, China

WU Shuangpin1

Resümee

Geographical real-scene 3D scenes are an important national digital infrastructure, which extends geographic information from 2D to 3D. Real-scene 3D data are stored and expressed in the form of an explicit 3D model, which has the problems of large amount of data and rough visualization effect. Neural Radiance Field (NeRF), realizing differentiable rendering based on neural implicit volume representation, is an innovative approach of high-quality view synthesis. First proposed by Mildenhall et al. (2020), NeRF has become one of the hottest research direction in the field of computer vision due to its realistic view synthesis effect. A large amount of literature about NeRF have been published since NeRF was proposed, and the application of NeRF in large-scale real-scene 3D visualization has begun to attract the attention of some published papers.View synthesis, which uses sparse 2D images to generate realistic new views at any viewpoint in 3D space without the reconstruction of 3D models, is a novel way to realize the representation of 3D scenes. The development of view synthesis technology has gone through several stages: image mosaicking, 3D model reprojection, view interpolation, and volume representing technology. NeRF, as an innovative approach of view synthesis, samples 5D coordinates (location and viewing direction) along camera rays, feeds those locations into a multilayer perceptron network to produce color and volume density, and uses volume rendering techniques to composite these values into a new image. NeRF not only produces remarkably higher-quality rendering than prior volumetric approaches but also requires just a fraction of the storage cost of other sampled volumetric representations. However, it faces problems such as requirements for high quality of source data, failure to support dynamic objects, low efficiency in processing, and single type of render target. Moreover, NeRF-related research are mostly conducted based on laboratory environment or standardized data at present. Due to these drawbacks, many obstacles need to be overcome before applying NeRF to large-scale real-scene 3D visualization.This paper reviews the workaround of unbounded scene, aliasing, luminosity consistency, scene relighting, and visibility field in the Block-NeRF algorithm, a variant of NeRF that can represent large-scale environments (Tancik et al. 2022). The Block-NeRF algorithm splits the environment into a set of Block-NeRFs that can be independently trained in parallel and composited during inference, and it selects relevant Block-NeRFs for rendering, which are then composited smoothly when traversing the scene. To aid with this compositing, Block-NeRF optimizes the appearance codes to match lighting conditions. It trains individual Block-NeRFs using techniques such as appearance embeddings, learned pose refinement, exposure input, transient objects, and visibility prediction.Nowadays, the studies of applying NeRF to large-scale real-scene 3D visualization are being conducted extensively, which has attracted much attention and plays a pioneering, leading role in further research. The results of these studies achieve the most basic 3D visualization large-scale real-scene effect, but they are limited by some conditions and their universality needs to be strongly improved. Because NeRF is still far from practical applications in producing large-scale real-scene 3D visualization, any slight progressive exploration is likely to become a continuous research hotspot. This paper identifies the challenges of NeRF research, including multisource data fusion, visual effect optimization, and virtual environment perception, which need more research.

Schlüsselwort

remote sensing;Neural Radiance Field;View synthesis;Implicit volume representing;computer vision;virtual geographic environment

References

  1. 1.
    Agarwal S, Furukawa Y, Snavely N, Simon I, Curless B, Seitz S M and Szeliski R. 2011. Building Rome in a day. Communications of the ACM, 54(10): 105-112
  2. 2.
    Athar S, Shu Z X and Samaras D. 2021. FLAME-in-NeRF: neural control of radiance fields for free view face animation. arXiv: 2108.04913
  3. 3.
    Attal B, Laidlaw E, Gokaslan A, Kim C, Richardt C, Tompkin J and O'Toole M. 2021. TöRF: time-of-flight radiance fields for dynamic scene view synthesis. arXiv: 2109.15271
  4. 4.
    Barron J T, Mildenhall B, Tancik M, Hedman P, Martin-Brualla R and Srinivasan P P. 2021. Mip-NeRF: a multiscale representation for anti-aliasing neural radiance fields//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 5835-5844
  5. 5.
    Barron J T, Mildenhall B, Verbin D, Srinivasan P P and Hedman P. 2022. Mip-NeRF 360: unbounded anti-aliased neural radiance fields//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 5460-5469
  6. 6.
    Boss M, Braun R, Jampani V, Barron J T, Liu C and Lensch H P A. 2021a. NeRD: neural reflectance decomposition from image collections//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 12664-12674
  7. 7.
    Boss M, Jampani V, Braun R, Liu C, Barron J T and Lensch H P A. 2021b. Neural-PIL: neural pre-integrated lighting for reflectance decomposition. arXiv:2110.14373
  8. 8.
    Buehler C, Bosse M, McMillan L, Gortler S and Cohen M. 2001. Unstructured lumigraph rendering//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. Los Angeles: ACM: 425-432
  9. 9.
    Chan E R, Monteiro M, Kellnhofer P, Wu J J and Wertzstein G. 2021. pi-GAN: periodic implicit generative adversarial networks for 3D-aware image synthesis//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE: 5795-5805
  10. 10.
    Chang Y and Gai M. 2021. A review on neural radiance fields based view synthesis. Journal of Graphics, 42(3): 376-384
  11. 11.
    Chen A P, Xu Z X, Zhao F Q, Zhang X S, Xiang F B, Yu J Y and Su H. 2021. MVSNeRF: fast generalizable radiance field reconstruction from multi-view stereo//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 14104-14113
  12. 12.
    Chen W Z, Gao J, Ling H, Smith E J, Lehtinen J, Jacobson A and Fidler S. 2019. Learning to predict 3D objects with an interpolation-based differentiable renderer//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc.: 9609-9619
  13. 13.
    Chen X Y, Zhang Q, Li X Y, Chen Y, Ying F, Wang X and Wang J. 2022. Hallucinated neural radiance fields in the wild//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 12933-12942
  14. 14.
    Cheng B W, Collins M D, Zhu Y K, Liu T, Huang T S, Adam H and Chen L C. 2020. Panoptic-DeepLab: a simple, strong, and fast baseline for bottom-up panoptic segmentation//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE: 12472-12482
  15. 15.
    Choi I, Gallo O, Troccoli A, Kim M H and Kautz J. 2019. Extreme view synthesis//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE: 7780-7789
  16. 16.
    Curless B and Levoy M. 1996. A volumetric method for building complex models from range images//Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. New Orleans: ACM: 303-312
  17. 17.
    Davis A, Levoy M and Durand F. 2012. Unstructured light fields. Computer Graphics Forum, 31: 305-314
  18. 18.
    Davison A J, Reid I D, Molton N D and Stasse O. 2007. MonoSLAM: real-time single camera SLAM. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6): 1052-1067
  19. 19.
    Deng K L, Liu A, Zhu J Y and Ramanan D. 2022. Depth-supervised NeRF: fewer views and faster training for free//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 12872-12881
  20. 20.
    Derksen D and Izzo D. 2021. Shadow neural radiance fields for multi-view satellite photogrammetry//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Nashville: IEEE: 1152-1161
  21. 21.
    Dey A, Ahmine Y and Comport A I. 2022. Mip-NeRF RGB-D: depth assisted fast neural radiance fields. arXiv: 2205.09351
  22. 22.
    Engel J, Schöps T and Cremers D. 2014. LSD-SLAM: large-scale direct monocular SLAM//13th European Conference on Computer Vision–ECCV 2014. Zürich: Springer: 834-849
  23. 23.
    Fang J M, Yi T R, Wang X G, Xie L X, Zhang X P, Liu W Y, Nießner M and Tian Q. 2022. Fast dynamic radiance fields with time-aware neural voxels//SIGGRAPH Asia 2022 Conference Papers. Daegu: ACM: 1-9
  24. 24.
    Früh C and Zakhor A. 2004. An automated method for large-scale, ground-based city model acquisition. International Journal of Computer Vision, 60(1): 5-24
  25. 25.
    Gao C, Shih Y, Lai W S, Liang C K and Huang J B. 2021. Portrait neural radiance fields from a single image. arXiv: 2012.05903
  26. 26.
    Genova K, Cole F, Sud A, Sarna A and Funkhouser T. 2020. Local deep implicit functions for 3D shape//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE: 4856-4865
  27. 27.
    Gong J H, Lin H, Xiao L B and Xie C J. 1999. Perspective on geo-visualization. Journal of Remote Sensing, 3(3): 236-244
  28. 28.
    Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A and Bengio Y. 2020. Generative adversarial networks. Communications of the ACM, 63(11): 139-144
  29. 29.
    Guo X, Chen G Y, Dai Y C, Ye X Q, Sun J D, Tan X and Ding E R. 2022a. Neural deformable voxel grid for fast optimization of dynamic view synthesis//Proceedings of the 16th Asian Conference on Computer Vision (ACCV). Macau, China: Springer: 450-468
  30. 30.
    Guo Y C, Kang D, Bao L C, He Y and Zhang S H. 2022b. NeRFReN: neural radiance fields with reflections//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 18388-18397
  31. 31.
    Haklay M and Weber P. 2008. OpenStreetMap: user-generated street maps. IEEE Pervasive Computing, 7(4): 12-18
  32. 32.
    Hedman P, Philip J, Price T, Frahm J M, Drettakis G and Brostow G. 2018. Deep blending for free-viewpoint image-based rendering. ACM Transactions on Graphics, 37(6): 257
  33. 33.
    Hedman P, Srinivasan P P, Mildenhall B, Barron J T and Debevec P. 2021. Baking neural radiance fields for real-time view synthesis//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 5855-5864
  34. 34.
    Hoang L, Lee S H, Kwon O H and Kwon K R. 2019. A deep learning method for 3D object classification using the wave kernel signature and a center point of the 3D-triangle mesh. Electronics, 8(10): 1196
  35. 35.
    Hong Y, Peng B, Xiao H Y, Liu L G and Zhang J Y. 2022. HeadNeRF: a realtime NeRF-based parametric head model//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 20342-20352
  36. 36.
    Huang X, Zhang Q, Feng Y, Li H D, Wang X and Wang Q. 2022a. HDR-NeRF: high dynamic range neural radiance fields//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 18377-18387
  37. 37.
    Huang Y H, He Y, Yuan Y J, Lai Y K and Gao L. 2022b. StylizedNeRF: consistent 3D scene stylization as stylized NeRF via 2D-3D mutual learning//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 18321-18331
  38. 38.
    Hwang I, Kim J and Kim Y M. 2023. Ev-NeRF: event based neural radiance field//Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa: IEEE: 837-847
  39. 39.
    Jain A, Tancik M and Abbeel P. 2021. Putting NeRF on a diet: semantically consistent few-shot view synthesis//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 5865-5874
  40. 40.
    Jang W and Agapito L. 2021. CodeNeRF: disentangled neural radiance fields for object categories//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 12929-12938
  41. 41.
    Jiang C Y, Sud A, Makadia A, Huang J W, Nießner M and Funkhouser T. 2020. Local implicit grid representations for 3D scenes//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE: 6000-6009
  42. 42.
    Jo K, Shim G, Jung S, Yang S and Choo J. 2023. CG-NeRF: conditional generative neural radiance fields for 3D-aware image synthesis//Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa: IEEE: 724-733
  43. 43.
    Johari M M, Lepoittevin Y and Fleuret F. 2022. GeoNeRF: generalizing NeRF with geometry priors//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 18344-18347
  44. 44.
    Kada M, Wichmann A, Filippovska Y and Hermes T. 2016. Animation strategies for smooth transformations between discrete LODs of 3D building models. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLI-B2: 413-420
  45. 45.
    Kada M, Wichmann A and Hermes T. 2015. Smooth transformations between generalized 3D building models for visualization purposes. Cartography and Geographic Information Science, 42(4): 306-314
  46. 46.
    Kania K, Yi K M, Kowalski M, Trzciński T and Tagliasacchi A. 2022. CoNeRF: controllable neural radiance fields//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 18602-18611
  47. 47.
    Khademi W and Ventura J. 2021. View synthesis in casually captured scenes using a cylindrical neural radiance field with exposure compensation//ACM SIGGRAPH 2021 Posters. Virtual Event: ACM: 28 [DOI: 10.1145/3450618.3469147]
  48. 48.
    Kingma D P and Welling M. 2022. Auto-encoding variational bayes. arXiv: 1312.6114
  49. 49.
    Kondo N, Ikeda Y, Tagliasacchi A, Matsuo Y, Ochiai Y and Gu S S. 2021. VaxNeRF: revisiting the classic for voxel-accelerated neural radiance field. arXiv: 2111.13112
  50. 50.
    Kosiorek A R, Strathmann H, Zoran D, Moreno P, Schneider R, Mokrá S and Rezende D J. 2021. NeRF-VAE: a geometry aware 3D scene generative model//Proceedings of the 38th International Conference on Machine Learning (ICML). Virtual: [s.n.]: 5742-5752
  51. 51.
    Kutulakos K N and Seitz S M. 2000. A theory of shape by space carving. International Journal of Computer Vision, 38(3): 199-218
  52. 52.
    Levoy M and Hanrahan P. 1996. Light field rendering//Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. New Orleans: ACM Press: 31-42
  53. 53.
    Li D R, Xiao X W, Guo B X, Jiang W S and Shi Y R. 2016. Oblique image based automatic aerotriangulation and its application in 3D city model reconstruction. Geomatics and Information Science of Wuhan University, 41(6): 711-721
  54. 54.
    Li J X, Feng Z J, She Q, Ding H H, Wang C H and Lee G H. 2021a. MINE: towards continuous depth MPI with NeRF for novel view synthesis//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 12558-12568
  55. 55.
    Li T M, Aittala M, Durand F and Lehtinen J. 2018. Differentiable Monte Carlo ray tracing through edge sampling. ACM Transactions on Graphics, 37(6): 222
  56. 56.
    Li Z P, Li L, Ma Z Y, Zhang P, Chen J B and Zhu J K. 2022. READ: large-scale neural scene rendering for autonomous driving. arXiv: 2205.05509
  57. 57.
    Li Z Q, Niklaus S, Snavely N and Wang O. 2021b. Neural scene flow fields for space-time view synthesis of dynamic scenes//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE: 6494-6504
  58. 58.
    Lin C H, Ma W C, Torralba A and Lucey S. 2021. BARF: bundle-adjusting neural radiance fields//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 5721-5731
  59. 59.
    Lin H T, Peng S D, Xu Z, Yan Y Z, Shuai Q, Bao H J and Zhou X W. 2022. Efficient neural radiance fields for interactive free-viewpoint video//SIGGRAPH Asia 2022 Conference Papers. Daegu: ACM: 39 [DOI: 10.1145/3550469.3555376]
  60. 60.
    Liu J W, Cao Y P, Mao W J, Zhang W Q, Zhang D J, Keppo J, Shan Y, Qie X H ang Shou M Z. 2022. DeVRF: fast deformable voxel radiance fields for dynamic scenes. arXiv: 2205.15723
  61. 61.
    Liu L J, Gu J T, Lin K Z, Chua T S and Theobalt C. 2021a. Neural sparse voxel fields. arXiv: 2007.11571
  62. 62.
    Liu S, Zhang X M, Zhang Z T, Zhang R, Zhu J Y and Russell B. 2021b. Editing conditional radiance fields//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 5753-5763
  63. 63.
    Lombardi S, Simon T, Saragih J, Schwartz G, Lehrmann A and Sheikh Y. 2019. Neural volumes: learning dynamic renderable volumes from images. ACM Transactions on Graphics, 38(4): 65
  64. 64.
    Loper M M and Black M J. 2014. OpenDR: an approximate differentiable renderer//13th European Conference on Computer Vision. Zürich: Springer International Publishing: 154-169
  65. 65.
    Luo H M, Chen A P, Zhang Q X, Pang B, Wu M Y, Xu L and Yu J Y. 2021. Convolutional neural opacity radiance fields//2021 IEEE International Conference on Computational Photography (ICCP). Haifa: IEEE: 1-12
  66. 66.
    Ma L, Li X Y, Liao J, Zhang Q, Wang X, Wang J and Sander P V. 2022. Deblur-NeRF: neural radiance fields from blurry images//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 12851-12860
  67. 67.
    Marí R, Facciolo G and Ehret T. 2022. Sat-NeRF: learning multi-view satellite photogrammetry with transient objects and shadow modeling using RPC cameras//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPR). New Orleans: IEEE: 1310-1320
  68. 68.
    Martin-Brualla R, Radwan N, Sajjadi M S M, Barron J T, Dosovitskiy A and Duckworth D. 2021. NeRF in the Wild: neural radiance fields for unconstrained photo collections//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE: 7206-7215
  69. 69.
    Meng Q, Chen A P, Luo H M, Wu M Y, Su H, Xu L, He X M and Yu J Y. 2021. GNeRF: GAN-based neural radiance field without posed camera//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 6351-6361
  70. 70.
    Mescheder L, Oechsle M, Niemeyer M, Nowozin S and Geiger A. 2019. Occupancy networks: learning 3D reconstruction in function space//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE: 4455-4465
  71. 71.
    Mildenhall B, Hedman P, Martin-Brualla R, Srinivasan P P and Barron J T. 2022. NeRF in the Dark: high dynamic range view synthesis from noisy raw images//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 16169-16178
  72. 72.
    Mildenhall B, Srinivasan P P, Tancik M, Barron J T, Ramamoorthi R and Ng R. 2020. NeRF: representing scenes as neural radiance fields for view synthesis. arXiv: 2003.08934
  73. 73.
    Moreau A, Piasco N, Tsishkou D, Stanciulescu B and de la Fortelle A. 2021. LENS: localization enhanced by NeRF synthesis//Proceedings of the 5th Conference on Robot Learning. London: PMLR: 1347-1356
  74. 74.
    Mur-Artal R, Montiel J M M and Tardós J D. 2015. ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Transactions on Robotics, 31(5): 1147-1163
  75. 75.
    Neff T, Stadlbauer P, Parger M, Kurz A, Mueller J H, Chaitanya C R A, Kaplanyan A and Steinberger M. 2021. DONeRF: towards real-time rendering of compact neural radiance fields using depth oracle networks. Computer Graphics Forum, 40(4): 45-59
  76. 76.
    Newcombe R A, Lovegrove S J and Davison A J. 2011. DTAM: dense tracking and mapping in real-time//Proceedings of the 2011 International Conference on Computer Vision. Barcelona: IEEE: 2320-2327
  77. 77.
    Nguyen-Phuoc T, Liu F and Xiao L. 2022. SNeRF: stylized neural implicit representations for 3D scenes. ACM Transactions on Graphics, 41(4): 142
  78. 78.
    Niemeyer M and Geiger A. 2021. GIRAFFE: representing scenes as compositional generative neural feature fields//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE: 11448-11459
  79. 79.
    Niemeyer M, Mescheder L, Oechsle M and Geiger A. 2020. Differentiable volumetric rendering: learning implicit 3D representations without 3D supervision//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE: 3501-3512
  80. 80.
    Ohori K A, Ledoux H, Biljecki F and Stoter J. 2015. Modeling a 3D city model and its levels of detail as a true 4D model. ISPRS International Journal of Geo-Information, 4(3): 1055-1075
  81. 81.
    Özyeşil O, Voroninski V, Basri R and Singer A. 2017. A survey of structure from motion. Acta Numerica, 26: 305-364
  82. 82.
    Park J J, Florence P, Straub J, Newcombe R and Lovegrove S. 2019. DeepSDF: learning continuous signed distance functions for shape representation//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE: 165-174
  83. 83.
    Park K, Sinha U, Hedman P, Barron J T, Bouaziz S, Goldman D B, Martin-Brualla R and Seitz S M. 2021. HyperNeRF: a higher-dimensional representation for topologically varying neural radiance fields. ACM Transactions on Graphics, 40(6): 238
  84. 84.
    Pearl N, Treibitz T and Korman S. 2022. NAN: noise-aware NeRFs for burst-denoising//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 12662-12671
  85. 85.
    Peng S Y, Niemeyer M, Mescheder L, Pollefeys M and Geiger A. 2020. Convolutional occupancy networks//16th European Conference on Computer Vision (ECCV). Glasgow: Springer International Publishing: 523-540
  86. 86.
    Penner E and Zhang L. 2017. Soft 3D reconstruction for view synthesis. ACM Transactions on Graphics, 36(6): 235
  87. 87.
    Piala M and Clark R. 2021. TermiNeRF: ray termination prediction for efficient neural rendering//2021 International Conference on 3D Vision (3DV). London: IEEE: 1106-1114
  88. 88.
    Pumarola A, Corona E, Pons-Moll G and Moreno-Noguer F. 2021. D-NeRF: neural radiance fields for dynamic scenes//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE: 10313-10322
  89. 89.
    Radford A, Kim J W, Hallacy C, Ramesh A, Goh G, Agarwal S, Sastry G, Askell A, Mishkin P, Clark J, Krueger G and Sutskever I. 2021. Learning transferable visual models from natural language supervision//Proceedings of the 38th International Conference on Machine Learning (ICML). Virtual: [s.n.]: 8748-8763
  90. 90.
    Raguram R, Wu C C, Frahm J M and Lazebnik S. 2011. Modeling and recognition of landmark image collections using iconic scene graphs. International Journal of Computer Vision, 95(3): 213-239
  91. 91.
    Rebain D, Jiang W, Yazdani S, Li K, Yi K M and Tagliasacchi A. 2021. DeRF: decomposed radiance fields//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE: 14148-14156
  92. 92.
    Reiser C, Peng S Y, Liao Y Y and Geiger A. 2021. KiloNeRF: speeding up neural radiance fields with thousands of tiny MLPs//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 14315-14325
  93. 93.
    Rematas K, Liu A, Srinivasan P, Barron J, Tagliasacchi A, Funkhouser T and Ferrari V. 2022. Urban radiance fields//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 12922-12932
  94. 94.
    Rematas K, Martin-Brualla R and Ferrari V. 2021. ShaRF: shape-conditioned radiance fields from a single view//Proceedings of the 38th International Conference on Machine Learning (ICML). Virtual: [s.n.]: 8948-8958
  95. 95.
    Roessle B, Barron J T, Mildenhall B, Srinivasan P P and NießnerM. 2022. Dense depth priors for neural radiance fields from sparse input views//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 12882-12891
  96. 96.
    Rosu R A and Behnke S. 2022. NeuralMVS: bridging multi-view stereo and novel view synthesis//2022 International Joint Conference on Neural Networks (IJCNN). Padua: IEEE: 1-7
  97. 97.
    Rudnev V, Elgharib M, Smith W, Liu L J, Golyanik V and Theobalt C. 2022. Neural radiance fields for outdoor scene relighting//17th European Conference on Computer Vision (ECCV). Tel Aviv: Springer: 615-631
  98. 98.
    Schönberger J L and Frahm J M. 2016. Structure-from-Motion revisited//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE: 4104-4113
  99. 99.
    Schwarz K, Liao Y Y, Niemeyer M and Geiger A. 2021. GRAF: generative radiance fields for 3D-aware image synthesis. arXiv:2007.02442
  100. 100.
    Seitz S M and Dyer C R. 1999. Photorealistic scene reconstruction by voxel coloring. International Journal of Computer Vision, 35(2): 151-173
  101. 101.
    Shan Q, Adams R, Curless B, Furukawa Y and Seitz S M. 2013. The visual turing test for scene reconstruction//2013 International Conference on 3D Vision (3DV). Seattle: IEEE: 25-32
  102. 102.
    She J F, Chen B, Tan J Z, Zhao Q and Ge R C. 2022. 3D building model simplification method considering both model mesh and building structure. Transactions in GIS, 26(3): 1182-1203
  103. 103.
    Shen J X, Ruiz A, Agudo A and Moreno-Noguer F. 2021a. Stochastic neural radiance fields: quantifying uncertainty in implicit 3D representations//2021 International Conference on 3D Vision (3DV). London: IEEE: 972-981
  104. 104.
    Shen S Y, Wang Z, Liu P, Pan Z Q, Li R Q, Gao T, Li S Y and Yu J Y. 2021b. Non-line-of-sight imaging via neural transient fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(7): 2257-2268
  105. 105.
    Sitzmann V, Zollhöefer M and Wetzstein G. 2019. Scene representation networks: continuous 3D-structure-aware neural scene representations//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc.: 1121-1132
  106. 106.
    Srinivasan P P, Deng B Y, Zhang X M, Tancik M, Mildenhall B and Barron J T. 2021. NeRV: neural reflectance and visibility fields for relighting and view synthesis//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE: 7491-7500
  107. 107.
    Su S Y, Yu F, Zollhoefer M and Rhodin H. 2021. A-NeRF: articulated neural radiance fields for learning human shape, appearance, and pose. arXiv: 2102.06199
  108. 108.
    Sun J X, Wang X, Zhang Y, Li X Y, Zhang Q, Liu Y B and Wang J. 2021. FENeRF: face editing in neural radiance fields//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 7662-7672
  109. 109.
    Szeliski R and Golland P. 1998. Stereo matching with transparency and matting//Sixth International Conference on Computer Vision. Bombay: IEEE: 517-524
  110. 110.
    Szeliski R and Shum H Y. 1997. Creating full view panoramic image mosaics and environment maps//Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. Los Angeles: ACM Press: 251-258
  111. 111.
    Takikawa T, Litalien J, Yin K X, Kreis K, Loop C, Nowrouzezahrai D, Jacobson A, McGuire M and Fidler S. 2021. Neural geometric level of detail: real-time rendering with implicit 3D shapes//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE: 11353-11362
  112. 112.
    Tancik M, Casser V, Yan X C, Pradhan S, Mildenhall B P, Srinivasan P, Barron J T and Kretzschmar H. 2022. Block-NeRF: scalable large scene neural view synthesis//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 8238-8248
  113. 113.
    Tretschk E, Tewari A, Golyanik V, Zollhöfer M, Lassner C and Theobalt C. 2021. Non-rigid neural radiance fields: reconstruction and novel view synthesis of a dynamic scene from monocular video//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 12939-12950
  114. 114.
    Trevithick A and Yang B. 2021. GRF: learning a general radiance field for 3D representation and rendering//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 15162-15172
  115. 115.
    Tucker R and Snavely N. 2020. Single-view view synthesis with multiplane images//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE: 548-557
  116. 116.
    Turki H, Ramanan D and Satyanarayanan M. 2022. Mega-NeRF: scalable construction of large-scale NeRFs for virtual fly-throughs//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 12912-12921
  117. 117.
    Verbin D, Hedman P, Mildenhall B, Zickler T, Barron J T and Srinivasan P P. 2022. Ref-NeRF: structured view-dependent appearance for neural radiance fields//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 5481-5490
  118. 118.
    Wang C, Chai M L, He M M, Chen D D and Liao J. 2022a. CLIP-NeRF: text-and-image driven manipulation of neural radiance fields//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 3825-3834
  119. 119.
    Wang Z R, Wu S Z, Xie W D, Chen M and Prisacariu V A. 2022b. NeRF——: neural radiance fields without known camera parameters. arXiv: 2102.07064
  120. 120.
    Wang Z Y, Wang L, Zhao F Q, Wu M Y, Xu L and Yu J Y. 2022c. MirrorNeRF: one-shot neural portrait radiance field from multi-mirror catadioptric imaging//2021 IEEE International Conference on Computational Photography (ICCP). Haifa: IEEE: 1-12
  121. 121.
    Wang Z Y, Yang W, Cao J M, Xu L, Yu J Q and Yu J Y. 2022d. NeReF: neural refractive field for fluid surface reconstruction and implicit representation. arXiv: 2203.04130
  122. 122.
    Wei Y, Liu S H, Rao Y M, Zhao W, Lu J W and Zhou J. 2021. NerfingMVS: guided optimization of neural radiance fields for indoor multi-view stereo//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 5590-5599
  123. 123.
    Wu J, Zhang Z X and Zhang J Q. 2006. 3D city modeling based on videogrammetry from helicopter. Journal of Image and Graphics, 11(8): 1161-1170
  124. 124.
    Wu L W, Lee J Y, Bhattad A, Wang Y X and Forsyth D. 2022a. DIVeR: real-time and accurate neural radiance fields with deterministic integration for volume rendering//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 16179-16188
  125. 125.
    Wu T H, Zhong F C, Tagliasacchi A, Cole F and Oztireli C. 2022b. D2NeRF: self-supervised decoupling of dynamic and static objects from a monocular video. arXiv: 2205.15838
  126. 126.
    Xian W Q, Huang J B, Kopf J and Kim C. 2021. Space-time neural irradiance fields for free-viewpoint video//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE: 9416-9426
  127. 127.
    Xiangli Y B, Xu L N, Pan X G, Zhao N X, Rao A Y, Theobalt C, Dai B and Lin D H. 2022. Bungeenerf: progressive neural radiance field for extreme multi-scale scene rendering//17th European Conference on Computer Vision (ECCV). Tel Aviv: Springer: 106-122
  128. 128.
    Xie C, Park K, Martin-Brualla R and Brown M. 2021. FiG-NeRF: figure-ground neural radiance fields for 3D object category modelling//2021 International Conference on 3D Vision (3DV). London: IEEE: 962-971
  129. 129.
    Xu H Y, Alldieck T and Sminchisescu C. 2021. H-NeRF: neural radiance fields for rendering and temporal reconstruction of humans in motion. arXiv: 2110.13746
  130. 130.
    Xu Q G, Xu Z X, Philip J, Bi S, Shu Z X, Sunkavalli K and Neumann U. 2022a. Point-NeRF: point-based neural radiance fields//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 5428-5438
  131. 131.
    Xu T H, Fujita Y and Matsumoto E. 2022b. Surface-aligned neural radiance fields for controllable 3D human synthesis//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 15862-15871
  132. 132.
    Yan S, Zhang M J, Fan Y C, Tan X H, Liu Y, Peng Y and Liu Y X. 2021. Progress in the large-scale outdoor image 3D reconstruction. Journal of Image and Graphics, 26(6): 1429-1449
  133. 133.
    Yang B B, Zhang Y D, Xu Y H, Li Y J, Zhou H, Bao H J, Zhang G F and Cui Z P. 2021. Learning object-compositional neural radiance field for editable scene rendering//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 13759-13768
  134. 134.
    Yang G W, Zhou W Y, Peng H Y, Liang D, Mu T J and Hu S M. 2023. Recursive-NeRF: an efficient and dynamically growing NeRF. IEEE Transactions on Visualization and Computer Graphics, 29(12): 5124-5136
  135. 135.
    Yen-Chen L, Florence P, Barron J T, Rodriguez A, Isola P and Lin T Y. 2021. INeRF: inverting neural radiance fields for pose estimation//2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Prague: IEEE: 1323-1330
  136. 136.
    Yu A, Li R L, Tancik M, Li H, Ng R and Kanazawa A. 2021. PlenOctrees for real-time rendering of neural radiance fields//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE: 5732-5741
  137. 137.
    Zhang K, Riegler G, Snavely N and Koltun V. 2020. NeRF++: analyzing and improving neural radiance fields. arXiv: 2010.07492
  138. 138.
    Zhang X M, Srinivasan P P, Deng B Y, Debevec P, Freeman W T and Barron J T. 2021. NeRFactor: neural factorization of shape and reflectance under an unknown illumination. ACM Transactions on Graphics, 40(6): 237
  139. 139.
    Zhao F Q, Yang W, Zhang J K, Lin P, Zhang Y L, Yu J Y and Xu L. 2022. HumanNeRF: efficiently generated human radiance field from sparse inputs//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE: 7733-7743
  140. 140.
    Zhou T H, Tucker R, Flynn J, Fyffe G and Snavely N. 2018. Stereo magnification: learning view synthesis using multiplane images. ACM Transactions on Graphics, 37(4): 65
  141. 141.
    Zhu S Y, Zhang R Z, Zhou L, Shen T W, Fang T, Tan P and Quan L. 2018. Very large-scale global SfM by distributed motion averaging//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 4568-4577
  142. 142.
    Zhuang Y Y, Zhu H, Sun X S and Cao X. 2022. MoFaNeRF: morphable facial neural radiance field//17th European Conference on Computer Vision. Tel Aviv: Springer: 268-285

Lesen Sie die ganze Passage

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website