Robustifying the Multi-Scale Representation of Neural Radiance Fields Article Swipe

MAKE IMPACT

Nishant Jain , Suryansh Kumar , Luc Van Gool · YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2210.04233

Neural Radiance Fields (NeRF) recently emerged as a new paradigm for object representation from multi-view (MV) images. Yet, it cannot handle multi-scale (MS) images and camera pose estimation errors, which generally is the case with multi-view images captured from a day-to-day commodity camera. Although recently proposed Mip-NeRF could handle multi-scale imaging problems with NeRF, it cannot handle camera pose estimation error. On the other hand, the newly proposed BARF can solve the camera pose problem with NeRF but fails if the images are multi-scale in nature. This paper presents a robust multi-scale neural radiance fields representation approach to simultaneously overcome both real-world imaging issues. Our method handles multi-scale imaging effects and camera-pose estimation problems with NeRF-inspired approaches by leveraging the fundamentals of scene rigidity. To reduce unpleasant aliasing artifacts due to multi-scale images in the ray space, we leverage Mip-NeRF multi-scale representation. For joint estimation of robust camera pose, we propose graph-neural network-based multiple motion averaging in the neural volume rendering framework. We demonstrate, with examples, that for an accurate neural representation of an object from day-to-day acquired multi-view images, it is crucial to have precise camera-pose estimates. Without considering robustness measures in the camera pose estimation, modeling for multi-scale aliasing artifacts via conical frustum can be counterproductive. We present extensive experiments on the benchmark datasets to demonstrate that our approach provides better results than the recent NeRF-inspired approaches for such realistic settings.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2210.04233

PDF

https://arxiv.org/pdf/2210.04233

OA Status

green

Cited By

6

Related Works

10

OpenAlex ID

https://openalex.org/W4304701439

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4304701439

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2210.04233

Digital Object Identifier

Title

Robustifying the Multi-Scale Representation of Neural Radiance Fields

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2022

Year of publication

Publication date

2022-10-09

Full publication date if available

Authors

Nishant Jain, Suryansh Kumar, Luc Van Gool

List of authors in order

Landing page

https://arxiv.org/abs/2210.04233

Publisher landing page

PDF URL

https://arxiv.org/pdf/2210.04233

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2210.04233

Direct OA link when available

Concepts

Artificial intelligence, Computer science, Computer vision, Pose, Rendering (computer graphics), Intrinsics, Robustness (evolution), Radiance, Gene, Chemistry, Biochemistry, Physics, Optics

Top concepts (fields/topics) attached by OpenAlex

Cited by

6

Total citation count in OpenAlex

Citations by year (recent)

2024: 1, 2023: 5

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload