Learning Robust Multi-scale Representation for Neural Radiance Fields from Unposed Images Article Swipe

MAKE IMPACT

Nishant Jain , Suryansh Kumar , Luc Van Gool · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.1007/s11263-023-01936-1

We introduce an improved solution to the neural image-based rendering problem in computer vision. Given a set of images taken from a freely moving camera at train time, the proposed approach could synthesize a realistic image of the scene from a novel viewpoint at test time. The key ideas presented in this paper are (i) Recovering accurate camera parameters via a robust pipeline from unposed day-to-day images is equally crucial in neural novel view synthesis problem; (ii) It is rather more practical to model object's content at different resolutions since dramatic camera motion is highly likely in day-to-day unposed images. To incorporate the key ideas, we leverage the fundamentals of scene rigidity, multi-scale neural scene representation, and single-image depth prediction. Concretely, the proposed approach makes the camera parameters as learnable in a neural fields-based modeling framework. By assuming per view depth prediction is given up to scale, we constrain the relative pose between successive frames. From the relative poses, absolute camera pose estimation is modeled via a graph-neural network-based multiple motion averaging within the multi-scale neural-fields network, leading to a single loss function. Optimizing the introduced loss function provides camera intrinsic, extrinsic, and image rendering from unposed images. We demonstrate, with examples, that for a unified framework to accurately model multiscale neural scene representation from day-to-day acquired unposed multi-view images, it is equally essential to have precise camera-pose estimates within the scene representation framework. Without considering robustness measures in the camera pose estimation pipeline, modeling for multi-scale aliasing artifacts can be counterproductive. We present extensive experiments on several benchmark datasets to demonstrate the suitability of our approach.

Type

article

Language

en

Landing Page

https://doi.org/10.1007/s11263-023-01936-1

OA Status

hybrid

Cited By

5

References

46

Related Works

10

OpenAlex ID

https://openalex.org/W4388593728

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4388593728

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.1007/s11263-023-01936-1

Digital Object Identifier

Title

Learning Robust Multi-scale Representation for Neural Radiance Fields from Unposed Images

Work title

Type

article

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-11-11

Full publication date if available

Authors

Nishant Jain, Suryansh Kumar, Luc Van Gool

List of authors in order

Landing page

https://doi.org/10.1007/s11263-023-01936-1

Publisher landing page

Open access

Yes

Whether a free full text is available

OA status

hybrid

Open access status per OpenAlex

OA URL

https://doi.org/10.1007/s11263-023-01936-1

Direct OA link when available

Concepts

Artificial intelligence, Computer vision, Computer science, Rendering (computer graphics), Artificial neural network, Robustness (evolution), Leverage (statistics), View synthesis, Gene, Chemistry, Biochemistry

Top concepts (fields/topics) attached by OpenAlex

Cited by

5

Total citation count in OpenAlex

Citations by year (recent)

2025: 5

Per-year citation counts (last 5 years)

References (count)

46

Number of works referenced by this work

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload