Ben Mildenhall
YOU?
Author Swipe
View article: Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering
Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering Open
State-of-the-art techniques for 3D reconstruction are largely based on volumetric scene representations, which require sampling multiple points to compute the color arriving along a ray. Using these representations for more general inverse…
View article: NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections
NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections Open
Neural Radiance Fields (NeRFs) typically struggle to reconstruct and render highly specular objects, whose appearance varies quickly with changes in viewpoint. Recent works have improved NeRF's ability to render detailed specular appearanc…
View article: GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation
GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation Open
Creating 4D fields of Gaussian Splatting from images or videos is a challenging task due to its under-constrained nature. While the optimization can draw photometric reference from the input videos or be regulated by generative models, dir…
View article: Disentangled 3D Scene Generation with Layout Learning
Disentangled 3D Scene Generation with Layout Learning Open
We introduce a method to generate 3D scenes that are disentangled into their component objects. This disentanglement is unsupervised, relying only on the knowledge of a large pretrained text-to-image model. Our key insight is that objects …
View article: Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis
Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis Open
While surface-based view synthesis algorithms are appealing due to their low computational requirements, they often struggle to reproduce thin structures. In contrast, more expensive methods that model the scene's geometry as a volumetric …
View article: Nuvo: Neural UV Mapping for Unruly 3D Representations
Nuvo: Neural UV Mapping for Unruly 3D Representations Open
Existing UV mapping algorithms are designed to operate on well-behaved meshes, instead of the geometry representations produced by state-of-the-art 3D reconstruction and generation techniques. As such, applying these methods to the volume …
View article: CamP: Camera Preconditioning for Neural Radiance Fields
CamP: Camera Preconditioning for Neural Radiance Fields Open
Neural Radiance Fields (NeRF) can be optimized to obtain high-fidelity 3D scene reconstructions of objects and large-scale scenes. However, NeRFs require accurate camera parameters as input --- inaccurate camera parameters result in blurry…
View article: ReconFusion: 3D Reconstruction with Diffusion Priors
ReconFusion: 3D Reconstruction with Diffusion Priors Open
3D reconstruction methods such as Neural Radiance Fields (NeRFs) excel at rendering photorealistic novel views of complex scenes. However, recovering a high-quality NeRF typically requires tens to hundreds of input images, resulting in a t…
View article: Generative Powers of Ten
Generative Powers of Ten Open
We present a method that uses a text-to-image model to generate consistent content across multiple image scales, enabling extreme semantic zooms into a scene, e.g., ranging from a wide-angle landscape view of a forest to a macro shot of an…
View article: State of the Art on Diffusion Models for Visual Computing
State of the Art on Diffusion Models for Visual Computing Open
The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes. …
View article: CamP: Camera Preconditioning for Neural Radiance Fields
CamP: Camera Preconditioning for Neural Radiance Fields Open
Neural Radiance Fields (NeRF) can be optimized to obtain high-fidelity 3D scene reconstructions of objects and large-scale scenes. However, NeRFs require accurate camera parameters as input -- inaccurate camera parameters result in blurry …
View article: BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis
BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis Open
We present a method for reconstructing high-quality meshes of large unbounded real-world scenes suitable for photorealistic novel view synthesis. We first optimize a hybrid neural volume-surface scene representation designed to have well-b…
View article: Eclipse: Disambiguating Illumination and Materials using Unintended Shadows
Eclipse: Disambiguating Illumination and Materials using Unintended Shadows Open
Decomposing an object's appearance into representations of its materials and the surrounding illumination is difficult, even when the object's 3D shape is known beforehand. This problem is especially challenging for diffuse objects: it is …
View article: Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields
Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields Open
Neural Radiance Field training can be accelerated through the use of grid-based representations in NeRF's learned mapping from spatial coordinates to colors and volumetric density. However, these grid-based approaches lack an explicit unde…
View article: DreamBooth3D: Subject-Driven Text-to-3D Generation
DreamBooth3D: Subject-Driven Text-to-3D Generation Open
We present DreamBooth3D, an approach to personalize text-to-3D generative models from as few as 3-6 casually captured images of a subject. Our approach combines recent advances in personalizing text-to-image models (DreamBooth) with text-t…
View article: BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis
BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis Open
We present a method for reconstructing high-quality meshes of large unbounded real-world scenes suitable for photorealistic novel view synthesis. We first optimize a hybrid neural volume-surface scene representation designed to have well-b…
View article: MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes
MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes Open
Neural radiance fields enable state-of-the-art photorealistic view synthesis. However, existing radiance field representations are either too compute-intensive for real-time rendering or require too much memory to scale to large scenes. We…
View article: AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training
AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training Open
Neural Radiance Fields (NeRFs) are a powerful representation for modeling a 3D scene as a continuous function. Though NeRF is able to render complex 3D scenes with view-dependent effects, few efforts have been devoted to exploring its limi…
View article: DreamFusion: Text-to-3D using 2D Diffusion
DreamFusion: Text-to-3D using 2D Diffusion Open
Recent breakthroughs in text-to-image synthesis have been driven by diffusion models trained on billions of image-text pairs. Adapting this approach to 3D synthesis would require large-scale datasets of labeled 3D data and efficient archit…
View article: Volume Rendering Digest (for NeRF)
Volume Rendering Digest (for NeRF) Open
Neural Radiance Fields employ simple volume rendering as a way to overcome the challenges of differentiating through ray-triangle intersections by leveraging a probabilistic notion of visibility. This is achieved by assuming the scene is c…
View article: Block-NeRF: Scalable Large Scene Neural View Synthesis
Block-NeRF: Scalable Large Scene Neural View Synthesis Open
We present Block-NeRF, a variant of Neural Radiance Fields that can represent large-scale environments. Specifically, we demonstrate that when scaling NeRF to render city-scale scenes spanning multiple blocks, it is vital to decompose the …
View article: Fast and High-Quality Image Denoising via Malleable Convolutions
Fast and High-Quality Image Denoising via Malleable Convolutions Open
Most image denoising networks apply a single set of static convolutional kernels across the entire input image. This is sub-optimal for natural images, as they often consist of heterogeneous visual patterns. Dynamic convolution tries to ad…
View article: NeRF
NeRF Open
We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene …
View article: Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields
Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields Open
Neural Radiance Fields (NeRF) is a popular view synthesis technique that represents a scene as a continuous volumetric function, parameterized by multilayer perceptrons that provide the volume density and view-dependent emitted radiance at…
View article: Dense Depth Priors for Neural Radiance Fields from Sparse Input Views
Dense Depth Priors for Neural Radiance Fields from Sparse Input Views Open
Neural radiance fields (NeRF) encode a scene into a neural representation that enables photo-realistic rendering of novel views. However, a successful reconstruction from RGB images requires a large number of input views taken under static…
View article: Zero-Shot Text-Guided Object Generation with Dream Fields
Zero-Shot Text-Guided Object Generation with Dream Fields Open
We combine neural rendering with multi-modal image and text representations to synthesize diverse 3D objects solely from natural language descriptions. Our method, Dream Fields, can generate the geometry and color of a wide range of object…
View article: RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs
RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs Open
Neural Radiance Fields (NeRF) have emerged as a powerful representation for the task of novel view synthesis due to their simplicity and state-of-the-art performance. Though NeRF can produce photorealistic renderings of unseen viewpoints w…
View article: NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images
NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images Open
Neural Radiance Fields (NeRF) is a technique for high quality novel view synthesis from a collection of posed input images. Like most view synthesis methods, NeRF uses tonemapped low dynamic range (LDR) as input; these images have been pro…
View article: Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields Open
Though neural radiance fields (NeRF) have demonstrated impressive view synthesis results on objects and small bounded regions of space, they struggle on "unbounded" scenes, where the camera may point in any direction and content may exist …