View synthesis
View article
NeRF Open
We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene …
View article
3D Gaussian Splatting for Real-Time Radiance Field Rendering Open
Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while re…
View article
Stereo magnification Open
The view synthesis problem---generating novel views of a scene from known imagery---has garnered recent attention due in part to compelling applications in virtual and augmented reality. In this paper, we explore an intriguing scenario for…
View article
Learning-based view synthesis for light field cameras Open
With the introduction of consumer light field cameras, light field imaging has recently become widespread. However, there is an inherent trade-off between the angular and spatial resolution, and thus, these cameras often sparsely sample in…
View article
Deep blending for free-viewpoint image-based rendering Open
Free-viewpoint image-based rendering (IBR) is a standing challenge. IBR methods combine warped versions of input photos to synthesize a novel view. The image quality of this combination is directly affected by geometric inaccuracies of mul…
View article
Soft 3D reconstruction for view synthesis Open
We present a novel algorithm for view synthesis that utilizes a soft 3D reconstruction to improve quality, continuity and robustness. Our main contribution is the formulation of a soft 3D representation that preserves depth uncertainty thr…
View article
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis Open
While 2D generative adversarial networks have enabled high-resolution image synthesis, they largely lack an understanding of the 3D world and the image formation process. Thus, they do not provide precise control over camera viewpoint or o…
View article
Deep appearance models for face rendering Open
We introduce a deep appearance model for rendering the human face. Inspired by Active Appearance Models, we develop a data-driven rendering pipeline that learns a joint representation of facial geometry and appearance from a multiview capt…
View article
NeRF--: Neural Radiance Fields Without Known Camera Parameters Open
Considering the problem of novel view synthesis (NVS) from only a set of 2D images, we simplify the training process of Neural Radiance Field (NeRF) on forward-facing scenes by removing the requirement of known or pre-computed camera param…
View article
Neural actor Open
We propose Neural Actor (NA), a new method for high-quality synthesis of humans from arbitrary viewpoints and under arbitrary controllable poses. Our method is developed upon recent neural scene representation and rendering works which lea…
View article
StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis Open
We propose StyleNeRF, a 3D-aware generative model for photo-realistic high-resolution image synthesis with high multi-view consistency, which can be trained on unstructured 2D images. Existing approaches either cannot synthesize high-resol…
View article
Unsupervised Person Image Synthesis in Arbitrary Poses Open
Trabajo presentado en la IEEE/CVF Conference on Computer Vision and Pattern Recognition, celebrada en Salt Lake City (UT, USA), del 18 al 23 de junio de 2018
View article
Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications Open
In this paper, a novel convolutional neural network (CNN)-based framework is developed for light field reconstruction from a sparse set of views. We indicate that the reconstruction can be efficiently modeled as angular restoration on an e…
View article
Stereo Magnification: Learning View Synthesis using Multiplane Images Open
The view synthesis problem--generating novel views of a scene from known imagery--has garnered recent attention due in part to compelling applications in virtual and augmented reality. In this paper, we explore an intriguing scenario for v…
View article
BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis Open
We present a method for reconstructing high-quality meshes of large unbounded real-world scenes suitable for photorealistic novel view synthesis. We first optimize a hybrid neural volume-surface scene representation designed to have well-b…
View article
Point‐Based Neural Rendering with Per‐View Optimization Open
There has recently been great interest in neural rendering methods. Some approaches use 3D geometry reconstructed with Multi‐View Stereo (MVS) but cannot recover from the errors of this process, while others directly learn a volumetric neu…
View article
Video-to-Video Synthesis Open
We study the problem of video-to-video synthesis, whose goal is to learn a mapping function from an input source video (e.g., a sequence of semantic segmentation masks) to an output photorealistic video that precisely depicts the content o…
View article
Learning Light Field Angular Super-Resolution via a Geometry-Aware Network Open
The acquisition of light field images with high angular resolution is costly. Although many methods have been proposed to improve the angular resolution of a sparsely-sampled light field, they always focus on the light field with a small b…
View article
GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose Open
We propose GeoNet, a jointly unsupervised learning framework for monocular depth, optical flow and ego-motion estimation from videos. The three components are coupled by the nature of 3D scene geometry, jointly learned by our framework in …
View article
Photo-Realistic Facial Details Synthesis From Single Image Open
We present a single-image 3D face synthesis technique that can handle challenging facial expressions while recovering fine geometric details. Our technique employs expression analysis for proxy face geometry generation and combines supervi…
View article
Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines Open
We present a practical and robust deep learning solution for capturing and rendering novel views of complex real world scenes for virtual exploration. Previous approaches either require intractably dense view sampling or provide little to …
View article
Deep view synthesis from sparse photometric images Open
The goal of light transport acquisition is to take images from a sparse set of lighting and viewing directions, and combine them to enable arbitrary relighting with changing view. While relighting from sparse images has received significan…
View article
X-Fields Open
We suggest to represent an X-Field ---a set of 2D images taken across different view, time or illumination conditions, i.e., video, lightfield, reflectance fields or combinations thereof---by learning a neural network (NN) to map their vie…
View article
Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations Open
Unsupervised learning with generative models has the potential of discovering rich representations of 3D scenes. While geometric deep learning has explored 3D-structure-aware representations of scene geometry, these models typically requir…
View article
DeepStereo: Learning to Predict New Views from the World's Imagery Open
Deep networks have recently enjoyed enormous success when applied to recognition and classification problems in computer vision, but their use in graphics problems has been limited. In this work, we present a novel deep architecture that p…
View article
An Implicit Parametric Morphable Dental Model Open
3D Morphable models of the human body capture variations among subjects and are useful in reconstruction and editing applications. Current dental models use an explicit mesh scene representation and model only the teeth, ignoring the gum. …
View article
ShadowGAN: Shadow synthesis for virtual objects with conditional adversarial networks Open
\n We introduce ShadowGAN, a generative adversarial network (GAN) for synthesizing shadows for virtual objects inserted in images. Given a target image containing several existing objects with shadows, and an input source object with a spe…
View article
Intuitive and efficient camera control with the toric space Open
A large range of computer graphics applications such as data visualization or virtual movie production require users to position and move viewpoints in 3D scenes to effectively convey visual information or tell stories. The desired viewpoi…
View article
Light field compression using depth image based view synthesis Open
International audience
View article
Geometry-Aware Deep Network for Single-Image Novel View Synthesis Open
This paper tackles the problem of novel view synthesis from a single image. In particular, we target real-world scenes with rich geometric structure, a challenging task due to the large appearance variations of such scenes and the lack of …