Robin Rombach
YOU?
Author Swipe
View article: FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space
FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space Open
We present evaluation results for FLUX.1 Kontext, a generative flow matching model that unifies image generation and editing. The model generates novel output views by incorporating semantic context from text and image inputs. Using a simp…
View article: test1
test1 Open
We present Stable Video Diffusion - a latent video diffusion model for high-resolution, state-of-the-art text-to-video and image-to-video generation. Recently, latent diffusion models trained for 2D image synthesis have been turned into ge…
View article: Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation
Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation Open
Diffusion models are the main driver of progress in image and video synthesis, but suffer from slow inference speed. Distillation methods, like the recently introduced adversarial diffusion distillation (ADD) aim to shift the model from ma…
View article: SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion Open
We present Stable Video 3D (SV3D) -- a latent video diffusion model for high-resolution, image-to-multi-view generation of orbital videos around a 3D object. Recent work on 3D generation propose techniques to adapt 2D generative models for…
View article: Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Scaling Rectified Flow Transformers for High-Resolution Image Synthesis Open
Diffusion models create data from noise by inverting the forward paths of data towards noise and have emerged as a powerful generative modeling technique for high-dimensional, perceptual data such as images and videos. Rectified flow is a …
View article: aMUSEd: An Open MUSE Reproduction
aMUSEd: An Open MUSE Reproduction Open
We present aMUSEd, an open-source, lightweight masked image model (MIM) for text-to-image generation based on MUSE. With 10 percent of MUSE's parameters, aMUSEd is focused on fast image generation. We believe MIM is under-explored compared…
View article: DiffusionSat: A Generative Foundation Model for Satellite Imagery
DiffusionSat: A Generative Foundation Model for Satellite Imagery Open
Diffusion models have achieved state-of-the-art results on many modalities including images, speech, and video. However, existing models are not tailored to support remote sensing data, which is widely used in important applications includ…
View article: Adversarial Diffusion Distillation
Adversarial Diffusion Distillation Open
We introduce Adversarial Diffusion Distillation (ADD), a novel training approach that efficiently samples large-scale foundational image diffusion models in just 1-4 steps while maintaining high image quality. We use score distillation to …
View article: SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis Open
We present SDXL, a latent diffusion model for text-to-image synthesis. Compared to previous versions of Stable Diffusion, SDXL leverages a three times larger UNet backbone: The increase of model parameters is mainly due to more attention b…
View article: NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models
NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models Open
Automatically generating high-quality real world 3D scenes is of enormous interest for applications such as virtual reality and robotics simulation. Towards this goal, we introduce NeuralField-LDM, a generative model capable of synthesizin…
View article: Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models Open
Latent Diffusion Models (LDMs) enable high-quality image synthesis while avoiding excessive compute demands by training a diffusion model in a compressed lower-dimensional latent space. Here, we apply the LDM paradigm to high-resolution vi…
View article: Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models
Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models Open
Novel architectures have recently improved generative image synthesis leading to excellent visual quality in various tasks. Of particular note is the field of ``AI-Art'', which has seen unprecedented growth with the emergence of powerful m…
View article: Semi-Parametric Neural Image Synthesis
Semi-Parametric Neural Image Synthesis Open
Novel architectures have recently improved generative image synthesis leading to excellent visual quality in various tasks. Much of this success is due to the scalability of these architectures and hence caused by a dramatic increase in mo…
View article: Invertible Neural Networks for Understanding Semantics of Invariances of CNN Representations
Invertible Neural Networks for Understanding Semantics of Invariances of CNN Representations Open
To tackle increasingly complex tasks, it has become an essential ability of neural networks to learn abstract representations. These task-specific representations and, particularly, the invariances they capture turn neural networks into bl…
View article: High-Resolution Image Synthesis with Latent Diffusion Models
High-Resolution Image Synthesis with Latent Diffusion Models Open
By decomposing the image formation process into a sequential application of denoising autoencoders, diffusion models (DMs) achieve state-of-the-art synthesis results on image data and beyond. Additionally, their formulation allows for a gu…
View article: Geometry-Free View Synthesis: Transformers and no 3D Priors
Geometry-Free View Synthesis: Transformers and no 3D Priors Open
Is a geometric model required to synthesize novel views from a single image? Being bound to local convolutions, CNNs need explicit 3D biases to model geometric transformations. In contrast, we demonstrate that a transformer-based model can…
View article: ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis
ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis Open
Autoregressive models and their sequential factorization of the data likelihood have recently demonstrated great potential for image representation and synthesis. Nevertheless, they incorporate image context in a linear 1D order by attendi…
View article: Taming Transformers for High-Resolution Image Synthesis
Taming Transformers for High-Resolution Image Synthesis Open
Designed to learn long-range interactions on sequential data, transformers continue to show state-of-the-art results on a wide variety of tasks. In contrast to CNNs, they contain no inductive bias that prioritizes local interactions. This …
View article: Stochastic Image-to-Video Synthesis using cINNs
Stochastic Image-to-Video Synthesis using cINNs Open
Video understanding calls for a model to learn the characteristic interplay between static scene content and its dynamics: Given an image, the model must be able to predict a future progression of the portrayed scene and, conversely, a vid…
View article: High-Resolution Complex Scene Synthesis with Transformers
High-Resolution Complex Scene Synthesis with Transformers Open
The use of coarse-grained layouts for controllable synthesis of complex scene images via deep generative models has recently gained popularity. However, results of current approaches still fall short of their promise of high-resolution syn…
View article: A Note on Data Biases in Generative Models
A Note on Data Biases in Generative Models Open
It is tempting to think that machines are less prone to unfairness and prejudice. However, machine learning approaches compute their outputs based on data. While biases can enter at any stage of the development pipeline, models are particu…
View article: Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs
Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs Open
To tackle increasingly complex tasks, it has become an essential ability of neural networks to learn abstract representations. These task-specific representations and, particularly, the invariances they capture turn neural networks into bl…
View article: Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs
Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs Open
To tackle increasingly complex tasks, it has become an essential ability of neural networks to learn abstract representations. These task-specific representations and, particularly, the invariances they capture turn neural networks into bl…
View article: A Disentangling Invertible Interpretation Network for Explaining Latent Representations
A Disentangling Invertible Interpretation Network for Explaining Latent Representations Open
Neural networks have greatly boosted performance in computer vision by learning powerful representations of input data. The drawback of end-to-end training for maximal overall performance are black-box models whose hidden representations a…
View article: Network-to-Network Translation with Conditional Invertible Neural Networks
Network-to-Network Translation with Conditional Invertible Neural Networks Open
Given the ever-increasing computational costs of modern machine learning models, we need to find new ways to reuse such expert models and thus tap into the resources that have been invested in their creation. Recent work suggests that the …
View article: Network Fusion for Content Creation with Conditional INNs.
Network Fusion for Content Creation with Conditional INNs. Open
Artificial Intelligence for Content Creation has the potential to reduce the amount of manual content creation work significantly. While automation of laborious work is welcome, it is only useful if it allows users to control aspects of th…
View article: Network-to-Network Translation with Conditional Invertible Neural\n Networks
Network-to-Network Translation with Conditional Invertible Neural\n Networks Open
Given the ever-increasing computational costs of modern machine learning\nmodels, we need to find new ways to reuse such expert models and thus tap into\nthe resources that have been invested in their creation. Recent work suggests\nthat t…