Karen Ullrich
YOU?
Author Swipe
View article: OpenApps: Simulating Environment Variations to Measure UI-Agent Reliability
OpenApps: Simulating Environment Variations to Measure UI-Agent Reliability Open
Reliability is key to realizing the promise of autonomous UI-Agents, multimodal agents that directly interact with apps in the same manner as humans, as users must be able to trust an agent to complete a given task. Current evaluations rel…
View article: A Single Character can Make or Break Your LLM Evals
A Single Character can Make or Break Your LLM Evals Open
Common Large Language model (LLM) evaluations rely on demonstration examples to steer models' responses to the desired style. While the number of examples used has been studied and standardized, the choice of how to format examples is less…
View article: LLM Output Homogenization is Task Dependent
LLM Output Homogenization is Task Dependent Open
A large language model can be less helpful if it exhibits output response homogenization. But whether two responses are considered homogeneous, and whether such homogenization is problematic, both depend on the task category. For instance,…
View article: DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models
DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models Open
Recent advances in text-to-image (T2I) models have achieved impressive quality and consistency. However, this has come at the cost of representation diversity. While automatic evaluation methods exist for benchmarking model diversity, they…
View article: Lossless Compression of Vector IDs for Approximate Nearest Neighbor Search
Lossless Compression of Vector IDs for Approximate Nearest Neighbor Search Open
Approximate nearest neighbor search for vectors relies on indexes that are most often accessed from RAM. Therefore, storage is the factor limiting the size of the database that can be served from a machine. Lossy vector compression, i.e., …
View article: EvalGIM: A Library for Evaluating Generative Image Models
EvalGIM: A Library for Evaluating Generative Image Models Open
As the use of text-to-image generative models increases, so does the adoption of automatic benchmarking methods used in their evaluation. However, while metrics and datasets abound, there are few unified benchmarking libraries that provide…
View article: Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles
Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles Open
Tokenization is associated with many poorly understood shortcomings in language models (LMs), yet remains an important component for long sequence scaling purposes. This work studies how tokenization impacts model performance by analyzing …
View article: Mission Impossible: A Statistical Perspective on Jailbreaking LLMs
Mission Impossible: A Statistical Perspective on Jailbreaking LLMs Open
Large language models (LLMs) are trained on a deluge of text data with limited quality control. As a result, LLMs can exhibit unintended or even harmful behaviours, such as leaking information, fake news or hate speech. Countermeasures, co…
View article: End-To-End Causal Effect Estimation from Unstructured Natural Language Data
End-To-End Causal Effect Estimation from Unstructured Natural Language Data Open
Knowing the effect of an intervention is critical for human decision-making, but current approaches for causal effect estimation rely on manual data collection and structuring, regardless of the causal assumptions. This increases both the …
View article: Understanding and Mitigating Tokenization Bias in Language Models
Understanding and Mitigating Tokenization Bias in Language Models Open
State-of-the-art language models are autoregressive and operate on subword units known as tokens. Specifically, one must encode the conditioning string into a list of tokens before passing to the language models for next-token prediction. …
View article: An Introduction to Vision-Language Modeling
An Introduction to Vision-Language Modeling Open
Following the recent popularity of Large Language Models (LLMs), several attempts have been made to extend them to the visual domain. From having a visual assistant that could guide us through unfamiliar environments to generative models t…
View article: On the Challenges and Opportunities in Generative AI
On the Challenges and Opportunities in Generative AI Open
The field of deep generative modeling has grown rapidly in the last few years. With the availability of massive amounts of training data coupled with advances in scalable unsupervised learning paradigms, recent large-scale generative model…
View article: Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models
Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models Open
Lossy image compression aims to represent images in as few bits as possible while maintaining fidelity to the original. Theoretical results indicate that optimizing distortion metrics such as PSNR or MS-SSIM necessarily leads to a discrepa…
View article: Latent Discretization for Continuous-time Sequence Compression
Latent Discretization for Continuous-time Sequence Compression Open
Neural compression offers a domain-agnostic approach to creating codecs for lossy or lossless compression via deep generative models. For sequence compression, however, most deep sequence models have costs that scale with the sequence leng…
View article: Image Compression with Product Quantized Masked Image Modeling
Image Compression with Product Quantized Masked Image Modeling Open
Recent neural compression methods have been based on the popular hyperprior framework. It relies on Scalar Quantization and offers a very strong compression performance. This contrasts from recent advances in image generation and represent…
View article: An optimal control perspective on diffusion-based generative modeling
An optimal control perspective on diffusion-based generative modeling Open
We establish a connection between stochastic optimal control and generative models based on stochastic differential equations (SDEs), such as recently developed diffusion probabilistic models. In particular, we derive a Hamilton-Jacobi-Bel…
View article: Compressing Multisets with Large Alphabets using Bits-Back Coding
Compressing Multisets with Large Alphabets using Bits-Back Coding Open
Current methods which compress multisets at an optimal rate have computational complexity that scales linearly with alphabet size, making them too slow to be practical in many real-world settings. We show how to convert a compression algor…
View article: Lossy Compression for Lossless Prediction
Lossy Compression for Lossless Prediction Open
Most data is automatically collected and only ever "seen" by algorithms. Yet, data compressors preserve perceptual fidelity rather than just the information needed by algorithms performing downstream tasks. In this paper, we characterize t…
View article: Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding
Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding Open
Latent variable models have been successfully applied in lossless compression with the bits-back coding algorithm. However, bits-back suffers from an increase in the bitrate equal to the KL divergence between the approximate posterior and …
View article: Neural Communication Systems with Bandwidth-limited Channel
Neural Communication Systems with Bandwidth-limited Channel Open
Reliably transmitting messages despite information loss due to a noisy channel is a core problem of information theory. One of the most important aspects of real world communication, e.g. via wifi, is that it may happen at varying levels o…
View article: A coding perspective on deep latent variable models
A coding perspective on deep latent variable models Open
In my thesis "A Coding Perspective on Deep Latent Variable Models", we discuss how statistical inference in Deep Latent Variable Models (DLVMs) relates to coding. In particular, we examine the minimum deception length (MDL) principle as a …
View article: Differentiable probabilistic models of scientific imaging with the Fourier slice theorem
Differentiable probabilistic models of scientific imaging with the Fourier slice theorem Open
Scientific imaging techniques such as optical and electron microscopy and computed tomography (CT) scanning are used to study the 3D structure of an object through 2D observations. These observations are related to the original 3D object t…
View article: Improved Bayesian Compression
Improved Bayesian Compression Open
Compression of Neural Networks (NN) has become a highly studied topic in recent years. The main reason for this is the demand for industrial scale usage of NNs such as deploying them on mobile devices, storing them efficiently, transmittin…
View article: Optical Music Recognition with Convolutional Sequence-to-Sequence Models.
Optical Music Recognition with Convolutional Sequence-to-Sequence Models. Open
Optical Music Recognition (OMR) is an important technology within Music Information Retrieval. Deep learning models show promising results on OMR tasks, but symbol-level annotated data sets of sufficient size to train such models are not a…
View article: Optical Music Recognition with Convolutional Sequence-to-Sequence Models
Optical Music Recognition with Convolutional Sequence-to-Sequence Models Open
Optical Music Recognition (OMR) is an important technology within Music Information Retrieval. Deep learning models show promising results on OMR tasks, but symbol-level annotated data sets of sufficient size to train such models are not a…
View article: Bayesian Compression for Deep Learning
Bayesian Compression for Deep Learning Open
Compression and computational efficiency in deep learning have become a problem of great significance. In this work, we argue that the most principled and effective way to attack this problem is by adopting a Bayesian point of view, where …
View article: Soft Weight-Sharing for Neural Network Compression
Soft Weight-Sharing for Neural Network Compression Open
The success of deep learning in numerous application domains created the de- sire to run and train them on mobile devices. This however, conflicts with their computationally, memory and energy intense nature, leading to a growing interest …
View article: Bayesian Compression for Deep Learning
Bayesian Compression for Deep Learning Open
Compression and computational efficiency in deep learning have become a problem of great significance. In this work, we argue that the most principled and effective way to attack this problem is by adopting a Bayesian point of view, where …