Benjamin Eysenbach
YOU?
Author Swipe
View article: BuilderBench -- A benchmark for generalist agents
BuilderBench -- A benchmark for generalist agents Open
Today's AI models learn primarily through mimicry and sharpening, so it is not surprising that they struggle to solve problems beyond the limits set by existing data. To solve novel problems, agents should acquire skills for exploring and …
View article: Self-Supervised Goal-Reaching Results in Multi-Agent Cooperation and Exploration
Self-Supervised Goal-Reaching Results in Multi-Agent Cooperation and Exploration Open
For groups of autonomous agents to achieve a particular goal, they must engage in coordination and long-horizon reasoning. However, designing reward functions to elicit such behavior is challenging. In this paper, we study how self-supervi…
View article: Contrastive Representations for Temporal Reasoning
Contrastive Representations for Temporal Reasoning Open
In classical AI, perception relies on learning state-based representations, while planning, which can be thought of as temporal reasoning over action sequences, is typically achieved through search. We study whether such reasoning can inst…
View article: Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning
Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning Open
Self-supervised feature learning and pretraining methods in reinforcement learning (RL) often rely on information-theoretic principles, termed mutual information skill learning (MISL). These methods aim to learn a representation of the env…
View article: Intention-Conditioned Flow Occupancy Models
Intention-Conditioned Flow Occupancy Models Open
Large-scale pre-training has fundamentally changed how machine learning research is done today: large foundation models are trained once, and then can be used by anyone in the community (including those without data or compute resources to…
View article: Horizon Reduction Makes RL Scalable
Horizon Reduction Makes RL Scalable Open
In this work, we study the scalability of offline reinforcement learning (RL) algorithms. In principle, a truly scalable offline RL algorithm should be able to solve any given problem, regardless of its complexity, given sufficient data, c…
View article: Normalizing Flows are Capable Models for RL
Normalizing Flows are Capable Models for RL Open
Modern reinforcement learning (RL) algorithms have found success by using powerful probabilistic models, such as transformers, energy-based models, and diffusion/flow-based models. To this end, RL researchers often choose to pay the price …
View article: The "Law" of the Unconscious Contrastive Learner: Probabilistic Alignment of Unpaired Modalities
The "Law" of the Unconscious Contrastive Learner: Probabilistic Alignment of Unpaired Modalities Open
While internet-scale data often comes in pairs (e.g., audio/image, image/text), we often want to perform inferences over modalities unseen together in the training data (e.g., audio/text). Empirically, this can often be addressed by learni…
View article: Horizon Generalization in Reinforcement Learning
Horizon Generalization in Reinforcement Learning Open
We study goal-conditioned RL through the lens of generalization, but not in the traditional sense of random augmentations and domain randomization. Rather, we aim to learn goal-directed policies that generalize with respect to the horizon:…
View article: Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning
Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning Open
Self-supervised learning has the potential of lifting several of the key challenges in reinforcement learning today, such as exploration, representation learning, and reward design. Recent work (METRA) has effectively argued that moving aw…
View article: Learning to Assist Humans without Inferring Rewards
Learning to Assist Humans without Inferring Rewards Open
Assistive agents should make humans' lives easier. Classically, such assistance is studied through the lens of inverse reinforcement learning, where an assistive agent (e.g., a chatbot, a robot) infers a human's intention and then selects …
View article: GHIL-Glue: Hierarchical Control with Filtered Subgoal Images
GHIL-Glue: Hierarchical Control with Filtered Subgoal Images Open
Image and video generative models that are pre-trained on Internet-scale data can greatly increase the generalization capacity of robot learning systems. These models can function as high-level planners, generating intermediate subgoals fo…
View article: OGBench: Benchmarking Offline Goal-Conditioned RL
OGBench: Benchmarking Offline Goal-Conditioned RL Open
Offline goal-conditioned reinforcement learning (GCRL) is a major problem in reinforcement learning (RL) because it provides a simple, unsupervised, and domain-agnostic way to acquire diverse behaviors and representations from unlabeled da…
View article: Accelerating Goal-Conditioned RL Algorithms and Research
Accelerating Goal-Conditioned RL Algorithms and Research Open
Self-supervision has the potential to transform reinforcement learning (RL), paralleling the breakthroughs it has enabled in other areas of machine learning. While self-supervised learning in other domains aims to find patterns in a fixed …
View article: A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals
A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals Open
In this paper, we present empirical evidence of skills and directed exploration emerging from a simple RL algorithm long before any successful trials are observed. For example, in a manipulation task, the agent is given a single observatio…
View article: Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making
Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making Open
Temporal distances lie at the heart of many algorithms for planning, control, and reinforcement learning that involve reaching goals, allowing one to estimate the transit time between two states. However, prior attempts to define such temp…
View article: Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference
Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference Open
Given time series data, how can we answer questions like "what will happen in the future?" and "how did we get here?" These sorts of probabilistic inference questions are challenging when observations are high-dimensional. In this paper, w…
View article: Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of View
Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of View Open
Some reinforcement learning (RL) algorithms can stitch pieces of experience to solve a task never seen before during training. This oft-sought property is one of the few ways in which RL methods based on dynamic-programming differ from RL …
View article: Bridging State and History Representations: Understanding Self-Predictive RL
Bridging State and History Representations: Understanding Self-Predictive RL Open
Representations are at the core of all deep reinforcement learning (RL) methods for both Markov decision processes (MDPs) and partially observable Markov decision processes (POMDPs). Many representation learning methods and theoretical fra…
View article: Learning to Reach Goals via Iterated Supervised Learning
Learning to Reach Goals via Iterated Supervised Learning Open
Current reinforcement learning (RL) algorithms can be brittle and difficult to use, especially when learning goal-reaching behaviors from sparse rewards. Although supervised imitation learning provides a simple and stable alternative, it r…
View article: Contrastive Difference Predictive Coding
Contrastive Difference Predictive Coding Open
Predicting and reasoning about the future lie at the heart of many time-series questions. For example, goal-conditioned reinforcement learning can be viewed as learning representations to predict which states are likely to be visited in th…
View article: A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning
A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning Open
As with any machine learning problem with limited data, effective offline RL algorithms require careful regularization to avoid overfitting. One-step methods perform regularization by doing just a single step of policy improvement, while c…
View article: Contrastive Example-Based Control
Contrastive Example-Based Control Open
While many real-world problems that might benefit from reinforcement learning, these problems rarely fit into the MDP mold: interacting with the environment is often expensive and specifying reward functions is challenging. Motivated by th…
View article: HIQL: Offline Goal-Conditioned RL with Latent States as Actions
HIQL: Offline Goal-Conditioned RL with Latent States as Actions Open
Unsupervised pre-training has recently become the bedrock for computer vision and natural language processing. In reinforcement learning (RL), goal-conditioned RL can potentially provide an analogous self-supervised approach for making use…
View article: When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment
When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment Open
Reinforcement learning (RL) algorithms face two distinct challenges: learning effective representations of past and present observations, and determining how actions influence future returns. Both challenges involve modeling long-term depe…
View article: Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data
Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data Open
Robotic systems that rely primarily on self-supervised learning have the potential to decrease the amount of human annotation and engineering effort required to learn control strategies. In the same way that prior robotic systems have leve…
View article: Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts
Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts Open
Training machine learning models robust to distribution shifts is critical for real-world applications. Some robust training algorithms (e.g., Group DRO) specialize to group shifts and require group information on all training points. Othe…
View article: Learning Options via Compression
Learning Options via Compression Open
Identifying statistical regularities in solutions to some tasks in multi-task reinforcement learning can accelerate the learning of new tasks. Skill learning offers one way of identifying these regularities by decomposing pre-collected exp…
View article: Contrastive Value Learning: Implicit Models for Simple Offline RL
Contrastive Value Learning: Implicit Models for Simple Offline RL Open
Model-based reinforcement learning (RL) methods are appealing in the offline setting because they allow an agent to reason about the consequences of actions without interacting with the environment. Prior methods learn a 1-step dynamics mo…
View article: Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective
Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective Open
While reinforcement learning (RL) methods that learn an internal model of the environment have the potential to be more sample efficient than their model-free counterparts, learning to model raw observations from high dimensional sensors c…