Liam Paull
YOU?
Author Swipe
View article: Agentic Scene Policies: Unifying Space, Semantics, and Affordances for Robot Action
Agentic Scene Policies: Unifying Space, Semantics, and Affordances for Robot Action Open
Executing open-ended natural language queries is a core problem in robotics. While recent advances in imitation learning and vision-language-actions models (VLAs) have enabled promising end-to-end policies, these models struggle when faced…
View article: Dynamic Objects Relocalization in Changing Environments with Flow Matching
Dynamic Objects Relocalization in Changing Environments with Flow Matching Open
Task and motion planning are long-standing challenges in robotics, especially when robots have to deal with dynamic environments exhibiting long-term dynamics, such as households or warehouses. In these environments, long-term dynamics mos…
View article: Perpetua: Multi-Hypothesis Persistence Modeling for Semi-Static Environments
Perpetua: Multi-Hypothesis Persistence Modeling for Semi-Static Environments Open
Many robotic systems require extended deployments in complex, dynamic environments. In such deployments, parts of the environment may change between subsequent robot observations. Most robotic mapping or environment modeling algorithms are…
View article: Poutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Driving
Poutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Driving Open
Maintaining good driving behavior in out-of-distribution scenarios remains a critical challenge in autonomous driving. A promising direction is to leverage the generalist knowledge and reasoning capabilities of large-language models by tre…
View article: Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes
Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes Open
Video diffusion techniques have advanced significantly in recent years; however, they struggle to generate realistic imagery of car crashes due to the scarcity of accident events in most driving datasets. Improving traffic safety requires …
View article: Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments
Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments Open
We introduce Scenario Dreamer, a fully data-driven generative simulator for autonomous vehicle planning that generates both the initial traffic scene - comprising a lane graph and agent bounding boxes - and closed-loop agent behaviours. Ex…
View article: OpenLex3D: A Tiered Evaluation Benchmark for Open-Vocabulary 3D Scene Representations
OpenLex3D: A Tiered Evaluation Benchmark for Open-Vocabulary 3D Scene Representations Open
3D scene understanding has been transformed by open-vocabulary language models that enable interaction via natural language. However, at present the evaluation of these representations is limited to datasets with closed-set semantics that …
View article: Safety Representations for Safer Policy Learning
Safety Representations for Safer Policy Learning Open
Reinforcement learning algorithms typically necessitate extensive exploration of the state space to find optimal policies. However, in safety-critical applications, the risks associated with such exploration can lead to catastrophic conseq…
View article: The Harmonic Exponential Filter for Nonparametric Estimation on Motion Groups
The Harmonic Exponential Filter for Nonparametric Estimation on Motion Groups Open
Bayesian estimation is a vital tool in robotics as it allows systems to\nupdate the robot state belief using incomplete information from noisy sensors.\nTo render the state estimation problem tractable, many systems assume that the\nmotion…
View article: The Bare Necessities: Designing Simple, Effective Open-Vocabulary Scene Graphs
The Bare Necessities: Designing Simple, Effective Open-Vocabulary Scene Graphs Open
3D open-vocabulary scene graph methods are a promising map representation for embodied agents, however many current approaches are computationally expensive. In this paper, we reexamine the critical design choices established in previous w…
View article: BACS: Background Aware Continual Semantic Segmentation
BACS: Background Aware Continual Semantic Segmentation Open
Semantic segmentation plays a crucial role in enabling comprehensive scene understanding for robotic systems. However, generating annotations is challenging, requiring labels for every pixel in an image. In scenarios like autonomous drivin…
View article: BACS: Background Aware Continual Semantic Segmentation
BACS: Background Aware Continual Semantic Segmentation Open
Semantic segmentation plays a crucial role in enabling comprehensive scene understanding for robotic systems. However, generating annotations is challenging, requiring labels for every pixel in an image. In scenarios like autonomous drivin…
View article: Rethinking Teacher-Student Curriculum Learning through the Cooperative Mechanics of Experience
Rethinking Teacher-Student Curriculum Learning through the Cooperative Mechanics of Experience Open
Teacher-Student Curriculum Learning (TSCL) is a curriculum learning framework that draws inspiration from human cultural transmission and learning. It involves a teacher algorithm shaping the learning process of a learner algorithm by expo…
View article: CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning Open
Evaluating autonomous vehicle stacks (AVs) in simulation typically involves replaying driving logs from real-world recorded traffic. However, agents replayed from offline data are not reactive and hard to intuitively control. Existing appr…
View article: A Survey on Small-Scale Testbeds for Connected and Automated Vehicles and Robot Swarms
A Survey on Small-Scale Testbeds for Connected and Automated Vehicles and Robot Swarms Open
Connected and automated vehicles and robot swarms hold transformative potential for enhancing safety, efficiency, and sustainability in the transportation and manufacturing sectors. Extensive testing and validation of these technologies is…
View article: GROOD: GRadient-Aware Out-of-Distribution Detection
GROOD: GRadient-Aware Out-of-Distribution Detection Open
Out-of-distribution (OOD) detection is crucial for ensuring the reliability of deep learning models in real-world applications. Existing methods typically focus on feature representations or output-space analysis, often assuming a distribu…
View article: Ghost on the Shell: An Expressive Representation of General 3D Shapes
Ghost on the Shell: An Expressive Representation of General 3D Shapes Open
The creation of photorealistic virtual worlds requires the accurate modeling of 3D surface geometry for a wide range of objects. For this, meshes are appealing since they 1) enable fast physics-based rendering with realistic material and l…
View article: ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning Open
For robots to perform a wide variety of tasks, they require a 3D representation of the world that is semantically rich, yet compact and efficient for task-driven perception and planning. Recent approaches have attempted to leverage feature…
View article: ConceptFusion: Open-set multimodal 3D mapping
ConceptFusion: Open-set multimodal 3D mapping Open
modalities such as natural language, images, and audio.We demonstrate that pixel-aligned open-set features can be fused into 3D maps via traditional SLAM and multi-view fusion approaches.This enables effective zero-shot spatial reasoning, …
View article: Estimating Regression Predictive Distributions with Sample Networks
Estimating Regression Predictive Distributions with Sample Networks Open
Estimating the uncertainty in deep neural network predictions is crucial for many real-world applications. A common approach to model uncertainty is to choose a parametric distribution and fit the data to it using maximum likelihood estima…
View article: MeshDiffusion: Score-based Generative 3D Mesh Modeling
MeshDiffusion: Score-based Generative 3D Mesh Modeling Open
We consider the task of generating realistic 3D shapes, which is useful for a variety of applications such as automatic scene generation and physical simulation. Compared to other 3D representations like voxels and point clouds, meshes are…
View article: One-4-All: Neural Potential Fields for Embodied Navigation
One-4-All: Neural Potential Fields for Embodied Navigation Open
A fundamental task in robotics is to navigate between two locations. In particular, real-world navigation can require long-horizon planning using high-dimensional RGB images, which poses a substantial challenge for end-to-end learning-base…
View article: ConceptFusion: Open-set Multimodal 3D Mapping
ConceptFusion: Open-set Multimodal 3D Mapping Open
Building 3D maps of the environment is central to robot navigation, planning, and interaction with objects in a scene. Most existing approaches that integrate semantic concepts with 3D maps largely remain confined to the closed-set setting…
View article: Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Loss
Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Loss Open
An effective framework for learning 3D representations for perception tasks is distilling rich self-supervised image features via contrastive learning. However, image-to point representation learning for autonomous driving datasets faces t…
View article: Multi-Agent Reinforcement Learning for Fast-Timescale Demand Response of Residential Loads
Multi-Agent Reinforcement Learning for Fast-Timescale Demand Response of Residential Loads Open
To integrate high amounts of renewable energy resources, electrical power grids must be able to cope with high amplitude, fast timescale variations in power generation. Frequency regulation through demand response has the potential to coor…
View article: Estimating Regression Predictive Distributions with Sample Networks
Estimating Regression Predictive Distributions with Sample Networks Open
Estimating the uncertainty in deep neural network predictions is crucial for many real-world applications. A common approach to model uncertainty is to choose a parametric distribution and fit the data to it using maximum likelihood estima…
View article: NeurIPS 2022 Competition: Driving SMARTS
NeurIPS 2022 Competition: Driving SMARTS Open
Driving SMARTS is a regular competition designed to tackle problems caused by the distribution shift in dynamic interaction contexts that are prevalent in real-world autonomous driving (AD). The proposed competition supports methodological…
View article: Robust and Controllable Object-Centric Learning through Energy-based Models
Robust and Controllable Object-Centric Learning through Energy-based Models Open
Humans are remarkably good at understanding and reasoning about complex visual scenes. The capability to decompose low-level observations into discrete objects allows us to build a grounded abstract representation and identify the composit…