Florian Golemo
YOU?
Author Swipe
View article: Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes
Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes Open
Video diffusion techniques have advanced significantly in recent years; however, they struggle to generate realistic imagery of car crashes due to the scarcity of accident events in most driving datasets. Improving traffic safety requires …
View article: DStruct2Design: Data and Benchmarks for Data Structure Driven Generative Floor Plan Design
DStruct2Design: Data and Benchmarks for Data Structure Driven Generative Floor Plan Design Open
Text conditioned generative models for images have yielded impressive results. Text conditioned floorplan generation as a special type of raster image generation task also received particular attention. However there are many use cases in …
View article: CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning Open
Evaluating autonomous vehicle stacks (AVs) in simulation typically involves replaying driving logs from real-world recorded traffic. However, agents replayed from offline data are not reactive and hard to intuitively control. Existing appr…
View article: GrowSpace: A reinforcement learning environment for plant architecture
GrowSpace: A reinforcement learning environment for plant architecture Open
Plants adapt over time to their surrounding conditions. We argue that no current plant simulator addresses plant responses to an environmental stimulus over time through gamification and sequential decision making. We propose the GrowSpace…
View article: Visual Question Answering From Another Perspective: CLEVR Mental Rotation Tests
Visual Question Answering From Another Perspective: CLEVR Mental Rotation Tests Open
Different types of mental rotation tests have been used extensively in psychology to understand human visual reasoning and perception. Understanding what an object or visual scene would look like from another viewpoint is a challenging pro…
View article: The Sandbox Environment for Generalizable Agent Research (SEGAR)
The Sandbox Environment for Generalizable Agent Research (SEGAR) Open
A broad challenge of research on generalization for sequential decision-making tasks in interactive environments is designing benchmarks that clearly landmark progress. While there has been notable headway, current benchmarks either do not…
View article: Kubric: A scalable dataset generator
Kubric: A scalable dataset generator Open
Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and training details. But collecting, processing and annotating real d…
View article: GrowSpace: Learning How to Shape Plants
GrowSpace: Learning How to Shape Plants Open
Plants are dynamic systems that are integral to our existence and survival. Plants face environment changes and adapt over time to their surrounding conditions. We argue that plant responses to an environmental stimulus are a good example …
View article: Sequoia: A Software Framework to Unify Continual Learning Research
Sequoia: A Software Framework to Unify Continual Learning Research Open
The field of Continual Learning (CL) seeks to develop algorithms that accumulate knowledge and skills over time through interaction with non-stationary environments. In practice, a plethora of evaluation procedures (settings) and algorithm…
View article: gradSim: Differentiable simulation for system identification and visuomotor control
gradSim: Differentiable simulation for system identification and visuomotor control Open
We consider the problem of estimating an object's physical properties such as mass, friction, and elasticity directly from video sequences. Such a system identification problem is fundamentally ill-posed due to the loss of information duri…
View article: gradSim: Differentiable simulation for system identification and visuomotor control
gradSim: Differentiable simulation for system identification and visuomotor control Open
We consider the problem of estimating an object's physical properties such as mass, friction, and elasticity directly from video sequences. Such a system identification problem is fundamentally ill-posed due to the loss of information duri…
View article: Touch-based Curiosity for Sparse-Reward Tasks
Touch-based Curiosity for Sparse-Reward Tasks Open
Robots in many real-world settings have access to force/torque sensors in their gripper and tactile sensing is often necessary in tasks that involve contact-rich motion. In this work, we leverage surprise from mismatches in touch feedback …
View article: Sim2Real in Robotics and Automation: Applications and Challenges
Sim2Real in Robotics and Automation: Applications and Challenges Open
To Perform reliably and consistently over sustained periods of time, large-scale automation critically relies on computer simulation. Simulation allows us and supervisory AI to effectively design, validate, and continuously improve complex…
View article: Autobots: Latent Variable Sequential Set Transformers
Autobots: Latent Variable Sequential Set Transformers Open
Robust multi-agent trajectory prediction is essential for the safe control of robots and vehicles that interact with humans. Many existing methods treat social and temporal information separately and therefore fall short of modelling the j…
View article: Latent Variable Nested Set Transformers & AutoBots.
Latent Variable Nested Set Transformers & AutoBots. Open
Humans have the innate ability to attend to the most relevant actors in their
vicinity and can forecast how they may behave in the future. This ability will
be crucial for the deployment of safety-critical agents such as robots or
vehicles…
View article: Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction
Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction Open
Robust multi-agent trajectory prediction is essential for the safe control of robotic systems. A major challenge is to efficiently learn a representation that approximates the true joint distribution of contextual, social, and temporal inf…
View article: Perspectives on Sim2Real Transfer for Robotics: A Summary of the R:SS 2020 Workshop
Perspectives on Sim2Real Transfer for Robotics: A Summary of the R:SS 2020 Workshop Open
This report presents the debates, posters, and discussions of the Sim2Real workshop held in conjunction with the 2020 edition of the "Robotics: Science and System" conference. Twelve leaders of the field took competing debate positions on …
View article: Unsupervised Learning of Dense Visual Representations
Unsupervised Learning of Dense Visual Representations Open
Contrastive self-supervised learning has emerged as a promising approach to unsupervised visual representation learning. In general, these methods learn global (image-level) representations that are invariant to different views (i.e., comp…
View article: Generating Automatic Curricula via Self-Supervised Active Domain Randomization
Generating Automatic Curricula via Self-Supervised Active Domain Randomization Open
Goal-directed Reinforcement Learning (RL) traditionally considers an agent interacting with an environment, prescribing a real-valued reward to an agent proportional to the completion of some goal. Goal-directed RL has seen large gains in …
View article: Robo-PlaNet: Learning to Poke in a Day
Robo-PlaNet: Learning to Poke in a Day Open
Recently, the Deep Planning Network (PlaNet) approach was introduced as a model-based reinforcement learning method that learns environment dynamics directly from pixel observations. This architecture is useful for learning tasks in which …
View article: Sidewalk Environment for Visual Navigation
Sidewalk Environment for Visual Navigation Open
This dataset contains low and high resolution panoramic images, coordinates, labels and a connectivity graph. In order to run this simulated environment, you will need at least one copy of the panoramic images, which are available in low (…
View article: Sidewalk Environment for Visual Navigation
Sidewalk Environment for Visual Navigation Open
This dataset contains low and high resolution panoramic images, coordinates, labels and a connectivity graph. In order to run this simulated environment, you will need at least one copy of the panoramic images, which are available in low (…
View article: Navigation Agents for the Visually Impaired: A Sidewalk Simulator and\n Experiments
Navigation Agents for the Visually Impaired: A Sidewalk Simulator and\n Experiments Open
Millions of blind and visually-impaired (BVI) people navigate urban\nenvironments every day, using smartphones for high-level path-planning and\nwhite canes or guide dogs for local information. However, many BVI people still\nstruggle to t…
View article: Navigation Agents for the Visually Impaired: A Sidewalk Simulator and Experiments
Navigation Agents for the Visually Impaired: A Sidewalk Simulator and Experiments Open
Millions of blind and visually-impaired (BVI) people navigate urban environments every day, using smartphones for high-level path-planning and white canes or guide dogs for local information. However, many BVI people still struggle to trav…
View article: Active Domain Randomization
Active Domain Randomization Open
Domain randomization is a popular technique for improving domain transfer, often used in a zero-shot setting when the target domain is unknown or cannot easily be used for training. In this work, we empirically examine the effects of domai…
View article: 3D IQ Test Task (3D-IQTT) - A Dataset for Quantitative Evaluation of 3D Reconstruction from 2D Images
3D IQ Test Task (3D-IQTT) - A Dataset for Quantitative Evaluation of 3D Reconstruction from 2D Images Open
3D reconstruction is mostly evaluated qualitatively. With this dataset, we are introducing a new difficult quantitative task, the 3D IQ test task (3D-IQTT). It is designed to be similar to mental rotation questions found in some IQ tests. …
View article: 3D IQ Test Task (3D-IQTT) - A Dataset for Quantitative Evaluation of 3D Reconstruction from 2D Images
3D IQ Test Task (3D-IQTT) - A Dataset for Quantitative Evaluation of 3D Reconstruction from 2D Images Open
3D reconstruction is mostly evaluated qualitatively. With this dataset, we are introducing a new difficult quantitative task, the 3D IQ test task (3D-IQTT). It is designed to be similar to mental rotation questions found in some IQ tests. …