Ken Caluwaerts
YOU?
Author Swipe
View article: Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfer
Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfer Open
General-purpose robots need a deep understanding of the physical world, advanced reasoning, and general and dexterous control. This report introduces the latest generation of the Gemini Robotics model family: Gemini Robotics 1.5, a multi-e…
View article: Gemini Robotics: Bringing AI into the Physical World
Gemini Robotics: Bringing AI into the Physical World Open
Recent advancements in large multimodal models have led to the emergence of remarkable generalist capabilities in digital domains, yet their translation to physical agents such as robots remains a significant challenge. This report introdu…
View article: Proc4Gem: Foundation models for physical agency through procedural generation
Proc4Gem: Foundation models for physical agency through procedural generation Open
In robot learning, it is common to either ignore the environment semantics, focusing on tasks like whole-body control which only require reasoning about robot-environment contacts, or conversely to ignore contact dynamics, focusing on grou…
View article: Prosody for Intuitive Robotic Interface Design: It's Not What You Said, It's How You Said It
Prosody for Intuitive Robotic Interface Design: It's Not What You Said, It's How You Said It Open
In this paper, we investigate the use of 'prosody' (the musical elements of speech) as a communicative signal for intuitive human-robot interaction interfaces. Our approach, rooted in Research through Design (RtD), examines the application…
View article: Learning to Learn Faster from Human Feedback with Language Model Predictive Control
Learning to Learn Faster from Human Feedback with Language Model Predictive Control Open
Large language models (LLMs) have been shown to exhibit a wide range of capabilities, such as writing robot code from language commands -- enabling non-experts to direct robot behaviors, modify them based on feedback, or compose them to pe…
View article: Barkour: Benchmarking Animal-level Agility with Quadruped Robots
Barkour: Benchmarking Animal-level Agility with Quadruped Robots Open
Animals have evolved various agile locomotion strategies, such as sprinting, leaping, and jumping. There is a growing interest in developing legged robots that move like their biological counterparts and show various agile skills to naviga…
View article: From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion
From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion Open
Legged robots navigating crowded scenes and complex terrains in the real world are required to execute dynamic leg movements while processing visual input for obstacle avoidance and path planning. We show that a quadruped robot can acquire…
View article: Learning Agile Locomotion Skills with a Mentor
Learning Agile Locomotion Skills with a Mentor Open
Developing agile behaviors for legged robots remains a challenging problem. While deep reinforcement learning is a promising approach, learning truly agile behaviors typically requires tedious reward shaping and careful curriculum design. …
View article: Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning
Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning Open
Learning adaptable policies is crucial for robots to operate autonomously in our complex and quickly changing world. In this work, we present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. In contras…
View article: Policies Modulating Trajectory Generators
Policies Modulating Trajectory Generators Open
We propose an architecture for learning complex controllable behaviors by having simple Policies Modulate Trajectory Generators (PMTG), a powerful combination that can provide both memory and prior knowledge to the controller. The result i…
View article: Data Efficient Reinforcement Learning for Legged Robots
Data Efficient Reinforcement Learning for Legged Robots Open
We present a model-based framework for robot locomotion that achieves walking based on only 4.5 minutes (45,000 control steps) of data collected on a quadruped robot. To accurately model the robot's dynamics over a long horizon, we introdu…
View article: Hierarchical Reinforcement Learning for Quadruped Locomotion
Hierarchical Reinforcement Learning for Quadruped Locomotion Open
Legged locomotion is a challenging task for learning algorithms, especially when the task requires a diverse set of primitive behaviors. To solve these problems, we introduce a hierarchical framework to automatically decompose complex loco…
View article: NoRML: No-Reward Meta Learning
NoRML: No-Reward Meta Learning Open
Efficiently adapting to new environments and changes in dynamics is critical for agents to successfully operate in the real world. Reinforcement learning (RL) based approaches typically rely on external reward feedback for adaptation. Howe…
View article: Deep Reinforcement Learning for Tensegrity Robot Locomotion
Deep Reinforcement Learning for Tensegrity Robot Locomotion Open
Tensegrity robots, composed of rigid rods connected by elastic cables, have a number of unique properties that make them appealing for use as planetary exploration rovers. However, control of tensegrity robots remains a difficult problem d…
View article: Reward-Modulated Hebbian Plasticity as Leverage for Partially Embodied Control in Compliant Robotics
Reward-Modulated Hebbian Plasticity as Leverage for Partially Embodied Control in Compliant Robotics Open
In embodied computation (or morphological computation), part of the complexity of motor control is offloaded to the body dynamics. We demonstrate that a simple Hebbian-like learning rule can be used to train systems with (partial) embodime…