Luca Carlone
YOU?
Author Swipe
View article: Advancing AI Challenges for the United States Department of the Air Force
Advancing AI Challenges for the United States Department of the Air Force Open
The DAF-MIT AI Accelerator is a collaboration between the United States Department of the Air Force (DAF) and the Massachusetts Institute of Technology (MIT). This program pioneers fundamental advances in artificial intelligence (AI) to ex…
View article: Structured Interfaces for Automated Reasoning with 3D Scene Graphs
Structured Interfaces for Automated Reasoning with 3D Scene Graphs Open
In order to provide a robot with the ability to understand and react to a user's natural language inputs, the natural language must be connected to the robot's underlying representations of the world. Recently, large language models (LLMs)…
View article: Non-submodular Visual Attention for Robot Navigation
Non-submodular Visual Attention for Robot Navigation Open
This paper presents a task-oriented computational framework to enhance Visual-Inertial Navigation (VIN) in robots, addressing challenges such as limited time and energy resources. The framework strategically selects visual features using a…
View article: Category-Level Object Shape and Pose Estimation in Less Than a Millisecond
Category-Level Object Shape and Pose Estimation in Less Than a Millisecond Open
Object shape and pose estimation is a foundational robotics problem, supporting tasks from manipulation to scene understanding and navigation. We present a fast local solver for shape and pose estimation which requires only category-level …
View article: A Roadmap for Climate-Relevant Robotics Research
A Roadmap for Climate-Relevant Robotics Research Open
Climate change is one of the defining challenges of the 21st century, and many in the robotics community are looking for ways to contribute. This paper presents a roadmap for climate-relevant robotics research, identifying high-impact oppo…
View article: Max Entropy Moment Kalman Filter for Polynomial Systems with Arbitrary Noise
Max Entropy Moment Kalman Filter for Polynomial Systems with Arbitrary Noise Open
Designing optimal Bayes filters for nonlinear non-Gaussian systems is a challenging task. The main difficulties are: 1) representing complex beliefs, 2) handling non-Gaussian noise, and 3) marginalizing past states. To address these challe…
View article: BUFFER-X: Towards Zero-Shot Point Cloud Registration in Diverse Scenes
BUFFER-X: Towards Zero-Shot Point Cloud Registration in Diverse Scenes Open
Recent advances in deep learning-based point cloud registration have improved generalization, yet most methods still require retraining or manual parameter tuning for each new environment. In this paper, we identify three key factors limit…
View article: Outlier-Robust Training of Machine Learning Models
Outlier-Robust Training of Machine Learning Models Open
Robust training of machine learning models in the presence of outliers has garnered attention across various domains. The use of robust losses is a popular approach and is known to mitigate the impact of outliers. We bring to light two lit…
View article: Integrating Vision Systems and STPA for Robust Landing and Take-Off in VTOL Aircraft
Integrating Vision Systems and STPA for Robust Landing and Take-Off in VTOL Aircraft Open
Vertical take-off and landing (VTOL) unmanned aerial vehicles (UAVs) are versatile platforms widely used in applications such as surveillance, search and rescue, and urban air mobility. Despite their potential, the critical phases of take-…
View article: CUPS: Improving Human Pose-Shape Estimators with Conformalized Deep Uncertainty
CUPS: Improving Human Pose-Shape Estimators with Conformalized Deep Uncertainty Open
We introduce CUPS, a novel method for learning sequence-to-sequence 3D human shapes and poses from RGB videos with uncertainty quantification. To improve on top of prior work, we develop a method to generate and score multiple hypotheses d…
View article: CRISP: Object Pose and Shape Estimation with Test-Time Adaptation
CRISP: Object Pose and Shape Estimation with Test-Time Adaptation Open
We consider the problem of estimating object pose and shape from an RGB-D image. Our first contribution is to introduce CRISP, a category-agnostic object pose and shape estimation pipeline. The pipeline implements an encoder-decoder model …
View article: KISS-Matcher: Fast and Robust Point Cloud Registration Revisited
KISS-Matcher: Fast and Robust Point Cloud Registration Revisited Open
While global point cloud registration systems have advanced significantly in all aspects, many studies have focused on specific components, such as feature extraction, graph-theoretic pruning, or pose solvers. In this paper, we take a holi…
View article: Test-Time Certifiable Self-Supervision to Bridge the Sim2Real Gap in Event-Based Satellite Pose Estimation
Test-Time Certifiable Self-Supervision to Bridge the Sim2Real Gap in Event-Based Satellite Pose Estimation Open
Deep learning plays a critical role in vision-based satellite pose estimation. However, the scarcity of real data from the space environment means that deep models need to be trained using synthetic data, which raises the Sim2Real domain g…
View article: High-speed aerial grasping using a soft drone with onboard perception
High-speed aerial grasping using a soft drone with onboard perception Open
Contrary to the stunning feats observed in birds of prey, aerial manipulation and grasping with flying robots still lack versatility and agility. Conventional approaches using rigid manipulators require precise positioning and are subject …
View article: A Comparative Study of the Impact of Virtual Reality on Student Learning and Satisfaction in Aerospace Education
A Comparative Study of the Impact of Virtual Reality on Student Learning and Satisfaction in Aerospace Education Open
Student Paper. Aerospace education is a continuously evolving field that is increasingly dependent on digital tools. However, shifting the existing paradigm to accommodate new cutting-edge technologies within engineering curriculum is an a…
View article: A Certifiable Algorithm for Simultaneous Shape Estimation and Object Tracking
A Certifiable Algorithm for Simultaneous Shape Estimation and Object Tracking Open
Applications from manipulation to autonomous vehicles rely on robust and general object tracking to safely perform tasks in dynamic environments. We propose the first certifiably optimal category-level approach for simultaneous shape estim…
View article: Mixed Diffusion for 3D Indoor Scene Synthesis
Mixed Diffusion for 3D Indoor Scene Synthesis Open
Generating realistic 3D scenes is an area of growing interest in computer vision and robotics. However, creating high-quality, diverse synthetic 3D content often requires expert intervention, making it costly and complex. Recently, efforts…
View article: CHAMP: Conformalized 3D Human Multi-Hypothesis Pose Estimators
CHAMP: Conformalized 3D Human Multi-Hypothesis Pose Estimators Open
We introduce CHAMP, a novel method for learning sequence-to-sequence, multi-hypothesis 3D human poses from 2D keypoints by leveraging a conditional distribution with a diffusion model. To predict a single output 3D pose sequence, we genera…
View article: Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs
Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs Open
We present a novel approach for long-term human trajectory prediction in indoor human-centric environments, which is essential for long-horizon robot planning in these environments. State-of-the-art human trajectory prediction methods are …
View article: Clio: Real-time Task-Driven Open-Set 3D Scene Graphs
Clio: Real-time Task-Driven Open-Set 3D Scene Graphs Open
Modern tools for class-agnostic image segmentation (e.g., SegmentAnything) and open-set semantic understanding (e.g., CLIP) provide unprecedented opportunities for robot perception and mapping. While traditional closed-set metric-semantic …
View article: Monitoring of Perception Systems: Deterministic, Probabilistic, and Learning-Based Fault Detection and Identification (Abstract Reprint)
Monitoring of Perception Systems: Deterministic, Probabilistic, and Learning-Based Fault Detection and Identification (Abstract Reprint) Open
This paper investigates runtime monitoring of perception systems. Perception is a critical component of high-integrity applications of robotics and autonomous systems, such as self-driving cars. In these applications, failure of perception…
View article: Task and Motion Planning in Hierarchical 3D Scene Graphs
Task and Motion Planning in Hierarchical 3D Scene Graphs Open
Recent work in the construction of 3D scene graphs has enabled mobile robots to build large-scale metric-semantic hierarchical representations of the world. These detailed models contain information that is useful for planning, however an …
View article: GMKF: Generalized Moment Kalman Filter for Polynomial Systems with Arbitrary Noise
GMKF: Generalized Moment Kalman Filter for Polynomial Systems with Arbitrary Noise Open
This paper develops a new filtering approach for state estimation in polynomial systems corrupted by arbitrary noise, which commonly arise in robotics. We first consider a batch setup where we perform state estimation using all data collec…
View article: Khronos: A Unified Approach for Spatio-Temporal Metric-Semantic SLAM in Dynamic Environments
Khronos: A Unified Approach for Spatio-Temporal Metric-Semantic SLAM in Dynamic Environments Open
Perceiving and understanding highly dynamic and changing environments is a crucial capability for robot autonomy. While large strides have been made towards developing dynamic SLAM approaches that estimate the robot pose accurately, a less…
View article: Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Clouds
Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Clouds Open
We investigate a variation of the 3D registration problem, named multi-model 3D registration. In the multi-model registration problem, we are given two point clouds picturing a set of objects at different poses (and possibly including poin…
View article: Kimera2: Robust and Accurate Metric-Semantic SLAM in the Real World
Kimera2: Robust and Accurate Metric-Semantic SLAM in the Real World Open
We present improvements to Kimera, an open-source metric-semantic visual-inertial SLAM library. In particular, we enhance Kimera-VIO, the visual-inertial odometry pipeline powering Kimera, to support better feature tracking, more efficient…
View article: Indoor and Outdoor 3D Scene Graph Generation via Language-Enabled Spatial Ontologies
Indoor and Outdoor 3D Scene Graph Generation via Language-Enabled Spatial Ontologies Open
This paper proposes an approach to build 3D scene graphs in arbitrary indoor and outdoor environments. Such extension is challenging; the hierarchy of concepts that describe an outdoor environment is more complex than for indoors, and manu…
View article: D-Lite: Navigation-Oriented Compression of 3D Scene Graphs for Multi-Robot Collaboration
D-Lite: Navigation-Oriented Compression of 3D Scene Graphs for Multi-Robot Collaboration Open
For a multi-robot team that collaboratively explores an unknown environment, it is of vital importance that the collected information is efficiently shared among robots in order to support exploration and navigation tasks. Practical constr…
View article: PyPose v0.6: The Imperative Programming Interface for Robotics
PyPose v0.6: The Imperative Programming Interface for Robotics Open
PyPose is an open-source library for robot learning. It combines a learning-based approach with physics-based optimization, which enables seamless end-to-end robot learning. It has been used in many tasks due to its meticulously designed a…
View article: VERF: Runtime Monitoring of Pose Estimation with Neural Radiance Fields
VERF: Runtime Monitoring of Pose Estimation with Neural Radiance Fields Open
We present VERF, a collection of two methods (VERF-PnP and VERF-Light) for providing runtime assurance on the correctness of a camera pose estimate of a monocular camera without relying on direct depth measurements. We leverage the ability…