PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability Article Swipe

MAKE IMPACT

Weijie Zhou , Manli Tao , Chaoyang Zhao , Haiyun Guo , Honghui Dong , Ming Tang , Jinqiao Wang · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2503.08481

Understanding the environment and a robot's physical reachability is crucial for task execution. While state-of-the-art vision-language models (VLMs) excel in environmental perception, they often generate inaccurate or impractical responses in embodied visual reasoning tasks due to a lack of understanding of robotic physical reachability. To address this issue, we propose a unified representation of physical reachability across diverse robots, i.e., Space-Physical Reachability Map (S-P Map), and PhysVLM, a vision-language model that integrates this reachability information into visual reasoning. Specifically, the S-P Map abstracts a robot's physical reachability into a generalized spatial representation, independent of specific robot configurations, allowing the model to focus on reachability features rather than robot-specific parameters. Subsequently, PhysVLM extends traditional VLM architectures by incorporating an additional feature encoder to process the S-P Map, enabling the model to reason about physical reachability without compromising its general vision-language capabilities. To train and evaluate PhysVLM, we constructed a large-scale multi-robot dataset, Phys100K, and a challenging benchmark, EQA-phys, which includes tasks for six different robots in both simulated and real-world environments. Experimental results demonstrate that PhysVLM outperforms existing models, achieving a 14\% improvement over GPT-4o on EQA-phys and surpassing advanced embodied VLMs such as RoboMamba and SpatialVLM on the RoboVQA-val and OpenEQA benchmarks. Additionally, the S-P Map shows strong compatibility with various VLMs, and its integration into GPT-4o-mini yields a 7.1\% performance improvement.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2503.08481

PDF

https://arxiv.org/pdf/2503.08481

OA Status

green

OpenAlex ID

https://openalex.org/W4416031512

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4416031512

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2503.08481

Digital Object Identifier

Title

PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-03-11

Full publication date if available

Authors

Weijie Zhou, Manli Tao, Chaoyang Zhao, Haiyun Guo, Honghui Dong, Ming Tang, Jinqiao Wang

List of authors in order

Landing page

https://arxiv.org/abs/2503.08481

Publisher landing page

PDF URL

https://arxiv.org/pdf/2503.08481

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2503.08481

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload