Training-free Generation of Temporally Consistent Rewards from VLMs Article Swipe

MAKE IMPACT

Yinuo Zhao , Jiale Yuan , Zhiyuan Xu , Xiaoshuai Hao , Xinyi Zhang , Kai Wu , Zhengping Che , Chi Harold Liu , Jian Tang · YOU? · · 2025 · Open Access ·

Recent advances in vision-language models (VLMs) have significantly improved performance in embodied tasks such as goal decomposition and visual comprehension. However, providing accurate rewards for robotic manipulation without fine-tuning VLMs remains challenging due to the absence of domain-specific robotic knowledge in pre-trained datasets and high computational costs that hinder real-time applicability. To address this, we propose $\mathrm{T}^2$-VLM, a novel training-free, temporally consistent framework that generates accurate rewards through tracking the status changes in VLM-derived subgoals. Specifically, our method first queries the VLM to establish spatially aware subgoals and an initial completion estimate before each round of interaction. We then employ a Bayesian tracking algorithm to update the goal completion status dynamically, using subgoal hidden states to generate structured rewards for reinforcement learning (RL) agents. This approach enhances long-horizon decision-making and improves failure recovery capabilities with RL. Extensive experiments indicate that $\mathrm{T}^2$-VLM achieves state-of-the-art performance in two robot manipulation benchmarks, demonstrating superior reward accuracy with reduced computation consumption. We believe our approach not only advances reward generation techniques but also contributes to the broader field of embodied AI. Project website: https://t2-vlm.github.io/.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2507.04789

PDF

https://arxiv.org/pdf/2507.04789

OA Status

green

OpenAlex ID

https://openalex.org/W4415163499

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4415163499

Canonical identifier for this work in OpenAlex

Title

Training-free Generation of Temporally Consistent Rewards from VLMs

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-07-07

Full publication date if available

Authors

Yinuo Zhao, Jiale Yuan, Zhiyuan Xu, Xiaoshuai Hao, Xinyi Zhang, Kai Wu, Zhengping Che, Chi Harold Liu, Jian Tang

List of authors in order

Landing page

https://arxiv.org/abs/2507.04789

Publisher landing page

PDF URL

https://arxiv.org/pdf/2507.04789

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2507.04789

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload