FreeAskWorld: An Interactive and Closed-Loop Simulator for Human-Centric Embodied AI Article Swipe

MAKE IMPACT

Yafeng Pan , Xinning He , Jun Yang , Xinyu Yin , Han Wang , Xiaoji Zheng , Chao Gao , Jiangtao Gong · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2511.13524

As embodied intelligence emerges as a core frontier in artificial intelligence research, simulation platforms must evolve beyond low-level physical interactions to capture complex, human-centered social behaviors. We introduce FreeAskWorld, an interactive simulation framework that integrates large language models (LLMs) for high-level behavior planning and semantically grounded interaction, informed by theories of intention and social cognition. Our framework supports scalable, realistic human-agent simulations and includes a modular data generation pipeline tailored for diverse embodied tasks.To validate the framework, we extend the classic Vision-and-Language Navigation (VLN) task into a interaction enriched Direction Inquiry setting, wherein agents can actively seek and interpret navigational guidance. We present and publicly release FreeAskWorld, a large-scale benchmark dataset comprising reconstructed environments, six diverse task types, 16 core object categories, 63,429 annotated sample frames, and more than 17 hours of interaction data to support training and evaluation of embodied AI systems. We benchmark VLN models, and human participants under both open-loop and closed-loop settings. Experimental results demonstrate that models fine-tuned on FreeAskWorld outperform their original counterparts, achieving enhanced semantic understanding and interaction competency. These findings underscore the efficacy of socially grounded simulation frameworks in advancing embodied AI systems toward sophisticated high-level planning and more naturalistic human-agent interaction. Importantly, our work underscores that interaction itself serves as an additional information modality.

Type

preprint

Landing Page

http://arxiv.org/abs/2511.13524

PDF

https://arxiv.org/pdf/2511.13524

OA Status

green

OpenAlex ID

https://openalex.org/W4416656756

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4416656756

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2511.13524

Digital Object Identifier

Title

FreeAskWorld: An Interactive and Closed-Loop Simulator for Human-Centric Embodied AI

Work title

Type

preprint

OpenAlex work type

Publication year

2025

Year of publication

Publication date

2025-11-17

Full publication date if available

Authors

Yafeng Pan, Xinning He, Jun Yang, Xinyu Yin, Han Wang, Xiaoji Zheng, Chao Gao, Jiangtao Gong

List of authors in order

Landing page

https://arxiv.org/abs/2511.13524

Publisher landing page

PDF URL

https://arxiv.org/pdf/2511.13524

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2511.13524

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload