Robust RL with LLM-Driven Data Synthesis and Policy Adaptation for Autonomous Driving Article Swipe

MAKE IMPACT

Saisai Wu , Jiaxu Liu , Xiangyu Yin , Guangliang Cheng , Fang Meng , Xingyu Zhao , Xinping Yi , Xiaowei Huang · YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2410.12568

The integration of Large Language Models (LLMs) into autonomous driving systems demonstrates strong common sense and reasoning abilities, effectively addressing the pitfalls of purely data-driven methods. Current LLM-based agents require lengthy inference times and face challenges in interacting with real-time autonomous driving environments. A key open question is whether we can effectively leverage the knowledge from LLMs to train an efficient and robust Reinforcement Learning (RL) agent. This paper introduces RAPID, a novel \underline{\textbf{R}}obust \underline{\textbf{A}}daptive \underline{\textbf{P}}olicy \underline{\textbf{I}}nfusion and \underline{\textbf{D}}istillation framework, which trains specialized mix-of-policy RL agents using data synthesized by an LLM-based driving agent and online adaptation. RAPID features three key designs: 1) utilization of offline data collected from an LLM agent to distil expert knowledge into RL policies for faster real-time inference; 2) introduction of robust distillation in RL to inherit both performance and robustness from LLM-based teacher; and 3) employment of a mix-of-policy approach for joint decision decoding with a policy adapter. Through fine-tuning via online environment interaction, RAPID reduces the forgetting of LLM knowledge while maintaining adaptability to different tasks. Extensive experiments demonstrate RAPID's capability to effectively integrate LLM knowledge into scaled-down RL policies in an efficient, adaptable, and robust way. Code and checkpoints will be made publicly available upon acceptance.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2410.12568

PDF

https://arxiv.org/pdf/2410.12568

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4403578360

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4403578360

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2410.12568

Digital Object Identifier

Title

Robust RL with LLM-Driven Data Synthesis and Policy Adaptation for Autonomous Driving

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2024

Year of publication

Publication date

2024-10-16

Full publication date if available

Authors

Saisai Wu, Jiaxu Liu, Xiangyu Yin, Guangliang Cheng, Fang Meng, Xingyu Zhao, Xinping Yi, Xiaowei Huang

List of authors in order

Landing page

https://arxiv.org/abs/2410.12568

Publisher landing page

PDF URL

https://arxiv.org/pdf/2410.12568

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2410.12568

Direct OA link when available

Concepts

Adaptation (eye), Computer science, Psychology, Neuroscience

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload