Diffusing States and Matching Scores: A New Framework for Imitation Learning Article Swipe

MAKE IMPACT

Runzhe Wu , Yiding Chen , Gokul Swamy , Kianté Brantley , Wen Sun · YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2410.13855

Adversarial Imitation Learning is traditionally framed as a two-player zero-sum game between a learner and an adversarially chosen cost function, and can therefore be thought of as the sequential generalization of a Generative Adversarial Network (GAN). However, in recent years, diffusion models have emerged as a non-adversarial alternative to GANs that merely require training a score function via regression, yet produce generations of higher quality. In response, we investigate how to lift insights from diffusion modeling to the sequential setting. We propose diffusing states and performing score-matching along diffused states to measure the discrepancy between the expert's and learner's states. Thus, our approach only requires training score functions to predict noises via standard regression, making it significantly easier and more stable to train than adversarial methods. Theoretically, we prove first- and second-order instance-dependent bounds with linear scaling in the horizon, proving that our approach avoids the compounding errors that stymie offline approaches to imitation learning. Empirically, we show our approach outperforms both GAN-style imitation learning baselines and discriminator-free imitation learning baselines across various continuous control problems, including complex tasks like controlling humanoids to walk, sit, crawl, and navigate through obstacles.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2410.13855

PDF

https://arxiv.org/pdf/2410.13855

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4403580245

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4403580245

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2410.13855

Digital Object Identifier

Title

Diffusing States and Matching Scores: A New Framework for Imitation Learning

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2024

Year of publication

Publication date

2024-10-17

Full publication date if available

Authors

Runzhe Wu, Yiding Chen, Gokul Swamy, Kianté Brantley, Wen Sun

List of authors in order

Landing page

https://arxiv.org/abs/2410.13855

Publisher landing page

PDF URL

https://arxiv.org/pdf/2410.13855

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2410.13855

Direct OA link when available

Concepts

Imitation, Matching (statistics), Statistical learning, Artificial intelligence, Psychology, Cognitive psychology, Econometrics, Computer science, Economics, Mathematics, Statistics, Social psychology

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2025: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload