SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution Article Swipe

MAKE IMPACT

Yuxiang Wei , Olivier Duchenne , Jade Copet , Quentin Carbonneaux , Lingming Zhang , Daniel Fried , Gabriel Synnaeve , Rishabh Singh , Sida I. Wang · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2502.18449

The recent DeepSeek-R1 release has demonstrated the immense potential of reinforcement learning (RL) in enhancing the general reasoning capabilities of large language models (LLMs). While DeepSeek-R1 and other follow-up work primarily focus on applying RL to competitive coding and math problems, this paper introduces SWE-RL, the first approach to scale RL-based LLM reasoning for real-world software engineering. Leveraging a lightweight rule-based reward (e.g., the similarity score between ground-truth and LLM-generated solutions), SWE-RL enables LLMs to autonomously recover a developer's reasoning processes and solutions by learning from extensive open-source software evolution data -- the record of a software's entire lifecycle, including its code snapshots, code changes, and events such as issues and pull requests. Trained on top of Llama 3, our resulting reasoning model, Llama3-SWE-RL-70B, achieves a 41.0% solve rate on SWE-bench Verified -- a human-verified collection of real-world GitHub issues. To our knowledge, this is the best performance reported for medium-sized (<100B) LLMs to date, even comparable to leading proprietary LLMs like GPT-4o. Surprisingly, despite performing RL solely on software evolution data, Llama3-SWE-RL has even emerged with generalized reasoning skills. For example, it shows improved results on five out-of-domain tasks, namely, function coding, library use, code reasoning, mathematics, and general language understanding, whereas a supervised-finetuning baseline even leads to performance degradation on average. Overall, SWE-RL opens up a new direction to improve the reasoning capabilities of LLMs through reinforcement learning on massive software engineering data.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2502.18449

PDF

https://arxiv.org/pdf/2502.18449

OA Status

green

OpenAlex ID

https://openalex.org/W4415189338

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4415189338

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2502.18449

Digital Object Identifier

Title

SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-02-25

Full publication date if available

Authors

Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, Sida I. Wang

List of authors in order

Landing page

https://arxiv.org/abs/2502.18449

Publisher landing page

PDF URL

https://arxiv.org/pdf/2502.18449

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2502.18449

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload