Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective Article Swipe

MAKE IMPACT

Zhoujun Cheng , Shibo Hao , Tianyang Liu , Fan Zhou , Y. H. Xie , Feng Yao , Yuexin Bian , Yonghao Zhuang , Nolan Dey , Yuanyuan Zha , Yi Gu , Kun Zhou , Haijun Yu , Yuan Li , Richard E. Fan , Jianshu She , Cunxu Gao , Abulhair Saparov , Haonan Li , Taylor W. Killian , Mikhail Yurochkin , Zhengzhong Liu , Eric P. Xing , Zhiting Hu · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2506.14965

Reinforcement learning (RL) has emerged as a promising approach to improve large language model (LLM) reasoning, yet most open efforts focus narrowly on math and code, limiting our understanding of its broader applicability to general reasoning. A key challenge lies in the lack of reliable, scalable RL reward signals across diverse reasoning domains. We introduce Guru, a curated RL reasoning corpus of 92K verifiable examples spanning six reasoning domains--Math, Code, Science, Logic, Simulation, and Tabular--each built through domain-specific reward design, deduplication, and filtering to ensure reliability and effectiveness for RL training. Based on Guru, we systematically revisit established findings in RL for LLM reasoning and observe significant variation across domains. For example, while prior work suggests that RL primarily elicits existing knowledge from pretrained models, our results reveal a more nuanced pattern: domains frequently seen during pretraining (Math, Code, Science) easily benefit from cross-domain RL training, while domains with limited pretraining exposure (Logic, Simulation, and Tabular) require in-domain training to achieve meaningful performance gains, suggesting that RL is likely to facilitate genuine skill acquisition. Finally, we present Guru-7B and Guru-32B, two models that achieve state-of-the-art performance among open models RL-trained with publicly available data, outperforming best baselines by 7.9% and 6.7% on our 17-task evaluation suite across six reasoning domains. We also show that our models effectively improve the Pass@k performance of their base models, particularly on complex tasks less likely to appear in pretraining data. We release data, models, training and evaluation code to facilitate general-purpose reasoning at: https://github.com/LLM360/Reasoning360

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2506.14965

PDF

https://arxiv.org/pdf/2506.14965

OA Status

green

OpenAlex ID

https://openalex.org/W4415333332

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4415333332

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2506.14965

Digital Object Identifier

Title

Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-06-17

Full publication date if available

Authors

Zhoujun Cheng, Shibo Hao, Tianyang Liu, Fan Zhou, Y. H. Xie, Feng Yao, Yuexin Bian, Yonghao Zhuang, Nolan Dey, Yuanyuan Zha, Yi Gu, Kun Zhou, Haijun Yu, Yuan Li, Richard E. Fan, Jianshu She, Cunxu Gao, Abulhair Saparov, Haonan Li, Taylor W. Killian, Mikhail Yurochkin, Zhengzhong Liu, Eric P. Xing, Zhiting Hu

List of authors in order

Landing page

https://arxiv.org/abs/2506.14965

Publisher landing page

PDF URL

https://arxiv.org/pdf/2506.14965

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2506.14965

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload