AdaCuRL: Adaptive Curriculum Reinforcement Learning with Invalid Sample Mitigation and Historical Revisiting Article Swipe

MAKE IMPACT

Rui Li , Fei Wei , Yong Wang , Xiangxiang Chu · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2511.09478

Reinforcement learning (RL) has demonstrated considerable potential for enhancing reasoning in large language models (LLMs). However, existing methods suffer from Gradient Starvation and Policy Degradation when training directly on samples with mixed difficulty. To mitigate this, prior approaches leverage Chain-of-Thought (CoT) data, but the construction of high-quality CoT annotations remains labor-intensive. Alternatively, curriculum learning strategies have been explored but frequently encounter challenges, such as difficulty mismatch, reliance on manual curriculum design, and catastrophic forgetting. To address these issues, we propose AdaCuRL, a Adaptive Curriculum Reinforcement Learning framework that integrates coarse-to-fine difficulty estimation with adaptive curriculum scheduling. This approach dynamically aligns data difficulty with model capability and incorporates a data revisitation mechanism to mitigate catastrophic forgetting. Furthermore, AdaCuRL employs adaptive reference and sparse KL strategies to prevent Policy Degradation. Extensive experiments across diverse reasoning benchmarks demonstrate that AdaCuRL consistently achieves significant performance improvements on both LLMs and MLLMs.

Type

preprint

Landing Page

http://arxiv.org/abs/2511.09478

PDF

https://arxiv.org/pdf/2511.09478

OA Status

green

OpenAlex ID

https://openalex.org/W4416223658

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4416223658

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2511.09478

Digital Object Identifier

Title

AdaCuRL: Adaptive Curriculum Reinforcement Learning with Invalid Sample Mitigation and Historical Revisiting

Work title

Type

preprint

OpenAlex work type

Publication year

2025

Year of publication

Publication date

2025-11-12

Full publication date if available

Authors

Rui Li, Fei Wei, Yong Wang, Xiangxiang Chu

List of authors in order

Landing page

https://arxiv.org/abs/2511.09478

Publisher landing page

PDF URL

https://arxiv.org/pdf/2511.09478

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2511.09478

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload