Writing-RL: Advancing Long-form Writing via Adaptive Curriculum Reinforcement Learning Article Swipe

MAKE IMPACT

Xuanyu Lei , Yuning Wu , Kaiming Liu , Weizhou Shen , Ming Yan , Zhang Ji , Yang Liu · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2506.05760

Recent advances in Large Language Models (LLMs) have enabled strong performance in long-form writing, yet existing supervised fine-tuning (SFT) approaches suffer from limitations such as data saturation and restricted learning capacity bounded by teacher signals. In this work, we present Writing-RL: an Adaptive Curriculum Reinforcement Learning framework to advance long-form writing capabilities beyond SFT. The framework consists of three key components: Margin-aware Data Selection strategy that prioritizes samples with high learning potential, Pairwise Comparison Reward mechanism that provides discriminative learning signals in the absence of verifiable rewards, and Dynamic Reference Scheduling approach, which plays a particularly critical role by adaptively adjusting task difficulty based on evolving model performance. Experiments on 7B-scale writer models show that our RL framework largely improves long-form writing performance over strong SFT baselines. Furthermore, we observe that models trained with long-output RL generalize surprisingly well to long-input reasoning tasks, potentially offering a promising perspective for rethinking long-context training.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2506.05760

PDF

https://arxiv.org/pdf/2506.05760

OA Status

green

OpenAlex ID

https://openalex.org/W4417097313

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4417097313

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2506.05760

Digital Object Identifier

Title

Writing-RL: Advancing Long-form Writing via Adaptive Curriculum Reinforcement Learning

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-06-06

Full publication date if available

Authors

Xuanyu Lei, Yuning Wu, Kaiming Liu, Weizhou Shen, Ming Yan, Zhang Ji, Yang Liu

List of authors in order

Landing page

https://arxiv.org/abs/2506.05760

Publisher landing page

PDF URL

https://arxiv.org/pdf/2506.05760

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2506.05760

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload