Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning Article Swipe

MAKE IMPACT

Xīn Gào , Zihang Pan , Yu Li , Juntao Li , Changming He · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2510.04081

Reasoning capability is pivotal for Large Language Models (LLMs) to solve complex tasks, yet achieving reliable and scalable reasoning remains challenging. While Chain-of-Thought (CoT) prompting has become a mainstream approach, existing methods often suffer from uncontrolled generation, insufficient quality, and limited diversity in reasoning paths. Recent efforts leverage code to enhance CoT by grounding reasoning in executable steps, but such methods are typically constrained to predefined mathematical problems, hindering scalability and generalizability. In this work, we propose Caco (Code-Assisted Chain-of-ThOught), a novel framework that automates the synthesis of high-quality, verifiable, and diverse instruction-CoT reasoning data through code-driven augmentation. Unlike prior work, Caco first fine-tunes a code-based CoT generator on existing math and programming solutions in a unified code format, then scales the data generation to a large amount of diverse reasoning traces. Crucially, we introduce automated validation via code execution and rule-based filtering to ensure logical correctness and structural diversity, followed by reverse-engineering filtered outputs into natural language instructions and language CoTs to enrich task adaptability. This closed-loop process enables fully automated, scalable synthesis of reasoning data with guaranteed executability. Experiments on our created Caco-1.3M dataset demonstrate that Caco-trained models achieve strong competitive performance on mathematical reasoning benchmarks, outperforming existing strong baselines. Further analysis reveals that Caco's code-anchored verification and instruction diversity contribute to superior generalization across unseen tasks. Our work establishes a paradigm for building self-sustaining, trustworthy reasoning systems without human intervention.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2510.04081

PDF

https://arxiv.org/pdf/2510.04081

OA Status

green

OpenAlex ID

https://openalex.org/W4416373971

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4416373971

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2510.04081

Digital Object Identifier

Title

Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-10-05

Full publication date if available

Authors

Xīn Gào, Zihang Pan, Yu Li, Juntao Li, Changming He

List of authors in order

Landing page

https://arxiv.org/abs/2510.04081

Publisher landing page

PDF URL

https://arxiv.org/pdf/2510.04081

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2510.04081

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload