Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuning Article Swipe

MAKE IMPACT

Hongran Li , Jike Zhong , Shitian Zhao , Yuxiang Lai , Haoquan Zhang , Wenpeng Zhu , Kaipeng Zhang · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2503.16188

This paper investigates the role of explicit thinking process in rule-based reinforcement fine-tuning (RFT) for MLLMs. We first propose CLS-RL for MLLM image classification, using verifiable rewards for fine-tuning. Experiments show CLS-RL significantly outperforms SFT and yields a cross-dataset generalization effect. We then rethink and question whether explicit thinking in RFT is always necessary. Challenging the convention that explicit thinking is crucial for the success of RFT, we introduce No-Thinking-RL, exploring RFT without thinking by introducing a simple equality accuracy reward. We evaluate No-Thinking-RL on 6 diverse tasks across different model sizes and types. Experimental results reveal three key findings: 1). Visual perception tasks do not require thinking during RFT, as No-Thinking-RL consistently outperforms or matches Thinking-based RFT across model sizes. 2).} Models with limited capabilities struggle to generate high-quality CoT for RFT, making Thinking-based RFT less effective than No-Thinking-RL. 3). There are inconsistencies between the answers in the thinking and answer tags for some responses of thinking-based RFT, which show lower accuracy than the overall accuracy. We hypothesize that explicit thinking before verifiable answers may hinder reward convergence and reduce performance. To test this hypothesis, we propose Think-After-Answer, which places thinking after the answer to mitigate this effect for experimental verification. Lastly, we conduct a pilot study to explore whether MLLMs can learn when to think during RFT, introducing an Adaptive-Thinking method. Experiments show that it converges to a specific prompt depending on model capability and task complexity, achieving comparable or better performance than both Thinking and No-Thinking-RL. This suggests MLLMs can adaptively decide to think or not based on their capabilities and task complexity.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2503.16188

PDF

https://arxiv.org/pdf/2503.16188

OA Status

green

OpenAlex ID

https://openalex.org/W4414688615

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4414688615

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2503.16188

Digital Object Identifier

Title

Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuning

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-03-20

Full publication date if available

Authors

Hongran Li, Jike Zhong, Shitian Zhao, Yuxiang Lai, Haoquan Zhang, Wenpeng Zhu, Kaipeng Zhang

List of authors in order

Landing page

https://arxiv.org/abs/2503.16188

Publisher landing page

PDF URL

https://arxiv.org/pdf/2503.16188

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2503.16188

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload