QLLM: Do We Really Need a Mixing Network for Credit Assignment in Multi-Agent Reinforcement Learning? Article Swipe

MAKE IMPACT

Zhouyang Jiang , Bin Zhang , Airong Wei , Zhiwei Xu · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2504.12961

Credit assignment has remained a fundamental challenge in multi-agent reinforcement learning (MARL). Previous studies have primarily addressed this issue through value decomposition methods under the centralized training with decentralized execution paradigm, where neural networks are utilized to approximate the nonlinear relationship between individual Q-values and the global Q-value. Although these approaches have achieved considerable success in various benchmark tasks, they still suffer from several limitations, including imprecise attribution of contributions, limited interpretability, and poor scalability in high-dimensional state spaces. To address these challenges, we propose a novel algorithm, \textbf{QLLM}, which facilitates the automatic construction of credit assignment functions using large language models (LLMs). Specifically, the concept of \textbf{TFCAF} is introduced, wherein the credit allocation process is represented as a direct and expressive nonlinear functional formulation. A custom-designed \textit{coder-evaluator} framework is further employed to guide the generation, verification, and refinement of executable code by LLMs, significantly mitigating issues such as hallucination and shallow reasoning during inference. Extensive experiments conducted on several standard MARL benchmarks demonstrate that the proposed method consistently outperforms existing state-of-the-art baselines. Moreover, QLLM exhibits strong generalization capability and maintains compatibility with a wide range of MARL algorithms that utilize mixing networks, positioning it as a promising and versatile solution for complex multi-agent scenarios.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2504.12961

PDF

https://arxiv.org/pdf/2504.12961

OA Status

green

OpenAlex ID

https://openalex.org/W4414975192

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4414975192

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2504.12961

Digital Object Identifier

Title

QLLM: Do We Really Need a Mixing Network for Credit Assignment in Multi-Agent Reinforcement Learning?

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-04-17

Full publication date if available

Authors

Zhouyang Jiang, Bin Zhang, Airong Wei, Zhiwei Xu

List of authors in order

Landing page

https://arxiv.org/abs/2504.12961

Publisher landing page

PDF URL

https://arxiv.org/pdf/2504.12961

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2504.12961

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload