Lookahead Routing for Large Language Models Article Swipe

MAKE IMPACT

Tianyuan Shi , Yi Zhu , Ruiyao Chen , Xiaojun Quan · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2510.19506

Large language model (LLM) routers improve the efficiency of multi-model systems by directing each query to the most appropriate model while leveraging the diverse strengths of heterogeneous LLMs. Most existing approaches frame routing as a classification problem based solely on the input query. While this reduces overhead by avoiding inference across all models, it overlooks valuable information that could be gleaned from potential outputs and fails to capture implicit intent or contextual nuances that often emerge only during response generation. These limitations can result in suboptimal routing decisions, particularly for complex or ambiguous queries that require deeper semantic understanding. To address this challenge, we propose Lookahead, a routing framework that "foresees" potential model outputs by predicting their latent representations and uses these predictions to guide model selection, thus enabling more informed routing without full inference. Within this framework, we implement two approaches based on causal and masked language models. Empirical evaluations across seven public benchmarks - spanning instruction following, mathematical reasoning, and code generation - show that Lookahead consistently outperforms existing routing baselines, achieving an average performance gain of 7.7% over the state-of-the-art. Our code is available at https://github.com/huangcb01/lookahead-routing.

Type

preprint

Landing Page

http://arxiv.org/abs/2510.19506

PDF

https://arxiv.org/pdf/2510.19506

OA Status

green

OpenAlex ID

https://openalex.org/W4416247041

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4416247041

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2510.19506

Digital Object Identifier

Title

Lookahead Routing for Large Language Models

Work title

Type

preprint

OpenAlex work type

Publication year

2025

Year of publication

Publication date

2025-10-22

Full publication date if available

Authors

Tianyuan Shi, Yi Zhu, Ruiyao Chen, Xiaojun Quan

List of authors in order

Landing page

https://arxiv.org/abs/2510.19506

Publisher landing page

PDF URL

https://arxiv.org/pdf/2510.19506

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2510.19506

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload