LlamaRestTest: Effective REST API Testing with Small Language Models Article Swipe

MAKE IMPACT

Myeongsoo Kim , Saurabh Sinha , Alessandro Orso · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.1145/3715737

Modern web services rely heavily on REST APIs, typically documented using the OpenAPI specification. The widespread adoption of this standard has resulted in the development of many black-box testing tools that generate tests based on OpenAPI specifications. Although Large Language Models (LLMs) have shown promising test-generation abilities, their application to REST API testing remains mostly unexplored. We present LlamaRestTest, a novel approach that employs two custom LLMs-created by fine-tuning and quantizing the Llama3-8B model using mined datasets of REST API example values and inter-parameter dependencies-to generate realistic test inputs and uncover inter-parameter dependencies during the testing process by analyzing server responses. We evaluated LlamaRestTest on 12 real-world services (including popular services such as Spotify), comparing it against RESTGPT, a GPT-powered specification-enhancement tool, as well as several state-of-the-art REST API testing tools, including RESTler, MoRest, EvoMaster, and ARAT-RL. Our results demonstrate that fine-tuning enables smaller models to outperform much larger models in detecting actionable parameter-dependency rules and generating valid inputs for REST API testing. We also evaluated different tool configurations, ranging from the base Llama3-8B model to fine-tuned versions, and explored multiple quantization techniques, including 2-bit, 4-bit, and 8-bit integer formats. Our study shows that small language models can perform as well as, or better than, large language models in REST API testing, balancing effectiveness and efficiency. Furthermore, LlamaRestTest outperforms state-of-the-art REST API testing tools in code coverage achieved and internal server errors identified, even when those tools use RESTGPT-enhanced specifications. Finally, through an ablation study, we show that each component of LlamaRestTest contributes to its overall performance.

Type

article

Language

en

Landing Page

https://doi.org/10.1145/3715737

OA Status

hybrid

Cited By

1

References

36

Related Works

10

OpenAlex ID

https://openalex.org/W4411449726

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4411449726

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.1145/3715737

Digital Object Identifier

Title

LlamaRestTest: Effective REST API Testing with Small Language Models

Work title

Type

article

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-06-19

Full publication date if available

Authors

Myeongsoo Kim, Saurabh Sinha, Alessandro Orso

List of authors in order

Landing page

https://doi.org/10.1145/3715737

Publisher landing page

Open access

Yes

Whether a free full text is available

OA status

hybrid

Open access status per OpenAlex

OA URL

https://doi.org/10.1145/3715737

Direct OA link when available

Concepts

Computer science, Rest (music), Fuzz testing, White-box testing, Programming language, Software, Software development, Software construction, Cardiology, Medicine

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2025: 1

Per-year citation counts (last 5 years)

References (count)

36

Number of works referenced by this work

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload