LlamaRestTest: Effective REST API Testing with Small Language Models Article Swipe

MAKE IMPACT

Myeongsoo Kim , Saurabh Sinha , Alessandro Orso · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2501.08598

Modern web services rely heavily on REST APIs, typically documented using the OpenAPI specification. The widespread adoption of this standard has resulted in the development of many black-box testing tools that generate tests based on OpenAPI specifications. Although Large Language Models (LLMs) have shown promising test-generation abilities, their application to REST API testing remains mostly unexplored. We present LlamaRestTest, a novel approach that employs two custom LLMs-created by fine-tuning and quantizing the Llama3-8B model using mined datasets of REST API example values and inter-parameter dependencies-to generate realistic test inputs and uncover inter-parameter dependencies during the testing process by analyzing server responses. We evaluated LlamaRestTest on 12 real-world services (including popular services such as Spotify), comparing it against RESTGPT, a GPT-powered specification-enhancement tool, as well as several state-of-the-art REST API testing tools, including RESTler, MoRest, EvoMaster, and ARAT-RL. Our results demonstrate that fine-tuning enables smaller models to outperform much larger models in detecting actionable parameter-dependency rules and generating valid inputs for REST API testing. We also evaluated different tool configurations, ranging from the base Llama3-8B model to fine-tuned versions, and explored multiple quantization techniques, including 2-bit, 4-bit, and 8-bit integer formats. Our study shows that small language models can perform as well as, or better than, large language models in REST API testing, balancing effectiveness and efficiency. Furthermore, LlamaRestTest outperforms state-of-the-art REST API testing tools in code coverage achieved and internal server errors identified, even when those tools use RESTGPT-enhanced specifications.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2501.08598

PDF

https://arxiv.org/pdf/2501.08598

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4406483036

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4406483036

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2501.08598

Digital Object Identifier

Title

LlamaRestTest: Effective REST API Testing with Small Language Models

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-01-15

Full publication date if available

Authors

Myeongsoo Kim, Saurabh Sinha, Alessandro Orso

List of authors in order

Landing page

https://arxiv.org/abs/2501.08598

Publisher landing page

PDF URL

https://arxiv.org/pdf/2501.08598

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2501.08598

Direct OA link when available

Concepts

Rest (music), Computer science, Medicine, Cardiology

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2025: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload