A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs Article Swipe

MAKE IMPACT

Myeongsoo Kim , Tyler Stennett , Saurabh Sinha , Alessandro Orso · YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2411.07098

As modern web services increasingly rely on REST APIs, their thorough testing has become crucial. Furthermore, the advent of REST API documentation languages, such as the OpenAPI Specification, has led to the emergence of many black-box REST API testing tools. However, these tools often focus on individual test elements in isolation (e.g., APIs, parameters, values), resulting in lower coverage and less effectiveness in fault detection. To address these limitations, we present AutoRestTest, the first black-box tool to adopt a dependency-embedded multi-agent approach for REST API testing that integrates multi-agent reinforcement learning (MARL) with a semantic property dependency graph (SPDG) and Large Language Models (LLMs). Our approach treats REST API testing as a separable problem, where four agents -- API, dependency, parameter, and value agents -- collaborate to optimize API exploration. LLMs handle domain-specific value generation, the SPDG model simplifies the search space for dependencies using a similarity score between API operations, and MARL dynamically optimizes the agents' behavior. Our evaluation of AutoRestTest on 12 real-world REST services shows that it outperforms the four leading black-box REST API testing tools, including those assisted by RESTGPT (which generates realistic test inputs using LLMs), in terms of code coverage, operation coverage, and fault detection. Notably, AutoRestTest is the only tool able to trigger an internal server error in the Spotify service. Our ablation study illustrates that each component of AutoRestTest -- the SPDG, the LLM, and the agent-learning mechanism -- contributes to its overall effectiveness.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2411.07098

PDF

https://arxiv.org/pdf/2411.07098

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4404391907

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4404391907

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2411.07098

Digital Object Identifier

Title

A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2024

Year of publication

Publication date

2024-11-11

Full publication date if available

Authors

Myeongsoo Kim, Tyler Stennett, Saurabh Sinha, Alessandro Orso

List of authors in order

Landing page

https://arxiv.org/abs/2411.07098

Publisher landing page

PDF URL

https://arxiv.org/pdf/2411.07098

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2411.07098

Direct OA link when available

Concepts

Rest (music), Computer science, Physics, Acoustics

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2025: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload