ANVIL: Anomaly-based Vulnerability Identification without Labelled Training Data Article Swipe

MAKE IMPACT

Weizhou Wang , Eric Liu , Xiangyu Guo , David Lie · YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2408.16028

Supervised-learning-based vulnerability detectors often fall short due to limited labelled training data. In contrast, Large Language Models (LLMs) like GPT-4 are trained on vast unlabelled code corpora, yet perform only marginally better than coin flips when directly prompted to detect vulnerabilities. In this paper, we reframe vulnerability detection as anomaly detection, based on the premise that vulnerable code is rare and thus anomalous relative to patterns learned by LLMs. We introduce ANVIL, which performs a masked code reconstruction task: the LLM reconstructs a masked line of code, and deviations from the original are scored as anomalies. We propose a hybrid anomaly score that combines exact match, cross-entropy loss, prediction confidence, and structural complexity. We evaluate our approach across multiple LLM families, scoring methods, and context sizes, and against vulnerabilities after the LLM's training cut-off. On the PrimeVul dataset, ANVIL outperforms state-of-the-art supervised detectors-LineVul, LineVD, and LLMAO-achieving up to 2x higher Top-3 accuracy, 75% better Normalized MFR, and a significant improvement on ROC-AUC. Finally, by integrating ANVIL with fuzzers, we uncover two previously unknown vulnerabilities, demonstrating the practical utility of anomaly-guided detection.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2408.16028

PDF

https://arxiv.org/pdf/2408.16028

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4402706306

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4402706306

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2408.16028

Digital Object Identifier

Title

ANVIL: Anomaly-based Vulnerability Identification without Labelled Training Data

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2024

Year of publication

Publication date

2024-08-28

Full publication date if available

Authors

Weizhou Wang, Eric Liu, Xiangyu Guo, David Lie

List of authors in order

Landing page

https://arxiv.org/abs/2408.16028

Publisher landing page

PDF URL

https://arxiv.org/pdf/2408.16028

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2408.16028

Direct OA link when available

Concepts

Identification (biology), Training (meteorology), Vulnerability (computing), Anomaly (physics), Anomaly detection, Computer science, Training set, Artificial intelligence, Computer security, Geography, Physics, Biology, Botany, Meteorology, Condensed matter physics

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload