Automatic Generation of a Cryptography Misuse Taxonomy Using Large Language Models Article Swipe

MAKE IMPACT

Jie Gui , Wenyi Ouyang , Yi Zhang , Liang Cheng , Chen Wu , Wenxin Hu · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2509.10814

The prevalence of cryptographic API misuse (CAM) is compromising the effectiveness of cryptography and in turn the security of modern systems and applications. Despite extensive efforts to develop CAM detection tools, these tools typically rely on a limited set of predefined rules from human-curated knowledge. This rigid, rule-based approach hinders adaptation to evolving CAM patterns in real practices. We propose leveraging large language models (LLMs), trained on publicly available cryptography-related data, to automatically detect and classify CAMs in real-world code to address this limitation. Our method enables the development and continuous expansion of a CAM taxonomy, supporting developers and detection tools in tracking and understanding emerging CAM patterns. Specifically, we develop an LLM-agnostic prompt engineering method to guide LLMs in detecting CAM instances from C/C++, Java, Python, and Go code, and then classifying them into a hierarchical taxonomy. Using a data set of 3,492 real-world software programs, we demonstrate the effectiveness of our approach with mainstream LLMs, including GPT, Llama, Gemini, and Claude. It also allows us to quantitatively measure and compare the performance of these LLMs in analyzing CAM in realistic code. Our evaluation produced a taxonomy with 279 base CAM categories, 36 of which are not addressed by existing taxonomies. To validate its practical value, we encode 11 newly identified CAM types into detection rules and integrate them into existing tools. Experiments show that such integration expands the tools' detection capabilities.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2509.10814

PDF

https://arxiv.org/pdf/2509.10814

OA Status

green

OpenAlex ID

https://openalex.org/W4415087066

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4415087066

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2509.10814

Digital Object Identifier

Title

Automatic Generation of a Cryptography Misuse Taxonomy Using Large Language Models

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-09-13

Full publication date if available

Authors

Jie Gui, Wenyi Ouyang, Yi Zhang, Liang Cheng, Chen Wu, Wenxin Hu

List of authors in order

Landing page

https://arxiv.org/abs/2509.10814

Publisher landing page

PDF URL

https://arxiv.org/pdf/2509.10814

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2509.10814

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload