A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems Article Swipe

MAKE IMPACT

Fangzhou Wu , Ning Zhang , Somesh Jha , Patrick McDaniel , Chaowei Xiao · YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2402.18649

Large Language Model (LLM) systems are inherently compositional, with individual LLM serving as the core foundation with additional layers of objects such as plugins, sandbox, and so on. Along with the great potential, there are also increasing concerns over the security of such probabilistic intelligent systems. However, existing studies on LLM security often focus on individual LLM, but without examining the ecosystem through the lens of LLM systems with other objects (e.g., Frontend, Webtool, Sandbox, and so on). In this paper, we systematically analyze the security of LLM systems, instead of focusing on the individual LLMs. To do so, we build on top of the information flow and formulate the security of LLM systems as constraints on the alignment of the information flow within LLM and between LLM and other objects. Based on this construction and the unique probabilistic nature of LLM, the attack surface of the LLM system can be decomposed into three key components: (1) multi-layer security analysis, (2) analysis of the existence of constraints, and (3) analysis of the robustness of these constraints. To ground this new attack surface, we propose a multi-layer and multi-step approach and apply it to the state-of-art LLM system, OpenAI GPT4. Our investigation exposes several security issues, not just within the LLM model itself but also in its integration with other components. We found that although the OpenAI GPT4 has designed numerous safety constraints to improve its safety features, these safety constraints are still vulnerable to attackers. To further demonstrate the real-world threats of our discovered vulnerabilities, we construct an end-to-end attack where an adversary can illicitly acquire the user's chat history, all without the need to manipulate the user's input or gain direct access to OpenAI GPT4. Our demo is in the link: https://fzwark.github.io/LLM-System-Attack-Demo/

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2402.18649

PDF

https://arxiv.org/pdf/2402.18649

OA Status

green

Cited By

17

Related Works

10

OpenAlex ID

https://openalex.org/W4401066072

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4401066072

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2402.18649

Digital Object Identifier

Title

A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2024

Year of publication

Publication date

2024-02-28

Full publication date if available

Authors

Fangzhou Wu, Ning Zhang, Somesh Jha, Patrick McDaniel, Chaowei Xiao

List of authors in order

Landing page

https://arxiv.org/abs/2402.18649

Publisher landing page

PDF URL

https://arxiv.org/pdf/2402.18649

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2402.18649

Direct OA link when available

Concepts

Political science, Computer security, Computer science

Top concepts (fields/topics) attached by OpenAlex

Cited by

17

Total citation count in OpenAlex

Citations by year (recent)

2025: 9, 2024: 8

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload