Targeting Alignment: Extracting Safety Classifiers of Aligned LLMs Article Swipe

MAKE IMPACT

Jean-Charles Noirot Ferrand , Yohan Beugin , Eric Pauley , Ryan Sheatsley , Patrick McDaniel · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2501.16534

Alignment in large language models (LLMs) is used to enforce guidelines such as safety. Yet, alignment fails in the face of jailbreak attacks that modify inputs to induce unsafe outputs. In this paper, we introduce and evaluate a new technique for jailbreak attacks. We observe that alignment embeds a safety classifier in the LLM responsible for deciding between refusal and compliance, and seek to extract an approximation of this classifier: a surrogate classifier. To this end, we build candidate classifiers from subsets of the LLM. We first evaluate the degree to which candidate classifiers approximate the LLM's safety classifier in benign and adversarial settings. Then, we attack the candidates and measure how well the resulting adversarial inputs transfer to the LLM. Our evaluation shows that the best candidates achieve accurate agreement (an F1 score above 80%) using as little as 20% of the model architecture. Further, we find that attacks mounted on the surrogate classifiers can be transferred to the LLM with high success. For example, a surrogate using only 50% of the Llama 2 model achieved an attack success rate (ASR) of 70% with half the memory footprint and runtime -- a substantial improvement over attacking the LLM directly, where we only observed a 22% ASR. These results show that extracting surrogate classifiers is an effective and efficient means for modeling (and therein addressing) the vulnerability of aligned models to jailbreaking attacks.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2501.16534

PDF

https://arxiv.org/pdf/2501.16534

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4406950075

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4406950075

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2501.16534

Digital Object Identifier

Title

Targeting Alignment: Extracting Safety Classifiers of Aligned LLMs

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-01-27

Full publication date if available

Authors

Jean-Charles Noirot Ferrand, Yohan Beugin, Eric Pauley, Ryan Sheatsley, Patrick McDaniel

List of authors in order

Landing page

https://arxiv.org/abs/2501.16534

Publisher landing page

PDF URL

https://arxiv.org/pdf/2501.16534

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2501.16534

Direct OA link when available

Concepts

Artificial intelligence, Computer science, Computational biology, Biology

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload