Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking Article Swipe

PDF

Jianjun Zhu , Liqiang Yan , Shuaiqiang Wang , Dawei Yin , Lei Sha ·

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2502.12970

Large Reasoning Models (LRMs) have recently demonstrated impressive performances across diverse domains. However, how the safety of Large Language Models (LLMs) benefits from enhanced reasoning capabilities against jailbreak queries remains unexplored. To bridge this gap, in this paper, we propose Reasoning-to-Defend (R2D), a novel training paradigm that integrates a safety-aware reasoning mechanism into LLMs' generation process. This enables self-evaluation at each step of the reasoning process, forming safety pivot tokens as indicators of the safety status of responses. Furthermore, in order to improve the accuracy of predicting pivot tokens, we propose Contrastive Pivot Optimization (CPO), which enhances the model's perception of the safety status of given dialogues. LLMs dynamically adjust their response strategies during reasoning, significantly enhancing their safety capabilities defending jailbreak attacks. Extensive experiments demonstrate that R2D effectively mitigates various attacks and improves overall safety, while maintaining the original performances. This highlights the substantial potential of safety-aware reasoning in improving robustness of LRMs and LLMs against various jailbreaks.

Related Topics

Concepts

Computer science Computer security Epistemology Philosophy

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2502.12970
PDF: https://arxiv.org/pdf/2502.12970
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4407759916

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4407759916

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2502.12970

Digital Object Identifier
Title: Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-02-18

Full publication date if available
Authors: Jianjun Zhu, Liqiang Yan, Shuaiqiang Wang, Dawei Yin, Lei Sha

List of authors in order
Landing page: https://arxiv.org/abs/2502.12970

Publisher landing page
PDF URL: https://arxiv.org/pdf/2502.12970

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2502.12970

Direct OA link when available
Concepts: Computer science, Computer security, Epistemology, Philosophy

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4407759916
doi	https://doi.org/10.48550/arxiv.2502.12970
ids.doi	https://doi.org/10.48550/arxiv.2502.12970
ids.openalex	https://openalex.org/W4407759916
fwci
type	preprint
title	Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T11689
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9801999926567078
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Adversarial Robustness in Machine Learning
topics[1].id	https://openalex.org/T12034
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9083999991416931
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1710
topics[1].subfield.display_name	Information Systems
topics[1].display_name	Digital and Cyber Forensics
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C41008148
concepts[0].level	0
concepts[0].score	0.46436306834220886
concepts[0].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[0].display_name	Computer science
concepts[1].id	https://openalex.org/C38652104
concepts[1].level	1
concepts[1].score	0.3567383289337158
concepts[1].wikidata	https://www.wikidata.org/wiki/Q3510521
concepts[1].display_name	Computer security
concepts[2].id	https://openalex.org/C111472728
concepts[2].level	1
concepts[2].score	0.3472856879234314
concepts[2].wikidata	https://www.wikidata.org/wiki/Q9471
concepts[2].display_name	Epistemology
concepts[3].id	https://openalex.org/C138885662
concepts[3].level	0
concepts[3].score	0.2299000322818756
concepts[3].wikidata	https://www.wikidata.org/wiki/Q5891
concepts[3].display_name	Philosophy
keywords[0].id	https://openalex.org/keywords/computer-science
keywords[0].score	0.46436306834220886
keywords[0].display_name	Computer science
keywords[1].id	https://openalex.org/keywords/computer-security
keywords[1].score	0.3567383289337158
keywords[1].display_name	Computer security
keywords[2].id	https://openalex.org/keywords/epistemology
keywords[2].score	0.3472856879234314
keywords[2].display_name	Epistemology
keywords[3].id	https://openalex.org/keywords/philosophy
keywords[3].score	0.2299000322818756
keywords[3].display_name	Philosophy
language	en
locations[0].id	pmh:oai:arXiv.org:2502.12970
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2502.12970
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2502.12970
locations[1].id	doi:10.48550/arxiv.2502.12970
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2502.12970
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5101956072
authorships[0].author.orcid	https://orcid.org/0000-0003-3613-5778
authorships[0].author.display_name	Jianjun Zhu
authorships[0].author_position	first
authorships[0].raw_author_name	Zhu, Junda
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5055520700
authorships[1].author.orcid	https://orcid.org/0000-0002-4633-4028
authorships[1].author.display_name	Liqiang Yan
authorships[1].author_position	middle
authorships[1].raw_author_name	Yan, Lingyong
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5050255638
authorships[2].author.orcid	https://orcid.org/0000-0002-9212-1947
authorships[2].author.display_name	Shuaiqiang Wang
authorships[2].author_position	middle
authorships[2].raw_author_name	Wang, Shuaiqiang
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5100766337
authorships[3].author.orcid	https://orcid.org/0000-0001-7295-4223
authorships[3].author.display_name	Dawei Yin
authorships[3].author_position	middle
authorships[3].raw_author_name	Yin, Dawei
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5079222154
authorships[4].author.orcid	https://orcid.org/0000-0001-5914-7590
authorships[4].author.display_name	Lei Sha
authorships[4].author_position	last
authorships[4].raw_author_name	Sha, Lei
authorships[4].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2502.12970
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T11689
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9801999926567078
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Adversarial Robustness in Machine Learning
related_works	https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2502.12970
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2502.12970
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2502.12970
primary_location.id	pmh:oai:arXiv.org:2502.12970
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2502.12970
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2502.12970
publication_date	2025-02-18
publication_year	2025
referenced_works_count	0
abstract_inverted_index.a	42, 48
abstract_inverted_index.To	31
abstract_inverted_index.as	70
abstract_inverted_index.at	59
abstract_inverted_index.in	35, 79, 149
abstract_inverted_index.of	16, 62, 72, 76, 85, 100, 104, 146, 152
abstract_inverted_index.to	81
abstract_inverted_index.we	38, 89
abstract_inverted_index.R2D	127
abstract_inverted_index.and	132, 154
abstract_inverted_index.how	13
abstract_inverted_index.the	14, 63, 73, 83, 97, 101, 138, 143
abstract_inverted_index.LLMs	107, 155
abstract_inverted_index.LRMs	153
abstract_inverted_index.This	56, 141
abstract_inverted_index.each	60
abstract_inverted_index.from	22
abstract_inverted_index.gap,	34
abstract_inverted_index.have	4
abstract_inverted_index.into	52
abstract_inverted_index.step	61
abstract_inverted_index.that	46, 126
abstract_inverted_index.this	33, 36
abstract_inverted_index.LLMs'	53
abstract_inverted_index.Large	0, 17
abstract_inverted_index.Pivot	92
abstract_inverted_index.given	105
abstract_inverted_index.novel	43
abstract_inverted_index.order	80
abstract_inverted_index.pivot	68, 87
abstract_inverted_index.their	110, 117
abstract_inverted_index.which	95
abstract_inverted_index.while	136
abstract_inverted_index.(CPO),	94
abstract_inverted_index.(LLMs)	20
abstract_inverted_index.(LRMs)	3
abstract_inverted_index.(R2D),	41
abstract_inverted_index.Models	2, 19
abstract_inverted_index.across	9
abstract_inverted_index.adjust	109
abstract_inverted_index.bridge	32
abstract_inverted_index.during	113
abstract_inverted_index.paper,	37
abstract_inverted_index.safety	15, 67, 74, 102, 118
abstract_inverted_index.status	75, 103
abstract_inverted_index.tokens	69
abstract_inverted_index.against	26, 156
abstract_inverted_index.attacks	131
abstract_inverted_index.diverse	10
abstract_inverted_index.enables	57
abstract_inverted_index.forming	66
abstract_inverted_index.improve	82
abstract_inverted_index.model's	98
abstract_inverted_index.overall	134
abstract_inverted_index.propose	39, 90
abstract_inverted_index.queries	28
abstract_inverted_index.remains	29
abstract_inverted_index.safety,	135
abstract_inverted_index.tokens,	88
abstract_inverted_index.various	130, 157
abstract_inverted_index.However,	12
abstract_inverted_index.Language	18
abstract_inverted_index.accuracy	84
abstract_inverted_index.attacks.	122
abstract_inverted_index.benefits	21
abstract_inverted_index.domains.	11
abstract_inverted_index.enhanced	23
abstract_inverted_index.enhances	96
abstract_inverted_index.improves	133
abstract_inverted_index.original	139
abstract_inverted_index.paradigm	45
abstract_inverted_index.process,	65
abstract_inverted_index.process.	55
abstract_inverted_index.recently	5
abstract_inverted_index.response	111
abstract_inverted_index.training	44
abstract_inverted_index.Extensive	123
abstract_inverted_index.Reasoning	1
abstract_inverted_index.defending	120
abstract_inverted_index.enhancing	116
abstract_inverted_index.improving	150
abstract_inverted_index.jailbreak	27, 121
abstract_inverted_index.mechanism	51
abstract_inverted_index.mitigates	129
abstract_inverted_index.potential	145
abstract_inverted_index.reasoning	24, 50, 64, 148
abstract_inverted_index.dialogues.	106
abstract_inverted_index.generation	54
abstract_inverted_index.highlights	142
abstract_inverted_index.impressive	7
abstract_inverted_index.indicators	71
abstract_inverted_index.integrates	47
abstract_inverted_index.perception	99
abstract_inverted_index.predicting	86
abstract_inverted_index.reasoning,	114
abstract_inverted_index.responses.	77
abstract_inverted_index.robustness	151
abstract_inverted_index.strategies	112
abstract_inverted_index.Contrastive	91
abstract_inverted_index.demonstrate	125
abstract_inverted_index.dynamically	108
abstract_inverted_index.effectively	128
abstract_inverted_index.experiments	124
abstract_inverted_index.jailbreaks.	158
abstract_inverted_index.maintaining	137
abstract_inverted_index.substantial	144
abstract_inverted_index.unexplored.	30
abstract_inverted_index.Furthermore,	78
abstract_inverted_index.Optimization	93
abstract_inverted_index.capabilities	25, 119
abstract_inverted_index.demonstrated	6
abstract_inverted_index.performances	8
abstract_inverted_index.safety-aware	49, 147
abstract_inverted_index.performances.	140
abstract_inverted_index.significantly	115
abstract_inverted_index.self-evaluation	58
abstract_inverted_index.Reasoning-to-Defend	40
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	5
citation_normalized_percentile