Utilize the Flow Before Stepping into the Same River Twice: Certainty Represented Knowledge Flow for Refusal-Aware Instruction Tuning Article Swipe

PDF

Rupeng Zhu , Zhiyuan Ma , Jiang Wu , Junyuan Gao , Jiaqi Wang , Dahua Lin , Changming He ·

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.1609/aaai.v39i24.34812

Refusal-Aware Instruction Tuning (RAIT) enables Large Language Models (LLMs) to refuse to answer unknown questions. By modifying responses of unknown questions in the training data to refusal responses such as ''I don't know", RAIT enhances the reliability of LLMs and reduces their hallucination. Generally, RAIT modifies training samples based on the correctness of the initial LLM's response. However, this crude approach can cause LLMs to excessively refuse answering questions they could have correctly answered, the problem we call over-refusal. In this paper, we explore two primary causes of over-refusal: Static conflict occurs when similar samples within the LLM’s feature space receive differing supervision signals (original vs. modified ''I don't know"). Dynamic conflict arises as the LLM's evolving knowledge during SFT enables it to answer previously unanswerable questions, but the now-answerable training samples still retain the original ''I don't know" supervision signals from the initial LLM state, leading to inconsistencies. These conflicts cause the trained LLM to misclassify known questions as unknown, resulting in over-refusal. To address this issue, we introduce Certainty Represented Knowledge Flow for Refusal-Aware Instructions Tuning (CRaFT). CRaFT centers on two main contributions: First, we additionally incorporate response certainty to selectively filter and modify data, reducing static conflicts. Second, we implement preliminary rehearsal training to characterize changes in the LLM's knowledge state, which helps mitigate dynamic conflicts during the fine-tuning process. We conducted extensive experiments on open-ended question answering and multiple-choice question task. Experiment results show that CRaFT can improve LLM's overall performance during the RAIT process.

Related Topics

Concepts

Certainty Flow (mathematics) Computer science Knowledge flow Mathematics Epistemology Knowledge management Philosophy Geometry

Metadata

Type: article
Language: en
Landing Page: https://doi.org/10.1609/aaai.v39i24.34812
PDF: https://ojs.aaai.org/index.php/AAAI/article/download/34812/36967
OA Status: diamond
Related Works: 10
OpenAlex ID: https://openalex.org/W4409362686

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4409362686

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.1609/aaai.v39i24.34812

Digital Object Identifier
Title: Utilize the Flow Before Stepping into the Same River Twice: Certainty Represented Knowledge Flow for Refusal-Aware Instruction Tuning

Work title
Type: article

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-04-11

Full publication date if available
Authors: Rupeng Zhu, Zhiyuan Ma, Jiang Wu, Junyuan Gao, Jiaqi Wang, Dahua Lin, Changming He

List of authors in order
Landing page: https://doi.org/10.1609/aaai.v39i24.34812

Publisher landing page
PDF URL: https://ojs.aaai.org/index.php/AAAI/article/download/34812/36967

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: diamond

Open access status per OpenAlex
OA URL: https://ojs.aaai.org/index.php/AAAI/article/download/34812/36967

Direct OA link when available
Concepts: Certainty, Flow (mathematics), Computer science, Knowledge flow, Mathematics, Epistemology, Knowledge management, Philosophy, Geometry

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4409362686
doi	https://doi.org/10.1609/aaai.v39i24.34812
ids.doi	https://doi.org/10.1609/aaai.v39i24.34812
ids.openalex	https://openalex.org/W4409362686
fwci	0.0
type	article
title	Utilize the Flow Before Stepping into the Same River Twice: Certainty Represented Knowledge Flow for Refusal-Aware Instruction Tuning
biblio.issue	24
biblio.volume	39
biblio.last_page	26165
biblio.first_page	26157
topics[0].id	https://openalex.org/T11902
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9851999878883362
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Intelligent Tutoring Systems and Adaptive Learning
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C7493553
concepts[0].level	2
concepts[0].score	0.8107709884643555
concepts[0].wikidata	https://www.wikidata.org/wiki/Q1520777
concepts[0].display_name	Certainty
concepts[1].id	https://openalex.org/C38349280
concepts[1].level	2
concepts[1].score	0.6543534994125366
concepts[1].wikidata	https://www.wikidata.org/wiki/Q1434290
concepts[1].display_name	Flow (mathematics)
concepts[2].id	https://openalex.org/C41008148
concepts[2].level	0
concepts[2].score	0.47354549169540405
concepts[2].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[2].display_name	Computer science
concepts[3].id	https://openalex.org/C2984563274
concepts[3].level	2
concepts[3].score	0.4139832854270935
concepts[3].wikidata	https://www.wikidata.org/wiki/Q2586354
concepts[3].display_name	Knowledge flow
concepts[4].id	https://openalex.org/C33923547
concepts[4].level	0
concepts[4].score	0.24158161878585815
concepts[4].wikidata	https://www.wikidata.org/wiki/Q395
concepts[4].display_name	Mathematics
concepts[5].id	https://openalex.org/C111472728
concepts[5].level	1
concepts[5].score	0.22372236847877502
concepts[5].wikidata	https://www.wikidata.org/wiki/Q9471
concepts[5].display_name	Epistemology
concepts[6].id	https://openalex.org/C56739046
concepts[6].level	1
concepts[6].score	0.13599470257759094
concepts[6].wikidata	https://www.wikidata.org/wiki/Q192060
concepts[6].display_name	Knowledge management
concepts[7].id	https://openalex.org/C138885662
concepts[7].level	0
concepts[7].score	0.13222801685333252
concepts[7].wikidata	https://www.wikidata.org/wiki/Q5891
concepts[7].display_name	Philosophy
concepts[8].id	https://openalex.org/C2524010
concepts[8].level	1
concepts[8].score	0.06676894426345825
concepts[8].wikidata	https://www.wikidata.org/wiki/Q8087
concepts[8].display_name	Geometry
keywords[0].id	https://openalex.org/keywords/certainty
keywords[0].score	0.8107709884643555
keywords[0].display_name	Certainty
keywords[1].id	https://openalex.org/keywords/flow
keywords[1].score	0.6543534994125366
keywords[1].display_name	Flow (mathematics)
keywords[2].id	https://openalex.org/keywords/computer-science
keywords[2].score	0.47354549169540405
keywords[2].display_name	Computer science
keywords[3].id	https://openalex.org/keywords/knowledge-flow
keywords[3].score	0.4139832854270935
keywords[3].display_name	Knowledge flow
keywords[4].id	https://openalex.org/keywords/mathematics
keywords[4].score	0.24158161878585815
keywords[4].display_name	Mathematics
keywords[5].id	https://openalex.org/keywords/epistemology
keywords[5].score	0.22372236847877502
keywords[5].display_name	Epistemology
keywords[6].id	https://openalex.org/keywords/knowledge-management
keywords[6].score	0.13599470257759094
keywords[6].display_name	Knowledge management
keywords[7].id	https://openalex.org/keywords/philosophy
keywords[7].score	0.13222801685333252
keywords[7].display_name	Philosophy
keywords[8].id	https://openalex.org/keywords/geometry
keywords[8].score	0.06676894426345825
keywords[8].display_name	Geometry
language	en
locations[0].id	doi:10.1609/aaai.v39i24.34812
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4210191458
locations[0].source.issn	2159-5399, 2374-3468
locations[0].source.type	conference
locations[0].source.is_oa	True
locations[0].source.issn_l	2159-5399
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	Proceedings of the AAAI Conference on Artificial Intelligence
locations[0].source.host_organization	https://openalex.org/P4310320058
locations[0].source.host_organization_name	Association for the Advancement of Artificial Intelligence
locations[0].source.host_organization_lineage	https://openalex.org/P4310320058
locations[0].source.host_organization_lineage_names	Association for the Advancement of Artificial Intelligence
locations[0].license
locations[0].pdf_url	https://ojs.aaai.org/index.php/AAAI/article/download/34812/36967
locations[0].version	publishedVersion
locations[0].raw_type	journal-article
locations[0].license_id
locations[0].is_accepted	True
locations[0].is_published	True
locations[0].raw_source_name	Proceedings of the AAAI Conference on Artificial Intelligence
locations[0].landing_page_url	https://doi.org/10.1609/aaai.v39i24.34812
indexed_in	crossref
authorships[0].author.id	https://openalex.org/A5030676323
authorships[0].author.orcid	https://orcid.org/0000-0002-6321-2663
authorships[0].author.display_name	Rupeng Zhu
authorships[0].author_position	first
authorships[0].raw_author_name	Runchuan Zhu
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5081053737
authorships[1].author.orcid	https://orcid.org/0000-0003-3270-6844
authorships[1].author.display_name	Zhiyuan Ma
authorships[1].author_position	middle
authorships[1].raw_author_name	Zhipeng Ma
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5100949430
authorships[2].author.orcid
authorships[2].author.display_name	Jiang Wu
authorships[2].author_position	middle
authorships[2].raw_author_name	Jiang Wu
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5111363114
authorships[3].author.orcid
authorships[3].author.display_name	Junyuan Gao
authorships[3].author_position	middle
authorships[3].raw_author_name	Junyuan Gao
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5100365347
authorships[4].author.orcid	https://orcid.org/0000-0001-8841-0124
authorships[4].author.display_name	Jiaqi Wang
authorships[4].author_position	middle
authorships[4].raw_author_name	Jiaqi Wang
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5010087030
authorships[5].author.orcid	https://orcid.org/0000-0002-8865-7896
authorships[5].author.display_name	Dahua Lin
authorships[5].author_position	middle
authorships[5].raw_author_name	Dahua Lin
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5075805415
authorships[6].author.orcid
authorships[6].author.display_name	Changming He
authorships[6].author_position	last
authorships[6].raw_author_name	Conghui He
authorships[6].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://ojs.aaai.org/index.php/AAAI/article/download/34812/36967
open_access.oa_status	diamond
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Utilize the Flow Before Stepping into the Same River Twice: Certainty Represented Knowledge Flow for Refusal-Aware Instruction Tuning
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T11902
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9851999878883362
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Intelligent Tutoring Systems and Adaptive Learning
related_works	https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2418227039, https://openalex.org/W2387971622, https://openalex.org/W1559214593, https://openalex.org/W2141935323, https://openalex.org/W2418299857, https://openalex.org/W2466655349, https://openalex.org/W2354806206
cited_by_count	0
locations_count	1
best_oa_location.id	doi:10.1609/aaai.v39i24.34812
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4210191458
best_oa_location.source.issn	2159-5399, 2374-3468
best_oa_location.source.type	conference
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l	2159-5399
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	Proceedings of the AAAI Conference on Artificial Intelligence
best_oa_location.source.host_organization	https://openalex.org/P4310320058
best_oa_location.source.host_organization_name	Association for the Advancement of Artificial Intelligence
best_oa_location.source.host_organization_lineage	https://openalex.org/P4310320058
best_oa_location.source.host_organization_lineage_names	Association for the Advancement of Artificial Intelligence
best_oa_location.license
best_oa_location.pdf_url	https://ojs.aaai.org/index.php/AAAI/article/download/34812/36967
best_oa_location.version	publishedVersion
best_oa_location.raw_type	journal-article
best_oa_location.license_id
best_oa_location.is_accepted	True
best_oa_location.is_published	True
best_oa_location.raw_source_name	Proceedings of the AAAI Conference on Artificial Intelligence
best_oa_location.landing_page_url	https://doi.org/10.1609/aaai.v39i24.34812
primary_location.id	doi:10.1609/aaai.v39i24.34812
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4210191458
primary_location.source.issn	2159-5399, 2374-3468
primary_location.source.type	conference
primary_location.source.is_oa	True
primary_location.source.issn_l	2159-5399
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	Proceedings of the AAAI Conference on Artificial Intelligence
primary_location.source.host_organization	https://openalex.org/P4310320058
primary_location.source.host_organization_name	Association for the Advancement of Artificial Intelligence
primary_location.source.host_organization_lineage	https://openalex.org/P4310320058
primary_location.source.host_organization_lineage_names	Association for the Advancement of Artificial Intelligence
primary_location.license
primary_location.pdf_url	https://ojs.aaai.org/index.php/AAAI/article/download/34812/36967
primary_location.version	publishedVersion
primary_location.raw_type	journal-article
primary_location.license_id
primary_location.is_accepted	True
primary_location.is_published	True
primary_location.raw_source_name	Proceedings of the AAAI Conference on Artificial Intelligence
primary_location.landing_page_url	https://doi.org/10.1609/aaai.v39i24.34812
publication_date	2025-04-11
publication_year	2025
referenced_works_count	0
abstract_inverted_index.By	15
abstract_inverted_index.In	79
abstract_inverted_index.To	164
abstract_inverted_index.We	223
abstract_inverted_index.as	29, 113, 159
abstract_inverted_index.in	21, 162, 209
abstract_inverted_index.it	121
abstract_inverted_index.of	18, 37, 52, 87
abstract_inverted_index.on	49, 181, 227
abstract_inverted_index.to	9, 11, 25, 64, 122, 147, 155, 191, 206
abstract_inverted_index.we	76, 82, 168, 186, 201
abstract_inverted_index.''I	30, 107, 136
abstract_inverted_index.LLM	144, 154
abstract_inverted_index.SFT	119
abstract_inverted_index.and	39, 194, 231
abstract_inverted_index.but	127
abstract_inverted_index.can	61, 240
abstract_inverted_index.for	174
abstract_inverted_index.the	22, 35, 50, 53, 74, 96, 114, 128, 134, 142, 152, 210, 220, 246
abstract_inverted_index.two	84, 182
abstract_inverted_index.vs.	105
abstract_inverted_index.Flow	173
abstract_inverted_index.LLMs	38, 63
abstract_inverted_index.RAIT	33, 44, 247
abstract_inverted_index.call	77
abstract_inverted_index.data	24
abstract_inverted_index.from	141
abstract_inverted_index.have	71
abstract_inverted_index.main	183
abstract_inverted_index.show	237
abstract_inverted_index.such	28
abstract_inverted_index.that	238
abstract_inverted_index.they	69
abstract_inverted_index.this	58, 80, 166
abstract_inverted_index.when	92
abstract_inverted_index.CRaFT	179, 239
abstract_inverted_index.LLM's	55, 115, 211, 242
abstract_inverted_index.Large	5
abstract_inverted_index.These	149
abstract_inverted_index.based	48
abstract_inverted_index.cause	62, 151
abstract_inverted_index.could	70
abstract_inverted_index.crude	59
abstract_inverted_index.data,	196
abstract_inverted_index.don't	31, 108, 137
abstract_inverted_index.helps	215
abstract_inverted_index.know"	138
abstract_inverted_index.known	157
abstract_inverted_index.space	99
abstract_inverted_index.still	132
abstract_inverted_index.task.	234
abstract_inverted_index.their	41
abstract_inverted_index.which	214
abstract_inverted_index.(LLMs)	8
abstract_inverted_index.(RAIT)	3
abstract_inverted_index.First,	185
abstract_inverted_index.Models	7
abstract_inverted_index.Static	89
abstract_inverted_index.Tuning	2, 177
abstract_inverted_index.answer	12, 123
abstract_inverted_index.arises	112
abstract_inverted_index.causes	86
abstract_inverted_index.during	118, 219, 245
abstract_inverted_index.filter	193
abstract_inverted_index.issue,	167
abstract_inverted_index.know",	32
abstract_inverted_index.modify	195
abstract_inverted_index.occurs	91
abstract_inverted_index.paper,	81
abstract_inverted_index.refuse	10, 66
abstract_inverted_index.retain	133
abstract_inverted_index.state,	145, 213
abstract_inverted_index.static	198
abstract_inverted_index.within	95
abstract_inverted_index.Dynamic	110
abstract_inverted_index.LLM’s	97
abstract_inverted_index.Second,	200
abstract_inverted_index.address	165
abstract_inverted_index.centers	180
abstract_inverted_index.changes	208
abstract_inverted_index.dynamic	217
abstract_inverted_index.enables	4, 120
abstract_inverted_index.explore	83
abstract_inverted_index.feature	98
abstract_inverted_index.improve	241
abstract_inverted_index.initial	54, 143
abstract_inverted_index.know").	109
abstract_inverted_index.leading	146
abstract_inverted_index.overall	243
abstract_inverted_index.primary	85
abstract_inverted_index.problem	75
abstract_inverted_index.receive	100
abstract_inverted_index.reduces	40
abstract_inverted_index.refusal	26
abstract_inverted_index.results	236
abstract_inverted_index.samples	47, 94, 131
abstract_inverted_index.signals	103, 140
abstract_inverted_index.similar	93
abstract_inverted_index.trained	153
abstract_inverted_index.unknown	13, 19
abstract_inverted_index.(CRaFT).	178
abstract_inverted_index.However,	57
abstract_inverted_index.Language	6
abstract_inverted_index.approach	60
abstract_inverted_index.conflict	90, 111
abstract_inverted_index.enhances	34
abstract_inverted_index.evolving	116
abstract_inverted_index.mitigate	216
abstract_inverted_index.modified	106
abstract_inverted_index.modifies	45
abstract_inverted_index.original	135
abstract_inverted_index.process.	222, 248
abstract_inverted_index.question	229, 233
abstract_inverted_index.reducing	197
abstract_inverted_index.response	189
abstract_inverted_index.training	23, 46, 130, 205
abstract_inverted_index.unknown,	160
abstract_inverted_index.(original	104
abstract_inverted_index.Certainty	170
abstract_inverted_index.Knowledge	172
abstract_inverted_index.answered,	73
abstract_inverted_index.answering	67, 230
abstract_inverted_index.certainty	190
abstract_inverted_index.conducted	224
abstract_inverted_index.conflicts	150, 218
abstract_inverted_index.correctly	72
abstract_inverted_index.differing	101
abstract_inverted_index.extensive	225
abstract_inverted_index.implement	202
abstract_inverted_index.introduce	169
abstract_inverted_index.knowledge	117, 212
abstract_inverted_index.modifying	16
abstract_inverted_index.questions	20, 68, 158
abstract_inverted_index.rehearsal	204
abstract_inverted_index.response.	56
abstract_inverted_index.responses	17, 27
abstract_inverted_index.resulting	161
abstract_inverted_index.Experiment	235
abstract_inverted_index.Generally,	43
abstract_inverted_index.conflicts.	199
abstract_inverted_index.open-ended	228
abstract_inverted_index.previously	124
abstract_inverted_index.questions,	126
abstract_inverted_index.questions.	14
abstract_inverted_index.Instruction	1
abstract_inverted_index.Represented	171
abstract_inverted_index.correctness	51
abstract_inverted_index.excessively	65
abstract_inverted_index.experiments	226
abstract_inverted_index.fine-tuning	221
abstract_inverted_index.incorporate	188
abstract_inverted_index.misclassify	156
abstract_inverted_index.performance	244
abstract_inverted_index.preliminary	203
abstract_inverted_index.reliability	36
abstract_inverted_index.selectively	192
abstract_inverted_index.supervision	102, 139
abstract_inverted_index.Instructions	176
abstract_inverted_index.additionally	187
abstract_inverted_index.characterize	207
abstract_inverted_index.unanswerable	125
abstract_inverted_index.Refusal-Aware	0, 175
abstract_inverted_index.over-refusal.	78, 163
abstract_inverted_index.over-refusal:	88
abstract_inverted_index.contributions:	184
abstract_inverted_index.hallucination.	42
abstract_inverted_index.now-answerable	129
abstract_inverted_index.multiple-choice	232
abstract_inverted_index.inconsistencies.	148
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	7
citation_normalized_percentile.value	0.19798995
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False