KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning Article Swipe
Yinyi Luo
,
Z.D. Zhou
,
Hao Chen
,
Kai Qiu
,
Shaoping Li
,
Jindong Wang
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2510.02392
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2510.02392
Knowledge editing and machine unlearning are two popular approaches for large language models (LLMs) to stay up-to-date. However, the knowledge updating mechanism of LLMs remains largely unexplored due to insufficient, isolated, and small-scale evaluation. For instance, are LLMs similar to humans in modifying certain knowledge? What differs editing and unlearning as training data increases? This paper proposes KnowledgeSmith, a unified framework to systematically understand the updating mechanism of LLMs. We first cast editing and unlearning as instances of one constrained optimization problem. Then, we propose an automatic dataset generator that provides structured interventions across multiple graph levels and data scales, enabling controlled studies of how different modification strategies propagate through model knowledge. Extensive experiments demonstrate nuanced insights over knowledge propagation, plasticity scaling, consistency, and robustness. For instance, our results show that LLMs do not exhibit similar updating as humans for different levels of knowledge, and there exists consistency-capacity trade-off. We hope our findings can offer suggestions to the design of more reliable and scalable strategies. Code: https://github.com/AIFrontierLab/KnowledgeSmith.git
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2510.02392
- https://arxiv.org/pdf/2510.02392
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4416368271
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4416368271Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2510.02392Digital Object Identifier
- Title
-
KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and UnlearningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-10-01Full publication date if available
- Authors
-
Yinyi Luo, Z.D. Zhou, Hao Chen, Kai Qiu, Shaoping Li, Jindong WangList of authors in order
- Landing page
-
https://arxiv.org/abs/2510.02392Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2510.02392Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2510.02392Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4416368271 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2510.02392 |
| ids.doi | https://doi.org/10.48550/arxiv.2510.02392 |
| ids.openalex | https://openalex.org/W4416368271 |
| fwci | |
| type | preprint |
| title | KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2510.02392 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2510.02392 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2510.02392 |
| locations[1].id | doi:10.48550/arxiv.2510.02392 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2510.02392 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5055364431 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Yinyi Luo |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Luo, Yinyi |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5113086010 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-9510-7104 |
| authorships[1].author.display_name | Z.D. Zhou |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Zhou, Zhexian |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5086037981 |
| authorships[2].author.orcid | https://orcid.org/0009-0004-1887-6630 |
| authorships[2].author.display_name | Hao Chen |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Chen, Hao |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5101785171 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-9486-1578 |
| authorships[3].author.display_name | Kai Qiu |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Qiu, Kai |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5107072741 |
| authorships[4].author.orcid | https://orcid.org/0000-0003-4255-6229 |
| authorships[4].author.display_name | Shaoping Li |
| authorships[4].author_position | last |
| authorships[4].raw_author_name | Li, Sharon |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5100700956 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-4833-0880 |
| authorships[5].author.display_name | Jindong Wang |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Wang, Jindong |
| authorships[5].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2510.02392 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-28T12:20:18.312131 |
| primary_topic | |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2510.02392 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2510.02392 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2510.02392 |
| primary_location.id | pmh:oai:arXiv.org:2510.02392 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2510.02392 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2510.02392 |
| publication_date | 2025-10-01 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 58 |
| abstract_inverted_index.We | 69, 149 |
| abstract_inverted_index.an | 85 |
| abstract_inverted_index.as | 50, 75, 137 |
| abstract_inverted_index.do | 132 |
| abstract_inverted_index.in | 41 |
| abstract_inverted_index.of | 22, 67, 77, 103, 142, 159 |
| abstract_inverted_index.to | 14, 28, 39, 61, 156 |
| abstract_inverted_index.we | 83 |
| abstract_inverted_index.For | 34, 125 |
| abstract_inverted_index.and | 2, 31, 48, 73, 97, 123, 144, 162 |
| abstract_inverted_index.are | 5, 36 |
| abstract_inverted_index.can | 153 |
| abstract_inverted_index.due | 27 |
| abstract_inverted_index.for | 9, 139 |
| abstract_inverted_index.how | 104 |
| abstract_inverted_index.not | 133 |
| abstract_inverted_index.one | 78 |
| abstract_inverted_index.our | 127, 151 |
| abstract_inverted_index.the | 18, 64, 157 |
| abstract_inverted_index.two | 6 |
| abstract_inverted_index.LLMs | 23, 37, 131 |
| abstract_inverted_index.This | 54 |
| abstract_inverted_index.What | 45 |
| abstract_inverted_index.cast | 71 |
| abstract_inverted_index.data | 52, 98 |
| abstract_inverted_index.hope | 150 |
| abstract_inverted_index.more | 160 |
| abstract_inverted_index.over | 117 |
| abstract_inverted_index.show | 129 |
| abstract_inverted_index.stay | 15 |
| abstract_inverted_index.that | 89, 130 |
| abstract_inverted_index.Code: | 165 |
| abstract_inverted_index.LLMs. | 68 |
| abstract_inverted_index.Then, | 82 |
| abstract_inverted_index.first | 70 |
| abstract_inverted_index.graph | 95 |
| abstract_inverted_index.large | 10 |
| abstract_inverted_index.model | 110 |
| abstract_inverted_index.offer | 154 |
| abstract_inverted_index.paper | 55 |
| abstract_inverted_index.there | 145 |
| abstract_inverted_index.(LLMs) | 13 |
| abstract_inverted_index.across | 93 |
| abstract_inverted_index.design | 158 |
| abstract_inverted_index.exists | 146 |
| abstract_inverted_index.humans | 40, 138 |
| abstract_inverted_index.levels | 96, 141 |
| abstract_inverted_index.models | 12 |
| abstract_inverted_index.certain | 43 |
| abstract_inverted_index.dataset | 87 |
| abstract_inverted_index.differs | 46 |
| abstract_inverted_index.editing | 1, 47, 72 |
| abstract_inverted_index.exhibit | 134 |
| abstract_inverted_index.largely | 25 |
| abstract_inverted_index.machine | 3 |
| abstract_inverted_index.nuanced | 115 |
| abstract_inverted_index.popular | 7 |
| abstract_inverted_index.propose | 84 |
| abstract_inverted_index.remains | 24 |
| abstract_inverted_index.results | 128 |
| abstract_inverted_index.scales, | 99 |
| abstract_inverted_index.similar | 38, 135 |
| abstract_inverted_index.studies | 102 |
| abstract_inverted_index.through | 109 |
| abstract_inverted_index.unified | 59 |
| abstract_inverted_index.However, | 17 |
| abstract_inverted_index.enabling | 100 |
| abstract_inverted_index.findings | 152 |
| abstract_inverted_index.insights | 116 |
| abstract_inverted_index.language | 11 |
| abstract_inverted_index.multiple | 94 |
| abstract_inverted_index.problem. | 81 |
| abstract_inverted_index.proposes | 56 |
| abstract_inverted_index.provides | 90 |
| abstract_inverted_index.reliable | 161 |
| abstract_inverted_index.scalable | 163 |
| abstract_inverted_index.scaling, | 121 |
| abstract_inverted_index.training | 51 |
| abstract_inverted_index.updating | 20, 65, 136 |
| abstract_inverted_index.Extensive | 112 |
| abstract_inverted_index.Knowledge | 0 |
| abstract_inverted_index.automatic | 86 |
| abstract_inverted_index.different | 105, 140 |
| abstract_inverted_index.framework | 60 |
| abstract_inverted_index.generator | 88 |
| abstract_inverted_index.instance, | 35, 126 |
| abstract_inverted_index.instances | 76 |
| abstract_inverted_index.isolated, | 30 |
| abstract_inverted_index.knowledge | 19, 118 |
| abstract_inverted_index.mechanism | 21, 66 |
| abstract_inverted_index.modifying | 42 |
| abstract_inverted_index.propagate | 108 |
| abstract_inverted_index.approaches | 8 |
| abstract_inverted_index.controlled | 101 |
| abstract_inverted_index.increases? | 53 |
| abstract_inverted_index.knowledge, | 143 |
| abstract_inverted_index.knowledge. | 111 |
| abstract_inverted_index.knowledge? | 44 |
| abstract_inverted_index.plasticity | 120 |
| abstract_inverted_index.strategies | 107 |
| abstract_inverted_index.structured | 91 |
| abstract_inverted_index.trade-off. | 148 |
| abstract_inverted_index.understand | 63 |
| abstract_inverted_index.unexplored | 26 |
| abstract_inverted_index.unlearning | 4, 49, 74 |
| abstract_inverted_index.constrained | 79 |
| abstract_inverted_index.demonstrate | 114 |
| abstract_inverted_index.evaluation. | 33 |
| abstract_inverted_index.experiments | 113 |
| abstract_inverted_index.robustness. | 124 |
| abstract_inverted_index.small-scale | 32 |
| abstract_inverted_index.strategies. | 164 |
| abstract_inverted_index.suggestions | 155 |
| abstract_inverted_index.up-to-date. | 16 |
| abstract_inverted_index.consistency, | 122 |
| abstract_inverted_index.modification | 106 |
| abstract_inverted_index.optimization | 80 |
| abstract_inverted_index.propagation, | 119 |
| abstract_inverted_index.insufficient, | 29 |
| abstract_inverted_index.interventions | 92 |
| abstract_inverted_index.systematically | 62 |
| abstract_inverted_index.KnowledgeSmith, | 57 |
| abstract_inverted_index.consistency-capacity | 147 |
| abstract_inverted_index.https://github.com/AIFrontierLab/KnowledgeSmith.git | 166 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 6 |
| citation_normalized_percentile |