Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering Article Swipe
Yucheng Shi
,
Qiaoyu Tan
,
Xuansheng Wu
,
Shaochen Zhong
,
Kaixiong Zhou
,
Ninghao Liu
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2403.19631
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2403.19631
Large Language Models (LLMs) have shown proficiency in question-answering tasks but often struggle to integrate real-time knowledge, leading to potentially outdated or inaccurate responses. This problem becomes even more challenging when dealing with multi-hop questions, since they require LLMs to update and integrate multiple knowledge pieces relevant to the questions. To tackle the problem, we propose the Retrieval-Augmented model Editing (RAE) framework for multi-hop question answering. RAE first retrieves edited facts and then refines the language model through in-context learning. Specifically, our retrieval approach, based on mutual information maximization, leverages the reasoning abilities of LLMs to identify chain facts that traditional similarity-based searches might miss. In addition, our framework includes a pruning strategy to eliminate redundant information from the retrieved facts, which enhances the editing accuracy and mitigates the hallucination problem. Our framework is supported by theoretical justification for its fact retrieval efficacy. Finally, comprehensive evaluation across various LLMs validates RAE's ability in providing accurate answers with updated knowledge. Our code is available at: https://github.com/sycny/RAE.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2403.19631
- https://arxiv.org/pdf/2403.19631
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4393336270
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4393336270Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2403.19631Digital Object Identifier
- Title
-
Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question AnsweringWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-03-28Full publication date if available
- Authors
-
Yucheng Shi, Qiaoyu Tan, Xuansheng Wu, Shaochen Zhong, Kaixiong Zhou, Ninghao LiuList of authors in order
- Landing page
-
https://arxiv.org/abs/2403.19631Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2403.19631Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2403.19631Direct OA link when available
- Concepts
-
Question answering, Computer science, Hop (telecommunications), Information retrieval, World Wide Web, Natural language processing, Computer networkTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4393336270 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2403.19631 |
| ids.doi | https://doi.org/10.48550/arxiv.2403.19631 |
| ids.openalex | https://openalex.org/W4393336270 |
| fwci | |
| type | preprint |
| title | Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9975000023841858 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9817000031471252 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| topics[2].id | https://openalex.org/T12031 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9745000004768372 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Speech and dialogue systems |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C44291984 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7194463610649109 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1074173 |
| concepts[0].display_name | Question answering |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.7155066728591919 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C25906391 |
| concepts[2].level | 2 |
| concepts[2].score | 0.669703483581543 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q1432381 |
| concepts[2].display_name | Hop (telecommunications) |
| concepts[3].id | https://openalex.org/C23123220 |
| concepts[3].level | 1 |
| concepts[3].score | 0.45926228165626526 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q816826 |
| concepts[3].display_name | Information retrieval |
| concepts[4].id | https://openalex.org/C136764020 |
| concepts[4].level | 1 |
| concepts[4].score | 0.42178019881248474 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q466 |
| concepts[4].display_name | World Wide Web |
| concepts[5].id | https://openalex.org/C204321447 |
| concepts[5].level | 1 |
| concepts[5].score | 0.3636243939399719 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[5].display_name | Natural language processing |
| concepts[6].id | https://openalex.org/C31258907 |
| concepts[6].level | 1 |
| concepts[6].score | 0.1765836477279663 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q1301371 |
| concepts[6].display_name | Computer network |
| keywords[0].id | https://openalex.org/keywords/question-answering |
| keywords[0].score | 0.7194463610649109 |
| keywords[0].display_name | Question answering |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.7155066728591919 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/hop |
| keywords[2].score | 0.669703483581543 |
| keywords[2].display_name | Hop (telecommunications) |
| keywords[3].id | https://openalex.org/keywords/information-retrieval |
| keywords[3].score | 0.45926228165626526 |
| keywords[3].display_name | Information retrieval |
| keywords[4].id | https://openalex.org/keywords/world-wide-web |
| keywords[4].score | 0.42178019881248474 |
| keywords[4].display_name | World Wide Web |
| keywords[5].id | https://openalex.org/keywords/natural-language-processing |
| keywords[5].score | 0.3636243939399719 |
| keywords[5].display_name | Natural language processing |
| keywords[6].id | https://openalex.org/keywords/computer-network |
| keywords[6].score | 0.1765836477279663 |
| keywords[6].display_name | Computer network |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2403.19631 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2403.19631 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2403.19631 |
| locations[1].id | doi:10.48550/arxiv.2403.19631 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2403.19631 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5016550690 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-8070-5363 |
| authorships[0].author.display_name | Yucheng Shi |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Shi, Yucheng |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5043697901 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-8999-968X |
| authorships[1].author.display_name | Qiaoyu Tan |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Tan, Qiaoyu |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5089884284 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-7816-7658 |
| authorships[2].author.display_name | Xuansheng Wu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Wu, Xuansheng |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5017672884 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Shaochen Zhong |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Zhong, Shaochen |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5071607114 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-5226-8736 |
| authorships[4].author.display_name | Kaixiong Zhou |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Zhou, Kaixiong |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5007489034 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-9170-2424 |
| authorships[5].author.display_name | Ninghao Liu |
| authorships[5].author_position | last |
| authorships[5].raw_author_name | Liu, Ninghao |
| authorships[5].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2403.19631 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-03-30T00:00:00 |
| display_name | Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9975000023841858 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| related_works | https://openalex.org/W2748952813, https://openalex.org/W2384605597, https://openalex.org/W2387743295, https://openalex.org/W2115758952, https://openalex.org/W2117210722, https://openalex.org/W2589759689, https://openalex.org/W3082787378, https://openalex.org/W2136007095, https://openalex.org/W2366230879, https://openalex.org/W3208425359 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2403.19631 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2403.19631 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2403.19631 |
| primary_location.id | pmh:oai:arXiv.org:2403.19631 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2403.19631 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2403.19631 |
| publication_date | 2024-03-28 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 110 |
| abstract_inverted_index.In | 105 |
| abstract_inverted_index.To | 50 |
| abstract_inverted_index.by | 135 |
| abstract_inverted_index.in | 7, 152 |
| abstract_inverted_index.is | 133, 161 |
| abstract_inverted_index.of | 93 |
| abstract_inverted_index.on | 85 |
| abstract_inverted_index.or | 21 |
| abstract_inverted_index.to | 13, 18, 39, 47, 95, 113 |
| abstract_inverted_index.we | 54 |
| abstract_inverted_index.Our | 131, 159 |
| abstract_inverted_index.RAE | 66 |
| abstract_inverted_index.and | 41, 71, 126 |
| abstract_inverted_index.at: | 163 |
| abstract_inverted_index.but | 10 |
| abstract_inverted_index.for | 62, 138 |
| abstract_inverted_index.its | 139 |
| abstract_inverted_index.our | 81, 107 |
| abstract_inverted_index.the | 48, 52, 56, 74, 90, 118, 123, 128 |
| abstract_inverted_index.LLMs | 38, 94, 148 |
| abstract_inverted_index.This | 24 |
| abstract_inverted_index.code | 160 |
| abstract_inverted_index.even | 27 |
| abstract_inverted_index.fact | 140 |
| abstract_inverted_index.from | 117 |
| abstract_inverted_index.have | 4 |
| abstract_inverted_index.more | 28 |
| abstract_inverted_index.that | 99 |
| abstract_inverted_index.then | 72 |
| abstract_inverted_index.they | 36 |
| abstract_inverted_index.when | 30 |
| abstract_inverted_index.with | 32, 156 |
| abstract_inverted_index.(RAE) | 60 |
| abstract_inverted_index.Large | 0 |
| abstract_inverted_index.RAE's | 150 |
| abstract_inverted_index.based | 84 |
| abstract_inverted_index.chain | 97 |
| abstract_inverted_index.facts | 70, 98 |
| abstract_inverted_index.first | 67 |
| abstract_inverted_index.might | 103 |
| abstract_inverted_index.miss. | 104 |
| abstract_inverted_index.model | 58, 76 |
| abstract_inverted_index.often | 11 |
| abstract_inverted_index.shown | 5 |
| abstract_inverted_index.since | 35 |
| abstract_inverted_index.tasks | 9 |
| abstract_inverted_index.which | 121 |
| abstract_inverted_index.(LLMs) | 3 |
| abstract_inverted_index.Models | 2 |
| abstract_inverted_index.across | 146 |
| abstract_inverted_index.edited | 69 |
| abstract_inverted_index.facts, | 120 |
| abstract_inverted_index.mutual | 86 |
| abstract_inverted_index.pieces | 45 |
| abstract_inverted_index.tackle | 51 |
| abstract_inverted_index.update | 40 |
| abstract_inverted_index.Editing | 59 |
| abstract_inverted_index.ability | 151 |
| abstract_inverted_index.answers | 155 |
| abstract_inverted_index.becomes | 26 |
| abstract_inverted_index.dealing | 31 |
| abstract_inverted_index.editing | 124 |
| abstract_inverted_index.leading | 17 |
| abstract_inverted_index.problem | 25 |
| abstract_inverted_index.propose | 55 |
| abstract_inverted_index.pruning | 111 |
| abstract_inverted_index.refines | 73 |
| abstract_inverted_index.require | 37 |
| abstract_inverted_index.through | 77 |
| abstract_inverted_index.updated | 157 |
| abstract_inverted_index.various | 147 |
| abstract_inverted_index.Finally, | 143 |
| abstract_inverted_index.Language | 1 |
| abstract_inverted_index.accuracy | 125 |
| abstract_inverted_index.accurate | 154 |
| abstract_inverted_index.enhances | 122 |
| abstract_inverted_index.identify | 96 |
| abstract_inverted_index.includes | 109 |
| abstract_inverted_index.language | 75 |
| abstract_inverted_index.multiple | 43 |
| abstract_inverted_index.outdated | 20 |
| abstract_inverted_index.problem, | 53 |
| abstract_inverted_index.problem. | 130 |
| abstract_inverted_index.question | 64 |
| abstract_inverted_index.relevant | 46 |
| abstract_inverted_index.searches | 102 |
| abstract_inverted_index.strategy | 112 |
| abstract_inverted_index.struggle | 12 |
| abstract_inverted_index.abilities | 92 |
| abstract_inverted_index.addition, | 106 |
| abstract_inverted_index.approach, | 83 |
| abstract_inverted_index.available | 162 |
| abstract_inverted_index.efficacy. | 142 |
| abstract_inverted_index.eliminate | 114 |
| abstract_inverted_index.framework | 61, 108, 132 |
| abstract_inverted_index.integrate | 14, 42 |
| abstract_inverted_index.knowledge | 44 |
| abstract_inverted_index.learning. | 79 |
| abstract_inverted_index.leverages | 89 |
| abstract_inverted_index.mitigates | 127 |
| abstract_inverted_index.multi-hop | 33, 63 |
| abstract_inverted_index.providing | 153 |
| abstract_inverted_index.real-time | 15 |
| abstract_inverted_index.reasoning | 91 |
| abstract_inverted_index.redundant | 115 |
| abstract_inverted_index.retrieval | 82, 141 |
| abstract_inverted_index.retrieved | 119 |
| abstract_inverted_index.retrieves | 68 |
| abstract_inverted_index.supported | 134 |
| abstract_inverted_index.validates | 149 |
| abstract_inverted_index.answering. | 65 |
| abstract_inverted_index.evaluation | 145 |
| abstract_inverted_index.in-context | 78 |
| abstract_inverted_index.inaccurate | 22 |
| abstract_inverted_index.knowledge, | 16 |
| abstract_inverted_index.knowledge. | 158 |
| abstract_inverted_index.questions, | 34 |
| abstract_inverted_index.questions. | 49 |
| abstract_inverted_index.responses. | 23 |
| abstract_inverted_index.challenging | 29 |
| abstract_inverted_index.information | 87, 116 |
| abstract_inverted_index.potentially | 19 |
| abstract_inverted_index.proficiency | 6 |
| abstract_inverted_index.theoretical | 136 |
| abstract_inverted_index.traditional | 100 |
| abstract_inverted_index.Specifically, | 80 |
| abstract_inverted_index.comprehensive | 144 |
| abstract_inverted_index.hallucination | 129 |
| abstract_inverted_index.justification | 137 |
| abstract_inverted_index.maximization, | 88 |
| abstract_inverted_index.similarity-based | 101 |
| abstract_inverted_index.question-answering | 8 |
| abstract_inverted_index.Retrieval-Augmented | 57 |
| abstract_inverted_index.https://github.com/sycny/RAE. | 164 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 6 |
| citation_normalized_percentile |