Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis Article Swipe
Reshmi Ghosh
,
Rahul Seetharaman
,
Hitesh Wadhwa
,
Somyaa Aggarwal
,
Samyadeep Basu
,
Soundararajan Srinivasan
,
Wenlong Zhao
,
Shreyas Chaudhari
,
Ehsan Aghazadeh
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2410.00857
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2410.00857
Retrieval Augmented Generation (RAG) is a widely used approach for leveraging external context in several natural language applications such as question answering and information retrieval. Yet, the exact nature in which a Language Model (LM) leverages this non-parametric memory or retrieved context isn't clearly understood. This paper mechanistically examines the RAG pipeline to highlight that LMs demonstrate a "shortcut'' effect and have a strong bias towards utilizing the retrieved context to answer questions, while relying minimally on model priors. We propose (a) Causal Mediation Analysis; for proving that parametric memory is minimally utilized when answering a question and (b) Attention Contributions and Knockouts for showing the last token residual stream do not get enriched from the subject token in the question, but gets enriched from tokens of RAG-context. We find this pronounced "shortcut'' behaviour to be true across both LLMs (e.g.,LlaMa) and SLMs (e.g., Phi)
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2410.00857
- https://arxiv.org/pdf/2410.00857
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4403821520
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4403821520Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2410.00857Digital Object Identifier
- Title
-
Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysisWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-10-01Full publication date if available
- Authors
-
Reshmi Ghosh, Rahul Seetharaman, Hitesh Wadhwa, Somyaa Aggarwal, Samyadeep Basu, Soundararajan Srinivasan, Wenlong Zhao, Shreyas Chaudhari, Ehsan AghazadehList of authors in order
- Landing page
-
https://arxiv.org/abs/2410.00857Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2410.00857Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2410.00857Direct OA link when available
- Concepts
-
Parametric statistics, Computer science, Information retrieval, Statistics, MathematicsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4403821520 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2410.00857 |
| ids.doi | https://doi.org/10.48550/arxiv.2410.00857 |
| ids.openalex | https://openalex.org/W4403821520 |
| fwci | 0.0 |
| type | preprint |
| title | Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10286 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.8027999997138977 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1710 |
| topics[0].subfield.display_name | Information Systems |
| topics[0].display_name | Information Retrieval and Search Behavior |
| topics[1].id | https://openalex.org/T10028 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.7268000245094299 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Topic Modeling |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C117251300 |
| concepts[0].level | 2 |
| concepts[0].score | 0.5987159609794617 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1849855 |
| concepts[0].display_name | Parametric statistics |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.5478286147117615 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C23123220 |
| concepts[2].level | 1 |
| concepts[2].score | 0.4068088233470917 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q816826 |
| concepts[2].display_name | Information retrieval |
| concepts[3].id | https://openalex.org/C105795698 |
| concepts[3].level | 1 |
| concepts[3].score | 0.23242685198783875 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q12483 |
| concepts[3].display_name | Statistics |
| concepts[4].id | https://openalex.org/C33923547 |
| concepts[4].level | 0 |
| concepts[4].score | 0.18179991841316223 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[4].display_name | Mathematics |
| keywords[0].id | https://openalex.org/keywords/parametric-statistics |
| keywords[0].score | 0.5987159609794617 |
| keywords[0].display_name | Parametric statistics |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.5478286147117615 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/information-retrieval |
| keywords[2].score | 0.4068088233470917 |
| keywords[2].display_name | Information retrieval |
| keywords[3].id | https://openalex.org/keywords/statistics |
| keywords[3].score | 0.23242685198783875 |
| keywords[3].display_name | Statistics |
| keywords[4].id | https://openalex.org/keywords/mathematics |
| keywords[4].score | 0.18179991841316223 |
| keywords[4].display_name | Mathematics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2410.00857 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2410.00857 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2410.00857 |
| locations[1].id | doi:10.48550/arxiv.2410.00857 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2410.00857 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5019507987 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-1814-2133 |
| authorships[0].author.display_name | Reshmi Ghosh |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Ghosh, Reshmi |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5104019728 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Rahul Seetharaman |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Seetharaman, Rahul |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5099282601 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Hitesh Wadhwa |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Wadhwa, Hitesh |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5111234400 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Somyaa Aggarwal |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Aggarwal, Somyaa |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5085795724 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Samyadeep Basu |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Basu, Samyadeep |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5102163048 |
| authorships[5].author.orcid | |
| authorships[5].author.display_name | Soundararajan Srinivasan |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Srinivasan, Soundararajan |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5102691719 |
| authorships[6].author.orcid | |
| authorships[6].author.display_name | Wenlong Zhao |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Zhao, Wenlong |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5113258513 |
| authorships[7].author.orcid | |
| authorships[7].author.display_name | Shreyas Chaudhari |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Chaudhari, Shreyas |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5099282602 |
| authorships[8].author.orcid | |
| authorships[8].author.display_name | Ehsan Aghazadeh |
| authorships[8].author_position | last |
| authorships[8].raw_author_name | Aghazadeh, Ehsan |
| authorships[8].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2410.00857 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-10-29T00:00:00 |
| display_name | Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10286 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.8027999997138977 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1710 |
| primary_topic.subfield.display_name | Information Systems |
| primary_topic.display_name | Information Retrieval and Search Behavior |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2410.00857 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2410.00857 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2410.00857 |
| primary_location.id | pmh:oai:arXiv.org:2410.00857 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2410.00857 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2410.00857 |
| publication_date | 2024-10-01 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 5, 31, 57, 62, 95 |
| abstract_inverted_index.We | 79, 128 |
| abstract_inverted_index.as | 19 |
| abstract_inverted_index.be | 135 |
| abstract_inverted_index.do | 110 |
| abstract_inverted_index.in | 13, 29, 118 |
| abstract_inverted_index.is | 4, 90 |
| abstract_inverted_index.of | 126 |
| abstract_inverted_index.on | 76 |
| abstract_inverted_index.or | 39 |
| abstract_inverted_index.to | 52, 70, 134 |
| abstract_inverted_index.(a) | 81 |
| abstract_inverted_index.(b) | 98 |
| abstract_inverted_index.LMs | 55 |
| abstract_inverted_index.RAG | 50 |
| abstract_inverted_index.and | 22, 60, 97, 101, 141 |
| abstract_inverted_index.but | 121 |
| abstract_inverted_index.for | 9, 85, 103 |
| abstract_inverted_index.get | 112 |
| abstract_inverted_index.not | 111 |
| abstract_inverted_index.the | 26, 49, 67, 105, 115, 119 |
| abstract_inverted_index.(LM) | 34 |
| abstract_inverted_index.LLMs | 139 |
| abstract_inverted_index.Phi) | 144 |
| abstract_inverted_index.SLMs | 142 |
| abstract_inverted_index.This | 45 |
| abstract_inverted_index.Yet, | 25 |
| abstract_inverted_index.bias | 64 |
| abstract_inverted_index.both | 138 |
| abstract_inverted_index.find | 129 |
| abstract_inverted_index.from | 114, 124 |
| abstract_inverted_index.gets | 122 |
| abstract_inverted_index.have | 61 |
| abstract_inverted_index.last | 106 |
| abstract_inverted_index.such | 18 |
| abstract_inverted_index.that | 54, 87 |
| abstract_inverted_index.this | 36, 130 |
| abstract_inverted_index.true | 136 |
| abstract_inverted_index.used | 7 |
| abstract_inverted_index.when | 93 |
| abstract_inverted_index.(RAG) | 3 |
| abstract_inverted_index.Model | 33 |
| abstract_inverted_index.exact | 27 |
| abstract_inverted_index.isn't | 42 |
| abstract_inverted_index.model | 77 |
| abstract_inverted_index.paper | 46 |
| abstract_inverted_index.token | 107, 117 |
| abstract_inverted_index.which | 30 |
| abstract_inverted_index.while | 73 |
| abstract_inverted_index.(e.g., | 143 |
| abstract_inverted_index.Causal | 82 |
| abstract_inverted_index.across | 137 |
| abstract_inverted_index.answer | 71 |
| abstract_inverted_index.effect | 59 |
| abstract_inverted_index.memory | 38, 89 |
| abstract_inverted_index.nature | 28 |
| abstract_inverted_index.stream | 109 |
| abstract_inverted_index.strong | 63 |
| abstract_inverted_index.tokens | 125 |
| abstract_inverted_index.widely | 6 |
| abstract_inverted_index.clearly | 43 |
| abstract_inverted_index.context | 12, 41, 69 |
| abstract_inverted_index.natural | 15 |
| abstract_inverted_index.priors. | 78 |
| abstract_inverted_index.propose | 80 |
| abstract_inverted_index.proving | 86 |
| abstract_inverted_index.relying | 74 |
| abstract_inverted_index.several | 14 |
| abstract_inverted_index.showing | 104 |
| abstract_inverted_index.subject | 116 |
| abstract_inverted_index.towards | 65 |
| abstract_inverted_index.Language | 32 |
| abstract_inverted_index.approach | 8 |
| abstract_inverted_index.enriched | 113, 123 |
| abstract_inverted_index.examines | 48 |
| abstract_inverted_index.external | 11 |
| abstract_inverted_index.language | 16 |
| abstract_inverted_index.pipeline | 51 |
| abstract_inverted_index.question | 20, 96 |
| abstract_inverted_index.residual | 108 |
| abstract_inverted_index.utilized | 92 |
| abstract_inverted_index.Analysis; | 84 |
| abstract_inverted_index.Attention | 99 |
| abstract_inverted_index.Augmented | 1 |
| abstract_inverted_index.Knockouts | 102 |
| abstract_inverted_index.Mediation | 83 |
| abstract_inverted_index.Retrieval | 0 |
| abstract_inverted_index.answering | 21, 94 |
| abstract_inverted_index.behaviour | 133 |
| abstract_inverted_index.highlight | 53 |
| abstract_inverted_index.leverages | 35 |
| abstract_inverted_index.minimally | 75, 91 |
| abstract_inverted_index.question, | 120 |
| abstract_inverted_index.retrieved | 40, 68 |
| abstract_inverted_index.utilizing | 66 |
| abstract_inverted_index.Generation | 2 |
| abstract_inverted_index.leveraging | 10 |
| abstract_inverted_index.parametric | 88 |
| abstract_inverted_index.pronounced | 131 |
| abstract_inverted_index.questions, | 72 |
| abstract_inverted_index.retrieval. | 24 |
| abstract_inverted_index."shortcut'' | 58, 132 |
| abstract_inverted_index.demonstrate | 56 |
| abstract_inverted_index.information | 23 |
| abstract_inverted_index.understood. | 44 |
| abstract_inverted_index.(e.g.,LlaMa) | 140 |
| abstract_inverted_index.RAG-context. | 127 |
| abstract_inverted_index.applications | 17 |
| abstract_inverted_index.Contributions | 100 |
| abstract_inverted_index.non-parametric | 37 |
| abstract_inverted_index.mechanistically | 47 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 9 |
| citation_normalized_percentile |