Inside-Out: Hidden Factual Knowledge in LLMs Article Swipe
Zorik Gekhman
,
Eyal Ben David
,
Hadas Orgad
,
E. O. Ofek
,
Yonatan Belinkov
,
Idan Szpektor
,
Jonathan Herzig
,
Roi Reichart
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2503.15299
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2503.15299
This work presents a framework for assessing whether large language models (LLMs) encode more factual knowledge in their parameters than what they express in their outputs. While a few studies hint at this possibility, none has clearly defined or demonstrated this phenomenon. We first propose a formal definition of knowledge, quantifying it for a given question as the fraction of correct-incorrect answer pairs where the correct one is ranked higher. This gives rise to external and internal knowledge, depending on the information used to score individual answer candidates: either the model's observable token-level probabilities or its intermediate computations. Hidden knowledge arises when internal knowledge exceeds external knowledge. We then present a case study, applying this framework to three popular open-weights LLMs in a closed-book QA setup. Our results indicate that: (1) LLMs consistently encode more factual knowledge internally than what they express externally, with an average relative gap of 40%. (2) Surprisingly, some knowledge is so deeply hidden that a model can internally know an answer perfectly, yet fail to generate it even once, despite large-scale repeated sampling of 1,000 answers. This reveals fundamental limitations in the generation capabilities of LLMs, which (3) put a practical constraint on scaling test-time compute via repeated answer sampling in closed-book QA: significant performance improvements remain inaccessible because some answers are practically never sampled, yet if they were, we would be guaranteed to rank them first.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2503.15299
- https://arxiv.org/pdf/2503.15299
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4414903222
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4414903222Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2503.15299Digital Object Identifier
- Title
-
Inside-Out: Hidden Factual Knowledge in LLMsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-03-19Full publication date if available
- Authors
-
Zorik Gekhman, Eyal Ben David, Hadas Orgad, E. O. Ofek, Yonatan Belinkov, Idan Szpektor, Jonathan Herzig, Roi ReichartList of authors in order
- Landing page
-
https://arxiv.org/abs/2503.15299Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2503.15299Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2503.15299Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4414903222 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2503.15299 |
| ids.doi | https://doi.org/10.48550/arxiv.2503.15299 |
| ids.openalex | https://openalex.org/W4414903222 |
| fwci | |
| type | preprint |
| title | Inside-Out: Hidden Factual Knowledge in LLMs |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T13643 |
| topics[0].field.id | https://openalex.org/fields/33 |
| topics[0].field.display_name | Social Sciences |
| topics[0].score | 0.9264000058174133 |
| topics[0].domain.id | https://openalex.org/domains/2 |
| topics[0].domain.display_name | Social Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/3320 |
| topics[0].subfield.display_name | Political Science and International Relations |
| topics[0].display_name | Artificial Intelligence in Law |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2503.15299 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2503.15299 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2503.15299 |
| locations[1].id | doi:10.48550/arxiv.2503.15299 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2503.15299 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5087260801 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Zorik Gekhman |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Gekhman, Zorik |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5038080229 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Eyal Ben David |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | David, Eyal Ben |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5017373229 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Hadas Orgad |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Orgad, Hadas |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5078910984 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-6786-8774 |
| authorships[3].author.display_name | E. O. Ofek |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Ofek, Eran |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5051184573 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Yonatan Belinkov |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Belinkov, Yonatan |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5026091724 |
| authorships[5].author.orcid | |
| authorships[5].author.display_name | Idan Szpektor |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Szpektor, Idan |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5071893787 |
| authorships[6].author.orcid | https://orcid.org/0009-0000-7227-6557 |
| authorships[6].author.display_name | Jonathan Herzig |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Herzig, Jonathan |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5054952724 |
| authorships[7].author.orcid | https://orcid.org/0000-0001-6918-0554 |
| authorships[7].author.display_name | Roi Reichart |
| authorships[7].author_position | last |
| authorships[7].raw_author_name | Reichart, Roi |
| authorships[7].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2503.15299 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Inside-Out: Hidden Factual Knowledge in LLMs |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T13643 |
| primary_topic.field.id | https://openalex.org/fields/33 |
| primary_topic.field.display_name | Social Sciences |
| primary_topic.score | 0.9264000058174133 |
| primary_topic.domain.id | https://openalex.org/domains/2 |
| primary_topic.domain.display_name | Social Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/3320 |
| primary_topic.subfield.display_name | Political Science and International Relations |
| primary_topic.display_name | Artificial Intelligence in Law |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2503.15299 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2503.15299 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2503.15299 |
| primary_location.id | pmh:oai:arXiv.org:2503.15299 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2503.15299 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2503.15299 |
| publication_date | 2025-03-19 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 3, 27, 45, 53, 110, 122, 159, 194 |
| abstract_inverted_index.QA | 124 |
| abstract_inverted_index.We | 42, 107 |
| abstract_inverted_index.an | 144, 164 |
| abstract_inverted_index.as | 56 |
| abstract_inverted_index.at | 31 |
| abstract_inverted_index.be | 226 |
| abstract_inverted_index.if | 221 |
| abstract_inverted_index.in | 16, 23, 121, 185, 205 |
| abstract_inverted_index.is | 67, 154 |
| abstract_inverted_index.it | 51, 171 |
| abstract_inverted_index.of | 48, 59, 148, 178, 189 |
| abstract_inverted_index.on | 79, 197 |
| abstract_inverted_index.or | 38, 94 |
| abstract_inverted_index.so | 155 |
| abstract_inverted_index.to | 73, 83, 116, 169, 228 |
| abstract_inverted_index.we | 224 |
| abstract_inverted_index.(1) | 130 |
| abstract_inverted_index.(2) | 150 |
| abstract_inverted_index.(3) | 192 |
| abstract_inverted_index.Our | 126 |
| abstract_inverted_index.QA: | 207 |
| abstract_inverted_index.and | 75 |
| abstract_inverted_index.are | 216 |
| abstract_inverted_index.can | 161 |
| abstract_inverted_index.few | 28 |
| abstract_inverted_index.for | 5, 52 |
| abstract_inverted_index.gap | 147 |
| abstract_inverted_index.has | 35 |
| abstract_inverted_index.its | 95 |
| abstract_inverted_index.one | 66 |
| abstract_inverted_index.put | 193 |
| abstract_inverted_index.the | 57, 64, 80, 89, 186 |
| abstract_inverted_index.via | 201 |
| abstract_inverted_index.yet | 167, 220 |
| abstract_inverted_index.40%. | 149 |
| abstract_inverted_index.LLMs | 120, 131 |
| abstract_inverted_index.This | 0, 70, 181 |
| abstract_inverted_index.case | 111 |
| abstract_inverted_index.even | 172 |
| abstract_inverted_index.fail | 168 |
| abstract_inverted_index.hint | 30 |
| abstract_inverted_index.know | 163 |
| abstract_inverted_index.more | 13, 134 |
| abstract_inverted_index.none | 34 |
| abstract_inverted_index.rank | 229 |
| abstract_inverted_index.rise | 72 |
| abstract_inverted_index.some | 152, 214 |
| abstract_inverted_index.than | 19, 138 |
| abstract_inverted_index.that | 158 |
| abstract_inverted_index.them | 230 |
| abstract_inverted_index.then | 108 |
| abstract_inverted_index.they | 21, 140, 222 |
| abstract_inverted_index.this | 32, 40, 114 |
| abstract_inverted_index.used | 82 |
| abstract_inverted_index.what | 20, 139 |
| abstract_inverted_index.when | 101 |
| abstract_inverted_index.with | 143 |
| abstract_inverted_index.work | 1 |
| abstract_inverted_index.1,000 | 179 |
| abstract_inverted_index.LLMs, | 190 |
| abstract_inverted_index.While | 26 |
| abstract_inverted_index.first | 43 |
| abstract_inverted_index.given | 54 |
| abstract_inverted_index.gives | 71 |
| abstract_inverted_index.large | 8 |
| abstract_inverted_index.model | 160 |
| abstract_inverted_index.never | 218 |
| abstract_inverted_index.once, | 173 |
| abstract_inverted_index.pairs | 62 |
| abstract_inverted_index.score | 84 |
| abstract_inverted_index.that: | 129 |
| abstract_inverted_index.their | 17, 24 |
| abstract_inverted_index.three | 117 |
| abstract_inverted_index.were, | 223 |
| abstract_inverted_index.where | 63 |
| abstract_inverted_index.which | 191 |
| abstract_inverted_index.would | 225 |
| abstract_inverted_index.(LLMs) | 11 |
| abstract_inverted_index.Hidden | 98 |
| abstract_inverted_index.answer | 61, 86, 165, 203 |
| abstract_inverted_index.arises | 100 |
| abstract_inverted_index.deeply | 156 |
| abstract_inverted_index.either | 88 |
| abstract_inverted_index.encode | 12, 133 |
| abstract_inverted_index.first. | 231 |
| abstract_inverted_index.formal | 46 |
| abstract_inverted_index.hidden | 157 |
| abstract_inverted_index.models | 10 |
| abstract_inverted_index.ranked | 68 |
| abstract_inverted_index.remain | 211 |
| abstract_inverted_index.setup. | 125 |
| abstract_inverted_index.study, | 112 |
| abstract_inverted_index.answers | 215 |
| abstract_inverted_index.average | 145 |
| abstract_inverted_index.because | 213 |
| abstract_inverted_index.clearly | 36 |
| abstract_inverted_index.compute | 200 |
| abstract_inverted_index.correct | 65 |
| abstract_inverted_index.defined | 37 |
| abstract_inverted_index.despite | 174 |
| abstract_inverted_index.exceeds | 104 |
| abstract_inverted_index.express | 22, 141 |
| abstract_inverted_index.factual | 14, 135 |
| abstract_inverted_index.higher. | 69 |
| abstract_inverted_index.model's | 90 |
| abstract_inverted_index.popular | 118 |
| abstract_inverted_index.present | 109 |
| abstract_inverted_index.propose | 44 |
| abstract_inverted_index.results | 127 |
| abstract_inverted_index.reveals | 182 |
| abstract_inverted_index.scaling | 198 |
| abstract_inverted_index.studies | 29 |
| abstract_inverted_index.whether | 7 |
| abstract_inverted_index.answers. | 180 |
| abstract_inverted_index.applying | 113 |
| abstract_inverted_index.external | 74, 105 |
| abstract_inverted_index.fraction | 58 |
| abstract_inverted_index.generate | 170 |
| abstract_inverted_index.indicate | 128 |
| abstract_inverted_index.internal | 76, 102 |
| abstract_inverted_index.language | 9 |
| abstract_inverted_index.outputs. | 25 |
| abstract_inverted_index.presents | 2 |
| abstract_inverted_index.question | 55 |
| abstract_inverted_index.relative | 146 |
| abstract_inverted_index.repeated | 176, 202 |
| abstract_inverted_index.sampled, | 219 |
| abstract_inverted_index.sampling | 177, 204 |
| abstract_inverted_index.assessing | 6 |
| abstract_inverted_index.depending | 78 |
| abstract_inverted_index.framework | 4, 115 |
| abstract_inverted_index.knowledge | 15, 99, 103, 136, 153 |
| abstract_inverted_index.practical | 195 |
| abstract_inverted_index.test-time | 199 |
| abstract_inverted_index.constraint | 196 |
| abstract_inverted_index.definition | 47 |
| abstract_inverted_index.generation | 187 |
| abstract_inverted_index.guaranteed | 227 |
| abstract_inverted_index.individual | 85 |
| abstract_inverted_index.internally | 137, 162 |
| abstract_inverted_index.knowledge, | 49, 77 |
| abstract_inverted_index.knowledge. | 106 |
| abstract_inverted_index.observable | 91 |
| abstract_inverted_index.parameters | 18 |
| abstract_inverted_index.perfectly, | 166 |
| abstract_inverted_index.candidates: | 87 |
| abstract_inverted_index.closed-book | 123, 206 |
| abstract_inverted_index.externally, | 142 |
| abstract_inverted_index.fundamental | 183 |
| abstract_inverted_index.information | 81 |
| abstract_inverted_index.large-scale | 175 |
| abstract_inverted_index.limitations | 184 |
| abstract_inverted_index.performance | 209 |
| abstract_inverted_index.phenomenon. | 41 |
| abstract_inverted_index.practically | 217 |
| abstract_inverted_index.quantifying | 50 |
| abstract_inverted_index.significant | 208 |
| abstract_inverted_index.token-level | 92 |
| abstract_inverted_index.capabilities | 188 |
| abstract_inverted_index.consistently | 132 |
| abstract_inverted_index.demonstrated | 39 |
| abstract_inverted_index.improvements | 210 |
| abstract_inverted_index.inaccessible | 212 |
| abstract_inverted_index.intermediate | 96 |
| abstract_inverted_index.open-weights | 119 |
| abstract_inverted_index.possibility, | 33 |
| abstract_inverted_index.Surprisingly, | 151 |
| abstract_inverted_index.computations. | 97 |
| abstract_inverted_index.probabilities | 93 |
| abstract_inverted_index.correct-incorrect | 60 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 8 |
| citation_normalized_percentile |