Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models Article Swipe

PDF

Laura Ruis , Maximilian Mozes , Jong Bin Bae , Siddhartha Rao Kamalakara , Dwarak Talupuru , A. Locatelli , Robert Kirk , Tim Rocktäschel , Edward Grefenstette , Max Bartolo ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2411.12580

The capabilities and limitations of Large Language Models have been sketched out in great detail in recent years, providing an intriguing yet conflicting picture. On the one hand, LLMs demonstrate a general ability to solve problems. On the other hand, they show surprising reasoning gaps when compared to humans, casting doubt on the robustness of their generalisation strategies. The sheer volume of data used in the design of LLMs has precluded us from applying the method traditionally used to measure generalisation: train-test set separation. To overcome this, we study what kind of generalisation strategies LLMs employ when performing reasoning tasks by investigating the pretraining data they rely on. For two models of different sizes (7B and 35B) and 2.5B of their pretraining tokens, we identify what documents influence the model outputs for three simple mathematical reasoning tasks and contrast this to the data that are influential for answering factual questions. We find that, while the models rely on mostly distinct sets of data for each factual question, a document often has a similar influence across different reasoning questions within the same task, indicating the presence of procedural knowledge. We further find that the answers to factual questions often show up in the most influential data. However, for reasoning questions the answers usually do not show up as highly influential, nor do the answers to the intermediate reasoning steps. When we characterise the top ranked documents for the reasoning questions qualitatively, we confirm that the influential documents often contain procedural knowledge, like demonstrating how to obtain a solution using formulae or code. Our findings indicate that the approach to reasoning the models use is unlike retrieval, and more like a generalisable strategy that synthesises procedural knowledge from documents doing a similar form of reasoning.

Related Topics

Computer Science

Concepts

Computer science Psychology Natural language processing Cognitive psychology Cognitive science

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2411.12580
PDF: https://arxiv.org/pdf/2411.12580
OA Status: green
Cited By: 1
Related Works: 10
OpenAlex ID: https://openalex.org/W4404574036

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4404574036

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2411.12580

Digital Object Identifier
Title: Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-11-19

Full publication date if available
Authors: Laura Ruis, Maximilian Mozes, Jong Bin Bae, Siddhartha Rao Kamalakara, Dwarak Talupuru, A. Locatelli, Robert Kirk, Tim Rocktäschel, Edward Grefenstette, Max Bartolo

List of authors in order
Landing page: https://arxiv.org/abs/2411.12580

Publisher landing page
PDF URL: https://arxiv.org/pdf/2411.12580

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2411.12580

Direct OA link when available
Concepts: Computer science, Psychology, Natural language processing, Cognitive psychology, Cognitive science

Top concepts (fields/topics) attached by OpenAlex
Cited by: 1

Total citation count in OpenAlex
Citations by year (recent): 2025: 1

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4404574036
doi	https://doi.org/10.48550/arxiv.2411.12580
ids.doi	https://doi.org/10.48550/arxiv.2411.12580
ids.openalex	https://openalex.org/W4404574036
fwci	0.0
type	preprint
title	Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10181
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9902999997138977
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Natural Language Processing Techniques
topics[1].id	https://openalex.org/T10028
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9610000252723694
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1702
topics[1].subfield.display_name	Artificial Intelligence
topics[1].display_name	Topic Modeling
topics[2].id	https://openalex.org/T10215
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.9495999813079834
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1702
topics[2].subfield.display_name	Artificial Intelligence
topics[2].display_name	Semantic Web and Ontologies
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C41008148
concepts[0].level	0
concepts[0].score	0.5535658597946167
concepts[0].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[0].display_name	Computer science
concepts[1].id	https://openalex.org/C15744967
concepts[1].level	0
concepts[1].score	0.3443676233291626
concepts[1].wikidata	https://www.wikidata.org/wiki/Q9418
concepts[1].display_name	Psychology
concepts[2].id	https://openalex.org/C204321447
concepts[2].level	1
concepts[2].score	0.33190488815307617
concepts[2].wikidata	https://www.wikidata.org/wiki/Q30642
concepts[2].display_name	Natural language processing
concepts[3].id	https://openalex.org/C180747234
concepts[3].level	1
concepts[3].score	0.3264217972755432
concepts[3].wikidata	https://www.wikidata.org/wiki/Q23373
concepts[3].display_name	Cognitive psychology
concepts[4].id	https://openalex.org/C188147891
concepts[4].level	1
concepts[4].score	0.32090920209884644
concepts[4].wikidata	https://www.wikidata.org/wiki/Q147638
concepts[4].display_name	Cognitive science
keywords[0].id	https://openalex.org/keywords/computer-science
keywords[0].score	0.5535658597946167
keywords[0].display_name	Computer science
keywords[1].id	https://openalex.org/keywords/psychology
keywords[1].score	0.3443676233291626
keywords[1].display_name	Psychology
keywords[2].id	https://openalex.org/keywords/natural-language-processing
keywords[2].score	0.33190488815307617
keywords[2].display_name	Natural language processing
keywords[3].id	https://openalex.org/keywords/cognitive-psychology
keywords[3].score	0.3264217972755432
keywords[3].display_name	Cognitive psychology
keywords[4].id	https://openalex.org/keywords/cognitive-science
keywords[4].score	0.32090920209884644
keywords[4].display_name	Cognitive science
language	en
locations[0].id	pmh:oai:arXiv.org:2411.12580
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2411.12580
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2411.12580
locations[1].id	doi:10.48550/arxiv.2411.12580
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2411.12580
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5060010430
authorships[0].author.orcid
authorships[0].author.display_name	Laura Ruis
authorships[0].author_position	first
authorships[0].raw_author_name	Ruis, Laura
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5038732160
authorships[1].author.orcid	https://orcid.org/0000-0001-8138-3792
authorships[1].author.display_name	Maximilian Mozes
authorships[1].author_position	middle
authorships[1].raw_author_name	Mozes, Maximilian
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5027005127
authorships[2].author.orcid	https://orcid.org/0000-0002-3913-1011
authorships[2].author.display_name	Jong Bin Bae
authorships[2].author_position	middle
authorships[2].raw_author_name	Bae, Juhan
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5054907218
authorships[3].author.orcid
authorships[3].author.display_name	Siddhartha Rao Kamalakara
authorships[3].author_position	middle
authorships[3].raw_author_name	Kamalakara, Siddhartha Rao
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5098901824
authorships[4].author.orcid
authorships[4].author.display_name	Dwarak Talupuru
authorships[4].author_position	middle
authorships[4].raw_author_name	Talupuru, Dwarak
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5086062922
authorships[5].author.orcid	https://orcid.org/0000-0002-7122-3311
authorships[5].author.display_name	A. Locatelli
authorships[5].author_position	middle
authorships[5].raw_author_name	Locatelli, Acyr
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5051243028
authorships[6].author.orcid	https://orcid.org/0000-0002-6541-5915
authorships[6].author.display_name	Robert Kirk
authorships[6].author_position	middle
authorships[6].raw_author_name	Kirk, Robert
authorships[6].is_corresponding	False
authorships[7].author.id	https://openalex.org/A5079315903
authorships[7].author.orcid
authorships[7].author.display_name	Tim Rocktäschel
authorships[7].author_position	middle
authorships[7].raw_author_name	Rocktäschel, Tim
authorships[7].is_corresponding	False
authorships[8].author.id	https://openalex.org/A5023508792
authorships[8].author.orcid	https://orcid.org/0000-0003-1164-8809
authorships[8].author.display_name	Edward Grefenstette
authorships[8].author_position	middle
authorships[8].raw_author_name	Grefenstette, Edward
authorships[8].is_corresponding	False
authorships[9].author.id	https://openalex.org/A5018864006
authorships[9].author.orcid	https://orcid.org/0009-0007-3301-7895
authorships[9].author.display_name	Max Bartolo
authorships[9].author_position	last
authorships[9].raw_author_name	Bartolo, Max
authorships[9].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2411.12580
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10181
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9902999997138977
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Natural Language Processing Techniques
related_works	https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052
cited_by_count	1
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	1
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2411.12580
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2411.12580
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2411.12580
primary_location.id	pmh:oai:arXiv.org:2411.12580
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2411.12580
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2411.12580
publication_date	2024-11-19
publication_year	2024
referenced_works_count	0
abstract_inverted_index.a	30, 167, 171, 255, 278, 288
abstract_inverted_index.On	24, 36
abstract_inverted_index.To	84
abstract_inverted_index.We	150, 188
abstract_inverted_index.an	19
abstract_inverted_index.as	216
abstract_inverted_index.by	100
abstract_inverted_index.do	212, 220
abstract_inverted_index.in	12, 15, 64, 200
abstract_inverted_index.is	272
abstract_inverted_index.of	4, 54, 61, 67, 91, 111, 119, 161, 185, 291
abstract_inverted_index.on	51, 157
abstract_inverted_index.or	259
abstract_inverted_index.to	33, 47, 78, 140, 194, 223, 253, 267
abstract_inverted_index.up	199, 215
abstract_inverted_index.us	71
abstract_inverted_index.we	87, 123, 229, 240
abstract_inverted_index.(7B	114
abstract_inverted_index.For	108
abstract_inverted_index.Our	261
abstract_inverted_index.The	0, 58
abstract_inverted_index.and	2, 115, 117, 137, 275
abstract_inverted_index.are	144
abstract_inverted_index.for	131, 146, 163, 206, 235
abstract_inverted_index.has	69, 170
abstract_inverted_index.how	252
abstract_inverted_index.nor	219
abstract_inverted_index.not	213
abstract_inverted_index.on.	107
abstract_inverted_index.one	26
abstract_inverted_index.out	11
abstract_inverted_index.set	82
abstract_inverted_index.the	25, 37, 52, 65, 74, 102, 128, 141, 154, 179, 183, 192, 201, 209, 221, 224, 231, 236, 243, 265, 269
abstract_inverted_index.top	232
abstract_inverted_index.two	109
abstract_inverted_index.use	271
abstract_inverted_index.yet	21
abstract_inverted_index.2.5B	118
abstract_inverted_index.35B)	116
abstract_inverted_index.LLMs	28, 68, 94
abstract_inverted_index.When	228
abstract_inverted_index.been	9
abstract_inverted_index.data	62, 104, 142, 162
abstract_inverted_index.each	164
abstract_inverted_index.find	151, 190
abstract_inverted_index.form	290
abstract_inverted_index.from	72, 285
abstract_inverted_index.gaps	44
abstract_inverted_index.have	8
abstract_inverted_index.kind	90
abstract_inverted_index.like	250, 277
abstract_inverted_index.more	276
abstract_inverted_index.most	202
abstract_inverted_index.rely	106, 156
abstract_inverted_index.same	180
abstract_inverted_index.sets	160
abstract_inverted_index.show	41, 198, 214
abstract_inverted_index.that	143, 191, 242, 264, 281
abstract_inverted_index.they	40, 105
abstract_inverted_index.this	139
abstract_inverted_index.used	63, 77
abstract_inverted_index.what	89, 125
abstract_inverted_index.when	45, 96
abstract_inverted_index.Large	5
abstract_inverted_index.code.	260
abstract_inverted_index.data.	204
abstract_inverted_index.doing	287
abstract_inverted_index.doubt	50
abstract_inverted_index.great	13
abstract_inverted_index.hand,	27, 39
abstract_inverted_index.model	129
abstract_inverted_index.often	169, 197, 246
abstract_inverted_index.other	38
abstract_inverted_index.sheer	59
abstract_inverted_index.sizes	113
abstract_inverted_index.solve	34
abstract_inverted_index.study	88
abstract_inverted_index.task,	181
abstract_inverted_index.tasks	99, 136
abstract_inverted_index.that,	152
abstract_inverted_index.their	55, 120
abstract_inverted_index.this,	86
abstract_inverted_index.three	132
abstract_inverted_index.using	257
abstract_inverted_index.while	153
abstract_inverted_index.Models	7
abstract_inverted_index.across	174
abstract_inverted_index.design	66
abstract_inverted_index.detail	14
abstract_inverted_index.employ	95
abstract_inverted_index.highly	217
abstract_inverted_index.method	75
abstract_inverted_index.models	110, 155, 270
abstract_inverted_index.mostly	158
abstract_inverted_index.obtain	254
abstract_inverted_index.ranked	233
abstract_inverted_index.recent	16
abstract_inverted_index.simple	133
abstract_inverted_index.steps.	227
abstract_inverted_index.unlike	273
abstract_inverted_index.volume	60
abstract_inverted_index.within	178
abstract_inverted_index.years,	17
abstract_inverted_index.ability	32
abstract_inverted_index.answers	193, 210, 222
abstract_inverted_index.casting	49
abstract_inverted_index.confirm	241
abstract_inverted_index.contain	247
abstract_inverted_index.factual	148, 165, 195
abstract_inverted_index.further	189
abstract_inverted_index.general	31
abstract_inverted_index.humans,	48
abstract_inverted_index.measure	79
abstract_inverted_index.outputs	130
abstract_inverted_index.similar	172, 289
abstract_inverted_index.tokens,	122
abstract_inverted_index.usually	211
abstract_inverted_index.However,	205
abstract_inverted_index.Language	6
abstract_inverted_index.applying	73
abstract_inverted_index.approach	266
abstract_inverted_index.compared	46
abstract_inverted_index.contrast	138
abstract_inverted_index.distinct	159
abstract_inverted_index.document	168
abstract_inverted_index.findings	262
abstract_inverted_index.formulae	258
abstract_inverted_index.identify	124
abstract_inverted_index.indicate	263
abstract_inverted_index.overcome	85
abstract_inverted_index.picture.	23
abstract_inverted_index.presence	184
abstract_inverted_index.sketched	10
abstract_inverted_index.solution	256
abstract_inverted_index.strategy	280
abstract_inverted_index.answering	147
abstract_inverted_index.different	112, 175
abstract_inverted_index.documents	126, 234, 245, 286
abstract_inverted_index.influence	127, 173
abstract_inverted_index.knowledge	284
abstract_inverted_index.precluded	70
abstract_inverted_index.problems.	35
abstract_inverted_index.providing	18
abstract_inverted_index.question,	166
abstract_inverted_index.questions	177, 196, 208, 238
abstract_inverted_index.reasoning	43, 98, 135, 176, 207, 226, 237, 268
abstract_inverted_index.indicating	182
abstract_inverted_index.intriguing	20
abstract_inverted_index.knowledge,	249
abstract_inverted_index.knowledge.	187
abstract_inverted_index.performing	97
abstract_inverted_index.procedural	186, 248, 283
abstract_inverted_index.questions.	149
abstract_inverted_index.reasoning.	292
abstract_inverted_index.retrieval,	274
abstract_inverted_index.robustness	53
abstract_inverted_index.strategies	93
abstract_inverted_index.surprising	42
abstract_inverted_index.train-test	81
abstract_inverted_index.conflicting	22
abstract_inverted_index.demonstrate	29
abstract_inverted_index.influential	145, 203, 244
abstract_inverted_index.limitations	3
abstract_inverted_index.pretraining	103, 121
abstract_inverted_index.separation.	83
abstract_inverted_index.strategies.	57
abstract_inverted_index.synthesises	282
abstract_inverted_index.capabilities	1
abstract_inverted_index.characterise	230
abstract_inverted_index.influential,	218
abstract_inverted_index.intermediate	225
abstract_inverted_index.mathematical	134
abstract_inverted_index.demonstrating	251
abstract_inverted_index.generalisable	279
abstract_inverted_index.investigating	101
abstract_inverted_index.traditionally	76
abstract_inverted_index.generalisation	56, 92
abstract_inverted_index.qualitatively,	239
abstract_inverted_index.generalisation:	80
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	10
citation_normalized_percentile