Phi-4 Technical Report Article Swipe

PDF

Marah Abdin , Jyoti Aneja , Harkirat Singh Behl , Sébastien Bubeck , Ronen Eldan , Suriya Gunasekar , Michael R. Harrison , Russell J. Hewett , Mojan Javaheripi , Piero Kauffmann , James R. Lee , Yin Tat Lee , Yuanzhi Li , Weishung Liu , Caio César Teodoro Mendes , Anh Nguyen , Eric Price , Gustavo de Rosa , Olli Saarikivi , Adil Salim , Shital Shah , Xin Wang , Rachel Ward , Yue Wu , Dingli Yu , Cyril Zhang , Yi Zhang ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2412.08905

We present phi-4, a 14-billion parameter language model developed with a training recipe that is centrally focused on data quality. Unlike most language models, where pre-training is based primarily on organic data sources such as web content or code, phi-4 strategically incorporates synthetic data throughout the training process. While previous models in the Phi family largely distill the capabilities of a teacher model (specifically GPT-4), phi-4 substantially surpasses its teacher model on STEM-focused QA capabilities, giving evidence that our data-generation and post-training techniques go beyond distillation. Despite minimal changes to the phi-3 architecture, phi-4 achieves strong performance relative to its size -- especially on reasoning-focused benchmarks -- due to improved data, training curriculum, and innovations in the post-training scheme.

Related Topics

Computer Science

Concepts

Computer science

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2412.08905
PDF: https://arxiv.org/pdf/2412.08905
OA Status: green
Cited By: 16
Related Works: 10
OpenAlex ID: https://openalex.org/W4405354744

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4405354744

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2412.08905

Digital Object Identifier
Title: Phi-4 Technical Report

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-12-12

Full publication date if available
Authors: Marah Abdin, Jyoti Aneja, Harkirat Singh Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael R. Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio César Teodoro Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, Yi Zhang

List of authors in order
Landing page: https://arxiv.org/abs/2412.08905

Publisher landing page
PDF URL: https://arxiv.org/pdf/2412.08905

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2412.08905

Direct OA link when available
Concepts: Computer science

Top concepts (fields/topics) attached by OpenAlex
Cited by: 16

Total citation count in OpenAlex
Citations by year (recent): 2025: 16

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4405354744
doi	https://doi.org/10.48550/arxiv.2412.08905
ids.doi	https://doi.org/10.48550/arxiv.2412.08905
ids.openalex	https://openalex.org/W4405354744
fwci
type	preprint
title	Phi-4 Technical Report
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T13771
topics[0].field.id	https://openalex.org/fields/26
topics[0].field.display_name	Mathematics
topics[0].score	0.04839999973773956
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/2611
topics[0].subfield.display_name	Modeling and Simulation
topics[0].display_name	Advanced Research in Science and Engineering
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C41008148
concepts[0].level	0
concepts[0].score	0.30298149585723877
concepts[0].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[0].display_name	Computer science
keywords[0].id	https://openalex.org/keywords/computer-science
keywords[0].score	0.30298149585723877
keywords[0].display_name	Computer science
language	en
locations[0].id	pmh:oai:arXiv.org:2412.08905
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2412.08905
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2412.08905
locations[1].id	doi:10.48550/arxiv.2412.08905
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2412.08905
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5068559432
authorships[0].author.orcid	https://orcid.org/0000-0002-8021-7108
authorships[0].author.display_name	Marah Abdin
authorships[0].author_position	first
authorships[0].raw_author_name	Abdin, Marah
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5083743598
authorships[1].author.orcid
authorships[1].author.display_name	Jyoti Aneja
authorships[1].author_position	middle
authorships[1].raw_author_name	Aneja, Jyoti
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5081255348
authorships[2].author.orcid
authorships[2].author.display_name	Harkirat Singh Behl
authorships[2].author_position	middle
authorships[2].raw_author_name	Behl, Harkirat
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5021842384
authorships[3].author.orcid	https://orcid.org/0000-0001-9122-4410
authorships[3].author.display_name	Sébastien Bubeck
authorships[3].author_position	middle
authorships[3].raw_author_name	Bubeck, Sébastien
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5011758954
authorships[4].author.orcid	https://orcid.org/0000-0002-0678-741X
authorships[4].author.display_name	Ronen Eldan
authorships[4].author_position	middle
authorships[4].raw_author_name	Eldan, Ronen
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5015603055
authorships[5].author.orcid
authorships[5].author.display_name	Suriya Gunasekar
authorships[5].author_position	middle
authorships[5].raw_author_name	Gunasekar, Suriya
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5016346717
authorships[6].author.orcid	https://orcid.org/0000-0003-1703-9879
authorships[6].author.display_name	Michael R. Harrison
authorships[6].author_position	middle
authorships[6].raw_author_name	Harrison, Michael
authorships[6].is_corresponding	False
authorships[7].author.id	https://openalex.org/A5057195161
authorships[7].author.orcid	https://orcid.org/0000-0001-8944-4705
authorships[7].author.display_name	Russell J. Hewett
authorships[7].author_position	middle
authorships[7].raw_author_name	Hewett, Russell J.
authorships[7].is_corresponding	False
authorships[8].author.id	https://openalex.org/A5052706169
authorships[8].author.orcid	https://orcid.org/0000-0003-4062-8807
authorships[8].author.display_name	Mojan Javaheripi
authorships[8].author_position	middle
authorships[8].raw_author_name	Javaheripi, Mojan
authorships[8].is_corresponding	False
authorships[9].author.id	https://openalex.org/A5092231018
authorships[9].author.orcid
authorships[9].author.display_name	Piero Kauffmann
authorships[9].author_position	middle
authorships[9].raw_author_name	Kauffmann, Piero
authorships[9].is_corresponding	False
authorships[10].author.id	https://openalex.org/A5028879014
authorships[10].author.orcid	https://orcid.org/0000-0002-3512-1617
authorships[10].author.display_name	James R. Lee
authorships[10].author_position	middle
authorships[10].raw_author_name	Lee, James R.
authorships[10].is_corresponding	False
authorships[11].author.id	https://openalex.org/A5032315762
authorships[11].author.orcid
authorships[11].author.display_name	Yin Tat Lee
authorships[11].author_position	middle
authorships[11].raw_author_name	Lee, Yin Tat
authorships[11].is_corresponding	False
authorships[12].author.id	https://openalex.org/A5101879889
authorships[12].author.orcid	https://orcid.org/0000-0002-3584-4735
authorships[12].author.display_name	Yuanzhi Li
authorships[12].author_position	middle
authorships[12].raw_author_name	Li, Yuanzhi
authorships[12].is_corresponding	False
authorships[13].author.id	https://openalex.org/A5038989193
authorships[13].author.orcid
authorships[13].author.display_name	Weishung Liu
authorships[13].author_position	middle
authorships[13].raw_author_name	Liu, Weishung
authorships[13].is_corresponding	False
authorships[14].author.id	https://openalex.org/A5101597983
authorships[14].author.orcid	https://orcid.org/0000-0002-3426-5969
authorships[14].author.display_name	Caio César Teodoro Mendes
authorships[14].author_position	middle
authorships[14].raw_author_name	Mendes, Caio C. T.
authorships[14].is_corresponding	False
authorships[15].author.id	https://openalex.org/A5089006585
authorships[15].author.orcid	https://orcid.org/0000-0002-1449-211X
authorships[15].author.display_name	Anh Nguyen
authorships[15].author_position	middle
authorships[15].raw_author_name	Nguyen, Anh
authorships[15].is_corresponding	False
authorships[16].author.id	https://openalex.org/A5083990461
authorships[16].author.orcid	https://orcid.org/0000-0002-3480-8054
authorships[16].author.display_name	Eric Price
authorships[16].author_position	middle
authorships[16].raw_author_name	Price, Eric
authorships[16].is_corresponding	False
authorships[17].author.id	https://openalex.org/A5108301210
authorships[17].author.orcid
authorships[17].author.display_name	Gustavo de Rosa
authorships[17].author_position	middle
authorships[17].raw_author_name	de Rosa, Gustavo
authorships[17].is_corresponding	False
authorships[18].author.id	https://openalex.org/A5001454502
authorships[18].author.orcid	https://orcid.org/0000-0001-7596-4734
authorships[18].author.display_name	Olli Saarikivi
authorships[18].author_position	middle
authorships[18].raw_author_name	Saarikivi, Olli
authorships[18].is_corresponding	False
authorships[19].author.id	https://openalex.org/A5044733184
authorships[19].author.orcid	https://orcid.org/0000-0002-3829-8864
authorships[19].author.display_name	Adil Salim
authorships[19].author_position	middle
authorships[19].raw_author_name	Salim, Adil
authorships[19].is_corresponding	False
authorships[20].author.id	https://openalex.org/A5101671828
authorships[20].author.orcid	https://orcid.org/0000-0002-8995-1704
authorships[20].author.display_name	Shital Shah
authorships[20].author_position	middle
authorships[20].raw_author_name	Shah, Shital
authorships[20].is_corresponding	False
authorships[21].author.id	https://openalex.org/A5115695322
authorships[21].author.orcid	https://orcid.org/0000-0002-6071-8837
authorships[21].author.display_name	Xin Wang
authorships[21].author_position	middle
authorships[21].raw_author_name	Wang, Xin
authorships[21].is_corresponding	False
authorships[22].author.id	https://openalex.org/A5071181313
authorships[22].author.orcid
authorships[22].author.display_name	Rachel Ward
authorships[22].author_position	middle
authorships[22].raw_author_name	Ward, Rachel
authorships[22].is_corresponding	False
authorships[23].author.id	https://openalex.org/A5051234612
authorships[23].author.orcid	https://orcid.org/0000-0003-2387-4766
authorships[23].author.display_name	Yue Wu
authorships[23].author_position	middle
authorships[23].raw_author_name	Wu, Yue
authorships[23].is_corresponding	False
authorships[24].author.id	https://openalex.org/A5034102301
authorships[24].author.orcid	https://orcid.org/0000-0003-3177-8942
authorships[24].author.display_name	Dingli Yu
authorships[24].author_position	middle
authorships[24].raw_author_name	Yu, Dingli
authorships[24].is_corresponding	False
authorships[25].author.id	https://openalex.org/A5043228083
authorships[25].author.orcid	https://orcid.org/0000-0002-8707-1279
authorships[25].author.display_name	Cyril Zhang
authorships[25].author_position	middle
authorships[25].raw_author_name	Zhang, Cyril
authorships[25].is_corresponding	False
authorships[26].author.id	https://openalex.org/A5100388328
authorships[26].author.orcid	https://orcid.org/0000-0003-3795-4812
authorships[26].author.display_name	Yi Zhang
authorships[26].author_position	last
authorships[26].raw_author_name	Zhang, Yi
authorships[26].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2412.08905
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Phi-4 Technical Report
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T13771
primary_topic.field.id	https://openalex.org/fields/26
primary_topic.field.display_name	Mathematics
primary_topic.score	0.04839999973773956
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/2611
primary_topic.subfield.display_name	Modeling and Simulation
primary_topic.display_name	Advanced Research in Science and Engineering
related_works	https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052
cited_by_count	16
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	16
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2412.08905
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2412.08905
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2412.08905
primary_location.id	pmh:oai:arXiv.org:2412.08905
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2412.08905
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2412.08905
publication_date	2024-12-12
publication_year	2024
referenced_works_count	0
abstract_inverted_index.a	3, 10, 60
abstract_inverted_index.--	101, 106
abstract_inverted_index.QA	73
abstract_inverted_index.We	0
abstract_inverted_index.as	34
abstract_inverted_index.go	83
abstract_inverted_index.in	51, 115
abstract_inverted_index.is	14, 26
abstract_inverted_index.of	59
abstract_inverted_index.on	17, 29, 71, 103
abstract_inverted_index.or	37
abstract_inverted_index.to	89, 98, 108
abstract_inverted_index.Phi	53
abstract_inverted_index.and	80, 113
abstract_inverted_index.due	107
abstract_inverted_index.its	68, 99
abstract_inverted_index.our	78
abstract_inverted_index.the	45, 52, 57, 90, 116
abstract_inverted_index.web	35
abstract_inverted_index.data	18, 31, 43
abstract_inverted_index.most	21
abstract_inverted_index.size	100
abstract_inverted_index.such	33
abstract_inverted_index.that	13, 77
abstract_inverted_index.with	9
abstract_inverted_index.While	48
abstract_inverted_index.based	27
abstract_inverted_index.code,	38
abstract_inverted_index.data,	110
abstract_inverted_index.model	7, 62, 70
abstract_inverted_index.phi-3	91
abstract_inverted_index.phi-4	39, 65, 93
abstract_inverted_index.where	24
abstract_inverted_index.Unlike	20
abstract_inverted_index.beyond	84
abstract_inverted_index.family	54
abstract_inverted_index.giving	75
abstract_inverted_index.models	50
abstract_inverted_index.phi-4,	2
abstract_inverted_index.recipe	12
abstract_inverted_index.strong	95
abstract_inverted_index.Despite	86
abstract_inverted_index.GPT-4),	64
abstract_inverted_index.changes	88
abstract_inverted_index.content	36
abstract_inverted_index.distill	56
abstract_inverted_index.focused	16
abstract_inverted_index.largely	55
abstract_inverted_index.minimal	87
abstract_inverted_index.models,	23
abstract_inverted_index.organic	30
abstract_inverted_index.present	1
abstract_inverted_index.scheme.	118
abstract_inverted_index.sources	32
abstract_inverted_index.teacher	61, 69
abstract_inverted_index.achieves	94
abstract_inverted_index.evidence	76
abstract_inverted_index.improved	109
abstract_inverted_index.language	6, 22
abstract_inverted_index.previous	49
abstract_inverted_index.process.	47
abstract_inverted_index.quality.	19
abstract_inverted_index.relative	97
abstract_inverted_index.training	11, 46, 111
abstract_inverted_index.centrally	15
abstract_inverted_index.developed	8
abstract_inverted_index.parameter	5
abstract_inverted_index.primarily	28
abstract_inverted_index.surpasses	67
abstract_inverted_index.synthetic	42
abstract_inverted_index.14-billion	4
abstract_inverted_index.benchmarks	105
abstract_inverted_index.especially	102
abstract_inverted_index.techniques	82
abstract_inverted_index.throughout	44
abstract_inverted_index.curriculum,	112
abstract_inverted_index.innovations	114
abstract_inverted_index.performance	96
abstract_inverted_index.STEM-focused	72
abstract_inverted_index.capabilities	58
abstract_inverted_index.incorporates	41
abstract_inverted_index.pre-training	25
abstract_inverted_index.(specifically	63
abstract_inverted_index.architecture,	92
abstract_inverted_index.capabilities,	74
abstract_inverted_index.distillation.	85
abstract_inverted_index.post-training	81, 117
abstract_inverted_index.strategically	40
abstract_inverted_index.substantially	66
abstract_inverted_index.data-generation	79
abstract_inverted_index.reasoning-focused	104
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	27
citation_normalized_percentile