LLM360: Towards Fully Transparent Open-Source LLMs Article Swipe

PDF

Zhengzhong Liu , Aurick Qiao , Willie Neiswanger , Hongyi Wang , Bowen Tan , Tianhua Tao , Junbo Li , Yuqi Wang , Suqi Sun , Omkar Pangarkar , Richard E. Fan , Yi Gu , Victor S. Miller , Yonghao Zhuang , Guowei He , Haonan Li , Fajri Koto , Liping Tang , Nikhil Ranjan , Zhiqiang Shen , Xuguang Ren , Roberto Iriondo , Cun Mu , Zhiting Hu , Mark Schulze , Preslav Nakov , Tim Baldwin , Eric P. Xing ·

YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2312.06550

The recent surge in open-source Large Language Models (LLMs), such as LLaMA, Falcon, and Mistral, provides diverse options for AI practitioners and researchers. However, most LLMs have only released partial artifacts, such as the final model weights or inference code, and technical reports increasingly limit their scope to high-level design choices and surface statistics. These choices hinder progress in the field by degrading transparency into the training of LLMs and forcing teams to rediscover many details in the training process. We present LLM360, an initiative to fully open-source LLMs, which advocates for all training code and data, model checkpoints, and intermediate results to be made available to the community. The goal of LLM360 is to support open and collaborative AI research by making the end-to-end LLM training process transparent and reproducible by everyone. As a first step of LLM360, we release two 7B parameter LLMs pre-trained from scratch, Amber and CrystalCoder, including their training code, data, intermediate checkpoints, and analyses (at https://www.llm360.ai). We are committed to continually pushing the boundaries of LLMs through this open-source effort. More large-scale and stronger models are underway and will be released in the future.

Related Topics

Concepts

Transparency (behavior) Forcing (mathematics) Computer science Process (computing) Computer security Programming language Climatology Geology

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2312.06550
PDF: https://arxiv.org/pdf/2312.06550
OA Status: green
Cited By: 3
Related Works: 10
OpenAlex ID: https://openalex.org/W4389708455

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4389708455

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2312.06550

Digital Object Identifier
Title: LLM360: Towards Fully Transparent Open-Source LLMs

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2023

Year of publication
Publication date: 2023-12-11

Full publication date if available
Authors: Zhengzhong Liu, Aurick Qiao, Willie Neiswanger, Hongyi Wang, Bowen Tan, Tianhua Tao, Junbo Li, Yuqi Wang, Suqi Sun, Omkar Pangarkar, Richard E. Fan, Yi Gu, Victor S. Miller, Yonghao Zhuang, Guowei He, Haonan Li, Fajri Koto, Liping Tang, Nikhil Ranjan, Zhiqiang Shen, Xuguang Ren, Roberto Iriondo, Cun Mu, Zhiting Hu, Mark Schulze, Preslav Nakov, Tim Baldwin, Eric P. Xing

List of authors in order
Landing page: https://arxiv.org/abs/2312.06550

Publisher landing page
PDF URL: https://arxiv.org/pdf/2312.06550

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2312.06550

Direct OA link when available
Concepts: Transparency (behavior), Forcing (mathematics), Computer science, Process (computing), Computer security, Programming language, Climatology, Geology

Top concepts (fields/topics) attached by OpenAlex
Cited by: 3

Total citation count in OpenAlex
Citations by year (recent): 2024: 3

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4389708455
doi	https://doi.org/10.48550/arxiv.2312.06550
ids.doi	https://doi.org/10.48550/arxiv.2312.06550
ids.openalex	https://openalex.org/W4389708455
fwci
type	preprint
title	LLM360: Towards Fully Transparent Open-Source LLMs
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10028
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9940000176429749
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Topic Modeling
topics[1].id	https://openalex.org/T10181
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9717000126838684
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1702
topics[1].subfield.display_name	Artificial Intelligence
topics[1].display_name	Natural Language Processing Techniques
topics[2].id	https://openalex.org/T11948
topics[2].field.id	https://openalex.org/fields/25
topics[2].field.display_name	Materials Science
topics[2].score	0.9570000171661377
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/2505
topics[2].subfield.display_name	Materials Chemistry
topics[2].display_name	Machine Learning in Materials Science
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C2780233690
concepts[0].level	2
concepts[0].score	0.5700200200080872
concepts[0].wikidata	https://www.wikidata.org/wiki/Q535347
concepts[0].display_name	Transparency (behavior)
concepts[1].id	https://openalex.org/C197115733
concepts[1].level	2
concepts[1].score	0.4852044880390167
concepts[1].wikidata	https://www.wikidata.org/wiki/Q1003136
concepts[1].display_name	Forcing (mathematics)
concepts[2].id	https://openalex.org/C41008148
concepts[2].level	0
concepts[2].score	0.46021831035614014
concepts[2].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[2].display_name	Computer science
concepts[3].id	https://openalex.org/C98045186
concepts[3].level	2
concepts[3].score	0.4356541633605957
concepts[3].wikidata	https://www.wikidata.org/wiki/Q205663
concepts[3].display_name	Process (computing)
concepts[4].id	https://openalex.org/C38652104
concepts[4].level	1
concepts[4].score	0.2464621663093567
concepts[4].wikidata	https://www.wikidata.org/wiki/Q3510521
concepts[4].display_name	Computer security
concepts[5].id	https://openalex.org/C199360897
concepts[5].level	1
concepts[5].score	0.11513042449951172
concepts[5].wikidata	https://www.wikidata.org/wiki/Q9143
concepts[5].display_name	Programming language
concepts[6].id	https://openalex.org/C49204034
concepts[6].level	1
concepts[6].score	0.0
concepts[6].wikidata	https://www.wikidata.org/wiki/Q52139
concepts[6].display_name	Climatology
concepts[7].id	https://openalex.org/C127313418
concepts[7].level	0
concepts[7].score	0.0
concepts[7].wikidata	https://www.wikidata.org/wiki/Q1069
concepts[7].display_name	Geology
keywords[0].id	https://openalex.org/keywords/transparency
keywords[0].score	0.5700200200080872
keywords[0].display_name	Transparency (behavior)
keywords[1].id	https://openalex.org/keywords/forcing
keywords[1].score	0.4852044880390167
keywords[1].display_name	Forcing (mathematics)
keywords[2].id	https://openalex.org/keywords/computer-science
keywords[2].score	0.46021831035614014
keywords[2].display_name	Computer science
keywords[3].id	https://openalex.org/keywords/process
keywords[3].score	0.4356541633605957
keywords[3].display_name	Process (computing)
keywords[4].id	https://openalex.org/keywords/computer-security
keywords[4].score	0.2464621663093567
keywords[4].display_name	Computer security
keywords[5].id	https://openalex.org/keywords/programming-language
keywords[5].score	0.11513042449951172
keywords[5].display_name	Programming language
language	en
locations[0].id	pmh:oai:arXiv.org:2312.06550
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2312.06550
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2312.06550
locations[1].id	doi:10.48550/arxiv.2312.06550
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2312.06550
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5029648803
authorships[0].author.orcid
authorships[0].author.display_name	Zhengzhong Liu
authorships[0].author_position	first
authorships[0].raw_author_name	Liu, Zhengzhong
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5073820927
authorships[1].author.orcid	https://orcid.org/0009-0004-9119-8696
authorships[1].author.display_name	Aurick Qiao
authorships[1].author_position	middle
authorships[1].raw_author_name	Qiao, Aurick
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5073310637
authorships[2].author.orcid	https://orcid.org/0000-0002-9619-5572
authorships[2].author.display_name	Willie Neiswanger
authorships[2].author_position	middle
authorships[2].raw_author_name	Neiswanger, Willie
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5100701032
authorships[3].author.orcid	https://orcid.org/0000-0002-1076-1140
authorships[3].author.display_name	Hongyi Wang
authorships[3].author_position	middle
authorships[3].raw_author_name	Wang, Hongyi
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5101403295
authorships[4].author.orcid	https://orcid.org/0000-0003-3036-2881
authorships[4].author.display_name	Bowen Tan
authorships[4].author_position	middle
authorships[4].raw_author_name	Tan, Bowen
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5081822522
authorships[5].author.orcid
authorships[5].author.display_name	Tianhua Tao
authorships[5].author_position	middle
authorships[5].raw_author_name	Tao, Tianhua
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5101548643
authorships[6].author.orcid
authorships[6].author.display_name	Junbo Li
authorships[6].author_position	middle
authorships[6].raw_author_name	Li, Junbo
authorships[6].is_corresponding	False
authorships[7].author.id	https://openalex.org/A5100451661
authorships[7].author.orcid	https://orcid.org/0000-0001-6230-0330
authorships[7].author.display_name	Yuqi Wang
authorships[7].author_position	middle
authorships[7].raw_author_name	Wang, Yuqi
authorships[7].is_corresponding	False
authorships[8].author.id	https://openalex.org/A5101279172
authorships[8].author.orcid
authorships[8].author.display_name	Suqi Sun
authorships[8].author_position	middle
authorships[8].raw_author_name	Sun, Suqi
authorships[8].is_corresponding	False
authorships[9].author.id	https://openalex.org/A5093487079
authorships[9].author.orcid
authorships[9].author.display_name	Omkar Pangarkar
authorships[9].author_position	middle
authorships[9].raw_author_name	Pangarkar, Omkar
authorships[9].is_corresponding	False
authorships[10].author.id	https://openalex.org/A5063971801
authorships[10].author.orcid	https://orcid.org/0000-0001-7716-0377
authorships[10].author.display_name	Richard E. Fan
authorships[10].author_position	middle
authorships[10].raw_author_name	Fan, Richard
authorships[10].is_corresponding	False
authorships[11].author.id	https://openalex.org/A5113681294
authorships[11].author.orcid
authorships[11].author.display_name	Yi Gu
authorships[11].author_position	middle
authorships[11].raw_author_name	Gu, Yi
authorships[11].is_corresponding	False
authorships[12].author.id	https://openalex.org/A5085804010
authorships[12].author.orcid
authorships[12].author.display_name	Victor S. Miller
authorships[12].author_position	middle
authorships[12].raw_author_name	Miller, Victor
authorships[12].is_corresponding	False
authorships[13].author.id	https://openalex.org/A5076407338
authorships[13].author.orcid
authorships[13].author.display_name	Yonghao Zhuang
authorships[13].author_position	middle
authorships[13].raw_author_name	Zhuang, Yonghao
authorships[13].is_corresponding	False
authorships[14].author.id	https://openalex.org/A5102088247
authorships[14].author.orcid
authorships[14].author.display_name	Guowei He
authorships[14].author_position	middle
authorships[14].raw_author_name	He, Guowei
authorships[14].is_corresponding	False
authorships[15].author.id	https://openalex.org/A5107274823
authorships[15].author.orcid	https://orcid.org/0000-0003-0357-0888
authorships[15].author.display_name	Haonan Li
authorships[15].author_position	middle
authorships[15].raw_author_name	Li, Haonan
authorships[15].is_corresponding	False
authorships[16].author.id	https://openalex.org/A5065822589
authorships[16].author.orcid
authorships[16].author.display_name	Fajri Koto
authorships[16].author_position	middle
authorships[16].raw_author_name	Koto, Fajri
authorships[16].is_corresponding	False
authorships[17].author.id	https://openalex.org/A5000863621
authorships[17].author.orcid	https://orcid.org/0000-0002-3393-7073
authorships[17].author.display_name	Liping Tang
authorships[17].author_position	middle
authorships[17].raw_author_name	Tang, Liping
authorships[17].is_corresponding	False
authorships[18].author.id	https://openalex.org/A5046696909
authorships[18].author.orcid
authorships[18].author.display_name	Nikhil Ranjan
authorships[18].author_position	middle
authorships[18].raw_author_name	Ranjan, Nikhil
authorships[18].is_corresponding	False
authorships[19].author.id	https://openalex.org/A5066530136
authorships[19].author.orcid	https://orcid.org/0000-0002-4560-5092
authorships[19].author.display_name	Zhiqiang Shen
authorships[19].author_position	middle
authorships[19].raw_author_name	Shen, Zhiqiang
authorships[19].is_corresponding	False
authorships[20].author.id	https://openalex.org/A5104213553
authorships[20].author.orcid
authorships[20].author.display_name	Xuguang Ren
authorships[20].author_position	middle
authorships[20].raw_author_name	Ren, Xuguang
authorships[20].is_corresponding	False
authorships[21].author.id	https://openalex.org/A5093487080
authorships[21].author.orcid
authorships[21].author.display_name	Roberto Iriondo
authorships[21].author_position	middle
authorships[21].raw_author_name	Iriondo, Roberto
authorships[21].is_corresponding	False
authorships[22].author.id	https://openalex.org/A5113070853
authorships[22].author.orcid
authorships[22].author.display_name	Cun Mu
authorships[22].author_position	middle
authorships[22].raw_author_name	Mu, Cun
authorships[22].is_corresponding	False
authorships[23].author.id	https://openalex.org/A5085608858
authorships[23].author.orcid	https://orcid.org/0000-0002-6239-5031
authorships[23].author.display_name	Zhiting Hu
authorships[23].author_position	middle
authorships[23].raw_author_name	Hu, Zhiting
authorships[23].is_corresponding	False
authorships[24].author.id	https://openalex.org/A5074925398
authorships[24].author.orcid	https://orcid.org/0000-0003-0438-4091
authorships[24].author.display_name	Mark Schulze
authorships[24].author_position	middle
authorships[24].raw_author_name	Schulze, Mark
authorships[24].is_corresponding	False
authorships[25].author.id	https://openalex.org/A5012055259
authorships[25].author.orcid	https://orcid.org/0000-0002-3600-1510
authorships[25].author.display_name	Preslav Nakov
authorships[25].author_position	middle
authorships[25].raw_author_name	Nakov, Preslav
authorships[25].is_corresponding	False
authorships[26].author.id	https://openalex.org/A5108603416
authorships[26].author.orcid
authorships[26].author.display_name	Tim Baldwin
authorships[26].author_position	middle
authorships[26].raw_author_name	Baldwin, Tim
authorships[26].is_corresponding	False
authorships[27].author.id	https://openalex.org/A5009547049
authorships[27].author.orcid	https://orcid.org/0009-0005-9158-4201
authorships[27].author.display_name	Eric P. Xing
authorships[27].author_position	last
authorships[27].raw_author_name	Xing, Eric P.
authorships[27].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2312.06550
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	LLM360: Towards Fully Transparent Open-Source LLMs
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10028
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9940000176429749
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Topic Modeling
related_works	https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W4382930947, https://openalex.org/W3081288631, https://openalex.org/W3152382318, https://openalex.org/W3004686567, https://openalex.org/W2738656338, https://openalex.org/W2603787370, https://openalex.org/W2555400967, https://openalex.org/W3214759741
cited_by_count	3
counts_by_year[0].year	2024
counts_by_year[0].cited_by_count	3
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2312.06550
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2312.06550
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2312.06550
primary_location.id	pmh:oai:arXiv.org:2312.06550
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2312.06550
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2312.06550
publication_date	2023-12-11
publication_year	2023
referenced_works_count	0
abstract_inverted_index.a	134
abstract_inverted_index.7B	142
abstract_inverted_index.AI	19, 119
abstract_inverted_index.As	133
abstract_inverted_index.We	80, 162
abstract_inverted_index.an	83
abstract_inverted_index.as	10, 32
abstract_inverted_index.be	103, 185
abstract_inverted_index.by	61, 121, 131
abstract_inverted_index.in	3, 58, 76, 187
abstract_inverted_index.is	113
abstract_inverted_index.of	67, 111, 137, 170
abstract_inverted_index.or	37
abstract_inverted_index.to	47, 72, 85, 102, 106, 114, 165
abstract_inverted_index.we	139
abstract_inverted_index.(at	160
abstract_inverted_index.LLM	125
abstract_inverted_index.The	0, 109
abstract_inverted_index.all	92
abstract_inverted_index.and	13, 21, 40, 51, 69, 95, 99, 117, 129, 149, 158, 178, 183
abstract_inverted_index.are	163, 181
abstract_inverted_index.for	18, 91
abstract_inverted_index.the	33, 59, 65, 77, 107, 123, 168, 188
abstract_inverted_index.two	141
abstract_inverted_index.LLMs	25, 68, 144, 171
abstract_inverted_index.More	176
abstract_inverted_index.code	94
abstract_inverted_index.from	146
abstract_inverted_index.goal	110
abstract_inverted_index.have	26
abstract_inverted_index.into	64
abstract_inverted_index.made	104
abstract_inverted_index.many	74
abstract_inverted_index.most	24
abstract_inverted_index.only	27
abstract_inverted_index.open	116
abstract_inverted_index.step	136
abstract_inverted_index.such	9, 31
abstract_inverted_index.this	173
abstract_inverted_index.will	184
abstract_inverted_index.Amber	148
abstract_inverted_index.LLMs,	88
abstract_inverted_index.Large	5
abstract_inverted_index.These	54
abstract_inverted_index.code,	39, 154
abstract_inverted_index.data,	96, 155
abstract_inverted_index.field	60
abstract_inverted_index.final	34
abstract_inverted_index.first	135
abstract_inverted_index.fully	86
abstract_inverted_index.limit	44
abstract_inverted_index.model	35, 97
abstract_inverted_index.scope	46
abstract_inverted_index.surge	2
abstract_inverted_index.teams	71
abstract_inverted_index.their	45, 152
abstract_inverted_index.which	89
abstract_inverted_index.LLM360	112
abstract_inverted_index.LLaMA,	11
abstract_inverted_index.Models	7
abstract_inverted_index.design	49
abstract_inverted_index.hinder	56
abstract_inverted_index.making	122
abstract_inverted_index.models	180
abstract_inverted_index.recent	1
abstract_inverted_index.(LLMs),	8
abstract_inverted_index.Falcon,	12
abstract_inverted_index.LLM360,	82, 138
abstract_inverted_index.choices	50, 55
abstract_inverted_index.details	75
abstract_inverted_index.diverse	16
abstract_inverted_index.effort.	175
abstract_inverted_index.forcing	70
abstract_inverted_index.future.	189
abstract_inverted_index.options	17
abstract_inverted_index.partial	29
abstract_inverted_index.present	81
abstract_inverted_index.process	127
abstract_inverted_index.pushing	167
abstract_inverted_index.release	140
abstract_inverted_index.reports	42
abstract_inverted_index.results	101
abstract_inverted_index.support	115
abstract_inverted_index.surface	52
abstract_inverted_index.through	172
abstract_inverted_index.weights	36
abstract_inverted_index.However,	23
abstract_inverted_index.Language	6
abstract_inverted_index.Mistral,	14
abstract_inverted_index.analyses	159
abstract_inverted_index.process.	79
abstract_inverted_index.progress	57
abstract_inverted_index.provides	15
abstract_inverted_index.released	28, 186
abstract_inverted_index.research	120
abstract_inverted_index.scratch,	147
abstract_inverted_index.stronger	179
abstract_inverted_index.training	66, 78, 93, 126, 153
abstract_inverted_index.underway	182
abstract_inverted_index.advocates	90
abstract_inverted_index.available	105
abstract_inverted_index.committed	164
abstract_inverted_index.degrading	62
abstract_inverted_index.everyone.	132
abstract_inverted_index.including	151
abstract_inverted_index.inference	38
abstract_inverted_index.parameter	143
abstract_inverted_index.technical	41
abstract_inverted_index.artifacts,	30
abstract_inverted_index.boundaries	169
abstract_inverted_index.community.	108
abstract_inverted_index.end-to-end	124
abstract_inverted_index.high-level	48
abstract_inverted_index.initiative	84
abstract_inverted_index.rediscover	73
abstract_inverted_index.continually	166
abstract_inverted_index.large-scale	177
abstract_inverted_index.open-source	4, 87, 174
abstract_inverted_index.pre-trained	145
abstract_inverted_index.statistics.	53
abstract_inverted_index.transparent	128
abstract_inverted_index.checkpoints,	98, 157
abstract_inverted_index.increasingly	43
abstract_inverted_index.intermediate	100, 156
abstract_inverted_index.reproducible	130
abstract_inverted_index.researchers.	22
abstract_inverted_index.transparency	63
abstract_inverted_index.CrystalCoder,	150
abstract_inverted_index.collaborative	118
abstract_inverted_index.practitioners	20
abstract_inverted_index.https://www.llm360.ai).	161
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	28
citation_normalized_percentile