On Optimizing the Communication of Model Parallelism Article Swipe

PDF

Yonghao Zhuang , Hexu Zhao , Lianmin Zheng , Zhuohan Li , Eric P. Xing , Qirong Ho , Joseph E. Gonzalez , Ion Stoica , Hao Zhang ·

YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2211.05322

We study a novel and important communication pattern in large-scale model-parallel deep learning (DL), which we call cross-mesh resharding. This pattern emerges when the two paradigms of model parallelism - intra-operator and inter-operator parallelism - are combined to support large models on large clusters. In cross-mesh resharding, a sharded tensor needs to be sent from a source device mesh to a destination device mesh, on which the tensor may be distributed with the same or different layouts. We formalize this as a many-to-many multicast communication problem, and show that existing approaches either are sub-optimal or do not generalize to different network topologies or tensor layouts, which result from different model architectures and parallelism strategies. We then propose two contributions to address cross-mesh resharding: an efficient broadcast-based communication system, and an "overlapping-friendly" pipeline schedule. On microbenchmarks, our overall system outperforms existing ones by up to 10x across various tensor and mesh layouts. On end-to-end training of two large models, GPT-3 and U-Transformer, we improve throughput by 10% and 50%, respectively.

Related Topics

Theoretical Computer Science

Concepts

Computer science Multicast Parallelism (grammar) Parallel computing Distributed computing Network topology Models of communication Pipeline (software) Operator (biology) Schedule Theoretical computer science Computer network Programming language Operating system Gene Repressor Chemistry Biochemistry Communication Sociology Transcription factor

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2211.05322
PDF: https://arxiv.org/pdf/2211.05322
OA Status: green
Cited By: 3
Related Works: 10
OpenAlex ID: https://openalex.org/W4308828369

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4308828369

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2211.05322

Digital Object Identifier
Title: On Optimizing the Communication of Model Parallelism

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2022

Year of publication
Publication date: 2022-11-10

Full publication date if available
Authors: Yonghao Zhuang, Hexu Zhao, Lianmin Zheng, Zhuohan Li, Eric P. Xing, Qirong Ho, Joseph E. Gonzalez, Ion Stoica, Hao Zhang

List of authors in order
Landing page: https://arxiv.org/abs/2211.05322

Publisher landing page
PDF URL: https://arxiv.org/pdf/2211.05322

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2211.05322

Direct OA link when available
Concepts: Computer science, Multicast, Parallelism (grammar), Parallel computing, Distributed computing, Network topology, Models of communication, Pipeline (software), Operator (biology), Schedule, Theoretical computer science, Computer network, Programming language, Operating system, Gene, Repressor, Chemistry, Biochemistry, Communication, Sociology, Transcription factor

Top concepts (fields/topics) attached by OpenAlex
Cited by: 3

Total citation count in OpenAlex
Citations by year (recent): 2025: 1, 2024: 2

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4308828369
doi	https://doi.org/10.48550/arxiv.2211.05322
ids.doi	https://doi.org/10.48550/arxiv.2211.05322
ids.openalex	https://openalex.org/W4308828369
fwci
type	preprint
title	On Optimizing the Communication of Model Parallelism
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10036
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9958000183105469
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1707
topics[0].subfield.display_name	Computer Vision and Pattern Recognition
topics[0].display_name	Advanced Neural Network Applications
topics[1].id	https://openalex.org/T12808
topics[1].field.id	https://openalex.org/fields/22
topics[1].field.display_name	Engineering
topics[1].score	0.994700014591217
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/2208
topics[1].subfield.display_name	Electrical and Electronic Engineering
topics[1].display_name	Ferroelectric and Negative Capacitance Devices
topics[2].id	https://openalex.org/T10054
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.993399977684021
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1708
topics[2].subfield.display_name	Hardware and Architecture
topics[2].display_name	Parallel Computing and Optimization Techniques
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C41008148
concepts[0].level	0
concepts[0].score	0.8460873365402222
concepts[0].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[0].display_name	Computer science
concepts[1].id	https://openalex.org/C32295351
concepts[1].level	2
concepts[1].score	0.7035799026489258
concepts[1].wikidata	https://www.wikidata.org/wiki/Q899288
concepts[1].display_name	Multicast
concepts[2].id	https://openalex.org/C2781172179
concepts[2].level	2
concepts[2].score	0.683171808719635
concepts[2].wikidata	https://www.wikidata.org/wiki/Q853109
concepts[2].display_name	Parallelism (grammar)
concepts[3].id	https://openalex.org/C173608175
concepts[3].level	1
concepts[3].score	0.6204637289047241
concepts[3].wikidata	https://www.wikidata.org/wiki/Q232661
concepts[3].display_name	Parallel computing
concepts[4].id	https://openalex.org/C120314980
concepts[4].level	1
concepts[4].score	0.4838391840457916
concepts[4].wikidata	https://www.wikidata.org/wiki/Q180634
concepts[4].display_name	Distributed computing
concepts[5].id	https://openalex.org/C199845137
concepts[5].level	2
concepts[5].score	0.4813588261604309
concepts[5].wikidata	https://www.wikidata.org/wiki/Q145490
concepts[5].display_name	Network topology
concepts[6].id	https://openalex.org/C158156997
concepts[6].level	2
concepts[6].score	0.4491894841194153
concepts[6].wikidata	https://www.wikidata.org/wiki/Q1416645
concepts[6].display_name	Models of communication
concepts[7].id	https://openalex.org/C43521106
concepts[7].level	2
concepts[7].score	0.4304533898830414
concepts[7].wikidata	https://www.wikidata.org/wiki/Q2165493
concepts[7].display_name	Pipeline (software)
concepts[8].id	https://openalex.org/C17020691
concepts[8].level	5
concepts[8].score	0.41290047764778137
concepts[8].wikidata	https://www.wikidata.org/wiki/Q139677
concepts[8].display_name	Operator (biology)
concepts[9].id	https://openalex.org/C68387754
concepts[9].level	2
concepts[9].score	0.4122838079929352
concepts[9].wikidata	https://www.wikidata.org/wiki/Q7271585
concepts[9].display_name	Schedule
concepts[10].id	https://openalex.org/C80444323
concepts[10].level	1
concepts[10].score	0.3747384250164032
concepts[10].wikidata	https://www.wikidata.org/wiki/Q2878974
concepts[10].display_name	Theoretical computer science
concepts[11].id	https://openalex.org/C31258907
concepts[11].level	1
concepts[11].score	0.18045195937156677
concepts[11].wikidata	https://www.wikidata.org/wiki/Q1301371
concepts[11].display_name	Computer network
concepts[12].id	https://openalex.org/C199360897
concepts[12].level	1
concepts[12].score	0.10308873653411865
concepts[12].wikidata	https://www.wikidata.org/wiki/Q9143
concepts[12].display_name	Programming language
concepts[13].id	https://openalex.org/C111919701
concepts[13].level	1
concepts[13].score	0.07229351997375488
concepts[13].wikidata	https://www.wikidata.org/wiki/Q9135
concepts[13].display_name	Operating system
concepts[14].id	https://openalex.org/C104317684
concepts[14].level	2
concepts[14].score	0.0
concepts[14].wikidata	https://www.wikidata.org/wiki/Q7187
concepts[14].display_name	Gene
concepts[15].id	https://openalex.org/C158448853
concepts[15].level	4
concepts[15].score	0.0
concepts[15].wikidata	https://www.wikidata.org/wiki/Q425218
concepts[15].display_name	Repressor
concepts[16].id	https://openalex.org/C185592680
concepts[16].level	0
concepts[16].score	0.0
concepts[16].wikidata	https://www.wikidata.org/wiki/Q2329
concepts[16].display_name	Chemistry
concepts[17].id	https://openalex.org/C55493867
concepts[17].level	1
concepts[17].score	0.0
concepts[17].wikidata	https://www.wikidata.org/wiki/Q7094
concepts[17].display_name	Biochemistry
concepts[18].id	https://openalex.org/C46312422
concepts[18].level	1
concepts[18].score	0.0
concepts[18].wikidata	https://www.wikidata.org/wiki/Q11024
concepts[18].display_name	Communication
concepts[19].id	https://openalex.org/C144024400
concepts[19].level	0
concepts[19].score	0.0
concepts[19].wikidata	https://www.wikidata.org/wiki/Q21201
concepts[19].display_name	Sociology
concepts[20].id	https://openalex.org/C86339819
concepts[20].level	3
concepts[20].score	0.0
concepts[20].wikidata	https://www.wikidata.org/wiki/Q407384
concepts[20].display_name	Transcription factor
keywords[0].id	https://openalex.org/keywords/computer-science
keywords[0].score	0.8460873365402222
keywords[0].display_name	Computer science
keywords[1].id	https://openalex.org/keywords/multicast
keywords[1].score	0.7035799026489258
keywords[1].display_name	Multicast
keywords[2].id	https://openalex.org/keywords/parallelism
keywords[2].score	0.683171808719635
keywords[2].display_name	Parallelism (grammar)
keywords[3].id	https://openalex.org/keywords/parallel-computing
keywords[3].score	0.6204637289047241
keywords[3].display_name	Parallel computing
keywords[4].id	https://openalex.org/keywords/distributed-computing
keywords[4].score	0.4838391840457916
keywords[4].display_name	Distributed computing
keywords[5].id	https://openalex.org/keywords/network-topology
keywords[5].score	0.4813588261604309
keywords[5].display_name	Network topology
keywords[6].id	https://openalex.org/keywords/models-of-communication
keywords[6].score	0.4491894841194153
keywords[6].display_name	Models of communication
keywords[7].id	https://openalex.org/keywords/pipeline
keywords[7].score	0.4304533898830414
keywords[7].display_name	Pipeline (software)
keywords[8].id	https://openalex.org/keywords/operator
keywords[8].score	0.41290047764778137
keywords[8].display_name	Operator (biology)
keywords[9].id	https://openalex.org/keywords/schedule
keywords[9].score	0.4122838079929352
keywords[9].display_name	Schedule
keywords[10].id	https://openalex.org/keywords/theoretical-computer-science
keywords[10].score	0.3747384250164032
keywords[10].display_name	Theoretical computer science
keywords[11].id	https://openalex.org/keywords/computer-network
keywords[11].score	0.18045195937156677
keywords[11].display_name	Computer network
keywords[12].id	https://openalex.org/keywords/programming-language
keywords[12].score	0.10308873653411865
keywords[12].display_name	Programming language
keywords[13].id	https://openalex.org/keywords/operating-system
keywords[13].score	0.07229351997375488
keywords[13].display_name	Operating system
language	en
locations[0].id	pmh:oai:arXiv.org:2211.05322
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2211.05322
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2211.05322
locations[1].id	doi:10.48550/arxiv.2211.05322
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2211.05322
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5076407338
authorships[0].author.orcid
authorships[0].author.display_name	Yonghao Zhuang
authorships[0].author_position	first
authorships[0].raw_author_name	Zhuang, Yonghao
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5030301481
authorships[1].author.orcid
authorships[1].author.display_name	Hexu Zhao
authorships[1].author_position	middle
authorships[1].raw_author_name	Zhao, Hexu
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5061339425
authorships[2].author.orcid	https://orcid.org/0000-0001-5812-731X
authorships[2].author.display_name	Lianmin Zheng
authorships[2].author_position	middle
authorships[2].raw_author_name	Zheng, Lianmin
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5009362721
authorships[3].author.orcid	https://orcid.org/0000-0001-5372-9450
authorships[3].author.display_name	Zhuohan Li
authorships[3].author_position	middle
authorships[3].raw_author_name	Li, Zhuohan
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5009547049
authorships[4].author.orcid	https://orcid.org/0009-0005-9158-4201
authorships[4].author.display_name	Eric P. Xing
authorships[4].author_position	middle
authorships[4].raw_author_name	Xing, Eric P.
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5012361506
authorships[5].author.orcid
authorships[5].author.display_name	Qirong Ho
authorships[5].author_position	middle
authorships[5].raw_author_name	Ho, Qirong
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5072427753
authorships[6].author.orcid	https://orcid.org/0000-0003-2921-956X
authorships[6].author.display_name	Joseph E. Gonzalez
authorships[6].author_position	middle
authorships[6].raw_author_name	Gonzalez, Joseph E.
authorships[6].is_corresponding	False
authorships[7].author.id	https://openalex.org/A5041920173
authorships[7].author.orcid	https://orcid.org/0000-0002-5373-0088
authorships[7].author.display_name	Ion Stoica
authorships[7].author_position	middle
authorships[7].raw_author_name	Stoica, Ion
authorships[7].is_corresponding	False
authorships[8].author.id	https://openalex.org/A5089432787
authorships[8].author.orcid	https://orcid.org/0000-0003-1923-589X
authorships[8].author.display_name	Hao Zhang
authorships[8].author_position	last
authorships[8].raw_author_name	Zhang, Hao
authorships[8].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2211.05322
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	On Optimizing the Communication of Model Parallelism
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10036
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9958000183105469
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1707
primary_topic.subfield.display_name	Computer Vision and Pattern Recognition
primary_topic.display_name	Advanced Neural Network Applications
related_works	https://openalex.org/W2033488476, https://openalex.org/W2131092765, https://openalex.org/W1589992863, https://openalex.org/W2277662012, https://openalex.org/W1617457131, https://openalex.org/W4377015086, https://openalex.org/W2369090769, https://openalex.org/W2997509936, https://openalex.org/W136477243, https://openalex.org/W2033862586
cited_by_count	3
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	1
counts_by_year[1].year	2024
counts_by_year[1].cited_by_count	2
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2211.05322
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2211.05322
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2211.05322
primary_location.id	pmh:oai:arXiv.org:2211.05322
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2211.05322
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2211.05322
publication_date	2022-11-10
publication_year	2022
referenced_works_count	0
abstract_inverted_index.-	29, 34
abstract_inverted_index.a	2, 47, 55, 60, 81
abstract_inverted_index.In	44
abstract_inverted_index.On	133, 151
abstract_inverted_index.We	0, 77, 114
abstract_inverted_index.an	123, 129
abstract_inverted_index.as	80
abstract_inverted_index.be	52, 69
abstract_inverted_index.by	141, 164
abstract_inverted_index.do	95
abstract_inverted_index.in	8
abstract_inverted_index.of	26, 154
abstract_inverted_index.on	41, 64
abstract_inverted_index.or	74, 94, 102
abstract_inverted_index.to	37, 51, 59, 98, 119, 143
abstract_inverted_index.up	142
abstract_inverted_index.we	15, 161
abstract_inverted_index.10%	165
abstract_inverted_index.10x	144
abstract_inverted_index.and	4, 31, 86, 111, 128, 148, 159, 166
abstract_inverted_index.are	35, 92
abstract_inverted_index.may	68
abstract_inverted_index.not	96
abstract_inverted_index.our	135
abstract_inverted_index.the	23, 66, 72
abstract_inverted_index.two	24, 117, 155
abstract_inverted_index.50%,	167
abstract_inverted_index.This	19
abstract_inverted_index.call	16
abstract_inverted_index.deep	11
abstract_inverted_index.from	54, 107
abstract_inverted_index.mesh	58, 149
abstract_inverted_index.ones	140
abstract_inverted_index.same	73
abstract_inverted_index.sent	53
abstract_inverted_index.show	87
abstract_inverted_index.that	88
abstract_inverted_index.then	115
abstract_inverted_index.this	79
abstract_inverted_index.when	22
abstract_inverted_index.with	71
abstract_inverted_index.(DL),	13
abstract_inverted_index.GPT-3	158
abstract_inverted_index.large	39, 42, 156
abstract_inverted_index.mesh,	63
abstract_inverted_index.model	27, 109
abstract_inverted_index.needs	50
abstract_inverted_index.novel	3
abstract_inverted_index.study	1
abstract_inverted_index.which	14, 65, 105
abstract_inverted_index.across	145
abstract_inverted_index.device	57, 62
abstract_inverted_index.either	91
abstract_inverted_index.models	40
abstract_inverted_index.result	106
abstract_inverted_index.source	56
abstract_inverted_index.system	137
abstract_inverted_index.tensor	49, 67, 103, 147
abstract_inverted_index.address	120
abstract_inverted_index.emerges	21
abstract_inverted_index.improve	162
abstract_inverted_index.models,	157
abstract_inverted_index.network	100
abstract_inverted_index.overall	136
abstract_inverted_index.pattern	7, 20
abstract_inverted_index.propose	116
abstract_inverted_index.sharded	48
abstract_inverted_index.support	38
abstract_inverted_index.system,	127
abstract_inverted_index.various	146
abstract_inverted_index.combined	36
abstract_inverted_index.existing	89, 139
abstract_inverted_index.layouts,	104
abstract_inverted_index.layouts.	76, 150
abstract_inverted_index.learning	12
abstract_inverted_index.pipeline	131
abstract_inverted_index.problem,	85
abstract_inverted_index.training	153
abstract_inverted_index.clusters.	43
abstract_inverted_index.different	75, 99, 108
abstract_inverted_index.efficient	124
abstract_inverted_index.formalize	78
abstract_inverted_index.important	5
abstract_inverted_index.multicast	83
abstract_inverted_index.paradigms	25
abstract_inverted_index.schedule.	132
abstract_inverted_index.approaches	90
abstract_inverted_index.cross-mesh	17, 45, 121
abstract_inverted_index.end-to-end	152
abstract_inverted_index.generalize	97
abstract_inverted_index.throughput	163
abstract_inverted_index.topologies	101
abstract_inverted_index.destination	61
abstract_inverted_index.distributed	70
abstract_inverted_index.large-scale	9
abstract_inverted_index.outperforms	138
abstract_inverted_index.parallelism	28, 33, 112
abstract_inverted_index.resharding,	46
abstract_inverted_index.resharding.	18
abstract_inverted_index.resharding:	122
abstract_inverted_index.strategies.	113
abstract_inverted_index.sub-optimal	93
abstract_inverted_index.many-to-many	82
abstract_inverted_index.architectures	110
abstract_inverted_index.communication	6, 84, 126
abstract_inverted_index.contributions	118
abstract_inverted_index.respectively.	168
abstract_inverted_index.U-Transformer,	160
abstract_inverted_index.inter-operator	32
abstract_inverted_index.intra-operator	30
abstract_inverted_index.model-parallel	10
abstract_inverted_index.broadcast-based	125
abstract_inverted_index.microbenchmarks,	134
abstract_inverted_index."overlapping-friendly"	130
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	9
sustainable_development_goals[0].id	https://metadata.un.org/sdg/9
sustainable_development_goals[0].score	0.4099999964237213
sustainable_development_goals[0].display_name	Industry, innovation and infrastructure
citation_normalized_percentile