Codabench: Flexible, easy-to-use, and reproducible meta-benchmark platform Article Swipe
Zhen Xu
,
Sérgio Escalera
,
Adrien Pavão
,
Magali Richard
,
Wei-Wei Tu
,
Quanming Yao
,
Huan Zhao
,
Isabelle Guyon
·
YOU?
·
· 2022
· Open Access
·
· DOI: https://doi.org/10.1016/j.patter.2022.100543
YOU?
·
· 2022
· Open Access
·
· DOI: https://doi.org/10.1016/j.patter.2022.100543
Obtaining a standardized benchmark of computational methods is a major issue in data-science communities. Dedicated frameworks enabling fair benchmarking in a unified environment are yet to be developed. Here, we introduce Codabench, a meta-benchmark platform that is open sourced and community driven for benchmarking algorithms or software agents versus datasets or tasks. A public instance of Codabench is open to everyone free of charge and allows benchmark organizers to fairly compare submissions under the same setting (software, hardware, data, algorithms), with custom protocols and data formats. Codabench has unique features facilitating easy organization of flexible and reproducible benchmarks, such as the possibility of reusing templates of benchmarks and supplying compute resources on demand. Codabench has been used internally and externally on various applications, receiving more than 130 users and 2,500 submissions. As illustrative use cases, we introduce four diverse benchmarks covering graph machine learning, cancer heterogeneity, clinical diagnosis, and reinforcement learning.
Related Topics
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://doi.org/10.1016/j.patter.2022.100543
- http://www.cell.com/article/S2666389922001465/pdf
- OA Status
- gold
- Cited By
- 62
- References
- 32
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4283462007
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4283462007Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.1016/j.patter.2022.100543Digital Object Identifier
- Title
-
Codabench: Flexible, easy-to-use, and reproducible meta-benchmark platformWork title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2022Year of publication
- Publication date
-
2022-06-24Full publication date if available
- Authors
-
Zhen Xu, Sérgio Escalera, Adrien Pavão, Magali Richard, Wei-Wei Tu, Quanming Yao, Huan Zhao, Isabelle GuyonList of authors in order
- Landing page
-
https://doi.org/10.1016/j.patter.2022.100543Publisher landing page
- PDF URL
-
https://www.cell.com/article/S2666389922001465/pdfDirect link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
goldOpen access status per OpenAlex
- OA URL
-
https://www.cell.com/article/S2666389922001465/pdfDirect OA link when available
- Concepts
-
Benchmark (surveying), Computer science, Metamodeling, Geology, Software engineering, GeodesyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
62Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 50, 2024: 8, 2023: 4Per-year citation counts (last 5 years)
- References (count)
-
32Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4283462007 |
|---|---|
| doi | https://doi.org/10.1016/j.patter.2022.100543 |
| ids.doi | https://doi.org/10.1016/j.patter.2022.100543 |
| ids.pmid | https://pubmed.ncbi.nlm.nih.gov/35845844 |
| ids.openalex | https://openalex.org/W4283462007 |
| fwci | 12.13952686 |
| type | article |
| title | Codabench: Flexible, easy-to-use, and reproducible meta-benchmark platform |
| biblio.issue | 7 |
| biblio.volume | 3 |
| biblio.last_page | 100543 |
| biblio.first_page | 100543 |
| topics[0].id | https://openalex.org/T12535 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9944999814033508 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Machine Learning and Data Classification |
| topics[1].id | https://openalex.org/T11986 |
| topics[1].field.id | https://openalex.org/fields/18 |
| topics[1].field.display_name | Decision Sciences |
| topics[1].score | 0.9865000247955322 |
| topics[1].domain.id | https://openalex.org/domains/2 |
| topics[1].domain.display_name | Social Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1802 |
| topics[1].subfield.display_name | Information Systems and Management |
| topics[1].display_name | Scientific Computing and Data Management |
| topics[2].id | https://openalex.org/T12292 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.982200026512146 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1707 |
| topics[2].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[2].display_name | Graph Theory and Algorithms |
| is_xpac | False |
| apc_list.value | 5200 |
| apc_list.currency | USD |
| apc_list.value_usd | 5200 |
| apc_paid.value | 5200 |
| apc_paid.currency | USD |
| apc_paid.value_usd | 5200 |
| concepts[0].id | https://openalex.org/C185798385 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7579210996627808 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1161707 |
| concepts[0].display_name | Benchmark (surveying) |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.618639349937439 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C86610423 |
| concepts[2].level | 2 |
| concepts[2].score | 0.414092481136322 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q1925081 |
| concepts[2].display_name | Metamodeling |
| concepts[3].id | https://openalex.org/C127313418 |
| concepts[3].level | 0 |
| concepts[3].score | 0.18481522798538208 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q1069 |
| concepts[3].display_name | Geology |
| concepts[4].id | https://openalex.org/C115903868 |
| concepts[4].level | 1 |
| concepts[4].score | 0.1718926727771759 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q80993 |
| concepts[4].display_name | Software engineering |
| concepts[5].id | https://openalex.org/C13280743 |
| concepts[5].level | 1 |
| concepts[5].score | 0.03728371858596802 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q131089 |
| concepts[5].display_name | Geodesy |
| keywords[0].id | https://openalex.org/keywords/benchmark |
| keywords[0].score | 0.7579210996627808 |
| keywords[0].display_name | Benchmark (surveying) |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.618639349937439 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/metamodeling |
| keywords[2].score | 0.414092481136322 |
| keywords[2].display_name | Metamodeling |
| keywords[3].id | https://openalex.org/keywords/geology |
| keywords[3].score | 0.18481522798538208 |
| keywords[3].display_name | Geology |
| keywords[4].id | https://openalex.org/keywords/software-engineering |
| keywords[4].score | 0.1718926727771759 |
| keywords[4].display_name | Software engineering |
| keywords[5].id | https://openalex.org/keywords/geodesy |
| keywords[5].score | 0.03728371858596802 |
| keywords[5].display_name | Geodesy |
| language | en |
| locations[0].id | doi:10.1016/j.patter.2022.100543 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4210198061 |
| locations[0].source.issn | 2666-3899 |
| locations[0].source.type | journal |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | 2666-3899 |
| locations[0].source.is_core | True |
| locations[0].source.is_in_doaj | True |
| locations[0].source.display_name | Patterns |
| locations[0].source.host_organization | https://openalex.org/P4310320990 |
| locations[0].source.host_organization_name | Elsevier BV |
| locations[0].source.host_organization_lineage | https://openalex.org/P4310320990 |
| locations[0].source.host_organization_lineage_names | Elsevier BV |
| locations[0].license | cc-by |
| locations[0].pdf_url | http://www.cell.com/article/S2666389922001465/pdf |
| locations[0].version | publishedVersion |
| locations[0].raw_type | journal-article |
| locations[0].license_id | https://openalex.org/licenses/cc-by |
| locations[0].is_accepted | True |
| locations[0].is_published | True |
| locations[0].raw_source_name | Patterns |
| locations[0].landing_page_url | https://doi.org/10.1016/j.patter.2022.100543 |
| locations[1].id | pmid:35845844 |
| locations[1].is_oa | False |
| locations[1].source.id | https://openalex.org/S4306525036 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | False |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | PubMed |
| locations[1].source.host_organization | https://openalex.org/I1299303238 |
| locations[1].source.host_organization_name | National Institutes of Health |
| locations[1].source.host_organization_lineage | https://openalex.org/I1299303238 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | publishedVersion |
| locations[1].raw_type | |
| locations[1].license_id | |
| locations[1].is_accepted | True |
| locations[1].is_published | True |
| locations[1].raw_source_name | Patterns (New York, N.Y.) |
| locations[1].landing_page_url | https://pubmed.ncbi.nlm.nih.gov/35845844 |
| locations[2].id | pmh:oai:HAL:hal-03374222v4 |
| locations[2].is_oa | True |
| locations[2].source.id | https://openalex.org/S4306402512 |
| locations[2].source.issn | |
| locations[2].source.type | repository |
| locations[2].source.is_oa | False |
| locations[2].source.issn_l | |
| locations[2].source.is_core | False |
| locations[2].source.is_in_doaj | False |
| locations[2].source.display_name | HAL (Le Centre pour la Communication Scientifique Directe) |
| locations[2].source.host_organization | https://openalex.org/I1294671590 |
| locations[2].source.host_organization_name | Centre National de la Recherche Scientifique |
| locations[2].source.host_organization_lineage | https://openalex.org/I1294671590 |
| locations[2].license | other-oa |
| locations[2].pdf_url | |
| locations[2].version | submittedVersion |
| locations[2].raw_type | Journal articles |
| locations[2].license_id | https://openalex.org/licenses/other-oa |
| locations[2].is_accepted | False |
| locations[2].is_published | False |
| locations[2].raw_source_name | Patterns, 2022, 3 (7), pp.100543. ⟨10.1016/j.patter.2022.100543⟩ |
| locations[2].landing_page_url | https://hal.science/hal-03374222 |
| locations[3].id | pmh:oai:pubmedcentral.nih.gov:9278500 |
| locations[3].is_oa | True |
| locations[3].source.id | https://openalex.org/S2764455111 |
| locations[3].source.issn | |
| locations[3].source.type | repository |
| locations[3].source.is_oa | False |
| locations[3].source.issn_l | |
| locations[3].source.is_core | False |
| locations[3].source.is_in_doaj | False |
| locations[3].source.display_name | PubMed Central |
| locations[3].source.host_organization | https://openalex.org/I1299303238 |
| locations[3].source.host_organization_name | National Institutes of Health |
| locations[3].source.host_organization_lineage | https://openalex.org/I1299303238 |
| locations[3].license | other-oa |
| locations[3].pdf_url | |
| locations[3].version | submittedVersion |
| locations[3].raw_type | Text |
| locations[3].license_id | https://openalex.org/licenses/other-oa |
| locations[3].is_accepted | False |
| locations[3].is_published | False |
| locations[3].raw_source_name | Patterns (N Y) |
| locations[3].landing_page_url | https://www.ncbi.nlm.nih.gov/pmc/articles/9278500 |
| locations[4].id | pmh:oai:pure.atira.dk:publications/5d8328c7-54a3-435f-ae55-44278142c6e2 |
| locations[4].is_oa | True |
| locations[4].source.id | https://openalex.org/S4306400216 |
| locations[4].source.issn | |
| locations[4].source.type | repository |
| locations[4].source.is_oa | False |
| locations[4].source.issn_l | |
| locations[4].source.is_core | False |
| locations[4].source.is_in_doaj | False |
| locations[4].source.display_name | Research Portal (King's College London) |
| locations[4].source.host_organization | https://openalex.org/I183935753 |
| locations[4].source.host_organization_name | King's College London |
| locations[4].source.host_organization_lineage | https://openalex.org/I183935753 |
| locations[4].license | cc-by |
| locations[4].pdf_url | |
| locations[4].version | submittedVersion |
| locations[4].raw_type | |
| locations[4].license_id | https://openalex.org/licenses/cc-by |
| locations[4].is_accepted | False |
| locations[4].is_published | False |
| locations[4].raw_source_name | |
| locations[4].landing_page_url | https://vbn.aau.dk/da/publications/5d8328c7-54a3-435f-ae55-44278142c6e2 |
| indexed_in | crossref, doaj, pubmed |
| authorships[0].author.id | https://openalex.org/A5102956776 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-8901-1819 |
| authorships[0].author.display_name | Zhen Xu |
| authorships[0].affiliations[0].raw_affiliation_string | 4Paradigm (Beijing, China - China) |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Zhen Xu |
| authorships[0].is_corresponding | False |
| authorships[0].raw_affiliation_strings | 4Paradigm (Beijing, China - China) |
| authorships[1].author.id | https://openalex.org/A5038228433 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-0617-8873 |
| authorships[1].author.display_name | Sérgio Escalera |
| authorships[1].countries | ES |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I71999127 |
| authorships[1].affiliations[0].raw_affiliation_string | UB - Universitat de Barcelona (Gran Via de les Corts Catalanes, 585, 08007 Barcelona - Spain) |
| authorships[1].institutions[0].id | https://openalex.org/I71999127 |
| authorships[1].institutions[0].ror | https://ror.org/021018s57 |
| authorships[1].institutions[0].type | education |
| authorships[1].institutions[0].lineage | https://openalex.org/I71999127 |
| authorships[1].institutions[0].country_code | ES |
| authorships[1].institutions[0].display_name | Universitat de Barcelona |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Sergio Escalera |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | UB - Universitat de Barcelona (Gran Via de les Corts Catalanes, 585, 08007 Barcelona - Spain) |
| authorships[2].author.id | https://openalex.org/A5108132519 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-7374-5095 |
| authorships[2].author.display_name | Adrien Pavão |
| authorships[2].countries | FR |
| authorships[2].affiliations[0].institution_ids | https://openalex.org/I277688954 |
| authorships[2].affiliations[0].raw_affiliation_string | TAU - TAckling the Underspecified (Université Paris Saclay Bâtiment 660 - Shannon ; 91405 Orsay Cedex - France) |
| authorships[2].institutions[0].id | https://openalex.org/I277688954 |
| authorships[2].institutions[0].ror | https://ror.org/03xjwb503 |
| authorships[2].institutions[0].type | education |
| authorships[2].institutions[0].lineage | https://openalex.org/I277688954 |
| authorships[2].institutions[0].country_code | FR |
| authorships[2].institutions[0].display_name | Université Paris-Saclay |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Adrien Pavão |
| authorships[2].is_corresponding | False |
| authorships[2].raw_affiliation_strings | TAU - TAckling the Underspecified (Université Paris Saclay Bâtiment 660 - Shannon ; 91405 Orsay Cedex - France) |
| authorships[3].author.id | https://openalex.org/A5080596509 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-3165-3218 |
| authorships[3].author.display_name | Magali Richard |
| authorships[3].affiliations[0].raw_affiliation_string | TIMC-MAGe - Modèles et Algorithmes pour la Génomique (Domaine de la Merci, 38706 La Tronche, France - France) |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Magali Richard |
| authorships[3].is_corresponding | False |
| authorships[3].raw_affiliation_strings | TIMC-MAGe - Modèles et Algorithmes pour la Génomique (Domaine de la Merci, 38706 La Tronche, France - France) |
| authorships[4].author.id | https://openalex.org/A5032604295 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-2407-0252 |
| authorships[4].author.display_name | Wei-Wei Tu |
| authorships[4].countries | US |
| authorships[4].affiliations[0].raw_affiliation_string | 4Paradigm (Beijing, China - China) |
| authorships[4].affiliations[1].institution_ids | https://openalex.org/I157066012 |
| authorships[4].affiliations[1].raw_affiliation_string | Chalearn (United States) |
| authorships[4].institutions[0].id | https://openalex.org/I157066012 |
| authorships[4].institutions[0].ror | https://ror.org/03bw04561 |
| authorships[4].institutions[0].type | company |
| authorships[4].institutions[0].lineage | https://openalex.org/I157066012 |
| authorships[4].institutions[0].country_code | US |
| authorships[4].institutions[0].display_name | Gleason (United States) |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Wei-Wei Tu |
| authorships[4].is_corresponding | False |
| authorships[4].raw_affiliation_strings | 4Paradigm (Beijing, China - China), Chalearn (United States) |
| authorships[5].author.id | https://openalex.org/A5072484211 |
| authorships[5].author.orcid | https://orcid.org/0000-0001-8944-8618 |
| authorships[5].author.display_name | Quanming Yao |
| authorships[5].countries | CN |
| authorships[5].affiliations[0].institution_ids | https://openalex.org/I99065089 |
| authorships[5].affiliations[0].raw_affiliation_string | THU - Tsinghua University [Beijing] (Beijing, 100084 P.R. - China) |
| authorships[5].institutions[0].id | https://openalex.org/I99065089 |
| authorships[5].institutions[0].ror | https://ror.org/03cve4549 |
| authorships[5].institutions[0].type | education |
| authorships[5].institutions[0].lineage | https://openalex.org/I99065089 |
| authorships[5].institutions[0].country_code | CN |
| authorships[5].institutions[0].display_name | Tsinghua University |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Quanming Yao |
| authorships[5].is_corresponding | False |
| authorships[5].raw_affiliation_strings | THU - Tsinghua University [Beijing] (Beijing, 100084 P.R. - China) |
| authorships[6].author.id | https://openalex.org/A5101478660 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-0320-8718 |
| authorships[6].author.display_name | Huan Zhao |
| authorships[6].affiliations[0].raw_affiliation_string | 4Paradigm (Beijing, China - China) |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Huan Zhao |
| authorships[6].is_corresponding | False |
| authorships[6].raw_affiliation_strings | 4Paradigm (Beijing, China - China) |
| authorships[7].author.id | https://openalex.org/A5006360335 |
| authorships[7].author.orcid | https://orcid.org/0000-0002-9266-1783 |
| authorships[7].author.display_name | Isabelle Guyon |
| authorships[7].countries | FR, US |
| authorships[7].affiliations[0].institution_ids | https://openalex.org/I277688954 |
| authorships[7].affiliations[0].raw_affiliation_string | TAU - TAckling the Underspecified (Université Paris Saclay Bâtiment 660 - Shannon ; 91405 Orsay Cedex - France) |
| authorships[7].affiliations[1].institution_ids | https://openalex.org/I157066012 |
| authorships[7].affiliations[1].raw_affiliation_string | Chalearn (United States) |
| authorships[7].institutions[0].id | https://openalex.org/I277688954 |
| authorships[7].institutions[0].ror | https://ror.org/03xjwb503 |
| authorships[7].institutions[0].type | education |
| authorships[7].institutions[0].lineage | https://openalex.org/I277688954 |
| authorships[7].institutions[0].country_code | FR |
| authorships[7].institutions[0].display_name | Université Paris-Saclay |
| authorships[7].institutions[1].id | https://openalex.org/I157066012 |
| authorships[7].institutions[1].ror | https://ror.org/03bw04561 |
| authorships[7].institutions[1].type | company |
| authorships[7].institutions[1].lineage | https://openalex.org/I157066012 |
| authorships[7].institutions[1].country_code | US |
| authorships[7].institutions[1].display_name | Gleason (United States) |
| authorships[7].author_position | last |
| authorships[7].raw_author_name | Isabelle Guyon |
| authorships[7].is_corresponding | False |
| authorships[7].raw_affiliation_strings | Chalearn (United States), TAU - TAckling the Underspecified (Université Paris Saclay Bâtiment 660 - Shannon ; 91405 Orsay Cedex - France) |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | http://www.cell.com/article/S2666389922001465/pdf |
| open_access.oa_status | gold |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Codabench: Flexible, easy-to-use, and reproducible meta-benchmark platform |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T12535 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9944999814033508 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Machine Learning and Data Classification |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2080167534, https://openalex.org/W3146440365, https://openalex.org/W2011315860, https://openalex.org/W2130588191, https://openalex.org/W2140792736, https://openalex.org/W2280241695, https://openalex.org/W4239214709 |
| cited_by_count | 62 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 50 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 8 |
| counts_by_year[2].year | 2023 |
| counts_by_year[2].cited_by_count | 4 |
| locations_count | 5 |
| best_oa_location.id | doi:10.1016/j.patter.2022.100543 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4210198061 |
| best_oa_location.source.issn | 2666-3899 |
| best_oa_location.source.type | journal |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | 2666-3899 |
| best_oa_location.source.is_core | True |
| best_oa_location.source.is_in_doaj | True |
| best_oa_location.source.display_name | Patterns |
| best_oa_location.source.host_organization | https://openalex.org/P4310320990 |
| best_oa_location.source.host_organization_name | Elsevier BV |
| best_oa_location.source.host_organization_lineage | https://openalex.org/P4310320990 |
| best_oa_location.source.host_organization_lineage_names | Elsevier BV |
| best_oa_location.license | cc-by |
| best_oa_location.pdf_url | http://www.cell.com/article/S2666389922001465/pdf |
| best_oa_location.version | publishedVersion |
| best_oa_location.raw_type | journal-article |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by |
| best_oa_location.is_accepted | True |
| best_oa_location.is_published | True |
| best_oa_location.raw_source_name | Patterns |
| best_oa_location.landing_page_url | https://doi.org/10.1016/j.patter.2022.100543 |
| primary_location.id | doi:10.1016/j.patter.2022.100543 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4210198061 |
| primary_location.source.issn | 2666-3899 |
| primary_location.source.type | journal |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | 2666-3899 |
| primary_location.source.is_core | True |
| primary_location.source.is_in_doaj | True |
| primary_location.source.display_name | Patterns |
| primary_location.source.host_organization | https://openalex.org/P4310320990 |
| primary_location.source.host_organization_name | Elsevier BV |
| primary_location.source.host_organization_lineage | https://openalex.org/P4310320990 |
| primary_location.source.host_organization_lineage_names | Elsevier BV |
| primary_location.license | cc-by |
| primary_location.pdf_url | http://www.cell.com/article/S2666389922001465/pdf |
| primary_location.version | publishedVersion |
| primary_location.raw_type | journal-article |
| primary_location.license_id | https://openalex.org/licenses/cc-by |
| primary_location.is_accepted | True |
| primary_location.is_published | True |
| primary_location.raw_source_name | Patterns |
| primary_location.landing_page_url | https://doi.org/10.1016/j.patter.2022.100543 |
| publication_date | 2022-06-24 |
| publication_year | 2022 |
| referenced_works | https://openalex.org/W2132862423, https://openalex.org/W2966048283, https://openalex.org/W6681934335, https://openalex.org/W6776488958, https://openalex.org/W6675354045, https://openalex.org/W6684207430, https://openalex.org/W2145680191, https://openalex.org/W6631362777, https://openalex.org/W6744557953, https://openalex.org/W2931367569, https://openalex.org/W2127795553, https://openalex.org/W3202268458, https://openalex.org/W3096486123, https://openalex.org/W2933601491, https://openalex.org/W3003644143, https://openalex.org/W2028145591, https://openalex.org/W2076785727, https://openalex.org/W2008626263, https://openalex.org/W3105259638, https://openalex.org/W4293651439, https://openalex.org/W4301410255, https://openalex.org/W2061212083, https://openalex.org/W2746791238, https://openalex.org/W3035524727, https://openalex.org/W4212774754, https://openalex.org/W3120740533, https://openalex.org/W4214717370, https://openalex.org/W2150066425, https://openalex.org/W4234482043, https://openalex.org/W4244531973, https://openalex.org/W2101234009, https://openalex.org/W2117539524 |
| referenced_works_count | 32 |
| abstract_inverted_index.A | 52 |
| abstract_inverted_index.a | 1, 8, 20, 32 |
| abstract_inverted_index.As | 131 |
| abstract_inverted_index.as | 99 |
| abstract_inverted_index.be | 26 |
| abstract_inverted_index.in | 11, 19 |
| abstract_inverted_index.is | 7, 36, 57 |
| abstract_inverted_index.of | 4, 55, 62, 93, 102, 105 |
| abstract_inverted_index.on | 111, 120 |
| abstract_inverted_index.or | 45, 50 |
| abstract_inverted_index.to | 25, 59, 68 |
| abstract_inverted_index.we | 29, 135 |
| abstract_inverted_index.130 | 126 |
| abstract_inverted_index.and | 39, 64, 83, 95, 107, 118, 128, 148 |
| abstract_inverted_index.are | 23 |
| abstract_inverted_index.for | 42 |
| abstract_inverted_index.has | 87, 114 |
| abstract_inverted_index.the | 73, 100 |
| abstract_inverted_index.use | 133 |
| abstract_inverted_index.yet | 24 |
| abstract_inverted_index.been | 115 |
| abstract_inverted_index.data | 84 |
| abstract_inverted_index.easy | 91 |
| abstract_inverted_index.fair | 17 |
| abstract_inverted_index.four | 137 |
| abstract_inverted_index.free | 61 |
| abstract_inverted_index.more | 124 |
| abstract_inverted_index.open | 37, 58 |
| abstract_inverted_index.same | 74 |
| abstract_inverted_index.such | 98 |
| abstract_inverted_index.than | 125 |
| abstract_inverted_index.that | 35 |
| abstract_inverted_index.used | 116 |
| abstract_inverted_index.with | 80 |
| abstract_inverted_index.2,500 | 129 |
| abstract_inverted_index.Here, | 28 |
| abstract_inverted_index.data, | 78 |
| abstract_inverted_index.graph | 141 |
| abstract_inverted_index.issue | 10 |
| abstract_inverted_index.major | 9 |
| abstract_inverted_index.under | 72 |
| abstract_inverted_index.users | 127 |
| abstract_inverted_index.agents | 47 |
| abstract_inverted_index.allows | 65 |
| abstract_inverted_index.cancer | 144 |
| abstract_inverted_index.cases, | 134 |
| abstract_inverted_index.charge | 63 |
| abstract_inverted_index.custom | 81 |
| abstract_inverted_index.driven | 41 |
| abstract_inverted_index.fairly | 69 |
| abstract_inverted_index.public | 53 |
| abstract_inverted_index.tasks. | 51 |
| abstract_inverted_index.unique | 88 |
| abstract_inverted_index.versus | 48 |
| abstract_inverted_index.compare | 70 |
| abstract_inverted_index.compute | 109 |
| abstract_inverted_index.demand. | 112 |
| abstract_inverted_index.diverse | 138 |
| abstract_inverted_index.machine | 142 |
| abstract_inverted_index.methods | 6 |
| abstract_inverted_index.reusing | 103 |
| abstract_inverted_index.setting | 75 |
| abstract_inverted_index.sourced | 38 |
| abstract_inverted_index.unified | 21 |
| abstract_inverted_index.various | 121 |
| abstract_inverted_index.clinical | 146 |
| abstract_inverted_index.covering | 140 |
| abstract_inverted_index.datasets | 49 |
| abstract_inverted_index.enabling | 16 |
| abstract_inverted_index.everyone | 60 |
| abstract_inverted_index.features | 89 |
| abstract_inverted_index.flexible | 94 |
| abstract_inverted_index.formats. | 85 |
| abstract_inverted_index.instance | 54 |
| abstract_inverted_index.platform | 34 |
| abstract_inverted_index.software | 46 |
| abstract_inverted_index.Codabench | 56, 86, 113 |
| abstract_inverted_index.Dedicated | 14 |
| abstract_inverted_index.Obtaining | 0 |
| abstract_inverted_index.benchmark | 3, 66 |
| abstract_inverted_index.community | 40 |
| abstract_inverted_index.hardware, | 77 |
| abstract_inverted_index.introduce | 30, 136 |
| abstract_inverted_index.learning, | 143 |
| abstract_inverted_index.learning. | 150 |
| abstract_inverted_index.protocols | 82 |
| abstract_inverted_index.receiving | 123 |
| abstract_inverted_index.resources | 110 |
| abstract_inverted_index.supplying | 108 |
| abstract_inverted_index.templates | 104 |
| abstract_inverted_index.(software, | 76 |
| abstract_inverted_index.Codabench, | 31 |
| abstract_inverted_index.algorithms | 44 |
| abstract_inverted_index.benchmarks | 106, 139 |
| abstract_inverted_index.developed. | 27 |
| abstract_inverted_index.diagnosis, | 147 |
| abstract_inverted_index.externally | 119 |
| abstract_inverted_index.frameworks | 15 |
| abstract_inverted_index.internally | 117 |
| abstract_inverted_index.organizers | 67 |
| abstract_inverted_index.benchmarks, | 97 |
| abstract_inverted_index.environment | 22 |
| abstract_inverted_index.possibility | 101 |
| abstract_inverted_index.submissions | 71 |
| abstract_inverted_index.algorithms), | 79 |
| abstract_inverted_index.benchmarking | 18, 43 |
| abstract_inverted_index.communities. | 13 |
| abstract_inverted_index.data-science | 12 |
| abstract_inverted_index.facilitating | 90 |
| abstract_inverted_index.illustrative | 132 |
| abstract_inverted_index.organization | 92 |
| abstract_inverted_index.reproducible | 96 |
| abstract_inverted_index.standardized | 2 |
| abstract_inverted_index.submissions. | 130 |
| abstract_inverted_index.applications, | 122 |
| abstract_inverted_index.computational | 5 |
| abstract_inverted_index.reinforcement | 149 |
| abstract_inverted_index.heterogeneity, | 145 |
| abstract_inverted_index.meta-benchmark | 33 |
| cited_by_percentile_year.max | 100 |
| cited_by_percentile_year.min | 97 |
| countries_distinct_count | 4 |
| institutions_distinct_count | 8 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/17 |
| sustainable_development_goals[0].score | 0.4699999988079071 |
| sustainable_development_goals[0].display_name | Partnerships for the goals |
| citation_normalized_percentile.value | 0.98214881 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | True |