AIBench: An Agile Domain-specific Benchmarking Methodology and an AI Benchmark Suite Article Swipe

PDF

Wanling Gao , Fei Tang , Jianfeng Zhan , Chuanxin Lan , Chunjie Luo , Lei Wang , Jiahui Dai , Zheng Cao , Xiongwang Xiong , Zihan Jiang , Tianshu Hao , Fanda Fan , Xu Wen , Fan Zhang , Yunyou Huang , Jianan Chen , Mengjia Du , Rui Ren , Zheng Chen , Daoyi Zheng , Haoning Tang , Kunlin Zhan , Biao Wang , Defei Kong , Minghe Yu , Chongkang Tan , Huan Li , Xinhui Tian , Yatao Li , Gang Lü , Junchao Shao , Zhenyu Wang , Xiaoyu Wang , Hainan Ye ·

YOU? · · 2020 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2002.07162

Domain-specific software and hardware co-design is encouraging as it is much easier to achieve efficiency for fewer tasks. Agile domain-specific benchmarking speeds up the process as it provides not only relevant design inputs but also relevant metrics, and tools. Unfortunately, modern workloads like Big data, AI, and Internet services dwarf the traditional one in terms of code size, deployment scale, and execution path, and hence raise serious benchmarking challenges. This paper proposes an agile domain-specific benchmarking methodology. Together with seventeen industry partners, we identify ten important end-to-end application scenarios, among which sixteen representative AI tasks are distilled as the AI component benchmarks. We propose the permutations of essential AI and non-AI component benchmarks as end-to-end benchmarks. An end-to-end benchmark is a distillation of the essential attributes of an industry-scale application. We design and implement a highly extensible, configurable, and flexible benchmark framework, on the basis of which, we propose the guideline for building end-to-end benchmarks, and present the first end-to-end Internet service AI benchmark. The preliminary evaluation shows the value of our benchmark suite---AIBench against MLPerf and TailBench for hardware and software designers, micro-architectural researchers, and code developers. The specifications, source code, testbed, and results are publicly available from the web site \url{http://www.benchcouncil.org/AIBench/index.html}.

Related Topics

Benchmark (Surveying)

Mathematical Analysis

Mathematics

Physics

Business

Concepts

Benchmarking Benchmark (surveying) Computer science Suite Agile software development Testbed Software engineering The Internet Domain (mathematical analysis) Software Component (thermodynamics) Operating system World Wide Web History Marketing Geography Archaeology Geodesy Mathematical analysis Thermodynamics Mathematics Physics Business

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2002.07162
PDF: https://arxiv.org/pdf/2002.07162
OA Status: green
Cited By: 1
References: 49
Related Works: 10
OpenAlex ID: https://openalex.org/W3006388426

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W3006388426

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2002.07162

Digital Object Identifier
Title: AIBench: An Agile Domain-specific Benchmarking Methodology and an AI Benchmark Suite

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2020

Year of publication
Publication date: 2020-02-17

Full publication date if available
Authors: Wanling Gao, Fei Tang, Jianfeng Zhan, Chuanxin Lan, Chunjie Luo, Lei Wang, Jiahui Dai, Zheng Cao, Xiongwang Xiong, Zihan Jiang, Tianshu Hao, Fanda Fan, Xu Wen, Fan Zhang, Yunyou Huang, Jianan Chen, Mengjia Du, Rui Ren, Zheng Chen, Daoyi Zheng, Haoning Tang, Kunlin Zhan, Biao Wang, Defei Kong, Minghe Yu, Chongkang Tan, Huan Li, Xinhui Tian, Yatao Li, Gang Lü, Junchao Shao, Zhenyu Wang, Xiaoyu Wang, Hainan Ye

List of authors in order
Landing page: https://arxiv.org/abs/2002.07162

Publisher landing page
PDF URL: https://arxiv.org/pdf/2002.07162

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2002.07162

Direct OA link when available
Concepts: Benchmarking, Benchmark (surveying), Computer science, Suite, Agile software development, Testbed, Software engineering, The Internet, Domain (mathematical analysis), Software, Component (thermodynamics), Operating system, World Wide Web, History, Marketing, Geography, Archaeology, Geodesy, Mathematical analysis, Thermodynamics, Mathematics, Physics, Business

Top concepts (fields/topics) attached by OpenAlex
Cited by: 1

Total citation count in OpenAlex
Citations by year (recent): 2024: 1

Per-year citation counts (last 5 years)
References (count): 49

Number of works referenced by this work
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W3006388426
doi	https://doi.org/10.48550/arxiv.2002.07162
ids.doi	https://doi.org/10.48550/arxiv.2002.07162
ids.mag	3006388426
ids.openalex	https://openalex.org/W3006388426
fwci
type	preprint
title	AIBench: An Agile Domain-specific Benchmarking Methodology and an AI Benchmark Suite
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T11948
topics[0].field.id	https://openalex.org/fields/25
topics[0].field.display_name	Materials Science
topics[0].score	0.9926999807357788
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/2505
topics[0].subfield.display_name	Materials Chemistry
topics[0].display_name	Machine Learning in Materials Science
topics[1].id	https://openalex.org/T12127
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9886999726295471
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1705
topics[1].subfield.display_name	Computer Networks and Communications
topics[1].display_name	Software System Performance and Reliability
topics[2].id	https://openalex.org/T12808
topics[2].field.id	https://openalex.org/fields/22
topics[2].field.display_name	Engineering
topics[2].score	0.9785000085830688
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/2208
topics[2].subfield.display_name	Electrical and Electronic Engineering
topics[2].display_name	Ferroelectric and Negative Capacitance Devices
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C86251818
concepts[0].level	2
concepts[0].score	0.8876969814300537
concepts[0].wikidata	https://www.wikidata.org/wiki/Q816754
concepts[0].display_name	Benchmarking
concepts[1].id	https://openalex.org/C185798385
concepts[1].level	2
concepts[1].score	0.7648545503616333
concepts[1].wikidata	https://www.wikidata.org/wiki/Q1161707
concepts[1].display_name	Benchmark (surveying)
concepts[2].id	https://openalex.org/C41008148
concepts[2].level	0
concepts[2].score	0.7601069211959839
concepts[2].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[2].display_name	Computer science
concepts[3].id	https://openalex.org/C79581498
concepts[3].level	2
concepts[3].score	0.6470437049865723
concepts[3].wikidata	https://www.wikidata.org/wiki/Q1367530
concepts[3].display_name	Suite
concepts[4].id	https://openalex.org/C14185376
concepts[4].level	2
concepts[4].score	0.6351307034492493
concepts[4].wikidata	https://www.wikidata.org/wiki/Q30232
concepts[4].display_name	Agile software development
concepts[5].id	https://openalex.org/C31395832
concepts[5].level	2
concepts[5].score	0.5796648263931274
concepts[5].wikidata	https://www.wikidata.org/wiki/Q1318674
concepts[5].display_name	Testbed
concepts[6].id	https://openalex.org/C115903868
concepts[6].level	1
concepts[6].score	0.5550695061683655
concepts[6].wikidata	https://www.wikidata.org/wiki/Q80993
concepts[6].display_name	Software engineering
concepts[7].id	https://openalex.org/C110875604
concepts[7].level	2
concepts[7].score	0.49180296063423157
concepts[7].wikidata	https://www.wikidata.org/wiki/Q75
concepts[7].display_name	The Internet
concepts[8].id	https://openalex.org/C36503486
concepts[8].level	2
concepts[8].score	0.47780948877334595
concepts[8].wikidata	https://www.wikidata.org/wiki/Q11235244
concepts[8].display_name	Domain (mathematical analysis)
concepts[9].id	https://openalex.org/C2777904410
concepts[9].level	2
concepts[9].score	0.43761906027793884
concepts[9].wikidata	https://www.wikidata.org/wiki/Q7397
concepts[9].display_name	Software
concepts[10].id	https://openalex.org/C168167062
concepts[10].level	2
concepts[10].score	0.42793571949005127
concepts[10].wikidata	https://www.wikidata.org/wiki/Q1117970
concepts[10].display_name	Component (thermodynamics)
concepts[11].id	https://openalex.org/C111919701
concepts[11].level	1
concepts[11].score	0.2633216381072998
concepts[11].wikidata	https://www.wikidata.org/wiki/Q9135
concepts[11].display_name	Operating system
concepts[12].id	https://openalex.org/C136764020
concepts[12].level	1
concepts[12].score	0.19866999983787537
concepts[12].wikidata	https://www.wikidata.org/wiki/Q466
concepts[12].display_name	World Wide Web
concepts[13].id	https://openalex.org/C95457728
concepts[13].level	0
concepts[13].score	0.0
concepts[13].wikidata	https://www.wikidata.org/wiki/Q309
concepts[13].display_name	History
concepts[14].id	https://openalex.org/C162853370
concepts[14].level	1
concepts[14].score	0.0
concepts[14].wikidata	https://www.wikidata.org/wiki/Q39809
concepts[14].display_name	Marketing
concepts[15].id	https://openalex.org/C205649164
concepts[15].level	0
concepts[15].score	0.0
concepts[15].wikidata	https://www.wikidata.org/wiki/Q1071
concepts[15].display_name	Geography
concepts[16].id	https://openalex.org/C166957645
concepts[16].level	1
concepts[16].score	0.0
concepts[16].wikidata	https://www.wikidata.org/wiki/Q23498
concepts[16].display_name	Archaeology
concepts[17].id	https://openalex.org/C13280743
concepts[17].level	1
concepts[17].score	0.0
concepts[17].wikidata	https://www.wikidata.org/wiki/Q131089
concepts[17].display_name	Geodesy
concepts[18].id	https://openalex.org/C134306372
concepts[18].level	1
concepts[18].score	0.0
concepts[18].wikidata	https://www.wikidata.org/wiki/Q7754
concepts[18].display_name	Mathematical analysis
concepts[19].id	https://openalex.org/C97355855
concepts[19].level	1
concepts[19].score	0.0
concepts[19].wikidata	https://www.wikidata.org/wiki/Q11473
concepts[19].display_name	Thermodynamics
concepts[20].id	https://openalex.org/C33923547
concepts[20].level	0
concepts[20].score	0.0
concepts[20].wikidata	https://www.wikidata.org/wiki/Q395
concepts[20].display_name	Mathematics
concepts[21].id	https://openalex.org/C121332964
concepts[21].level	0
concepts[21].score	0.0
concepts[21].wikidata	https://www.wikidata.org/wiki/Q413
concepts[21].display_name	Physics
concepts[22].id	https://openalex.org/C144133560
concepts[22].level	0
concepts[22].score	0.0
concepts[22].wikidata	https://www.wikidata.org/wiki/Q4830453
concepts[22].display_name	Business
keywords[0].id	https://openalex.org/keywords/benchmarking
keywords[0].score	0.8876969814300537
keywords[0].display_name	Benchmarking
keywords[1].id	https://openalex.org/keywords/benchmark
keywords[1].score	0.7648545503616333
keywords[1].display_name	Benchmark (surveying)
keywords[2].id	https://openalex.org/keywords/computer-science
keywords[2].score	0.7601069211959839
keywords[2].display_name	Computer science
keywords[3].id	https://openalex.org/keywords/suite
keywords[3].score	0.6470437049865723
keywords[3].display_name	Suite
keywords[4].id	https://openalex.org/keywords/agile-software-development
keywords[4].score	0.6351307034492493
keywords[4].display_name	Agile software development
keywords[5].id	https://openalex.org/keywords/testbed
keywords[5].score	0.5796648263931274
keywords[5].display_name	Testbed
keywords[6].id	https://openalex.org/keywords/software-engineering
keywords[6].score	0.5550695061683655
keywords[6].display_name	Software engineering
keywords[7].id	https://openalex.org/keywords/the-internet
keywords[7].score	0.49180296063423157
keywords[7].display_name	The Internet
keywords[8].id	https://openalex.org/keywords/domain
keywords[8].score	0.47780948877334595
keywords[8].display_name	Domain (mathematical analysis)
keywords[9].id	https://openalex.org/keywords/software
keywords[9].score	0.43761906027793884
keywords[9].display_name	Software
keywords[10].id	https://openalex.org/keywords/component
keywords[10].score	0.42793571949005127
keywords[10].display_name	Component (thermodynamics)
keywords[11].id	https://openalex.org/keywords/operating-system
keywords[11].score	0.2633216381072998
keywords[11].display_name	Operating system
keywords[12].id	https://openalex.org/keywords/world-wide-web
keywords[12].score	0.19866999983787537
keywords[12].display_name	World Wide Web
language	en
locations[0].id	pmh:oai:arXiv.org:2002.07162
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2002.07162
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2002.07162
locations[1].id	doi:10.48550/arxiv.2002.07162
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2002.07162
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5075110031
authorships[0].author.orcid	https://orcid.org/0000-0002-3911-9389
authorships[0].author.display_name	Wanling Gao
authorships[0].author_position	first
authorships[0].raw_author_name	Wanling Gao
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5047051172
authorships[1].author.orcid	https://orcid.org/0000-0003-2521-4825
authorships[1].author.display_name	Fei Tang
authorships[1].author_position	middle
authorships[1].raw_author_name	Fei Tang
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5085963553
authorships[2].author.orcid	https://orcid.org/0000-0002-3728-6837
authorships[2].author.display_name	Jianfeng Zhan
authorships[2].author_position	middle
authorships[2].raw_author_name	Jianfeng Zhan
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5075639248
authorships[3].author.orcid
authorships[3].author.display_name	Chuanxin Lan
authorships[3].author_position	middle
authorships[3].raw_author_name	Chuanxin Lan
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5033347503
authorships[4].author.orcid	https://orcid.org/0000-0002-6977-929X
authorships[4].author.display_name	Chunjie Luo
authorships[4].author_position	middle
authorships[4].raw_author_name	Chunjie Luo
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5100435839
authorships[5].author.orcid	https://orcid.org/0000-0002-0537-1105
authorships[5].author.display_name	Lei Wang
authorships[5].author_position	middle
authorships[5].raw_author_name	Lei Wang
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5007492406
authorships[6].author.orcid
authorships[6].author.display_name	Jiahui Dai
authorships[6].author_position	middle
authorships[6].raw_author_name	Jiahui Dai
authorships[6].is_corresponding	False
authorships[7].author.id	https://openalex.org/A5029024939
authorships[7].author.orcid	https://orcid.org/0000-0002-1565-3683
authorships[7].author.display_name	Zheng Cao
authorships[7].author_position	middle
authorships[7].raw_author_name	Zheng Cao
authorships[7].is_corresponding	False
authorships[8].author.id	https://openalex.org/A5039320210
authorships[8].author.orcid
authorships[8].author.display_name	Xiongwang Xiong
authorships[8].author_position	middle
authorships[8].raw_author_name	Xiongwang Xiong
authorships[8].is_corresponding	False
authorships[9].author.id	https://openalex.org/A5013767183
authorships[9].author.orcid	https://orcid.org/0000-0003-0632-7402
authorships[9].author.display_name	Zihan Jiang
authorships[9].author_position	middle
authorships[9].raw_author_name	Zihan Jiang
authorships[9].is_corresponding	False
authorships[10].author.id	https://openalex.org/A5002465838
authorships[10].author.orcid	https://orcid.org/0000-0001-5540-8722
authorships[10].author.display_name	Tianshu Hao
authorships[10].author_position	middle
authorships[10].raw_author_name	Tianshu Hao
authorships[10].is_corresponding	False
authorships[11].author.id	https://openalex.org/A5083043281
authorships[11].author.orcid	https://orcid.org/0000-0002-5214-0959
authorships[11].author.display_name	Fanda Fan
authorships[11].author_position	middle
authorships[11].raw_author_name	Fanda Fan
authorships[11].is_corresponding	False
authorships[12].author.id	https://openalex.org/A5101418436
authorships[12].author.orcid	https://orcid.org/0000-0002-0658-0208
authorships[12].author.display_name	Xu Wen
authorships[12].author_position	middle
authorships[12].raw_author_name	Xu Wen
authorships[12].is_corresponding	False
authorships[13].author.id	https://openalex.org/A5100403400
authorships[13].author.orcid	https://orcid.org/0000-0001-7886-6144
authorships[13].author.display_name	Fan Zhang
authorships[13].author_position	middle
authorships[13].raw_author_name	Fan Zhang
authorships[13].is_corresponding	False
authorships[14].author.id	https://openalex.org/A5029016037
authorships[14].author.orcid	https://orcid.org/0000-0003-1236-0372
authorships[14].author.display_name	Yunyou Huang
authorships[14].author_position	middle
authorships[14].raw_author_name	Yunyou Huang
authorships[14].is_corresponding	False
authorships[15].author.id	https://openalex.org/A5100701920
authorships[15].author.orcid	https://orcid.org/0000-0002-6673-6884
authorships[15].author.display_name	Jianan Chen
authorships[15].author_position	middle
authorships[15].raw_author_name	Jianan Chen
authorships[15].is_corresponding	False
authorships[16].author.id	https://openalex.org/A5020051295
authorships[16].author.orcid
authorships[16].author.display_name	Mengjia Du
authorships[16].author_position	middle
authorships[16].raw_author_name	Mengjia Du
authorships[16].is_corresponding	False
authorships[17].author.id	https://openalex.org/A5101763272
authorships[17].author.orcid	https://orcid.org/0000-0002-7723-6754
authorships[17].author.display_name	Rui Ren
authorships[17].author_position	middle
authorships[17].raw_author_name	Rui Ren
authorships[17].is_corresponding	False
authorships[18].author.id	https://openalex.org/A5100665770
authorships[18].author.orcid	https://orcid.org/0000-0001-5252-4752
authorships[18].author.display_name	Zheng Chen
authorships[18].author_position	middle
authorships[18].raw_author_name	Chen Zheng
authorships[18].is_corresponding	False
authorships[19].author.id	https://openalex.org/A5041094593
authorships[19].author.orcid
authorships[19].author.display_name	Daoyi Zheng
authorships[19].author_position	middle
authorships[19].raw_author_name	Daoyi Zheng
authorships[19].is_corresponding	False
authorships[20].author.id	https://openalex.org/A5044559413
authorships[20].author.orcid
authorships[20].author.display_name	Haoning Tang
authorships[20].author_position	middle
authorships[20].raw_author_name	Haoning Tang
authorships[20].is_corresponding	False
authorships[21].author.id	https://openalex.org/A5087790662
authorships[21].author.orcid
authorships[21].author.display_name	Kunlin Zhan
authorships[21].author_position	middle
authorships[21].raw_author_name	Kunlin Zhan
authorships[21].is_corresponding	False
authorships[22].author.id	https://openalex.org/A5100439502
authorships[22].author.orcid	https://orcid.org/0000-0003-1089-1049
authorships[22].author.display_name	Biao Wang
authorships[22].author_position	middle
authorships[22].raw_author_name	Biao Wang
authorships[22].is_corresponding	False
authorships[23].author.id	https://openalex.org/A5026217516
authorships[23].author.orcid
authorships[23].author.display_name	Defei Kong
authorships[23].author_position	middle
authorships[23].raw_author_name	Defei Kong
authorships[23].is_corresponding	False
authorships[24].author.id	https://openalex.org/A5073691234
authorships[24].author.orcid	https://orcid.org/0000-0002-0287-8867
authorships[24].author.display_name	Minghe Yu
authorships[24].author_position	middle
authorships[24].raw_author_name	Minghe Yu
authorships[24].is_corresponding	False
authorships[25].author.id	https://openalex.org/A5005225012
authorships[25].author.orcid
authorships[25].author.display_name	Chongkang Tan
authorships[25].author_position	middle
authorships[25].raw_author_name	Chongkang Tan
authorships[25].is_corresponding	False
authorships[26].author.id	https://openalex.org/A5100319254
authorships[26].author.orcid	https://orcid.org/0000-0003-4229-2796
authorships[26].author.display_name	Huan Li
authorships[26].author_position	middle
authorships[26].raw_author_name	Huan Li
authorships[26].is_corresponding	False
authorships[27].author.id	https://openalex.org/A5102811848
authorships[27].author.orcid	https://orcid.org/0000-0003-3687-7923
authorships[27].author.display_name	Xinhui Tian
authorships[27].author_position	middle
authorships[27].raw_author_name	Xinhui Tian
authorships[27].is_corresponding	False
authorships[28].author.id	https://openalex.org/A5101898982
authorships[28].author.orcid	https://orcid.org/0000-0002-0696-4631
authorships[28].author.display_name	Yatao Li
authorships[28].author_position	middle
authorships[28].raw_author_name	Yatao Li
authorships[28].is_corresponding	False
authorships[29].author.id	https://openalex.org/A5070951824
authorships[29].author.orcid	https://orcid.org/0000-0002-0045-6433
authorships[29].author.display_name	Gang Lü
authorships[29].author_position	middle
authorships[29].raw_author_name	Gang Lu
authorships[29].is_corresponding	False
authorships[30].author.id	https://openalex.org/A5030902617
authorships[30].author.orcid
authorships[30].author.display_name	Junchao Shao
authorships[30].author_position	middle
authorships[30].raw_author_name	Junchao Shao
authorships[30].is_corresponding	False
authorships[31].author.id	https://openalex.org/A5100344954
authorships[31].author.orcid	https://orcid.org/0000-0002-4857-7234
authorships[31].author.display_name	Zhenyu Wang
authorships[31].author_position	middle
authorships[31].raw_author_name	Zhenyu Wang
authorships[31].is_corresponding	False
authorships[32].author.id	https://openalex.org/A5100357613
authorships[32].author.orcid	https://orcid.org/0000-0002-6431-8822
authorships[32].author.display_name	Xiaoyu Wang
authorships[32].author_position	middle
authorships[32].raw_author_name	Xiaoyu Wang
authorships[32].is_corresponding	False
authorships[33].author.id	https://openalex.org/A5042268564
authorships[33].author.orcid
authorships[33].author.display_name	Hainan Ye
authorships[33].author_position	last
authorships[33].raw_author_name	Hainan Ye
authorships[33].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2002.07162
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	AIBench: An Agile Domain-specific Benchmarking Methodology and an AI Benchmark Suite
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T11948
primary_topic.field.id	https://openalex.org/fields/25
primary_topic.field.display_name	Materials Science
primary_topic.score	0.9926999807357788
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/2505
primary_topic.subfield.display_name	Materials Chemistry
primary_topic.display_name	Machine Learning in Materials Science
related_works	https://openalex.org/W4379115841, https://openalex.org/W2083794993, https://openalex.org/W1511772879, https://openalex.org/W4394660363, https://openalex.org/W2186315912, https://openalex.org/W2248125223, https://openalex.org/W2588591308, https://openalex.org/W2127898439, https://openalex.org/W3082894236, https://openalex.org/W3195664246
cited_by_count	1
counts_by_year[0].year	2024
counts_by_year[0].cited_by_count	1
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2002.07162
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2002.07162
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2002.07162
primary_location.id	pmh:oai:arXiv.org:2002.07162
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2002.07162
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2002.07162
publication_date	2020-02-17
publication_year	2020
referenced_works	https://openalex.org/W2219888463, https://openalex.org/W2952173606, https://openalex.org/W2112121929, https://openalex.org/W2883242673, https://openalex.org/W2900608469, https://openalex.org/W2463955103, https://openalex.org/W1782590233, https://openalex.org/W2808847742, https://openalex.org/W2991650863, https://openalex.org/W1602980622, https://openalex.org/W2790501674, https://openalex.org/W2605350416, https://openalex.org/W2190691619, https://openalex.org/W2794670651, https://openalex.org/W2803058261, https://openalex.org/W2060204338, https://openalex.org/W2614794251, https://openalex.org/W2963402657, https://openalex.org/W2341349540, https://openalex.org/W2563351168, https://openalex.org/W1494198834, https://openalex.org/W2515080096, https://openalex.org/W2340897893, https://openalex.org/W2194775991, https://openalex.org/W2779888504, https://openalex.org/W2794581605, https://openalex.org/W2979245724, https://openalex.org/W2110953678, https://openalex.org/W2613718673, https://openalex.org/W2953795911, https://openalex.org/W2962793481, https://openalex.org/W3099206234, https://openalex.org/W967544008, https://openalex.org/W603908379, https://openalex.org/W2803718882, https://openalex.org/W2606722458, https://openalex.org/W2587914027, https://openalex.org/W2963929190, https://openalex.org/W2193413348, https://openalex.org/W2108598243, https://openalex.org/W2912955126, https://openalex.org/W2551540143, https://openalex.org/W2963403868, https://openalex.org/W2963839617, https://openalex.org/W2963149687, https://openalex.org/W2919115771, https://openalex.org/W2271840356, https://openalex.org/W1877705782, https://openalex.org/W1861492603
referenced_works_count	49
abstract_inverted_index.a	120, 134
abstract_inverted_index.AI	93, 99, 108, 162
abstract_inverted_index.An	116
abstract_inverted_index.We	102, 130
abstract_inverted_index.an	72, 127
abstract_inverted_index.as	7, 25, 97, 113
abstract_inverted_index.in	53
abstract_inverted_index.is	5, 9, 119
abstract_inverted_index.it	8, 26
abstract_inverted_index.of	55, 106, 122, 126, 145, 170
abstract_inverted_index.on	142
abstract_inverted_index.to	12
abstract_inverted_index.up	22
abstract_inverted_index.we	82, 147
abstract_inverted_index.AI,	45
abstract_inverted_index.Big	43
abstract_inverted_index.The	164, 188
abstract_inverted_index.and	2, 37, 46, 60, 63, 109, 132, 138, 155, 176, 180, 185, 193
abstract_inverted_index.are	95, 195
abstract_inverted_index.but	33
abstract_inverted_index.for	15, 151, 178
abstract_inverted_index.not	28
abstract_inverted_index.one	52
abstract_inverted_index.our	171
abstract_inverted_index.ten	84
abstract_inverted_index.the	23, 50, 98, 104, 123, 143, 149, 157, 168, 199
abstract_inverted_index.web	200
abstract_inverted_index.This	69
abstract_inverted_index.also	34
abstract_inverted_index.code	56, 186
abstract_inverted_index.from	198
abstract_inverted_index.like	42
abstract_inverted_index.much	10
abstract_inverted_index.only	29
abstract_inverted_index.site	201
abstract_inverted_index.with	78
abstract_inverted_index.Agile	18
abstract_inverted_index.agile	73
abstract_inverted_index.among	89
abstract_inverted_index.basis	144
abstract_inverted_index.code,	191
abstract_inverted_index.data,	44
abstract_inverted_index.dwarf	49
abstract_inverted_index.fewer	16
abstract_inverted_index.first	158
abstract_inverted_index.hence	64
abstract_inverted_index.paper	70
abstract_inverted_index.path,	62
abstract_inverted_index.raise	65
abstract_inverted_index.shows	167
abstract_inverted_index.size,	57
abstract_inverted_index.tasks	94
abstract_inverted_index.terms	54
abstract_inverted_index.value	169
abstract_inverted_index.which	90
abstract_inverted_index.MLPerf	175
abstract_inverted_index.design	31, 131
abstract_inverted_index.easier	11
abstract_inverted_index.highly	135
abstract_inverted_index.inputs	32
abstract_inverted_index.modern	40
abstract_inverted_index.non-AI	110
abstract_inverted_index.scale,	59
abstract_inverted_index.source	190
abstract_inverted_index.speeds	21
abstract_inverted_index.tasks.	17
abstract_inverted_index.tools.	38
abstract_inverted_index.which,	146
abstract_inverted_index.achieve	13
abstract_inverted_index.against	174
abstract_inverted_index.present	156
abstract_inverted_index.process	24
abstract_inverted_index.propose	103, 148
abstract_inverted_index.results	194
abstract_inverted_index.serious	66
abstract_inverted_index.service	161
abstract_inverted_index.sixteen	91
abstract_inverted_index.Internet	47, 160
abstract_inverted_index.Together	77
abstract_inverted_index.building	152
abstract_inverted_index.flexible	139
abstract_inverted_index.hardware	3, 179
abstract_inverted_index.identify	83
abstract_inverted_index.industry	80
abstract_inverted_index.metrics,	36
abstract_inverted_index.proposes	71
abstract_inverted_index.provides	27
abstract_inverted_index.publicly	196
abstract_inverted_index.relevant	30, 35
abstract_inverted_index.services	48
abstract_inverted_index.software	1, 181
abstract_inverted_index.testbed,	192
abstract_inverted_index.TailBench	177
abstract_inverted_index.available	197
abstract_inverted_index.benchmark	118, 140, 172
abstract_inverted_index.co-design	4
abstract_inverted_index.component	100, 111
abstract_inverted_index.distilled	96
abstract_inverted_index.essential	107, 124
abstract_inverted_index.execution	61
abstract_inverted_index.guideline	150
abstract_inverted_index.implement	133
abstract_inverted_index.important	85
abstract_inverted_index.partners,	81
abstract_inverted_index.seventeen	79
abstract_inverted_index.workloads	41
abstract_inverted_index.attributes	125
abstract_inverted_index.benchmark.	163
abstract_inverted_index.benchmarks	112
abstract_inverted_index.deployment	58
abstract_inverted_index.designers,	182
abstract_inverted_index.efficiency	14
abstract_inverted_index.end-to-end	86, 114, 117, 153, 159
abstract_inverted_index.evaluation	166
abstract_inverted_index.framework,	141
abstract_inverted_index.scenarios,	88
abstract_inverted_index.application	87
abstract_inverted_index.benchmarks,	154
abstract_inverted_index.benchmarks.	101, 115
abstract_inverted_index.challenges.	68
abstract_inverted_index.developers.	187
abstract_inverted_index.encouraging	6
abstract_inverted_index.extensible,	136
abstract_inverted_index.preliminary	165
abstract_inverted_index.traditional	51
abstract_inverted_index.application.	129
abstract_inverted_index.benchmarking	20, 67, 75
abstract_inverted_index.distillation	121
abstract_inverted_index.methodology.	76
abstract_inverted_index.permutations	105
abstract_inverted_index.researchers,	184
abstract_inverted_index.configurable,	137
abstract_inverted_index.Unfortunately,	39
abstract_inverted_index.industry-scale	128
abstract_inverted_index.representative	92
abstract_inverted_index.Domain-specific	0
abstract_inverted_index.domain-specific	19, 74
abstract_inverted_index.specifications,	189
abstract_inverted_index.suite---AIBench	173
abstract_inverted_index.micro-architectural	183
abstract_inverted_index.\url{http://www.benchcouncil.org/AIBench/index.html}.	202
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	34
sustainable_development_goals[0].id	https://metadata.un.org/sdg/9
sustainable_development_goals[0].score	0.6299999952316284
sustainable_development_goals[0].display_name	Industry, innovation and infrastructure
citation_normalized_percentile