First-Generation Inference Accelerator Deployment at Facebook Article Swipe

PDF

Michael J. Anderson , Benny Chen , Stephen Chen , Summer Deng , Jordan Fix , Michael Gschwind , Aravind Kalaiah , Changkyu Kim , Jaewon Lee , Jason Liang , Haixin Liu , Yinghai Lu , Jack G. Montgomery , Arun S. Moorthy , Satish Nadathur , Sam Naghshineh , Avinash Nayak , Jongsoo Park , Chris Petersen , Martin Schatz , Sundaram Narayanan , Bangsheng Tang , Peter Tang , Amy Yang , Jiecao Yu , Hector Yuen , Ying Zhang , Aravind Anbudurai , Vandana Balan , Harsha Bojja , Joe Boyd , Matthew Breitbach , Claudio Caldato , Anna Calvo , Garret Catron , Sneh Chandwani , Panos Christeas , Brad Cottel , Brian Coutinho , Arun Dalli , Abhishek Dhanotia , Oniel Duncan , Roman Dzhabarov , Simon Elmir , Chunli Fu , Wenyin Fu , Michael Fulthorp , Adi Gangidi , Nick Gibson , Sean Gordon , Beatriz Padilla Hernandez , Daniel E. Ho , Yucheng Huang , Olof Johansson , Shishir Juluri ·

YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2107.04140

In this paper, we provide a deep dive into the deployment of inference accelerators at Facebook. Many of our ML workloads have unique characteristics, such as sparse memory accesses, large model sizes, as well as high compute, memory and network bandwidth requirements. We co-designed a high-performance, energy-efficient inference accelerator platform based on these requirements. We describe the inference accelerator platform ecosystem we developed and deployed at Facebook: both hardware, through Open Compute Platform (OCP), and software framework and tooling, through Pytorch/Caffe2/Glow. A characteristic of this ecosystem from the start is its openness to enable a variety of AI accelerators from different vendors. This platform, with six low-power accelerator cards alongside a single-socket host CPU, allows us to serve models of high complexity that cannot be easily or efficiently run on CPUs. We describe various performance optimizations, at both platform and accelerator level, which enables this platform to serve production traffic at Facebook. We also share deployment challenges, lessons learned during performance optimization, as well as provide guidance for future inference hardware co-design.

Related Topics

Computer Science

Artificial Intelligence

Software Engineering

Concepts

Software deployment Inference Computer science Artificial intelligence Software engineering

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2107.04140
PDF: https://arxiv.org/pdf/2107.04140
OA Status: green
Cited By: 19
References: 51
Related Works: 10
OpenAlex ID: https://openalex.org/W3177865674

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W3177865674

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2107.04140

Digital Object Identifier
Title: First-Generation Inference Accelerator Deployment at Facebook

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2021

Year of publication
Publication date: 2021-07-08

Full publication date if available
Authors: Michael J. Anderson, Benny Chen, Stephen Chen, Summer Deng, Jordan Fix, Michael Gschwind, Aravind Kalaiah, Changkyu Kim, Jaewon Lee, Jason Liang, Haixin Liu, Yinghai Lu, Jack G. Montgomery, Arun S. Moorthy, Satish Nadathur, Sam Naghshineh, Avinash Nayak, Jongsoo Park, Chris Petersen, Martin Schatz, Sundaram Narayanan, Bangsheng Tang, Peter Tang, Amy Yang, Jiecao Yu, Hector Yuen, Ying Zhang, Aravind Anbudurai, Vandana Balan, Harsha Bojja, Joe Boyd, Matthew Breitbach, Claudio Caldato, Anna Calvo, Garret Catron, Sneh Chandwani, Panos Christeas, Brad Cottel, Brian Coutinho, Arun Dalli, Abhishek Dhanotia, Oniel Duncan, Roman Dzhabarov, Simon Elmir, Chunli Fu, Wenyin Fu, Michael Fulthorp, Adi Gangidi, Nick Gibson, Sean Gordon, Beatriz Padilla Hernandez, Daniel E. Ho, Yucheng Huang, Olof Johansson, Shishir Juluri

List of authors in order
Landing page: https://arxiv.org/abs/2107.04140

Publisher landing page
PDF URL: https://arxiv.org/pdf/2107.04140

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2107.04140

Direct OA link when available
Concepts: Software deployment, Inference, Computer science, Artificial intelligence, Software engineering

Top concepts (fields/topics) attached by OpenAlex
Cited by: 19

Total citation count in OpenAlex
Citations by year (recent): 2024: 1, 2023: 5, 2022: 8, 2021: 5

Per-year citation counts (last 5 years)
References (count): 51

Number of works referenced by this work
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W3177865674
doi	https://doi.org/10.48550/arxiv.2107.04140
ids.doi	https://doi.org/10.48550/arxiv.2107.04140
ids.mag	3177865674
ids.openalex	https://openalex.org/W3177865674
fwci
type	preprint
title	First-Generation Inference Accelerator Deployment at Facebook
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10715
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9916999936103821
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1705
topics[0].subfield.display_name	Computer Networks and Communications
topics[0].display_name	Distributed and Parallel Computing Systems
topics[1].id	https://openalex.org/T12016
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9901000261306763
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1710
topics[1].subfield.display_name	Information Systems
topics[1].display_name	Web Data Mining and Analysis
topics[2].id	https://openalex.org/T11986
topics[2].field.id	https://openalex.org/fields/18
topics[2].field.display_name	Decision Sciences
topics[2].score	0.9900000095367432
topics[2].domain.id	https://openalex.org/domains/2
topics[2].domain.display_name	Social Sciences
topics[2].subfield.id	https://openalex.org/subfields/1802
topics[2].subfield.display_name	Information Systems and Management
topics[2].display_name	Scientific Computing and Data Management
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C105339364
concepts[0].level	2
concepts[0].score	0.8338873386383057
concepts[0].wikidata	https://www.wikidata.org/wiki/Q2297740
concepts[0].display_name	Software deployment
concepts[1].id	https://openalex.org/C2776214188
concepts[1].level	2
concepts[1].score	0.7489145398139954
concepts[1].wikidata	https://www.wikidata.org/wiki/Q408386
concepts[1].display_name	Inference
concepts[2].id	https://openalex.org/C41008148
concepts[2].level	0
concepts[2].score	0.5439668893814087
concepts[2].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[2].display_name	Computer science
concepts[3].id	https://openalex.org/C154945302
concepts[3].level	1
concepts[3].score	0.2627740502357483
concepts[3].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[3].display_name	Artificial intelligence
concepts[4].id	https://openalex.org/C115903868
concepts[4].level	1
concepts[4].score	0.16226032376289368
concepts[4].wikidata	https://www.wikidata.org/wiki/Q80993
concepts[4].display_name	Software engineering
keywords[0].id	https://openalex.org/keywords/software-deployment
keywords[0].score	0.8338873386383057
keywords[0].display_name	Software deployment
keywords[1].id	https://openalex.org/keywords/inference
keywords[1].score	0.7489145398139954
keywords[1].display_name	Inference
keywords[2].id	https://openalex.org/keywords/computer-science
keywords[2].score	0.5439668893814087
keywords[2].display_name	Computer science
keywords[3].id	https://openalex.org/keywords/artificial-intelligence
keywords[3].score	0.2627740502357483
keywords[3].display_name	Artificial intelligence
keywords[4].id	https://openalex.org/keywords/software-engineering
keywords[4].score	0.16226032376289368
keywords[4].display_name	Software engineering
language	en
locations[0].id	pmh:oai:arXiv.org:2107.04140
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2107.04140
locations[0].version	submittedVersion
locations[0].raw_type
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2107.04140
locations[1].id	doi:10.48550/arxiv.2107.04140
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2107.04140
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5104652429
authorships[0].author.orcid
authorships[0].author.display_name	Michael J. Anderson
authorships[0].author_position	first
authorships[0].raw_author_name	Michael J. Anderson
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5084400616
authorships[1].author.orcid
authorships[1].author.display_name	Benny Chen
authorships[1].author_position	middle
authorships[1].raw_author_name	Benny Chen
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5044711688
authorships[2].author.orcid	https://orcid.org/0000-0001-8657-6200
authorships[2].author.display_name	Stephen Chen
authorships[2].author_position	middle
authorships[2].raw_author_name	Stephen Chen
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5024307057
authorships[3].author.orcid
authorships[3].author.display_name	Summer Deng
authorships[3].author_position	middle
authorships[3].raw_author_name	Summer Deng
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5004456963
authorships[4].author.orcid	https://orcid.org/0009-0005-0523-4767
authorships[4].author.display_name	Jordan Fix
authorships[4].author_position	middle
authorships[4].raw_author_name	Jordan Fix
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5049413685
authorships[5].author.orcid	https://orcid.org/0009-0001-4963-4915
authorships[5].author.display_name	Michael Gschwind
authorships[5].author_position	middle
authorships[5].raw_author_name	Michael Gschwind
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5058488778
authorships[6].author.orcid
authorships[6].author.display_name	Aravind Kalaiah
authorships[6].author_position	middle
authorships[6].raw_author_name	Aravind Kalaiah
authorships[6].is_corresponding	False
authorships[7].author.id	https://openalex.org/A5101667271
authorships[7].author.orcid	https://orcid.org/0000-0002-0283-8371
authorships[7].author.display_name	Changkyu Kim
authorships[7].author_position	middle
authorships[7].raw_author_name	Changkyu Kim
authorships[7].is_corresponding	False
authorships[8].author.id	https://openalex.org/A5100329194
authorships[8].author.orcid	https://orcid.org/0000-0002-0768-384X
authorships[8].author.display_name	Jaewon Lee
authorships[8].author_position	middle
authorships[8].raw_author_name	Jaewon Lee
authorships[8].is_corresponding	False
authorships[9].author.id	https://openalex.org/A5049065304
authorships[9].author.orcid	https://orcid.org/0000-0001-8742-8824
authorships[9].author.display_name	Jason Liang
authorships[9].author_position	middle
authorships[9].raw_author_name	Jason Liang
authorships[9].is_corresponding	False
authorships[10].author.id	https://openalex.org/A5101732340
authorships[10].author.orcid	https://orcid.org/0009-0000-9411-3062
authorships[10].author.display_name	Haixin Liu
authorships[10].author_position	middle
authorships[10].raw_author_name	Haixin Liu
authorships[10].is_corresponding	False
authorships[11].author.id	https://openalex.org/A5027861933
authorships[11].author.orcid	https://orcid.org/0009-0003-1993-8648
authorships[11].author.display_name	Yinghai Lu
authorships[11].author_position	middle
authorships[11].raw_author_name	Yinghai Lu
authorships[11].is_corresponding	False
authorships[12].author.id	https://openalex.org/A5013247173
authorships[12].author.orcid	https://orcid.org/0009-0006-3450-7362
authorships[12].author.display_name	Jack G. Montgomery
authorships[12].author_position	middle
authorships[12].raw_author_name	Jack Montgomery
authorships[12].is_corresponding	False
authorships[13].author.id	https://openalex.org/A5000451241
authorships[13].author.orcid	https://orcid.org/0000-0002-5988-1389
authorships[13].author.display_name	Arun S. Moorthy
authorships[13].author_position	middle
authorships[13].raw_author_name	Arun Moorthy
authorships[13].is_corresponding	False
authorships[14].author.id	https://openalex.org/A5047513659
authorships[14].author.orcid
authorships[14].author.display_name	Satish Nadathur
authorships[14].author_position	middle
authorships[14].raw_author_name	Nadathur Satish
authorships[14].is_corresponding	False
authorships[15].author.id	https://openalex.org/A5047403657
authorships[15].author.orcid	https://orcid.org/0000-0002-6511-1866
authorships[15].author.display_name	Sam Naghshineh
authorships[15].author_position	middle
authorships[15].raw_author_name	Sam Naghshineh
authorships[15].is_corresponding	False
authorships[16].author.id	https://openalex.org/A5019595876
authorships[16].author.orcid	https://orcid.org/0000-0001-7913-7189
authorships[16].author.display_name	Avinash Nayak
authorships[16].author_position	middle
authorships[16].raw_author_name	Avinash Nayak
authorships[16].is_corresponding	False
authorships[17].author.id	https://openalex.org/A5101876582
authorships[17].author.orcid	https://orcid.org/0000-0002-4750-9440
authorships[17].author.display_name	Jongsoo Park
authorships[17].author_position	middle
authorships[17].raw_author_name	Jongsoo Park
authorships[17].is_corresponding	False
authorships[18].author.id	https://openalex.org/A5007553043
authorships[18].author.orcid
authorships[18].author.display_name	Chris Petersen
authorships[18].author_position	middle
authorships[18].raw_author_name	Chris Petersen
authorships[18].is_corresponding	False
authorships[19].author.id	https://openalex.org/A5103044975
authorships[19].author.orcid	https://orcid.org/0000-0002-6059-0490
authorships[19].author.display_name	Martin Schatz
authorships[19].author_position	middle
authorships[19].raw_author_name	Martin Schatz
authorships[19].is_corresponding	False
authorships[20].author.id	https://openalex.org/A5001916597
authorships[20].author.orcid
authorships[20].author.display_name	Sundaram Narayanan
authorships[20].author_position	middle
authorships[20].raw_author_name	Narayanan Sundaram
authorships[20].is_corresponding	False
authorships[21].author.id	https://openalex.org/A5049161422
authorships[21].author.orcid
authorships[21].author.display_name	Bangsheng Tang
authorships[21].author_position	middle
authorships[21].raw_author_name	Bangsheng Tang
authorships[21].is_corresponding	False
authorships[22].author.id	https://openalex.org/A5106715619
authorships[22].author.orcid
authorships[22].author.display_name	Peter Tang
authorships[22].author_position	middle
authorships[22].raw_author_name	Peter Tang
authorships[22].is_corresponding	False
authorships[23].author.id	https://openalex.org/A5103608781
authorships[23].author.orcid
authorships[23].author.display_name	Amy Yang
authorships[23].author_position	middle
authorships[23].raw_author_name	Amy Yang
authorships[23].is_corresponding	False
authorships[24].author.id	https://openalex.org/A5008163714
authorships[24].author.orcid	https://orcid.org/0000-0003-2085-0312
authorships[24].author.display_name	Jiecao Yu
authorships[24].author_position	middle
authorships[24].raw_author_name	Jiecao Yu
authorships[24].is_corresponding	False
authorships[25].author.id	https://openalex.org/A5060639309
authorships[25].author.orcid	https://orcid.org/0000-0003-3429-8338
authorships[25].author.display_name	Hector Yuen
authorships[25].author_position	middle
authorships[25].raw_author_name	Hector Yuen
authorships[25].is_corresponding	False
authorships[26].author.id	https://openalex.org/A5100386171
authorships[26].author.orcid	https://orcid.org/0000-0002-7557-2965
authorships[26].author.display_name	Ying Zhang
authorships[26].author_position	middle
authorships[26].raw_author_name	Ying Zhang
authorships[26].is_corresponding	False
authorships[27].author.id	https://openalex.org/A5057884599
authorships[27].author.orcid
authorships[27].author.display_name	Aravind Anbudurai
authorships[27].author_position	middle
authorships[27].raw_author_name	Aravind Anbudurai
authorships[27].is_corresponding	False
authorships[28].author.id	https://openalex.org/A5076789004
authorships[28].author.orcid	https://orcid.org/0000-0001-6183-9633
authorships[28].author.display_name	Vandana Balan
authorships[28].author_position	middle
authorships[28].raw_author_name	Vandana Balan
authorships[28].is_corresponding	False
authorships[29].author.id	https://openalex.org/A5085822183
authorships[29].author.orcid
authorships[29].author.display_name	Harsha Bojja
authorships[29].author_position	middle
authorships[29].raw_author_name	Harsha Bojja
authorships[29].is_corresponding	False
authorships[30].author.id	https://openalex.org/A5027688064
authorships[30].author.orcid
authorships[30].author.display_name	Joe Boyd
authorships[30].author_position	middle
authorships[30].raw_author_name	Joe Boyd
authorships[30].is_corresponding	False
authorships[31].author.id	https://openalex.org/A5001555877
authorships[31].author.orcid
authorships[31].author.display_name	Matthew Breitbach
authorships[31].author_position	middle
authorships[31].raw_author_name	Matthew Breitbach
authorships[31].is_corresponding	False
authorships[32].author.id	https://openalex.org/A5064351966
authorships[32].author.orcid
authorships[32].author.display_name	Claudio Caldato
authorships[32].author_position	middle
authorships[32].raw_author_name	Claudio Caldato
authorships[32].is_corresponding	False
authorships[33].author.id	https://openalex.org/A5109508650
authorships[33].author.orcid
authorships[33].author.display_name	Anna Calvo
authorships[33].author_position	middle
authorships[33].raw_author_name	Anna Calvo
authorships[33].is_corresponding	False
authorships[34].author.id	https://openalex.org/A5069108225
authorships[34].author.orcid
authorships[34].author.display_name	Garret Catron
authorships[34].author_position	middle
authorships[34].raw_author_name	Garret Catron
authorships[34].is_corresponding	False
authorships[35].author.id	https://openalex.org/A5047673834
authorships[35].author.orcid
authorships[35].author.display_name	Sneh Chandwani
authorships[35].author_position	middle
authorships[35].raw_author_name	Sneh Chandwani
authorships[35].is_corresponding	False
authorships[36].author.id	https://openalex.org/A5079364544
authorships[36].author.orcid
authorships[36].author.display_name	Panos Christeas
authorships[36].author_position	middle
authorships[36].raw_author_name	Panos Christeas
authorships[36].is_corresponding	False
authorships[37].author.id	https://openalex.org/A5041648069
authorships[37].author.orcid
authorships[37].author.display_name	Brad Cottel
authorships[37].author_position	middle
authorships[37].raw_author_name	Brad Cottel
authorships[37].is_corresponding	False
authorships[38].author.id	https://openalex.org/A5090093796
authorships[38].author.orcid
authorships[38].author.display_name	Brian Coutinho
authorships[38].author_position	middle
authorships[38].raw_author_name	Brian Coutinho
authorships[38].is_corresponding	False
authorships[39].author.id	https://openalex.org/A5035453925
authorships[39].author.orcid
authorships[39].author.display_name	Arun Dalli
authorships[39].author_position	middle
authorships[39].raw_author_name	Arun Dalli
authorships[39].is_corresponding	False
authorships[40].author.id	https://openalex.org/A5041380104
authorships[40].author.orcid	https://orcid.org/0000-0002-5916-9383
authorships[40].author.display_name	Abhishek Dhanotia
authorships[40].author_position	middle
authorships[40].raw_author_name	Abhishek Dhanotia
authorships[40].is_corresponding	False
authorships[41].author.id	https://openalex.org/A5047230649
authorships[41].author.orcid
authorships[41].author.display_name	Oniel Duncan
authorships[41].author_position	middle
authorships[41].raw_author_name	Oniel Duncan
authorships[41].is_corresponding	False
authorships[42].author.id	https://openalex.org/A5066626575
authorships[42].author.orcid
authorships[42].author.display_name	Roman Dzhabarov
authorships[42].author_position	middle
authorships[42].raw_author_name	Roman Dzhabarov
authorships[42].is_corresponding	False
authorships[43].author.id	https://openalex.org/A5074456508
authorships[43].author.orcid
authorships[43].author.display_name	Simon Elmir
authorships[43].author_position	middle
authorships[43].raw_author_name	Simon Elmir
authorships[43].is_corresponding	False
authorships[44].author.id	https://openalex.org/A5049709983
authorships[44].author.orcid	https://orcid.org/0000-0003-4378-0863
authorships[44].author.display_name	Chunli Fu
authorships[44].author_position	middle
authorships[44].raw_author_name	Chunli Fu
authorships[44].is_corresponding	False
authorships[45].author.id	https://openalex.org/A5108399988
authorships[45].author.orcid
authorships[45].author.display_name	Wenyin Fu
authorships[45].author_position	middle
authorships[45].raw_author_name	Wenyin Fu
authorships[45].is_corresponding	False
authorships[46].author.id	https://openalex.org/A5079057009
authorships[46].author.orcid
authorships[46].author.display_name	Michael Fulthorp
authorships[46].author_position	middle
authorships[46].raw_author_name	Michael Fulthorp
authorships[46].is_corresponding	False
authorships[47].author.id	https://openalex.org/A5071326128
authorships[47].author.orcid
authorships[47].author.display_name	Adi Gangidi
authorships[47].author_position	middle
authorships[47].raw_author_name	Adi Gangidi
authorships[47].is_corresponding	False
authorships[48].author.id	https://openalex.org/A5112475113
authorships[48].author.orcid
authorships[48].author.display_name	Nick Gibson
authorships[48].author_position	middle
authorships[48].raw_author_name	Nick Gibson
authorships[48].is_corresponding	False
authorships[49].author.id	https://openalex.org/A5034034559
authorships[49].author.orcid
authorships[49].author.display_name	Sean Gordon
authorships[49].author_position	middle
authorships[49].raw_author_name	Sean Gordon
authorships[49].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2107.04140
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	First-Generation Inference Accelerator Deployment at Facebook
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10715
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9916999936103821
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1705
primary_topic.subfield.display_name	Computer Networks and Communications
primary_topic.display_name	Distributed and Parallel Computing Systems
related_works	https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W2770234245, https://openalex.org/W96612179, https://openalex.org/W4229499248, https://openalex.org/W2566006169, https://openalex.org/W2987774938, https://openalex.org/W4256492088, https://openalex.org/W632915154, https://openalex.org/W2055733372
cited_by_count	19
counts_by_year[0].year	2024
counts_by_year[0].cited_by_count	1
counts_by_year[1].year	2023
counts_by_year[1].cited_by_count	5
counts_by_year[2].year	2022
counts_by_year[2].cited_by_count	8
counts_by_year[3].year	2021
counts_by_year[3].cited_by_count	5
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2107.04140
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2107.04140
best_oa_location.version	submittedVersion
best_oa_location.raw_type
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2107.04140
primary_location.id	pmh:oai:arXiv.org:2107.04140
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2107.04140
primary_location.version	submittedVersion
primary_location.raw_type
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2107.04140
publication_date	2021-07-08
publication_year	2021
referenced_works	https://openalex.org/W2515287984, https://openalex.org/W2794670651, https://openalex.org/W3043023836, https://openalex.org/W2549139847, https://openalex.org/W2949591530, https://openalex.org/W2285660444, https://openalex.org/W3016832937, https://openalex.org/W3016430712, https://openalex.org/W3016265891, https://openalex.org/W3043406639, https://openalex.org/W2988396473, https://openalex.org/W2900810680, https://openalex.org/W2798956872, https://openalex.org/W3094502228, https://openalex.org/W3042495273, https://openalex.org/W3016339201, https://openalex.org/W2980020162, https://openalex.org/W2799269579, https://openalex.org/W2048266589, https://openalex.org/W2562773490, https://openalex.org/W3107855596, https://openalex.org/W3016939927, https://openalex.org/W2790925711, https://openalex.org/W2798341898, https://openalex.org/W2513554817, https://openalex.org/W2938458886, https://openalex.org/W2947737663, https://openalex.org/W3034098129, https://openalex.org/W2947629474, https://openalex.org/W2901839763, https://openalex.org/W2981758446, https://openalex.org/W2809273748, https://openalex.org/W2986514577, https://openalex.org/W2606722458, https://openalex.org/W3016842236, https://openalex.org/W3092816080, https://openalex.org/W3092379737, https://openalex.org/W3030163527, https://openalex.org/W2076618162, https://openalex.org/W2883929540, https://openalex.org/W2999012726, https://openalex.org/W2289252105, https://openalex.org/W3036878841, https://openalex.org/W3109610142, https://openalex.org/W2984287396, https://openalex.org/W2979719709, https://openalex.org/W3113087954, https://openalex.org/W2295739661, https://openalex.org/W3095776306, https://openalex.org/W3008591352, https://openalex.org/W3035390927
referenced_works_count	51
abstract_inverted_index.A	81
abstract_inverted_index.a	5, 44, 94, 110
abstract_inverted_index.AI	97
abstract_inverted_index.In	0
abstract_inverted_index.ML	19
abstract_inverted_index.We	42, 54, 131, 152
abstract_inverted_index.as	25, 32, 34, 162, 164
abstract_inverted_index.at	14, 65, 136, 150
abstract_inverted_index.be	124
abstract_inverted_index.is	89
abstract_inverted_index.of	11, 17, 83, 96, 119
abstract_inverted_index.on	51, 129
abstract_inverted_index.or	126
abstract_inverted_index.to	92, 116, 146
abstract_inverted_index.us	115
abstract_inverted_index.we	3, 61
abstract_inverted_index.and	38, 63, 74, 77, 139
abstract_inverted_index.for	167
abstract_inverted_index.its	90
abstract_inverted_index.our	18
abstract_inverted_index.run	128
abstract_inverted_index.six	105
abstract_inverted_index.the	9, 56, 87
abstract_inverted_index.CPU,	113
abstract_inverted_index.Many	16
abstract_inverted_index.Open	70
abstract_inverted_index.This	102
abstract_inverted_index.also	153
abstract_inverted_index.both	67, 137
abstract_inverted_index.deep	6
abstract_inverted_index.dive	7
abstract_inverted_index.from	86, 99
abstract_inverted_index.have	21
abstract_inverted_index.high	35, 120
abstract_inverted_index.host	112
abstract_inverted_index.into	8
abstract_inverted_index.such	24
abstract_inverted_index.that	122
abstract_inverted_index.this	1, 84, 144
abstract_inverted_index.well	33, 163
abstract_inverted_index.with	104
abstract_inverted_index.CPUs.	130
abstract_inverted_index.based	50
abstract_inverted_index.cards	108
abstract_inverted_index.large	29
abstract_inverted_index.model	30
abstract_inverted_index.serve	117, 147
abstract_inverted_index.share	154
abstract_inverted_index.start	88
abstract_inverted_index.these	52
abstract_inverted_index.which	142
abstract_inverted_index.(OCP),	73
abstract_inverted_index.allows	114
abstract_inverted_index.cannot	123
abstract_inverted_index.during	159
abstract_inverted_index.easily	125
abstract_inverted_index.enable	93
abstract_inverted_index.future	168
abstract_inverted_index.level,	141
abstract_inverted_index.memory	27, 37
abstract_inverted_index.models	118
abstract_inverted_index.paper,	2
abstract_inverted_index.sizes,	31
abstract_inverted_index.sparse	26
abstract_inverted_index.unique	22
abstract_inverted_index.Compute	71
abstract_inverted_index.enables	143
abstract_inverted_index.learned	158
abstract_inverted_index.lessons	157
abstract_inverted_index.network	39
abstract_inverted_index.provide	4, 165
abstract_inverted_index.through	69, 79
abstract_inverted_index.traffic	149
abstract_inverted_index.variety	95
abstract_inverted_index.various	133
abstract_inverted_index.Platform	72
abstract_inverted_index.compute,	36
abstract_inverted_index.deployed	64
abstract_inverted_index.describe	55, 132
abstract_inverted_index.guidance	166
abstract_inverted_index.hardware	170
abstract_inverted_index.openness	91
abstract_inverted_index.platform	49, 59, 138, 145
abstract_inverted_index.software	75
abstract_inverted_index.tooling,	78
abstract_inverted_index.vendors.	101
abstract_inverted_index.Facebook.	15, 151
abstract_inverted_index.Facebook:	66
abstract_inverted_index.accesses,	28
abstract_inverted_index.alongside	109
abstract_inverted_index.bandwidth	40
abstract_inverted_index.developed	62
abstract_inverted_index.different	100
abstract_inverted_index.ecosystem	60, 85
abstract_inverted_index.framework	76
abstract_inverted_index.hardware,	68
abstract_inverted_index.inference	12, 47, 57, 169
abstract_inverted_index.low-power	106
abstract_inverted_index.platform,	103
abstract_inverted_index.workloads	20
abstract_inverted_index.co-design.	171
abstract_inverted_index.complexity	121
abstract_inverted_index.deployment	10, 155
abstract_inverted_index.production	148
abstract_inverted_index.accelerator	48, 58, 107, 140
abstract_inverted_index.challenges,	156
abstract_inverted_index.co-designed	43
abstract_inverted_index.efficiently	127
abstract_inverted_index.performance	134, 160
abstract_inverted_index.accelerators	13, 98
abstract_inverted_index.optimization,	161
abstract_inverted_index.requirements.	41, 53
abstract_inverted_index.single-socket	111
abstract_inverted_index.characteristic	82
abstract_inverted_index.optimizations,	135
abstract_inverted_index.characteristics,	23
abstract_inverted_index.energy-efficient	46
abstract_inverted_index.high-performance,	45
abstract_inverted_index.Pytorch/Caffe2/Glow.	80
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	55
sustainable_development_goals[0].id	https://metadata.un.org/sdg/7
sustainable_development_goals[0].score	0.6200000047683716
sustainable_development_goals[0].display_name	Affordable and clean energy
citation_normalized_percentile