Buddy Compression: Enabling Larger Memory for Deep Learning and HPC\n Workloads on GPUs Article Swipe

PDF

Esha Choukse , Michael Sullivan , Mike O’Connor , Mattan Erez , Jeff Pool , David Nellans , Steve Keckler ·

YOU? · · 2019 · Open Access · · DOI: https://doi.org/10.48550/arxiv.1903.02596

GPUs offer orders-of-magnitude higher memory bandwidth than traditional\nCPU-only systems. However, GPU device memory tends to be relatively small and\nthe memory capacity can not be increased by the user. This paper describes\nBuddy Compression, a scheme to increase both the effective GPU memory capacity\nand bandwidth while avoiding the downsides of conventional memory-expanding\nstrategies. Buddy Compression compresses GPU memory, splitting each compressed\nmemory entry between high-speed device memory and a slower-but-larger\ndisaggregated memory pool (or system memory). Highly-compressible memory\nentries can thus be accessed completely from device memory, while\nincompressible entries source their data using both on and off-device accesses.\nIncreasing the effective GPU memory capacity enables us to run\nlarger-memory-footprint HPC workloads and larger batch-sizes or models for DL\nworkloads than current memory capacities would allow. We show that our solution\nachieves an average compression ratio of 2.2x on HPC workloads and 1.5x on DL\nworkloads, with a slowdown of just 1~2%.\n

Related Topics

Concepts

Computer science Parallel computing Memory bandwidth Interleaved memory Memory footprint Registered memory Memory management Flat memory model Computer memory Uniform memory access Semiconductor memory High memory Supercomputer Compression (physics) Extended memory Computer hardware Operating system Materials science Composite material

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/1903.02596
PDF: https://arxiv.org/pdf/1903.02596
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4288419222

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4288419222

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.1903.02596

Digital Object Identifier
Title: Buddy Compression: Enabling Larger Memory for Deep Learning and HPC\n Workloads on GPUs

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2019

Year of publication
Publication date: 2019-03-06

Full publication date if available
Authors: Esha Choukse, Michael Sullivan, Mike O’Connor, Mattan Erez, Jeff Pool, David Nellans, Steve Keckler

List of authors in order
Landing page: https://arxiv.org/abs/1903.02596

Publisher landing page
PDF URL: https://arxiv.org/pdf/1903.02596

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/1903.02596

Direct OA link when available
Concepts: Computer science, Parallel computing, Memory bandwidth, Interleaved memory, Memory footprint, Registered memory, Memory management, Flat memory model, Computer memory, Uniform memory access, Semiconductor memory, High memory, Supercomputer, Compression (physics), Extended memory, Computer hardware, Operating system, Materials science, Composite material

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4288419222
doi	https://doi.org/10.48550/arxiv.1903.02596
ids.openalex	https://openalex.org/W4288419222
fwci	0.0
type	preprint
title	Buddy Compression: Enabling Larger Memory for Deep Learning and HPC\n Workloads on GPUs
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10054
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9990000128746033
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1708
topics[0].subfield.display_name	Hardware and Architecture
topics[0].display_name	Parallel Computing and Optimization Techniques
topics[1].id	https://openalex.org/T11181
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9987999796867371
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1705
topics[1].subfield.display_name	Computer Networks and Communications
topics[1].display_name	Advanced Data Storage Technologies
topics[2].id	https://openalex.org/T10101
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.992900013923645
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1710
topics[2].subfield.display_name	Information Systems
topics[2].display_name	Cloud Computing and Resource Management
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C41008148
concepts[0].level	0
concepts[0].score	0.787583589553833
concepts[0].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[0].display_name	Computer science
concepts[1].id	https://openalex.org/C173608175
concepts[1].level	1
concepts[1].score	0.6870089769363403
concepts[1].wikidata	https://www.wikidata.org/wiki/Q232661
concepts[1].display_name	Parallel computing
concepts[2].id	https://openalex.org/C188045654
concepts[2].level	2
concepts[2].score	0.6517577767372131
concepts[2].wikidata	https://www.wikidata.org/wiki/Q17148339
concepts[2].display_name	Memory bandwidth
concepts[3].id	https://openalex.org/C63511323
concepts[3].level	4
concepts[3].score	0.63016676902771
concepts[3].wikidata	https://www.wikidata.org/wiki/Q908936
concepts[3].display_name	Interleaved memory
concepts[4].id	https://openalex.org/C74912251
concepts[4].level	2
concepts[4].score	0.6049656271934509
concepts[4].wikidata	https://www.wikidata.org/wiki/Q6815727
concepts[4].display_name	Memory footprint
concepts[5].id	https://openalex.org/C93446704
concepts[5].level	3
concepts[5].score	0.5185468792915344
concepts[5].wikidata	https://www.wikidata.org/wiki/Q449328
concepts[5].display_name	Registered memory
concepts[6].id	https://openalex.org/C176649486
concepts[6].level	3
concepts[6].score	0.5150432586669922
concepts[6].wikidata	https://www.wikidata.org/wiki/Q2308807
concepts[6].display_name	Memory management
concepts[7].id	https://openalex.org/C57863822
concepts[7].level	4
concepts[7].score	0.5045171976089478
concepts[7].wikidata	https://www.wikidata.org/wiki/Q905488
concepts[7].display_name	Flat memory model
concepts[8].id	https://openalex.org/C92855701
concepts[8].level	3
concepts[8].score	0.49208587408065796
concepts[8].wikidata	https://www.wikidata.org/wiki/Q5830907
concepts[8].display_name	Computer memory
concepts[9].id	https://openalex.org/C51290061
concepts[9].level	4
concepts[9].score	0.47464069724082947
concepts[9].wikidata	https://www.wikidata.org/wiki/Q1936765
concepts[9].display_name	Uniform memory access
concepts[10].id	https://openalex.org/C98986596
concepts[10].level	2
concepts[10].score	0.459823876619339
concepts[10].wikidata	https://www.wikidata.org/wiki/Q1143031
concepts[10].display_name	Semiconductor memory
concepts[11].id	https://openalex.org/C2781357197
concepts[11].level	2
concepts[11].score	0.4551924467086792
concepts[11].wikidata	https://www.wikidata.org/wiki/Q5757597
concepts[11].display_name	High memory
concepts[12].id	https://openalex.org/C83283714
concepts[12].level	2
concepts[12].score	0.4501035213470459
concepts[12].wikidata	https://www.wikidata.org/wiki/Q121117
concepts[12].display_name	Supercomputer
concepts[13].id	https://openalex.org/C180016635
concepts[13].level	2
concepts[13].score	0.4254900813102722
concepts[13].wikidata	https://www.wikidata.org/wiki/Q2712821
concepts[13].display_name	Compression (physics)
concepts[14].id	https://openalex.org/C171675096
concepts[14].level	4
concepts[14].score	0.42528587579727173
concepts[14].wikidata	https://www.wikidata.org/wiki/Q1143380
concepts[14].display_name	Extended memory
concepts[15].id	https://openalex.org/C9390403
concepts[15].level	1
concepts[15].score	0.3568421006202698
concepts[15].wikidata	https://www.wikidata.org/wiki/Q3966
concepts[15].display_name	Computer hardware
concepts[16].id	https://openalex.org/C111919701
concepts[16].level	1
concepts[16].score	0.19625455141067505
concepts[16].wikidata	https://www.wikidata.org/wiki/Q9135
concepts[16].display_name	Operating system
concepts[17].id	https://openalex.org/C192562407
concepts[17].level	0
concepts[17].score	0.0777445137500763
concepts[17].wikidata	https://www.wikidata.org/wiki/Q228736
concepts[17].display_name	Materials science
concepts[18].id	https://openalex.org/C159985019
concepts[18].level	1
concepts[18].score	0.0
concepts[18].wikidata	https://www.wikidata.org/wiki/Q181790
concepts[18].display_name	Composite material
keywords[0].id	https://openalex.org/keywords/computer-science
keywords[0].score	0.787583589553833
keywords[0].display_name	Computer science
keywords[1].id	https://openalex.org/keywords/parallel-computing
keywords[1].score	0.6870089769363403
keywords[1].display_name	Parallel computing
keywords[2].id	https://openalex.org/keywords/memory-bandwidth
keywords[2].score	0.6517577767372131
keywords[2].display_name	Memory bandwidth
keywords[3].id	https://openalex.org/keywords/interleaved-memory
keywords[3].score	0.63016676902771
keywords[3].display_name	Interleaved memory
keywords[4].id	https://openalex.org/keywords/memory-footprint
keywords[4].score	0.6049656271934509
keywords[4].display_name	Memory footprint
keywords[5].id	https://openalex.org/keywords/registered-memory
keywords[5].score	0.5185468792915344
keywords[5].display_name	Registered memory
keywords[6].id	https://openalex.org/keywords/memory-management
keywords[6].score	0.5150432586669922
keywords[6].display_name	Memory management
keywords[7].id	https://openalex.org/keywords/flat-memory-model
keywords[7].score	0.5045171976089478
keywords[7].display_name	Flat memory model
keywords[8].id	https://openalex.org/keywords/computer-memory
keywords[8].score	0.49208587408065796
keywords[8].display_name	Computer memory
keywords[9].id	https://openalex.org/keywords/uniform-memory-access
keywords[9].score	0.47464069724082947
keywords[9].display_name	Uniform memory access
keywords[10].id	https://openalex.org/keywords/semiconductor-memory
keywords[10].score	0.459823876619339
keywords[10].display_name	Semiconductor memory
keywords[11].id	https://openalex.org/keywords/high-memory
keywords[11].score	0.4551924467086792
keywords[11].display_name	High memory
keywords[12].id	https://openalex.org/keywords/supercomputer
keywords[12].score	0.4501035213470459
keywords[12].display_name	Supercomputer
keywords[13].id	https://openalex.org/keywords/compression
keywords[13].score	0.4254900813102722
keywords[13].display_name	Compression (physics)
keywords[14].id	https://openalex.org/keywords/extended-memory
keywords[14].score	0.42528587579727173
keywords[14].display_name	Extended memory
keywords[15].id	https://openalex.org/keywords/computer-hardware
keywords[15].score	0.3568421006202698
keywords[15].display_name	Computer hardware
keywords[16].id	https://openalex.org/keywords/operating-system
keywords[16].score	0.19625455141067505
keywords[16].display_name	Operating system
keywords[17].id	https://openalex.org/keywords/materials-science
keywords[17].score	0.0777445137500763
keywords[17].display_name	Materials science
language	en
locations[0].id	pmh:oai:arXiv.org:1903.02596
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/1903.02596
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/1903.02596
indexed_in	arxiv
authorships[0].author.id	https://openalex.org/A5045708568
authorships[0].author.orcid	https://orcid.org/0000-0003-0371-5522
authorships[0].author.display_name	Esha Choukse
authorships[0].author_position	first
authorships[0].raw_author_name	Choukse, Esha
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5069505762
authorships[1].author.orcid	https://orcid.org/0000-0002-8606-5889
authorships[1].author.display_name	Michael Sullivan
authorships[1].author_position	middle
authorships[1].raw_author_name	Sullivan, Michael
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5055699570
authorships[2].author.orcid	https://orcid.org/0000-0003-0944-2393
authorships[2].author.display_name	Mike O’Connor
authorships[2].author_position	middle
authorships[2].raw_author_name	O'Connor, Mike
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5013680653
authorships[3].author.orcid	https://orcid.org/0000-0002-1567-4097
authorships[3].author.display_name	Mattan Erez
authorships[3].author_position	middle
authorships[3].raw_author_name	Erez, Mattan
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5063722719
authorships[4].author.orcid
authorships[4].author.display_name	Jeff Pool
authorships[4].author_position	middle
authorships[4].raw_author_name	Pool, Jeff
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5031781240
authorships[5].author.orcid	https://orcid.org/0000-0001-5203-8367
authorships[5].author.display_name	David Nellans
authorships[5].author_position	middle
authorships[5].raw_author_name	Nellans, David
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5110238116
authorships[6].author.orcid
authorships[6].author.display_name	Steve Keckler
authorships[6].author_position	last
authorships[6].raw_author_name	Keckler, Steve
authorships[6].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/1903.02596
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Buddy Compression: Enabling Larger Memory for Deep Learning and HPC\n Workloads on GPUs
has_fulltext	True
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T10054
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9990000128746033
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1708
primary_topic.subfield.display_name	Hardware and Architecture
primary_topic.display_name	Parallel Computing and Optimization Techniques
related_works	https://openalex.org/W2155373950, https://openalex.org/W2587873888, https://openalex.org/W2354036839, https://openalex.org/W4248614727, https://openalex.org/W2296275612, https://openalex.org/W4321458411, https://openalex.org/W3048967625, https://openalex.org/W2612506697, https://openalex.org/W2565280077, https://openalex.org/W2782503170
cited_by_count	0
locations_count	1
best_oa_location.id	pmh:oai:arXiv.org:1903.02596
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/1903.02596
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/1903.02596
primary_location.id	pmh:oai:arXiv.org:1903.02596
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/1903.02596
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/1903.02596
publication_date	2019-03-06
publication_year	2019
referenced_works_count	0
abstract_inverted_index.a	32, 64, 135
abstract_inverted_index.We	116
abstract_inverted_index.an	121
abstract_inverted_index.be	15, 23, 75
abstract_inverted_index.by	25
abstract_inverted_index.of	47, 125, 137
abstract_inverted_index.on	88, 127, 132
abstract_inverted_index.or	106
abstract_inverted_index.to	14, 34, 99
abstract_inverted_index.us	98
abstract_inverted_index.(or	68
abstract_inverted_index.GPU	10, 39, 53, 94
abstract_inverted_index.HPC	101, 128
abstract_inverted_index.and	63, 89, 103, 130
abstract_inverted_index.can	21, 73
abstract_inverted_index.for	108
abstract_inverted_index.not	22
abstract_inverted_index.our	119
abstract_inverted_index.the	26, 37, 45, 92
abstract_inverted_index.1.5x	131
abstract_inverted_index.2.2x	126
abstract_inverted_index.GPUs	0
abstract_inverted_index.This	28
abstract_inverted_index.both	36, 87
abstract_inverted_index.data	85
abstract_inverted_index.each	56
abstract_inverted_index.from	78
abstract_inverted_index.just	138
abstract_inverted_index.pool	67
abstract_inverted_index.show	117
abstract_inverted_index.than	6, 110
abstract_inverted_index.that	118
abstract_inverted_index.thus	74
abstract_inverted_index.with	134
abstract_inverted_index.Buddy	50
abstract_inverted_index.entry	58
abstract_inverted_index.offer	1
abstract_inverted_index.paper	29
abstract_inverted_index.ratio	124
abstract_inverted_index.small	17
abstract_inverted_index.tends	13
abstract_inverted_index.their	84
abstract_inverted_index.user.	27
abstract_inverted_index.using	86
abstract_inverted_index.while	43
abstract_inverted_index.would	114
abstract_inverted_index.allow.	115
abstract_inverted_index.device	11, 61, 79
abstract_inverted_index.higher	3
abstract_inverted_index.larger	104
abstract_inverted_index.memory	4, 12, 19, 40, 62, 66, 95, 112
abstract_inverted_index.models	107
abstract_inverted_index.scheme	33
abstract_inverted_index.source	83
abstract_inverted_index.system	69
abstract_inverted_index.1~2%.\n	139
abstract_inverted_index.average	122
abstract_inverted_index.between	59
abstract_inverted_index.current	111
abstract_inverted_index.enables	97
abstract_inverted_index.entries	82
abstract_inverted_index.memory,	54, 80
abstract_inverted_index.However,	9
abstract_inverted_index.accessed	76
abstract_inverted_index.and\nthe	18
abstract_inverted_index.avoiding	44
abstract_inverted_index.capacity	20, 96
abstract_inverted_index.increase	35
abstract_inverted_index.memory).	70
abstract_inverted_index.slowdown	136
abstract_inverted_index.systems.	8
abstract_inverted_index.bandwidth	5, 42
abstract_inverted_index.downsides	46
abstract_inverted_index.effective	38, 93
abstract_inverted_index.increased	24
abstract_inverted_index.splitting	55
abstract_inverted_index.workloads	102, 129
abstract_inverted_index.capacities	113
abstract_inverted_index.completely	77
abstract_inverted_index.compresses	52
abstract_inverted_index.high-speed	60
abstract_inverted_index.off-device	90
abstract_inverted_index.relatively	16
abstract_inverted_index.Compression	51
abstract_inverted_index.batch-sizes	105
abstract_inverted_index.compression	123
abstract_inverted_index.Compression,	31
abstract_inverted_index.conventional	48
abstract_inverted_index.DL\nworkloads	109
abstract_inverted_index.capacity\nand	41
abstract_inverted_index.DL\nworkloads,	133
abstract_inverted_index.memory\nentries	72
abstract_inverted_index.describes\nBuddy	30
abstract_inverted_index.compressed\nmemory	57
abstract_inverted_index.solution\nachieves	120
abstract_inverted_index.Highly-compressible	71
abstract_inverted_index.orders-of-magnitude	2
abstract_inverted_index.accesses.\nIncreasing	91
abstract_inverted_index.traditional\nCPU-only	7
abstract_inverted_index.while\nincompressible	81
abstract_inverted_index.run\nlarger-memory-footprint	100
abstract_inverted_index.memory-expanding\nstrategies.	49
abstract_inverted_index.slower-but-larger\ndisaggregated	65
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	7
sustainable_development_goals[0].id	https://metadata.un.org/sdg/7
sustainable_development_goals[0].score	0.7400000095367432
sustainable_development_goals[0].display_name	Affordable and clean energy
citation_normalized_percentile.value	0.33613011
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False