GNNAutoScale: Scalable and Expressive Graph Neural Networks via\n Historical Embeddings Article Swipe
Matthias Fey
,
Jan Eric Lenssen
,
Frank Weichert
,
Jure Leskovec
·
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2106.05609
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2106.05609
We present GNNAutoScale (GAS), a framework for scaling arbitrary\nmessage-passing GNNs to large graphs. GAS prunes entire sub-trees of the\ncomputation graph by utilizing historical embeddings from prior training\niterations, leading to constant GPU memory consumption in respect to input node\nsize without dropping any data. While existing solutions weaken the expressive\npower of message passing due to sub-sampling of edges or non-trainable\npropagations, our approach is provably able to maintain the expressive power of\nthe original GNN. We achieve this by providing approximation error bounds of\nhistorical embeddings and show how to tighten them in practice. Empirically, we\nshow that the practical realization of our framework, PyGAS, an easy-to-use\nextension for PyTorch Geometric, is both fast and memory-efficient, learns\nexpressive node representations, closely resembles the performance of their\nnon-scaling counterparts, and reaches state-of-the-art performance on\nlarge-scale graphs.\n
Related Topics
Concepts
Metadata
- Type
- preprint
- Landing Page
- http://arxiv.org/abs/2106.05609
- https://arxiv.org/pdf/2106.05609
- OA Status
- green
- Cited By
- 16
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4287121301
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4287121301Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2106.05609Digital Object Identifier
- Title
-
GNNAutoScale: Scalable and Expressive Graph Neural Networks via\n Historical EmbeddingsWork title
- Type
-
preprintOpenAlex work type
- Publication year
-
2021Year of publication
- Publication date
-
2021-06-10Full publication date if available
- Authors
-
Matthias Fey, Jan Eric Lenssen, Frank Weichert, Jure LeskovecList of authors in order
- Landing page
-
https://arxiv.org/abs/2106.05609Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2106.05609Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2106.05609Direct OA link when available
- Concepts
-
Computer science, Scalability, Expressive power, Scaling, Theoretical computer science, Computation, Graph, Node (physics), Realization (probability), Algorithm, Mathematics, Geometry, Statistics, Structural engineering, Engineering, DatabaseTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
16Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 2, 2024: 3, 2023: 6, 2022: 5Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4287121301 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2106.05609 |
| ids.openalex | https://openalex.org/W4287121301 |
| fwci | 1.97545005 |
| type | preprint |
| title | GNNAutoScale: Scalable and Expressive Graph Neural Networks via\n Historical Embeddings |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11273 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9988999962806702 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Advanced Graph Neural Networks |
| topics[1].id | https://openalex.org/T10036 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9771000146865845 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1707 |
| topics[1].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[1].display_name | Advanced Neural Network Applications |
| topics[2].id | https://openalex.org/T12292 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9718999862670898 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1707 |
| topics[2].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[2].display_name | Graph Theory and Algorithms |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.8255747556686401 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C48044578 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7901897430419922 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q727490 |
| concepts[1].display_name | Scalability |
| concepts[2].id | https://openalex.org/C195818886 |
| concepts[2].level | 2 |
| concepts[2].score | 0.7076860666275024 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q5421724 |
| concepts[2].display_name | Expressive power |
| concepts[3].id | https://openalex.org/C99844830 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5761661529541016 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q102441924 |
| concepts[3].display_name | Scaling |
| concepts[4].id | https://openalex.org/C80444323 |
| concepts[4].level | 1 |
| concepts[4].score | 0.5643763542175293 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q2878974 |
| concepts[4].display_name | Theoretical computer science |
| concepts[5].id | https://openalex.org/C45374587 |
| concepts[5].level | 2 |
| concepts[5].score | 0.5617804527282715 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q12525525 |
| concepts[5].display_name | Computation |
| concepts[6].id | https://openalex.org/C132525143 |
| concepts[6].level | 2 |
| concepts[6].score | 0.5416905879974365 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q141488 |
| concepts[6].display_name | Graph |
| concepts[7].id | https://openalex.org/C62611344 |
| concepts[7].level | 2 |
| concepts[7].score | 0.5360994338989258 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q1062658 |
| concepts[7].display_name | Node (physics) |
| concepts[8].id | https://openalex.org/C2781089630 |
| concepts[8].level | 2 |
| concepts[8].score | 0.42653200030326843 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q21856745 |
| concepts[8].display_name | Realization (probability) |
| concepts[9].id | https://openalex.org/C11413529 |
| concepts[9].level | 1 |
| concepts[9].score | 0.21200335025787354 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[9].display_name | Algorithm |
| concepts[10].id | https://openalex.org/C33923547 |
| concepts[10].level | 0 |
| concepts[10].score | 0.1105826199054718 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[10].display_name | Mathematics |
| concepts[11].id | https://openalex.org/C2524010 |
| concepts[11].level | 1 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q8087 |
| concepts[11].display_name | Geometry |
| concepts[12].id | https://openalex.org/C105795698 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q12483 |
| concepts[12].display_name | Statistics |
| concepts[13].id | https://openalex.org/C66938386 |
| concepts[13].level | 1 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q633538 |
| concepts[13].display_name | Structural engineering |
| concepts[14].id | https://openalex.org/C127413603 |
| concepts[14].level | 0 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[14].display_name | Engineering |
| concepts[15].id | https://openalex.org/C77088390 |
| concepts[15].level | 1 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q8513 |
| concepts[15].display_name | Database |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.8255747556686401 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/scalability |
| keywords[1].score | 0.7901897430419922 |
| keywords[1].display_name | Scalability |
| keywords[2].id | https://openalex.org/keywords/expressive-power |
| keywords[2].score | 0.7076860666275024 |
| keywords[2].display_name | Expressive power |
| keywords[3].id | https://openalex.org/keywords/scaling |
| keywords[3].score | 0.5761661529541016 |
| keywords[3].display_name | Scaling |
| keywords[4].id | https://openalex.org/keywords/theoretical-computer-science |
| keywords[4].score | 0.5643763542175293 |
| keywords[4].display_name | Theoretical computer science |
| keywords[5].id | https://openalex.org/keywords/computation |
| keywords[5].score | 0.5617804527282715 |
| keywords[5].display_name | Computation |
| keywords[6].id | https://openalex.org/keywords/graph |
| keywords[6].score | 0.5416905879974365 |
| keywords[6].display_name | Graph |
| keywords[7].id | https://openalex.org/keywords/node |
| keywords[7].score | 0.5360994338989258 |
| keywords[7].display_name | Node (physics) |
| keywords[8].id | https://openalex.org/keywords/realization |
| keywords[8].score | 0.42653200030326843 |
| keywords[8].display_name | Realization (probability) |
| keywords[9].id | https://openalex.org/keywords/algorithm |
| keywords[9].score | 0.21200335025787354 |
| keywords[9].display_name | Algorithm |
| keywords[10].id | https://openalex.org/keywords/mathematics |
| keywords[10].score | 0.1105826199054718 |
| keywords[10].display_name | Mathematics |
| language | |
| locations[0].id | pmh:oai:arXiv.org:2106.05609 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2106.05609 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2106.05609 |
| indexed_in | arxiv |
| authorships[0].author.id | https://openalex.org/A5002742256 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-5727-0701 |
| authorships[0].author.display_name | Matthias Fey |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Fey, Matthias |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5073462022 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-4093-9840 |
| authorships[1].author.display_name | Jan Eric Lenssen |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Lenssen, Jan E. |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5004108638 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-2530-8197 |
| authorships[2].author.display_name | Frank Weichert |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Weichert, Frank |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5091272738 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-5411-923X |
| authorships[3].author.display_name | Jure Leskovec |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Leskovec, Jure |
| authorships[3].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2106.05609 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2022-07-25T00:00:00 |
| display_name | GNNAutoScale: Scalable and Expressive Graph Neural Networks via\n Historical Embeddings |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T11273 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9988999962806702 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Advanced Graph Neural Networks |
| related_works | https://openalex.org/W2022544890, https://openalex.org/W2394097730, https://openalex.org/W2475378634, https://openalex.org/W4312353617, https://openalex.org/W2113405914, https://openalex.org/W2260963831, https://openalex.org/W4235249401, https://openalex.org/W2331305369, https://openalex.org/W2043523297, https://openalex.org/W2015200445 |
| cited_by_count | 16 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 2 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 3 |
| counts_by_year[2].year | 2023 |
| counts_by_year[2].cited_by_count | 6 |
| counts_by_year[3].year | 2022 |
| counts_by_year[3].cited_by_count | 5 |
| locations_count | 1 |
| best_oa_location.id | pmh:oai:arXiv.org:2106.05609 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2106.05609 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2106.05609 |
| primary_location.id | pmh:oai:arXiv.org:2106.05609 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2106.05609 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2106.05609 |
| publication_date | 2021-06-10 |
| publication_year | 2021 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 4 |
| abstract_inverted_index.We | 0, 71 |
| abstract_inverted_index.an | 99 |
| abstract_inverted_index.by | 20, 74 |
| abstract_inverted_index.in | 33, 87 |
| abstract_inverted_index.is | 60, 104 |
| abstract_inverted_index.of | 17, 48, 54, 95, 116 |
| abstract_inverted_index.or | 56 |
| abstract_inverted_index.to | 10, 28, 35, 52, 63, 84 |
| abstract_inverted_index.GAS | 13 |
| abstract_inverted_index.GPU | 30 |
| abstract_inverted_index.and | 81, 107, 119 |
| abstract_inverted_index.any | 40 |
| abstract_inverted_index.due | 51 |
| abstract_inverted_index.for | 6, 101 |
| abstract_inverted_index.how | 83 |
| abstract_inverted_index.our | 58, 96 |
| abstract_inverted_index.the | 46, 65, 92, 114 |
| abstract_inverted_index.GNN. | 70 |
| abstract_inverted_index.GNNs | 9 |
| abstract_inverted_index.able | 62 |
| abstract_inverted_index.both | 105 |
| abstract_inverted_index.fast | 106 |
| abstract_inverted_index.from | 24 |
| abstract_inverted_index.node | 110 |
| abstract_inverted_index.show | 82 |
| abstract_inverted_index.that | 91 |
| abstract_inverted_index.them | 86 |
| abstract_inverted_index.this | 73 |
| abstract_inverted_index.While | 42 |
| abstract_inverted_index.data. | 41 |
| abstract_inverted_index.edges | 55 |
| abstract_inverted_index.error | 77 |
| abstract_inverted_index.graph | 19 |
| abstract_inverted_index.input | 36 |
| abstract_inverted_index.large | 11 |
| abstract_inverted_index.power | 67 |
| abstract_inverted_index.prior | 25 |
| abstract_inverted_index.(GAS), | 3 |
| abstract_inverted_index.PyGAS, | 98 |
| abstract_inverted_index.bounds | 78 |
| abstract_inverted_index.entire | 15 |
| abstract_inverted_index.memory | 31 |
| abstract_inverted_index.prunes | 14 |
| abstract_inverted_index.weaken | 45 |
| abstract_inverted_index.PyTorch | 102 |
| abstract_inverted_index.achieve | 72 |
| abstract_inverted_index.closely | 112 |
| abstract_inverted_index.graphs. | 12 |
| abstract_inverted_index.leading | 27 |
| abstract_inverted_index.message | 49 |
| abstract_inverted_index.of\nthe | 68 |
| abstract_inverted_index.passing | 50 |
| abstract_inverted_index.present | 1 |
| abstract_inverted_index.reaches | 120 |
| abstract_inverted_index.respect | 34 |
| abstract_inverted_index.scaling | 7 |
| abstract_inverted_index.tighten | 85 |
| abstract_inverted_index.without | 38 |
| abstract_inverted_index.approach | 59 |
| abstract_inverted_index.constant | 29 |
| abstract_inverted_index.dropping | 39 |
| abstract_inverted_index.existing | 43 |
| abstract_inverted_index.maintain | 64 |
| abstract_inverted_index.original | 69 |
| abstract_inverted_index.provably | 61 |
| abstract_inverted_index.we\nshow | 90 |
| abstract_inverted_index.framework | 5 |
| abstract_inverted_index.graphs.\n | 124 |
| abstract_inverted_index.practical | 93 |
| abstract_inverted_index.practice. | 88 |
| abstract_inverted_index.providing | 75 |
| abstract_inverted_index.resembles | 113 |
| abstract_inverted_index.solutions | 44 |
| abstract_inverted_index.sub-trees | 16 |
| abstract_inverted_index.utilizing | 21 |
| abstract_inverted_index.Geometric, | 103 |
| abstract_inverted_index.embeddings | 23, 80 |
| abstract_inverted_index.expressive | 66 |
| abstract_inverted_index.framework, | 97 |
| abstract_inverted_index.historical | 22 |
| abstract_inverted_index.node\nsize | 37 |
| abstract_inverted_index.consumption | 32 |
| abstract_inverted_index.performance | 115, 122 |
| abstract_inverted_index.realization | 94 |
| abstract_inverted_index.Empirically, | 89 |
| abstract_inverted_index.GNNAutoScale | 2 |
| abstract_inverted_index.sub-sampling | 53 |
| abstract_inverted_index.approximation | 76 |
| abstract_inverted_index.counterparts, | 118 |
| abstract_inverted_index.of\nhistorical | 79 |
| abstract_inverted_index.on\nlarge-scale | 123 |
| abstract_inverted_index.representations, | 111 |
| abstract_inverted_index.state-of-the-art | 121 |
| abstract_inverted_index.the\ncomputation | 18 |
| abstract_inverted_index.expressive\npower | 47 |
| abstract_inverted_index.memory-efficient, | 108 |
| abstract_inverted_index.learns\nexpressive | 109 |
| abstract_inverted_index.their\nnon-scaling | 117 |
| abstract_inverted_index.training\niterations, | 26 |
| abstract_inverted_index.easy-to-use\nextension | 100 |
| abstract_inverted_index.arbitrary\nmessage-passing | 8 |
| abstract_inverted_index.non-trainable\npropagations, | 57 |
| cited_by_percentile_year.max | 98 |
| cited_by_percentile_year.min | 95 |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile.value | 0.89121952 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |