Residual connections provably mitigate oversmoothing in graph neural networks Article Swipe

PDF

Ziang Chen , Zhiping Lin , Shi Chen , Yury Polyanskiy , Philippe Rigollet ·

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2501.00762

Graph neural networks (GNNs) have achieved remarkable empirical success in processing and representing graph-structured data across various domains. However, a significant challenge known as "oversmoothing" persists, where vertex features become nearly indistinguishable in deep GNNs, severely restricting their expressive power and practical utility. In this work, we analyze the asymptotic oversmoothing rates of deep GNNs with and without residual connections by deriving explicit convergence rates for a normalized vertex similarity measure. Our analytical framework is grounded in the multiplicative ergodic theorem. Furthermore, we demonstrate that adding residual connections effectively mitigates or prevents oversmoothing across several broad families of parameter distributions. The theoretical findings are strongly supported by numerical experiments.

Related Topics

Mathematics

Computer Science

Discrete Mathematics

Mathematical Analysis

Concepts

Ergodic theory Multiplicative function Graph Mathematics Computer science Discrete mathematics Pure mathematics Mathematical analysis

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2501.00762
PDF: https://arxiv.org/pdf/2501.00762
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4406032773

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4406032773

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2501.00762

Digital Object Identifier
Title: Residual connections provably mitigate oversmoothing in graph neural networks

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-01-01

Full publication date if available
Authors: Ziang Chen, Zhiping Lin, Shi Chen, Yury Polyanskiy, Philippe Rigollet

List of authors in order
Landing page: https://arxiv.org/abs/2501.00762

Publisher landing page
PDF URL: https://arxiv.org/pdf/2501.00762

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2501.00762

Direct OA link when available
Concepts: Ergodic theory, Multiplicative function, Graph, Mathematics, Computer science, Discrete mathematics, Pure mathematics, Mathematical analysis

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4406032773
doi	https://doi.org/10.48550/arxiv.2501.00762
ids.doi	https://doi.org/10.48550/arxiv.2501.00762
ids.openalex	https://openalex.org/W4406032773
fwci
type	preprint
title	Residual connections provably mitigate oversmoothing in graph neural networks
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10320
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.8123999834060669
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Neural Networks and Applications
topics[1].id	https://openalex.org/T10241
topics[1].field.id	https://openalex.org/fields/28
topics[1].field.display_name	Neuroscience
topics[1].score	0.8084999918937683
topics[1].domain.id	https://openalex.org/domains/1
topics[1].domain.display_name	Life Sciences
topics[1].subfield.id	https://openalex.org/subfields/2805
topics[1].subfield.display_name	Cognitive Neuroscience
topics[1].display_name	Functional Brain Connectivity Studies
topics[2].id	https://openalex.org/T11520
topics[2].field.id	https://openalex.org/fields/31
topics[2].field.display_name	Physics and Astronomy
topics[2].score	0.8033999800682068
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/3109
topics[2].subfield.display_name	Statistical and Nonlinear Physics
topics[2].display_name	Advanced Thermodynamics and Statistical Mechanics
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C122044880
concepts[0].level	2
concepts[0].score	0.8101941347122192
concepts[0].wikidata	https://www.wikidata.org/wiki/Q5498822
concepts[0].display_name	Ergodic theory
concepts[1].id	https://openalex.org/C42747912
concepts[1].level	2
concepts[1].score	0.7065719962120056
concepts[1].wikidata	https://www.wikidata.org/wiki/Q1048447
concepts[1].display_name	Multiplicative function
concepts[2].id	https://openalex.org/C132525143
concepts[2].level	2
concepts[2].score	0.5297585129737854
concepts[2].wikidata	https://www.wikidata.org/wiki/Q141488
concepts[2].display_name	Graph
concepts[3].id	https://openalex.org/C33923547
concepts[3].level	0
concepts[3].score	0.3687708079814911
concepts[3].wikidata	https://www.wikidata.org/wiki/Q395
concepts[3].display_name	Mathematics
concepts[4].id	https://openalex.org/C41008148
concepts[4].level	0
concepts[4].score	0.3684704899787903
concepts[4].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[4].display_name	Computer science
concepts[5].id	https://openalex.org/C118615104
concepts[5].level	1
concepts[5].score	0.22428733110427856
concepts[5].wikidata	https://www.wikidata.org/wiki/Q121416
concepts[5].display_name	Discrete mathematics
concepts[6].id	https://openalex.org/C202444582
concepts[6].level	1
concepts[6].score	0.09126311540603638
concepts[6].wikidata	https://www.wikidata.org/wiki/Q837863
concepts[6].display_name	Pure mathematics
concepts[7].id	https://openalex.org/C134306372
concepts[7].level	1
concepts[7].score	0.0
concepts[7].wikidata	https://www.wikidata.org/wiki/Q7754
concepts[7].display_name	Mathematical analysis
keywords[0].id	https://openalex.org/keywords/ergodic-theory
keywords[0].score	0.8101941347122192
keywords[0].display_name	Ergodic theory
keywords[1].id	https://openalex.org/keywords/multiplicative-function
keywords[1].score	0.7065719962120056
keywords[1].display_name	Multiplicative function
keywords[2].id	https://openalex.org/keywords/graph
keywords[2].score	0.5297585129737854
keywords[2].display_name	Graph
keywords[3].id	https://openalex.org/keywords/mathematics
keywords[3].score	0.3687708079814911
keywords[3].display_name	Mathematics
keywords[4].id	https://openalex.org/keywords/computer-science
keywords[4].score	0.3684704899787903
keywords[4].display_name	Computer science
keywords[5].id	https://openalex.org/keywords/discrete-mathematics
keywords[5].score	0.22428733110427856
keywords[5].display_name	Discrete mathematics
keywords[6].id	https://openalex.org/keywords/pure-mathematics
keywords[6].score	0.09126311540603638
keywords[6].display_name	Pure mathematics
language	en
locations[0].id	pmh:oai:arXiv.org:2501.00762
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2501.00762
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2501.00762
locations[1].id	doi:10.48550/arxiv.2501.00762
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2501.00762
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5101457896
authorships[0].author.orcid	https://orcid.org/0000-0002-8298-5223
authorships[0].author.display_name	Ziang Chen
authorships[0].author_position	first
authorships[0].raw_author_name	Chen, Ziang
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5049506273
authorships[1].author.orcid	https://orcid.org/0000-0002-1587-1226
authorships[1].author.display_name	Zhiping Lin
authorships[1].author_position	middle
authorships[1].raw_author_name	Lin, Zhengjiang
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5100709711
authorships[2].author.orcid	https://orcid.org/0000-0002-3524-3577
authorships[2].author.display_name	Shi Chen
authorships[2].author_position	middle
authorships[2].raw_author_name	Chen, Shi
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5031031216
authorships[3].author.orcid
authorships[3].author.display_name	Yury Polyanskiy
authorships[3].author_position	middle
authorships[3].raw_author_name	Polyanskiy, Yury
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5001421646
authorships[4].author.orcid	https://orcid.org/0000-0002-1554-8912
authorships[4].author.display_name	Philippe Rigollet
authorships[4].author_position	last
authorships[4].raw_author_name	Rigollet, Philippe
authorships[4].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2501.00762
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-01-04T00:00:00
display_name	Residual connections provably mitigate oversmoothing in graph neural networks
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10320
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.8123999834060669
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Neural Networks and Applications
related_works	https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W4391375266, https://openalex.org/W2390585021, https://openalex.org/W412641959, https://openalex.org/W50969306, https://openalex.org/W4247405823, https://openalex.org/W1479682422, https://openalex.org/W2966636294, https://openalex.org/W4288265580
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2501.00762
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2501.00762
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2501.00762
primary_location.id	pmh:oai:arXiv.org:2501.00762
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2501.00762
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2501.00762
publication_date	2025-01-01
publication_year	2025
referenced_works_count	0
abstract_inverted_index.a	19, 66
abstract_inverted_index.In	43
abstract_inverted_index.as	23
abstract_inverted_index.by	60, 106
abstract_inverted_index.in	9, 32, 76
abstract_inverted_index.is	74
abstract_inverted_index.of	52, 97
abstract_inverted_index.or	90
abstract_inverted_index.we	46, 82
abstract_inverted_index.Our	71
abstract_inverted_index.The	100
abstract_inverted_index.and	11, 40, 56
abstract_inverted_index.are	103
abstract_inverted_index.for	65
abstract_inverted_index.the	48, 77
abstract_inverted_index.GNNs	54
abstract_inverted_index.data	14
abstract_inverted_index.deep	33, 53
abstract_inverted_index.have	4
abstract_inverted_index.that	84
abstract_inverted_index.this	44
abstract_inverted_index.with	55
abstract_inverted_index.GNNs,	34
abstract_inverted_index.Graph	0
abstract_inverted_index.broad	95
abstract_inverted_index.known	22
abstract_inverted_index.power	39
abstract_inverted_index.rates	51, 64
abstract_inverted_index.their	37
abstract_inverted_index.where	26
abstract_inverted_index.work,	45
abstract_inverted_index.(GNNs)	3
abstract_inverted_index.across	15, 93
abstract_inverted_index.adding	85
abstract_inverted_index.become	29
abstract_inverted_index.nearly	30
abstract_inverted_index.neural	1
abstract_inverted_index.vertex	27, 68
abstract_inverted_index.analyze	47
abstract_inverted_index.ergodic	79
abstract_inverted_index.several	94
abstract_inverted_index.success	8
abstract_inverted_index.various	16
abstract_inverted_index.without	57
abstract_inverted_index.However,	18
abstract_inverted_index.achieved	5
abstract_inverted_index.deriving	61
abstract_inverted_index.domains.	17
abstract_inverted_index.explicit	62
abstract_inverted_index.families	96
abstract_inverted_index.features	28
abstract_inverted_index.findings	102
abstract_inverted_index.grounded	75
abstract_inverted_index.measure.	70
abstract_inverted_index.networks	2
abstract_inverted_index.prevents	91
abstract_inverted_index.residual	58, 86
abstract_inverted_index.severely	35
abstract_inverted_index.strongly	104
abstract_inverted_index.theorem.	80
abstract_inverted_index.utility.	42
abstract_inverted_index.challenge	21
abstract_inverted_index.empirical	7
abstract_inverted_index.framework	73
abstract_inverted_index.mitigates	89
abstract_inverted_index.numerical	107
abstract_inverted_index.parameter	98
abstract_inverted_index.persists,	25
abstract_inverted_index.practical	41
abstract_inverted_index.supported	105
abstract_inverted_index.analytical	72
abstract_inverted_index.asymptotic	49
abstract_inverted_index.expressive	38
abstract_inverted_index.normalized	67
abstract_inverted_index.processing	10
abstract_inverted_index.remarkable	6
abstract_inverted_index.similarity	69
abstract_inverted_index.connections	59, 87
abstract_inverted_index.convergence	63
abstract_inverted_index.demonstrate	83
abstract_inverted_index.effectively	88
abstract_inverted_index.restricting	36
abstract_inverted_index.significant	20
abstract_inverted_index.theoretical	101
abstract_inverted_index.Furthermore,	81
abstract_inverted_index.experiments.	108
abstract_inverted_index.representing	12
abstract_inverted_index.oversmoothing	50, 92
abstract_inverted_index.distributions.	99
abstract_inverted_index.multiplicative	78
abstract_inverted_index."oversmoothing"	24
abstract_inverted_index.graph-structured	13
abstract_inverted_index.indistinguishable	31
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	5
citation_normalized_percentile