Symmetry in Neural Network Parameter Spaces Article Swipe

PDF

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2506.13018

Modern deep learning models are highly overparameterized, resulting in large sets of parameter configurations that yield the same outputs. A significant portion of this redundancy is explained by symmetries in the parameter space--transformations that leave the network function unchanged. These symmetries shape the loss landscape and constrain learning dynamics, offering a new lens for understanding optimization, generalization, and model complexity that complements existing theory of deep learning. This survey provides an overview of parameter space symmetry. We summarize existing literature, uncover connections between symmetry and learning theory, and identify gaps and opportunities in this emerging field.

Related Topics

Neural Correlates Of Consciousness

Convolutional Neural Network

Network (1976 Film)

Usa Network

Recurrent Neural Network

Mastodon (Social Network)

Network Effect (Novel)

Gma Network

Mlb Network

Network-Attached Storage

Origin Of Symmetry

Rectifier (Neural Networks)

Concepts

No concepts available.

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2506.13018
PDF: https://arxiv.org/pdf/2506.13018
OA Status: green
OpenAlex ID: https://openalex.org/W4415124299

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4415124299

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2506.13018

Digital Object Identifier
Title: Symmetry in Neural Network Parameter Spaces

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-06-16

Full publication date if available
Authors: Bo Zhao, Robin Walters, Rose Yu

List of authors in order
Landing page: https://arxiv.org/abs/2506.13018

Publisher landing page
PDF URL: https://arxiv.org/pdf/2506.13018

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2506.13018

Direct OA link when available
Cited by: 0

Total citation count in OpenAlex

Full payload

id	https://openalex.org/W4415124299
doi	https://doi.org/10.48550/arxiv.2506.13018
ids.doi	https://doi.org/10.48550/arxiv.2506.13018
ids.openalex	https://openalex.org/W4415124299
fwci
type	preprint
title	Symmetry in Neural Network Parameter Spaces
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10320
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.7806000113487244
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Neural Networks and Applications
is_xpac	False
apc_list
apc_paid
language	en
locations[0].id	pmh:oai:arXiv.org:2506.13018
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2506.13018
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2506.13018
locations[1].id	doi:10.48550/arxiv.2506.13018
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2506.13018
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5100631708
authorships[0].author.orcid	https://orcid.org/0000-0003-3218-6471
authorships[0].author.display_name	Bo Zhao
authorships[0].author_position	first
authorships[0].raw_author_name	Zhao, Bo
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5020217201
authorships[1].author.orcid	https://orcid.org/0000-0001-7751-2379
authorships[1].author.display_name	Robin Walters
authorships[1].author_position	middle
authorships[1].raw_author_name	Walters, Robin
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5057778679
authorships[2].author.orcid	https://orcid.org/0000-0002-8491-7937
authorships[2].author.display_name	Rose Yu
authorships[2].author_position	last
authorships[2].raw_author_name	Yu, Rose
authorships[2].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2506.13018
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-14T00:00:00
display_name	Symmetry in Neural Network Parameter Spaces
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10320
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.7806000113487244
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Neural Networks and Applications
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2506.13018
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2506.13018
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2506.13018
primary_location.id	pmh:oai:arXiv.org:2506.13018
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2506.13018
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2506.13018
publication_date	2025-06-16
publication_year	2025
referenced_works_count	0
abstract_inverted_index.A	19
abstract_inverted_index.a	50
abstract_inverted_index.We	76
abstract_inverted_index.an	70
abstract_inverted_index.by	27
abstract_inverted_index.in	8, 29, 92
abstract_inverted_index.is	25
abstract_inverted_index.of	11, 22, 64, 72
abstract_inverted_index.and	45, 57, 84, 87, 90
abstract_inverted_index.are	4
abstract_inverted_index.for	53
abstract_inverted_index.new	51
abstract_inverted_index.the	16, 30, 35, 42
abstract_inverted_index.This	67
abstract_inverted_index.deep	1, 65
abstract_inverted_index.gaps	89
abstract_inverted_index.lens	52
abstract_inverted_index.loss	43
abstract_inverted_index.same	17
abstract_inverted_index.sets	10
abstract_inverted_index.that	14, 33, 60
abstract_inverted_index.this	23, 93
abstract_inverted_index.These	39
abstract_inverted_index.large	9
abstract_inverted_index.leave	34
abstract_inverted_index.model	58
abstract_inverted_index.shape	41
abstract_inverted_index.space	74
abstract_inverted_index.yield	15
abstract_inverted_index.Modern	0
abstract_inverted_index.field.	95
abstract_inverted_index.highly	5
abstract_inverted_index.models	3
abstract_inverted_index.survey	68
abstract_inverted_index.theory	63
abstract_inverted_index.between	82
abstract_inverted_index.network	36
abstract_inverted_index.portion	21
abstract_inverted_index.theory,	86
abstract_inverted_index.uncover	80
abstract_inverted_index.emerging	94
abstract_inverted_index.existing	62, 78
abstract_inverted_index.function	37
abstract_inverted_index.identify	88
abstract_inverted_index.learning	2, 47, 85
abstract_inverted_index.offering	49
abstract_inverted_index.outputs.	18
abstract_inverted_index.overview	71
abstract_inverted_index.provides	69
abstract_inverted_index.symmetry	83
abstract_inverted_index.constrain	46
abstract_inverted_index.dynamics,	48
abstract_inverted_index.explained	26
abstract_inverted_index.landscape	44
abstract_inverted_index.learning.	66
abstract_inverted_index.parameter	12, 31, 73
abstract_inverted_index.resulting	7
abstract_inverted_index.summarize	77
abstract_inverted_index.symmetry.	75
abstract_inverted_index.complexity	59
abstract_inverted_index.redundancy	24
abstract_inverted_index.symmetries	28, 40
abstract_inverted_index.unchanged.	38
abstract_inverted_index.complements	61
abstract_inverted_index.connections	81
abstract_inverted_index.literature,	79
abstract_inverted_index.significant	20
abstract_inverted_index.opportunities	91
abstract_inverted_index.optimization,	55
abstract_inverted_index.understanding	54
abstract_inverted_index.configurations	13
abstract_inverted_index.generalization,	56
abstract_inverted_index.overparameterized,	6
abstract_inverted_index.space--transformations	32
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	3
citation_normalized_percentile