CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations Article Swipe

PDF

Jules J. Berman , Benjamin Peherstorfer ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2402.14646

This work introduces reduced models based on Continuous Low Rank Adaptation (CoLoRA) that pre-train neural networks for a given partial differential equation and then continuously adapt low-rank weights in time to rapidly predict the evolution of solution fields at new physics parameters and new initial conditions. The adaptation can be either purely data-driven or via an equation-driven variational approach that provides Galerkin-optimal approximations. Because CoLoRA approximates solution fields locally in time, the rank of the weights can be kept small, which means that only few training trajectories are required offline so that CoLoRA is well suited for data-scarce regimes. Predictions with CoLoRA are orders of magnitude faster than with classical methods and their accuracy and parameter efficiency is higher compared to other neural network approaches.

Related Topics

Partial Differential Equation

Computer Science

Mathematics

Mathematical Analysis

Algorithm

Artificial Intelligence

Neuroscience

Combinatorics

Concepts

Parameterized complexity Adaptation (eye) Rank (graph theory) Partial differential equation Computer science Applied mathematics Artificial neural network Mathematics Control theory (sociology) Mathematical analysis Algorithm Artificial intelligence Neuroscience Psychology Combinatorics Control (management)

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2402.14646
PDF: https://arxiv.org/pdf/2402.14646
OA Status: green
Cited By: 3
Related Works: 10
OpenAlex ID: https://openalex.org/W4392121003

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4392121003

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2402.14646

Digital Object Identifier
Title: CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-02-22

Full publication date if available
Authors: Jules J. Berman, Benjamin Peherstorfer

List of authors in order
Landing page: https://arxiv.org/abs/2402.14646

Publisher landing page
PDF URL: https://arxiv.org/pdf/2402.14646

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2402.14646

Direct OA link when available
Concepts: Parameterized complexity, Adaptation (eye), Rank (graph theory), Partial differential equation, Computer science, Applied mathematics, Artificial neural network, Mathematics, Control theory (sociology), Mathematical analysis, Algorithm, Artificial intelligence, Neuroscience, Psychology, Combinatorics, Control (management)

Top concepts (fields/topics) attached by OpenAlex
Cited by: 3

Total citation count in OpenAlex
Citations by year (recent): 2025: 1, 2024: 2

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4392121003
doi	https://doi.org/10.48550/arxiv.2402.14646
ids.doi	https://doi.org/10.48550/arxiv.2402.14646
ids.openalex	https://openalex.org/W4392121003
fwci
type	preprint
title	CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T11206
topics[0].field.id	https://openalex.org/fields/31
topics[0].field.display_name	Physics and Astronomy
topics[0].score	0.9990000128746033
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/3109
topics[0].subfield.display_name	Statistical and Nonlinear Physics
topics[0].display_name	Model Reduction and Neural Networks
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C165464430
concepts[0].level	2
concepts[0].score	0.8565719127655029
concepts[0].wikidata	https://www.wikidata.org/wiki/Q1570441
concepts[0].display_name	Parameterized complexity
concepts[1].id	https://openalex.org/C139807058
concepts[1].level	2
concepts[1].score	0.7131462693214417
concepts[1].wikidata	https://www.wikidata.org/wiki/Q352374
concepts[1].display_name	Adaptation (eye)
concepts[2].id	https://openalex.org/C164226766
concepts[2].level	2
concepts[2].score	0.6492642164230347
concepts[2].wikidata	https://www.wikidata.org/wiki/Q7293202
concepts[2].display_name	Rank (graph theory)
concepts[3].id	https://openalex.org/C93779851
concepts[3].level	2
concepts[3].score	0.6176738739013672
concepts[3].wikidata	https://www.wikidata.org/wiki/Q271977
concepts[3].display_name	Partial differential equation
concepts[4].id	https://openalex.org/C41008148
concepts[4].level	0
concepts[4].score	0.43790870904922485
concepts[4].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[4].display_name	Computer science
concepts[5].id	https://openalex.org/C28826006
concepts[5].level	1
concepts[5].score	0.42897891998291016
concepts[5].wikidata	https://www.wikidata.org/wiki/Q33521
concepts[5].display_name	Applied mathematics
concepts[6].id	https://openalex.org/C50644808
concepts[6].level	2
concepts[6].score	0.42732366919517517
concepts[6].wikidata	https://www.wikidata.org/wiki/Q192776
concepts[6].display_name	Artificial neural network
concepts[7].id	https://openalex.org/C33923547
concepts[7].level	0
concepts[7].score	0.42162787914276123
concepts[7].wikidata	https://www.wikidata.org/wiki/Q395
concepts[7].display_name	Mathematics
concepts[8].id	https://openalex.org/C47446073
concepts[8].level	3
concepts[8].score	0.34808528423309326
concepts[8].wikidata	https://www.wikidata.org/wiki/Q5165890
concepts[8].display_name	Control theory (sociology)
concepts[9].id	https://openalex.org/C134306372
concepts[9].level	1
concepts[9].score	0.23401528596878052
concepts[9].wikidata	https://www.wikidata.org/wiki/Q7754
concepts[9].display_name	Mathematical analysis
concepts[10].id	https://openalex.org/C11413529
concepts[10].level	1
concepts[10].score	0.22694671154022217
concepts[10].wikidata	https://www.wikidata.org/wiki/Q8366
concepts[10].display_name	Algorithm
concepts[11].id	https://openalex.org/C154945302
concepts[11].level	1
concepts[11].score	0.1947016417980194
concepts[11].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[11].display_name	Artificial intelligence
concepts[12].id	https://openalex.org/C169760540
concepts[12].level	1
concepts[12].score	0.10352164506912231
concepts[12].wikidata	https://www.wikidata.org/wiki/Q207011
concepts[12].display_name	Neuroscience
concepts[13].id	https://openalex.org/C15744967
concepts[13].level	0
concepts[13].score	0.09436583518981934
concepts[13].wikidata	https://www.wikidata.org/wiki/Q9418
concepts[13].display_name	Psychology
concepts[14].id	https://openalex.org/C114614502
concepts[14].level	1
concepts[14].score	0.06535694003105164
concepts[14].wikidata	https://www.wikidata.org/wiki/Q76592
concepts[14].display_name	Combinatorics
concepts[15].id	https://openalex.org/C2775924081
concepts[15].level	2
concepts[15].score	0.0
concepts[15].wikidata	https://www.wikidata.org/wiki/Q55608371
concepts[15].display_name	Control (management)
keywords[0].id	https://openalex.org/keywords/parameterized-complexity
keywords[0].score	0.8565719127655029
keywords[0].display_name	Parameterized complexity
keywords[1].id	https://openalex.org/keywords/adaptation
keywords[1].score	0.7131462693214417
keywords[1].display_name	Adaptation (eye)
keywords[2].id	https://openalex.org/keywords/rank
keywords[2].score	0.6492642164230347
keywords[2].display_name	Rank (graph theory)
keywords[3].id	https://openalex.org/keywords/partial-differential-equation
keywords[3].score	0.6176738739013672
keywords[3].display_name	Partial differential equation
keywords[4].id	https://openalex.org/keywords/computer-science
keywords[4].score	0.43790870904922485
keywords[4].display_name	Computer science
keywords[5].id	https://openalex.org/keywords/applied-mathematics
keywords[5].score	0.42897891998291016
keywords[5].display_name	Applied mathematics
keywords[6].id	https://openalex.org/keywords/artificial-neural-network
keywords[6].score	0.42732366919517517
keywords[6].display_name	Artificial neural network
keywords[7].id	https://openalex.org/keywords/mathematics
keywords[7].score	0.42162787914276123
keywords[7].display_name	Mathematics
keywords[8].id	https://openalex.org/keywords/control-theory
keywords[8].score	0.34808528423309326
keywords[8].display_name	Control theory (sociology)
keywords[9].id	https://openalex.org/keywords/mathematical-analysis
keywords[9].score	0.23401528596878052
keywords[9].display_name	Mathematical analysis
keywords[10].id	https://openalex.org/keywords/algorithm
keywords[10].score	0.22694671154022217
keywords[10].display_name	Algorithm
keywords[11].id	https://openalex.org/keywords/artificial-intelligence
keywords[11].score	0.1947016417980194
keywords[11].display_name	Artificial intelligence
keywords[12].id	https://openalex.org/keywords/neuroscience
keywords[12].score	0.10352164506912231
keywords[12].display_name	Neuroscience
keywords[13].id	https://openalex.org/keywords/psychology
keywords[13].score	0.09436583518981934
keywords[13].display_name	Psychology
keywords[14].id	https://openalex.org/keywords/combinatorics
keywords[14].score	0.06535694003105164
keywords[14].display_name	Combinatorics
language	en
locations[0].id	pmh:oai:arXiv.org:2402.14646
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2402.14646
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2402.14646
locations[1].id	doi:10.48550/arxiv.2402.14646
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2402.14646
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5064983883
authorships[0].author.orcid
authorships[0].author.display_name	Jules J. Berman
authorships[0].author_position	first
authorships[0].raw_author_name	Berman, Jules
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5027402421
authorships[1].author.orcid	https://orcid.org/0000-0002-1558-6775
authorships[1].author.display_name	Benjamin Peherstorfer
authorships[1].author_position	last
authorships[1].raw_author_name	Peherstorfer, Benjamin
authorships[1].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2402.14646
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2024-02-24T00:00:00
display_name	CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T11206
primary_topic.field.id	https://openalex.org/fields/31
primary_topic.field.display_name	Physics and Astronomy
primary_topic.score	0.9990000128746033
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/3109
primary_topic.subfield.display_name	Statistical and Nonlinear Physics
primary_topic.display_name	Model Reduction and Neural Networks
related_works	https://openalex.org/W2051058708, https://openalex.org/W154868527, https://openalex.org/W1494268238, https://openalex.org/W1983207144, https://openalex.org/W2490706771, https://openalex.org/W2480116122, https://openalex.org/W1976468483, https://openalex.org/W4255576661, https://openalex.org/W1516574938, https://openalex.org/W2563912921
cited_by_count	3
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	1
counts_by_year[1].year	2024
counts_by_year[1].cited_by_count	2
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2402.14646
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2402.14646
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2402.14646
primary_location.id	pmh:oai:arXiv.org:2402.14646
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2402.14646
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2402.14646
publication_date	2024-02-22
publication_year	2024
referenced_works_count	0
abstract_inverted_index.a	17
abstract_inverted_index.an	55
abstract_inverted_index.at	38
abstract_inverted_index.be	49, 77
abstract_inverted_index.in	28, 69
abstract_inverted_index.is	93, 117
abstract_inverted_index.of	35, 73, 104
abstract_inverted_index.on	6
abstract_inverted_index.or	53
abstract_inverted_index.so	90
abstract_inverted_index.to	30, 120
abstract_inverted_index.Low	8
abstract_inverted_index.The	46
abstract_inverted_index.and	22, 42, 111, 114
abstract_inverted_index.are	87, 102
abstract_inverted_index.can	48, 76
abstract_inverted_index.few	84
abstract_inverted_index.for	16, 96
abstract_inverted_index.new	39, 43
abstract_inverted_index.the	33, 71, 74
abstract_inverted_index.via	54
abstract_inverted_index.Rank	9
abstract_inverted_index.This	0
abstract_inverted_index.kept	78
abstract_inverted_index.only	83
abstract_inverted_index.rank	72
abstract_inverted_index.than	107
abstract_inverted_index.that	12, 59, 82, 91
abstract_inverted_index.then	23
abstract_inverted_index.time	29
abstract_inverted_index.well	94
abstract_inverted_index.with	100, 108
abstract_inverted_index.work	1
abstract_inverted_index.adapt	25
abstract_inverted_index.based	5
abstract_inverted_index.given	18
abstract_inverted_index.means	81
abstract_inverted_index.other	121
abstract_inverted_index.their	112
abstract_inverted_index.time,	70
abstract_inverted_index.which	80
abstract_inverted_index.CoLoRA	64, 92, 101
abstract_inverted_index.either	50
abstract_inverted_index.faster	106
abstract_inverted_index.fields	37, 67
abstract_inverted_index.higher	118
abstract_inverted_index.models	4
abstract_inverted_index.neural	14, 122
abstract_inverted_index.orders	103
abstract_inverted_index.purely	51
abstract_inverted_index.small,	79
abstract_inverted_index.suited	95
abstract_inverted_index.Because	63
abstract_inverted_index.initial	44
abstract_inverted_index.locally	68
abstract_inverted_index.methods	110
abstract_inverted_index.network	123
abstract_inverted_index.offline	89
abstract_inverted_index.partial	19
abstract_inverted_index.physics	40
abstract_inverted_index.predict	32
abstract_inverted_index.rapidly	31
abstract_inverted_index.reduced	3
abstract_inverted_index.weights	27, 75
abstract_inverted_index.(CoLoRA)	11
abstract_inverted_index.accuracy	113
abstract_inverted_index.approach	58
abstract_inverted_index.compared	119
abstract_inverted_index.equation	21
abstract_inverted_index.low-rank	26
abstract_inverted_index.networks	15
abstract_inverted_index.provides	60
abstract_inverted_index.regimes.	98
abstract_inverted_index.required	88
abstract_inverted_index.solution	36, 66
abstract_inverted_index.training	85
abstract_inverted_index.classical	109
abstract_inverted_index.evolution	34
abstract_inverted_index.magnitude	105
abstract_inverted_index.parameter	115
abstract_inverted_index.pre-train	13
abstract_inverted_index.Adaptation	10
abstract_inverted_index.Continuous	7
abstract_inverted_index.adaptation	47
abstract_inverted_index.efficiency	116
abstract_inverted_index.introduces	2
abstract_inverted_index.parameters	41
abstract_inverted_index.Predictions	99
abstract_inverted_index.approaches.	124
abstract_inverted_index.conditions.	45
abstract_inverted_index.data-driven	52
abstract_inverted_index.data-scarce	97
abstract_inverted_index.variational	57
abstract_inverted_index.approximates	65
abstract_inverted_index.continuously	24
abstract_inverted_index.differential	20
abstract_inverted_index.trajectories	86
abstract_inverted_index.approximations.	62
abstract_inverted_index.equation-driven	56
abstract_inverted_index.Galerkin-optimal	61
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	2
citation_normalized_percentile