Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers Article Swipe

MAKE IMPACT

Kai Lagemann , Christian Lagemann , Sach Mukherjee · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2306.12077

We propose a method for learning dynamical systems from high-dimensional empirical data that combines variational autoencoders and (spatio-)temporal attention within a framework designed to enforce certain scientifically-motivated invariances. We focus on the setting in which data are available from multiple different instances of a system whose underlying dynamical model is entirely unknown at the outset. The approach rests on a separation into an instance-specific encoding (capturing initial conditions, constants etc.) and a latent dynamics model that is itself universal across all instances/realizations of the system. The separation is achieved in an automated, data-driven manner and only empirical data are required as inputs to the model. The approach allows effective inference of system behaviour at any continuous time but does not require an explicit neural ODE formulation, which makes it efficient and highly scalable. We study behaviour through simple theoretical analyses and extensive experiments on synthetic and real-world datasets. The latter investigate learning the dynamics of complex systems based on finite data and show that the proposed approach can outperform state-of-the-art neural-dynamical models. We study also more general inductive bias in the context of transfer to data obtained under entirely novel system interventions. Overall, our results provide a promising new framework for efficiently learning dynamical models from heterogeneous data with potential applications in a wide range of fields including physics, medicine, biology and engineering.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2306.12077

PDF

https://arxiv.org/pdf/2306.12077

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4381713703

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4381713703

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2306.12077

Digital Object Identifier

Title

Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-06-21

Full publication date if available

Authors

Kai Lagemann, Christian Lagemann, Sach Mukherjee

List of authors in order

Landing page

https://arxiv.org/abs/2306.12077

Publisher landing page

PDF URL

https://arxiv.org/pdf/2306.12077

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2306.12077

Direct OA link when available

Concepts

Dynamical systems theory, Computer science, Ode, Scalability, Inductive bias, Inference, Artificial intelligence, Synthetic data, Machine learning, Theoretical computer science, Mathematics, Multi-task learning, Physics, Task (project management), Management, Economics, Database, Applied mathematics, Quantum mechanics

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2025: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload