Physics Steering: Causal Control of Cross-Domain Concepts in a Physics Foundation Model Article Swipe

MAKE IMPACT

Payel Mukhopadhyay , Michael T. McCabe , Miles Cranmer · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2511.20798

Recent advances in mechanistic interpretability have revealed that large language models (LLMs) develop internal representations corresponding not only to concrete entities but also distinct, human-understandable abstract concepts and behaviour. Moreover, these hidden features can be directly manipulated to steer model behaviour. However, it remains an open question whether this phenomenon is unique to models trained on inherently structured data (ie. language, images) or if it is a general property of foundation models. In this work, we investigate the internal representations of a large physics-focused foundation model. Inspired by recent work identifying single directions in activation space for complex behaviours in LLMs, we extract activation vectors from the model during forward passes over simulation datasets for different physical regimes. We then compute "delta" representations between the two regimes. These delta tensors act as concept directions in activation space, encoding specific physical features. By injecting these concept directions back into the model during inference, we can steer its predictions, demonstrating causal control over physical behaviours, such as inducing or removing some particular physical feature from a simulation. These results suggest that scientific foundation models learn generalised representations of physical principles. They do not merely rely on superficial correlations and patterns in the simulations. Our findings open new avenues for understanding and controlling scientific foundation models and has implications for AI-enabled scientific discovery.

Type

preprint

Landing Page

http://arxiv.org/abs/2511.20798

PDF

https://arxiv.org/pdf/2511.20798

OA Status

green

OpenAlex ID

https://openalex.org/W4416778131

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4416778131

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2511.20798

Digital Object Identifier

Title

Physics Steering: Causal Control of Cross-Domain Concepts in a Physics Foundation Model

Work title

Type

preprint

OpenAlex work type

Publication year

2025

Year of publication

Publication date

2025-11-25

Full publication date if available

Authors

Payel Mukhopadhyay, Michael T. McCabe, Miles Cranmer

List of authors in order

Landing page

https://arxiv.org/abs/2511.20798

Publisher landing page

PDF URL

https://arxiv.org/pdf/2511.20798

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2511.20798

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload