Lifting Manifolds to Mitigate Pseudo-Alignment in LLM4TS Article Swipe

MAKE IMPACT

Liangwei Nathan Zheng , Wenhao Liang , Wei Emma Zhang , Miao Xu , Olaf Maennel , Weitong Chen · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2510.12847

Pseudo-Alignment is a pervasive challenge in many large language models for time series (LLM4TS) models, often causing them to underperform compared to linear models or randomly initialised backbones. However, there is limited discussion in the community for the reasons that pseudo-alignment occurs. In this work, we conduct a thorough investigation into the root causes of pseudo-alignment in LLM4TS and build a connection of pseudo-alignment to the cone effect in LLM. We demonstrate that pseudo-alignment arises from the interplay of cone effect within pretrained LLM components and the intrinsically low-dimensional manifold of time-series data. In addition, we also introduce \textit{\textbf{TimeSUP}}, a novel technique designed to mitigate this issue and improve forecast performance in existing LLM4TS approaches. TimeSUP addresses this by increasing the time series manifold to more closely match the intrinsic dimension of language embeddings, allowing the model to distinguish temporal signals clearly while still capturing shared structures across modalities. As a result, representations for time and language tokens remain distinct yet exhibit high cosine similarity, signifying that the model preserves each modality unique features while learning their commonalities in a unified embedding space. Empirically, TimeSUP consistently outperforms state-of-the-art LLM4TS methods and other lightweight baselines on long-term forecasting performance. Furthermore, it can be seamlessly integrated into four existing LLM4TS pipelines and delivers significant improvements in forecasting performance.

Type

preprint

Landing Page

http://arxiv.org/abs/2510.12847

PDF

https://arxiv.org/pdf/2510.12847

OA Status

green

OpenAlex ID

https://openalex.org/W4415273582

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4415273582

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2510.12847

Digital Object Identifier

Title

Lifting Manifolds to Mitigate Pseudo-Alignment in LLM4TS

Work title

Type

preprint

OpenAlex work type

Publication year

2025

Year of publication

Publication date

2025-10-14

Full publication date if available

Authors

Liangwei Nathan Zheng, Wenhao Liang, Wei Emma Zhang, Miao Xu, Olaf Maennel, Weitong Chen

List of authors in order

Landing page

https://arxiv.org/abs/2510.12847

Publisher landing page

PDF URL

https://arxiv.org/pdf/2510.12847

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2510.12847

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload