Repetitive Contrastive Learning Enhances Mamba's Selectivity in Time Series Prediction Article Swipe
Wenbo Yan
,
H. Henry Cao
,
Ying Tan
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2504.09185
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2504.09185
Long sequence prediction is a key challenge in time series forecasting. While Mamba-based models have shown strong performance due to their sequence selection capabilities, they still struggle with insufficient focus on critical time steps and incomplete noise suppression, caused by limited selective abilities. To address this, we introduce Repetitive Contrastive Learning (RCL), a token-level contrastive pretraining framework aimed at enhancing Mamba's selective capabilities. RCL pretrains a single Mamba block to strengthen its selective abilities and then transfers these pretrained parameters to initialize Mamba blocks in various backbone models, improving their temporal prediction performance. RCL uses sequence augmentation with Gaussian noise and applies inter-sequence and intra-sequence contrastive learning to help the Mamba module prioritize information-rich time steps while ignoring noisy ones. Extensive experiments show that RCL consistently boosts the performance of backbone models, surpassing existing methods and achieving state-of-the-art results. Additionally, we propose two metrics to quantify Mamba's selective capabilities, providing theoretical, qualitative, and quantitative evidence for the improvements brought by RCL.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2504.09185
- https://arxiv.org/pdf/2504.09185
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4415155348
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4415155348Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2504.09185Digital Object Identifier
- Title
-
Repetitive Contrastive Learning Enhances Mamba's Selectivity in Time Series PredictionWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-04-12Full publication date if available
- Authors
-
Wenbo Yan, H. Henry Cao, Ying TanList of authors in order
- Landing page
-
https://arxiv.org/abs/2504.09185Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2504.09185Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2504.09185Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4415155348 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2504.09185 |
| ids.doi | https://doi.org/10.48550/arxiv.2504.09185 |
| ids.openalex | https://openalex.org/W4415155348 |
| fwci | |
| type | preprint |
| title | Repetitive Contrastive Learning Enhances Mamba's Selectivity in Time Series Prediction |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10320 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9639999866485596 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Neural Networks and Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2504.09185 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2504.09185 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2504.09185 |
| locations[1].id | doi:10.48550/arxiv.2504.09185 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2504.09185 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5050783869 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-6977-6283 |
| authorships[0].author.display_name | Wenbo Yan |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Yan, Wenbo |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5009424792 |
| authorships[1].author.orcid | https://orcid.org/0009-0004-2774-8243 |
| authorships[1].author.display_name | H. Henry Cao |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Cao, Hanzhong |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5078570390 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-8495-0246 |
| authorships[2].author.display_name | Ying Tan |
| authorships[2].author_position | last |
| authorships[2].raw_author_name | Tan, Ying |
| authorships[2].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2504.09185 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-14T00:00:00 |
| display_name | Repetitive Contrastive Learning Enhances Mamba's Selectivity in Time Series Prediction |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10320 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9639999866485596 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Neural Networks and Applications |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2504.09185 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2504.09185 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2504.09185 |
| primary_location.id | pmh:oai:arXiv.org:2504.09185 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2504.09185 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2504.09185 |
| publication_date | 2025-04-12 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 4, 52, 65 |
| abstract_inverted_index.To | 43 |
| abstract_inverted_index.at | 58 |
| abstract_inverted_index.by | 39, 159 |
| abstract_inverted_index.in | 7, 84 |
| abstract_inverted_index.is | 3 |
| abstract_inverted_index.of | 129 |
| abstract_inverted_index.on | 30 |
| abstract_inverted_index.to | 19, 69, 80, 107, 144 |
| abstract_inverted_index.we | 46, 140 |
| abstract_inverted_index.RCL | 63, 93, 124 |
| abstract_inverted_index.and | 34, 74, 100, 103, 135, 152 |
| abstract_inverted_index.due | 18 |
| abstract_inverted_index.for | 155 |
| abstract_inverted_index.its | 71 |
| abstract_inverted_index.key | 5 |
| abstract_inverted_index.the | 109, 127, 156 |
| abstract_inverted_index.two | 142 |
| abstract_inverted_index.Long | 0 |
| abstract_inverted_index.RCL. | 160 |
| abstract_inverted_index.have | 14 |
| abstract_inverted_index.help | 108 |
| abstract_inverted_index.show | 122 |
| abstract_inverted_index.that | 123 |
| abstract_inverted_index.then | 75 |
| abstract_inverted_index.they | 24 |
| abstract_inverted_index.time | 8, 32, 114 |
| abstract_inverted_index.uses | 94 |
| abstract_inverted_index.with | 27, 97 |
| abstract_inverted_index.Mamba | 67, 82, 110 |
| abstract_inverted_index.While | 11 |
| abstract_inverted_index.aimed | 57 |
| abstract_inverted_index.block | 68 |
| abstract_inverted_index.focus | 29 |
| abstract_inverted_index.noise | 36, 99 |
| abstract_inverted_index.noisy | 118 |
| abstract_inverted_index.ones. | 119 |
| abstract_inverted_index.shown | 15 |
| abstract_inverted_index.steps | 33, 115 |
| abstract_inverted_index.still | 25 |
| abstract_inverted_index.their | 20, 89 |
| abstract_inverted_index.these | 77 |
| abstract_inverted_index.this, | 45 |
| abstract_inverted_index.while | 116 |
| abstract_inverted_index.(RCL), | 51 |
| abstract_inverted_index.blocks | 83 |
| abstract_inverted_index.boosts | 126 |
| abstract_inverted_index.caused | 38 |
| abstract_inverted_index.models | 13 |
| abstract_inverted_index.module | 111 |
| abstract_inverted_index.series | 9 |
| abstract_inverted_index.single | 66 |
| abstract_inverted_index.strong | 16 |
| abstract_inverted_index.Mamba's | 60, 146 |
| abstract_inverted_index.address | 44 |
| abstract_inverted_index.applies | 101 |
| abstract_inverted_index.brought | 158 |
| abstract_inverted_index.limited | 40 |
| abstract_inverted_index.methods | 134 |
| abstract_inverted_index.metrics | 143 |
| abstract_inverted_index.models, | 87, 131 |
| abstract_inverted_index.propose | 141 |
| abstract_inverted_index.various | 85 |
| abstract_inverted_index.Gaussian | 98 |
| abstract_inverted_index.Learning | 50 |
| abstract_inverted_index.backbone | 86, 130 |
| abstract_inverted_index.critical | 31 |
| abstract_inverted_index.evidence | 154 |
| abstract_inverted_index.existing | 133 |
| abstract_inverted_index.ignoring | 117 |
| abstract_inverted_index.learning | 106 |
| abstract_inverted_index.quantify | 145 |
| abstract_inverted_index.results. | 138 |
| abstract_inverted_index.sequence | 1, 21, 95 |
| abstract_inverted_index.struggle | 26 |
| abstract_inverted_index.temporal | 90 |
| abstract_inverted_index.Extensive | 120 |
| abstract_inverted_index.abilities | 73 |
| abstract_inverted_index.achieving | 136 |
| abstract_inverted_index.challenge | 6 |
| abstract_inverted_index.enhancing | 59 |
| abstract_inverted_index.framework | 56 |
| abstract_inverted_index.improving | 88 |
| abstract_inverted_index.introduce | 47 |
| abstract_inverted_index.pretrains | 64 |
| abstract_inverted_index.providing | 149 |
| abstract_inverted_index.selection | 22 |
| abstract_inverted_index.selective | 41, 61, 72, 147 |
| abstract_inverted_index.transfers | 76 |
| abstract_inverted_index.Repetitive | 48 |
| abstract_inverted_index.abilities. | 42 |
| abstract_inverted_index.incomplete | 35 |
| abstract_inverted_index.initialize | 81 |
| abstract_inverted_index.parameters | 79 |
| abstract_inverted_index.prediction | 2, 91 |
| abstract_inverted_index.pretrained | 78 |
| abstract_inverted_index.prioritize | 112 |
| abstract_inverted_index.strengthen | 70 |
| abstract_inverted_index.surpassing | 132 |
| abstract_inverted_index.Contrastive | 49 |
| abstract_inverted_index.Mamba-based | 12 |
| abstract_inverted_index.contrastive | 54, 105 |
| abstract_inverted_index.experiments | 121 |
| abstract_inverted_index.performance | 17, 128 |
| abstract_inverted_index.pretraining | 55 |
| abstract_inverted_index.token-level | 53 |
| abstract_inverted_index.augmentation | 96 |
| abstract_inverted_index.consistently | 125 |
| abstract_inverted_index.forecasting. | 10 |
| abstract_inverted_index.improvements | 157 |
| abstract_inverted_index.insufficient | 28 |
| abstract_inverted_index.performance. | 92 |
| abstract_inverted_index.qualitative, | 151 |
| abstract_inverted_index.quantitative | 153 |
| abstract_inverted_index.suppression, | 37 |
| abstract_inverted_index.theoretical, | 150 |
| abstract_inverted_index.Additionally, | 139 |
| abstract_inverted_index.capabilities, | 23, 148 |
| abstract_inverted_index.capabilities. | 62 |
| abstract_inverted_index.inter-sequence | 102 |
| abstract_inverted_index.intra-sequence | 104 |
| abstract_inverted_index.information-rich | 113 |
| abstract_inverted_index.state-of-the-art | 137 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 3 |
| citation_normalized_percentile |