Binary Event-Driven Spiking Transformer Article Swipe
Honglin Cao
,
Zijian Zhou
,
Wenjie Wei
,
Ammar Belatreche
,
Liang Yu
,
Dehao Zhang
,
Malu Zhang
,
Yang Yang
,
Haizhou Li
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2501.05904
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2501.05904
Transformer-based Spiking Neural Networks (SNNs) introduce a novel event-driven self-attention paradigm that combines the high performance of Transformers with the energy efficiency of SNNs. However, the larger model size and increased computational demands of the Transformer structure limit their practicality in resource-constrained scenarios. In this paper, we integrate binarization techniques into Transformer-based SNNs and propose the Binary Event-Driven Spiking Transformer, i.e. BESTformer. The proposed BESTformer can significantly reduce storage and computational demands by representing weights and attention maps with a mere 1-bit. However, BESTformer suffers from a severe performance drop from its full-precision counterpart due to the limited representation capability of binarization. To address this issue, we propose a Coupled Information Enhancement (CIE) method, which consists of a reversible framework and information enhancement distillation. By maximizing the mutual information between the binary model and its full-precision counterpart, the CIE method effectively mitigates the performance degradation of the BESTformer. Extensive experiments on static and neuromorphic datasets demonstrate that our method achieves superior performance to other binary SNNs, showcasing its potential as a compact yet high-performance model for resource-limited edge devices.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2501.05904
- https://arxiv.org/pdf/2501.05904
- OA Status
- green
- Cited By
- 1
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4406317755
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4406317755Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2501.05904Digital Object Identifier
- Title
-
Binary Event-Driven Spiking TransformerWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-01-10Full publication date if available
- Authors
-
Honglin Cao, Zijian Zhou, Wenjie Wei, Ammar Belatreche, Liang Yu, Dehao Zhang, Malu Zhang, Yang Yang, Haizhou LiList of authors in order
- Landing page
-
https://arxiv.org/abs/2501.05904Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2501.05904Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2501.05904Direct OA link when available
- Concepts
-
Transformer, Event (particle physics), Computer science, Electrical engineering, Engineering, Physics, Voltage, Quantum mechanicsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4406317755 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2501.05904 |
| ids.doi | https://doi.org/10.48550/arxiv.2501.05904 |
| ids.openalex | https://openalex.org/W4406317755 |
| fwci | |
| type | preprint |
| title | Binary Event-Driven Spiking Transformer |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10502 |
| topics[0].field.id | https://openalex.org/fields/22 |
| topics[0].field.display_name | Engineering |
| topics[0].score | 0.9994999766349792 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2208 |
| topics[0].subfield.display_name | Electrical and Electronic Engineering |
| topics[0].display_name | Advanced Memory and Neural Computing |
| topics[1].id | https://openalex.org/T12808 |
| topics[1].field.id | https://openalex.org/fields/22 |
| topics[1].field.display_name | Engineering |
| topics[1].score | 0.9789000153541565 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2208 |
| topics[1].subfield.display_name | Electrical and Electronic Engineering |
| topics[1].display_name | Ferroelectric and Negative Capacitance Devices |
| topics[2].id | https://openalex.org/T10472 |
| topics[2].field.id | https://openalex.org/fields/22 |
| topics[2].field.display_name | Engineering |
| topics[2].score | 0.9778000116348267 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2208 |
| topics[2].subfield.display_name | Electrical and Electronic Engineering |
| topics[2].display_name | Semiconductor materials and devices |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C66322947 |
| concepts[0].level | 3 |
| concepts[0].score | 0.5364959836006165 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q11658 |
| concepts[0].display_name | Transformer |
| concepts[1].id | https://openalex.org/C2779662365 |
| concepts[1].level | 2 |
| concepts[1].score | 0.428215891122818 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q5416694 |
| concepts[1].display_name | Event (particle physics) |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.40505409240722656 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C119599485 |
| concepts[3].level | 1 |
| concepts[3].score | 0.1675432026386261 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q43035 |
| concepts[3].display_name | Electrical engineering |
| concepts[4].id | https://openalex.org/C127413603 |
| concepts[4].level | 0 |
| concepts[4].score | 0.13268589973449707 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[4].display_name | Engineering |
| concepts[5].id | https://openalex.org/C121332964 |
| concepts[5].level | 0 |
| concepts[5].score | 0.13083913922309875 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[5].display_name | Physics |
| concepts[6].id | https://openalex.org/C165801399 |
| concepts[6].level | 2 |
| concepts[6].score | 0.11309003829956055 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q25428 |
| concepts[6].display_name | Voltage |
| concepts[7].id | https://openalex.org/C62520636 |
| concepts[7].level | 1 |
| concepts[7].score | 0.0 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q944 |
| concepts[7].display_name | Quantum mechanics |
| keywords[0].id | https://openalex.org/keywords/transformer |
| keywords[0].score | 0.5364959836006165 |
| keywords[0].display_name | Transformer |
| keywords[1].id | https://openalex.org/keywords/event |
| keywords[1].score | 0.428215891122818 |
| keywords[1].display_name | Event (particle physics) |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.40505409240722656 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/electrical-engineering |
| keywords[3].score | 0.1675432026386261 |
| keywords[3].display_name | Electrical engineering |
| keywords[4].id | https://openalex.org/keywords/engineering |
| keywords[4].score | 0.13268589973449707 |
| keywords[4].display_name | Engineering |
| keywords[5].id | https://openalex.org/keywords/physics |
| keywords[5].score | 0.13083913922309875 |
| keywords[5].display_name | Physics |
| keywords[6].id | https://openalex.org/keywords/voltage |
| keywords[6].score | 0.11309003829956055 |
| keywords[6].display_name | Voltage |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2501.05904 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2501.05904 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2501.05904 |
| locations[1].id | doi:10.48550/arxiv.2501.05904 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2501.05904 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5103986115 |
| authorships[0].author.orcid | https://orcid.org/0009-0002-7851-3810 |
| authorships[0].author.display_name | Honglin Cao |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Cao, Honglin |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5101548247 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-7965-0617 |
| authorships[1].author.display_name | Zijian Zhou |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Zhou, Zijian |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5041639029 |
| authorships[2].author.orcid | https://orcid.org/0009-0002-7753-2948 |
| authorships[2].author.display_name | Wenjie Wei |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Wei, Wenjie |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5042364148 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-1927-9366 |
| authorships[3].author.display_name | Ammar Belatreche |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Belatreche, Ammar |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5101814743 |
| authorships[4].author.orcid | https://orcid.org/0009-0007-3922-3454 |
| authorships[4].author.display_name | Liang Yu |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Liang, Yu |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5014979144 |
| authorships[5].author.orcid | |
| authorships[5].author.display_name | Dehao Zhang |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Zhang, Dehao |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5031886937 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-2345-0974 |
| authorships[6].author.display_name | Malu Zhang |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Zhang, Malu |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5100397725 |
| authorships[7].author.orcid | https://orcid.org/0000-0003-0608-9408 |
| authorships[7].author.display_name | Yang Yang |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Yang, Yang |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5032690182 |
| authorships[8].author.orcid | https://orcid.org/0000-0001-9158-9401 |
| authorships[8].author.display_name | Haizhou Li |
| authorships[8].author_position | last |
| authorships[8].raw_author_name | Li, Haizhou |
| authorships[8].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2501.05904 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Binary Event-Driven Spiking Transformer |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10502 |
| primary_topic.field.id | https://openalex.org/fields/22 |
| primary_topic.field.display_name | Engineering |
| primary_topic.score | 0.9994999766349792 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2208 |
| primary_topic.subfield.display_name | Electrical and Electronic Engineering |
| primary_topic.display_name | Advanced Memory and Neural Computing |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2501.05904 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2501.05904 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2501.05904 |
| primary_location.id | pmh:oai:arXiv.org:2501.05904 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2501.05904 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2501.05904 |
| publication_date | 2025-01-10 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 6, 79, 86, 108, 117, 170 |
| abstract_inverted_index.By | 124 |
| abstract_inverted_index.In | 43 |
| abstract_inverted_index.To | 102 |
| abstract_inverted_index.as | 169 |
| abstract_inverted_index.by | 72 |
| abstract_inverted_index.in | 40 |
| abstract_inverted_index.of | 16, 22, 33, 100, 116, 145 |
| abstract_inverted_index.on | 150 |
| abstract_inverted_index.to | 95, 162 |
| abstract_inverted_index.we | 46, 106 |
| abstract_inverted_index.CIE | 138 |
| abstract_inverted_index.The | 62 |
| abstract_inverted_index.and | 29, 53, 69, 75, 120, 133, 152 |
| abstract_inverted_index.can | 65 |
| abstract_inverted_index.due | 94 |
| abstract_inverted_index.for | 175 |
| abstract_inverted_index.its | 91, 134, 167 |
| abstract_inverted_index.our | 157 |
| abstract_inverted_index.the | 13, 19, 25, 34, 55, 96, 126, 130, 137, 142, 146 |
| abstract_inverted_index.yet | 172 |
| abstract_inverted_index.SNNs | 52 |
| abstract_inverted_index.drop | 89 |
| abstract_inverted_index.edge | 177 |
| abstract_inverted_index.from | 85, 90 |
| abstract_inverted_index.high | 14 |
| abstract_inverted_index.i.e. | 60 |
| abstract_inverted_index.into | 50 |
| abstract_inverted_index.maps | 77 |
| abstract_inverted_index.mere | 80 |
| abstract_inverted_index.size | 28 |
| abstract_inverted_index.that | 11, 156 |
| abstract_inverted_index.this | 44, 104 |
| abstract_inverted_index.with | 18, 78 |
| abstract_inverted_index.(CIE) | 112 |
| abstract_inverted_index.SNNs, | 165 |
| abstract_inverted_index.SNNs. | 23 |
| abstract_inverted_index.limit | 37 |
| abstract_inverted_index.model | 27, 132, 174 |
| abstract_inverted_index.novel | 7 |
| abstract_inverted_index.other | 163 |
| abstract_inverted_index.their | 38 |
| abstract_inverted_index.which | 114 |
| abstract_inverted_index.(SNNs) | 4 |
| abstract_inverted_index.1-bit. | 81 |
| abstract_inverted_index.Binary | 56 |
| abstract_inverted_index.Neural | 2 |
| abstract_inverted_index.binary | 131, 164 |
| abstract_inverted_index.energy | 20 |
| abstract_inverted_index.issue, | 105 |
| abstract_inverted_index.larger | 26 |
| abstract_inverted_index.method | 139, 158 |
| abstract_inverted_index.mutual | 127 |
| abstract_inverted_index.paper, | 45 |
| abstract_inverted_index.reduce | 67 |
| abstract_inverted_index.severe | 87 |
| abstract_inverted_index.static | 151 |
| abstract_inverted_index.Coupled | 109 |
| abstract_inverted_index.Spiking | 1, 58 |
| abstract_inverted_index.address | 103 |
| abstract_inverted_index.between | 129 |
| abstract_inverted_index.compact | 171 |
| abstract_inverted_index.demands | 32, 71 |
| abstract_inverted_index.limited | 97 |
| abstract_inverted_index.method, | 113 |
| abstract_inverted_index.propose | 54, 107 |
| abstract_inverted_index.storage | 68 |
| abstract_inverted_index.suffers | 84 |
| abstract_inverted_index.weights | 74 |
| abstract_inverted_index.However, | 24, 82 |
| abstract_inverted_index.Networks | 3 |
| abstract_inverted_index.achieves | 159 |
| abstract_inverted_index.combines | 12 |
| abstract_inverted_index.consists | 115 |
| abstract_inverted_index.datasets | 154 |
| abstract_inverted_index.devices. | 178 |
| abstract_inverted_index.paradigm | 10 |
| abstract_inverted_index.proposed | 63 |
| abstract_inverted_index.superior | 160 |
| abstract_inverted_index.Extensive | 148 |
| abstract_inverted_index.attention | 76 |
| abstract_inverted_index.framework | 119 |
| abstract_inverted_index.increased | 30 |
| abstract_inverted_index.integrate | 47 |
| abstract_inverted_index.introduce | 5 |
| abstract_inverted_index.mitigates | 141 |
| abstract_inverted_index.potential | 168 |
| abstract_inverted_index.structure | 36 |
| abstract_inverted_index.BESTformer | 64, 83 |
| abstract_inverted_index.capability | 99 |
| abstract_inverted_index.efficiency | 21 |
| abstract_inverted_index.maximizing | 125 |
| abstract_inverted_index.reversible | 118 |
| abstract_inverted_index.scenarios. | 42 |
| abstract_inverted_index.showcasing | 166 |
| abstract_inverted_index.techniques | 49 |
| abstract_inverted_index.BESTformer. | 61, 147 |
| abstract_inverted_index.Enhancement | 111 |
| abstract_inverted_index.Information | 110 |
| abstract_inverted_index.Transformer | 35 |
| abstract_inverted_index.counterpart | 93 |
| abstract_inverted_index.degradation | 144 |
| abstract_inverted_index.demonstrate | 155 |
| abstract_inverted_index.effectively | 140 |
| abstract_inverted_index.enhancement | 122 |
| abstract_inverted_index.experiments | 149 |
| abstract_inverted_index.information | 121, 128 |
| abstract_inverted_index.performance | 15, 88, 143, 161 |
| abstract_inverted_index.Event-Driven | 57 |
| abstract_inverted_index.Transformer, | 59 |
| abstract_inverted_index.Transformers | 17 |
| abstract_inverted_index.binarization | 48 |
| abstract_inverted_index.counterpart, | 136 |
| abstract_inverted_index.event-driven | 8 |
| abstract_inverted_index.neuromorphic | 153 |
| abstract_inverted_index.practicality | 39 |
| abstract_inverted_index.representing | 73 |
| abstract_inverted_index.binarization. | 101 |
| abstract_inverted_index.computational | 31, 70 |
| abstract_inverted_index.distillation. | 123 |
| abstract_inverted_index.significantly | 66 |
| abstract_inverted_index.full-precision | 92, 135 |
| abstract_inverted_index.representation | 98 |
| abstract_inverted_index.self-attention | 9 |
| abstract_inverted_index.high-performance | 173 |
| abstract_inverted_index.resource-limited | 176 |
| abstract_inverted_index.Transformer-based | 0, 51 |
| abstract_inverted_index.resource-constrained | 41 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 9 |
| citation_normalized_percentile |