Feedforward Sequential Memory Neural Networks without Recurrent Feedback Article Swipe
Shiliang Zhang
,
Hui Jiang
,
Si Wei
,
Li-Rong Dai
·
YOU?
·
· 2015
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.1510.02693
YOU?
·
· 2015
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.1510.02693
We introduce a new structure for memory neural networks, called feedforward sequential memory networks (FSMN), which can learn long-term dependency without using recurrent feedback. The proposed FSMN is a standard feedforward neural networks equipped with learnable sequential memory blocks in the hidden layers. In this work, we have applied FSMN to several language modeling (LM) tasks. Experimental results have shown that the memory blocks in FSMN can learn effective representations of long history. Experiments have shown that FSMN based language models can significantly outperform not only feedforward neural network (FNN) based LMs but also the popular recurrent neural network (RNN) LMs.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/1510.02693
- https://arxiv.org/pdf/1510.02693
- OA Status
- green
- Cited By
- 20
- References
- 17
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W1920942766
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W1920942766Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.1510.02693Digital Object Identifier
- Title
-
Feedforward Sequential Memory Neural Networks without Recurrent FeedbackWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2015Year of publication
- Publication date
-
2015-10-09Full publication date if available
- Authors
-
Shiliang Zhang, Hui Jiang, Si Wei, Li-Rong DaiList of authors in order
- Landing page
-
https://arxiv.org/abs/1510.02693Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/1510.02693Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/1510.02693Direct OA link when available
- Concepts
-
Recurrent neural network, Feed forward, Computer science, Feedforward neural network, Artificial neural network, Time delay neural network, Dependency (UML), Artificial intelligence, Long short term memory, Control engineering, EngineeringTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
20Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1, 2023: 2, 2021: 4, 2020: 5, 2018: 4Per-year citation counts (last 5 years)
- References (count)
-
17Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W1920942766 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.1510.02693 |
| ids.doi | https://doi.org/10.48550/arxiv.1510.02693 |
| ids.mag | 1920942766 |
| ids.openalex | https://openalex.org/W1920942766 |
| fwci | |
| type | preprint |
| title | Feedforward Sequential Memory Neural Networks without Recurrent Feedback |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9995999932289124 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9993000030517578 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| topics[2].id | https://openalex.org/T10201 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.993399977684021 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Speech Recognition and Synthesis |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C147168706 |
| concepts[0].level | 3 |
| concepts[0].score | 0.8491915464401245 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1457734 |
| concepts[0].display_name | Recurrent neural network |
| concepts[1].id | https://openalex.org/C38858127 |
| concepts[1].level | 2 |
| concepts[1].score | 0.838442325592041 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q5441228 |
| concepts[1].display_name | Feed forward |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.7724353075027466 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C47702885 |
| concepts[3].level | 3 |
| concepts[3].score | 0.6853480339050293 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q5441227 |
| concepts[3].display_name | Feedforward neural network |
| concepts[4].id | https://openalex.org/C50644808 |
| concepts[4].level | 2 |
| concepts[4].score | 0.6506324410438538 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q192776 |
| concepts[4].display_name | Artificial neural network |
| concepts[5].id | https://openalex.org/C175202392 |
| concepts[5].level | 3 |
| concepts[5].score | 0.48460254073143005 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q2434543 |
| concepts[5].display_name | Time delay neural network |
| concepts[6].id | https://openalex.org/C19768560 |
| concepts[6].level | 2 |
| concepts[6].score | 0.4726002812385559 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q320727 |
| concepts[6].display_name | Dependency (UML) |
| concepts[7].id | https://openalex.org/C154945302 |
| concepts[7].level | 1 |
| concepts[7].score | 0.4723542630672455 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[7].display_name | Artificial intelligence |
| concepts[8].id | https://openalex.org/C133488467 |
| concepts[8].level | 4 |
| concepts[8].score | 0.4304826259613037 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q6673524 |
| concepts[8].display_name | Long short term memory |
| concepts[9].id | https://openalex.org/C133731056 |
| concepts[9].level | 1 |
| concepts[9].score | 0.10964643955230713 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q4917288 |
| concepts[9].display_name | Control engineering |
| concepts[10].id | https://openalex.org/C127413603 |
| concepts[10].level | 0 |
| concepts[10].score | 0.0 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[10].display_name | Engineering |
| keywords[0].id | https://openalex.org/keywords/recurrent-neural-network |
| keywords[0].score | 0.8491915464401245 |
| keywords[0].display_name | Recurrent neural network |
| keywords[1].id | https://openalex.org/keywords/feed-forward |
| keywords[1].score | 0.838442325592041 |
| keywords[1].display_name | Feed forward |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.7724353075027466 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/feedforward-neural-network |
| keywords[3].score | 0.6853480339050293 |
| keywords[3].display_name | Feedforward neural network |
| keywords[4].id | https://openalex.org/keywords/artificial-neural-network |
| keywords[4].score | 0.6506324410438538 |
| keywords[4].display_name | Artificial neural network |
| keywords[5].id | https://openalex.org/keywords/time-delay-neural-network |
| keywords[5].score | 0.48460254073143005 |
| keywords[5].display_name | Time delay neural network |
| keywords[6].id | https://openalex.org/keywords/dependency |
| keywords[6].score | 0.4726002812385559 |
| keywords[6].display_name | Dependency (UML) |
| keywords[7].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[7].score | 0.4723542630672455 |
| keywords[7].display_name | Artificial intelligence |
| keywords[8].id | https://openalex.org/keywords/long-short-term-memory |
| keywords[8].score | 0.4304826259613037 |
| keywords[8].display_name | Long short term memory |
| keywords[9].id | https://openalex.org/keywords/control-engineering |
| keywords[9].score | 0.10964643955230713 |
| keywords[9].display_name | Control engineering |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:1510.02693 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/1510.02693 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/1510.02693 |
| locations[1].id | doi:10.48550/arxiv.1510.02693 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.1510.02693 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5101777589 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-7795-437X |
| authorships[0].author.display_name | Shiliang Zhang |
| authorships[0].countries | CN |
| authorships[0].affiliations[0].institution_ids | https://openalex.org/I126520041 |
| authorships[0].affiliations[0].raw_affiliation_string | University of Science and Technology of China |
| authorships[0].institutions[0].id | https://openalex.org/I126520041 |
| authorships[0].institutions[0].ror | https://ror.org/04c4dkn09 |
| authorships[0].institutions[0].type | education |
| authorships[0].institutions[0].lineage | https://openalex.org/I126520041, https://openalex.org/I19820366 |
| authorships[0].institutions[0].country_code | CN |
| authorships[0].institutions[0].display_name | University of Science and Technology of China |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | ShiLiang Zhang |
| authorships[0].is_corresponding | False |
| authorships[0].raw_affiliation_strings | University of Science and Technology of China |
| authorships[1].author.id | https://openalex.org/A5101454723 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-4062-7206 |
| authorships[1].author.display_name | Hui Jiang |
| authorships[1].countries | CA |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I192455969 |
| authorships[1].affiliations[0].raw_affiliation_string | York University |
| authorships[1].institutions[0].id | https://openalex.org/I192455969 |
| authorships[1].institutions[0].ror | https://ror.org/05fq50484 |
| authorships[1].institutions[0].type | education |
| authorships[1].institutions[0].lineage | https://openalex.org/I192455969 |
| authorships[1].institutions[0].country_code | CA |
| authorships[1].institutions[0].display_name | York University |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Hui Jiang |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | York University |
| authorships[2].author.id | https://openalex.org/A5050660824 |
| authorships[2].author.orcid | https://orcid.org/0009-0009-5748-699X |
| authorships[2].author.display_name | Si Wei |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Si Wei |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5057227915 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-0859-2827 |
| authorships[3].author.display_name | Li-Rong Dai |
| authorships[3].countries | CN |
| authorships[3].affiliations[0].institution_ids | https://openalex.org/I126520041 |
| authorships[3].affiliations[0].raw_affiliation_string | University of Science and Technology of China |
| authorships[3].institutions[0].id | https://openalex.org/I126520041 |
| authorships[3].institutions[0].ror | https://ror.org/04c4dkn09 |
| authorships[3].institutions[0].type | education |
| authorships[3].institutions[0].lineage | https://openalex.org/I126520041, https://openalex.org/I19820366 |
| authorships[3].institutions[0].country_code | CN |
| authorships[3].institutions[0].display_name | University of Science and Technology of China |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | LiRong Dai |
| authorships[3].is_corresponding | False |
| authorships[3].raw_affiliation_strings | University of Science and Technology of China |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/1510.02693 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Feedforward Sequential Memory Neural Networks without Recurrent Feedback |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9995999932289124 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| related_works | https://openalex.org/W2115072676, https://openalex.org/W2912153778, https://openalex.org/W4387163678, https://openalex.org/W4288108708, https://openalex.org/W4311212821, https://openalex.org/W2045727192, https://openalex.org/W1529660427, https://openalex.org/W2102065768, https://openalex.org/W2109916967, https://openalex.org/W2950022897 |
| cited_by_count | 20 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2023 |
| counts_by_year[1].cited_by_count | 2 |
| counts_by_year[2].year | 2021 |
| counts_by_year[2].cited_by_count | 4 |
| counts_by_year[3].year | 2020 |
| counts_by_year[3].cited_by_count | 5 |
| counts_by_year[4].year | 2018 |
| counts_by_year[4].cited_by_count | 4 |
| counts_by_year[5].year | 2016 |
| counts_by_year[5].cited_by_count | 3 |
| counts_by_year[6].year | 2015 |
| counts_by_year[6].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:1510.02693 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/1510.02693 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/1510.02693 |
| primary_location.id | pmh:oai:arXiv.org:1510.02693 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/1510.02693 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/1510.02693 |
| publication_date | 2015-10-09 |
| publication_year | 2015 |
| referenced_works | https://openalex.org/W1924770834, https://openalex.org/W2251640092, https://openalex.org/W4303633609, https://openalex.org/W580074167, https://openalex.org/W1810943226, https://openalex.org/W4298154995, https://openalex.org/W2107878631, https://openalex.org/W2171928131, https://openalex.org/W2118776487, https://openalex.org/W4301604551, https://openalex.org/W179875071, https://openalex.org/W2950527759, https://openalex.org/W1533861849, https://openalex.org/W2951008357, https://openalex.org/W2064675550, https://openalex.org/W2949626814, https://openalex.org/W4285719527 |
| referenced_works_count | 17 |
| abstract_inverted_index.a | 2, 28 |
| abstract_inverted_index.In | 43 |
| abstract_inverted_index.We | 0 |
| abstract_inverted_index.in | 39, 64 |
| abstract_inverted_index.is | 27 |
| abstract_inverted_index.of | 70 |
| abstract_inverted_index.to | 50 |
| abstract_inverted_index.we | 46 |
| abstract_inverted_index.LMs | 91 |
| abstract_inverted_index.The | 24 |
| abstract_inverted_index.but | 92 |
| abstract_inverted_index.can | 16, 66, 81 |
| abstract_inverted_index.for | 5 |
| abstract_inverted_index.new | 3 |
| abstract_inverted_index.not | 84 |
| abstract_inverted_index.the | 40, 61, 94 |
| abstract_inverted_index.(LM) | 54 |
| abstract_inverted_index.FSMN | 26, 49, 65, 77 |
| abstract_inverted_index.LMs. | 100 |
| abstract_inverted_index.also | 93 |
| abstract_inverted_index.have | 47, 58, 74 |
| abstract_inverted_index.long | 71 |
| abstract_inverted_index.only | 85 |
| abstract_inverted_index.that | 60, 76 |
| abstract_inverted_index.this | 44 |
| abstract_inverted_index.with | 34 |
| abstract_inverted_index.(FNN) | 89 |
| abstract_inverted_index.(RNN) | 99 |
| abstract_inverted_index.based | 78, 90 |
| abstract_inverted_index.learn | 17, 67 |
| abstract_inverted_index.shown | 59, 75 |
| abstract_inverted_index.using | 21 |
| abstract_inverted_index.which | 15 |
| abstract_inverted_index.work, | 45 |
| abstract_inverted_index.blocks | 38, 63 |
| abstract_inverted_index.called | 9 |
| abstract_inverted_index.hidden | 41 |
| abstract_inverted_index.memory | 6, 12, 37, 62 |
| abstract_inverted_index.models | 80 |
| abstract_inverted_index.neural | 7, 31, 87, 97 |
| abstract_inverted_index.tasks. | 55 |
| abstract_inverted_index.(FSMN), | 14 |
| abstract_inverted_index.applied | 48 |
| abstract_inverted_index.layers. | 42 |
| abstract_inverted_index.network | 88, 98 |
| abstract_inverted_index.popular | 95 |
| abstract_inverted_index.results | 57 |
| abstract_inverted_index.several | 51 |
| abstract_inverted_index.without | 20 |
| abstract_inverted_index.equipped | 33 |
| abstract_inverted_index.history. | 72 |
| abstract_inverted_index.language | 52, 79 |
| abstract_inverted_index.modeling | 53 |
| abstract_inverted_index.networks | 13, 32 |
| abstract_inverted_index.proposed | 25 |
| abstract_inverted_index.standard | 29 |
| abstract_inverted_index.effective | 68 |
| abstract_inverted_index.feedback. | 23 |
| abstract_inverted_index.introduce | 1 |
| abstract_inverted_index.learnable | 35 |
| abstract_inverted_index.long-term | 18 |
| abstract_inverted_index.networks, | 8 |
| abstract_inverted_index.recurrent | 22, 96 |
| abstract_inverted_index.structure | 4 |
| abstract_inverted_index.dependency | 19 |
| abstract_inverted_index.outperform | 83 |
| abstract_inverted_index.sequential | 11, 36 |
| abstract_inverted_index.Experiments | 73 |
| abstract_inverted_index.feedforward | 10, 30, 86 |
| abstract_inverted_index.Experimental | 56 |
| abstract_inverted_index.significantly | 82 |
| abstract_inverted_index.representations | 69 |
| cited_by_percentile_year | |
| countries_distinct_count | 2 |
| institutions_distinct_count | 4 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.5299999713897705 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile |