Optical Music Recognition with Convolutional Sequence-to-Sequence Models. Article Swipe
Eelco van der Wel
,
Karen Ullrich
·
YOU?
·
· 2017
· Open Access
·
· DOI: https://doi.org/10.5072/zenodo.243774
YOU?
·
· 2017
· Open Access
·
· DOI: https://doi.org/10.5072/zenodo.243774
Optical Music Recognition (OMR) is an important technology within Music Information Retrieval. Deep learning models show promising results on OMR tasks, but symbol-level annotated data sets of sufficient size to train such models are not available and difficult to develop. We present a deep learning architecture called a Convolutional Sequence-to-Sequence model to both move towards an end-to-end trainable OMR pipeline, and apply a learning process that trains on full sentences of sheet music instead of individually labeled symbols. The model is trained and evaluated on a human generated data set, with various image augmentations based on real-world scenarios. This data set is the first publicly available set in OMR research with sufficient size to train and evaluate deep learning models. With the introduced augmentations a pitch recognition accuracy of 81% and a duration accuracy of 94% is achieved, resulting in a note level accuracy of 80%. Finally, the model is compared to commercially available methods, showing a large improvements over these applications.
Related Topics
Concepts
Computer science
Deep learning
Artificial intelligence
Pipeline (software)
Convolutional neural network
Sequence (biology)
Set (abstract data type)
Data set
Process (computing)
Pattern recognition (psychology)
Speech recognition
Machine learning
Operating system
Genetics
Programming language
Biology
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://arxiv.org/abs/1707.04877
- OA Status
- green
- Cited By
- 7
- References
- 7
- Related Works
- 20
- OpenAlex ID
- https://openalex.org/W2963043364
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W2963043364Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.5072/zenodo.243774Digital Object Identifier
- Title
-
Optical Music Recognition with Convolutional Sequence-to-Sequence Models.Work title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2017Year of publication
- Publication date
-
2017-10-23Full publication date if available
- Authors
-
Eelco van der Wel, Karen UllrichList of authors in order
- Landing page
-
https://arxiv.org/abs/1707.04877Publisher landing page
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/abs/1707.04877Direct OA link when available
- Concepts
-
Computer science, Deep learning, Artificial intelligence, Pipeline (software), Convolutional neural network, Sequence (biology), Set (abstract data type), Data set, Process (computing), Pattern recognition (psychology), Speech recognition, Machine learning, Operating system, Genetics, Programming language, BiologyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
7Total citation count in OpenAlex
- Citations by year (recent)
-
2021: 2, 2020: 4, 2018: 1Per-year citation counts (last 5 years)
- References (count)
-
7Number of works referenced by this work
- Related works (count)
-
20Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W2963043364 |
|---|---|
| doi | https://doi.org/10.5072/zenodo.243774 |
| ids.mag | 2963043364 |
| ids.openalex | https://openalex.org/W2963043364 |
| fwci | 1.01176064 |
| type | article |
| title | Optical Music Recognition with Convolutional Sequence-to-Sequence Models. |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | 737 |
| biblio.first_page | 731 |
| topics[0].id | https://openalex.org/T11309 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 1.0 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1711 |
| topics[0].subfield.display_name | Signal Processing |
| topics[0].display_name | Music and Audio Processing |
| topics[1].id | https://openalex.org/T11349 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9965999722480774 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1707 |
| topics[1].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[1].display_name | Music Technology and Sound Studies |
| topics[2].id | https://openalex.org/T10860 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9929999709129333 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1711 |
| topics[2].subfield.display_name | Signal Processing |
| topics[2].display_name | Speech and Audio Processing |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.8118720054626465 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C108583219 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6984421014785767 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q197536 |
| concepts[1].display_name | Deep learning |
| concepts[2].id | https://openalex.org/C154945302 |
| concepts[2].level | 1 |
| concepts[2].score | 0.6776504516601562 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[2].display_name | Artificial intelligence |
| concepts[3].id | https://openalex.org/C43521106 |
| concepts[3].level | 2 |
| concepts[3].score | 0.6694397330284119 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q2165493 |
| concepts[3].display_name | Pipeline (software) |
| concepts[4].id | https://openalex.org/C81363708 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5970979928970337 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q17084460 |
| concepts[4].display_name | Convolutional neural network |
| concepts[5].id | https://openalex.org/C2778112365 |
| concepts[5].level | 2 |
| concepts[5].score | 0.5851228833198547 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q3511065 |
| concepts[5].display_name | Sequence (biology) |
| concepts[6].id | https://openalex.org/C177264268 |
| concepts[6].level | 2 |
| concepts[6].score | 0.5053198933601379 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q1514741 |
| concepts[6].display_name | Set (abstract data type) |
| concepts[7].id | https://openalex.org/C58489278 |
| concepts[7].level | 2 |
| concepts[7].score | 0.48002949357032776 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q1172284 |
| concepts[7].display_name | Data set |
| concepts[8].id | https://openalex.org/C98045186 |
| concepts[8].level | 2 |
| concepts[8].score | 0.4680706262588501 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q205663 |
| concepts[8].display_name | Process (computing) |
| concepts[9].id | https://openalex.org/C153180895 |
| concepts[9].level | 2 |
| concepts[9].score | 0.43804931640625 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q7148389 |
| concepts[9].display_name | Pattern recognition (psychology) |
| concepts[10].id | https://openalex.org/C28490314 |
| concepts[10].level | 1 |
| concepts[10].score | 0.433197557926178 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q189436 |
| concepts[10].display_name | Speech recognition |
| concepts[11].id | https://openalex.org/C119857082 |
| concepts[11].level | 1 |
| concepts[11].score | 0.33298259973526 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[11].display_name | Machine learning |
| concepts[12].id | https://openalex.org/C111919701 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q9135 |
| concepts[12].display_name | Operating system |
| concepts[13].id | https://openalex.org/C54355233 |
| concepts[13].level | 1 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q7162 |
| concepts[13].display_name | Genetics |
| concepts[14].id | https://openalex.org/C199360897 |
| concepts[14].level | 1 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[14].display_name | Programming language |
| concepts[15].id | https://openalex.org/C86803240 |
| concepts[15].level | 0 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q420 |
| concepts[15].display_name | Biology |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.8118720054626465 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/deep-learning |
| keywords[1].score | 0.6984421014785767 |
| keywords[1].display_name | Deep learning |
| keywords[2].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[2].score | 0.6776504516601562 |
| keywords[2].display_name | Artificial intelligence |
| keywords[3].id | https://openalex.org/keywords/pipeline |
| keywords[3].score | 0.6694397330284119 |
| keywords[3].display_name | Pipeline (software) |
| keywords[4].id | https://openalex.org/keywords/convolutional-neural-network |
| keywords[4].score | 0.5970979928970337 |
| keywords[4].display_name | Convolutional neural network |
| keywords[5].id | https://openalex.org/keywords/sequence |
| keywords[5].score | 0.5851228833198547 |
| keywords[5].display_name | Sequence (biology) |
| keywords[6].id | https://openalex.org/keywords/set |
| keywords[6].score | 0.5053198933601379 |
| keywords[6].display_name | Set (abstract data type) |
| keywords[7].id | https://openalex.org/keywords/data-set |
| keywords[7].score | 0.48002949357032776 |
| keywords[7].display_name | Data set |
| keywords[8].id | https://openalex.org/keywords/process |
| keywords[8].score | 0.4680706262588501 |
| keywords[8].display_name | Process (computing) |
| keywords[9].id | https://openalex.org/keywords/pattern-recognition |
| keywords[9].score | 0.43804931640625 |
| keywords[9].display_name | Pattern recognition (psychology) |
| keywords[10].id | https://openalex.org/keywords/speech-recognition |
| keywords[10].score | 0.433197557926178 |
| keywords[10].display_name | Speech recognition |
| keywords[11].id | https://openalex.org/keywords/machine-learning |
| keywords[11].score | 0.33298259973526 |
| keywords[11].display_name | Machine learning |
| language | en |
| locations[0].id | mag:2963043364 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | |
| locations[0].version | |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | |
| locations[0].raw_source_name | arXiv (Cornell University) |
| locations[0].landing_page_url | https://arxiv.org/abs/1707.04877 |
| authorships[0].author.id | https://openalex.org/A5061649477 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Eelco van der Wel |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Eelco van der Wel |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5058031547 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Karen Ullrich |
| authorships[1].countries | DE |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I137479271 |
| authorships[1].affiliations[0].raw_affiliation_string | Helmholtz Centre for Environmental Research, Leipzig, Germany |
| authorships[1].institutions[0].id | https://openalex.org/I137479271 |
| authorships[1].institutions[0].ror | https://ror.org/000h6jb29 |
| authorships[1].institutions[0].type | facility |
| authorships[1].institutions[0].lineage | https://openalex.org/I1305996414, https://openalex.org/I137479271 |
| authorships[1].institutions[0].country_code | DE |
| authorships[1].institutions[0].display_name | Helmholtz Centre for Environmental Research |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Karen Ullrich |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | Helmholtz Centre for Environmental Research, Leipzig, Germany |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/abs/1707.04877 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Optical Music Recognition with Convolutional Sequence-to-Sequence Models. |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-10-10T17:16:08.811792 |
| primary_topic.id | https://openalex.org/T11309 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 1.0 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1711 |
| primary_topic.subfield.display_name | Signal Processing |
| primary_topic.display_name | Music and Audio Processing |
| related_works | https://openalex.org/W2739344165, https://openalex.org/W2963224290, https://openalex.org/W2889118961, https://openalex.org/W2811221988, https://openalex.org/W2785680402, https://openalex.org/W2194187530, https://openalex.org/W2990795634, https://openalex.org/W1489833900, https://openalex.org/W2755802490, https://openalex.org/W2907410141, https://openalex.org/W3128750130, https://openalex.org/W2964327310, https://openalex.org/W998260344, https://openalex.org/W2939763029, https://openalex.org/W3168378459, https://openalex.org/W1827297289, https://openalex.org/W2791153761, https://openalex.org/W2983626510, https://openalex.org/W3198112749, https://openalex.org/W2963169965 |
| cited_by_count | 7 |
| counts_by_year[0].year | 2021 |
| counts_by_year[0].cited_by_count | 2 |
| counts_by_year[1].year | 2020 |
| counts_by_year[1].cited_by_count | 4 |
| counts_by_year[2].year | 2018 |
| counts_by_year[2].cited_by_count | 1 |
| locations_count | 1 |
| best_oa_location.id | mag:2963043364 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | |
| best_oa_location.version | |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | arXiv (Cornell University) |
| best_oa_location.landing_page_url | https://arxiv.org/abs/1707.04877 |
| primary_location.id | mag:2963043364 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | |
| primary_location.version | |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | arXiv (Cornell University) |
| primary_location.landing_page_url | https://arxiv.org/abs/1707.04877 |
| publication_date | 2017-10-23 |
| publication_year | 2017 |
| referenced_works | https://openalex.org/W2949888546, https://openalex.org/W2613159350, https://openalex.org/W150004333, https://openalex.org/W2140825458, https://openalex.org/W158393211, https://openalex.org/W2336763592, https://openalex.org/W1522301498 |
| referenced_works_count | 7 |
| abstract_inverted_index.a | 42, 47, 62, 85, 124, 131, 140, 156 |
| abstract_inverted_index.We | 40 |
| abstract_inverted_index.an | 5, 55 |
| abstract_inverted_index.in | 107, 139 |
| abstract_inverted_index.is | 4, 80, 101, 136, 149 |
| abstract_inverted_index.of | 26, 70, 74, 128, 134, 144 |
| abstract_inverted_index.on | 18, 67, 84, 95 |
| abstract_inverted_index.to | 29, 38, 51, 113, 151 |
| abstract_inverted_index.81% | 129 |
| abstract_inverted_index.94% | 135 |
| abstract_inverted_index.OMR | 19, 58, 108 |
| abstract_inverted_index.The | 78 |
| abstract_inverted_index.and | 36, 60, 82, 115, 130 |
| abstract_inverted_index.are | 33 |
| abstract_inverted_index.but | 21 |
| abstract_inverted_index.not | 34 |
| abstract_inverted_index.set | 100, 106 |
| abstract_inverted_index.the | 102, 121, 147 |
| abstract_inverted_index.80%. | 145 |
| abstract_inverted_index.Deep | 12 |
| abstract_inverted_index.This | 98 |
| abstract_inverted_index.With | 120 |
| abstract_inverted_index.both | 52 |
| abstract_inverted_index.data | 24, 88, 99 |
| abstract_inverted_index.deep | 43, 117 |
| abstract_inverted_index.full | 68 |
| abstract_inverted_index.move | 53 |
| abstract_inverted_index.note | 141 |
| abstract_inverted_index.over | 159 |
| abstract_inverted_index.set, | 89 |
| abstract_inverted_index.sets | 25 |
| abstract_inverted_index.show | 15 |
| abstract_inverted_index.size | 28, 112 |
| abstract_inverted_index.such | 31 |
| abstract_inverted_index.that | 65 |
| abstract_inverted_index.with | 90, 110 |
| abstract_inverted_index.(OMR) | 3 |
| abstract_inverted_index.Music | 1, 9 |
| abstract_inverted_index.apply | 61 |
| abstract_inverted_index.based | 94 |
| abstract_inverted_index.first | 103 |
| abstract_inverted_index.human | 86 |
| abstract_inverted_index.image | 92 |
| abstract_inverted_index.large | 157 |
| abstract_inverted_index.level | 142 |
| abstract_inverted_index.model | 50, 79, 148 |
| abstract_inverted_index.music | 72 |
| abstract_inverted_index.pitch | 125 |
| abstract_inverted_index.sheet | 71 |
| abstract_inverted_index.these | 160 |
| abstract_inverted_index.train | 30, 114 |
| abstract_inverted_index.called | 46 |
| abstract_inverted_index.models | 14, 32 |
| abstract_inverted_index.tasks, | 20 |
| abstract_inverted_index.trains | 66 |
| abstract_inverted_index.within | 8 |
| abstract_inverted_index.Optical | 0 |
| abstract_inverted_index.instead | 73 |
| abstract_inverted_index.labeled | 76 |
| abstract_inverted_index.models. | 119 |
| abstract_inverted_index.present | 41 |
| abstract_inverted_index.process | 64 |
| abstract_inverted_index.results | 17 |
| abstract_inverted_index.showing | 155 |
| abstract_inverted_index.towards | 54 |
| abstract_inverted_index.trained | 81 |
| abstract_inverted_index.various | 91 |
| abstract_inverted_index.Finally, | 146 |
| abstract_inverted_index.accuracy | 127, 133, 143 |
| abstract_inverted_index.compared | 150 |
| abstract_inverted_index.develop. | 39 |
| abstract_inverted_index.duration | 132 |
| abstract_inverted_index.evaluate | 116 |
| abstract_inverted_index.learning | 13, 44, 63, 118 |
| abstract_inverted_index.methods, | 154 |
| abstract_inverted_index.publicly | 104 |
| abstract_inverted_index.research | 109 |
| abstract_inverted_index.symbols. | 77 |
| abstract_inverted_index.achieved, | 137 |
| abstract_inverted_index.annotated | 23 |
| abstract_inverted_index.available | 35, 105, 153 |
| abstract_inverted_index.difficult | 37 |
| abstract_inverted_index.evaluated | 83 |
| abstract_inverted_index.generated | 87 |
| abstract_inverted_index.important | 6 |
| abstract_inverted_index.pipeline, | 59 |
| abstract_inverted_index.promising | 16 |
| abstract_inverted_index.resulting | 138 |
| abstract_inverted_index.sentences | 69 |
| abstract_inverted_index.trainable | 57 |
| abstract_inverted_index.Retrieval. | 11 |
| abstract_inverted_index.end-to-end | 56 |
| abstract_inverted_index.introduced | 122 |
| abstract_inverted_index.real-world | 96 |
| abstract_inverted_index.scenarios. | 97 |
| abstract_inverted_index.sufficient | 27, 111 |
| abstract_inverted_index.technology | 7 |
| abstract_inverted_index.Information | 10 |
| abstract_inverted_index.Recognition | 2 |
| abstract_inverted_index.recognition | 126 |
| abstract_inverted_index.architecture | 45 |
| abstract_inverted_index.commercially | 152 |
| abstract_inverted_index.improvements | 158 |
| abstract_inverted_index.individually | 75 |
| abstract_inverted_index.symbol-level | 22 |
| abstract_inverted_index.Convolutional | 48 |
| abstract_inverted_index.applications. | 161 |
| abstract_inverted_index.augmentations | 93, 123 |
| abstract_inverted_index.Sequence-to-Sequence | 49 |
| cited_by_percentile_year.max | 97 |
| cited_by_percentile_year.min | 90 |
| countries_distinct_count | 1 |
| institutions_distinct_count | 2 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.7099999785423279 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile.value | 0.778009 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |