DECOR: Improving Coherence in L2 English Writing with a Novel Benchmark for Incoherence Detection, Reasoning, and Rewriting Article Swipe
Xuanming Zhang
,
Anthony Diaz
,
Zixun Chen
,
Qingyang Wu
,
Kun Qian
,
Erik Voss
,
Yu Zhou
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2406.19650
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2406.19650
Coherence in writing, an aspect that second-language (L2) English learners often struggle with, is crucial in assessing L2 English writing. Existing automated writing evaluation systems primarily use basic surface linguistic features to detect coherence in writing. However, little effort has been made to correct the detected incoherence, which could significantly benefit L2 language learners seeking to improve their writing. To bridge this gap, we introduce DECOR, a novel benchmark that includes expert annotations for detecting incoherence in L2 English writing, identifying the underlying reasons, and rewriting the incoherent sentences. To our knowledge, DECOR is the first coherence assessment dataset specifically designed for improving L2 English writing, featuring pairs of original incoherent sentences alongside their expert-rewritten counterparts. Additionally, we fine-tuned models to automatically detect and rewrite incoherence in student essays. We find that incorporating specific reasons for incoherence during fine-tuning consistently improves the quality of the rewrites, achieving a result that is favored in both automatic and human evaluations.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2406.19650
- https://arxiv.org/pdf/2406.19650
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4400222469
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4400222469Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2406.19650Digital Object Identifier
- Title
-
DECOR: Improving Coherence in L2 English Writing with a Novel Benchmark for Incoherence Detection, Reasoning, and RewritingWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-06-28Full publication date if available
- Authors
-
Xuanming Zhang, Anthony Diaz, Zixun Chen, Qingyang Wu, Kun Qian, Erik Voss, Yu ZhouList of authors in order
- Landing page
-
https://arxiv.org/abs/2406.19650Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2406.19650Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2406.19650Direct OA link when available
- Concepts
-
Rewriting, Benchmark (surveying), Coherence (philosophical gambling strategy), Computer science, Artificial intelligence, Programming language, Physics, Geography, Quantum mechanics, GeodesyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4400222469 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2406.19650 |
| ids.doi | https://doi.org/10.48550/arxiv.2406.19650 |
| ids.openalex | https://openalex.org/W4400222469 |
| fwci | 0.0 |
| type | preprint |
| title | DECOR: Improving Coherence in L2 English Writing with a Novel Benchmark for Incoherence Detection, Reasoning, and Rewriting |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T13629 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.7930999994277954 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Text Readability and Simplification |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C154690210 |
| concepts[0].level | 2 |
| concepts[0].score | 0.9010676145553589 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1668499 |
| concepts[0].display_name | Rewriting |
| concepts[1].id | https://openalex.org/C185798385 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7502539157867432 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q1161707 |
| concepts[1].display_name | Benchmark (surveying) |
| concepts[2].id | https://openalex.org/C2781181686 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6995032429695129 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q4226068 |
| concepts[2].display_name | Coherence (philosophical gambling strategy) |
| concepts[3].id | https://openalex.org/C41008148 |
| concepts[3].level | 0 |
| concepts[3].score | 0.6303305625915527 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[3].display_name | Computer science |
| concepts[4].id | https://openalex.org/C154945302 |
| concepts[4].level | 1 |
| concepts[4].score | 0.3375885784626007 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[4].display_name | Artificial intelligence |
| concepts[5].id | https://openalex.org/C199360897 |
| concepts[5].level | 1 |
| concepts[5].score | 0.3261699676513672 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[5].display_name | Programming language |
| concepts[6].id | https://openalex.org/C121332964 |
| concepts[6].level | 0 |
| concepts[6].score | 0.15408650040626526 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[6].display_name | Physics |
| concepts[7].id | https://openalex.org/C205649164 |
| concepts[7].level | 0 |
| concepts[7].score | 0.08280709385871887 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q1071 |
| concepts[7].display_name | Geography |
| concepts[8].id | https://openalex.org/C62520636 |
| concepts[8].level | 1 |
| concepts[8].score | 0.0 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q944 |
| concepts[8].display_name | Quantum mechanics |
| concepts[9].id | https://openalex.org/C13280743 |
| concepts[9].level | 1 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q131089 |
| concepts[9].display_name | Geodesy |
| keywords[0].id | https://openalex.org/keywords/rewriting |
| keywords[0].score | 0.9010676145553589 |
| keywords[0].display_name | Rewriting |
| keywords[1].id | https://openalex.org/keywords/benchmark |
| keywords[1].score | 0.7502539157867432 |
| keywords[1].display_name | Benchmark (surveying) |
| keywords[2].id | https://openalex.org/keywords/coherence |
| keywords[2].score | 0.6995032429695129 |
| keywords[2].display_name | Coherence (philosophical gambling strategy) |
| keywords[3].id | https://openalex.org/keywords/computer-science |
| keywords[3].score | 0.6303305625915527 |
| keywords[3].display_name | Computer science |
| keywords[4].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[4].score | 0.3375885784626007 |
| keywords[4].display_name | Artificial intelligence |
| keywords[5].id | https://openalex.org/keywords/programming-language |
| keywords[5].score | 0.3261699676513672 |
| keywords[5].display_name | Programming language |
| keywords[6].id | https://openalex.org/keywords/physics |
| keywords[6].score | 0.15408650040626526 |
| keywords[6].display_name | Physics |
| keywords[7].id | https://openalex.org/keywords/geography |
| keywords[7].score | 0.08280709385871887 |
| keywords[7].display_name | Geography |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2406.19650 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2406.19650 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2406.19650 |
| locations[1].id | doi:10.48550/arxiv.2406.19650 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2406.19650 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5063061958 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-9208-1845 |
| authorships[0].author.display_name | Xuanming Zhang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Zhang, Xuanming |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5085164592 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Anthony Diaz |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Diaz, Anthony |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5090783407 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-6970-2156 |
| authorships[2].author.display_name | Zixun Chen |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Chen, Zixun |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5010638511 |
| authorships[3].author.orcid | https://orcid.org/0000-0001-7888-4940 |
| authorships[3].author.display_name | Qingyang Wu |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Wu, Qingyang |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5005099484 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-6765-9446 |
| authorships[4].author.display_name | Kun Qian |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Qian, Kun |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5016701546 |
| authorships[5].author.orcid | https://orcid.org/0000-0001-7011-3084 |
| authorships[5].author.display_name | Erik Voss |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Voss, Erik |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5101754653 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-8572-9778 |
| authorships[6].author.display_name | Yu Zhou |
| authorships[6].author_position | last |
| authorships[6].raw_author_name | Yu, Zhou |
| authorships[6].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2406.19650 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-07-02T00:00:00 |
| display_name | DECOR: Improving Coherence in L2 English Writing with a Novel Benchmark for Incoherence Detection, Reasoning, and Rewriting |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T13629 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.7930999994277954 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Text Readability and Simplification |
| related_works | https://openalex.org/W2120204135, https://openalex.org/W2139396251, https://openalex.org/W1796293478, https://openalex.org/W1577544887, https://openalex.org/W2168276503, https://openalex.org/W1573537275, https://openalex.org/W2105713543, https://openalex.org/W4205908955, https://openalex.org/W174435416, https://openalex.org/W2132239740 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2406.19650 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2406.19650 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2406.19650 |
| primary_location.id | pmh:oai:arXiv.org:2406.19650 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2406.19650 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2406.19650 |
| publication_date | 2024-06-28 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 66, 147 |
| abstract_inverted_index.L2 | 17, 51, 77, 103 |
| abstract_inverted_index.To | 59, 89 |
| abstract_inverted_index.We | 129 |
| abstract_inverted_index.an | 3 |
| abstract_inverted_index.in | 1, 15, 34, 76, 126, 152 |
| abstract_inverted_index.is | 13, 93, 150 |
| abstract_inverted_index.of | 108, 143 |
| abstract_inverted_index.to | 31, 42, 55, 120 |
| abstract_inverted_index.we | 63, 117 |
| abstract_inverted_index.and | 84, 123, 155 |
| abstract_inverted_index.for | 73, 101, 135 |
| abstract_inverted_index.has | 39 |
| abstract_inverted_index.our | 90 |
| abstract_inverted_index.the | 44, 81, 86, 94, 141, 144 |
| abstract_inverted_index.use | 26 |
| abstract_inverted_index.(L2) | 7 |
| abstract_inverted_index.been | 40 |
| abstract_inverted_index.both | 153 |
| abstract_inverted_index.find | 130 |
| abstract_inverted_index.gap, | 62 |
| abstract_inverted_index.made | 41 |
| abstract_inverted_index.that | 5, 69, 131, 149 |
| abstract_inverted_index.this | 61 |
| abstract_inverted_index.DECOR | 92 |
| abstract_inverted_index.basic | 27 |
| abstract_inverted_index.could | 48 |
| abstract_inverted_index.first | 95 |
| abstract_inverted_index.human | 156 |
| abstract_inverted_index.novel | 67 |
| abstract_inverted_index.often | 10 |
| abstract_inverted_index.pairs | 107 |
| abstract_inverted_index.their | 57, 113 |
| abstract_inverted_index.which | 47 |
| abstract_inverted_index.with, | 12 |
| abstract_inverted_index.DECOR, | 65 |
| abstract_inverted_index.aspect | 4 |
| abstract_inverted_index.bridge | 60 |
| abstract_inverted_index.detect | 32, 122 |
| abstract_inverted_index.during | 137 |
| abstract_inverted_index.effort | 38 |
| abstract_inverted_index.expert | 71 |
| abstract_inverted_index.little | 37 |
| abstract_inverted_index.models | 119 |
| abstract_inverted_index.result | 148 |
| abstract_inverted_index.English | 8, 18, 78, 104 |
| abstract_inverted_index.benefit | 50 |
| abstract_inverted_index.correct | 43 |
| abstract_inverted_index.crucial | 14 |
| abstract_inverted_index.dataset | 98 |
| abstract_inverted_index.essays. | 128 |
| abstract_inverted_index.favored | 151 |
| abstract_inverted_index.improve | 56 |
| abstract_inverted_index.quality | 142 |
| abstract_inverted_index.reasons | 134 |
| abstract_inverted_index.rewrite | 124 |
| abstract_inverted_index.seeking | 54 |
| abstract_inverted_index.student | 127 |
| abstract_inverted_index.surface | 28 |
| abstract_inverted_index.systems | 24 |
| abstract_inverted_index.writing | 22 |
| abstract_inverted_index.Existing | 20 |
| abstract_inverted_index.However, | 36 |
| abstract_inverted_index.designed | 100 |
| abstract_inverted_index.detected | 45 |
| abstract_inverted_index.features | 30 |
| abstract_inverted_index.improves | 140 |
| abstract_inverted_index.includes | 70 |
| abstract_inverted_index.language | 52 |
| abstract_inverted_index.learners | 9, 53 |
| abstract_inverted_index.original | 109 |
| abstract_inverted_index.reasons, | 83 |
| abstract_inverted_index.specific | 133 |
| abstract_inverted_index.struggle | 11 |
| abstract_inverted_index.writing, | 2, 79, 105 |
| abstract_inverted_index.writing. | 19, 35, 58 |
| abstract_inverted_index.Coherence | 0 |
| abstract_inverted_index.achieving | 146 |
| abstract_inverted_index.alongside | 112 |
| abstract_inverted_index.assessing | 16 |
| abstract_inverted_index.automated | 21 |
| abstract_inverted_index.automatic | 154 |
| abstract_inverted_index.benchmark | 68 |
| abstract_inverted_index.coherence | 33, 96 |
| abstract_inverted_index.detecting | 74 |
| abstract_inverted_index.featuring | 106 |
| abstract_inverted_index.improving | 102 |
| abstract_inverted_index.introduce | 64 |
| abstract_inverted_index.primarily | 25 |
| abstract_inverted_index.rewrites, | 145 |
| abstract_inverted_index.rewriting | 85 |
| abstract_inverted_index.sentences | 111 |
| abstract_inverted_index.assessment | 97 |
| abstract_inverted_index.evaluation | 23 |
| abstract_inverted_index.fine-tuned | 118 |
| abstract_inverted_index.incoherent | 87, 110 |
| abstract_inverted_index.knowledge, | 91 |
| abstract_inverted_index.linguistic | 29 |
| abstract_inverted_index.sentences. | 88 |
| abstract_inverted_index.underlying | 82 |
| abstract_inverted_index.annotations | 72 |
| abstract_inverted_index.fine-tuning | 138 |
| abstract_inverted_index.identifying | 80 |
| abstract_inverted_index.incoherence | 75, 125, 136 |
| abstract_inverted_index.consistently | 139 |
| abstract_inverted_index.evaluations. | 157 |
| abstract_inverted_index.incoherence, | 46 |
| abstract_inverted_index.specifically | 99 |
| abstract_inverted_index.Additionally, | 116 |
| abstract_inverted_index.automatically | 121 |
| abstract_inverted_index.counterparts. | 115 |
| abstract_inverted_index.incorporating | 132 |
| abstract_inverted_index.significantly | 49 |
| abstract_inverted_index.second-language | 6 |
| abstract_inverted_index.expert-rewritten | 114 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 7 |
| citation_normalized_percentile |