Multi-Modal Requirements Data-based Acceptance Criteria Generation using LLMs Article Swipe
Fanyu Wang
,
Chetan Arora
,
Yonghui Liu
,
Kao Li Huang
,
Dishan Sambathkumar
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2508.06888
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2508.06888
Acceptance criteria (ACs) play a critical role in software development by clearly defining the conditions under which a software feature satisfies stakeholder expectations. However, manually creating accurate, comprehensive, and unambiguous acceptance criteria is challenging, particularly in user interface-intensive applications, due to the reliance on domain-specific knowledge and visual context that is not always captured by textual requirements alone. To address these challenges, we propose RAGcceptance M2RE, a novel approach that leverages Retrieval-Augmented Generation (RAG) to generate acceptance criteria from multi-modal requirements data, including both textual documentation and visual UI information. We systematically evaluated our approach in an industrial case study involving an education-focused software system used by approximately 100,000 users. The results indicate that integrating multi-modal information significantly enhances the relevance, correctness, and comprehensibility of the generated ACs. Moreover, practitioner evaluations confirm that our approach effectively reduces manual effort, captures nuanced stakeholder intent, and provides valuable criteria that domain experts may overlook, demonstrating practical utility and significant potential for industry adoption. This research underscores the potential of multi-modal RAG techniques in streamlining software validation processes and improving development efficiency. We also make our implementation and a dataset available.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2508.06888
- https://arxiv.org/pdf/2508.06888
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4416177830
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4416177830Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2508.06888Digital Object Identifier
- Title
-
Multi-Modal Requirements Data-based Acceptance Criteria Generation using LLMsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-08-09Full publication date if available
- Authors
-
Fanyu Wang, Chetan Arora, Yonghui Liu, Kao Li Huang, Dishan SambathkumarList of authors in order
- Landing page
-
https://arxiv.org/abs/2508.06888Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2508.06888Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2508.06888Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4416177830 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2508.06888 |
| ids.doi | https://doi.org/10.48550/arxiv.2508.06888 |
| ids.openalex | https://openalex.org/W4416177830 |
| fwci | |
| type | preprint |
| title | Multi-Modal Requirements Data-based Acceptance Criteria Generation using LLMs |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2508.06888 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2508.06888 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2508.06888 |
| locations[1].id | doi:10.48550/arxiv.2508.06888 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2508.06888 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5108050054 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-9937-8534 |
| authorships[0].author.display_name | Fanyu Wang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Wang, Fanyu |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5019739552 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-1466-7386 |
| authorships[1].author.display_name | Chetan Arora |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Arora, Chetan |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5100622983 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-7548-5100 |
| authorships[2].author.display_name | Yonghui Liu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Liu, Yonghui |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5084406562 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Kao Li Huang |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Huang, Kaicheng |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5120397964 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Dishan Sambathkumar |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Sambathkumar, Dishan |
| authorships[4].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2508.06888 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Multi-Modal Requirements Data-based Acceptance Criteria Generation using LLMs |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-28T07:06:52.247252 |
| primary_topic | |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2508.06888 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2508.06888 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2508.06888 |
| primary_location.id | pmh:oai:arXiv.org:2508.06888 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2508.06888 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2508.06888 |
| publication_date | 2025-08-09 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 4, 17, 66, 185 |
| abstract_inverted_index.To | 58 |
| abstract_inverted_index.UI | 88 |
| abstract_inverted_index.We | 90, 179 |
| abstract_inverted_index.an | 96, 101 |
| abstract_inverted_index.by | 10, 54, 106 |
| abstract_inverted_index.in | 7, 35, 95, 170 |
| abstract_inverted_index.is | 32, 50 |
| abstract_inverted_index.of | 124, 166 |
| abstract_inverted_index.on | 43 |
| abstract_inverted_index.to | 40, 74 |
| abstract_inverted_index.we | 62 |
| abstract_inverted_index.RAG | 168 |
| abstract_inverted_index.The | 110 |
| abstract_inverted_index.and | 28, 46, 86, 122, 143, 155, 175, 184 |
| abstract_inverted_index.due | 39 |
| abstract_inverted_index.for | 158 |
| abstract_inverted_index.may | 150 |
| abstract_inverted_index.not | 51 |
| abstract_inverted_index.our | 93, 133, 182 |
| abstract_inverted_index.the | 13, 41, 119, 125, 164 |
| abstract_inverted_index.ACs. | 127 |
| abstract_inverted_index.This | 161 |
| abstract_inverted_index.also | 180 |
| abstract_inverted_index.both | 83 |
| abstract_inverted_index.case | 98 |
| abstract_inverted_index.from | 78 |
| abstract_inverted_index.make | 181 |
| abstract_inverted_index.play | 3 |
| abstract_inverted_index.role | 6 |
| abstract_inverted_index.that | 49, 69, 113, 132, 147 |
| abstract_inverted_index.used | 105 |
| abstract_inverted_index.user | 36 |
| abstract_inverted_index.(ACs) | 2 |
| abstract_inverted_index.(RAG) | 73 |
| abstract_inverted_index.M2RE, | 65 |
| abstract_inverted_index.data, | 81 |
| abstract_inverted_index.novel | 67 |
| abstract_inverted_index.study | 99 |
| abstract_inverted_index.these | 60 |
| abstract_inverted_index.under | 15 |
| abstract_inverted_index.which | 16 |
| abstract_inverted_index.alone. | 57 |
| abstract_inverted_index.always | 52 |
| abstract_inverted_index.domain | 148 |
| abstract_inverted_index.manual | 137 |
| abstract_inverted_index.system | 104 |
| abstract_inverted_index.users. | 109 |
| abstract_inverted_index.visual | 47, 87 |
| abstract_inverted_index.100,000 | 108 |
| abstract_inverted_index.address | 59 |
| abstract_inverted_index.clearly | 11 |
| abstract_inverted_index.confirm | 131 |
| abstract_inverted_index.context | 48 |
| abstract_inverted_index.dataset | 186 |
| abstract_inverted_index.effort, | 138 |
| abstract_inverted_index.experts | 149 |
| abstract_inverted_index.feature | 19 |
| abstract_inverted_index.intent, | 142 |
| abstract_inverted_index.nuanced | 140 |
| abstract_inverted_index.propose | 63 |
| abstract_inverted_index.reduces | 136 |
| abstract_inverted_index.results | 111 |
| abstract_inverted_index.textual | 55, 84 |
| abstract_inverted_index.utility | 154 |
| abstract_inverted_index.However, | 23 |
| abstract_inverted_index.approach | 68, 94, 134 |
| abstract_inverted_index.captured | 53 |
| abstract_inverted_index.captures | 139 |
| abstract_inverted_index.creating | 25 |
| abstract_inverted_index.criteria | 1, 31, 77, 146 |
| abstract_inverted_index.critical | 5 |
| abstract_inverted_index.defining | 12 |
| abstract_inverted_index.enhances | 118 |
| abstract_inverted_index.generate | 75 |
| abstract_inverted_index.indicate | 112 |
| abstract_inverted_index.industry | 159 |
| abstract_inverted_index.manually | 24 |
| abstract_inverted_index.provides | 144 |
| abstract_inverted_index.reliance | 42 |
| abstract_inverted_index.research | 162 |
| abstract_inverted_index.software | 8, 18, 103, 172 |
| abstract_inverted_index.valuable | 145 |
| abstract_inverted_index.Moreover, | 128 |
| abstract_inverted_index.accurate, | 26 |
| abstract_inverted_index.adoption. | 160 |
| abstract_inverted_index.evaluated | 92 |
| abstract_inverted_index.generated | 126 |
| abstract_inverted_index.improving | 176 |
| abstract_inverted_index.including | 82 |
| abstract_inverted_index.involving | 100 |
| abstract_inverted_index.knowledge | 45 |
| abstract_inverted_index.leverages | 70 |
| abstract_inverted_index.overlook, | 151 |
| abstract_inverted_index.potential | 157, 165 |
| abstract_inverted_index.practical | 153 |
| abstract_inverted_index.processes | 174 |
| abstract_inverted_index.satisfies | 20 |
| abstract_inverted_index.Acceptance | 0 |
| abstract_inverted_index.Generation | 72 |
| abstract_inverted_index.acceptance | 30, 76 |
| abstract_inverted_index.available. | 187 |
| abstract_inverted_index.conditions | 14 |
| abstract_inverted_index.industrial | 97 |
| abstract_inverted_index.relevance, | 120 |
| abstract_inverted_index.techniques | 169 |
| abstract_inverted_index.validation | 173 |
| abstract_inverted_index.challenges, | 61 |
| abstract_inverted_index.development | 9, 177 |
| abstract_inverted_index.effectively | 135 |
| abstract_inverted_index.efficiency. | 178 |
| abstract_inverted_index.evaluations | 130 |
| abstract_inverted_index.information | 116 |
| abstract_inverted_index.integrating | 114 |
| abstract_inverted_index.multi-modal | 79, 115, 167 |
| abstract_inverted_index.significant | 156 |
| abstract_inverted_index.stakeholder | 21, 141 |
| abstract_inverted_index.unambiguous | 29 |
| abstract_inverted_index.underscores | 163 |
| abstract_inverted_index.RAGcceptance | 64 |
| abstract_inverted_index.challenging, | 33 |
| abstract_inverted_index.correctness, | 121 |
| abstract_inverted_index.information. | 89 |
| abstract_inverted_index.particularly | 34 |
| abstract_inverted_index.practitioner | 129 |
| abstract_inverted_index.requirements | 56, 80 |
| abstract_inverted_index.streamlining | 171 |
| abstract_inverted_index.applications, | 38 |
| abstract_inverted_index.approximately | 107 |
| abstract_inverted_index.demonstrating | 152 |
| abstract_inverted_index.documentation | 85 |
| abstract_inverted_index.expectations. | 22 |
| abstract_inverted_index.significantly | 117 |
| abstract_inverted_index.comprehensive, | 27 |
| abstract_inverted_index.implementation | 183 |
| abstract_inverted_index.systematically | 91 |
| abstract_inverted_index.domain-specific | 44 |
| abstract_inverted_index.comprehensibility | 123 |
| abstract_inverted_index.education-focused | 102 |
| abstract_inverted_index.Retrieval-Augmented | 71 |
| abstract_inverted_index.interface-intensive | 37 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 5 |
| citation_normalized_percentile |