The Right Prompts for the Job: Repair Code-Review Defects with Large Language Model Article Swipe
Zelin Zhao
,
Zhaogui Xu
,
Jialong Zhu
,
Di Peng
,
Yuan Yao
,
Xiaoxing Ma
·
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2312.17485
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2312.17485
Automatic program repair (APR) techniques have the potential to reduce manual efforts in uncovering and repairing program defects during the code review (CR) process. However, the limited accuracy and considerable time costs associated with existing APR approaches hinder their adoption in industrial practice. One key factor is the under-utilization of review comments, which provide valuable insights into defects and potential fixes. Recent advancements in Large Language Models (LLMs) have enhanced their ability to comprehend natural and programming languages, enabling them to generate patches based on review comments. This paper conducts a comprehensive investigation into the effective utilization of LLMs for repairing CR defects. In this study, various prompts are designed and compared across mainstream LLMs using two distinct datasets from human reviewers and automated checkers. Experimental results demonstrate a remarkable repair rate of 72.97% with the best prompt, highlighting a substantial improvement in the effectiveness and practicality of automatic repair techniques.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2312.17485
- https://arxiv.org/pdf/2312.17485
- OA Status
- green
- Cited By
- 1
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4390489717
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4390489717Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2312.17485Digital Object Identifier
- Title
-
The Right Prompts for the Job: Repair Code-Review Defects with Large Language ModelWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-12-29Full publication date if available
- Authors
-
Zelin Zhao, Zhaogui Xu, Jialong Zhu, Di Peng, Yuan Yao, Xiaoxing MaList of authors in order
- Landing page
-
https://arxiv.org/abs/2312.17485Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2312.17485Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2312.17485Direct OA link when available
- Concepts
-
Computer science, Process (computing), Mainstream, Code (set theory), Risk analysis (engineering), Key (lock), Data science, Computer security, Programming language, Medicine, Political science, Law, Set (abstract data type)Top concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2024: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4390489717 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2312.17485 |
| ids.doi | https://doi.org/10.48550/arxiv.2312.17485 |
| ids.openalex | https://openalex.org/W4390489717 |
| fwci | |
| type | preprint |
| title | The Right Prompts for the Job: Repair Code-Review Defects with Large Language Model |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10260 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9994999766349792 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1710 |
| topics[0].subfield.display_name | Information Systems |
| topics[0].display_name | Software Engineering Research |
| topics[1].id | https://openalex.org/T10743 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9984999895095825 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1712 |
| topics[1].subfield.display_name | Software |
| topics[1].display_name | Software Testing and Debugging Techniques |
| topics[2].id | https://openalex.org/T12423 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9983999729156494 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1712 |
| topics[2].subfield.display_name | Software |
| topics[2].display_name | Software Reliability and Analysis Research |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.645577609539032 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C98045186 |
| concepts[1].level | 2 |
| concepts[1].score | 0.578289806842804 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q205663 |
| concepts[1].display_name | Process (computing) |
| concepts[2].id | https://openalex.org/C2777617010 |
| concepts[2].level | 2 |
| concepts[2].score | 0.5618278980255127 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q18957 |
| concepts[2].display_name | Mainstream |
| concepts[3].id | https://openalex.org/C2776760102 |
| concepts[3].level | 3 |
| concepts[3].score | 0.5317203998565674 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q5139990 |
| concepts[3].display_name | Code (set theory) |
| concepts[4].id | https://openalex.org/C112930515 |
| concepts[4].level | 1 |
| concepts[4].score | 0.46145567297935486 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q4389547 |
| concepts[4].display_name | Risk analysis (engineering) |
| concepts[5].id | https://openalex.org/C26517878 |
| concepts[5].level | 2 |
| concepts[5].score | 0.46029743552207947 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q228039 |
| concepts[5].display_name | Key (lock) |
| concepts[6].id | https://openalex.org/C2522767166 |
| concepts[6].level | 1 |
| concepts[6].score | 0.3330715596675873 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q2374463 |
| concepts[6].display_name | Data science |
| concepts[7].id | https://openalex.org/C38652104 |
| concepts[7].level | 1 |
| concepts[7].score | 0.1732768416404724 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q3510521 |
| concepts[7].display_name | Computer security |
| concepts[8].id | https://openalex.org/C199360897 |
| concepts[8].level | 1 |
| concepts[8].score | 0.1460910439491272 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[8].display_name | Programming language |
| concepts[9].id | https://openalex.org/C71924100 |
| concepts[9].level | 0 |
| concepts[9].score | 0.1451033055782318 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q11190 |
| concepts[9].display_name | Medicine |
| concepts[10].id | https://openalex.org/C17744445 |
| concepts[10].level | 0 |
| concepts[10].score | 0.10589626431465149 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q36442 |
| concepts[10].display_name | Political science |
| concepts[11].id | https://openalex.org/C199539241 |
| concepts[11].level | 1 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q7748 |
| concepts[11].display_name | Law |
| concepts[12].id | https://openalex.org/C177264268 |
| concepts[12].level | 2 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q1514741 |
| concepts[12].display_name | Set (abstract data type) |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.645577609539032 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/process |
| keywords[1].score | 0.578289806842804 |
| keywords[1].display_name | Process (computing) |
| keywords[2].id | https://openalex.org/keywords/mainstream |
| keywords[2].score | 0.5618278980255127 |
| keywords[2].display_name | Mainstream |
| keywords[3].id | https://openalex.org/keywords/code |
| keywords[3].score | 0.5317203998565674 |
| keywords[3].display_name | Code (set theory) |
| keywords[4].id | https://openalex.org/keywords/risk-analysis |
| keywords[4].score | 0.46145567297935486 |
| keywords[4].display_name | Risk analysis (engineering) |
| keywords[5].id | https://openalex.org/keywords/key |
| keywords[5].score | 0.46029743552207947 |
| keywords[5].display_name | Key (lock) |
| keywords[6].id | https://openalex.org/keywords/data-science |
| keywords[6].score | 0.3330715596675873 |
| keywords[6].display_name | Data science |
| keywords[7].id | https://openalex.org/keywords/computer-security |
| keywords[7].score | 0.1732768416404724 |
| keywords[7].display_name | Computer security |
| keywords[8].id | https://openalex.org/keywords/programming-language |
| keywords[8].score | 0.1460910439491272 |
| keywords[8].display_name | Programming language |
| keywords[9].id | https://openalex.org/keywords/medicine |
| keywords[9].score | 0.1451033055782318 |
| keywords[9].display_name | Medicine |
| keywords[10].id | https://openalex.org/keywords/political-science |
| keywords[10].score | 0.10589626431465149 |
| keywords[10].display_name | Political science |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2312.17485 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2312.17485 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2312.17485 |
| locations[1].id | doi:10.48550/arxiv.2312.17485 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2312.17485 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5089593268 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-2638-0414 |
| authorships[0].author.display_name | Zelin Zhao |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Zhao, Zelin |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5044027332 |
| authorships[1].author.orcid | https://orcid.org/0009-0009-3975-2481 |
| authorships[1].author.display_name | Zhaogui Xu |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Xu, Zhaogui |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5101000545 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Jialong Zhu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Zhu, Jialong |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5029153155 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-8116-5215 |
| authorships[3].author.display_name | Di Peng |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Di, Peng |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5000537533 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-9359-2030 |
| authorships[4].author.display_name | Yuan Yao |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Yao, Yuan |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5041674680 |
| authorships[5].author.orcid | https://orcid.org/0000-0001-7970-1384 |
| authorships[5].author.display_name | Xiaoxing Ma |
| authorships[5].author_position | last |
| authorships[5].raw_author_name | Ma, Xiaoxing |
| authorships[5].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2312.17485 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-01-02T00:00:00 |
| display_name | The Right Prompts for the Job: Repair Code-Review Defects with Large Language Model |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10260 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9994999766349792 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1710 |
| primary_topic.subfield.display_name | Information Systems |
| primary_topic.display_name | Software Engineering Research |
| related_works | https://openalex.org/W1583826057, https://openalex.org/W2377237701, https://openalex.org/W2360099860, https://openalex.org/W4323893170, https://openalex.org/W2352463596, https://openalex.org/W2380850119, https://openalex.org/W2101450440, https://openalex.org/W2383675217, https://openalex.org/W2376151201, https://openalex.org/W2393898889 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2024 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2312.17485 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2312.17485 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2312.17485 |
| primary_location.id | pmh:oai:arXiv.org:2312.17485 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2312.17485 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2312.17485 |
| publication_date | 2023-12-29 |
| publication_year | 2023 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 90, 128, 139 |
| abstract_inverted_index.CR | 101 |
| abstract_inverted_index.In | 103 |
| abstract_inverted_index.in | 12, 40, 63, 142 |
| abstract_inverted_index.is | 46 |
| abstract_inverted_index.of | 49, 97, 132, 147 |
| abstract_inverted_index.on | 84 |
| abstract_inverted_index.to | 8, 72, 80 |
| abstract_inverted_index.APR | 35 |
| abstract_inverted_index.One | 43 |
| abstract_inverted_index.and | 14, 28, 58, 75, 110, 122, 145 |
| abstract_inverted_index.are | 108 |
| abstract_inverted_index.for | 99 |
| abstract_inverted_index.key | 44 |
| abstract_inverted_index.the | 6, 19, 25, 47, 94, 135, 143 |
| abstract_inverted_index.two | 116 |
| abstract_inverted_index.(CR) | 22 |
| abstract_inverted_index.LLMs | 98, 114 |
| abstract_inverted_index.This | 87 |
| abstract_inverted_index.best | 136 |
| abstract_inverted_index.code | 20 |
| abstract_inverted_index.from | 119 |
| abstract_inverted_index.have | 5, 68 |
| abstract_inverted_index.into | 56, 93 |
| abstract_inverted_index.rate | 131 |
| abstract_inverted_index.them | 79 |
| abstract_inverted_index.this | 104 |
| abstract_inverted_index.time | 30 |
| abstract_inverted_index.with | 33, 134 |
| abstract_inverted_index.(APR) | 3 |
| abstract_inverted_index.Large | 64 |
| abstract_inverted_index.based | 83 |
| abstract_inverted_index.costs | 31 |
| abstract_inverted_index.human | 120 |
| abstract_inverted_index.paper | 88 |
| abstract_inverted_index.their | 38, 70 |
| abstract_inverted_index.using | 115 |
| abstract_inverted_index.which | 52 |
| abstract_inverted_index.(LLMs) | 67 |
| abstract_inverted_index.72.97% | 133 |
| abstract_inverted_index.Models | 66 |
| abstract_inverted_index.Recent | 61 |
| abstract_inverted_index.across | 112 |
| abstract_inverted_index.during | 18 |
| abstract_inverted_index.factor | 45 |
| abstract_inverted_index.fixes. | 60 |
| abstract_inverted_index.hinder | 37 |
| abstract_inverted_index.manual | 10 |
| abstract_inverted_index.reduce | 9 |
| abstract_inverted_index.repair | 2, 130, 149 |
| abstract_inverted_index.review | 21, 50, 85 |
| abstract_inverted_index.study, | 105 |
| abstract_inverted_index.ability | 71 |
| abstract_inverted_index.defects | 17, 57 |
| abstract_inverted_index.efforts | 11 |
| abstract_inverted_index.limited | 26 |
| abstract_inverted_index.natural | 74 |
| abstract_inverted_index.patches | 82 |
| abstract_inverted_index.program | 1, 16 |
| abstract_inverted_index.prompt, | 137 |
| abstract_inverted_index.prompts | 107 |
| abstract_inverted_index.provide | 53 |
| abstract_inverted_index.results | 126 |
| abstract_inverted_index.various | 106 |
| abstract_inverted_index.However, | 24 |
| abstract_inverted_index.Language | 65 |
| abstract_inverted_index.accuracy | 27 |
| abstract_inverted_index.adoption | 39 |
| abstract_inverted_index.compared | 111 |
| abstract_inverted_index.conducts | 89 |
| abstract_inverted_index.datasets | 118 |
| abstract_inverted_index.defects. | 102 |
| abstract_inverted_index.designed | 109 |
| abstract_inverted_index.distinct | 117 |
| abstract_inverted_index.enabling | 78 |
| abstract_inverted_index.enhanced | 69 |
| abstract_inverted_index.existing | 34 |
| abstract_inverted_index.generate | 81 |
| abstract_inverted_index.insights | 55 |
| abstract_inverted_index.process. | 23 |
| abstract_inverted_index.valuable | 54 |
| abstract_inverted_index.Automatic | 0 |
| abstract_inverted_index.automated | 123 |
| abstract_inverted_index.automatic | 148 |
| abstract_inverted_index.checkers. | 124 |
| abstract_inverted_index.comments, | 51 |
| abstract_inverted_index.comments. | 86 |
| abstract_inverted_index.effective | 95 |
| abstract_inverted_index.potential | 7, 59 |
| abstract_inverted_index.practice. | 42 |
| abstract_inverted_index.repairing | 15, 100 |
| abstract_inverted_index.reviewers | 121 |
| abstract_inverted_index.approaches | 36 |
| abstract_inverted_index.associated | 32 |
| abstract_inverted_index.comprehend | 73 |
| abstract_inverted_index.industrial | 41 |
| abstract_inverted_index.languages, | 77 |
| abstract_inverted_index.mainstream | 113 |
| abstract_inverted_index.remarkable | 129 |
| abstract_inverted_index.techniques | 4 |
| abstract_inverted_index.uncovering | 13 |
| abstract_inverted_index.demonstrate | 127 |
| abstract_inverted_index.improvement | 141 |
| abstract_inverted_index.programming | 76 |
| abstract_inverted_index.substantial | 140 |
| abstract_inverted_index.techniques. | 150 |
| abstract_inverted_index.utilization | 96 |
| abstract_inverted_index.Experimental | 125 |
| abstract_inverted_index.advancements | 62 |
| abstract_inverted_index.considerable | 29 |
| abstract_inverted_index.highlighting | 138 |
| abstract_inverted_index.practicality | 146 |
| abstract_inverted_index.comprehensive | 91 |
| abstract_inverted_index.effectiveness | 144 |
| abstract_inverted_index.investigation | 92 |
| abstract_inverted_index.under-utilization | 48 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 6 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.49000000953674316 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile |