When Large Language Models Confront Repository-Level Automatic Program Repair: How Well They Done? Article Swipe
Yuxiao Chen
,
Jingzheng Wu
,
Xiang Ling
,
Changjiang Li
,
Zhiqing Rui
,
Tianyue Luo
,
Yanjun Wu
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.1145/3639478.3647633
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.1145/3639478.3647633
In recent years, large language models (LLMs) have demonstrated substantial potential in addressing automatic program repair (APR) tasks. However, the current evaluation of these models for APR tasks focuses solely on the limited context of the single function or file where the bug is located, overlooking the valuable information in the repository-level context. This paper investigates the performance of popular LLMs in handling repository-level repair tasks. We introduce RepoBugs, a new benchmark comprising 124 typical repository-level bugs from open-source repositories. Preliminary experiments using GPT3.5 based on the function where the error is located, reveal that the repair rate on RepoBugs is only 22.58%, significantly diverging from the performance of GPT3.5 on function-level bugs in related studies. This underscores the importance of providing repository-level context when addressing bugs at this level. However, the repository-level context offered by the preliminary method often proves redundant and imprecise and easily exceeds the prompt length limit of LLMs. To solve the problem, we propose a simple and universal repository-level context extraction method (RLCE) designed to provide more precise context for repository-level code repair tasks. Evaluations of three mainstream LLMs show that RLCE significantly enhances the ability to repair repository-level bugs. The improvement reaches a maximum of 160% compared to the preliminary method. Additionally, we conduct a comprehensive analysis of the effectiveness and limitations of RLCE, along with the capacity of LLMs to address repository-level bugs, offering valuable insights for future research.
Related Topics
Concepts
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://doi.org/10.1145/3639478.3647633
- https://dl.acm.org/doi/pdf/10.1145/3639478.3647633
- OA Status
- gold
- Cited By
- 9
- References
- 20
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4398239154
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4398239154Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.1145/3639478.3647633Digital Object Identifier
- Title
-
When Large Language Models Confront Repository-Level Automatic Program Repair: How Well They Done?Work title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-04-14Full publication date if available
- Authors
-
Yuxiao Chen, Jingzheng Wu, Xiang Ling, Changjiang Li, Zhiqing Rui, Tianyue Luo, Yanjun WuList of authors in order
- Landing page
-
https://doi.org/10.1145/3639478.3647633Publisher landing page
- PDF URL
-
https://dl.acm.org/doi/pdf/10.1145/3639478.3647633Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
goldOpen access status per OpenAlex
- OA URL
-
https://dl.acm.org/doi/pdf/10.1145/3639478.3647633Direct OA link when available
- Concepts
-
Context (archaeology), Computer science, Benchmark (surveying), Function (biology), Data science, Geodesy, Biology, Evolutionary biology, Geography, PaleontologyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
9Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 9Per-year citation counts (last 5 years)
- References (count)
-
20Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4398239154 |
|---|---|
| doi | https://doi.org/10.1145/3639478.3647633 |
| ids.doi | https://doi.org/10.1145/3639478.3647633 |
| ids.openalex | https://openalex.org/W4398239154 |
| fwci | 13.74885237 |
| type | article |
| title | When Large Language Models Confront Repository-Level Automatic Program Repair: How Well They Done? |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | 471 |
| biblio.first_page | 459 |
| topics[0].id | https://openalex.org/T10260 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9997000098228455 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1710 |
| topics[0].subfield.display_name | Information Systems |
| topics[0].display_name | Software Engineering Research |
| topics[1].id | https://openalex.org/T10743 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9997000098228455 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1712 |
| topics[1].subfield.display_name | Software |
| topics[1].display_name | Software Testing and Debugging Techniques |
| topics[2].id | https://openalex.org/T12423 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9973999857902527 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1712 |
| topics[2].subfield.display_name | Software |
| topics[2].display_name | Software Reliability and Analysis Research |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2779343474 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7461041212081909 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q3109175 |
| concepts[0].display_name | Context (archaeology) |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.7124510407447815 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C185798385 |
| concepts[2].level | 2 |
| concepts[2].score | 0.7000001668930054 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q1161707 |
| concepts[2].display_name | Benchmark (surveying) |
| concepts[3].id | https://openalex.org/C14036430 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5180342793464661 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q3736076 |
| concepts[3].display_name | Function (biology) |
| concepts[4].id | https://openalex.org/C2522767166 |
| concepts[4].level | 1 |
| concepts[4].score | 0.34810736775398254 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q2374463 |
| concepts[4].display_name | Data science |
| concepts[5].id | https://openalex.org/C13280743 |
| concepts[5].level | 1 |
| concepts[5].score | 0.0 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q131089 |
| concepts[5].display_name | Geodesy |
| concepts[6].id | https://openalex.org/C86803240 |
| concepts[6].level | 0 |
| concepts[6].score | 0.0 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q420 |
| concepts[6].display_name | Biology |
| concepts[7].id | https://openalex.org/C78458016 |
| concepts[7].level | 1 |
| concepts[7].score | 0.0 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q840400 |
| concepts[7].display_name | Evolutionary biology |
| concepts[8].id | https://openalex.org/C205649164 |
| concepts[8].level | 0 |
| concepts[8].score | 0.0 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q1071 |
| concepts[8].display_name | Geography |
| concepts[9].id | https://openalex.org/C151730666 |
| concepts[9].level | 1 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q7205 |
| concepts[9].display_name | Paleontology |
| keywords[0].id | https://openalex.org/keywords/context |
| keywords[0].score | 0.7461041212081909 |
| keywords[0].display_name | Context (archaeology) |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.7124510407447815 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/benchmark |
| keywords[2].score | 0.7000001668930054 |
| keywords[2].display_name | Benchmark (surveying) |
| keywords[3].id | https://openalex.org/keywords/function |
| keywords[3].score | 0.5180342793464661 |
| keywords[3].display_name | Function (biology) |
| keywords[4].id | https://openalex.org/keywords/data-science |
| keywords[4].score | 0.34810736775398254 |
| keywords[4].display_name | Data science |
| language | en |
| locations[0].id | doi:10.1145/3639478.3647633 |
| locations[0].is_oa | True |
| locations[0].source | |
| locations[0].license | cc-by |
| locations[0].pdf_url | https://dl.acm.org/doi/pdf/10.1145/3639478.3647633 |
| locations[0].version | publishedVersion |
| locations[0].raw_type | proceedings-article |
| locations[0].license_id | https://openalex.org/licenses/cc-by |
| locations[0].is_accepted | True |
| locations[0].is_published | True |
| locations[0].raw_source_name | Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings |
| locations[0].landing_page_url | https://doi.org/10.1145/3639478.3647633 |
| indexed_in | crossref |
| authorships[0].author.id | https://openalex.org/A5003717748 |
| authorships[0].author.orcid | https://orcid.org/0009-0008-9553-5433 |
| authorships[0].author.display_name | Yuxiao Chen |
| authorships[0].countries | CN |
| authorships[0].affiliations[0].institution_ids | https://openalex.org/I4210128818, https://openalex.org/I4210165038 |
| authorships[0].affiliations[0].raw_affiliation_string | Institute of Software, Chinese Academy of Sciences, Beijing, China University of Chinese Academy of Sciences, Beijing, China |
| authorships[0].affiliations[1].institution_ids | https://openalex.org/I4210165038 |
| authorships[0].affiliations[1].raw_affiliation_string | University of Chinese Academy of Sciences, Beijing, China |
| authorships[0].institutions[0].id | https://openalex.org/I4210128818 |
| authorships[0].institutions[0].ror | https://ror.org/033dfsn42 |
| authorships[0].institutions[0].type | facility |
| authorships[0].institutions[0].lineage | https://openalex.org/I19820366, https://openalex.org/I4210128818 |
| authorships[0].institutions[0].country_code | CN |
| authorships[0].institutions[0].display_name | Institute of Software |
| authorships[0].institutions[1].id | https://openalex.org/I4210165038 |
| authorships[0].institutions[1].ror | https://ror.org/05qbk4x57 |
| authorships[0].institutions[1].type | education |
| authorships[0].institutions[1].lineage | https://openalex.org/I19820366, https://openalex.org/I4210165038 |
| authorships[0].institutions[1].country_code | CN |
| authorships[0].institutions[1].display_name | University of Chinese Academy of Sciences |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Yuxiao Chen |
| authorships[0].is_corresponding | False |
| authorships[0].raw_affiliation_strings | Institute of Software, Chinese Academy of Sciences, Beijing, China University of Chinese Academy of Sciences, Beijing, China, University of Chinese Academy of Sciences, Beijing, China |
| authorships[1].author.id | https://openalex.org/A5101024367 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-5561-9829 |
| authorships[1].author.display_name | Jingzheng Wu |
| authorships[1].countries | CN |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I19820366, https://openalex.org/I4210128818 |
| authorships[1].affiliations[0].raw_affiliation_string | Institute of Software, The Chinese Academy of Sciences, Beijing, China |
| authorships[1].institutions[0].id | https://openalex.org/I19820366 |
| authorships[1].institutions[0].ror | https://ror.org/034t30j35 |
| authorships[1].institutions[0].type | government |
| authorships[1].institutions[0].lineage | https://openalex.org/I19820366 |
| authorships[1].institutions[0].country_code | CN |
| authorships[1].institutions[0].display_name | Chinese Academy of Sciences |
| authorships[1].institutions[1].id | https://openalex.org/I4210128818 |
| authorships[1].institutions[1].ror | https://ror.org/033dfsn42 |
| authorships[1].institutions[1].type | facility |
| authorships[1].institutions[1].lineage | https://openalex.org/I19820366, https://openalex.org/I4210128818 |
| authorships[1].institutions[1].country_code | CN |
| authorships[1].institutions[1].display_name | Institute of Software |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Jingzheng Wu |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | Institute of Software, The Chinese Academy of Sciences, Beijing, China |
| authorships[2].author.id | https://openalex.org/A5101416036 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-7377-7844 |
| authorships[2].author.display_name | Xiang Ling |
| authorships[2].countries | CN |
| authorships[2].affiliations[0].institution_ids | https://openalex.org/I19820366, https://openalex.org/I4210128818 |
| authorships[2].affiliations[0].raw_affiliation_string | Institute of Software, Chinese Academy of Sciences, Beijing, China |
| authorships[2].institutions[0].id | https://openalex.org/I19820366 |
| authorships[2].institutions[0].ror | https://ror.org/034t30j35 |
| authorships[2].institutions[0].type | government |
| authorships[2].institutions[0].lineage | https://openalex.org/I19820366 |
| authorships[2].institutions[0].country_code | CN |
| authorships[2].institutions[0].display_name | Chinese Academy of Sciences |
| authorships[2].institutions[1].id | https://openalex.org/I4210128818 |
| authorships[2].institutions[1].ror | https://ror.org/033dfsn42 |
| authorships[2].institutions[1].type | facility |
| authorships[2].institutions[1].lineage | https://openalex.org/I19820366, https://openalex.org/I4210128818 |
| authorships[2].institutions[1].country_code | CN |
| authorships[2].institutions[1].display_name | Institute of Software |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Xiang Ling |
| authorships[2].is_corresponding | False |
| authorships[2].raw_affiliation_strings | Institute of Software, Chinese Academy of Sciences, Beijing, China |
| authorships[3].author.id | https://openalex.org/A5101504674 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-1671-7183 |
| authorships[3].author.display_name | Changjiang Li |
| authorships[3].countries | US |
| authorships[3].affiliations[0].institution_ids | https://openalex.org/I59553526 |
| authorships[3].affiliations[0].raw_affiliation_string | Stony Brook University, New York, USA |
| authorships[3].institutions[0].id | https://openalex.org/I59553526 |
| authorships[3].institutions[0].ror | https://ror.org/05qghxh33 |
| authorships[3].institutions[0].type | education |
| authorships[3].institutions[0].lineage | https://openalex.org/I59553526 |
| authorships[3].institutions[0].country_code | US |
| authorships[3].institutions[0].display_name | Stony Brook University |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Changjiang Li |
| authorships[3].is_corresponding | False |
| authorships[3].raw_affiliation_strings | Stony Brook University, New York, USA |
| authorships[4].author.id | https://openalex.org/A5002459090 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-9444-0845 |
| authorships[4].author.display_name | Zhiqing Rui |
| authorships[4].countries | CN |
| authorships[4].affiliations[0].institution_ids | https://openalex.org/I4210165038 |
| authorships[4].affiliations[0].raw_affiliation_string | University of Chinese Academy of Sciences, Beijing, China |
| authorships[4].affiliations[1].institution_ids | https://openalex.org/I4210128818, https://openalex.org/I4210165038 |
| authorships[4].affiliations[1].raw_affiliation_string | Institute of Software, Chinese Academy of Sciences, Beijing, China University of Chinese Academy of Sciences, Beijing, China |
| authorships[4].institutions[0].id | https://openalex.org/I4210128818 |
| authorships[4].institutions[0].ror | https://ror.org/033dfsn42 |
| authorships[4].institutions[0].type | facility |
| authorships[4].institutions[0].lineage | https://openalex.org/I19820366, https://openalex.org/I4210128818 |
| authorships[4].institutions[0].country_code | CN |
| authorships[4].institutions[0].display_name | Institute of Software |
| authorships[4].institutions[1].id | https://openalex.org/I4210165038 |
| authorships[4].institutions[1].ror | https://ror.org/05qbk4x57 |
| authorships[4].institutions[1].type | education |
| authorships[4].institutions[1].lineage | https://openalex.org/I19820366, https://openalex.org/I4210165038 |
| authorships[4].institutions[1].country_code | CN |
| authorships[4].institutions[1].display_name | University of Chinese Academy of Sciences |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Zhiqing Rui |
| authorships[4].is_corresponding | False |
| authorships[4].raw_affiliation_strings | Institute of Software, Chinese Academy of Sciences, Beijing, China University of Chinese Academy of Sciences, Beijing, China, University of Chinese Academy of Sciences, Beijing, China |
| authorships[5].author.id | https://openalex.org/A5021069086 |
| authorships[5].author.orcid | https://orcid.org/0009-0000-9454-2924 |
| authorships[5].author.display_name | Tianyue Luo |
| authorships[5].countries | CN |
| authorships[5].affiliations[0].institution_ids | https://openalex.org/I19820366, https://openalex.org/I4210128818 |
| authorships[5].affiliations[0].raw_affiliation_string | Institute of Software, Chinese Academy of Sciences, Beijing, China |
| authorships[5].institutions[0].id | https://openalex.org/I19820366 |
| authorships[5].institutions[0].ror | https://ror.org/034t30j35 |
| authorships[5].institutions[0].type | government |
| authorships[5].institutions[0].lineage | https://openalex.org/I19820366 |
| authorships[5].institutions[0].country_code | CN |
| authorships[5].institutions[0].display_name | Chinese Academy of Sciences |
| authorships[5].institutions[1].id | https://openalex.org/I4210128818 |
| authorships[5].institutions[1].ror | https://ror.org/033dfsn42 |
| authorships[5].institutions[1].type | facility |
| authorships[5].institutions[1].lineage | https://openalex.org/I19820366, https://openalex.org/I4210128818 |
| authorships[5].institutions[1].country_code | CN |
| authorships[5].institutions[1].display_name | Institute of Software |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Tianyue Luo |
| authorships[5].is_corresponding | False |
| authorships[5].raw_affiliation_strings | Institute of Software, Chinese Academy of Sciences, Beijing, China |
| authorships[6].author.id | https://openalex.org/A5101053493 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-1823-0459 |
| authorships[6].author.display_name | Yanjun Wu |
| authorships[6].countries | CN |
| authorships[6].affiliations[0].institution_ids | https://openalex.org/I19820366, https://openalex.org/I4210128818 |
| authorships[6].affiliations[0].raw_affiliation_string | Institute of Software, Chinese Academy of Sciences, Beijing, China |
| authorships[6].institutions[0].id | https://openalex.org/I19820366 |
| authorships[6].institutions[0].ror | https://ror.org/034t30j35 |
| authorships[6].institutions[0].type | government |
| authorships[6].institutions[0].lineage | https://openalex.org/I19820366 |
| authorships[6].institutions[0].country_code | CN |
| authorships[6].institutions[0].display_name | Chinese Academy of Sciences |
| authorships[6].institutions[1].id | https://openalex.org/I4210128818 |
| authorships[6].institutions[1].ror | https://ror.org/033dfsn42 |
| authorships[6].institutions[1].type | facility |
| authorships[6].institutions[1].lineage | https://openalex.org/I19820366, https://openalex.org/I4210128818 |
| authorships[6].institutions[1].country_code | CN |
| authorships[6].institutions[1].display_name | Institute of Software |
| authorships[6].author_position | last |
| authorships[6].raw_author_name | Yanjun Wu |
| authorships[6].is_corresponding | False |
| authorships[6].raw_affiliation_strings | Institute of Software, Chinese Academy of Sciences, Beijing, China |
| has_content.pdf | True |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://dl.acm.org/doi/pdf/10.1145/3639478.3647633 |
| open_access.oa_status | gold |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | When Large Language Models Confront Repository-Level Automatic Program Repair: How Well They Done? |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10260 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9997000098228455 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1710 |
| primary_topic.subfield.display_name | Information Systems |
| primary_topic.display_name | Software Engineering Research |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W2378211422, https://openalex.org/W4321353415, https://openalex.org/W2745001401, https://openalex.org/W2130974462, https://openalex.org/W2028665553, https://openalex.org/W2086519370, https://openalex.org/W972276598, https://openalex.org/W2087343574 |
| cited_by_count | 9 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 9 |
| locations_count | 1 |
| best_oa_location.id | doi:10.1145/3639478.3647633 |
| best_oa_location.is_oa | True |
| best_oa_location.source | |
| best_oa_location.license | cc-by |
| best_oa_location.pdf_url | https://dl.acm.org/doi/pdf/10.1145/3639478.3647633 |
| best_oa_location.version | publishedVersion |
| best_oa_location.raw_type | proceedings-article |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by |
| best_oa_location.is_accepted | True |
| best_oa_location.is_published | True |
| best_oa_location.raw_source_name | Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings |
| best_oa_location.landing_page_url | https://doi.org/10.1145/3639478.3647633 |
| primary_location.id | doi:10.1145/3639478.3647633 |
| primary_location.is_oa | True |
| primary_location.source | |
| primary_location.license | cc-by |
| primary_location.pdf_url | https://dl.acm.org/doi/pdf/10.1145/3639478.3647633 |
| primary_location.version | publishedVersion |
| primary_location.raw_type | proceedings-article |
| primary_location.license_id | https://openalex.org/licenses/cc-by |
| primary_location.is_accepted | True |
| primary_location.is_published | True |
| primary_location.raw_source_name | Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings |
| primary_location.landing_page_url | https://doi.org/10.1145/3639478.3647633 |
| publication_date | 2024-04-14 |
| publication_year | 2024 |
| referenced_works | https://openalex.org/W2114334695, https://openalex.org/W2907705732, https://openalex.org/W2605202003, https://openalex.org/W2898644864, https://openalex.org/W3134686229, https://openalex.org/W2156723666, https://openalex.org/W3169424470, https://openalex.org/W2762550985, https://openalex.org/W2924629359, https://openalex.org/W2908354350, https://openalex.org/W3185341429, https://openalex.org/W3043761819, https://openalex.org/W2034256728, https://openalex.org/W4226278401, https://openalex.org/W2134119432, https://openalex.org/W4253053708, https://openalex.org/W2741422284, https://openalex.org/W4221143046, https://openalex.org/W2122947685, https://openalex.org/W3161027892 |
| referenced_works_count | 20 |
| abstract_inverted_index.a | 69, 159, 198, 210 |
| abstract_inverted_index.In | 0 |
| abstract_inverted_index.To | 153 |
| abstract_inverted_index.We | 66 |
| abstract_inverted_index.at | 127 |
| abstract_inverted_index.by | 135 |
| abstract_inverted_index.in | 11, 49, 61, 113 |
| abstract_inverted_index.is | 43, 91, 100 |
| abstract_inverted_index.of | 22, 34, 58, 108, 120, 151, 180, 200, 213, 218, 224 |
| abstract_inverted_index.on | 30, 85, 98, 110 |
| abstract_inverted_index.or | 38 |
| abstract_inverted_index.to | 169, 191, 203, 226 |
| abstract_inverted_index.we | 157, 208 |
| abstract_inverted_index.124 | 73 |
| abstract_inverted_index.APR | 26 |
| abstract_inverted_index.The | 195 |
| abstract_inverted_index.and | 142, 144, 161, 216 |
| abstract_inverted_index.bug | 42 |
| abstract_inverted_index.for | 25, 174, 233 |
| abstract_inverted_index.new | 70 |
| abstract_inverted_index.the | 19, 31, 35, 41, 46, 50, 56, 86, 89, 95, 106, 118, 131, 136, 147, 155, 189, 204, 214, 222 |
| abstract_inverted_index.160% | 201 |
| abstract_inverted_index.LLMs | 60, 183, 225 |
| abstract_inverted_index.RLCE | 186 |
| abstract_inverted_index.This | 53, 116 |
| abstract_inverted_index.bugs | 76, 112, 126 |
| abstract_inverted_index.code | 176 |
| abstract_inverted_index.file | 39 |
| abstract_inverted_index.from | 77, 105 |
| abstract_inverted_index.have | 7 |
| abstract_inverted_index.more | 171 |
| abstract_inverted_index.only | 101 |
| abstract_inverted_index.rate | 97 |
| abstract_inverted_index.show | 184 |
| abstract_inverted_index.that | 94, 185 |
| abstract_inverted_index.this | 128 |
| abstract_inverted_index.when | 124 |
| abstract_inverted_index.with | 221 |
| abstract_inverted_index.(APR) | 16 |
| abstract_inverted_index.LLMs. | 152 |
| abstract_inverted_index.RLCE, | 219 |
| abstract_inverted_index.along | 220 |
| abstract_inverted_index.based | 84 |
| abstract_inverted_index.bugs, | 229 |
| abstract_inverted_index.bugs. | 194 |
| abstract_inverted_index.error | 90 |
| abstract_inverted_index.large | 3 |
| abstract_inverted_index.limit | 150 |
| abstract_inverted_index.often | 139 |
| abstract_inverted_index.paper | 54 |
| abstract_inverted_index.solve | 154 |
| abstract_inverted_index.tasks | 27 |
| abstract_inverted_index.these | 23 |
| abstract_inverted_index.three | 181 |
| abstract_inverted_index.using | 82 |
| abstract_inverted_index.where | 40, 88 |
| abstract_inverted_index.(LLMs) | 6 |
| abstract_inverted_index.(RLCE) | 167 |
| abstract_inverted_index.GPT3.5 | 83, 109 |
| abstract_inverted_index.easily | 145 |
| abstract_inverted_index.future | 234 |
| abstract_inverted_index.length | 149 |
| abstract_inverted_index.level. | 129 |
| abstract_inverted_index.method | 138, 166 |
| abstract_inverted_index.models | 5, 24 |
| abstract_inverted_index.prompt | 148 |
| abstract_inverted_index.proves | 140 |
| abstract_inverted_index.recent | 1 |
| abstract_inverted_index.repair | 15, 64, 96, 177, 192 |
| abstract_inverted_index.reveal | 93 |
| abstract_inverted_index.simple | 160 |
| abstract_inverted_index.single | 36 |
| abstract_inverted_index.solely | 29 |
| abstract_inverted_index.tasks. | 17, 65, 178 |
| abstract_inverted_index.years, | 2 |
| abstract_inverted_index.22.58%, | 102 |
| abstract_inverted_index.ability | 190 |
| abstract_inverted_index.address | 227 |
| abstract_inverted_index.conduct | 209 |
| abstract_inverted_index.context | 33, 123, 133, 164, 173 |
| abstract_inverted_index.current | 20 |
| abstract_inverted_index.exceeds | 146 |
| abstract_inverted_index.focuses | 28 |
| abstract_inverted_index.limited | 32 |
| abstract_inverted_index.maximum | 199 |
| abstract_inverted_index.method. | 206 |
| abstract_inverted_index.offered | 134 |
| abstract_inverted_index.popular | 59 |
| abstract_inverted_index.precise | 172 |
| abstract_inverted_index.program | 14 |
| abstract_inverted_index.propose | 158 |
| abstract_inverted_index.provide | 170 |
| abstract_inverted_index.reaches | 197 |
| abstract_inverted_index.related | 114 |
| abstract_inverted_index.typical | 74 |
| abstract_inverted_index.However, | 18, 130 |
| abstract_inverted_index.RepoBugs | 99 |
| abstract_inverted_index.analysis | 212 |
| abstract_inverted_index.capacity | 223 |
| abstract_inverted_index.compared | 202 |
| abstract_inverted_index.context. | 52 |
| abstract_inverted_index.designed | 168 |
| abstract_inverted_index.enhances | 188 |
| abstract_inverted_index.function | 37, 87 |
| abstract_inverted_index.handling | 62 |
| abstract_inverted_index.insights | 232 |
| abstract_inverted_index.language | 4 |
| abstract_inverted_index.located, | 44, 92 |
| abstract_inverted_index.offering | 230 |
| abstract_inverted_index.problem, | 156 |
| abstract_inverted_index.studies. | 115 |
| abstract_inverted_index.valuable | 47, 231 |
| abstract_inverted_index.RepoBugs, | 68 |
| abstract_inverted_index.automatic | 13 |
| abstract_inverted_index.benchmark | 71 |
| abstract_inverted_index.diverging | 104 |
| abstract_inverted_index.imprecise | 143 |
| abstract_inverted_index.introduce | 67 |
| abstract_inverted_index.potential | 10 |
| abstract_inverted_index.providing | 121 |
| abstract_inverted_index.redundant | 141 |
| abstract_inverted_index.research. | 235 |
| abstract_inverted_index.universal | 162 |
| abstract_inverted_index.addressing | 12, 125 |
| abstract_inverted_index.comprising | 72 |
| abstract_inverted_index.evaluation | 21 |
| abstract_inverted_index.extraction | 165 |
| abstract_inverted_index.importance | 119 |
| abstract_inverted_index.mainstream | 182 |
| abstract_inverted_index.Evaluations | 179 |
| abstract_inverted_index.Preliminary | 80 |
| abstract_inverted_index.experiments | 81 |
| abstract_inverted_index.improvement | 196 |
| abstract_inverted_index.information | 48 |
| abstract_inverted_index.limitations | 217 |
| abstract_inverted_index.open-source | 78 |
| abstract_inverted_index.overlooking | 45 |
| abstract_inverted_index.performance | 57, 107 |
| abstract_inverted_index.preliminary | 137, 205 |
| abstract_inverted_index.substantial | 9 |
| abstract_inverted_index.underscores | 117 |
| abstract_inverted_index.demonstrated | 8 |
| abstract_inverted_index.investigates | 55 |
| abstract_inverted_index.Additionally, | 207 |
| abstract_inverted_index.comprehensive | 211 |
| abstract_inverted_index.effectiveness | 215 |
| abstract_inverted_index.repositories. | 79 |
| abstract_inverted_index.significantly | 103, 187 |
| abstract_inverted_index.function-level | 111 |
| abstract_inverted_index.repository-level | 51, 63, 75, 122, 132, 163, 175, 193, 228 |
| cited_by_percentile_year.max | 99 |
| cited_by_percentile_year.min | 98 |
| countries_distinct_count | 2 |
| institutions_distinct_count | 7 |
| citation_normalized_percentile.value | 0.97792888 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | True |