AUCAD: Automated Construction of Alignment Dataset from Log-Related Issues for Enhancing LLM-based Log Generation Article Swipe
Hao Zhang
,
Dong‐Jun Yu
,
Lei Zhang
,
Guoping Rong
,
Yongda Yu
,
Haifeng Shen
,
Qi Chen
,
Dong Shao
,
Hongyu Kuang
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2412.18835
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2412.18835
Log statements have become an integral part of modern software systems. Prior research efforts have focused on supporting the decisions of placing log statements, such as where/what to log. With the increasing adoption of Large Language Models (LLMs) for code-related tasks such as code completion or generation, automated approaches for generating log statements have gained much momentum. However, the performance of these approaches still has a long way to go. This paper explores enhancing the performance of LLM-based solutions for automated log statement generation by post-training LLMs with a purpose-built dataset. Thus the primary contribution is a novel approach called AUCAD, which automatically constructs such a dataset with information extracting from log-related issues. Researchers have long noticed that a significant portion of the issues in the open-source community are related to log statements. However, distilling this portion of data requires manual efforts, which is labor-intensive and costly, rendering it impractical. Utilizing our approach, we automatically extract log-related issues from 1,537 entries of log data across 88 projects and identify 808 code snippets (i.e., methods) with retrievable source code both before and after modification of each issue (including log statements) to construct a dataset. Each entry in the dataset consists of a data pair representing high-quality and problematic log statements, respectively. With this dataset, we proceed to post-train multiple LLMs (primarily from the Llama series) for automated log statement generation. Both human and experimental evaluations indicate that these models significantly outperform existing LLM-based solutions, thereby validating the efficacy of our method for constructing a post-training dataset to enhance LLM-based log statement generation.
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2412.18835
- https://arxiv.org/pdf/2412.18835
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4405900643
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4405900643Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2412.18835Digital Object Identifier
- Title
-
AUCAD: Automated Construction of Alignment Dataset from Log-Related Issues for Enhancing LLM-based Log GenerationWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-12-25Full publication date if available
- Authors
-
Hao Zhang, Dong‐Jun Yu, Lei Zhang, Guoping Rong, Yongda Yu, Haifeng Shen, Qi Chen, Dong Shao, Hongyu KuangList of authors in order
- Landing page
-
https://arxiv.org/abs/2412.18835Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2412.18835Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2412.18835Direct OA link when available
- Concepts
-
Computer science, Programming language, Natural language processingTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4405900643 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2412.18835 |
| ids.doi | https://doi.org/10.48550/arxiv.2412.18835 |
| ids.openalex | https://openalex.org/W4405900643 |
| fwci | |
| type | preprint |
| title | AUCAD: Automated Construction of Alignment Dataset from Log-Related Issues for Enhancing LLM-based Log Generation |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10181 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9797000288963318 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Natural Language Processing Techniques |
| topics[1].id | https://openalex.org/T10215 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9729999899864197 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Semantic Web and Ontologies |
| topics[2].id | https://openalex.org/T10028 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9467999935150146 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Topic Modeling |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.5742788910865784 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C199360897 |
| concepts[1].level | 1 |
| concepts[1].score | 0.3697492480278015 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[1].display_name | Programming language |
| concepts[2].id | https://openalex.org/C204321447 |
| concepts[2].level | 1 |
| concepts[2].score | 0.3537198305130005 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[2].display_name | Natural language processing |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.5742788910865784 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/programming-language |
| keywords[1].score | 0.3697492480278015 |
| keywords[1].display_name | Programming language |
| keywords[2].id | https://openalex.org/keywords/natural-language-processing |
| keywords[2].score | 0.3537198305130005 |
| keywords[2].display_name | Natural language processing |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2412.18835 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2412.18835 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2412.18835 |
| locations[1].id | doi:10.48550/arxiv.2412.18835 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2412.18835 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5100396844 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-0404-6941 |
| authorships[0].author.display_name | Hao Zhang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Zhang, Hao |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5008960436 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-6786-8053 |
| authorships[1].author.display_name | Dong‐Jun Yu |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Yu, Dongjun |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5116592761 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-1520-9360 |
| authorships[2].author.display_name | Lei Zhang |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Zhang, Lei |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5073751973 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-4576-0524 |
| authorships[3].author.display_name | Guoping Rong |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Rong, Guoping |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5102242486 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-6713-2364 |
| authorships[4].author.display_name | Yongda Yu |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Yu, Yongda |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5084809225 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-8221-981X |
| authorships[5].author.display_name | Haifeng Shen |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Shen, Haifeng |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5118092732 |
| authorships[6].author.orcid | https://orcid.org/0009-0001-2316-4521 |
| authorships[6].author.display_name | Qi Chen |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Zhang, He |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5101193832 |
| authorships[7].author.orcid | https://orcid.org/0000-0001-6500-0341 |
| authorships[7].author.display_name | Dong Shao |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Shao, Dong |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5043601793 |
| authorships[8].author.orcid | https://orcid.org/0009-0003-8702-2826 |
| authorships[8].author.display_name | Hongyu Kuang |
| authorships[8].author_position | last |
| authorships[8].raw_author_name | Kuang, Hongyu |
| authorships[8].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2412.18835 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-12-31T00:00:00 |
| display_name | AUCAD: Automated Construction of Alignment Dataset from Log-Related Issues for Enhancing LLM-based Log Generation |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T10181 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9797000288963318 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Natural Language Processing Techniques |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2412.18835 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2412.18835 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2412.18835 |
| primary_location.id | pmh:oai:arXiv.org:2412.18835 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2412.18835 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2412.18835 |
| publication_date | 2024-12-25 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 65, 88, 96, 105, 118, 191, 200, 252 |
| abstract_inverted_index.88 | 165 |
| abstract_inverted_index.an | 4 |
| abstract_inverted_index.as | 25, 42 |
| abstract_inverted_index.by | 84 |
| abstract_inverted_index.in | 124, 195 |
| abstract_inverted_index.is | 95, 143 |
| abstract_inverted_index.it | 148 |
| abstract_inverted_index.of | 7, 20, 33, 60, 76, 121, 137, 161, 183, 199, 247 |
| abstract_inverted_index.on | 16 |
| abstract_inverted_index.or | 45 |
| abstract_inverted_index.to | 27, 68, 130, 189, 215, 255 |
| abstract_inverted_index.we | 153, 213 |
| abstract_inverted_index.808 | 169 |
| abstract_inverted_index.Log | 0 |
| abstract_inverted_index.and | 145, 167, 180, 205, 231 |
| abstract_inverted_index.are | 128 |
| abstract_inverted_index.for | 38, 49, 79, 224, 250 |
| abstract_inverted_index.go. | 69 |
| abstract_inverted_index.has | 64 |
| abstract_inverted_index.log | 22, 51, 81, 131, 162, 187, 207, 226, 258 |
| abstract_inverted_index.our | 151, 248 |
| abstract_inverted_index.the | 18, 30, 58, 74, 92, 122, 125, 196, 221, 245 |
| abstract_inverted_index.way | 67 |
| abstract_inverted_index.Both | 229 |
| abstract_inverted_index.Each | 193 |
| abstract_inverted_index.LLMs | 86, 218 |
| abstract_inverted_index.This | 70 |
| abstract_inverted_index.Thus | 91 |
| abstract_inverted_index.With | 29, 210 |
| abstract_inverted_index.both | 178 |
| abstract_inverted_index.code | 43, 170, 177 |
| abstract_inverted_index.data | 138, 163, 201 |
| abstract_inverted_index.each | 184 |
| abstract_inverted_index.from | 110, 158, 220 |
| abstract_inverted_index.have | 2, 14, 53, 114 |
| abstract_inverted_index.log. | 28 |
| abstract_inverted_index.long | 66, 115 |
| abstract_inverted_index.much | 55 |
| abstract_inverted_index.pair | 202 |
| abstract_inverted_index.part | 6 |
| abstract_inverted_index.such | 24, 41, 104 |
| abstract_inverted_index.that | 117, 235 |
| abstract_inverted_index.this | 135, 211 |
| abstract_inverted_index.with | 87, 107, 174 |
| abstract_inverted_index.1,537 | 159 |
| abstract_inverted_index.Large | 34 |
| abstract_inverted_index.Llama | 222 |
| abstract_inverted_index.Prior | 11 |
| abstract_inverted_index.after | 181 |
| abstract_inverted_index.entry | 194 |
| abstract_inverted_index.human | 230 |
| abstract_inverted_index.issue | 185 |
| abstract_inverted_index.novel | 97 |
| abstract_inverted_index.paper | 71 |
| abstract_inverted_index.still | 63 |
| abstract_inverted_index.tasks | 40 |
| abstract_inverted_index.these | 61, 236 |
| abstract_inverted_index.which | 101, 142 |
| abstract_inverted_index.(LLMs) | 37 |
| abstract_inverted_index.(i.e., | 172 |
| abstract_inverted_index.AUCAD, | 100 |
| abstract_inverted_index.Models | 36 |
| abstract_inverted_index.across | 164 |
| abstract_inverted_index.become | 3 |
| abstract_inverted_index.before | 179 |
| abstract_inverted_index.called | 99 |
| abstract_inverted_index.gained | 54 |
| abstract_inverted_index.issues | 123, 157 |
| abstract_inverted_index.manual | 140 |
| abstract_inverted_index.method | 249 |
| abstract_inverted_index.models | 237 |
| abstract_inverted_index.modern | 8 |
| abstract_inverted_index.source | 176 |
| abstract_inverted_index.costly, | 146 |
| abstract_inverted_index.dataset | 106, 197, 254 |
| abstract_inverted_index.efforts | 13 |
| abstract_inverted_index.enhance | 256 |
| abstract_inverted_index.entries | 160 |
| abstract_inverted_index.extract | 155 |
| abstract_inverted_index.focused | 15 |
| abstract_inverted_index.issues. | 112 |
| abstract_inverted_index.noticed | 116 |
| abstract_inverted_index.placing | 21 |
| abstract_inverted_index.portion | 120, 136 |
| abstract_inverted_index.primary | 93 |
| abstract_inverted_index.proceed | 214 |
| abstract_inverted_index.related | 129 |
| abstract_inverted_index.series) | 223 |
| abstract_inverted_index.thereby | 243 |
| abstract_inverted_index.However, | 57, 133 |
| abstract_inverted_index.Language | 35 |
| abstract_inverted_index.adoption | 32 |
| abstract_inverted_index.approach | 98 |
| abstract_inverted_index.consists | 198 |
| abstract_inverted_index.dataset, | 212 |
| abstract_inverted_index.dataset. | 90, 192 |
| abstract_inverted_index.efficacy | 246 |
| abstract_inverted_index.efforts, | 141 |
| abstract_inverted_index.existing | 240 |
| abstract_inverted_index.explores | 72 |
| abstract_inverted_index.identify | 168 |
| abstract_inverted_index.indicate | 234 |
| abstract_inverted_index.integral | 5 |
| abstract_inverted_index.methods) | 173 |
| abstract_inverted_index.multiple | 217 |
| abstract_inverted_index.projects | 166 |
| abstract_inverted_index.requires | 139 |
| abstract_inverted_index.research | 12 |
| abstract_inverted_index.snippets | 171 |
| abstract_inverted_index.software | 9 |
| abstract_inverted_index.systems. | 10 |
| abstract_inverted_index.LLM-based | 77, 241, 257 |
| abstract_inverted_index.Utilizing | 150 |
| abstract_inverted_index.approach, | 152 |
| abstract_inverted_index.automated | 47, 80, 225 |
| abstract_inverted_index.community | 127 |
| abstract_inverted_index.construct | 190 |
| abstract_inverted_index.decisions | 19 |
| abstract_inverted_index.enhancing | 73 |
| abstract_inverted_index.momentum. | 56 |
| abstract_inverted_index.rendering | 147 |
| abstract_inverted_index.solutions | 78 |
| abstract_inverted_index.statement | 82, 227, 259 |
| abstract_inverted_index.(including | 186 |
| abstract_inverted_index.(primarily | 219 |
| abstract_inverted_index.approaches | 48, 62 |
| abstract_inverted_index.completion | 44 |
| abstract_inverted_index.constructs | 103 |
| abstract_inverted_index.distilling | 134 |
| abstract_inverted_index.extracting | 109 |
| abstract_inverted_index.generating | 50 |
| abstract_inverted_index.generation | 83 |
| abstract_inverted_index.increasing | 31 |
| abstract_inverted_index.outperform | 239 |
| abstract_inverted_index.post-train | 216 |
| abstract_inverted_index.solutions, | 242 |
| abstract_inverted_index.statements | 1, 52 |
| abstract_inverted_index.supporting | 17 |
| abstract_inverted_index.validating | 244 |
| abstract_inverted_index.where/what | 26 |
| abstract_inverted_index.Researchers | 113 |
| abstract_inverted_index.evaluations | 233 |
| abstract_inverted_index.generation, | 46 |
| abstract_inverted_index.generation. | 228, 260 |
| abstract_inverted_index.information | 108 |
| abstract_inverted_index.log-related | 111, 156 |
| abstract_inverted_index.open-source | 126 |
| abstract_inverted_index.performance | 59, 75 |
| abstract_inverted_index.problematic | 206 |
| abstract_inverted_index.retrievable | 175 |
| abstract_inverted_index.significant | 119 |
| abstract_inverted_index.statements) | 188 |
| abstract_inverted_index.statements, | 23, 208 |
| abstract_inverted_index.statements. | 132 |
| abstract_inverted_index.code-related | 39 |
| abstract_inverted_index.constructing | 251 |
| abstract_inverted_index.contribution | 94 |
| abstract_inverted_index.experimental | 232 |
| abstract_inverted_index.high-quality | 204 |
| abstract_inverted_index.impractical. | 149 |
| abstract_inverted_index.modification | 182 |
| abstract_inverted_index.representing | 203 |
| abstract_inverted_index.automatically | 102, 154 |
| abstract_inverted_index.post-training | 85, 253 |
| abstract_inverted_index.purpose-built | 89 |
| abstract_inverted_index.respectively. | 209 |
| abstract_inverted_index.significantly | 238 |
| abstract_inverted_index.labor-intensive | 144 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 9 |
| citation_normalized_percentile |