Evaluating Generative Models for Graph-to-Text Generation Article Swipe
Shuzhou Yuan
,
Michael Färber
·
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.26615/978-954-452-092-2_133
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.26615/978-954-452-092-2_133
Large language models (LLMs) have been widely employed for graph-to-text generation tasks.However, the process of finetuning LLMs requires significant training resources and annotation work.In this paper, we explore the capability of generative models to generate descriptive text from graph data in a zeroshot setting.Specifically, we evaluate GPT-3 and ChatGPT on two graph-to-text datasets and compare their performance with that of finetuned LLM models such as T5 and BART.Our results demonstrate that generative models are capable of generating fluent and coherent text, achieving BLEU scores of 10.57 and 11.08 for the AGENDA and WebNLG datasets, respectively.However, our error analysis reveals that generative models still struggle with understanding the semantic relations between entities, and they also tend to generate text with hallucinations or irrelevant information.As a part of error analysis, we utilize BERT to detect machine-generated text and achieve high macro-F1 scores.We have made the text generated by generative models publicly available.
Related Topics
Concepts
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://doi.org/10.26615/978-954-452-092-2_133
- https://doi.org/10.26615/978-954-452-092-2_133
- OA Status
- gold
- Cited By
- 4
- References
- 34
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4390682215
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4390682215Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.26615/978-954-452-092-2_133Digital Object Identifier
- Title
-
Evaluating Generative Models for Graph-to-Text GenerationWork title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-01-01Full publication date if available
- Authors
-
Shuzhou Yuan, Michael FärberList of authors in order
- Landing page
-
https://doi.org/10.26615/978-954-452-092-2_133Publisher landing page
- PDF URL
-
https://doi.org/10.26615/978-954-452-092-2_133Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
goldOpen access status per OpenAlex
- OA URL
-
https://doi.org/10.26615/978-954-452-092-2_133Direct OA link when available
- Concepts
-
Computer science, Generative grammar, Text generation, Generative model, Natural language processing, Graph, Artificial intelligence, Macro, Annotation, Information retrieval, Machine learning, Theoretical computer science, Programming languageTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
4Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 2, 2024: 2Per-year citation counts (last 5 years)
- References (count)
-
34Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4390682215 |
|---|---|
| doi | https://doi.org/10.26615/978-954-452-092-2_133 |
| ids.doi | https://doi.org/10.26615/978-954-452-092-2_133 |
| ids.openalex | https://openalex.org/W4390682215 |
| fwci | 1.02177155 |
| type | article |
| title | Evaluating Generative Models for Graph-to-Text Generation |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | 1264 |
| biblio.first_page | 1256 |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9998999834060669 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T11273 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9923999905586243 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Advanced Graph Neural Networks |
| topics[2].id | https://openalex.org/T10181 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9912999868392944 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Natural Language Processing Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.8295820951461792 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C39890363 |
| concepts[1].level | 2 |
| concepts[1].score | 0.8217658996582031 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q36108 |
| concepts[1].display_name | Generative grammar |
| concepts[2].id | https://openalex.org/C2985684807 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6867835521697998 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q1513879 |
| concepts[2].display_name | Text generation |
| concepts[3].id | https://openalex.org/C167966045 |
| concepts[3].level | 3 |
| concepts[3].score | 0.6210788488388062 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q5532625 |
| concepts[3].display_name | Generative model |
| concepts[4].id | https://openalex.org/C204321447 |
| concepts[4].level | 1 |
| concepts[4].score | 0.6202152967453003 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[4].display_name | Natural language processing |
| concepts[5].id | https://openalex.org/C132525143 |
| concepts[5].level | 2 |
| concepts[5].score | 0.6176518201828003 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q141488 |
| concepts[5].display_name | Graph |
| concepts[6].id | https://openalex.org/C154945302 |
| concepts[6].level | 1 |
| concepts[6].score | 0.6054823994636536 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[6].display_name | Artificial intelligence |
| concepts[7].id | https://openalex.org/C166955791 |
| concepts[7].level | 2 |
| concepts[7].score | 0.5136120319366455 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q629579 |
| concepts[7].display_name | Macro |
| concepts[8].id | https://openalex.org/C2776321320 |
| concepts[8].level | 2 |
| concepts[8].score | 0.4221782088279724 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q857525 |
| concepts[8].display_name | Annotation |
| concepts[9].id | https://openalex.org/C23123220 |
| concepts[9].level | 1 |
| concepts[9].score | 0.39102548360824585 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q816826 |
| concepts[9].display_name | Information retrieval |
| concepts[10].id | https://openalex.org/C119857082 |
| concepts[10].level | 1 |
| concepts[10].score | 0.3403175473213196 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[10].display_name | Machine learning |
| concepts[11].id | https://openalex.org/C80444323 |
| concepts[11].level | 1 |
| concepts[11].score | 0.19712835550308228 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q2878974 |
| concepts[11].display_name | Theoretical computer science |
| concepts[12].id | https://openalex.org/C199360897 |
| concepts[12].level | 1 |
| concepts[12].score | 0.10711762309074402 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[12].display_name | Programming language |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.8295820951461792 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/generative-grammar |
| keywords[1].score | 0.8217658996582031 |
| keywords[1].display_name | Generative grammar |
| keywords[2].id | https://openalex.org/keywords/text-generation |
| keywords[2].score | 0.6867835521697998 |
| keywords[2].display_name | Text generation |
| keywords[3].id | https://openalex.org/keywords/generative-model |
| keywords[3].score | 0.6210788488388062 |
| keywords[3].display_name | Generative model |
| keywords[4].id | https://openalex.org/keywords/natural-language-processing |
| keywords[4].score | 0.6202152967453003 |
| keywords[4].display_name | Natural language processing |
| keywords[5].id | https://openalex.org/keywords/graph |
| keywords[5].score | 0.6176518201828003 |
| keywords[5].display_name | Graph |
| keywords[6].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[6].score | 0.6054823994636536 |
| keywords[6].display_name | Artificial intelligence |
| keywords[7].id | https://openalex.org/keywords/macro |
| keywords[7].score | 0.5136120319366455 |
| keywords[7].display_name | Macro |
| keywords[8].id | https://openalex.org/keywords/annotation |
| keywords[8].score | 0.4221782088279724 |
| keywords[8].display_name | Annotation |
| keywords[9].id | https://openalex.org/keywords/information-retrieval |
| keywords[9].score | 0.39102548360824585 |
| keywords[9].display_name | Information retrieval |
| keywords[10].id | https://openalex.org/keywords/machine-learning |
| keywords[10].score | 0.3403175473213196 |
| keywords[10].display_name | Machine learning |
| keywords[11].id | https://openalex.org/keywords/theoretical-computer-science |
| keywords[11].score | 0.19712835550308228 |
| keywords[11].display_name | Theoretical computer science |
| keywords[12].id | https://openalex.org/keywords/programming-language |
| keywords[12].score | 0.10711762309074402 |
| keywords[12].display_name | Programming language |
| language | en |
| locations[0].id | doi:10.26615/978-954-452-092-2_133 |
| locations[0].is_oa | True |
| locations[0].source | |
| locations[0].license | |
| locations[0].pdf_url | https://doi.org/10.26615/978-954-452-092-2_133 |
| locations[0].version | publishedVersion |
| locations[0].raw_type | proceedings-article |
| locations[0].license_id | |
| locations[0].is_accepted | True |
| locations[0].is_published | True |
| locations[0].raw_source_name | Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Processings |
| locations[0].landing_page_url | https://doi.org/10.26615/978-954-452-092-2_133 |
| indexed_in | crossref |
| authorships[0].author.id | https://openalex.org/A5008824325 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Shuzhou Yuan |
| authorships[0].countries | DE |
| authorships[0].affiliations[0].institution_ids | https://openalex.org/I102335020 |
| authorships[0].affiliations[0].raw_affiliation_string | Karlsruhe Institute of Technology (KIT) |
| authorships[0].institutions[0].id | https://openalex.org/I102335020 |
| authorships[0].institutions[0].ror | https://ror.org/04t3en479 |
| authorships[0].institutions[0].type | education |
| authorships[0].institutions[0].lineage | https://openalex.org/I102335020, https://openalex.org/I1305996414 |
| authorships[0].institutions[0].country_code | DE |
| authorships[0].institutions[0].display_name | Karlsruhe Institute of Technology |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Shuzhou Yuan |
| authorships[0].is_corresponding | False |
| authorships[0].raw_affiliation_strings | Karlsruhe Institute of Technology (KIT) |
| authorships[1].author.id | https://openalex.org/A5031600582 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-5458-8645 |
| authorships[1].author.display_name | Michael Färber |
| authorships[1].countries | DE |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I102335020 |
| authorships[1].affiliations[0].raw_affiliation_string | Karlsruhe Institute of Technology (KIT) |
| authorships[1].institutions[0].id | https://openalex.org/I102335020 |
| authorships[1].institutions[0].ror | https://ror.org/04t3en479 |
| authorships[1].institutions[0].type | education |
| authorships[1].institutions[0].lineage | https://openalex.org/I102335020, https://openalex.org/I1305996414 |
| authorships[1].institutions[0].country_code | DE |
| authorships[1].institutions[0].display_name | Karlsruhe Institute of Technology |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Michael Färber |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | Karlsruhe Institute of Technology (KIT) |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://doi.org/10.26615/978-954-452-092-2_133 |
| open_access.oa_status | gold |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-01-13T00:00:00 |
| display_name | Evaluating Generative Models for Graph-to-Text Generation |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9998999834060669 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| related_works | https://openalex.org/W4365211920, https://openalex.org/W3014948380, https://openalex.org/W4380551139, https://openalex.org/W4317695495, https://openalex.org/W4287117424, https://openalex.org/W4387506531, https://openalex.org/W2087346071, https://openalex.org/W2967848559, https://openalex.org/W4299831724, https://openalex.org/W4283803360 |
| cited_by_count | 4 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 2 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 2 |
| locations_count | 1 |
| best_oa_location.id | doi:10.26615/978-954-452-092-2_133 |
| best_oa_location.is_oa | True |
| best_oa_location.source | |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://doi.org/10.26615/978-954-452-092-2_133 |
| best_oa_location.version | publishedVersion |
| best_oa_location.raw_type | proceedings-article |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | True |
| best_oa_location.is_published | True |
| best_oa_location.raw_source_name | Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Processings |
| best_oa_location.landing_page_url | https://doi.org/10.26615/978-954-452-092-2_133 |
| primary_location.id | doi:10.26615/978-954-452-092-2_133 |
| primary_location.is_oa | True |
| primary_location.source | |
| primary_location.license | |
| primary_location.pdf_url | https://doi.org/10.26615/978-954-452-092-2_133 |
| primary_location.version | publishedVersion |
| primary_location.raw_type | proceedings-article |
| primary_location.license_id | |
| primary_location.is_accepted | True |
| primary_location.is_published | True |
| primary_location.raw_source_name | Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Processings |
| primary_location.landing_page_url | https://doi.org/10.26615/978-954-452-092-2_133 |
| publication_date | 2023-01-01 |
| publication_year | 2023 |
| referenced_works | https://openalex.org/W3214637961, https://openalex.org/W3088227725, https://openalex.org/W3115328016, https://openalex.org/W4389524085, https://openalex.org/W2931198394, https://openalex.org/W2963718112, https://openalex.org/W4319793302, https://openalex.org/W4292779060, https://openalex.org/W2758950307, https://openalex.org/W3035252911, https://openalex.org/W4385245566, https://openalex.org/W2908510526, https://openalex.org/W2786660442, https://openalex.org/W3034987089, https://openalex.org/W2154652894, https://openalex.org/W2963341956, https://openalex.org/W2740107682, https://openalex.org/W4285174063, https://openalex.org/W4226278401, https://openalex.org/W2962950136, https://openalex.org/W3034080136, https://openalex.org/W2964116568, https://openalex.org/W2798749466, https://openalex.org/W2924961378, https://openalex.org/W4389524534, https://openalex.org/W4229543565, https://openalex.org/W2101105183, https://openalex.org/W2123301721, https://openalex.org/W2604799547, https://openalex.org/W4320167623, https://openalex.org/W102708294, https://openalex.org/W2971187756, https://openalex.org/W2963374482, https://openalex.org/W3177423701 |
| referenced_works_count | 34 |
| abstract_inverted_index.a | 41, 123 |
| abstract_inverted_index.T5 | 65 |
| abstract_inverted_index.as | 64 |
| abstract_inverted_index.by | 145 |
| abstract_inverted_index.in | 40 |
| abstract_inverted_index.of | 14, 30, 59, 75, 84, 125 |
| abstract_inverted_index.on | 49 |
| abstract_inverted_index.or | 120 |
| abstract_inverted_index.to | 33, 115, 131 |
| abstract_inverted_index.we | 26, 44, 128 |
| abstract_inverted_index.LLM | 61 |
| abstract_inverted_index.and | 21, 47, 53, 66, 78, 86, 91, 111, 135 |
| abstract_inverted_index.are | 73 |
| abstract_inverted_index.for | 8, 88 |
| abstract_inverted_index.our | 95 |
| abstract_inverted_index.the | 12, 28, 89, 106, 142 |
| abstract_inverted_index.two | 50 |
| abstract_inverted_index.BERT | 130 |
| abstract_inverted_index.BLEU | 82 |
| abstract_inverted_index.LLMs | 16 |
| abstract_inverted_index.also | 113 |
| abstract_inverted_index.been | 5 |
| abstract_inverted_index.data | 39 |
| abstract_inverted_index.from | 37 |
| abstract_inverted_index.have | 4, 140 |
| abstract_inverted_index.high | 137 |
| abstract_inverted_index.made | 141 |
| abstract_inverted_index.part | 124 |
| abstract_inverted_index.such | 63 |
| abstract_inverted_index.tend | 114 |
| abstract_inverted_index.text | 36, 117, 134, 143 |
| abstract_inverted_index.that | 58, 70, 99 |
| abstract_inverted_index.they | 112 |
| abstract_inverted_index.this | 24 |
| abstract_inverted_index.with | 57, 104, 118 |
| abstract_inverted_index.10.57 | 85 |
| abstract_inverted_index.11.08 | 87 |
| abstract_inverted_index.GPT-3 | 46 |
| abstract_inverted_index.Large | 0 |
| abstract_inverted_index.error | 96, 126 |
| abstract_inverted_index.graph | 38 |
| abstract_inverted_index.still | 102 |
| abstract_inverted_index.text, | 80 |
| abstract_inverted_index.their | 55 |
| abstract_inverted_index.(LLMs) | 3 |
| abstract_inverted_index.AGENDA | 90 |
| abstract_inverted_index.WebNLG | 92 |
| abstract_inverted_index.detect | 132 |
| abstract_inverted_index.fluent | 77 |
| abstract_inverted_index.models | 2, 32, 62, 72, 101, 147 |
| abstract_inverted_index.paper, | 25 |
| abstract_inverted_index.scores | 83 |
| abstract_inverted_index.widely | 6 |
| abstract_inverted_index.ChatGPT | 48 |
| abstract_inverted_index.achieve | 136 |
| abstract_inverted_index.between | 109 |
| abstract_inverted_index.capable | 74 |
| abstract_inverted_index.compare | 54 |
| abstract_inverted_index.explore | 27 |
| abstract_inverted_index.process | 13 |
| abstract_inverted_index.results | 68 |
| abstract_inverted_index.reveals | 98 |
| abstract_inverted_index.utilize | 129 |
| abstract_inverted_index.work.In | 23 |
| abstract_inverted_index.BART.Our | 67 |
| abstract_inverted_index.analysis | 97 |
| abstract_inverted_index.coherent | 79 |
| abstract_inverted_index.datasets | 52 |
| abstract_inverted_index.employed | 7 |
| abstract_inverted_index.evaluate | 45 |
| abstract_inverted_index.generate | 34, 116 |
| abstract_inverted_index.language | 1 |
| abstract_inverted_index.macro-F1 | 138 |
| abstract_inverted_index.publicly | 148 |
| abstract_inverted_index.requires | 17 |
| abstract_inverted_index.semantic | 107 |
| abstract_inverted_index.struggle | 103 |
| abstract_inverted_index.training | 19 |
| abstract_inverted_index.zeroshot | 42 |
| abstract_inverted_index.achieving | 81 |
| abstract_inverted_index.analysis, | 127 |
| abstract_inverted_index.datasets, | 93 |
| abstract_inverted_index.entities, | 110 |
| abstract_inverted_index.finetuned | 60 |
| abstract_inverted_index.generated | 144 |
| abstract_inverted_index.relations | 108 |
| abstract_inverted_index.resources | 20 |
| abstract_inverted_index.scores.We | 139 |
| abstract_inverted_index.annotation | 22 |
| abstract_inverted_index.available. | 149 |
| abstract_inverted_index.capability | 29 |
| abstract_inverted_index.finetuning | 15 |
| abstract_inverted_index.generating | 76 |
| abstract_inverted_index.generation | 10 |
| abstract_inverted_index.generative | 31, 71, 100, 146 |
| abstract_inverted_index.irrelevant | 121 |
| abstract_inverted_index.demonstrate | 69 |
| abstract_inverted_index.descriptive | 35 |
| abstract_inverted_index.performance | 56 |
| abstract_inverted_index.significant | 18 |
| abstract_inverted_index.graph-to-text | 9, 51 |
| abstract_inverted_index.understanding | 105 |
| abstract_inverted_index.hallucinations | 119 |
| abstract_inverted_index.information.As | 122 |
| abstract_inverted_index.tasks.However, | 11 |
| abstract_inverted_index.machine-generated | 133 |
| abstract_inverted_index.respectively.However, | 94 |
| abstract_inverted_index.setting.Specifically, | 43 |
| cited_by_percentile_year.max | 97 |
| cited_by_percentile_year.min | 94 |
| countries_distinct_count | 1 |
| institutions_distinct_count | 2 |
| citation_normalized_percentile.value | 0.79373721 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |