TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models Article Swipe
Zorik Gekhman
,
Jonathan Herzig
,
Roee Aharoni
,
Chen Elkind
,
Idan Szpektor
·
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2305.11171
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2305.11171
Factual consistency evaluation is often conducted using Natural Language Inference (NLI) models, yet these models exhibit limited success in evaluating summaries. Previous work improved such models with synthetic training data. However, the data is typically based on perturbed human-written summaries, which often differ in their characteristics from real model-generated summaries and have limited coverage of possible factual errors. Alternatively, large language models (LLMs) have recently shown promising results in directly evaluating generative tasks, but are too computationally expensive for practical use. Motivated by these limitations, we introduce TrueTeacher, a method for generating synthetic data by annotating diverse model-generated summaries using a LLM. Unlike prior work, TrueTeacher does not rely on human-written summaries, and is multilingual by nature. Experiments on the TRUE benchmark show that a student model trained using our data, substantially outperforms both the state-of-the-art model with similar capacity, and the LLM teacher. In a systematic study, we compare TrueTeacher to existing synthetic data generation methods and demonstrate its superiority and robustness to domain-shift. We also show that our method generalizes to multilingual scenarios. Lastly, we release our large scale synthetic dataset (1.4M examples), generated using TrueTeacher, and a checkpoint trained on this data.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2305.11171
- https://arxiv.org/pdf/2305.11171
- OA Status
- green
- Cited By
- 3
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4377164430
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4377164430Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2305.11171Digital Object Identifier
- Title
-
TrueTeacher: Learning Factual Consistency Evaluation with Large Language ModelsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-05-18Full publication date if available
- Authors
-
Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, Idan SzpektorList of authors in order
- Landing page
-
https://arxiv.org/abs/2305.11171Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2305.11171Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2305.11171Direct OA link when available
- Concepts
-
Computer science, Robustness (evolution), Consistency (knowledge bases), Synthetic data, Inference, Artificial intelligence, Generative grammar, Language model, Benchmark (surveying), Machine learning, Generative model, Natural language processing, Chemistry, Gene, Biochemistry, Geography, GeodesyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
3Total citation count in OpenAlex
- Citations by year (recent)
-
2024: 2, 2023: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4377164430 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2305.11171 |
| ids.doi | https://doi.org/10.48550/arxiv.2305.11171 |
| ids.openalex | https://openalex.org/W4377164430 |
| fwci | |
| type | preprint |
| title | TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9998000264167786 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9979000091552734 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| topics[2].id | https://openalex.org/T10260 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9922999739646912 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1710 |
| topics[2].subfield.display_name | Information Systems |
| topics[2].display_name | Software Engineering Research |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.8215246200561523 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C63479239 |
| concepts[1].level | 3 |
| concepts[1].score | 0.695936918258667 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q7353546 |
| concepts[1].display_name | Robustness (evolution) |
| concepts[2].id | https://openalex.org/C2776436953 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6890193819999695 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q5163215 |
| concepts[2].display_name | Consistency (knowledge bases) |
| concepts[3].id | https://openalex.org/C160920958 |
| concepts[3].level | 2 |
| concepts[3].score | 0.629523515701294 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q7662746 |
| concepts[3].display_name | Synthetic data |
| concepts[4].id | https://openalex.org/C2776214188 |
| concepts[4].level | 2 |
| concepts[4].score | 0.6186560392379761 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q408386 |
| concepts[4].display_name | Inference |
| concepts[5].id | https://openalex.org/C154945302 |
| concepts[5].level | 1 |
| concepts[5].score | 0.5438836216926575 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[5].display_name | Artificial intelligence |
| concepts[6].id | https://openalex.org/C39890363 |
| concepts[6].level | 2 |
| concepts[6].score | 0.5434224605560303 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q36108 |
| concepts[6].display_name | Generative grammar |
| concepts[7].id | https://openalex.org/C137293760 |
| concepts[7].level | 2 |
| concepts[7].score | 0.5406931638717651 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q3621696 |
| concepts[7].display_name | Language model |
| concepts[8].id | https://openalex.org/C185798385 |
| concepts[8].level | 2 |
| concepts[8].score | 0.5354006290435791 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q1161707 |
| concepts[8].display_name | Benchmark (surveying) |
| concepts[9].id | https://openalex.org/C119857082 |
| concepts[9].level | 1 |
| concepts[9].score | 0.500401496887207 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[9].display_name | Machine learning |
| concepts[10].id | https://openalex.org/C167966045 |
| concepts[10].level | 3 |
| concepts[10].score | 0.41480696201324463 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q5532625 |
| concepts[10].display_name | Generative model |
| concepts[11].id | https://openalex.org/C204321447 |
| concepts[11].level | 1 |
| concepts[11].score | 0.38560330867767334 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[11].display_name | Natural language processing |
| concepts[12].id | https://openalex.org/C185592680 |
| concepts[12].level | 0 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q2329 |
| concepts[12].display_name | Chemistry |
| concepts[13].id | https://openalex.org/C104317684 |
| concepts[13].level | 2 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q7187 |
| concepts[13].display_name | Gene |
| concepts[14].id | https://openalex.org/C55493867 |
| concepts[14].level | 1 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q7094 |
| concepts[14].display_name | Biochemistry |
| concepts[15].id | https://openalex.org/C205649164 |
| concepts[15].level | 0 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q1071 |
| concepts[15].display_name | Geography |
| concepts[16].id | https://openalex.org/C13280743 |
| concepts[16].level | 1 |
| concepts[16].score | 0.0 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q131089 |
| concepts[16].display_name | Geodesy |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.8215246200561523 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/robustness |
| keywords[1].score | 0.695936918258667 |
| keywords[1].display_name | Robustness (evolution) |
| keywords[2].id | https://openalex.org/keywords/consistency |
| keywords[2].score | 0.6890193819999695 |
| keywords[2].display_name | Consistency (knowledge bases) |
| keywords[3].id | https://openalex.org/keywords/synthetic-data |
| keywords[3].score | 0.629523515701294 |
| keywords[3].display_name | Synthetic data |
| keywords[4].id | https://openalex.org/keywords/inference |
| keywords[4].score | 0.6186560392379761 |
| keywords[4].display_name | Inference |
| keywords[5].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[5].score | 0.5438836216926575 |
| keywords[5].display_name | Artificial intelligence |
| keywords[6].id | https://openalex.org/keywords/generative-grammar |
| keywords[6].score | 0.5434224605560303 |
| keywords[6].display_name | Generative grammar |
| keywords[7].id | https://openalex.org/keywords/language-model |
| keywords[7].score | 0.5406931638717651 |
| keywords[7].display_name | Language model |
| keywords[8].id | https://openalex.org/keywords/benchmark |
| keywords[8].score | 0.5354006290435791 |
| keywords[8].display_name | Benchmark (surveying) |
| keywords[9].id | https://openalex.org/keywords/machine-learning |
| keywords[9].score | 0.500401496887207 |
| keywords[9].display_name | Machine learning |
| keywords[10].id | https://openalex.org/keywords/generative-model |
| keywords[10].score | 0.41480696201324463 |
| keywords[10].display_name | Generative model |
| keywords[11].id | https://openalex.org/keywords/natural-language-processing |
| keywords[11].score | 0.38560330867767334 |
| keywords[11].display_name | Natural language processing |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2305.11171 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2305.11171 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2305.11171 |
| locations[1].id | doi:10.48550/arxiv.2305.11171 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2305.11171 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5087260801 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Zorik Gekhman |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Gekhman, Zorik |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5071893787 |
| authorships[1].author.orcid | https://orcid.org/0009-0000-7227-6557 |
| authorships[1].author.display_name | Jonathan Herzig |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Herzig, Jonathan |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5027223654 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Roee Aharoni |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Aharoni, Roee |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5091986043 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Chen Elkind |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Elkind, Chen |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5026091724 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Idan Szpektor |
| authorships[4].author_position | last |
| authorships[4].raw_author_name | Szpektor, Idan |
| authorships[4].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2305.11171 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2023-05-21T00:00:00 |
| display_name | TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9998000264167786 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| related_works | https://openalex.org/W4365211920, https://openalex.org/W3014948380, https://openalex.org/W4380551139, https://openalex.org/W4317695495, https://openalex.org/W2280377497, https://openalex.org/W4387506531, https://openalex.org/W4238433571, https://openalex.org/W3174044702, https://openalex.org/W2967848559, https://openalex.org/W4283803360 |
| cited_by_count | 3 |
| counts_by_year[0].year | 2024 |
| counts_by_year[0].cited_by_count | 2 |
| counts_by_year[1].year | 2023 |
| counts_by_year[1].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2305.11171 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2305.11171 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2305.11171 |
| primary_location.id | pmh:oai:arXiv.org:2305.11171 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2305.11171 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2305.11171 |
| publication_date | 2023-05-18 |
| publication_year | 2023 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 88, 100, 124, 145, 189 |
| abstract_inverted_index.In | 144 |
| abstract_inverted_index.We | 165 |
| abstract_inverted_index.by | 82, 94, 115 |
| abstract_inverted_index.in | 18, 43, 68 |
| abstract_inverted_index.is | 3, 33, 113 |
| abstract_inverted_index.of | 54 |
| abstract_inverted_index.on | 36, 109, 118, 192 |
| abstract_inverted_index.to | 151, 163, 172 |
| abstract_inverted_index.we | 85, 148, 176 |
| abstract_inverted_index.LLM | 142 |
| abstract_inverted_index.and | 50, 112, 140, 157, 161, 188 |
| abstract_inverted_index.are | 74 |
| abstract_inverted_index.but | 73 |
| abstract_inverted_index.for | 78, 90 |
| abstract_inverted_index.its | 159 |
| abstract_inverted_index.not | 107 |
| abstract_inverted_index.our | 129, 169, 178 |
| abstract_inverted_index.the | 31, 119, 134, 141 |
| abstract_inverted_index.too | 75 |
| abstract_inverted_index.yet | 12 |
| abstract_inverted_index.LLM. | 101 |
| abstract_inverted_index.TRUE | 120 |
| abstract_inverted_index.also | 166 |
| abstract_inverted_index.both | 133 |
| abstract_inverted_index.data | 32, 93, 154 |
| abstract_inverted_index.does | 106 |
| abstract_inverted_index.from | 46 |
| abstract_inverted_index.have | 51, 63 |
| abstract_inverted_index.real | 47 |
| abstract_inverted_index.rely | 108 |
| abstract_inverted_index.show | 122, 167 |
| abstract_inverted_index.such | 24 |
| abstract_inverted_index.that | 123, 168 |
| abstract_inverted_index.this | 193 |
| abstract_inverted_index.use. | 80 |
| abstract_inverted_index.with | 26, 137 |
| abstract_inverted_index.work | 22 |
| abstract_inverted_index.(1.4M | 183 |
| abstract_inverted_index.(NLI) | 10 |
| abstract_inverted_index.based | 35 |
| abstract_inverted_index.data, | 130 |
| abstract_inverted_index.data. | 29, 194 |
| abstract_inverted_index.large | 59, 179 |
| abstract_inverted_index.model | 126, 136 |
| abstract_inverted_index.often | 4, 41 |
| abstract_inverted_index.prior | 103 |
| abstract_inverted_index.scale | 180 |
| abstract_inverted_index.shown | 65 |
| abstract_inverted_index.their | 44 |
| abstract_inverted_index.these | 13, 83 |
| abstract_inverted_index.using | 6, 99, 128, 186 |
| abstract_inverted_index.which | 40 |
| abstract_inverted_index.work, | 104 |
| abstract_inverted_index.(LLMs) | 62 |
| abstract_inverted_index.Unlike | 102 |
| abstract_inverted_index.differ | 42 |
| abstract_inverted_index.method | 89, 170 |
| abstract_inverted_index.models | 14, 25, 61 |
| abstract_inverted_index.study, | 147 |
| abstract_inverted_index.tasks, | 72 |
| abstract_inverted_index.Factual | 0 |
| abstract_inverted_index.Lastly, | 175 |
| abstract_inverted_index.Natural | 7 |
| abstract_inverted_index.compare | 149 |
| abstract_inverted_index.dataset | 182 |
| abstract_inverted_index.diverse | 96 |
| abstract_inverted_index.errors. | 57 |
| abstract_inverted_index.exhibit | 15 |
| abstract_inverted_index.factual | 56 |
| abstract_inverted_index.limited | 16, 52 |
| abstract_inverted_index.methods | 156 |
| abstract_inverted_index.models, | 11 |
| abstract_inverted_index.nature. | 116 |
| abstract_inverted_index.release | 177 |
| abstract_inverted_index.results | 67 |
| abstract_inverted_index.similar | 138 |
| abstract_inverted_index.student | 125 |
| abstract_inverted_index.success | 17 |
| abstract_inverted_index.trained | 127, 191 |
| abstract_inverted_index.However, | 30 |
| abstract_inverted_index.Language | 8 |
| abstract_inverted_index.Previous | 21 |
| abstract_inverted_index.coverage | 53 |
| abstract_inverted_index.directly | 69 |
| abstract_inverted_index.existing | 152 |
| abstract_inverted_index.improved | 23 |
| abstract_inverted_index.language | 60 |
| abstract_inverted_index.possible | 55 |
| abstract_inverted_index.recently | 64 |
| abstract_inverted_index.teacher. | 143 |
| abstract_inverted_index.training | 28 |
| abstract_inverted_index.Inference | 9 |
| abstract_inverted_index.Motivated | 81 |
| abstract_inverted_index.benchmark | 121 |
| abstract_inverted_index.capacity, | 139 |
| abstract_inverted_index.conducted | 5 |
| abstract_inverted_index.expensive | 77 |
| abstract_inverted_index.generated | 185 |
| abstract_inverted_index.introduce | 86 |
| abstract_inverted_index.perturbed | 37 |
| abstract_inverted_index.practical | 79 |
| abstract_inverted_index.promising | 66 |
| abstract_inverted_index.summaries | 49, 98 |
| abstract_inverted_index.synthetic | 27, 92, 153, 181 |
| abstract_inverted_index.typically | 34 |
| abstract_inverted_index.annotating | 95 |
| abstract_inverted_index.checkpoint | 190 |
| abstract_inverted_index.evaluating | 19, 70 |
| abstract_inverted_index.evaluation | 2 |
| abstract_inverted_index.examples), | 184 |
| abstract_inverted_index.generating | 91 |
| abstract_inverted_index.generation | 155 |
| abstract_inverted_index.generative | 71 |
| abstract_inverted_index.robustness | 162 |
| abstract_inverted_index.scenarios. | 174 |
| abstract_inverted_index.summaries, | 39, 111 |
| abstract_inverted_index.summaries. | 20 |
| abstract_inverted_index.systematic | 146 |
| abstract_inverted_index.Experiments | 117 |
| abstract_inverted_index.TrueTeacher | 105, 150 |
| abstract_inverted_index.consistency | 1 |
| abstract_inverted_index.demonstrate | 158 |
| abstract_inverted_index.generalizes | 171 |
| abstract_inverted_index.outperforms | 132 |
| abstract_inverted_index.superiority | 160 |
| abstract_inverted_index.TrueTeacher, | 87, 187 |
| abstract_inverted_index.limitations, | 84 |
| abstract_inverted_index.multilingual | 114, 173 |
| abstract_inverted_index.domain-shift. | 164 |
| abstract_inverted_index.human-written | 38, 110 |
| abstract_inverted_index.substantially | 131 |
| abstract_inverted_index.Alternatively, | 58 |
| abstract_inverted_index.characteristics | 45 |
| abstract_inverted_index.computationally | 76 |
| abstract_inverted_index.model-generated | 48, 97 |
| abstract_inverted_index.state-of-the-art | 135 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 5 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.8500000238418579 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile |