Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models Article Swipe
Bernd Bohnet
,
Vinh Q. Tran
,
Pat Verga
,
Roee Aharoni
,
Daniel Andor
,
Livio Baldini Soares
,
Jacob Eisenstein
,
Kuzman Ganchev
,
Jonathan Herzig
,
Kai Hui
,
Tom Kwiatkowski
,
Ji Ma
,
Jianmo Ni
,
Tal Schuster
,
William W. Cohen
,
Michael J. Collins
,
Dipanjan Das
,
Donald Metzler
,
Slav Petrov
,
Kellie Webster
·
YOU?
·
· 2022
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2212.08037
YOU?
·
· 2022
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2212.08037
Large language models (LLMs) have shown impressive results while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial in this setting. We formulate and study Attributed QA as a key first step in the development of attributed LLMs. We propose a reproducible evaluation framework for the task and benchmark a broad set of architectures. We take human annotations as a gold standard and show that a correlated automatic metric is suitable for development. Our experimental work gives concrete answers to two key questions (How to measure attribution?, and How well do current state-of-the-art methods perform on attribution?), and give some hints as to how to address a third (How to build LLMs with attribution?).
Related Topics
Concepts
Attribution
Benchmark (surveying)
Metric (unit)
Key (lock)
Set (abstract data type)
Task (project management)
Computer science
Ranking (information retrieval)
Natural language processing
Artificial intelligence
Psychology
Social psychology
Computer security
Programming language
Engineering
Operations management
Geography
Geodesy
Systems engineering
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2212.08037
- https://arxiv.org/pdf/2212.08037
- OA Status
- green
- Cited By
- 25
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4311731003
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4311731003Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2212.08037Digital Object Identifier
- Title
-
Attributed Question Answering: Evaluation and Modeling for Attributed Large Language ModelsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2022Year of publication
- Publication date
-
2022-12-15Full publication date if available
- Authors
-
Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, Tom Kwiatkowski, Ji Ma, Jianmo Ni, Tal Schuster, William W. Cohen, Michael J. Collins, Dipanjan Das, Donald Metzler, Slav Petrov, Kellie WebsterList of authors in order
- Landing page
-
https://arxiv.org/abs/2212.08037Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2212.08037Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2212.08037Direct OA link when available
- Concepts
-
Attribution, Benchmark (surveying), Metric (unit), Key (lock), Set (abstract data type), Task (project management), Computer science, Ranking (information retrieval), Natural language processing, Artificial intelligence, Psychology, Social psychology, Computer security, Programming language, Engineering, Operations management, Geography, Geodesy, Systems engineeringTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
25Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 3, 2024: 6, 2023: 16Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4311731003 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2212.08037 |
| ids.doi | https://doi.org/10.48550/arxiv.2212.08037 |
| ids.openalex | https://openalex.org/W4311731003 |
| fwci | |
| type | preprint |
| title | Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9998000264167786 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.998199999332428 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| topics[2].id | https://openalex.org/T12026 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.930899977684021 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Explainable Artificial Intelligence (XAI) |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C143299363 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7628248929977417 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q900584 |
| concepts[0].display_name | Attribution |
| concepts[1].id | https://openalex.org/C185798385 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6968781352043152 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q1161707 |
| concepts[1].display_name | Benchmark (surveying) |
| concepts[2].id | https://openalex.org/C176217482 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6396690011024475 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q860554 |
| concepts[2].display_name | Metric (unit) |
| concepts[3].id | https://openalex.org/C26517878 |
| concepts[3].level | 2 |
| concepts[3].score | 0.6241942048072815 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q228039 |
| concepts[3].display_name | Key (lock) |
| concepts[4].id | https://openalex.org/C177264268 |
| concepts[4].level | 2 |
| concepts[4].score | 0.6239905953407288 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q1514741 |
| concepts[4].display_name | Set (abstract data type) |
| concepts[5].id | https://openalex.org/C2780451532 |
| concepts[5].level | 2 |
| concepts[5].score | 0.608769416809082 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q759676 |
| concepts[5].display_name | Task (project management) |
| concepts[6].id | https://openalex.org/C41008148 |
| concepts[6].level | 0 |
| concepts[6].score | 0.5437475442886353 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[6].display_name | Computer science |
| concepts[7].id | https://openalex.org/C189430467 |
| concepts[7].level | 2 |
| concepts[7].score | 0.44375860691070557 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q7293293 |
| concepts[7].display_name | Ranking (information retrieval) |
| concepts[8].id | https://openalex.org/C204321447 |
| concepts[8].level | 1 |
| concepts[8].score | 0.34778475761413574 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[8].display_name | Natural language processing |
| concepts[9].id | https://openalex.org/C154945302 |
| concepts[9].level | 1 |
| concepts[9].score | 0.3163989782333374 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[9].display_name | Artificial intelligence |
| concepts[10].id | https://openalex.org/C15744967 |
| concepts[10].level | 0 |
| concepts[10].score | 0.26360830664634705 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[10].display_name | Psychology |
| concepts[11].id | https://openalex.org/C77805123 |
| concepts[11].level | 1 |
| concepts[11].score | 0.21920916438102722 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q161272 |
| concepts[11].display_name | Social psychology |
| concepts[12].id | https://openalex.org/C38652104 |
| concepts[12].level | 1 |
| concepts[12].score | 0.17846819758415222 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q3510521 |
| concepts[12].display_name | Computer security |
| concepts[13].id | https://openalex.org/C199360897 |
| concepts[13].level | 1 |
| concepts[13].score | 0.1059102714061737 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[13].display_name | Programming language |
| concepts[14].id | https://openalex.org/C127413603 |
| concepts[14].level | 0 |
| concepts[14].score | 0.10209959745407104 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[14].display_name | Engineering |
| concepts[15].id | https://openalex.org/C21547014 |
| concepts[15].level | 1 |
| concepts[15].score | 0.08341583609580994 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q1423657 |
| concepts[15].display_name | Operations management |
| concepts[16].id | https://openalex.org/C205649164 |
| concepts[16].level | 0 |
| concepts[16].score | 0.07964572310447693 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q1071 |
| concepts[16].display_name | Geography |
| concepts[17].id | https://openalex.org/C13280743 |
| concepts[17].level | 1 |
| concepts[17].score | 0.0 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q131089 |
| concepts[17].display_name | Geodesy |
| concepts[18].id | https://openalex.org/C201995342 |
| concepts[18].level | 1 |
| concepts[18].score | 0.0 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q682496 |
| concepts[18].display_name | Systems engineering |
| keywords[0].id | https://openalex.org/keywords/attribution |
| keywords[0].score | 0.7628248929977417 |
| keywords[0].display_name | Attribution |
| keywords[1].id | https://openalex.org/keywords/benchmark |
| keywords[1].score | 0.6968781352043152 |
| keywords[1].display_name | Benchmark (surveying) |
| keywords[2].id | https://openalex.org/keywords/metric |
| keywords[2].score | 0.6396690011024475 |
| keywords[2].display_name | Metric (unit) |
| keywords[3].id | https://openalex.org/keywords/key |
| keywords[3].score | 0.6241942048072815 |
| keywords[3].display_name | Key (lock) |
| keywords[4].id | https://openalex.org/keywords/set |
| keywords[4].score | 0.6239905953407288 |
| keywords[4].display_name | Set (abstract data type) |
| keywords[5].id | https://openalex.org/keywords/task |
| keywords[5].score | 0.608769416809082 |
| keywords[5].display_name | Task (project management) |
| keywords[6].id | https://openalex.org/keywords/computer-science |
| keywords[6].score | 0.5437475442886353 |
| keywords[6].display_name | Computer science |
| keywords[7].id | https://openalex.org/keywords/ranking |
| keywords[7].score | 0.44375860691070557 |
| keywords[7].display_name | Ranking (information retrieval) |
| keywords[8].id | https://openalex.org/keywords/natural-language-processing |
| keywords[8].score | 0.34778475761413574 |
| keywords[8].display_name | Natural language processing |
| keywords[9].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[9].score | 0.3163989782333374 |
| keywords[9].display_name | Artificial intelligence |
| keywords[10].id | https://openalex.org/keywords/psychology |
| keywords[10].score | 0.26360830664634705 |
| keywords[10].display_name | Psychology |
| keywords[11].id | https://openalex.org/keywords/social-psychology |
| keywords[11].score | 0.21920916438102722 |
| keywords[11].display_name | Social psychology |
| keywords[12].id | https://openalex.org/keywords/computer-security |
| keywords[12].score | 0.17846819758415222 |
| keywords[12].display_name | Computer security |
| keywords[13].id | https://openalex.org/keywords/programming-language |
| keywords[13].score | 0.1059102714061737 |
| keywords[13].display_name | Programming language |
| keywords[14].id | https://openalex.org/keywords/engineering |
| keywords[14].score | 0.10209959745407104 |
| keywords[14].display_name | Engineering |
| keywords[15].id | https://openalex.org/keywords/operations-management |
| keywords[15].score | 0.08341583609580994 |
| keywords[15].display_name | Operations management |
| keywords[16].id | https://openalex.org/keywords/geography |
| keywords[16].score | 0.07964572310447693 |
| keywords[16].display_name | Geography |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2212.08037 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2212.08037 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2212.08037 |
| locations[1].id | doi:10.48550/arxiv.2212.08037 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2212.08037 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5066427994 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Bernd Bohnet |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Bohnet, Bernd |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5102435043 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Vinh Q. Tran |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Tran, Vinh Q. |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5063157850 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Pat Verga |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Verga, Pat |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5027223654 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Roee Aharoni |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Aharoni, Roee |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5026230181 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Daniel Andor |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Andor, Daniel |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5076699933 |
| authorships[5].author.orcid | |
| authorships[5].author.display_name | Livio Baldini Soares |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Soares, Livio Baldini |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5047699861 |
| authorships[6].author.orcid | |
| authorships[6].author.display_name | Jacob Eisenstein |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Eisenstein, Jacob |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5081573359 |
| authorships[7].author.orcid | |
| authorships[7].author.display_name | Kuzman Ganchev |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Ganchev, Kuzman |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5071893787 |
| authorships[8].author.orcid | https://orcid.org/0009-0000-7227-6557 |
| authorships[8].author.display_name | Jonathan Herzig |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Herzig, Jonathan |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5101182015 |
| authorships[9].author.orcid | |
| authorships[9].author.display_name | Kai Hui |
| authorships[9].author_position | middle |
| authorships[9].raw_author_name | Hui, Kai |
| authorships[9].is_corresponding | False |
| authorships[10].author.id | https://openalex.org/A5011150187 |
| authorships[10].author.orcid | |
| authorships[10].author.display_name | Tom Kwiatkowski |
| authorships[10].author_position | middle |
| authorships[10].raw_author_name | Kwiatkowski, Tom |
| authorships[10].is_corresponding | False |
| authorships[11].author.id | https://openalex.org/A5100460563 |
| authorships[11].author.orcid | https://orcid.org/0000-0003-2181-5121 |
| authorships[11].author.display_name | Ji Ma |
| authorships[11].author_position | middle |
| authorships[11].raw_author_name | Ma, Ji |
| authorships[11].is_corresponding | False |
| authorships[12].author.id | https://openalex.org/A5077817759 |
| authorships[12].author.orcid | https://orcid.org/0000-0002-6863-8073 |
| authorships[12].author.display_name | Jianmo Ni |
| authorships[12].author_position | middle |
| authorships[12].raw_author_name | Ni, Jianmo |
| authorships[12].is_corresponding | False |
| authorships[13].author.id | https://openalex.org/A5058037733 |
| authorships[13].author.orcid | https://orcid.org/0000-0002-7772-8230 |
| authorships[13].author.display_name | Tal Schuster |
| authorships[13].author_position | middle |
| authorships[13].raw_author_name | Schuster, Tal |
| authorships[13].is_corresponding | False |
| authorships[14].author.id | https://openalex.org/A5051617344 |
| authorships[14].author.orcid | |
| authorships[14].author.display_name | William W. Cohen |
| authorships[14].author_position | middle |
| authorships[14].raw_author_name | Cohen, William W. |
| authorships[14].is_corresponding | False |
| authorships[15].author.id | https://openalex.org/A5074839699 |
| authorships[15].author.orcid | https://orcid.org/0000-0001-5226-5498 |
| authorships[15].author.display_name | Michael J. Collins |
| authorships[15].author_position | middle |
| authorships[15].raw_author_name | Collins, Michael |
| authorships[15].is_corresponding | False |
| authorships[16].author.id | https://openalex.org/A5021055773 |
| authorships[16].author.orcid | https://orcid.org/0000-0003-2325-0646 |
| authorships[16].author.display_name | Dipanjan Das |
| authorships[16].author_position | middle |
| authorships[16].raw_author_name | Das, Dipanjan |
| authorships[16].is_corresponding | False |
| authorships[17].author.id | https://openalex.org/A5000115067 |
| authorships[17].author.orcid | https://orcid.org/0000-0003-4276-6269 |
| authorships[17].author.display_name | Donald Metzler |
| authorships[17].author_position | middle |
| authorships[17].raw_author_name | Metzler, Donald |
| authorships[17].is_corresponding | False |
| authorships[18].author.id | https://openalex.org/A5062886906 |
| authorships[18].author.orcid | |
| authorships[18].author.display_name | Slav Petrov |
| authorships[18].author_position | middle |
| authorships[18].raw_author_name | Petrov, Slav |
| authorships[18].is_corresponding | False |
| authorships[19].author.id | https://openalex.org/A5088842522 |
| authorships[19].author.orcid | |
| authorships[19].author.display_name | Kellie Webster |
| authorships[19].author_position | last |
| authorships[19].raw_author_name | Webster, Kellie |
| authorships[19].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2212.08037 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2022-12-28T00:00:00 |
| display_name | Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9998000264167786 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| related_works | https://openalex.org/W2035546108, https://openalex.org/W2378211422, https://openalex.org/W2376361520, https://openalex.org/W2133328864, https://openalex.org/W1482441085, https://openalex.org/W2966858528, https://openalex.org/W51364034, https://openalex.org/W2151687600, https://openalex.org/W2898073868, https://openalex.org/W4284663758 |
| cited_by_count | 25 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 3 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 6 |
| counts_by_year[2].year | 2023 |
| counts_by_year[2].cited_by_count | 16 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2212.08037 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2212.08037 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2212.08037 |
| primary_location.id | pmh:oai:arXiv.org:2212.08037 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2212.08037 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2212.08037 |
| publication_date | 2022-12-15 |
| publication_year | 2022 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 57, 69, 78, 88, 94, 135 |
| abstract_inverted_index.QA | 55 |
| abstract_inverted_index.We | 28, 50, 67, 83 |
| abstract_inverted_index.an | 33 |
| abstract_inverted_index.as | 56, 87, 130 |
| abstract_inverted_index.be | 45 |
| abstract_inverted_index.do | 119 |
| abstract_inverted_index.in | 25, 47, 61 |
| abstract_inverted_index.is | 17, 42, 98 |
| abstract_inverted_index.it | 40 |
| abstract_inverted_index.no | 12 |
| abstract_inverted_index.of | 32, 64, 81 |
| abstract_inverted_index.on | 124 |
| abstract_inverted_index.or | 11 |
| abstract_inverted_index.to | 35, 44, 108, 113, 131, 133, 138 |
| abstract_inverted_index.How | 117 |
| abstract_inverted_index.LLM | 34 |
| abstract_inverted_index.Our | 102 |
| abstract_inverted_index.and | 52, 76, 91, 116, 126 |
| abstract_inverted_index.for | 73, 100 |
| abstract_inverted_index.how | 132 |
| abstract_inverted_index.key | 58, 110 |
| abstract_inverted_index.may | 22 |
| abstract_inverted_index.set | 80 |
| abstract_inverted_index.the | 30, 37, 62, 74 |
| abstract_inverted_index.two | 109 |
| abstract_inverted_index.(How | 112, 137 |
| abstract_inverted_index.LLMs | 21, 140 |
| abstract_inverted_index.give | 127 |
| abstract_inverted_index.gold | 89 |
| abstract_inverted_index.have | 4, 23 |
| abstract_inverted_index.show | 92 |
| abstract_inverted_index.some | 128 |
| abstract_inverted_index.step | 60 |
| abstract_inverted_index.take | 84 |
| abstract_inverted_index.task | 75 |
| abstract_inverted_index.text | 38 |
| abstract_inverted_index.that | 20, 39, 93 |
| abstract_inverted_index.this | 48 |
| abstract_inverted_index.well | 118 |
| abstract_inverted_index.with | 141 |
| abstract_inverted_index.work | 104 |
| abstract_inverted_index.LLMs. | 66 |
| abstract_inverted_index.Large | 0 |
| abstract_inverted_index.broad | 79 |
| abstract_inverted_index.build | 139 |
| abstract_inverted_index.first | 59 |
| abstract_inverted_index.gives | 105 |
| abstract_inverted_index.hints | 129 |
| abstract_inverted_index.human | 85 |
| abstract_inverted_index.shown | 5 |
| abstract_inverted_index.study | 53 |
| abstract_inverted_index.there | 16 |
| abstract_inverted_index.third | 136 |
| abstract_inverted_index.while | 8 |
| abstract_inverted_index.(LLMs) | 3 |
| abstract_inverted_index.direct | 13 |
| abstract_inverted_index.likely | 43 |
| abstract_inverted_index.little | 10 |
| abstract_inverted_index.metric | 97 |
| abstract_inverted_index.models | 2 |
| abstract_inverted_index.ability | 31 |
| abstract_inverted_index.address | 134 |
| abstract_inverted_index.answers | 107 |
| abstract_inverted_index.believe | 29 |
| abstract_inverted_index.crucial | 46 |
| abstract_inverted_index.current | 120 |
| abstract_inverted_index.measure | 114 |
| abstract_inverted_index.methods | 122 |
| abstract_inverted_index.perform | 123 |
| abstract_inverted_index.propose | 68 |
| abstract_inverted_index.results | 7 |
| abstract_inverted_index.Further, | 15 |
| abstract_inverted_index.concrete | 106 |
| abstract_inverted_index.evidence | 19 |
| abstract_inverted_index.language | 1 |
| abstract_inverted_index.mounting | 18 |
| abstract_inverted_index.setting. | 49 |
| abstract_inverted_index.standard | 90 |
| abstract_inverted_index.suitable | 99 |
| abstract_inverted_index.attribute | 36 |
| abstract_inverted_index.automatic | 96 |
| abstract_inverted_index.benchmark | 77 |
| abstract_inverted_index.formulate | 51 |
| abstract_inverted_index.framework | 72 |
| abstract_inverted_index.generates | 41 |
| abstract_inverted_index.potential | 24 |
| abstract_inverted_index.questions | 111 |
| abstract_inverted_index.requiring | 9 |
| abstract_inverted_index.Attributed | 54 |
| abstract_inverted_index.attributed | 65 |
| abstract_inverted_index.correlated | 95 |
| abstract_inverted_index.evaluation | 71 |
| abstract_inverted_index.impressive | 6 |
| abstract_inverted_index.scenarios. | 27 |
| abstract_inverted_index.annotations | 86 |
| abstract_inverted_index.development | 63 |
| abstract_inverted_index.development. | 101 |
| abstract_inverted_index.experimental | 103 |
| abstract_inverted_index.reproducible | 70 |
| abstract_inverted_index.supervision. | 14 |
| abstract_inverted_index.attribution?, | 115 |
| abstract_inverted_index.architectures. | 82 |
| abstract_inverted_index.attribution?), | 125 |
| abstract_inverted_index.attribution?). | 142 |
| abstract_inverted_index.state-of-the-art | 121 |
| abstract_inverted_index.information-seeking | 26 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 20 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.7300000190734863 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile |