Large Language Models are Interpretable Learners Article Swipe
Ruochen Wang
,
Si Si
,
Felix Yu
,
D. Wiesmann
,
Cho‐Jui Hsieh
,
Inderjit S. Dhillon
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2406.17224
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2406.17224
The trade-off between expressiveness and interpretability remains a core challenge when building human-centric predictive models for classification and decision-making. While symbolic rules offer interpretability, they often lack expressiveness, whereas neural networks excel in performance but are known for being black boxes. In this paper, we show a combination of Large Language Models (LLMs) and symbolic programs can bridge this gap. In the proposed LLM-based Symbolic Programs (LSPs), the pretrained LLM with natural language prompts provides a massive set of interpretable modules that can transform raw input into natural language concepts. Symbolic programs then integrate these modules into an interpretable decision rule. To train LSPs, we develop a divide-and-conquer approach to incrementally build the program from scratch, where the learning process of each step is guided by LLMs. To evaluate the effectiveness of LSPs in extracting interpretable and accurate knowledge from data, we introduce IL-Bench, a collection of diverse tasks, including both synthetic and real-world scenarios across different modalities. Empirical results demonstrate LSP's superior performance compared to traditional neurosymbolic programs and vanilla automatic prompt tuning methods. Moreover, as the knowledge learned by LSP is a combination of natural language descriptions and symbolic rules, it is easily transferable to humans (interpretable), and other LLMs, and generalizes well to out-of-distribution samples.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2406.17224
- https://arxiv.org/pdf/2406.17224
- OA Status
- green
- Cited By
- 2
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4400064758
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4400064758Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2406.17224Digital Object Identifier
- Title
-
Large Language Models are Interpretable LearnersWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-06-25Full publication date if available
- Authors
-
Ruochen Wang, Si Si, Felix Yu, D. Wiesmann, Cho‐Jui Hsieh, Inderjit S. DhillonList of authors in order
- Landing page
-
https://arxiv.org/abs/2406.17224Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2406.17224Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2406.17224Direct OA link when available
- Concepts
-
Computer science, Natural language processing, Artificial intelligence, Linguistics, PhilosophyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
2Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 2Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4400064758 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2406.17224 |
| ids.doi | https://doi.org/10.48550/arxiv.2406.17224 |
| ids.openalex | https://openalex.org/W4400064758 |
| fwci | |
| type | preprint |
| title | Large Language Models are Interpretable Learners |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.8605999946594238 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.7838000059127808 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.5858905911445618 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C204321447 |
| concepts[1].level | 1 |
| concepts[1].score | 0.5462777018547058 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[1].display_name | Natural language processing |
| concepts[2].id | https://openalex.org/C154945302 |
| concepts[2].level | 1 |
| concepts[2].score | 0.45876210927963257 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[2].display_name | Artificial intelligence |
| concepts[3].id | https://openalex.org/C41895202 |
| concepts[3].level | 1 |
| concepts[3].score | 0.4033655524253845 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q8162 |
| concepts[3].display_name | Linguistics |
| concepts[4].id | https://openalex.org/C138885662 |
| concepts[4].level | 0 |
| concepts[4].score | 0.07413944602012634 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q5891 |
| concepts[4].display_name | Philosophy |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.5858905911445618 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/natural-language-processing |
| keywords[1].score | 0.5462777018547058 |
| keywords[1].display_name | Natural language processing |
| keywords[2].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[2].score | 0.45876210927963257 |
| keywords[2].display_name | Artificial intelligence |
| keywords[3].id | https://openalex.org/keywords/linguistics |
| keywords[3].score | 0.4033655524253845 |
| keywords[3].display_name | Linguistics |
| keywords[4].id | https://openalex.org/keywords/philosophy |
| keywords[4].score | 0.07413944602012634 |
| keywords[4].display_name | Philosophy |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2406.17224 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2406.17224 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2406.17224 |
| locations[1].id | doi:10.48550/arxiv.2406.17224 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2406.17224 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5080683301 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-2014-4282 |
| authorships[0].author.display_name | Ruochen Wang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Wang, Ruochen |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5001215167 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-2406-7221 |
| authorships[1].author.display_name | Si Si |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Si, Si |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5111045439 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Felix Yu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Yu, Felix |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5082313228 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | D. Wiesmann |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Wiesmann, Dorothea |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5010841999 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-3520-9627 |
| authorships[4].author.display_name | Cho‐Jui Hsieh |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Hsieh, Cho-Jui |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5063459703 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-2759-1416 |
| authorships[5].author.display_name | Inderjit S. Dhillon |
| authorships[5].author_position | last |
| authorships[5].raw_author_name | Dhillon, Inderjit |
| authorships[5].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2406.17224 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Large Language Models are Interpretable Learners |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.8605999946594238 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052, https://openalex.org/W2382290278, https://openalex.org/W3204019825 |
| cited_by_count | 2 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 2 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2406.17224 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2406.17224 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2406.17224 |
| primary_location.id | pmh:oai:arXiv.org:2406.17224 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2406.17224 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2406.17224 |
| publication_date | 2024-06-25 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 7, 46, 75, 106, 144, 183 |
| abstract_inverted_index.In | 41, 60 |
| abstract_inverted_index.To | 101, 127 |
| abstract_inverted_index.an | 97 |
| abstract_inverted_index.as | 176 |
| abstract_inverted_index.by | 125, 180 |
| abstract_inverted_index.in | 32, 133 |
| abstract_inverted_index.is | 123, 182, 193 |
| abstract_inverted_index.it | 192 |
| abstract_inverted_index.of | 48, 78, 120, 131, 146, 185 |
| abstract_inverted_index.to | 109, 165, 196, 205 |
| abstract_inverted_index.we | 44, 104, 141 |
| abstract_inverted_index.LLM | 69 |
| abstract_inverted_index.LSP | 181 |
| abstract_inverted_index.The | 0 |
| abstract_inverted_index.and | 4, 17, 53, 136, 152, 169, 189, 199, 202 |
| abstract_inverted_index.are | 35 |
| abstract_inverted_index.but | 34 |
| abstract_inverted_index.can | 56, 82 |
| abstract_inverted_index.for | 15, 37 |
| abstract_inverted_index.raw | 84 |
| abstract_inverted_index.set | 77 |
| abstract_inverted_index.the | 61, 67, 112, 117, 129, 177 |
| abstract_inverted_index.LSPs | 132 |
| abstract_inverted_index.both | 150 |
| abstract_inverted_index.core | 8 |
| abstract_inverted_index.each | 121 |
| abstract_inverted_index.from | 114, 139 |
| abstract_inverted_index.gap. | 59 |
| abstract_inverted_index.into | 86, 96 |
| abstract_inverted_index.lack | 26 |
| abstract_inverted_index.show | 45 |
| abstract_inverted_index.step | 122 |
| abstract_inverted_index.that | 81 |
| abstract_inverted_index.then | 92 |
| abstract_inverted_index.they | 24 |
| abstract_inverted_index.this | 42, 58 |
| abstract_inverted_index.well | 204 |
| abstract_inverted_index.when | 10 |
| abstract_inverted_index.with | 70 |
| abstract_inverted_index.LLMs, | 201 |
| abstract_inverted_index.LLMs. | 126 |
| abstract_inverted_index.LSP's | 161 |
| abstract_inverted_index.LSPs, | 103 |
| abstract_inverted_index.Large | 49 |
| abstract_inverted_index.While | 19 |
| abstract_inverted_index.being | 38 |
| abstract_inverted_index.black | 39 |
| abstract_inverted_index.build | 111 |
| abstract_inverted_index.data, | 140 |
| abstract_inverted_index.excel | 31 |
| abstract_inverted_index.input | 85 |
| abstract_inverted_index.known | 36 |
| abstract_inverted_index.offer | 22 |
| abstract_inverted_index.often | 25 |
| abstract_inverted_index.other | 200 |
| abstract_inverted_index.rule. | 100 |
| abstract_inverted_index.rules | 21 |
| abstract_inverted_index.these | 94 |
| abstract_inverted_index.train | 102 |
| abstract_inverted_index.where | 116 |
| abstract_inverted_index.(LLMs) | 52 |
| abstract_inverted_index.Models | 51 |
| abstract_inverted_index.across | 155 |
| abstract_inverted_index.boxes. | 40 |
| abstract_inverted_index.bridge | 57 |
| abstract_inverted_index.easily | 194 |
| abstract_inverted_index.guided | 124 |
| abstract_inverted_index.humans | 197 |
| abstract_inverted_index.models | 14 |
| abstract_inverted_index.neural | 29 |
| abstract_inverted_index.paper, | 43 |
| abstract_inverted_index.prompt | 172 |
| abstract_inverted_index.rules, | 191 |
| abstract_inverted_index.tasks, | 148 |
| abstract_inverted_index.tuning | 173 |
| abstract_inverted_index.(LSPs), | 66 |
| abstract_inverted_index.between | 2 |
| abstract_inverted_index.develop | 105 |
| abstract_inverted_index.diverse | 147 |
| abstract_inverted_index.learned | 179 |
| abstract_inverted_index.massive | 76 |
| abstract_inverted_index.modules | 80, 95 |
| abstract_inverted_index.natural | 71, 87, 186 |
| abstract_inverted_index.process | 119 |
| abstract_inverted_index.program | 113 |
| abstract_inverted_index.prompts | 73 |
| abstract_inverted_index.remains | 6 |
| abstract_inverted_index.results | 159 |
| abstract_inverted_index.vanilla | 170 |
| abstract_inverted_index.whereas | 28 |
| abstract_inverted_index.Language | 50 |
| abstract_inverted_index.Programs | 65 |
| abstract_inverted_index.Symbolic | 64, 90 |
| abstract_inverted_index.accurate | 137 |
| abstract_inverted_index.approach | 108 |
| abstract_inverted_index.building | 11 |
| abstract_inverted_index.compared | 164 |
| abstract_inverted_index.decision | 99 |
| abstract_inverted_index.evaluate | 128 |
| abstract_inverted_index.language | 72, 88, 187 |
| abstract_inverted_index.learning | 118 |
| abstract_inverted_index.methods. | 174 |
| abstract_inverted_index.networks | 30 |
| abstract_inverted_index.programs | 55, 91, 168 |
| abstract_inverted_index.proposed | 62 |
| abstract_inverted_index.provides | 74 |
| abstract_inverted_index.samples. | 207 |
| abstract_inverted_index.scratch, | 115 |
| abstract_inverted_index.superior | 162 |
| abstract_inverted_index.symbolic | 20, 54, 190 |
| abstract_inverted_index.Empirical | 158 |
| abstract_inverted_index.IL-Bench, | 143 |
| abstract_inverted_index.LLM-based | 63 |
| abstract_inverted_index.Moreover, | 175 |
| abstract_inverted_index.automatic | 171 |
| abstract_inverted_index.challenge | 9 |
| abstract_inverted_index.concepts. | 89 |
| abstract_inverted_index.different | 156 |
| abstract_inverted_index.including | 149 |
| abstract_inverted_index.integrate | 93 |
| abstract_inverted_index.introduce | 142 |
| abstract_inverted_index.knowledge | 138, 178 |
| abstract_inverted_index.scenarios | 154 |
| abstract_inverted_index.synthetic | 151 |
| abstract_inverted_index.trade-off | 1 |
| abstract_inverted_index.transform | 83 |
| abstract_inverted_index.collection | 145 |
| abstract_inverted_index.extracting | 134 |
| abstract_inverted_index.predictive | 13 |
| abstract_inverted_index.pretrained | 68 |
| abstract_inverted_index.real-world | 153 |
| abstract_inverted_index.combination | 47, 184 |
| abstract_inverted_index.demonstrate | 160 |
| abstract_inverted_index.generalizes | 203 |
| abstract_inverted_index.modalities. | 157 |
| abstract_inverted_index.performance | 33, 163 |
| abstract_inverted_index.traditional | 166 |
| abstract_inverted_index.descriptions | 188 |
| abstract_inverted_index.transferable | 195 |
| abstract_inverted_index.effectiveness | 130 |
| abstract_inverted_index.human-centric | 12 |
| abstract_inverted_index.incrementally | 110 |
| abstract_inverted_index.interpretable | 79, 98, 135 |
| abstract_inverted_index.neurosymbolic | 167 |
| abstract_inverted_index.classification | 16 |
| abstract_inverted_index.expressiveness | 3 |
| abstract_inverted_index.expressiveness, | 27 |
| abstract_inverted_index.(interpretable), | 198 |
| abstract_inverted_index.decision-making. | 18 |
| abstract_inverted_index.interpretability | 5 |
| abstract_inverted_index.interpretability, | 23 |
| abstract_inverted_index.divide-and-conquer | 107 |
| abstract_inverted_index.out-of-distribution | 206 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 6 |
| citation_normalized_percentile |