InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task Planning Article Swipe
Muzhi Han
,
Yifeng Zhu
,
Song-Chun Zhu
,
Ying Wu
,
Yuke Zhu
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2405.19758
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2405.19758
Learning abstract state representations and knowledge is crucial for long-horizon robot planning. We present InterPreT, an LLM-powered framework for robots to learn symbolic predicates from language feedback of human non-experts during embodied interaction. The learned predicates provide relational abstractions of the environment state, facilitating the learning of symbolic operators that capture action preconditions and effects. By compiling the learned predicates and operators into a PDDL domain on-the-fly, InterPreT allows effective planning toward arbitrary in-domain goals using a PDDL planner. In both simulated and real-world robot manipulation domains, we demonstrate that InterPreT reliably uncovers the key predicates and operators governing the environment dynamics. Although learned from simple training tasks, these predicates and operators exhibit strong generalization to novel tasks with significantly higher complexity. In the most challenging generalization setting, InterPreT attains success rates of 73% in simulation and 40% in the real world, substantially outperforming baseline methods.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2405.19758
- https://arxiv.org/pdf/2405.19758
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4399252291
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4399252291Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2405.19758Digital Object Identifier
- Title
-
InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task PlanningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-05-30Full publication date if available
- Authors
-
Muzhi Han, Yifeng Zhu, Song-Chun Zhu, Ying Wu, Yuke ZhuList of authors in order
- Landing page
-
https://arxiv.org/abs/2405.19758Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2405.19758Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2405.19758Direct OA link when available
- Concepts
-
Computer science, Task (project management), Predicate (mathematical logic), Natural language processing, Artificial intelligence, Cognitive psychology, Psychology, Programming language, Engineering, Systems engineeringTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4399252291 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2405.19758 |
| ids.doi | https://doi.org/10.48550/arxiv.2405.19758 |
| ids.openalex | https://openalex.org/W4399252291 |
| fwci | |
| type | preprint |
| title | InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task Planning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10181 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9873999953269958 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Natural Language Processing Techniques |
| topics[1].id | https://openalex.org/T12031 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9757000207901001 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Speech and dialogue systems |
| topics[2].id | https://openalex.org/T10028 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.944100022315979 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Topic Modeling |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.6544294357299805 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C2780451532 |
| concepts[1].level | 2 |
| concepts[1].score | 0.5769216418266296 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q759676 |
| concepts[1].display_name | Task (project management) |
| concepts[2].id | https://openalex.org/C140146324 |
| concepts[2].level | 2 |
| concepts[2].score | 0.5745483636856079 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q1144319 |
| concepts[2].display_name | Predicate (mathematical logic) |
| concepts[3].id | https://openalex.org/C204321447 |
| concepts[3].level | 1 |
| concepts[3].score | 0.4509144425392151 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[3].display_name | Natural language processing |
| concepts[4].id | https://openalex.org/C154945302 |
| concepts[4].level | 1 |
| concepts[4].score | 0.3772643804550171 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[4].display_name | Artificial intelligence |
| concepts[5].id | https://openalex.org/C180747234 |
| concepts[5].level | 1 |
| concepts[5].score | 0.3233446180820465 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q23373 |
| concepts[5].display_name | Cognitive psychology |
| concepts[6].id | https://openalex.org/C15744967 |
| concepts[6].level | 0 |
| concepts[6].score | 0.2726517915725708 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[6].display_name | Psychology |
| concepts[7].id | https://openalex.org/C199360897 |
| concepts[7].level | 1 |
| concepts[7].score | 0.1189558207988739 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[7].display_name | Programming language |
| concepts[8].id | https://openalex.org/C127413603 |
| concepts[8].level | 0 |
| concepts[8].score | 0.06568291783332825 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[8].display_name | Engineering |
| concepts[9].id | https://openalex.org/C201995342 |
| concepts[9].level | 1 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q682496 |
| concepts[9].display_name | Systems engineering |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.6544294357299805 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/task |
| keywords[1].score | 0.5769216418266296 |
| keywords[1].display_name | Task (project management) |
| keywords[2].id | https://openalex.org/keywords/predicate |
| keywords[2].score | 0.5745483636856079 |
| keywords[2].display_name | Predicate (mathematical logic) |
| keywords[3].id | https://openalex.org/keywords/natural-language-processing |
| keywords[3].score | 0.4509144425392151 |
| keywords[3].display_name | Natural language processing |
| keywords[4].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[4].score | 0.3772643804550171 |
| keywords[4].display_name | Artificial intelligence |
| keywords[5].id | https://openalex.org/keywords/cognitive-psychology |
| keywords[5].score | 0.3233446180820465 |
| keywords[5].display_name | Cognitive psychology |
| keywords[6].id | https://openalex.org/keywords/psychology |
| keywords[6].score | 0.2726517915725708 |
| keywords[6].display_name | Psychology |
| keywords[7].id | https://openalex.org/keywords/programming-language |
| keywords[7].score | 0.1189558207988739 |
| keywords[7].display_name | Programming language |
| keywords[8].id | https://openalex.org/keywords/engineering |
| keywords[8].score | 0.06568291783332825 |
| keywords[8].display_name | Engineering |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2405.19758 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2405.19758 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2405.19758 |
| locations[1].id | doi:10.48550/arxiv.2405.19758 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2405.19758 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5070386263 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-2649-4577 |
| authorships[0].author.display_name | Muzhi Han |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Han, Muzhi |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5102214085 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Yifeng Zhu |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Zhu, Yifeng |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5031660884 |
| authorships[2].author.orcid | https://orcid.org/0009-0009-9458-5583 |
| authorships[2].author.display_name | Song-Chun Zhu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Zhu, Song-Chun |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5101780958 |
| authorships[3].author.orcid | https://orcid.org/0009-0001-6768-5118 |
| authorships[3].author.display_name | Ying Wu |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Wu, Ying Nian |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5111217717 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Yuke Zhu |
| authorships[4].author_position | last |
| authorships[4].raw_author_name | Zhu, Yuke |
| authorships[4].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2405.19758 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task Planning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10181 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9873999953269958 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Natural Language Processing Techniques |
| related_works | https://openalex.org/W1547206149, https://openalex.org/W2054891275, https://openalex.org/W2364373416, https://openalex.org/W2042218267, https://openalex.org/W1577005250, https://openalex.org/W2352272361, https://openalex.org/W1871461366, https://openalex.org/W2379635044, https://openalex.org/W3204019825, https://openalex.org/W4211075255 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2405.19758 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2405.19758 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2405.19758 |
| primary_location.id | pmh:oai:arXiv.org:2405.19758 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2405.19758 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2405.19758 |
| publication_date | 2024-05-30 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 63, 76 |
| abstract_inverted_index.By | 55 |
| abstract_inverted_index.In | 79, 122 |
| abstract_inverted_index.We | 12 |
| abstract_inverted_index.an | 15 |
| abstract_inverted_index.in | 134, 138 |
| abstract_inverted_index.is | 6 |
| abstract_inverted_index.of | 27, 39, 46, 132 |
| abstract_inverted_index.to | 20, 115 |
| abstract_inverted_index.we | 87 |
| abstract_inverted_index.40% | 137 |
| abstract_inverted_index.73% | 133 |
| abstract_inverted_index.The | 33 |
| abstract_inverted_index.and | 4, 53, 60, 82, 96, 110, 136 |
| abstract_inverted_index.for | 8, 18 |
| abstract_inverted_index.key | 94 |
| abstract_inverted_index.the | 40, 44, 57, 93, 99, 123, 139 |
| abstract_inverted_index.PDDL | 64, 77 |
| abstract_inverted_index.both | 80 |
| abstract_inverted_index.from | 24, 104 |
| abstract_inverted_index.into | 62 |
| abstract_inverted_index.most | 124 |
| abstract_inverted_index.real | 140 |
| abstract_inverted_index.that | 49, 89 |
| abstract_inverted_index.with | 118 |
| abstract_inverted_index.goals | 74 |
| abstract_inverted_index.human | 28 |
| abstract_inverted_index.learn | 21 |
| abstract_inverted_index.novel | 116 |
| abstract_inverted_index.rates | 131 |
| abstract_inverted_index.robot | 10, 84 |
| abstract_inverted_index.state | 2 |
| abstract_inverted_index.tasks | 117 |
| abstract_inverted_index.these | 108 |
| abstract_inverted_index.using | 75 |
| abstract_inverted_index.action | 51 |
| abstract_inverted_index.allows | 68 |
| abstract_inverted_index.domain | 65 |
| abstract_inverted_index.during | 30 |
| abstract_inverted_index.higher | 120 |
| abstract_inverted_index.robots | 19 |
| abstract_inverted_index.simple | 105 |
| abstract_inverted_index.state, | 42 |
| abstract_inverted_index.strong | 113 |
| abstract_inverted_index.tasks, | 107 |
| abstract_inverted_index.toward | 71 |
| abstract_inverted_index.world, | 141 |
| abstract_inverted_index.attains | 129 |
| abstract_inverted_index.capture | 50 |
| abstract_inverted_index.crucial | 7 |
| abstract_inverted_index.exhibit | 112 |
| abstract_inverted_index.learned | 34, 58, 103 |
| abstract_inverted_index.present | 13 |
| abstract_inverted_index.provide | 36 |
| abstract_inverted_index.success | 130 |
| abstract_inverted_index.Although | 102 |
| abstract_inverted_index.Learning | 0 |
| abstract_inverted_index.abstract | 1 |
| abstract_inverted_index.baseline | 144 |
| abstract_inverted_index.domains, | 86 |
| abstract_inverted_index.effects. | 54 |
| abstract_inverted_index.embodied | 31 |
| abstract_inverted_index.feedback | 26 |
| abstract_inverted_index.language | 25 |
| abstract_inverted_index.learning | 45 |
| abstract_inverted_index.methods. | 145 |
| abstract_inverted_index.planner. | 78 |
| abstract_inverted_index.planning | 70 |
| abstract_inverted_index.reliably | 91 |
| abstract_inverted_index.setting, | 127 |
| abstract_inverted_index.symbolic | 22, 47 |
| abstract_inverted_index.training | 106 |
| abstract_inverted_index.uncovers | 92 |
| abstract_inverted_index.InterPreT | 67, 90, 128 |
| abstract_inverted_index.arbitrary | 72 |
| abstract_inverted_index.compiling | 56 |
| abstract_inverted_index.dynamics. | 101 |
| abstract_inverted_index.effective | 69 |
| abstract_inverted_index.framework | 17 |
| abstract_inverted_index.governing | 98 |
| abstract_inverted_index.in-domain | 73 |
| abstract_inverted_index.knowledge | 5 |
| abstract_inverted_index.operators | 48, 61, 97, 111 |
| abstract_inverted_index.planning. | 11 |
| abstract_inverted_index.simulated | 81 |
| abstract_inverted_index.InterPreT, | 14 |
| abstract_inverted_index.predicates | 23, 35, 59, 95, 109 |
| abstract_inverted_index.real-world | 83 |
| abstract_inverted_index.relational | 37 |
| abstract_inverted_index.simulation | 135 |
| abstract_inverted_index.LLM-powered | 16 |
| abstract_inverted_index.challenging | 125 |
| abstract_inverted_index.complexity. | 121 |
| abstract_inverted_index.demonstrate | 88 |
| abstract_inverted_index.environment | 41, 100 |
| abstract_inverted_index.non-experts | 29 |
| abstract_inverted_index.on-the-fly, | 66 |
| abstract_inverted_index.abstractions | 38 |
| abstract_inverted_index.facilitating | 43 |
| abstract_inverted_index.interaction. | 32 |
| abstract_inverted_index.long-horizon | 9 |
| abstract_inverted_index.manipulation | 85 |
| abstract_inverted_index.outperforming | 143 |
| abstract_inverted_index.preconditions | 52 |
| abstract_inverted_index.significantly | 119 |
| abstract_inverted_index.substantially | 142 |
| abstract_inverted_index.generalization | 114, 126 |
| abstract_inverted_index.representations | 3 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 5 |
| citation_normalized_percentile |