OMuleT: Orchestrating Multiple Tools for Practicable Conversational Recommendation Article Swipe

PDF

Se-eun Yoon , Xiu-Shen Wei , Yabing Jiang , Rachit Pareek , Frank Ong , Kai Gao , Julian McAuley , Minghui Gong ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2411.19352

In this paper, we present a systematic effort to design, evaluate, and implement a realistic conversational recommender system (CRS). The objective of our system is to allow users to input free-form text to request recommendations, and then receive a list of relevant and diverse items. While previous work on synthetic queries augments large language models (LLMs) with 1-3 tools, we argue that a more extensive toolbox is necessary to effectively handle real user requests. As such, we propose a novel approach that equips LLMs with over 10 tools, providing them access to the internal knowledge base and API calls used in production. We evaluate our model on a dataset of real users and show that it generates relevant, novel, and diverse recommendations compared to vanilla LLMs. Furthermore, we conduct ablation studies to demonstrate the effectiveness of using the full range of tools in our toolbox. We share our designs and lessons learned from deploying the system for internal alpha release. Our contribution is the addressing of all four key aspects of a practicable CRS: (1) real user requests, (2) augmenting LLMs with a wide variety of tools, (3) extensive evaluation, and (4) deployment insights.

Related Topics

Computer Science

Human–Computer Interaction

Concepts

Computer science Human–computer interaction

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2411.19352
PDF: https://arxiv.org/pdf/2411.19352
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4405030261

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4405030261

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2411.19352

Digital Object Identifier
Title: OMuleT: Orchestrating Multiple Tools for Practicable Conversational Recommendation

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-11-28

Full publication date if available
Authors: Se-eun Yoon, Xiu-Shen Wei, Yabing Jiang, Rachit Pareek, Frank Ong, Kai Gao, Julian McAuley, Minghui Gong

List of authors in order
Landing page: https://arxiv.org/abs/2411.19352

Publisher landing page
PDF URL: https://arxiv.org/pdf/2411.19352

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2411.19352

Direct OA link when available
Concepts: Computer science, Human–computer interaction

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4405030261
doi	https://doi.org/10.48550/arxiv.2411.19352
ids.doi	https://doi.org/10.48550/arxiv.2411.19352
ids.openalex	https://openalex.org/W4405030261
fwci
type	preprint
title	OMuleT: Orchestrating Multiple Tools for Practicable Conversational Recommendation
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10028
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9786999821662903
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Topic Modeling
topics[1].id	https://openalex.org/T12031
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9695000052452087
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1702
topics[1].subfield.display_name	Artificial Intelligence
topics[1].display_name	Speech and dialogue systems
topics[2].id	https://openalex.org/T10203
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.9585000276565552
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1710
topics[2].subfield.display_name	Information Systems
topics[2].display_name	Recommender Systems and Techniques
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C41008148
concepts[0].level	0
concepts[0].score	0.573157012462616
concepts[0].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[0].display_name	Computer science
concepts[1].id	https://openalex.org/C107457646
concepts[1].level	1
concepts[1].score	0.32793164253234863
concepts[1].wikidata	https://www.wikidata.org/wiki/Q207434
concepts[1].display_name	Human–computer interaction
keywords[0].id	https://openalex.org/keywords/computer-science
keywords[0].score	0.573157012462616
keywords[0].display_name	Computer science
keywords[1].id	https://openalex.org/keywords/human–computer-interaction
keywords[1].score	0.32793164253234863
keywords[1].display_name	Human–computer interaction
language	en
locations[0].id	pmh:oai:arXiv.org:2411.19352
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2411.19352
locations[0].version	submittedVersion
locations[0].raw_type
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2411.19352
locations[1].id	doi:10.48550/arxiv.2411.19352
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2411.19352
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5054796820
authorships[0].author.orcid	https://orcid.org/0009-0004-7653-7012
authorships[0].author.display_name	Se-eun Yoon
authorships[0].author_position	first
authorships[0].raw_author_name	Yoon, Se-eun
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5066964304
authorships[1].author.orcid	https://orcid.org/0000-0002-8200-1845
authorships[1].author.display_name	Xiu-Shen Wei
authorships[1].author_position	middle
authorships[1].raw_author_name	Wei, Xiaokai
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5071959677
authorships[2].author.orcid
authorships[2].author.display_name	Yabing Jiang
authorships[2].author_position	middle
authorships[2].raw_author_name	Jiang, Yexi
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5115002510
authorships[3].author.orcid
authorships[3].author.display_name	Rachit Pareek
authorships[3].author_position	middle
authorships[3].raw_author_name	Pareek, Rachit
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5104030376
authorships[4].author.orcid
authorships[4].author.display_name	Frank Ong
authorships[4].author_position	middle
authorships[4].raw_author_name	Ong, Frank
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5022088613
authorships[5].author.orcid	https://orcid.org/0000-0001-6427-2038
authorships[5].author.display_name	Kai Gao
authorships[5].author_position	middle
authorships[5].raw_author_name	Gao, Kevin
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5021827617
authorships[6].author.orcid	https://orcid.org/0000-0003-0955-7588
authorships[6].author.display_name	Julian McAuley
authorships[6].author_position	middle
authorships[6].raw_author_name	McAuley, Julian
authorships[6].is_corresponding	False
authorships[7].author.id	https://openalex.org/A5020905664
authorships[7].author.orcid	https://orcid.org/0000-0003-4731-2784
authorships[7].author.display_name	Minghui Gong
authorships[7].author_position	last
authorships[7].raw_author_name	Gong, Michelle
authorships[7].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2411.19352
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2024-12-05T00:00:00
display_name	OMuleT: Orchestrating Multiple Tools for Practicable Conversational Recommendation
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10028
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9786999821662903
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Topic Modeling
related_works	https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2411.19352
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2411.19352
best_oa_location.version	submittedVersion
best_oa_location.raw_type
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2411.19352
primary_location.id	pmh:oai:arXiv.org:2411.19352
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2411.19352
primary_location.version	submittedVersion
primary_location.raw_type
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2411.19352
publication_date	2024-11-28
publication_year	2024
referenced_works_count	0
abstract_inverted_index.a	5, 13, 38, 62, 78, 107, 171, 182
abstract_inverted_index.10	86
abstract_inverted_index.As	74
abstract_inverted_index.In	0
abstract_inverted_index.We	102, 145
abstract_inverted_index.in	100, 142
abstract_inverted_index.is	24, 66, 162
abstract_inverted_index.it	115
abstract_inverted_index.of	21, 40, 109, 135, 140, 165, 170, 185
abstract_inverted_index.on	48, 106
abstract_inverted_index.to	8, 25, 28, 32, 68, 91, 123, 131
abstract_inverted_index.we	3, 59, 76, 127
abstract_inverted_index.(1)	174
abstract_inverted_index.(2)	178
abstract_inverted_index.(3)	187
abstract_inverted_index.(4)	191
abstract_inverted_index.1-3	57
abstract_inverted_index.API	97
abstract_inverted_index.Our	160
abstract_inverted_index.The	19
abstract_inverted_index.all	166
abstract_inverted_index.and	11, 35, 42, 96, 112, 119, 149, 190
abstract_inverted_index.for	156
abstract_inverted_index.key	168
abstract_inverted_index.our	22, 104, 143, 147
abstract_inverted_index.the	92, 133, 137, 154, 163
abstract_inverted_index.CRS:	173
abstract_inverted_index.LLMs	83, 180
abstract_inverted_index.base	95
abstract_inverted_index.four	167
abstract_inverted_index.from	152
abstract_inverted_index.full	138
abstract_inverted_index.list	39
abstract_inverted_index.more	63
abstract_inverted_index.over	85
abstract_inverted_index.real	71, 110, 175
abstract_inverted_index.show	113
abstract_inverted_index.text	31
abstract_inverted_index.that	61, 81, 114
abstract_inverted_index.them	89
abstract_inverted_index.then	36
abstract_inverted_index.this	1
abstract_inverted_index.used	99
abstract_inverted_index.user	72, 176
abstract_inverted_index.wide	183
abstract_inverted_index.with	56, 84, 181
abstract_inverted_index.work	47
abstract_inverted_index.LLMs.	125
abstract_inverted_index.While	45
abstract_inverted_index.allow	26
abstract_inverted_index.alpha	158
abstract_inverted_index.argue	60
abstract_inverted_index.calls	98
abstract_inverted_index.input	29
abstract_inverted_index.large	52
abstract_inverted_index.model	105
abstract_inverted_index.novel	79
abstract_inverted_index.range	139
abstract_inverted_index.share	146
abstract_inverted_index.such,	75
abstract_inverted_index.tools	141
abstract_inverted_index.users	27, 111
abstract_inverted_index.using	136
abstract_inverted_index.(CRS).	18
abstract_inverted_index.(LLMs)	55
abstract_inverted_index.access	90
abstract_inverted_index.effort	7
abstract_inverted_index.equips	82
abstract_inverted_index.handle	70
abstract_inverted_index.items.	44
abstract_inverted_index.models	54
abstract_inverted_index.novel,	118
abstract_inverted_index.paper,	2
abstract_inverted_index.system	17, 23, 155
abstract_inverted_index.tools,	58, 87, 186
abstract_inverted_index.aspects	169
abstract_inverted_index.conduct	128
abstract_inverted_index.dataset	108
abstract_inverted_index.design,	9
abstract_inverted_index.designs	148
abstract_inverted_index.diverse	43, 120
abstract_inverted_index.learned	151
abstract_inverted_index.lessons	150
abstract_inverted_index.present	4
abstract_inverted_index.propose	77
abstract_inverted_index.queries	50
abstract_inverted_index.receive	37
abstract_inverted_index.request	33
abstract_inverted_index.studies	130
abstract_inverted_index.toolbox	65
abstract_inverted_index.vanilla	124
abstract_inverted_index.variety	184
abstract_inverted_index.ablation	129
abstract_inverted_index.approach	80
abstract_inverted_index.augments	51
abstract_inverted_index.compared	122
abstract_inverted_index.evaluate	103
abstract_inverted_index.internal	93, 157
abstract_inverted_index.language	53
abstract_inverted_index.previous	46
abstract_inverted_index.release.	159
abstract_inverted_index.relevant	41
abstract_inverted_index.toolbox.	144
abstract_inverted_index.deploying	153
abstract_inverted_index.evaluate,	10
abstract_inverted_index.extensive	64, 188
abstract_inverted_index.free-form	30
abstract_inverted_index.generates	116
abstract_inverted_index.implement	12
abstract_inverted_index.insights.	193
abstract_inverted_index.knowledge	94
abstract_inverted_index.necessary	67
abstract_inverted_index.objective	20
abstract_inverted_index.providing	88
abstract_inverted_index.realistic	14
abstract_inverted_index.relevant,	117
abstract_inverted_index.requests,	177
abstract_inverted_index.requests.	73
abstract_inverted_index.synthetic	49
abstract_inverted_index.addressing	164
abstract_inverted_index.augmenting	179
abstract_inverted_index.deployment	192
abstract_inverted_index.systematic	6
abstract_inverted_index.demonstrate	132
abstract_inverted_index.effectively	69
abstract_inverted_index.evaluation,	189
abstract_inverted_index.practicable	172
abstract_inverted_index.production.	101
abstract_inverted_index.recommender	16
abstract_inverted_index.Furthermore,	126
abstract_inverted_index.contribution	161
abstract_inverted_index.effectiveness	134
abstract_inverted_index.conversational	15
abstract_inverted_index.recommendations	121
abstract_inverted_index.recommendations,	34
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	8
citation_normalized_percentile