How Does Conversation Length Impact User's Satisfaction? A Case Study of Length-Controlled Conversations with LLM-Powered Chatbots Article Swipe
Shih-Hong Huang
,
Ya-Fang Lin
,
Zeyu He
,
Chieh-Yang Huang
,
Ting-Hao Huang
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2404.17025
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2404.17025
Users can discuss a wide range of topics with large language models (LLMs), but they do not always prefer solving problems or getting information through lengthy conversations. This raises an intriguing HCI question: How does instructing LLMs to engage in longer or shorter conversations affect conversation quality? In this paper, we developed two Slack chatbots using GPT-4 with the ability to vary conversation lengths and conducted a user study. Participants asked the chatbots both highly and less conversable questions, engaging in dialogues with 0, 3, 5, and 7 conversational turns. We found that the conversation quality does not differ drastically across different conditions, while participants had mixed reactions. Our study demonstrates LLMs' ability to change conversation length and the potential benefits for users resulting from such changes, but we caution that changes in text form may not necessarily imply changes in quality or content.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2404.17025
- https://arxiv.org/pdf/2404.17025
- OA Status
- green
- Cited By
- 1
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4396243501
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4396243501Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2404.17025Digital Object Identifier
- Title
-
How Does Conversation Length Impact User's Satisfaction? A Case Study of Length-Controlled Conversations with LLM-Powered ChatbotsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-04-25Full publication date if available
- Authors
-
Shih-Hong Huang, Ya-Fang Lin, Zeyu He, Chieh-Yang Huang, Ting-Hao HuangList of authors in order
- Landing page
-
https://arxiv.org/abs/2404.17025Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2404.17025Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2404.17025Direct OA link when available
- Concepts
-
Conversation, User satisfaction, Computer science, Psychology, Human–computer interaction, CommunicationTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2024: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4396243501 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2404.17025 |
| ids.doi | https://doi.org/10.48550/arxiv.2404.17025 |
| ids.openalex | https://openalex.org/W4396243501 |
| fwci | |
| type | preprint |
| title | How Does Conversation Length Impact User's Satisfaction? A Case Study of Length-Controlled Conversations with LLM-Powered Chatbots |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12128 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9952999949455261 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | AI in Service Interactions |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2777200299 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8663485646247864 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q52943 |
| concepts[0].display_name | Conversation |
| concepts[1].id | https://openalex.org/C3017893058 |
| concepts[1].level | 2 |
| concepts[1].score | 0.5548992156982422 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q999278 |
| concepts[1].display_name | User satisfaction |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.42619356513023376 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C15744967 |
| concepts[3].level | 0 |
| concepts[3].score | 0.3893277049064636 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[3].display_name | Psychology |
| concepts[4].id | https://openalex.org/C107457646 |
| concepts[4].level | 1 |
| concepts[4].score | 0.2969106137752533 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q207434 |
| concepts[4].display_name | Human–computer interaction |
| concepts[5].id | https://openalex.org/C46312422 |
| concepts[5].level | 1 |
| concepts[5].score | 0.1345607340335846 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11024 |
| concepts[5].display_name | Communication |
| keywords[0].id | https://openalex.org/keywords/conversation |
| keywords[0].score | 0.8663485646247864 |
| keywords[0].display_name | Conversation |
| keywords[1].id | https://openalex.org/keywords/user-satisfaction |
| keywords[1].score | 0.5548992156982422 |
| keywords[1].display_name | User satisfaction |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.42619356513023376 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/psychology |
| keywords[3].score | 0.3893277049064636 |
| keywords[3].display_name | Psychology |
| keywords[4].id | https://openalex.org/keywords/human–computer-interaction |
| keywords[4].score | 0.2969106137752533 |
| keywords[4].display_name | Human–computer interaction |
| keywords[5].id | https://openalex.org/keywords/communication |
| keywords[5].score | 0.1345607340335846 |
| keywords[5].display_name | Communication |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2404.17025 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2404.17025 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2404.17025 |
| locations[1].id | doi:10.48550/arxiv.2404.17025 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2404.17025 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5074527184 |
| authorships[0].author.orcid | https://orcid.org/0009-0001-4298-5325 |
| authorships[0].author.display_name | Shih-Hong Huang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Huang, Shih-Hong |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5076681896 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-3689-5910 |
| authorships[1].author.display_name | Ya-Fang Lin |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Lin, Ya-Fang |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5101817568 |
| authorships[2].author.orcid | https://orcid.org/0009-0007-7115-2692 |
| authorships[2].author.display_name | Zeyu He |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | He, Zeyu |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5102729368 |
| authorships[3].author.orcid | https://orcid.org/0009-0001-6736-9959 |
| authorships[3].author.display_name | Chieh-Yang Huang |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Huang, Chieh-Yang |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5083675499 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-7021-4627 |
| authorships[4].author.display_name | Ting-Hao Huang |
| authorships[4].author_position | last |
| authorships[4].raw_author_name | Huang, Ting-Hao 'Kenneth' |
| authorships[4].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2404.17025 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | How Does Conversation Length Impact User's Satisfaction? A Case Study of Length-Controlled Conversations with LLM-Powered Chatbots |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12128 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9952999949455261 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | AI in Service Interactions |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W1968552888, https://openalex.org/W2374116601, https://openalex.org/W3093134843, https://openalex.org/W2772323916, https://openalex.org/W1511346092, https://openalex.org/W1527532029, https://openalex.org/W2378167147, https://openalex.org/W3210777354 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2024 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2404.17025 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2404.17025 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2404.17025 |
| primary_location.id | pmh:oai:arXiv.org:2404.17025 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2404.17025 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2404.17025 |
| publication_date | 2024-04-25 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.7 | 87 |
| abstract_inverted_index.a | 3, 66 |
| abstract_inverted_index.0, | 83 |
| abstract_inverted_index.3, | 84 |
| abstract_inverted_index.5, | 85 |
| abstract_inverted_index.In | 47 |
| abstract_inverted_index.We | 90 |
| abstract_inverted_index.an | 29 |
| abstract_inverted_index.do | 15 |
| abstract_inverted_index.in | 39, 80, 132, 140 |
| abstract_inverted_index.of | 6 |
| abstract_inverted_index.or | 21, 41, 142 |
| abstract_inverted_index.to | 37, 60, 113 |
| abstract_inverted_index.we | 50, 128 |
| abstract_inverted_index.HCI | 31 |
| abstract_inverted_index.How | 33 |
| abstract_inverted_index.Our | 108 |
| abstract_inverted_index.and | 64, 75, 86, 117 |
| abstract_inverted_index.but | 13, 127 |
| abstract_inverted_index.can | 1 |
| abstract_inverted_index.for | 121 |
| abstract_inverted_index.had | 105 |
| abstract_inverted_index.may | 135 |
| abstract_inverted_index.not | 16, 97, 136 |
| abstract_inverted_index.the | 58, 71, 93, 118 |
| abstract_inverted_index.two | 52 |
| abstract_inverted_index.LLMs | 36 |
| abstract_inverted_index.This | 27 |
| abstract_inverted_index.both | 73 |
| abstract_inverted_index.does | 34, 96 |
| abstract_inverted_index.form | 134 |
| abstract_inverted_index.from | 124 |
| abstract_inverted_index.less | 76 |
| abstract_inverted_index.such | 125 |
| abstract_inverted_index.text | 133 |
| abstract_inverted_index.that | 92, 130 |
| abstract_inverted_index.they | 14 |
| abstract_inverted_index.this | 48 |
| abstract_inverted_index.user | 67 |
| abstract_inverted_index.vary | 61 |
| abstract_inverted_index.wide | 4 |
| abstract_inverted_index.with | 8, 57, 82 |
| abstract_inverted_index.GPT-4 | 56 |
| abstract_inverted_index.LLMs' | 111 |
| abstract_inverted_index.Slack | 53 |
| abstract_inverted_index.Users | 0 |
| abstract_inverted_index.asked | 70 |
| abstract_inverted_index.found | 91 |
| abstract_inverted_index.imply | 138 |
| abstract_inverted_index.large | 9 |
| abstract_inverted_index.mixed | 106 |
| abstract_inverted_index.range | 5 |
| abstract_inverted_index.study | 109 |
| abstract_inverted_index.users | 122 |
| abstract_inverted_index.using | 55 |
| abstract_inverted_index.while | 103 |
| abstract_inverted_index.across | 100 |
| abstract_inverted_index.affect | 44 |
| abstract_inverted_index.always | 17 |
| abstract_inverted_index.change | 114 |
| abstract_inverted_index.differ | 98 |
| abstract_inverted_index.engage | 38 |
| abstract_inverted_index.highly | 74 |
| abstract_inverted_index.length | 116 |
| abstract_inverted_index.longer | 40 |
| abstract_inverted_index.models | 11 |
| abstract_inverted_index.paper, | 49 |
| abstract_inverted_index.prefer | 18 |
| abstract_inverted_index.raises | 28 |
| abstract_inverted_index.study. | 68 |
| abstract_inverted_index.topics | 7 |
| abstract_inverted_index.turns. | 89 |
| abstract_inverted_index.(LLMs), | 12 |
| abstract_inverted_index.ability | 59, 112 |
| abstract_inverted_index.caution | 129 |
| abstract_inverted_index.changes | 131, 139 |
| abstract_inverted_index.discuss | 2 |
| abstract_inverted_index.getting | 22 |
| abstract_inverted_index.lengths | 63 |
| abstract_inverted_index.lengthy | 25 |
| abstract_inverted_index.quality | 95, 141 |
| abstract_inverted_index.shorter | 42 |
| abstract_inverted_index.solving | 19 |
| abstract_inverted_index.through | 24 |
| abstract_inverted_index.benefits | 120 |
| abstract_inverted_index.changes, | 126 |
| abstract_inverted_index.chatbots | 54, 72 |
| abstract_inverted_index.content. | 143 |
| abstract_inverted_index.engaging | 79 |
| abstract_inverted_index.language | 10 |
| abstract_inverted_index.problems | 20 |
| abstract_inverted_index.quality? | 46 |
| abstract_inverted_index.conducted | 65 |
| abstract_inverted_index.developed | 51 |
| abstract_inverted_index.dialogues | 81 |
| abstract_inverted_index.different | 101 |
| abstract_inverted_index.potential | 119 |
| abstract_inverted_index.question: | 32 |
| abstract_inverted_index.resulting | 123 |
| abstract_inverted_index.intriguing | 30 |
| abstract_inverted_index.questions, | 78 |
| abstract_inverted_index.reactions. | 107 |
| abstract_inverted_index.conditions, | 102 |
| abstract_inverted_index.conversable | 77 |
| abstract_inverted_index.drastically | 99 |
| abstract_inverted_index.information | 23 |
| abstract_inverted_index.instructing | 35 |
| abstract_inverted_index.necessarily | 137 |
| abstract_inverted_index.Participants | 69 |
| abstract_inverted_index.conversation | 45, 62, 94, 115 |
| abstract_inverted_index.demonstrates | 110 |
| abstract_inverted_index.participants | 104 |
| abstract_inverted_index.conversations | 43 |
| abstract_inverted_index.conversational | 88 |
| abstract_inverted_index.conversations. | 26 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 5 |
| citation_normalized_percentile |