Feeling Textiles through AI: An exploration into Multimodal Language Models and Human Perception Alignment Article Swipe
Shu Zhong
,
Elia Gatti
,
Youngjun Cho
,
Marianna Obrist
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.1145/3678957.3685756
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.1145/3678957.3685756
Human-artificial intelligence (AI) alignment ensures that AI systems align with human goals and behaviors. This paper introduces \nperceptual alignment as a critical aspect of this alignment, focusing \non the concurrence between human judgments and AI evaluations \nacross sensory modalities. We particularly explore how Multimodal \nLarge Language Models (MLLMs), which process both visual and \ntextual data, interpret the tactile qualities of textiles—a significant \nchallenge in online shopping environments. Our research analyzes \nsix vision-based MLLMs to see how they describe the tactile experience of textiles and compares these AI-generated descriptions \nwith human assessments. Through semantic similarity measures \nand in-person evaluations, we investigate the extent of alignment \nbetween human perceptions and AI descriptions. Our findings indicate significant variability in the AI’s ability to interpret different \ntextiles, highlighting both the potential and limitations of current \nAI models in achieving perceptual alignment. This work contributes \nto understanding the complexities of aligning AI capabilities with \nhuman touch sensory experiences.
Related Topics
Concepts
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://doi.org/10.1145/3678957.3685756
- OA Status
- gold
- References
- 9
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4403913334
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4403913334Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.1145/3678957.3685756Digital Object Identifier
- Title
-
Feeling Textiles through AI: An exploration into Multimodal Language Models and Human Perception AlignmentWork title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-10-30Full publication date if available
- Authors
-
Shu Zhong, Elia Gatti, Youngjun Cho, Marianna ObristList of authors in order
- Landing page
-
https://doi.org/10.1145/3678957.3685756Publisher landing page
- Open access
-
YesWhether a free full text is available
- OA status
-
goldOpen access status per OpenAlex
- OA URL
-
https://doi.org/10.1145/3678957.3685756Direct OA link when available
- Concepts
-
Feeling, Perception, Computer science, Artificial intelligence, Human–computer interaction, Cognitive science, Natural language processing, Psychology, Neuroscience, Social psychologyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- References (count)
-
9Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4403913334 |
|---|---|
| doi | https://doi.org/10.1145/3678957.3685756 |
| ids.doi | https://doi.org/10.1145/3678957.3685756 |
| ids.openalex | https://openalex.org/W4403913334 |
| fwci | 0.0 |
| type | article |
| title | Feeling Textiles through AI: An exploration into Multimodal Language Models and Human Perception Alignment |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | 37 |
| biblio.first_page | 33 |
| topics[0].id | https://openalex.org/T12496 |
| topics[0].field.id | https://openalex.org/fields/32 |
| topics[0].field.display_name | Psychology |
| topics[0].score | 0.9959999918937683 |
| topics[0].domain.id | https://openalex.org/domains/2 |
| topics[0].domain.display_name | Social Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/3207 |
| topics[0].subfield.display_name | Social Psychology |
| topics[0].display_name | Color perception and design |
| topics[1].id | https://openalex.org/T12650 |
| topics[1].field.id | https://openalex.org/fields/28 |
| topics[1].field.display_name | Neuroscience |
| topics[1].score | 0.9685999751091003 |
| topics[1].domain.id | https://openalex.org/domains/1 |
| topics[1].domain.display_name | Life Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2805 |
| topics[1].subfield.display_name | Cognitive Neuroscience |
| topics[1].display_name | Aesthetic Perception and Analysis |
| topics[2].id | https://openalex.org/T12032 |
| topics[2].field.id | https://openalex.org/fields/32 |
| topics[2].field.display_name | Psychology |
| topics[2].score | 0.9628999829292297 |
| topics[2].domain.id | https://openalex.org/domains/2 |
| topics[2].domain.display_name | Social Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/3205 |
| topics[2].subfield.display_name | Experimental and Cognitive Psychology |
| topics[2].display_name | Multisensory perception and integration |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C122980154 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8652253150939941 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q205555 |
| concepts[0].display_name | Feeling |
| concepts[1].id | https://openalex.org/C26760741 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6617452502250671 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q160402 |
| concepts[1].display_name | Perception |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.6332523226737976 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C154945302 |
| concepts[3].level | 1 |
| concepts[3].score | 0.48573046922683716 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[3].display_name | Artificial intelligence |
| concepts[4].id | https://openalex.org/C107457646 |
| concepts[4].level | 1 |
| concepts[4].score | 0.42752018570899963 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q207434 |
| concepts[4].display_name | Human–computer interaction |
| concepts[5].id | https://openalex.org/C188147891 |
| concepts[5].level | 1 |
| concepts[5].score | 0.40926575660705566 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q147638 |
| concepts[5].display_name | Cognitive science |
| concepts[6].id | https://openalex.org/C204321447 |
| concepts[6].level | 1 |
| concepts[6].score | 0.35824257135391235 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[6].display_name | Natural language processing |
| concepts[7].id | https://openalex.org/C15744967 |
| concepts[7].level | 0 |
| concepts[7].score | 0.24142423272132874 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[7].display_name | Psychology |
| concepts[8].id | https://openalex.org/C169760540 |
| concepts[8].level | 1 |
| concepts[8].score | 0.061973243951797485 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q207011 |
| concepts[8].display_name | Neuroscience |
| concepts[9].id | https://openalex.org/C77805123 |
| concepts[9].level | 1 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q161272 |
| concepts[9].display_name | Social psychology |
| keywords[0].id | https://openalex.org/keywords/feeling |
| keywords[0].score | 0.8652253150939941 |
| keywords[0].display_name | Feeling |
| keywords[1].id | https://openalex.org/keywords/perception |
| keywords[1].score | 0.6617452502250671 |
| keywords[1].display_name | Perception |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.6332523226737976 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[3].score | 0.48573046922683716 |
| keywords[3].display_name | Artificial intelligence |
| keywords[4].id | https://openalex.org/keywords/human–computer-interaction |
| keywords[4].score | 0.42752018570899963 |
| keywords[4].display_name | Human–computer interaction |
| keywords[5].id | https://openalex.org/keywords/cognitive-science |
| keywords[5].score | 0.40926575660705566 |
| keywords[5].display_name | Cognitive science |
| keywords[6].id | https://openalex.org/keywords/natural-language-processing |
| keywords[6].score | 0.35824257135391235 |
| keywords[6].display_name | Natural language processing |
| keywords[7].id | https://openalex.org/keywords/psychology |
| keywords[7].score | 0.24142423272132874 |
| keywords[7].display_name | Psychology |
| keywords[8].id | https://openalex.org/keywords/neuroscience |
| keywords[8].score | 0.061973243951797485 |
| keywords[8].display_name | Neuroscience |
| language | en |
| locations[0].id | doi:10.1145/3678957.3685756 |
| locations[0].is_oa | True |
| locations[0].source | |
| locations[0].license | cc-by |
| locations[0].pdf_url | |
| locations[0].version | publishedVersion |
| locations[0].raw_type | proceedings-article |
| locations[0].license_id | https://openalex.org/licenses/cc-by |
| locations[0].is_accepted | True |
| locations[0].is_published | True |
| locations[0].raw_source_name | International Conference on Multimodel Interaction |
| locations[0].landing_page_url | https://doi.org/10.1145/3678957.3685756 |
| locations[1].id | pmh:oai:eprints.ucl.ac.uk.OAI2:10196394 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400024 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | False |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | UCL Discovery (University College London) |
| locations[1].source.host_organization | https://openalex.org/I45129253 |
| locations[1].source.host_organization_name | University College London |
| locations[1].source.host_organization_lineage | https://openalex.org/I45129253 |
| locations[1].license | |
| locations[1].pdf_url | https://discovery.ucl.ac.uk/10196394/2/Zhong_icmi24-59.pdf |
| locations[1].version | submittedVersion |
| locations[1].raw_type | Proceedings paper |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | False |
| locations[1].raw_source_name | In: Proceedings of the 26th ACM International Conference on Multimodal Interaction. ACM: San Jose, Costa Rica. (2024) (In press). |
| locations[1].landing_page_url | https://discovery.ucl.ac.uk/id/eprint/10196394/ |
| indexed_in | crossref |
| authorships[0].author.id | https://openalex.org/A5012790743 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-1820-6424 |
| authorships[0].author.display_name | Shu Zhong |
| authorships[0].countries | GB |
| authorships[0].affiliations[0].institution_ids | https://openalex.org/I45129253 |
| authorships[0].affiliations[0].raw_affiliation_string | Computer Science, University College London, United Kingdom |
| authorships[0].institutions[0].id | https://openalex.org/I45129253 |
| authorships[0].institutions[0].ror | https://ror.org/02jx3x895 |
| authorships[0].institutions[0].type | education |
| authorships[0].institutions[0].lineage | https://openalex.org/I124357947, https://openalex.org/I45129253 |
| authorships[0].institutions[0].country_code | GB |
| authorships[0].institutions[0].display_name | University College London |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Shu Zhong |
| authorships[0].is_corresponding | False |
| authorships[0].raw_affiliation_strings | Computer Science, University College London, United Kingdom |
| authorships[1].author.id | https://openalex.org/A5066364876 |
| authorships[1].author.orcid | https://orcid.org/0009-0008-1675-1323 |
| authorships[1].author.display_name | Elia Gatti |
| authorships[1].countries | GB |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I45129253 |
| authorships[1].affiliations[0].raw_affiliation_string | Computer Science, University College London, United Kingdom |
| authorships[1].institutions[0].id | https://openalex.org/I45129253 |
| authorships[1].institutions[0].ror | https://ror.org/02jx3x895 |
| authorships[1].institutions[0].type | education |
| authorships[1].institutions[0].lineage | https://openalex.org/I124357947, https://openalex.org/I45129253 |
| authorships[1].institutions[0].country_code | GB |
| authorships[1].institutions[0].display_name | University College London |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Elia Gatti |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | Computer Science, University College London, United Kingdom |
| authorships[2].author.id | https://openalex.org/A5069669556 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-5695-0759 |
| authorships[2].author.display_name | Youngjun Cho |
| authorships[2].countries | GB |
| authorships[2].affiliations[0].institution_ids | https://openalex.org/I45129253 |
| authorships[2].affiliations[0].raw_affiliation_string | Computer Science, University College London, United Kingdom |
| authorships[2].institutions[0].id | https://openalex.org/I45129253 |
| authorships[2].institutions[0].ror | https://ror.org/02jx3x895 |
| authorships[2].institutions[0].type | education |
| authorships[2].institutions[0].lineage | https://openalex.org/I124357947, https://openalex.org/I45129253 |
| authorships[2].institutions[0].country_code | GB |
| authorships[2].institutions[0].display_name | University College London |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Youngjun Cho |
| authorships[2].is_corresponding | False |
| authorships[2].raw_affiliation_strings | Computer Science, University College London, United Kingdom |
| authorships[3].author.id | https://openalex.org/A5090754516 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-4009-1627 |
| authorships[3].author.display_name | Marianna Obrist |
| authorships[3].countries | GB |
| authorships[3].affiliations[0].institution_ids | https://openalex.org/I45129253 |
| authorships[3].affiliations[0].raw_affiliation_string | Computer Science, University College London, United Kingdom |
| authorships[3].institutions[0].id | https://openalex.org/I45129253 |
| authorships[3].institutions[0].ror | https://ror.org/02jx3x895 |
| authorships[3].institutions[0].type | education |
| authorships[3].institutions[0].lineage | https://openalex.org/I124357947, https://openalex.org/I45129253 |
| authorships[3].institutions[0].country_code | GB |
| authorships[3].institutions[0].display_name | University College London |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Marianna Obrist |
| authorships[3].is_corresponding | False |
| authorships[3].raw_affiliation_strings | Computer Science, University College London, United Kingdom |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://doi.org/10.1145/3678957.3685756 |
| open_access.oa_status | gold |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Feeling Textiles through AI: An exploration into Multimodal Language Models and Human Perception Alignment |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T12496 |
| primary_topic.field.id | https://openalex.org/fields/32 |
| primary_topic.field.display_name | Psychology |
| primary_topic.score | 0.9959999918937683 |
| primary_topic.domain.id | https://openalex.org/domains/2 |
| primary_topic.domain.display_name | Social Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/3207 |
| primary_topic.subfield.display_name | Social Psychology |
| primary_topic.display_name | Color perception and design |
| related_works | https://openalex.org/W2291261743, https://openalex.org/W1540119434, https://openalex.org/W4229598134, https://openalex.org/W2335262298, https://openalex.org/W2765982327, https://openalex.org/W2071657884, https://openalex.org/W213396585, https://openalex.org/W4237671428, https://openalex.org/W2068152137, https://openalex.org/W2358660055 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | doi:10.1145/3678957.3685756 |
| best_oa_location.is_oa | True |
| best_oa_location.source | |
| best_oa_location.license | cc-by |
| best_oa_location.pdf_url | |
| best_oa_location.version | publishedVersion |
| best_oa_location.raw_type | proceedings-article |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by |
| best_oa_location.is_accepted | True |
| best_oa_location.is_published | True |
| best_oa_location.raw_source_name | International Conference on Multimodel Interaction |
| best_oa_location.landing_page_url | https://doi.org/10.1145/3678957.3685756 |
| primary_location.id | doi:10.1145/3678957.3685756 |
| primary_location.is_oa | True |
| primary_location.source | |
| primary_location.license | cc-by |
| primary_location.pdf_url | |
| primary_location.version | publishedVersion |
| primary_location.raw_type | proceedings-article |
| primary_location.license_id | https://openalex.org/licenses/cc-by |
| primary_location.is_accepted | True |
| primary_location.is_published | True |
| primary_location.raw_source_name | International Conference on Multimodel Interaction |
| primary_location.landing_page_url | https://doi.org/10.1145/3678957.3685756 |
| publication_date | 2024-10-30 |
| publication_year | 2024 |
| referenced_works | https://openalex.org/W6778883912, https://openalex.org/W4309618884, https://openalex.org/W1550358375, https://openalex.org/W3047544760, https://openalex.org/W3097599689, https://openalex.org/W4226278401, https://openalex.org/W1577704899, https://openalex.org/W610473259, https://openalex.org/W4300011764 |
| referenced_works_count | 9 |
| abstract_inverted_index.a | 19 |
| abstract_inverted_index.AI | 6, 32, 98, 133 |
| abstract_inverted_index.We | 36 |
| abstract_inverted_index.as | 18 |
| abstract_inverted_index.in | 57, 105, 121 |
| abstract_inverted_index.of | 22, 54, 74, 93, 118, 131 |
| abstract_inverted_index.to | 66, 109 |
| abstract_inverted_index.we | 89 |
| abstract_inverted_index.Our | 61, 100 |
| abstract_inverted_index.and | 12, 31, 76, 97, 116 |
| abstract_inverted_index.how | 39, 68 |
| abstract_inverted_index.see | 67 |
| abstract_inverted_index.the | 26, 51, 71, 91, 106, 114, 129 |
| abstract_inverted_index.(AI) | 2 |
| abstract_inverted_index.This | 14, 125 |
| abstract_inverted_index.both | 46, 113 |
| abstract_inverted_index.that | 5 |
| abstract_inverted_index.they | 69 |
| abstract_inverted_index.this | 23 |
| abstract_inverted_index.with | 9 |
| abstract_inverted_index.work | 126 |
| abstract_inverted_index.MLLMs | 65 |
| abstract_inverted_index.align | 8 |
| abstract_inverted_index.data, | 49 |
| abstract_inverted_index.goals | 11 |
| abstract_inverted_index.human | 10, 29, 81, 95 |
| abstract_inverted_index.paper | 15 |
| abstract_inverted_index.these | 78 |
| abstract_inverted_index.touch | 136 |
| abstract_inverted_index.which | 44 |
| abstract_inverted_index.AI’s | 107 |
| abstract_inverted_index.Models | 42 |
| abstract_inverted_index.aspect | 21 |
| abstract_inverted_index.extent | 92 |
| abstract_inverted_index.models | 120 |
| abstract_inverted_index.online | 58 |
| abstract_inverted_index.visual | 47 |
| abstract_inverted_index.Through | 83 |
| abstract_inverted_index.ability | 108 |
| abstract_inverted_index.between | 28 |
| abstract_inverted_index.ensures | 4 |
| abstract_inverted_index.explore | 38 |
| abstract_inverted_index.process | 45 |
| abstract_inverted_index.sensory | 34, 137 |
| abstract_inverted_index.systems | 7 |
| abstract_inverted_index.tactile | 52, 72 |
| abstract_inverted_index.(MLLMs), | 43 |
| abstract_inverted_index.Language | 41 |
| abstract_inverted_index.aligning | 132 |
| abstract_inverted_index.compares | 77 |
| abstract_inverted_index.critical | 20 |
| abstract_inverted_index.describe | 70 |
| abstract_inverted_index.findings | 101 |
| abstract_inverted_index.indicate | 102 |
| abstract_inverted_index.research | 62 |
| abstract_inverted_index.semantic | 84 |
| abstract_inverted_index.shopping | 59 |
| abstract_inverted_index.textiles | 75 |
| abstract_inverted_index.achieving | 122 |
| abstract_inverted_index.alignment | 3, 17 |
| abstract_inverted_index.in-person | 87 |
| abstract_inverted_index.interpret | 50, 110 |
| abstract_inverted_index.judgments | 30 |
| abstract_inverted_index.potential | 115 |
| abstract_inverted_index.qualities | 53 |
| abstract_inverted_index.alignment, | 24 |
| abstract_inverted_index.alignment. | 124 |
| abstract_inverted_index.behaviors. | 13 |
| abstract_inverted_index.experience | 73 |
| abstract_inverted_index.perceptual | 123 |
| abstract_inverted_index.similarity | 85 |
| abstract_inverted_index.concurrence | 27 |
| abstract_inverted_index.investigate | 90 |
| abstract_inverted_index.limitations | 117 |
| abstract_inverted_index.modalities. | 35 |
| abstract_inverted_index.perceptions | 96 |
| abstract_inverted_index.significant | 103 |
| abstract_inverted_index.variability | 104 |
| abstract_inverted_index.AI-generated | 79 |
| abstract_inverted_index.assessments. | 82 |
| abstract_inverted_index.capabilities | 134 |
| abstract_inverted_index.complexities | 130 |
| abstract_inverted_index.evaluations, | 88 |
| abstract_inverted_index.experiences. | 138 |
| abstract_inverted_index.highlighting | 112 |
| abstract_inverted_index.intelligence | 1 |
| abstract_inverted_index.particularly | 37 |
| abstract_inverted_index.textiles—a | 55 |
| abstract_inverted_index.vision-based | 64 |
| abstract_inverted_index.descriptions. | 99 |
| abstract_inverted_index.environments. | 60 |
| abstract_inverted_index.understanding | 128 |
| abstract_inverted_index.Human-artificial | 0 |
| abstract_inverted_index.current \nAI | 119 |
| abstract_inverted_index.with \nhuman | 135 |
| abstract_inverted_index.and \ntextual | 48 |
| abstract_inverted_index.focusing \non | 25 |
| abstract_inverted_index.analyzes \nsix | 63 |
| abstract_inverted_index.measures \nand | 86 |
| abstract_inverted_index.contributes \nto | 127 |
| abstract_inverted_index.Multimodal \nLarge | 40 |
| abstract_inverted_index.alignment \nbetween | 94 |
| abstract_inverted_index.descriptions \nwith | 80 |
| abstract_inverted_index.evaluations \nacross | 33 |
| abstract_inverted_index.different \ntextiles, | 111 |
| abstract_inverted_index.introduces \nperceptual | 16 |
| abstract_inverted_index.significant \nchallenge | 56 |
| cited_by_percentile_year | |
| countries_distinct_count | 1 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile.value | 0.28971324 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |