Beyond the Desktop: XR-Driven Segmentation with Meta Quest 3 and MX Ink Article Swipe

PDF

Lisle Faray de Paiva , Gijs Luijten , A. S. Santos , Moon Kim , Behrus Puladi , Jens Kleesiek ·

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2506.04858

Medical imaging segmentation is essential in clinical settings for diagnosing diseases, planning surgeries, and other procedures. However, manual annotation is a cumbersome and effortful task. To mitigate these aspects, this study implements and evaluates the usability and clinical applicability of an extended reality (XR)-based segmentation tool for anatomical CT scans, using the Meta Quest 3 headset and Logitech MX Ink stylus. We develop an immersive interface enabling real-time interaction with 2D and 3D medical imaging data in a customizable workspace designed to mitigate workflow fragmentation and cognitive demands inherent to conventional manual segmentation tools. The platform combines stylus-driven annotation, mirroring traditional pen-on-paper workflows, with instant 3D volumetric rendering. A user study with a public craniofacial CT dataset demonstrated the tool's foundational viability, achieving a System Usability Scale (SUS) score of 66, within the expected range for medical applications. Participants highlighted the system's intuitive controls (scoring 4.1/5 for self-descriptiveness on ISONORM metrics) and spatial interaction design, with qualitative feedback highlighting strengths in hybrid 2D/3D navigation and realistic stylus ergonomics. While users identified opportunities to enhance task-specific precision and error management, the platform's core workflow enabled dynamic slice adjustment, reducing cognitive load compared to desktop tools. Results position the XR-stylus paradigm as a promising foundation for immersive segmentation tools, with iterative refinements targeting haptic feedback calibration and workflow personalization to advance adoption in preoperative planning.

Related Topics

The Place Beyond The Pines

Just Beyond

Criminal Minds: Beyond Borders Season 1

Beyond Paradise (Tv Series)

Beyond The Gates (Tv Series)

Desktop Computer

Mad Max Beyond Thunderdome

Beyond The Sea (Black Mirror)

V/H/S/Beyond

Beyond (Band)

Beyond Good & Evil (Video Game)

Beyond Good And Evil

Concepts

No concepts available.

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2506.04858
PDF: https://arxiv.org/pdf/2506.04858
OA Status: green
OpenAlex ID: https://openalex.org/W4416133883

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4416133883

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2506.04858

Digital Object Identifier
Title: Beyond the Desktop: XR-Driven Segmentation with Meta Quest 3 and MX Ink

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-06-05

Full publication date if available
Authors: Lisle Faray de Paiva, Gijs Luijten, A. S. Santos, Moon Kim, Behrus Puladi, Jens Kleesiek

List of authors in order
Landing page: https://arxiv.org/abs/2506.04858

Publisher landing page
PDF URL: https://arxiv.org/pdf/2506.04858

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2506.04858

Direct OA link when available
Cited by: 0

Total citation count in OpenAlex

Full payload

id	https://openalex.org/W4416133883
doi	https://doi.org/10.48550/arxiv.2506.04858
ids.doi	https://doi.org/10.48550/arxiv.2506.04858
ids.openalex	https://openalex.org/W4416133883
fwci
type	preprint
title	Beyond the Desktop: XR-Driven Segmentation with Meta Quest 3 and MX Ink
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
is_xpac	False
apc_list
apc_paid
language	en
locations[0].id	pmh:oai:arXiv.org:2506.04858
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license	cc-by-nc-sa
locations[0].pdf_url	https://arxiv.org/pdf/2506.04858
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id	https://openalex.org/licenses/cc-by-nc-sa
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2506.04858
locations[1].id	doi:10.48550/arxiv.2506.04858
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2506.04858
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5116235176
authorships[0].author.orcid
authorships[0].author.display_name	Lisle Faray de Paiva
authorships[0].author_position	first
authorships[0].raw_author_name	de Paiva, Lisle Faray
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5065432808
authorships[1].author.orcid	https://orcid.org/0009-0000-3404-1917
authorships[1].author.display_name	Gijs Luijten
authorships[1].author_position	middle
authorships[1].raw_author_name	Luijten, Gijs
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5110704027
authorships[2].author.orcid
authorships[2].author.display_name	A. S. Santos
authorships[2].author_position	middle
authorships[2].raw_author_name	Santos, Ana Sofia Ferreira
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5055346551
authorships[3].author.orcid	https://orcid.org/0000-0002-7546-3910
authorships[3].author.display_name	Moon Kim
authorships[3].author_position	middle
authorships[3].raw_author_name	Kim, Moon
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5034899775
authorships[4].author.orcid	https://orcid.org/0000-0001-5909-6105
authorships[4].author.display_name	Behrus Puladi
authorships[4].author_position	middle
authorships[4].raw_author_name	Puladi, Behrus
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5017161970
authorships[5].author.orcid	https://orcid.org/0000-0001-8686-0682
authorships[5].author.display_name	Jens Kleesiek
authorships[5].author_position	last
authorships[5].raw_author_name	Kleesiek, Jens
authorships[5].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2506.04858
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Beyond the Desktop: XR-Driven Segmentation with Meta Quest 3 and MX Ink
has_fulltext	False
is_retracted	False
updated_date	2025-11-28T05:57:33.614661
primary_topic
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2506.04858
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license	cc-by-nc-sa
best_oa_location.pdf_url	https://arxiv.org/pdf/2506.04858
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id	https://openalex.org/licenses/cc-by-nc-sa
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2506.04858
primary_location.id	pmh:oai:arXiv.org:2506.04858
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license	cc-by-nc-sa
primary_location.pdf_url	https://arxiv.org/pdf/2506.04858
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id	https://openalex.org/licenses/cc-by-nc-sa
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2506.04858
publication_date	2025-06-05
publication_year	2025
referenced_works_count	0
abstract_inverted_index.3	54
abstract_inverted_index.A	108
abstract_inverted_index.a	20, 77, 112, 123, 200
abstract_inverted_index.2D	70
abstract_inverted_index.3D	72, 105
abstract_inverted_index.CT	48, 115
abstract_inverted_index.MX	58
abstract_inverted_index.To	25
abstract_inverted_index.We	61
abstract_inverted_index.an	40, 63
abstract_inverted_index.as	199
abstract_inverted_index.in	5, 76, 160, 220
abstract_inverted_index.is	3, 19
abstract_inverted_index.of	39, 129
abstract_inverted_index.on	148
abstract_inverted_index.to	81, 89, 172, 191, 217
abstract_inverted_index.66,	130
abstract_inverted_index.Ink	59
abstract_inverted_index.The	94
abstract_inverted_index.and	13, 22, 32, 36, 56, 71, 85, 151, 164, 176, 214
abstract_inverted_index.for	8, 46, 135, 146, 203
abstract_inverted_index.the	34, 51, 118, 132, 140, 179, 196
abstract_inverted_index.Meta	52
abstract_inverted_index.core	181
abstract_inverted_index.data	75
abstract_inverted_index.load	189
abstract_inverted_index.this	29
abstract_inverted_index.tool	45
abstract_inverted_index.user	109
abstract_inverted_index.with	69, 103, 111, 155, 207
abstract_inverted_index.(SUS)	127
abstract_inverted_index.2D/3D	162
abstract_inverted_index.4.1/5	145
abstract_inverted_index.Quest	53
abstract_inverted_index.Scale	126
abstract_inverted_index.While	168
abstract_inverted_index.error	177
abstract_inverted_index.other	14
abstract_inverted_index.range	134
abstract_inverted_index.score	128
abstract_inverted_index.slice	185
abstract_inverted_index.study	30, 110
abstract_inverted_index.task.	24
abstract_inverted_index.these	27
abstract_inverted_index.users	169
abstract_inverted_index.using	50
abstract_inverted_index.System	124
abstract_inverted_index.haptic	211
abstract_inverted_index.hybrid	161
abstract_inverted_index.manual	17, 91
abstract_inverted_index.public	113
abstract_inverted_index.scans,	49
abstract_inverted_index.stylus	166
abstract_inverted_index.tool's	119
abstract_inverted_index.tools,	206
abstract_inverted_index.tools.	93, 193
abstract_inverted_index.within	131
abstract_inverted_index.ISONORM	149
abstract_inverted_index.Medical	0
abstract_inverted_index.Results	194
abstract_inverted_index.advance	218
abstract_inverted_index.dataset	116
abstract_inverted_index.demands	87
abstract_inverted_index.design,	154
abstract_inverted_index.desktop	192
abstract_inverted_index.develop	62
abstract_inverted_index.dynamic	184
abstract_inverted_index.enabled	183
abstract_inverted_index.enhance	173
abstract_inverted_index.headset	55
abstract_inverted_index.imaging	1, 74
abstract_inverted_index.instant	104
abstract_inverted_index.medical	73, 136
abstract_inverted_index.reality	42
abstract_inverted_index.spatial	152
abstract_inverted_index.stylus.	60
abstract_inverted_index.(scoring	144
abstract_inverted_index.However,	16
abstract_inverted_index.Logitech	57
abstract_inverted_index.adoption	219
abstract_inverted_index.aspects,	28
abstract_inverted_index.clinical	6, 37
abstract_inverted_index.combines	96
abstract_inverted_index.compared	190
abstract_inverted_index.controls	143
abstract_inverted_index.designed	80
abstract_inverted_index.enabling	66
abstract_inverted_index.expected	133
abstract_inverted_index.extended	41
abstract_inverted_index.feedback	157, 212
abstract_inverted_index.inherent	88
abstract_inverted_index.metrics)	150
abstract_inverted_index.mitigate	26, 82
abstract_inverted_index.paradigm	198
abstract_inverted_index.planning	11
abstract_inverted_index.platform	95
abstract_inverted_index.position	195
abstract_inverted_index.reducing	187
abstract_inverted_index.settings	7
abstract_inverted_index.system's	141
abstract_inverted_index.workflow	83, 182, 215
abstract_inverted_index.Usability	125
abstract_inverted_index.XR-stylus	197
abstract_inverted_index.achieving	122
abstract_inverted_index.cognitive	86, 188
abstract_inverted_index.diseases,	10
abstract_inverted_index.effortful	23
abstract_inverted_index.essential	4
abstract_inverted_index.evaluates	33
abstract_inverted_index.immersive	64, 204
abstract_inverted_index.interface	65
abstract_inverted_index.intuitive	142
abstract_inverted_index.iterative	208
abstract_inverted_index.mirroring	99
abstract_inverted_index.planning.	222
abstract_inverted_index.precision	175
abstract_inverted_index.promising	201
abstract_inverted_index.real-time	67
abstract_inverted_index.realistic	165
abstract_inverted_index.strengths	159
abstract_inverted_index.targeting	210
abstract_inverted_index.usability	35
abstract_inverted_index.workspace	79
abstract_inverted_index.(XR)-based	43
abstract_inverted_index.anatomical	47
abstract_inverted_index.annotation	18
abstract_inverted_index.cumbersome	21
abstract_inverted_index.diagnosing	9
abstract_inverted_index.foundation	202
abstract_inverted_index.identified	170
abstract_inverted_index.implements	31
abstract_inverted_index.navigation	163
abstract_inverted_index.platform's	180
abstract_inverted_index.rendering.	107
abstract_inverted_index.surgeries,	12
abstract_inverted_index.viability,	121
abstract_inverted_index.volumetric	106
abstract_inverted_index.workflows,	102
abstract_inverted_index.adjustment,	186
abstract_inverted_index.annotation,	98
abstract_inverted_index.calibration	213
abstract_inverted_index.ergonomics.	167
abstract_inverted_index.highlighted	139
abstract_inverted_index.interaction	68, 153
abstract_inverted_index.management,	178
abstract_inverted_index.procedures.	15
abstract_inverted_index.qualitative	156
abstract_inverted_index.refinements	209
abstract_inverted_index.traditional	100
abstract_inverted_index.Participants	138
abstract_inverted_index.conventional	90
abstract_inverted_index.craniofacial	114
abstract_inverted_index.customizable	78
abstract_inverted_index.demonstrated	117
abstract_inverted_index.foundational	120
abstract_inverted_index.highlighting	158
abstract_inverted_index.pen-on-paper	101
abstract_inverted_index.preoperative	221
abstract_inverted_index.segmentation	2, 44, 92, 205
abstract_inverted_index.applicability	38
abstract_inverted_index.applications.	137
abstract_inverted_index.fragmentation	84
abstract_inverted_index.opportunities	171
abstract_inverted_index.stylus-driven	97
abstract_inverted_index.task-specific	174
abstract_inverted_index.personalization	216
abstract_inverted_index.self-descriptiveness	147
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	6
citation_normalized_percentile