Evaluating the Usefulness of Translation Technologies for Emergency Response Communication: A Scenario-Based Study (Preprint) Article Swipe

View

Anne M. Turner , Yong Kyung Choi , Kristin Dew , Ming-Tse Tsai , Alyssa Bosold , Esther Wu , Donahue Smith , Hendrika Meischke ·

YOU? · · 2018 · Open Access · · DOI: https://doi.org/10.2196/preprints.11171

BACKGROUND In the United States, language barriers pose challenges to communication in emergency response and impact emergency care delivery and quality for individuals who are limited English proficient (LEP). There is a growing interest among Emergency Medical Services (EMS) personnel in using automated translation tools to improve communications with LEP individuals in the field. However, little is known about whether automated translation software can be used successfully in EMS settings to improve communication with LEP individuals. OBJECTIVE The objective of this work is to use scenario-based methods with EMS providers and nonnative English-speaking users who identified themselves as LEP (henceforth referred to as LEP participants) to evaluate the potential of two automated translation technologies in improving emergency communication. METHODS We developed mock emergency scenarios and enacted them in simulation sessions with EMS personnel and Spanish-speaking and Chinese-speaking (Mandarin) LEP participants using two automated language translation tools: an EMS domain-specific fixed-sentence translation tool (QuickSpeak) and a statistical machine translation tool (Google Translate). At the end of the sessions, we gathered feedback from both groups through a postsession questionnaire. EMS participants also completed the System Usability Scale (SUS). RESULTS We conducted a total of 5 group sessions (3 Chinese and 2 Spanish) with 12 Chinese-speaking LEP participants, 14 Spanish-speaking LEP participants, and 17 EMS personnel. Overall, communications between EMS and LEP participants remained limited, even with the use of the two translation tools. QuickSpeak had higher mean SUS scores than Google Translate (65.3 vs 48.4; P=.04). Although both tools were deemed less than satisfactory, LEP participants showed preference toward the domain-specific system with fixed questions (QuickSpeak) over the free-text translation tool (Google Translate) in terms of understanding the EMS personnel’s questions (Chinese 11/12, 92% vs 3/12, 25%; Spanish 12/14, 86% vs 4/14, 29%). While both EMS and LEP participants appreciated the flexibility of the free-text tool, multiple translation errors and difficulty responding to questions limited its usefulness. CONCLUSIONS Technologies are emerging that have the potential to assist with language translation in emergency response; however, improvements in accuracy and usability are needed before these technologies can be used safely in the field.

Related Topics

Human–Computer Interaction

Philosophy

Concepts

Usability Machine translation Language barrier Computer science Language translation Natural language processing Human–computer interaction Linguistics Philosophy

Metadata

Type: preprint
Language: en
Landing Page: https://doi.org/10.2196/preprints.11171
OA Status: gold
References: 13
Related Works: 10
OpenAlex ID: https://openalex.org/W4249598644

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4249598644

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.2196/preprints.11171

Digital Object Identifier
Title: Evaluating the Usefulness of Translation Technologies for Emergency Response Communication: A Scenario-Based Study (Preprint)

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2018

Year of publication
Publication date: 2018-05-31

Full publication date if available
Authors: Anne M. Turner, Yong Kyung Choi, Kristin Dew, Ming-Tse Tsai, Alyssa Bosold, Esther Wu, Donahue Smith, Hendrika Meischke

List of authors in order
Landing page: https://doi.org/10.2196/preprints.11171

Publisher landing page
Open access: Yes

Whether a free full text is available
OA status: gold

Open access status per OpenAlex
OA URL: https://doi.org/10.2196/preprints.11171

Direct OA link when available
Concepts: Usability, Machine translation, Language barrier, Computer science, Language translation, Natural language processing, Human–computer interaction, Linguistics, Philosophy

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
References (count): 13

Number of works referenced by this work
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4249598644
doi	https://doi.org/10.2196/preprints.11171
ids.doi	https://doi.org/10.2196/preprints.11171
ids.openalex	https://openalex.org/W4249598644
fwci	0.0
type	preprint
title	Evaluating the Usefulness of Translation Technologies for Emergency Response Communication: A Scenario-Based Study (Preprint)
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T12151
topics[0].field.id	https://openalex.org/fields/36
topics[0].field.display_name	Health Professions
topics[0].score	0.9998000264167786
topics[0].domain.id	https://openalex.org/domains/4
topics[0].domain.display_name	Health Sciences
topics[0].subfield.id	https://openalex.org/subfields/3600
topics[0].subfield.display_name	General Health Professions
topics[0].display_name	Interpreting and Communication in Healthcare
topics[1].id	https://openalex.org/T11095
topics[1].field.id	https://openalex.org/fields/27
topics[1].field.display_name	Medicine
topics[1].score	0.9577000141143799
topics[1].domain.id	https://openalex.org/domains/4
topics[1].domain.display_name	Health Sciences
topics[1].subfield.id	https://openalex.org/subfields/2711
topics[1].subfield.display_name	Emergency Medicine
topics[1].display_name	Emergency and Acute Care Studies
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C170130773
concepts[0].level	2
concepts[0].score	0.6809834241867065
concepts[0].wikidata	https://www.wikidata.org/wiki/Q216378
concepts[0].display_name	Usability
concepts[1].id	https://openalex.org/C203005215
concepts[1].level	2
concepts[1].score	0.5989954471588135
concepts[1].wikidata	https://www.wikidata.org/wiki/Q79798
concepts[1].display_name	Machine translation
concepts[2].id	https://openalex.org/C47602998
concepts[2].level	2
concepts[2].score	0.5347152352333069
concepts[2].wikidata	https://www.wikidata.org/wiki/Q2588869
concepts[2].display_name	Language barrier
concepts[3].id	https://openalex.org/C41008148
concepts[3].level	0
concepts[3].score	0.472909539937973
concepts[3].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[3].display_name	Computer science
concepts[4].id	https://openalex.org/C2986862884
concepts[4].level	3
concepts[4].score	0.42997974157333374
concepts[4].wikidata	https://www.wikidata.org/wiki/Q7553
concepts[4].display_name	Language translation
concepts[5].id	https://openalex.org/C204321447
concepts[5].level	1
concepts[5].score	0.27440837025642395
concepts[5].wikidata	https://www.wikidata.org/wiki/Q30642
concepts[5].display_name	Natural language processing
concepts[6].id	https://openalex.org/C107457646
concepts[6].level	1
concepts[6].score	0.12863078713417053
concepts[6].wikidata	https://www.wikidata.org/wiki/Q207434
concepts[6].display_name	Human–computer interaction
concepts[7].id	https://openalex.org/C41895202
concepts[7].level	1
concepts[7].score	0.0
concepts[7].wikidata	https://www.wikidata.org/wiki/Q8162
concepts[7].display_name	Linguistics
concepts[8].id	https://openalex.org/C138885662
concepts[8].level	0
concepts[8].score	0.0
concepts[8].wikidata	https://www.wikidata.org/wiki/Q5891
concepts[8].display_name	Philosophy
keywords[0].id	https://openalex.org/keywords/usability
keywords[0].score	0.6809834241867065
keywords[0].display_name	Usability
keywords[1].id	https://openalex.org/keywords/machine-translation
keywords[1].score	0.5989954471588135
keywords[1].display_name	Machine translation
keywords[2].id	https://openalex.org/keywords/language-barrier
keywords[2].score	0.5347152352333069
keywords[2].display_name	Language barrier
keywords[3].id	https://openalex.org/keywords/computer-science
keywords[3].score	0.472909539937973
keywords[3].display_name	Computer science
keywords[4].id	https://openalex.org/keywords/language-translation
keywords[4].score	0.42997974157333374
keywords[4].display_name	Language translation
keywords[5].id	https://openalex.org/keywords/natural-language-processing
keywords[5].score	0.27440837025642395
keywords[5].display_name	Natural language processing
keywords[6].id	https://openalex.org/keywords/human–computer-interaction
keywords[6].score	0.12863078713417053
keywords[6].display_name	Human–computer interaction
language	en
locations[0].id	doi:10.2196/preprints.11171
locations[0].is_oa	True
locations[0].source
locations[0].license	cc-by
locations[0].pdf_url
locations[0].version	acceptedVersion
locations[0].raw_type	posted-content
locations[0].license_id	https://openalex.org/licenses/cc-by
locations[0].is_accepted	True
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	https://doi.org/10.2196/preprints.11171
indexed_in	crossref
authorships[0].author.id	https://openalex.org/A5031179413
authorships[0].author.orcid	https://orcid.org/0000-0002-1915-016X
authorships[0].author.display_name	Anne M. Turner
authorships[0].author_position	first
authorships[0].raw_author_name	Anne M Turner
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5054185226
authorships[1].author.orcid	https://orcid.org/0000-0001-7882-4358
authorships[1].author.display_name	Yong Kyung Choi
authorships[1].author_position	middle
authorships[1].raw_author_name	Yong K Choi
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5087314213
authorships[2].author.orcid	https://orcid.org/0000-0001-6365-9708
authorships[2].author.display_name	Kristin Dew
authorships[2].author_position	middle
authorships[2].raw_author_name	Kristin Dew
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5015411330
authorships[3].author.orcid	https://orcid.org/0000-0002-0466-6737
authorships[3].author.display_name	Ming-Tse Tsai
authorships[3].author_position	middle
authorships[3].raw_author_name	Ming-Tse Tsai
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5009836097
authorships[4].author.orcid	https://orcid.org/0000-0002-7105-2188
authorships[4].author.display_name	Alyssa Bosold
authorships[4].author_position	middle
authorships[4].raw_author_name	Alyssa L Bosold
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5088424537
authorships[5].author.orcid	https://orcid.org/0000-0003-2792-8063
authorships[5].author.display_name	Esther Wu
authorships[5].author_position	middle
authorships[5].raw_author_name	Shuyang Wu
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5041450814
authorships[6].author.orcid	https://orcid.org/0000-0002-2718-3965
authorships[6].author.display_name	Donahue Smith
authorships[6].author_position	middle
authorships[6].raw_author_name	Donahue Smith
authorships[6].is_corresponding	False
authorships[7].author.id	https://openalex.org/A5013800003
authorships[7].author.orcid	https://orcid.org/0000-0001-7195-0028
authorships[7].author.display_name	Hendrika Meischke
authorships[7].author_position	last
authorships[7].raw_author_name	Hendrika Meischke
authorships[7].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://doi.org/10.2196/preprints.11171
open_access.oa_status	gold
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Evaluating the Usefulness of Translation Technologies for Emergency Response Communication: A Scenario-Based Study (Preprint)
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T12151
primary_topic.field.id	https://openalex.org/fields/36
primary_topic.field.display_name	Health Professions
primary_topic.score	0.9998000264167786
primary_topic.domain.id	https://openalex.org/domains/4
primary_topic.domain.display_name	Health Sciences
primary_topic.subfield.id	https://openalex.org/subfields/3600
primary_topic.subfield.display_name	General Health Professions
primary_topic.display_name	Interpreting and Communication in Healthcare
related_works	https://openalex.org/W4362495644, https://openalex.org/W2134714056, https://openalex.org/W2997015083, https://openalex.org/W4321065355, https://openalex.org/W2172459317, https://openalex.org/W2892009878, https://openalex.org/W4379381477, https://openalex.org/W4293577743, https://openalex.org/W3127743011, https://openalex.org/W3208367986
cited_by_count	0
locations_count	1
best_oa_location.id	doi:10.2196/preprints.11171
best_oa_location.is_oa	True
best_oa_location.source
best_oa_location.license	cc-by
best_oa_location.pdf_url
best_oa_location.version	acceptedVersion
best_oa_location.raw_type	posted-content
best_oa_location.license_id	https://openalex.org/licenses/cc-by
best_oa_location.is_accepted	True
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	https://doi.org/10.2196/preprints.11171
primary_location.id	doi:10.2196/preprints.11171
primary_location.is_oa	True
primary_location.source
primary_location.license	cc-by
primary_location.pdf_url
primary_location.version	acceptedVersion
primary_location.raw_type	posted-content
primary_location.license_id	https://openalex.org/licenses/cc-by
primary_location.is_accepted	True
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	https://doi.org/10.2196/preprints.11171
publication_date	2018-05-31
publication_year	2018
referenced_works	https://openalex.org/W2071144678, https://openalex.org/W2051741606, https://openalex.org/W2066137044, https://openalex.org/W2272026655, https://openalex.org/W2182628044, https://openalex.org/W1972389364, https://openalex.org/W2213110262, https://openalex.org/W2147805454, https://openalex.org/W2011189417, https://openalex.org/W2754799758, https://openalex.org/W2032191478, https://openalex.org/W2162216745, https://openalex.org/W2126877592
referenced_works_count	13
abstract_inverted_index.2	205
abstract_inverted_index.5	199
abstract_inverted_index.a	32, 159, 179, 196
abstract_inverted_index.(3	202
abstract_inverted_index.12	208
abstract_inverted_index.14	212
abstract_inverted_index.17	217
abstract_inverted_index.At	166
abstract_inverted_index.In	2
abstract_inverted_index.We	124, 194
abstract_inverted_index.an	151
abstract_inverted_index.as	100, 105
abstract_inverted_index.be	65, 352
abstract_inverted_index.in	12, 41, 52, 68, 117, 132, 278, 337, 342, 355
abstract_inverted_index.is	31, 57, 85
abstract_inverted_index.of	82, 112, 169, 198, 233, 280, 307
abstract_inverted_index.to	10, 46, 71, 86, 104, 108, 317, 332
abstract_inverted_index.vs	248, 289, 295
abstract_inverted_index.we	172
abstract_inverted_index.86%	294
abstract_inverted_index.92%	288
abstract_inverted_index.EMS	69, 91, 136, 152, 182, 218, 223, 283, 300
abstract_inverted_index.LEP	50, 75, 101, 106, 143, 210, 214, 225, 259, 302
abstract_inverted_index.SUS	242
abstract_inverted_index.The	80
abstract_inverted_index.and	15, 20, 93, 129, 138, 140, 158, 204, 216, 224, 301, 314, 344
abstract_inverted_index.are	25, 326, 346
abstract_inverted_index.can	64, 351
abstract_inverted_index.end	168
abstract_inverted_index.for	22
abstract_inverted_index.had	239
abstract_inverted_index.its	320
abstract_inverted_index.the	3, 53, 110, 167, 170, 186, 231, 234, 264, 272, 282, 305, 308, 330, 356
abstract_inverted_index.two	113, 146, 235
abstract_inverted_index.use	87, 232
abstract_inverted_index.who	24, 97
abstract_inverted_index.25%;	291
abstract_inverted_index.also	184
abstract_inverted_index.both	176, 252, 299
abstract_inverted_index.care	18
abstract_inverted_index.even	229
abstract_inverted_index.from	175
abstract_inverted_index.have	329
abstract_inverted_index.less	256
abstract_inverted_index.mean	241
abstract_inverted_index.mock	126
abstract_inverted_index.over	271
abstract_inverted_index.pose	8
abstract_inverted_index.than	244, 257
abstract_inverted_index.that	328
abstract_inverted_index.them	131
abstract_inverted_index.this	83
abstract_inverted_index.tool	156, 163, 275
abstract_inverted_index.used	66, 353
abstract_inverted_index.were	254
abstract_inverted_index.with	49, 74, 90, 135, 207, 230, 267, 334
abstract_inverted_index.work	84
abstract_inverted_index.(65.3	247
abstract_inverted_index.(EMS)	39
abstract_inverted_index.29%).	297
abstract_inverted_index.3/12,	290
abstract_inverted_index.4/14,	296
abstract_inverted_index.48.4;	249
abstract_inverted_index.<sec>	0, 78, 122, 192, 323
abstract_inverted_index.Scale	189
abstract_inverted_index.There	30
abstract_inverted_index.While	298
abstract_inverted_index.about	59
abstract_inverted_index.among	35
abstract_inverted_index.fixed	268
abstract_inverted_index.group	200
abstract_inverted_index.known	58
abstract_inverted_index.terms	279
abstract_inverted_index.these	349
abstract_inverted_index.tool,	310
abstract_inverted_index.tools	45, 253
abstract_inverted_index.total	197
abstract_inverted_index.users	96
abstract_inverted_index.using	42, 145
abstract_inverted_index.(LEP).	29
abstract_inverted_index.(SUS).	190
abstract_inverted_index.11/12,	287
abstract_inverted_index.12/14,	293
abstract_inverted_index.</sec>	77, 121, 191, 322, 358
abstract_inverted_index.Google	245
abstract_inverted_index.System	187
abstract_inverted_index.United	4
abstract_inverted_index.assist	333
abstract_inverted_index.before	348
abstract_inverted_index.deemed	255
abstract_inverted_index.errors	313
abstract_inverted_index.field.	54, 357
abstract_inverted_index.groups	177
abstract_inverted_index.higher	240
abstract_inverted_index.impact	16
abstract_inverted_index.little	56
abstract_inverted_index.needed	347
abstract_inverted_index.safely	354
abstract_inverted_index.scores	243
abstract_inverted_index.showed	261
abstract_inverted_index.system	266
abstract_inverted_index.tools.	237
abstract_inverted_index.tools:	150
abstract_inverted_index.toward	263
abstract_inverted_index.(Google	164, 276
abstract_inverted_index.Chinese	203
abstract_inverted_index.English	27
abstract_inverted_index.Medical	37
abstract_inverted_index.P=.04).	250
abstract_inverted_index.Spanish	292
abstract_inverted_index.States,	5
abstract_inverted_index.between	222
abstract_inverted_index.enacted	130
abstract_inverted_index.growing	33
abstract_inverted_index.improve	47, 72
abstract_inverted_index.limited	26, 319
abstract_inverted_index.machine	161
abstract_inverted_index.methods	89
abstract_inverted_index.quality	21
abstract_inverted_index.through	178
abstract_inverted_index.whether	60
abstract_inverted_index.(Chinese	286
abstract_inverted_index.Although	251
abstract_inverted_index.However,	55
abstract_inverted_index.Overall,	220
abstract_inverted_index.Services	38
abstract_inverted_index.Spanish)	206
abstract_inverted_index.accuracy	343
abstract_inverted_index.barriers	7
abstract_inverted_index.delivery	19
abstract_inverted_index.emerging	327
abstract_inverted_index.evaluate	109
abstract_inverted_index.feedback	174
abstract_inverted_index.gathered	173
abstract_inverted_index.however,	340
abstract_inverted_index.interest	34
abstract_inverted_index.language	6, 148, 335
abstract_inverted_index.limited,	228
abstract_inverted_index.multiple	311
abstract_inverted_index.referred	103
abstract_inverted_index.remained	227
abstract_inverted_index.response	14
abstract_inverted_index.sessions	134, 201
abstract_inverted_index.settings	70
abstract_inverted_index.software	63
abstract_inverted_index.Emergency	36
abstract_inverted_index.Translate	246
abstract_inverted_index.Usability	188
abstract_inverted_index.automated	43, 61, 114, 147
abstract_inverted_index.completed	185
abstract_inverted_index.conducted	195
abstract_inverted_index.developed	125
abstract_inverted_index.emergency	13, 17, 119, 127, 338
abstract_inverted_index.free-text	273, 309
abstract_inverted_index.improving	118
abstract_inverted_index.nonnative	94
abstract_inverted_index.objective	81
abstract_inverted_index.personnel	40, 137
abstract_inverted_index.potential	111, 331
abstract_inverted_index.providers	92
abstract_inverted_index.questions	269, 285, 318
abstract_inverted_index.response;	339
abstract_inverted_index.scenarios	128
abstract_inverted_index.sessions,	171
abstract_inverted_index.usability	345
abstract_inverted_index.(Mandarin)	142
abstract_inverted_index.QuickSpeak	238
abstract_inverted_index.Translate)	277
abstract_inverted_index.challenges	9
abstract_inverted_index.difficulty	315
abstract_inverted_index.identified	98
abstract_inverted_index.personnel.	219
abstract_inverted_index.preference	262
abstract_inverted_index.proficient	28
abstract_inverted_index.responding	316
abstract_inverted_index.simulation	133
abstract_inverted_index.themselves	99
abstract_inverted_index.(henceforth	102
abstract_inverted_index.Translate).	165
abstract_inverted_index.appreciated	304
abstract_inverted_index.flexibility	306
abstract_inverted_index.individuals	23, 51
abstract_inverted_index.postsession	180
abstract_inverted_index.statistical	160
abstract_inverted_index.translation	44, 62, 115, 149, 155, 162, 236, 274, 312, 336
abstract_inverted_index.usefulness.	321
abstract_inverted_index.(QuickSpeak)	157, 270
abstract_inverted_index.Technologies	325
abstract_inverted_index.improvements	341
abstract_inverted_index.individuals.	76
abstract_inverted_index.participants	144, 183, 226, 260, 303
abstract_inverted_index.successfully	67
abstract_inverted_index.technologies	116, 350
abstract_inverted_index.communication	11, 73
abstract_inverted_index.participants)	107
abstract_inverted_index.participants,	211, 215
abstract_inverted_index.personnel’s	284
abstract_inverted_index.satisfactory,	258
abstract_inverted_index.understanding	281
abstract_inverted_index.communication.	120
abstract_inverted_index.communications	48, 221
abstract_inverted_index.fixed-sentence	154
abstract_inverted_index.questionnaire.	181
abstract_inverted_index.scenario-based	88
abstract_inverted_index.domain-specific	153, 265
abstract_inverted_index.Chinese-speaking	141, 209
abstract_inverted_index.English-speaking	95
abstract_inverted_index.Spanish-speaking	139, 213
abstract_inverted_index.<title>METHODS</title>	123
abstract_inverted_index.<title>RESULTS</title>	193
abstract_inverted_index.<title>OBJECTIVE</title>	79
abstract_inverted_index.<title>BACKGROUND</title>	1
abstract_inverted_index.<title>CONCLUSIONS</title>	324
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	8
sustainable_development_goals[0].id	https://metadata.un.org/sdg/4
sustainable_development_goals[0].score	0.6800000071525574
sustainable_development_goals[0].display_name	Quality Education
citation_normalized_percentile.value	0.55821358
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False