In-situ workflow auto-tuning through combining component models Article Swipe

PDF

Tong Shu , Yanfei Guo , Justin M. Wozniak , Xiaoning Ding , Ian Foster , Tahsin Kurç ·

YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.1145/3437801.3441615

In-situ parallel workflows couple multiple component applications via streaming data transfer to avoid data exchange via shared file systems. Such workflows are challenging to configure for optimal performance due to the huge space of possible configurations. Here, we propose an in-situ workflow auto-tuning method, ALIC, which integrates machine learning techniques with knowledge of in-situ workflow structures to enable automated workflow configuration with a limited number of performance measurements. Experiments with real applications show that ALIC identify better configurations than existing methods given a computer time budget.

Related Topics

Concepts

Workflow Computer science Component (thermodynamics) Workflow technology Distributed computing Workflow engine Workflow management system In situ Database Meteorology Thermodynamics Physics

Metadata

Type: article
Language: en
Landing Page: https://doi.org/10.1145/3437801.3441615
PDF: https://dl.acm.org/doi/pdf/10.1145/3437801.3441615
OA Status: gold
Cited By: 9
References: 10
Related Works: 10
OpenAlex ID: https://openalex.org/W3129852841

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W3129852841

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.1145/3437801.3441615

Digital Object Identifier
Title: In-situ workflow auto-tuning through combining component models

Work title
Type: article

OpenAlex work type
Language: en

Primary language
Publication year: 2021

Year of publication
Publication date: 2021-02-17

Full publication date if available
Authors: Tong Shu, Yanfei Guo, Justin M. Wozniak, Xiaoning Ding, Ian Foster, Tahsin Kurç

List of authors in order
Landing page: https://doi.org/10.1145/3437801.3441615

Publisher landing page
PDF URL: https://dl.acm.org/doi/pdf/10.1145/3437801.3441615

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: gold

Open access status per OpenAlex
OA URL: https://dl.acm.org/doi/pdf/10.1145/3437801.3441615

Direct OA link when available
Concepts: Workflow, Computer science, Component (thermodynamics), Workflow technology, Distributed computing, Workflow engine, Workflow management system, In situ, Database, Meteorology, Thermodynamics, Physics

Top concepts (fields/topics) attached by OpenAlex
Cited by: 9

Total citation count in OpenAlex
Citations by year (recent): 2025: 1, 2024: 3, 2023: 3, 2021: 2

Per-year citation counts (last 5 years)
References (count): 10

Number of works referenced by this work
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W3129852841
doi	https://doi.org/10.1145/3437801.3441615
ids.doi	https://doi.org/10.1145/3437801.3441615
ids.mag	3129852841
ids.openalex	https://openalex.org/W3129852841
fwci	1.32505405
type	article
title	In-situ workflow auto-tuning through combining component models
awards[0].id	https://openalex.org/G6859237865
awards[0].funder_id	https://openalex.org/F4320306084
awards[0].display_name
awards[0].funder_award_id	17-SC-20-SC
awards[0].funder_display_name	U.S. Department of Energy
biblio.issue
biblio.volume
biblio.last_page	468
biblio.first_page	467
topics[0].id	https://openalex.org/T11181
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9990000128746033
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1705
topics[0].subfield.display_name	Computer Networks and Communications
topics[0].display_name	Advanced Data Storage Technologies
topics[1].id	https://openalex.org/T11986
topics[1].field.id	https://openalex.org/fields/18
topics[1].field.display_name	Decision Sciences
topics[1].score	0.9970999956130981
topics[1].domain.id	https://openalex.org/domains/2
topics[1].domain.display_name	Social Sciences
topics[1].subfield.id	https://openalex.org/subfields/1802
topics[1].subfield.display_name	Information Systems and Management
topics[1].display_name	Scientific Computing and Data Management
topics[2].id	https://openalex.org/T10715
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.9965999722480774
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1705
topics[2].subfield.display_name	Computer Networks and Communications
topics[2].display_name	Distributed and Parallel Computing Systems
funders[0].id	https://openalex.org/F4320306084
funders[0].ror	https://ror.org/01bj3aw27
funders[0].display_name	U.S. Department of Energy
funders[1].id	https://openalex.org/F4320310628
funders[1].ror	https://ror.org/05vz28418
funders[1].display_name	Southern Illinois University
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C177212765
concepts[0].level	2
concepts[0].score	0.936674952507019
concepts[0].wikidata	https://www.wikidata.org/wiki/Q627335
concepts[0].display_name	Workflow
concepts[1].id	https://openalex.org/C41008148
concepts[1].level	0
concepts[1].score	0.789832353591919
concepts[1].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[1].display_name	Computer science
concepts[2].id	https://openalex.org/C168167062
concepts[2].level	2
concepts[2].score	0.7078703045845032
concepts[2].wikidata	https://www.wikidata.org/wiki/Q1117970
concepts[2].display_name	Component (thermodynamics)
concepts[3].id	https://openalex.org/C19612761
concepts[3].level	3
concepts[3].score	0.6231963634490967
concepts[3].wikidata	https://www.wikidata.org/wiki/Q8034836
concepts[3].display_name	Workflow technology
concepts[4].id	https://openalex.org/C120314980
concepts[4].level	1
concepts[4].score	0.5939803123474121
concepts[4].wikidata	https://www.wikidata.org/wiki/Q180634
concepts[4].display_name	Distributed computing
concepts[5].id	https://openalex.org/C188220564
concepts[5].level	3
concepts[5].score	0.540287435054779
concepts[5].wikidata	https://www.wikidata.org/wiki/Q3325097
concepts[5].display_name	Workflow engine
concepts[6].id	https://openalex.org/C140824633
concepts[6].level	3
concepts[6].score	0.4965572953224182
concepts[6].wikidata	https://www.wikidata.org/wiki/Q2808660
concepts[6].display_name	Workflow management system
concepts[7].id	https://openalex.org/C2777822432
concepts[7].level	2
concepts[7].score	0.4545021951198578
concepts[7].wikidata	https://www.wikidata.org/wiki/Q216681
concepts[7].display_name	In situ
concepts[8].id	https://openalex.org/C77088390
concepts[8].level	1
concepts[8].score	0.34889814257621765
concepts[8].wikidata	https://www.wikidata.org/wiki/Q8513
concepts[8].display_name	Database
concepts[9].id	https://openalex.org/C153294291
concepts[9].level	1
concepts[9].score	0.0
concepts[9].wikidata	https://www.wikidata.org/wiki/Q25261
concepts[9].display_name	Meteorology
concepts[10].id	https://openalex.org/C97355855
concepts[10].level	1
concepts[10].score	0.0
concepts[10].wikidata	https://www.wikidata.org/wiki/Q11473
concepts[10].display_name	Thermodynamics
concepts[11].id	https://openalex.org/C121332964
concepts[11].level	0
concepts[11].score	0.0
concepts[11].wikidata	https://www.wikidata.org/wiki/Q413
concepts[11].display_name	Physics
keywords[0].id	https://openalex.org/keywords/workflow
keywords[0].score	0.936674952507019
keywords[0].display_name	Workflow
keywords[1].id	https://openalex.org/keywords/computer-science
keywords[1].score	0.789832353591919
keywords[1].display_name	Computer science
keywords[2].id	https://openalex.org/keywords/component
keywords[2].score	0.7078703045845032
keywords[2].display_name	Component (thermodynamics)
keywords[3].id	https://openalex.org/keywords/workflow-technology
keywords[3].score	0.6231963634490967
keywords[3].display_name	Workflow technology
keywords[4].id	https://openalex.org/keywords/distributed-computing
keywords[4].score	0.5939803123474121
keywords[4].display_name	Distributed computing
keywords[5].id	https://openalex.org/keywords/workflow-engine
keywords[5].score	0.540287435054779
keywords[5].display_name	Workflow engine
keywords[6].id	https://openalex.org/keywords/workflow-management-system
keywords[6].score	0.4965572953224182
keywords[6].display_name	Workflow management system
keywords[7].id	https://openalex.org/keywords/in-situ
keywords[7].score	0.4545021951198578
keywords[7].display_name	In situ
keywords[8].id	https://openalex.org/keywords/database
keywords[8].score	0.34889814257621765
keywords[8].display_name	Database
language	en
locations[0].id	doi:10.1145/3437801.3441615
locations[0].is_oa	True
locations[0].source
locations[0].license
locations[0].pdf_url	https://dl.acm.org/doi/pdf/10.1145/3437801.3441615
locations[0].version	publishedVersion
locations[0].raw_type	proceedings-article
locations[0].license_id
locations[0].is_accepted	True
locations[0].is_published	True
locations[0].raw_source_name	Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
locations[0].landing_page_url	https://doi.org/10.1145/3437801.3441615
indexed_in	crossref
authorships[0].author.id	https://openalex.org/A5103106461
authorships[0].author.orcid	https://orcid.org/0000-0001-5952-3090
authorships[0].author.display_name	Tong Shu
authorships[0].affiliations[0].raw_affiliation_string	Southern Illinois University
authorships[0].author_position	first
authorships[0].raw_author_name	Tong Shu
authorships[0].is_corresponding	False
authorships[0].raw_affiliation_strings	Southern Illinois University
authorships[1].author.id	https://openalex.org/A5024620270
authorships[1].author.orcid	https://orcid.org/0000-0002-3731-5423
authorships[1].author.display_name	Yanfei Guo
authorships[1].countries	US
authorships[1].affiliations[0].institution_ids	https://openalex.org/I1282105669
authorships[1].affiliations[0].raw_affiliation_string	Argonne National Laboratory
authorships[1].institutions[0].id	https://openalex.org/I1282105669
authorships[1].institutions[0].ror	https://ror.org/05gvnxz63
authorships[1].institutions[0].type	facility
authorships[1].institutions[0].lineage	https://openalex.org/I1282105669, https://openalex.org/I1330989302, https://openalex.org/I39565521, https://openalex.org/I40347166
authorships[1].institutions[0].country_code	US
authorships[1].institutions[0].display_name	Argonne National Laboratory
authorships[1].author_position	middle
authorships[1].raw_author_name	Yanfei Guo
authorships[1].is_corresponding	False
authorships[1].raw_affiliation_strings	Argonne National Laboratory
authorships[2].author.id	https://openalex.org/A5020776475
authorships[2].author.orcid	https://orcid.org/0000-0002-2441-2048
authorships[2].author.display_name	Justin M. Wozniak
authorships[2].countries	US
authorships[2].affiliations[0].institution_ids	https://openalex.org/I1282105669
authorships[2].affiliations[0].raw_affiliation_string	Argonne National Laboratory
authorships[2].institutions[0].id	https://openalex.org/I1282105669
authorships[2].institutions[0].ror	https://ror.org/05gvnxz63
authorships[2].institutions[0].type	facility
authorships[2].institutions[0].lineage	https://openalex.org/I1282105669, https://openalex.org/I1330989302, https://openalex.org/I39565521, https://openalex.org/I40347166
authorships[2].institutions[0].country_code	US
authorships[2].institutions[0].display_name	Argonne National Laboratory
authorships[2].author_position	middle
authorships[2].raw_author_name	Justin Wozniak
authorships[2].is_corresponding	False
authorships[2].raw_affiliation_strings	Argonne National Laboratory
authorships[3].author.id	https://openalex.org/A5023013886
authorships[3].author.orcid	https://orcid.org/0000-0002-9947-0437
authorships[3].author.display_name	Xiaoning Ding
authorships[3].countries	US
authorships[3].affiliations[0].institution_ids	https://openalex.org/I118118575
authorships[3].affiliations[0].raw_affiliation_string	New Jersey Institute of Technology
authorships[3].institutions[0].id	https://openalex.org/I118118575
authorships[3].institutions[0].ror	https://ror.org/05e74xb87
authorships[3].institutions[0].type	education
authorships[3].institutions[0].lineage	https://openalex.org/I118118575
authorships[3].institutions[0].country_code	US
authorships[3].institutions[0].display_name	New Jersey Institute of Technology
authorships[3].author_position	middle
authorships[3].raw_author_name	Xiaoning Ding
authorships[3].is_corresponding	False
authorships[3].raw_affiliation_strings	New Jersey Institute of Technology
authorships[4].author.id	https://openalex.org/A5032231503
authorships[4].author.orcid	https://orcid.org/0000-0003-2129-5269
authorships[4].author.display_name	Ian Foster
authorships[4].affiliations[0].raw_affiliation_string	Univ. Chicago
authorships[4].author_position	middle
authorships[4].raw_author_name	Ian Foster
authorships[4].is_corresponding	False
authorships[4].raw_affiliation_strings	Univ. Chicago
authorships[5].author.id	https://openalex.org/A5037383891
authorships[5].author.orcid	https://orcid.org/0000-0001-9237-4306
authorships[5].author.display_name	Tahsin Kurç
authorships[5].countries	US
authorships[5].affiliations[0].institution_ids	https://openalex.org/I59553526
authorships[5].affiliations[0].raw_affiliation_string	Stony Brook University
authorships[5].institutions[0].id	https://openalex.org/I59553526
authorships[5].institutions[0].ror	https://ror.org/05qghxh33
authorships[5].institutions[0].type	education
authorships[5].institutions[0].lineage	https://openalex.org/I59553526
authorships[5].institutions[0].country_code	US
authorships[5].institutions[0].display_name	Stony Brook University
authorships[5].author_position	last
authorships[5].raw_author_name	Tahsin Kurc
authorships[5].is_corresponding	False
authorships[5].raw_affiliation_strings	Stony Brook University
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://dl.acm.org/doi/pdf/10.1145/3437801.3441615
open_access.oa_status	gold
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	In-situ workflow auto-tuning through combining component models
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T11181
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9990000128746033
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1705
primary_topic.subfield.display_name	Computer Networks and Communications
primary_topic.display_name	Advanced Data Storage Technologies
related_works	https://openalex.org/W2352774566, https://openalex.org/W2049143847, https://openalex.org/W2356466503, https://openalex.org/W2376671203, https://openalex.org/W2391168633, https://openalex.org/W2385499106, https://openalex.org/W2354357087, https://openalex.org/W1574425962, https://openalex.org/W2387628393, https://openalex.org/W2370201914
cited_by_count	9
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	1
counts_by_year[1].year	2024
counts_by_year[1].cited_by_count	3
counts_by_year[2].year	2023
counts_by_year[2].cited_by_count	3
counts_by_year[3].year	2021
counts_by_year[3].cited_by_count	2
locations_count	1
best_oa_location.id	doi:10.1145/3437801.3441615
best_oa_location.is_oa	True
best_oa_location.source
best_oa_location.license
best_oa_location.pdf_url	https://dl.acm.org/doi/pdf/10.1145/3437801.3441615
best_oa_location.version	publishedVersion
best_oa_location.raw_type	proceedings-article
best_oa_location.license_id
best_oa_location.is_accepted	True
best_oa_location.is_published	True
best_oa_location.raw_source_name	Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
best_oa_location.landing_page_url	https://doi.org/10.1145/3437801.3441615
primary_location.id	doi:10.1145/3437801.3441615
primary_location.is_oa	True
primary_location.source
primary_location.license
primary_location.pdf_url	https://dl.acm.org/doi/pdf/10.1145/3437801.3441615
primary_location.version	publishedVersion
primary_location.raw_type	proceedings-article
primary_location.license_id
primary_location.is_accepted	True
primary_location.is_published	True
primary_location.raw_source_name	Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
primary_location.landing_page_url	https://doi.org/10.1145/3437801.3441615
publication_date	2021-02-17
publication_year	2021
referenced_works	https://openalex.org/W2295598076, https://openalex.org/W2922457781, https://openalex.org/W2889745160, https://openalex.org/W2913064927, https://openalex.org/W2944973524, https://openalex.org/W2770670265, https://openalex.org/W2763642597, https://openalex.org/W2739642345, https://openalex.org/W2808379700, https://openalex.org/W3102476541
referenced_works_count	10
abstract_inverted_index.a	62, 82
abstract_inverted_index.an	39
abstract_inverted_index.of	33, 52, 65
abstract_inverted_index.to	11, 23, 29, 56
abstract_inverted_index.we	37
abstract_inverted_index.are	21
abstract_inverted_index.due	28
abstract_inverted_index.for	25
abstract_inverted_index.the	30
abstract_inverted_index.via	7, 15
abstract_inverted_index.ALIC	74
abstract_inverted_index.Such	19
abstract_inverted_index.data	9, 13
abstract_inverted_index.file	17
abstract_inverted_index.huge	31
abstract_inverted_index.real	70
abstract_inverted_index.show	72
abstract_inverted_index.than	78
abstract_inverted_index.that	73
abstract_inverted_index.time	84
abstract_inverted_index.with	50, 61, 69
abstract_inverted_index.ALIC,	44
abstract_inverted_index.Here,	36
abstract_inverted_index.avoid	12
abstract_inverted_index.given	81
abstract_inverted_index.space	32
abstract_inverted_index.which	45
abstract_inverted_index.better	76
abstract_inverted_index.couple	3
abstract_inverted_index.enable	57
abstract_inverted_index.number	64
abstract_inverted_index.shared	16
abstract_inverted_index.In-situ	0
abstract_inverted_index.budget.	85
abstract_inverted_index.in-situ	40, 53
abstract_inverted_index.limited	63
abstract_inverted_index.machine	47
abstract_inverted_index.method,	43
abstract_inverted_index.methods	80
abstract_inverted_index.optimal	26
abstract_inverted_index.propose	38
abstract_inverted_index.computer	83
abstract_inverted_index.exchange	14
abstract_inverted_index.existing	79
abstract_inverted_index.identify	75
abstract_inverted_index.learning	48
abstract_inverted_index.multiple	4
abstract_inverted_index.parallel	1
abstract_inverted_index.possible	34
abstract_inverted_index.systems.	18
abstract_inverted_index.transfer	10
abstract_inverted_index.workflow	41, 54, 59
abstract_inverted_index.automated	58
abstract_inverted_index.component	5
abstract_inverted_index.configure	24
abstract_inverted_index.knowledge	51
abstract_inverted_index.streaming	8
abstract_inverted_index.workflows	2, 20
abstract_inverted_index.integrates	46
abstract_inverted_index.structures	55
abstract_inverted_index.techniques	49
abstract_inverted_index.Experiments	68
abstract_inverted_index.auto-tuning	42
abstract_inverted_index.challenging	22
abstract_inverted_index.performance	27, 66
abstract_inverted_index.applications	6, 71
abstract_inverted_index.configuration	60
abstract_inverted_index.measurements.	67
abstract_inverted_index.configurations	77
abstract_inverted_index.configurations.	35
cited_by_percentile_year.max	97
cited_by_percentile_year.min	91
countries_distinct_count	1
institutions_distinct_count	6
sustainable_development_goals[0].id	https://metadata.un.org/sdg/9
sustainable_development_goals[0].score	0.5
sustainable_development_goals[0].display_name	Industry, innovation and infrastructure
citation_normalized_percentile.value	0.80062361
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False