Testing DBMS Performance with Mutations Article Swipe
Xinyu Liu
,
Qi Zhou
,
Joy Arulraj
,
Alessandro Orso
·
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2105.10016
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2105.10016
Because database systems are the critical component of modern data-intensive applications, it is important to ensure that they operate correctly. To this end, developers extensively test these systems to eliminate bugs that negatively affect functionality. In addition to functional bugs, however, there is another important class of bugs: performance bugs. These bugs negatively affect the response time of a database system and can therefore affect the overall performance of the system. Despite their impact on end-user experience, performance bugs have received considerably less attention than functional bugs. In this paper, we present AMOEBA, a system for automatically detecting performance bugs in database systems. The core idea behind AMOEBA is to construct query pairs that are semantically equivalent to each other and then compare their response time on the same database system. If the queries exhibit a significant difference in their runtime performance, then the root cause is likely a performance bug in the system. We propose a novel set of structure and predicate mutation rules for constructing query pairs that are likely to uncover performance bugs. We introduce feedback mechanisms for improving the efficacy and computational efficiency of the tool. We evaluate AMOEBA on two widely-used DBMSs, namely PostgreSQL and CockroachDB. AMOEBA has discovered 20 previously-unknown performance bugs, among which developers have already confirmed 14 and fixed 4.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2105.10016
- https://arxiv.org/pdf/2105.10016
- OA Status
- green
- References
- 34
- Related Works
- 20
- OpenAlex ID
- https://openalex.org/W3198210017
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W3198210017Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2105.10016Digital Object Identifier
- Title
-
Testing DBMS Performance with MutationsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2021Year of publication
- Publication date
-
2021-05-20Full publication date if available
- Authors
-
Xinyu Liu, Qi Zhou, Joy Arulraj, Alessandro OrsoList of authors in order
- Landing page
-
https://arxiv.org/abs/2105.10016Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2105.10016Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2105.10016Direct OA link when available
- Concepts
-
Computer science, Database, Construct (python library), Distributed computing, Programming languageTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- References (count)
-
34Number of works referenced by this work
- Related works (count)
-
20Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W3198210017 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2105.10016 |
| ids.doi | https://doi.org/10.48550/arxiv.2105.10016 |
| ids.mag | 3198210017 |
| ids.openalex | https://openalex.org/W3198210017 |
| fwci | |
| type | preprint |
| title | Testing DBMS Performance with Mutations |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10317 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9991000294685364 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1705 |
| topics[0].subfield.display_name | Computer Networks and Communications |
| topics[0].display_name | Advanced Database Systems and Queries |
| topics[1].id | https://openalex.org/T12127 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9980999827384949 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1705 |
| topics[1].subfield.display_name | Computer Networks and Communications |
| topics[1].display_name | Software System Performance and Reliability |
| topics[2].id | https://openalex.org/T10101 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9977999925613403 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1710 |
| topics[2].subfield.display_name | Information Systems |
| topics[2].display_name | Cloud Computing and Resource Management |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.82526695728302 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C77088390 |
| concepts[1].level | 1 |
| concepts[1].score | 0.5070915222167969 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q8513 |
| concepts[1].display_name | Database |
| concepts[2].id | https://openalex.org/C2780801425 |
| concepts[2].level | 2 |
| concepts[2].score | 0.478918194770813 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q5164392 |
| concepts[2].display_name | Construct (python library) |
| concepts[3].id | https://openalex.org/C120314980 |
| concepts[3].level | 1 |
| concepts[3].score | 0.3504958748817444 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q180634 |
| concepts[3].display_name | Distributed computing |
| concepts[4].id | https://openalex.org/C199360897 |
| concepts[4].level | 1 |
| concepts[4].score | 0.28881293535232544 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[4].display_name | Programming language |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.82526695728302 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/database |
| keywords[1].score | 0.5070915222167969 |
| keywords[1].display_name | Database |
| keywords[2].id | https://openalex.org/keywords/construct |
| keywords[2].score | 0.478918194770813 |
| keywords[2].display_name | Construct (python library) |
| keywords[3].id | https://openalex.org/keywords/distributed-computing |
| keywords[3].score | 0.3504958748817444 |
| keywords[3].display_name | Distributed computing |
| keywords[4].id | https://openalex.org/keywords/programming-language |
| keywords[4].score | 0.28881293535232544 |
| keywords[4].display_name | Programming language |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2105.10016 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2105.10016 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2105.10016 |
| locations[1].id | mag:3198210017 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | submittedVersion |
| locations[1].raw_type | |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | False |
| locations[1].raw_source_name | arXiv (Cornell University) |
| locations[1].landing_page_url | https://arxiv.org/pdf/2105.10016 |
| locations[2].id | doi:10.48550/arxiv.2105.10016 |
| locations[2].is_oa | True |
| locations[2].source.id | https://openalex.org/S4306400194 |
| locations[2].source.issn | |
| locations[2].source.type | repository |
| locations[2].source.is_oa | True |
| locations[2].source.issn_l | |
| locations[2].source.is_core | False |
| locations[2].source.is_in_doaj | False |
| locations[2].source.display_name | arXiv (Cornell University) |
| locations[2].source.host_organization | https://openalex.org/I205783295 |
| locations[2].source.host_organization_name | Cornell University |
| locations[2].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[2].license | |
| locations[2].pdf_url | |
| locations[2].version | |
| locations[2].raw_type | article |
| locations[2].license_id | |
| locations[2].is_accepted | False |
| locations[2].is_published | |
| locations[2].raw_source_name | |
| locations[2].landing_page_url | https://doi.org/10.48550/arxiv.2105.10016 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5100446429 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-2797-2083 |
| authorships[0].author.display_name | Xinyu Liu |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Xinyu Liu |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5077864517 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-6116-2036 |
| authorships[1].author.display_name | Qi Zhou |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Qi Zhou |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5060680349 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-7706-6978 |
| authorships[2].author.display_name | Joy Arulraj |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Joy Arulraj |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5030961858 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-4516-9320 |
| authorships[3].author.display_name | Alessandro Orso |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Alessandro Orso |
| authorships[3].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2105.10016 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Testing DBMS Performance with Mutations |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10317 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9991000294685364 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1705 |
| primary_topic.subfield.display_name | Computer Networks and Communications |
| primary_topic.display_name | Advanced Database Systems and Queries |
| related_works | https://openalex.org/W3164786800, https://openalex.org/W3125459478, https://openalex.org/W2968620057, https://openalex.org/W3107659790, https://openalex.org/W2514538448, https://openalex.org/W2344916214, https://openalex.org/W1978905120, https://openalex.org/W2521352760, https://openalex.org/W2114054012, https://openalex.org/W2572445426, https://openalex.org/W3033465675, https://openalex.org/W2943179835, https://openalex.org/W3156367898, https://openalex.org/W2899083685, https://openalex.org/W2117741559, https://openalex.org/W2921619209, https://openalex.org/W1597481093, https://openalex.org/W3097133472, https://openalex.org/W2036804147, https://openalex.org/W3203687540 |
| cited_by_count | 0 |
| locations_count | 3 |
| best_oa_location.id | pmh:oai:arXiv.org:2105.10016 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2105.10016 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2105.10016 |
| primary_location.id | pmh:oai:arXiv.org:2105.10016 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2105.10016 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2105.10016 |
| publication_date | 2021-05-20 |
| publication_year | 2021 |
| referenced_works | https://openalex.org/W1988504473, https://openalex.org/W2753710282, https://openalex.org/W1968779970, https://openalex.org/W2037085022, https://openalex.org/W2971026710, https://openalex.org/W2982547214, https://openalex.org/W3108403875, https://openalex.org/W1527501358, https://openalex.org/W3106731144, https://openalex.org/W2921282339, https://openalex.org/W3097728505, https://openalex.org/W2150669190, https://openalex.org/W2117480765, https://openalex.org/W2141575847, https://openalex.org/W3008797115, https://openalex.org/W2112100340, https://openalex.org/W2475801470, https://openalex.org/W109452506, https://openalex.org/W3099032206, https://openalex.org/W2061040095, https://openalex.org/W2299503697, https://openalex.org/W1504218467, https://openalex.org/W2158237121, https://openalex.org/W2584709123, https://openalex.org/W2066623874, https://openalex.org/W2293726547, https://openalex.org/W3159820712, https://openalex.org/W2049251595, https://openalex.org/W2109574129, https://openalex.org/W135863099, https://openalex.org/W2396309311, https://openalex.org/W2806878452, https://openalex.org/W2295144739, https://openalex.org/W1556422040 |
| referenced_works_count | 34 |
| abstract_inverted_index.a | 58, 93, 135, 148, 156 |
| abstract_inverted_index.14 | 214 |
| abstract_inverted_index.20 | 204 |
| abstract_inverted_index.4. | 217 |
| abstract_inverted_index.If | 131 |
| abstract_inverted_index.In | 35, 87 |
| abstract_inverted_index.To | 20 |
| abstract_inverted_index.We | 154, 176, 190 |
| abstract_inverted_index.in | 100, 138, 151 |
| abstract_inverted_index.is | 12, 42, 108, 146 |
| abstract_inverted_index.it | 11 |
| abstract_inverted_index.of | 7, 46, 57, 68, 159, 187 |
| abstract_inverted_index.on | 74, 126, 193 |
| abstract_inverted_index.to | 14, 28, 37, 109, 117, 172 |
| abstract_inverted_index.we | 90 |
| abstract_inverted_index.The | 103 |
| abstract_inverted_index.and | 61, 120, 161, 184, 199, 215 |
| abstract_inverted_index.are | 3, 114, 170 |
| abstract_inverted_index.bug | 150 |
| abstract_inverted_index.can | 62 |
| abstract_inverted_index.for | 95, 165, 180 |
| abstract_inverted_index.has | 202 |
| abstract_inverted_index.set | 158 |
| abstract_inverted_index.the | 4, 54, 65, 69, 127, 132, 143, 152, 182, 188 |
| abstract_inverted_index.two | 194 |
| abstract_inverted_index.bugs | 30, 51, 78, 99 |
| abstract_inverted_index.core | 104 |
| abstract_inverted_index.each | 118 |
| abstract_inverted_index.end, | 22 |
| abstract_inverted_index.have | 79, 211 |
| abstract_inverted_index.idea | 105 |
| abstract_inverted_index.less | 82 |
| abstract_inverted_index.root | 144 |
| abstract_inverted_index.same | 128 |
| abstract_inverted_index.test | 25 |
| abstract_inverted_index.than | 84 |
| abstract_inverted_index.that | 16, 31, 113, 169 |
| abstract_inverted_index.then | 121, 142 |
| abstract_inverted_index.they | 17 |
| abstract_inverted_index.this | 21, 88 |
| abstract_inverted_index.time | 56, 125 |
| abstract_inverted_index.These | 50 |
| abstract_inverted_index.among | 208 |
| abstract_inverted_index.bugs, | 39, 207 |
| abstract_inverted_index.bugs. | 49, 86, 175 |
| abstract_inverted_index.bugs: | 47 |
| abstract_inverted_index.cause | 145 |
| abstract_inverted_index.class | 45 |
| abstract_inverted_index.fixed | 216 |
| abstract_inverted_index.novel | 157 |
| abstract_inverted_index.other | 119 |
| abstract_inverted_index.pairs | 112, 168 |
| abstract_inverted_index.query | 111, 167 |
| abstract_inverted_index.rules | 164 |
| abstract_inverted_index.their | 72, 123, 139 |
| abstract_inverted_index.there | 41 |
| abstract_inverted_index.these | 26 |
| abstract_inverted_index.tool. | 189 |
| abstract_inverted_index.which | 209 |
| abstract_inverted_index.AMOEBA | 107, 192, 201 |
| abstract_inverted_index.DBMSs, | 196 |
| abstract_inverted_index.affect | 33, 53, 64 |
| abstract_inverted_index.behind | 106 |
| abstract_inverted_index.ensure | 15 |
| abstract_inverted_index.impact | 73 |
| abstract_inverted_index.likely | 147, 171 |
| abstract_inverted_index.modern | 8 |
| abstract_inverted_index.namely | 197 |
| abstract_inverted_index.paper, | 89 |
| abstract_inverted_index.system | 60, 94 |
| abstract_inverted_index.AMOEBA, | 92 |
| abstract_inverted_index.Because | 0 |
| abstract_inverted_index.Despite | 71 |
| abstract_inverted_index.already | 212 |
| abstract_inverted_index.another | 43 |
| abstract_inverted_index.compare | 122 |
| abstract_inverted_index.exhibit | 134 |
| abstract_inverted_index.operate | 18 |
| abstract_inverted_index.overall | 66 |
| abstract_inverted_index.present | 91 |
| abstract_inverted_index.propose | 155 |
| abstract_inverted_index.queries | 133 |
| abstract_inverted_index.runtime | 140 |
| abstract_inverted_index.system. | 70, 130, 153 |
| abstract_inverted_index.systems | 2, 27 |
| abstract_inverted_index.uncover | 173 |
| abstract_inverted_index.addition | 36 |
| abstract_inverted_index.critical | 5 |
| abstract_inverted_index.database | 1, 59, 101, 129 |
| abstract_inverted_index.efficacy | 183 |
| abstract_inverted_index.end-user | 75 |
| abstract_inverted_index.evaluate | 191 |
| abstract_inverted_index.feedback | 178 |
| abstract_inverted_index.however, | 40 |
| abstract_inverted_index.mutation | 163 |
| abstract_inverted_index.received | 80 |
| abstract_inverted_index.response | 55, 124 |
| abstract_inverted_index.systems. | 102 |
| abstract_inverted_index.attention | 83 |
| abstract_inverted_index.component | 6 |
| abstract_inverted_index.confirmed | 213 |
| abstract_inverted_index.construct | 110 |
| abstract_inverted_index.detecting | 97 |
| abstract_inverted_index.eliminate | 29 |
| abstract_inverted_index.important | 13, 44 |
| abstract_inverted_index.improving | 181 |
| abstract_inverted_index.introduce | 177 |
| abstract_inverted_index.predicate | 162 |
| abstract_inverted_index.structure | 160 |
| abstract_inverted_index.therefore | 63 |
| abstract_inverted_index.PostgreSQL | 198 |
| abstract_inverted_index.correctly. | 19 |
| abstract_inverted_index.developers | 23, 210 |
| abstract_inverted_index.difference | 137 |
| abstract_inverted_index.discovered | 203 |
| abstract_inverted_index.efficiency | 186 |
| abstract_inverted_index.equivalent | 116 |
| abstract_inverted_index.functional | 38, 85 |
| abstract_inverted_index.mechanisms | 179 |
| abstract_inverted_index.negatively | 32, 52 |
| abstract_inverted_index.experience, | 76 |
| abstract_inverted_index.extensively | 24 |
| abstract_inverted_index.performance | 48, 67, 77, 98, 149, 174, 206 |
| abstract_inverted_index.significant | 136 |
| abstract_inverted_index.widely-used | 195 |
| abstract_inverted_index.CockroachDB. | 200 |
| abstract_inverted_index.considerably | 81 |
| abstract_inverted_index.constructing | 166 |
| abstract_inverted_index.performance, | 141 |
| abstract_inverted_index.semantically | 115 |
| abstract_inverted_index.applications, | 10 |
| abstract_inverted_index.automatically | 96 |
| abstract_inverted_index.computational | 185 |
| abstract_inverted_index.data-intensive | 9 |
| abstract_inverted_index.functionality. | 34 |
| abstract_inverted_index.previously-unknown | 205 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile |