Visual High Dimensional Hypothesis Testing Article Swipe
In exploratory data analysis of known classes of high dimensional data, a central question is how distinct are the classes? The Direction Projection Permutation (DiProPerm) hypothesis test provides an answer to this that is directly connected to a visual analysis of the data. In this paper, we propose an improved DiProPerm test that solves 3 major challenges of the original version. First, we implement only balanced permutations to increase the test power for data with strong signals. Second, our mathematical analysis leads to an adjustment to correct the null behavior of both balanced and the conventional all permutations. Third, new confidence intervals (reflecting permutation variation) for test significance are also proposed for comparison of results across different contexts. This improvement of DiProPerm inference is illustrated in the context of comparing cancer types in examples from The Cancer Genome Atlas.
Related Topics
- Type
- preprint
- Language
- en
- Landing Page
- http://export.arxiv.org/pdf/2101.00362
- OA Status
- green
- Cited By
- 1
- References
- 8
- Related Works
- 20
- OpenAlex ID
- https://openalex.org/W3120747838
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W3120747838Canonical identifier for this work in OpenAlex
- Title
-
Visual High Dimensional Hypothesis TestingWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2021Year of publication
- Publication date
-
2021-01-02Full publication date if available
- Authors
-
Xi Yang, Jan Hannig, J. S. MarronList of authors in order
- Landing page
-
https://export.arxiv.org/pdf/2101.00362Publisher landing page
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://export.arxiv.org/pdf/2101.00362Direct OA link when available
- Concepts
-
Permutation (music), Inference, Computer science, Null (SQL), Null hypothesis, Resampling, Context (archaeology), Multiple comparisons problem, Exploratory data analysis, Statistical hypothesis testing, Projection (relational algebra), Data mining, Theoretical computer science, Algorithm, Artificial intelligence, Mathematics, Statistics, Biology, Acoustics, Physics, PaleontologyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2021: 1Per-year citation counts (last 5 years)
- References (count)
-
8Number of works referenced by this work
- Related works (count)
-
20Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W3120747838 |
|---|---|
| doi | |
| ids.mag | 3120747838 |
| ids.openalex | https://openalex.org/W3120747838 |
| fwci | |
| type | preprint |
| title | Visual High Dimensional Hypothesis Testing |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10885 |
| topics[0].field.id | https://openalex.org/fields/13 |
| topics[0].field.display_name | Biochemistry, Genetics and Molecular Biology |
| topics[0].score | 0.9980999827384949 |
| topics[0].domain.id | https://openalex.org/domains/1 |
| topics[0].domain.display_name | Life Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1312 |
| topics[0].subfield.display_name | Molecular Biology |
| topics[0].display_name | Gene expression and cancer classification |
| topics[1].id | https://openalex.org/T10862 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.993399977684021 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | AI in cancer detection |
| topics[2].id | https://openalex.org/T12859 |
| topics[2].field.id | https://openalex.org/fields/13 |
| topics[2].field.display_name | Biochemistry, Genetics and Molecular Biology |
| topics[2].score | 0.9901999831199646 |
| topics[2].domain.id | https://openalex.org/domains/1 |
| topics[2].domain.display_name | Life Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1304 |
| topics[2].subfield.display_name | Biophysics |
| topics[2].display_name | Cell Image Analysis Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C21308566 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7138020992279053 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q7169365 |
| concepts[0].display_name | Permutation (music) |
| concepts[1].id | https://openalex.org/C2776214188 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6201766729354858 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q408386 |
| concepts[1].display_name | Inference |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.5797587633132935 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C203763787 |
| concepts[3].level | 2 |
| concepts[3].score | 0.569597601890564 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q371029 |
| concepts[3].display_name | Null (SQL) |
| concepts[4].id | https://openalex.org/C191988596 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5240464210510254 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q628374 |
| concepts[4].display_name | Null hypothesis |
| concepts[5].id | https://openalex.org/C150921843 |
| concepts[5].level | 2 |
| concepts[5].score | 0.5034100413322449 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q1170431 |
| concepts[5].display_name | Resampling |
| concepts[6].id | https://openalex.org/C2779343474 |
| concepts[6].level | 2 |
| concepts[6].score | 0.502082109451294 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q3109175 |
| concepts[6].display_name | Context (archaeology) |
| concepts[7].id | https://openalex.org/C183905921 |
| concepts[7].level | 2 |
| concepts[7].score | 0.4922391474246979 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q1038757 |
| concepts[7].display_name | Multiple comparisons problem |
| concepts[8].id | https://openalex.org/C120894424 |
| concepts[8].level | 2 |
| concepts[8].score | 0.4916554093360901 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q1322871 |
| concepts[8].display_name | Exploratory data analysis |
| concepts[9].id | https://openalex.org/C87007009 |
| concepts[9].level | 2 |
| concepts[9].score | 0.4823150634765625 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q210832 |
| concepts[9].display_name | Statistical hypothesis testing |
| concepts[10].id | https://openalex.org/C57493831 |
| concepts[10].level | 2 |
| concepts[10].score | 0.41706210374832153 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q3134666 |
| concepts[10].display_name | Projection (relational algebra) |
| concepts[11].id | https://openalex.org/C124101348 |
| concepts[11].level | 1 |
| concepts[11].score | 0.3642727732658386 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q172491 |
| concepts[11].display_name | Data mining |
| concepts[12].id | https://openalex.org/C80444323 |
| concepts[12].level | 1 |
| concepts[12].score | 0.3608901798725128 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q2878974 |
| concepts[12].display_name | Theoretical computer science |
| concepts[13].id | https://openalex.org/C11413529 |
| concepts[13].level | 1 |
| concepts[13].score | 0.347731351852417 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[13].display_name | Algorithm |
| concepts[14].id | https://openalex.org/C154945302 |
| concepts[14].level | 1 |
| concepts[14].score | 0.3317158818244934 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[14].display_name | Artificial intelligence |
| concepts[15].id | https://openalex.org/C33923547 |
| concepts[15].level | 0 |
| concepts[15].score | 0.3030775487422943 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[15].display_name | Mathematics |
| concepts[16].id | https://openalex.org/C105795698 |
| concepts[16].level | 1 |
| concepts[16].score | 0.24668467044830322 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q12483 |
| concepts[16].display_name | Statistics |
| concepts[17].id | https://openalex.org/C86803240 |
| concepts[17].level | 0 |
| concepts[17].score | 0.09430965781211853 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q420 |
| concepts[17].display_name | Biology |
| concepts[18].id | https://openalex.org/C24890656 |
| concepts[18].level | 1 |
| concepts[18].score | 0.0 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q82811 |
| concepts[18].display_name | Acoustics |
| concepts[19].id | https://openalex.org/C121332964 |
| concepts[19].level | 0 |
| concepts[19].score | 0.0 |
| concepts[19].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[19].display_name | Physics |
| concepts[20].id | https://openalex.org/C151730666 |
| concepts[20].level | 1 |
| concepts[20].score | 0.0 |
| concepts[20].wikidata | https://www.wikidata.org/wiki/Q7205 |
| concepts[20].display_name | Paleontology |
| keywords[0].id | https://openalex.org/keywords/permutation |
| keywords[0].score | 0.7138020992279053 |
| keywords[0].display_name | Permutation (music) |
| keywords[1].id | https://openalex.org/keywords/inference |
| keywords[1].score | 0.6201766729354858 |
| keywords[1].display_name | Inference |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.5797587633132935 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/null |
| keywords[3].score | 0.569597601890564 |
| keywords[3].display_name | Null (SQL) |
| keywords[4].id | https://openalex.org/keywords/null-hypothesis |
| keywords[4].score | 0.5240464210510254 |
| keywords[4].display_name | Null hypothesis |
| keywords[5].id | https://openalex.org/keywords/resampling |
| keywords[5].score | 0.5034100413322449 |
| keywords[5].display_name | Resampling |
| keywords[6].id | https://openalex.org/keywords/context |
| keywords[6].score | 0.502082109451294 |
| keywords[6].display_name | Context (archaeology) |
| keywords[7].id | https://openalex.org/keywords/multiple-comparisons-problem |
| keywords[7].score | 0.4922391474246979 |
| keywords[7].display_name | Multiple comparisons problem |
| keywords[8].id | https://openalex.org/keywords/exploratory-data-analysis |
| keywords[8].score | 0.4916554093360901 |
| keywords[8].display_name | Exploratory data analysis |
| keywords[9].id | https://openalex.org/keywords/statistical-hypothesis-testing |
| keywords[9].score | 0.4823150634765625 |
| keywords[9].display_name | Statistical hypothesis testing |
| keywords[10].id | https://openalex.org/keywords/projection |
| keywords[10].score | 0.41706210374832153 |
| keywords[10].display_name | Projection (relational algebra) |
| keywords[11].id | https://openalex.org/keywords/data-mining |
| keywords[11].score | 0.3642727732658386 |
| keywords[11].display_name | Data mining |
| keywords[12].id | https://openalex.org/keywords/theoretical-computer-science |
| keywords[12].score | 0.3608901798725128 |
| keywords[12].display_name | Theoretical computer science |
| keywords[13].id | https://openalex.org/keywords/algorithm |
| keywords[13].score | 0.347731351852417 |
| keywords[13].display_name | Algorithm |
| keywords[14].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[14].score | 0.3317158818244934 |
| keywords[14].display_name | Artificial intelligence |
| keywords[15].id | https://openalex.org/keywords/mathematics |
| keywords[15].score | 0.3030775487422943 |
| keywords[15].display_name | Mathematics |
| keywords[16].id | https://openalex.org/keywords/statistics |
| keywords[16].score | 0.24668467044830322 |
| keywords[16].display_name | Statistics |
| keywords[17].id | https://openalex.org/keywords/biology |
| keywords[17].score | 0.09430965781211853 |
| keywords[17].display_name | Biology |
| language | en |
| locations[0].id | mag:3120747838 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | arXiv (Cornell University) |
| locations[0].landing_page_url | http://export.arxiv.org/pdf/2101.00362 |
| authorships[0].author.id | https://openalex.org/A5101899583 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-1277-9539 |
| authorships[0].author.display_name | Xi Yang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Xi Yang |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5084746497 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-4164-0173 |
| authorships[1].author.display_name | Jan Hannig |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Jan Hannig |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5060651495 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | J. S. Marron |
| authorships[2].author_position | last |
| authorships[2].raw_author_name | J. S. Marron |
| authorships[2].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | http://export.arxiv.org/pdf/2101.00362 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Visual High Dimensional Hypothesis Testing |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-10-10T17:16:08.811792 |
| primary_topic.id | https://openalex.org/T10885 |
| primary_topic.field.id | https://openalex.org/fields/13 |
| primary_topic.field.display_name | Biochemistry, Genetics and Molecular Biology |
| primary_topic.score | 0.9980999827384949 |
| primary_topic.domain.id | https://openalex.org/domains/1 |
| primary_topic.domain.display_name | Life Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1312 |
| primary_topic.subfield.display_name | Molecular Biology |
| primary_topic.display_name | Gene expression and cancer classification |
| related_works | https://openalex.org/W1606491053, https://openalex.org/W2068079611, https://openalex.org/W1995316801, https://openalex.org/W2187584467, https://openalex.org/W1563116596, https://openalex.org/W2008840001, https://openalex.org/W75271608, https://openalex.org/W1995248017, https://openalex.org/W2120557245, https://openalex.org/W146057680, https://openalex.org/W1992178209, https://openalex.org/W3004732852, https://openalex.org/W2977783618, https://openalex.org/W2049817623, https://openalex.org/W1980437782, https://openalex.org/W2011154619, https://openalex.org/W2782976553, https://openalex.org/W2005111994, https://openalex.org/W2279373597, https://openalex.org/W2803334329 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2021 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 1 |
| best_oa_location.id | mag:3120747838 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | arXiv (Cornell University) |
| best_oa_location.landing_page_url | http://export.arxiv.org/pdf/2101.00362 |
| primary_location.id | mag:3120747838 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | arXiv (Cornell University) |
| primary_location.landing_page_url | http://export.arxiv.org/pdf/2101.00362 |
| publication_date | 2021-01-02 |
| publication_year | 2021 |
| referenced_works | https://openalex.org/W2235688562, https://openalex.org/W2011058684, https://openalex.org/W2094439664, https://openalex.org/W2319794630, https://openalex.org/W2413573154, https://openalex.org/W2795989238, https://openalex.org/W2024806449, https://openalex.org/W2086258274 |
| referenced_works_count | 8 |
| abstract_inverted_index.3 | 54 |
| abstract_inverted_index.a | 11, 37 |
| abstract_inverted_index.In | 0, 43 |
| abstract_inverted_index.an | 28, 48, 83 |
| abstract_inverted_index.in | 125, 132 |
| abstract_inverted_index.is | 14, 33, 123 |
| abstract_inverted_index.of | 4, 7, 40, 57, 90, 113, 120, 128 |
| abstract_inverted_index.to | 30, 36, 67, 82, 85 |
| abstract_inverted_index.we | 46, 62 |
| abstract_inverted_index.The | 20, 135 |
| abstract_inverted_index.all | 96 |
| abstract_inverted_index.and | 93 |
| abstract_inverted_index.are | 17, 108 |
| abstract_inverted_index.for | 72, 105, 111 |
| abstract_inverted_index.how | 15 |
| abstract_inverted_index.new | 99 |
| abstract_inverted_index.our | 78 |
| abstract_inverted_index.the | 18, 41, 58, 69, 87, 94, 126 |
| abstract_inverted_index.This | 118 |
| abstract_inverted_index.also | 109 |
| abstract_inverted_index.both | 91 |
| abstract_inverted_index.data | 2, 73 |
| abstract_inverted_index.from | 134 |
| abstract_inverted_index.high | 8 |
| abstract_inverted_index.null | 88 |
| abstract_inverted_index.only | 64 |
| abstract_inverted_index.test | 26, 51, 70, 106 |
| abstract_inverted_index.that | 32, 52 |
| abstract_inverted_index.this | 31, 44 |
| abstract_inverted_index.with | 74 |
| abstract_inverted_index.data, | 10 |
| abstract_inverted_index.data. | 42 |
| abstract_inverted_index.known | 5 |
| abstract_inverted_index.leads | 81 |
| abstract_inverted_index.major | 55 |
| abstract_inverted_index.power | 71 |
| abstract_inverted_index.types | 131 |
| abstract_inverted_index.Atlas. | 138 |
| abstract_inverted_index.Cancer | 136 |
| abstract_inverted_index.First, | 61 |
| abstract_inverted_index.Genome | 137 |
| abstract_inverted_index.Third, | 98 |
| abstract_inverted_index.across | 115 |
| abstract_inverted_index.answer | 29 |
| abstract_inverted_index.cancer | 130 |
| abstract_inverted_index.paper, | 45 |
| abstract_inverted_index.solves | 53 |
| abstract_inverted_index.strong | 75 |
| abstract_inverted_index.visual | 38 |
| abstract_inverted_index.Second, | 77 |
| abstract_inverted_index.central | 12 |
| abstract_inverted_index.classes | 6 |
| abstract_inverted_index.context | 127 |
| abstract_inverted_index.correct | 86 |
| abstract_inverted_index.propose | 47 |
| abstract_inverted_index.results | 114 |
| abstract_inverted_index.analysis | 3, 39, 80 |
| abstract_inverted_index.balanced | 65, 92 |
| abstract_inverted_index.behavior | 89 |
| abstract_inverted_index.classes? | 19 |
| abstract_inverted_index.directly | 34 |
| abstract_inverted_index.distinct | 16 |
| abstract_inverted_index.examples | 133 |
| abstract_inverted_index.improved | 49 |
| abstract_inverted_index.increase | 68 |
| abstract_inverted_index.original | 59 |
| abstract_inverted_index.proposed | 110 |
| abstract_inverted_index.provides | 27 |
| abstract_inverted_index.question | 13 |
| abstract_inverted_index.signals. | 76 |
| abstract_inverted_index.version. | 60 |
| abstract_inverted_index.DiProPerm | 50, 121 |
| abstract_inverted_index.Direction | 21 |
| abstract_inverted_index.comparing | 129 |
| abstract_inverted_index.connected | 35 |
| abstract_inverted_index.contexts. | 117 |
| abstract_inverted_index.different | 116 |
| abstract_inverted_index.implement | 63 |
| abstract_inverted_index.inference | 122 |
| abstract_inverted_index.intervals | 101 |
| abstract_inverted_index.Projection | 22 |
| abstract_inverted_index.adjustment | 84 |
| abstract_inverted_index.challenges | 56 |
| abstract_inverted_index.comparison | 112 |
| abstract_inverted_index.confidence | 100 |
| abstract_inverted_index.hypothesis | 25 |
| abstract_inverted_index.variation) | 104 |
| abstract_inverted_index.(DiProPerm) | 24 |
| abstract_inverted_index.(reflecting | 102 |
| abstract_inverted_index.Permutation | 23 |
| abstract_inverted_index.dimensional | 9 |
| abstract_inverted_index.exploratory | 1 |
| abstract_inverted_index.illustrated | 124 |
| abstract_inverted_index.improvement | 119 |
| abstract_inverted_index.permutation | 103 |
| abstract_inverted_index.conventional | 95 |
| abstract_inverted_index.mathematical | 79 |
| abstract_inverted_index.permutations | 66 |
| abstract_inverted_index.significance | 107 |
| abstract_inverted_index.permutations. | 97 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 3 |
| citation_normalized_percentile |