Linking in Style: Understanding learned features in deep learning models Article Swipe
Maren H. Wehrheim
,
Pamela Osuna-Vargas
,
Matthias Kaschube
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2409.16865
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2409.16865
Convolutional neural networks (CNNs) learn abstract features to perform object classification, but understanding these features remains challenging due to difficult-to-interpret results or high computational costs. We propose an automatic method to visualize and systematically analyze learned features in CNNs. Specifically, we introduce a linking network that maps the penultimate layer of a pre-trained classifier to the latent space of a generative model (StyleGAN-XL), thereby enabling an interpretable, human-friendly visualization of the classifier's representations. Our findings indicate a congruent semantic order in both spaces, enabling a direct linear mapping between them. Training the linking network is computationally inexpensive and decoupled from training both the GAN and the classifier. We introduce an automatic pipeline that utilizes such GAN-based visualizations to quantify learned representations by analyzing activation changes in the classifier in the image domain. This quantification allows us to systematically study the learned representations in several thousand units simultaneously and to extract and visualize units selective for specific semantic concepts. Further, we illustrate how our method can be used to quantify and interpret the classifier's decision boundary using counterfactual examples. Overall, our method offers systematic and objective perspectives on learned abstract representations in CNNs. https://github.com/kaschube-lab/LinkingInStyle.git
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2409.16865
- https://arxiv.org/pdf/2409.16865
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4403784607
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4403784607Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2409.16865Digital Object Identifier
- Title
-
Linking in Style: Understanding learned features in deep learning modelsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-09-25Full publication date if available
- Authors
-
Maren H. Wehrheim, Pamela Osuna-Vargas, Matthias KaschubeList of authors in order
- Landing page
-
https://arxiv.org/abs/2409.16865Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2409.16865Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2409.16865Direct OA link when available
- Concepts
-
Style (visual arts), Deep learning, Artificial intelligence, Computer science, Natural language processing, Cognitive science, Psychology, History, ArchaeologyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4403784607 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2409.16865 |
| ids.doi | https://doi.org/10.48550/arxiv.2409.16865 |
| ids.openalex | https://openalex.org/W4403784607 |
| fwci | |
| type | preprint |
| title | Linking in Style: Understanding learned features in deep learning models |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10320 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.04650000110268593 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Neural Networks and Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2776445246 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7701114416122437 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1792644 |
| concepts[0].display_name | Style (visual arts) |
| concepts[1].id | https://openalex.org/C108583219 |
| concepts[1].level | 2 |
| concepts[1].score | 0.5260878801345825 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q197536 |
| concepts[1].display_name | Deep learning |
| concepts[2].id | https://openalex.org/C154945302 |
| concepts[2].level | 1 |
| concepts[2].score | 0.4976647198200226 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[2].display_name | Artificial intelligence |
| concepts[3].id | https://openalex.org/C41008148 |
| concepts[3].level | 0 |
| concepts[3].score | 0.45620203018188477 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[3].display_name | Computer science |
| concepts[4].id | https://openalex.org/C204321447 |
| concepts[4].level | 1 |
| concepts[4].score | 0.3232388496398926 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[4].display_name | Natural language processing |
| concepts[5].id | https://openalex.org/C188147891 |
| concepts[5].level | 1 |
| concepts[5].score | 0.32221847772598267 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q147638 |
| concepts[5].display_name | Cognitive science |
| concepts[6].id | https://openalex.org/C15744967 |
| concepts[6].level | 0 |
| concepts[6].score | 0.2831485867500305 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[6].display_name | Psychology |
| concepts[7].id | https://openalex.org/C95457728 |
| concepts[7].level | 0 |
| concepts[7].score | 0.1719077229499817 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q309 |
| concepts[7].display_name | History |
| concepts[8].id | https://openalex.org/C166957645 |
| concepts[8].level | 1 |
| concepts[8].score | 0.11442902684211731 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q23498 |
| concepts[8].display_name | Archaeology |
| keywords[0].id | https://openalex.org/keywords/style |
| keywords[0].score | 0.7701114416122437 |
| keywords[0].display_name | Style (visual arts) |
| keywords[1].id | https://openalex.org/keywords/deep-learning |
| keywords[1].score | 0.5260878801345825 |
| keywords[1].display_name | Deep learning |
| keywords[2].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[2].score | 0.4976647198200226 |
| keywords[2].display_name | Artificial intelligence |
| keywords[3].id | https://openalex.org/keywords/computer-science |
| keywords[3].score | 0.45620203018188477 |
| keywords[3].display_name | Computer science |
| keywords[4].id | https://openalex.org/keywords/natural-language-processing |
| keywords[4].score | 0.3232388496398926 |
| keywords[4].display_name | Natural language processing |
| keywords[5].id | https://openalex.org/keywords/cognitive-science |
| keywords[5].score | 0.32221847772598267 |
| keywords[5].display_name | Cognitive science |
| keywords[6].id | https://openalex.org/keywords/psychology |
| keywords[6].score | 0.2831485867500305 |
| keywords[6].display_name | Psychology |
| keywords[7].id | https://openalex.org/keywords/history |
| keywords[7].score | 0.1719077229499817 |
| keywords[7].display_name | History |
| keywords[8].id | https://openalex.org/keywords/archaeology |
| keywords[8].score | 0.11442902684211731 |
| keywords[8].display_name | Archaeology |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2409.16865 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2409.16865 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2409.16865 |
| locations[1].id | doi:10.48550/arxiv.2409.16865 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2409.16865 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5019922546 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-3197-9947 |
| authorships[0].author.display_name | Maren H. Wehrheim |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Wehrheim, Maren H. |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5093315757 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Pamela Osuna-Vargas |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Osuna-Vargas, Pamela |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5066601492 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-5145-7487 |
| authorships[2].author.display_name | Matthias Kaschube |
| authorships[2].author_position | last |
| authorships[2].raw_author_name | Kaschube, Matthias |
| authorships[2].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2409.16865 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Linking in Style: Understanding learned features in deep learning models |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10320 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.04650000110268593 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Neural Networks and Applications |
| related_works | https://openalex.org/W2731899572, https://openalex.org/W3215138031, https://openalex.org/W4321369474, https://openalex.org/W3009238340, https://openalex.org/W4360585206, https://openalex.org/W4285208911, https://openalex.org/W3082895349, https://openalex.org/W4213079790, https://openalex.org/W2248239756, https://openalex.org/W4323565446 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2409.16865 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2409.16865 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2409.16865 |
| primary_location.id | pmh:oai:arXiv.org:2409.16865 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2409.16865 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2409.16865 |
| publication_date | 2024-09-25 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 42, 51, 59, 76, 84 |
| abstract_inverted_index.We | 25, 107 |
| abstract_inverted_index.an | 27, 65, 109 |
| abstract_inverted_index.be | 165 |
| abstract_inverted_index.by | 121 |
| abstract_inverted_index.in | 37, 80, 125, 128, 142, 190 |
| abstract_inverted_index.is | 94 |
| abstract_inverted_index.of | 50, 58, 69 |
| abstract_inverted_index.on | 186 |
| abstract_inverted_index.or | 21 |
| abstract_inverted_index.to | 7, 18, 30, 54, 117, 136, 148, 167 |
| abstract_inverted_index.us | 135 |
| abstract_inverted_index.we | 40, 159 |
| abstract_inverted_index.GAN | 103 |
| abstract_inverted_index.Our | 73 |
| abstract_inverted_index.and | 32, 97, 104, 147, 150, 169, 183 |
| abstract_inverted_index.but | 11 |
| abstract_inverted_index.can | 164 |
| abstract_inverted_index.due | 17 |
| abstract_inverted_index.for | 154 |
| abstract_inverted_index.how | 161 |
| abstract_inverted_index.our | 162, 179 |
| abstract_inverted_index.the | 47, 55, 70, 91, 102, 105, 126, 129, 139, 171 |
| abstract_inverted_index.This | 132 |
| abstract_inverted_index.both | 81, 101 |
| abstract_inverted_index.from | 99 |
| abstract_inverted_index.high | 22 |
| abstract_inverted_index.maps | 46 |
| abstract_inverted_index.such | 114 |
| abstract_inverted_index.that | 45, 112 |
| abstract_inverted_index.used | 166 |
| abstract_inverted_index.CNNs. | 38, 191 |
| abstract_inverted_index.image | 130 |
| abstract_inverted_index.layer | 49 |
| abstract_inverted_index.learn | 4 |
| abstract_inverted_index.model | 61 |
| abstract_inverted_index.order | 79 |
| abstract_inverted_index.space | 57 |
| abstract_inverted_index.study | 138 |
| abstract_inverted_index.them. | 89 |
| abstract_inverted_index.these | 13 |
| abstract_inverted_index.units | 145, 152 |
| abstract_inverted_index.using | 175 |
| abstract_inverted_index.(CNNs) | 3 |
| abstract_inverted_index.allows | 134 |
| abstract_inverted_index.costs. | 24 |
| abstract_inverted_index.direct | 85 |
| abstract_inverted_index.latent | 56 |
| abstract_inverted_index.linear | 86 |
| abstract_inverted_index.method | 29, 163, 180 |
| abstract_inverted_index.neural | 1 |
| abstract_inverted_index.object | 9 |
| abstract_inverted_index.offers | 181 |
| abstract_inverted_index.analyze | 34 |
| abstract_inverted_index.between | 88 |
| abstract_inverted_index.changes | 124 |
| abstract_inverted_index.domain. | 131 |
| abstract_inverted_index.extract | 149 |
| abstract_inverted_index.learned | 35, 119, 140, 187 |
| abstract_inverted_index.linking | 43, 92 |
| abstract_inverted_index.mapping | 87 |
| abstract_inverted_index.network | 44, 93 |
| abstract_inverted_index.perform | 8 |
| abstract_inverted_index.propose | 26 |
| abstract_inverted_index.remains | 15 |
| abstract_inverted_index.results | 20 |
| abstract_inverted_index.several | 143 |
| abstract_inverted_index.spaces, | 82 |
| abstract_inverted_index.thereby | 63 |
| abstract_inverted_index.Further, | 158 |
| abstract_inverted_index.Overall, | 178 |
| abstract_inverted_index.Training | 90 |
| abstract_inverted_index.abstract | 5, 188 |
| abstract_inverted_index.boundary | 174 |
| abstract_inverted_index.decision | 173 |
| abstract_inverted_index.enabling | 64, 83 |
| abstract_inverted_index.features | 6, 14, 36 |
| abstract_inverted_index.findings | 74 |
| abstract_inverted_index.indicate | 75 |
| abstract_inverted_index.networks | 2 |
| abstract_inverted_index.pipeline | 111 |
| abstract_inverted_index.quantify | 118, 168 |
| abstract_inverted_index.semantic | 78, 156 |
| abstract_inverted_index.specific | 155 |
| abstract_inverted_index.thousand | 144 |
| abstract_inverted_index.training | 100 |
| abstract_inverted_index.utilizes | 113 |
| abstract_inverted_index.GAN-based | 115 |
| abstract_inverted_index.analyzing | 122 |
| abstract_inverted_index.automatic | 28, 110 |
| abstract_inverted_index.concepts. | 157 |
| abstract_inverted_index.congruent | 77 |
| abstract_inverted_index.decoupled | 98 |
| abstract_inverted_index.examples. | 177 |
| abstract_inverted_index.interpret | 170 |
| abstract_inverted_index.introduce | 41, 108 |
| abstract_inverted_index.objective | 184 |
| abstract_inverted_index.selective | 153 |
| abstract_inverted_index.visualize | 31, 151 |
| abstract_inverted_index.activation | 123 |
| abstract_inverted_index.classifier | 53, 127 |
| abstract_inverted_index.generative | 60 |
| abstract_inverted_index.illustrate | 160 |
| abstract_inverted_index.systematic | 182 |
| abstract_inverted_index.challenging | 16 |
| abstract_inverted_index.classifier. | 106 |
| abstract_inverted_index.inexpensive | 96 |
| abstract_inverted_index.penultimate | 48 |
| abstract_inverted_index.pre-trained | 52 |
| abstract_inverted_index.classifier's | 71, 172 |
| abstract_inverted_index.perspectives | 185 |
| abstract_inverted_index.Convolutional | 0 |
| abstract_inverted_index.Specifically, | 39 |
| abstract_inverted_index.computational | 23 |
| abstract_inverted_index.understanding | 12 |
| abstract_inverted_index.visualization | 68 |
| abstract_inverted_index.(StyleGAN-XL), | 62 |
| abstract_inverted_index.counterfactual | 176 |
| abstract_inverted_index.human-friendly | 67 |
| abstract_inverted_index.interpretable, | 66 |
| abstract_inverted_index.quantification | 133 |
| abstract_inverted_index.simultaneously | 146 |
| abstract_inverted_index.systematically | 33, 137 |
| abstract_inverted_index.visualizations | 116 |
| abstract_inverted_index.classification, | 10 |
| abstract_inverted_index.computationally | 95 |
| abstract_inverted_index.representations | 120, 141, 189 |
| abstract_inverted_index.representations. | 72 |
| abstract_inverted_index.difficult-to-interpret | 19 |
| abstract_inverted_index.https://github.com/kaschube-lab/LinkingInStyle.git | 192 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 3 |
| citation_normalized_percentile |