Concept-Centric Token Interpretation for Vector-Quantized Generative Models Article Swipe

PDF

Tianze Yang , Yucheng Shi , Mengnan Du , Xuansheng Wu , Qiaoyu Tan , Jing Sun , Ninghao Liu ·

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2506.00698

Vector-Quantized Generative Models (VQGMs) have emerged as powerful tools for image generation. However, the key component of VQGMs -- the codebook of discrete tokens -- is still not well understood, e.g., which tokens are critical to generate an image of a certain concept? This paper introduces Concept-Oriented Token Explanation (CORTEX), a novel approach for interpreting VQGMs by identifying concept-specific token combinations. Our framework employs two methods: (1) a sample-level explanation method that analyzes token importance scores in individual images, and (2) a codebook-level explanation method that explores the entire codebook to find globally relevant tokens. Experimental results demonstrate CORTEX's efficacy in providing clear explanations of token usage in the generative process, outperforming baselines across multiple pretrained VQGMs. Besides enhancing VQGMs transparency, CORTEX is useful in applications such as targeted image editing and shortcut feature detection. Our code is available at https://github.com/YangTianze009/CORTEX.

Related Topics

Concepts

No concepts available.

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2506.00698
PDF: https://arxiv.org/pdf/2506.00698
OA Status: green
OpenAlex ID: https://openalex.org/W4414891767

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4414891767

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2506.00698

Digital Object Identifier
Title: Concept-Centric Token Interpretation for Vector-Quantized Generative Models

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-05-31

Full publication date if available
Authors: Tianze Yang, Yucheng Shi, Mengnan Du, Xuansheng Wu, Qiaoyu Tan, Jing Sun, Ninghao Liu

List of authors in order
Landing page: https://arxiv.org/abs/2506.00698

Publisher landing page
PDF URL: https://arxiv.org/pdf/2506.00698

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2506.00698

Direct OA link when available
Cited by: 0

Total citation count in OpenAlex

Full payload

id	https://openalex.org/W4414891767
doi	https://doi.org/10.48550/arxiv.2506.00698
ids.doi	https://doi.org/10.48550/arxiv.2506.00698
ids.openalex	https://openalex.org/W4414891767
fwci
type	preprint
title	Concept-Centric Token Interpretation for Vector-Quantized Generative Models
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10215
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9860000014305115
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Semantic Web and Ontologies
topics[1].id	https://openalex.org/T10317
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9416000247001648
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1705
topics[1].subfield.display_name	Computer Networks and Communications
topics[1].display_name	Advanced Database Systems and Queries
topics[2].id	https://openalex.org/T11986
topics[2].field.id	https://openalex.org/fields/18
topics[2].field.display_name	Decision Sciences
topics[2].score	0.9332000017166138
topics[2].domain.id	https://openalex.org/domains/2
topics[2].domain.display_name	Social Sciences
topics[2].subfield.id	https://openalex.org/subfields/1802
topics[2].subfield.display_name	Information Systems and Management
topics[2].display_name	Scientific Computing and Data Management
is_xpac	False
apc_list
apc_paid
language	en
locations[0].id	pmh:oai:arXiv.org:2506.00698
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2506.00698
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2506.00698
locations[1].id	doi:10.48550/arxiv.2506.00698
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2506.00698
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5101301594
authorships[0].author.orcid
authorships[0].author.display_name	Tianze Yang
authorships[0].author_position	first
authorships[0].raw_author_name	Yang, Tianze
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5016550690
authorships[1].author.orcid	https://orcid.org/0000-0002-8070-5363
authorships[1].author.display_name	Yucheng Shi
authorships[1].author_position	middle
authorships[1].raw_author_name	Shi, Yucheng
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5072191151
authorships[2].author.orcid	https://orcid.org/0000-0002-1614-6069
authorships[2].author.display_name	Mengnan Du
authorships[2].author_position	middle
authorships[2].raw_author_name	Du, Mengnan
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5089884284
authorships[3].author.orcid	https://orcid.org/0000-0002-7816-7658
authorships[3].author.display_name	Xuansheng Wu
authorships[3].author_position	middle
authorships[3].raw_author_name	Wu, Xuansheng
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5102981159
authorships[4].author.orcid	https://orcid.org/0000-0002-1206-8446
authorships[4].author.display_name	Qiaoyu Tan
authorships[4].author_position	middle
authorships[4].raw_author_name	Tan, Qiaoyu
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5084282975
authorships[5].author.orcid	https://orcid.org/0000-0001-7254-454X
authorships[5].author.display_name	Jing Sun
authorships[5].author_position	middle
authorships[5].raw_author_name	Sun, Jin
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5007489034
authorships[6].author.orcid	https://orcid.org/0000-0002-9170-2424
authorships[6].author.display_name	Ninghao Liu
authorships[6].author_position	last
authorships[6].raw_author_name	Liu, Ninghao
authorships[6].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2506.00698
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Concept-Centric Token Interpretation for Vector-Quantized Generative Models
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10215
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9860000014305115
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Semantic Web and Ontologies
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2506.00698
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2506.00698
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2506.00698
primary_location.id	pmh:oai:arXiv.org:2506.00698
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2506.00698
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2506.00698
publication_date	2025-05-31
publication_year	2025
referenced_works_count	0
abstract_inverted_index.a	40, 50, 67, 81
abstract_inverted_index.--	18, 24
abstract_inverted_index.an	37
abstract_inverted_index.as	6, 127
abstract_inverted_index.at	139
abstract_inverted_index.by	56
abstract_inverted_index.in	76, 100, 107, 124
abstract_inverted_index.is	25, 122, 137
abstract_inverted_index.of	16, 21, 39, 104
abstract_inverted_index.to	35, 90
abstract_inverted_index.(1)	66
abstract_inverted_index.(2)	80
abstract_inverted_index.Our	61, 135
abstract_inverted_index.and	79, 131
abstract_inverted_index.are	33
abstract_inverted_index.for	9, 53
abstract_inverted_index.key	14
abstract_inverted_index.not	27
abstract_inverted_index.the	13, 19, 87, 108
abstract_inverted_index.two	64
abstract_inverted_index.This	43
abstract_inverted_index.code	136
abstract_inverted_index.find	91
abstract_inverted_index.have	4
abstract_inverted_index.such	126
abstract_inverted_index.that	71, 85
abstract_inverted_index.well	28
abstract_inverted_index.Token	47
abstract_inverted_index.VQGMs	17, 55, 119
abstract_inverted_index.clear	102
abstract_inverted_index.e.g.,	30
abstract_inverted_index.image	10, 38, 129
abstract_inverted_index.novel	51
abstract_inverted_index.paper	44
abstract_inverted_index.still	26
abstract_inverted_index.token	59, 73, 105
abstract_inverted_index.tools	8
abstract_inverted_index.usage	106
abstract_inverted_index.which	31
abstract_inverted_index.CORTEX	121
abstract_inverted_index.Models	2
abstract_inverted_index.VQGMs.	116
abstract_inverted_index.across	113
abstract_inverted_index.entire	88
abstract_inverted_index.method	70, 84
abstract_inverted_index.scores	75
abstract_inverted_index.tokens	23, 32
abstract_inverted_index.useful	123
abstract_inverted_index.(VQGMs)	3
abstract_inverted_index.Besides	117
abstract_inverted_index.certain	41
abstract_inverted_index.editing	130
abstract_inverted_index.emerged	5
abstract_inverted_index.employs	63
abstract_inverted_index.feature	133
abstract_inverted_index.images,	78
abstract_inverted_index.results	96
abstract_inverted_index.tokens.	94
abstract_inverted_index.CORTEX's	98
abstract_inverted_index.However,	12
abstract_inverted_index.analyzes	72
abstract_inverted_index.approach	52
abstract_inverted_index.codebook	20, 89
abstract_inverted_index.concept?	42
abstract_inverted_index.critical	34
abstract_inverted_index.discrete	22
abstract_inverted_index.efficacy	99
abstract_inverted_index.explores	86
abstract_inverted_index.generate	36
abstract_inverted_index.globally	92
abstract_inverted_index.methods:	65
abstract_inverted_index.multiple	114
abstract_inverted_index.powerful	7
abstract_inverted_index.process,	110
abstract_inverted_index.relevant	93
abstract_inverted_index.shortcut	132
abstract_inverted_index.targeted	128
abstract_inverted_index.(CORTEX),	49
abstract_inverted_index.available	138
abstract_inverted_index.baselines	112
abstract_inverted_index.component	15
abstract_inverted_index.enhancing	118
abstract_inverted_index.framework	62
abstract_inverted_index.providing	101
abstract_inverted_index.Generative	1
abstract_inverted_index.detection.	134
abstract_inverted_index.generative	109
abstract_inverted_index.importance	74
abstract_inverted_index.individual	77
abstract_inverted_index.introduces	45
abstract_inverted_index.pretrained	115
abstract_inverted_index.Explanation	48
abstract_inverted_index.demonstrate	97
abstract_inverted_index.explanation	69, 83
abstract_inverted_index.generation.	11
abstract_inverted_index.identifying	57
abstract_inverted_index.understood,	29
abstract_inverted_index.Experimental	95
abstract_inverted_index.applications	125
abstract_inverted_index.explanations	103
abstract_inverted_index.interpreting	54
abstract_inverted_index.sample-level	68
abstract_inverted_index.combinations.	60
abstract_inverted_index.outperforming	111
abstract_inverted_index.transparency,	120
abstract_inverted_index.codebook-level	82
abstract_inverted_index.Concept-Oriented	46
abstract_inverted_index.Vector-Quantized	0
abstract_inverted_index.concept-specific	58
abstract_inverted_index.https://github.com/YangTianze009/CORTEX.	140
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	7
citation_normalized_percentile