HHFT: Hierarchical Heterogeneous Feature Transformer for Recommendation Systems Article Swipe

PDF

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2511.20235

We propose HHFT (Hierarchical Heterogeneous Feature Transformer), a Transformer-based architecture tailored for industrial CTR prediction. HHFT addresses the limitations of DNN through three key designs: (1) Semantic Feature Partitioning: Grouping heterogeneous features (e.g. user profile, item information, behaviour sequennce) into semantically coherent blocks to preserve domain-specific information; (2) Heterogeneous Transformer Encoder: Adopting block-specific QKV projections and FFNs to avoid semantic confusion between distinct feature types; (3) Hiformer Layer: Capturing high-order interactions across features. Our findings reveal that Transformers significantly outperform DNN baselines, achieving a +0.4% improvement in CTR AUC at scale. We have successfully deployed the model on Taobao's production platform, observing a significant uplift in key business metrics, including a +0.6% increase in Gross Merchandise Value (GMV).

Related Topics

Truth And Reconciliation Commission Of Canada

Concepts

No concepts available.

Metadata

Type: preprint
Landing Page: http://arxiv.org/abs/2511.20235
PDF: https://arxiv.org/pdf/2511.20235
OA Status: green
OpenAlex ID: https://openalex.org/W4416771125

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4416771125

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2511.20235

Digital Object Identifier
Title: HHFT: Hierarchical Heterogeneous Feature Transformer for Recommendation Systems

Work title
Type: preprint

OpenAlex work type
Publication year: 2025

Year of publication
Publication date: 2025-11-25

Full publication date if available
Authors: Liang Yu, Wenming Zhang, Dan Ou

List of authors in order
Landing page: https://arxiv.org/abs/2511.20235

Publisher landing page
PDF URL: https://arxiv.org/pdf/2511.20235

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2511.20235

Direct OA link when available
Cited by: 0

Total citation count in OpenAlex

Full payload

id	https://openalex.org/W4416771125
doi	https://doi.org/10.48550/arxiv.2511.20235
ids.doi	https://doi.org/10.48550/arxiv.2511.20235
ids.openalex	https://openalex.org/W4416771125
fwci
type	preprint
title	HHFT: Hierarchical Heterogeneous Feature Transformer for Recommendation Systems
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
is_xpac	False
apc_list
apc_paid
language
locations[0].id	pmh:oai:arXiv.org:2511.20235
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2511.20235
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2511.20235
locations[1].id	doi:10.48550/arxiv.2511.20235
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2511.20235
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5101814743
authorships[0].author.orcid	https://orcid.org/0009-0007-3922-3454
authorships[0].author.display_name	Liang Yu
authorships[0].author_position	first
authorships[0].raw_author_name	Yu, Liren
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5073386986
authorships[1].author.orcid	https://orcid.org/0000-0001-6743-1006
authorships[1].author.display_name	Wenming Zhang
authorships[1].author_position	middle
authorships[1].raw_author_name	Zhang, Wenming
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5013687810
authorships[2].author.orcid	https://orcid.org/0000-0001-9485-6042
authorships[2].author.display_name	Dan Ou
authorships[2].author_position	middle
authorships[2].raw_author_name	Ou, Dan
authorships[2].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2511.20235
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-11-28T00:00:00
display_name	HHFT: Hierarchical Heterogeneous Feature Transformer for Recommendation Systems
has_fulltext	False
is_retracted	False
updated_date	2025-11-28T23:01:20.083199
primary_topic
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2511.20235
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2511.20235
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2511.20235
primary_location.id	pmh:oai:arXiv.org:2511.20235
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2511.20235
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2511.20235
publication_date	2025-11-25
publication_year	2025
referenced_works_count	0
abstract_inverted_index.a	7, 83, 102, 110
abstract_inverted_index.We	0, 91
abstract_inverted_index.at	89
abstract_inverted_index.in	86, 105, 113
abstract_inverted_index.of	19
abstract_inverted_index.on	97
abstract_inverted_index.to	43, 57
abstract_inverted_index.(1)	25
abstract_inverted_index.(2)	47
abstract_inverted_index.(3)	65
abstract_inverted_index.AUC	88
abstract_inverted_index.CTR	13, 87
abstract_inverted_index.DNN	20, 80
abstract_inverted_index.Our	73
abstract_inverted_index.QKV	53
abstract_inverted_index.and	55
abstract_inverted_index.for	11
abstract_inverted_index.key	23, 106
abstract_inverted_index.the	17, 95
abstract_inverted_index.FFNs	56
abstract_inverted_index.HHFT	2, 15
abstract_inverted_index.have	92
abstract_inverted_index.into	39
abstract_inverted_index.item	35
abstract_inverted_index.that	76
abstract_inverted_index.user	33
abstract_inverted_index.(e.g.	32
abstract_inverted_index.+0.4%	84
abstract_inverted_index.+0.6%	111
abstract_inverted_index.Gross	114
abstract_inverted_index.Value	116
abstract_inverted_index.avoid	58
abstract_inverted_index.model	96
abstract_inverted_index.three	22
abstract_inverted_index.(GMV).	117
abstract_inverted_index.Layer:	67
abstract_inverted_index.across	71
abstract_inverted_index.blocks	42
abstract_inverted_index.reveal	75
abstract_inverted_index.scale.	90
abstract_inverted_index.types;	64
abstract_inverted_index.uplift	104
abstract_inverted_index.Feature	5, 27
abstract_inverted_index.between	61
abstract_inverted_index.feature	63
abstract_inverted_index.propose	1
abstract_inverted_index.through	21
abstract_inverted_index.Adopting	51
abstract_inverted_index.Encoder:	50
abstract_inverted_index.Grouping	29
abstract_inverted_index.Hiformer	66
abstract_inverted_index.Semantic	26
abstract_inverted_index.Taobao's	98
abstract_inverted_index.business	107
abstract_inverted_index.coherent	41
abstract_inverted_index.deployed	94
abstract_inverted_index.designs:	24
abstract_inverted_index.distinct	62
abstract_inverted_index.features	31
abstract_inverted_index.findings	74
abstract_inverted_index.increase	112
abstract_inverted_index.metrics,	108
abstract_inverted_index.preserve	44
abstract_inverted_index.profile,	34
abstract_inverted_index.semantic	59
abstract_inverted_index.tailored	10
abstract_inverted_index.Capturing	68
abstract_inverted_index.achieving	82
abstract_inverted_index.addresses	16
abstract_inverted_index.behaviour	37
abstract_inverted_index.confusion	60
abstract_inverted_index.features.	72
abstract_inverted_index.including	109
abstract_inverted_index.observing	101
abstract_inverted_index.platform,	100
abstract_inverted_index.baselines,	81
abstract_inverted_index.high-order	69
abstract_inverted_index.industrial	12
abstract_inverted_index.outperform	79
abstract_inverted_index.production	99
abstract_inverted_index.sequennce)	38
abstract_inverted_index.Merchandise	115
abstract_inverted_index.Transformer	49
abstract_inverted_index.improvement	85
abstract_inverted_index.limitations	18
abstract_inverted_index.prediction.	14
abstract_inverted_index.projections	54
abstract_inverted_index.significant	103
abstract_inverted_index.Transformers	77
abstract_inverted_index.architecture	9
abstract_inverted_index.information,	36
abstract_inverted_index.information;	46
abstract_inverted_index.interactions	70
abstract_inverted_index.semantically	40
abstract_inverted_index.successfully	93
abstract_inverted_index.(Hierarchical	3
abstract_inverted_index.Heterogeneous	4, 48
abstract_inverted_index.Partitioning:	28
abstract_inverted_index.Transformer),	6
abstract_inverted_index.heterogeneous	30
abstract_inverted_index.significantly	78
abstract_inverted_index.block-specific	52
abstract_inverted_index.domain-specific	45
abstract_inverted_index.Transformer-based	8
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	3
citation_normalized_percentile