Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent Article Swipe

PDF

Xingwu Sun , Yanfeng Chen , Yiqing Huang , Ruobing Xie , Jiaqi Zhu , Kai Zhang , Shuaipeng Li , Zhen Yang , J. L. Han , Shu Xiao-bo , Jiahao Bu , Z C Chen , X. P. Huang , Fengzong Lian , S. Yang , Jianfeng Yan , Yuyuan Zeng , Xiaoqin Ren , Chao Yu , Lulu Wu , Yue Mao , Jun Xia , Tao Yang , Suncong Zheng , Kai Wu , Dian Jiao , Jinbao Xue , Xipeng Zhang , Decheng Wu , Kai Liu , Dan Wu , Guanghui Xu , Shaohua Chen , Shuang Chen , Xiao Feng , Yigeng Hong , Junqiang Zheng , Chengcheng Xu , Zongwei Li , Xiang Kuang , Jiaochan Hu , Yiqi Chen , Yuchi Deng , Guiyang Li , Ao Liu , Chenchen Zhang , Shihui Hu , Zilong Zhao , Zifan Wu , Yao Ding , Weichao Wang , H. B. Liu , Rong Wang , Fei Hao , Peijie Yu , Ze Zhao , Xun Cao , Hai Wang , Fenfen Xiang , Mengyuan Huang , Zhiyuan Xiong , Bin Hu , X. T. Hou , Lei Jiang , Jianqiang Ma , Jiajia Wu , Yaping Deng , Yi Shen , Qian Wang , Weijie Liu , Jie Liu , Meng Chen , Liang Dong , W.H. Jia , Chen Hu , Feifei Liu , Rui Yuan , Huilin Xu , Z.-Y. Yan , Tengfei Cao , Zhichao Hu , Xin‐Hua Feng , Du Dong , Tao Yu , Yangyu Tao , Feng Zhang , Jianchen Zhu , Chengzhong Xu , Xirui Li , Chang‐Sheng Zha , Wen Ouyang , Yinben Xia , Xiang Li , Zhidong He , R Chen , Jiawei Song , Ruibin Chen , Fan Jiang , Chongqing Zhao , Wei Wang ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2411.02265

In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens. We conduct a thorough evaluation of Hunyuan-Large's superior performance across various benchmarks including language understanding and generation, logical reasoning, mathematical problem-solving, coding, long-context, and aggregated tasks, where it outperforms LLama3.1-70B and exhibits comparable performance when compared to the significantly larger LLama3.1-405B model. Key practice of Hunyuan-Large include large-scale synthetic data that is orders larger than in previous literature, a mixed expert routing strategy, a key-value cache compression technique, and an expert-specific learning rate strategy. Additionally, we also investigate the scaling laws and learning rate schedule of mixture of experts models, providing valuable insights and guidances for future model development and optimization. The code and checkpoints of Hunyuan-Large are released to facilitate future innovations and applications. Codes: https://github.com/Tencent/Hunyuan-Large Models: https://huggingface.co/tencent/Tencent-Hunyuan-Large

Related Topics

Computer Science

Concepts

Open source Computer science Operating system Software

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2411.02265
PDF: https://arxiv.org/pdf/2411.02265
OA Status: green
Cited By: 5
Related Works: 10
OpenAlex ID: https://openalex.org/W4404354211

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4404354211

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2411.02265

Digital Object Identifier
Title: Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-11-04

Full publication date if available
Authors: Xingwu Sun, Yanfeng Chen, Yiqing Huang, Ruobing Xie, Jiaqi Zhu, Kai Zhang, Shuaipeng Li, Zhen Yang, J. L. Han, Shu Xiao-bo, Jiahao Bu, Z C Chen, X. P. Huang, Fengzong Lian, S. Yang, Jianfeng Yan, Yuyuan Zeng, Xiaoqin Ren, Chao Yu, Lulu Wu, Yue Mao, Jun Xia, Tao Yang, Suncong Zheng, Kai Wu, Dian Jiao, Jinbao Xue, Xipeng Zhang, Decheng Wu, Kai Liu, Dan Wu, Guanghui Xu, Shaohua Chen, Shuang Chen, Xiao Feng, Yigeng Hong, Junqiang Zheng, Chengcheng Xu, Zongwei Li, Xiang Kuang, Jiaochan Hu, Yiqi Chen, Yuchi Deng, Guiyang Li, Ao Liu, Chenchen Zhang, Shihui Hu, Zilong Zhao, Zifan Wu, Yao Ding, Weichao Wang, H. B. Liu, Rong Wang, Fei Hao, Peijie Yu, Ze Zhao, Xun Cao, Hai Wang, Fenfen Xiang, Mengyuan Huang, Zhiyuan Xiong, Bin Hu, X. T. Hou, Lei Jiang, Jianqiang Ma, Jiajia Wu, Yaping Deng, Yi Shen, Qian Wang, Weijie Liu, Jie Liu, Meng Chen, Liang Dong, W.H. Jia, Chen Hu, Feifei Liu, Rui Yuan, Huilin Xu, Z.-Y. Yan, Tengfei Cao, Zhichao Hu, Xin‐Hua Feng, Du Dong, Tao Yu, Yangyu Tao, Feng Zhang, Jianchen Zhu, Chengzhong Xu, Xirui Li, Chang‐Sheng Zha, Wen Ouyang, Yinben Xia, Xiang Li, Zhidong He, R Chen, Jiawei Song, Ruibin Chen, Fan Jiang, Chongqing Zhao, Wei Wang

List of authors in order
Landing page: https://arxiv.org/abs/2411.02265

Publisher landing page
PDF URL: https://arxiv.org/pdf/2411.02265

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2411.02265

Direct OA link when available
Concepts: Open source, Computer science, Operating system, Software

Top concepts (fields/topics) attached by OpenAlex
Cited by: 5

Total citation count in OpenAlex
Citations by year (recent): 2025: 5

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4404354211
doi	https://doi.org/10.48550/arxiv.2411.02265
ids.doi	https://doi.org/10.48550/arxiv.2411.02265
ids.openalex	https://openalex.org/W4404354211
fwci
type	preprint
title	Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T13553
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.7493000030517578
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1705
topics[0].subfield.display_name	Computer Networks and Communications
topics[0].display_name	Age of Information Optimization
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C3018397939
concepts[0].level	3
concepts[0].score	0.6476789712905884
concepts[0].wikidata	https://www.wikidata.org/wiki/Q3644502
concepts[0].display_name	Open source
concepts[1].id	https://openalex.org/C41008148
concepts[1].level	0
concepts[1].score	0.3558697998523712
concepts[1].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[1].display_name	Computer science
concepts[2].id	https://openalex.org/C111919701
concepts[2].level	1
concepts[2].score	0.1830577552318573
concepts[2].wikidata	https://www.wikidata.org/wiki/Q9135
concepts[2].display_name	Operating system
concepts[3].id	https://openalex.org/C2777904410
concepts[3].level	2
concepts[3].score	0.08893615007400513
concepts[3].wikidata	https://www.wikidata.org/wiki/Q7397
concepts[3].display_name	Software
keywords[0].id	https://openalex.org/keywords/open-source
keywords[0].score	0.6476789712905884
keywords[0].display_name	Open source
keywords[1].id	https://openalex.org/keywords/computer-science
keywords[1].score	0.3558697998523712
keywords[1].display_name	Computer science
keywords[2].id	https://openalex.org/keywords/operating-system
keywords[2].score	0.1830577552318573
keywords[2].display_name	Operating system
keywords[3].id	https://openalex.org/keywords/software
keywords[3].score	0.08893615007400513
keywords[3].display_name	Software
language	en
locations[0].id	pmh:oai:arXiv.org:2411.02265
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2411.02265
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2411.02265
locations[1].id	doi:10.48550/arxiv.2411.02265
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2411.02265
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5039337290
authorships[0].author.orcid
authorships[0].author.display_name	Xingwu Sun
authorships[0].author_position	first
authorships[0].raw_author_name	Sun, Xingwu
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5082902686
authorships[1].author.orcid	https://orcid.org/0000-0002-5377-2093
authorships[1].author.display_name	Yanfeng Chen
authorships[1].author_position	middle
authorships[1].raw_author_name	Chen, Yanfeng
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5086362466
authorships[2].author.orcid	https://orcid.org/0000-0001-5341-7369
authorships[2].author.display_name	Yiqing Huang
authorships[2].author_position	middle
authorships[2].raw_author_name	Huang, Yiqing
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5101577090
authorships[3].author.orcid	https://orcid.org/0000-0003-3170-5647
authorships[3].author.display_name	Ruobing Xie
authorships[3].author_position	middle
authorships[3].raw_author_name	Xie, Ruobing
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5007671115
authorships[4].author.orcid	https://orcid.org/0000-0002-0886-4845
authorships[4].author.display_name	Jiaqi Zhu
authorships[4].author_position	middle
authorships[4].raw_author_name	Zhu, Jiaqi
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5101886594
authorships[5].author.orcid	https://orcid.org/0009-0005-1294-2876
authorships[5].author.display_name	Kai Zhang
authorships[5].author_position	middle
authorships[5].raw_author_name	Zhang, Kai
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5113103847
authorships[6].author.orcid	https://orcid.org/0000-0002-9182-072X
authorships[6].author.display_name	Shuaipeng Li
authorships[6].author_position	middle
authorships[6].raw_author_name	Li, Shuaipeng
authorships[6].is_corresponding	False
authorships[7].author.id	https://openalex.org/A5042463866
authorships[7].author.orcid	https://orcid.org/0000-0001-9763-6766
authorships[7].author.display_name	Zhen Yang
authorships[7].author_position	middle
authorships[7].raw_author_name	Yang, Zhen
authorships[7].is_corresponding	False
authorships[8].author.id	https://openalex.org/A5002756705
authorships[8].author.orcid	https://orcid.org/0000-0002-9274-3092
authorships[8].author.display_name	J. L. Han
authorships[8].author_position	middle
authorships[8].raw_author_name	Han, Jonny
authorships[8].is_corresponding	False
authorships[9].author.id	https://openalex.org/A5039304812
authorships[9].author.orcid
authorships[9].author.display_name	Shu Xiao-bo
authorships[9].author_position	middle
authorships[9].raw_author_name	Shu, Xiaobo
authorships[9].is_corresponding	False
authorships[10].author.id	https://openalex.org/A5110748360
authorships[10].author.orcid
authorships[10].author.display_name	Jiahao Bu
authorships[10].author_position	middle
authorships[10].raw_author_name	Bu, Jiahao
authorships[10].is_corresponding	False
authorships[11].author.id	https://openalex.org/A5087551278
authorships[11].author.orcid
authorships[11].author.display_name	Z C Chen
authorships[11].author_position	middle
authorships[11].raw_author_name	Chen, Zhongzhi
authorships[11].is_corresponding	False
authorships[12].author.id	https://openalex.org/A5049888558
authorships[12].author.orcid	https://orcid.org/0009-0005-8098-1551
authorships[12].author.display_name	X. P. Huang
authorships[12].author_position	middle
authorships[12].raw_author_name	Huang, Xuemeng
authorships[12].is_corresponding	False
authorships[13].author.id	https://openalex.org/A5091705422
authorships[13].author.orcid	https://orcid.org/0000-0002-9871-3038
authorships[13].author.display_name	Fengzong Lian
authorships[13].author_position	middle
authorships[13].raw_author_name	Lian, Fengzong
authorships[13].is_corresponding	False
authorships[14].author.id	https://openalex.org/A5034205662
authorships[14].author.orcid	https://orcid.org/0000-0002-2898-6532
authorships[14].author.display_name	S. Yang
authorships[14].author_position	middle
authorships[14].raw_author_name	Yang, Saiyong
authorships[14].is_corresponding	False
authorships[15].author.id	https://openalex.org/A5053937579
authorships[15].author.orcid
authorships[15].author.display_name	Jianfeng Yan
authorships[15].author_position	middle
authorships[15].raw_author_name	Yan, Jianfeng
authorships[15].is_corresponding	False
authorships[16].author.id	https://openalex.org/A5076901150
authorships[16].author.orcid
authorships[16].author.display_name	Yuyuan Zeng
authorships[16].author_position	middle
authorships[16].raw_author_name	Zeng, Yuyuan
authorships[16].is_corresponding	False
authorships[17].author.id	https://openalex.org/A5006385457
authorships[17].author.orcid	https://orcid.org/0000-0002-0765-5101
authorships[17].author.display_name	Xiaoqin Ren
authorships[17].author_position	middle
authorships[17].raw_author_name	Ren, Xiaoqin
authorships[17].is_corresponding	False
authorships[18].author.id	https://openalex.org/A5056581550
authorships[18].author.orcid	https://orcid.org/0000-0002-9244-9416
authorships[18].author.display_name	Chao Yu
authorships[18].author_position	middle
authorships[18].raw_author_name	Yu, Chao
authorships[18].is_corresponding	False
authorships[19].author.id	https://openalex.org/A5115597673
authorships[19].author.orcid
authorships[19].author.display_name	Lulu Wu
authorships[19].author_position	middle
authorships[19].raw_author_name	Wu, Lulu
authorships[19].is_corresponding	False
authorships[20].author.id	https://openalex.org/A5052943433
authorships[20].author.orcid	https://orcid.org/0000-0003-4680-9916
authorships[20].author.display_name	Yue Mao
authorships[20].author_position	middle
authorships[20].raw_author_name	Mao, Yue
authorships[20].is_corresponding	False
authorships[21].author.id	https://openalex.org/A5075655523
authorships[21].author.orcid	https://orcid.org/0000-0002-9864-7835
authorships[21].author.display_name	Jun Xia
authorships[21].author_position	middle
authorships[21].raw_author_name	Xia, Jun
authorships[21].is_corresponding	False
authorships[22].author.id	https://openalex.org/A5115602250
authorships[22].author.orcid	https://orcid.org/0000-0002-7795-6081
authorships[22].author.display_name	Tao Yang
authorships[22].author_position	middle
authorships[22].raw_author_name	Yang, Tao
authorships[22].is_corresponding	False
authorships[23].author.id	https://openalex.org/A5110447612
authorships[23].author.orcid
authorships[23].author.display_name	Suncong Zheng
authorships[23].author_position	middle
authorships[23].raw_author_name	Zheng, Suncong
authorships[23].is_corresponding	False
authorships[24].author.id	https://openalex.org/A5038287652
authorships[24].author.orcid	https://orcid.org/0000-0002-5016-0251
authorships[24].author.display_name	Kai Wu
authorships[24].author_position	middle
authorships[24].raw_author_name	Wu, Kan
authorships[24].is_corresponding	False
authorships[25].author.id	https://openalex.org/A5102733016
authorships[25].author.orcid	https://orcid.org/0009-0000-7339-7246
authorships[25].author.display_name	Dian Jiao
authorships[25].author_position	middle
authorships[25].raw_author_name	Jiao, Dian
authorships[25].is_corresponding	False
authorships[26].author.id	https://openalex.org/A5025328651
authorships[26].author.orcid
authorships[26].author.display_name	Jinbao Xue
authorships[26].author_position	middle
authorships[26].raw_author_name	Xue, Jinbao
authorships[26].is_corresponding	False
authorships[27].author.id	https://openalex.org/A5102022377
authorships[27].author.orcid	https://orcid.org/0000-0002-2434-8226
authorships[27].author.display_name	Xipeng Zhang
authorships[27].author_position	middle
authorships[27].raw_author_name	Zhang, Xipeng
authorships[27].is_corresponding	False
authorships[28].author.id	https://openalex.org/A5065900428
authorships[28].author.orcid	https://orcid.org/0000-0002-6605-8550
authorships[28].author.display_name	Decheng Wu
authorships[28].author_position	middle
authorships[28].raw_author_name	Wu, Decheng
authorships[28].is_corresponding	False
authorships[29].author.id	https://openalex.org/A5115602464
authorships[29].author.orcid	https://orcid.org/0000-0003-2933-0356
authorships[29].author.display_name	Kai Liu
authorships[29].author_position	middle
authorships[29].raw_author_name	Liu, Kai
authorships[29].is_corresponding	False
authorships[30].author.id	https://openalex.org/A5021449674
authorships[30].author.orcid	https://orcid.org/0000-0003-0849-0297
authorships[30].author.display_name	Dan Wu
authorships[30].author_position	middle
authorships[30].raw_author_name	Wu, Dengpeng
authorships[30].is_corresponding	False
authorships[31].author.id	https://openalex.org/A5101552980
authorships[31].author.orcid	https://orcid.org/0000-0003-2910-9870
authorships[31].author.display_name	Guanghui Xu
authorships[31].author_position	middle
authorships[31].raw_author_name	Xu, Guanghui
authorships[31].is_corresponding	False
authorships[32].author.id	https://openalex.org/A5107772022
authorships[32].author.orcid	https://orcid.org/0000-0001-8500-2635
authorships[32].author.display_name	Shaohua Chen
authorships[32].author_position	middle
authorships[32].raw_author_name	Chen, Shaohua
authorships[32].is_corresponding	False
authorships[33].author.id	https://openalex.org/A5112225939
authorships[33].author.orcid	https://orcid.org/0009-0005-6148-9721
authorships[33].author.display_name	Shuang Chen
authorships[33].author_position	middle
authorships[33].raw_author_name	Chen, Shuang
authorships[33].is_corresponding	False
authorships[34].author.id	https://openalex.org/A5102645221
authorships[34].author.orcid	https://orcid.org/0009-0006-6267-466X
authorships[34].author.display_name	Xiao Feng
authorships[34].author_position	middle
authorships[34].raw_author_name	Feng, Xiao
authorships[34].is_corresponding	False
authorships[35].author.id	https://openalex.org/A5071710119
authorships[35].author.orcid	https://orcid.org/0009-0002-6845-0596
authorships[35].author.display_name	Yigeng Hong
authorships[35].author_position	middle
authorships[35].raw_author_name	Hong, Yigeng
authorships[35].is_corresponding	False
authorships[36].author.id	https://openalex.org/A5021562828
authorships[36].author.orcid	https://orcid.org/0000-0001-8612-4589
authorships[36].author.display_name	Junqiang Zheng
authorships[36].author_position	middle
authorships[36].raw_author_name	Zheng, Junqiang
authorships[36].is_corresponding	False
authorships[37].author.id	https://openalex.org/A5101869355
authorships[37].author.orcid	https://orcid.org/0000-0001-5278-3201
authorships[37].author.display_name	Chengcheng Xu
authorships[37].author_position	middle
authorships[37].raw_author_name	Xu, Chengcheng
authorships[37].is_corresponding	False
authorships[38].author.id	https://openalex.org/A5101530714
authorships[38].author.orcid	https://orcid.org/0000-0003-1682-6096
authorships[38].author.display_name	Zongwei Li
authorships[38].author_position	middle
authorships[38].raw_author_name	Li, Zongwei
authorships[38].is_corresponding	False
authorships[39].author.id	https://openalex.org/A5104214336
authorships[39].author.orcid
authorships[39].author.display_name	Xiang Kuang
authorships[39].author_position	middle
authorships[39].raw_author_name	Kuang, Xiong
authorships[39].is_corresponding	False
authorships[40].author.id	https://openalex.org/A5103935829
authorships[40].author.orcid	https://orcid.org/0009-0008-2586-9366
authorships[40].author.display_name	Jiaochan Hu
authorships[40].author_position	middle
authorships[40].raw_author_name	Hu, Jianglu
authorships[40].is_corresponding	False
authorships[41].author.id	https://openalex.org/A5111072505
authorships[41].author.orcid
authorships[41].author.display_name	Yiqi Chen
authorships[41].author_position	middle
authorships[41].raw_author_name	Chen, Yiqi
authorships[41].is_corresponding	False
authorships[42].author.id	https://openalex.org/A5113169015
authorships[42].author.orcid
authorships[42].author.display_name	Yuchi Deng
authorships[42].author_position	middle
authorships[42].raw_author_name	Deng, Yuchi
authorships[42].is_corresponding	False
authorships[43].author.id	https://openalex.org/A5006198528
authorships[43].author.orcid	https://orcid.org/0000-0002-2802-5766
authorships[43].author.display_name	Guiyang Li
authorships[43].author_position	middle
authorships[43].raw_author_name	Li, Guiyang
authorships[43].is_corresponding	False
authorships[44].author.id	https://openalex.org/A5100346418
authorships[44].author.orcid	https://orcid.org/0000-0002-8412-6414
authorships[44].author.display_name	Ao Liu
authorships[44].author_position	middle
authorships[44].raw_author_name	Liu, Ao
authorships[44].is_corresponding	False
authorships[45].author.id	https://openalex.org/A5100408849
authorships[45].author.orcid	https://orcid.org/0000-0001-7498-9147
authorships[45].author.display_name	Chenchen Zhang
authorships[45].author_position	middle
authorships[45].raw_author_name	Zhang, Chenchen
authorships[45].is_corresponding	False
authorships[46].author.id	https://openalex.org/A5004694705
authorships[46].author.orcid
authorships[46].author.display_name	Shihui Hu
authorships[46].author_position	middle
authorships[46].raw_author_name	Hu, Shihui
authorships[46].is_corresponding	False
authorships[47].author.id	https://openalex.org/A5005929442
authorships[47].author.orcid	https://orcid.org/0000-0001-6607-4596
authorships[47].author.display_name	Zilong Zhao
authorships[47].author_position	middle
authorships[47].raw_author_name	Zhao, Zilong
authorships[47].is_corresponding	False
authorships[48].author.id	https://openalex.org/A5015473154
authorships[48].author.orcid	https://orcid.org/0000-0002-8568-9622
authorships[48].author.display_name	Zifan Wu
authorships[48].author_position	middle
authorships[48].raw_author_name	Wu, Zifan
authorships[48].is_corresponding	False
authorships[49].author.id	https://openalex.org/A5014913742
authorships[49].author.orcid	https://orcid.org/0000-0002-6654-0996
authorships[49].author.display_name	Yao Ding
authorships[49].author_position	middle
authorships[49].raw_author_name	Ding, Yao
authorships[49].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2411.02265
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2024-11-15T00:00:00
display_name	Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T13553
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.7493000030517578
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1705
primary_topic.subfield.display_name	Computer Networks and Communications
primary_topic.display_name	Age of Information Optimization
related_works	https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052
cited_by_count	5
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	5
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2411.02265
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2411.02265
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2411.02265
primary_location.id	pmh:oai:arXiv.org:2411.02265
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2411.02265
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2411.02265
publication_date	2024-11-04
publication_year	2024
referenced_works_count	0
abstract_inverted_index.a	18, 38, 94, 99
abstract_inverted_index.52	25
abstract_inverted_index.In	0
abstract_inverted_index.We	36
abstract_inverted_index.an	105
abstract_inverted_index.in	91
abstract_inverted_index.is	7, 87
abstract_inverted_index.it	63
abstract_inverted_index.of	14, 20, 30, 41, 80, 121, 123, 141
abstract_inverted_index.to	33, 72, 145
abstract_inverted_index.up	32
abstract_inverted_index.we	3, 111
abstract_inverted_index.389	21
abstract_inverted_index.Key	78
abstract_inverted_index.The	137
abstract_inverted_index.and	24, 51, 59, 66, 104, 117, 129, 135, 139, 149
abstract_inverted_index.are	143
abstract_inverted_index.for	131
abstract_inverted_index.the	9, 73, 114
abstract_inverted_index.256K	34
abstract_inverted_index.also	112
abstract_inverted_index.code	138
abstract_inverted_index.data	85
abstract_inverted_index.laws	116
abstract_inverted_index.rate	108, 119
abstract_inverted_index.than	90
abstract_inverted_index.that	86
abstract_inverted_index.this	1
abstract_inverted_index.when	70
abstract_inverted_index.with	17
abstract_inverted_index.cache	101
abstract_inverted_index.mixed	95
abstract_inverted_index.model	133
abstract_inverted_index.total	19
abstract_inverted_index.where	62
abstract_inverted_index.which	6
abstract_inverted_index.Codes:	151
abstract_inverted_index.across	45
abstract_inverted_index.expert	96
abstract_inverted_index.future	132, 147
abstract_inverted_index.larger	75, 89
abstract_inverted_index.model,	16
abstract_inverted_index.model.	77
abstract_inverted_index.orders	88
abstract_inverted_index.paper,	2
abstract_inverted_index.tasks,	61
abstract_inverted_index.Models:	153
abstract_inverted_index.billion	22, 26
abstract_inverted_index.capable	29
abstract_inverted_index.coding,	57
abstract_inverted_index.conduct	37
abstract_inverted_index.experts	15, 124
abstract_inverted_index.include	82
abstract_inverted_index.largest	10
abstract_inverted_index.logical	53
abstract_inverted_index.mixture	13, 122
abstract_inverted_index.models,	125
abstract_inverted_index.routing	97
abstract_inverted_index.scaling	115
abstract_inverted_index.tokens.	35
abstract_inverted_index.various	46
abstract_inverted_index.compared	71
abstract_inverted_index.exhibits	67
abstract_inverted_index.handling	31
abstract_inverted_index.insights	128
abstract_inverted_index.language	49
abstract_inverted_index.learning	107, 118
abstract_inverted_index.practice	79
abstract_inverted_index.previous	92
abstract_inverted_index.released	144
abstract_inverted_index.schedule	120
abstract_inverted_index.superior	43
abstract_inverted_index.thorough	39
abstract_inverted_index.valuable	127
abstract_inverted_index.currently	8
abstract_inverted_index.guidances	130
abstract_inverted_index.including	48
abstract_inverted_index.introduce	4
abstract_inverted_index.key-value	100
abstract_inverted_index.providing	126
abstract_inverted_index.strategy,	98
abstract_inverted_index.strategy.	109
abstract_inverted_index.synthetic	84
abstract_inverted_index.activation	27
abstract_inverted_index.aggregated	60
abstract_inverted_index.benchmarks	47
abstract_inverted_index.comparable	68
abstract_inverted_index.evaluation	40
abstract_inverted_index.facilitate	146
abstract_inverted_index.parameters	23
abstract_inverted_index.reasoning,	54
abstract_inverted_index.technique,	103
abstract_inverted_index.checkpoints	140
abstract_inverted_index.compression	102
abstract_inverted_index.development	134
abstract_inverted_index.generation,	52
abstract_inverted_index.innovations	148
abstract_inverted_index.investigate	113
abstract_inverted_index.large-scale	83
abstract_inverted_index.literature,	93
abstract_inverted_index.open-source	11
abstract_inverted_index.outperforms	64
abstract_inverted_index.parameters,	28
abstract_inverted_index.performance	44, 69
abstract_inverted_index.LLama3.1-70B	65
abstract_inverted_index.mathematical	55
abstract_inverted_index.Additionally,	110
abstract_inverted_index.Hunyuan-Large	81, 142
abstract_inverted_index.LLama3.1-405B	76
abstract_inverted_index.applications.	150
abstract_inverted_index.long-context,	58
abstract_inverted_index.optimization.	136
abstract_inverted_index.significantly	74
abstract_inverted_index.understanding	50
abstract_inverted_index.Hunyuan-Large,	5
abstract_inverted_index.Hunyuan-Large's	42
abstract_inverted_index.expert-specific	106
abstract_inverted_index.problem-solving,	56
abstract_inverted_index.Transformer-based	12
abstract_inverted_index.https://github.com/Tencent/Hunyuan-Large	152
abstract_inverted_index.https://huggingface.co/tencent/Tencent-Hunyuan-Large	154
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	108
citation_normalized_percentile