KDExplainer: A Task-oriented Attention Model for Explaining Knowledge Distillation Article Swipe
Mengqi Xue
,
Jie Song
,
Xinchao Wang
,
Ying Chen
,
Xingen Wang
,
Mingli Song
·
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2105.04181
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2105.04181
Knowledge distillation (KD) has recently emerged as an efficacious scheme for learning compact deep neural networks (DNNs). Despite the promising results achieved, the rationale that interprets the behavior of KD has yet remained largely understudied. In this paper, we introduce a novel task-oriented attention model, termed as KDExplainer, to shed light on the working mechanism underlying the vanilla KD. At the heart of KDExplainer is a Hierarchical Mixture of Experts (HME), in which a multi-class classification is reformulated as a multi-task binary one. Through distilling knowledge from a free-form pre-trained DNN to KDExplainer, we observe that KD implicitly modulates the knowledge conflicts between different subtasks, and in reality has much more to offer than label smoothing. Based on such findings, we further introduce a portable tool, dubbed as virtual attention module (VAM), that can be seamlessly integrated with various DNNs to enhance their performance under KD. Experimental results demonstrate that with a negligible additional cost, student models equipped with VAM consistently outperform their non-VAM counterparts across different benchmarks. Furthermore, when combined with other KD methods, VAM remains competent in promoting results, even though it is only motivated by vanilla KD. The code is available at https://github.com/zju-vipa/KDExplainer.
Related Topics
Concepts
Computer science
Task (project management)
Smoothing
Code (set theory)
Scheme (mathematics)
Artificial intelligence
Class (philosophy)
Deep neural networks
Binary number
Artificial neural network
Distillation
Machine learning
Deep learning
Programming language
Chemistry
Arithmetic
Engineering
Mathematics
Mathematical analysis
Organic chemistry
Systems engineering
Computer vision
Set (abstract data type)
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2105.04181
- https://arxiv.org/pdf/2105.04181
- OA Status
- green
- References
- 14
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W3162413315
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W3162413315Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2105.04181Digital Object Identifier
- Title
-
KDExplainer: A Task-oriented Attention Model for Explaining Knowledge DistillationWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2021Year of publication
- Publication date
-
2021-05-10Full publication date if available
- Authors
-
Mengqi Xue, Jie Song, Xinchao Wang, Ying Chen, Xingen Wang, Mingli SongList of authors in order
- Landing page
-
https://arxiv.org/abs/2105.04181Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2105.04181Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2105.04181Direct OA link when available
- Concepts
-
Computer science, Task (project management), Smoothing, Code (set theory), Scheme (mathematics), Artificial intelligence, Class (philosophy), Deep neural networks, Binary number, Artificial neural network, Distillation, Machine learning, Deep learning, Programming language, Chemistry, Arithmetic, Engineering, Mathematics, Mathematical analysis, Organic chemistry, Systems engineering, Computer vision, Set (abstract data type)Top concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- References (count)
-
14Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W3162413315 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2105.04181 |
| ids.doi | https://doi.org/10.48550/arxiv.2105.04181 |
| ids.mag | 3162413315 |
| ids.openalex | https://openalex.org/W3162413315 |
| fwci | |
| type | preprint |
| title | KDExplainer: A Task-oriented Attention Model for Explaining Knowledge Distillation |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11689 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9991000294685364 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Adversarial Robustness in Machine Learning |
| topics[1].id | https://openalex.org/T12026 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9987999796867371 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Explainable Artificial Intelligence (XAI) |
| topics[2].id | https://openalex.org/T11714 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9969000220298767 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1707 |
| topics[2].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[2].display_name | Multimodal Machine Learning Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.7464028000831604 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C2780451532 |
| concepts[1].level | 2 |
| concepts[1].score | 0.66949462890625 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q759676 |
| concepts[1].display_name | Task (project management) |
| concepts[2].id | https://openalex.org/C3770464 |
| concepts[2].level | 2 |
| concepts[2].score | 0.5987833142280579 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q775963 |
| concepts[2].display_name | Smoothing |
| concepts[3].id | https://openalex.org/C2776760102 |
| concepts[3].level | 3 |
| concepts[3].score | 0.5979092121124268 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q5139990 |
| concepts[3].display_name | Code (set theory) |
| concepts[4].id | https://openalex.org/C77618280 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5912370085716248 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q1155772 |
| concepts[4].display_name | Scheme (mathematics) |
| concepts[5].id | https://openalex.org/C154945302 |
| concepts[5].level | 1 |
| concepts[5].score | 0.5795614719390869 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[5].display_name | Artificial intelligence |
| concepts[6].id | https://openalex.org/C2777212361 |
| concepts[6].level | 2 |
| concepts[6].score | 0.572369396686554 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q5127848 |
| concepts[6].display_name | Class (philosophy) |
| concepts[7].id | https://openalex.org/C2984842247 |
| concepts[7].level | 3 |
| concepts[7].score | 0.5641700029373169 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q197536 |
| concepts[7].display_name | Deep neural networks |
| concepts[8].id | https://openalex.org/C48372109 |
| concepts[8].level | 2 |
| concepts[8].score | 0.5195130109786987 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q3913 |
| concepts[8].display_name | Binary number |
| concepts[9].id | https://openalex.org/C50644808 |
| concepts[9].level | 2 |
| concepts[9].score | 0.5054759979248047 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q192776 |
| concepts[9].display_name | Artificial neural network |
| concepts[10].id | https://openalex.org/C204030448 |
| concepts[10].level | 2 |
| concepts[10].score | 0.4874131977558136 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q101017 |
| concepts[10].display_name | Distillation |
| concepts[11].id | https://openalex.org/C119857082 |
| concepts[11].level | 1 |
| concepts[11].score | 0.4648401141166687 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[11].display_name | Machine learning |
| concepts[12].id | https://openalex.org/C108583219 |
| concepts[12].level | 2 |
| concepts[12].score | 0.4226198196411133 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q197536 |
| concepts[12].display_name | Deep learning |
| concepts[13].id | https://openalex.org/C199360897 |
| concepts[13].level | 1 |
| concepts[13].score | 0.08164125680923462 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[13].display_name | Programming language |
| concepts[14].id | https://openalex.org/C185592680 |
| concepts[14].level | 0 |
| concepts[14].score | 0.07254490256309509 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q2329 |
| concepts[14].display_name | Chemistry |
| concepts[15].id | https://openalex.org/C94375191 |
| concepts[15].level | 1 |
| concepts[15].score | 0.0687556266784668 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q11205 |
| concepts[15].display_name | Arithmetic |
| concepts[16].id | https://openalex.org/C127413603 |
| concepts[16].level | 0 |
| concepts[16].score | 0.06675606966018677 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[16].display_name | Engineering |
| concepts[17].id | https://openalex.org/C33923547 |
| concepts[17].level | 0 |
| concepts[17].score | 0.0 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[17].display_name | Mathematics |
| concepts[18].id | https://openalex.org/C134306372 |
| concepts[18].level | 1 |
| concepts[18].score | 0.0 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q7754 |
| concepts[18].display_name | Mathematical analysis |
| concepts[19].id | https://openalex.org/C178790620 |
| concepts[19].level | 1 |
| concepts[19].score | 0.0 |
| concepts[19].wikidata | https://www.wikidata.org/wiki/Q11351 |
| concepts[19].display_name | Organic chemistry |
| concepts[20].id | https://openalex.org/C201995342 |
| concepts[20].level | 1 |
| concepts[20].score | 0.0 |
| concepts[20].wikidata | https://www.wikidata.org/wiki/Q682496 |
| concepts[20].display_name | Systems engineering |
| concepts[21].id | https://openalex.org/C31972630 |
| concepts[21].level | 1 |
| concepts[21].score | 0.0 |
| concepts[21].wikidata | https://www.wikidata.org/wiki/Q844240 |
| concepts[21].display_name | Computer vision |
| concepts[22].id | https://openalex.org/C177264268 |
| concepts[22].level | 2 |
| concepts[22].score | 0.0 |
| concepts[22].wikidata | https://www.wikidata.org/wiki/Q1514741 |
| concepts[22].display_name | Set (abstract data type) |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.7464028000831604 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/task |
| keywords[1].score | 0.66949462890625 |
| keywords[1].display_name | Task (project management) |
| keywords[2].id | https://openalex.org/keywords/smoothing |
| keywords[2].score | 0.5987833142280579 |
| keywords[2].display_name | Smoothing |
| keywords[3].id | https://openalex.org/keywords/code |
| keywords[3].score | 0.5979092121124268 |
| keywords[3].display_name | Code (set theory) |
| keywords[4].id | https://openalex.org/keywords/scheme |
| keywords[4].score | 0.5912370085716248 |
| keywords[4].display_name | Scheme (mathematics) |
| keywords[5].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[5].score | 0.5795614719390869 |
| keywords[5].display_name | Artificial intelligence |
| keywords[6].id | https://openalex.org/keywords/class |
| keywords[6].score | 0.572369396686554 |
| keywords[6].display_name | Class (philosophy) |
| keywords[7].id | https://openalex.org/keywords/deep-neural-networks |
| keywords[7].score | 0.5641700029373169 |
| keywords[7].display_name | Deep neural networks |
| keywords[8].id | https://openalex.org/keywords/binary-number |
| keywords[8].score | 0.5195130109786987 |
| keywords[8].display_name | Binary number |
| keywords[9].id | https://openalex.org/keywords/artificial-neural-network |
| keywords[9].score | 0.5054759979248047 |
| keywords[9].display_name | Artificial neural network |
| keywords[10].id | https://openalex.org/keywords/distillation |
| keywords[10].score | 0.4874131977558136 |
| keywords[10].display_name | Distillation |
| keywords[11].id | https://openalex.org/keywords/machine-learning |
| keywords[11].score | 0.4648401141166687 |
| keywords[11].display_name | Machine learning |
| keywords[12].id | https://openalex.org/keywords/deep-learning |
| keywords[12].score | 0.4226198196411133 |
| keywords[12].display_name | Deep learning |
| keywords[13].id | https://openalex.org/keywords/programming-language |
| keywords[13].score | 0.08164125680923462 |
| keywords[13].display_name | Programming language |
| keywords[14].id | https://openalex.org/keywords/chemistry |
| keywords[14].score | 0.07254490256309509 |
| keywords[14].display_name | Chemistry |
| keywords[15].id | https://openalex.org/keywords/arithmetic |
| keywords[15].score | 0.0687556266784668 |
| keywords[15].display_name | Arithmetic |
| keywords[16].id | https://openalex.org/keywords/engineering |
| keywords[16].score | 0.06675606966018677 |
| keywords[16].display_name | Engineering |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2105.04181 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2105.04181 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2105.04181 |
| locations[1].id | doi:10.48550/arxiv.2105.04181 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2105.04181 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5040860593 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-4936-4887 |
| authorships[0].author.display_name | Mengqi Xue |
| authorships[0].countries | CN |
| authorships[0].affiliations[0].institution_ids | https://openalex.org/I76130692 |
| authorships[0].affiliations[0].raw_affiliation_string | (Zhejiang University) |
| authorships[0].institutions[0].id | https://openalex.org/I76130692 |
| authorships[0].institutions[0].ror | https://ror.org/00a2xv884 |
| authorships[0].institutions[0].type | education |
| authorships[0].institutions[0].lineage | https://openalex.org/I76130692 |
| authorships[0].institutions[0].country_code | CN |
| authorships[0].institutions[0].display_name | Zhejiang University |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Mengqi Xue |
| authorships[0].is_corresponding | False |
| authorships[0].raw_affiliation_strings | (Zhejiang University) |
| authorships[1].author.id | https://openalex.org/A5047371218 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-3671-6521 |
| authorships[1].author.display_name | Jie Song |
| authorships[1].countries | CN |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I76130692 |
| authorships[1].affiliations[0].raw_affiliation_string | (Zhejiang University) |
| authorships[1].institutions[0].id | https://openalex.org/I76130692 |
| authorships[1].institutions[0].ror | https://ror.org/00a2xv884 |
| authorships[1].institutions[0].type | education |
| authorships[1].institutions[0].lineage | https://openalex.org/I76130692 |
| authorships[1].institutions[0].country_code | CN |
| authorships[1].institutions[0].display_name | Zhejiang University |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Jie Song |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | (Zhejiang University) |
| authorships[2].author.id | https://openalex.org/A5015574447 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-0057-1404 |
| authorships[2].author.display_name | Xinchao Wang |
| authorships[2].countries | SG |
| authorships[2].affiliations[0].institution_ids | https://openalex.org/I165932596 |
| authorships[2].affiliations[0].raw_affiliation_string | National University of Singapore, |
| authorships[2].institutions[0].id | https://openalex.org/I165932596 |
| authorships[2].institutions[0].ror | https://ror.org/01tgyzw49 |
| authorships[2].institutions[0].type | education |
| authorships[2].institutions[0].lineage | https://openalex.org/I165932596 |
| authorships[2].institutions[0].country_code | SG |
| authorships[2].institutions[0].display_name | National University of Singapore |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Xinchao Wang |
| authorships[2].is_corresponding | False |
| authorships[2].raw_affiliation_strings | National University of Singapore, |
| authorships[3].author.id | https://openalex.org/A5100383139 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Ying Chen |
| authorships[3].countries | CN |
| authorships[3].affiliations[0].institution_ids | https://openalex.org/I76130692 |
| authorships[3].affiliations[0].raw_affiliation_string | (Zhejiang University) |
| authorships[3].institutions[0].id | https://openalex.org/I76130692 |
| authorships[3].institutions[0].ror | https://ror.org/00a2xv884 |
| authorships[3].institutions[0].type | education |
| authorships[3].institutions[0].lineage | https://openalex.org/I76130692 |
| authorships[3].institutions[0].country_code | CN |
| authorships[3].institutions[0].display_name | Zhejiang University |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Ying Chen |
| authorships[3].is_corresponding | False |
| authorships[3].raw_affiliation_strings | (Zhejiang University) |
| authorships[4].author.id | https://openalex.org/A5042307193 |
| authorships[4].author.orcid | https://orcid.org/0000-0003-0507-5020 |
| authorships[4].author.display_name | Xingen Wang |
| authorships[4].countries | CN |
| authorships[4].affiliations[0].institution_ids | https://openalex.org/I76130692 |
| authorships[4].affiliations[0].raw_affiliation_string | (Zhejiang University) |
| authorships[4].institutions[0].id | https://openalex.org/I76130692 |
| authorships[4].institutions[0].ror | https://ror.org/00a2xv884 |
| authorships[4].institutions[0].type | education |
| authorships[4].institutions[0].lineage | https://openalex.org/I76130692 |
| authorships[4].institutions[0].country_code | CN |
| authorships[4].institutions[0].display_name | Zhejiang University |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Xingen Wang |
| authorships[4].is_corresponding | False |
| authorships[4].raw_affiliation_strings | (Zhejiang University) |
| authorships[5].author.id | https://openalex.org/A5026532752 |
| authorships[5].author.orcid | https://orcid.org/0000-0003-2621-6048 |
| authorships[5].author.display_name | Mingli Song |
| authorships[5].countries | CN |
| authorships[5].affiliations[0].institution_ids | https://openalex.org/I76130692 |
| authorships[5].affiliations[0].raw_affiliation_string | (Zhejiang University) |
| authorships[5].institutions[0].id | https://openalex.org/I76130692 |
| authorships[5].institutions[0].ror | https://ror.org/00a2xv884 |
| authorships[5].institutions[0].type | education |
| authorships[5].institutions[0].lineage | https://openalex.org/I76130692 |
| authorships[5].institutions[0].country_code | CN |
| authorships[5].institutions[0].display_name | Zhejiang University |
| authorships[5].author_position | last |
| authorships[5].raw_author_name | Mingli Song |
| authorships[5].is_corresponding | False |
| authorships[5].raw_affiliation_strings | (Zhejiang University) |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2105.04181 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2021-05-24T00:00:00 |
| display_name | KDExplainer: A Task-oriented Attention Model for Explaining Knowledge Distillation |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11689 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9991000294685364 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Adversarial Robustness in Machine Learning |
| related_works | https://openalex.org/W1978572805, https://openalex.org/W2383807498, https://openalex.org/W1997992934, https://openalex.org/W4377865163, https://openalex.org/W3193857078, https://openalex.org/W2888956734, https://openalex.org/W3000197790, https://openalex.org/W4315865067, https://openalex.org/W2979433843, https://openalex.org/W3208304128 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2105.04181 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2105.04181 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2105.04181 |
| primary_location.id | pmh:oai:arXiv.org:2105.04181 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2105.04181 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2105.04181 |
| publication_date | 2021-05-10 |
| publication_year | 2021 |
| referenced_works | https://openalex.org/W2951527505, https://openalex.org/W2995607862, https://openalex.org/W1686810756, https://openalex.org/W2949117887, https://openalex.org/W2561238782, https://openalex.org/W2964118293, https://openalex.org/W2963125010, https://openalex.org/W2963534679, https://openalex.org/W2526468814, https://openalex.org/W2970726176, https://openalex.org/W2964111476, https://openalex.org/W1514535095, https://openalex.org/W2401231614, https://openalex.org/W2963122961 |
| referenced_works_count | 14 |
| abstract_inverted_index.a | 40, 65, 73, 79, 87, 123, 151 |
| abstract_inverted_index.At | 59 |
| abstract_inverted_index.In | 35 |
| abstract_inverted_index.KD | 29, 96, 173 |
| abstract_inverted_index.an | 7 |
| abstract_inverted_index.as | 6, 46, 78, 127 |
| abstract_inverted_index.at | 194 |
| abstract_inverted_index.be | 134 |
| abstract_inverted_index.by | 187 |
| abstract_inverted_index.in | 71, 106, 178 |
| abstract_inverted_index.is | 64, 76, 184, 192 |
| abstract_inverted_index.it | 183 |
| abstract_inverted_index.of | 28, 62, 68 |
| abstract_inverted_index.on | 51, 117 |
| abstract_inverted_index.to | 48, 91, 111, 140 |
| abstract_inverted_index.we | 38, 93, 120 |
| abstract_inverted_index.DNN | 90 |
| abstract_inverted_index.KD. | 58, 145, 189 |
| abstract_inverted_index.The | 190 |
| abstract_inverted_index.VAM | 159, 175 |
| abstract_inverted_index.and | 105 |
| abstract_inverted_index.can | 133 |
| abstract_inverted_index.for | 10 |
| abstract_inverted_index.has | 3, 30, 108 |
| abstract_inverted_index.the | 18, 22, 26, 52, 56, 60, 99 |
| abstract_inverted_index.yet | 31 |
| abstract_inverted_index.(KD) | 2 |
| abstract_inverted_index.DNNs | 139 |
| abstract_inverted_index.code | 191 |
| abstract_inverted_index.deep | 13 |
| abstract_inverted_index.even | 181 |
| abstract_inverted_index.from | 86 |
| abstract_inverted_index.more | 110 |
| abstract_inverted_index.much | 109 |
| abstract_inverted_index.one. | 82 |
| abstract_inverted_index.only | 185 |
| abstract_inverted_index.shed | 49 |
| abstract_inverted_index.such | 118 |
| abstract_inverted_index.than | 113 |
| abstract_inverted_index.that | 24, 95, 132, 149 |
| abstract_inverted_index.this | 36 |
| abstract_inverted_index.when | 169 |
| abstract_inverted_index.with | 137, 150, 158, 171 |
| abstract_inverted_index.Based | 116 |
| abstract_inverted_index.cost, | 154 |
| abstract_inverted_index.heart | 61 |
| abstract_inverted_index.label | 114 |
| abstract_inverted_index.light | 50 |
| abstract_inverted_index.novel | 41 |
| abstract_inverted_index.offer | 112 |
| abstract_inverted_index.other | 172 |
| abstract_inverted_index.their | 142, 162 |
| abstract_inverted_index.tool, | 125 |
| abstract_inverted_index.under | 144 |
| abstract_inverted_index.which | 72 |
| abstract_inverted_index.(HME), | 70 |
| abstract_inverted_index.(VAM), | 131 |
| abstract_inverted_index.across | 165 |
| abstract_inverted_index.binary | 81 |
| abstract_inverted_index.dubbed | 126 |
| abstract_inverted_index.model, | 44 |
| abstract_inverted_index.models | 156 |
| abstract_inverted_index.module | 130 |
| abstract_inverted_index.neural | 14 |
| abstract_inverted_index.paper, | 37 |
| abstract_inverted_index.scheme | 9 |
| abstract_inverted_index.termed | 45 |
| abstract_inverted_index.though | 182 |
| abstract_inverted_index.(DNNs). | 16 |
| abstract_inverted_index.Despite | 17 |
| abstract_inverted_index.Experts | 69 |
| abstract_inverted_index.Mixture | 67 |
| abstract_inverted_index.Through | 83 |
| abstract_inverted_index.between | 102 |
| abstract_inverted_index.compact | 12 |
| abstract_inverted_index.emerged | 5 |
| abstract_inverted_index.enhance | 141 |
| abstract_inverted_index.further | 121 |
| abstract_inverted_index.largely | 33 |
| abstract_inverted_index.non-VAM | 163 |
| abstract_inverted_index.observe | 94 |
| abstract_inverted_index.reality | 107 |
| abstract_inverted_index.remains | 176 |
| abstract_inverted_index.results | 20, 147 |
| abstract_inverted_index.student | 155 |
| abstract_inverted_index.vanilla | 57, 188 |
| abstract_inverted_index.various | 138 |
| abstract_inverted_index.virtual | 128 |
| abstract_inverted_index.working | 53 |
| abstract_inverted_index.behavior | 27 |
| abstract_inverted_index.combined | 170 |
| abstract_inverted_index.equipped | 157 |
| abstract_inverted_index.learning | 11 |
| abstract_inverted_index.methods, | 174 |
| abstract_inverted_index.networks | 15 |
| abstract_inverted_index.portable | 124 |
| abstract_inverted_index.recently | 4 |
| abstract_inverted_index.remained | 32 |
| abstract_inverted_index.results, | 180 |
| abstract_inverted_index.Knowledge | 0 |
| abstract_inverted_index.achieved, | 21 |
| abstract_inverted_index.attention | 43, 129 |
| abstract_inverted_index.available | 193 |
| abstract_inverted_index.competent | 177 |
| abstract_inverted_index.conflicts | 101 |
| abstract_inverted_index.different | 103, 166 |
| abstract_inverted_index.findings, | 119 |
| abstract_inverted_index.free-form | 88 |
| abstract_inverted_index.introduce | 39, 122 |
| abstract_inverted_index.knowledge | 85, 100 |
| abstract_inverted_index.mechanism | 54 |
| abstract_inverted_index.modulates | 98 |
| abstract_inverted_index.motivated | 186 |
| abstract_inverted_index.promising | 19 |
| abstract_inverted_index.promoting | 179 |
| abstract_inverted_index.rationale | 23 |
| abstract_inverted_index.subtasks, | 104 |
| abstract_inverted_index.additional | 153 |
| abstract_inverted_index.distilling | 84 |
| abstract_inverted_index.implicitly | 97 |
| abstract_inverted_index.integrated | 136 |
| abstract_inverted_index.interprets | 25 |
| abstract_inverted_index.multi-task | 80 |
| abstract_inverted_index.negligible | 152 |
| abstract_inverted_index.outperform | 161 |
| abstract_inverted_index.seamlessly | 135 |
| abstract_inverted_index.smoothing. | 115 |
| abstract_inverted_index.underlying | 55 |
| abstract_inverted_index.KDExplainer | 63 |
| abstract_inverted_index.benchmarks. | 167 |
| abstract_inverted_index.demonstrate | 148 |
| abstract_inverted_index.efficacious | 8 |
| abstract_inverted_index.multi-class | 74 |
| abstract_inverted_index.performance | 143 |
| abstract_inverted_index.pre-trained | 89 |
| abstract_inverted_index.Experimental | 146 |
| abstract_inverted_index.Furthermore, | 168 |
| abstract_inverted_index.Hierarchical | 66 |
| abstract_inverted_index.KDExplainer, | 47, 92 |
| abstract_inverted_index.consistently | 160 |
| abstract_inverted_index.counterparts | 164 |
| abstract_inverted_index.distillation | 1 |
| abstract_inverted_index.reformulated | 77 |
| abstract_inverted_index.task-oriented | 42 |
| abstract_inverted_index.understudied. | 34 |
| abstract_inverted_index.classification | 75 |
| abstract_inverted_index.https://github.com/zju-vipa/KDExplainer. | 195 |
| cited_by_percentile_year | |
| countries_distinct_count | 2 |
| institutions_distinct_count | 6 |
| citation_normalized_percentile |