On the Analysis and Distillation of Emergent Outlier Properties in Pre-trained Language Models Article Swipe

PDF

Tianyang Zhao , Kunwar Yashraj Singh , Srikar Appalaraju , Peng Tang , Ying Wu , Li Erran Li ·

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.18653/v1/2025.naacl-long.430

Related Topics

Distillation

Computer Science

Artificial Intelligence

Machine Learning

Chemistry

Chromatography

Concepts

Distillation Outlier Computer science Natural language processing Artificial intelligence Machine learning Chemistry Chromatography

Metadata

Type: article
Language: en
Landing Page: https://doi.org/10.18653/v1/2025.naacl-long.430
PDF: https://aclanthology.org/2025.naacl-long.430.pdf
OA Status: gold
Related Works: 10
OpenAlex ID: https://openalex.org/W4411119419

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4411119419

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.18653/v1/2025.naacl-long.430

Digital Object Identifier
Title: On the Analysis and Distillation of Emergent Outlier Properties in Pre-trained Language Models

Work title
Type: article

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-01-01

Full publication date if available
Authors: Tianyang Zhao, Kunwar Yashraj Singh, Srikar Appalaraju, Peng Tang, Ying Wu, Li Erran Li

List of authors in order
Landing page: https://doi.org/10.18653/v1/2025.naacl-long.430

Publisher landing page
PDF URL: https://aclanthology.org/2025.naacl-long.430.pdf

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: gold

Open access status per OpenAlex
OA URL: https://aclanthology.org/2025.naacl-long.430.pdf

Direct OA link when available
Concepts: Distillation, Outlier, Computer science, Natural language processing, Artificial intelligence, Machine learning, Chemistry, Chromatography

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4411119419
doi	https://doi.org/10.18653/v1/2025.naacl-long.430
ids.doi	https://doi.org/10.18653/v1/2025.naacl-long.430
ids.openalex	https://openalex.org/W4411119419
fwci	0.0
type	article
title	On the Analysis and Distillation of Emergent Outlier Properties in Pre-trained Language Models
biblio.issue
biblio.volume
biblio.last_page	8507
biblio.first_page	8475
topics[0].id	https://openalex.org/T10181
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9825999736785889
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Natural Language Processing Techniques
topics[1].id	https://openalex.org/T10028
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9653000235557556
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1702
topics[1].subfield.display_name	Artificial Intelligence
topics[1].display_name	Topic Modeling
topics[2].id	https://openalex.org/T10201
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.9132000207901001
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1702
topics[2].subfield.display_name	Artificial Intelligence
topics[2].display_name	Speech Recognition and Synthesis
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C204030448
concepts[0].level	2
concepts[0].score	0.7383458614349365
concepts[0].wikidata	https://www.wikidata.org/wiki/Q101017
concepts[0].display_name	Distillation
concepts[1].id	https://openalex.org/C79337645
concepts[1].level	2
concepts[1].score	0.7108233571052551
concepts[1].wikidata	https://www.wikidata.org/wiki/Q779824
concepts[1].display_name	Outlier
concepts[2].id	https://openalex.org/C41008148
concepts[2].level	0
concepts[2].score	0.6889725923538208
concepts[2].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[2].display_name	Computer science
concepts[3].id	https://openalex.org/C204321447
concepts[3].level	1
concepts[3].score	0.518360435962677
concepts[3].wikidata	https://www.wikidata.org/wiki/Q30642
concepts[3].display_name	Natural language processing
concepts[4].id	https://openalex.org/C154945302
concepts[4].level	1
concepts[4].score	0.5080513954162598
concepts[4].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[4].display_name	Artificial intelligence
concepts[5].id	https://openalex.org/C119857082
concepts[5].level	1
concepts[5].score	0.3742826581001282
concepts[5].wikidata	https://www.wikidata.org/wiki/Q2539
concepts[5].display_name	Machine learning
concepts[6].id	https://openalex.org/C185592680
concepts[6].level	0
concepts[6].score	0.07985755801200867
concepts[6].wikidata	https://www.wikidata.org/wiki/Q2329
concepts[6].display_name	Chemistry
concepts[7].id	https://openalex.org/C43617362
concepts[7].level	1
concepts[7].score	0.05292558670043945
concepts[7].wikidata	https://www.wikidata.org/wiki/Q170050
concepts[7].display_name	Chromatography
keywords[0].id	https://openalex.org/keywords/distillation
keywords[0].score	0.7383458614349365
keywords[0].display_name	Distillation
keywords[1].id	https://openalex.org/keywords/outlier
keywords[1].score	0.7108233571052551
keywords[1].display_name	Outlier
keywords[2].id	https://openalex.org/keywords/computer-science
keywords[2].score	0.6889725923538208
keywords[2].display_name	Computer science
keywords[3].id	https://openalex.org/keywords/natural-language-processing
keywords[3].score	0.518360435962677
keywords[3].display_name	Natural language processing
keywords[4].id	https://openalex.org/keywords/artificial-intelligence
keywords[4].score	0.5080513954162598
keywords[4].display_name	Artificial intelligence
keywords[5].id	https://openalex.org/keywords/machine-learning
keywords[5].score	0.3742826581001282
keywords[5].display_name	Machine learning
keywords[6].id	https://openalex.org/keywords/chemistry
keywords[6].score	0.07985755801200867
keywords[6].display_name	Chemistry
keywords[7].id	https://openalex.org/keywords/chromatography
keywords[7].score	0.05292558670043945
keywords[7].display_name	Chromatography
language	en
locations[0].id	doi:10.18653/v1/2025.naacl-long.430
locations[0].is_oa	True
locations[0].source
locations[0].license	cc-by
locations[0].pdf_url	https://aclanthology.org/2025.naacl-long.430.pdf
locations[0].version	publishedVersion
locations[0].raw_type	proceedings-article
locations[0].license_id	https://openalex.org/licenses/cc-by
locations[0].is_accepted	True
locations[0].is_published	True
locations[0].raw_source_name	Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
locations[0].landing_page_url	https://doi.org/10.18653/v1/2025.naacl-long.430
indexed_in	crossref
authorships[0].author.id	https://openalex.org/A5031912756
authorships[0].author.orcid	https://orcid.org/0000-0003-0184-6553
authorships[0].author.display_name	Tianyang Zhao
authorships[0].author_position	first
authorships[0].raw_author_name	Tianyang Zhao
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5088891617
authorships[1].author.orcid
authorships[1].author.display_name	Kunwar Yashraj Singh
authorships[1].author_position	middle
authorships[1].raw_author_name	Kunwar Yashraj Singh
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5003062075
authorships[2].author.orcid
authorships[2].author.display_name	Srikar Appalaraju
authorships[2].author_position	middle
authorships[2].raw_author_name	Srikar Appalaraju
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5021603326
authorships[3].author.orcid	https://orcid.org/0000-0001-6607-1280
authorships[3].author.display_name	Peng Tang
authorships[3].author_position	middle
authorships[3].raw_author_name	Peng Tang
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5101780958
authorships[4].author.orcid	https://orcid.org/0009-0001-6768-5118
authorships[4].author.display_name	Ying Wu
authorships[4].author_position	middle
authorships[4].raw_author_name	Ying Nian Wu
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5053710007
authorships[5].author.orcid
authorships[5].author.display_name	Li Erran Li
authorships[5].author_position	last
authorships[5].raw_author_name	Li Erran Li
authorships[5].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://aclanthology.org/2025.naacl-long.430.pdf
open_access.oa_status	gold
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	On the Analysis and Distillation of Emergent Outlier Properties in Pre-trained Language Models
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T10181
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9825999736785889
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Natural Language Processing Techniques
related_works	https://openalex.org/W2961085424, https://openalex.org/W4306674287, https://openalex.org/W4387369504, https://openalex.org/W4394896187, https://openalex.org/W3170094116, https://openalex.org/W4386462264, https://openalex.org/W3107602296, https://openalex.org/W4364306694, https://openalex.org/W4312192474, https://openalex.org/W4283697347
cited_by_count	0
locations_count	1
best_oa_location.id	doi:10.18653/v1/2025.naacl-long.430
best_oa_location.is_oa	True
best_oa_location.source
best_oa_location.license	cc-by
best_oa_location.pdf_url	https://aclanthology.org/2025.naacl-long.430.pdf
best_oa_location.version	publishedVersion
best_oa_location.raw_type	proceedings-article
best_oa_location.license_id	https://openalex.org/licenses/cc-by
best_oa_location.is_accepted	True
best_oa_location.is_published	True
best_oa_location.raw_source_name	Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
best_oa_location.landing_page_url	https://doi.org/10.18653/v1/2025.naacl-long.430
primary_location.id	doi:10.18653/v1/2025.naacl-long.430
primary_location.is_oa	True
primary_location.source
primary_location.license	cc-by
primary_location.pdf_url	https://aclanthology.org/2025.naacl-long.430.pdf
primary_location.version	publishedVersion
primary_location.raw_type	proceedings-article
primary_location.license_id	https://openalex.org/licenses/cc-by
primary_location.is_accepted	True
primary_location.is_published	True
primary_location.raw_source_name	Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
primary_location.landing_page_url	https://doi.org/10.18653/v1/2025.naacl-long.430
publication_date	2025-01-01
publication_year	2025
referenced_works_count	0
abstract_inverted_index
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	6
sustainable_development_goals[0].id	https://metadata.un.org/sdg/4
sustainable_development_goals[0].score	0.6800000071525574
sustainable_development_goals[0].display_name	Quality Education
citation_normalized_percentile.value	0.09159451
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	True