Isolating authorship from content with semantic embeddings and contrastive learning Article Swipe
Javier Huertas‐Tato
,
Adrián Girón-Jiménez
,
Alejandro Martín
,
David Camacho
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2411.18472
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2411.18472
Authorship has entangled style and content inside. Authors frequently write about the same topics in the same style, so when different authors write about the exact same topic the easiest way out to distinguish them is by understanding the nuances of their style. Modern neural models for authorship can pick up these features using contrastive learning, however, some amount of content leakage is always present. Our aim is to reduce the inevitable impact and correlation between content and authorship. We present a technique to use contrastive learning (InfoNCE) with additional hard negatives synthetically created using a semantic similarity model. This disentanglement technique aims to distance the content embedding space from the style embedding space, leading to embeddings more informed by style. We demonstrate the performance with ablations on two different datasets and compare them on out-of-domain challenges. Improvements are clearly shown on challenging evaluations on prolific authors with up to a 10% increase in accuracy when the settings are particularly hard. Trials on challenges also demonstrate the preservation of zero-shot capabilities of this method as fine tuning.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2411.18472
- https://arxiv.org/pdf/2411.18472
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4404992442
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4404992442Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2411.18472Digital Object Identifier
- Title
-
Isolating authorship from content with semantic embeddings and contrastive learningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-11-27Full publication date if available
- Authors
-
Javier Huertas‐Tato, Adrián Girón-Jiménez, Alejandro Martín, David CamachoList of authors in order
- Landing page
-
https://arxiv.org/abs/2411.18472Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2411.18472Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2411.18472Direct OA link when available
- Concepts
-
Natural language processing, Psychology, Content (measure theory), Computer science, Artificial intelligence, Linguistics, Information retrieval, Mathematics, Philosophy, Mathematical analysisTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4404992442 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2411.18472 |
| ids.doi | https://doi.org/10.48550/arxiv.2411.18472 |
| ids.openalex | https://openalex.org/W4404992442 |
| fwci | |
| type | preprint |
| title | Isolating authorship from content with semantic embeddings and contrastive learning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12380 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9970999956130981 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Authorship Attribution and Profiling |
| topics[1].id | https://openalex.org/T10028 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9556999802589417 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Topic Modeling |
| topics[2].id | https://openalex.org/T11147 |
| topics[2].field.id | https://openalex.org/fields/33 |
| topics[2].field.display_name | Social Sciences |
| topics[2].score | 0.9556000232696533 |
| topics[2].domain.id | https://openalex.org/domains/2 |
| topics[2].domain.display_name | Social Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/3312 |
| topics[2].subfield.display_name | Sociology and Political Science |
| topics[2].display_name | Misinformation and Its Impacts |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C204321447 |
| concepts[0].level | 1 |
| concepts[0].score | 0.561836838722229 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[0].display_name | Natural language processing |
| concepts[1].id | https://openalex.org/C15744967 |
| concepts[1].level | 0 |
| concepts[1].score | 0.5390597581863403 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[1].display_name | Psychology |
| concepts[2].id | https://openalex.org/C2778152352 |
| concepts[2].level | 2 |
| concepts[2].score | 0.44233253598213196 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q5165061 |
| concepts[2].display_name | Content (measure theory) |
| concepts[3].id | https://openalex.org/C41008148 |
| concepts[3].level | 0 |
| concepts[3].score | 0.4114103615283966 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[3].display_name | Computer science |
| concepts[4].id | https://openalex.org/C154945302 |
| concepts[4].level | 1 |
| concepts[4].score | 0.41075029969215393 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[4].display_name | Artificial intelligence |
| concepts[5].id | https://openalex.org/C41895202 |
| concepts[5].level | 1 |
| concepts[5].score | 0.3895542621612549 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q8162 |
| concepts[5].display_name | Linguistics |
| concepts[6].id | https://openalex.org/C23123220 |
| concepts[6].level | 1 |
| concepts[6].score | 0.32968252897262573 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q816826 |
| concepts[6].display_name | Information retrieval |
| concepts[7].id | https://openalex.org/C33923547 |
| concepts[7].level | 0 |
| concepts[7].score | 0.15661904215812683 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[7].display_name | Mathematics |
| concepts[8].id | https://openalex.org/C138885662 |
| concepts[8].level | 0 |
| concepts[8].score | 0.11523661017417908 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q5891 |
| concepts[8].display_name | Philosophy |
| concepts[9].id | https://openalex.org/C134306372 |
| concepts[9].level | 1 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q7754 |
| concepts[9].display_name | Mathematical analysis |
| keywords[0].id | https://openalex.org/keywords/natural-language-processing |
| keywords[0].score | 0.561836838722229 |
| keywords[0].display_name | Natural language processing |
| keywords[1].id | https://openalex.org/keywords/psychology |
| keywords[1].score | 0.5390597581863403 |
| keywords[1].display_name | Psychology |
| keywords[2].id | https://openalex.org/keywords/content |
| keywords[2].score | 0.44233253598213196 |
| keywords[2].display_name | Content (measure theory) |
| keywords[3].id | https://openalex.org/keywords/computer-science |
| keywords[3].score | 0.4114103615283966 |
| keywords[3].display_name | Computer science |
| keywords[4].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[4].score | 0.41075029969215393 |
| keywords[4].display_name | Artificial intelligence |
| keywords[5].id | https://openalex.org/keywords/linguistics |
| keywords[5].score | 0.3895542621612549 |
| keywords[5].display_name | Linguistics |
| keywords[6].id | https://openalex.org/keywords/information-retrieval |
| keywords[6].score | 0.32968252897262573 |
| keywords[6].display_name | Information retrieval |
| keywords[7].id | https://openalex.org/keywords/mathematics |
| keywords[7].score | 0.15661904215812683 |
| keywords[7].display_name | Mathematics |
| keywords[8].id | https://openalex.org/keywords/philosophy |
| keywords[8].score | 0.11523661017417908 |
| keywords[8].display_name | Philosophy |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2411.18472 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2411.18472 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2411.18472 |
| locations[1].id | doi:10.48550/arxiv.2411.18472 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2411.18472 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5040465870 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-4127-5505 |
| authorships[0].author.display_name | Javier Huertas‐Tato |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Huertas-Tato, Javier |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5114989272 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Adrián Girón-Jiménez |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Girón-Jiménez, Adrián |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5101979842 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-0800-7632 |
| authorships[2].author.display_name | Alejandro Martín |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Martín, Alejandro |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5025314362 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-5051-3475 |
| authorships[3].author.display_name | David Camacho |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Camacho, David |
| authorships[3].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2411.18472 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-12-05T00:00:00 |
| display_name | Isolating authorship from content with semantic embeddings and contrastive learning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12380 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9970999956130981 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Authorship Attribution and Profiling |
| related_works | https://openalex.org/W4250539519, https://openalex.org/W4294018197, https://openalex.org/W4233433299, https://openalex.org/W2379384513, https://openalex.org/W2213288308, https://openalex.org/W2884856691, https://openalex.org/W4244821991, https://openalex.org/W2125732910, https://openalex.org/W2909905999, https://openalex.org/W3204019825 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2411.18472 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2411.18472 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2411.18472 |
| primary_location.id | pmh:oai:arXiv.org:2411.18472 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2411.18472 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2411.18472 |
| publication_date | 2024-11-27 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 81, 95, 150 |
| abstract_inverted_index.We | 79, 121 |
| abstract_inverted_index.as | 174 |
| abstract_inverted_index.by | 36, 119 |
| abstract_inverted_index.in | 14, 153 |
| abstract_inverted_index.is | 35, 62, 67 |
| abstract_inverted_index.of | 40, 59, 168, 171 |
| abstract_inverted_index.on | 127, 134, 141, 144, 162 |
| abstract_inverted_index.so | 18 |
| abstract_inverted_index.to | 32, 68, 83, 103, 115, 149 |
| abstract_inverted_index.up | 50, 148 |
| abstract_inverted_index.10% | 151 |
| abstract_inverted_index.Our | 65 |
| abstract_inverted_index.aim | 66 |
| abstract_inverted_index.and | 4, 73, 77, 131 |
| abstract_inverted_index.are | 138, 158 |
| abstract_inverted_index.can | 48 |
| abstract_inverted_index.for | 46 |
| abstract_inverted_index.has | 1 |
| abstract_inverted_index.out | 31 |
| abstract_inverted_index.the | 11, 15, 24, 28, 38, 70, 105, 110, 123, 156, 166 |
| abstract_inverted_index.two | 128 |
| abstract_inverted_index.use | 84 |
| abstract_inverted_index.way | 30 |
| abstract_inverted_index.This | 99 |
| abstract_inverted_index.aims | 102 |
| abstract_inverted_index.also | 164 |
| abstract_inverted_index.fine | 175 |
| abstract_inverted_index.from | 109 |
| abstract_inverted_index.hard | 90 |
| abstract_inverted_index.more | 117 |
| abstract_inverted_index.pick | 49 |
| abstract_inverted_index.same | 12, 16, 26 |
| abstract_inverted_index.some | 57 |
| abstract_inverted_index.them | 34, 133 |
| abstract_inverted_index.this | 172 |
| abstract_inverted_index.when | 19, 155 |
| abstract_inverted_index.with | 88, 125, 147 |
| abstract_inverted_index.about | 10, 23 |
| abstract_inverted_index.exact | 25 |
| abstract_inverted_index.hard. | 160 |
| abstract_inverted_index.shown | 140 |
| abstract_inverted_index.space | 108 |
| abstract_inverted_index.style | 3, 111 |
| abstract_inverted_index.their | 41 |
| abstract_inverted_index.these | 51 |
| abstract_inverted_index.topic | 27 |
| abstract_inverted_index.using | 53, 94 |
| abstract_inverted_index.write | 9, 22 |
| abstract_inverted_index.Modern | 43 |
| abstract_inverted_index.Trials | 161 |
| abstract_inverted_index.always | 63 |
| abstract_inverted_index.amount | 58 |
| abstract_inverted_index.impact | 72 |
| abstract_inverted_index.method | 173 |
| abstract_inverted_index.model. | 98 |
| abstract_inverted_index.models | 45 |
| abstract_inverted_index.neural | 44 |
| abstract_inverted_index.reduce | 69 |
| abstract_inverted_index.space, | 113 |
| abstract_inverted_index.style, | 17 |
| abstract_inverted_index.style. | 42, 120 |
| abstract_inverted_index.topics | 13 |
| abstract_inverted_index.Authors | 7 |
| abstract_inverted_index.authors | 21, 146 |
| abstract_inverted_index.between | 75 |
| abstract_inverted_index.clearly | 139 |
| abstract_inverted_index.compare | 132 |
| abstract_inverted_index.content | 5, 60, 76, 106 |
| abstract_inverted_index.created | 93 |
| abstract_inverted_index.easiest | 29 |
| abstract_inverted_index.inside. | 6 |
| abstract_inverted_index.leading | 114 |
| abstract_inverted_index.leakage | 61 |
| abstract_inverted_index.nuances | 39 |
| abstract_inverted_index.present | 80 |
| abstract_inverted_index.tuning. | 176 |
| abstract_inverted_index.accuracy | 154 |
| abstract_inverted_index.datasets | 130 |
| abstract_inverted_index.distance | 104 |
| abstract_inverted_index.features | 52 |
| abstract_inverted_index.however, | 56 |
| abstract_inverted_index.increase | 152 |
| abstract_inverted_index.informed | 118 |
| abstract_inverted_index.learning | 86 |
| abstract_inverted_index.present. | 64 |
| abstract_inverted_index.prolific | 145 |
| abstract_inverted_index.semantic | 96 |
| abstract_inverted_index.settings | 157 |
| abstract_inverted_index.(InfoNCE) | 87 |
| abstract_inverted_index.ablations | 126 |
| abstract_inverted_index.different | 20, 129 |
| abstract_inverted_index.embedding | 107, 112 |
| abstract_inverted_index.entangled | 2 |
| abstract_inverted_index.learning, | 55 |
| abstract_inverted_index.negatives | 91 |
| abstract_inverted_index.technique | 82, 101 |
| abstract_inverted_index.zero-shot | 169 |
| abstract_inverted_index.Authorship | 0 |
| abstract_inverted_index.additional | 89 |
| abstract_inverted_index.authorship | 47 |
| abstract_inverted_index.challenges | 163 |
| abstract_inverted_index.embeddings | 116 |
| abstract_inverted_index.frequently | 8 |
| abstract_inverted_index.inevitable | 71 |
| abstract_inverted_index.similarity | 97 |
| abstract_inverted_index.authorship. | 78 |
| abstract_inverted_index.challenges. | 136 |
| abstract_inverted_index.challenging | 142 |
| abstract_inverted_index.contrastive | 54, 85 |
| abstract_inverted_index.correlation | 74 |
| abstract_inverted_index.demonstrate | 122, 165 |
| abstract_inverted_index.distinguish | 33 |
| abstract_inverted_index.evaluations | 143 |
| abstract_inverted_index.performance | 124 |
| abstract_inverted_index.Improvements | 137 |
| abstract_inverted_index.capabilities | 170 |
| abstract_inverted_index.particularly | 159 |
| abstract_inverted_index.preservation | 167 |
| abstract_inverted_index.out-of-domain | 135 |
| abstract_inverted_index.synthetically | 92 |
| abstract_inverted_index.understanding | 37 |
| abstract_inverted_index.disentanglement | 100 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile |