Alignment and stability of embeddings: measurement and inference improvement Article Swipe
Furkan Gürsoy
,
Mounir Haddad
,
Cécile Bothorel
·
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2101.07251
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2101.07251
Representation learning (RL) methods learn objects' latent embeddings where information is preserved by distances. Since distances are invariant to certain linear transformations, one may obtain different embeddings while preserving the same information. In dynamic systems, a temporal difference in embeddings may be explained by the stability of the system or by the misalignment of embeddings due to arbitrary transformations. In the literature, embedding alignment has not been defined formally, explored theoretically, or analyzed empirically. Here, we explore the embedding alignment and its parts, provide the first formal definitions, propose novel metrics to measure alignment and stability, and show their suitability through synthetic experiments. Real-world experiments show that both static and dynamic RL methods are prone to produce misaligned embeddings and such misalignment worsens the performance of dynamic network inference tasks. By ensuring alignment, the prediction accuracy raises by up to 90% in static and by 40% in dynamic RL methods.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- https://doi.org/10.48550/arxiv.2101.07251
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4383316345
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4383316345Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2101.07251Digital Object Identifier
- Title
-
Alignment and stability of embeddings: measurement and inference improvementWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2021Year of publication
- Publication date
-
2021-01-18Full publication date if available
- Authors
-
Furkan Gürsoy, Mounir Haddad, Cécile BothorelList of authors in order
- Landing page
-
https://doi.org/10.48550/arxiv.2101.07251Publisher landing page
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://doi.org/10.48550/arxiv.2101.07251Direct OA link when available
- Concepts
-
Embedding, Inference, Computer science, Representation (politics), Invariant (physics), Stability (learning theory), Measure (data warehouse), Algorithm, Theoretical computer science, Artificial intelligence, Mathematics, Machine learning, Data mining, Mathematical physics, Politics, Law, Political scienceTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4383316345 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2101.07251 |
| ids.doi | https://doi.org/10.48550/arxiv.2101.07251 |
| ids.openalex | https://openalex.org/W4383316345 |
| fwci | |
| type | preprint |
| title | Alignment and stability of embeddings: measurement and inference improvement |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11273 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9944999814033508 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Advanced Graph Neural Networks |
| topics[1].id | https://openalex.org/T11307 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.96670001745224 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Domain Adaptation and Few-Shot Learning |
| topics[2].id | https://openalex.org/T12676 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9053999781608582 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Machine Learning and ELM |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41608201 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8700330853462219 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q980509 |
| concepts[0].display_name | Embedding |
| concepts[1].id | https://openalex.org/C2776214188 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7494330406188965 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q408386 |
| concepts[1].display_name | Inference |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.7066542506217957 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C2776359362 |
| concepts[3].level | 3 |
| concepts[3].score | 0.6726742386817932 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q2145286 |
| concepts[3].display_name | Representation (politics) |
| concepts[4].id | https://openalex.org/C190470478 |
| concepts[4].level | 2 |
| concepts[4].score | 0.6329073905944824 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q2370229 |
| concepts[4].display_name | Invariant (physics) |
| concepts[5].id | https://openalex.org/C112972136 |
| concepts[5].level | 2 |
| concepts[5].score | 0.5808982253074646 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q7595718 |
| concepts[5].display_name | Stability (learning theory) |
| concepts[6].id | https://openalex.org/C2780009758 |
| concepts[6].level | 2 |
| concepts[6].score | 0.5674006342887878 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q6804172 |
| concepts[6].display_name | Measure (data warehouse) |
| concepts[7].id | https://openalex.org/C11413529 |
| concepts[7].level | 1 |
| concepts[7].score | 0.46181997656822205 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[7].display_name | Algorithm |
| concepts[8].id | https://openalex.org/C80444323 |
| concepts[8].level | 1 |
| concepts[8].score | 0.45950740575790405 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q2878974 |
| concepts[8].display_name | Theoretical computer science |
| concepts[9].id | https://openalex.org/C154945302 |
| concepts[9].level | 1 |
| concepts[9].score | 0.37750884890556335 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[9].display_name | Artificial intelligence |
| concepts[10].id | https://openalex.org/C33923547 |
| concepts[10].level | 0 |
| concepts[10].score | 0.23159149289131165 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[10].display_name | Mathematics |
| concepts[11].id | https://openalex.org/C119857082 |
| concepts[11].level | 1 |
| concepts[11].score | 0.22201889753341675 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[11].display_name | Machine learning |
| concepts[12].id | https://openalex.org/C124101348 |
| concepts[12].level | 1 |
| concepts[12].score | 0.19398176670074463 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q172491 |
| concepts[12].display_name | Data mining |
| concepts[13].id | https://openalex.org/C37914503 |
| concepts[13].level | 1 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q156495 |
| concepts[13].display_name | Mathematical physics |
| concepts[14].id | https://openalex.org/C94625758 |
| concepts[14].level | 2 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q7163 |
| concepts[14].display_name | Politics |
| concepts[15].id | https://openalex.org/C199539241 |
| concepts[15].level | 1 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q7748 |
| concepts[15].display_name | Law |
| concepts[16].id | https://openalex.org/C17744445 |
| concepts[16].level | 0 |
| concepts[16].score | 0.0 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q36442 |
| concepts[16].display_name | Political science |
| keywords[0].id | https://openalex.org/keywords/embedding |
| keywords[0].score | 0.8700330853462219 |
| keywords[0].display_name | Embedding |
| keywords[1].id | https://openalex.org/keywords/inference |
| keywords[1].score | 0.7494330406188965 |
| keywords[1].display_name | Inference |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.7066542506217957 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/representation |
| keywords[3].score | 0.6726742386817932 |
| keywords[3].display_name | Representation (politics) |
| keywords[4].id | https://openalex.org/keywords/invariant |
| keywords[4].score | 0.6329073905944824 |
| keywords[4].display_name | Invariant (physics) |
| keywords[5].id | https://openalex.org/keywords/stability |
| keywords[5].score | 0.5808982253074646 |
| keywords[5].display_name | Stability (learning theory) |
| keywords[6].id | https://openalex.org/keywords/measure |
| keywords[6].score | 0.5674006342887878 |
| keywords[6].display_name | Measure (data warehouse) |
| keywords[7].id | https://openalex.org/keywords/algorithm |
| keywords[7].score | 0.46181997656822205 |
| keywords[7].display_name | Algorithm |
| keywords[8].id | https://openalex.org/keywords/theoretical-computer-science |
| keywords[8].score | 0.45950740575790405 |
| keywords[8].display_name | Theoretical computer science |
| keywords[9].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[9].score | 0.37750884890556335 |
| keywords[9].display_name | Artificial intelligence |
| keywords[10].id | https://openalex.org/keywords/mathematics |
| keywords[10].score | 0.23159149289131165 |
| keywords[10].display_name | Mathematics |
| keywords[11].id | https://openalex.org/keywords/machine-learning |
| keywords[11].score | 0.22201889753341675 |
| keywords[11].display_name | Machine learning |
| keywords[12].id | https://openalex.org/keywords/data-mining |
| keywords[12].score | 0.19398176670074463 |
| keywords[12].display_name | Data mining |
| language | en |
| locations[0].id | doi:10.48550/arxiv.2101.07251 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | |
| locations[0].version | |
| locations[0].raw_type | article |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | https://doi.org/10.48550/arxiv.2101.07251 |
| indexed_in | datacite |
| authorships[0].author.id | https://openalex.org/A5017257146 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-9701-2814 |
| authorships[0].author.display_name | Furkan Gürsoy |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Gürsoy, Furkan |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5006054575 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-1824-4647 |
| authorships[1].author.display_name | Mounir Haddad |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Haddad, Mounir |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5063950837 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-6602-1769 |
| authorships[2].author.display_name | Cécile Bothorel |
| authorships[2].author_position | last |
| authorships[2].raw_author_name | Bothorel, Cécile |
| authorships[2].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://doi.org/10.48550/arxiv.2101.07251 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2023-07-07T00:00:00 |
| display_name | Alignment and stability of embeddings: measurement and inference improvement |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11273 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9944999814033508 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Advanced Graph Neural Networks |
| related_works | https://openalex.org/W2081900870, https://openalex.org/W4255837520, https://openalex.org/W2387011115, https://openalex.org/W2183306018, https://openalex.org/W2549990292, https://openalex.org/W2345479200, https://openalex.org/W4234808182, https://openalex.org/W2055243143, https://openalex.org/W2382043075, https://openalex.org/W2951819827 |
| cited_by_count | 0 |
| locations_count | 1 |
| best_oa_location.id | doi:10.48550/arxiv.2101.07251 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | |
| best_oa_location.version | |
| best_oa_location.raw_type | article |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | https://doi.org/10.48550/arxiv.2101.07251 |
| primary_location.id | doi:10.48550/arxiv.2101.07251 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | |
| primary_location.version | |
| primary_location.raw_type | article |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | https://doi.org/10.48550/arxiv.2101.07251 |
| publication_date | 2021-01-18 |
| publication_year | 2021 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 35 |
| abstract_inverted_index.By | 130 |
| abstract_inverted_index.In | 32, 59 |
| abstract_inverted_index.RL | 111, 148 |
| abstract_inverted_index.be | 41 |
| abstract_inverted_index.by | 12, 43, 50, 137, 144 |
| abstract_inverted_index.in | 38, 141, 146 |
| abstract_inverted_index.is | 10 |
| abstract_inverted_index.of | 46, 53, 125 |
| abstract_inverted_index.or | 49, 71 |
| abstract_inverted_index.to | 18, 56, 91, 115, 139 |
| abstract_inverted_index.up | 138 |
| abstract_inverted_index.we | 75 |
| abstract_inverted_index.40% | 145 |
| abstract_inverted_index.90% | 140 |
| abstract_inverted_index.and | 80, 94, 96, 109, 119, 143 |
| abstract_inverted_index.are | 16, 113 |
| abstract_inverted_index.due | 55 |
| abstract_inverted_index.has | 64 |
| abstract_inverted_index.its | 81 |
| abstract_inverted_index.may | 23, 40 |
| abstract_inverted_index.not | 65 |
| abstract_inverted_index.one | 22 |
| abstract_inverted_index.the | 29, 44, 47, 51, 60, 77, 84, 123, 133 |
| abstract_inverted_index.(RL) | 2 |
| abstract_inverted_index.been | 66 |
| abstract_inverted_index.both | 107 |
| abstract_inverted_index.same | 30 |
| abstract_inverted_index.show | 97, 105 |
| abstract_inverted_index.such | 120 |
| abstract_inverted_index.that | 106 |
| abstract_inverted_index.Here, | 74 |
| abstract_inverted_index.Since | 14 |
| abstract_inverted_index.first | 85 |
| abstract_inverted_index.learn | 4 |
| abstract_inverted_index.novel | 89 |
| abstract_inverted_index.prone | 114 |
| abstract_inverted_index.their | 98 |
| abstract_inverted_index.where | 8 |
| abstract_inverted_index.while | 27 |
| abstract_inverted_index.formal | 86 |
| abstract_inverted_index.latent | 6 |
| abstract_inverted_index.linear | 20 |
| abstract_inverted_index.obtain | 24 |
| abstract_inverted_index.parts, | 82 |
| abstract_inverted_index.raises | 136 |
| abstract_inverted_index.static | 108, 142 |
| abstract_inverted_index.system | 48 |
| abstract_inverted_index.tasks. | 129 |
| abstract_inverted_index.certain | 19 |
| abstract_inverted_index.defined | 67 |
| abstract_inverted_index.dynamic | 33, 110, 126, 147 |
| abstract_inverted_index.explore | 76 |
| abstract_inverted_index.measure | 92 |
| abstract_inverted_index.methods | 3, 112 |
| abstract_inverted_index.metrics | 90 |
| abstract_inverted_index.network | 127 |
| abstract_inverted_index.produce | 116 |
| abstract_inverted_index.propose | 88 |
| abstract_inverted_index.provide | 83 |
| abstract_inverted_index.through | 100 |
| abstract_inverted_index.worsens | 122 |
| abstract_inverted_index.accuracy | 135 |
| abstract_inverted_index.analyzed | 72 |
| abstract_inverted_index.ensuring | 131 |
| abstract_inverted_index.explored | 69 |
| abstract_inverted_index.learning | 1 |
| abstract_inverted_index.methods. | 149 |
| abstract_inverted_index.objects' | 5 |
| abstract_inverted_index.systems, | 34 |
| abstract_inverted_index.temporal | 36 |
| abstract_inverted_index.alignment | 63, 79, 93 |
| abstract_inverted_index.arbitrary | 57 |
| abstract_inverted_index.different | 25 |
| abstract_inverted_index.distances | 15 |
| abstract_inverted_index.embedding | 62, 78 |
| abstract_inverted_index.explained | 42 |
| abstract_inverted_index.formally, | 68 |
| abstract_inverted_index.inference | 128 |
| abstract_inverted_index.invariant | 17 |
| abstract_inverted_index.preserved | 11 |
| abstract_inverted_index.stability | 45 |
| abstract_inverted_index.synthetic | 101 |
| abstract_inverted_index.Real-world | 103 |
| abstract_inverted_index.alignment, | 132 |
| abstract_inverted_index.difference | 37 |
| abstract_inverted_index.distances. | 13 |
| abstract_inverted_index.embeddings | 7, 26, 39, 54, 118 |
| abstract_inverted_index.misaligned | 117 |
| abstract_inverted_index.prediction | 134 |
| abstract_inverted_index.preserving | 28 |
| abstract_inverted_index.stability, | 95 |
| abstract_inverted_index.experiments | 104 |
| abstract_inverted_index.information | 9 |
| abstract_inverted_index.literature, | 61 |
| abstract_inverted_index.performance | 124 |
| abstract_inverted_index.suitability | 99 |
| abstract_inverted_index.definitions, | 87 |
| abstract_inverted_index.empirically. | 73 |
| abstract_inverted_index.experiments. | 102 |
| abstract_inverted_index.information. | 31 |
| abstract_inverted_index.misalignment | 52, 121 |
| abstract_inverted_index.Representation | 0 |
| abstract_inverted_index.theoretically, | 70 |
| abstract_inverted_index.transformations, | 21 |
| abstract_inverted_index.transformations. | 58 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 3 |
| citation_normalized_percentile |