A Survey of the Self Supervised Learning Mechanisms for Vision Transformers Article Swipe
Asifullah Khan
,
Anabia Sohail
,
Mustansar Fiaz
,
Mehdi Hassan
,
Tariq Habib Afridi
,
Sibghat Ullah Marwat
,
Farzeen Munir
,
Safdar Ali
,
Hannan Naseem
,
Muhammad Zaigham Zaheer
,
Kamran Ali
,
Tangina Sultana
,
Ziaurrehman Tanoli
,
Naeem Akhter
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2408.17059
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2408.17059
Advances in deep learning are re-defining how visual data is processed and understand by the machines. Vision Transformers (ViTs) have recently demonstrated prominent performance in computer vision related tasks. However, their performance improves with increasing numbers of labeled data, indicating reliance on labeled data. Humanly annotated data are difficult to acquire and thus shifted the focus from traditional annotations to unsupervised learning strategies that learn structures inside the data. In response to this challenge, self-supervised learning (SSL) has emerged as a promising technique. SSL utilize inherent relationships within the data as a form of supervision. This technique can reduce the dependence on manual annotations and offers a more scalable and resource-effective approach to training models. Taking these strengths into account, it is necessary to assess the combination of SSL methods with ViTs, especially in the cases of limited labeled data. Inspired by this evolving trend, this survey aims to systematically review SSL mechanisms tailored for ViTs. We propose a comprehensive taxonomy to classify SSL techniques based on their representations and pre-training tasks. Furthermore, we highlighted the motivations behind the study of SSL, reviewed prominent pre-training tasks, and highlight advancements and challenges in this field. Furthermore, we conduct a comparative analysis of various SSL methods designed for ViTs, evaluating their strengths, limitations, and applicability to different scenarios.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2408.17059
- https://arxiv.org/pdf/2408.17059
- OA Status
- green
- Cited By
- 5
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4403713395
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4403713395Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2408.17059Digital Object Identifier
- Title
-
A Survey of the Self Supervised Learning Mechanisms for Vision TransformersWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-08-30Full publication date if available
- Authors
-
Asifullah Khan, Anabia Sohail, Mustansar Fiaz, Mehdi Hassan, Tariq Habib Afridi, Sibghat Ullah Marwat, Farzeen Munir, Safdar Ali, Hannan Naseem, Muhammad Zaigham Zaheer, Kamran Ali, Tangina Sultana, Ziaurrehman Tanoli, Naeem AkhterList of authors in order
- Landing page
-
https://arxiv.org/abs/2408.17059Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2408.17059Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2408.17059Direct OA link when available
- Concepts
-
Computer science, Transformer, Artificial intelligence, Computer vision, Machine learning, Engineering, Electrical engineering, VoltageTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
5Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 5Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4403713395 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2408.17059 |
| ids.doi | https://doi.org/10.48550/arxiv.2408.17059 |
| ids.openalex | https://openalex.org/W4403713395 |
| fwci | |
| type | preprint |
| title | A Survey of the Self Supervised Learning Mechanisms for Vision Transformers |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12111 |
| topics[0].field.id | https://openalex.org/fields/22 |
| topics[0].field.display_name | Engineering |
| topics[0].score | 0.6769000291824341 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2209 |
| topics[0].subfield.display_name | Industrial and Manufacturing Engineering |
| topics[0].display_name | Industrial Vision Systems and Defect Detection |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.5450513362884521 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C66322947 |
| concepts[1].level | 3 |
| concepts[1].score | 0.4950060248374939 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q11658 |
| concepts[1].display_name | Transformer |
| concepts[2].id | https://openalex.org/C154945302 |
| concepts[2].level | 1 |
| concepts[2].score | 0.46372267603874207 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[2].display_name | Artificial intelligence |
| concepts[3].id | https://openalex.org/C31972630 |
| concepts[3].level | 1 |
| concepts[3].score | 0.3692508935928345 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q844240 |
| concepts[3].display_name | Computer vision |
| concepts[4].id | https://openalex.org/C119857082 |
| concepts[4].level | 1 |
| concepts[4].score | 0.35884004831314087 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[4].display_name | Machine learning |
| concepts[5].id | https://openalex.org/C127413603 |
| concepts[5].level | 0 |
| concepts[5].score | 0.2586136758327484 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[5].display_name | Engineering |
| concepts[6].id | https://openalex.org/C119599485 |
| concepts[6].level | 1 |
| concepts[6].score | 0.16145920753479004 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q43035 |
| concepts[6].display_name | Electrical engineering |
| concepts[7].id | https://openalex.org/C165801399 |
| concepts[7].level | 2 |
| concepts[7].score | 0.0 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q25428 |
| concepts[7].display_name | Voltage |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.5450513362884521 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/transformer |
| keywords[1].score | 0.4950060248374939 |
| keywords[1].display_name | Transformer |
| keywords[2].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[2].score | 0.46372267603874207 |
| keywords[2].display_name | Artificial intelligence |
| keywords[3].id | https://openalex.org/keywords/computer-vision |
| keywords[3].score | 0.3692508935928345 |
| keywords[3].display_name | Computer vision |
| keywords[4].id | https://openalex.org/keywords/machine-learning |
| keywords[4].score | 0.35884004831314087 |
| keywords[4].display_name | Machine learning |
| keywords[5].id | https://openalex.org/keywords/engineering |
| keywords[5].score | 0.2586136758327484 |
| keywords[5].display_name | Engineering |
| keywords[6].id | https://openalex.org/keywords/electrical-engineering |
| keywords[6].score | 0.16145920753479004 |
| keywords[6].display_name | Electrical engineering |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2408.17059 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2408.17059 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2408.17059 |
| locations[1].id | doi:10.48550/arxiv.2408.17059 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2408.17059 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5083112369 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-2039-5305 |
| authorships[0].author.display_name | Asifullah Khan |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Khan, Asifullah |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5026855523 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-2961-6098 |
| authorships[1].author.display_name | Anabia Sohail |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Sohail, Anabia |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5066756540 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-2289-2284 |
| authorships[2].author.display_name | Mustansar Fiaz |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Fiaz, Mustansar |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5082020424 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-4629-2582 |
| authorships[3].author.display_name | Mehdi Hassan |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Hassan, Mehdi |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5074260339 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-1535-8203 |
| authorships[4].author.display_name | Tariq Habib Afridi |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Afridi, Tariq Habib |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5114389164 |
| authorships[5].author.orcid | |
| authorships[5].author.display_name | Sibghat Ullah Marwat |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Marwat, Sibghat Ullah |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5036926878 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-6760-4373 |
| authorships[6].author.display_name | Farzeen Munir |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Munir, Farzeen |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5080262652 |
| authorships[7].author.orcid | https://orcid.org/0000-0003-1208-8655 |
| authorships[7].author.display_name | Safdar Ali |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Ali, Safdar |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5114389165 |
| authorships[8].author.orcid | |
| authorships[8].author.display_name | Hannan Naseem |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Naseem, Hannan |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5004839726 |
| authorships[9].author.orcid | https://orcid.org/0000-0001-8272-1351 |
| authorships[9].author.display_name | Muhammad Zaigham Zaheer |
| authorships[9].author_position | middle |
| authorships[9].raw_author_name | Zaheer, Muhammad Zaigham |
| authorships[9].is_corresponding | False |
| authorships[10].author.id | https://openalex.org/A5084192501 |
| authorships[10].author.orcid | https://orcid.org/0000-0002-3122-6729 |
| authorships[10].author.display_name | Kamran Ali |
| authorships[10].author_position | middle |
| authorships[10].raw_author_name | Ali, Kamran |
| authorships[10].is_corresponding | False |
| authorships[11].author.id | https://openalex.org/A5004667864 |
| authorships[11].author.orcid | https://orcid.org/0000-0002-3896-5591 |
| authorships[11].author.display_name | Tangina Sultana |
| authorships[11].author_position | middle |
| authorships[11].raw_author_name | Sultana, Tangina |
| authorships[11].is_corresponding | False |
| authorships[12].author.id | https://openalex.org/A5084617577 |
| authorships[12].author.orcid | https://orcid.org/0000-0003-2435-9862 |
| authorships[12].author.display_name | Ziaurrehman Tanoli |
| authorships[12].author_position | middle |
| authorships[12].raw_author_name | Tanoli, Ziaurrehman |
| authorships[12].is_corresponding | False |
| authorships[13].author.id | https://openalex.org/A5029709862 |
| authorships[13].author.orcid | https://orcid.org/0000-0001-5922-1674 |
| authorships[13].author.display_name | Naeem Akhter |
| authorships[13].author_position | last |
| authorships[13].raw_author_name | Akhter, Naeem |
| authorships[13].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2408.17059 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | A Survey of the Self Supervised Learning Mechanisms for Vision Transformers |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12111 |
| primary_topic.field.id | https://openalex.org/fields/22 |
| primary_topic.field.display_name | Engineering |
| primary_topic.score | 0.6769000291824341 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2209 |
| primary_topic.subfield.display_name | Industrial and Manufacturing Engineering |
| primary_topic.display_name | Industrial Vision Systems and Defect Detection |
| related_works | https://openalex.org/W2961085424, https://openalex.org/W4306674287, https://openalex.org/W2772917594, https://openalex.org/W2036807459, https://openalex.org/W2058170566, https://openalex.org/W2755342338, https://openalex.org/W2166024367, https://openalex.org/W3116076068, https://openalex.org/W2229312674, https://openalex.org/W2951359407 |
| cited_by_count | 5 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 5 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2408.17059 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2408.17059 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2408.17059 |
| primary_location.id | pmh:oai:arXiv.org:2408.17059 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2408.17059 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2408.17059 |
| publication_date | 2024-08-30 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 80, 91, 106, 158, 197 |
| abstract_inverted_index.In | 69 |
| abstract_inverted_index.We | 156 |
| abstract_inverted_index.as | 79, 90 |
| abstract_inverted_index.by | 13, 141 |
| abstract_inverted_index.in | 1, 24, 133, 191 |
| abstract_inverted_index.is | 9, 121 |
| abstract_inverted_index.it | 120 |
| abstract_inverted_index.of | 36, 93, 127, 136, 180, 200 |
| abstract_inverted_index.on | 41, 101, 166 |
| abstract_inverted_index.to | 49, 59, 71, 112, 123, 148, 161, 213 |
| abstract_inverted_index.we | 173, 195 |
| abstract_inverted_index.SSL | 83, 128, 151, 163, 202 |
| abstract_inverted_index.and | 11, 51, 104, 109, 169, 186, 189, 211 |
| abstract_inverted_index.are | 4, 47 |
| abstract_inverted_index.can | 97 |
| abstract_inverted_index.for | 154, 205 |
| abstract_inverted_index.has | 77 |
| abstract_inverted_index.how | 6 |
| abstract_inverted_index.the | 14, 54, 67, 88, 99, 125, 134, 175, 178 |
| abstract_inverted_index.SSL, | 181 |
| abstract_inverted_index.This | 95 |
| abstract_inverted_index.aims | 147 |
| abstract_inverted_index.data | 8, 46, 89 |
| abstract_inverted_index.deep | 2 |
| abstract_inverted_index.form | 92 |
| abstract_inverted_index.from | 56 |
| abstract_inverted_index.have | 19 |
| abstract_inverted_index.into | 118 |
| abstract_inverted_index.more | 107 |
| abstract_inverted_index.that | 63 |
| abstract_inverted_index.this | 72, 142, 145, 192 |
| abstract_inverted_index.thus | 52 |
| abstract_inverted_index.with | 33, 130 |
| abstract_inverted_index.(SSL) | 76 |
| abstract_inverted_index.ViTs, | 131, 206 |
| abstract_inverted_index.ViTs. | 155 |
| abstract_inverted_index.based | 165 |
| abstract_inverted_index.cases | 135 |
| abstract_inverted_index.data, | 38 |
| abstract_inverted_index.data. | 43, 68, 139 |
| abstract_inverted_index.focus | 55 |
| abstract_inverted_index.learn | 64 |
| abstract_inverted_index.study | 179 |
| abstract_inverted_index.their | 30, 167, 208 |
| abstract_inverted_index.these | 116 |
| abstract_inverted_index.(ViTs) | 18 |
| abstract_inverted_index.Taking | 115 |
| abstract_inverted_index.Vision | 16 |
| abstract_inverted_index.assess | 124 |
| abstract_inverted_index.behind | 177 |
| abstract_inverted_index.field. | 193 |
| abstract_inverted_index.inside | 66 |
| abstract_inverted_index.manual | 102 |
| abstract_inverted_index.offers | 105 |
| abstract_inverted_index.reduce | 98 |
| abstract_inverted_index.review | 150 |
| abstract_inverted_index.survey | 146 |
| abstract_inverted_index.tasks, | 185 |
| abstract_inverted_index.tasks. | 28, 171 |
| abstract_inverted_index.trend, | 144 |
| abstract_inverted_index.vision | 26 |
| abstract_inverted_index.visual | 7 |
| abstract_inverted_index.within | 87 |
| abstract_inverted_index.Humanly | 44 |
| abstract_inverted_index.acquire | 50 |
| abstract_inverted_index.conduct | 196 |
| abstract_inverted_index.emerged | 78 |
| abstract_inverted_index.labeled | 37, 42, 138 |
| abstract_inverted_index.limited | 137 |
| abstract_inverted_index.methods | 129, 203 |
| abstract_inverted_index.models. | 114 |
| abstract_inverted_index.numbers | 35 |
| abstract_inverted_index.propose | 157 |
| abstract_inverted_index.related | 27 |
| abstract_inverted_index.shifted | 53 |
| abstract_inverted_index.utilize | 84 |
| abstract_inverted_index.various | 201 |
| abstract_inverted_index.Advances | 0 |
| abstract_inverted_index.However, | 29 |
| abstract_inverted_index.Inspired | 140 |
| abstract_inverted_index.account, | 119 |
| abstract_inverted_index.analysis | 199 |
| abstract_inverted_index.approach | 111 |
| abstract_inverted_index.classify | 162 |
| abstract_inverted_index.computer | 25 |
| abstract_inverted_index.designed | 204 |
| abstract_inverted_index.evolving | 143 |
| abstract_inverted_index.improves | 32 |
| abstract_inverted_index.inherent | 85 |
| abstract_inverted_index.learning | 3, 61, 75 |
| abstract_inverted_index.recently | 20 |
| abstract_inverted_index.reliance | 40 |
| abstract_inverted_index.response | 70 |
| abstract_inverted_index.reviewed | 182 |
| abstract_inverted_index.scalable | 108 |
| abstract_inverted_index.tailored | 153 |
| abstract_inverted_index.taxonomy | 160 |
| abstract_inverted_index.training | 113 |
| abstract_inverted_index.annotated | 45 |
| abstract_inverted_index.different | 214 |
| abstract_inverted_index.difficult | 48 |
| abstract_inverted_index.highlight | 187 |
| abstract_inverted_index.machines. | 15 |
| abstract_inverted_index.necessary | 122 |
| abstract_inverted_index.processed | 10 |
| abstract_inverted_index.prominent | 22, 183 |
| abstract_inverted_index.promising | 81 |
| abstract_inverted_index.strengths | 117 |
| abstract_inverted_index.technique | 96 |
| abstract_inverted_index.challenge, | 73 |
| abstract_inverted_index.challenges | 190 |
| abstract_inverted_index.dependence | 100 |
| abstract_inverted_index.especially | 132 |
| abstract_inverted_index.evaluating | 207 |
| abstract_inverted_index.increasing | 34 |
| abstract_inverted_index.indicating | 39 |
| abstract_inverted_index.mechanisms | 152 |
| abstract_inverted_index.scenarios. | 215 |
| abstract_inverted_index.strategies | 62 |
| abstract_inverted_index.strengths, | 209 |
| abstract_inverted_index.structures | 65 |
| abstract_inverted_index.technique. | 82 |
| abstract_inverted_index.techniques | 164 |
| abstract_inverted_index.understand | 12 |
| abstract_inverted_index.annotations | 58, 103 |
| abstract_inverted_index.combination | 126 |
| abstract_inverted_index.comparative | 198 |
| abstract_inverted_index.highlighted | 174 |
| abstract_inverted_index.motivations | 176 |
| abstract_inverted_index.performance | 23, 31 |
| abstract_inverted_index.re-defining | 5 |
| abstract_inverted_index.traditional | 57 |
| abstract_inverted_index.Furthermore, | 172, 194 |
| abstract_inverted_index.Transformers | 17 |
| abstract_inverted_index.advancements | 188 |
| abstract_inverted_index.demonstrated | 21 |
| abstract_inverted_index.limitations, | 210 |
| abstract_inverted_index.pre-training | 170, 184 |
| abstract_inverted_index.supervision. | 94 |
| abstract_inverted_index.unsupervised | 60 |
| abstract_inverted_index.applicability | 212 |
| abstract_inverted_index.comprehensive | 159 |
| abstract_inverted_index.relationships | 86 |
| abstract_inverted_index.systematically | 149 |
| abstract_inverted_index.representations | 168 |
| abstract_inverted_index.self-supervised | 74 |
| abstract_inverted_index.resource-effective | 110 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 14 |
| citation_normalized_percentile |