Pose-Aware Self-Supervised Learning with Viewpoint Trajectory Regularization Article Swipe
Jiayun Wang
,
Stella X. Yu
,
Yubei Chen
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2403.14973
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2403.14973
Learning visual features from unlabeled images has proven successful for semantic categorization, often by mapping different $views$ of the same object to the same feature to achieve recognition invariance. However, visual recognition involves not only identifying $what$ an object is but also understanding $how$ it is presented. For example, seeing a car from the side versus head-on is crucial for deciding whether to stay put or jump out of the way. While unsupervised feature learning for downstream viewpoint reasoning is important, it remains under-explored, partly due to the lack of a standardized evaluation method and benchmarks. We introduce a new dataset of adjacent image triplets obtained from a viewpoint trajectory, without any semantic or pose labels. We benchmark both semantic classification and pose estimation accuracies on the same visual feature. Additionally, we propose a viewpoint trajectory regularization loss for learning features from unlabeled image triplets. Our experiments demonstrate that this approach helps develop a visual representation that encodes object identity and organizes objects by their poses, retaining semantic classification accuracy while achieving emergent global pose awareness and better generalization to novel objects. Our dataset and code are available at http://pwang.pw/trajSSL/.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2403.14973
- https://arxiv.org/pdf/2403.14973
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4393178062
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4393178062Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2403.14973Digital Object Identifier
- Title
-
Pose-Aware Self-Supervised Learning with Viewpoint Trajectory RegularizationWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-03-22Full publication date if available
- Authors
-
Jiayun Wang, Stella X. Yu, Yubei ChenList of authors in order
- Landing page
-
https://arxiv.org/abs/2403.14973Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2403.14973Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2403.14973Direct OA link when available
- Concepts
-
Regularization (linguistics), Representation (politics), Computer science, Trajectory, Artificial intelligence, Mathematics, Physics, Political science, Law, Astronomy, PoliticsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4393178062 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2403.14973 |
| ids.doi | https://doi.org/10.48550/arxiv.2403.14973 |
| ids.openalex | https://openalex.org/W4393178062 |
| fwci | |
| type | preprint |
| title | Pose-Aware Self-Supervised Learning with Viewpoint Trajectory Regularization |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T14339 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.8219000101089478 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Image Processing and 3D Reconstruction |
| topics[1].id | https://openalex.org/T10320 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.7930999994277954 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Neural Networks and Applications |
| topics[2].id | https://openalex.org/T13067 |
| topics[2].field.id | https://openalex.org/fields/19 |
| topics[2].field.display_name | Earth and Planetary Sciences |
| topics[2].score | 0.7192999720573425 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1906 |
| topics[2].subfield.display_name | Geochemistry and Petrology |
| topics[2].display_name | Geological Modeling and Analysis |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2776135515 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7591370344161987 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q17143721 |
| concepts[0].display_name | Regularization (linguistics) |
| concepts[1].id | https://openalex.org/C2776359362 |
| concepts[1].level | 3 |
| concepts[1].score | 0.5874763131141663 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q2145286 |
| concepts[1].display_name | Representation (politics) |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.4837055802345276 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C13662910 |
| concepts[3].level | 2 |
| concepts[3].score | 0.47273126244544983 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q193139 |
| concepts[3].display_name | Trajectory |
| concepts[4].id | https://openalex.org/C154945302 |
| concepts[4].level | 1 |
| concepts[4].score | 0.4543078541755676 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[4].display_name | Artificial intelligence |
| concepts[5].id | https://openalex.org/C33923547 |
| concepts[5].level | 0 |
| concepts[5].score | 0.4254779517650604 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[5].display_name | Mathematics |
| concepts[6].id | https://openalex.org/C121332964 |
| concepts[6].level | 0 |
| concepts[6].score | 0.14650842547416687 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[6].display_name | Physics |
| concepts[7].id | https://openalex.org/C17744445 |
| concepts[7].level | 0 |
| concepts[7].score | 0.0 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q36442 |
| concepts[7].display_name | Political science |
| concepts[8].id | https://openalex.org/C199539241 |
| concepts[8].level | 1 |
| concepts[8].score | 0.0 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q7748 |
| concepts[8].display_name | Law |
| concepts[9].id | https://openalex.org/C1276947 |
| concepts[9].level | 1 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q333 |
| concepts[9].display_name | Astronomy |
| concepts[10].id | https://openalex.org/C94625758 |
| concepts[10].level | 2 |
| concepts[10].score | 0.0 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q7163 |
| concepts[10].display_name | Politics |
| keywords[0].id | https://openalex.org/keywords/regularization |
| keywords[0].score | 0.7591370344161987 |
| keywords[0].display_name | Regularization (linguistics) |
| keywords[1].id | https://openalex.org/keywords/representation |
| keywords[1].score | 0.5874763131141663 |
| keywords[1].display_name | Representation (politics) |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.4837055802345276 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/trajectory |
| keywords[3].score | 0.47273126244544983 |
| keywords[3].display_name | Trajectory |
| keywords[4].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[4].score | 0.4543078541755676 |
| keywords[4].display_name | Artificial intelligence |
| keywords[5].id | https://openalex.org/keywords/mathematics |
| keywords[5].score | 0.4254779517650604 |
| keywords[5].display_name | Mathematics |
| keywords[6].id | https://openalex.org/keywords/physics |
| keywords[6].score | 0.14650842547416687 |
| keywords[6].display_name | Physics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2403.14973 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2403.14973 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2403.14973 |
| locations[1].id | doi:10.48550/arxiv.2403.14973 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2403.14973 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5038575919 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-1823-8829 |
| authorships[0].author.display_name | Jiayun Wang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Wang, Jiayun |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5042014034 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-3507-5761 |
| authorships[1].author.display_name | Stella X. Yu |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Yu, Stella X. |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5013970773 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-8930-3512 |
| authorships[2].author.display_name | Yubei Chen |
| authorships[2].author_position | last |
| authorships[2].raw_author_name | Chen, Yubei |
| authorships[2].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2403.14973 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-03-26T00:00:00 |
| display_name | Pose-Aware Self-Supervised Learning with Viewpoint Trajectory Regularization |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T14339 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.8219000101089478 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Image Processing and 3D Reconstruction |
| related_works | https://openalex.org/W1979597421, https://openalex.org/W2007980826, https://openalex.org/W2061531152, https://openalex.org/W3002753104, https://openalex.org/W2077600819, https://openalex.org/W2142036596, https://openalex.org/W2072657027, https://openalex.org/W2600246793, https://openalex.org/W4238204885, https://openalex.org/W2963966623 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2403.14973 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2403.14973 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2403.14973 |
| primary_location.id | pmh:oai:arXiv.org:2403.14973 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2403.14973 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2403.14973 |
| publication_date | 2024-03-22 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 50, 90, 98, 107, 133, 153 |
| abstract_inverted_index.We | 96, 116 |
| abstract_inverted_index.an | 37 |
| abstract_inverted_index.at | 188 |
| abstract_inverted_index.by | 13, 163 |
| abstract_inverted_index.is | 39, 45, 57, 79 |
| abstract_inverted_index.it | 44, 81 |
| abstract_inverted_index.of | 17, 68, 89, 101 |
| abstract_inverted_index.on | 125 |
| abstract_inverted_index.or | 65, 113 |
| abstract_inverted_index.to | 21, 25, 62, 86, 179 |
| abstract_inverted_index.we | 131 |
| abstract_inverted_index.For | 47 |
| abstract_inverted_index.Our | 145, 182 |
| abstract_inverted_index.and | 94, 121, 160, 176, 184 |
| abstract_inverted_index.any | 111 |
| abstract_inverted_index.are | 186 |
| abstract_inverted_index.but | 40 |
| abstract_inverted_index.car | 51 |
| abstract_inverted_index.due | 85 |
| abstract_inverted_index.for | 9, 59, 75, 138 |
| abstract_inverted_index.has | 6 |
| abstract_inverted_index.new | 99 |
| abstract_inverted_index.not | 33 |
| abstract_inverted_index.out | 67 |
| abstract_inverted_index.put | 64 |
| abstract_inverted_index.the | 18, 22, 53, 69, 87, 126 |
| abstract_inverted_index.also | 41 |
| abstract_inverted_index.both | 118 |
| abstract_inverted_index.code | 185 |
| abstract_inverted_index.from | 3, 52, 106, 141 |
| abstract_inverted_index.jump | 66 |
| abstract_inverted_index.lack | 88 |
| abstract_inverted_index.loss | 137 |
| abstract_inverted_index.only | 34 |
| abstract_inverted_index.pose | 114, 122, 174 |
| abstract_inverted_index.same | 19, 23, 127 |
| abstract_inverted_index.side | 54 |
| abstract_inverted_index.stay | 63 |
| abstract_inverted_index.that | 148, 156 |
| abstract_inverted_index.this | 149 |
| abstract_inverted_index.way. | 70 |
| abstract_inverted_index.$how$ | 43 |
| abstract_inverted_index.While | 71 |
| abstract_inverted_index.helps | 151 |
| abstract_inverted_index.image | 103, 143 |
| abstract_inverted_index.novel | 180 |
| abstract_inverted_index.often | 12 |
| abstract_inverted_index.their | 164 |
| abstract_inverted_index.while | 170 |
| abstract_inverted_index.$what$ | 36 |
| abstract_inverted_index.better | 177 |
| abstract_inverted_index.global | 173 |
| abstract_inverted_index.images | 5 |
| abstract_inverted_index.method | 93 |
| abstract_inverted_index.object | 20, 38, 158 |
| abstract_inverted_index.partly | 84 |
| abstract_inverted_index.poses, | 165 |
| abstract_inverted_index.proven | 7 |
| abstract_inverted_index.seeing | 49 |
| abstract_inverted_index.versus | 55 |
| abstract_inverted_index.visual | 1, 30, 128, 154 |
| abstract_inverted_index.$views$ | 16 |
| abstract_inverted_index.achieve | 26 |
| abstract_inverted_index.crucial | 58 |
| abstract_inverted_index.dataset | 100, 183 |
| abstract_inverted_index.develop | 152 |
| abstract_inverted_index.encodes | 157 |
| abstract_inverted_index.feature | 24, 73 |
| abstract_inverted_index.head-on | 56 |
| abstract_inverted_index.labels. | 115 |
| abstract_inverted_index.mapping | 14 |
| abstract_inverted_index.objects | 162 |
| abstract_inverted_index.propose | 132 |
| abstract_inverted_index.remains | 82 |
| abstract_inverted_index.whether | 61 |
| abstract_inverted_index.without | 110 |
| abstract_inverted_index.However, | 29 |
| abstract_inverted_index.Learning | 0 |
| abstract_inverted_index.accuracy | 169 |
| abstract_inverted_index.adjacent | 102 |
| abstract_inverted_index.approach | 150 |
| abstract_inverted_index.deciding | 60 |
| abstract_inverted_index.emergent | 172 |
| abstract_inverted_index.example, | 48 |
| abstract_inverted_index.feature. | 129 |
| abstract_inverted_index.features | 2, 140 |
| abstract_inverted_index.identity | 159 |
| abstract_inverted_index.involves | 32 |
| abstract_inverted_index.learning | 74, 139 |
| abstract_inverted_index.objects. | 181 |
| abstract_inverted_index.obtained | 105 |
| abstract_inverted_index.semantic | 10, 112, 119, 167 |
| abstract_inverted_index.triplets | 104 |
| abstract_inverted_index.achieving | 171 |
| abstract_inverted_index.available | 187 |
| abstract_inverted_index.awareness | 175 |
| abstract_inverted_index.benchmark | 117 |
| abstract_inverted_index.different | 15 |
| abstract_inverted_index.introduce | 97 |
| abstract_inverted_index.organizes | 161 |
| abstract_inverted_index.reasoning | 78 |
| abstract_inverted_index.retaining | 166 |
| abstract_inverted_index.triplets. | 144 |
| abstract_inverted_index.unlabeled | 4, 142 |
| abstract_inverted_index.viewpoint | 77, 108, 134 |
| abstract_inverted_index.accuracies | 124 |
| abstract_inverted_index.downstream | 76 |
| abstract_inverted_index.estimation | 123 |
| abstract_inverted_index.evaluation | 92 |
| abstract_inverted_index.important, | 80 |
| abstract_inverted_index.presented. | 46 |
| abstract_inverted_index.successful | 8 |
| abstract_inverted_index.trajectory | 135 |
| abstract_inverted_index.benchmarks. | 95 |
| abstract_inverted_index.demonstrate | 147 |
| abstract_inverted_index.experiments | 146 |
| abstract_inverted_index.identifying | 35 |
| abstract_inverted_index.invariance. | 28 |
| abstract_inverted_index.recognition | 27, 31 |
| abstract_inverted_index.trajectory, | 109 |
| abstract_inverted_index.standardized | 91 |
| abstract_inverted_index.unsupervised | 72 |
| abstract_inverted_index.Additionally, | 130 |
| abstract_inverted_index.understanding | 42 |
| abstract_inverted_index.classification | 120, 168 |
| abstract_inverted_index.generalization | 178 |
| abstract_inverted_index.regularization | 136 |
| abstract_inverted_index.representation | 155 |
| abstract_inverted_index.categorization, | 11 |
| abstract_inverted_index.under-explored, | 83 |
| abstract_inverted_index.http://pwang.pw/trajSSL/. | 189 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 3 |
| citation_normalized_percentile |