Two Stream Self-Supervised Learning for Action Recognition Article Swipe
Ahmed Taha
,
Moustafa Meshry
,
Xitong Yang
,
Yi‐Ting Chen
,
Larry S. Davis
·
YOU?
·
· 2018
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.1806.07383
YOU?
·
· 2018
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.1806.07383
We present a self-supervised approach using spatio-temporal signals between video frames for action recognition. A two-stream architecture is leveraged to tangle spatial and temporal representation learning. Our task is formulated as both a sequence verification and spatio-temporal alignment tasks. The former task requires motion temporal structure understanding while the latter couples the learned motion with the spatial representation. The self-supervised pre-trained weights effectiveness is validated on the action recognition task. Quantitative evaluation shows the self-supervised approach competence on three datasets: HMDB51, UCF101, and Honda driving dataset (HDD). Further investigations to boost performance and generalize validity are still required.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/1806.07383
- https://arxiv.org/pdf/1806.07383
- OA Status
- green
- Cited By
- 6
- References
- 3
- Related Works
- 20
- OpenAlex ID
- https://openalex.org/W2809446765
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W2809446765Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.1806.07383Digital Object Identifier
- Title
-
Two Stream Self-Supervised Learning for Action RecognitionWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2018Year of publication
- Publication date
-
2018-06-16Full publication date if available
- Authors
-
Ahmed Taha, Moustafa Meshry, Xitong Yang, Yi‐Ting Chen, Larry S. DavisList of authors in order
- Landing page
-
https://arxiv.org/abs/1806.07383Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/1806.07383Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/1806.07383Direct OA link when available
- Concepts
-
Computer science, Artificial intelligence, Action recognition, Task (project management), Pattern recognition (psychology), Representation (politics), Machine learning, Motion (physics), Feature learning, Multi-task learning, Class (philosophy), Law, Management, Economics, Political science, PoliticsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
6Total citation count in OpenAlex
- Citations by year (recent)
-
2021: 1, 2020: 3, 2019: 2Per-year citation counts (last 5 years)
- References (count)
-
3Number of works referenced by this work
- Related works (count)
-
20Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W2809446765 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.1806.07383 |
| ids.doi | https://doi.org/10.48550/arxiv.1806.07383 |
| ids.mag | 2809446765 |
| ids.openalex | https://openalex.org/W2809446765 |
| fwci | |
| type | preprint |
| title | Two Stream Self-Supervised Learning for Action Recognition |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10812 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9998999834060669 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Human Pose and Action Recognition |
| topics[1].id | https://openalex.org/T11512 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9965999722480774 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Anomaly Detection Techniques and Applications |
| topics[2].id | https://openalex.org/T12740 |
| topics[2].field.id | https://openalex.org/fields/22 |
| topics[2].field.display_name | Engineering |
| topics[2].score | 0.9940000176429749 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2204 |
| topics[2].subfield.display_name | Biomedical Engineering |
| topics[2].display_name | Gait Recognition and Analysis |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.7572470903396606 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C154945302 |
| concepts[1].level | 1 |
| concepts[1].score | 0.6804115772247314 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[1].display_name | Artificial intelligence |
| concepts[2].id | https://openalex.org/C2987834672 |
| concepts[2].level | 3 |
| concepts[2].score | 0.6606600284576416 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q4677630 |
| concepts[2].display_name | Action recognition |
| concepts[3].id | https://openalex.org/C2780451532 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5980467200279236 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q759676 |
| concepts[3].display_name | Task (project management) |
| concepts[4].id | https://openalex.org/C153180895 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5573164224624634 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q7148389 |
| concepts[4].display_name | Pattern recognition (psychology) |
| concepts[5].id | https://openalex.org/C2776359362 |
| concepts[5].level | 3 |
| concepts[5].score | 0.5358191132545471 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q2145286 |
| concepts[5].display_name | Representation (politics) |
| concepts[6].id | https://openalex.org/C119857082 |
| concepts[6].level | 1 |
| concepts[6].score | 0.4853886365890503 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[6].display_name | Machine learning |
| concepts[7].id | https://openalex.org/C104114177 |
| concepts[7].level | 2 |
| concepts[7].score | 0.4738486409187317 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q79782 |
| concepts[7].display_name | Motion (physics) |
| concepts[8].id | https://openalex.org/C59404180 |
| concepts[8].level | 2 |
| concepts[8].score | 0.44787320494651794 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q17013334 |
| concepts[8].display_name | Feature learning |
| concepts[9].id | https://openalex.org/C28006648 |
| concepts[9].level | 3 |
| concepts[9].score | 0.4384491741657257 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q6934509 |
| concepts[9].display_name | Multi-task learning |
| concepts[10].id | https://openalex.org/C2777212361 |
| concepts[10].level | 2 |
| concepts[10].score | 0.07105275988578796 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q5127848 |
| concepts[10].display_name | Class (philosophy) |
| concepts[11].id | https://openalex.org/C199539241 |
| concepts[11].level | 1 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q7748 |
| concepts[11].display_name | Law |
| concepts[12].id | https://openalex.org/C187736073 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q2920921 |
| concepts[12].display_name | Management |
| concepts[13].id | https://openalex.org/C162324750 |
| concepts[13].level | 0 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q8134 |
| concepts[13].display_name | Economics |
| concepts[14].id | https://openalex.org/C17744445 |
| concepts[14].level | 0 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q36442 |
| concepts[14].display_name | Political science |
| concepts[15].id | https://openalex.org/C94625758 |
| concepts[15].level | 2 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q7163 |
| concepts[15].display_name | Politics |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.7572470903396606 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[1].score | 0.6804115772247314 |
| keywords[1].display_name | Artificial intelligence |
| keywords[2].id | https://openalex.org/keywords/action-recognition |
| keywords[2].score | 0.6606600284576416 |
| keywords[2].display_name | Action recognition |
| keywords[3].id | https://openalex.org/keywords/task |
| keywords[3].score | 0.5980467200279236 |
| keywords[3].display_name | Task (project management) |
| keywords[4].id | https://openalex.org/keywords/pattern-recognition |
| keywords[4].score | 0.5573164224624634 |
| keywords[4].display_name | Pattern recognition (psychology) |
| keywords[5].id | https://openalex.org/keywords/representation |
| keywords[5].score | 0.5358191132545471 |
| keywords[5].display_name | Representation (politics) |
| keywords[6].id | https://openalex.org/keywords/machine-learning |
| keywords[6].score | 0.4853886365890503 |
| keywords[6].display_name | Machine learning |
| keywords[7].id | https://openalex.org/keywords/motion |
| keywords[7].score | 0.4738486409187317 |
| keywords[7].display_name | Motion (physics) |
| keywords[8].id | https://openalex.org/keywords/feature-learning |
| keywords[8].score | 0.44787320494651794 |
| keywords[8].display_name | Feature learning |
| keywords[9].id | https://openalex.org/keywords/multi-task-learning |
| keywords[9].score | 0.4384491741657257 |
| keywords[9].display_name | Multi-task learning |
| keywords[10].id | https://openalex.org/keywords/class |
| keywords[10].score | 0.07105275988578796 |
| keywords[10].display_name | Class (philosophy) |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:1806.07383 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/1806.07383 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/1806.07383 |
| locations[1].id | mag:2809446765 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | submittedVersion |
| locations[1].raw_type | |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | False |
| locations[1].raw_source_name | arXiv (Cornell University) |
| locations[1].landing_page_url | https://arxiv.org/pdf/1806.07383.pdf |
| locations[2].id | doi:10.48550/arxiv.1806.07383 |
| locations[2].is_oa | True |
| locations[2].source.id | https://openalex.org/S4306400194 |
| locations[2].source.issn | |
| locations[2].source.type | repository |
| locations[2].source.is_oa | True |
| locations[2].source.issn_l | |
| locations[2].source.is_core | False |
| locations[2].source.is_in_doaj | False |
| locations[2].source.display_name | arXiv (Cornell University) |
| locations[2].source.host_organization | https://openalex.org/I205783295 |
| locations[2].source.host_organization_name | Cornell University |
| locations[2].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[2].license | |
| locations[2].pdf_url | |
| locations[2].version | |
| locations[2].raw_type | article |
| locations[2].license_id | |
| locations[2].is_accepted | False |
| locations[2].is_published | |
| locations[2].raw_source_name | |
| locations[2].landing_page_url | https://doi.org/10.48550/arxiv.1806.07383 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5071123879 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-4214-9247 |
| authorships[0].author.display_name | Ahmed Taha |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Ahmed Taha |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5002875160 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Moustafa Meshry |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Moustafa Meshry |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5091064356 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-4372-241X |
| authorships[2].author.display_name | Xitong Yang |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Xitong Yang |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5100348259 |
| authorships[3].author.orcid | https://orcid.org/0000-0001-7924-5649 |
| authorships[3].author.display_name | Yi‐Ting Chen |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Yi-Ting Chen |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5111454036 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Larry S. Davis |
| authorships[4].author_position | last |
| authorships[4].raw_author_name | Larry Davis |
| authorships[4].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/1806.07383 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Two Stream Self-Supervised Learning for Action Recognition |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10812 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9998999834060669 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Human Pose and Action Recognition |
| related_works | https://openalex.org/W2194775991, https://openalex.org/W2963446712, https://openalex.org/W2108598243, https://openalex.org/W2930478971, https://openalex.org/W2997907976, https://openalex.org/W3106939553, https://openalex.org/W3200037705, https://openalex.org/W2941505483, https://openalex.org/W3034381931, https://openalex.org/W3159212182, https://openalex.org/W3180472782, https://openalex.org/W3197097949, https://openalex.org/W3036085800, https://openalex.org/W2888101838, https://openalex.org/W3206854705, https://openalex.org/W3104870221, https://openalex.org/W3175181692, https://openalex.org/W2994869114, https://openalex.org/W3163676133, https://openalex.org/W3177339245 |
| cited_by_count | 6 |
| counts_by_year[0].year | 2021 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2020 |
| counts_by_year[1].cited_by_count | 3 |
| counts_by_year[2].year | 2019 |
| counts_by_year[2].cited_by_count | 2 |
| locations_count | 3 |
| best_oa_location.id | pmh:oai:arXiv.org:1806.07383 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/1806.07383 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/1806.07383 |
| primary_location.id | pmh:oai:arXiv.org:1806.07383 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/1806.07383 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/1806.07383 |
| publication_date | 2018-06-16 |
| publication_year | 2018 |
| referenced_works | https://openalex.org/W2550462002, https://openalex.org/W2964037671, https://openalex.org/W2798338071 |
| referenced_works_count | 3 |
| abstract_inverted_index.A | 14 |
| abstract_inverted_index.a | 2, 32 |
| abstract_inverted_index.We | 0 |
| abstract_inverted_index.as | 30 |
| abstract_inverted_index.is | 17, 28, 63 |
| abstract_inverted_index.on | 65, 77 |
| abstract_inverted_index.to | 19, 89 |
| abstract_inverted_index.Our | 26 |
| abstract_inverted_index.The | 39, 58 |
| abstract_inverted_index.and | 22, 35, 82, 92 |
| abstract_inverted_index.are | 95 |
| abstract_inverted_index.for | 11 |
| abstract_inverted_index.the | 48, 51, 55, 66, 73 |
| abstract_inverted_index.both | 31 |
| abstract_inverted_index.task | 27, 41 |
| abstract_inverted_index.with | 54 |
| abstract_inverted_index.Honda | 83 |
| abstract_inverted_index.boost | 90 |
| abstract_inverted_index.shows | 72 |
| abstract_inverted_index.still | 96 |
| abstract_inverted_index.task. | 69 |
| abstract_inverted_index.three | 78 |
| abstract_inverted_index.using | 5 |
| abstract_inverted_index.video | 9 |
| abstract_inverted_index.while | 47 |
| abstract_inverted_index.(HDD). | 86 |
| abstract_inverted_index.action | 12, 67 |
| abstract_inverted_index.former | 40 |
| abstract_inverted_index.frames | 10 |
| abstract_inverted_index.latter | 49 |
| abstract_inverted_index.motion | 43, 53 |
| abstract_inverted_index.tangle | 20 |
| abstract_inverted_index.tasks. | 38 |
| abstract_inverted_index.Further | 87 |
| abstract_inverted_index.HMDB51, | 80 |
| abstract_inverted_index.UCF101, | 81 |
| abstract_inverted_index.between | 8 |
| abstract_inverted_index.couples | 50 |
| abstract_inverted_index.dataset | 85 |
| abstract_inverted_index.driving | 84 |
| abstract_inverted_index.learned | 52 |
| abstract_inverted_index.present | 1 |
| abstract_inverted_index.signals | 7 |
| abstract_inverted_index.spatial | 21, 56 |
| abstract_inverted_index.weights | 61 |
| abstract_inverted_index.approach | 4, 75 |
| abstract_inverted_index.requires | 42 |
| abstract_inverted_index.sequence | 33 |
| abstract_inverted_index.temporal | 23, 44 |
| abstract_inverted_index.validity | 94 |
| abstract_inverted_index.alignment | 37 |
| abstract_inverted_index.datasets: | 79 |
| abstract_inverted_index.learning. | 25 |
| abstract_inverted_index.leveraged | 18 |
| abstract_inverted_index.required. | 97 |
| abstract_inverted_index.structure | 45 |
| abstract_inverted_index.validated | 64 |
| abstract_inverted_index.competence | 76 |
| abstract_inverted_index.evaluation | 71 |
| abstract_inverted_index.formulated | 29 |
| abstract_inverted_index.generalize | 93 |
| abstract_inverted_index.two-stream | 15 |
| abstract_inverted_index.performance | 91 |
| abstract_inverted_index.pre-trained | 60 |
| abstract_inverted_index.recognition | 68 |
| abstract_inverted_index.Quantitative | 70 |
| abstract_inverted_index.architecture | 16 |
| abstract_inverted_index.recognition. | 13 |
| abstract_inverted_index.verification | 34 |
| abstract_inverted_index.effectiveness | 62 |
| abstract_inverted_index.understanding | 46 |
| abstract_inverted_index.investigations | 88 |
| abstract_inverted_index.representation | 24 |
| abstract_inverted_index.representation. | 57 |
| abstract_inverted_index.self-supervised | 3, 59, 74 |
| abstract_inverted_index.spatio-temporal | 6, 36 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 5 |
| citation_normalized_percentile |