Using Motion Cues to Supervise Single-Frame Body Pose and Shape Estimation in Low Data Regimes Article Swipe
Andrey V. Davydov
,
Alexey Sidnev
,
Artsiom Sanakoyeu
,
Yuhua Chen
,
Mathieu Salzmann
,
Pascal Fua
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2402.02736
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2402.02736
When enough annotated training data is available, supervised deep-learning algorithms excel at estimating human body pose and shape using a single camera. The effects of too little such data being available can be mitigated by using other information sources, such as databases of body shapes, to learn priors. Unfortunately, such sources are not always available either. We show that, in such cases, easy-to-obtain unannotated videos can be used instead to provide the required supervisory signals. Given a trained model using too little annotated data, we compute poses in consecutive frames along with the optical flow between them. We then enforce consistency between the image optical flow and the one that can be inferred from the change in pose from one frame to the next. This provides enough additional supervision to effectively refine the network weights and to perform on par with methods trained using far more annotated data.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2402.02736
- https://arxiv.org/pdf/2402.02736
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4391591316
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4391591316Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2402.02736Digital Object Identifier
- Title
-
Using Motion Cues to Supervise Single-Frame Body Pose and Shape Estimation in Low Data RegimesWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-02-05Full publication date if available
- Authors
-
Andrey V. Davydov, Alexey Sidnev, Artsiom Sanakoyeu, Yuhua Chen, Mathieu Salzmann, Pascal FuaList of authors in order
- Landing page
-
https://arxiv.org/abs/2402.02736Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2402.02736Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2402.02736Direct OA link when available
- Concepts
-
Frame (networking), Motion (physics), Pose, Computer vision, Computer science, Artificial intelligence, Motion capture, TelecommunicationsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4391591316 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2402.02736 |
| ids.doi | https://doi.org/10.48550/arxiv.2402.02736 |
| ids.openalex | https://openalex.org/W4391591316 |
| fwci | |
| type | preprint |
| title | Using Motion Cues to Supervise Single-Frame Body Pose and Shape Estimation in Low Data Regimes |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10719 |
| topics[0].field.id | https://openalex.org/fields/22 |
| topics[0].field.display_name | Engineering |
| topics[0].score | 0.9703999757766724 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2206 |
| topics[0].subfield.display_name | Computational Mechanics |
| topics[0].display_name | 3D Shape Modeling and Analysis |
| topics[1].id | https://openalex.org/T10531 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9661999940872192 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1707 |
| topics[1].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[1].display_name | Advanced Vision and Imaging |
| topics[2].id | https://openalex.org/T12290 |
| topics[2].field.id | https://openalex.org/fields/22 |
| topics[2].field.display_name | Engineering |
| topics[2].score | 0.9628999829292297 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2207 |
| topics[2].subfield.display_name | Control and Systems Engineering |
| topics[2].display_name | Human Motion and Animation |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C126042441 |
| concepts[0].level | 2 |
| concepts[0].score | 0.6173064708709717 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1324888 |
| concepts[0].display_name | Frame (networking) |
| concepts[1].id | https://openalex.org/C104114177 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6127853989601135 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q79782 |
| concepts[1].display_name | Motion (physics) |
| concepts[2].id | https://openalex.org/C52102323 |
| concepts[2].level | 2 |
| concepts[2].score | 0.5920872092247009 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q1671968 |
| concepts[2].display_name | Pose |
| concepts[3].id | https://openalex.org/C31972630 |
| concepts[3].level | 1 |
| concepts[3].score | 0.5448101162910461 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q844240 |
| concepts[3].display_name | Computer vision |
| concepts[4].id | https://openalex.org/C41008148 |
| concepts[4].level | 0 |
| concepts[4].score | 0.5050609707832336 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[4].display_name | Computer science |
| concepts[5].id | https://openalex.org/C154945302 |
| concepts[5].level | 1 |
| concepts[5].score | 0.47986119985580444 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[5].display_name | Artificial intelligence |
| concepts[6].id | https://openalex.org/C48007421 |
| concepts[6].level | 3 |
| concepts[6].score | 0.4470491111278534 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q676252 |
| concepts[6].display_name | Motion capture |
| concepts[7].id | https://openalex.org/C76155785 |
| concepts[7].level | 1 |
| concepts[7].score | 0.07361850142478943 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q418 |
| concepts[7].display_name | Telecommunications |
| keywords[0].id | https://openalex.org/keywords/frame |
| keywords[0].score | 0.6173064708709717 |
| keywords[0].display_name | Frame (networking) |
| keywords[1].id | https://openalex.org/keywords/motion |
| keywords[1].score | 0.6127853989601135 |
| keywords[1].display_name | Motion (physics) |
| keywords[2].id | https://openalex.org/keywords/pose |
| keywords[2].score | 0.5920872092247009 |
| keywords[2].display_name | Pose |
| keywords[3].id | https://openalex.org/keywords/computer-vision |
| keywords[3].score | 0.5448101162910461 |
| keywords[3].display_name | Computer vision |
| keywords[4].id | https://openalex.org/keywords/computer-science |
| keywords[4].score | 0.5050609707832336 |
| keywords[4].display_name | Computer science |
| keywords[5].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[5].score | 0.47986119985580444 |
| keywords[5].display_name | Artificial intelligence |
| keywords[6].id | https://openalex.org/keywords/motion-capture |
| keywords[6].score | 0.4470491111278534 |
| keywords[6].display_name | Motion capture |
| keywords[7].id | https://openalex.org/keywords/telecommunications |
| keywords[7].score | 0.07361850142478943 |
| keywords[7].display_name | Telecommunications |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2402.02736 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2402.02736 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2402.02736 |
| locations[1].id | pmh:oai:infoscience.epfl.ch:20.500.14299/242053 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400487 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | Infoscience (Ecole Polytechnique Fédérale de Lausanne) |
| locations[1].source.host_organization | |
| locations[1].source.host_organization_name | |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | submittedVersion |
| locations[1].raw_type | research article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | False |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://infoscience.epfl.ch/handle/20.500.14299/242053 |
| locations[2].id | doi:10.48550/arxiv.2402.02736 |
| locations[2].is_oa | True |
| locations[2].source.id | https://openalex.org/S4306400194 |
| locations[2].source.issn | |
| locations[2].source.type | repository |
| locations[2].source.is_oa | True |
| locations[2].source.issn_l | |
| locations[2].source.is_core | False |
| locations[2].source.is_in_doaj | False |
| locations[2].source.display_name | arXiv (Cornell University) |
| locations[2].source.host_organization | https://openalex.org/I205783295 |
| locations[2].source.host_organization_name | Cornell University |
| locations[2].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[2].license | |
| locations[2].pdf_url | |
| locations[2].version | |
| locations[2].raw_type | article |
| locations[2].license_id | |
| locations[2].is_accepted | False |
| locations[2].is_published | |
| locations[2].raw_source_name | |
| locations[2].landing_page_url | https://doi.org/10.48550/arxiv.2402.02736 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5103363377 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-7061-5460 |
| authorships[0].author.display_name | Andrey V. Davydov |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Davydov, Andrey |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5008211117 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Alexey Sidnev |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Sidnev, Alexey |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5053620423 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Artsiom Sanakoyeu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Sanakoyeu, Artsiom |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5100384521 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-3218-1304 |
| authorships[3].author.display_name | Yuhua Chen |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Chen, Yuhua |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5049300388 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-8347-8637 |
| authorships[4].author.display_name | Mathieu Salzmann |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Salzmann, Mathieu |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5038674741 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-6702-9970 |
| authorships[5].author.display_name | Pascal Fua |
| authorships[5].author_position | last |
| authorships[5].raw_author_name | Fua, Pascal |
| authorships[5].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2402.02736 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-02-07T00:00:00 |
| display_name | Using Motion Cues to Supervise Single-Frame Body Pose and Shape Estimation in Low Data Regimes |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10719 |
| primary_topic.field.id | https://openalex.org/fields/22 |
| primary_topic.field.display_name | Engineering |
| primary_topic.score | 0.9703999757766724 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2206 |
| primary_topic.subfield.display_name | Computational Mechanics |
| primary_topic.display_name | 3D Shape Modeling and Analysis |
| related_works | https://openalex.org/W2736638679, https://openalex.org/W1968716783, https://openalex.org/W4313046826, https://openalex.org/W1827696521, https://openalex.org/W2039848376, https://openalex.org/W2173450654, https://openalex.org/W2621720158, https://openalex.org/W2130272765, https://openalex.org/W2091722187, https://openalex.org/W2006196742 |
| cited_by_count | 0 |
| locations_count | 3 |
| best_oa_location.id | pmh:oai:arXiv.org:2402.02736 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2402.02736 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2402.02736 |
| primary_location.id | pmh:oai:arXiv.org:2402.02736 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2402.02736 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2402.02736 |
| publication_date | 2024-02-05 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 19, 76 |
| abstract_inverted_index.We | 56, 97 |
| abstract_inverted_index.as | 40 |
| abstract_inverted_index.at | 11 |
| abstract_inverted_index.be | 32, 66, 111 |
| abstract_inverted_index.by | 34 |
| abstract_inverted_index.in | 59, 87, 116 |
| abstract_inverted_index.is | 5 |
| abstract_inverted_index.of | 24, 42 |
| abstract_inverted_index.on | 138 |
| abstract_inverted_index.to | 45, 69, 121, 129, 136 |
| abstract_inverted_index.we | 84 |
| abstract_inverted_index.The | 22 |
| abstract_inverted_index.and | 16, 106, 135 |
| abstract_inverted_index.are | 51 |
| abstract_inverted_index.can | 31, 65, 110 |
| abstract_inverted_index.far | 144 |
| abstract_inverted_index.not | 52 |
| abstract_inverted_index.one | 108, 119 |
| abstract_inverted_index.par | 139 |
| abstract_inverted_index.the | 71, 92, 102, 107, 114, 122, 132 |
| abstract_inverted_index.too | 25, 80 |
| abstract_inverted_index.This | 124 |
| abstract_inverted_index.When | 0 |
| abstract_inverted_index.body | 14, 43 |
| abstract_inverted_index.data | 4, 28 |
| abstract_inverted_index.flow | 94, 105 |
| abstract_inverted_index.from | 113, 118 |
| abstract_inverted_index.more | 145 |
| abstract_inverted_index.pose | 15, 117 |
| abstract_inverted_index.show | 57 |
| abstract_inverted_index.such | 27, 39, 49, 60 |
| abstract_inverted_index.that | 109 |
| abstract_inverted_index.then | 98 |
| abstract_inverted_index.used | 67 |
| abstract_inverted_index.with | 91, 140 |
| abstract_inverted_index.Given | 75 |
| abstract_inverted_index.along | 90 |
| abstract_inverted_index.being | 29 |
| abstract_inverted_index.data, | 83 |
| abstract_inverted_index.data. | 147 |
| abstract_inverted_index.excel | 10 |
| abstract_inverted_index.frame | 120 |
| abstract_inverted_index.human | 13 |
| abstract_inverted_index.image | 103 |
| abstract_inverted_index.learn | 46 |
| abstract_inverted_index.model | 78 |
| abstract_inverted_index.next. | 123 |
| abstract_inverted_index.other | 36 |
| abstract_inverted_index.poses | 86 |
| abstract_inverted_index.shape | 17 |
| abstract_inverted_index.that, | 58 |
| abstract_inverted_index.them. | 96 |
| abstract_inverted_index.using | 18, 35, 79, 143 |
| abstract_inverted_index.always | 53 |
| abstract_inverted_index.cases, | 61 |
| abstract_inverted_index.change | 115 |
| abstract_inverted_index.enough | 1, 126 |
| abstract_inverted_index.frames | 89 |
| abstract_inverted_index.little | 26, 81 |
| abstract_inverted_index.refine | 131 |
| abstract_inverted_index.single | 20 |
| abstract_inverted_index.videos | 64 |
| abstract_inverted_index.between | 95, 101 |
| abstract_inverted_index.camera. | 21 |
| abstract_inverted_index.compute | 85 |
| abstract_inverted_index.effects | 23 |
| abstract_inverted_index.either. | 55 |
| abstract_inverted_index.enforce | 99 |
| abstract_inverted_index.instead | 68 |
| abstract_inverted_index.methods | 141 |
| abstract_inverted_index.network | 133 |
| abstract_inverted_index.optical | 93, 104 |
| abstract_inverted_index.perform | 137 |
| abstract_inverted_index.priors. | 47 |
| abstract_inverted_index.provide | 70 |
| abstract_inverted_index.shapes, | 44 |
| abstract_inverted_index.sources | 50 |
| abstract_inverted_index.trained | 77, 142 |
| abstract_inverted_index.weights | 134 |
| abstract_inverted_index.inferred | 112 |
| abstract_inverted_index.provides | 125 |
| abstract_inverted_index.required | 72 |
| abstract_inverted_index.signals. | 74 |
| abstract_inverted_index.sources, | 38 |
| abstract_inverted_index.training | 3 |
| abstract_inverted_index.annotated | 2, 82, 146 |
| abstract_inverted_index.available | 30, 54 |
| abstract_inverted_index.databases | 41 |
| abstract_inverted_index.mitigated | 33 |
| abstract_inverted_index.additional | 127 |
| abstract_inverted_index.algorithms | 9 |
| abstract_inverted_index.available, | 6 |
| abstract_inverted_index.estimating | 12 |
| abstract_inverted_index.supervised | 7 |
| abstract_inverted_index.consecutive | 88 |
| abstract_inverted_index.consistency | 100 |
| abstract_inverted_index.effectively | 130 |
| abstract_inverted_index.information | 37 |
| abstract_inverted_index.supervision | 128 |
| abstract_inverted_index.supervisory | 73 |
| abstract_inverted_index.unannotated | 63 |
| abstract_inverted_index.deep-learning | 8 |
| abstract_inverted_index.Unfortunately, | 48 |
| abstract_inverted_index.easy-to-obtain | 62 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 6 |
| citation_normalized_percentile |