Study of detecting behavioral signatures within DeepFake videos Article Swipe
YOU?
·
· 2022
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2208.03561
There is strong interest in the generation of synthetic video imagery of people talking for various purposes, including entertainment, communication, training, and advertisement. With the development of deep fake generation models, synthetic video imagery will soon be visually indistinguishable to the naked eye from a naturally capture video. In addition, many methods are continuing to improve to avoid more careful, forensic visual analysis. Some deep fake videos are produced through the use of facial puppetry, which directly controls the head and face of the synthetic image through the movements of the actor, allow the actor to 'puppet' the image of another. In this paper, we address the question of whether one person's movements can be distinguished from the original speaker by controlling the visual appearance of the speaker but transferring the behavior signals from another source. We conduct a study by comparing synthetic imagery that: 1) originates from a different person speaking a different utterance, 2) originates from the same person speaking a different utterance, and 3) originates from a different person speaking the same utterance. Our study shows that synthetic videos in all three cases are seen as less real and less engaging than the original source video. Our results indicate that there could be a behavioral signature that is detectable from a person's movements that is separate from their visual appearance, and that this behavioral signature could be used to distinguish a deep fake from a properly captured video.
Related Topics
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2208.03561
- https://arxiv.org/pdf/2208.03561
- OA Status
- green
- Cited By
- 2
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4302391656
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4302391656Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2208.03561Digital Object Identifier
- Title
-
Study of detecting behavioral signatures within DeepFake videosWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2022Year of publication
- Publication date
-
2022-08-06Full publication date if available
- Authors
-
Qiaomu Miao, Sin‐Hwa Kang, Stacy Marsella, Steve DiPaola, Chao Wang, Ari ShapiroList of authors in order
- Landing page
-
https://arxiv.org/abs/2208.03561Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2208.03561Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2208.03561Direct OA link when available
- Concepts
-
Utterance, Computer science, Entertainment, Face (sociological concept), Artificial intelligence, Computer vision, Linguistics, Visual arts, Art, PhilosophyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
2Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1, 2024: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4302391656 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2208.03561 |
| ids.doi | https://doi.org/10.48550/arxiv.2208.03561 |
| ids.openalex | https://openalex.org/W4302391656 |
| fwci | |
| type | preprint |
| title | Study of detecting behavioral signatures within DeepFake videos |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12357 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9994000196456909 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Digital Media Forensic Detection |
| topics[1].id | https://openalex.org/T10775 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9908000230789185 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1707 |
| topics[1].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[1].display_name | Generative Adversarial Networks and Image Synthesis |
| topics[2].id | https://openalex.org/T11448 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9599999785423279 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1707 |
| topics[2].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[2].display_name | Face recognition and analysis |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2775852435 |
| concepts[0].level | 2 |
| concepts[0].score | 0.81467205286026 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q258403 |
| concepts[0].display_name | Utterance |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.6510649919509888 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C512170562 |
| concepts[2].level | 2 |
| concepts[2].score | 0.5252560973167419 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q173799 |
| concepts[2].display_name | Entertainment |
| concepts[3].id | https://openalex.org/C2779304628 |
| concepts[3].level | 2 |
| concepts[3].score | 0.48706749081611633 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q3503480 |
| concepts[3].display_name | Face (sociological concept) |
| concepts[4].id | https://openalex.org/C154945302 |
| concepts[4].level | 1 |
| concepts[4].score | 0.4527164399623871 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[4].display_name | Artificial intelligence |
| concepts[5].id | https://openalex.org/C31972630 |
| concepts[5].level | 1 |
| concepts[5].score | 0.35179632902145386 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q844240 |
| concepts[5].display_name | Computer vision |
| concepts[6].id | https://openalex.org/C41895202 |
| concepts[6].level | 1 |
| concepts[6].score | 0.11171665787696838 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q8162 |
| concepts[6].display_name | Linguistics |
| concepts[7].id | https://openalex.org/C153349607 |
| concepts[7].level | 1 |
| concepts[7].score | 0.09007570147514343 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q36649 |
| concepts[7].display_name | Visual arts |
| concepts[8].id | https://openalex.org/C142362112 |
| concepts[8].level | 0 |
| concepts[8].score | 0.0 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q735 |
| concepts[8].display_name | Art |
| concepts[9].id | https://openalex.org/C138885662 |
| concepts[9].level | 0 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q5891 |
| concepts[9].display_name | Philosophy |
| keywords[0].id | https://openalex.org/keywords/utterance |
| keywords[0].score | 0.81467205286026 |
| keywords[0].display_name | Utterance |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.6510649919509888 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/entertainment |
| keywords[2].score | 0.5252560973167419 |
| keywords[2].display_name | Entertainment |
| keywords[3].id | https://openalex.org/keywords/face |
| keywords[3].score | 0.48706749081611633 |
| keywords[3].display_name | Face (sociological concept) |
| keywords[4].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[4].score | 0.4527164399623871 |
| keywords[4].display_name | Artificial intelligence |
| keywords[5].id | https://openalex.org/keywords/computer-vision |
| keywords[5].score | 0.35179632902145386 |
| keywords[5].display_name | Computer vision |
| keywords[6].id | https://openalex.org/keywords/linguistics |
| keywords[6].score | 0.11171665787696838 |
| keywords[6].display_name | Linguistics |
| keywords[7].id | https://openalex.org/keywords/visual-arts |
| keywords[7].score | 0.09007570147514343 |
| keywords[7].display_name | Visual arts |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2208.03561 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2208.03561 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2208.03561 |
| locations[1].id | doi:10.48550/arxiv.2208.03561 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2208.03561 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5007347641 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Qiaomu Miao |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Miao, Qiaomu |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5074475670 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-2619-5021 |
| authorships[1].author.display_name | Sin‐Hwa Kang |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Kang, Sinhwa |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5058433199 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-5711-7934 |
| authorships[2].author.display_name | Stacy Marsella |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Marsella, Stacy |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5008452037 |
| authorships[3].author.orcid | https://orcid.org/0000-0001-7549-9545 |
| authorships[3].author.display_name | Steve DiPaola |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | DiPaola, Steve |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5115076700 |
| authorships[4].author.orcid | https://orcid.org/0000-0003-4887-923X |
| authorships[4].author.display_name | Chao Wang |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Wang, Chao |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5062006413 |
| authorships[5].author.orcid | https://orcid.org/0000-0001-8626-4119 |
| authorships[5].author.display_name | Ari Shapiro |
| authorships[5].author_position | last |
| authorships[5].raw_author_name | Shapiro, Ari |
| authorships[5].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2208.03561 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Study of detecting behavioral signatures within DeepFake videos |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12357 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9994000196456909 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Digital Media Forensic Detection |
| related_works | https://openalex.org/W2058170566, https://openalex.org/W2755342338, https://openalex.org/W2772917594, https://openalex.org/W2775347418, https://openalex.org/W2166024367, https://openalex.org/W3116076068, https://openalex.org/W2229312674, https://openalex.org/W2951359407, https://openalex.org/W2079911747, https://openalex.org/W1969923398 |
| cited_by_count | 2 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2208.03561 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2208.03561 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2208.03561 |
| primary_location.id | pmh:oai:arXiv.org:2208.03561 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2208.03561 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2208.03561 |
| publication_date | 2022-08-06 |
| publication_year | 2022 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 44, 138, 148, 152, 162, 169, 206, 213, 233, 237 |
| abstract_inverted_index.1) | 145 |
| abstract_inverted_index.2) | 155 |
| abstract_inverted_index.3) | 166 |
| abstract_inverted_index.In | 48, 101 |
| abstract_inverted_index.We | 136 |
| abstract_inverted_index.as | 188 |
| abstract_inverted_index.be | 36, 114, 205, 229 |
| abstract_inverted_index.by | 120, 140 |
| abstract_inverted_index.in | 4, 182 |
| abstract_inverted_index.is | 1, 210, 217 |
| abstract_inverted_index.of | 7, 11, 26, 72, 82, 89, 99, 108, 125 |
| abstract_inverted_index.to | 39, 54, 56, 95, 231 |
| abstract_inverted_index.we | 104 |
| abstract_inverted_index.Our | 176, 199 |
| abstract_inverted_index.all | 183 |
| abstract_inverted_index.and | 21, 80, 165, 191, 223 |
| abstract_inverted_index.are | 52, 67, 186 |
| abstract_inverted_index.but | 128 |
| abstract_inverted_index.can | 113 |
| abstract_inverted_index.eye | 42 |
| abstract_inverted_index.for | 14 |
| abstract_inverted_index.one | 110 |
| abstract_inverted_index.the | 5, 24, 40, 70, 78, 83, 87, 90, 93, 97, 106, 117, 122, 126, 130, 158, 173, 195 |
| abstract_inverted_index.use | 71 |
| abstract_inverted_index.Some | 63 |
| abstract_inverted_index.With | 23 |
| abstract_inverted_index.deep | 27, 64, 234 |
| abstract_inverted_index.face | 81 |
| abstract_inverted_index.fake | 28, 65, 235 |
| abstract_inverted_index.from | 43, 116, 133, 147, 157, 168, 212, 219, 236 |
| abstract_inverted_index.head | 79 |
| abstract_inverted_index.less | 189, 192 |
| abstract_inverted_index.many | 50 |
| abstract_inverted_index.more | 58 |
| abstract_inverted_index.real | 190 |
| abstract_inverted_index.same | 159, 174 |
| abstract_inverted_index.seen | 187 |
| abstract_inverted_index.soon | 35 |
| abstract_inverted_index.than | 194 |
| abstract_inverted_index.that | 179, 202, 209, 216, 224 |
| abstract_inverted_index.this | 102, 225 |
| abstract_inverted_index.used | 230 |
| abstract_inverted_index.will | 34 |
| abstract_inverted_index.There | 0 |
| abstract_inverted_index.actor | 94 |
| abstract_inverted_index.allow | 92 |
| abstract_inverted_index.avoid | 57 |
| abstract_inverted_index.cases | 185 |
| abstract_inverted_index.could | 204, 228 |
| abstract_inverted_index.image | 85, 98 |
| abstract_inverted_index.naked | 41 |
| abstract_inverted_index.shows | 178 |
| abstract_inverted_index.study | 139, 177 |
| abstract_inverted_index.that: | 144 |
| abstract_inverted_index.their | 220 |
| abstract_inverted_index.there | 203 |
| abstract_inverted_index.three | 184 |
| abstract_inverted_index.video | 9, 32 |
| abstract_inverted_index.which | 75 |
| abstract_inverted_index.actor, | 91 |
| abstract_inverted_index.facial | 73 |
| abstract_inverted_index.paper, | 103 |
| abstract_inverted_index.people | 12 |
| abstract_inverted_index.person | 150, 160, 171 |
| abstract_inverted_index.source | 197 |
| abstract_inverted_index.strong | 2 |
| abstract_inverted_index.video. | 47, 198, 240 |
| abstract_inverted_index.videos | 66, 181 |
| abstract_inverted_index.visual | 61, 123, 221 |
| abstract_inverted_index.address | 105 |
| abstract_inverted_index.another | 134 |
| abstract_inverted_index.capture | 46 |
| abstract_inverted_index.conduct | 137 |
| abstract_inverted_index.imagery | 10, 33, 143 |
| abstract_inverted_index.improve | 55 |
| abstract_inverted_index.methods | 51 |
| abstract_inverted_index.models, | 30 |
| abstract_inverted_index.results | 200 |
| abstract_inverted_index.signals | 132 |
| abstract_inverted_index.source. | 135 |
| abstract_inverted_index.speaker | 119, 127 |
| abstract_inverted_index.talking | 13 |
| abstract_inverted_index.through | 69, 86 |
| abstract_inverted_index.various | 15 |
| abstract_inverted_index.whether | 109 |
| abstract_inverted_index.'puppet' | 96 |
| abstract_inverted_index.another. | 100 |
| abstract_inverted_index.behavior | 131 |
| abstract_inverted_index.captured | 239 |
| abstract_inverted_index.careful, | 59 |
| abstract_inverted_index.controls | 77 |
| abstract_inverted_index.directly | 76 |
| abstract_inverted_index.engaging | 193 |
| abstract_inverted_index.forensic | 60 |
| abstract_inverted_index.indicate | 201 |
| abstract_inverted_index.interest | 3 |
| abstract_inverted_index.original | 118, 196 |
| abstract_inverted_index.person's | 111, 214 |
| abstract_inverted_index.produced | 68 |
| abstract_inverted_index.properly | 238 |
| abstract_inverted_index.question | 107 |
| abstract_inverted_index.separate | 218 |
| abstract_inverted_index.speaking | 151, 161, 172 |
| abstract_inverted_index.visually | 37 |
| abstract_inverted_index.addition, | 49 |
| abstract_inverted_index.analysis. | 62 |
| abstract_inverted_index.comparing | 141 |
| abstract_inverted_index.different | 149, 153, 163, 170 |
| abstract_inverted_index.including | 17 |
| abstract_inverted_index.movements | 88, 112, 215 |
| abstract_inverted_index.naturally | 45 |
| abstract_inverted_index.puppetry, | 74 |
| abstract_inverted_index.purposes, | 16 |
| abstract_inverted_index.signature | 208, 227 |
| abstract_inverted_index.synthetic | 8, 31, 84, 142, 180 |
| abstract_inverted_index.training, | 20 |
| abstract_inverted_index.appearance | 124 |
| abstract_inverted_index.behavioral | 207, 226 |
| abstract_inverted_index.continuing | 53 |
| abstract_inverted_index.detectable | 211 |
| abstract_inverted_index.generation | 6, 29 |
| abstract_inverted_index.originates | 146, 156, 167 |
| abstract_inverted_index.utterance, | 154, 164 |
| abstract_inverted_index.utterance. | 175 |
| abstract_inverted_index.appearance, | 222 |
| abstract_inverted_index.controlling | 121 |
| abstract_inverted_index.development | 25 |
| abstract_inverted_index.distinguish | 232 |
| abstract_inverted_index.transferring | 129 |
| abstract_inverted_index.distinguished | 115 |
| abstract_inverted_index.advertisement. | 22 |
| abstract_inverted_index.communication, | 19 |
| abstract_inverted_index.entertainment, | 18 |
| abstract_inverted_index.indistinguishable | 38 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 6 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/16 |
| sustainable_development_goals[0].score | 0.4699999988079071 |
| sustainable_development_goals[0].display_name | Peace, Justice and strong institutions |
| citation_normalized_percentile |