SHIFT3D: Synthesizing Hard Inputs For Tricking 3D Detectors Article Swipe
Hongge Chen
,
Chen Zhao
,
Gregory P. Meyer
,
Dennis Park
,
Carl Vondrick
,
Ashish Shrivastava
,
Yuning Chai
·
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2309.05810
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2309.05810
We present SHIFT3D, a differentiable pipeline for generating 3D shapes that are structurally plausible yet challenging to 3D object detectors. In safety-critical applications like autonomous driving, discovering such novel challenging objects can offer insight into unknown vulnerabilities of 3D detectors. By representing objects with a signed distanced function (SDF), we show that gradient error signals allow us to smoothly deform the shape or pose of a 3D object in order to confuse a downstream 3D detector. Importantly, the objects generated by SHIFT3D physically differ from the baseline object yet retain a semantically recognizable shape. Our approach provides interpretable failure modes for modern 3D object detectors, and can aid in preemptive discovery of potential safety risks within 3D perception systems before these risks become critical failures.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2309.05810
- https://arxiv.org/pdf/2309.05810
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4386721608
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4386721608Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2309.05810Digital Object Identifier
- Title
-
SHIFT3D: Synthesizing Hard Inputs For Tricking 3D DetectorsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-09-11Full publication date if available
- Authors
-
Hongge Chen, Chen Zhao, Gregory P. Meyer, Dennis Park, Carl Vondrick, Ashish Shrivastava, Yuning ChaiList of authors in order
- Landing page
-
https://arxiv.org/abs/2309.05810Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2309.05810Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2309.05810Direct OA link when available
- Concepts
-
Pipeline (software), Object (grammar), Computer science, Detector, Differentiable function, Function (biology), Artificial intelligence, Perception, Computer vision, Programming language, Mathematics, Psychology, Evolutionary biology, Telecommunications, Biology, Mathematical analysis, NeuroscienceTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4386721608 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2309.05810 |
| ids.doi | https://doi.org/10.48550/arxiv.2309.05810 |
| ids.openalex | https://openalex.org/W4386721608 |
| fwci | |
| type | preprint |
| title | SHIFT3D: Synthesizing Hard Inputs For Tricking 3D Detectors |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10036 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9987999796867371 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Advanced Neural Network Applications |
| topics[1].id | https://openalex.org/T12111 |
| topics[1].field.id | https://openalex.org/fields/22 |
| topics[1].field.display_name | Engineering |
| topics[1].score | 0.9923999905586243 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2209 |
| topics[1].subfield.display_name | Industrial and Manufacturing Engineering |
| topics[1].display_name | Industrial Vision Systems and Defect Detection |
| topics[2].id | https://openalex.org/T11689 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9916999936103821 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Adversarial Robustness in Machine Learning |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C43521106 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7072852849960327 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q2165493 |
| concepts[0].display_name | Pipeline (software) |
| concepts[1].id | https://openalex.org/C2781238097 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7047172784805298 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q175026 |
| concepts[1].display_name | Object (grammar) |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.6866883635520935 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C94915269 |
| concepts[3].level | 2 |
| concepts[3].score | 0.680065393447876 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q1834857 |
| concepts[3].display_name | Detector |
| concepts[4].id | https://openalex.org/C202615002 |
| concepts[4].level | 2 |
| concepts[4].score | 0.61845463514328 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q783507 |
| concepts[4].display_name | Differentiable function |
| concepts[5].id | https://openalex.org/C14036430 |
| concepts[5].level | 2 |
| concepts[5].score | 0.5907547473907471 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q3736076 |
| concepts[5].display_name | Function (biology) |
| concepts[6].id | https://openalex.org/C154945302 |
| concepts[6].level | 1 |
| concepts[6].score | 0.4636372923851013 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[6].display_name | Artificial intelligence |
| concepts[7].id | https://openalex.org/C26760741 |
| concepts[7].level | 2 |
| concepts[7].score | 0.42110076546669006 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q160402 |
| concepts[7].display_name | Perception |
| concepts[8].id | https://openalex.org/C31972630 |
| concepts[8].level | 1 |
| concepts[8].score | 0.3877992331981659 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q844240 |
| concepts[8].display_name | Computer vision |
| concepts[9].id | https://openalex.org/C199360897 |
| concepts[9].level | 1 |
| concepts[9].score | 0.117632657289505 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[9].display_name | Programming language |
| concepts[10].id | https://openalex.org/C33923547 |
| concepts[10].level | 0 |
| concepts[10].score | 0.09078678488731384 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[10].display_name | Mathematics |
| concepts[11].id | https://openalex.org/C15744967 |
| concepts[11].level | 0 |
| concepts[11].score | 0.07195845246315002 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[11].display_name | Psychology |
| concepts[12].id | https://openalex.org/C78458016 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q840400 |
| concepts[12].display_name | Evolutionary biology |
| concepts[13].id | https://openalex.org/C76155785 |
| concepts[13].level | 1 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q418 |
| concepts[13].display_name | Telecommunications |
| concepts[14].id | https://openalex.org/C86803240 |
| concepts[14].level | 0 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q420 |
| concepts[14].display_name | Biology |
| concepts[15].id | https://openalex.org/C134306372 |
| concepts[15].level | 1 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q7754 |
| concepts[15].display_name | Mathematical analysis |
| concepts[16].id | https://openalex.org/C169760540 |
| concepts[16].level | 1 |
| concepts[16].score | 0.0 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q207011 |
| concepts[16].display_name | Neuroscience |
| keywords[0].id | https://openalex.org/keywords/pipeline |
| keywords[0].score | 0.7072852849960327 |
| keywords[0].display_name | Pipeline (software) |
| keywords[1].id | https://openalex.org/keywords/object |
| keywords[1].score | 0.7047172784805298 |
| keywords[1].display_name | Object (grammar) |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.6866883635520935 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/detector |
| keywords[3].score | 0.680065393447876 |
| keywords[3].display_name | Detector |
| keywords[4].id | https://openalex.org/keywords/differentiable-function |
| keywords[4].score | 0.61845463514328 |
| keywords[4].display_name | Differentiable function |
| keywords[5].id | https://openalex.org/keywords/function |
| keywords[5].score | 0.5907547473907471 |
| keywords[5].display_name | Function (biology) |
| keywords[6].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[6].score | 0.4636372923851013 |
| keywords[6].display_name | Artificial intelligence |
| keywords[7].id | https://openalex.org/keywords/perception |
| keywords[7].score | 0.42110076546669006 |
| keywords[7].display_name | Perception |
| keywords[8].id | https://openalex.org/keywords/computer-vision |
| keywords[8].score | 0.3877992331981659 |
| keywords[8].display_name | Computer vision |
| keywords[9].id | https://openalex.org/keywords/programming-language |
| keywords[9].score | 0.117632657289505 |
| keywords[9].display_name | Programming language |
| keywords[10].id | https://openalex.org/keywords/mathematics |
| keywords[10].score | 0.09078678488731384 |
| keywords[10].display_name | Mathematics |
| keywords[11].id | https://openalex.org/keywords/psychology |
| keywords[11].score | 0.07195845246315002 |
| keywords[11].display_name | Psychology |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2309.05810 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2309.05810 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2309.05810 |
| locations[1].id | doi:10.48550/arxiv.2309.05810 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2309.05810 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5062177930 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-7519-7341 |
| authorships[0].author.display_name | Hongge Chen |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Chen, Hongge |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5100351992 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-9843-6758 |
| authorships[1].author.display_name | Chen Zhao |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Chen, Zhao |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5067267836 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-9444-5577 |
| authorships[2].author.display_name | Gregory P. Meyer |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Meyer, Gregory P. |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5077824676 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Dennis Park |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Park, Dennis |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5103033393 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-5676-2387 |
| authorships[4].author.display_name | Carl Vondrick |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Vondrick, Carl |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5002740493 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-2465-0708 |
| authorships[5].author.display_name | Ashish Shrivastava |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Shrivastava, Ashish |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5079961895 |
| authorships[6].author.orcid | |
| authorships[6].author.display_name | Yuning Chai |
| authorships[6].author_position | last |
| authorships[6].raw_author_name | Chai, Yuning |
| authorships[6].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2309.05810 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | SHIFT3D: Synthesizing Hard Inputs For Tricking 3D Detectors |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10036 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9987999796867371 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Advanced Neural Network Applications |
| related_works | https://openalex.org/W4285277090, https://openalex.org/W4327738859, https://openalex.org/W2348722996, https://openalex.org/W2334570605, https://openalex.org/W3181683615, https://openalex.org/W4286826125, https://openalex.org/W1633485514, https://openalex.org/W1604739066, https://openalex.org/W2115878407, https://openalex.org/W1980454230 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2309.05810 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2309.05810 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2309.05810 |
| primary_location.id | pmh:oai:arXiv.org:2309.05810 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2309.05810 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2309.05810 |
| publication_date | 2023-09-11 |
| publication_year | 2023 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 3, 44, 65, 72, 90 |
| abstract_inverted_index.3D | 8, 17, 38, 66, 74, 102, 116 |
| abstract_inverted_index.By | 40 |
| abstract_inverted_index.In | 20 |
| abstract_inverted_index.We | 0 |
| abstract_inverted_index.by | 80 |
| abstract_inverted_index.in | 68, 108 |
| abstract_inverted_index.of | 37, 64, 111 |
| abstract_inverted_index.or | 62 |
| abstract_inverted_index.to | 16, 57, 70 |
| abstract_inverted_index.us | 56 |
| abstract_inverted_index.we | 49 |
| abstract_inverted_index.Our | 94 |
| abstract_inverted_index.aid | 107 |
| abstract_inverted_index.and | 105 |
| abstract_inverted_index.are | 11 |
| abstract_inverted_index.can | 31, 106 |
| abstract_inverted_index.for | 6, 100 |
| abstract_inverted_index.the | 60, 77, 85 |
| abstract_inverted_index.yet | 14, 88 |
| abstract_inverted_index.from | 84 |
| abstract_inverted_index.into | 34 |
| abstract_inverted_index.like | 23 |
| abstract_inverted_index.pose | 63 |
| abstract_inverted_index.show | 50 |
| abstract_inverted_index.such | 27 |
| abstract_inverted_index.that | 10, 51 |
| abstract_inverted_index.with | 43 |
| abstract_inverted_index.allow | 55 |
| abstract_inverted_index.error | 53 |
| abstract_inverted_index.modes | 99 |
| abstract_inverted_index.novel | 28 |
| abstract_inverted_index.offer | 32 |
| abstract_inverted_index.order | 69 |
| abstract_inverted_index.risks | 114, 121 |
| abstract_inverted_index.shape | 61 |
| abstract_inverted_index.these | 120 |
| abstract_inverted_index.(SDF), | 48 |
| abstract_inverted_index.become | 122 |
| abstract_inverted_index.before | 119 |
| abstract_inverted_index.deform | 59 |
| abstract_inverted_index.differ | 83 |
| abstract_inverted_index.modern | 101 |
| abstract_inverted_index.object | 18, 67, 87, 103 |
| abstract_inverted_index.retain | 89 |
| abstract_inverted_index.safety | 113 |
| abstract_inverted_index.shape. | 93 |
| abstract_inverted_index.shapes | 9 |
| abstract_inverted_index.signed | 45 |
| abstract_inverted_index.within | 115 |
| abstract_inverted_index.SHIFT3D | 81 |
| abstract_inverted_index.confuse | 71 |
| abstract_inverted_index.failure | 98 |
| abstract_inverted_index.insight | 33 |
| abstract_inverted_index.objects | 30, 42, 78 |
| abstract_inverted_index.present | 1 |
| abstract_inverted_index.signals | 54 |
| abstract_inverted_index.systems | 118 |
| abstract_inverted_index.unknown | 35 |
| abstract_inverted_index.SHIFT3D, | 2 |
| abstract_inverted_index.approach | 95 |
| abstract_inverted_index.baseline | 86 |
| abstract_inverted_index.critical | 123 |
| abstract_inverted_index.driving, | 25 |
| abstract_inverted_index.function | 47 |
| abstract_inverted_index.gradient | 52 |
| abstract_inverted_index.pipeline | 5 |
| abstract_inverted_index.provides | 96 |
| abstract_inverted_index.smoothly | 58 |
| abstract_inverted_index.detector. | 75 |
| abstract_inverted_index.discovery | 110 |
| abstract_inverted_index.distanced | 46 |
| abstract_inverted_index.failures. | 124 |
| abstract_inverted_index.generated | 79 |
| abstract_inverted_index.plausible | 13 |
| abstract_inverted_index.potential | 112 |
| abstract_inverted_index.autonomous | 24 |
| abstract_inverted_index.detectors, | 104 |
| abstract_inverted_index.detectors. | 19, 39 |
| abstract_inverted_index.downstream | 73 |
| abstract_inverted_index.generating | 7 |
| abstract_inverted_index.perception | 117 |
| abstract_inverted_index.physically | 82 |
| abstract_inverted_index.preemptive | 109 |
| abstract_inverted_index.challenging | 15, 29 |
| abstract_inverted_index.discovering | 26 |
| abstract_inverted_index.Importantly, | 76 |
| abstract_inverted_index.applications | 22 |
| abstract_inverted_index.recognizable | 92 |
| abstract_inverted_index.representing | 41 |
| abstract_inverted_index.semantically | 91 |
| abstract_inverted_index.structurally | 12 |
| abstract_inverted_index.interpretable | 97 |
| abstract_inverted_index.differentiable | 4 |
| abstract_inverted_index.safety-critical | 21 |
| abstract_inverted_index.vulnerabilities | 36 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 7 |
| citation_normalized_percentile |