Fast Sparse View Guided NeRF Update for Object Reconfigurations Article Swipe
Ziqi Lu
,
Jianbo Ye
,
Xiaohan Fei
,
Xiaolong Li
,
Jiawei Mo
,
Ashwin Swaminathan
,
Stefano Soatto
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2403.11024
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2403.11024
Neural Radiance Field (NeRF), as an implicit 3D scene representation, lacks inherent ability to accommodate changes made to the initial static scene. If objects are reconfigured, it is difficult to update the NeRF to reflect the new state of the scene without time-consuming data re-capturing and NeRF re-training. To address this limitation, we develop the first update method for NeRFs to physical changes. Our method takes only sparse new images (e.g. 4) of the altered scene as extra inputs and update the pre-trained NeRF in around 1 to 2 minutes. Particularly, we develop a pipeline to identify scene changes and update the NeRF accordingly. Our core idea is the use of a second helper NeRF to learn the local geometry and appearance changes, which sidesteps the optimization difficulties in direct NeRF fine-tuning. The interpolation power of the helper NeRF is the key to accurately reconstruct the un-occluded objects regions under sparse view supervision. Our method imposes no constraints on NeRF pre-training, and requires no extra user input or explicit semantic priors. It is an order of magnitude faster than re-training NeRF from scratch while maintaining on-par and even superior performance.
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2403.11024
- https://arxiv.org/pdf/2403.11024
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4393023425
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4393023425Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2403.11024Digital Object Identifier
- Title
-
Fast Sparse View Guided NeRF Update for Object ReconfigurationsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-03-16Full publication date if available
- Authors
-
Ziqi Lu, Jianbo Ye, Xiaohan Fei, Xiaolong Li, Jiawei Mo, Ashwin Swaminathan, Stefano SoattoList of authors in order
- Landing page
-
https://arxiv.org/abs/2403.11024Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2403.11024Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2403.11024Direct OA link when available
- Concepts
-
Computer science, Object (grammar), Artificial intelligenceTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4393023425 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2403.11024 |
| ids.doi | https://doi.org/10.48550/arxiv.2403.11024 |
| ids.openalex | https://openalex.org/W4393023425 |
| fwci | |
| type | preprint |
| title | Fast Sparse View Guided NeRF Update for Object Reconfigurations |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10036 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9574999809265137 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Advanced Neural Network Applications |
| topics[1].id | https://openalex.org/T10054 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9538999795913696 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1708 |
| topics[1].subfield.display_name | Hardware and Architecture |
| topics[1].display_name | Parallel Computing and Optimization Techniques |
| topics[2].id | https://openalex.org/T10522 |
| topics[2].field.id | https://openalex.org/fields/27 |
| topics[2].field.display_name | Medicine |
| topics[2].score | 0.927299976348877 |
| topics[2].domain.id | https://openalex.org/domains/4 |
| topics[2].domain.display_name | Health Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2741 |
| topics[2].subfield.display_name | Radiology, Nuclear Medicine and Imaging |
| topics[2].display_name | Medical Imaging Techniques and Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.7009350061416626 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C2781238097 |
| concepts[1].level | 2 |
| concepts[1].score | 0.5590269565582275 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q175026 |
| concepts[1].display_name | Object (grammar) |
| concepts[2].id | https://openalex.org/C154945302 |
| concepts[2].level | 1 |
| concepts[2].score | 0.3128805160522461 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[2].display_name | Artificial intelligence |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.7009350061416626 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/object |
| keywords[1].score | 0.5590269565582275 |
| keywords[1].display_name | Object (grammar) |
| keywords[2].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[2].score | 0.3128805160522461 |
| keywords[2].display_name | Artificial intelligence |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2403.11024 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2403.11024 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2403.11024 |
| locations[1].id | doi:10.48550/arxiv.2403.11024 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2403.11024 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5100313072 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Ziqi Lu |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Lu, Ziqi |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5031373536 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-4612-6429 |
| authorships[1].author.display_name | Jianbo Ye |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Ye, Jianbo |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5087030407 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-1030-2286 |
| authorships[2].author.display_name | Xiaohan Fei |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Fei, Xiaohan |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5100371535 |
| authorships[3].author.orcid | https://orcid.org/0000-0001-7493-2650 |
| authorships[3].author.display_name | Xiaolong Li |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Li, Xiaolong |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5087799311 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-1043-5353 |
| authorships[4].author.display_name | Jiawei Mo |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Mo, Jiawei |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5084355679 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-4279-369X |
| authorships[5].author.display_name | Ashwin Swaminathan |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Swaminathan, Ashwin |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5038328783 |
| authorships[6].author.orcid | https://orcid.org/0000-0003-2902-6362 |
| authorships[6].author.display_name | Stefano Soatto |
| authorships[6].author_position | last |
| authorships[6].raw_author_name | Soatto, Stefano |
| authorships[6].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2403.11024 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Fast Sparse View Guided NeRF Update for Object Reconfigurations |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10036 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9574999809265137 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Advanced Neural Network Applications |
| related_works | https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W2382290278, https://openalex.org/W2478288626, https://openalex.org/W4391913857, https://openalex.org/W2350741829, https://openalex.org/W2975200075 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2403.11024 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2403.11024 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2403.11024 |
| primary_location.id | pmh:oai:arXiv.org:2403.11024 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2403.11024 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2403.11024 |
| publication_date | 2024-03-16 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.1 | 86 |
| abstract_inverted_index.2 | 88 |
| abstract_inverted_index.a | 93, 111 |
| abstract_inverted_index.3D | 7 |
| abstract_inverted_index.4) | 71 |
| abstract_inverted_index.If | 22 |
| abstract_inverted_index.It | 171 |
| abstract_inverted_index.To | 48 |
| abstract_inverted_index.an | 5, 173 |
| abstract_inverted_index.as | 4, 76 |
| abstract_inverted_index.in | 84, 128 |
| abstract_inverted_index.is | 27, 107, 139, 172 |
| abstract_inverted_index.it | 26 |
| abstract_inverted_index.no | 156, 163 |
| abstract_inverted_index.of | 38, 72, 110, 135, 175 |
| abstract_inverted_index.on | 158 |
| abstract_inverted_index.or | 167 |
| abstract_inverted_index.to | 13, 17, 29, 33, 60, 87, 95, 115, 142 |
| abstract_inverted_index.we | 52, 91 |
| abstract_inverted_index.Our | 63, 104, 153 |
| abstract_inverted_index.The | 132 |
| abstract_inverted_index.and | 45, 79, 99, 120, 161, 186 |
| abstract_inverted_index.are | 24 |
| abstract_inverted_index.for | 58 |
| abstract_inverted_index.key | 141 |
| abstract_inverted_index.new | 36, 68 |
| abstract_inverted_index.the | 18, 31, 35, 39, 54, 73, 81, 101, 108, 117, 125, 136, 140, 145 |
| abstract_inverted_index.use | 109 |
| abstract_inverted_index.NeRF | 32, 46, 83, 102, 114, 130, 138, 159, 180 |
| abstract_inverted_index.core | 105 |
| abstract_inverted_index.data | 43 |
| abstract_inverted_index.even | 187 |
| abstract_inverted_index.from | 181 |
| abstract_inverted_index.idea | 106 |
| abstract_inverted_index.made | 16 |
| abstract_inverted_index.only | 66 |
| abstract_inverted_index.than | 178 |
| abstract_inverted_index.this | 50 |
| abstract_inverted_index.user | 165 |
| abstract_inverted_index.view | 151 |
| abstract_inverted_index.(e.g. | 70 |
| abstract_inverted_index.Field | 2 |
| abstract_inverted_index.NeRFs | 59 |
| abstract_inverted_index.extra | 77, 164 |
| abstract_inverted_index.first | 55 |
| abstract_inverted_index.input | 166 |
| abstract_inverted_index.lacks | 10 |
| abstract_inverted_index.learn | 116 |
| abstract_inverted_index.local | 118 |
| abstract_inverted_index.order | 174 |
| abstract_inverted_index.power | 134 |
| abstract_inverted_index.scene | 8, 40, 75, 97 |
| abstract_inverted_index.state | 37 |
| abstract_inverted_index.takes | 65 |
| abstract_inverted_index.under | 149 |
| abstract_inverted_index.which | 123 |
| abstract_inverted_index.while | 183 |
| abstract_inverted_index.Neural | 0 |
| abstract_inverted_index.around | 85 |
| abstract_inverted_index.direct | 129 |
| abstract_inverted_index.faster | 177 |
| abstract_inverted_index.helper | 113, 137 |
| abstract_inverted_index.images | 69 |
| abstract_inverted_index.inputs | 78 |
| abstract_inverted_index.method | 57, 64, 154 |
| abstract_inverted_index.on-par | 185 |
| abstract_inverted_index.scene. | 21 |
| abstract_inverted_index.second | 112 |
| abstract_inverted_index.sparse | 67, 150 |
| abstract_inverted_index.static | 20 |
| abstract_inverted_index.update | 30, 56, 80, 100 |
| abstract_inverted_index.(NeRF), | 3 |
| abstract_inverted_index.ability | 12 |
| abstract_inverted_index.address | 49 |
| abstract_inverted_index.altered | 74 |
| abstract_inverted_index.changes | 15, 98 |
| abstract_inverted_index.develop | 53, 92 |
| abstract_inverted_index.imposes | 155 |
| abstract_inverted_index.initial | 19 |
| abstract_inverted_index.objects | 23, 147 |
| abstract_inverted_index.priors. | 170 |
| abstract_inverted_index.reflect | 34 |
| abstract_inverted_index.regions | 148 |
| abstract_inverted_index.scratch | 182 |
| abstract_inverted_index.without | 41 |
| abstract_inverted_index.Radiance | 1 |
| abstract_inverted_index.changes, | 122 |
| abstract_inverted_index.changes. | 62 |
| abstract_inverted_index.explicit | 168 |
| abstract_inverted_index.geometry | 119 |
| abstract_inverted_index.identify | 96 |
| abstract_inverted_index.implicit | 6 |
| abstract_inverted_index.inherent | 11 |
| abstract_inverted_index.minutes. | 89 |
| abstract_inverted_index.physical | 61 |
| abstract_inverted_index.pipeline | 94 |
| abstract_inverted_index.requires | 162 |
| abstract_inverted_index.semantic | 169 |
| abstract_inverted_index.superior | 188 |
| abstract_inverted_index.difficult | 28 |
| abstract_inverted_index.magnitude | 176 |
| abstract_inverted_index.sidesteps | 124 |
| abstract_inverted_index.accurately | 143 |
| abstract_inverted_index.appearance | 121 |
| abstract_inverted_index.accommodate | 14 |
| abstract_inverted_index.constraints | 157 |
| abstract_inverted_index.limitation, | 51 |
| abstract_inverted_index.maintaining | 184 |
| abstract_inverted_index.pre-trained | 82 |
| abstract_inverted_index.re-training | 179 |
| abstract_inverted_index.reconstruct | 144 |
| abstract_inverted_index.un-occluded | 146 |
| abstract_inverted_index.accordingly. | 103 |
| abstract_inverted_index.difficulties | 127 |
| abstract_inverted_index.fine-tuning. | 131 |
| abstract_inverted_index.optimization | 126 |
| abstract_inverted_index.performance. | 189 |
| abstract_inverted_index.re-capturing | 44 |
| abstract_inverted_index.re-training. | 47 |
| abstract_inverted_index.supervision. | 152 |
| abstract_inverted_index.Particularly, | 90 |
| abstract_inverted_index.interpolation | 133 |
| abstract_inverted_index.pre-training, | 160 |
| abstract_inverted_index.reconfigured, | 25 |
| abstract_inverted_index.time-consuming | 42 |
| abstract_inverted_index.representation, | 9 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 7 |
| citation_normalized_percentile |