BASED: Bundle-Adjusting Surgical Endoscopic Dynamic Video Reconstruction using Neural Radiance Fields Article Swipe
S. Saha
,
Sainan Liu
,
Shan Lin
,
Jingpei Lu
,
Michael C. Yip
·
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2309.15329
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2309.15329
Reconstruction of deformable scenes from endoscopic videos is important for many applications such as intraoperative navigation, surgical visual perception, and robotic surgery. It is a foundational requirement for realizing autonomous robotic interventions for minimally invasive surgery. However, previous approaches in this domain have been limited by their modular nature and are confined to specific camera and scene settings. Our work adopts the Neural Radiance Fields (NeRF) approach to learning 3D implicit representations of scenes that are both dynamic and deformable over time, and furthermore with unknown camera poses. We demonstrate this approach on endoscopic surgical scenes from robotic surgery. This work removes the constraints of known camera poses and overcomes the drawbacks of the state-of-the-art unstructured dynamic scene reconstruction technique, which relies on the static part of the scene for accurate reconstruction. Through several experimental datasets, we demonstrate the versatility of our proposed model to adapt to diverse camera and scene settings, and show its promise for both current and future robotic surgical systems.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2309.15329
- https://arxiv.org/pdf/2309.15329
- OA Status
- green
- Cited By
- 2
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4387156856
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4387156856Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2309.15329Digital Object Identifier
- Title
-
BASED: Bundle-Adjusting Surgical Endoscopic Dynamic Video Reconstruction using Neural Radiance FieldsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-09-27Full publication date if available
- Authors
-
S. Saha, Sainan Liu, Shan Lin, Jingpei Lu, Michael C. YipList of authors in order
- Landing page
-
https://arxiv.org/abs/2309.15329Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2309.15329Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2309.15329Direct OA link when available
- Concepts
-
Computer vision, Artificial intelligence, Computer science, Modular design, Bundle adjustment, Domain (mathematical analysis), Robot, Radiance, Image (mathematics), Mathematics, Optics, Physics, Mathematical analysis, Operating systemTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
2Total citation count in OpenAlex
- Citations by year (recent)
-
2024: 1, 2023: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4387156856 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2309.15329 |
| ids.doi | https://doi.org/10.48550/arxiv.2309.15329 |
| ids.openalex | https://openalex.org/W4387156856 |
| fwci | |
| type | preprint |
| title | BASED: Bundle-Adjusting Surgical Endoscopic Dynamic Video Reconstruction using Neural Radiance Fields |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10531 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9983000159263611 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Advanced Vision and Imaging |
| topics[1].id | https://openalex.org/T10191 |
| topics[1].field.id | https://openalex.org/fields/22 |
| topics[1].field.display_name | Engineering |
| topics[1].score | 0.9969000220298767 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2202 |
| topics[1].subfield.display_name | Aerospace Engineering |
| topics[1].display_name | Robotics and Sensor-Based Localization |
| topics[2].id | https://openalex.org/T10481 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9929999709129333 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1704 |
| topics[2].subfield.display_name | Computer Graphics and Computer-Aided Design |
| topics[2].display_name | Computer Graphics and Visualization Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C31972630 |
| concepts[0].level | 1 |
| concepts[0].score | 0.7533949017524719 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q844240 |
| concepts[0].display_name | Computer vision |
| concepts[1].id | https://openalex.org/C154945302 |
| concepts[1].level | 1 |
| concepts[1].score | 0.7531617879867554 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[1].display_name | Artificial intelligence |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.7123312950134277 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C101468663 |
| concepts[3].level | 2 |
| concepts[3].score | 0.6437517404556274 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q1620158 |
| concepts[3].display_name | Modular design |
| concepts[4].id | https://openalex.org/C179458375 |
| concepts[4].level | 3 |
| concepts[4].score | 0.5366306900978088 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q1020763 |
| concepts[4].display_name | Bundle adjustment |
| concepts[5].id | https://openalex.org/C36503486 |
| concepts[5].level | 2 |
| concepts[5].score | 0.459955096244812 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11235244 |
| concepts[5].display_name | Domain (mathematical analysis) |
| concepts[6].id | https://openalex.org/C90509273 |
| concepts[6].level | 2 |
| concepts[6].score | 0.431460440158844 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q11012 |
| concepts[6].display_name | Robot |
| concepts[7].id | https://openalex.org/C23690007 |
| concepts[7].level | 2 |
| concepts[7].score | 0.42581087350845337 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q1411145 |
| concepts[7].display_name | Radiance |
| concepts[8].id | https://openalex.org/C115961682 |
| concepts[8].level | 2 |
| concepts[8].score | 0.1823396384716034 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q860623 |
| concepts[8].display_name | Image (mathematics) |
| concepts[9].id | https://openalex.org/C33923547 |
| concepts[9].level | 0 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[9].display_name | Mathematics |
| concepts[10].id | https://openalex.org/C120665830 |
| concepts[10].level | 1 |
| concepts[10].score | 0.0 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q14620 |
| concepts[10].display_name | Optics |
| concepts[11].id | https://openalex.org/C121332964 |
| concepts[11].level | 0 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[11].display_name | Physics |
| concepts[12].id | https://openalex.org/C134306372 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q7754 |
| concepts[12].display_name | Mathematical analysis |
| concepts[13].id | https://openalex.org/C111919701 |
| concepts[13].level | 1 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q9135 |
| concepts[13].display_name | Operating system |
| keywords[0].id | https://openalex.org/keywords/computer-vision |
| keywords[0].score | 0.7533949017524719 |
| keywords[0].display_name | Computer vision |
| keywords[1].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[1].score | 0.7531617879867554 |
| keywords[1].display_name | Artificial intelligence |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.7123312950134277 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/modular-design |
| keywords[3].score | 0.6437517404556274 |
| keywords[3].display_name | Modular design |
| keywords[4].id | https://openalex.org/keywords/bundle-adjustment |
| keywords[4].score | 0.5366306900978088 |
| keywords[4].display_name | Bundle adjustment |
| keywords[5].id | https://openalex.org/keywords/domain |
| keywords[5].score | 0.459955096244812 |
| keywords[5].display_name | Domain (mathematical analysis) |
| keywords[6].id | https://openalex.org/keywords/robot |
| keywords[6].score | 0.431460440158844 |
| keywords[6].display_name | Robot |
| keywords[7].id | https://openalex.org/keywords/radiance |
| keywords[7].score | 0.42581087350845337 |
| keywords[7].display_name | Radiance |
| keywords[8].id | https://openalex.org/keywords/image |
| keywords[8].score | 0.1823396384716034 |
| keywords[8].display_name | Image (mathematics) |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2309.15329 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | cc-by-nc-sa |
| locations[0].pdf_url | https://arxiv.org/pdf/2309.15329 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | https://openalex.org/licenses/cc-by-nc-sa |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2309.15329 |
| locations[1].id | doi:10.48550/arxiv.2309.15329 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2309.15329 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5105267591 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-9296-1498 |
| authorships[0].author.display_name | S. Saha |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Saha, Shreya |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5101864952 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-6505-8196 |
| authorships[1].author.display_name | Sainan Liu |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Liu, Sainan |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5003166096 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-6362-2972 |
| authorships[2].author.display_name | Shan Lin |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Lin, Shan |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5081984167 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-9136-6096 |
| authorships[3].author.display_name | Jingpei Lu |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Lu, Jingpei |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5054598974 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-9689-0172 |
| authorships[4].author.display_name | Michael C. Yip |
| authorships[4].author_position | last |
| authorships[4].raw_author_name | Yip, Michael |
| authorships[4].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2309.15329 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | BASED: Bundle-Adjusting Surgical Endoscopic Dynamic Video Reconstruction using Neural Radiance Fields |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10531 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9983000159263611 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Advanced Vision and Imaging |
| related_works | https://openalex.org/W2896728493, https://openalex.org/W2392142157, https://openalex.org/W2043512367, https://openalex.org/W4321518006, https://openalex.org/W2331836163, https://openalex.org/W1994657804, https://openalex.org/W2005276308, https://openalex.org/W1970182911, https://openalex.org/W2024462231, https://openalex.org/W2347721387 |
| cited_by_count | 2 |
| counts_by_year[0].year | 2024 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2023 |
| counts_by_year[1].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2309.15329 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | cc-by-nc-sa |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2309.15329 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by-nc-sa |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2309.15329 |
| primary_location.id | pmh:oai:arXiv.org:2309.15329 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | cc-by-nc-sa |
| primary_location.pdf_url | https://arxiv.org/pdf/2309.15329 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | https://openalex.org/licenses/cc-by-nc-sa |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2309.15329 |
| publication_date | 2023-09-27 |
| publication_year | 2023 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 24 |
| abstract_inverted_index.3D | 69 |
| abstract_inverted_index.It | 22 |
| abstract_inverted_index.We | 88 |
| abstract_inverted_index.as | 13 |
| abstract_inverted_index.by | 45 |
| abstract_inverted_index.in | 39 |
| abstract_inverted_index.is | 7, 23 |
| abstract_inverted_index.of | 1, 72, 104, 112, 126, 140 |
| abstract_inverted_index.on | 92, 122 |
| abstract_inverted_index.to | 52, 67, 144, 146 |
| abstract_inverted_index.we | 136 |
| abstract_inverted_index.Our | 58 |
| abstract_inverted_index.and | 19, 49, 55, 78, 82, 108, 149, 152, 159 |
| abstract_inverted_index.are | 50, 75 |
| abstract_inverted_index.for | 9, 27, 32, 129, 156 |
| abstract_inverted_index.its | 154 |
| abstract_inverted_index.our | 141 |
| abstract_inverted_index.the | 61, 102, 110, 113, 123, 127, 138 |
| abstract_inverted_index.This | 99 |
| abstract_inverted_index.been | 43 |
| abstract_inverted_index.both | 76, 157 |
| abstract_inverted_index.from | 4, 96 |
| abstract_inverted_index.have | 42 |
| abstract_inverted_index.many | 10 |
| abstract_inverted_index.over | 80 |
| abstract_inverted_index.part | 125 |
| abstract_inverted_index.show | 153 |
| abstract_inverted_index.such | 12 |
| abstract_inverted_index.that | 74 |
| abstract_inverted_index.this | 40, 90 |
| abstract_inverted_index.with | 84 |
| abstract_inverted_index.work | 59, 100 |
| abstract_inverted_index.adapt | 145 |
| abstract_inverted_index.known | 105 |
| abstract_inverted_index.model | 143 |
| abstract_inverted_index.poses | 107 |
| abstract_inverted_index.scene | 56, 117, 128, 150 |
| abstract_inverted_index.their | 46 |
| abstract_inverted_index.time, | 81 |
| abstract_inverted_index.which | 120 |
| abstract_inverted_index.(NeRF) | 65 |
| abstract_inverted_index.Fields | 64 |
| abstract_inverted_index.Neural | 62 |
| abstract_inverted_index.adopts | 60 |
| abstract_inverted_index.camera | 54, 86, 106, 148 |
| abstract_inverted_index.domain | 41 |
| abstract_inverted_index.future | 160 |
| abstract_inverted_index.nature | 48 |
| abstract_inverted_index.poses. | 87 |
| abstract_inverted_index.relies | 121 |
| abstract_inverted_index.scenes | 3, 73, 95 |
| abstract_inverted_index.static | 124 |
| abstract_inverted_index.videos | 6 |
| abstract_inverted_index.visual | 17 |
| abstract_inverted_index.Through | 132 |
| abstract_inverted_index.current | 158 |
| abstract_inverted_index.diverse | 147 |
| abstract_inverted_index.dynamic | 77, 116 |
| abstract_inverted_index.limited | 44 |
| abstract_inverted_index.modular | 47 |
| abstract_inverted_index.promise | 155 |
| abstract_inverted_index.removes | 101 |
| abstract_inverted_index.robotic | 20, 30, 97, 161 |
| abstract_inverted_index.several | 133 |
| abstract_inverted_index.unknown | 85 |
| abstract_inverted_index.However, | 36 |
| abstract_inverted_index.Radiance | 63 |
| abstract_inverted_index.accurate | 130 |
| abstract_inverted_index.approach | 66, 91 |
| abstract_inverted_index.confined | 51 |
| abstract_inverted_index.implicit | 70 |
| abstract_inverted_index.invasive | 34 |
| abstract_inverted_index.learning | 68 |
| abstract_inverted_index.previous | 37 |
| abstract_inverted_index.proposed | 142 |
| abstract_inverted_index.specific | 53 |
| abstract_inverted_index.surgery. | 21, 35, 98 |
| abstract_inverted_index.surgical | 16, 94, 162 |
| abstract_inverted_index.systems. | 163 |
| abstract_inverted_index.datasets, | 135 |
| abstract_inverted_index.drawbacks | 111 |
| abstract_inverted_index.important | 8 |
| abstract_inverted_index.minimally | 33 |
| abstract_inverted_index.overcomes | 109 |
| abstract_inverted_index.realizing | 28 |
| abstract_inverted_index.settings, | 151 |
| abstract_inverted_index.settings. | 57 |
| abstract_inverted_index.approaches | 38 |
| abstract_inverted_index.autonomous | 29 |
| abstract_inverted_index.deformable | 2, 79 |
| abstract_inverted_index.endoscopic | 5, 93 |
| abstract_inverted_index.technique, | 119 |
| abstract_inverted_index.constraints | 103 |
| abstract_inverted_index.demonstrate | 89, 137 |
| abstract_inverted_index.furthermore | 83 |
| abstract_inverted_index.navigation, | 15 |
| abstract_inverted_index.perception, | 18 |
| abstract_inverted_index.requirement | 26 |
| abstract_inverted_index.versatility | 139 |
| abstract_inverted_index.applications | 11 |
| abstract_inverted_index.experimental | 134 |
| abstract_inverted_index.foundational | 25 |
| abstract_inverted_index.unstructured | 115 |
| abstract_inverted_index.interventions | 31 |
| abstract_inverted_index.Reconstruction | 0 |
| abstract_inverted_index.intraoperative | 14 |
| abstract_inverted_index.reconstruction | 118 |
| abstract_inverted_index.reconstruction. | 131 |
| abstract_inverted_index.representations | 71 |
| abstract_inverted_index.state-of-the-art | 114 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 5 |
| citation_normalized_percentile |