Single-view Neural Radiance Fields with Depth Teacher Article Swipe

PDF

Yu-Rui Chen , Chun Gu , Feihu Zhang , Li Zhang ·

YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2303.09952

Neural Radiance Fields (NeRF) have been proposed for photorealistic novel view rendering. However, it requires many different views of one scene for training. Moreover, it has poor generalizations to new scenes and requires retraining or fine-tuning on each scene. In this paper, we develop a new NeRF model for novel view synthesis using only a single image as input. We propose to combine the (coarse) planar rendering and the (fine) volume rendering to achieve higher rendering quality and better generalizations. We also design a depth teacher net that predicts dense pseudo depth maps to supervise the joint rendering mechanism and boost the learning of consistent 3D geometry. We evaluate our method on three challenging datasets. It outperforms state-of-the-art single-view NeRFs by achieving 5$\sim$20\% improvements in PSNR and reducing 20$\sim$50\% of the errors in the depth rendering. It also shows excellent generalization abilities to unseen data without the need to fine-tune on each new scene.

Related Topics

Rendering (Computer Graphics)

Computer Science

Radiance

Artificial Intelligence

Deep Learning

Worker Adjustment And Retraining Notification Act Of 1988

Concepts

Rendering (computer graphics) Computer science Radiance Artificial intelligence Image-based modeling and rendering Deep neural networks Retraining Computer vision Artificial neural network 3D rendering Computer graphics (images) Volume rendering Depth map Image (mathematics) Geology Remote sensing Business International trade

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2303.09952
PDF: https://arxiv.org/pdf/2303.09952
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4327992529

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4327992529

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2303.09952

Digital Object Identifier
Title: Single-view Neural Radiance Fields with Depth Teacher

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2023

Year of publication
Publication date: 2023-03-17

Full publication date if available
Authors: Yu-Rui Chen, Chun Gu, Feihu Zhang, Li Zhang

List of authors in order
Landing page: https://arxiv.org/abs/2303.09952

Publisher landing page
PDF URL: https://arxiv.org/pdf/2303.09952

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2303.09952

Direct OA link when available
Concepts: Rendering (computer graphics), Computer science, Radiance, Artificial intelligence, Image-based modeling and rendering, Deep neural networks, Retraining, Computer vision, Artificial neural network, 3D rendering, Computer graphics (images), Volume rendering, Depth map, Image (mathematics), Geology, Remote sensing, Business, International trade

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4327992529
doi	https://doi.org/10.48550/arxiv.2303.09952
ids.doi	https://doi.org/10.48550/arxiv.2303.09952
ids.openalex	https://openalex.org/W4327992529
fwci
type	preprint
title	Single-view Neural Radiance Fields with Depth Teacher
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10531
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9998999834060669
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1707
topics[0].subfield.display_name	Computer Vision and Pattern Recognition
topics[0].display_name	Advanced Vision and Imaging
topics[1].id	https://openalex.org/T10481
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9973000288009644
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1704
topics[1].subfield.display_name	Computer Graphics and Computer-Aided Design
topics[1].display_name	Computer Graphics and Visualization Techniques
topics[2].id	https://openalex.org/T10719
topics[2].field.id	https://openalex.org/fields/22
topics[2].field.display_name	Engineering
topics[2].score	0.9936000108718872
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/2206
topics[2].subfield.display_name	Computational Mechanics
topics[2].display_name	3D Shape Modeling and Analysis
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C205711294
concepts[0].level	2
concepts[0].score	0.8490224480628967
concepts[0].wikidata	https://www.wikidata.org/wiki/Q176953
concepts[0].display_name	Rendering (computer graphics)
concepts[1].id	https://openalex.org/C41008148
concepts[1].level	0
concepts[1].score	0.7528026103973389
concepts[1].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[1].display_name	Computer science
concepts[2].id	https://openalex.org/C23690007
concepts[2].level	2
concepts[2].score	0.7327061891555786
concepts[2].wikidata	https://www.wikidata.org/wiki/Q1411145
concepts[2].display_name	Radiance
concepts[3].id	https://openalex.org/C154945302
concepts[3].level	1
concepts[3].score	0.6120465397834778
concepts[3].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[3].display_name	Artificial intelligence
concepts[4].id	https://openalex.org/C44185422
concepts[4].level	3
concepts[4].score	0.52445387840271
concepts[4].wikidata	https://www.wikidata.org/wiki/Q6002064
concepts[4].display_name	Image-based modeling and rendering
concepts[5].id	https://openalex.org/C2984842247
concepts[5].level	3
concepts[5].score	0.485825777053833
concepts[5].wikidata	https://www.wikidata.org/wiki/Q197536
concepts[5].display_name	Deep neural networks
concepts[6].id	https://openalex.org/C2778712577
concepts[6].level	2
concepts[6].score	0.4743807911872864
concepts[6].wikidata	https://www.wikidata.org/wiki/Q3505966
concepts[6].display_name	Retraining
concepts[7].id	https://openalex.org/C31972630
concepts[7].level	1
concepts[7].score	0.4674661457538605
concepts[7].wikidata	https://www.wikidata.org/wiki/Q844240
concepts[7].display_name	Computer vision
concepts[8].id	https://openalex.org/C50644808
concepts[8].level	2
concepts[8].score	0.462988018989563
concepts[8].wikidata	https://www.wikidata.org/wiki/Q192776
concepts[8].display_name	Artificial neural network
concepts[9].id	https://openalex.org/C36816356
concepts[9].level	3
concepts[9].score	0.4518173038959503
concepts[9].wikidata	https://www.wikidata.org/wiki/Q16911860
concepts[9].display_name	3D rendering
concepts[10].id	https://openalex.org/C121684516
concepts[10].level	1
concepts[10].score	0.4284575879573822
concepts[10].wikidata	https://www.wikidata.org/wiki/Q7600677
concepts[10].display_name	Computer graphics (images)
concepts[11].id	https://openalex.org/C30769735
concepts[11].level	3
concepts[11].score	0.4253585934638977
concepts[11].wikidata	https://www.wikidata.org/wiki/Q2165951
concepts[11].display_name	Volume rendering
concepts[12].id	https://openalex.org/C141268832
concepts[12].level	3
concepts[12].score	0.4118196368217468
concepts[12].wikidata	https://www.wikidata.org/wiki/Q2940499
concepts[12].display_name	Depth map
concepts[13].id	https://openalex.org/C115961682
concepts[13].level	2
concepts[13].score	0.1944969892501831
concepts[13].wikidata	https://www.wikidata.org/wiki/Q860623
concepts[13].display_name	Image (mathematics)
concepts[14].id	https://openalex.org/C127313418
concepts[14].level	0
concepts[14].score	0.10616573691368103
concepts[14].wikidata	https://www.wikidata.org/wiki/Q1069
concepts[14].display_name	Geology
concepts[15].id	https://openalex.org/C62649853
concepts[15].level	1
concepts[15].score	0.07219627499580383
concepts[15].wikidata	https://www.wikidata.org/wiki/Q199687
concepts[15].display_name	Remote sensing
concepts[16].id	https://openalex.org/C144133560
concepts[16].level	0
concepts[16].score	0.0
concepts[16].wikidata	https://www.wikidata.org/wiki/Q4830453
concepts[16].display_name	Business
concepts[17].id	https://openalex.org/C155202549
concepts[17].level	1
concepts[17].score	0.0
concepts[17].wikidata	https://www.wikidata.org/wiki/Q178803
concepts[17].display_name	International trade
keywords[0].id	https://openalex.org/keywords/rendering
keywords[0].score	0.8490224480628967
keywords[0].display_name	Rendering (computer graphics)
keywords[1].id	https://openalex.org/keywords/computer-science
keywords[1].score	0.7528026103973389
keywords[1].display_name	Computer science
keywords[2].id	https://openalex.org/keywords/radiance
keywords[2].score	0.7327061891555786
keywords[2].display_name	Radiance
keywords[3].id	https://openalex.org/keywords/artificial-intelligence
keywords[3].score	0.6120465397834778
keywords[3].display_name	Artificial intelligence
keywords[4].id	https://openalex.org/keywords/image-based-modeling-and-rendering
keywords[4].score	0.52445387840271
keywords[4].display_name	Image-based modeling and rendering
keywords[5].id	https://openalex.org/keywords/deep-neural-networks
keywords[5].score	0.485825777053833
keywords[5].display_name	Deep neural networks
keywords[6].id	https://openalex.org/keywords/retraining
keywords[6].score	0.4743807911872864
keywords[6].display_name	Retraining
keywords[7].id	https://openalex.org/keywords/computer-vision
keywords[7].score	0.4674661457538605
keywords[7].display_name	Computer vision
keywords[8].id	https://openalex.org/keywords/artificial-neural-network
keywords[8].score	0.462988018989563
keywords[8].display_name	Artificial neural network
keywords[9].id	https://openalex.org/keywords/3d-rendering
keywords[9].score	0.4518173038959503
keywords[9].display_name	3D rendering
keywords[10].id	https://openalex.org/keywords/computer-graphics
keywords[10].score	0.4284575879573822
keywords[10].display_name	Computer graphics (images)
keywords[11].id	https://openalex.org/keywords/volume-rendering
keywords[11].score	0.4253585934638977
keywords[11].display_name	Volume rendering
keywords[12].id	https://openalex.org/keywords/depth-map
keywords[12].score	0.4118196368217468
keywords[12].display_name	Depth map
keywords[13].id	https://openalex.org/keywords/image
keywords[13].score	0.1944969892501831
keywords[13].display_name	Image (mathematics)
keywords[14].id	https://openalex.org/keywords/geology
keywords[14].score	0.10616573691368103
keywords[14].display_name	Geology
keywords[15].id	https://openalex.org/keywords/remote-sensing
keywords[15].score	0.07219627499580383
keywords[15].display_name	Remote sensing
language	en
locations[0].id	pmh:oai:arXiv.org:2303.09952
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2303.09952
locations[0].version	submittedVersion
locations[0].raw_type
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2303.09952
locations[1].id	doi:10.48550/arxiv.2303.09952
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2303.09952
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5049569903
authorships[0].author.orcid	https://orcid.org/0000-0002-1564-7750
authorships[0].author.display_name	Yu-Rui Chen
authorships[0].author_position	first
authorships[0].raw_author_name	Chen, Yurui
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5100537474
authorships[1].author.orcid
authorships[1].author.display_name	Chun Gu
authorships[1].author_position	middle
authorships[1].raw_author_name	Gu, Chun
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5101421861
authorships[2].author.orcid	https://orcid.org/0000-0003-0712-5722
authorships[2].author.display_name	Feihu Zhang
authorships[2].author_position	middle
authorships[2].raw_author_name	Zhang, Feihu
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5100425666
authorships[3].author.orcid	https://orcid.org/0000-0002-9208-1949
authorships[3].author.display_name	Li Zhang
authorships[3].author_position	last
authorships[3].raw_author_name	Zhang, Li
authorships[3].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2303.09952
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Single-view Neural Radiance Fields with Depth Teacher
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10531
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9998999834060669
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1707
primary_topic.subfield.display_name	Computer Vision and Pattern Recognition
primary_topic.display_name	Advanced Vision and Imaging
related_works	https://openalex.org/W2519546352, https://openalex.org/W2000818808, https://openalex.org/W4244795592, https://openalex.org/W2105109486, https://openalex.org/W2028900186, https://openalex.org/W2415504098, https://openalex.org/W2067132795, https://openalex.org/W2098484428, https://openalex.org/W2117511222, https://openalex.org/W2117760822
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2303.09952
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2303.09952
best_oa_location.version	submittedVersion
best_oa_location.raw_type
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2303.09952
primary_location.id	pmh:oai:arXiv.org:2303.09952
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2303.09952
primary_location.version	submittedVersion
primary_location.raw_type
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2303.09952
publication_date	2023-03-17
publication_year	2023
referenced_works_count	0
abstract_inverted_index.a	44, 54, 83
abstract_inverted_index.3D	105
abstract_inverted_index.In	39
abstract_inverted_index.It	115, 136
abstract_inverted_index.We	59, 80, 107
abstract_inverted_index.as	57
abstract_inverted_index.by	120
abstract_inverted_index.in	124, 132
abstract_inverted_index.it	13, 24
abstract_inverted_index.of	18, 103, 129
abstract_inverted_index.on	36, 111, 150
abstract_inverted_index.or	34
abstract_inverted_index.to	28, 61, 72, 93, 142, 148
abstract_inverted_index.we	42
abstract_inverted_index.and	31, 67, 77, 99, 126
abstract_inverted_index.for	7, 21, 48
abstract_inverted_index.has	25
abstract_inverted_index.net	86
abstract_inverted_index.new	29, 45, 152
abstract_inverted_index.one	19
abstract_inverted_index.our	109
abstract_inverted_index.the	63, 68, 95, 101, 130, 133, 146
abstract_inverted_index.NeRF	46
abstract_inverted_index.PSNR	125
abstract_inverted_index.also	81, 137
abstract_inverted_index.been	5
abstract_inverted_index.data	144
abstract_inverted_index.each	37, 151
abstract_inverted_index.have	4
abstract_inverted_index.many	15
abstract_inverted_index.maps	92
abstract_inverted_index.need	147
abstract_inverted_index.only	53
abstract_inverted_index.poor	26
abstract_inverted_index.that	87
abstract_inverted_index.this	40
abstract_inverted_index.view	10, 50
abstract_inverted_index.NeRFs	119
abstract_inverted_index.boost	100
abstract_inverted_index.dense	89
abstract_inverted_index.depth	84, 91, 134
abstract_inverted_index.image	56
abstract_inverted_index.joint	96
abstract_inverted_index.model	47
abstract_inverted_index.novel	9, 49
abstract_inverted_index.scene	20
abstract_inverted_index.shows	138
abstract_inverted_index.three	112
abstract_inverted_index.using	52
abstract_inverted_index.views	17
abstract_inverted_index.(NeRF)	3
abstract_inverted_index.(fine)	69
abstract_inverted_index.Fields	2
abstract_inverted_index.Neural	0
abstract_inverted_index.better	78
abstract_inverted_index.design	82
abstract_inverted_index.errors	131
abstract_inverted_index.higher	74
abstract_inverted_index.input.	58
abstract_inverted_index.method	110
abstract_inverted_index.paper,	41
abstract_inverted_index.planar	65
abstract_inverted_index.pseudo	90
abstract_inverted_index.scene.	38, 153
abstract_inverted_index.scenes	30
abstract_inverted_index.single	55
abstract_inverted_index.unseen	143
abstract_inverted_index.volume	70
abstract_inverted_index.achieve	73
abstract_inverted_index.combine	62
abstract_inverted_index.develop	43
abstract_inverted_index.propose	60
abstract_inverted_index.quality	76
abstract_inverted_index.teacher	85
abstract_inverted_index.without	145
abstract_inverted_index.(coarse)	64
abstract_inverted_index.However,	12
abstract_inverted_index.Radiance	1
abstract_inverted_index.evaluate	108
abstract_inverted_index.learning	102
abstract_inverted_index.predicts	88
abstract_inverted_index.proposed	6
abstract_inverted_index.reducing	127
abstract_inverted_index.requires	14, 32
abstract_inverted_index.Moreover,	23
abstract_inverted_index.abilities	141
abstract_inverted_index.achieving	121
abstract_inverted_index.datasets.	114
abstract_inverted_index.different	16
abstract_inverted_index.excellent	139
abstract_inverted_index.fine-tune	149
abstract_inverted_index.geometry.	106
abstract_inverted_index.mechanism	98
abstract_inverted_index.rendering	66, 71, 75, 97
abstract_inverted_index.supervise	94
abstract_inverted_index.synthesis	51
abstract_inverted_index.training.	22
abstract_inverted_index.consistent	104
abstract_inverted_index.rendering.	11, 135
abstract_inverted_index.retraining	33
abstract_inverted_index.5$\sim$20\%	122
abstract_inverted_index.challenging	113
abstract_inverted_index.fine-tuning	35
abstract_inverted_index.outperforms	116
abstract_inverted_index.single-view	118
abstract_inverted_index.20$\sim$50\%	128
abstract_inverted_index.improvements	123
abstract_inverted_index.generalization	140
abstract_inverted_index.photorealistic	8
abstract_inverted_index.generalizations	27
abstract_inverted_index.generalizations.	79
abstract_inverted_index.state-of-the-art	117
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	4
sustainable_development_goals[0].id	https://metadata.un.org/sdg/4
sustainable_development_goals[0].score	0.7599999904632568
sustainable_development_goals[0].display_name	Quality Education
citation_normalized_percentile