Semantic Scene Completion via Integrating Instances and Scene in-the-Loop Article Swipe

MAKE IMPACT

Yingjie Cai , Xuesong Chen , Chao Zhang , Kwan-Yee Lin , Xiaogang Wang , Hongsheng Li · YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2104.03640

Semantic Scene Completion aims at reconstructing a complete 3D scene with precise voxel-wise semantics from a single-view depth or RGBD image. It is a crucial but challenging problem for indoor scene understanding. In this work, we present a novel framework named Scene-Instance-Scene Network (\textit{SISNet}), which takes advantages of both instance and scene level semantic information. Our method is capable of inferring fine-grained shape details as well as nearby objects whose semantic categories are easily mixed-up. The key insight is that we decouple the instances from a coarsely completed semantic scene instead of a raw input image to guide the reconstruction of instances and the overall scene. SISNet conducts iterative scene-to-instance (SI) and instance-to-scene (IS) semantic completion. Specifically, the SI is able to encode objects' surrounding context for effectively decoupling instances from the scene and each instance could be voxelized into higher resolution to capture finer details. With IS, fine-grained instance information can be integrated back into the 3D scene and thus leads to more accurate semantic scene completion. Utilizing such an iterative mechanism, the scene and instance completion benefits each other to achieve higher completion accuracy. Extensively experiments show that our proposed method consistently outperforms state-of-the-art methods on both real NYU, NYUCAD and synthetic SUNCG-RGBD datasets. The code and the supplementary material will be available at \url{https://github.com/yjcaimeow/SISNet}.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2104.03640

PDF

https://arxiv.org/pdf/2104.03640

OA Status

green

Cited By

7

References

43

Related Works

10

OpenAlex ID

https://openalex.org/W3145127101

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W3145127101

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2104.03640

Digital Object Identifier

Title

Semantic Scene Completion via Integrating Instances and Scene in-the-Loop

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2021

Year of publication

Publication date

2021-04-08

Full publication date if available

Authors

Yingjie Cai, Xuesong Chen, Chao Zhang, Kwan-Yee Lin, Xiaogang Wang, Hongsheng Li

List of authors in order

Landing page

https://arxiv.org/abs/2104.03640

Publisher landing page

PDF URL

https://arxiv.org/pdf/2104.03640

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2104.03640

Direct OA link when available

Concepts

Computer science, Artificial intelligence, Semantics (computer science), Computer vision, Code (set theory), Context (archaeology), Image (mathematics), ENCODE, Scene graph, Key (lock), Rendering (computer graphics), Paleontology, Chemistry, Biology, Gene, Biochemistry, Computer security, Set (abstract data type), Programming language

Top concepts (fields/topics) attached by OpenAlex

Cited by

7

Total citation count in OpenAlex

Citations by year (recent)

2025: 1, 2024: 3, 2023: 2, 2022: 1

Per-year citation counts (last 5 years)

References (count)

43

Number of works referenced by this work

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload