3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior Article Swipe

MAKE IMPACT

Xiaokang Chen , Kwan-Yee Lin , Qian Chen , Gang Zeng , Hongsheng Li · YOU? · · 2020 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2003.14052

The goal of the Semantic Scene Completion (SSC) task is to simultaneously predict a completed 3D voxel representation of volumetric occupancy and semantic labels of objects in the scene from a single-view observation. Since the computational cost generally increases explosively along with the growth of voxel resolution, most current state-of-the-arts have to tailor their framework into a low-resolution representation with the sacrifice of detail prediction. Thus, voxel resolution becomes one of the crucial difficulties that lead to the performance bottleneck. In this paper, we propose to devise a new geometry-based strategy to embed depth information with low-resolution voxel representation, which could still be able to encode sufficient geometric information, e.g., room layout, object's sizes and shapes, to infer the invisible areas of the scene with well structure-preserving details. To this end, we first propose a novel 3D sketch-aware feature embedding to explicitly encode geometric information effectively and efficiently. With the 3D sketch in hand, we further devise a simple yet effective semantic scene completion framework that incorporates a light-weight 3D Sketch Hallucination module to guide the inference of occupancy and the semantic labels via a semi-supervised structure prior learning strategy. We demonstrate that our proposed geometric embedding works better than the depth feature learning from habitual SSC frameworks. Our final model surpasses state-of-the-arts consistently on three public benchmarks, which only requires 3D volumes of 60 x 36 x 60 resolution for both input and output. The code and the supplementary material will be available at https://charlesCXK.github.io.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2003.14052

PDF

https://arxiv.org/pdf/2003.14052

OA Status

green

Cited By

13

References

46

Related Works

10

OpenAlex ID

https://openalex.org/W3014459315

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W3014459315

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2003.14052

Digital Object Identifier

Title

3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2020

Year of publication

Publication date

2020-03-31

Full publication date if available

Authors

Xiaokang Chen, Kwan-Yee Lin, Qian Chen, Gang Zeng, Hongsheng Li

List of authors in order

Landing page

https://arxiv.org/abs/2003.14052

Publisher landing page

PDF URL

https://arxiv.org/pdf/2003.14052

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2003.14052

Direct OA link when available

Concepts

Computer science, Artificial intelligence, Feature (linguistics), Sketch, Representation (politics), Embedding, Voxel, Inference, Object (grammar), Computer vision, Feature learning, Pattern recognition (psychology), Natural language processing, Algorithm, Politics, Philosophy, Law, Political science, Linguistics

Top concepts (fields/topics) attached by OpenAlex

Cited by

13

Total citation count in OpenAlex

Citations by year (recent)

2025: 1, 2024: 3, 2023: 3, 2022: 3, 2021: 3

Per-year citation counts (last 5 years)

References (count)

46

Number of works referenced by this work

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload