TGP: Two-modal occupancy prediction with 3D Gaussian and sparse points for 3D Environment Awareness Article Swipe

MAKE IMPACT

Chen Mu , Wenyu Chen , Mingchuan Yang , Yuan Zhang , Tao Han , Xinchi Li , Yunlong Li , Huaici Zhao · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2503.09941

3D semantic occupancy has rapidly become a research focus in the fields of robotics and autonomous driving environment perception due to its ability to provide more realistic geometric perception and its closer integration with downstream tasks. By performing occupancy prediction of the 3D space in the environment, the ability and robustness of scene understanding can be effectively improved. However, existing occupancy prediction tasks are primarily modeled using voxel or point cloud-based approaches: voxel-based network structures often suffer from the loss of spatial information due to the voxelization process, while point cloud-based methods, although better at retaining spatial location information, face limitations in representing volumetric structural details. To address this issue, we propose a dual-modal prediction method based on 3D Gaussian sets and sparse points, which balances both spatial location and volumetric structural information, achieving higher accuracy in semantic occupancy prediction. Specifically, our method adopts a Transformer-based architecture, taking 3D Gaussian sets, sparse points, and queries as inputs. Through the multi-layer structure of the Transformer, the enhanced queries and 3D Gaussian sets jointly contribute to the semantic occupancy prediction, and an adaptive fusion mechanism integrates the semantic outputs of both modalities to generate the final prediction results. Additionally, to further improve accuracy, we dynamically refine the point cloud at each layer, allowing for more precise location information during occupancy prediction. We conducted experiments on the Occ3DnuScenes dataset, and the experimental results demonstrate superior performance of the proposed method on IoU based metrics.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2503.09941

PDF

https://arxiv.org/pdf/2503.09941

OA Status

green

OpenAlex ID

https://openalex.org/W4416038072

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4416038072

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2503.09941

Digital Object Identifier

Title

TGP: Two-modal occupancy prediction with 3D Gaussian and sparse points for 3D Environment Awareness

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-03-13

Full publication date if available

Authors

Chen Mu, Wenyu Chen, Mingchuan Yang, Yuan Zhang, Tao Han, Xinchi Li, Yunlong Li, Huaici Zhao

List of authors in order

Landing page

https://arxiv.org/abs/2503.09941

Publisher landing page

PDF URL

https://arxiv.org/pdf/2503.09941

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2503.09941

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload