UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding Article Swipe
Yueming Xu
,
Jiahui Zhang
,
Ze Huang
,
Yu-Rui Chen
,
Yanpeng Zhou
,
Zhen‐Yu Chen
,
Yu-Jie Yuan
,
Ping Xia
,
Guowei Huang
,
Xinyue Cai
,
Zhongang Qi
,
Xingyue Quan
,
Jianye Hao
,
Hang Xu
,
Zhang Li
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2508.11952
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2508.11952
Despite the impressive progress on understanding and generating images shown by the recent unified architectures, the integration of 3D tasks remains challenging and largely unexplored. In this paper, we introduce UniUGG, the first unified understanding and generation framework for 3D modalities. Our unified framework employs an LLM to comprehend and decode sentences and 3D representations. At its core, we propose a spatial decoder leveraging a latent diffusion model to generate high-quality 3D representations. This allows for the generation and imagination of 3D scenes based on a reference image and an arbitrary view transformation, while remaining supports for spatial visual question answering (VQA) tasks. Additionally, we propose a geometric-semantic learning strategy to pretrain the vision encoder. This design jointly captures the input's semantic and geometric cues, enhancing both spatial understanding and generation. Extensive experimental results demonstrate the superiority of our method in visual representation, spatial understanding, and 3D generation. The source code will be released upon paper acceptance.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2508.11952
- https://arxiv.org/pdf/2508.11952
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4414459917
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4414459917Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2508.11952Digital Object Identifier
- Title
-
UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic EncodingWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-08-16Full publication date if available
- Authors
-
Yueming Xu, Jiahui Zhang, Ze Huang, Yu-Rui Chen, Yanpeng Zhou, Zhen‐Yu Chen, Yu-Jie Yuan, Ping Xia, Guowei Huang, Xinyue Cai, Zhongang Qi, Xingyue Quan, Jianye Hao, Hang Xu, Zhang LiList of authors in order
- Landing page
-
https://arxiv.org/abs/2508.11952Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2508.11952Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2508.11952Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4414459917 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2508.11952 |
| ids.doi | https://doi.org/10.48550/arxiv.2508.11952 |
| ids.openalex | https://openalex.org/W4414459917 |
| fwci | |
| type | preprint |
| title | UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T14339 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9987999796867371 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Image Processing and 3D Reconstruction |
| topics[1].id | https://openalex.org/T10719 |
| topics[1].field.id | https://openalex.org/fields/22 |
| topics[1].field.display_name | Engineering |
| topics[1].score | 0.996399998664856 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2206 |
| topics[1].subfield.display_name | Computational Mechanics |
| topics[1].display_name | 3D Shape Modeling and Analysis |
| topics[2].id | https://openalex.org/T10481 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9941999912261963 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1704 |
| topics[2].subfield.display_name | Computer Graphics and Computer-Aided Design |
| topics[2].display_name | Computer Graphics and Visualization Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2508.11952 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2508.11952 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2508.11952 |
| locations[1].id | doi:10.48550/arxiv.2508.11952 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2508.11952 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5100514477 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-5720-1175 |
| authorships[0].author.display_name | Yueming Xu |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Xu, Yueming |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5100383592 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-4949-6217 |
| authorships[1].author.display_name | Jiahui Zhang |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Zhang, Jiahui |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5089403097 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Ze Huang |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Huang, Ze |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5049569903 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-1564-7750 |
| authorships[3].author.display_name | Yu-Rui Chen |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Chen, Yurui |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5051703879 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Yanpeng Zhou |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Zhou, Yanpeng |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5100422927 |
| authorships[5].author.orcid | https://orcid.org/0000-0001-5615-1682 |
| authorships[5].author.display_name | Zhen‐Yu Chen |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Chen, Zhenyu |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5101893134 |
| authorships[6].author.orcid | https://orcid.org/0000-0001-5646-6211 |
| authorships[6].author.display_name | Yu-Jie Yuan |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Yuan, Yu-Jie |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5101719121 |
| authorships[7].author.orcid | https://orcid.org/0000-0002-5675-3293 |
| authorships[7].author.display_name | Ping Xia |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Xia, Pengxiang |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5101896195 |
| authorships[8].author.orcid | https://orcid.org/0000-0002-7442-3722 |
| authorships[8].author.display_name | Guowei Huang |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Huang, Guowei |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5043352291 |
| authorships[9].author.orcid | |
| authorships[9].author.display_name | Xinyue Cai |
| authorships[9].author_position | middle |
| authorships[9].raw_author_name | Cai, Xinyue |
| authorships[9].is_corresponding | False |
| authorships[10].author.id | https://openalex.org/A5101500719 |
| authorships[10].author.orcid | https://orcid.org/0000-0001-8298-4063 |
| authorships[10].author.display_name | Zhongang Qi |
| authorships[10].author_position | middle |
| authorships[10].raw_author_name | Qi, Zhongang |
| authorships[10].is_corresponding | False |
| authorships[11].author.id | https://openalex.org/A5113273845 |
| authorships[11].author.orcid | |
| authorships[11].author.display_name | Xingyue Quan |
| authorships[11].author_position | middle |
| authorships[11].raw_author_name | Quan, Xingyue |
| authorships[11].is_corresponding | False |
| authorships[12].author.id | https://openalex.org/A5072138293 |
| authorships[12].author.orcid | |
| authorships[12].author.display_name | Jianye Hao |
| authorships[12].author_position | middle |
| authorships[12].raw_author_name | Hao, Jianye |
| authorships[12].is_corresponding | False |
| authorships[13].author.id | https://openalex.org/A5101361782 |
| authorships[13].author.orcid | https://orcid.org/0009-0003-0132-4058 |
| authorships[13].author.display_name | Hang Xu |
| authorships[13].author_position | middle |
| authorships[13].raw_author_name | Xu, Hang |
| authorships[13].is_corresponding | False |
| authorships[14].author.id | https://openalex.org/A5100418954 |
| authorships[14].author.orcid | https://orcid.org/0000-0001-9242-0763 |
| authorships[14].author.display_name | Zhang Li |
| authorships[14].author_position | last |
| authorships[14].raw_author_name | Zhang, Li |
| authorships[14].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2508.11952 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T14339 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9987999796867371 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Image Processing and 3D Reconstruction |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2508.11952 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2508.11952 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2508.11952 |
| primary_location.id | pmh:oai:arXiv.org:2508.11952 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2508.11952 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2508.11952 |
| publication_date | 2025-08-16 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 60, 64, 85, 106 |
| abstract_inverted_index.3D | 18, 39, 53, 71, 81, 146 |
| abstract_inverted_index.At | 55 |
| abstract_inverted_index.In | 25 |
| abstract_inverted_index.an | 45, 89 |
| abstract_inverted_index.be | 152 |
| abstract_inverted_index.by | 10 |
| abstract_inverted_index.in | 140 |
| abstract_inverted_index.of | 17, 80, 137 |
| abstract_inverted_index.on | 4, 84 |
| abstract_inverted_index.to | 47, 68, 110 |
| abstract_inverted_index.we | 28, 58, 104 |
| abstract_inverted_index.LLM | 46 |
| abstract_inverted_index.Our | 41 |
| abstract_inverted_index.The | 148 |
| abstract_inverted_index.and | 6, 22, 35, 49, 52, 78, 88, 122, 129, 145 |
| abstract_inverted_index.for | 38, 75, 96 |
| abstract_inverted_index.its | 56 |
| abstract_inverted_index.our | 138 |
| abstract_inverted_index.the | 1, 11, 15, 31, 76, 112, 119, 135 |
| abstract_inverted_index.This | 73, 115 |
| abstract_inverted_index.both | 126 |
| abstract_inverted_index.code | 150 |
| abstract_inverted_index.this | 26 |
| abstract_inverted_index.upon | 154 |
| abstract_inverted_index.view | 91 |
| abstract_inverted_index.will | 151 |
| abstract_inverted_index.(VQA) | 101 |
| abstract_inverted_index.based | 83 |
| abstract_inverted_index.core, | 57 |
| abstract_inverted_index.cues, | 124 |
| abstract_inverted_index.first | 32 |
| abstract_inverted_index.image | 87 |
| abstract_inverted_index.model | 67 |
| abstract_inverted_index.paper | 155 |
| abstract_inverted_index.shown | 9 |
| abstract_inverted_index.tasks | 19 |
| abstract_inverted_index.while | 93 |
| abstract_inverted_index.allows | 74 |
| abstract_inverted_index.decode | 50 |
| abstract_inverted_index.design | 116 |
| abstract_inverted_index.images | 8 |
| abstract_inverted_index.latent | 65 |
| abstract_inverted_index.method | 139 |
| abstract_inverted_index.paper, | 27 |
| abstract_inverted_index.recent | 12 |
| abstract_inverted_index.scenes | 82 |
| abstract_inverted_index.source | 149 |
| abstract_inverted_index.tasks. | 102 |
| abstract_inverted_index.vision | 113 |
| abstract_inverted_index.visual | 98, 141 |
| abstract_inverted_index.Despite | 0 |
| abstract_inverted_index.UniUGG, | 30 |
| abstract_inverted_index.decoder | 62 |
| abstract_inverted_index.employs | 44 |
| abstract_inverted_index.input's | 120 |
| abstract_inverted_index.jointly | 117 |
| abstract_inverted_index.largely | 23 |
| abstract_inverted_index.propose | 59, 105 |
| abstract_inverted_index.remains | 20 |
| abstract_inverted_index.results | 133 |
| abstract_inverted_index.spatial | 61, 97, 127, 143 |
| abstract_inverted_index.unified | 13, 33, 42 |
| abstract_inverted_index.captures | 118 |
| abstract_inverted_index.encoder. | 114 |
| abstract_inverted_index.generate | 69 |
| abstract_inverted_index.learning | 108 |
| abstract_inverted_index.pretrain | 111 |
| abstract_inverted_index.progress | 3 |
| abstract_inverted_index.question | 99 |
| abstract_inverted_index.released | 153 |
| abstract_inverted_index.semantic | 121 |
| abstract_inverted_index.strategy | 109 |
| abstract_inverted_index.supports | 95 |
| abstract_inverted_index.Extensive | 131 |
| abstract_inverted_index.answering | 100 |
| abstract_inverted_index.arbitrary | 90 |
| abstract_inverted_index.diffusion | 66 |
| abstract_inverted_index.enhancing | 125 |
| abstract_inverted_index.framework | 37, 43 |
| abstract_inverted_index.geometric | 123 |
| abstract_inverted_index.introduce | 29 |
| abstract_inverted_index.reference | 86 |
| abstract_inverted_index.remaining | 94 |
| abstract_inverted_index.sentences | 51 |
| abstract_inverted_index.comprehend | 48 |
| abstract_inverted_index.generating | 7 |
| abstract_inverted_index.generation | 36, 77 |
| abstract_inverted_index.impressive | 2 |
| abstract_inverted_index.leveraging | 63 |
| abstract_inverted_index.acceptance. | 156 |
| abstract_inverted_index.challenging | 21 |
| abstract_inverted_index.demonstrate | 134 |
| abstract_inverted_index.generation. | 130, 147 |
| abstract_inverted_index.imagination | 79 |
| abstract_inverted_index.integration | 16 |
| abstract_inverted_index.modalities. | 40 |
| abstract_inverted_index.superiority | 136 |
| abstract_inverted_index.unexplored. | 24 |
| abstract_inverted_index.experimental | 132 |
| abstract_inverted_index.high-quality | 70 |
| abstract_inverted_index.Additionally, | 103 |
| abstract_inverted_index.understanding | 5, 34, 128 |
| abstract_inverted_index.architectures, | 14 |
| abstract_inverted_index.understanding, | 144 |
| abstract_inverted_index.representation, | 142 |
| abstract_inverted_index.transformation, | 92 |
| abstract_inverted_index.representations. | 54, 72 |
| abstract_inverted_index.geometric-semantic | 107 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 15 |
| citation_normalized_percentile |