YouDream: Generating Anatomically Controllable Consistent Text-to-3D Animals Article Swipe
Sandeep Mishra
,
Oindrila Saha
,
Alan C. Bovik
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2406.16273
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2406.16273
3D generation guided by text-to-image diffusion models enables the creation of visually compelling assets. However previous methods explore generation based on image or text. The boundaries of creativity are limited by what can be expressed through words or the images that can be sourced. We present YouDream, a method to generate high-quality anatomically controllable animals. YouDream is guided using a text-to-image diffusion model controlled by 2D views of a 3D pose prior. Our method generates 3D animals that are not possible to create using previous text-to-3D generative methods. Additionally, our method is capable of preserving anatomic consistency in the generated animals, an area where prior text-to-3D approaches often struggle. Moreover, we design a fully automated pipeline for generating commonly found animals. To circumvent the need for human intervention to create a 3D pose, we propose a multi-agent LLM that adapts poses from a limited library of animal 3D poses to represent the desired animal. A user study conducted on the outcomes of YouDream demonstrates the preference of the animal models generated by our method over others. Turntable results and code are released at https://youdream3d.github.io/
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2406.16273
- https://arxiv.org/pdf/2406.16273
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4400023746
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4400023746Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2406.16273Digital Object Identifier
- Title
-
YouDream: Generating Anatomically Controllable Consistent Text-to-3D AnimalsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-06-24Full publication date if available
- Authors
-
Sandeep Mishra, Oindrila Saha, Alan C. BovikList of authors in order
- Landing page
-
https://arxiv.org/abs/2406.16273Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2406.16273Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2406.16273Direct OA link when available
- Concepts
-
Computer science, Biology, Natural language processing, Evolutionary biology, AnatomyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4400023746 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2406.16273 |
| ids.doi | https://doi.org/10.48550/arxiv.2406.16273 |
| ids.openalex | https://openalex.org/W4400023746 |
| fwci | |
| type | preprint |
| title | YouDream: Generating Anatomically Controllable Consistent Text-to-3D Animals |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T14339 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9599000215530396 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Image Processing and 3D Reconstruction |
| topics[1].id | https://openalex.org/T12290 |
| topics[1].field.id | https://openalex.org/fields/22 |
| topics[1].field.display_name | Engineering |
| topics[1].score | 0.9358999729156494 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2207 |
| topics[1].subfield.display_name | Control and Systems Engineering |
| topics[1].display_name | Human Motion and Animation |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.4390435516834259 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C86803240 |
| concepts[1].level | 0 |
| concepts[1].score | 0.34647977352142334 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q420 |
| concepts[1].display_name | Biology |
| concepts[2].id | https://openalex.org/C204321447 |
| concepts[2].level | 1 |
| concepts[2].score | 0.3443741202354431 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[2].display_name | Natural language processing |
| concepts[3].id | https://openalex.org/C78458016 |
| concepts[3].level | 1 |
| concepts[3].score | 0.32578781247138977 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q840400 |
| concepts[3].display_name | Evolutionary biology |
| concepts[4].id | https://openalex.org/C105702510 |
| concepts[4].level | 1 |
| concepts[4].score | 0.3241545557975769 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q514 |
| concepts[4].display_name | Anatomy |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.4390435516834259 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/biology |
| keywords[1].score | 0.34647977352142334 |
| keywords[1].display_name | Biology |
| keywords[2].id | https://openalex.org/keywords/natural-language-processing |
| keywords[2].score | 0.3443741202354431 |
| keywords[2].display_name | Natural language processing |
| keywords[3].id | https://openalex.org/keywords/evolutionary-biology |
| keywords[3].score | 0.32578781247138977 |
| keywords[3].display_name | Evolutionary biology |
| keywords[4].id | https://openalex.org/keywords/anatomy |
| keywords[4].score | 0.3241545557975769 |
| keywords[4].display_name | Anatomy |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2406.16273 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2406.16273 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2406.16273 |
| locations[1].id | doi:10.48550/arxiv.2406.16273 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2406.16273 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5078490593 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-5893-9243 |
| authorships[0].author.display_name | Sandeep Mishra |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Mishra, Sandeep |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5036639026 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Oindrila Saha |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Saha, Oindrila |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5075463806 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-6067-710X |
| authorships[2].author.display_name | Alan C. Bovik |
| authorships[2].author_position | last |
| authorships[2].raw_author_name | Bovik, Alan C. |
| authorships[2].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2406.16273 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-06-26T00:00:00 |
| display_name | YouDream: Generating Anatomically Controllable Consistent Text-to-3D Animals |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T14339 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9599000215530396 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Image Processing and 3D Reconstruction |
| related_works | https://openalex.org/W2748952813, https://openalex.org/W4391375266, https://openalex.org/W2082860237, https://openalex.org/W2119695867, https://openalex.org/W2130076355, https://openalex.org/W1990804418, https://openalex.org/W1993764875, https://openalex.org/W2046158694, https://openalex.org/W2788277189, https://openalex.org/W2061542922 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2406.16273 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2406.16273 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2406.16273 |
| primary_location.id | pmh:oai:arXiv.org:2406.16273 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2406.16273 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2406.16273 |
| publication_date | 2024-06-24 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.A | 154 |
| abstract_inverted_index.a | 47, 59, 68, 112, 130, 135, 142 |
| abstract_inverted_index.2D | 65 |
| abstract_inverted_index.3D | 0, 69, 75, 131, 147 |
| abstract_inverted_index.To | 121 |
| abstract_inverted_index.We | 44 |
| abstract_inverted_index.an | 101 |
| abstract_inverted_index.at | 182 |
| abstract_inverted_index.be | 33, 42 |
| abstract_inverted_index.by | 3, 30, 64, 171 |
| abstract_inverted_index.in | 97 |
| abstract_inverted_index.is | 56, 91 |
| abstract_inverted_index.of | 10, 26, 67, 93, 145, 161, 166 |
| abstract_inverted_index.on | 20, 158 |
| abstract_inverted_index.or | 22, 37 |
| abstract_inverted_index.to | 49, 81, 128, 149 |
| abstract_inverted_index.we | 110, 133 |
| abstract_inverted_index.LLM | 137 |
| abstract_inverted_index.Our | 72 |
| abstract_inverted_index.The | 24 |
| abstract_inverted_index.and | 178 |
| abstract_inverted_index.are | 28, 78, 180 |
| abstract_inverted_index.can | 32, 41 |
| abstract_inverted_index.for | 116, 125 |
| abstract_inverted_index.not | 79 |
| abstract_inverted_index.our | 89, 172 |
| abstract_inverted_index.the | 8, 38, 98, 123, 151, 159, 164, 167 |
| abstract_inverted_index.area | 102 |
| abstract_inverted_index.code | 179 |
| abstract_inverted_index.from | 141 |
| abstract_inverted_index.need | 124 |
| abstract_inverted_index.over | 174 |
| abstract_inverted_index.pose | 70 |
| abstract_inverted_index.that | 40, 77, 138 |
| abstract_inverted_index.user | 155 |
| abstract_inverted_index.what | 31 |
| abstract_inverted_index.based | 19 |
| abstract_inverted_index.found | 119 |
| abstract_inverted_index.fully | 113 |
| abstract_inverted_index.human | 126 |
| abstract_inverted_index.image | 21 |
| abstract_inverted_index.model | 62 |
| abstract_inverted_index.often | 107 |
| abstract_inverted_index.pose, | 132 |
| abstract_inverted_index.poses | 140, 148 |
| abstract_inverted_index.prior | 104 |
| abstract_inverted_index.study | 156 |
| abstract_inverted_index.text. | 23 |
| abstract_inverted_index.using | 58, 83 |
| abstract_inverted_index.views | 66 |
| abstract_inverted_index.where | 103 |
| abstract_inverted_index.words | 36 |
| abstract_inverted_index.adapts | 139 |
| abstract_inverted_index.animal | 146, 168 |
| abstract_inverted_index.create | 82, 129 |
| abstract_inverted_index.design | 111 |
| abstract_inverted_index.guided | 2, 57 |
| abstract_inverted_index.images | 39 |
| abstract_inverted_index.method | 48, 73, 90, 173 |
| abstract_inverted_index.models | 6, 169 |
| abstract_inverted_index.prior. | 71 |
| abstract_inverted_index.However | 14 |
| abstract_inverted_index.animal. | 153 |
| abstract_inverted_index.animals | 76 |
| abstract_inverted_index.assets. | 13 |
| abstract_inverted_index.capable | 92 |
| abstract_inverted_index.desired | 152 |
| abstract_inverted_index.enables | 7 |
| abstract_inverted_index.explore | 17 |
| abstract_inverted_index.library | 144 |
| abstract_inverted_index.limited | 29, 143 |
| abstract_inverted_index.methods | 16 |
| abstract_inverted_index.others. | 175 |
| abstract_inverted_index.present | 45 |
| abstract_inverted_index.propose | 134 |
| abstract_inverted_index.results | 177 |
| abstract_inverted_index.through | 35 |
| abstract_inverted_index.YouDream | 55, 162 |
| abstract_inverted_index.anatomic | 95 |
| abstract_inverted_index.animals, | 100 |
| abstract_inverted_index.animals. | 54, 120 |
| abstract_inverted_index.commonly | 118 |
| abstract_inverted_index.creation | 9 |
| abstract_inverted_index.generate | 50 |
| abstract_inverted_index.methods. | 87 |
| abstract_inverted_index.outcomes | 160 |
| abstract_inverted_index.pipeline | 115 |
| abstract_inverted_index.possible | 80 |
| abstract_inverted_index.previous | 15, 84 |
| abstract_inverted_index.released | 181 |
| abstract_inverted_index.sourced. | 43 |
| abstract_inverted_index.visually | 11 |
| abstract_inverted_index.Moreover, | 109 |
| abstract_inverted_index.Turntable | 176 |
| abstract_inverted_index.YouDream, | 46 |
| abstract_inverted_index.automated | 114 |
| abstract_inverted_index.conducted | 157 |
| abstract_inverted_index.diffusion | 5, 61 |
| abstract_inverted_index.expressed | 34 |
| abstract_inverted_index.generated | 99, 170 |
| abstract_inverted_index.generates | 74 |
| abstract_inverted_index.represent | 150 |
| abstract_inverted_index.struggle. | 108 |
| abstract_inverted_index.approaches | 106 |
| abstract_inverted_index.boundaries | 25 |
| abstract_inverted_index.circumvent | 122 |
| abstract_inverted_index.compelling | 12 |
| abstract_inverted_index.controlled | 63 |
| abstract_inverted_index.creativity | 27 |
| abstract_inverted_index.generating | 117 |
| abstract_inverted_index.generation | 1, 18 |
| abstract_inverted_index.generative | 86 |
| abstract_inverted_index.preference | 165 |
| abstract_inverted_index.preserving | 94 |
| abstract_inverted_index.text-to-3D | 85, 105 |
| abstract_inverted_index.consistency | 96 |
| abstract_inverted_index.multi-agent | 136 |
| abstract_inverted_index.anatomically | 52 |
| abstract_inverted_index.controllable | 53 |
| abstract_inverted_index.demonstrates | 163 |
| abstract_inverted_index.high-quality | 51 |
| abstract_inverted_index.intervention | 127 |
| abstract_inverted_index.Additionally, | 88 |
| abstract_inverted_index.text-to-image | 4, 60 |
| abstract_inverted_index.https://youdream3d.github.io/ | 183 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 3 |
| citation_normalized_percentile |