OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models Article Swipe
Zhe Kong
,
Yong Zhang
,
Tianyu Yang
,
Tao Wang
,
Kaihao Zhang
,
Bizhu Wu
,
Guanying Chen
,
Wei Liu
,
Wenhan Luo
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2403.10983
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2403.10983
Personalization is an important topic in text-to-image generation, especially the challenging multi-concept personalization. Current multi-concept methods are struggling with identity preservation, occlusion, and the harmony between foreground and background. In this work, we propose OMG, an occlusion-friendly personalized generation framework designed to seamlessly integrate multiple concepts within a single image. We propose a novel two-stage sampling solution. The first stage takes charge of layout generation and visual comprehension information collection for handling occlusions. The second one utilizes the acquired visual comprehension information and the designed noise blending to integrate multiple concepts while considering occlusions. We also observe that the initiation denoising timestep for noise blending is the key to identity preservation and layout. Moreover, our method can be combined with various single-concept models, such as LoRA and InstantID without additional tuning. Especially, LoRA models on civitai.com can be exploited directly. Extensive experiments demonstrate that OMG exhibits superior performance in multi-concept personalization.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2403.10983
- https://arxiv.org/pdf/2403.10983
- OA Status
- green
- Cited By
- 1
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4393023484
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4393023484Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2403.10983Digital Object Identifier
- Title
-
OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion ModelsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-03-16Full publication date if available
- Authors
-
Zhe Kong, Yong Zhang, Tianyu Yang, Tao Wang, Kaihao Zhang, Bizhu Wu, Guanying Chen, Wei Liu, Wenhan LuoList of authors in order
- Landing page
-
https://arxiv.org/abs/2403.10983Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2403.10983Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2403.10983Direct OA link when available
- Concepts
-
Computer science, Diffusion, User Friendly, Occlusion, Medicine, Internal medicine, Programming language, Physics, ThermodynamicsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4393023484 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2403.10983 |
| ids.doi | https://doi.org/10.48550/arxiv.2403.10983 |
| ids.openalex | https://openalex.org/W4393023484 |
| fwci | |
| type | preprint |
| title | OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12016 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9642999768257141 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1710 |
| topics[0].subfield.display_name | Information Systems |
| topics[0].display_name | Web Data Mining and Analysis |
| topics[1].id | https://openalex.org/T10028 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9527000188827515 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Topic Modeling |
| topics[2].id | https://openalex.org/T10203 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9355999827384949 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1710 |
| topics[2].subfield.display_name | Information Systems |
| topics[2].display_name | Recommender Systems and Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.5711761713027954 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C69357855 |
| concepts[1].level | 2 |
| concepts[1].score | 0.5349672436714172 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q163214 |
| concepts[1].display_name | Diffusion |
| concepts[2].id | https://openalex.org/C61437842 |
| concepts[2].level | 2 |
| concepts[2].score | 0.5122663378715515 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q281291 |
| concepts[2].display_name | User Friendly |
| concepts[3].id | https://openalex.org/C2776268601 |
| concepts[3].level | 2 |
| concepts[3].score | 0.43810051679611206 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q968808 |
| concepts[3].display_name | Occlusion |
| concepts[4].id | https://openalex.org/C71924100 |
| concepts[4].level | 0 |
| concepts[4].score | 0.25349539518356323 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11190 |
| concepts[4].display_name | Medicine |
| concepts[5].id | https://openalex.org/C126322002 |
| concepts[5].level | 1 |
| concepts[5].score | 0.1475624442100525 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11180 |
| concepts[5].display_name | Internal medicine |
| concepts[6].id | https://openalex.org/C199360897 |
| concepts[6].level | 1 |
| concepts[6].score | 0.10232099890708923 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[6].display_name | Programming language |
| concepts[7].id | https://openalex.org/C121332964 |
| concepts[7].level | 0 |
| concepts[7].score | 0.09730106592178345 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[7].display_name | Physics |
| concepts[8].id | https://openalex.org/C97355855 |
| concepts[8].level | 1 |
| concepts[8].score | 0.0 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q11473 |
| concepts[8].display_name | Thermodynamics |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.5711761713027954 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/diffusion |
| keywords[1].score | 0.5349672436714172 |
| keywords[1].display_name | Diffusion |
| keywords[2].id | https://openalex.org/keywords/user-friendly |
| keywords[2].score | 0.5122663378715515 |
| keywords[2].display_name | User Friendly |
| keywords[3].id | https://openalex.org/keywords/occlusion |
| keywords[3].score | 0.43810051679611206 |
| keywords[3].display_name | Occlusion |
| keywords[4].id | https://openalex.org/keywords/medicine |
| keywords[4].score | 0.25349539518356323 |
| keywords[4].display_name | Medicine |
| keywords[5].id | https://openalex.org/keywords/internal-medicine |
| keywords[5].score | 0.1475624442100525 |
| keywords[5].display_name | Internal medicine |
| keywords[6].id | https://openalex.org/keywords/programming-language |
| keywords[6].score | 0.10232099890708923 |
| keywords[6].display_name | Programming language |
| keywords[7].id | https://openalex.org/keywords/physics |
| keywords[7].score | 0.09730106592178345 |
| keywords[7].display_name | Physics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2403.10983 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2403.10983 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2403.10983 |
| locations[1].id | doi:10.48550/arxiv.2403.10983 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2403.10983 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5055425346 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-7430-534X |
| authorships[0].author.display_name | Zhe Kong |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Kong, Zhe |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5100419741 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-8803-2055 |
| authorships[1].author.display_name | Yong Zhang |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Zhang, Yong |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5101998585 |
| authorships[2].author.orcid | https://orcid.org/0009-0001-4646-085X |
| authorships[2].author.display_name | Tianyu Yang |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Yang, Tianyu |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5100453601 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-8406-8672 |
| authorships[3].author.display_name | Tao Wang |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Wang, Tao |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5103144603 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-4317-660X |
| authorships[4].author.display_name | Kaihao Zhang |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Zhang, Kaihao |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5101320959 |
| authorships[5].author.orcid | |
| authorships[5].author.display_name | Bizhu Wu |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Wu, Bizhu |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5074655512 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-4789-1176 |
| authorships[6].author.display_name | Guanying Chen |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Chen, Guanying |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5102644156 |
| authorships[7].author.orcid | https://orcid.org/0009-0005-9131-1211 |
| authorships[7].author.display_name | Wei Liu |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Liu, Wei |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5004450394 |
| authorships[8].author.orcid | https://orcid.org/0000-0002-5697-4168 |
| authorships[8].author.display_name | Wenhan Luo |
| authorships[8].author_position | last |
| authorships[8].raw_author_name | Luo, Wenhan |
| authorships[8].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2403.10983 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12016 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9642999768257141 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1710 |
| primary_topic.subfield.display_name | Information Systems |
| primary_topic.display_name | Web Data Mining and Analysis |
| related_works | https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W2382290278, https://openalex.org/W2478288626, https://openalex.org/W4391913857, https://openalex.org/W2350741829, https://openalex.org/W2530322880 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2403.10983 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2403.10983 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2403.10983 |
| primary_location.id | pmh:oai:arXiv.org:2403.10983 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2403.10983 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2403.10983 |
| publication_date | 2024-03-16 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 47, 52 |
| abstract_inverted_index.In | 29 |
| abstract_inverted_index.We | 50, 94 |
| abstract_inverted_index.an | 2, 35 |
| abstract_inverted_index.as | 124 |
| abstract_inverted_index.be | 117, 137 |
| abstract_inverted_index.in | 5, 148 |
| abstract_inverted_index.is | 1, 105 |
| abstract_inverted_index.of | 62 |
| abstract_inverted_index.on | 134 |
| abstract_inverted_index.to | 41, 87, 108 |
| abstract_inverted_index.we | 32 |
| abstract_inverted_index.OMG | 144 |
| abstract_inverted_index.The | 57, 73 |
| abstract_inverted_index.and | 22, 27, 65, 82, 111, 126 |
| abstract_inverted_index.are | 16 |
| abstract_inverted_index.can | 116, 136 |
| abstract_inverted_index.for | 70, 102 |
| abstract_inverted_index.key | 107 |
| abstract_inverted_index.one | 75 |
| abstract_inverted_index.our | 114 |
| abstract_inverted_index.the | 9, 23, 77, 83, 98, 106 |
| abstract_inverted_index.LoRA | 125, 132 |
| abstract_inverted_index.OMG, | 34 |
| abstract_inverted_index.also | 95 |
| abstract_inverted_index.such | 123 |
| abstract_inverted_index.that | 97, 143 |
| abstract_inverted_index.this | 30 |
| abstract_inverted_index.with | 18, 119 |
| abstract_inverted_index.first | 58 |
| abstract_inverted_index.noise | 85, 103 |
| abstract_inverted_index.novel | 53 |
| abstract_inverted_index.stage | 59 |
| abstract_inverted_index.takes | 60 |
| abstract_inverted_index.topic | 4 |
| abstract_inverted_index.while | 91 |
| abstract_inverted_index.work, | 31 |
| abstract_inverted_index.charge | 61 |
| abstract_inverted_index.image. | 49 |
| abstract_inverted_index.layout | 63 |
| abstract_inverted_index.method | 115 |
| abstract_inverted_index.models | 133 |
| abstract_inverted_index.second | 74 |
| abstract_inverted_index.single | 48 |
| abstract_inverted_index.visual | 66, 79 |
| abstract_inverted_index.within | 46 |
| abstract_inverted_index.Current | 13 |
| abstract_inverted_index.between | 25 |
| abstract_inverted_index.harmony | 24 |
| abstract_inverted_index.layout. | 112 |
| abstract_inverted_index.methods | 15 |
| abstract_inverted_index.models, | 122 |
| abstract_inverted_index.observe | 96 |
| abstract_inverted_index.propose | 33, 51 |
| abstract_inverted_index.tuning. | 130 |
| abstract_inverted_index.various | 120 |
| abstract_inverted_index.without | 128 |
| abstract_inverted_index.acquired | 78 |
| abstract_inverted_index.blending | 86, 104 |
| abstract_inverted_index.combined | 118 |
| abstract_inverted_index.concepts | 45, 90 |
| abstract_inverted_index.designed | 40, 84 |
| abstract_inverted_index.exhibits | 145 |
| abstract_inverted_index.handling | 71 |
| abstract_inverted_index.identity | 19, 109 |
| abstract_inverted_index.multiple | 44, 89 |
| abstract_inverted_index.sampling | 55 |
| abstract_inverted_index.superior | 146 |
| abstract_inverted_index.timestep | 101 |
| abstract_inverted_index.utilizes | 76 |
| abstract_inverted_index.Extensive | 140 |
| abstract_inverted_index.InstantID | 127 |
| abstract_inverted_index.Moreover, | 113 |
| abstract_inverted_index.denoising | 100 |
| abstract_inverted_index.directly. | 139 |
| abstract_inverted_index.exploited | 138 |
| abstract_inverted_index.framework | 39 |
| abstract_inverted_index.important | 3 |
| abstract_inverted_index.integrate | 43, 88 |
| abstract_inverted_index.solution. | 56 |
| abstract_inverted_index.two-stage | 54 |
| abstract_inverted_index.additional | 129 |
| abstract_inverted_index.collection | 69 |
| abstract_inverted_index.especially | 8 |
| abstract_inverted_index.foreground | 26 |
| abstract_inverted_index.generation | 38, 64 |
| abstract_inverted_index.initiation | 99 |
| abstract_inverted_index.occlusion, | 21 |
| abstract_inverted_index.seamlessly | 42 |
| abstract_inverted_index.struggling | 17 |
| abstract_inverted_index.Especially, | 131 |
| abstract_inverted_index.background. | 28 |
| abstract_inverted_index.challenging | 10 |
| abstract_inverted_index.civitai.com | 135 |
| abstract_inverted_index.considering | 92 |
| abstract_inverted_index.demonstrate | 142 |
| abstract_inverted_index.experiments | 141 |
| abstract_inverted_index.generation, | 7 |
| abstract_inverted_index.information | 68, 81 |
| abstract_inverted_index.occlusions. | 72, 93 |
| abstract_inverted_index.performance | 147 |
| abstract_inverted_index.personalized | 37 |
| abstract_inverted_index.preservation | 110 |
| abstract_inverted_index.comprehension | 67, 80 |
| abstract_inverted_index.multi-concept | 11, 14, 149 |
| abstract_inverted_index.preservation, | 20 |
| abstract_inverted_index.text-to-image | 6 |
| abstract_inverted_index.single-concept | 121 |
| abstract_inverted_index.Personalization | 0 |
| abstract_inverted_index.personalization. | 12, 150 |
| abstract_inverted_index.occlusion-friendly | 36 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 9 |
| citation_normalized_percentile |