FPAN: Mitigating Replication in Diffusion Models through the Fine-Grained Probabilistic Addition of Noise to Token Embeddings Article Swipe
Jun Xu
,
Chenghao Li
,
Yuke Zhang
,
Peter A. Beerel
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2505.21848
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2505.21848
Diffusion models have demonstrated remarkable potential in generating high-quality images. However, their tendency to replicate training data raises serious privacy concerns, particularly when the training datasets contain sensitive or private information. Existing mitigation strategies primarily focus on reducing image duplication, modifying the cross-attention mechanism, and altering the denoising backbone architecture of diffusion models. Moreover, recent work has shown that adding a consistent small amount of noise to text embeddings can reduce replication to some degree. In this work, we begin by analyzing the impact of adding varying amounts of noise. Based on our analysis, we propose a fine-grained noise injection technique that probabilistically adds a larger amount of noise to token embeddings. We refer to our method as Fine-grained Probabilistic Addition of Noise (FPAN). Through our extensive experiments, we show that our proposed FPAN can reduce replication by an average of 28.78% compared to the baseline diffusion model without significantly impacting image quality, and outperforms the prior consistent-magnitude-noise-addition approach by 26.51%. Moreover, when combined with other existing mitigation methods, our FPAN approach can further reduce replication by up to 16.82% with similar, if not improved, image quality.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2505.21848
- https://arxiv.org/pdf/2505.21848
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4416047528
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4416047528Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2505.21848Digital Object Identifier
- Title
-
FPAN: Mitigating Replication in Diffusion Models through the Fine-Grained Probabilistic Addition of Noise to Token EmbeddingsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-05-28Full publication date if available
- Authors
-
Jun Xu, Chenghao Li, Yuke Zhang, Peter A. BeerelList of authors in order
- Landing page
-
https://arxiv.org/abs/2505.21848Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2505.21848Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2505.21848Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4416047528 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2505.21848 |
| ids.doi | https://doi.org/10.48550/arxiv.2505.21848 |
| ids.openalex | https://openalex.org/W4416047528 |
| fwci | |
| type | preprint |
| title | FPAN: Mitigating Replication in Diffusion Models through the Fine-Grained Probabilistic Addition of Noise to Token Embeddings |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2505.21848 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2505.21848 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2505.21848 |
| locations[1].id | doi:10.48550/arxiv.2505.21848 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2505.21848 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5022865022 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-9829-4382 |
| authorships[0].author.display_name | Jun Xu |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Xu, Jingqi |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5100756593 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-8680-1655 |
| authorships[1].author.display_name | Chenghao Li |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Li, Chenghao |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5019021341 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-5253-5478 |
| authorships[2].author.display_name | Yuke Zhang |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Zhang, Yuke |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5084205024 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-8283-0168 |
| authorships[3].author.display_name | Peter A. Beerel |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Beerel, Peter A. |
| authorships[3].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2505.21848 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | FPAN: Mitigating Replication in Diffusion Models through the Fine-Grained Probabilistic Addition of Noise to Token Embeddings |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-12-01T00:03:43.161839 |
| primary_topic | |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2505.21848 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2505.21848 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2505.21848 |
| primary_location.id | pmh:oai:arXiv.org:2505.21848 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2505.21848 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2505.21848 |
| publication_date | 2025-05-28 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 60, 96, 104 |
| abstract_inverted_index.In | 75 |
| abstract_inverted_index.We | 112 |
| abstract_inverted_index.an | 138 |
| abstract_inverted_index.as | 117 |
| abstract_inverted_index.by | 80, 137, 159, 176 |
| abstract_inverted_index.if | 182 |
| abstract_inverted_index.in | 6 |
| abstract_inverted_index.of | 50, 64, 84, 88, 107, 121, 140 |
| abstract_inverted_index.on | 36, 91 |
| abstract_inverted_index.or | 28 |
| abstract_inverted_index.to | 13, 66, 72, 109, 114, 143, 178 |
| abstract_inverted_index.up | 177 |
| abstract_inverted_index.we | 78, 94, 128 |
| abstract_inverted_index.and | 44, 153 |
| abstract_inverted_index.can | 69, 134, 172 |
| abstract_inverted_index.has | 56 |
| abstract_inverted_index.not | 183 |
| abstract_inverted_index.our | 92, 115, 125, 131, 169 |
| abstract_inverted_index.the | 23, 41, 46, 82, 144, 155 |
| abstract_inverted_index.FPAN | 133, 170 |
| abstract_inverted_index.adds | 103 |
| abstract_inverted_index.data | 16 |
| abstract_inverted_index.have | 2 |
| abstract_inverted_index.show | 129 |
| abstract_inverted_index.some | 73 |
| abstract_inverted_index.text | 67 |
| abstract_inverted_index.that | 58, 101, 130 |
| abstract_inverted_index.this | 76 |
| abstract_inverted_index.when | 22, 162 |
| abstract_inverted_index.with | 164, 180 |
| abstract_inverted_index.work | 55 |
| abstract_inverted_index.Based | 90 |
| abstract_inverted_index.Noise | 122 |
| abstract_inverted_index.begin | 79 |
| abstract_inverted_index.focus | 35 |
| abstract_inverted_index.image | 38, 151, 185 |
| abstract_inverted_index.model | 147 |
| abstract_inverted_index.noise | 65, 98, 108 |
| abstract_inverted_index.other | 165 |
| abstract_inverted_index.prior | 156 |
| abstract_inverted_index.refer | 113 |
| abstract_inverted_index.shown | 57 |
| abstract_inverted_index.small | 62 |
| abstract_inverted_index.their | 11 |
| abstract_inverted_index.token | 110 |
| abstract_inverted_index.work, | 77 |
| abstract_inverted_index.16.82% | 179 |
| abstract_inverted_index.28.78% | 141 |
| abstract_inverted_index.adding | 59, 85 |
| abstract_inverted_index.amount | 63, 106 |
| abstract_inverted_index.impact | 83 |
| abstract_inverted_index.larger | 105 |
| abstract_inverted_index.method | 116 |
| abstract_inverted_index.models | 1 |
| abstract_inverted_index.noise. | 89 |
| abstract_inverted_index.raises | 17 |
| abstract_inverted_index.recent | 54 |
| abstract_inverted_index.reduce | 70, 135, 174 |
| abstract_inverted_index.(FPAN). | 123 |
| abstract_inverted_index.26.51%. | 160 |
| abstract_inverted_index.Through | 124 |
| abstract_inverted_index.amounts | 87 |
| abstract_inverted_index.average | 139 |
| abstract_inverted_index.contain | 26 |
| abstract_inverted_index.degree. | 74 |
| abstract_inverted_index.further | 173 |
| abstract_inverted_index.images. | 9 |
| abstract_inverted_index.models. | 52 |
| abstract_inverted_index.privacy | 19 |
| abstract_inverted_index.private | 29 |
| abstract_inverted_index.propose | 95 |
| abstract_inverted_index.serious | 18 |
| abstract_inverted_index.varying | 86 |
| abstract_inverted_index.without | 148 |
| abstract_inverted_index.Addition | 120 |
| abstract_inverted_index.Existing | 31 |
| abstract_inverted_index.However, | 10 |
| abstract_inverted_index.altering | 45 |
| abstract_inverted_index.approach | 158, 171 |
| abstract_inverted_index.backbone | 48 |
| abstract_inverted_index.baseline | 145 |
| abstract_inverted_index.combined | 163 |
| abstract_inverted_index.compared | 142 |
| abstract_inverted_index.datasets | 25 |
| abstract_inverted_index.existing | 166 |
| abstract_inverted_index.methods, | 168 |
| abstract_inverted_index.proposed | 132 |
| abstract_inverted_index.quality, | 152 |
| abstract_inverted_index.quality. | 186 |
| abstract_inverted_index.reducing | 37 |
| abstract_inverted_index.similar, | 181 |
| abstract_inverted_index.tendency | 12 |
| abstract_inverted_index.training | 15, 24 |
| abstract_inverted_index.Diffusion | 0 |
| abstract_inverted_index.Moreover, | 53, 161 |
| abstract_inverted_index.analysis, | 93 |
| abstract_inverted_index.analyzing | 81 |
| abstract_inverted_index.concerns, | 20 |
| abstract_inverted_index.denoising | 47 |
| abstract_inverted_index.diffusion | 51, 146 |
| abstract_inverted_index.extensive | 126 |
| abstract_inverted_index.impacting | 150 |
| abstract_inverted_index.improved, | 184 |
| abstract_inverted_index.injection | 99 |
| abstract_inverted_index.modifying | 40 |
| abstract_inverted_index.potential | 5 |
| abstract_inverted_index.primarily | 34 |
| abstract_inverted_index.replicate | 14 |
| abstract_inverted_index.sensitive | 27 |
| abstract_inverted_index.technique | 100 |
| abstract_inverted_index.consistent | 61 |
| abstract_inverted_index.embeddings | 68 |
| abstract_inverted_index.generating | 7 |
| abstract_inverted_index.mechanism, | 43 |
| abstract_inverted_index.mitigation | 32, 167 |
| abstract_inverted_index.remarkable | 4 |
| abstract_inverted_index.strategies | 33 |
| abstract_inverted_index.embeddings. | 111 |
| abstract_inverted_index.outperforms | 154 |
| abstract_inverted_index.replication | 71, 136, 175 |
| abstract_inverted_index.Fine-grained | 118 |
| abstract_inverted_index.architecture | 49 |
| abstract_inverted_index.demonstrated | 3 |
| abstract_inverted_index.duplication, | 39 |
| abstract_inverted_index.experiments, | 127 |
| abstract_inverted_index.fine-grained | 97 |
| abstract_inverted_index.high-quality | 8 |
| abstract_inverted_index.information. | 30 |
| abstract_inverted_index.particularly | 21 |
| abstract_inverted_index.Probabilistic | 119 |
| abstract_inverted_index.significantly | 149 |
| abstract_inverted_index.cross-attention | 42 |
| abstract_inverted_index.probabilistically | 102 |
| abstract_inverted_index.consistent-magnitude-noise-addition | 157 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile |