Privacy Preserving In-Context-Learning Framework for Large Language Models Article Swipe

PDF

Bishnu Bhusal , Manoj Acharya , Colin Samplawski , Adam D. Cobb , Susmit Jha ·

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2509.13625

Large language models (LLMs) have significantly transformed natural language understanding and generation, but they raise privacy concerns due to potential exposure of sensitive information. Studies have highlighted the risk of information leakage, where adversaries can extract sensitive information embedded in the prompts. In this work, we introduce a novel private prediction framework for generating high-quality synthetic text with strong privacy guarantees. Our approach leverages the Differential Privacy (DP) framework to ensure worst-case theoretical bounds on information leakage without requiring any fine-tuning of the underlying models. The proposed method performs inference on private records and aggregates the resulting per-token output distributions. This enables the generation of longer and coherent synthetic text while maintaining privacy guarantees. Additionally, we propose a simple blending operation that combines private and public inference to further enhance utility. Empirical evaluations demonstrate that our approach outperforms previous state-of-the-art methods on in-context-learning (ICL) tasks, making it a promising direction for privacy-preserving text generation while maintaining high utility. Our code is available at https://github.com/bhusalb/privacy-preserving-icl.

Related Topics

Right To Privacy

Invasion Of Privacy (Album)

Privacy

Privacy Laws Of The United States

Gnu Privacy Guard

Privacy Act Of 1974

Privacy International

Privacy Badger

Concepts

No concepts available.

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2509.13625
PDF: https://arxiv.org/pdf/2509.13625
OA Status: green
OpenAlex ID: https://openalex.org/W4416254678

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4416254678

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2509.13625

Digital Object Identifier
Title: Privacy Preserving In-Context-Learning Framework for Large Language Models

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-09-17

Full publication date if available
Authors: Bishnu Bhusal, Manoj Acharya, Colin Samplawski, Adam D. Cobb, Susmit Jha

List of authors in order
Landing page: https://arxiv.org/abs/2509.13625

Publisher landing page
PDF URL: https://arxiv.org/pdf/2509.13625

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2509.13625

Direct OA link when available
Cited by: 0

Total citation count in OpenAlex

Full payload

id	https://openalex.org/W4416254678
doi	https://doi.org/10.48550/arxiv.2509.13625
ids.doi	https://doi.org/10.48550/arxiv.2509.13625
ids.openalex	https://openalex.org/W4416254678
fwci
type	preprint
title	Privacy Preserving In-Context-Learning Framework for Large Language Models
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
is_xpac	False
apc_list
apc_paid
language	en
locations[0].id	pmh:oai:arXiv.org:2509.13625
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2509.13625
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2509.13625
locations[1].id	doi:10.48550/arxiv.2509.13625
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2509.13625
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5092885542
authorships[0].author.orcid	https://orcid.org/0000-0001-7522-5878
authorships[0].author.display_name	Bishnu Bhusal
authorships[0].author_position	first
authorships[0].raw_author_name	Bhusal, Bishnu
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5019134968
authorships[1].author.orcid	https://orcid.org/0000-0003-0223-3556
authorships[1].author.display_name	Manoj Acharya
authorships[1].author_position	middle
authorships[1].raw_author_name	Acharya, Manoj
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5011915615
authorships[2].author.orcid
authorships[2].author.display_name	Colin Samplawski
authorships[2].author_position	middle
authorships[2].raw_author_name	Samplawski, Colin
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5102898820
authorships[3].author.orcid	https://orcid.org/0000-0002-6638-1788
authorships[3].author.display_name	Adam D. Cobb
authorships[3].author_position	middle
authorships[3].raw_author_name	Cobb, Adam D.
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5035902535
authorships[4].author.orcid	https://orcid.org/0000-0001-5983-9095
authorships[4].author.display_name	Susmit Jha
authorships[4].author_position	middle
authorships[4].raw_author_name	Jha, Susmit
authorships[4].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2509.13625
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Privacy Preserving In-Context-Learning Framework for Large Language Models
has_fulltext	False
is_retracted	False
updated_date	2025-11-28T09:29:53.416291
primary_topic
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2509.13625
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2509.13625
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2509.13625
primary_location.id	pmh:oai:arXiv.org:2509.13625
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2509.13625
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2509.13625
publication_date	2025-09-17
publication_year	2025
referenced_works_count	0
abstract_inverted_index.a	47, 117, 147
abstract_inverted_index.In	42
abstract_inverted_index.at	162
abstract_inverted_index.in	39
abstract_inverted_index.is	160
abstract_inverted_index.it	146
abstract_inverted_index.of	21, 29, 81, 104
abstract_inverted_index.on	74, 90, 141
abstract_inverted_index.to	18, 69, 127
abstract_inverted_index.we	45, 115
abstract_inverted_index.Our	61, 158
abstract_inverted_index.The	85
abstract_inverted_index.and	10, 93, 106, 124
abstract_inverted_index.any	79
abstract_inverted_index.but	12
abstract_inverted_index.can	34
abstract_inverted_index.due	17
abstract_inverted_index.for	52, 150
abstract_inverted_index.our	135
abstract_inverted_index.the	27, 40, 64, 82, 95, 102
abstract_inverted_index.(DP)	67
abstract_inverted_index.This	100
abstract_inverted_index.code	159
abstract_inverted_index.have	4, 25
abstract_inverted_index.high	156
abstract_inverted_index.risk	28
abstract_inverted_index.text	56, 109, 152
abstract_inverted_index.that	121, 134
abstract_inverted_index.they	13
abstract_inverted_index.this	43
abstract_inverted_index.with	57
abstract_inverted_index.(ICL)	143
abstract_inverted_index.Large	0
abstract_inverted_index.novel	48
abstract_inverted_index.raise	14
abstract_inverted_index.where	32
abstract_inverted_index.while	110, 154
abstract_inverted_index.work,	44
abstract_inverted_index.(LLMs)	3
abstract_inverted_index.bounds	73
abstract_inverted_index.ensure	70
abstract_inverted_index.longer	105
abstract_inverted_index.making	145
abstract_inverted_index.method	87
abstract_inverted_index.models	2
abstract_inverted_index.output	98
abstract_inverted_index.public	125
abstract_inverted_index.simple	118
abstract_inverted_index.strong	58
abstract_inverted_index.tasks,	144
abstract_inverted_index.Privacy	66
abstract_inverted_index.Studies	24
abstract_inverted_index.enables	101
abstract_inverted_index.enhance	129
abstract_inverted_index.extract	35
abstract_inverted_index.further	128
abstract_inverted_index.leakage	76
abstract_inverted_index.methods	140
abstract_inverted_index.models.	84
abstract_inverted_index.natural	7
abstract_inverted_index.privacy	15, 59, 112
abstract_inverted_index.private	49, 91, 123
abstract_inverted_index.propose	116
abstract_inverted_index.records	92
abstract_inverted_index.without	77
abstract_inverted_index.approach	62, 136
abstract_inverted_index.blending	119
abstract_inverted_index.coherent	107
abstract_inverted_index.combines	122
abstract_inverted_index.concerns	16
abstract_inverted_index.embedded	38
abstract_inverted_index.exposure	20
abstract_inverted_index.language	1, 8
abstract_inverted_index.leakage,	31
abstract_inverted_index.performs	88
abstract_inverted_index.previous	138
abstract_inverted_index.prompts.	41
abstract_inverted_index.proposed	86
abstract_inverted_index.utility.	130, 157
abstract_inverted_index.Empirical	131
abstract_inverted_index.available	161
abstract_inverted_index.direction	149
abstract_inverted_index.framework	51, 68
abstract_inverted_index.inference	89, 126
abstract_inverted_index.introduce	46
abstract_inverted_index.leverages	63
abstract_inverted_index.operation	120
abstract_inverted_index.per-token	97
abstract_inverted_index.potential	19
abstract_inverted_index.promising	148
abstract_inverted_index.requiring	78
abstract_inverted_index.resulting	96
abstract_inverted_index.sensitive	22, 36
abstract_inverted_index.synthetic	55, 108
abstract_inverted_index.aggregates	94
abstract_inverted_index.generating	53
abstract_inverted_index.generation	103, 153
abstract_inverted_index.prediction	50
abstract_inverted_index.underlying	83
abstract_inverted_index.worst-case	71
abstract_inverted_index.adversaries	33
abstract_inverted_index.demonstrate	133
abstract_inverted_index.evaluations	132
abstract_inverted_index.fine-tuning	80
abstract_inverted_index.generation,	11
abstract_inverted_index.guarantees.	60, 113
abstract_inverted_index.highlighted	26
abstract_inverted_index.information	30, 37, 75
abstract_inverted_index.maintaining	111, 155
abstract_inverted_index.outperforms	137
abstract_inverted_index.theoretical	72
abstract_inverted_index.transformed	6
abstract_inverted_index.Differential	65
abstract_inverted_index.high-quality	54
abstract_inverted_index.information.	23
abstract_inverted_index.Additionally,	114
abstract_inverted_index.significantly	5
abstract_inverted_index.understanding	9
abstract_inverted_index.distributions.	99
abstract_inverted_index.state-of-the-art	139
abstract_inverted_index.privacy-preserving	151
abstract_inverted_index.in-context-learning	142
abstract_inverted_index.https://github.com/bhusalb/privacy-preserving-icl.	163
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	5
citation_normalized_percentile