Santiago Zanella-Béguelin
YOU?
Author Swipe
View article: Beyond Membership: Limitations of Add/Remove Adjacency in Differential Privacy
Beyond Membership: Limitations of Add/Remove Adjacency in Differential Privacy Open
Training machine learning models with differential privacy (DP) limits an adversary's ability to infer sensitive information about the training data. It can be interpreted as a bound on adversary's capability to distinguish two adjacent da…
View article: The Price of Intelligence
The Price of Intelligence Open
Three risks inherent in LLMs.
View article: A Systematization of Security Vulnerabilities in Computer Use Agents
A Systematization of Security Vulnerabilities in Computer Use Agents Open
Computer Use Agents (CUAs), autonomous systems that interact with software interfaces via browsers or virtual machines, are rapidly being deployed in consumer and enterprise environments. These agents introduce novel attack surfaces and tr…
View article: A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks
A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks Open
Recent research has demonstrated that state-of-the-art LLMs and defenses remain susceptible to multi-turn jailbreak attacks. These attacks require only closed-box model access and are often easy to perform manually, posing a significant th…
View article: Securing AI Agents with Information-Flow Control
Securing AI Agents with Information-Flow Control Open
As AI agents become increasingly autonomous and capable, ensuring their security against vulnerabilities such as prompt injection becomes critical. This paper explores the use of information-flow control (IFC) to provide security guarantee…
View article: The Price of Intelligence
The Price of Intelligence Open
The vulnerability of LLMs to hallucination, prompt injection, and jailbreaks poses a significant but surmountable challenge to their widespread adoption and responsible use. We have argued that these problems are inherent, certainly in the…
View article: Permissive Information-Flow Analysis for Large Language Models
Permissive Information-Flow Analysis for Large Language Models Open
Large Language Models (LLMs) are rapidly becoming commodity components of larger software systems. This poses natural security and privacy problems: poisoned data retrieved from one component can change the model's behavior and compromise …
View article: Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competition
Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competition Open
Large language model systems face important security risks from maliciously crafted messages that aim to overwrite the system's original instructions or leak private data. To study this problem, we organized a capture-the-flag competition …
View article: Closed-Form Bounds for DP-SGD against Record-level Inference
Closed-Form Bounds for DP-SGD against Record-level Inference Open
Machine learning models trained with differentially-private (DP) algorithms such as DP-SGD enjoy resilience against a wide range of privacy attacks. Although it is possible to derive bounds for some attacks based solely on an $(\varepsilon…
View article: Rethinking Privacy in Machine Learning Pipelines from an Information Flow Control Perspective
Rethinking Privacy in Machine Learning Pipelines from an Information Flow Control Perspective Open
Modern machine learning systems use models trained on ever-growing corpora. Typically, metadata such as ownership, access control, or licensing information is ignored during training. Instead, to mitigate privacy risks, we rely on generic …
View article: On the Efficacy of Differentially Private Few-shot Image Classification
On the Efficacy of Differentially Private Few-shot Image Classification Open
There has been significant recent progress in training differentially private (DP) models which achieve accuracy that approaches the best non-private models. These DP models are typically pretrained on large public datasets and then fine-t…
View article: Analyzing Leakage of Personally Identifiable Information in Language Models
Analyzing Leakage of Personally Identifiable Information in Language Models Open
Language Models (LMs) have been shown to leak information about training data through sentence-level membership inference and reconstruction attacks. Understanding the risk of LMs leaking Personally Identifiable Information (PII) has recei…
View article: SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning
SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning Open
Deploying machine learning models in production may allow adversaries to infer sensitive information about training data. There is a vast literature analyzing different types of inference risks, ranging from membership inference to reconst…
View article: Bayesian Estimation of Differential Privacy
Bayesian Estimation of Differential Privacy Open
Algorithms such as Differentially Private SGD enable training machine learning models with formal privacy guarantees. However, there is a discrepancy between the protection that such algorithms guarantee in theory and the protection they a…
View article: Analyzing Information Leakage of Updates to Natural Language Models
Analyzing Information Leakage of Updates to Natural Language Models Open
To continuously improve quality and reflect changes in data, machine learning\napplications have to regularly retrain and update their core models. We show\nthat a differential analysis of language model snapshots before and after an\nupda…
View article: HACLxN: Verified Generic SIMD Crypto (for all your favourite platforms)
HACLxN: Verified Generic SIMD Crypto (for all your favourite platforms) Open
We present a new methodology for building formally verified cryptographic libraries that are optimized for multiple architectures. In particular, we show how to write and verify generic crypto code in the F* programming language that explo…
View article: EverCrypt: A Fast, Verified, Cross-Platform Cryptographic Provider
EverCrypt: A Fast, Verified, Cross-Platform Cryptographic Provider Open
International audience
View article: Analyzing Privacy Loss in Updates of Natural Language Models.
Analyzing Privacy Loss in Updates of Natural Language Models. Open
To continuously improve quality and reflect changes in data, machine learning-based services have to regularly re-train and update their core models. In the setting of language models, we show that a comparative analysis of model snapshots…
View article: Imperfect forward secrecy
Imperfect forward secrecy Open
We investigate the security of Diffie-Hellman key exchange as used in popular Internet protocols and find it to be less secure than widely believed. First, we present Logjam, a novel flaw in TLS that lets a man-in-the-middle downgrade conn…
View article: Towards Automated Proving of Relational Properties of Probabilistic Programs (Invited Talk)
Towards Automated Proving of Relational Properties of Probabilistic Programs (Invited Talk) Open
Some security properties go beyond what is expressible in terms of an individual execution of a single program. In particular, many security policies in cryptography can be naturally phrased as relational properties of two open probabilist…
View article: A monadic framework for relational verification: applied to information security, program equivalence, and optimizations
A monadic framework for relational verification: applied to information security, program equivalence, and optimizations Open
International audience
View article: Verified low-level programming embedded in F*
Verified low-level programming embedded in F* Open
We present Low*, a language for low-level programming and verification, and its application to high-assurance optimized cryptographic libraries. Low* is a shallow embedding of a small, sequential, well-behaved subset of C in F*, a dependen…
View article: Implementing and Proving the TLS 1.3 Record Layer
Implementing and Proving the TLS 1.3 Record Layer Open
International audience
View article: Verified Low-Level Programming Embedded in F*
Verified Low-Level Programming Embedded in F* Open
We present Low*, a language for low-level programming and verification, and its application to high-assurance optimized cryptographic libraries. Low* is a shallow embedding of a small, sequential, well-behaved subset of C in F*, a dependen…
View article: A Monadic Framework for Relational Verification (Functional Pearl).
A Monadic Framework for Relational Verification (Functional Pearl). Open
Relational properties describe multiple runs of one or more programs. They characterize many useful notions of security, program refinement, and equivalence for programs with diverse computational effects, and they have received much atten…
View article: A Monadic Framework for Relational Verification: Applied to Information Security, Program Equivalence, and Optimizations
A Monadic Framework for Relational Verification: Applied to Information Security, Program Equivalence, and Optimizations Open
Relational properties describe multiple runs of one or more programs. They characterize many useful notions of security, program refinement, and equivalence for programs with diverse computational effects, and they have received much atten…
View article: Formal Verification of Smart Contracts
Formal Verification of Smart Contracts Open
International audience
View article: Dependent types and multi-monadic effects in F*
Dependent types and multi-monadic effects in F* Open
International audience
View article: Imperfect Forward Secrecy
Imperfect Forward Secrecy Open
International audience