3LegRace: Privacy-Preserving DNN Training over TEEs and GPUs Article Swipe

MAKE IMPACT

Yue Niu , Ramy E. Ali , Salman Avestimehr · YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.56553/popets-2022-0105

Leveraging parallel hardware (e.g. GPUs) for deep neural network (DNN) training brings high computing performance. However, it raises data privacy concerns as GPUs lack a trusted environment to protect the data. Trusted execution environments (TEEs) have emerged as a promising solution to achieve privacypreserving learning. Unfortunately, TEEs’ limited computing power renders them not comparable to GPUs in performance. To improve the trade-off among privacy, computing performance, and model accuracy, we propose an asymmetric model decomposition framework, AsymML, to (1) accelerate training using parallel hardware; and (2) achieve a strong privacy guarantee using TEEs and differential privacy (DP) with much less accuracy compromised compared to DP-only methods. By exploiting the low-rank characteristics in training data and intermediate features, AsymML asymmetrically decomposes inputs and intermediate activations into low-rank and residual parts. With the decomposed data, the target DNN model is accordingly split into a trusted and an untrusted part. The trusted part performs computations on low-rank data, with low compute and memory costs. The untrusted part is fed with residuals perturbed by very small noise. Privacy, computing performance, and model accuracy are well managed by respectively delegating the trusted and the untrusted part to TEEs and GPUs. We provide a formal DP guarantee that demonstrates that, for the same privacy guarantee, combining asymmetric data decomposition and DP requires much smaller noise compared to solely using DP without decomposition. This improves the privacy-utility trade-off significantly compared to using only DP methods without decomposition. Furthermore, we present a rank bound analysis showing that the low-rank structure is preserved after each layer across the entire model. Our extensive evaluations on DNN models show that AsymML delivers 7.6× speedup in training compared to the TEE-only executions while ensuring privacy. We also demonstrate that AsymML is effective in protecting data under common attacks such as model inversion and gradient attacks.

Type

article

Language

en

Landing Page

https://doi.org/10.56553/popets-2022-0105

OA Status

hybrid

Cited By

11

Related Works

10

OpenAlex ID

https://openalex.org/W4293783501

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4293783501

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.56553/popets-2022-0105

Digital Object Identifier

Title

3LegRace: Privacy-Preserving DNN Training over TEEs and GPUs

Work title

Type

article

OpenAlex work type

Language

en

Primary language

Publication year

2022

Year of publication

Publication date

2022-08-31

Full publication date if available

Authors

Yue Niu, Ramy E. Ali, Salman Avestimehr

List of authors in order

Landing page

https://doi.org/10.56553/popets-2022-0105

Publisher landing page

Open access

Yes

Whether a free full text is available

OA status

hybrid

Open access status per OpenAlex

OA URL

https://doi.org/10.56553/popets-2022-0105

Direct OA link when available

Concepts

Computer science, Differential privacy, Scheme (mathematics), Noise (video), Decomposition, Rank (graph theory), Information privacy, Trusted Network Connect, Computation, Distributed computing, Computer engineering, Parallel computing, Trusted Computing, Algorithm, Artificial intelligence, Computer security, Mathematical analysis, Mathematics, Image (mathematics), Ecology, Combinatorics, Biology

Top concepts (fields/topics) attached by OpenAlex

Cited by

11

Total citation count in OpenAlex

Citations by year (recent)

2025: 1, 2024: 6, 2023: 4

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload