Efficient Skip Connections Realization for Secure Inference on Encrypted Data

Nir Drucker; Itamar Zimerman

doi:10.1007/978-3-031-34671-2_5

Efficient Skip Connections Realization for Secure Inference on Encrypted Data

Nir Drucker, Itamar Zimerman

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Homomorphic Encryption (HE) is a cryptographic tool that allows performing computation under encryption, which is used by many privacy-preserving machine learning solutions, for example, to perform secure classification. Modern deep learning applications yield good performance for example in image processing tasks benchmarks by including many skip connections. The latter appears to be very costly when attempting to execute model inference under HE. In this paper, we show that by replacing (mid-term) skip connections with (short-term) Dirac parameterization and (long-term) shared-source skip connection we were able to reduce the skip connections burden for HE-based solutions, achieving × 1.3 computing power improvement for the sameaccuracy.

Original language	English
Title of host publication	Cyber Security, Cryptology, and Machine Learning - 7th International Symposium, CSCML 2023, Proceedings
Editors	Shlomi Dolev, Ehud Gudes, Pascal Paillier
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	65-73
Number of pages	9
ISBN (Print)	9783031346705
DOIs	https://doi.org/10.1007/978-3-031-34671-2_5
State	Published - 2023
Externally published	Yes
Event	7th International Symposium on Cyber Security, Cryptology, and Machine Learning, CSCML 2023 - Be'er Sheva, Israel Duration: 29 Jun 2023 → 30 Jun 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13914 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	7th International Symposium on Cyber Security, Cryptology, and Machine Learning, CSCML 2023
Country/Territory	Israel
City	Be'er Sheva
Period	29/06/23 → 30/06/23

Bibliographical note

Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Keywords

deep neural networks
Dirac networks
Dirac parameterization
encrypted neural networks
homomorphic encryption
PPML
privacy preserving machine learning
shared-source skip connections

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-031-34671-2_5

Cite this

Drucker, N., & Zimerman, I. (2023). Efficient Skip Connections Realization for Secure Inference on Encrypted Data. In S. Dolev, E. Gudes, & P. Paillier (Eds.), Cyber Security, Cryptology, and Machine Learning - 7th International Symposium, CSCML 2023, Proceedings (pp. 65-73). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13914 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-34671-2_5

Drucker, Nir ; Zimerman, Itamar. / Efficient Skip Connections Realization for Secure Inference on Encrypted Data. Cyber Security, Cryptology, and Machine Learning - 7th International Symposium, CSCML 2023, Proceedings. editor / Shlomi Dolev ; Ehud Gudes ; Pascal Paillier. Springer Science and Business Media Deutschland GmbH, 2023. pp. 65-73 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{37b1e1395e1b463cbe97aabb7cf7e50e,

title = "Efficient Skip Connections Realization for Secure Inference on Encrypted Data",

abstract = "Homomorphic Encryption (HE) is a cryptographic tool that allows performing computation under encryption, which is used by many privacy-preserving machine learning solutions, for example, to perform secure classification. Modern deep learning applications yield good performance for example in image processing tasks benchmarks by including many skip connections. The latter appears to be very costly when attempting to execute model inference under HE. In this paper, we show that by replacing (mid-term) skip connections with (short-term) Dirac parameterization and (long-term) shared-source skip connection we were able to reduce the skip connections burden for HE-based solutions, achieving × 1.3 computing power improvement for the sameaccuracy.",

keywords = "deep neural networks, Dirac networks, Dirac parameterization, encrypted neural networks, homomorphic encryption, PPML, privacy preserving machine learning, shared-source skip connections",

author = "Nir Drucker and Itamar Zimerman",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 7th International Symposium on Cyber Security, Cryptology, and Machine Learning, CSCML 2023 ; Conference date: 29-06-2023 Through 30-06-2023",

year = "2023",

doi = "10.1007/978-3-031-34671-2_5",

language = "English",

isbn = "9783031346705",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "65--73",

editor = "Shlomi Dolev and Ehud Gudes and Pascal Paillier",

booktitle = "Cyber Security, Cryptology, and Machine Learning - 7th International Symposium, CSCML 2023, Proceedings",

address = "Germany",

}

Drucker, N & Zimerman, I 2023, Efficient Skip Connections Realization for Secure Inference on Encrypted Data. in S Dolev, E Gudes & P Paillier (eds), Cyber Security, Cryptology, and Machine Learning - 7th International Symposium, CSCML 2023, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13914 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 65-73, 7th International Symposium on Cyber Security, Cryptology, and Machine Learning, CSCML 2023, Be'er Sheva, Israel, 29/06/23. https://doi.org/10.1007/978-3-031-34671-2_5

Efficient Skip Connections Realization for Secure Inference on Encrypted Data. / Drucker, Nir; Zimerman, Itamar.
Cyber Security, Cryptology, and Machine Learning - 7th International Symposium, CSCML 2023, Proceedings. ed. / Shlomi Dolev; Ehud Gudes; Pascal Paillier. Springer Science and Business Media Deutschland GmbH, 2023. p. 65-73 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13914 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Efficient Skip Connections Realization for Secure Inference on Encrypted Data

AU - Drucker, Nir

AU - Zimerman, Itamar

PY - 2023

Y1 - 2023

N2 - Homomorphic Encryption (HE) is a cryptographic tool that allows performing computation under encryption, which is used by many privacy-preserving machine learning solutions, for example, to perform secure classification. Modern deep learning applications yield good performance for example in image processing tasks benchmarks by including many skip connections. The latter appears to be very costly when attempting to execute model inference under HE. In this paper, we show that by replacing (mid-term) skip connections with (short-term) Dirac parameterization and (long-term) shared-source skip connection we were able to reduce the skip connections burden for HE-based solutions, achieving × 1.3 computing power improvement for the sameaccuracy.

AB - Homomorphic Encryption (HE) is a cryptographic tool that allows performing computation under encryption, which is used by many privacy-preserving machine learning solutions, for example, to perform secure classification. Modern deep learning applications yield good performance for example in image processing tasks benchmarks by including many skip connections. The latter appears to be very costly when attempting to execute model inference under HE. In this paper, we show that by replacing (mid-term) skip connections with (short-term) Dirac parameterization and (long-term) shared-source skip connection we were able to reduce the skip connections burden for HE-based solutions, achieving × 1.3 computing power improvement for the sameaccuracy.

KW - deep neural networks

KW - Dirac networks

KW - Dirac parameterization

KW - encrypted neural networks

KW - homomorphic encryption

KW - PPML

KW - privacy preserving machine learning

KW - shared-source skip connections

UR - http://www.scopus.com/inward/record.url?scp=85164957161&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-34671-2_5

DO - 10.1007/978-3-031-34671-2_5

M3 - Conference contribution

AN - SCOPUS:85164957161

SN - 9783031346705

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 65

EP - 73

BT - Cyber Security, Cryptology, and Machine Learning - 7th International Symposium, CSCML 2023, Proceedings

A2 - Dolev, Shlomi

A2 - Gudes, Ehud

A2 - Paillier, Pascal

PB - Springer Science and Business Media Deutschland GmbH

T2 - 7th International Symposium on Cyber Security, Cryptology, and Machine Learning, CSCML 2023

Y2 - 29 June 2023 through 30 June 2023

ER -

Drucker N, Zimerman I. Efficient Skip Connections Realization for Secure Inference on Encrypted Data. In Dolev S, Gudes E, Paillier P, editors, Cyber Security, Cryptology, and Machine Learning - 7th International Symposium, CSCML 2023, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 65-73. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-34671-2_5