Primula: A practical shuffle/sort operator for serverless computing

Marc Sánchez-Artigas; Germán T. Eizaguirre; Gil Vernik; Lachlan Stuart; Pedro Garciá-López

doi:10.1145/3429357.3430522

Primula: A practical shuffle/sort operator for serverless computing

Marc Sánchez-Artigas, Germán T. Eizaguirre, Gil Vernik, Lachlan Stuart, Pedro Garciá-López

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Serverless computing has recently gained much attention as a feasible alternative to always-on IaaS for data processing. However, existing severless frameworks are not (yet) usable enough to reach out to a large number of users. To wit, they still require developers to specify the number of serverless functions for a simple sort job. We report our experience in designing Primula, a serverless sort operator that abstracts away users from the complexities of resource provisioning, skewed data and stragglers, yielding the most accessible sort primitive to date. Our evaluation on the IBM Cloud platform demonstrates the usability of Primula without abandoning performance (e.g., 3x faster than a serverless Spark backend and 62% slower than a hybrid serverless/IaaS solution).

Original language	English
Title of host publication	Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020
Publisher	Association for Computing Machinery, Inc
Pages	31-37
Number of pages	7
ISBN (Electronic)	9781450382014
DOIs	https://doi.org/10.1145/3429357.3430522
State	Published - 7 Dec 2020
Externally published	Yes
Event	21st International Middleware Conference Industrial Track, Middleware Industry 2020 - Part of Middleware 2020 - Virtual, Online, Netherlands Duration: 7 Dec 2020 → 11 Dec 2020

Publication series

Name	Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020

Conference

Conference	21st International Middleware Conference Industrial Track, Middleware Industry 2020 - Part of Middleware 2020
Country/Territory	Netherlands
City	Virtual, Online
Period	7/12/20 → 11/12/20

Bibliographical note

Publisher Copyright:
© 2020 ACM.

Keywords

Function-as-a-service
Serverless computing

ASJC Scopus subject areas

Computer Networks and Communications
Hardware and Architecture
Software

Access to Document

10.1145/3429357.3430522

Cite this

Sánchez-Artigas, M., Eizaguirre, G. T., Vernik, G., Stuart, L., & Garciá-López, P. (2020). Primula: A practical shuffle/sort operator for serverless computing. In Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020 (pp. 31-37). (Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020). Association for Computing Machinery, Inc. https://doi.org/10.1145/3429357.3430522

Sánchez-Artigas, Marc ; Eizaguirre, Germán T. ; Vernik, Gil et al. / Primula : A practical shuffle/sort operator for serverless computing. Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020. Association for Computing Machinery, Inc, 2020. pp. 31-37 (Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020).

@inproceedings{839594f36130459c9ba50b8d15e26d8f,

title = "Primula: A practical shuffle/sort operator for serverless computing",

abstract = "Serverless computing has recently gained much attention as a feasible alternative to always-on IaaS for data processing. However, existing severless frameworks are not (yet) usable enough to reach out to a large number of users. To wit, they still require developers to specify the number of serverless functions for a simple sort job. We report our experience in designing Primula, a serverless sort operator that abstracts away users from the complexities of resource provisioning, skewed data and stragglers, yielding the most accessible sort primitive to date. Our evaluation on the IBM Cloud platform demonstrates the usability of Primula without abandoning performance (e.g., 3x faster than a serverless Spark backend and 62% slower than a hybrid serverless/IaaS solution).",

keywords = "Function-as-a-service, Serverless computing",

author = "Marc S{\'a}nchez-Artigas and Eizaguirre, {Germ{\'a}n T.} and Gil Vernik and Lachlan Stuart and Pedro Garci{\'a}-L{\'o}pez",

note = "Publisher Copyright: {\textcopyright} 2020 ACM.; 21st International Middleware Conference Industrial Track, Middleware Industry 2020 - Part of Middleware 2020 ; Conference date: 07-12-2020 Through 11-12-2020",

year = "2020",

month = dec,

day = "7",

doi = "10.1145/3429357.3430522",

language = "English",

series = "Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020",

publisher = "Association for Computing Machinery, Inc",

pages = "31--37",

booktitle = "Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020",

}

Sánchez-Artigas, M, Eizaguirre, GT, Vernik, G, Stuart, L & Garciá-López, P 2020, Primula: A practical shuffle/sort operator for serverless computing. in Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020. Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020, Association for Computing Machinery, Inc, pp. 31-37, 21st International Middleware Conference Industrial Track, Middleware Industry 2020 - Part of Middleware 2020, Virtual, Online, Netherlands, 7/12/20. https://doi.org/10.1145/3429357.3430522

Primula: A practical shuffle/sort operator for serverless computing. / Sánchez-Artigas, Marc; Eizaguirre, Germán T.; Vernik, Gil et al.
Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020. Association for Computing Machinery, Inc, 2020. p. 31-37 (Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Primula

T2 - 21st International Middleware Conference Industrial Track, Middleware Industry 2020 - Part of Middleware 2020

AU - Sánchez-Artigas, Marc

AU - Eizaguirre, Germán T.

AU - Vernik, Gil

AU - Stuart, Lachlan

AU - Garciá-López, Pedro

PY - 2020/12/7

Y1 - 2020/12/7

N2 - Serverless computing has recently gained much attention as a feasible alternative to always-on IaaS for data processing. However, existing severless frameworks are not (yet) usable enough to reach out to a large number of users. To wit, they still require developers to specify the number of serverless functions for a simple sort job. We report our experience in designing Primula, a serverless sort operator that abstracts away users from the complexities of resource provisioning, skewed data and stragglers, yielding the most accessible sort primitive to date. Our evaluation on the IBM Cloud platform demonstrates the usability of Primula without abandoning performance (e.g., 3x faster than a serverless Spark backend and 62% slower than a hybrid serverless/IaaS solution).

AB - Serverless computing has recently gained much attention as a feasible alternative to always-on IaaS for data processing. However, existing severless frameworks are not (yet) usable enough to reach out to a large number of users. To wit, they still require developers to specify the number of serverless functions for a simple sort job. We report our experience in designing Primula, a serverless sort operator that abstracts away users from the complexities of resource provisioning, skewed data and stragglers, yielding the most accessible sort primitive to date. Our evaluation on the IBM Cloud platform demonstrates the usability of Primula without abandoning performance (e.g., 3x faster than a serverless Spark backend and 62% slower than a hybrid serverless/IaaS solution).

KW - Function-as-a-service

KW - Serverless computing

UR - http://www.scopus.com/inward/record.url?scp=85100496382&partnerID=8YFLogxK

U2 - 10.1145/3429357.3430522

DO - 10.1145/3429357.3430522

M3 - Conference contribution

AN - SCOPUS:85100496382

T3 - Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020

SP - 31

EP - 37

BT - Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020

PB - Association for Computing Machinery, Inc

Y2 - 7 December 2020 through 11 December 2020

ER -

Sánchez-Artigas M, Eizaguirre GT, Vernik G, Stuart L, Garciá-López P. Primula: A practical shuffle/sort operator for serverless computing. In Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020. Association for Computing Machinery, Inc. 2020. p. 31-37. (Middleware Industry 2020 - Proceedings of the 2020 21st International Middleware Conference Industrial Track, Part of Middleware 2020). doi: 10.1145/3429357.3430522

Primula: A practical shuffle/sort operator for serverless computing

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this