Abstract
Unexpectedly, the rise of serverless computing has also collaterally started the “democratization” of massive-scale data parallelism. This new trend heralded by PyWren pursues to enable untrained users to execute single-machine code in the cloud at massive scale through platforms like AWS Lambda. Inspired by this vision, this industry paper presents IBM-PyWren, which continues the pioneering work begun by PyWren in this field. It must be noted that IBM-PyWren is not, however, just a mere reimplementation of PyWren’s API atop IBM Cloud Functions. Rather, it is must be viewed as an advanced extension of PyWren to run broader MapReduce jobs. We describe the design, innovative features (API extensions, data discovering & partitioning, composability, etc.) and performance of IBM-PyWren, along with the challenges encountered during its implementation.
| Original language | English |
|---|---|
| Title of host publication | Middleware Industry 2018 - Proceedings of the 2018 ACM/IFIP/USENIX Middleware Conference (Industrial Track) |
| Publisher | Association for Computing Machinery, Inc |
| Pages | 1-8 |
| Number of pages | 8 |
| ISBN (Electronic) | 9781450360166 |
| DOIs | |
| State | Published - 10 Dec 2018 |
| Externally published | Yes |
| Event | 19th ACM IFIP USENIX Middleware Conference, Middleware 2018 - Rennes, France Duration: 10 Dec 2018 → 14 Dec 2018 |
Publication series
| Name | Middleware Industry 2018 - Proceedings of the 2018 ACM/IFIP/USENIX Middleware Conference (Industrial Track) |
|---|
Conference
| Conference | 19th ACM IFIP USENIX Middleware Conference, Middleware 2018 |
|---|---|
| Country/Territory | France |
| City | Rennes |
| Period | 10/12/18 → 14/12/18 |
Bibliographical note
Publisher Copyright:© 2018 Association for Computing Machinery.
Keywords
- Distributed computing
- IBM cloud functions
- IBM cloud object storage
- PyWren
- Serverless computing
ASJC Scopus subject areas
- Information Systems
- Software
