A survey on the distributed computing stack
In this paper, we review the background and the state of the art of the Distributed Computing software stack. We aim to provide the readers with a comprehensive overview of this area by supplying a detailed big-picture of the latest technologies. First, we introduce the general background of Distrib...
| Authors: | , , , , , |
|---|---|
| Format: | article |
| Publication Date: | 2021 |
| Country: | España |
| Institution: | Universitat Politècnica de Catalunya (UPC) |
| Repository: | UPCommons. Portal del coneixement obert de la UPC |
| Language: | English |
| OAI Identifier: | oai:upcommons.upc.edu:2117/362356 |
| Online Access: | https://hdl.handle.net/2117/362356 https://dx.doi.org/10.1016/j.cosrev.2021.100422 |
| Access Level: | Open access |
| Keyword: | Electronic data processing -- Distributed processing Distributed systems Distributed programming models Distributed computing Cloud computing Task-based Workflows Dataflows Graph processing Streaming Data sharing Resource management Infrastructure managers Processament distribuït de dades Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors::Arquitectures distribuïdes |
| Summary: | In this paper, we review the background and the state of the art of the Distributed Computing software stack. We aim to provide the readers with a comprehensive overview of this area by supplying a detailed big-picture of the latest technologies. First, we introduce the general background of Distributed Computing and propose a layered top–bottom classification of the latest available software. Next, we focus on each abstraction layer, i.e. Application Development (including Task-based Workflows, Dataflows, and Graph Processing), Platform (including Data Sharing and Resource Management), Communication (including Remote Invocation, Message Passing, and Message Queuing), and Infrastructure (including Batch and Interactive systems). For each layer, we give a general background, discuss its technical challenges, review the latest programming languages, programming models, frameworks, libraries, and tools, and provide a summary table comparing the features of each alternative. Finally, we conclude this survey with a discussion of open problems and future directions. |
|---|