BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks

Efficient asset management is essential for optimizing the performance and scalability of modern Big Data (BD) frameworks. However, traditional resource allocation methods often suffer from static partitioning, inefficient resource utilization, and high operational costs, limiting their ability to a...

ver descrição completa

Detalhes bibliográficos
Autores: VázquezCaderno, Pablo, Awaysheh, Feras, Cabaleiro Domínguez, José Carlos, Fernández Pena, Anselmo Tomás
Formato: artículo
Fecha de publicación:2025
País:España
Recursos:Universidad de Santiago de Compostela (USC)
Repositorio:Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Idioma:inglés
OAI Identifier:oai:minerva.usc.gal:10347/42844
Acesso em linha:https://hdl.handle.net/10347/42844
Access Level:acceso abierto
Palavra-chave:DB
Apache Spark
Opportunistic scheduling
Dynamic resource provisioning
Resource allocation
Elastic computing
Green computing
id ES_9b9a201c6d2a1a3dbd2ad2c95f5462ec
oai_identifier_str oai:minerva.usc.gal:10347/42844
network_acronym_str ES
network_name_str España
repository_id_str
spelling BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworksVázquezCaderno, PabloAwaysheh, FerasCabaleiro Domínguez, José CarlosFernández Pena, Anselmo TomásDBApache SparkOpportunistic schedulingDynamic resource provisioningResource allocationElastic computingGreen computingEfficient asset management is essential for optimizing the performance and scalability of modern Big Data (BD) frameworks. However, traditional resource allocation methods often suffer from static partitioning, inefficient resource utilization, and high operational costs, limiting their ability to adapt to fluctuating workloads dynamically. This paper introduces BigOPERA, an opportunistic and elastic resource allocation framework designed to enhance BD processing environments by integrating dedicated and non-dedicated computing assets. Leveraging containerization and a two-tiered scheduling mechanism, BigOPERA dynamically manages available resources to improve workload execution efficiency. Experimental results demonstrate that BigOPERA achieves up to 35% performance improvement over native Apache Spark configurations, significantly enhancing computational throughput while optimizing resource consumption. Our findings highlight the potential of BigOPERA in scalable, cost-effective, and sustainable BD processing.Springer NatureUniversidade de Santiago de Compostela. Centro de Investigación en Tecnoloxías Intelixentes da USC (CiTIUS)Universidade de Santiago de Compostela. Departamento de Electrónica e Computación20252025-07-1620252025-07-16journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/10347/42844reponame:Minerva. Repositorio Institucional de la Universidad de Santiago de Compostelainstname:Universidad de Santiago de Compostela (USC)InglésengAgencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2022-141623NB-I00open accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:minerva.usc.gal:10347/428442026-06-15T12:47:27Z
dc.title.none.fl_str_mv BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks
title BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks
spellingShingle BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks
VázquezCaderno, Pablo
DB
Apache Spark
Opportunistic scheduling
Dynamic resource provisioning
Resource allocation
Elastic computing
Green computing
title_short BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks
title_full BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks
title_fullStr BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks
title_full_unstemmed BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks
title_sort BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks
dc.creator.none.fl_str_mv VázquezCaderno, Pablo
Awaysheh, Feras
Cabaleiro Domínguez, José Carlos
Fernández Pena, Anselmo Tomás
author VázquezCaderno, Pablo
author_facet VázquezCaderno, Pablo
Awaysheh, Feras
Cabaleiro Domínguez, José Carlos
Fernández Pena, Anselmo Tomás
author_role author
author2 Awaysheh, Feras
Cabaleiro Domínguez, José Carlos
Fernández Pena, Anselmo Tomás
author2_role author
author
author
dc.contributor.none.fl_str_mv Universidade de Santiago de Compostela. Centro de Investigación en Tecnoloxías Intelixentes da USC (CiTIUS)
Universidade de Santiago de Compostela. Departamento de Electrónica e Computación
dc.subject.none.fl_str_mv DB
Apache Spark
Opportunistic scheduling
Dynamic resource provisioning
Resource allocation
Elastic computing
Green computing
topic DB
Apache Spark
Opportunistic scheduling
Dynamic resource provisioning
Resource allocation
Elastic computing
Green computing
description Efficient asset management is essential for optimizing the performance and scalability of modern Big Data (BD) frameworks. However, traditional resource allocation methods often suffer from static partitioning, inefficient resource utilization, and high operational costs, limiting their ability to adapt to fluctuating workloads dynamically. This paper introduces BigOPERA, an opportunistic and elastic resource allocation framework designed to enhance BD processing environments by integrating dedicated and non-dedicated computing assets. Leveraging containerization and a two-tiered scheduling mechanism, BigOPERA dynamically manages available resources to improve workload execution efficiency. Experimental results demonstrate that BigOPERA achieves up to 35% performance improvement over native Apache Spark configurations, significantly enhancing computational throughput while optimizing resource consumption. Our findings highlight the potential of BigOPERA in scalable, cost-effective, and sustainable BD processing.
publishDate 2025
dc.date.none.fl_str_mv 2025
2025-07-16
2025
2025-07-16
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/10347/42844
url https://hdl.handle.net/10347/42844
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Agencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2022-141623NB-I00
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Springer Nature
publisher.none.fl_str_mv Springer Nature
dc.source.none.fl_str_mv reponame:Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
instname:Universidad de Santiago de Compostela (USC)
instname_str Universidad de Santiago de Compostela (USC)
reponame_str Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
collection Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869414551843241984
score 15.228081

Registos relacionados