Workflow allocations and scheduling on IaaS platforms, from theory to practice

Workflow allocations and scheduling on IaaS platforms, from theory to practice

Par Adrian Muresan
Date : vendredi 12 octobre 2012 11h40
Durée : 45 minutes
Langue : English
Tags : budget cloud workflows

The workflow application pattern is very common across scientific domains. In a traditional grid environment, scientist have to compete with each
other for compute resources which often leads to over-provisioning and under-use of resources. In addition, workflow applications have a non-constant demand of resources and therefore in a traditional environment involving static resource allocations, a part of the resources will not be used. This is an inefficient use of resources.

The current work addresses the problem of resource allocation for non-deterministic workflows under budget constraints. This talk will present an approach for using on-demand resource allocations for scientific workflow applications. With on-demand resources we can
satisfy the application usage needs without over-provisioning which leads to a higher usage efficiency. We present a way of transforming the initial problem into sub-problems that have been studied before. We propose two new allocation algorithms that are capable of determining resource allocations under budget constraints and we present ways of using them to address the problem at hand.

We have tested our approach by using DIET (MADag)
as a workflow engine, Nimbus as an IaaS resource provider and a real-world scientific workflow application called RAMSES, used for n-body simulations of dark-matter particles that represent the backbone of galaxies.