Virtualization / Cloud Abstractions in SimGrid

1. MSG VM API

Contact
Takahiro Hirofuchi, Adrien Lebre
URL (C API)
http://simgrid.gforge.inria.fr/simgrid/3.11/doc/group__msg__VMs.html
URL (JAVA API)
http://simgrid.gforge.inria.fr/simgrid/3.11/doc/javadoc/index.html
Description

The MSG VM API allows users to launch hundreds of thousands of VMs on their simulation programs and control VMs in the same manner as in the real world (e.g., suspend/resume and migrate).

Users can execute computation and communication tasks on physical machines (PMs) and VMs through the same SimGrid API. It includes a live migration model implementing the precopy migration algorithm. This model correctly calculates the migration time as well as the migration traffic, taking account of resource contention caused by other computations and data exchanges within the whole system. This allows user to obtain accurate results of dynamic virtualized systems.

From the viewpoint of the SimGrid APIs, a VM object (msg_vm_t ) supports the same API as a PM object; for example, you can create a process on a VM through the same API, MSG_process_create().

But, a VM object also supports VM-specific API, such as suspend/resume and migration.

  void example(msg_host_t pm) {
    /* 0. Create a VM (named VM0) on a PM. */
    msg_vm_t vm = MSG_vm_create_core(pm, "VM0");
    MSG_vm_start(vm);

    /* 1. Launch a process on the VM. */
    msg_process_t pr = MSG_process_create("worker", worker_main, NULL, vm);

    /* 2. Suspend the VM for 10 seconds. */
    MSG_vm_suspend(vm);
    MSG_process_sleep(10);
    MSG_vm_resume(vm);

    /* 3.  Migrate the VM to another PM. */
    MSG_vm_migrate(vm, pm1);

    /* 4. Clean up. */
    MSG_process_kill(pr);
    MSG_vm_shutdown(vm);
    MSG_vm_destroy(vm);
  }

There is some example code using the MSG VM API under the ./examples/msg/cloud directory.

examples/msg/cloud/master_worker_vm.c
Perform master/worker jobs on VMs. Good for the first tutorial.
examples/msg/cloud/simple_vm.c
Test computation and communication tasks on a VM.
examples/msg/cloud/migrate_vm.c
Perform live migrations of VMs.
examples/msg/cloud/bound.c
Limit the CPU usage of a VM and a computation task.
examples/msg/cloud/multicore.c
Set the CPU affinity of a VM on a multicore PM.
examples/msg/cloud/scale.c
Test scalability.

Similar tests are available for the JAVA bindings, please see the ./examples/java/cloud directory.

2. VMPlaceS - VM load Injector

Contact
Adrien Lebre
URL (JAVA)
https://github.com/BeyondTheClouds/VMPlaces
Description
A dedicated framework to evaluate and compare VM placement algorithms.

At coarse-grained, the framework is composed of two major components: the injector and the VM placement algorithm. The injector is the generic part of the framework (i.e. the one you can directly use) while the VM placement algorithm is the part you want to study (or compare with available algorithms).

The injector makes load changes of VMs during a predefined time: Every \(t\) seconds, the injector selects one VM and changes its CPU load according to a Gaussian distribution. \(t\) is a random variable that follows an exponential distribution with rate parameter \(\lambda\). The Gaussian distribution is defined by a mean (\(\mu\)) as well as a standard deviation (\(\sigma\)) that are given at the beginning of each simulation. The parameters of the simulation are defined in the Simulator.properties file. By default, the duration of each experiment is set to 3600 seconds. The parameters are \(\lambda=\mathit{Nb\_VMs}/300\) and \(\mu=70\), \(\sigma=30\). Concretely, the load of each VM starts from 0% and varies on average every 5 minutes in steps of 10 (with a significant part between 40% and 100% of CPU usage).

At the beginning, the simulation create \(n\) VMs, each of which is based on one of predefined VM classes. A VM class is a template of the specification of a VM and its workload. It is described as nb_cpu:ramsize:net_bw:mig_speed:mem_speed. VMs are launched on PMs in a round-robin manner, i.e., each PM has almost the same number of VMs. According to the investigated algorithm, VMs are relocated through the different PMs during the whole execution of the simulation.

The ultimate objective of the SimGrid Injector VM framework is to deliver a building block that will enable to compare fairly different VM placement algorithms. A complete research report should be available by January 2015.

3. SchIaaS

Contact
Julien Gossa / Luke Bertot
URL (JAVA)
http://schiaas.gforge.inria.fr/
Description
Semi-abstract and extensible CloudManager supporting:
  • Run, terminate, suspend, resume and describe instances;
  • Description of available resources;
  • Image and instance types management;
  • Automatic VM placement on the clusters;
  • Boot and other VM life-cycle processes;
  • Cloud storage management.

The purpose of SchIaaS is to provide a common interface to any cloud engine build upon SimGrid, so that cloud application simulations can be ported to different cloud simulator with very few adaptation efforts.

schiaas.png

3.1. RICE: Reduces Implementation of Compute Engine (Engine 1 - everything except billing)

Contact
Julien Gossa / Luke Bertot
URL
http://schiaas.gforge.inria.fr/
Description
RICE is designed to be as simple and extensible as possible. It adds the strict necessary operations only, like image storage and transfer, or VM scheduling over physical host, and keeps everything extensible. Thus, it is the good entry point to simple cloud simulation, simple VM scheduling algorithm simulation, or building more complex cloud engines.

3.2. RISE: Reduces Implementation of Storage Engine (Engine 1 - everything except billing)

Contact
Julien Gossa / Luke Bertot
URL
http://schiaas.gforge.inria.fr/
Description
RISE is designed to be as simple and extensible as possible. It adds the strict necessary operations only, like data storage and transfer, and keeps eveything extensible. Thus, it is the good entry point to simple storage support, or building more complex storage engines.

4. SGCB

Contact
Jonathan Rouzaud-Cornabas (jonathan.rouzaud-cornabas@inria.fr)
URL
https://gforge.inria.fr/project/admin/?group_id=4532
Research report
http://hal.inria.fr/hal-00909120
Simulation blocks
compute, storage (block and key-value), accounting and billing
Description
SimGrid Cloud Broker provides a simulation of a Cloud and multiple Clouds as seen by a Cloud User. As most of the interactions between a Cloud and a User is done through an API interface, SGCB reproduces the same approach. Moreover, as AWS is the most well known and used Cloud and its related APIs (EC2 and S3) are defacto standards, SGCB replicates them to ease the transition between SGCB and AWS or other Clouds providing the APIs. Finally, even if at the moment most of Cloud users only host their applications in one Cloud or one Region of AWS, the trend quickly evolves to multi-Clouds and multi-Regions applications mainly due to availability reasons. Therefore, our simulator is able to present multiple Clouds and AWS regions to the user. Thanks to an easy interface to the simulator, we are able to improve it over time and add new features and new models.