The SimGrid Contrib Section

1. A contrib section? Why?

Many people have come to ask us for more complex examples than the simple ones that are distributed with the main branch of the SimGrid project. We do not want to include complex examples in the main branch but we think it is a good idea that users share their experience. That is why we have created a contrib branch that can be used by people who have a nice piece of work using SimGrid and accept to share it. The license of this code should be LGPL if possible. If you can't license your code with LGPL, just tell us about it and we will try to sort out whether it makes sense to publish it here.

2. Projects that facilitates SimGrid usability

2.1. Visualizing traces with R

Since a few years, we have realized that develop ping and maintaining GUIS to analyze simulations was very time and resource consuming. This is why we have invested on a much more agile approach leveraging modern data analytics tools. We recommand to have a look at the page describing how to leverage R to analyze SimGrid traces.

2.2. TRIVA/VIVA

VIVA (formerly TRIVA) is an open-source tool used to analyze traces (in the pajé format) registered during the execution of parallel applications. It is mainly developped by Lucas Mello Schnorr. The tool serves also as a sandbox to the development of new visualization techniques. Some features include:

  • Temporal integration using dynamic time-intervals
  • Spatial aggregation through hierarchical traces
  • Scalable visual analysis with Squarified Treemaps
  • A Custom Graph Visualization

SimGrid includes a tracing system capable to generate traces that can be easily visualized using Triva. It is so since version 3.4 (and works seemly since version 3.6). It is very useful when you to understand why and how your simulation behaves, so let's have a look to the vids on TRIVA website.

2.3. SimGrid Eclipse Plug-in

The SimGrid Eclipse Plug-in helps new users to get a first step in SimGrid simulation by automatically generating files needed in order to run a simulation using SimGrid.

One of its main strength is the platform editor that allows easy visualization and editing of platform description files, that can also be appreciated by advanced users.

Main Features:

  • Helps you to create a new SimGrid project with a new project wizard
  • Graphic editing of platform description files
  • Support for SimGrid native C and SimGrid-Java bindings

More information, download, install and doc are available on the SimGrid Eclipse Plug-In Website

2.4. Jedule

Jedule is yet another Gantt chart viewer. It is an open-source tool developped by Sascha Hunold and is available on sourceforge. It is used to analyze traces (in an XML format) registered during the execution of parallel applications. It is more oriented toward the visualisation of parallel jobs than other Gantt chart tools such as Paje or ViTE.

SimGrid includes a tracing system capable to generate traces that can be easily visualized using Jedule. For now it can be activated only with the SimDAG API with functions like jedule_sd_init, jedule_log_sd_event, jedule_sd_dump, …

2.5. ATLAS-NDGF workload generator

The generator creates synthetic grid workloads according to the data model built for the ATLAS experiment at LHC at CERN (or more specifically, for a fraction of the total ATLAS workload that is processed on the resources of the Nordic DataGrid Facility). Detailed information about the model can be found on the web site, or in the SuperComputing 2012 publication titled "ATLAS grid workload on NDGF resources: analysis, modeling, and workload generation". In particular, the model puts emphasis on data requirements by jobs and captures a variety of correlations, e.g., between the size of input and output files or between job execution times and size of the input data.

The generator produces a scalable-size workload with any given number of jobs while trying to retain adherence to the distributions in the model. It is written in Python and able to produce a workload with many thousand jobs within a few seconds on a typical desktop computer. The generator is able to produce workloads in the DAX format, support for which is embedded in SimGrid, and thus can be used as input supplier to SimGrid-based simulation.

The description of the generator, download link and instructions of how to use it with SimGrid are available here.

2.6. Topo5K

Topo5k is a command line tool to retrieve Grid'5000 topology and possibly export it as a SimGrid platform file.

3. Projects relying on MSG

3.1. Volunteer Computing

SimBOINC

SimBOINC is a simulator for heterogeneous and volatile desktop grids and volunteer computing systems. The goal of this project is to provide a simulator by which to test new scheduling strategies in BOINC, and other desktop and volunteer systems, in general.

It is contributed by Derrick Kondo and can be found here: http://simboinc.gforge.inria.fr/ Unfortunately, it relies on an old version of the boinc client and an old version of SimGrid and requires significant work to use.

SimGrid-BOINC

SimGrid-BOINC is a simulator developped originally by Bruno Donassolo during his master with Arnaud Legrand. Unlike SimBOINC, it does not use the BOINC client simulator but a simpler version that is quite faithful. It has grounded the following publications:

  • Bruno Donassolo, Henri Casanova, Arnaud Legrand, and Pedro Velho. Fast and Scalable Simulation of Volunteer Computing Systems Using SimGrid. In Proceedings of the Workshop on Large-Scale System and Application Performance (LSAP), Chicago, IL, June 2010.
  • Bruno Donassolo, Arnaud Legrand, and Claudio Geyer. *Non-Cooperative Scheduling Considered Harmful in Collaborative Volunteer Computing Environments*. In Proceedings of the 11th IEEE International Symposium on Cluster Computing and the Grid (CCGrid'11), May 2011. IEEE Computer Society Press. [PDF]
  • Lucas Mello Schnorr, Arnaud Legrand, and Jean-Marc Vincent. Detection and analysis of resource usage anomalies in large distributed systems through multi-scale visualization. Concurrency and Computation: Practice and Experience, 2011. [WWW]

More recently Duco Van Amstel has developped several models of the server part to explore different designs.

It is available at the following git repository: 

    git://scm.gforge.inria.fr/simgrid/simgrid-boinc.git

or 

    git+ssh://scm.gforge.inria.fr/gitroot//simgrid/simgrid-boinc.git

3.2. Peer to Peer

If you're into P2P, you may have heard of the PeerSim Simulator. The API it provides makes it particularly suited to test/design new P2P protocols. The underlying network models are however simpler than those available in SimGrid. Some colleagues who wanted some more in-depth understanding of the network resource usage have thus developed an interface in Java that allows users to simulate and execute their code under PeerSim or Simgrid, using PeerSim implementation policy.

It is available here: https://gforge.inria.fr/projects/psg/

3.3. Grid and Batch Scheduling

Grid Matrix

Grid Matrix is an application that lets you create your grid network and simulate the execution of distributed applications from a Graphic User Interface (GUI).

  • Features:
  • Complete design of the network.
  • Bindings for MSG.
  • Simulation run within the GUI.
  • Pause / Stop simulation.
  • Multiple trace options.
  • Python output and errors redirected to the GUI.
  • Grid Nodes change their red intensity to show activity.
  • Custom activity can be set by Python scripts.
  • Create graphics from scripts.
  • Requirements:
  • Microsoft Visual Studio VS2005 SP1 runtime
  • Future works:
  • Support other APIs in SimGrid.
  • Add a form to set process parameters.
  • Linux support.

This is contributed by Pablo Yabo, and can be found here: http://research.nektra.com/Grid_Matrix

GarSim

GarSim is a simulator of a scheduling meta-system or just of a batch system. This code is written in C++ and has many dependencies (with python, sqlite, some parts of condor, …).

It used to be maintained by Vincent Garonne (garonne::lal.in2p3.fr), and is part of the SimGrid SVN (directory contrib/garSim). If you need to access it, just send a mail on the simgrid-user mailing list.

Simbatch

This program simulates the behavior of a batch scheduler managing a cluster.

This program is part of the SimGrid SVN (directory contrib/Simbatch) and is not maintained anymore by its author. If you need to access it, just send a mail on the simgrid-user mailing list.

3.4. MapReduce

MRSG

Ever dreamt of studying Hadoop/MapReduce applications using SimGrid ? This project from Wagner Kolberg is for you: https://github.com/MRSG/MRSG

This project seems to have lead to at least three forks from Julio Anjos:

4. Projects related to SMPI

4.1. Calibration Procedure

To obtain faithful predictions with SMPI, the performance of both the network and the MPI layer need to be calibrated. This set of links describe how we compute the SMPI parameters based on a set of point-to-point MPI measurements. The R code produces the XML snippet to put in the simgrid XML file.

  • The procedure to calibrate point-to-point MPI communications is described here. Originally, the R script was described here and the article first building on it was published at PMBS/SC'13.
  • The description of the saturation procedure required to model the internals of the cluster topology is ongoing. A first draft is available here.

4.2. Integration with Paraver

The SimGrid team is currently working with colleagues from the BSC to see how SMPI can help when analyzing MPI codes within the paraver framework. The description of the current status of this work is described here.

5. Projects related to Virtualization and Cloud Computing Concerns

Please give a look at Cloud Contribs page that describes the MSG VM API, VMPlacS, SchIaaS and SGCB.

5.1. AMALTHEA-SimGrid

AMALTHEA-SimGrid allows users to easily specify mappings for cloud applications described using the AMALTHEA model and to assess the quality for these mappings. The two quality metrics provided by the framework are execution time and energy consumption.

AMALTHEA-SimGrid has been developed at the LIRMM lab (Montpellier) in the context of the DreamCloud European project and can be downloaded here. A description of the simulator has been published in the DreamCloud 2016 workshop collocated with Hipeac conference. The paper can be found here.