Task Farming of Atomic Grid Jobs: Dynamo

Download (V1.2)	Documentation	Science Case	Flash Demo

Motivation

What many scientists demand from grid computing is simply this: A large number of compute nodes and an easy way to run many independend jobs on them. Access should be somewhat flexible, but still simple, and it should take less than a day to set up the package to run with a specified program.
The Dynamo scripts are designed to do exactly this. The package was originally an application for a magnetohydrodynamic simulation (see Science Case ), but it has been developed further so it can be generally used.

Execution Sequence

The package will start any number of serial, independent calculations on D-Grid from within an one page shell script.

A program, together with corresponding input settings and data, is uploaded to a grid resource, e.g. a cluster node.
The binary is executed (a grid job "stage in"). This process is repeated as often as necessary, i.e. as many input data directories are found. The script uses the Globus Toolkit 4 command globusrun-ws for this.
When the individual runs are finished, the output is transferred back to the submitting machine (a "stage out").

Using the package

This very simple principle can be adapted to many scientific programs where a large number of input data or parameters must be processed. Understanding the given implementation of "dynamo" and then adapting the scripts to a different program can be done in less than a day. Possible applications are data reduction, model fitting or other theoretical calculations: If the a large number of runs is needed for a specific program, where the input changes but all runs are otherwise independent of each other, the "dynamo" script package will be a suitable and fast solution that runs with limited effort for the user.

Realtime visualisation of a "Dynamo" run

To download the scripts and learn more, please follow the links at the top of the page.