Sponsored by BMBF Logo

Atomic Grid Jobs - the AstroGrid-D Use Case Dynamo


What many scientists demand from grid computing is simply this: A large number of compute nodes and an easy way to run many jobs on them. Access should be somewhat flexible, but still simple, and it should take less than a day to set up the package to run with a specified program.

The Dynamo scripts are designed to do exactly this. They show how a number of serial, independent calculations can be performed on D-Grid from within a one page shell script. It does this in three steps:
1. A program, together with corresponding input settings and data, is uploaded to a grid resource, e.g. a cluster node.
2. The binary is executed (a grid job "stage in"). This process is repeated as often as necessary, i.e. as many input data directories are found.
3. When the individual runs are finished, the output is transferred back to the submitting machine (a "stage out").

This very simple principle can be adapted to many scientific programs where a large number of input data or parameters must be processed. Understanding the given implementation of "dynamo" and then adapting the scripts to a different program can be done in little time, usually less than half a day. No matter if for data reduction, model fitting or other theoretical calculations: If the a larg number of runs is needed, for a specific program where the input changes but all runs are otherwise independent of each other, the "dynamo" script package will be a very suitable and fast solution with very little effort for the user.