EDGI introduces new tool to submit 10,000 jobs to the grid
The European Desktop Grid Initiative (EDGI) project successfully demonstrated the use of the new MetaJob feature at the EGI User Forum in Vilnius. MetaJob enables users to submit jobs to the EDGI Desktop Grid through gLite using one single command and can be used to wrap or describe thousands of jobs into one single MetaJob.
“Desktop Grids are especially useful for applications like parameter sweeps where you have thousands and thousands of similar jobs,” explains Ad Emmen, from AlmereGrid. “MetaJob is a command line ‘script’ that helps an EGI user to create these jobs for his/her application, and submit them automatically through standard gLite interfaces.”
The efficiency of MetaJob was demonstrated with great success at the EDGI booth. During the day, the EDGI team submitted 10,000 jobs several times to show the EGI User Forum visitors how this can easily be processed by BOINC, without disrupting the gLite infrastructure.
For Jozsef Kovacs, from the MTA SZTAKI (Computer and Automation Research Institute, Hungarian Academy of Sciences), MetaJob makes it easier to handle a high number of jobs. MetaJob also allows “to increase the scalability of the service grid to desktop grid bridging services deployed by the EDGI project by decreasing the load of the service grid components – for example gLite wms,” he adds.
From a technical point of view the solution is based on the well-known Service Cluster Grid to Desktop Grid bridge, where an extra MetaJob des-cription file (containing the definition of a huge number of jobs) is attached to the job as input file. This MetaJob input file is recognised by the 3GBridge component, which then creates the jobs and inserts them into BOINC. This solution also works with XtremWeb on the Desktop Grid side or ARC on the Service/Cluster Grid side without any modification.
“This successful EDGI demonstration clearly shows that Desktop Grids can be used effortlessly for scientific day-to-day calculations,” said Peter Kacsuk from the EDGI project. “The relevance of Desktop Grids for e-Science has been proven once and for all.”
