The Virgo Scientific Collaboration is using the EGI Workload Manager

Alessandro Paolini reports on the new collaboration

Virgo is a giant laser interferometer designed to detect gravitational waves and located at the European Gravitational Observatory (EGO) site in Italy. Virgo is a collaboration between the French National Center for Scientific Research (CNRS) and the National Institute for Nuclear Physics (INFN) and is now operated by an international consortium.

The Virgo collaboration has demanding needs for storing and analysing a large amount of data recorded by the interferometer. A typical analysis run can last for about 6 months, and depending on the amount of data, hundreds of thousands of jobs can be submitted to the computing resources, with each job lasting anywhere between a few hours to a full day.

That is why Virgo chose to use the EGI Workload Manager – to easily dispatch jobs to computing resources and to manage the data necessary as input for the jobs or produced as output.

The EGI Workload Manager (also called DIRAC4EGI) is an EGI service based on the DIRAC technology and is suitable for users that need to exploit distributed resources in an optimised and transparent way. The types of resources that DIRAC can support include computing resources (grid, cloud, and batch systems), storage and catalog resources. The access to DIRAC4EGI, a multi-VO DIRAC server, is provided by EGI to the communities that don’t have enough resources for installing and managing an own dedicated server or are simply looking to try the functionalities of the tool.

Virgo is now performing tests using this instance. The fact that DIRAC is already used by many communities as a mature tool was an important factor in the decision.

In addition to the EGI Workload Manager, the Virgo collaboration also decided to test distributed data management solution to better understand its potential. For that, it was agreed to set-up a dedicated DIRAC file catalog component as well, hosted at the INFN data centre in Bologna, Italy.

The tests conducted so far show good performance results. For example, the catalog was populated with millions of records, and the performances were good even with a number of records similar to the real numbers that are expected to be in production. The tests also allowed to find and fix some misconfigurations on the resource centres currently available in France, Italy, and the Netherlands. In the following months, more sites will be involved and there are plans to move and register the production data between the sites, using the DIRAC data transfer feature.

To help the VIRGO community understand this technology, a winter school will be jointly organised by EGI, DIRAC and VIRGO in late 2018.

The school targets new VIRGO members, postdocs, and senior researchers and will provide courses on the following topics:

  • grid & cloud concepts and the EGI e-infrastructure
  • high-level solutions developed by INFN for provisioning, creating, managing and accessing pool of heterogeneous computing resources (DODAS – Dynamic On Demand Analysis Service)
  • use of the DIRAC system to handle user payloads running on any EGI grid- and cloud-based resources and on other computing resources.

The Virgo detector is located in Italy, within the site of the European Gravitational Observatory.

More information

The observation of gravitational waves by the Virgo and LIGO Scientific Collaborations paved the way to the Physics Nobel Prize awarded in 2017 to Rainer Weiss, Kip Thorne and Barry Barish.

Alessandro Paolini is Operations Officer at the EGI Foundation.