Making data and cloud resources interoperable using EUDAT and EGI services

Sara Garavelli on the close collaboration between EUDAT an EGI

From 22 to 25 January 2018, Porto hosted over 230 participants of EUDAT’s Conference “Putting the EOSC vision into practice”. Attendees included policy makers, service providers and research communities representatives from 25 countries working on various data challenges and disciplines.

The conference was opened by Augusto Burgueño Arjona, Head of the eInfrastructure Unit of the Directorate General for Communications Networks, Content and Technology (DG CONNECT), who presented EOSC as an open science instrument supporting the collaboration between e-Infrastructures & research infrastructures: “EOSC has to be an inclusive ecosystem where horizontal and thematic service providers work together to meet the user needs”.

The discussion on how to put the EOSC vision into practice was addressed with a set of breakout sessions themed around the creation of a thriving data economy. The topics approached were in the range of: interoperability of services, the role of research infrastructures as thematic service providers, business models and sustainability of data infrastructures, legal issues.

One of the sessions was dedicated to the results accomplished by the collaboration between EGI and EUDAT for the implementation of a production cross-infrastructure offering seamless access to data and high-throughput computing resources. The work of the collaboration involved concrete user communities in the design process. This helped both EGI and EUDAT to better shape their services so as to match real needs of their users.

Two major use cases were brought in by the ICOS and ENES research communities.

The ICOS use case focused on the new web-based service offered on the ICOS Carbon Portal to perform 3-dimensional Stochastic Time-Inverted Lagrangian Transport (STILT) atmospheric transport model calculations. The input data consists of meteorological air transport data (from ECMWF), data on greenhouse gas emissions (from EDGAR), and atmospheric observations (from ICOS and other sources). The output data shows time series of concentrations of greenhouse gases and their resulting footprints at selected locations. The data was successfully handled via a combination of EUDAT B2STAGE and B2SAFE services and other network file management systems, while the production model was visualised using EGI on-demand computing services.

The ENES use case addressed the volume increase of the climate data archive by employing the EUDAT General Execution Framework (GEF) Workflow API, in combination with EUDAT B2 services, and interfacing with the EGI Federated Cloud. Post-processing results (e.g. data on carbon gas emissions) are sent back to be displayed and further processed at the IS-ENES platform. They can also be downloaded in different common data formats as tailored products via a simple website interface. The input data – typically Coupled Model Intercomparison Project phase 5/6 (CMIP5/CMIP6) data – is now being downloaded locally by climate impact researchers and makes room for a more sustainable data workflow.

To conclude, the session at the EUDAT conference was a great opportunity to present examples of the collaboration between EGI & EUDAT and between generic and thematic service providers in their effort to support researchers’ needs.

More information

Sara Garavelli is the Outreach Manager of the EUDAT Collaborative Data Infrastructure (CDI).