The EGI DataHub service available in the EGI catalogue

Andrea Manzi introduces a new EGI service

We spoke to Andrea Manzi, Data Solutions Manager at the EGI Foundation, about the EGI DataHub service. Andrea tells us that he is proud to announce that DataHub has been successfully included in the EGI catalogue of services. We asked Andrea to share more about the service.

The EGI DataHub prototyping phase has started back in 2018 using the Onedata technology with the aim to support the following use cases:

  • Bringing data close to the computing to exploit it efficiently.
  • Publishing a dataset and make it available to a specific community or worldwide across federated sites.

The service has been successfully integrated with the EGI AAI (Checkin) service allowing users to make their data available using different levels of access: from completely unrestricted open access to open data to authenticated access to closed data sets.

The data hosted on the EGI DataHub can be readily accessible by cloud Virtual Machines (VMs), running grid jobs or Jupyter notebooks thanks to full integration with EGI Federated Cloud, High-Throughput compute resources and the EGI Notebooks service. The use of standard protocols and web services guarantees easy and scalable access to data from cloud, HTC applications and Jupyter. This ensures maximum compatibility with existing applications and minimum hassle for developers and users alike.

The service delivers metadata handling, and the possibility to register Handle and PID systems to allow assigning Persistent Identifiers to data, thus implementing FAIR principles. DataHub is managed following the FitSM service management principles and it’s integrated in the EGI ecosystem (Helpdesk, Monitoring).

A number of pilots have been set up over the last period by different communities to exploit the service:

Users can access the service for evaluation using a testing area (PLAYGROUND shared space), or request support to publish their data and have dedicated storage assigned.

The service is funded by the EGI-ACE project, EGI Foundation and Cyfronet.

More information

EGI DataHub

Andrea Manzi is Data Solutions Manager at the EGI Foundation.