The eXtreme-DataCloud project: data management for eXtreme scale computing

The eXtreme DataCloud (XDC) project was a 3-year Horizon 2020 project which finished in April 2020 and its aims were to develop and release advanced data management services within current and future e-Infrastructures in Europe including the EGI Federation, the European Open Science Cloud (EOSC) and the Worldwide LHC Computing Grid (WLCG).

XDC ensured that its outputs were continuously validated against the scientific use cases driving the project by including the use cases as part of the project: Lifewatch, European Clinical Research infrastructure Network (ECRIN), Cherenkov Telescope Array (CTA), The Worldwide LHC Computing Grid (WLCG) and the European XFEL. In doing this, XDC ensured that its services work at scale against the most demanding, data-intensive experiments across Europe and worldwide.

The XDC project built on the data aspects of the previous successful Horizon 2020  INDIGO-DataCloud project which laid the groundwork by developing a comprehensive platform and tools to meet the cloud, storage and network challenges faced by scientific researchers. XDC extended this framework by further improving federated data management services by identifying and adding missing functionalities, enhancing the user experience in these services within the backdrop of highly dynamic modern infrastructures.

Primary outputs of the project were two consolidated software releases containing the components involved of the project, containing the newly developed functionality: XDC-1 (codename: Pulsar) and XDC-2 (codename: Quasar). The components included: CachingOnDemand, dCache, Dynafed, EOSC, FTS, Onedata, PaaS Orchestrator and the PaaS Orchestrator Dashboard, Orchent, RUCIO, TOSCA types and templates plugin and XDC-http-cache.

EGI contributed to XDC over the course of the project via its user support and dissemination activities and on exploitation of the project output with piloting activities with external use cases, the distribution of XDC software via the Unified Middleware Distribution (UMD) as well as pre-production testing using Staged Rollout and coordination of service providers.

For more information please see:

XDC video presentation

XDC Service Catalogue