Data and cloud federations: EGI, EOSC and GAIA-X case studies

In a recently published article, Tiziana Ferrari and Mark Dietrich put their magnifying lens on the architecture and governance of some of the major European federated initiatives: EGI, EOSC, and GAIA-X

The EGI Federation started in 2004 and has been growing ever since. The EGI Federated Cloud was launched in 2014 and is now the largest research cloud federation in Europe, providing distributed access to data and offering scientific tools that facilitate data computing.

A popular computing platform, the EGI Federated Cloud has seen its use doubled in 2020 and 2021. A constantly growing number of providers participates by providing service provisioning agreements with research communities. Joining the federation allows these providers to easily support international research communities in their distributed data processing and data  analytics tasks, while keeping complete control of the allocation of their cloud services and resources.

The EGI Cloud federation builds on a standard-based software stack capable of interoperating  with a wide range of cloud-native tools and platforms, including Kubernetes for container execution and IaaS orchestrators (e.g. Infrastructure Manager and Terraform), to deploy and run their applications with minimal or no modifications. Besides the common IaaS features for  managing computing  resources as virtual machines, EGI cloud enables portability with  features such as resource discovery to programmatically locate the best provider for the execution of users workloads, and a central Virtual Machine catalogue. The automated  distribution of images so users can find their custom software ready to be used at the providers  without worrying about the low level details.

Federation initiatives have successfully demonstrated that data sharing in distributed research infrastructures produces great benefits for the members of the federation. These initiatives facilitate collaboration, from development of networks and communities of practice, to creating technology platforms and formal business agreements. Among all, EOSC and GAIA-X have been emerging as important new federated initiatives respectively targeting the research and industrial community. In recent years, discussion has turned to the opportunities and challenges associated with exchanging data between collaborating parties.

In the recently published article ‘Governance, Architectures and Business Models for Data and Cloud Federations: the EOSC and GAIA-X Case Studies’ (https://zenodo.org/record/4929021) we compare the technical architecture and organizational models of these initiatives and propose the following recommendations to maximize collaboration and effective data sharing across sectors:

  • Recommendation 1: Data sharing and protection. Federated cloud and data initiatives will benefit from sharing good practices on cross-domain/organisation data sharing policies and FAIR best practices that enable data exploitation. Cross-domain and cross-initiative use cases offer the opportunity to collaborate on data protection and GDPR compliance aspects, a common concern in data sharing.
  • Recommendation 2: Compliance and federated governance. To ensure compatibility of data reuse across sectors and initiatives, it is important to adopt compatible models for data ecosystem business models, federation architectures and related roles, and service integration and delivery models including concerted approaches to develop resource and service registries. Existing and new federated infrastructures will succeed to the extent that their respective stakeholders can gain benefits from federation participation. Initiatives should collaborate to study federation architectures and standards to define and implement the related enterprise systems.
  • Recommendation 3. Technical interoperability. Federated initiatives will be able to more quickly advance their technical implementation plans by joining efforts on technical and operational interoperability frameworks that make data exchanges between providers possible.

These three recommendations are meant to improve collaboration across organizational boundaries, countries and continents. These are essential to innovation, productivity and discovery. Collaboration in research has been encouraged for decades, and international collaboration is the hallmark of key scientific efforts.

Likewise, collaboration in business has been identified as critical to superior business performance and sustainability, offsetting natural incentives to compete. Collaboration and integration in sectors such as mobility, energy, health, agriculture, finance, and sustainable development have been identified by the European Commission as essential to enabling improved efficiency and effectiveness in those sectors while also enabling achievement of the EU’s policies on reduction of greenhouse gas emissions, and increased resource sustainability.

More information

Recommended reading:

The Potential of Cloud Federation, Horizon Cloud Project (2020), DOI:

10.5281/zenodo.4733565 (https://zenodo.org/record/4733565).

Find out more about the EGI Federated Cloud.