About
EGI has set up a collaboration with a group of SMEs (RHEA, SixSq, EOproc) to develop a demonstrator for an open integrated platform that delivers a multi-cloud EO data processing service.
Challenge
The main goal was to prevent cloud vendor lock-in situations while addressing end-to-end processing of EO data in multiple clouds environments. The service was intended to work with any EO data located anywhere and with any EO processor that can be rapidly deployed with flexibility and at a low cost, and which provides services accessed via public and standardised APIs.
Impact
The multi-cloud EO data processing platform has been demonstrated with the EGI Federated Cloud, the Innovation Platform Testbed (IPT) Poland, and the Amazon Web Services (AWS) cloud. The platform presents the following capabilities:
- Multi-cloud data discovery: a data discovery service allows discovering and selecting EO scenes relevant to an EO application. It uses a metadata catalogue currently tracking nine satellites including Landsat-8, Sentinel-1 and 2, and able to hold up to 10+ millions of scenes.
- Multi-cloud data management and access: a data management service provides a data location API that allows exposing and staging EO data on the cloud infrastructure where the EO application is deployed and executed. It uses a global high-performance data management system that provides access to distributed storage resources.
- Multi-cloud application deployment: an application deployment service allows selecting the cloud infrastructure on which to deploy the EO application, and provides automated deployment, execution and monitoring. It uses a multi-cloud application management service automating the full application management lifecycle, and providing connectors to multiple clouds, public and private, leveraging both open source and proprietary cloud APIs.