Competence Centres: results in EGI-Engage

Gergely Sipos outlines the main achievements of the Competence Centres

EGI-Engage pioneered a new model of engagement and support for Research Infrastructures, based on distributed centres where national initiatives, user communities, technology and service providers join forces to collect and analyse requirements, integrate community-specific applications into state-of-the-art services, foster interoperability across e-Infrastructures, and evolve services through a user-centric development model.

We called them the Competence Centres – or CCs for short. EGI-Engage launched eight CCs 2.5 years ago and while these continue to operate beyond the project, I am pleased to report that the initiative was a success and we plan to take the CC model into the new EOSC-hub project, which is due to start in January 2018.

Here is a summary of what we achieved together with the research communities:

ELIXIR – Life sciences

The ELIXIR CC aimed at evaluating, adopting and promoting technologies and resources from EGI to the wider ELIXIR research community. The team collected representative life science use cases that could benefit from EGI services and then set up a federated cloud infrastructure combining the EGI Federated Cloud with ELIXIR cloud providers and with the ELIXIR Authentication and Authorisation system to implement those use cases. The results were:

  • The cBioPortal from CESNET is now ported to and hosted 24/7 to the CESNET cloud site.
  • The compute-intensive part of the META-Pipe metagenomics pipeline use case from CSC and Marine metagenomics use case from EMBL-EBI were successfully ported to the federated cloud resources.
  • Life scientists are now able to instantiate their own data analysis environment in the cloud (Insyght Comparative Genomics use case from CNRS IFB and PhenoMeNal project use case from EMBL-EBI).
  • Users in the US and in Europe can access the same tools and run them on local clouds. (JetStream interoperability use case from the US University of Indiana).

DARIAH – Arts and humanities

The goal of the DARIAH CC was to raise awareness of e-Infrastructures’ benefits. To achieve this the CC:

  • Established a VO to collect compute and storage resources for the DARIAH community provided by the EGI data centres and cloud sites.
  • Developed and deployed the DARIAH Science Gateway with applications (Simple Semantic Search Engine, Parallel Semantic Search Engine, and DBO@Cloud) and three services (Cloud Access, Workflow Development, and File Transfer), and enabled a federated login.
  • Established a Working Group within the DARIAH-ERIC community to provide advisory support and promote the benefits of using Cloud infrastructure and the DARIAH CC services beyond the time limits of the EGI-Engage project.
  • Coordinated participation and contribution to 15 external events to promote and disseminate the achievements.

MoBrain – Structural biology

The MoBrain CC aimed at lowering barriers for scientists to access online portals and tools for structural biology, building on the work of the WeNMR/ INSTRUCT and NeuGrid4You teams. The CC:

  • Implemented GPGPU-enabled web interfaces for the AMBER and DisVis online portals, providing an enhanced service by exploiting the faster performance of accelerated computing.
  • The Scipion cloud framework was deployed into the EGI Federated Cloud to allow researchers to obtain 3D maps of macromolecular complexes.
  • Continued to support a continuous robust use of HTC resources – the HADDOCK portal, for example, has been sending about 10 million jobs per year.
  • In collaboration with the INDIGO-DataCloud project the CC put into production two new web portals making use of the available grid GPGPU resources via Docker containers: DisVis (114 registered users) and PowerFit (79 registered users).

BBMRI – Biobanking

The BBMRI CC was set up to develop and pilot data processing workflows for sensitive personal data. The work resulted in:

  • Expansion of the BiobankCloud platform with the authentication and authorization mechanisms to allow integration with common AAIs (e.g., BBMRI-ERIC AAI, EGI Check-in).
  • A demonstrator was ported to the private cluster of the MMCI hospital (in the Czech Republic), where the data analysis workflow has been performed on data of real patients.
  • The selection of the biobank workflows most suited as use cases (from CZ, NL, SE).

LifeWatch – Biodiversity sciences

The CC’s goal was to assess and implement requirements of LifeWatch research communities for e-infrastructure services. Throughout the project, the CC:

  • Integrated pattern recognition tools and data flow handlers with the IFCA cloud site.
  • Compiled a LifeWatch service catalogue of 16 services covering support for ecological observatories, workflows, virtual labs and citizen science.
  • Supported the deployment of services via Federated Cloud resources and used 5.5 million CPU hours during the project.

EISCAT_3D – Ionosphere and atmosphere observatory

The CC worked on the development of the EISCAT_3D user portal backed by EGI federated HTC and cloud services. This portal will provide scientists with services to discover, access and analyse (e.g. visualise, mine) data generated by EISCAT_3D.
The EISCAT_3D portal has a working access control and interfaces for data discovery and download as well as a function for analysis job submissions. Moreover, the system facilitated the development of data models and modelling tools within the EISCAT_3D community, and the applicability of operating a central portal service for scientists to interact and compute with EISCAT data.

EPOS – Earth sciences

The CC collected, analysed and compared community needs with EGI technical offerings, resulting in three pilots:

  • AAI: demonstrated interoperability between the EPOS AAI and the EGI Check-in service; the prototype was developed based on the UNITY IDM technology and interfaced with Check-in.
  • Earthquake simulation (MISFIT): showed how an existing seismology application can be improved by integration with the EGI Federated Cloud.
  • Satellite Data: set up an environment with EGI to support the development of new services for satellite data processing. The pilot deployed an EPOS service on top of the Geohazard Thematic Exploitation Platform by Terradue from the satellite data TCS, and linked it to the EGI Federated Cloud to exploit its computing and storage resources.

Disaster mitigation

The CC worked to develop customised IT services to support climate and disaster mitigation researchers in Asia and produced:

  • Two web portals to simulate tsunami wave propagation (iCOMCOT) and weather conditions (WRF).
  • The two portals provide stand-alone and ease-of-use simulation tools for the entire lifecycle of a tsunami event and numerical weather prediction.
More information

Gergely Sipos is the EGI Foundation Customer and Technical Outreach Manager.

CCs technical details, milestones and deliverables