New EGI services: Notebooks and Workload Manager

Yin Chen and Enol Fernández introduce the new services

Notebooks

Notebooks is a new service introduced as beta in the EGI service catalogue. Notebooks provides an interactive data analysis environment based on the Jupyter system, running on top of the EGI Federated Cloud & using Check-in for authentication and authorisation.

Users of the Notebooks service can create and share documents that combine text, mathematics, computations and their rich media output to perform any kind of research task, e.g. data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more on the federated EGI resources. Individual users can use a catch-all instance of the service with limited resources and communities can request the setup of specific instance for their needs.

The service is operated by the EGI Foundation and runs on resources provided by INFN-Catania and CESGA cloud sites. A number of communities, such as AGINFRA+, EPOS, OpenDreamKit and D4Science, and individual users are testing the service during this beta phase and will be moved to full production during 2019.

Workload Manager

Workload Manager was recently included in the EGI service catalogue also as a beta service and is based on DIRAC technology. Workload Manager has a user-friendly interface and allows easy extensions for the needs of specific applications via APIs.

Workload Manager is suitable for users who need to exploit distributed resources in a transparent way. The service optimises the distribution of their computing tasks among the available resources, both HTC and Cloud.

The service is coordinated by the EGI Foundation and operated by IN2P3 on resources provided by CYFRONET. A number of EGI communities have already adopted the Workload Manager service, including WeNMR, OPENCoastS, EISCAT-3D and VIRGO. For example, the WeNMR community uses Workload Manager for a number of their tools, and reported an improvement from previous 70% to 99% with DIRAC job submission.

More information:

Yin Chen is Senior Technical Outreach Expert at the EGI Foundation.

Enol Fernández is Cloud Technologist at the EGI Foundation.

Notebooks

Workload Manager