EGI Federation Home

Notebook integration with DIRAC is now in production

Updated 14/11/2022

Written by Yin Chen and Enol Fernandez

Notebook and Workload manager services are two popular EGI services used by many scientific communities. Notebook provides an interactive data analysis interface relying on EGI Cloud and storage. On the other hand, when computation becomes large-scale, the DIRAC-based Workload Manager becomes useful for load balancing and maximising the usage of computational resources. 

New requirements were raised by EISCAT Competence Center when they integrated both services to build a data access portal; in particular, they wanted the Notebook to directly access the DIRAC client toolkits, allowing EISCAT users to retrieve data from DIRAC storage and manage job submission without leaving the Notebook environment. 

The identified solution foresees using CVMFS to provide DIRAC client components via POSIX interface. The integration solution has been tested by EISCAT CC in their newly released EISCAT Data Access Portal

Today, we are pleased to announce this integration of Notebook with DIRAC-based Workload manager is in production. As it is generic, any community with similar requirements can consider its adoption. 

This was a collaborative effort involving EGI Technical Solutions, CESNET and DIRAC teams. We would also give special thanks to EISCAT CC who lead the testing – all these valuable community requirements help advance the technology evolution in EGI.

Find out more on the EGI Services

Notebooks

Create interactive documents with live code, visualisations and text

Workload Manager

Manage computing workloads in an efficient wayCreate interactive documents with live code, visualisations and text