EGI Federation Home
Environmental Science

EISCAT_3D

Data portal and cloud infrastructure to study the Earth’s atmosphere

EISCAT is a non-profit scientific organisation that operates four radar antenna sites to enable research on the lower, middle and upper atmosphere and ionosphere using the incoherent scatter radar technique. Its research facilities are located in three main countries: Finland, Norway and Sweden, North of the Arctic circle.

The challenge

The EISCAT_3D radar system will provide a thirty-year update to EISCAT’s existing mainland facilities, exploiting the capabilities of state-of-the-art electronics, networking, storage and computing. In doing so, it will outstrip the capabilities of any such radar operating in the world and become the centrepiece of the international network of instruments monitoring the Earth’s upper atmosphere and space environment in the first half of the twenty-first century. The new system will open up new opportunities for physicists to explore a variety of research fields, but it comes with significant challenges in handling large-scale experimental data. The incoming data rates from all 60000 receivers amounting to 100Tb/s in total need to be reduced to a few PB/year to be within funding limits.

The EISCAT_3D research infrastructure has to be set up to make this data browsable and analysable for researchers, as well as archiving it for the long-term (it is planned to archive around 2PB per year). This primary mission is achieved by procuring sufficiently big storage and network capacities at a few data/compute centres (for back-up or load distribution). The data will be transferred to these locations after initial filtering and calibration at the data source(s). The data/compute centres are responsible for data archival, data curation, generation of derived data and for sharing the data with scientists. Data must be shared with scientists via a data access portal and programming APIs. 

The EISCAT Data Access Portal should offer the following main features:

Authentication and Authorization: Any researcher should be able to access the portal and browse metadata. The portal grants/denies access to data and processing based on affiliation.

Data browser: Authorised researchers should be able to browse reference applications in the portal, select an application for use, feed their data in, visualise or download the analysis result.

Data download: Authorised researchers should be able to select the EISCAT_3D data they are interested in for download.

Online computing: Authorised researchers should be able to analyse data without downloading them, using cloud resources, reference applications or user’s own software.

The solution

To support the development of a working prototype of the EISCAT Data Access Portal, the following EGI services have been used/investigated:

The EGI Workload Manager, is customized for the community requirements that provides searching, retrieval and reprocessing (visualisation, analysis) of data products generated by EISCAT radars.

The EGI Cloud Compute and the cloud-based EGI Online Storage to distribute the computational task to a scalable compute platform and to store intermediate results from the user jobs.

The EGI Check-In and Perun are integrated to enable user registration, authenticated and authorised access to the EISCAT Data Access portal and to the underlying distributed compute infrastructure.

Services provided by the EGI Federation

EGI Workload Manager

Manage computing workloads in an efficient way

EGI Cloud Compute

Run virtual machines on demand with complete control over computing resources

EGI Online Storage

Store, share and access your files and their metadata on a global scale

EGI Check-in

Login with your own credentials

Impact

12 members

6 associated member states, 6 affiliated member institutes, 3 continents

10TB of data per year

produced by three EISCAT radars 

3-level datasets

accessed by thousands users every year from all continents

50 publications

per year, in refereed journals, with multi-national contributions

Highlights

Publications

The list of publications submitted by the EISCAT Scientific Association can be found in the EISCAT repository. A total number of 69 publications were published during the period 2020-2021. 

Tutorial materials and courses

User documentation was made available and training has been delivered during EISCAT user forums. Training materials are available in the EISCAT repo.

Supporting projects

EOSC-hub

EOSC-hub brought together multiple service providers to create the Hub: a single contact point for...

Participating Partners

Explore the latest news

EISCAT_3D gets go ahead for construction

The construction of EISCAT_3D, a new international research radar for space weather research, has been...

A federation of cloud resources beyond Europe

Integration of China’s CSTCloud with the EGI Federation has been recently completed. CSTCloud is a...

New service level agreements: EISCAT, Binareé Oy, Cos4Cloud and C-SCALE

We are pleased to announce the following new service level agreements: EISCAT EISCAT_3D will be the...

New participants joining the EGI Federation in 2021

Exciting news! We are happy to announce that from the start of next year EISCAT...

4th ENVRI week

The 4th ENVRI week will take place in Grenoble, France, from the 15th to the...