Overview of cluster

The Esrum cluster is a cluster managed by the Data Analytics Platform (formerly the Phenomics Platform) at CBMR. Hosting and technical support is handled by UCPH-IT.

In addition to the documentation provided here, UCPH-IT also provides documentation for the UCPH computing/HPC Systems on KUnet.

Architecture

The cluster consists of a head node, 12 compute nodes, 1 GPU / high-memory node, 3 GPU nodes, 2 RStudio web servers, and 1 server for running containers. A Shiny server server managed by UCPH-IT is also available.

Users connect to the "head" node, from which jobs can be submitted to the individual compute nodes using the Slurm Workload Manager:

Node

RAM

CPUs

GPUs

Name(s)

1

Head

2 TB

48 cores

esrumhead01fl

12

Compute

2 TB

64 cores

esrumcmpn01fl - esrumcmpn12fl

1

GPU / high-memory

4 TB

64 cores

2x NVIDIA A100 80GB

esrumgpun01fl

3

GPU

2 TB

64 cores

2x NVIDIA H100 80GB

esrumgpun02fl - esrumgpun04fl

2

Rstudio

2 TB

64 cores

esrumweb01fl, esrumweb02fl

1

Container*

2 TB

64 cores

esrumcont01fl

* The container node is used as a compute node when not being used to run containers.

Scratch space

Each node furthermore has at least 1.5 TB of local scratch space:

  • Most nodes have ~3 TB of local scratch

  • esrumcmpn07fl to esrumcmpn07fl each have ~1.5 TB of local scratch

  • esrumcmpn12fl has ~18 TB of local scratch

Software

The nodes all run Red Hat Enterprise Linux 8 and a range of scientific and other software is made available using environment modules. Missing software can be requested via UCPH-IT and/or data-analytics. See the Requesting software modules section for more information.

Projects and data-shares

Access is managed on a per-project level, and is administered by the individual project owners, with each project folder containing a standard set of sub-folders (apps, data, people, scratch).

Datasets used by several projects may be made available via read-only network shares. As with projects, access is administered by the data owner.

See the respective pages for accessing existing projects/data-shared and for creating new projects/data-shared.

Backup policies and quotas

Your /home folder and the apps. data, and people folders in projects are automatically backed up. The scratch folders are NOT backed up. The specific frequency and duration of backups differ for each type of folder and may also differ for individual projects.

As a rule, folders for projects involving GDPR protected data (indicated by the project name ending with -AUDIT) is subject to more frequent backups. However, on-site backups are kept for a shorter time to prevent the unauthorized recovery of intentionally deleted data.

See Data storage on Esrum for more information.

Security and risks

While UCPH-IT has performed a formal risk assessment of the Esrum HPC cluster, this assessment is not publicly available at the time of writing. A less formal and non-exhaustive overview of data security at Esrum can be found here, in English and Danish. This summary was authored by the Data Analytics Platform.

Note that you must be connected to the UCPH VPN to access this document. See the Connecting to the cluster page for information about using the VPN.

Additional resources