Jupyter Notebooks¶

Jupyter Notebooks are available via the module system on Esrum and can be run on regular compute nodes or on the GPU/high-memory node, depending on the kind of analyses you wish to run and the size of your workload.

By default, Jupyter only includes support for Python notebooks, but instructions are included below for how to add support for R.

Note

We are currently working on making Jupyter available along with the Troubleshooting. We will announce when this service is ready.

Starting a Jupyter notebook¶

To start a notebook on a node, run the following commands:

$ module load jupyter-notebook
$ srun --pty -- jupyter notebook --no-browser --ip=0.0.0.0 --port=XXXXX

The number used in the argument --port=XXXXX must be a value in the range 49152 to 65535, and must not be a number used by another user on Esrum. The number shown here was randomly selected for you, and you can refresh this page for a different suggestion.

Warning

A random port could not be selected, because Javascript is disabled. Please choose a number in the range 49152 to 65535, and use this number whenever the documentation says XXXXX

This will allocate a single CPU and ~16 GB of RAM to your notebook. If you need additional resources for your notebook, then please see the Reserving resources section for instructions on how to reserve additional CPUs and RAM, and the GPU / high-memory jobs page for instructions on how to reserve GPUs or large amounts of memory. The srun accepts the same options as sbatch.

Tip

It is recommended that you execute the srun command in a tmux or screen session, to avoid the notebook shutting down if you lose connection to the head node. See Persistent sessions with tmux for more information.

Connecting to the Jupyter Notebook¶

To connect to the notebook server, you will first need to set up a connection from your PC to the compute node on which your notebook is running. This is called "port forwarding" and is described on the Port forwarding page.

However, to do so you must first determine on which compute node your job is running. This can be done in a couple of ways:

Look for the URLs printed by Jupyter when you started it on Esrum:

To access the notebook, open this file in a browser:
    file:///home/abc123/.local/share/jupyter/runtime/nbserver-2082873-open.html
  Or copy and paste one of these URLs:
      http://esrumcmpn07fl.unicph.domain:XXXXX/?token=0123456789abcdefghijklmnopqrstuvwxyz
   or http://127.0.0.1:XXXXX/?token=0123456789abcdefghijklmnopqrstuvwxyz

In this example, our notebook is running on the esrumcmpn07fl node. All Esrum node names end with .unicph.domain, but we do not need to include this part of the name.

Alternatively, run the following command on the head node in a separate terminal:
```
$ squeue --me --name jupyter
JOBID  PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
551600 standardq  jupyter   abc123  R       8:49      1 esrumcmpn07fl
```
By looking in the NODELIST column, we can see that the notebook is running on esrumcmpn07fl, as above.

Once you've determined what node your notebook is running on, go to the Port forwarding page and setup port forwarding to that node and the port you used when starting Jupyter (e.g. XXXXX).

Adding an R kernel to Jupyter¶

The Jupyter module only comes with a Python kernel. If you instead wish to use R in your Jupyter notebook, you can add an R Kernel for the specific version of R that you wish to use.

To do so, run the following commands, replacing R/4.3.3 with the version of R that you wish to use:

$ module load jupyter-notebook/6.5.4
$ module load --auto R/4.3.3
$ R
> install.packages('IRkernel')
> name <- paste("ir", gsub("\\.", "", getRversion()), sep="")
> displayname <- paste("R", getRversion())
> IRkernel::installspec(name=name, displayname=displayname)
> quit(save="no")

This will make an R kernel with the name R 4.3.3 available in Jupyter. You can repeat these commands for each version of R that you wish to make available as a kernel. Run the command module purge between each, to ensure that you have loaded only the expected version of R and gcc that R depends on.

Once you are done adding R versions, you start notebook as shown above:

$ module load jupyter-notebook/6.5.4
$ srun --pty -- jupyter notebook --no-browser --port=XXXXX

While you do not need to load the R module first, if you only wish to run R code, you must do so if you wish to install R libraries via the notebook:

$ module load jupyter-notebook/6.5.4
$ module load --auto R/4.3.3
$ srun --pty -- jupyter notebook --no-browser --port=XXXXX

Troubleshooting¶

Jupyter Notebooks: Browser error when opening URL¶

Depending on your browser you may receive one of the following errors. The typical causes are listed, but the exact error message will depend on your browser. It is therefore helpful to review all possible causes listed here.

When using Chrome, the cause is typically listed below the line that says "This site can't be reached".

The connection was reset

This typically indicates that Jupyter Notebook isn't running on the server, or that it is running on a different port than the one you've forwarded. Check that Jupyter Notebook is running and make sure that your forwarded ports match those used by Jupyter Notebook on Esrum.
Localhost refused to connect or Unable to connect

This typically indicates that port forwarding isn't active, or that you have entered the wrong port number in your browser. Therefore,
- Verify that port forwarding is active: On OSX/Linux that means verifying that an ssh command is running as described in the Port forwarding for OSX/Linux users section, and on Windows that means activating port forwarding in MobaXterm as described in the Port forwarding for Windows users section.
- If using the instructions for Linux/OSX, verify that you ran the ssh command on your laptop or desktop, and not on the Esrum head node.
- Verify that either of these are using the same port number as in the jupyter command you ran or as in the http://127.0.0.1 URL printed by Jupyter.
- Verify that you are using the second URL that Jupyter prints on the terminal, namely the URL starting with http://127.0.0.1:XXXX:
```
To access the notebook, open this file in a browser:
    file:///home/abc123/.local/share/jupyter/runtime/nbserver-2082873-open.html
    Or copy and paste one of these URLs:
        http://esrumcmpn07fl.unicph.domain:XXXXX/?token=0123456789abcdefghijklmnopqrstuvwxyz
    or http://127.0.0.1:XXXXX/?token=0123456789abcdefghijklmnopqrstuvwxyz
```
  For security reasons it is not possible to connect directly to the compute nodes.