Jupyter Notebooks#

Jupyter Notebooks are available via the module system on Esrum and can be run on regular compute nodes or on the GPU/high-memory node, depending on the kind of analyses you wish to run and the size of your workload:

To start a notebook on a node, run the following commands:

$ module load jupyter-notebook
$ srun --pty -- jupyter notebook --no-browser --port=XXXXX

By default, this will allocate a single CPU and ~16 GB of RAM to your notebook. Please see the Reserving resources section for instructions on how to reserve additional resources and the Using the GPU / high-memory nodes page for instructions on how to reserve GPU / high-memory resources. The srun accepts the same options as sbatch.

To connect to the notebook server, you will first need to set up a connection from your PC to the node where your notebook is running. This is called "port forwarding" and is described below.

Tip

See the Running jobs using Slurm pages for information about how to reserve CPUs, memory, and/or GPUs when using srun. The srun command takes the same command-line options as sbatch, as described here.

Tip

It is recommended that you execute the above commands in a tmux or screen session, to avoid the notebook shutting down if you lose connection to the server. See Persistent sessions with tmux for more information.

Port forwarding in Windows (MobaXterm)#

The following instructions assume that you are using MobaXterm. If not, then please refer to the documentation for your tool of choice.

  1. Run the following command on the head node to determine on which node which your notebook is running:

    $ squeue --me --name jupyter
JOBID  PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
551600 standardq  jupyter   abc123  R       8:49      1 esrumcmpn07fl

    In this example the notebook is running on esrumcmpn07fl, but your notebook may be running on any of the nodes on Esrum!

  2. Install and configure MobaXterm as described in Configuring MobaXterm.

  3. Click the middle Tunneling button on the toolbar.

    ../_images/mobaxterm_tunnel_01.png

  4. Click the bottom-left New SSH Tunnel button.

    ../_images/mobaxterm_tunnel_02.png

  5. Follow these steps to configure the tunnel:

    ../_images/mobaxterm_tunnel_03.png

    1. In middle-left box, write your chosen port number (e.g. XXXXX) where the screenshot shows 12345.

    2. In the top-right pair of boxes, replace localhost with the name of the node where your notebook is running (this was esrumcmpn07fl in the example above, but your notebook will likely be running on a different node), and replace 12345 with your chosen port number (e.g. XXXXX).

    3. In the middle-right trio of boxes, write the full name of the head node (esrumhead01fl.unicph.domain), write your UCPH username where the screenshot has abc123, and set the final value to 22.

    4. Finally, click Save.

  6. If the tunnel does not start automatically, press either the "Play" button or the Start all tunnels button:

    ../_images/mobaxterm_tunnel_04.png

  7. Enter your password and your SSH tunnel should now be active.

Once you have configured MobaXterm and enabled port forwarding, you can open your notebook via the http://localhost:XXXXX/?token=${long_text_here} URL that Jupyter Notebook printed in your terminal.

Port forwarding on Linux/OSX#

It is recommended to enable port forwarding using your ~/.ssh/config file. This is accomplished by adding a LocalForward line to your entry for Esrum as shown below (see also the section about Connecting on Linux):

Host esrum esrumhead01fl esrumhead01fl.unicph.domain
    HostName esrumhead01fl.unicph.domain
    User abc123

    LocalForward XXXXX localhost:XXXXX

The LocalForward option consists of two parts: The port used by the notebook on Esrum (XXXXX), and the address via which the notebook on Esrum should be accessible on your PC (localhost:XXXXX).

Alternatively, you can start/stop port forwarding on demand by using an explicit SSH command. The -N option is optional and stops ssh from starting a shell once it has connected to Esrum:

$ ssh -S none -N -L XXXXX:localhost:XXXXX abc123@esrumhead01fl.unicph.domain

Once you have port forwarding is enabled, you can open your notebook via the http://localhost:XXXXX/?token=${long_text_here} URL that Jupyter Notebook printed in your terminal.

Note

The -S none option is recommended in case shared connections are enabled (see the ControlMaster section in man ssh), in which case the ssh command may otherwise not open the specified ports if a connection already exists.

Troubleshooting#

Jupyter Notebooks: Browser error when opening URL#

Depending on your browser you may receive one of the following errors. The typical causes are listed, but the exact error message will depend on your browser. It is therefore helpful to review all possible causes listed here.

When using Chrome, the cause is typically listed below the line that says "This site can't be reached".

  • "The connection was reset"

    This typically indicates that Jupyter Notebook isn't running on the server, or that it is running on a different port than the one you've forwarded. Check that Jupyter Notebook is running and make sure that your forwarded ports match those used by Jupyter Notebook on Esrum.

  • "Localhost refused to connect" or "Unable to connect"

    This typically indicates that port forwarding isn't active, or that you have entered the wrong port number in your browser. Verify that port forwarding is active and that you are using the correct port number in the localhost URL.

  • "Check if there is a typo in esrumweb01fl" or "We're having trouble finding that site"

    You are most likely connecting from a network outside UCPH. Make sure that you are using a wired connection at CBMR and/or that the VPN is activated and try again.