Jupyter

Jupyter (jupyter.org) is the language agnostic framework for managing code “notebooks” of cells, mixed code, text, and graphics. The Juypter project was spun out of IPython’s “Notebook” feature and, while supporting a variety of languages, is primarily associated with Python and IPython’s interactive interface. Jupyter is available in many of the most recent python modules on SCC.

Jupyter documentation:

 

Jupyter Initial Setup

The following section describes the one-time creation of the Jupyter configuration file and password setup to secure your notebooks. Some python versions will require this, but it is a good idea to configure the password in any version when using a multi-user system like the Shared Computing Cluster.

  1. Load a recent python module
    scc1% module load python/2.7.13
  2. Generate the Jupyter configuration file
    scc1% jupyter notebook --generate-config
    Writing default config to: ~/.jupyter/jupyter_notebook_config.py
  3. Use Python to create an encrypted password hash and exit Python.
    scc1% python
    Python 2.7.13 (default, Feb  8 2017, 12:51:33)
    [GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from notebook.auth import passwd; passwd()
    Enter password: 
    Verify password: 
    'Sha1:1243ab45678c:1a234b5678c901de234567f890g123hij4567890'
    >>> exit()
  4. Add the hashed password to your Jupyter configuration file.
    Open the Jupyter configuration file ~/.jupyter/jupyter_notebook_config.py in your preferred text editor and edit the c.NotebookApp.password option (line 217). You will need to first uncomment the line by removing the preceding “#” symbol and then add your encrypted password hash as the value.The original line:

    #c.NotebookApp.password = u''

    Should be modified to reflect your new password hash:

    c.NotebookApp.password = u'Sha1:1243ab45678c:1a234b5678c901de234567f890g123hij4567890'

Jupyter Launch and Display Through X-Forwarding

To launch Jupyter notebook and use a web-browser on SCC, load Python and execute the jupyter command with the notebook subcommand. This will start the Jupyter server and open a Firefox browser automatically.

Note: The Firefox browser display will run through X-Forwarding and can sometimes render slowly. The computation will happen at full speed, but a better graphical experience can be configured using port forwarding or through VNC.
  1. Load Firefox and Python modules
    scc1% module load python/2.7.13
  2. Start Jupyter Notebook
    scc1% jupyter notebook
    [I 16:19:41.896 NotebookApp] Serving notebooks from local directory: ~
    [I 16:19:41.896 NotebookApp] 0 active kernels 
    [I 16:19:41.896 NotebookApp] The Jupyter Notebook is running at: http://localhost:8892/?token=bb55926e6659feabf5706d93b95
    [I 16:19:41.896 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
    [C 16:19:41.899 NotebookApp] 
        
        Copy/paste this URL into your browser when you connect for the first time,
        to login with a token:
            http://localhost:8892/?token=bb55926e6659feabf5706d93b95
    
    [I 16:19:43.378 NotebookApp] Accepting one-time-token-authenticated connection from 127.0.0.1
  3. Firefox will open automatically.
    jupyter
  4. To terminate the notebook, use the Control-C key combination and confirm that you wish to shutdown the notebook.
    ^C
    [I 16:20:02.532 NotebookApp] interrupted
    Serving notebooks from local directory: ~
    0 active kernels 
    The Jupyter Notebook is running at: http://localhost:8892/?token=bb55926e6659feabf592
    Shutdown this notebook server (y/[n])? y
    
    [C 16:20:05.184 NotebookApp] Shutdown confirmed
    [I 16:20:05.186 NotebookApp] Shutting down kernels

Jupyter Remote Access

When running Jupyter Notebook on the SCC, the Firefox browser is also automatically started to interact with your notebooks. If working remotely or on a slow internet connection, applications with graphical user interfaces can sometimes render slowly.  The computation will happen at full speed, but a better graphical experience can be configured using either VNC or port forwarding.

VNC

Connecting to the SCC through a VNC session requires some additional setup, but will usually resolve any graphical user interface related issues. Once connected through VNC, you can follow the instructions in the previous section inside the VNC session.

Port Forwarding

With some additional setup, it is possible start the Jupyter server on the SCC, forward the connection to a local system, and connect using a local browser for a better browser experience without VNC. Here we forgo loading firefox on the SCC and instruct Jupyter to start without a browser by executing the jupyter command with the notebook subcommand and --no-browser option. This will start the Jupyter server on SCC and provide information about the ports that need to be forwarded to your local system.

scc1% module load python/2.7.13
scc1% jupyter notebook --no-browser
[I 09:41:51.909 NotebookApp] The port 8888 is already in use, trying another port.
[I 09:41:51.909 NotebookApp] The port 8889 is already in use, trying another port.
[I 09:41:51.909 NotebookApp] The port 8890 is already in use, trying another port.
[I 09:41:51.910 NotebookApp] The port 8891 is already in use, trying another port.
[I 09:41:51.949 NotebookApp] Serving notebooks from local directory: ~
[I 09:41:51.950 NotebookApp] 0 active kernels 
[I 09:41:51.950 NotebookApp] The Jupyter Notebook is running at: http://localhost:8892/
[I 09:41:51.950 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

Unlike the X-Forwarding instructions, this process does not start firefox on SCC and instead provides the information needed to forward the port to a local system. Pay special attention to the line stating “The Jupyter Notebook is running at: http://localhost:8892/”, the port number (8892 in this example) will be different for each session. We need to open a new connection from your local system to the SCC login node running the jupyter server (scc1, scc2, geo, or scc4) with port forwarding to bring this port (8892) to a port on local system (we use 7070 in this example).

localhost% ssh userID@scc1.bu.edu -L 7070:localhost:8892

With the port forwarded from SCC to you local system (7070 in this example), you can open any browser on your local computer to “localhost:7070”.

jupyter_portforward