From UABgrid Documentation
Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click here.
Jupyter On Demand
As of 2019, UAB Research Computing allows access to cheaha via On Demand. To access.
1. Click On Demand
2. Select Interactive App and pick Jupyter Notebook
convert: unable to open file `/research/apps/rnb/docs/htdocs/tgw/images/5/56/JupyterNotebookStart.png' @ error/png.c/ReadPNGImage/3639.
convert: missing an image filename `/tmp/transform_1467976-1.png' @ error/convert.c/ConvertImageCommand/3015.
3. Load in Anaconda
module load Anaconda3/5.3.1
The following should also work for an updated version of Anaconda.
module load Anaconda3
4. If you require running on a GPU, please add the following to your environment.
module load cuda92/toolkit/9.2.88 module load CUDA/9.2.88-GCC-7.3.0-2.30
Additionally, you will need to request a GPU as shown below by including the pascalenodes argument:
convert: unable to open file `/research/apps/rnb/docs/htdocs/tgw/images/5/57/PascalNodes.png' @ error/png.c/ReadPNGImage/3639.
convert: missing an image filename `/tmp/transform_58d953-1.png' @ error/convert.c/ConvertImageCommand/3015.
5. Click Launch
convert: unable to open file `/research/apps/rnb/docs/htdocs/tgw/images/8/81/LaunchJupyter.png' @ error/png.c/ReadPNGImage/3639.
convert: missing an image filename `/tmp/transform_5b35c10-1.png' @ error/convert.c/ConvertImageCommand/3015.
Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).
6. Connect to Jupyter Notebook
7. Test Pytorch with a new notebook
convert: unable to open file `/research/apps/rnb/docs/htdocs/tgw/images/5/56/TestPytorch.png' @ error/png.c/ReadPNGImage/3639.
convert: missing an image filename `/tmp/transform_4576d1c-1.png' @ error/convert.c/ConvertImageCommand/3015.
Adding Custom Conda Environments to Jupyter
See Conda Cheet Sheet for some helpful Conda documentation. Additional info (some specific to cheaha) is below.
Preparing to build a conda env on cheaha
Do NOT build your new conda env on the head node, use an interactive job to create the new environment by using the command below.
srun --ntasks=1 --cpus-per-task=2 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
Set up your cheaha .condarc file (Cheahea)
This tells conda where to look for the installed environments. This file should be in your home directory on Cheaha (/data/user/username/.condarc). Make sure to include that '.' in the beginning of the filename.
channels: - defaults envs_dirs: - /data/user/username/python/env
Use the data directory rather than the home directory for your conda environment as it can get quite large. The file will be hidden and not visible in Jupyter environment. To view or edit it, start terminal and run 'nano ~/.condarc' or 'vi ~/.condarc'.
Build your conda environment
Once you are off the root node, you can start building your conda environment. conda create --name mynlp python=3.7
To update an existing environment:
conda env update --prefix ./scibert --file scibert.yml --prune
Add ipykernel to your conda environment
Add this conda module is what gets your environment to show up in Jupyter notebooks. Insert these lines in your .yml file. You can edit the file in Jupyter environment.
name: scibert channels: - defaults dependencies: - ipykernel=5.1.2 - _libgcc_mutex=0.1=main - alabaster=0.7.12=py37_0
Add other packages of interest
Add additional package of interest to your conda environment, for example "torch", "transformers", etc.. Pip can be used as well.
conda install torch
You can see your conda environment packages using
(Optional) Export YAML file containing your environment to cheaha
If 8you already have a functional conda environment elsewhere (like your laptop) you can export it to cheaha. The command below exports the "scibert" environment built on another machine. You can create any filename, but keep the extension .yml
conda env export --no-builds > scibert.yml
Copy the file to Cheaha. You can use "Upload" button in Jupyter
To replicate the new conda environment on cheaha execute the following commands:
conda env create --file scibert.yml --prefix /data/user/username/python/env/scibert
If you use --name argument instead to specify the environment name, conda will create a subdirectory with same name as the yml filename. That is confusing, so using --prefix makes the environment path cleaner
Load environment on Jupyter Notebook
Jupyter by Proxy
(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)
The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an interactive job.
One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, a Developers Command Prompt (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been tested, but does not work reliably on cheaha for proxying connections.
The Jupyter notebooks is built with Anaconda,a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own Python_Virtual_Environment just for that purpose.
1. Start the Jupyter Notebook
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash module load Anaconda3/5.2.0 unset XDG_RUNTIME_DIR jupyter notebook --no-browser --ip=$host
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.
2. Proxy Connection Locally
Now, start up a new tab/terminal/window on your client machine and relogin to cheaha, using
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
- c00XX is the compute node where you started the jupyter notebook, for example c0047
- 88XX is the port that the notebook is running, for example 8888
- For windows users, you can find instructions for port forwarding, here
3. Copy notebook URL
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
Copy/paste this URL into your browser when you connect for the first time, to login with a token: http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).
4. Access Notebook through Local Browser via Proxy Connection
Now access the link on your client machine browser locally using the link generated by jupyter notebook by substituting in localhost instead of c00XX. Make sure you have the correct port as well.
A Jupyter notebook should then open in your browser connected to the compute node.
DeepNLP option (development in progress)
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
Heavy Data IO option
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
Memory Heavy option
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
Finally, if your job requires a GPU then add the gres and partition arguments as shown below:
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash