Jupyter - Revision history

Mmoo97@uab.edu: /* Build your conda environment */

2020-09-24T13:13:33Z

Build your conda environment

← Older revision		Revision as of 13:13, 24 September 2020
Line 68:		Line 68:
	== Build your conda environment ==		== Build your conda environment ==
	Once you are off the root node, you can start building your conda environment.		Once you are off the root node, you can start building your conda environment.
			<pre>
	conda create --name mynlp python=3.7		conda create --name mynlp python=3.7
			</pre>

	To update an existing environment:		To update an existing environment:

Mmoo97@uab.edu: /* Set up your cheaha .condarc file (Cheahea) */

2020-09-24T13:12:38Z

Set up your cheaha .condarc file (Cheahea)

← Older revision		Revision as of 13:12, 24 September 2020
Line 55:		Line 55:
	</pre>		</pre>

	== Set up your cheaha .condarc file (~~Cheahea~~) ==		== Set up your cheaha .condarc file (Cheaha) ==
	This tells conda where to look for the installed environments. This file should be in your home directory on Cheaha (/data/user/username/.condarc). Make sure to include that '.' in the beginning of the filename.		This tells conda where to look for the installed environments. This file should be in your home directory on Cheaha (/data/user/username/.condarc). Make sure to include that '.' in the beginning of the filename.
	<pre>		<pre>

Ozborn@uab.edu: More information on custom building conda environments on cheahea

2020-08-28T19:03:02Z

More information on custom building conda environments on cheahea

← Older revision		Revision as of 19:03, 28 August 2020
Line 46:		Line 46:

	= Adding Custom Conda Environments to Jupyter =		= Adding Custom Conda Environments to Jupyter =
	== ~~Export YAML file containing your environment~~ to cheaha ==
	~~Wherever~~ your ~~working environment is~~, ~~export it~~ to cheaha. ~~Below exports~~ the ~~scibert environment which I set up~~ on ~~a different server~~. ~~You~~ can ~~create any filename~~, ~~but keep~~ the ~~extension~~ .~~yml~~		See [https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf Conda Cheet Sheet] for some helpful Conda documentation. Additional info (some specific to cheaha) is below.

			== Preparing to build a conda env on cheaha==
			Do NOT build your new conda env on the head node, use an interactive job to create the new environment by using the command below.
			<pre>
			srun --ntasks=1 --cpus-per-task=2 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
			</pre>

			== Set up your cheaha .condarc file (Cheahea) ==
			This tells conda where to look for the installed environments. This file should be in your home directory on Cheaha (/data/user/username/.condarc). Make sure to include that '.' in the beginning of the filename.
			<pre>
			channels:
			- defaults
			envs_dirs:
			- /data/user/username/python/env
			</pre>
			Use the data directory rather than the home directory for your conda environment as it can get quite large.
			The file will be hidden and not visible in Jupyter environment. To view or edit it, start terminal and run 'nano ~/.condarc' or 'vi ~/.condarc'.

			== Build your conda environment ==
			Once you are off the root node, you can start building your conda environment.
			conda create --name mynlp python=3.7

			To update an existing environment:
	<pre>		<pre>
	conda env ~~export~~ --no-~~builds >~~ scibert.yml		conda env update --prefix ./scibert --file scibert.yml --prune
	</pre>		</pre>
	~~Copy the file to Cheaha. You can use "Upload" button in Jupyter~~

	== Add ipykernel to your conda environment ==		== Add ipykernel to your conda environment ==
Line 65:		Line 87:
	</pre>		</pre>

	== ~~Build your conda environment~~ ==		== Add other packages of interest ==
	~~Do NOT run it on the head node, use an interactive job~~ to ~~create the new~~ environment.		Add additional package of interest to your conda environment, for example "torch", "transformers", etc.. Pip can be used as well.
	<pre>		<pre>
	~~srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash~~		conda install torch
	</pre>		</pre>

	~~To build a new~~ environment:		You can see your conda environment packages using
	<pre>		<pre>
	conda ~~env create --file scibert.yml --prefix /data/user/username/python/env/scibert~~		conda list
	</pre>		</pre>
	If you use --name argument instead to specify the environment name, conda will create a subdirectory with same name as the yml filename. That is confusing, so using --prefix makes the environment path cleaner

	~~TO update an existing~~ environment:
			== (Optional) Export YAML file containing your environment to cheaha ==
			If 8you already have a functional conda environment elsewhere (like your laptop) you can export it to cheaha. The command below exports the "scibert" environment built on another machine. You can create any filename, but keep the extension .yml
	<pre>		<pre>
	conda env ~~update~~ --~~prefix ./scibert~~ -~~-file~~ scibert.yml ~~--prune~~		conda env export --no-builds > scibert.yml
	</pre>		</pre>
			Copy the file to Cheaha. You can use "Upload" button in Jupyter

	~~== Set up your cheaha .condarc file ==~~		To replicate the new conda environment on cheaha execute the following commands:
	~~This tells~~ conda ~~where to look for the installed environments. This file should be in your home directory~~ on ~~Cheaha (/data/user/username/.condarc). Make sure to include that '.' in~~ the ~~beginning of the filename.~~		<pre>
	<pre>		conda env create --file scibert.yml --prefix /data/user/username/python/env/scibert
	~~channels:~~
	- ~~defaults~~
	~~envs_dirs:~~
	- /data/user/username/python/env
	</pre>		</pre>
	~~Use~~ the ~~data directory rather than the home directory for your~~ conda ~~environment~~ as ~~it can get quite large~~.		If you use --name argument instead to specify the environment name, conda will create a subdirectory with same name as the yml filename. That is confusing, so using --prefix makes the environment path cleaner
	~~The file will be hidden and not visible in Jupyter~~ environment~~. To view or edit it, start terminal and run 'nano ~/.condarc'.~~

	== Load environment on Jupyter Notebook ==		== Load environment on Jupyter Notebook ==

Louistw@uab.edu at 17:07, 20 April 2020

2020-04-20T17:07:43Z

← Older revision		Revision as of 17:07, 20 April 2020
Line 5:		Line 5:
	As of 2019, UAB Research Computing allows access to cheaha via [https://rc.uab.edu On Demand]. To access.		As of 2019, UAB Research Computing allows access to cheaha via [https://rc.uab.edu On Demand]. To access.

	== 1. Click [~~http~~://rc.uab.edu On Demand] ==		== 1. Click [https://rc.uab.edu On Demand] ==

Louistw@uab.edu: update ondemand link from http to https

2020-04-20T17:07:03Z

update ondemand link from http to https

← Older revision		Revision as of 17:07, 20 April 2020
Line 3:		Line 3:
	= Jupyter On Demand =		= Jupyter On Demand =

	As of 2019, UAB Research Computing allows access to cheaha via [~~http~~://rc.uab.edu On Demand]. To access.		As of 2019, UAB Research Computing allows access to cheaha via [https://rc.uab.edu On Demand]. To access.

	== 1. Click [http://rc.uab.edu On Demand] ==		== 1. Click [http://rc.uab.edu On Demand] ==

Azotov@uab.edu at 20:03, 10 February 2020

2020-02-10T20:03:06Z

← Older revision		Revision as of 20:03, 10 February 2020
Line 51:		Line 51:
	conda env export --no-builds > scibert.yml		conda env export --no-builds > scibert.yml
	</pre>		</pre>
			Copy the file to Cheaha. You can use "Upload" button in Jupyter
	~~== Set up your cheaha .condarc file ==~~
	~~This~~ file ~~should be in your home directory on cheaha~~.
	~~<pre>~~
	~~channels:~~
	~~- defaults~~
	~~envs_dirs:~~
	~~- /data/user/ozborn/Conda_Env~~
	~~</pre>~~
	~~Use the data directory rather than the home directory for your conda environment as it~~ can ~~get quite large~~

	== Add ipykernel to your conda environment ==		== Add ipykernel to your conda environment ==
	Add this conda module is what gets your environment to show up in Jupyter notebooks		Add this conda module is what gets your environment to show up in Jupyter notebooks. Insert these lines in your .yml file. You can edit the file in Jupyter environment.
	<pre>		<pre>
	name: scibert		name: scibert
Line 82:		Line 73:
	To build a new environment:		To build a new environment:
	<pre>		<pre>
	conda env create --file scibert.yml --~~name~~ scibert		conda env create --file scibert.yml --prefix /data/user/username/python/env/scibert
	</pre>		</pre>
			If you use --name argument instead to specify the environment name, conda will create a subdirectory with same name as the yml filename. That is confusing, so using --prefix makes the environment path cleaner

	TO update an existing environment:		TO update an existing environment:
Line 89:		Line 81:
	conda env update --prefix ./scibert --file scibert.yml --prune		conda env update --prefix ./scibert --file scibert.yml --prune
	</pre>		</pre>

			== Set up your cheaha .condarc file ==
			This tells conda where to look for the installed environments. This file should be in your home directory on Cheaha (/data/user/username/.condarc). Make sure to include that '.' in the beginning of the filename.
			<pre>
			channels:
			- defaults
			envs_dirs:
			- /data/user/username/python/env
			</pre>
			Use the data directory rather than the home directory for your conda environment as it can get quite large.
			The file will be hidden and not visible in Jupyter environment. To view or edit it, start terminal and run 'nano ~/.condarc'.

	== Load environment on Jupyter Notebook ==		== Load environment on Jupyter Notebook ==

Ozborn@uab.edu: /* Build your conda environment */ How to update

2020-02-08T01:19:40Z

Build your conda environment: How to update

← Older revision		Revision as of 01:19, 8 February 2020
Line 80:		Line 80:
	</pre>		</pre>

			To build a new environment:
	<pre>		<pre>
	conda env create --file scibert.yml --name scibert		conda env create --file scibert.yml --name scibert
			</pre>

			TO update an existing environment:
			<pre>
			conda env update --prefix ./scibert --file scibert.yml --prune
	</pre>		</pre>

Ozborn@uab.edu: --no-builds option

2020-02-07T02:33:32Z

--no-builds option

← Older revision		Revision as of 02:33, 7 February 2020
Line 49:		Line 49:
	Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server. You can create any filename, but keep the extension .yml		Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server. You can create any filename, but keep the extension .yml
	<pre>		<pre>
	conda env export > scibert.yml		conda env export --no-builds > scibert.yml
	</pre>		</pre>

Azotov@uab.edu: /* Export YAML file containing your environment to cheaha */

2020-01-31T16:22:59Z

Export YAML file containing your environment to cheaha

← Older revision		Revision as of 16:22, 31 January 2020
Line 47:		Line 47:
	= Adding Custom Conda Environments to Jupyter =		= Adding Custom Conda Environments to Jupyter =
	== Export YAML file containing your environment to cheaha ==		== Export YAML file containing your environment to cheaha ==
	Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server.		Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server. You can create any filename, but keep the extension .yml
	<pre>		<pre>
	conda env export > scibert.yml		conda env export > scibert.yml

Ozborn@uab.edu: /* Build your conda environment */

2020-01-15T03:26:45Z

Build your conda environment

← Older revision		Revision as of 03:26, 15 January 2020
Line 80:		Line 80:
	</pre>		</pre>

			<pre>
			conda env create --file scibert.yml --name scibert
			</pre>

	== Load environment on Jupyter Notebook ==		== Load environment on Jupyter Notebook ==