Python Virtual Environment

Python virtual environment is a method of creating an isolated environment for Python projects. It enables each project to have its own dependencies, regardless of what dependencies every other project has. To read more about Python virtual environments, click here.

Creating a Python Virtual Environment

Load one of the Python modules available on Cheaha in your environment.

[snoopy@c1 ~]$ module avail Python

-------------------------- /share/apps/rc/modules/all --------------------------
Python/2.7.10-goolf-1.7.20       Python/2.7.13-intel-2017a
Python/2.7.10-intel-2015b        Python/2.7.3-foss-2016a
Python/2.7.11-foss-2016a         Python/2.7.3-goolf-1.7.20
Python/2.7.11-foss-2016b         Python/2.7.5-goolf-1.7.20
Python/2.7.11-goolf-1.7.20       Python/2.7.8-intel-2015b
Python/2.7.11-intel-2015b        Python/2.7.9-goolf-1.7.20
Python/2.7.11-intel-2016a        Python/2.7.9-intel-2015b
Python/2.7.12-foss-2016a         Python/3.2.3-goolf-1.7.20
Python/2.7.12-foss-2016b         Python/3.5.1-foss-2016a
Python/2.7.12-intel-2015b        Python/3.5.1-intel-2016a
Python/2.7.12-intel-2016a        Python/3.6.1-intel-2017a
Python/2.7.13-GCCcore-6.3.0-bare Python/3.6.3-intel-2017a

Once you have loaded Python, we would use virtualenv to create and manage virtual environments.

[snoopy@c1 Python_Environments]$ module load Python/3.6.3-intel-2017a 
[snoopy@c1 Python_Environments]$ virtualenv test_environment
Using base prefix '/share/apps/rc/software/Python/3.6.3-intel-2017a'
New python executable in /data/user/snoopy/Python_Environments/test_environment/bin/python
Installing setuptools, pip, wheel...done.
[snoopy@c1 Python_Environments]$

Activating a Virtual Environment

Once a virtual environment has been created, you need to activate it to be in the virtual environment.

[snoopy@c1 Python_Environments]$ source test_environment/bin/activate
(test_environment) [snoopy@c1 Python_Environments]$

Activating the virtual environment will change your shell’s prompt to show what virtual environment you’re using, test_environment in the above case, and modify the environment so that you can install Python packages for that particular environment.

Maintaining a Virtual Environment

After this you can install the packages that you would like for this environment, using pip. pip is a package management system used to install and manage software packages written in Python.

(test_environment) [snoopy@c1 Python_Environments]$ pip install numpy
Collecting numpy
  Downloading numpy-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (17.2MB)
    100% |████████████████████████████████| 17.2MB 77kB/s 
Installing collected packages: numpy
Successfully installed numpy-1.14.0
(test_environment) [snoopy@c1 Python_Environments]$ ls test_environment/lib/python3.6/site-packages/
easy_install.py         pip-9.0.1.dist-info  setuptools-38.4.0.dist-info
numpy                   pkg_resources        wheel
numpy-1.14.0.dist-info  __pycache__          wheel-0.30.0.dist-info
pip                     setuptools
(test_environment) [snoopy@c1 Python_Environments]$

You can use this method to install a Python application alongside all the dependencies that it requires.

Deactivating a Virtual Environment

After you are done using the virtual environment, you can use deactivate command to go back to your bash shell environemnt.

(test_environment) [snoopy@c1 Python_Environments]$ deactivate 
[snoopy@c1 Python_Environments]$

It would change your shell's prompt and remove the name of the virtual environment that you were in.

Sharing a virtual environment

You can use pip freeze to list all the packages in a virtual environment and copy it to a requirement.txt.file

pip freeze > requirements.txt

Now you can create new virtualenv and after activating that virtual environment, install all the packages using the following command.

pip install -r requirements.txt