There are many packages that extend Python’s standard functionality. While we try to satisfy all requests for specific Python packages, we may not be able to meet your needs for various reasons. When this is the case, we provide directions below to help you install Python packages on your own.

Note: These instructions are for any of the Python 2.7.X versions, which you can access by loading one of the Python modules, but these instructions do not apply to the default Python, version 2.6.6. If you are interested in other versions of Python, please contact us.

First we recommend requesting that SCV install a python package globally, and if we cannot immediately meet your needs, we will recommend one of the following options:

  1. How to install Python packages in your home directory
  2. How to install packages for a group of users to a commonly shared install location.
  3. How to make a custom copy of python using a tool called virtualenv. This local copy of python allows you to easily install Python packages in the same location. This is useful if you want to try out a development version of a package without breaking your other projects that use the stable version. It is also useful when you track the dependencies of your Python programming project. This option is typically used to create a programming-project-specific version of python.

How to install packages locally

easy_install is a tool that automatically searches for your package at pypi.org, an online repository, downloads the files, and then installs them for you. To install packages in the default location, easy_install requires you to have special permissions. Fortunately, there are several ways to control the default location, and --user is the easiest option (run easy_install --help to see the others). For example, this is how one might install BioPython in their home directory:

scc1% easy_install --user biopython

With this command, BioPython is installed to ~/.local/lib/python2.7/site-packages, which is a directory Python automatically looks for when loading Python packages. If you have the package source code, and you want to install the package using its setup.py script, then you can use this flag as well:

scc1% python setup.py build
scc1% python setup.py install --user

If your package installs scripts, then you will want to load the module python_modules/local, which simply prepends $HOME/.local/bin to your PATH environment variable. In other words, it tells the shell/terminal where to look for your scripts.

What about pip?

pip is an alternative to easy_install. Very often installation documentation recommends using pip. Currently pip is not installed with all versions of Python on the SCC, and we recommend using easy_install, which is universally available on the SCC. If you would prefer to use pip, you can install pip yourself using the previous directions:

scc1% easy_install --user pip

How to install shared packages locally

Installing code to a shared directory follows a similar process, but you use a different flag. Here is a step by step guide using the following example shared directory, which you can customize to your liking: /projectnb/your_project/python_packages.

1. Make the directories where Python libraries and scripts will go:

scc1% mkdir -p /projectnb/your_project/python_packages/lib/python2.7/site-packages
scc1% mkdir -p /projectnb/your_project/python_packages/bin

2. Define important environment variables (interactively then again in your login script)

For (t)csh:

scc1% setenv PYTHONPATH /projectnb/your_project/python_packages/lib/python2.7/site-packages:$PYTHONPATH
scc1% setenv PATH /projectnb/your_project/python_packages/bin:$PATH

For bash:

scc1% export PYTHONPATH=/projectnb/your_project/python_packages/lib/python2.7/site-packages:$PYTHONPATH
scc1% export PATH=/projectnb/your_project/python_packages/bin:$PATH

3. Use easy_install or python setup.py install with the --prefix option:

scc1% easy_install --prefix=projectnb/your_project/python_packages biopython

Now you and your colleagues will need to set the PYTHONPATH and PATH environment variables in each of your login (.bashrc or .cshrc) scripts to get access to these packages every time you login.

virtualenv

virtualenv is a tool to setup an isolated Python installation using a currently installed Python. It’s very easy to setup an isolated Python environment that you have control over. Typically this is done on a per-programming-project basis, so that each project can have access to their unique list of (Python) dependencies. To have access to virtualenv, you will need to load python_modules/virtualenv. Here is a step by step guide to create a virtual environment, which will be called in this example venv.

1. Initialize with virtualenv, which will create a directory (venv in this example) and populate it with several tools including a local copy of Python:

scc1% module load python_modules/virutalenv
scc1% virtualenv --system-site-packages venv

The option --system-site-packages tells virtualenv to include all the currently installed python packages in your virtual environment. Do not use this flag if you want a clean virtual environment, that only includes standard modules. Such an environment is useful to document your programming project’s dependencies.

2. activate the virtual environment by sourcing the activate (activate.csh for (t)csh) script found in venv/bin:

scc1% source venv/bin/activate
(venv)scc1%

You will find that the prompt will change and it will carry the name of your virtual environment (venv in this case). You will need to activate a project every time you want to use that specific version of python. Each time you login, you will need to reactivate the virtualenv, that is repeat this step (step 2).

3. Install a package, using any of easy_install, pip, or setup.py:

(venv)scc1% easy_install biopython
(venv)scc1% pip install sumatra

Notice, you no longer need to use the flag --user, because the new default prefix for the installation is venv, which you have permissions to modify. Also, you will not need to adjust the PATH and PYTHONPATH environment variables.

4. deactivate when you no longer want to use this custom virtual environment.

(venv)scc1% deactivate
scc1%

deactivate is a command that the virtual environment creates for you, and it only exists once you have activated an existing environment. Once deactivated, the command prompt will return to it’s original form.