There are many packages that extend Python’s standard functionality. While we try to satisfy all requests for specific Python packages, we may not be able to meet your needs for various reasons. When this is the case, we provide directions below to help you install Python packages on your own.
First we recommend requesting that SCV install a python package globally, and if we cannot immediately meet your needs, we will recommend one of the following options:
- How to install Python packages in your home directory
- How to install packages for a group of users to a commonly shared install location.
- How to make a custom copy of python using a tool called virtualenv. This local copy of python allows you to easily install Python packages in the same location. This is useful if you want to try out a development version of a package without breaking your other projects that use the stable version. It is also useful when you track the dependencies of your Python programming project. This option is typically used to create a programming-project-specific version of python.
easy_install is a tool that automatically searches for your package at pypi.org, an online repository, downloads the files, and then installs them for you. To install packages in the default location,
easy_install requires you to have special permissions. Fortunately, there are several ways to control the default location, and
--user is the easiest option (run
easy_install --help to see the others). For example, this is how one might install
BioPython in their home directory:
scc1% easy_install --user biopython
With this command,
BioPython is installed to
~/.local/lib/python2.7/site-packages, which is a directory Python automatically looks for when loading Python packages. If you have the package source code, and you want to install the package using its
setup.py script, then you can use this flag as well:
scc1% python setup.py build scc1% python setup.py install --user
If your package installs scripts, then you will want to load the module
python_modules/local, which simply prepends
$HOME/.local/bin to your
PATH environment variable. In other words, it tells the shell/terminal where to look for your scripts.
What about pip?
pip is an alternative to
easy_install. Very often installation documentation recommends using
pip is not installed with all versions of Python on the SCC, and we recommend using
easy_install, which is universally available on the SCC. If you would prefer to use
pip, you can install
pip yourself using the previous directions:
scc1% easy_install --user pip
Installing code to a shared directory follows a similar process, but you use a different flag. Here is a step by step guide using the following example shared directory, which you can customize to your liking:
1. Make the directories where Python libraries and scripts will go:
scc1% mkdir -p /projectnb/your_project/python_packages/lib/python2.7/site-packages scc1% mkdir -p /projectnb/your_project/python_packages/bin
2. Define important environment variables (interactively then again in your login script)
scc1% setenv PYTHONPATH /projectnb/your_project/python_packages/lib/python2.7/site-packages:$PYTHONPATH scc1% setenv PATH /projectnb/your_project/python_packages/bin:$PATH
scc1% export PYTHONPATH=/projectnb/your_project/python_packages/lib/python2.7/site-packages:$PYTHONPATH scc1% export PATH=/projectnb/your_project/python_packages/bin:$PATH
python setup.py install with the
scc1% easy_install --prefix=projectnb/your_project/python_packages biopython
Now you and your colleagues will need to set the
PATH environment variables in each of your login (
.cshrc) scripts to get access to these packages every time you login.
virtualenv is a tool to setup an isolated Python installation using a currently installed Python. It’s very easy to setup an isolated Python environment that you have control over. Typically this is done on a per-programming-project basis, so that each project can have access to their unique list of (Python) dependencies. To have access to
virtualenv, you will need to load
python_modules/virtualenv. Here is a step by step guide to create a virtual environment, which will be called in this example
1. Initialize with
virtualenv, which will create a directory (
venv in this example) and populate it with several tools including a local copy of Python:
scc1% module load python_modules/virutalenv scc1% virtualenv --system-site-packages venv
virtualenv to include all the currently installed python packages in your virtual environment. Do not use this flag if you want a clean virtual environment, that only includes standard modules. Such an environment is useful to document your programming project’s dependencies.
2. activate the virtual environment by sourcing the activate (activate.csh for (t)csh) script found in
scc1% source venv/bin/activate (venv)scc1%
You will find that the prompt will change and it will carry the name of your virtual environment (
venv in this case). You will need to activate a project every time you want to use that specific version of python. Each time you login, you will need to reactivate the virtualenv, that is repeat this step (step 2).
3. Install a package, using any of
(venv)scc1% easy_install biopython (venv)scc1% pip install sumatra
Notice, you no longer need to use the flag
--user, because the new default prefix for the installation is venv, which you have permissions to modify. Also, you will not need to adjust the
PYTHONPATH environment variables.
4. deactivate when you no longer want to use this custom virtual environment.
(venv)scc1% deactivate scc1%
deactivate is a command that the virtual environment creates for you, and it only exists once you have activated an existing environment. Once deactivated, the command prompt will return to it’s original form.