Sections

 

Miniconda Modules

When you load Miniconda for the first time you will get a warning message, like the one below, suggesting you should create a .condarc file using the setup_scc_condarc.sh bash script to avoid exceeding the quota for your Home directory.

[rcs@scc1 ~] module load miniconda
------------------------------------------------------------------------------
WARNING:
You do not have a .condarc file in your home directory. This file is used
to configure the default location for environments and packages. By
default conda will store environments and download packages in your
home directory. Home directory quotas are limited to 10GB and this is easily
reached with the installation of a few environments.

Before using conda on the SCC run this script to create a default .condarc
file that will use your /projectnb directory for storage:

    setup_scc_condarc.sh

------------------------------------------------------------------------------

Run the command below to run the bash script and follow the instructions in the prompt.

[rcs@scc1 ~] setup_scc_condarc.sh

After creating the .condarc you are all set to create an environment. Visit the conda documentation to learn more about how to use the conda command line utility and make sure to follow the Linux instructions.

Configuring .condarc File Manually

Miniconda allows you to create an isolated programming environment known as a container. This requires installing all of the Python packages required to run your code for each environment. These environments can take up a considerable amount of disk space and should be saved in your Project Disk Space. To do this, in your home directory create the file ~/.condarc and add the following, making sure to replace your_project and your_loginname appropriately:

envs_dirs:
    - /projectnb/your_project/your_loginname/.conda/envs
    - ~/.conda/envs
pkgs_dirs:
    - /projectnb/your_project/your_loginname/.conda/pkgs
    - ~/.conda/pkgs
env_prompt: ({name})

Also, replace /projectnb with /restricted/projectnb if that is where your project has disk space available.

Avoid Doing These Things On The SCC

You should avoid doing the following on the SCC while using conda, as they can cause unexpected behavior in jobs or when using OnDemand:

  • Do not run conda init as this will cause changes to your .bashrc file.
  • Don’t add conda environment activation to your .bashrc file, for example by loading the miniconda module and then calling conda activate env_name.

Increase Disk Space by Cleaning Conda Cached Packages

Miniconda stores an index cache, lock files, unused cache packages, and tarballs when packages are installed into environments. This is convenient for creating environments quickly when they contain similar packages as existing environments; however, you can delete these files to free up storage space. To remove (or clean) these cached files, run:

[rcs@scc1 ~] module load miniconda
[rcs@scc1 ~] conda clean -a

This will give you a summary of the files it will delete and ask for you to confirm if you want to proceed with the deletions.

Back to top

Last updated: Loading…