{"id":148408,"date":"2023-10-20T10:45:42","date_gmt":"2023-10-20T14:45:42","guid":{"rendered":"http:\/\/www.bu.edu\/tech\/?page_id=148408"},"modified":"2025-01-03T13:50:21","modified_gmt":"2025-01-03T18:50:21","slug":"miniconda-modules","status":"publish","type":"page","link":"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/common-languages\/python\/python-software\/miniconda-modules\/","title":{"rendered":"Miniconda Modules"},"content":{"rendered":"<p><a name=\"top\"><\/a><br \/>\n<strong>Sections<\/strong><\/p>\n<ul>\n<li><a href=\"#Conda Modules\">Miniconda Modules<\/a>\n<ul>\n<li><a href=\"#Manual\">Configuring A .condarc File Manually<\/a><\/li>\n<li><a href=\"#Avoid\">Avoid Doing These Things On The SCC<\/a><\/li>\n<li><a href=\"#Space\">Increase Disk Space by Cleaning Conda Cached Packages<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p><a name=\"Miniconda Modules\"><\/a><\/p>\n<h2>Miniconda Modules<\/h2>\n<p>When you load Miniconda for the first time you will get a warning message, like the one below, suggesting you should create a <code>.condarc<\/code> file using the <code>setup_scc_condarc.sh<\/code> bash script to avoid exceeding the quota for your Home directory.<\/p>\n<pre><code class=\"code-block\"><span class=\"prompt\">[rcs@scc1 ~]<\/span> <span class=\"command\">module load miniconda<\/span>\r\n<span class=\"output\">------------------------------------------------------------------------------\r\nWARNING:\r\nYou do not have a .condarc file in your home directory. This file is used\r\nto configure the default location for environments and packages. By\r\ndefault conda will store environments and download packages in your\r\nhome directory. Home directory quotas are limited to 10GB and this is easily\r\nreached with the installation of a few environments.\r\n\r\nBefore using conda on the SCC run this script to create a default .condarc\r\nfile that will use your \/projectnb directory for storage:\r\n\r\n    setup_scc_condarc.sh\r\n\r\n------------------------------------------------------------------------------<\/span>\r\n<\/code><\/pre>\n<p>Run the command below to run the bash script and follow the instructions in the prompt.<\/p>\n<pre><code class=\"code-block\"><span class=\"prompt\">[rcs@scc1 ~]<\/span> <span class=\"command\">setup_scc_condarc.sh<\/span>\r\n<\/code><\/pre>\n<p>After creating the <code>.condarc<\/code> you are all set to create an environment. Visit the <a href=\"https:\/\/docs.conda.io\/projects\/conda\/en\/latest\/user-guide\/getting-started.html#managing-envs\">conda documentation<\/a> to learn more about how to use the <code><span class=\"command\">conda<\/span><\/code> command line utility and make sure to follow the Linux instructions.<br \/>\n<a name=\"Manual\"><\/a><\/p>\n<h3>Configuring .condarc File Manually<\/h3>\n<p>Miniconda allows you to create an isolated programming environment known as a <i>container<\/i>. This requires installing all of the Python packages required to run your code for each environment. These environments can take up a considerable amount of disk space and should be saved in your Project Disk Space. To do this, in your home directory create the file <code>~\/.condarc<\/code> and add the following, making sure to replace <code><span class=\"placeholder\">your_project<\/span><\/code> and <code><span class=\"placeholder\">your_loginname<\/span><\/code> appropriately:<\/p>\n<pre><code>envs_dirs:\r\n    - \/projectnb\/<span class=\"placeholder\">your_project<\/span>\/<span class=\"placeholder\">your_loginname<\/span>\/.conda\/envs\r\n    - ~\/.conda\/envs\r\npkgs_dirs:\r\n    - \/projectnb\/<span class=\"placeholder\">your_project<\/span>\/<span class=\"placeholder\">your_loginname<\/span>\/.conda\/pkgs\r\n    - ~\/.conda\/pkgs\r\nenv_prompt: ({name})<\/code><\/pre>\n<p>Also, replace <code>\/projectnb<\/code> with <code>\/restricted\/projectnb<\/code> if that is where your project has disk space available.<br \/>\n<a name=\"Avoid\"><\/a><\/p>\n<h3>Avoid Doing These Things On The SCC<\/h3>\n<p>You should avoid doing the following on the SCC while using <code><span class=\"command\">conda<\/span><\/code>, as they can cause unexpected behavior in jobs or when using OnDemand:<\/p>\n<ul>\n<li>Do not run <code><span class=\"command\">conda init<\/span><\/code> as this will cause changes to your <code>.bashrc<\/code> file.<\/li>\n<li>Don&#8217;t add conda environment activation to your <code>.bashrc<\/code> file, for example by loading the <code>miniconda<\/code> module and then calling <code>conda activate env_name<\/code>.<\/li>\n<\/ul>\n<p><a name=\"Space\"><\/a><\/p>\n<h3>Increase Disk Space by Cleaning Conda Cached Packages<\/h3>\n<p>Miniconda stores an index cache, lock files, unused cache packages, and tarballs when packages are installed into environments. This is convenient for creating environments quickly when they contain similar packages as existing environments; however, you can delete these files to free up storage space. To remove (or clean) these cached files, run:<\/p>\n<pre><code class=\"code-block\"><span class=\"prompt\">[rcs@scc1 ~]<\/span> <span class=\"command\">module load miniconda<\/span>\r\n<span class=\"prompt\">[rcs@scc1 ~]<\/span> <span class=\"command\">conda clean -a<\/span><\/code><\/pre>\n<p>This will give you a summary of the files it will delete and ask for you to confirm if you want to proceed with the deletions.<\/p>\n<p><a href=\"#top\">Back to top<\/a><\/p>\n<div style=\"float: right;\">\n<p id=\"last-modified-timestamp\" style=\"margin: 0;\">Last updated: Loading&#8230;<\/p>\n<\/div>\n<p><script>\r\ndocument.addEventListener('DOMContentLoaded', function() {\r\n    \/\/ Get the content of the meta tag\r\n    var lastUpdatedContent = document.querySelector('meta[name=\"last-updated\"]').content;\r\n    \r\n    \/\/ Parse the content into a Date object\r\n    var lastUpdatedDate = new Date(lastUpdatedContent);\r\n    \r\n    \/\/ Format the date\r\n    var formattedDate = lastUpdatedDate.toLocaleDateString(undefined, { year: 'numeric', month: 'long', day: 'numeric' });\r\n    \r\n    \/\/ Update the HTML element with the formatted date\r\n    document.getElementById('last-modified-timestamp').innerHTML = 'Last updated: ' + formattedDate;\r\n});\r\n<\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Sections Miniconda Modules Configuring A .condarc File Manually Avoid Doing These Things On The SCC Increase Disk Space by Cleaning Conda Cached Packages &nbsp; Miniconda Modules When you load Miniconda for the first time you will get a warning message, like the one below, suggesting you should create a .condarc file using the setup_scc_condarc.sh bash&#8230;<\/p>\n","protected":false},"author":1692,"featured_media":0,"parent":147618,"menu_order":2,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/148408"}],"collection":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/users\/1692"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/comments?post=148408"}],"version-history":[{"count":18,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/148408\/revisions"}],"predecessor-version":[{"id":155302,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/148408\/revisions\/155302"}],"up":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/147618"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/media?parent=148408"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}