{"id":149128,"date":"2023-11-28T13:44:31","date_gmt":"2023-11-28T18:44:31","guid":{"rendered":"http:\/\/www.bu.edu\/tech\/?page_id=149128"},"modified":"2026-01-13T10:38:11","modified_gmt":"2026-01-13T15:38:11","slug":"academic-machine-learning-environment","status":"publish","type":"page","link":"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/common-languages\/python\/python-ml\/academic-machine-learning-environment\/","title":{"rendered":"Academic Machine Learning Environment"},"content":{"rendered":"<p><a name=\"top\"><\/a><br \/>\nThe module <code>academic-ml<\/code> provides pre-configured <code>conda<\/code> environments environments containing popular machine learning libraries and essential Python packages for data science workflows. Designed to support academic coursework on the SCC (Shared Computing Cluster), this module is equally suitable for research projects as it streamlines the setup of ML frameworks. The module is updated twice yearly before the Spring and Fall semesters to ensure access to the latest stable versions of included libraries and security patches.<\/p>\n<p>A new version of the <code>academic-ml<\/code> module will be installed before the start of the Spring and Fall semesters so that the latest updates to the various libraries will be available.<\/p>\n<h3>Sections<\/h3>\n<ul>\n<li><a href=\"#use\">Using the academic-ml Module<\/a><\/li>\n<li><a href=\"#custom\">Customizing the academic-ml Module Environments<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p><a name=\"use\"><\/a><\/p>\n<h2>Using the <code>academic-ml<\/code> Module<\/h2>\n<p>This module includes three conda environments based on PyTorch, Tensorflow, and Jax libraries. The GPU capabilities are automatically used when these environments are used on GPU-equipped compute nodes :<\/p>\n<ul>\n<li><code><strong>spring-2026-pyt<\/strong><\/code>\u00a0 &#8211; PyTorch 2.9.1<\/li>\n<li><code><strong>spring-2026-tf<\/strong><\/code>\u00a0 &#8211; Tensorflow 2.20.0<\/li>\n<li><code><strong>spring-2026-jax<\/strong><\/code>\u00a0 &#8211; Jax 0.8.2<\/li>\n<\/ul>\n<p>Aside from the core ML software and associated libraries, these environments are mostly identical, and a <a href=\"http:\/\/sccsvc.bu.edu\/software\/#\/package\/academic-ml\/\">summary of the available software<\/a> is provided. Each one provides development software such as Jupyter Notebooks, Spyder, the Nvidia CUDA SDK,\u00a0 ML libraries like Transformers, and common Python libraries such as Numpy, Scipy, and Pandas. A complete list of every installed library with all versions is available by running the <code>conda list<\/code> command after activating the environments.<\/p>\n<h2>Using the environments<\/h2>\n<p>To use any of these environments, load the <code>miniconda<\/code> module, then the <code>academic-ml<\/code> module. First-time users of the miniconda module will prompted to run a setup script. See the <a href=\"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/common-languages\/python\/python-software\/miniconda-modules\/\">miniconda documentation<\/a> for more details:<\/p>\n<pre class=\"code-block\"><code>module load miniconda\r\nmodule load academic-ml\/spring-2026\r\n<span class=\"comment\"># see a summary of installed libraries:<\/span>\r\nmodule help academic-ml\/spring-2026<\/code><\/pre>\n<p>Loading the <code>academic-ml<\/code> module in a terminal prints a message with instructions to activate these environments, and these commands are shown here:<\/p>\n<pre class=\"code-block\"><code><span class=\"comment\"># Activate the PyTorch environment:<\/span>\r\nconda activate spring-2026-pyt\r\n<span class=\"comment\"># Or activate the Tensorflow environment:<\/span>\r\nconda activate spring-2026-tf\r\n<span class=\"comment\">#Or activate the Jax-based environment run:<\/span>\r\nconda activate spring-2026-jax\r\n<\/code><\/pre>\n<p>When requesting a Jupyter Notebook, Jupyter Lab, VS Code Server, or Spyder OnDemand session, enter the conda activation command in the <strong>Pre-Launch Command<\/strong> field. This screenshot shows an example of starting a Jupyter Notebook session with the <code>spring-2026-tf<\/code> environment.<\/p>\n<p><img loading=\"lazy\" src=\"\/tech\/files\/2026\/01\/spring-2026-tf-notebook-screenshot-636x350.png\" alt=\"\" width=\"509\" height=\"240\" class=\"alignnone size-medium wp-image-160690\" \/><br \/>\n<a name=\"custom\"><\/a><\/p>\n<h2>Cached Model Files<\/h2>\n<p>The main ML libraries and popular libraries like Hugging Face will automatically download and cache model files and datasets. By default the cache location is in a user&#8217;s home directory, but this cache can quickly hit the home directory&#8217;s 10GB storage limit. Each library has its own particular means of setting the location of its cache directory, either using environment variables or using library function calls. The <code>academic-ml\/spring-2026<\/code> conda environments will automatically set the cache directory location for PyTorch, Tensorflow, Jax, and Hugging Face upon activation by setting the TORCH_HOME, TFHUB_DIR, JAX_COMPILATION_CACHE_DIR, and HF_HOME variables, respectively. The location will be in the same directory as conda environments, as configured in a user&#8217;s <code>~\/.condarc<\/code> file. The cache directory location can be printed out after activating one of the environments with the command <code>echo $HF_HOME <\/code>.<\/p>\n<h2>Customizing the <code>academic-ml<\/code> Module Environments<\/h2>\n<p>If you want to customize the conda environments, for example to add additional libraries for an academic course or for your own work, you will need to re-create one of these conda environment in your project&#8217;s <code>\/projectnb<\/code> (or <code>\/restricted\/projectnb<\/code>) directory. Once the environment is recreated you can modify it as you like. There are two environment variables set when the module is loaded that provide the location of a .yml file for each of the two environments. These .yml files install exact clones of their environments:<\/p>\n<ul>\n<li><code><strong>$SCC_ACADEMIC_ML_YML_PYT<\/strong><\/code> &#8211; PyTorch .yml file<\/li>\n<li><code><strong>$SCC_ACADEMIC_ML_YML_TF<\/strong><\/code> &#8211; Tensorflow .yml file<\/li>\n<li><code><strong>$SCC_ACADEMIC_ML_YML_JAX<\/strong><\/code> &#8211; Jax.yml file<\/li>\n<\/ul>\n<p>There are two additional environment variables with a simplified package list, generally adding capabilities like Jupyter and Spyder to the main machine learning library. These allow you to set up an environment for easier customization:<\/p>\n<ul>\n<li><code><strong>$SCC_ACADEMIC_ML_YML_BASIC_PYT<\/strong><\/code> &#8211; PyTorch .yml file<\/li>\n<li><code><strong>$SCC_ACADEMIC_ML_YML_BASIC_TF<\/strong><\/code> &#8211; Tensorflow .yml file<\/li>\n<li><code><strong>$SCC_ACADEMIC_ML_YML_BASIC_JAX<\/strong><\/code> &#8211; Jax.yml file<\/li>\n<\/ul>\n<p>These are used to create your project&#8217;s copy of the environment.\u00a0 As an example, let&#8217;s make a clone of one of the environments and add an additional library to it for use in a course.<\/p>\n<pre class=\"code-block\"><code>module load miniconda\r\nmodule load academic-ml\/spring-2026\r\n<span class=\"comment\"># make sure you are in the class \/projectnb directory. <\/span>\r\ncd \/projectnb\/course_number\r\n\r\n# Change to the materials subdirectory\r\ncd materials\r\n<span class=\"comment\"># make a software subdirectory<\/span>\r\nmkdir software\r\ncd software\r\n\r\n<span class=\"comment\"># Make a clone of the spring-2026-pyt PyTorch-based environment. \r\n# This will use ~17GB of space.<\/span>\r\nconda env create -p .\/class-custom-pyt -f  $SCC_ACADEMIC_ML_YML_PYT \r\n<span class=\"comment\"># Wait 45+ minutes...<\/span>\r\nconda activate  .\/class-custom-pyt\r\n\r\n<span class=\"comment\"># <strong>IMPORTANT:<\/strong> Run this script to set up the\r\n# environment to automatically handle the model caching environment variables\r\n# regardless of which .yml file <\/span>\r\n$SCC_ACADEMIC_ML_DIR\/scripts\/install_model_cache_dir.sh\r\n\r\n<span class=\"comment\"># Install the emoji library, for adding emojis to your Python print statements.<\/span>\r\nconda install emoji -c conda-forge\r\n \r\n<span class=\"comment\"># In the future, to use your custom conda environment, use:<\/span>\r\nconda activate \/projectnb\/course_number\/materials\/software\/class-custom-pyt  <\/code><\/pre>\n<p><a href=\"#top\">Back to top<\/a><\/p>\n<p><script>\r\nconst date = new Date(document.lastModified);\r\ndocument.getElementById(\"demo\").innerHTML = date;\r\n<\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The module academic-ml provides pre-configured conda environments environments containing popular machine learning libraries and essential Python packages for data science workflows. Designed to support academic coursework on the SCC (Shared Computing Cluster), this module is equally suitable for research projects as it streamlines the setup of ML frameworks. The module is updated twice yearly before&#8230;<\/p>\n","protected":false},"author":22336,"featured_media":0,"parent":147628,"menu_order":5,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/149128"}],"collection":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/users\/22336"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/comments?post=149128"}],"version-history":[{"count":32,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/149128\/revisions"}],"predecessor-version":[{"id":160691,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/149128\/revisions\/160691"}],"up":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/147628"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/media?parent=149128"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}