{"id":105977,"date":"2017-04-06T12:20:25","date_gmt":"2017-04-06T16:20:25","guid":{"rendered":"http:\/\/www.bu.edu\/tech\/?page_id=105977"},"modified":"2021-07-12T15:34:26","modified_gmt":"2021-07-12T19:34:26","slug":"versions","status":"publish","type":"page","link":"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/common-languages\/versions\/","title":{"rendered":"Python Versions"},"content":{"rendered":"<p>We\u00a0maintain several versions of Python on the Shared Computing Cluster (SCC) and try to keep both the Python versions and associated packages up to date. Each Python installation\u00a0has many packages pre-configured and readily available. The packages include popular libraries like NumPy, Pandas, SciPy among many others. Researchers have a choice of Python versions on SCC, but most will prefer the\u00a0<a href=\"#general\">general purpose installations<\/a>. Each installation can be loaded using the\u00a0<a href=\"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/software-and-applications\/modules\/\">module system<\/a>.<\/p>\n<p>The categories of our Python installations are as follows:<\/p>\n<ul>\n<li><a href=\"#general\">General Purpose Python Modules<\/a><\/li>\n<li><a href=\"#intel\">Intel Distribution for Python<\/a><\/li>\n<li><a href=\"#anaconda\">Anaconda Python<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p>The most recent listing of python installations\u00a0on SCC can be queried using the <a href=\"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/software-and-applications\/modules\/\">module system<\/a>.<\/p>\n<pre><code class=\"code-block\"><span class=\"prompt\">scc1$<\/span> <span class=\"command\">module avail python<\/span><\/code><\/pre>\n<h2><a name=\"general\"><\/a>General Purpose Python Modules<\/h2>\n<p>For most computational use cases, the most recent version of these general purpose installations is the best choice. We offer python2,\u00a0python3, and maintain older versions for reproducibility. These modules are configured for general use and already contain many commonly used python packages.<\/p>\n<p>To load the default version of python3, use the following command in terminal:<\/p>\n<pre><code class=\"code-block\"><span class=\"prompt\">scc1$<\/span> <span class=\"command\">module load python3\/3.8.10<\/span><\/code><\/pre>\n<p>We recommend always specifying the version of the module you will load. This prevents breaking your workflow\/pipeline in case we upgrade the default version of a module to a newer one.<br \/>\n&nbsp;<\/p>\n<p><strong>Update Frequency:<\/strong><\/p>\n<p>We install new Python modules approximately every 6 months. The Python libraries installed within the Python modules are updated to their latest versions when the module is installed.<\/p>\n<h2><a name=\"intel\"><\/a>Intel Distribution for Python<\/h2>\n<p>The Intel Distribution for Python behaves like regular Python, but leverages Intel technologies to speed up many of the core python libraries, including NumPy, SciPy, Pandas, Scikit-Learn, Jupyter, matplotlib, and mpi4py. This distribution also integrates Intel Math Kernel Library (Intel MKL), Intel Data Analytics Acceleration Library (DAAL) and pyDAAL, Intel MPI Library, and Intel Threading Building Blocks (TBB). The following modules offer significant speedups for some computational workloads at the cost of potential incompatibility with other python packages.<\/p>\n<p>To load the most recent version of Intel Python 3.7, issue the following command:<\/p>\n<pre><code class=\"code-block\"><span class=\"prompt\">scc1$<\/span> <span class=\"command\">module load python3-intel\/2021.1.1<span>\r\n<\/span><\/span><\/code><\/pre>\n<p>The optimizations available in the Intel Distribution of Python depend both OpenMP and Intel multi-processing libraries. Two environment variables are used\u00a0coordinate the automatic parallel processing: <code><span class=\"command\">OMP_NUM_THREADS<\/span><\/code> and <code><span class=\"command\">MKL_NUM_THREADS<\/span><\/code>. It is important to understand the impact of these variables on your code and define appropriate values when running your code. Most commonly,\u00a0you should set the value equal to the number of slots (<code><span class=\"command\">$NSLOTS<\/span><\/code>) your job requests, but not always. Please read\u00a0<a href=\"https:\/\/software.intel.com\/en-us\/articles\/intel-math-kernel-library-intel-mkl-using-intel-mkl-with-threaded-applications\">Intel&#8217;s guidance on threaded applications<\/a>.<\/p>\n<p>&nbsp;<\/p>\n<table>\n<tbody>\n<tr>\n<th>Module Name<\/th>\n<th>Description<\/th>\n<th>Default Value<\/th>\n<th>Recommended<\/th>\n<\/tr>\n<tr>\n<td><code><span class=\"command\">OMP_NUM_THREADS<\/span><\/code><\/td>\n<td>Sets the number of threads for OpenMP<\/td>\n<td style=\"text-align: center;\">1<\/td>\n<td style=\"text-align: center;\"><code><span class=\"command\">$NSLOTS<\/span><\/code><\/td>\n<\/tr>\n<tr>\n<td><code><span class=\"command\">MKL_NUM_THREADS<\/span><\/code><\/td>\n<td>Sets the number of threads for Intel Math Kernel Library<\/td>\n<td style=\"text-align: center;\">1<\/td>\n<td style=\"text-align: center;\"><code><span class=\"command\">$NSLOTS<\/span><\/code><\/td>\n<\/tr>\n<tr>\n<td><code><span class=\"command\">OPENBLAS_NUM_THREADS<\/span><\/code><\/td>\n<td>Sets the number of threads for OpenBLAS<\/td>\n<td style=\"text-align: center;\">1<\/td>\n<td style=\"text-align: center;\"><code><span class=\"command\">$NSLOTS<\/span><\/code><\/td>\n<\/tr>\n<tr>\n<td><code><span class=\"command\">NUMBA_NUM_THREADS<\/span><\/code><\/td>\n<td>Sets the number of threads for NUMBA<\/td>\n<td style=\"text-align: center;\">1<\/td>\n<td style=\"text-align: center;\"><code><span class=\"command\">$NSLOTS<\/span><\/code><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<h2><a name=\"anaconda\"><\/a>Anaconda Distribution (conda)<\/h2>\n<p>Anaconda is an open-source package and environment manager for Python. It has gained traction for ease of packaging and replicating modules or entire python environments on different systems. The distribution includes a set of core python packages and additional user packages can be installed from remote &#8220;channels&#8221;. Anaconda has also been known to cause confusion and package dependency issues in complex environments; <a href=\"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/common-languages\/python\/virtualenv\/\">virtualenv<\/a>\u00a0is an alternative for a virtual environment configuration within the cluster if you are not working with existing software that depends on a conda environment. <\/p>\n<p><b>We recommend the use of the <strong>miniconda<\/strong> module to utilize Conda environments. The anaconda2 and anaconda3 modules are no longer updated and will be deprecated in the future.<\/b> Note that you can only load and use either a Python or a miniconda module. You should <b>not<\/b> attempt to load miniconda and python modules at the same time.<\/p>\n<p>Instructions for using Anaconda are found on the <a href=\"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/common-languages\/python\/anaconda\/\">Anaconda Python<\/a> page.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We\u00a0maintain several versions of Python on the Shared Computing Cluster (SCC) and try to keep both the Python versions and associated packages up to date. Each Python installation\u00a0has many packages pre-configured and readily available. The packages include popular libraries like NumPy, Pandas, SciPy among many others. Researchers have a choice of Python versions on SCC,&#8230;<\/p>\n","protected":false},"author":3593,"featured_media":0,"parent":76567,"menu_order":1,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/105977"}],"collection":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/users\/3593"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/comments?post=105977"}],"version-history":[{"count":48,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/105977\/revisions"}],"predecessor-version":[{"id":136109,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/105977\/revisions\/136109"}],"up":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/76567"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/media?parent=105977"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}