{"id":148420,"date":"2023-10-20T10:52:35","date_gmt":"2023-10-20T14:52:35","guid":{"rendered":"http:\/\/www.bu.edu\/tech\/?page_id=148420"},"modified":"2025-04-04T15:14:17","modified_gmt":"2025-04-04T19:14:17","slug":"huggingface","status":"publish","type":"page","link":"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/common-languages\/python\/python-ml\/huggingface\/","title":{"rendered":"Hugging Face"},"content":{"rendered":"<p><a name=\"top\"><\/a><\/p>\n<div style=\"width: 100%;\">\n<div style=\"float: right; width: 28%;\"><img loading=\"lazy\" src=\"\/tech\/files\/2023\/12\/hf-e1702919451901.png\" alt=\"\" width=\"400\" height=\"400\" class=\"alignnone size-full wp-image-149641\" \/><\/div>\n<div style=\"float: none;\">Hugging Face contains a number of useful packages that can be used in various machine learning applications. The full list is provided in the <a href=\"https:\/\/huggingface.co\/docs\">Hugging Face Docs<\/a>. To use these packages, you need to install them into a virtual environment or conda environment. The instructions for two example packages, <a href=\"https:\/\/huggingface.co\/docs\/transformers\/index\">Transformers<\/a> and <a href=\"https:\/\/huggingface.co\/docs\/datasets\/index\">Datasets<\/a>, are provided below.<\/div>\n<\/div>\n<p><strong>Sections<\/strong><\/p>\n<ul>\n<li><a href=\"#install\">Installation<\/a>\n<ul>\n<li><a href=\"#venv\">Python Virtual Environment<\/a><\/li>\n<li><a href=\"#conda\">Conda Environment<\/a><\/li>\n<li><a href=\"#pycode\">Setting HF_HOME in Python<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p><a name=\"install\"><\/a><\/p>\n<h2>Installation<\/h2>\n<p><a name=\"venv\"><\/a><\/p>\n<h3>Python Virtual Environment<\/h3>\n<ol>\n<li>Create and activate your virtualenv as explained on the <a href=\"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/common-languages\/python\/python-installs\/virtualenv\/\">Python Installs page<\/a>.<\/li>\n<li>Set the following path to specify a location (other than your home directory) to store cached datasets and models. The path should be in the <code>\/projectnb<\/code> or <code>\/rprojectnb<\/code> directory for your project, preferably in a subfolder that you&#8217;ve created:\n<pre><code class=\"code-block\"><span class=\"prompt\">(my_newenv) [rcs@scc1 ~]<\/span> <span class=\"command\">export HF_HOME=<span class=\"placeholder\">\/projectnb\/YOUR_PROJECT\/YOUR_FOLDER\/hf_cache<\/span><\/span><\/code><\/pre>\n<p>For further details on dataset cache setup please see the <a href=\"https:\/\/huggingface.co\/docs\/datasets\/main\/en\/cache\">Hugging Face Docs<\/a>.<\/li>\n<li>Install <a href=\"https:\/\/huggingface.co\/docs\/transformers\/index\">Transformers<\/a> and <a href=\"https:\/\/huggingface.co\/docs\/datasets\/index\">Datasets<\/a> with <code><b>pip<\/b><\/code>:\n<pre><code class=\"code-block\"><span class=\"prompt\">(my_newenv) [rcs@scc1 ~]<\/span> <span class=\"command\">pip install transformers datasets<\/span><\/code><\/pre>\n<\/li>\n<\/ol>\n<p><a name=\"conda\"><\/a><\/p>\n<h3>Conda Environment<\/h3>\n<ol>\n<li>Create and activate your conda env as explained on the <a href=\"https:\/\/www.bu.edu\/tech\/support\/research\/software-and-programming\/common-languages\/python\/python-installs\/conda\/\">Miniconda Installs page<\/a>.<\/li>\n<li>Set the following path to specify a location (other than your home directory) to cache downloads as described above:\n<pre><code class=\"code-block\"><span class=\"prompt\">(my_newenv) [rcs@scc1 ~]<\/span> <span class=\"command\">export HF_HOME=<span class=\"placeholder\">\/projectnb\/YOUR_PROJECT\/YOUR_FOLDER\/hf_cache<\/span><\/span><\/code><\/pre>\n<p>For further details on dataset cache setup please see the <a href=\"https:\/\/huggingface.co\/docs\/datasets\/v2.15.0\/en\/cache#cache-directory\">Hugging Face Docs<\/a>.<\/li>\n<li>Install <a href=\"https:\/\/huggingface.co\/docs\/transformers\/index\">Transformers<\/a> and <a href=\"https:\/\/huggingface.co\/docs\/datasets\/index\">Datasets<\/a> with <code><b>conda install<\/b><\/code>:\n<pre><code class=\"code-block\"><span class=\"prompt\">(my_newenv) [rcs@scc1 ~]<\/span> <span class=\"command\">conda install -c huggingface -c conda-forge transformers datasets<\/span><\/code><\/pre>\n<\/li>\n<\/ol>\n<p><a name=\"pycode\"><\/a><\/p>\n<h3>Setting HF_HOME in Python<\/h3>\n<p>The value of the cache directory can be set in a Python script. To do so, insert the following lines of code into your Python script before any Hugging Face libraries are imported:<\/p>\n<pre><code class=\"code-block\">import os\r\nos.environ['HF_HOME'] = '\/projectnb\/YOUR_PROJECT\/YOUR_FOLDER\/hf_cache'<\/code><\/pre>\n<p><a href=\"#top\">Back to top<\/a><\/p>\n<p><script>\r\nconst date = new Date(document.lastModified);\r\ndocument.getElementById(\"demo\").innerHTML = date;\r\n<\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hugging Face contains a number of useful packages that can be used in various machine learning applications. The full list is provided in the Hugging Face Docs. To use these packages, you need to install them into a virtual environment or conda environment. The instructions for two example packages, Transformers and Datasets, are provided below&#8230;.<\/p>\n","protected":false},"author":1692,"featured_media":0,"parent":147628,"menu_order":3,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/148420"}],"collection":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/users\/1692"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/comments?post=148420"}],"version-history":[{"count":24,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/148420\/revisions"}],"predecessor-version":[{"id":156693,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/148420\/revisions\/156693"}],"up":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/147628"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/media?parent=148420"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}