{"id":139572,"date":"2022-04-01T17:18:35","date_gmt":"2022-04-01T21:18:35","guid":{"rendered":"http:\/\/www.bu.edu\/tech\/?page_id=139572"},"modified":"2025-09-15T11:52:35","modified_gmt":"2025-09-15T15:52:35","slug":"bu-fileservers","status":"publish","type":"page","link":"https:\/\/www.bu.edu\/tech\/support\/research\/system-usage\/transferring-files\/bu-fileservers\/","title":{"rendered":"Transferring Files to\/from Fileservers at BU"},"content":{"rendered":"<p>Researchers on the SCC can connect to remote file systems on the BU network for the purpose of transferring data. The steps to connect to these shares will depend on how the remote system exports the data share. Two common protocols and associated applications are described on this page.<\/p>\n<ol>\n<li><a href=\"#SMB\">Windows SMB\/CIFS Shares (e.g. IS&amp;T NAS1, BUMC-IT Y-Drive)<\/a><\/li>\n<li><a href=\"#NFS\">Network File System &#8220;NFS&#8221; Shares (e.g. Linux Systems, ENGNAS)<\/a><\/li>\n<\/ol>\n<div class=\"highlight-yellow\">RCS recommends copying data from the remote file system to your SCC Project Space for faster data access in batch jobs.<\/div>\n<h2><a name=\"SMB\"><\/a>Windows SMB\/CIFS Shares<\/h2>\n<p>Windows SMB\/CIFS shares on the BU network can be accessed from the SCC using either graphical file browsers or command line utilities. This is commonly used to connect to shares on the <a href=\"https:\/\/www.bu.edu\/tech\/services\/infrastructure\/storage-backup\/network-file-storage\/\">IS&amp;T Network File Storage (nas1.bu.edu)<\/a> and <a href=\"https:\/\/www.bumc.bu.edu\/it\/support\/storage-options\/y-drive\/\">BUMC Y-Drive<\/a> services, but the same approach may be applied to other remote data shares.<\/p>\n<div class=\"bu_collapsible_container \" aria-live=\"polite\" data-customize-animation=\"false\"><h3 class=\"bu_collapsible\" aria-expanded=\"false\"tabindex=\"0\" role=\"button\">Using SCC OnDemand Interactive Desktop Session<\/h3><div class=\"bu_collapsible_section\" style=\"display: none;\"><\/p>\n<p>The SCC OnDemand Interactive Desktop app provides a file manager that supports network shares.<\/p>\n<ol>\n<li>Log into <a href=\"https:\/\/scc-ondemand.bu.edu\/\">SCC OnDemand<\/a> and Launch an Interactive Desktop Session<\/li>\n<li>Open the \u201cFile Manager\u201d<br \/>\n<img loading=\"lazy\" src=\"\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-1.png\" alt=\"\" width=\"872\" height=\"546\" class=\"alignnone size-full wp-image-148040\" srcset=\"https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-1.png 872w, https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-1-636x398.png 636w, https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-1-768x481.png 768w\" sizes=\"(max-width: 872px) 100vw, 872px\" \/><\/li>\n<li>Enter the path for the remote share.<br \/>\nThe path for the share will be comprised of the following components using the BUMC Y-Drive as an example. The path for the Y-Drive is `smb:\/\/ad.bu.edu\/bumcfiles`.<\/p>\n<ol>\n<li>The protocol: smb:\/\/<\/li>\n<li>The servername: ad.bu.edu<\/li>\n<li>The share name: bumcfiles<br \/>\n<img loading=\"lazy\" src=\"\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-2.1-1.png\" alt=\"\" width=\"872\" height=\"546\" class=\"alignnone size-full wp-image-148047\" srcset=\"https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-2.1-1.png 872w, https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-2.1-1-636x398.png 636w, https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-2.1-1-768x481.png 768w\" sizes=\"(max-width: 872px) 100vw, 872px\" \/><\/li>\n<\/ol>\n<\/li>\n<li>Enter your credentials for the share.<br \/>\nFor BU services, this will be your BU username and password with \u201cAD\u201d as the domain.<br \/>\n<img loading=\"lazy\" src=\"\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-3.1.png\" alt=\"\" width=\"872\" height=\"546\" class=\"alignnone size-full wp-image-148048\" srcset=\"https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-3.1.png 872w, https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-3.1-636x398.png 636w, https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-3.1-768x481.png 768w\" sizes=\"(max-width: 872px) 100vw, 872px\" \/><\/li>\n<li>Navigate the file manager to find your files on the remote share.<br \/>\n<img loading=\"lazy\" src=\"\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-4.png\" alt=\"\" width=\"872\" height=\"546\" class=\"alignnone size-full wp-image-148043\" srcset=\"https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-4.png 872w, https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-4-636x398.png 636w, https:\/\/www.bu.edu\/tech\/files\/2023\/10\/rcs_xferfiles_ood_desktop-4-768x481.png 768w\" sizes=\"(max-width: 872px) 100vw, 872px\" \/><\/li>\n<\/ol>\n<ol start=\"4\"><\/ol>\n<ul><\/ul>\n<p><\/div>\n<\/div>\n\n<div class=\"bu_collapsible_container \" aria-live=\"polite\" data-customize-animation=\"false\"><h3 class=\"bu_collapsible\" aria-expanded=\"false\"tabindex=\"0\" role=\"button\">Using <code>smbclient<\/code> from the Command Line<\/h3><div class=\"bu_collapsible_section\" style=\"display: none;\"><\/p>\n<p>The <code><span class=\"command\">smbclient<\/span><\/code> utility provides directory navigation and file transfer for remote file systems using the SMB\/CIFS protocol. Official documentation for <code><span class=\"command\">smbclient<\/span><\/code> can be found on the <a href=\"https:\/\/www.samba.org\/samba\/docs\/man\/\">Samba website<\/a> or directly from the SCC by typing <code><span class=\"command\">man smbclient<\/span><\/code>.<\/p>\n<p>The example below demonstrates a connection to the BUMC Y-Drive.<\/p>\n<ol>\n<li>Connect to a remote share\n<pre style=\"margin-bottom: 30px;\"><code class=\"code-block\" style=\"padding: 0em 1em 0em 1em;\"><span class=\"prompt\">scc1$<\/span> <span class=\"command\">smbclient<\/span> \\\\\\\\ad.bu.edu\\\\bumcfiles -W ad.bu.edu -U <span class=\"placeholder\">username<\/span>\r\n<span class=\"output\">Enter AD.BU.EDU\\<span class=\"placeholder\">username<\/span>\u2019s password:\r\nTry \"help\" to get a list of possible commands.<\/span>\r\n<span class=\"prompt\">smb: \\&gt;<\/span><\/code><\/pre>\n<\/li>\n<li>Navigate the share<br \/>\nUse the <code><span class=\"command\">smbclient<\/span><\/code> subcommands <code><span class=\"command\">ls<\/span><\/code> and <code><span class=\"command\">cd<\/span><\/code> to navigate the remote filesystem. Here we navigate to the SPH Environmental Health share and find <code>myfile.txt<\/code>. If the directory that you want to use has spaces, then the path should be put in quotes.<\/p>\n<pre style=\"margin-bottom: 30px;\"><code class=\"code-block\" style=\"padding: 0em 1em 0em 1em;\"><span class=\"prompt\">smb: \\&gt;<\/span> <span class=\"command\">cd<\/span> SPH\\EH\\\r\n<span class=\"prompt\">smb: \\SPH\\EH\\&gt;<\/span> <span class=\"command\">ls<\/span>\r\n<span class=\"output\">.            D  0  Mon Feb 14 09:35:09 2022\r\n..           D  0  Mon May 17 15:36:02 2021\r\nmyfile.txt   F  1  Thu Jan 01 00:00:00 1970<\/span>\r\n<\/code><\/pre>\n<\/li>\n<li>Transfer data<br \/>\nUse the <code><span class=\"command\">smbclient<\/span><\/code> subcommands <code><span class=\"command\">get<\/span><\/code> and <code><span class=\"command\">put<\/span><\/code> to transfer data to and from the SCC. The perspective for these sub-commands is from the SCC: a <code><span class=\"command\">get<\/span><\/code> copies the specified file from the remote share to the SCC directory and <code><span class=\"command\">put<\/span><\/code> will copy files from the SCC to the remote share. To transfer multiple files you can use the <code><span class=\"command\">mget<\/span><\/code> and <code><span class=\"command\">mput<\/span><\/code> commands.<\/p>\n<pre style=\"margin-bottom: 30px;\"><code class=\"code-block\" style=\"padding: 0em 1em 0em 1em;\"><span class=\"prompt\">smb: \\&gt;<\/span> <span class=\"command\">get<\/span> \"\\SPH\\EH\\myfile.txt\" \"\/projectnb\/mySCCproject\/myfile.txt\"\r\n<span class=\"output\">getting file \\SPH\\EH\\myfile.txt of size 18944 as \/projectnb\/mySCCproject\/myfile.txt<\/span><\/code><\/pre>\n<\/li>\n<li>Close the connection<br \/>\nUse the <code><span class=\"command\">exit<\/span><\/code> command or <code>Ctrl-D<\/code> to close the connection. You will see the standard SCC prompt once the program exits.<\/p>\n<pre style=\"margin-bottom: 30px;\"><code class=\"code-block\" style=\"padding: 0em 1em 0em 1em;\"><span class=\"prompt\">smb: \\&gt;<\/span> <span class=\"command\">exit<\/span>\r\n<span class=\"prompt\">scc1$<\/span> <\/code><\/pre>\n<\/li>\n<\/ol>\n<p><\/div>\n<\/div>\n\n<h2><span>Network File System (\u201cNFS\u201d) Shares<\/span><\/h2>\n<p>Network File System <span>(&#8220;NFS&#8221;)<\/span> shares on the BU network can be accessed from the SCC using the automounter. <span>The automounter utility is used to mount <a href=\"#NFSCONFIG\">appropriately configured NFS shares<\/a> from remote systems to the <code>\/net\/<\/code> directory on the SCC so that data can be transferred to and from the SCC. This utility is always available on the SCC and a researcher need only navigate to the mount point to access the data. At BU, common NFS shares include <a href=\"https:\/\/www.bu.edu\/tech\/services\/infrastructure\/storage-backup\/network-file-storage\/\">IS&amp;T Network File Storage<\/a>, the <a href=\"https:\/\/www.bu.edu\/engit\/services\/college-of-engineering-storage\/\">ENGNAS<\/a>\u00a0and lab workstations.<\/span><\/p>\n<div class=\"bu_collapsible_container \" aria-live=\"polite\" data-customize-animation=\"false\"><h3 class=\"bu_collapsible\" aria-expanded=\"false\"tabindex=\"0\" role=\"button\">Using the Automounter to connect to NFS shares<\/h3><div class=\"bu_collapsible_section\" style=\"display: none;\"><br \/>\nA researcher with access to a data share on <code><span class=\"placeholder\">myremotesystem<\/span><\/code> could do the following. In this example, <code><span class=\"placeholder\">myremotesystem<\/span><\/code> is the hostname or IP address of the remote file server. This directory will not appear in the <code>\/net<\/code> directory until it is accessed (e.g by using the <code><span class=\"command\">cd<\/span><\/code> or <code><span class=\"command\">ls<\/span><\/code> command) and the automounter mounts the share.<\/p>\n<pre style=\"margin-bottom: 30px;\"><code class=\"code-block\" style=\"padding: 0em 1em 0em 1em;\"><span class=\"prompt\">scc1$<\/span> <span class=\"command\">ls<\/span> \/net\/<span class=\"placeholder\">myremotesystem<\/span> <\/code><\/pre>\n<p>Data stored on the IS&amp;T Network File Storage service with NFS enabled can be accessed as follows. Note that not all IS&amp;T Network File Storage shares are NFS enabled and you may need to use the SMB instructions in the section above.<\/p>\n<pre style=\"margin-bottom: 30px;\"><code class=\"code-block\" style=\"padding: 0em 1em 0em 1em;\"><span class=\"prompt\">scc1$<\/span> <span class=\"command\">ls<\/span> \/net\/nas1.bu.edu\/proj\/<span class=\"placeholder\">PROJECT_GROUP_NAME<\/span> <\/code><\/pre>\n<p><\/div>\n<\/div>\n\n<p><a name=\"ENGNAS\"><\/a><div class=\"bu_collapsible_container \" aria-live=\"polite\" data-customize-animation=\"false\"><h3 class=\"bu_collapsible\" aria-expanded=\"false\"tabindex=\"0\" role=\"button\">Using the BU Engineering Share (ENGNAS)<\/h3><div class=\"bu_collapsible_section\" style=\"display: none;\"><\/p>\n<p>The College of Engineering Information Technology department provides a network file storage system called <a href=\"https:\/\/www.bu.edu\/engit\/services\/college-of-engineering-storage\/\">ENGNAS<\/a> to engineering research labs, courses, projects, student groups, and administrative units. The ENGNAS is accessible from the SCC via NFS as described in the previous section and is additionally linked to the path <code>\/ad\/eng\/<\/code> on the SCC for convenience. The <code>\/ad\/eng\/<\/code> path mirrors that of ENGNAS on ENG-Grid and on ENG-IT managed workstations to allow researchers to seamlessly move between computing environments using the same paths for scripts, data, and software. Additional information for the organization of this network share is detailed on the <a href=\"https:\/\/www.bu.edu\/engit\/knowledge-base\/grid\/file-storage\/\">ENG-IT File Storage Knowledgebase<\/a>.<\/p>\n<p>The ENG-GRID file system is accessible through\u00a0<code>\/ad\/eng\/<\/code>:<\/p>\n<pre><code class=\"code-block\"><span class=\"prompt\">scc1$<\/span> <span class=\"command\"> ls -l \/ad\/eng <\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 25 Jun 10 2019 bin      -&gt; \/net\/engnas\/eng_linux\/bin<\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 19 Jan 12 2020 courses  -&gt; \/net\/engnas\/Courses<\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 25 Jun 10 2019 etc      -&gt; \/net\/engnas\/eng_linux\/etc<\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 25 Jun 10 2019 lib      -&gt; \/net\/engnas\/eng_linux\/lib<\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 25 Jun 10 2019 opt      -&gt; \/net\/engnas\/eng_linux\/opt<\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 25 Jun 10 2019 pkg      -&gt; \/net\/engnas\/eng_linux\/pkg<\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 27 Jun 10 2019 pkg.7    -&gt; \/net\/engnas\/eng_linux\/pkg.7<\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 20 Jan 12 2020 projects -&gt; \/net\/engnas\/Projects<\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 20 Jan 12 2020 research -&gt; \/net\/engnas\/Research<\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 26 Jun 10 2019 sbin     -&gt; \/net\/engnas\/eng_linux\/sbin<\/span>\r\n<span class=\"output\">lrwxrwxrwx. 1 root root 17 Jan 12 2020 users    -&gt; \/net\/engnas\/Users<\/span><\/code><\/pre>\n<p><\/div>\n<\/div>\n<\/p>\n<div class=\"bu_collapsible_container \" aria-live=\"polite\" data-customize-animation=\"false\"><h3 class=\"bu_collapsible\" aria-expanded=\"false\"tabindex=\"0\" role=\"button\">NFS Configuration Notes<\/h3><div class=\"bu_collapsible_section\" style=\"display: none;\"><br \/>\n<a name=\"NFSCONFIG\"><\/a>The following instructions are intended for system administrators of remote file servers that need to be accessed from the SCC. Research Computing staff are unable to directly assist with configuration of remote file servers.<\/p>\n<h4>Required Steps<\/h4>\n<ol>\n<li>The remote filesystem must be on a BU network.<\/li>\n<li>Enable NFS on the remote system.<\/li>\n<li>NFS export the desired volume or directory on the remote system to the network ranges below.\n<ul>\n<li><code>192.12.187.128\/26<\/code> &#8212; Login Nodes (scc1, scc2, geo) and public gateway to compute nodes.<\/li>\n<li><code>10.48.225.0\/24<\/code> &#8212; SCC4, VirtualGL nodes, and private gateway to compute nodes.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<h4>Important considerations and notes<\/h4>\n<p>The remote filesystem should use BU usernames and group IDs, user accounts should have group membership matching that of the SCC, and the permissions of exported directories should be reviewed. Inconsistent IDs, group membership, and permissions can cause data access or exposure issues. If the user accounts on the remote file system are not BU compliant, then it may be needed to set squash on the export and map the user or group ids to something known on the SCC.<br \/>\n<\/div>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"<p>Researchers on the SCC can connect to remote file systems on the BU network for the purpose of transferring data. The steps to connect to these shares will depend on how the remote system exports the data share. Two common protocols and associated applications are described on this page. Windows SMB\/CIFS Shares (e.g. IS&amp;T NAS1,&#8230;<\/p>\n","protected":false},"author":1692,"featured_media":0,"parent":137947,"menu_order":5,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/139572"}],"collection":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/users\/1692"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/comments?post=139572"}],"version-history":[{"count":25,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/139572\/revisions"}],"predecessor-version":[{"id":159656,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/139572\/revisions\/159656"}],"up":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/137947"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/media?parent=139572"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}