{"id":99034,"date":"2016-05-10T14:01:36","date_gmt":"2016-05-10T18:01:36","guid":{"rendered":"http:\/\/www.bu.edu\/tech\/?page_id=99034"},"modified":"2021-04-22T16:09:55","modified_gmt":"2021-04-22T20:09:55","slug":"sizing-guidelines","status":"publish","type":"page","link":"https:\/\/www.bu.edu\/tech\/services\/infrastructure\/storage-backup\/network-file-storage\/sizing-guidelines\/","title":{"rendered":"Sizing Guidelines"},"content":{"rendered":"<h3>Summary<\/h3>\n<p><strong>We recommend requesting at least 25% more storage space than the size of the files that you&#8217;d like to store. <\/strong><\/p>\n<p>File size can be greatly reduced by compressing your files using an archive format such as .zip, .tar, or .cpio, and file compression is a particularly good idea if you don&#8217;t access the contents of those files very often.<\/p>\n<p>If you&#8217;d like more information on determining file size or compressing files, assistance can be <a href=\"\/tech\/contact\/\" class=\"myhelp_activate\">provided by the IT Help Center<\/a> or your <a href=\"https:\/\/www.bu.edu\/tech\/support\/other\/\">local IT support organization<\/a>. Also <a href=\"http:\/\/www.ehow.com\/how_2093605_compress-files-before-sending-them.html\">basic instructions for using .zip can also be found online.<\/a><\/p>\n<div class=\"bu_collapsible_container \" aria-live=\"polite\" data-customize-animation=\"false\"><h3 class=\"bu_collapsible\" aria-expanded=\"false\"tabindex=\"0\" role=\"button\">Technical Details &amp; Background Information for Administrators<\/h3><div class=\"bu_collapsible_section\" style=\"display: none;\"><\/p>\n<p>Content transferred to the Network File Storage\u00a0service is allocated on disk using\u00a0the physical storage requirement, which includes data protection blocks.\u00a0This will be at least 25% greater than the logical storage size reported on a traditional filesystem, which does not report parity\/raid overhead (most desktop operating systems will report storage usage this way). If you store files with average size exceeding 4MB, expect your content to have an overhead requirement of 25-30%. (See background information below for additional details.)<\/p>\n<p>If you have multiple terabytes, quota allocations for Network File Storage\u00a0\u201cterabytes\u201d are calculated using powers of 2, not base-10 calculation typically used by disk vendors. This is more formally called a\u00a0<a href=\"http:\/\/en.wikipedia.org\/wiki\/Tebibyte\">tebibyte<\/a> and is about 10% larger than a \u201cterabyte:\u201d<\/p>\n<ul>\n<li>1000 B\/kB * 1000 kB\/MB * 1000MB\/GB * 1000 Gb\/Tb\u00a0 = 10<sup>12<\/sup><\/li>\n<li>1024 B\/KiB * 1024 KiB\/MiB* 1024 MiB\/GiB * 1024 GiB\/TiB = 2<sup>40<\/sup><\/li>\n<\/ul>\n<p>The data copied into the Network File Storage shares\u00a0using native formats have had varied results from 26% (combination of full and incremental backup volumes), to 35% (native format A\/V classroom recordings), to 300% (sample with\u00a0millions of very small, 12k files not bundled into .tar or .zip archive format).\u00a0If you have large quantities of data which are structured as relatively small files, plan to store your data using backup images or archive file formats like .zip, .tar, or .cpio \u00a0to achieve 4Mb+ file sizes (the most efficient to store).<\/p>\n<p>The Isilon storage filesystem (OneFS) organizes disk space using traditional 8k disk blocks and exposes the ability to select the degree of \u201craid\u201d protection on a folder-by-folder basis. An Isilon fileserver deployment is assembled as a cluster of storage nodes with a high-speed backplane for internal communication. The top two levels of the filesystem are replicated across all storage nodes. Boston University\u2019s Isilon\u00a0deployment uses \u201cN:2+1\u201d protection level, which Isilon recommends for archive and near-line storage deployments. The term \u201cN:2+1\u201d means that data will remain available without interruption or data loss if any two disks or a disk and a node(CPU+36 disks) fail at the same time. \u00a0A benefit of the implementation is that recovery from disk failure is much faster and less I\/O intensive than is the case for traditional hardware raid implementations; this reduces the probability that a second disk will fail during recovery operation.<\/p>\n<p>The OneFS filesystem associates data blocks required to meet protection level for each file and directory as part of the stored object.\u00a0 This mechanism is relatively efficient for files that are 4Mb or larger which, with 5-node cluster, will require 25% physical storage \u201coverhead\u201d to meet protection requirement. Smaller files require more physical data blocks as a percentage of file size in 8k blocks and files less than 128k are stored as a 3-way mirror to guarantee N:2+1 protection.<\/p>\n<p><\/div>\n<\/div>\n\n","protected":false},"excerpt":{"rendered":"<p>Summary We recommend requesting at least 25% more storage space than the size of the files that you&#8217;d like to store. File size can be greatly reduced by compressing your files using an archive format such as .zip, .tar, or .cpio, and file compression is a particularly good idea if you don&#8217;t access the contents&#8230;<\/p>\n","protected":false},"author":1301,"featured_media":0,"parent":93732,"menu_order":2,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/99034"}],"collection":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/users\/1301"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/comments?post=99034"}],"version-history":[{"count":3,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/99034\/revisions"}],"predecessor-version":[{"id":99350,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/99034\/revisions\/99350"}],"up":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/93732"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/media?parent=99034"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}