{"id":150677,"date":"2024-03-01T14:45:32","date_gmt":"2024-03-01T19:45:32","guid":{"rendered":"http:\/\/www.bu.edu\/tech\/?page_id=150677"},"modified":"2024-07-12T10:52:46","modified_gmt":"2024-07-12T14:52:46","slug":"mpi","status":"publish","type":"page","link":"https:\/\/www.bu.edu\/tech\/support\/research\/computing-resources\/external\/access\/mpi\/","title":{"rendered":"ACCESS MPI Resources"},"content":{"rendered":"<p><a name=\"top\"><\/a><\/p>\n<style>\n  p { text-align: justify; }\n  .image-container { float: right; width: 48%; margin-left: 20px; }\n<\/style>\n<div style=\"width:100%;\">\n<div class=\"image-container\">\n    <img loading=\"lazy\" src=\"\/tech\/files\/2024\/03\/stampede3-pressrelease-banner.jpg__1200x1200_q85_subsampling-2-e1711916380956.jpg\" alt=\"\" width=\"600\" height=\"360\" class=\"alignnone size-full wp-image-151293\" \/>\n  <\/div>\n<p>If your main focus is on running MPI computations, we suggest allocating your project Service Units (SUs) on <a href=\"https:\/\/docs.tacc.utexas.edu\/hpc\/stampede3\/\">Stampede3<\/a>. After researching all available ACCESS resource options, we&#8217;ve found that Stampede3 provides the largest node configuration, with queues divided into three options based on the number of nodes that can be requested in a single job. You can explore other options on the <a href=\"https:\/\/allocations.access-ci.org\/resources\">ACCESS Resource list<\/a>.<\/p>\n<p><strong>Sections<\/strong><\/p>\n<ul>\n<li><a href=\"#stampede\">Stampede3<\/a><\/li>\n<li><a href=\"#start\">Getting Started on Stampede3<\/a><\/li>\n<li><a href=\"#help\">Stampede3 Support<\/a><\/li>\n<li><a href=\"#example\">Example MPI Job on Stampede3<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p><a name=\"stampede\"><\/a><\/p>\n<h2>Stampede3<\/h2>\n<p>Stampede3 is a compute resource funded by the NSF and administered by the Texas Advanced Computing Center (<a href=\"https:\/\/docs.tacc.utexas.edu\/\">TACC<\/a>) at the University of Texas at Austin. <\/p>\n<div style=\"margin-left: 20px;\">\n<h3>Architecture<\/h3>\n<p>The cluster is composed of 3 CPU models:<\/p>\n<ul>\n<li><a href=\"https:\/\/docs.tacc.utexas.edu\/hpc\/stampede3\/#system-skx\">Skylake<\/a><\/li>\n<li><a href=\"https:\/\/docs.tacc.utexas.edu\/hpc\/stampede3\/#system-spr\">Sapphire Rapid<\/a><\/li>\n<li><a href=\"https:\/\/docs.tacc.utexas.edu\/hpc\/stampede3\/#system-icx\">Icelake<\/a><\/li>\n<\/ul>\n<h3>Queues<\/h3>\n<p>These resources are accessible via Stampede3&#8217;s Slurm Workload Manager. Execute <b><code><span class=\"code\">qlimits<\/b><\/span><\/code> for real-time information regarding limits on available queues. An example output is copied below, where the columns are:<\/p>\n<ul>\n<li><b>MinNode<\/b>&nbsp;: minimum number of nodes per job<\/li>\n<li><b>MaxNode<\/b>&nbsp;: maximum number of nodes allowed per job<\/li>\n<li><b>MaxWall<\/b>&nbsp;: maximum wall-clock time allowed per job<\/li>\n<li><b>MaxNodePU<\/b>&nbsp;: maximum number of nodes allowed per user concurrently<\/li>\n<li><b>MaxJobsPU<\/b>&nbsp;: maximum number of jobs allowed per user running concurrently<\/li>\n<li><b>MaxSubmit<\/b>&nbsp;: maximum number of jobs allowed per user running &#038; queued concurrently<\/li>\n<\/ul>\n<pre><code class=\"code-block\"><span class=\"command\">Name             MinNode  MaxNode     MaxWall  MaxNodePU  MaxJobsPU   MaxSubmit\r\nicx                    1       16  1-00:00:00         24          6          20\r\nskx                    1       32  2-00:00:00         64         20          50\r\nskx-dev                1        4    02:00:00          6          1           3\r\n<\/span><\/code><\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<p><a name=\"start\"><\/a><\/p>\n<h2>Getting Started on Stampede3<\/h2>\n<ol>\n<li> After allocating your project SUs to Stampede3, login with your ACCESS credentials to the <a href=\"https:\/\/tacc.utexas.edu\/portal\/login\">TACC portal<\/a><\/li>\n<li> Once logged in, you must setup <a href=\"https:\/\/docs.tacc.utexas.edu\/basics\/mfa\">multi-factor authentication<\/a> under the <strong>Manage Account<\/strong> menu.<\/li>\n<li> The standard way to connect to Stampede3 is via <b><code><span class=\"code\">ssh<\/span><\/code><\/b>:\n<pre><code class=\"code-block\"><span class=\"prompt\">[localhost$ ~] <\/span><span class=\"command\">ssh<\/span> <span class=\"placeholder\">myusername<\/span><span class=\"command\">@stampede3.tacc.utexas.edu<\/span><\/code><\/pre>\n<\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<p><a name=\"help\"><\/a><\/p>\n<h2>Stampede3 Support<\/h2>\n<p>The help desk for Stampede3 is TACC Consulting which operates 8am to 5pm CST, Monday through Friday, except for holidays. You can <a href=\"https:\/\/tacc.utexas.edu\/about\/help\/\">submit a help desk ticket<\/a> at any time via the TACC User Portal with <strong>Stampede3<\/strong> in the resource field. While the consultants can address system issues and answer questions about Stampede3, they can&#8217;t teach parallel programming in a ticket. They may offer general advice that will help you build, debug, optimize, or modify your code, but you shouldn&#8217;t expect them to do these things for you.<br \/>\n&nbsp;<\/p>\n<p><a name=\"example\"><\/a><\/p>\n<h2>Example MPI Job on Stampede3<\/h2>\n<p>Refer to our <a href=\"https:\/\/github.com\/bu-rcs\/access-use-case-demos\/tree\/main\/MPI\">ACCESS GitHub page<\/a> for an example MPI job submission.<br \/>\n&nbsp;<\/p>\n<hr width=\"50%\">\n<div style=\"overflow: hidden;\">\n<div style=\"float: left;\">\n        <i>This page was heavily inspired by the <a href=\"https:\/\/docs.tacc.utexas.edu\/hpc\/stampede3\/\">Stampede3 User Guide at UT-Austin<\/a>.<\/i>\n    <\/div>\n<\/div>\n<div style=\"float: right;\">\n<p id=\"last-modified-timestamp\" style=\"margin: 0;\">Last updated: Loading&#8230;<\/p>\n<\/p><\/div>\n<p><script>\r\ndocument.addEventListener('DOMContentLoaded', function() {\r\n    \/\/ Get the content of the meta tag\r\n    var lastUpdatedContent = document.querySelector('meta[name=\"last-updated\"]').content;\r\n    \r\n    \/\/ Parse the content into a Date object\r\n    var lastUpdatedDate = new Date(lastUpdatedContent);\r\n    \r\n    \/\/ Format the date\r\n    var formattedDate = lastUpdatedDate.toLocaleDateString(undefined, { year: 'numeric', month: 'long', day: 'numeric' });\r\n    \r\n    \/\/ Update the HTML element with the formatted date\r\n    document.getElementById('last-modified-timestamp').innerHTML = 'Last updated: ' + formattedDate;\r\n});\r\n<\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>If your main focus is on running MPI computations, we suggest allocating your project Service Units (SUs) on Stampede3. After researching all available ACCESS resource options, we&#8217;ve found that Stampede3 provides the largest node configuration, with queues divided into three options based on the number of nodes that can be requested in a single job&#8230;.<\/p>\n","protected":false},"author":1692,"featured_media":0,"parent":47915,"menu_order":3,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/150677"}],"collection":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/users\/1692"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/comments?post=150677"}],"version-history":[{"count":28,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/150677\/revisions"}],"predecessor-version":[{"id":152222,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/150677\/revisions\/152222"}],"up":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/47915"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/media?parent=150677"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}