If your main focus is on running MPI computations, we suggest allocating your project Service Units (SUs) on Stampede3. After researching all available ACCESS resource options, we’ve found that Stampede3 provides the largest node configuration, with queues divided into three options based on the number of nodes that can be requested in a single job. You can explore other options on the ACCESS Resource list.

Sections

 

Stampede3

Stampede3 is a compute resource funded by the NSF and administered by the Texas Advanced Computing Center (TACC) at the University of Texas at Austin.

Architecture

The cluster is composed of 3 CPU models:

Queues

These resources are accessible via Stampede3’s Slurm Workload Manager. Execute qlimits for real-time information regarding limits on available queues. An example output is copied below, where the columns are:

  • MinNode : minimum number of nodes per job
  • MaxNode : maximum number of nodes allowed per job
  • MaxWall : maximum wall-clock time allowed per job
  • MaxNodePU : maximum number of nodes allowed per user concurrently
  • MaxJobsPU : maximum number of jobs allowed per user running concurrently
  • MaxSubmit : maximum number of jobs allowed per user running & queued concurrently
Name             MinNode  MaxNode     MaxWall  MaxNodePU  MaxJobsPU   MaxSubmit
icx                    1       16  1-00:00:00         24          6          20
skx                    1       32  2-00:00:00         64         20          50
skx-dev                1        4    02:00:00          6          1           3

 

Getting Started on Stampede3

  1. After allocating your project SUs to Stampede3, login with your ACCESS credentials to the TACC portal
  2. Once logged in, you must setup multi-factor authentication under the Manage Account menu.
  3. The standard way to connect to Stampede3 is via ssh: 
    [localhost$ ~] ssh myusername@stampede3.tacc.utexas.edu

 

Stampede3 Support

The help desk for Stampede3 is TACC Consulting which operates 8am to 5pm CST, Monday through Friday, except for holidays. You can submit a help desk ticket at any time via the TACC User Portal with Stampede3 in the resource field. While the consultants can address system issues and answer questions about Stampede3, they can’t teach parallel programming in a ticket. They may offer general advice that will help you build, debug, optimize, or modify your code, but you shouldn’t expect them to do these things for you.
 

Example MPI Job on Stampede3

Refer to our ACCESS GitHub page for an example MPI job submission.
 

This page was heavily inspired by the Stampede3 User Guide at UT-Austin.

Last updated: Loading…