Most traditional computing clusters (including the SCC) are designed primarily for use cases that have the following features:

  • Run via batch jobs or interactive jobs with a fixed running time limit.
  • Compute resources are generally available only to the particular person that requested them.
  • Run on the native host hardware/software of the cluster.

The above model serves a broad range of needs, but does not map well to certain types of use cases:

  • Persistent servers or databases.
  • Persistent websites or data that needs to be accessible to the public or to a selected group.
  • Tasks that require fine grained control or administrator privleges to the host system.
  • Specific operating system or other unusual software needs that can’t be accommodated by a Singularity container.

Use cases that fall into the above categories do not map well to the more traditional cluster computing structure of the SCC, but may be accommodated by running on the Jetstream2 system.

Sections

 

Jetstream2

Jetstream2 describes itself as:

Cloud-based and on-demand, the 24/7 system includes discipline-specific apps. You can even create virtual machines that look and feel like your lab workstation or home machine, with thousands of times the computing power.

The technical details for each of the node types on Jetstream2 can be found on their Technical Details page. Broadly the system has the following types of nodes:

  • Compute nodes
  • Large Memory nodes
  • GPU nodes

 

Getting Started on Jetstream2

Broadly you will need to do the following steps to run on Jetstream2:

  1. Login to Jetstream2
  2. Create your instance
  3. Install/configure necessary software
  4. Create an image of your configured instance for re-use

You can find more detail about each of these steps and more at the Getting Started page.

Logging in to Jetstream2

The Jetstream2 documentation has a helpful page that details what is needed. You will need to make sure you have used ACCESS credits to receive SUs on Jetstream2. Next you will need to log in to Jetstream2. There are several interfaces one could use, though a beginner-friendly option is Exosphere. Follow the steps on the Exosphere login page to successfully log in to Jetstream2 via Exosphere. Note that during this process you will need to select a login method and the allocation under which you plan to work.

Create your Instance

After logging in and selecting the allocation to work under, you can now select this allocation to create your instance under. You can read in greater detail about the process of creating instances on the Create Instance documentation page. A reasonable first option to play around with is:

Instance > By Type > AlmaLinux8 > Flavor: m3.tiny

Wait around 20 minutes for your instance to build. Once the status for your instance shows “Ready” you can now connect to it: Connect to > Web Shell. This gives you a shell in the instance where you can now install and/or configure the necessary software.

Install/configure Necessary Software

The general page on Installing Software on Jetstream2 contains details about installing software in general. As you will have access to “sudo” privileges within your instance you will be able to install anything needed. However the most common software is already available in any of the featured instances via the Jetstream2 “Software Collection”, you can find more details about this on the Using Software page.

Create an Image for Re-use

Once your instance is configured to your liking and ready to use, you can save a snapshot of it as an “image”. Then in the future you can use this image to create new instances that will exactly replicate the state of the instance when you took a snapshot. More details about this process can be found on the Snapshots and Images page. Pay careful attention to the warnings documented on the page as several steps are necessary to ensure your newly created image works as expected when used.

 

Example Use Case

To provide a concrete example of how one might use Jetstream2, let’s examine a possible use case. Suppose you have a database that contains data that you would like to share publicly. To do this we will need to:

  1. Create a website
  2. Create a database
  3. Make the database accessible from the website
  4. Make the website publicly accessible

There are many ways to achieve this, but one approach is to use Flask, SQLite, and Python.

Example code is available that implements the above idea and can be found in this Github repo. You can download or clone this repo to your Jetstream2 instance and try it out. This also can serve as a template for those looking to implement similar functionality for themselves.
 

Last updated: Loading…