LoadLeveler Basics

Description

LoadLeveler is the batch system used on the IBM BlueGene. LoadLeveler schedules jobs, and provides functions for building, submitting, and processing jobs in a dynamic environment.

Availability and Setup

LoadLeveler is available exclusively on the BlueGene system. All BlueGene jobs must be submitted through the LoadLeveler batch system and run on the back-end.

Using LoadLeveler

LoadLeveler Commands for Users:
Command Purpose
llsubmit Submit a job
llq

Query your job(s) status in queue
llstatus Query machine status
llcancel Cancel a submitted job (query JobID with llq)
llhold Hold or release a submitted job

For more details on LoadLeveler, please see Workload Management With LoadLeveler.

  • Job Submission Example
    With a batch script, say, myscript, job is submitted through llsubmit as follows:

       lee % llsubmit myscript
   llsubmit: The job "fe1.bgl.bu.edu.162" has been submitted.
  • llhold Usage Example
    A submitted job may be put on hold for various reasons. A user can use llhold to withhold jobs. Likewise, user-withheld jobs can be released from hold status with llhold.

    lee % llq
Id                     Owner    Submitted   ST PRI Class    Running On
---------------------- -------- ----------- -- --- -------- --------------
fe1.bgl.bu.edu.162.0   kadin     9/26 08:44 H  50  bgl      fe1.bgl.bu.edu

 lee % llhold -r fe1.bgl.bu.edu.162

    Please note the omission of the suffix “.0″ when using llhold.

  • Miscellaneous BG commands:
    • lee % bglstat
      Shows current allocation of the Blue Gene machine.
    • lee % qstat
      Similar to llq but more details.
  • Batch Job Rules

    There are four batch configurations available in the BG: 32-, 128-, 512-, and 1024-node clusters. Under the default COprocessor mode, one processor of a node is dedicated to communication and the other is for computing. From a computational standpoint, a node is hence synonymous with a processor in the coprocessor mode. Depending on the requested number of processors, LoadLeveler will automatically assign the job to the smallest cluster that can accommodate your processor request. For example, a job requesting, say, 100 processors will be assigned the 128-node cluster, although only 100 will be used. Similarly, a job requesting 16 processors will be assigned a 32-node cluster.

    If Virtual Node mode (i.e., -mode VN) is used, both processors of a node are used for computation. A job requesting, say, 1000 processors will be assigned a 512-node cluster.

Additional Help/Documentation

Additional details are in the IBM documentation on LoadLeveler.

Document Name: loadleveler
Author/Maintainer: Aaron D. Fuegi (aarondf@bu.edu) and Kadin Tseng (kadin@bu.edu)
Executable: /opt/ibmll/LoadL/full/bin/llsubmit, /opt/ibmll/LoadL/full/bin/llq, /opt/ibmll/LoadL/full/bin/llstatus, /opt/ibmll/LoadL/full/bin/llcancel, /opt/ibmll/LoadL/full/bin/llhold
Keywords: load, sharing, batch
Machines List: IBM Blue Gene
Related Man Pages: llsubmit, llq, llstatus, llcancel, llhold
Created July 13, 2007; Last Revised July 13, 2007; Last Modified February 12, 2009
URL of this document: http://www.bu.edu/tech/research/training/scv-software-packages/loadleveler/
Go up to Software Help Pages