MPI jobs typically require a lot of computational resources. Running MPI jobs on login nodes slows down the system and impacts the works of other users. Therefore it is highly recommended to run MPI jobs on compute nodes. You can submit a batch job for productive and massive MPI runs. If you want to run MPI jobs interactively, for example to debug or test MPI programs, you can request an interactive session to do that.

How to run MPI jobs in an interactive session

To start an interactive MPI session, use qrsh command with -pe mpi_#_tasks_per_node option, where # is the number of MPI tasks per node. Based on the currently available resources on SCC, the number denoted by the symbol # in the above flag could be 4, 8, 12, 16 or 28. For example, to request one compute node with 16 CPU cores:

scc % qrsh -pe mpi_16_tasks_per_node 16 

To request an interactive session with multiple nodes, specify the total number of MPI tasks to be a multiple of 4, 8, 12, 16 or 28. For example, to request two 16-core compute nodes:

scc % qrsh -pe mpi_16_tasks_per_node 32 

Since MPI queues on SCC are busy for most of the time, it is likely that your MPI interactive session can not start immediately. If so, you can use -now n option to make your job waiting in the queue until the requested resources are available:

scc % qrsh -pe mpi_16_tasks_per_node 32 -now n

In an interactive session, necessary MPI environment variables have been set by the system, so you can run an MPI program with mpirun, simply specifying the total number of MPI tasks with the flag -np (or -n):

scc-ha1 % mpirun -np 32 ./your_mpi_binary

Note that the host file required to run an MPI program on multiple nodes is automatically handled by the system and you do not need to explicitly provide the host file for mpirun.

Please refer to this page for MPI programming and examples.

For further questions on running MPI jobs in an interactive session, please contact us at help@scc.bu.edu .