Types of Parallel Jobs
- Distributed, or task-parallel, jobs:
A distributed job is a job with:
Multiple tasks running independently on multiple workers with no information passed among them. On Katana, a distributed job is a series of single-processor batch jobs. This is also known as task-parallel, or “embarrassingly parallel,” jobs.
Examples of distributed jobs are Monte Carlo simulations and image processing. The primary PCT utility for this type of application is dfeval. More on this and other PCT utility functions later …
- Parallel, or data-parallel, jobs:
A parallel job is:
A single task running concurrently on multiple workers that may communicate with each other. On Katana, this results in one batch job with multiple processors running in parallel. This is also known as a data-parallel job.
Examples of a parallel job include many linear algebra applications: matrix multiply, linear system of equation solvers, or Eigen solvers. Some of these applications may run efficiently in parallel while others may not. It depends on the underlying algorithms and operations. This also include jobs that mix serial and parallel processing. Some of the more important PCT utilities for these types of jobs are spmd, parfor, drange.