This page imported from: /afs/bu.edu/cwis/webuser/web/s/c/scv/documentation/tutorials/MPI/alliance/communicators/MPI_Comm_split.html

MPI_Comm_split

MPI_Comm_split



MPI_Comm_split forms new communicators from an existing one.

In many scientific and engineering computations, we often deal with matrices or
grids, especially cartesian grids, with rows and columns. On occasions however,
we may need to deal with them in less traditional manners. For
example, instead of dealing with individual rows, it may be necessary to deal with
groups of rows or even more generally, other arbitrary configurations.
MPI_Comm_split permits the creation of new communicators with such flexibilities.

The input variable color identifies the group while
the key variable specifies a member of the group.

Fortran Syntax

Subroutine MPI_Comm_split(old_comm, color, key, new_comm, ierr)

C Syntax

int MPI_Comm_split(MPI_Comm old_comm, int color, int key, MPI_Comm *new_comm)

Example in Fortran

For a 2D logical grid, create subgrids of rows and columns


c**logical 2D topology with nrow rows and mcol columns
      irow = Iam/mcol       !! logical row number
      jcol = mod(Iam, mcol) !! logical column number
      comm2D = MPI_COMM_WORLD

      call MPI_Comm_split(comm2D, irow, jcol, row_comm, ierr)
      call MPI_Comm_split(comm2D, jcol, irow, col_comm, ierr)

To demonstrate the results of this example, lets say that we have
6 processes (0, 1, …, 5) at our disposal. Mathematically (and
topologically), we think of these
processes as arranged in a 3-by-2 logical grid, as shown in
Figure a below.
The number in parentheses represents the rank number associated with the
logical grid. Irow and jcol, both functions of the calling process number,
Iam, are defined as the row and column numbers respectively.
A tabulation of irow
and jcol versus Iam is shown below:

Iam 0 1 2 3 4 5
irow 0 0 1 1 2 2
jcol 0 1 0 1 0 1

The first MPI_Comm_split call specifies irow as the “color” (or group)
with jcol as the “key” (or distinct member identity within group). This results
in processes on each row classified as a separate group, as shown in Figure b.
The second MPI_Comm_split call on the other hand defines jcol as the
color and irow as the key. This joins all processes in
a column as belonging to a group, as shown in Figure c.

Figure a.
2D logical Grid
(0)
(0)		 (1)
(1)
(2)
(2)		 (3)
(3)
(4)
(4)		 (5)
(5)
Figure b.
3 Row Subgrids
(0)
(0)		 (1)
(1)
(2)
(0)		 (3)
(1)
(4)
(0)		 (5)
(1)
Figure c.
2 Column Subgrids
(0)
(0)		 (1)
(0)
(2)
(1)		 (3)
(1)
(4)
(2)		 (5)
(2)

Here, the rank numbers (shown above in parentheses) are assigned using C’s
“row-major” rule. The numbers in black represent the “old” rank numbers
(see Figure a) while those in green denote the rank numbers
for the individual row groups (see Figure b) and column groups (see Figure c).

Note that in the above, we chose to think of the logical grids as
two-dimensional, they could very well be thought of as

Figure d.
1D logical Grid
(0)
(0)		 (1)
(1)		 (2)
(2)		 (3)
(3)		 (4)
(4)		 (5)
(5)
Figure e.
3 Subgrids
(0)
(0)		 (1)
(1)		 (2)
(0)		 (3)
(1)		 (4)
(0)		 (5)
(1)
Figure f.
2 Subgrids
(0)
(0)		 (1)
(0)		 (2)
(1)		 (3)
(1)		 (4)
(2)		 (5)
(2)

As another example of MPI_Comm_split, lets split the rows into two groups
of rows: the first consists of rows 1 and 2 while the second represents row
3. The code fragment that does that is: 


C**MPI_Comm_split is more general than MPI_Cart_sub
C**simple example of 6 processes divided into 2 groups;
C**1st 4 belongs to group 0 and remaining two to group 1
      group = Iam/4               ! group0:0,1,2,3; group1:4,5
      index = Iam - row_group*4   ! group0:0,1,2,3; group1:0,1
      call MPI_Comm_split(comm2D, group, index, 
     &                    row_comm, ierr)

The above table is illustrated in the figure below and the output of the
fortran code for this particular arrangement is shown below:

Figure g. Another example.
(0)
(0)		 (1)
(1)
(2)
(2)		 (3)
(3)
(4)
(0)		 (5)
(1)

We have just demonstrated the use of MPI_Comm_split to divide a logical grid into
arbitrary subgrids.

Note that:

  • This routine is similar to
    MPI_Cart_sub.

    • MPI_Comm_split is more general than MPI_Cart_create.
    • MPI_Comm_split creates logical grid and is referred to by
      its linear rank number; MPI_Cart_sub creates cartesian grid
      and rank can be referred to by cartesian coordinates.
      For instance, for a 2D cartesian grid, a grid cell is
      known by its (irow,jcol) index pair.
  • If two processes have the same key, the resulting rank numbers
    of these two processes in the new communicator are ordered
    according to their respective ranks in the old communicator.
    [Recalls that ranks in a communicator are by definition unique
    and ordered as (0, 1, 2, …, p-1).
    (See example)

  • A corollary of the above is that if no specific ordering
    is required of the new communicator, setting key to a
    constant results in MPI to order their new ranks following
    their rank order in the old communicator.