MPI_Cart_create

MPI_Cart_create

MPI_Cart_create creates a new communicator using a cartesian topology.

Fortran Syntax

Subroutine MPI_Cart_create(old_comm, ndims, dim_size, periods, reorder, new_comm, ierr)

C Syntax

int MPI_Cart_create(MPI_Comm old_comm, int ndims, int *dim_size,
int *periods, int reorder, MPI_Comm *new_comm)

Example in Fortran


      include "mpif.h"
      integer old_comm, new_comm, ndims, reorder, ierr
      integer dim_size(2)
      logical periods(0:1)

      old_comm = MPI_COMM_WORLD
      ndims = 2            ! 2D matrx/grid
      dim_size(1) = 3      ! rows
      dim_size(2) = 2      ! columns
      periods(0) = .true.  ! row-periodic (each column wraps around)
      periods(1) = .false. ! column-nonperiodic
      reorder = 1

      call MPI_Cart_create(old_comm, ndims, dim_size,
     &       periods, reorder, new_comm, ierr)

In the above example we use MPI_Cart_create to map (or rename) 6 processes from a linear ordering (<i.e., 0,1,2,3,4,5) into a two-dimensional matrix ordering of 3 rows by 2 columns ( i.e., (0,0), (0,1), …, (2,1) ). Figure a below depicts the resulting cartesian grid representation for the processes. The index pair “i,j” represent row “i” and column “j”. The number in parentheses represents the (linear) rank number corresponding to the specific cartesian coordinates.

Figure a. Cartesian Grid
0,0 (0) 0,1 (1)
1,0 (2) 1,1 (3)
2,0 (4) 2,1 (5)

With processes renamed in a 2D grid topology, we will be able to
assign or distribute work, or distinguish among the processes by their grid topology rather than by their linear process ranks. Additionally, we imposed periodicity along the first dimension ( periods(0)=.true. ) which means that any reference beyond the first or last entry of any row will be wrapped around cyclically. For example, row index i=-1, due to periodicity, is mapped into i=2; similarly, i=-2 is mapped to i=1. Likewise, i=3 is the same as i=0. There is no periodicity imposed on the second dimension ( periods(1)=.false. ). Any reference to the column index outside of its defined range ( in this case 0 to 1) will result in a negative process rank (equal to MPI_PROC_NULL which is -1) which signifies that it is out of range.

Similarly, if periodicity is defined only for the column index (i.e., periods(0)=.false.; periods(1)=.true.), each row will wrap around itself. The effects of periodic columns and periodic rows are depicted in figures b and c, respectively. The tan-colored cells indicate cyclic boundary condition in effect.

Figure b. periods(0)=.true.;periods(1)=.false.
-1,0 (4) -1,1 (5)
0,-1(-1) 0,0 (0) 0,1 (1) 0,2(-1)
1,-1(-1) 1,0
(2)		 1,1
(3)

1,2
(-1)
2,-1(-1) 2,0 (4) 2,1 (5) 2,2 (-1)
3,0 (0) 3,1 (1)
Figure c. periods(0)=.false.;periods(1)=.true.
  -1,0 (-1) -1,1 (-1)  
0,-1 (1) 0, 0 (0) 0, 1 (1) 0, 2 (0)
1,-1 (3) 1, 0 (2) 1, 1 (3) 1, 2 (2)
2,-1 (5) 2, 0 (4) 2, 1 (5) 2, 2 (4)
  3,0 (-1) 3,1 (-1)  

While having the processes laid out in the cartesian topology help the programmer to write code that’s conceivably more readable, many MPI routines only recognize rank number and hence knowing the relationship between ranks and cartesian coordinates (as shown in the tables above) is key to the opportunity to exploit the topology for computational expediency. In the following sections, we will discuss two subroutines that will provide us with this information. They are: MPI_Cart_coords and MPI_Cart_rank.

Here is the fortran code used to generate
the above tables.

Example in C


 #include "mpi.h";
 MPI_Comm  old_comm, new_comm;
 int ndims, reorder, ierr;
 int dim_size[2], periods[2];

 old_comm = MPI_COMM_WORLD;
 ndims = 2;
 dim_size[0] = 3;
 dim_size[1] = 2;
 periods[0] = 1;
 periods[1] = 0;
 reorder = 1;

 MPI_Cart_create(old_comm, ndims, dim_size,
             periods, reorder, &new_comm);