Described in the following is a procedure to run multiple independent MATLAB tasks. Typically, users who want to run MATLAB jobs in this mode do so for parametric studies of the same code under different conditions. These conditions are often controlled by simple parameters or by input file. Since multiple copies of MATLAB are processed concurrently, the computed results from each must be saved in a unique file. While you can submit multiple single-processor MATLAB jobs one at a time, shown below is a batch submission script that submit N instances of single-processor MATLAB jobs. To distinguish one job from another, a loop index, essentially, is passed as an environment variable to your m-file. This index may be used as a parameter for your m-file to perform a specific task or it may be used as part of an input file for the specific task.

  1. The batch submission script is called submitmjobs, the purpose is to submit multiple single-processor MATLAB jobs to the batch queue:
    #!/bin/csh
    #
    # Purpose: Running this script launches multiple serial jobs in batch.
    # Usage: scc1% submitmjobs ntasks batch_script "SGE options"
    # Runtime input parameters to submitmjobs:
    #   ntasks -- the number of tasks submitted to batch queue
    #   batch_script -- such as run_matlab_job, for MATLAB apps
    #   The OPTIONAL input must be valid SGE batch parameters.
    # Example:
    # scc1% submitmjobs 4 run_matlab_job "-l h_rt=09:00:00"
    #
    # In the above, running submitmjobs launches 4  run_matlab_job with a
    # wallclock limit of 9 hours for each of 4 tasks. The optional third
    # input specifies runtime limit. Without it, the default wallclock limit
    # is 2 hours for each task. Each of the 4 tasks is assigned a unique
    # index, task, ranged between 1 to 4. You can use
    # task's value to determine what data or action to take.
    #
    # The Sun Grid Engine batch preserves the current dir in batch.
    # A different dir may also be used
    # cd my_dir
    
    if ($# == 0) then
      echo Usage is : submitmjobs 4 run_matlab_job ...
      exit
    endif
    
    @ ntasks = $1
    set batch_script = $2
    set options = "$3"
    # Name your batch job; otherwise default to script name
    set myjob = myBatchJobName
    setenv MATLAB_NTASKS $ntasks
    foreach task (`/usr/bin/seq $ntasks`)
      setenv MATLAB_TASK $task
      qsub -N $myjob$task $options -V $batch_script
    end
    <span class="# The next job performs post processing AFTER the above jobs are done.
    # If you need this operation, remove # to activate
    #qsub -hold_jid "$myjob*" -V post_process_job
    # keep this line to ensure newline

    Shown below is the batch script for a MATLAB application, run_matlab_job:

    #!/bin/csh
    # this script is a companion script for submitmjobs for MATLAB apps
    # it demonstrates how the passed "task" can be used to make each of
    # the ntasks to perform a different task or use different data. If your
    # app is a function m-file, "task" needs to be passed as input argument.
    # If your app is a script m-file, "task" is automatically available
    # because it shares the same workspace as myscript.
    # Use one form or the other.
    
    unsetenv DISPLAY
    # IMPORTANT: DONOT indent any of the below statements
    matlab -nojvm -singleCompThread  >! myoutput_$MATLAB_TASK << MATLAB_ENV
    % The below works like a script m-file between MATLAB_ENVs
    task = $MATLAB_TASK       % define matlab variable "task" from env variable
    ntasks = $MATLAB_NTASKS   % define number of tasks
    outname = ['output' num2str(task)] % associate output file name with "task"
    myfunc(task, outname)     % example to pass "task" to a function "myfunc"
    myscript                  % example to pass "task" to script "myscript"
    exit                      % don't forget to exit MATLAB
    MATLAB_ENV
    # keep this line to ensure newline

    myfunc.m:

    function myfunc(rank, outname)
    %function myfunc(rank, outname)
    disp('This is a MATLAB function m-file')
    if rank == 1
      disp(['Perform a task with rank ' num2str(rank)])
    else
      disp(['Perform another task with rank ' num2str(rank)])
    end
    % alternatively, may construct outname here with "rank"
    save(outname,'rank');    % save data as mat-file
    end
    

    myscript.m:

    disp('This is a MATLAB script m-file')
    % "task" is passed from submitmjob to run_matlab_job to myscript
    disp(['Do work based on the value of task = ' num2str(task)])
    

    post_process_job.m:

    #!/bin/csh
    # this script is a companion script for submitmjobs for MATLAB apps
    # a number of individual tasks has been submitted to queue via submitmjobs
    # With hold_jid, this queued post processing job will start only after all
    # previous jobs have been completed. For this example, myfunc writes 
    # the rank (task)  number to a file for each task. The post processing job
    # then load these files and write them to a single file, combine_output.mat:
    
    unsetenv DISPLAY
    # IMPORTANT: DONOT indent any of the below statements
    matlab -nojvm -singleCompThread  << MATLAB_ENV
    ntasks = $MATLAB_NTASKS
    for task=1:ntasks
    load(['output' num2str(task)],'rank')
    a(task) = rank;
    end
    save('combine_output','a')
    exit
    MATLAB_ENV
    # keep this line to ensure a newline

    In the above example, four instances of MATLAB are launched. The myoutput_* files trap any output sent to the screen by each MATLAB instance.

  2. Change submitmjobs and run_matlab_job execute attribute
    scc1% chmod +x submitmjobs
    scc1% chmod +x run_matlab_job
    scc1% chmod +x post_process_job
  3. Submit 4 single-processor jobs to the batch queue with a (default) two hours wallclock time each.
    scc1% submitmjobs 4 run_matlab_job

    The following demonstrates a process to launch 16 single-processor batch jobs with a wallclock limit of 9 hours each.

    scc1% submitmjobs 16 run_matlab_job "-l h_rt=09:00:00"

Please call (617-353-8294) or email (kadin@bu.edu) Kadin Tseng if you need help or if you’d like to explore other possibilities of multiprocessing with MATLAB.