Described in the following is a procedure to run multiple independent MATLAB tasks. Typically, users who want to run MATLAB jobs in this mode do so for parametric studies of the same code under different conditions. These conditions are often controlled by simple parameters or by input file. Since multiple copies of MATLAB are processed concurrently, the computed results from each must be saved in a unique file. While you can submit multiple single-processor MATLAB jobs one at a time, shown below is a batch submission script that submit N instances of single-processor MATLAB jobs. To distinguish one job from another, a loop index, essentially, is passed as an environment variable to your m-file. This index may be used as a parameter for your m-file to perform a specific task or it may be used as part of an input file for the specific task.
- The batch submission script is called
submitmjobs, the purpose is to submit multiple single-processor MATLAB jobs to the batch queue:
#!/bin/csh # # Purpose: Running this script launches multiple serial jobs in batch. # Usage: scc1% submitmjobs ntasks batch_script "SGE options" # Runtime input parameters to submitmjobs: # ntasks -- the number of tasks submitted to batch queue # batch_script -- such as run_matlab_job, for MATLAB apps # The OPTIONAL input must be valid SGE batch parameters. # Example: # scc1% submitmjobs 4 run_matlab_job "-l h_rt=09:00:00" # # In the above, running submitmjobs launches 4 run_matlab_job with a # wallclock limit of 9 hours for each of 4 tasks. The optional third # input specifies runtime limit. Without it, the default wallclock limit # is 2 hours for each task. Each of the 4 tasks is assigned a unique # index, task, ranged between 1 to 4. You can use # task's value to determine what data or action to take. # # The Sun Grid Engine batch preserves the current dir in batch. # A different dir may also be used # cd my_dir if ($# == 0) then echo Usage is : submitmjobs 4 run_matlab_job ... exit endif @ ntasks = $1 set batch_script = $2 set options = "$3" # Name your batch job; otherwise default to script name set myjob = myBatchJobName setenv MATLAB_NTASKS $ntasks foreach task (`/usr/bin/seq $ntasks`) setenv MATLAB_TASK $task qsub -N $myjob$task $options -V $batch_script end <span class="# The next job performs post processing AFTER the above jobs are done. # If you need this operation, remove # to activate #qsub -hold_jid "$myjob*" -V post_process_job # keep this line to ensure newline
Shown below is the batch script for a MATLAB application,
#!/bin/csh # this script is a companion script for submitmjobs for MATLAB apps # it demonstrates how the passed "task" can be used to make each of # the ntasks to perform a different task or use different data. If your # app is a function m-file, "task" needs to be passed as input argument. # If your app is a script m-file, "task" is automatically available # because it shares the same workspace as myscript. # Use one form or the other. unsetenv DISPLAY # IMPORTANT: DONOT indent any of the below statements matlab -nojvm -singleCompThread >! myoutput_$MATLAB_TASK << MATLAB_ENV % The below works like a script m-file between MATLAB_ENVs task = $MATLAB_TASK % define matlab variable "task" from env variable ntasks = $MATLAB_NTASKS % define number of tasks outname = ['output' num2str(task)] % associate output file name with "task" myfunc(task, outname) % example to pass "task" to a function "myfunc" myscript % example to pass "task" to script "myscript" exit % don't forget to exit MATLAB MATLAB_ENV # keep this line to ensure newline
function myfunc(rank, outname) %function myfunc(rank, outname) disp('This is a MATLAB function m-file') if rank == 1 disp(['Perform a task with rank ' num2str(rank)]) else disp(['Perform another task with rank ' num2str(rank)]) end % alternatively, may construct outname here with "rank" save(outname,'rank'); % save data as mat-file end
disp('This is a MATLAB script m-file') % "task" is passed from submitmjob to run_matlab_job to myscript disp(['Do work based on the value of task = ' num2str(task)])
#!/bin/csh # this script is a companion script for submitmjobs for MATLAB apps # a number of individual tasks has been submitted to queue via submitmjobs # With hold_jid, this queued post processing job will start only after all # previous jobs have been completed. For this example, myfunc writes # the rank (task) number to a file for each task. The post processing job # then load these files and write them to a single file, combine_output.mat: unsetenv DISPLAY # IMPORTANT: DONOT indent any of the below statements matlab -nojvm -singleCompThread << MATLAB_ENV ntasks = $MATLAB_NTASKS for task=1:ntasks load(['output' num2str(task)],'rank') a(task) = rank; end save('combine_output','a') exit MATLAB_ENV # keep this line to ensure a newline
In the above example, four instances of MATLAB are launched. The
myoutput_*files trap any output sent to the screen by each MATLAB instance.
scc1% chmod +x submitmjobs scc1% chmod +x run_matlab_job scc1% chmod +x post_process_job
- Submit 4 single-processor jobs to the batch queue with a (default) two hours wallclock time each.
scc1% submitmjobs 4 run_matlab_job
The following demonstrates a process to launch 16 single-processor batch jobs with a wallclock limit of 9 hours each.
scc1% submitmjobs 16 run_matlab_job "-l h_rt=09:00:00"
Please call (617-353-8294) or email (email@example.com) Kadin Tseng if you need help or if you’d like to explore other possibilities of multiprocessing with MATLAB.