MATLAB DCS: Difference between revisions

From Cheaha
Jump to navigation Jump to search
(Migrated SSH configuration to MatLab SSH Config page)
No edit summary
 
(48 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{Matlab_deprecated}}
The [http://www.mathworks.com/products/distriben MATLAB Distributed Computing Server (MATLAB DCS)] is a parallel computing extension to MATLAB that enables processing to be spread across a large number of worker nodes, accelerating the speed at which compute intensive operations can complete.
The [http://www.mathworks.com/products/distriben MATLAB Distributed Computing Server (MATLAB DCS)] is a parallel computing extension to MATLAB that enables processing to be spread across a large number of worker nodes, accelerating the speed at which compute intensive operations can complete.


UAB IT Research Computing maintains a 128 worker node license for the the [[Cheaha]] computing platform. In order to use DCS on Cheaha, you will need to [[MATLAB#Using_MATLAB|use a MATLAB instance] with the Parallel Computing Toolbox installed.   
UAB IT Research Computing maintains a 128 worker node license for the the [[Cheaha]] computing platform. In order to use DCS on Cheaha, you will need to [[MATLAB#Using_MATLAB|use a MATLAB instance]] with the Parallel Computing Toolbox installed.   


In order to leverage the MATLAB worker nodes on [[Cheaha]] the Parallel Computing Toolbox will need to be configured to communicate with [[Cheaha]] by following the steps in this document.
In order to leverage the MATLAB worker nodes on [[Cheaha]] the Parallel Computing Toolbox will need to be configured to submit compute tasks to [[Cheaha]] by following the steps in this document.


{{MatlabAppPage}}
{{MatlabAppPage}}


== References ==
== Overview ==
[http://www.mathworks.com/access/helpdesk/help/pdf_doc/distcomp/distcomp.pdf Parallel Computing Toolbox User's Guide] - This is the official 655 page MATLAB user guide for the Parallel Computing Toolbox and is recommended reading!
 
The following outline highlights the steps involved to configure your MatLab install and write programs that submit tasks to the worker nodes of the Distributed Computing Server on Cheaha:
 
* Configure the Task Submit Environment (One-time Setup)
** [[MatLab|Install MatLab]] with the Parallel Computing Toolbox on your Windows / Linux / Mac workstation
** Download and extract the MatLab task submission functions to your workstation MatLab environment
** Define the "cheaha" parallel configuration in your workstation MatLab environment to submit tasks to Cheaha
** Run the validation tests to ensure your "cheaha" parallel configuration works
* Develop and Run Parallel Computing Applications
** Write, test and debug your parallel code on your local workstation using the default "local" parallel configuration
** Once your code works, select the "cheaha" parallel configuration to submit tasks to the Cheaha cluster. Note: your workstation MatLab application does not need to keep running after the tasks are submitted.
** You will receive an email when the tasks you submitted are complete
** Use your workstation MatLab application to retrieve the results
** When you are finished with your job contexts, clean up the job related content to free disk space
 
== Using MATLAB DCS ==
 
The MATLAB Distributed Computing Services (DCS) are accessed via the Parallel Computing Toolbox (PCT) which is installed as part of your desktop [[MATLAB|MATLAB installation]]. The PCT allows MATLAB running on your workstation to send MATLAB code and data (tasks) to the cluster directly from the comfort of your familiar MATLAB environment on your desktop. This makes the expanded compute power of [[Cheaha]] available to processes work loads that exceed the capabilities of your desktop computer.  Once your tasks are submitted to [[Cheaha]], your desktop MATLAB is also free to move on to other tasks or be closed completely, freeing your desktop or laptop for your other activities.


== Overview ==
Configuring the Parallel Computing Toolbox involves three steps documented below:
The following is an overview of the process to run MATLAB jobs on cheaha using the Distributed Computing Server:
# install MATLAB submit functions on your workstation
* Initial Setup
# configure the "cheaha" parallel computing target to which PCT tasks can be submitted
** Install and license both MATLAB client and Parallel Computing Toolbox on your Windows / Linux / Mac workstation
# run the validation tests to confirm a working installation.
** Configure SSH on your workstation (see the section on 'SSH Keys' below)
** Download and extract the MATLAB submission functions to your MATLAB PATH
** Add Cheaha to the Parallel Computing Toolbox configuration
* Running Jobs
** Write, test and debug your parallel code on your local workstation
** Once the code is ready for production, run it on Cheaha using the Parallel Computing Toolbox
** After the job completes (you will receive an email), retrieve the results
** Destroy the job related content on cheaha to clean up


== Client Firewalls ==
This page documents the DCS configuration for MATLAB 2010b and later.  For DCS configuration instructions on previous versions of MATLAB, please see the page [[MatLab DCS R2010a and Earlier]]
The information below was obtained from [http://www.mathworks.com/support/solutions/en/data/1-39LCCA/index.html?product=DW this MATLAB FAQ page].
=== Linux ===
You are running a firewall on your Linux workstation, aren't you? Good. In order for MATLAB communicate properly with the Distributed Computing Server on cheaha, you'll need to open a range of ports, specifically TCP ports 27370 thru 27470. A typical entry in an iptables firewall chain would look like the following (where 138.26.84.72 is the IP address for cheaha.uabgrid.uab.edu)
<pre>
# 20100915 - Mike Hanby - MATLAB Parallel computing toolbox port range
-A INPUT -s 138.26.84.72 -m state --state NEW -m tcp -p tcp --dport 27370:27470 -j ACCEPT
# end MATLAB
</pre>


Once the firewall script has been updated, the firewall must be reloaded / restarted.
Using MATLAB DCS requires you have a cluster account on [[Cheaha]].  Please request an account by sending an email to [[mailto:support@listserv.uab.edu]] and include your campus affiliation and a brief statement of your research interests for using the cluster.


== MATLAB from Your Desktop ==
=== MATLAB DCS from Your Desktop ===
Once SSH has been successfully configured, you are ready to continue setting up the MATLAB client.


MATLAB Parallel Computing Toolbox enables you to submit your MATLAB code to the cluster without leaving the graphical interface.
==== MATLAB Submit Functions ====


This section discusses the following steps:
The MATLAB submit functions create a cluster job context for your code and are responsible for transferring your code and the data it analyzes to the cluster for processing.
# Download and extract the MATLAB submission functions
# Start the MATLAB client
# Add the cheaha Parallel Computing Toolbox configuration
# Perform the validation tests


=== MATLAB Submit Functions ===
These submit functions must be installed on your computer and must be accessible to MATLAB via the MATLAB PATH environment.  The easiest way to accomplish this is to copy the submit functions to the default directory created for by MATLAB.  These directories on the respective operating systems are listed below.
In order for Parallel Computing Toolbox to work with our cluster, you will have to copy the special submit functions to your MATLAB PATH.


Please note that starting with MATLAB 2010b, the submit functions have been changed. If you are running a version prior to 2010b, please download the older functions:
All operating systems (Windows, Linux and Mac) are supported by the same set of submit functions. The functions are written in MATLAB making them  cross-platform and only dependent on the version of MATLAB in use.


==== MATLAB Versions R2010b and later ====
# Download the MATLAB submit functions
All operating systems (Windows, Linux and Mac) are supported by a single set of submit functions:
#* [http://projects.uabgrid.uab.edu/matlab/browser/trunk/distributables/matlab-R2013a-nonshared.zip?format=raw Submit Functions for MATLAB R2013a] -(updated 09/04/2013)
# Download the MATLAB submit functions for R2010b and later
#* [http://projects.uabgrid.uab.edu/matlab/browser/trunk/distributables/matlab-R2012a-nonshared.zip?format=raw Submit Functions for MATLAB R2012a] -(updated 03/07/2012)
#* [http://projects.uabgrid.uab.edu/matlab/browser/trunk/distributables/matlab-nonshared.zip?format=raw MATLAB R2010b and later Submit Functions] - '''Updated 02/21/2011'''
#* [http://projects.uabgrid.uab.edu/matlab/browser/trunk/distributables/matlab-R2011b-nonshared.zip?format=raw Submit Functions for MATLAB R2010b, R2011a, R2011b] -(updated 02/21/2011)
# Unzip the files to a directory on your MATLAB PATH. Typical MATLAB PATH locations are:
# Unzip the files to a directory included in your MATLAB PATH setting. Recommended locations are:
#* Windows:  My Documents\MATLAB
#* Windows:  <pre>My Documents\MATLAB</pre>
#* Linux:    $HOME/Documents/MATLAB
#* Linux:    <pre>$HOME/Documents/MATLAB</pre>
#* Mac:      $HOME/Documents/MATLAB
#* Mac:      <pre>$HOME/Documents/MATLAB</pre>


Once the submit function files have been downloaded and unzipped to your MATLAB PATH, start/restart the MATLAB client.
Once the submit function files have been downloaded and unzipped in the above paths, restart MATLAB to ensure they are properly loaded in your environment.


==== MATLAB Versions R2010a and prior ====
NOTE: If you choose not to use the above path recommendations, your MATLAB PATH may be viewed/altered by starting the MATLAB client on your workstation and clicking File -> Set Path and adding the path in which you unpacked the submit functions.
# Download the appropriate zipped file for your operating system (note that both Linux and Mac use the 'unix' submit functions)
#* [http://projects.uabgrid.uab.edu/matlab/browser/trunk/distributables/matlab-R2010a-windows-nonshared.zip?format=raw Windows MATLAB R2010a and prior Submit Functions] - '''Updated 05/13/2010'''
#* [http://projects.uabgrid.uab.edu/matlab/browser/trunk/distributables/matlab-R2010a-unix-nonshared.zip?format=raw Linux MATLAB R2010a and prior Submit Functions] - '''Updated 05/13/2010'''
#* [http://projects.uabgrid.uab.edu/matlab/browser/trunk/distributables/matlab-R2010a-unix-nonshared.zip?format=raw Mac MATLAB R2010a and prior Submit Functions] - '''Updated 05/13/2010'''
# Unzip the files to a directory on your MATLAB PATH. Typical MATLAB PATH locations are:
#* All Systems: $MATLABROOT/toolbox/local
#* Windows:    My Documents\MATLAB
#* Linux:  $HOME/Documents/MATLAB
#* Mac: ???


Once the submit function files have been downloaded and unzipped to your MATLAB PATH, start/restart the MATLAB client.
==== Parallel Computing Toolbox Configuration ====


NOTE: Your MATLAB PATH may be viewed/altered by starting the MATLAB client on your workstation and clicking File -> Set Path
The Parallel Computing Toolbox (PCT) enables language extensions in MATLAB that support dividing your application into tasks that can be executed in parallel.  By default, all of these tasks will run on your local workstation using the pre-defined "local" PCT configuration.


=== Parallel Computing Toolbox Configuration ===
To run these tasks on the Cheaha compute cluster, a new configuration for the PCT must be defined.  In this section we will create the "cheaha" configuration and run a quick validation test to confirm its operation.
In this section we will add the cheaha configuration to the Parallel Computing Toolbox followed by a quick validation test.


Please follow the instructions for the appropriate version of MATLAB
'''Prior to continuing''', make sure you:
==== MATLAB R2010b and Later ====
* can establish an SSH connection to Cheaha
'''Prior to continuing''', make sure that you've completed the following:
* have followed the steps in the previous section
# Configured SSH and successfully tested passwordless authentication to Cheaha
# Loaded your SSH key
#* Windows users - means starting PuTTY Pagent and loading your SSH key
#* Linux users - means running ssh-agent to load the SSH key
#* Mac users - same as for Linux, run ssh-agent
# Downloaded and extracted the latest MATLAB submission scripts to your MATLAB PATH


Now that the prerequisites are complete:
===== Create the "cheaha" PCT Configuration =====
# Download and save the [http://projects.uabgrid.uab.edu/matlab/browser/trunk/parallel-configs/cheaha.mat?format=raw Cheaha cluster configuration file], this will be imported into MATLAB further down
Download and save the Cheaha cluster configuration file for your MATLAB version
# Start the MATLAB client on your workstation (Windows users should have a MATLAB icon on the desktop), Linux may have to use the command line. On Mac, from '''Finder''' click on the MATLAB icon located at '''/Applications/'''
# [http://projects.uabgrid.uab.edu/matlab/browser/trunk/parallel-configs/cheaha-R2011b.mat?format=raw R2010b, R2011a, R2011b] cluster configuration file
# Click the Parallel menu
#* Start MATLAB on your workstation
# Click Manage Configurations
#* Click the "Parallel" menu
# Click File -> Import
#* Click "Manage Configurations"
# Browse to the location where you saved the '''cheaha.mat''' file select it and click Open
#* In the "Configurations Manager" window, click "File -> Import"
#* Browse to the location where you saved the '''cheaha-R2011b.mat''' file, select it, and click "Open"
# [http://projects.uabgrid.uab.edu/matlab/browser/trunk/parallel-configs/cheaha-R2012a.settings?format=raw R2012a] cluster configuration file
#* Start MATLAB R2012a on your workstation
#* Click the "Parallel" menu
#* Click "Manage Cluster Profiles"
#* In the "Cluster Profile Manager" window, click the "Import" button on the toolbar
#* Browse to the location where you saved the '''cheaha-R2012a.settings''' file, select it, and click "Open"


The Configuration Manager should now list a new entry for cheaha.
The Configuration Manager for R2011b and prior should now list a new entry named "cheaha" as shown in the following image, R2012a and later will also have a new entry in the Cluster Profile list:
[[Image:Matlab-configuration-manager.jpg]]
[[Image:2011_config_mngr.png|none|x400px]]


* Double click on cheaha in the Configuration Manager window to open the configuration window
===== Personalize the "cheaha" PCT Configuration -2011b and earlier =====
** Stretch the window to the right so that you can view all of the text in the SubmitFcn field (makes it easier to modify)
** '''ClusterMatlabRoot''': If necessary, change the Root directory of MATLAB installation for workers to the correct version, R2010b in this example: '''/share/apps/mathworks/R2010b'''
** '''DataLocation'''    : Change the local directory where job data is stored to an existing directory on your workstation: '''C:\jobs\matlab'''
** '''ParallelSubmitFcn''': Change "YOURUSERID" to your Cheaha user id
** '''SubmitFcn'''        : Change "YOURUSERID" to your Cheaha user id


The configuration will look similar to this screen shot.
#Double click on cheaha in the Configuration Manager window to open the configuration editor. (Note: stretch the "Generic Scheduler Configuration Properties" window to the right so that you can view all of the text in the fields making it easier to read and edit correctly.)
# Edit the following fields to use your personal data directories
#* '''ClusterMatlabRoot''': Make sure that the Root directory of MATLAB installation for workers matches the exact version of MATLAB you are using on your workstation. In this example '''/share/apps/mathworks/R2011a''' matches a MATLAB R2011a workstation install.  Change the "R2011a" to match your workstation MATLAB version.
#* '''DataLocation'''    : Change the directory path where job data is stored to an existing directory on your workstation where MATLAB can stage job files.
#* '''ParallelSubmitFcn''': Change the text "YOURUSERID" to your login id on Cheaha
#* '''SubmitFcn'''        : Change the text "YOURUSERID" to your login id on Cheaha
# Click 'OK'to save the configuration
# SSH to cheaha and make sure to create the $USER_SCRATCH/matlab directory. If this directory does not exist, the parallel computing toolbox jobs will fail.


[[Image:cheaha-parallel-config-R2010b.jpg]]
The initial configuration will look similar to this screen shot.  You will need to edit the fields as describe in the preceding steps before you can use the configuration.  '''NOTE: be sure to replace the template user name settings "YOURUSERNAME" with the appropriate settings for your desktop and cluster account.'''


==== MATLAB R2010a and Earlier ====
[[Image:Cheaha_parallel_config.png|none|x650px]]
'''Prior to continuing''', make sure that you've completed the following:
# Configured SSH and successfully tested passwordless authentication to Cheaha
# Loaded your SSH key
#* Windows users - means starting PuTTY Pagent and loading your SSH key
#* Linux users - means running ssh-agent to load the SSH key
#* Mac users - same as for Linux, run ssh-agent
# Downloaded and extracted the latest MATLAB submission scripts to your MATLAB PATH


Now that the prerequisites are complete:
===== Personalize the "cheaha" PCT Configuration -2012a  =====
# Download and save the [http://projects.uabgrid.uab.edu/matlab/browser/trunk/parallel-configs/cheaha-R2010a.mat?format=raw Cheaha cluster configuration file for MATLAB R2010a and prior], this will be imported into MATLAB further down
# Start the MATLAB client on your workstation (Windows users should have a MATLAB icon on the desktop), Linux may have to use the command line. On Mac, from '''Finder''' click on the MATLAB icon located at '''/Applications/'''
# Click the Parallel menu
# Click Manage Configurations
# Click File -> Import
# Browse to the location where you saved the cheaha-R2010a.mat file select it and click Open


The Configuration Manager should now list a new entry for cheaha.
#Double click on cheaha in the Configuration Manager window to open the configuration editor. (Note: stretch the "Generic Scheduler Configuration Properties" window to the right so that you can view all of the text in the fields making it easier to read and edit correctly.)
[[Image:Matlab-configuration-manager.jpg]]
# Edit the following fields to use your personal data directories
#* '''ClusterMatlabRoot''': Make sure that the Root directory of MATLAB installation for workers matches the exact version of MATLAB you are using on your workstation. In this example '''/share/apps/mathworks/R2012a''' matches a MATLAB R2012a workstation install.  Change the "R2012a" to match your workstation MATLAB version.
#* '''DataLocation'''    : Change the directory path where job data is stored to an existing directory on your workstation where MATLAB can stage job files.
#* '''independentSubmitFcn''': Change the text "YOURUSERID" to your login id on Cheaha
#* '''communicatingSubmitFcn'''        : Change the text "YOURUSERID" to your login id on Cheaha
# Click 'OK'to save the configuration
# SSH to cheaha and make sure to create the $USER_SCRATCH/matlab directory. If this directory does not exist, the parallel computing toolbox jobs will fail.


You may have to change the following settings (double click on cheaha in the Configuration Manager window):
The initial configuration will look similar to this screen shot.  You will need to edit the fields as describe in the preceding steps before you can use the configuration.  '''NOTE: be sure to replace the template user name settings "YOURUSERNAME" and "YOURUSERID" with the appropriate settings for your desktop and cluster account.'''  
* Root directory of MATLAB installation for workers (ClusterMatlabRoot): '''/share/apps/mathworks/R2010a'''
* Directory where job data is stored (DataLocation): '''C:\jobs\matlab'''


The configuration will look similar to this screen shot
[[Image:MATLAB_R2012a_configuration.png|none|x650px]]
[[Image:cheaha-parallel-config.jpg]]


=== Examples ===
===== Validate the "cheaha" PCT Configuration =====
==== Parfor ====
This example uses two files:
* myWave.m - a script with a parfor loop generates a wave form
* rParforWave.m - the submission script


The job will use 4 total slots/ MATLAB workers on the cluster (3 workers plus the master worker process).
# Before starting validation please make sure the directory 'lustre/scratch/YOURUSERID/matlab' (please convert all settings to point to the new preferred location) or ''''/scratch/user/YOURUSERID/matlab'''' (preferred)  exists on the scratch space on the Cheaha. If it does not please SSH into Cheaha and create the directory before proceeding.
# Select Cheaha on the configuration manager page and click 'Start Validation'
# Wait for the validation to complete. This might take a few minutes and you ask for User credentials on Cheaha. All tests other than 'Matlabpool' validate on the Cheaha and the output is as shown.


<ol><li>Create the myWave.m script containing this code
[[Image:Validation.png|none|x400px]]
<pre>
parfor i=1:1024


  A(i) = sin(i*2*pi/1024);
===== Begin Using MATLAB DCS from your Desktop =====


end
The MATLAB DCS is now configured for Desktop usage. A simple parallel wave job "rParforWave" to test the configuration is described in [[MatLab_DCS_Examples]].
</pre>
<li>Next create the rParforWave.m script making sure to change the '''YOUREMAIL''' string to a working email address</li>
<pre>
% Always set these variables
email          = 'YOUREMAIL';
email_opt      = 'eas';      % qsub email options
s_rt            = '00:05:00'; % soft wall time
h_rt            = '00:07:00'; % hard wall time
vf              = '1G';      % Amount of memory need per task
min_cpu_slots  = 2;          % Min number of cpu slots needed for the job
max_cpu_slots  = 3;          % Max number of cpu slots needed for the job


% Configure the scheduler - Do NOT modify these
A summary of the above steps is available at [[MATLAB_workshop_2011]] with additional examples and submit scripts available in the [[MATLAB_workshop_2011#Workshop_Demo.27s | workshop demo ]] section.
sge_options = ['-l vf=', vf, ',h_rt=', h_rt, ',s_rt=', s_rt, ' -m ', email_opt, ' -M ', email];
SGEClusterInfo.setExtraParameter(sge_options);
sched = findResource();
% End of scheduler configuration


% start of user specific commands
=== MATLAB DCS from Cheaha ===
job = batch('myWave', 'matlabpool', max_cpu_slots, 'FileDependencies', {'myWave.m'});


% The following commands can be run once the job is submitted to view the results
MATLAB can be started interactively from [[Cheaha]] via an SSH session using the [[MatLab CLI|command line]], X windows forwarding, or VNC.  This is very similar to using MATLAB from your desktop: in order to leverage the compute power of the cluster the Parallel Computing Toolbox must be configured to send tasks to the cluster scheduler.
% >> waitForState(job)
% >> load(job, 'A')
% >> plot(A)


% Once the job is complete, permanently remove its data
If you do not follow these configuration steps, parallel tasks will be executed locally on the cluster log in node (head node). This will negatively impact your own and others interactive use of this log in node and may lead to your computations being stopped administratively.
% >> destroy(job)
</pre>
<li>Select cheaha as the parallel configuration by clicking '''Parallel -> Select Configuration -> cheaha''' in the main MATLAB window
<li>Run the rParforWave.m code by opening the script in the MATLAB editor and clicking the green run arrow
<li>After several seconds you should see output similar to the following in the MATLAB Command Window
<ul><li>The '''job 294773''' is the job number assigned by the Cheaha scheduler
<li>The '''Job1''' is the job name and number used by MATLAB to reference the job. In most cases, this is the number that you'll use to interact with MATLAB to load the results, clean up the job, etc...
<pre>
Your job 294773 ("Job1") has been submitted
</pre></ul>
<li>Now that the job has been submitted, instruct MATLAB to wait for the job to complete using [http://www.mathworks.com/access/helpdesk/help/toolbox/distcomp/waitforstate.html waitForState]. MATLAB will show as 'Busy' until the job completes, at which time the >> prompt will appear. You can verify that the job is complete by running the job.State function call.
<pre>
>> waitForState(job)
>> job.State


ans =
==== MATLAB Submit Functions ====


finished
The MATLAB submit functions create a cluster job context for your code. When running MATLAB from Cheaha, the submit functions are already installed and no additional actions are required by the user for this step.
</pre>
<li>Now that the job has completed, to view the results first use the [http://www.mathworks.com/access/helpdesk/help/toolbox/distcomp/load.html load] function to load the workspace variable '''A''' from our batch job:
<pre>
>> load(job, 'A')
</pre>
<li>Next, display the [http://www.mathworks.com/access/helpdesk/help/techdoc/ref/plot.html plot]
<pre>
>> plot(A)
</pre>
[[Image:RParforWave-plot.png]]
<li>Once you are done with the job, make sure to run the [http://www.mathworks.com/access/helpdesk/help/toolbox/distcomp/destroy.html destroy] the job to clean up the space used on the cluster
<pre>
>> job.destroy
</pre>
</ol>


In the case of longer running jobs, you probably don't want to tie up your MATLAB client by using [http://www.mathworks.com/access/helpdesk/help/toolbox/distcomp/waitforstate.html waitForState].
==== Parallel Computing Toolbox Configuration ====


This will allow you to perform other tasks in MATLAB or exit entirely without having to wait for your job to complete.
The Parallel Computing Toolbox (PCT) for your copy of MATLAB running in your cluster account must be configured to submit tasks to the compute nodes of the cluster.  Keep in mind that running MATLAB interactively on the cluster is the same as running it from your own desktop: MATLAB runs all tasks on the machine on which it is running unless it is told to send the work to another computer.  This condition holds even "inside" the Cheaha cluster: the cluster compute nodes are physically separate computers and MATLAB must request access via the scheduler just like any other job running on the cluster.


MATLAB provides a function to load a previously submitted job back into the workspace, [http://www.mathworks.com/access/helpdesk/help/toolbox/distcomp/findjob.html findJob].
Configuring the PCT when running MATLAB interactively from Cheaha is just like the configuration when running MATLAB from your desktop with two exceptions:
* you must transfer any code or data to your Cheaha account explicitly outside of MATLAB using standard cluster procedures, aka SSH.
* when MATLAB submits your tasks to the compute nodes, it benefits from the shared storage on the cluster and does not need to further copy your code and data to the compute nodes


In the example above, our MATLAB job name was '''Job1''' and from that we can deduce that the MATLAB job number was '''1'''. It can be loaded as follows (make sure the correct Parallel configuration is selected):
To address these differences, follow the [[#Parallel Computing Toolbox Configuration|PCT instructions above]] and when [[#Personalize the "cheaha" PCT Configuration|editing the "cheaha" configuration]] modify the steps for the following fields:
<pre>
# '''Folder where job data is stored (DataLocation)''': specify a directory in your personal Cheaha account
>> sched = findResource();
# '''Function called when submitting parallel jobs (ParallelSubmitFnc)''': change the value to "{@ParallelSubmitFnc}"
>> job = findJob(sched, 'ID', 1);
# '''Function called when submitting distributed jobs (DistributedSubmitFnc)''': change the value to "{@DistributedSubmitFnc}"
>> job.State
# '''Job data location is accessible from both client and cluster nodes''': change this value to "True"


ans =
Now save the "cheaha" configuring by clicking OK and proceeding to the [[#Validate the "cheaha" PCT Configuration|validation tests described above]].  Note: when running MATLAB interactively on Cheaha MatlabPool works and the validation tests are expected to pass (you'll see a green checkmark).


finished
== References ==
</pre>
* [http://www.mathworks.com/help/toolbox/distcomp/ Parallel Computing Toolbox on-line documentation]
* [http://www.mathworks.com/access/helpdesk/help/pdf_doc/distcomp/distcomp.pdf Parallel Computing Toolbox User's Guide (pdf)] - The 655 page MATLAB User Guide for the Parallel Computing Toolbox and is recommended reading!


It is important to clean up after the job using [http://www.mathworks.com/access/helpdesk/help/toolbox/distcomp/destroy.html destroy] to free up hard disk space on both your desktop and the cluster. Any output that is to be saved should be copied to another location on your Desktop prior to running [http://www.mathworks.com/access/helpdesk/help/toolbox/distcomp/destroy.html destroy].
<pre>
>> findJob(sched, 'ID', 1)
>> job.destroy
</pre>
{{MATLAB Support}}
{{MATLAB Support}}


[[Category:MATLAB]]
[[Category:MATLAB]][[Category:MATLAB installation]]

Latest revision as of 19:41, 11 February 2019

Ambox important.png

This page is Deprecated
To Use MATLAB with the SLURM scheduler on Cheaha please click the link below

MATLAB with SLURM


The MATLAB Distributed Computing Server (MATLAB DCS) is a parallel computing extension to MATLAB that enables processing to be spread across a large number of worker nodes, accelerating the speed at which compute intensive operations can complete.

UAB IT Research Computing maintains a 128 worker node license for the the Cheaha computing platform. In order to use DCS on Cheaha, you will need to use a MATLAB instance with the Parallel Computing Toolbox installed.

In order to leverage the MATLAB worker nodes on Cheaha the Parallel Computing Toolbox will need to be configured to submit compute tasks to Cheaha by following the steps in this document.

Please see the MATLAB application page for more information and a general overview of MATLAB and its use at UAB.

Overview

The following outline highlights the steps involved to configure your MatLab install and write programs that submit tasks to the worker nodes of the Distributed Computing Server on Cheaha:

  • Configure the Task Submit Environment (One-time Setup)
    • Install MatLab with the Parallel Computing Toolbox on your Windows / Linux / Mac workstation
    • Download and extract the MatLab task submission functions to your workstation MatLab environment
    • Define the "cheaha" parallel configuration in your workstation MatLab environment to submit tasks to Cheaha
    • Run the validation tests to ensure your "cheaha" parallel configuration works
  • Develop and Run Parallel Computing Applications
    • Write, test and debug your parallel code on your local workstation using the default "local" parallel configuration
    • Once your code works, select the "cheaha" parallel configuration to submit tasks to the Cheaha cluster. Note: your workstation MatLab application does not need to keep running after the tasks are submitted.
    • You will receive an email when the tasks you submitted are complete
    • Use your workstation MatLab application to retrieve the results
    • When you are finished with your job contexts, clean up the job related content to free disk space

Using MATLAB DCS

The MATLAB Distributed Computing Services (DCS) are accessed via the Parallel Computing Toolbox (PCT) which is installed as part of your desktop MATLAB installation. The PCT allows MATLAB running on your workstation to send MATLAB code and data (tasks) to the cluster directly from the comfort of your familiar MATLAB environment on your desktop. This makes the expanded compute power of Cheaha available to processes work loads that exceed the capabilities of your desktop computer. Once your tasks are submitted to Cheaha, your desktop MATLAB is also free to move on to other tasks or be closed completely, freeing your desktop or laptop for your other activities.

Configuring the Parallel Computing Toolbox involves three steps documented below:

  1. install MATLAB submit functions on your workstation
  2. configure the "cheaha" parallel computing target to which PCT tasks can be submitted
  3. run the validation tests to confirm a working installation.

This page documents the DCS configuration for MATLAB 2010b and later. For DCS configuration instructions on previous versions of MATLAB, please see the page MatLab DCS R2010a and Earlier

Using MATLAB DCS requires you have a cluster account on Cheaha. Please request an account by sending an email to [[1]] and include your campus affiliation and a brief statement of your research interests for using the cluster.

MATLAB DCS from Your Desktop

MATLAB Submit Functions

The MATLAB submit functions create a cluster job context for your code and are responsible for transferring your code and the data it analyzes to the cluster for processing.

These submit functions must be installed on your computer and must be accessible to MATLAB via the MATLAB PATH environment. The easiest way to accomplish this is to copy the submit functions to the default directory created for by MATLAB. These directories on the respective operating systems are listed below.

All operating systems (Windows, Linux and Mac) are supported by the same set of submit functions. The functions are written in MATLAB making them cross-platform and only dependent on the version of MATLAB in use.

  1. Download the MATLAB submit functions
  2. Unzip the files to a directory included in your MATLAB PATH setting. Recommended locations are:
    • Windows:
      My Documents\MATLAB
    • Linux:
      $HOME/Documents/MATLAB
    • Mac:
      $HOME/Documents/MATLAB

Once the submit function files have been downloaded and unzipped in the above paths, restart MATLAB to ensure they are properly loaded in your environment.

NOTE: If you choose not to use the above path recommendations, your MATLAB PATH may be viewed/altered by starting the MATLAB client on your workstation and clicking File -> Set Path and adding the path in which you unpacked the submit functions.

Parallel Computing Toolbox Configuration

The Parallel Computing Toolbox (PCT) enables language extensions in MATLAB that support dividing your application into tasks that can be executed in parallel. By default, all of these tasks will run on your local workstation using the pre-defined "local" PCT configuration.

To run these tasks on the Cheaha compute cluster, a new configuration for the PCT must be defined. In this section we will create the "cheaha" configuration and run a quick validation test to confirm its operation.

Prior to continuing, make sure you:

  • can establish an SSH connection to Cheaha
  • have followed the steps in the previous section
Create the "cheaha" PCT Configuration

Download and save the Cheaha cluster configuration file for your MATLAB version

  1. R2010b, R2011a, R2011b cluster configuration file
    • Start MATLAB on your workstation
    • Click the "Parallel" menu
    • Click "Manage Configurations"
    • In the "Configurations Manager" window, click "File -> Import"
    • Browse to the location where you saved the cheaha-R2011b.mat file, select it, and click "Open"
  2. R2012a cluster configuration file
    • Start MATLAB R2012a on your workstation
    • Click the "Parallel" menu
    • Click "Manage Cluster Profiles"
    • In the "Cluster Profile Manager" window, click the "Import" button on the toolbar
    • Browse to the location where you saved the cheaha-R2012a.settings file, select it, and click "Open"

The Configuration Manager for R2011b and prior should now list a new entry named "cheaha" as shown in the following image, R2012a and later will also have a new entry in the Cluster Profile list:

2011 config mngr.png
Personalize the "cheaha" PCT Configuration -2011b and earlier
  1. Double click on cheaha in the Configuration Manager window to open the configuration editor. (Note: stretch the "Generic Scheduler Configuration Properties" window to the right so that you can view all of the text in the fields making it easier to read and edit correctly.)
  2. Edit the following fields to use your personal data directories
    • ClusterMatlabRoot: Make sure that the Root directory of MATLAB installation for workers matches the exact version of MATLAB you are using on your workstation. In this example /share/apps/mathworks/R2011a matches a MATLAB R2011a workstation install. Change the "R2011a" to match your workstation MATLAB version.
    • DataLocation  : Change the directory path where job data is stored to an existing directory on your workstation where MATLAB can stage job files.
    • ParallelSubmitFcn: Change the text "YOURUSERID" to your login id on Cheaha
    • SubmitFcn  : Change the text "YOURUSERID" to your login id on Cheaha
  3. Click 'OK'to save the configuration
  4. SSH to cheaha and make sure to create the $USER_SCRATCH/matlab directory. If this directory does not exist, the parallel computing toolbox jobs will fail.

The initial configuration will look similar to this screen shot. You will need to edit the fields as describe in the preceding steps before you can use the configuration. NOTE: be sure to replace the template user name settings "YOURUSERNAME" with the appropriate settings for your desktop and cluster account.

Cheaha parallel config.png
Personalize the "cheaha" PCT Configuration -2012a
  1. Double click on cheaha in the Configuration Manager window to open the configuration editor. (Note: stretch the "Generic Scheduler Configuration Properties" window to the right so that you can view all of the text in the fields making it easier to read and edit correctly.)
  2. Edit the following fields to use your personal data directories
    • ClusterMatlabRoot: Make sure that the Root directory of MATLAB installation for workers matches the exact version of MATLAB you are using on your workstation. In this example /share/apps/mathworks/R2012a matches a MATLAB R2012a workstation install. Change the "R2012a" to match your workstation MATLAB version.
    • DataLocation  : Change the directory path where job data is stored to an existing directory on your workstation where MATLAB can stage job files.
    • independentSubmitFcn: Change the text "YOURUSERID" to your login id on Cheaha
    • communicatingSubmitFcn  : Change the text "YOURUSERID" to your login id on Cheaha
  3. Click 'OK'to save the configuration
  4. SSH to cheaha and make sure to create the $USER_SCRATCH/matlab directory. If this directory does not exist, the parallel computing toolbox jobs will fail.

The initial configuration will look similar to this screen shot. You will need to edit the fields as describe in the preceding steps before you can use the configuration. NOTE: be sure to replace the template user name settings "YOURUSERNAME" and "YOURUSERID" with the appropriate settings for your desktop and cluster account.

MATLAB R2012a configuration.png
Validate the "cheaha" PCT Configuration
  1. Before starting validation please make sure the directory 'lustre/scratch/YOURUSERID/matlab' (please convert all settings to point to the new preferred location) or '/scratch/user/YOURUSERID/matlab' (preferred) exists on the scratch space on the Cheaha. If it does not please SSH into Cheaha and create the directory before proceeding.
  2. Select Cheaha on the configuration manager page and click 'Start Validation'
  3. Wait for the validation to complete. This might take a few minutes and you ask for User credentials on Cheaha. All tests other than 'Matlabpool' validate on the Cheaha and the output is as shown.
Validation.png
Begin Using MATLAB DCS from your Desktop

The MATLAB DCS is now configured for Desktop usage. A simple parallel wave job "rParforWave" to test the configuration is described in MatLab_DCS_Examples.

A summary of the above steps is available at MATLAB_workshop_2011 with additional examples and submit scripts available in the workshop demo section.

MATLAB DCS from Cheaha

MATLAB can be started interactively from Cheaha via an SSH session using the command line, X windows forwarding, or VNC. This is very similar to using MATLAB from your desktop: in order to leverage the compute power of the cluster the Parallel Computing Toolbox must be configured to send tasks to the cluster scheduler.

If you do not follow these configuration steps, parallel tasks will be executed locally on the cluster log in node (head node). This will negatively impact your own and others interactive use of this log in node and may lead to your computations being stopped administratively.

MATLAB Submit Functions

The MATLAB submit functions create a cluster job context for your code. When running MATLAB from Cheaha, the submit functions are already installed and no additional actions are required by the user for this step.

Parallel Computing Toolbox Configuration

The Parallel Computing Toolbox (PCT) for your copy of MATLAB running in your cluster account must be configured to submit tasks to the compute nodes of the cluster. Keep in mind that running MATLAB interactively on the cluster is the same as running it from your own desktop: MATLAB runs all tasks on the machine on which it is running unless it is told to send the work to another computer. This condition holds even "inside" the Cheaha cluster: the cluster compute nodes are physically separate computers and MATLAB must request access via the scheduler just like any other job running on the cluster.

Configuring the PCT when running MATLAB interactively from Cheaha is just like the configuration when running MATLAB from your desktop with two exceptions:

  • you must transfer any code or data to your Cheaha account explicitly outside of MATLAB using standard cluster procedures, aka SSH.
  • when MATLAB submits your tasks to the compute nodes, it benefits from the shared storage on the cluster and does not need to further copy your code and data to the compute nodes

To address these differences, follow the PCT instructions above and when editing the "cheaha" configuration modify the steps for the following fields:

  1. Folder where job data is stored (DataLocation): specify a directory in your personal Cheaha account
  2. Function called when submitting parallel jobs (ParallelSubmitFnc): change the value to "{@ParallelSubmitFnc}"
  3. Function called when submitting distributed jobs (DistributedSubmitFnc): change the value to "{@DistributedSubmitFnc}"
  4. Job data location is accessible from both client and cluster nodes: change this value to "True"

Now save the "cheaha" configuring by clicking OK and proceeding to the validation tests described above. Note: when running MATLAB interactively on Cheaha MatlabPool works and the validation tests are expected to pass (you'll see a green checkmark).

References

MATLAB Support / Mailing List

As with any application or computer language, learning to use MATLAB to analyze data or to develop or modify MATLAB applications is an individual responsibility. There is ample application documentation available from the Mathworks website, potential outreach to colleagues who also use MATLAB, and options for consultation with Mathworks. Mathworks also host on-campus training seminars several times a year and provides many on-line learning tutorials.

Installation support for MATLAB at UAB is provided by your local IT support organization and the Docs wiki.

Mathworks Website

Your first and best option for application-specific questions on MATLAB is to refer to the on-line MATLAB documentation. The Mathworks site also provides a a support matrix and an on-line knowledge base.

UAB MATLAB Wiki

The MATLAB page on the Docs wiki is the starting point for installing MATLAB at UAB and, optionally, configuring it to use cluster computing. All users are encouraged to contribute to the MATLAB knowledge in this wiki, especially if you see areas where improvements are needed. Remember, this knowledge base is only as good as the people who contribute to it.

Contributing to the wiki is as easy as clicking the login link on the top-right of the page and signing in with your UAB BlazerID. If you are unsure about making an edit, you can make suggestions for improvement on the page's Discussion tab or discuss the proposed improvement in the MATLAB user group.

UAB MATLAB User Group

At UAB, MATLAB installation support is provided by your local IT support group. Support for application specific questions is available from peers in your research group. We realize that some people are not as familiar with MATLAB as others. For this reason, we have established a MATLAB user forum (mailing list) where users of MATLAB at UAB can help answer each others questions.

This is a network of volunteers sharing their knowledge with peers. You are encouraged to reach out to this community for questions on using MATLAB by

Archives of MATLAB user group discussions are available on-line at https://vo.uabgrid.uab.edu/sympa/arc/matlab-user. You may find your question is already answered in these archives.


UAB MATLAB announce mailing list

To receive information about UAB's MATLAB license and announcements please subscribe to the matlab-annc mailing list by