MATLAB DCS

From Cheaha
Revision as of 14:11, 7 September 2011 by Jpr@uab.edu (talk | contribs) (→‎Using MATLAB DCS: Rewrite the section intro to provide an overview and adapt to the streamlined documentation)
Jump to navigation Jump to search


Attention: Research Computing Documentation has Moved
https://docs.rc.uab.edu/


Please use the new documentation url https://docs.rc.uab.edu/ for all Research Computing documentation needs.


As a result of this move, we have deprecated use of this wiki for documentation. We are providing read-only access to the content to facilitate migration of bookmarks and to serve as an historical record. All content updates should be made at the new documentation site. The original wiki will not receive further updates.

Thank you,

The Research Computing Team

The MATLAB Distributed Computing Server (MATLAB DCS) is a parallel computing extension to MATLAB that enables processing to be spread across a large number of worker nodes, accelerating the speed at which compute intensive operations can complete.

UAB IT Research Computing maintains a 128 worker node license for the the Cheaha computing platform. In order to use DCS on Cheaha, you will need to use a MATLAB instance with the Parallel Computing Toolbox installed.

In order to leverage the MATLAB worker nodes on Cheaha the Parallel Computing Toolbox will need to be configured to submit compute tasks to Cheaha by following the steps in this document.

Please see the MATLAB application page for more information and a general overview of MATLAB and its use at UAB.

Overview

The following outline highlights the steps involved to configure your MatLab install and write programs that submit tasks to the worker nodes of the Distributed Computing Server on Cheaha:

  • Configure the Task Submit Environment (One-time Setup)
    • Install MatLab with the Parallel Computing Toolbox on your Windows / Linux / Mac workstation
    • Download and extract the MatLab task submission functions to your workstation MatLab environment
    • Define the "cheaha" parallel configuration in your workstation MatLab environment to submit tasks to Cheaha
    • Run the validation tests to ensure your "cheaha" parallel configuration works
  • Develop and Run Parallel Computing Applications
    • Write, test and debug your parallel code on your local workstation using the default "local" parallel configuration
    • Once the code is ready for production, select the "cheaha" parallel configuration to submit tasks to the Cheaha cluster. Note: your workstation MatLab application does not need to keep running after the tasks are submitted.
    • You will receive an email when the tasks you submitted are complete
    • Use your workstation MatLab application to retrieve the results
    • When you are finished with your job contexts, clean up the job related content to free disk space

Using MATLAB DCS

The MATLAB Distributed Computing Services (DCS) are accessed via the Parallel Computing Toolbox (PCT) which is installed as part of your desktop MATLAB installation. The PCT allows MATLAB running on your workstation to send MATLAB code and data (tasks) to the cluster directly from the comfort of your familiar MATLAB environment on your desktop. This makes the expanded compute power of Cheaha available to processes work loads that exceed the capabilities of your desktop computer. Once your tasks are submitted to Cheaha, your desktop MATLAB is also free to move on to other tasks or be closed completely, freeing your desktop or laptop for your other activities.

Configuring the Parallel Computing Toolbox involves three steps documented below: 1) install MATLAB submit functions on your workstation 2) configure the "cheaha" parallel computing target to which PCT tasks can be submitted and 3) run the validation tests to confirm a working installation.

Using the MATLAB DCS requires you have a cluster account on Cheaha. Please request an account by sending an email to [[1]] and include your campus affiliation and a brief statement of your research interests for using the cluster.

MATLAB DCS from Your Desktop

MATLAB Submit Functions

In order for Parallel Computing Toolbox to work with our cluster, you will have to copy the special submit functions to your MATLAB PATH.

These steps document the DCS configuration for MatLab 2010b and later. For DCS configuration instructions on previous versions of MatLab, please visit the page MatLab DCS R2010a and Earlier

All operating systems (Windows, Linux and Mac) are supported by a single set of submit functions:

  1. Download the MATLAB submit functions for R2010b and later
  2. Unzip the files to a directory on your MATLAB PATH. Typical MATLAB PATH locations are:
    • Windows: My Documents\MATLAB
    • Linux: $HOME/Documents/MATLAB
    • Mac: $HOME/Documents/MATLAB

Once the submit function files have been downloaded and unzipped to your MATLAB PATH, start/restart the MATLAB client.

NOTE: Your MATLAB PATH may be viewed/altered by starting the MATLAB client on your workstation and clicking File -> Set Path

Parallel Computing Toolbox Configuration

In this section we will add the cheaha configuration to the Parallel Computing Toolbox followed by a quick validation test.

Please follow the instructions for the appropriate version of MATLAB

Prior to continuing, make sure that you've completed the following:

  1. Configured SSH and successfully tested passwordless authentication to Cheaha
  2. Loaded your SSH key
    • Windows users - means starting PuTTY Pagent and loading your SSH key
    • Linux users - means running ssh-agent to load the SSH key
    • Mac users - same as for Linux, run ssh-agent
  3. Downloaded and extracted the latest MATLAB submission scripts to your MATLAB PATH

Now that the prerequisites are complete:

  1. Download and save the Cheaha cluster configuration file, this will be imported into MATLAB further down
  2. Start the MATLAB client on your workstation (Windows users should have a MATLAB icon on the desktop), Linux may have to use the command line. On Mac, from Finder click on the MATLAB icon located at /Applications/
  3. Click the Parallel menu
  4. Click Manage Configurations
  5. Click File -> Import
  6. Browse to the location where you saved the cheaha.mat file select it and click Open

The Configuration Manager should now list a new entry for cheaha. Matlab-configuration-manager.jpg

  • Double click on cheaha in the Configuration Manager window to open the configuration window
    • Stretch the window to the right so that you can view all of the text in the SubmitFcn field (makes it easier to modify)
    • ClusterMatlabRoot: If necessary, change the Root directory of MATLAB installation for workers to the correct version, R2010b in this example: /share/apps/mathworks/R2010b
    • DataLocation  : Change the local directory where job data is stored to an existing directory on your workstation: C:\jobs\matlab
    • ParallelSubmitFcn: Change "YOURUSERID" to your Cheaha user id
    • SubmitFcn  : Change "YOURUSERID" to your Cheaha user id

The configuration will look similar to this screen shot.

Cheaha-parallel-config-R2010b.jpg


  • Click 'OK'to save the configuration
  • Select Cheaha on the configuration manager page and click 'Start Validation'
  • Allow validation to complete. It might take a few minutes and ask for User credentials on Cheaha. All tests other than 'Matlabpool' validate on the Cheaha and the output is as shown.

Validation.png


MATLAB DCS from Cheaha

References

Parallel Computing Toolbox User's Guide - This is the official 655 page MATLAB user guide for the Parallel Computing Toolbox and is recommended reading!

MATLAB Support / Mailing List

As with any application or computer language, learning to use MATLAB to analyze data or to develop or modify MATLAB applications is an individual responsibility. There is ample application documentation available from the Mathworks website, potential outreach to colleagues who also use MATLAB, and options for consultation with Mathworks. Mathworks also host on-campus training seminars several times a year and provides many on-line learning tutorials.

Installation support for MATLAB at UAB is provided by your local IT support organization and the Docs wiki.

Mathworks Website

Your first and best option for application-specific questions on MATLAB is to refer to the on-line MATLAB documentation. The Mathworks site also provides a a support matrix and an on-line knowledge base.

UAB MATLAB Wiki

The MATLAB page on the Docs wiki is the starting point for installing MATLAB at UAB and, optionally, configuring it to use cluster computing. All users are encouraged to contribute to the MATLAB knowledge in this wiki, especially if you see areas where improvements are needed. Remember, this knowledge base is only as good as the people who contribute to it.

Contributing to the wiki is as easy as clicking the login link on the top-right of the page and signing in with your UAB BlazerID. If you are unsure about making an edit, you can make suggestions for improvement on the page's Discussion tab or discuss the proposed improvement in the MATLAB user group.

UAB MATLAB User Group

At UAB, MATLAB installation support is provided by your local IT support group. Support for application specific questions is available from peers in your research group. We realize that some people are not as familiar with MATLAB as others. For this reason, we have established a MATLAB user forum (mailing list) where users of MATLAB at UAB can help answer each others questions.

This is a network of volunteers sharing their knowledge with peers. You are encouraged to reach out to this community for questions on using MATLAB by

Archives of MATLAB user group discussions are available on-line at https://vo.uabgrid.uab.edu/sympa/arc/matlab-user. You may find your question is already answered in these archives.


UAB MATLAB announce mailing list

To receive information about UAB's MATLAB license and announcements please subscribe to the matlab-annc mailing list by

Retrieved from "https://docs.uabgrid.uab.edu/w/index.php?title=MATLAB_DCS&oldid=3063"