MatLab CLI: Difference between revisions
(14 intermediate revisions by 3 users not shown) | |||
Line 10: | Line 10: | ||
The following is an example output of the command and doesn't necessarily represent the currently installed versions: | The following is an example output of the command and doesn't necessarily represent the currently installed versions: | ||
<pre> | <pre> | ||
$ module avail | $ module avail rc/matlab | ||
------------------------------------ / | ------------------------------------ /share/apps/modulefiles ------------------------------------- | ||
rc/matlab/R2012a | |||
rc/matlab/R2012aa | |||
rc/matlab/R2013a | |||
rc/matlab/R2015a | |||
rc/matlab/R2015b | |||
</pre> | </pre> | ||
=== Simple MATLAB Test === | === Simple MATLAB Test === | ||
A simple test to verify that the MATLAB client on Cheaha can check out a license from your server. | A simple test to verify that the MATLAB client on Cheaha can check out a license from your server (or from the campus license server 27000@lmgr.uabgrid.uab.edu ). | ||
Set up your environment with the command: | Set up your environment with the command: | ||
<pre> | <pre> | ||
$ module load | $ module load rc/matlab/R2012a | ||
</pre> | </pre> | ||
As a test, you can run MatLab and access your license server with | As a test, you can run MatLab and access your license server with | ||
<pre> | <pre> | ||
$ matlab | $ matlab -nodesktop -nojvm -r 'rand, pause(0), exit' | ||
</pre> | |||
or create an 'm file' and run it as | |||
<pre> | |||
$ matlab -nodesktop -nojvm <temp.m | |||
</pre> | </pre> | ||
For example: | For example: | ||
<pre> | <pre> | ||
$ module load | $ module load rc/matlab/R2012a | ||
$ matlab | $ matlab -nodesktop -nojvm -r 'rand, pause(0), exit' | ||
< M A T L A B (R) > | |||
Copyright 1984-2010 The MathWorks, Inc. | |||
Version 7.11.0.584 (R2010b) 64-bit (glnxa64) | |||
August 16, 2010 | |||
Line 46: | Line 52: | ||
For product information, visit www.mathworks.com. | For product information, visit www.mathworks.com. | ||
ans = | ans = | ||
Line 52: | Line 58: | ||
</pre> | </pre> | ||
This will start matlab without a graphical display and without Java support. This is good just to | This will start matlab without a graphical display and without Java support. This is good just to | ||
verify things work, but do not run any significant computations on the Cheaha head node! | verify things work, but do not run any significant computations on the Cheaha head node! | ||
MatLab computational work must be run on the compute nodes by submitting a job submission script to the SGE scheduler | MatLab computational work must be run on the compute nodes by submitting a job submission script to the SGE scheduler or by using the Parallel Computing Toolbox on your desktop. | ||
=== Serial MATLAB === | === Serial MATLAB === | ||
Line 69: | Line 78: | ||
* YOUR_EMAIL_ADDRESS | * YOUR_EMAIL_ADDRESS | ||
* h_rt and s_rt to appropriate hard and soft runtime limits | * h_rt and s_rt to appropriate hard and soft runtime limits | ||
* | * vf to the maximum amount of memory that your job will use | ||
Note, the matlab command switch '-nojvm' dramatically decreases the amount of RAM needed to start MATLAB. The work performed by your job will determine whether or not it can be run with '-nojvm' | |||
<pre> | <pre> | ||
$ mkdir -p $ | $ mkdir -p $USER_SCRATCH/jobs/matlab/serial01/output | ||
$ cd $ | $ cd $USER_SCRATCH/jobs/matlab/serial01 | ||
</pre> | </pre> | ||
<pre> | <pre> | ||
Line 80: | Line 91: | ||
# | # | ||
#$ -N serialMatlab | #$ -N serialMatlab | ||
#$ -l h_rt=00:10:00,s_rt=00:08:00, | #$ -l h_rt=00:10:00,s_rt=00:08:00,vf=2G | ||
#$ -j y | #$ -j y | ||
# | # | ||
Line 86: | Line 97: | ||
#$ -m eas | #$ -m eas | ||
# | # | ||
module load mathworks/ | module load mathworks/R2011a | ||
#$ -V | #$ -V | ||
matlab | matlab -nodisplay -nojvm < matlab-script | ||
</pre> | </pre> | ||
Then submit the script to the scheduler with | Then submit the script to the scheduler with | ||
<pre> | <pre> | ||
$ cd $ | $ cd $USER_SCRATCH/jobs/matlab/serial01 | ||
$ qsub matlabtest.qsub | $ qsub matlabtest.qsub | ||
</pre> | </pre> | ||
Line 107: | Line 118: | ||
Distributed MATLAB jobs use the following licenses: | Distributed MATLAB jobs use the following licenses: | ||
* | * MATLAB client license with the Parallel Computing Toolbox - Uses 1 client license | ||
* The Cheaha Distributed Computing license | * The Cheaha Distributed Computing license - 1 DCS license per cpu slot | ||
The client license will only be needed for as long as it takes MATLAB to start the job on the compute nodes (unless you keep the client open, for example using "waitForState(job)" in your MATLAB script). | The client license will only be needed for as long as it takes MATLAB to start the job on the compute nodes (unless you keep the client open, for example using "waitForState(job)" in your MATLAB script). This is mostly relevant if you are using a client license from a source other than the UAB MATLAB site license where there are a limited number of licenses available. | ||
The instructions are a work in progress, so please contact Research Computing support with any questions or corrections. | The instructions are a work in progress, so please contact Research Computing support with any questions or corrections. | ||
Line 116: | Line 127: | ||
First, create the working directory for the job | First, create the working directory for the job | ||
<pre> | <pre> | ||
$ mkdir -p $ | $ mkdir -p $USER_SCRATCH/jobs/matlab/distrib01/output | ||
$ cd $ | $ cd $USER_SCRATCH/jobs/matlab/distrib01 | ||
</pre> | </pre> | ||
Line 131: | Line 142: | ||
<pre> | <pre> | ||
% Always set these variables | % Always set these variables | ||
matlab_ver = ' | matlab_ver = 'R2011a'; % (MATLAB release supported by your license) R2009a R2009b R2010a | ||
email = 'YOUREMAIL'; % your email address | email = 'YOUREMAIL'; % your email address | ||
email_opt = 'eas'; % qsub email options | email_opt = 'eas'; % qsub email options | ||
Line 137: | Line 148: | ||
h_rt = '00:07:00'; % hard wall time | h_rt = '00:07:00'; % hard wall time | ||
vf = '1G'; % Amount of memory need per task | vf = '1G'; % Amount of memory need per task | ||
scratch = getenv(' | scratch = getenv('USER_SCRATCH'); | ||
outputDirectory = [scratch, '/jobs/matlab/distrib01/output']; | outputDirectory = [scratch, '/jobs/matlab/distrib01/output']; | ||
Line 157: | Line 168: | ||
createTask(job, @rand, 1, {3,3}); | createTask(job, @rand, 1, {3,3}); | ||
createTask(job, @rand, 1, {3,3}); | createTask(job, @rand, 1, {3,3}); | ||
% createTask(job, @USERFUNCTION, 1, {options}); % this can call a user m file instead of an inbuilt | |||
% function , please make sure that the user defined | |||
% function is in the MATLAB path) | |||
submit(job) | submit(job) | ||
Line 162: | Line 177: | ||
Running the MATLAB script will submit 2 SGE single slot (CPU) jobs, one for each task. The Parallel Computing Toolbox requires Java VM, so notice that for this job we do not include the "-nojvm" switch! | Running the MATLAB script will submit 2 SGE single slot (CPU) jobs, one for each task. The Parallel Computing Toolbox requires Java VM, so notice that for this job we do not include the "-nojvm" switch! | ||
Additional examples and submit scripts available on the [[MATLAB_workshop_2011#Workshop_Demo.27s | MATLAB workshop 2011 demo ]] page. | |||
<pre> | <pre> | ||
$ module load mathworks/R2010a | $ module load mathworks/R2010a | ||
$ matlab | $ matlab -nodisplay < sharedDistrib01.m | ||
</pre> | </pre> | ||
Line 190: | Line 208: | ||
First, create the working directory for the job | First, create the working directory for the job | ||
<pre> | <pre> | ||
$ mkdir -p $ | $ mkdir -p $USER_SCRATCH/jobs/matlab/paralle01/output | ||
$ cd $ | $ cd $USER_SCRATCH/jobs/matlab/parallel01 | ||
</pre> | </pre> | ||
Line 205: | Line 223: | ||
<pre> | <pre> | ||
% Always set these variables | % Always set these variables | ||
matlab_ver = ' | matlab_ver = 'R2011a'; % (MATLAB release supported by your license) R2009a R2009b R2010a | ||
email = 'YOUREMAIL'; % your email address | email = 'YOUREMAIL'; % your email address | ||
email_opt = 'eas'; % qsub email options | email_opt = 'eas'; % qsub email options | ||
Line 213: | Line 231: | ||
min_cpu_slots = 4; % Minimum number of cpu slots needed for the parallel job | min_cpu_slots = 4; % Minimum number of cpu slots needed for the parallel job | ||
max_cpu_slots = 8; % Maximum number of cpu slots needed for the parallel job | max_cpu_slots = 8; % Maximum number of cpu slots needed for the parallel job | ||
scratch = getenv(' | scratch = getenv('USER_SCRATCH'); | ||
outputDirectory = [scratch, '/jobs/matlab/parallel01/output']; | outputDirectory = [scratch, '/jobs/matlab/parallel01/output']; | ||
Line 240: | Line 258: | ||
Running the MATLAB script will submit 1 SGE job requesting 4 slots (cpu cores). The Parallel Computing Toolbox requires Java VM, so notice that for this job we do not include the "-nojvm" switch! | Running the MATLAB script will submit 1 SGE job requesting 4 slots (cpu cores). The Parallel Computing Toolbox requires Java VM, so notice that for this job we do not include the "-nojvm" switch! | ||
<pre> | <pre> | ||
$ module load mathworks/ | $ module load mathworks/R2011a | ||
$ matlab | $ matlab -nodisplay < sharedPar01.m | ||
</pre> | </pre> | ||
Line 256: | Line 274: | ||
The job output can be found in the "output" directory | The job output can be found in the "output" directory | ||
=== ParFor Parallel Example === | === Matlabpool/ParFor Parallel Example === | ||
This example will utilize the parfor parallel loop as [http://www.mathworks.com/products/parallel-computing/parallel/accelerate.html defined here]. | This example will utilize the parfor parallel loop as [http://www.mathworks.com/products/parallel-computing/parallel/accelerate.html defined here]. | ||
Matlabpool can be run on the [[Cheaha]] from the head node. | |||
<pre> | |||
1. Login to the Cheaha head node with X using: ssh -X USERID@cheaha.uabgrid.uab.edu | |||
2. load the Matlab module: module load mathworks/R2011a | |||
3. Start Matlab in X using: matlab | |||
</pre> | |||
This should open up a Matlab window on your Desktop. User needs to setup the Parallel Config and copy the required files to the user folder on the Cheaha. A similar process for the same on a desktop is mentioned [[MatLab_DCS#MATLAB_from_Your_Desktop| here]]. | |||
Validate the parallel configuration. | |||
<pre> | |||
Open matlabpool from the MATLAB command window using: matlabpool open cheaha 'numberOfProcessors'. | |||
Output: Starting matlabpool using the 'cheaha' configuration ... connected to 'numberOfProcessors' labs. | |||
Run Matlab job as you would run it using a local configuration on your Desktop. | |||
Close matlabpool using: matlabpool close all | |||
</pre> | |||
{{MATLAB Support}} | |||
[[Category:MATLAB]] | [[Category:MATLAB]] |
Latest revision as of 18:29, 28 October 2016
While MATLAB is typically used via GUI workspace environment on a local desktop, MATLAB can be accessed from a command line environment in order to call MATLAB scripts from other scripts, issue commands to a running instances, or simply to support using a text-based terminal for users who prefer or need that functionality.
The instructions below describe command-line MATLAB interaction for the HPC environment on Cheaha.
Please see the MATLAB application page for more information and a general overview of MATLAB and its use at UAB.
MATLAB Versions
Use the 'module' command to view a list of available MATLAB versions. If the version that you require isn't listed, please open a help desk ticket to request the installation.
The following is an example output of the command and doesn't necessarily represent the currently installed versions:
$ module avail rc/matlab ------------------------------------ /share/apps/modulefiles ------------------------------------- rc/matlab/R2012a rc/matlab/R2012aa rc/matlab/R2013a rc/matlab/R2015a rc/matlab/R2015b
Simple MATLAB Test
A simple test to verify that the MATLAB client on Cheaha can check out a license from your server (or from the campus license server 27000@lmgr.uabgrid.uab.edu ).
Set up your environment with the command:
$ module load rc/matlab/R2012a
As a test, you can run MatLab and access your license server with
$ matlab -nodesktop -nojvm -r 'rand, pause(0), exit'
or create an 'm file' and run it as
$ matlab -nodesktop -nojvm <temp.m
For example:
$ module load rc/matlab/R2012a $ matlab -nodesktop -nojvm -r 'rand, pause(0), exit' < M A T L A B (R) > Copyright 1984-2010 The MathWorks, Inc. Version 7.11.0.584 (R2010b) 64-bit (glnxa64) August 16, 2010 To get started, type one of these: helpwin, helpdesk, or demo. For product information, visit www.mathworks.com. ans = 0.8147
This will start matlab without a graphical display and without Java support. This is good just to verify things work, but do not run any significant computations on the Cheaha head node!
MatLab computational work must be run on the compute nodes by submitting a job submission script to the SGE scheduler or by using the Parallel Computing Toolbox on your desktop.
Serial MATLAB
Serial MATLAB jobs have the following characteristics:
* Consumes one of your client licenses for the duration of the job * Does not use the distributed licenses available on cheaha * Does not require the Parallel Computing Toolbox * Restricted to a single CPU core (slot)
See the next section for an example using the distributed computing license.
Create a job script "matlabtest.qsub" making sure to change:
* YOUR_EMAIL_ADDRESS * h_rt and s_rt to appropriate hard and soft runtime limits * vf to the maximum amount of memory that your job will use
Note, the matlab command switch '-nojvm' dramatically decreases the amount of RAM needed to start MATLAB. The work performed by your job will determine whether or not it can be run with '-nojvm'
$ mkdir -p $USER_SCRATCH/jobs/matlab/serial01/output $ cd $USER_SCRATCH/jobs/matlab/serial01
#!/bin/bash #$ -S /bin/bash #$ -cwd # #$ -N serialMatlab #$ -l h_rt=00:10:00,s_rt=00:08:00,vf=2G #$ -j y # #$ -M YOUR_EMAIL_ADDRESS #$ -m eas # module load mathworks/R2011a #$ -V matlab -nodisplay -nojvm < matlab-script
Then submit the script to the scheduler with
$ cd $USER_SCRATCH/jobs/matlab/serial01 $ qsub matlabtest.qsub
Check on it with qstat.
$ qstat -u $USER
Distributed MATLAB
These instructions provide an example of how to create and submit a distributed MATLAB job on cheaha.
Distributed MATLAB jobs use the following licenses:
- MATLAB client license with the Parallel Computing Toolbox - Uses 1 client license
- The Cheaha Distributed Computing license - 1 DCS license per cpu slot
The client license will only be needed for as long as it takes MATLAB to start the job on the compute nodes (unless you keep the client open, for example using "waitForState(job)" in your MATLAB script). This is mostly relevant if you are using a client license from a source other than the UAB MATLAB site license where there are a limited number of licenses available.
The instructions are a work in progress, so please contact Research Computing support with any questions or corrections.
First, create the working directory for the job
$ mkdir -p $USER_SCRATCH/jobs/matlab/distrib01/output $ cd $USER_SCRATCH/jobs/matlab/distrib01
Next, create a simple 2 task distributed MATLAB script called "sharedDistrib01.m" make sure to change:
* email to your email address * s_rt to an appropriate soft wall time limit * h_rt to the maximum wall time for your job * vf to the maximum memory needed for each task * outputDirectory to the directory where results should be stored
Don't make any changes to the section labeled "Configure the scheduler"
% Always set these variables matlab_ver = 'R2011a'; % (MATLAB release supported by your license) R2009a R2009b R2010a email = 'YOUREMAIL'; % your email address email_opt = 'eas'; % qsub email options s_rt = '00:05:00'; % soft wall time h_rt = '00:07:00'; % hard wall time vf = '1G'; % Amount of memory need per task scratch = getenv('USER_SCRATCH'); outputDirectory = [scratch, '/jobs/matlab/distrib01/output']; % Configure the scheduler sched = findResource('scheduler', 'type', 'generic'); set(sched, 'DataLocation' , outputDirectory); set(sched, 'ClusterMatlabRoot', ['/share/apps/mathworks/', matlab_ver]); set(sched, 'HasSharedFilesystem', true); set(sched, 'ClusterOsType' , 'unix'); set(sched, 'SubmitFcn', {@sgeSubmitFcn}); set(sched, 'DestroyJobFcn', {@sgeDestroyJob}); set(sched, 'GetJobStateFcn', {@sgeGetJobState}); sge_options = ['-pe matlab 1 -l matlab_dcs=1,vf=', vf, ',h_rt=', h_rt, ',s_rt=', s_rt, ' -m ', email_opt, ' -M ', email]; SGEClusterInfo.setExtraParameter(sge_options); get(sched) job = createJob(sched); % start of user specific commands createTask(job, @rand, 1, {3,3}); createTask(job, @rand, 1, {3,3}); % createTask(job, @USERFUNCTION, 1, {options}); % this can call a user m file instead of an inbuilt % function , please make sure that the user defined % function is in the MATLAB path) submit(job)
Running the MATLAB script will submit 2 SGE single slot (CPU) jobs, one for each task. The Parallel Computing Toolbox requires Java VM, so notice that for this job we do not include the "-nojvm" switch!
Additional examples and submit scripts available on the MATLAB workshop 2011 demo page.
$ module load mathworks/R2010a $ matlab -nodisplay < sharedDistrib01.m
Check qstat to see that the scheduler now has 2 jobs running, one for each task
$ qstat -u $USER job-ID prior name user state submit/start at queue slots ja-task-ID ----------------------------------------------------------------------------------------------------------------- 110839 0.50167 Job1.1 jdoe r 03/10/2010 16:32:37 all.q@compute-0-12.local 1 110840 0.50083 Job1.2 jdoe r 03/10/2010 16:32:37 all.q@compute-0-12.local 1
The job output can be found in the "output" directory
Parallel MATLAB
These instructions provide an example of how to create and submit a parallel MATLAB job on cheaha. Parallel MATLAB jobs require two separate licenses:
* Your own client license that includes the Parallel Computing Toolbox * The Cheaha 128 node Distributed Computing license
The client license will only be needed for as long as it takes MATLAB to start the job on the compute nodes (unless you keep the client open, for example using "waitForState(job)" in your MATLAB script).
Check out this MATLAB Help Page for a quick overview of using parallel code in your MATLAB scripts.
First, create the working directory for the job
$ mkdir -p $USER_SCRATCH/jobs/matlab/paralle01/output $ cd $USER_SCRATCH/jobs/matlab/parallel01
Next, create a simple 4 slot parallel MATLAB script called "sharedPar01.m" make sure to change:
* email to your email address * s_rt to an appropriate soft wall time limit * h_rt to the maximum wall time for your job * vf to the maximum memory needed for each task * outputDirectory to the directory where results should be stored
Don't make any changes to the section labeled "Configure the scheduler"
% Always set these variables matlab_ver = 'R2011a'; % (MATLAB release supported by your license) R2009a R2009b R2010a email = 'YOUREMAIL'; % your email address email_opt = 'eas'; % qsub email options s_rt = '00:05:00'; % soft wall time h_rt = '00:07:00'; % hard wall time vf = '1G'; % Amount of memory need per task min_cpu_slots = 4; % Minimum number of cpu slots needed for the parallel job max_cpu_slots = 8; % Maximum number of cpu slots needed for the parallel job scratch = getenv('USER_SCRATCH'); outputDirectory = [scratch, '/jobs/matlab/parallel01/output']; % Configure the scheduler sched = findResource('scheduler', 'type', 'generic'); set(sched, 'DataLocation' , outputDirectory); set(sched, 'ClusterMatlabRoot', ['/share/apps/mathworks/', matlab_ver]); set(sched, 'HasSharedFilesystem', true); set(sched, 'ClusterOsType' , 'unix'); set(sched, 'ParallelSubmitFcn', {@sgeParallelSubmitFcn}); set(sched, 'DestroyJobFcn', {@sgeDestroyJob}); set(sched, 'GetJobStateFcn', {@sgeGetJobState}); sge_options = ['-l matlab_dcs=1,vf=', vf, ',h_rt=', h_rt, ',s_rt=', s_rt, ' -m ', email_opt, ' -M ', email]; SGEClusterInfo.setExtraParameter(sge_options); get(sched) pjob = createParallelJob(sched); % start of user specific commands createTask(pjob, 'rand', 1, {4}); set(pjob, 'MinimumNumberOfWorkers', min_cpu_slots); set(pjob, 'MaximumNumberOfWorkers', max_cpu_slots); submit(pjob)
Running the MATLAB script will submit 1 SGE job requesting 4 slots (cpu cores). The Parallel Computing Toolbox requires Java VM, so notice that for this job we do not include the "-nojvm" switch!
$ module load mathworks/R2011a $ matlab -nodisplay < sharedPar01.m
Check qstat to see that the scheduler now has 2 jobs running, one for each task
$ qstat -u $USER job-ID prior name user state submit/start at queue slots ja-task-ID ----------------------------------------------------------------------------------------------------------------- 110857 0.00000 Job1 jdoe r 03/10/2010 17:20:08 4
The job output can be found in the "output" directory
Matlabpool/ParFor Parallel Example
This example will utilize the parfor parallel loop as defined here.
Matlabpool can be run on the Cheaha from the head node.
1. Login to the Cheaha head node with X using: ssh -X USERID@cheaha.uabgrid.uab.edu 2. load the Matlab module: module load mathworks/R2011a 3. Start Matlab in X using: matlab
This should open up a Matlab window on your Desktop. User needs to setup the Parallel Config and copy the required files to the user folder on the Cheaha. A similar process for the same on a desktop is mentioned here.
Validate the parallel configuration.
Open matlabpool from the MATLAB command window using: matlabpool open cheaha 'numberOfProcessors'. Output: Starting matlabpool using the 'cheaha' configuration ... connected to 'numberOfProcessors' labs. Run Matlab job as you would run it using a local configuration on your Desktop. Close matlabpool using: matlabpool close all
MATLAB Support / Mailing List
As with any application or computer language, learning to use MATLAB to analyze data or to develop or modify MATLAB applications is an individual responsibility. There is ample application documentation available from the Mathworks website, potential outreach to colleagues who also use MATLAB, and options for consultation with Mathworks. Mathworks also host on-campus training seminars several times a year and provides many on-line learning tutorials.
Installation support for MATLAB at UAB is provided by your local IT support organization and the Docs wiki.
Mathworks Website
Your first and best option for application-specific questions on MATLAB is to refer to the on-line MATLAB documentation. The Mathworks site also provides a a support matrix and an on-line knowledge base.
UAB MATLAB Wiki
The MATLAB page on the Docs wiki is the starting point for installing MATLAB at UAB and, optionally, configuring it to use cluster computing. All users are encouraged to contribute to the MATLAB knowledge in this wiki, especially if you see areas where improvements are needed. Remember, this knowledge base is only as good as the people who contribute to it.
Contributing to the wiki is as easy as clicking the login link on the top-right of the page and signing in with your UAB BlazerID. If you are unsure about making an edit, you can make suggestions for improvement on the page's Discussion tab or discuss the proposed improvement in the MATLAB user group.
UAB MATLAB User Group
At UAB, MATLAB installation support is provided by your local IT support group. Support for application specific questions is available from peers in your research group. We realize that some people are not as familiar with MATLAB as others. For this reason, we have established a MATLAB user forum (mailing list) where users of MATLAB at UAB can help answer each others questions.
This is a network of volunteers sharing their knowledge with peers. You are encouraged to reach out to this community for questions on using MATLAB by
- Joining the MATLAB user group - alternately send an email to sympa@vo.uabgrid.uab.edu with the following text in the body of the email subscribe matlab-user
- After joining the group, you can email questions to the group via matlab-user@vo.uabgrid.uab.edu
Archives of MATLAB user group discussions are available on-line at https://vo.uabgrid.uab.edu/sympa/arc/matlab-user. You may find your question is already answered in these archives.
UAB MATLAB announce mailing list
To receive information about UAB's MATLAB license and announcements please subscribe to the matlab-annc mailing list by
- Joining the MATLAB announce mailing list - alternately send an email to sympa@vo.uabgrid.uab.edu with the following text in the body of the email subscribe matlab-annc