COMSOL

COMSOL (formerly FEMLAB) is a finite element analysis, solver and Simulation software / FEA Software package for various physics and engineering applications, especially coupled phenomena, or multiphysics. COMSOL Multiphysics also offers an extensive interface to MATLAB and its toolboxes for a large variety of programming, preprocessing and postprocessing possibilities. The packages are cross-platform (Windows, Mac, Linux, Unix). In addition to conventional physics-based user-interfaces, COMSOL Multiphysics also allows for entering coupled systems of partial differential equations (PDEs).

COMSOL was started by graduate students to Germund Dahlquist based on codes developed for a graduate course at the Royal Institute of Technology (KTH) in Stockholm, Sweden.

The current version of COMSOL Multiphysiscs installed on the Cheaha is ver 4.2

Overview

Comsol can be used in a variety of interactive and batch modes on the Cheaha cluster.

1. Submit cluster enabled batch jobs from the linux command prompt or scheduler script (Preferred method)
2. Start Comsol desktop GUI on Cheaha and work interactively on the cluster
3. Start Comsol desktop GUI on a local PC/MAC and connect to cluster enabled Comsol server on the Cheaha

Using Comsol with the Cluster

Submit cluster enabled batch jobs from command line or scheduler script (Preferred method)

Check for the version of Comsol on Cheaha

 $ module avail comsol

------------------------------- /etc/modulefiles -------------------------------
comsol/comsol42

First, create the working directory for the job- Replace 'USERNAME' with the account associated username on Cheaha. You can create any directory to run your job. It is recommended that the job directory be on the scratch (i.e. lustre filesystem) instead of the user home directory.

$ mkdir -p /lustre/scratch/USERNAME/jobs/COMSOL 
$ cd /lustre/scratch/USERNAME/jobs/COMSOL

Next, Copy all the files required for COMSOL to the working directory Cosmol requires at least the input model file *.mph (this model can be created on the local machine) From your local machine using terminal or putty (If on Windows) transfer the files to Cheaha in the directory created above

$ scp -r model.mph USERNAME@cheaha.uabgrid.uab.edu:/lustre/scratch/USERNAME/jobs/COMSOL 

Next, create a job submit script as shown below called 'comsolSubmit', make sure to edit the following parameters:

* -l h_rt to the maximum wall time for your job
* -N - job name
* -M - user email
* -pe smp #numberOfProcessors  (-pe smp 8 - run the code in parallel on 8  processors on Cheaha. Maximum smp available is 12)
* -l vf to the maximum memory needed for each task (if you ask for 4GB and -pe smp 8, it will give you a total of 4GB * 8 processors =32GB for the job. Maximum memory 47GB usable on the sipsey nodes)
* cd to the current working directory where the job data is stored
* -np #numberOfProcessors. (Please make sure this is same as number of processors  defined by the -pe option above)

#!/bin/bash
#$ -S /bin/bash
#$ -cwd
# Specify parallel environment and number of cores required
#$ -pe smp 4
#
# Specify job memory(if you specify smp of more than 1, total job memory will be vf*smp) 
#$ -l vf=4G
#
# Specify job runtime(hh:mm:ss)
#$ -l h_rt=1:00:00
#$ -j y
#
# Specify jobname and email address
#$ -N comsol_test
#$ -M USERNAME@uab.edu
#$ -V
# 
# Specify the job queue (leave commented for all queues)
#$ -q sipsey.q
#
# Load modules appropriate for Comsol
 module load comsol/comsol42



comsol -np 4 batch -inputfile model1.mph -outputfile outputName.mph >> outputNAME.txt



exit

Submit the script to the scheduler with

$ qsub comsolSubmit

The output will be

Your job 8913121 ("comsol_test") has been submitted

You can check the status of the jobs using the 'qstat' command

qstat -u $USER
job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID 
-----------------------------------------------------------------------------------------------------------------
8913121 0.51232 comsol_test tanthony     r     11/9/2011 10:39:56 sipsey.q@sipsey-compute-1-11.l    4 

The job output can be found in the output directory specified earlier.

If another Comsol job is already running then we cannot checkout more licenses and the job will fail(since the current license has a limit 1) In that case the output file will read:

COMSOL 4.2 (Build: 150) starting in batch mode
*******************************************
***COMSOL 4.2.0.150 progress output file***
*******************************************
Tue Nov 22 10:40:19 CST 2011
Opening: /home/tanthony/bagvanth/Nov7th_2011_1.mph
Exception:
    com.comsol.nativejni.FlNativeException: License error -4 Licensed number
 of users already reached
    (rethrown as com.comsol.util.exceptions.LicenseException)
Messages:
    Could not obtain license for#RF Module

    '''License error -4 Licensed number of users already reached'''

Start Comsol desktop GUI on Cheaha and work interactively on the cluster

COMSOL can be run on the Cheaha interactively. (please do not run COMSOL directly on the head node) Running COMSOL interactively requires a Highspeed internet connection. A line with 1Gb/s is recommended for optimal use)

1. Login to the Cheaha head node with X using:

$ ssh -X USERID@cheaha.uabgrid.uab.edu 

2. Use qlogin to request resources for the interactive session (here we are asking to be on the sipsey nodes of the Cheaha, utilizing 4 cores and 4GB ram/core =16GB ram total, for 1.00 hour )

$ qlogin -q sipsey.q -l vf=4G,h_rt=1:00:00 -pe smp 4 

Your job 8219234 ("QLOGIN") has been submitted
waiting for interactive job to be scheduled ...
Your interactive job 8219234 has been successfully scheduled.
Establishing /share/apps/scripts/qlogin_ssh_wrapper.sh  session to host sipsey-compute-1-5.local ...
Rocks Compute Node
Rocks 5.4 (Maverick)
Profile built 16:15 08-Aug-2011

Kickstarted 16:46 08-Aug-2011
[tanthony@sipsey-compute-1-5 ~]$ 

3. load the COMSOL module

$ module load comsol/comsol42 

4. Start Comsol in X using

$ comsol 

This should open the COMSOL desktop GUI. (This operation takes some time and is dependent on your internet conenction. Please be patient)

Start Comsol desktop GUI on a local PC/MAC and connect to cluster enabled Comsol server on the Cheaha

Work in progress to provide this functionality for COMSOL on the Cheaha