From UABgrid Documentation
COMSOL (formerly FEMLAB) is a finite element analysis, solver and Simulation software / FEA Software package for various physics and engineering applications, especially coupled phenomena, or multiphysics. COMSOL Multiphysics also offers an extensive interface to MATLAB and its toolboxes for a large variety of programming, preprocessing and postprocessing possibilities. The packages are cross-platform (Windows, Mac, Linux, Unix). In addition to conventional physics-based user-interfaces, COMSOL Multiphysics also allows for entering coupled systems of partial differential equations (PDEs).
COMSOL was started by graduate students to Germund Dahlquist based on codes developed for a graduate course at the Royal Institute of Technology (KTH) in Stockholm, Sweden.
The current version of COMSOL Multiphysiscs installed on the Cheaha is ver 4.2
Comsol can be used in a variety of interactive and batch modes on the Cheaha cluster.
- Submit cluster enabled batch jobs from the linux command prompt or scheduler script (Preferred method)
- Start Comsol desktop GUI on Cheaha and work interactively on the cluster
- Start Comsol desktop GUI on a local PC/MAC and connect to cluster enabled Comsol server on the Cheaha
Using Comsol with the Cluster
Submit cluster enabled batch jobs from command line or scheduler script (Preferred method)
Check for the version of Comsol on Cheaha
$ module avail comsol ------------------------------- /etc/modulefiles ------------------------------- comsol/comsol42
First, create the working directory for the job- Replace 'USERNAME' with the account associated username on Cheaha. You can create any directory to run your job. It is recommended that the job directory be on the scratch (i.e. lustre filesystem) instead of the user home directory.
$ mkdir -p /lustre/scratch/USERNAME/jobs/COMSOL $ cd /lustre/scratch/USERNAME/jobs/COMSOL
Next, Copy all the files required for COMSOL to the working directory Cosmol requires at least the input model file *.mph (this model can be created on the local machine) From your local machine using terminal or putty (If on Windows) transfer the files to Cheaha in the directory created above
$ scp -r model.mph USERNAME@cheaha.uabgrid.uab.edu:/lustre/scratch/USERNAME/jobs/COMSOL
Next, create a job submit script as shown below called 'comsolSubmit', make sure to edit the following parameters:
* -l h_rt to the maximum wall time for your job * -N - job name * -M - user email * -pe smp #numberOfProcessors (-pe smp 8 - run the code in parallel on 8 processors on Cheaha. Maximum smp available is 12) * -l vf to the maximum memory needed for each task (if you ask for 4GB and -pe smp 8, it will give you a total of 4GB * 8 processors =32GB for the job. Maximum memory 47GB usable on the sipsey nodes) * cd to the current working directory where the job data is stored * -np #numberOfProcessors. (Please make sure this is same as number of processors defined by the -pe option above)
#!/bin/bash #$ -S /bin/bash #$ -cwd # Specify parallel environment and number of cores required #$ -pe smp 4 # # Specify job memory(if you specify smp of more than 1, total job memory will be vf*smp) #$ -l vf=4G # # Specify job runtime(hh:mm:ss) #$ -l h_rt=1:00:00 #$ -j y # # Specify jobname and email address #$ -N comsol_test #$ -M USERNAME@uab.edu #$ -V # # Specify the job queue (leave commented for all queues) #$ -q sipsey.q # # Load modules appropriate for Comsol module load comsol/comsol42 comsol -np 4 batch -inputfile model1.mph -outputfile outputName.mph >> outputNAME.txt exit
Submit the script to the scheduler with
$ qsub comsolSubmit
The output will be
Your job 8913121 ("comsol_test") has been submitted
You can check the status of the jobs using the 'qstat' command
qstat -u $USER job-ID prior name user state submit/start at queue slots ja-task-ID ----------------------------------------------------------------------------------------------------------------- 8913121 0.51232 comsol_test tanthony r 11/9/2011 10:39:56 email@example.com 4
The job output can be found in the output directory specified earlier.
If another Comsol job is already running then we cannot checkout more licenses and the job will fail(since the current license has a limit 1) In that case the output file will read:
COMSOL 4.2 (Build: 150) starting in batch mode ******************************************* ***COMSOL 188.8.131.52 progress output file*** ******************************************* Tue Nov 22 10:40:19 CST 2011 Opening: /home/tanthony/bagvanth/Nov7th_2011_1.mph Exception: com.comsol.nativejni.FlNativeException: License error -4 Licensed number of users already reached (rethrown as com.comsol.util.exceptions.LicenseException) Messages: Could not obtain license for#RF Module '''License error -4 Licensed number of users already reached'''
Start Comsol desktop GUI on Cheaha and work interactively on the cluster
COMSOL can be run on the Cheaha interactively. (please do not run COMSOL directly on the head node) Running COMSOL interactively requires a Highspeed internet connection. A line with 1Gb/s is recommended for optimal use)
1. Login to the Cheaha head node with X using:
$ ssh -X USERID@cheaha.uabgrid.uab.edu
2. Use qlogin to request resources for the interactive session (here we are asking to be on the sipsey nodes of the Cheaha, utilizing 4 cores and 4GB ram/core =16GB ram total, for 1.00 hour )
$ qlogin -q sipsey.q -l vf=4G,h_rt=1:00:00 -pe smp 4
Your job 8219234 ("QLOGIN") has been submitted waiting for interactive job to be scheduled ... Your interactive job 8219234 has been successfully scheduled. Establishing /share/apps/scripts/qlogin_ssh_wrapper.sh session to host sipsey-compute-1-5.local ... Rocks Compute Node Rocks 5.4 (Maverick) Profile built 16:15 08-Aug-2011 Kickstarted 16:46 08-Aug-2011 [tanthony@sipsey-compute-1-5 ~]$
3. load the COMSOL module
$ module load comsol/comsol42
4. Start Comsol in X using
This should open the COMSOL desktop GUI. (This operation takes some time and is dependent on your internet conenction. Please be patient)
Start Comsol desktop GUI on a local PC/MAC and connect to cluster enabled Comsol server on the Cheaha
Work in progress to provide this functionality for COMSOL on the Cheaha