Open OnDemand Sandbox: Difference between revisions

From Cheaha
Jump to navigation Jump to search
(→‎Building your first dev app: Emphasis role or script.sh.erb)
Line 14: Line 14:
===Building your first dev app===
===Building your first dev app===


Over here we provide instructions to create your first example dev script:
Over here you have instructions to create your first example dev app:
 
====IGV====


* Under '''Develop''' tab , click on '''My SandBox Apps (Development)'''
* Under '''Develop''' tab , click on '''My SandBox Apps (Development)'''
Line 25: Line 27:
<pre>
<pre>
Directory Name: IGV
Directory Name: IGV
Git Remote: https://github.com/uabrc/ood_igv.git
Git Remote: https://github.com/uabrc/sandbox_app_template
</pre>
</pre>
and '''Submit'''
and '''Submit'''
Line 33: Line 35:
* Now click on '''Files''' to take a look at the files for the app (local git repo working directory), that you just cloned.
* Now click on '''Files''' to take a look at the files for the app (local git repo working directory), that you just cloned.


* Highlight '''form.yml''' and click on '''Edit''' . Change the file to the following content:
* Let's modify '''manifest.yml''' file (highlight the file, and click on '''edit''') to change metadata for the sandbox app. Change the SOFTWARE_NAME and SOFTWARE_LINK variables appropriately.
In this case change then to following:
<pre>
<pre>
---
SOFTWARE_NAME : IGV
cluster: "SLURM_CLUSTER"
SOFTWARE_LINK : http://software.broadinstitute.org/software/igv/
attributes:
</pre>
  bc_num_hours:
    value: 1
 
  bc_num_slots:
    label: Number of CPU
    value: 1
    min: 1
    max: 24
    step: 1
 
  bc_num_mems:
    widget: "number_field"
    label: Memory per CPU (GB)
    value: 4
    min: 1
    max: 128
    step: 1
 
  bc_partition:
    widget: select
    label: Partition
    options:
      - [ "express", "express" ]
      - [ "short", "short" ]
      - [ "medium", "medium" ]
      - [ "long", "long" ]
      - [ "interactive", "interactive" ]
      - [ "pascalnodes", "pascalnodes" ]


* Next, highlight '''form.yml''' and click on '''Edit''' . Uncomment version section in the file, and change that section as follows:
<pre>
   version:
   version:
     widget: select
     widget: select
Line 73: Line 50:
     options:
     options:
       - [ "2.5.0-Java-11", "IGV/2.5.0-Java-11" ]
       - [ "2.5.0-Java-11", "IGV/2.5.0-Java-11" ]
form:
  - version
  - bc_num_hours
  - bc_partition
  - bc_num_slots
  - bc_num_mems
  - bc_email_on_started
</pre>
</pre>
and save the file, and close the tab.
and save the file, and close the tab.
Change '''SOFTWARE_NAME''' appropriately. Under options change '''display_version''' to the the version that you want to display on the form, and change '''corresponding_cheaha_module''' to matching module name on cheaha.


Here we are specifying what the form for the app is going to look like, and changing available parameters for Cheaha.
Here we are specifying what the form for the app is going to look like, and changing available parameters for Cheaha.


* Now again in the file explorer for your app, click on '''New File''' with name '''submit.yml.erb'''
The "native" section in '''submit.yml.erb''' includes the slurm parameters you would use with sbatch and describes the resources needed for the app.  These can include variable names defined in the form above which will get substituted for the user provided values when the app is run.


* Edit the file submit.yml.erb, with the following contentThe "native" sectino includes the slurm parameters you would use with sbatch and describes the resources needed for the app.  These can include variable names defined in the form above which will get substituted for the user provided values when the app is run.
* Now, go into '''template''' and edit file '''script.sh.erb''', by adding following lines at the end of the fileThis is the actual job script that will be submitted to the cluster, just like you would submit with sbatch from the command line.
<pre>
<pre>
---
igvtools_gui.command &
batch_connect:
igv.sh
  template: "vnc"
script:
  native:
    - "-N 1"
    - "-n <%= bc_num_slots.blank? ? 1 : bc_num_slots.to_i %>"
    - "--mem-per-cpu=<%= bc_num_mems.blank? ? 4 : bc_num_mems.to_i %>G"
    - "--partition=<%= bc_partition %>"
    - "--time=<%= bc_num_hours.blank? ? 1 : bc_num_hours.to_i %>:00:00"
    - "--job-name=ood-igv"
<%- if bc_partition == "pascalnodes" -%>
    - "--gres=gpu:1"
<%- end -%>
</pre>
</pre>
SAve the file, and close the tab
Save the file, and close the tab.
This script runs the app executable after loading the appropriate module, based on the version you choose in the form.
 
* Close the file browser for your app.
 
* Click on '''Launch IGV''', and submit the job with desired parameters.


* Now, go into '''template''' folder and create a new file '''script.sh.erb''', and add following content to it.  This is the actual job script that will be submitted to the cluster, just like you would submit with sbatch from the command line.
You should have your first dev app (IGV) running.
<pre>
#!/usr/bin/env bash




# Clean the environment
====FSL====
module reset
* Under '''Develop''' tab , click on '''My SandBox Apps (Development)'''


# Set working directory to home directory
* Now click on '''New App''' tab
cd "${HOME}"
 
* Now we are going to import an already existing git repo for an app, by clicking on '''Clone Existing App'''


#
* Add the following values in the form:
# Launch Xfce Window Manager and Panel
<pre>
#
Directory Name: FSL
Git Remote: https://github.com/uabrc/sandbox_app_template
</pre>
and '''Submit'''


(
'''Note:''' You can also check the line, about creating a new git repo, if you want to use it to maintain your own instance of the app.
  export SEND_256_COLORS_TO_REMOTE=1
  export XDG_CONFIG_HOME="<%= session.staged_root.join("config") %>"
  export XDG_DATA_HOME="<%= session.staged_root.join("share") %>"
  export XDG_CACHE_HOME="$(mktemp -d)"
  module restore
  set -x
  xfwm4 --compositor=off --daemon --sm-client-disable
  xsetroot -solid "#D3D3D3"
  xfsettingsd --sm-client-disable
  xfce4-panel --sm-client-disable
) &


#
* Now click on '''Files''' to take a look at the files for the app (local git repo working directory), that you just cloned.
# Start Integrative Genomics Viewer
#


# Load the required environment
* Let's modify '''manifest.yml''' file (highlight the file, and click on '''edit''') to change metadata for the sandbox app. Change the SOFTWARE_NAME and SOFTWARE_LINK variables appropriately.
module load <%= context.version %>
In this case change then to following:
# Launch IGV
<pre>
module list
SOFTWARE_NAME : FSL
set -x
SOFTWARE_LINK : https://www.fmrib.ox.ac.uk/fsl
igvtools_gui.command &
igv.sh
</pre>
</pre>
Save the file, and close the tab. Now click on '''Open in terminal''', and run
 
* Next, highlight '''form.yml''' and click on '''Edit''' . Uncomment version section in the file, and change that section as follows:
<pre>
<pre>
chmod +x script.sh.erb
  version:
    widget: select
    label: "FSL version"
    help: "This defines the version of FSL you want to load."
    options:
      - [ "6.0.3", "FSL/6.0.3" ]
</pre>
</pre>
, and then close the terminal. This script needs to be an executable file.
and save the file, and close the tab.
 
Change '''SOFTWARE_NAME''' appropriately. Under options change '''display_version''' to the the version that you want to display on the form, and change '''corresponding_cheaha_module''' to matching module name on cheaha.
 
Here we are specifying what the form for the app is going to look like, and changing available parameters for Cheaha.


The "native" section in '''submit.yml.erb''' includes the slurm parameters you would use with sbatch and describes the resources needed for the app.  These can include variable names defined in the form above which will get substituted for the user provided values when the app is run.
* Now, go into '''template''' and edit file '''script.sh.erb''', by adding following lines at the end of the file.  This is the actual job script that will be submitted to the cluster, just like you would submit with sbatch from the command line.
<pre>
source /share/apps/rc/software/FSL/6.0.3/fsl/etc/fslconf/fsl.sh
fsl
</pre>
Save the file, and close the tab.
This script runs the app executable after loading the appropriate module, based on the version you choose in the form.
This script runs the app executable after loading the appropriate module, based on the version you choose in the form.


* Now close the file browser for your app.
* Close the file browser for your app.


* Now click, on 'Launch IGV', and submit the job.
* Click on '''Launch IGV''', and submit the job with desired parameters.


You should have your first dev app running.
You should have your first dev app (IGV) running.


===More info===
===More info===

Revision as of 15:12, 10 August 2020

Open OnDemand is a web interface to Cheaha HPC resources. We have setup multiple general purpose apps, such as HPC Desktop, Jupyter, RStudio etc, on cheaha, to make the process of accessing these basic softwares easier. OnDemand also allows users to setup their own personal apps in the OnDemand, as sandbox apps.

Setting up sandbox for your profile

Setting up sandbox is pretty simple and straightforward.

  • You need to create a dev folder in your $USER_DATA/ondemand folder
mkdir -p $USER_DATA/ondemand/dev
  • Then go to Cheaha OnDemand , login, and under Help, click on Restart Web Server
  • You have successfully enabled sandbox for your environment, and you should see a new Develop tab.

Building your first dev app

Over here you have instructions to create your first example dev app:

IGV

  • Under Develop tab , click on My SandBox Apps (Development)
  • Now click on New App tab
  • Now we are going to import an already existing git repo for an app, by clicking on Clone Existing App
  • Add the following values in the form:
Directory Name: IGV
Git Remote: https://github.com/uabrc/sandbox_app_template

and Submit

Note: You can also check the line, about creating a new git repo, if you want to use it to maintain your own instance of the app.

  • Now click on Files to take a look at the files for the app (local git repo working directory), that you just cloned.
  • Let's modify manifest.yml file (highlight the file, and click on edit) to change metadata for the sandbox app. Change the SOFTWARE_NAME and SOFTWARE_LINK variables appropriately.

In this case change then to following:

SOFTWARE_NAME : IGV
SOFTWARE_LINK : http://software.broadinstitute.org/software/igv/
  • Next, highlight form.yml and click on Edit . Uncomment version section in the file, and change that section as follows:
  version:
    widget: select
    label: "IGV version"
    help: "This defines the version of IGV you want to load."
    options:
      - [ "2.5.0-Java-11", "IGV/2.5.0-Java-11" ]

and save the file, and close the tab.

Change SOFTWARE_NAME appropriately. Under options change display_version to the the version that you want to display on the form, and change corresponding_cheaha_module to matching module name on cheaha.

Here we are specifying what the form for the app is going to look like, and changing available parameters for Cheaha.

The "native" section in submit.yml.erb includes the slurm parameters you would use with sbatch and describes the resources needed for the app. These can include variable names defined in the form above which will get substituted for the user provided values when the app is run.

  • Now, go into template and edit file script.sh.erb, by adding following lines at the end of the file. This is the actual job script that will be submitted to the cluster, just like you would submit with sbatch from the command line.
igvtools_gui.command &
igv.sh

Save the file, and close the tab. This script runs the app executable after loading the appropriate module, based on the version you choose in the form.

  • Close the file browser for your app.
  • Click on Launch IGV, and submit the job with desired parameters.

You should have your first dev app (IGV) running.


FSL

  • Under Develop tab , click on My SandBox Apps (Development)
  • Now click on New App tab
  • Now we are going to import an already existing git repo for an app, by clicking on Clone Existing App
  • Add the following values in the form:
Directory Name: FSL
Git Remote: https://github.com/uabrc/sandbox_app_template

and Submit

Note: You can also check the line, about creating a new git repo, if you want to use it to maintain your own instance of the app.

  • Now click on Files to take a look at the files for the app (local git repo working directory), that you just cloned.
  • Let's modify manifest.yml file (highlight the file, and click on edit) to change metadata for the sandbox app. Change the SOFTWARE_NAME and SOFTWARE_LINK variables appropriately.

In this case change then to following:

SOFTWARE_NAME : FSL
SOFTWARE_LINK : https://www.fmrib.ox.ac.uk/fsl
  • Next, highlight form.yml and click on Edit . Uncomment version section in the file, and change that section as follows:
  version:
    widget: select
    label: "FSL version"
    help: "This defines the version of FSL you want to load."
    options:
      - [ "6.0.3", "FSL/6.0.3" ]

and save the file, and close the tab.

Change SOFTWARE_NAME appropriately. Under options change display_version to the the version that you want to display on the form, and change corresponding_cheaha_module to matching module name on cheaha.

Here we are specifying what the form for the app is going to look like, and changing available parameters for Cheaha.

The "native" section in submit.yml.erb includes the slurm parameters you would use with sbatch and describes the resources needed for the app. These can include variable names defined in the form above which will get substituted for the user provided values when the app is run.

  • Now, go into template and edit file script.sh.erb, by adding following lines at the end of the file. This is the actual job script that will be submitted to the cluster, just like you would submit with sbatch from the command line.
source /share/apps/rc/software/FSL/6.0.3/fsl/etc/fslconf/fsl.sh
fsl

Save the file, and close the tab. This script runs the app executable after loading the appropriate module, based on the version you choose in the form.

  • Close the file browser for your app.
  • Click on Launch IGV, and submit the job with desired parameters.

You should have your first dev app (IGV) running.

More info

To get more example apps to build or to build passenger apps, look here.


Contact us if you run into any issues, or to discuss any new app.