Galaxy Importfs

UAB Galaxy can import data from a user's importfs directory on Cheaha cluster. This is a two step process as follows:

1. Stage data files in importfs directory - /scratch/importfs/galaxy/$USER
2. Select files that need to be uploaded into the galaxy

The first step of data staging in importfs directory happens outside of Galaxy. In the second step, a Galaxy user can select files in importfs staging directory and import them into Galaxy. When files in importfs staging directory are uploaded in Galaxy, their original version in importfs staging directory is deleted by Galaxy application.

Following documentation outlines steps involved in using UAB Galaxy's data import method.

Account setup

1. You need a Cheaha cluster account in order to transfer your data files to '/scratch/importfs/galaxy/$USER' directory. Please refer to Cheaha_GettingStarted#Access page for getting Cheaha account.
2. Your '/scratch/importfs/galaxy/$USER' directory should get configured within 30-minutes after you get your Cheaha account.
3. Make sure you can login to Cheaha cluster as documented Cheaha_GettingStarted#Login.

Data Transfer

Data transfer to Cheaha is briefly described in Cheaha_GettingStarted#Uploading_Data. Following examples cover Galaxy specific use cases to transfer data to '/scratch/importfs/galaxy/$USER' directory.

Pulling data from external URLs

A file accessible via external FTP or HTTP location can be downloaded using FTP or HTTP network download tools. Below is an example using wget network downloader tool which should work for both ftp and http URLs.

# Change to Directory "/scratch/importfs/galaxy/$USER"
$ cd /scratch/importfs/galaxy/$USER
$ wget http://pavgi.uabgrid.uab.edu/seq.bam

Note: wget has many other useful options which may help in downloading data. Refer to wget manual (command: man wget) to see all options.

Copying data to Cheaha using scp

scp is a file transfer protocol to securely copy files between two computer systems. Mac and Linux systems natively comes with scp support, however, Windows systems will need an external client like pscp or WinSCP. Following example shows native scp client usage from Linux or Mac system. It should work in a similar manner with other scp clients.

# scp <path-to-file-on-local-desktop> <BlazerID>@cheaha.uabgrid.uab.edu:/scratch/importfs/galaxy/<BlazerID>/
$ scp users.csv pavgi@cheaha.uabgrid.uab.edu:/scratch/importfs/galaxy/pavgi/

Above command will transfer a file on a local desktop system to '/scratch/importfs/galaxy/<BlazerID>/' directory on Cheaha.

Data import in Galaxy

Files deposited in '/scratch/importfs/galaxy/$USER' directory can be seen by Galaxy application and they are listed in data upload method as shown below.

Above example shows only file staged in importfs directory, however, you can multiple files staged and selected for data import in Galaxy. All files successfully imported in Galaxy will show up right-side history panel and they will be deleted from importfs directory.