Galaxy File Uploads: Difference between revisions
Jump to navigation
Jump to search
(updated ftp related to sections with importfs and revised other sections aswell) |
No edit summary |
||
| Line 4: | Line 4: | ||
# Importing existing files on Cheaha file system | # Importing existing files on Cheaha file system | ||
Web browser based file upload is a convenient approach, but not recommended for files larger than 2 GB in size because of browser limitations. Also, web browser based upload in Galaxy doesn't provide any feedback on upload progress and it can be an unreliable | Web browser based file upload is a convenient approach, but not recommended for files larger than 2 GB in size because of browser limitations. Also, web browser based upload in Galaxy doesn't provide any feedback on upload progress and it can be an unreliable operation. Hence, it's recommended to stage data on Galaxy accessible file-system and then import it in Galaxy. | ||
UAB Galaxy instance is configured to look for files in '/scratch/importfs/galaxy/$USER' and '/scratch/user/$USER' directories on Cheaha. Data files can be copied to Cheaha using [wikipedia:Secure_copy scp] or they can be downloaded using tools like wget, curl or ftp. Please refer to [[Cheaha_GettingStarted#Access]] page for getting access to Cheaha. | UAB Galaxy instance is configured to look for files in '/scratch/importfs/galaxy/$USER' and '/scratch/user/$USER' directories on Cheaha. Data files can be copied to Cheaha using [wikipedia:Secure_copy scp] or they can be downloaded using tools like wget, curl or ftp. Please refer to [[Cheaha_GettingStarted#Access]] page for getting access to Cheaha. | ||
| Line 11: | Line 11: | ||
# importfs or file drop-off mode: UAB Galaxy platform is configured to import files in '/scratch/importfs/galaxy/$USER' directory. Galaxy application 'moves' files from imports directory to it's internal datasets directory. See [[Galaxy_Data_Import]] page for more details on the importfs method. | # importfs or file drop-off mode: UAB Galaxy platform is configured to import files in '/scratch/importfs/galaxy/$USER' directory. Galaxy application 'moves' files from imports directory to it's internal datasets directory. See [[Galaxy_Data_Import]] page for more details on the importfs method. | ||
# Data Library: Galaxy has a concept of 'Data Libraries' which is a data container to organize files in an hierarchical manner, similar to directories on a desktop. Data libraries provide other features for data organization and sharing as well. Data libraries support direct files uploads from a web browser, data fetch from external URLs and also file system imports. The file system import options is similar to | # Data Library: Galaxy has a concept of 'Data Libraries' which is a data container to organize files in an hierarchical manner, similar to directories on a desktop. Data libraries provide other features for data organization and sharing as well. Data libraries support direct files uploads from a web browser, data fetch from external URLs and also file system imports. The file system import options is similar to importfs option described above, however, it copies file to internal datasets directory rather than moving it. UAB Galaxy platform is configured to import files in '/scratch/user/$USER' directory. See [[Galaxy_Data_Library_Import]] page for more details on this data import method. | ||
Revision as of 02:42, 8 March 2013
UAB Galaxy supports data import in three ways:
- Direct file uploads to using a web browser
- Fetching data from external URLs through Galaxy
- Importing existing files on Cheaha file system
Web browser based file upload is a convenient approach, but not recommended for files larger than 2 GB in size because of browser limitations. Also, web browser based upload in Galaxy doesn't provide any feedback on upload progress and it can be an unreliable operation. Hence, it's recommended to stage data on Galaxy accessible file-system and then import it in Galaxy.
UAB Galaxy instance is configured to look for files in '/scratch/importfs/galaxy/$USER' and '/scratch/user/$USER' directories on Cheaha. Data files can be copied to Cheaha using [wikipedia:Secure_copy scp] or they can be downloaded using tools like wget, curl or ftp. Please refer to Cheaha_GettingStarted#Access page for getting access to Cheaha.
Following sections provide an overview of UAB Galaxy import methods.
- importfs or file drop-off mode: UAB Galaxy platform is configured to import files in '/scratch/importfs/galaxy/$USER' directory. Galaxy application 'moves' files from imports directory to it's internal datasets directory. See Galaxy_Data_Import page for more details on the importfs method.
- Data Library: Galaxy has a concept of 'Data Libraries' which is a data container to organize files in an hierarchical manner, similar to directories on a desktop. Data libraries provide other features for data organization and sharing as well. Data libraries support direct files uploads from a web browser, data fetch from external URLs and also file system imports. The file system import options is similar to importfs option described above, however, it copies file to internal datasets directory rather than moving it. UAB Galaxy platform is configured to import files in '/scratch/user/$USER' directory. See Galaxy_Data_Library_Import page for more details on this data import method.