Galaxy: Difference between revisions

From Cheaha
Jump to navigation Jump to search
(Removed Cheaha Iinstallation Information, now on own wiki)
(Removed previous content; Started with user oriented documentation)
Line 2: Line 2:


= Overview =
= Overview =
[http://bitbucket.org/galaxy/galaxy-central/wiki/Home Galaxy] is an easy-to-use, open-source, scalable framework for tool and data integration. Galaxy provides access to tools (mainly comparative genomics) through an interface (e.g., a web-based interface). The [http://bitbucket.org/galaxy/galaxy-central/wiki/ImplementationInfo Galaxy framework] is implemented in the Python programming language.
[http://bitbucket.org/galaxy/galaxy-central/wiki/Home Galaxy] is an easy-to-use, open-source, scalable framework for tool and data integration. Galaxy provides access to tools (mainly comparative genomics) through an interface (e.g., a web-based interface). The [http://bitbucket.org/galaxy/galaxy-central/wiki/ImplementationInfo Galaxy framework] is implemented in the Python programming language.  


= End Users =
= Galaxy@UAB =
The UAB Galaxy instance can be accessed at http://galaxy.uabgrid.uab.edu using BlazerID credentials. We are using Galaxy revision 50e249442c5a on campus installation.


A public instance of Galaxy maintained by Penn State University is at http://usegalaxy.org/
== Hardware ==
Behind the scenes the Galaxy server at UAB is powered by [http://docs.uabgrid.uab.edu/wiki/Cheaha Cheaha cluster].


= Developers and Core Production Installation =
== Available Tools ==  


== Quickstart ==
{| border="1"
To get started with installing your own Galaxy instance, the only required component is Python (2.4,2.5, and 2.6). Four simple steps will [http://bitbucket.org/galaxy/galaxy-central/wiki/GetGalaxy get you started with your own Galaxy instance]
|+
# clone the mercurial galaxy distribution <pre>hg clone http://www.bx.psu.edu/hg/galaxy galaxy_dist</pre> or [http://bitbucket.org/galaxy/galaxy-dist/get/tip.tar.gz get a source tarball]   
! Software !! Version !! Information
# execute setup.sh
|-
# execute run.sh
! bwa
# go to http://localhost:8080  
| 0.5.9 || Further information
|-
! bowtie || 0.12.7
|-
! lastz
| 1.02.00 || Further information
|-
! samtools
| 0.1.12a || Further information
|-
! Legacy blast (megablast)
| 2.2.25 || Further information
|-
! srma
| 0.1.15 || Further information
|-
! velvet
| 1.1.03 || Further information
|-
! Top Hat
| 1.2.0 || Further information
|-
! Cuff Links
| 1.0.1 || Further information
|-
! Lift Over
| 26-Apr-2011 18:26  2.6M || Further information
|-
! R
| R-2.13.0 || Further information
|-
! RPy
| 1.0.3 || Further information
|-
! ps2pdf
| ?? || Further information
|-
! MACS
| 1.4.0rc2 || Further information
|-
! taxonomy2tree
| r3 || Further information
|-
! sputnik
| NA || Further information
|-
! beam2
| Unknown || Further information
|-
! addscores
| NA || Further information
|-
! clustalw
| 2.1 || Further information
|-
! gmaj
| NA || Further information
|-
! gpass
| NA ||  Further information
|-
! HYPHY
| 2.0020110330 beta ||  Further information
|-
! laj
| NA ||  Further information
|-
! pass2
| NA ||  Further information
|-
! twoBitToFa
| NA ||  Further information
|-
! Perl
| revision 5 version 8 subversion 8 ||  Further information
|-
! perM
| 3.3 ||  Further information
|-
! GNUPlot
| 4.4.3 ||  Further information
|-  
! Numpy
| 1.6.0 ||  Further information
|-
! numexpr
| 1.4.2 ||  Further information
|-  
! hdf5
| 1.8.7 ||  Further information
|-
! Cython
| 0.14.1 ||  Further information
|-
! Python Tables (tables)
| 2.2.1||  Further information
|-
! FastX Toolkit
| 0.0.13 || Further information


* Galaxy runs on a local webserver, [http://pythonpaste.org/script/ PasteScript] written in Python. PasteScript is based on Python's library module [http://docs.python.org/library/simplehttpserver.html simplehttpserver]and implemented with the help of python package, [http://www.owlfish.com/software/wsgiutils/index.html WSGIUtils]
|}


== CentOS School of Medicine Installation ==
== Available datasets ==  
This describes galaxy installation on a CentOS 5.5. I would suggest to others considering installation galaxy to try a distribution not based on Red Hat to avoid dealing with python system issues. Failing that, I would use virtualenv or start with a completely separate installation of python 2.6.


=== Temporary Directory and the $TEMP variable ===
It is advisable to set the $TEMP variable to a directory with lots of space, otherwise /tmp will be used which often isn't large enough to handle large files. See:
<br>
http://lists.bx.psu.edu/pipermail/galaxy-user/2009-September/000704.html
<BR>
In this case I modified run.sh (which wasn't picking up my TEMP export) to export TEMP=/home/galaxy/galaxy_temp_dir


=== Apache and Postgres Setup ===
= Public instance =
 
A public instance of Galaxy maintained by Penn State University is at http://usegalaxy.org/
* For [http://bitbucket.org/galaxy/galaxy-central/wiki/Config/ProductionServer deployment to production environments], Galaxy documentation suggests using a proxy server like Apache/Nginx to serve up static content and for handling authnz. As mentioned in [http://bitbucket.org/galaxy/galaxy-central/wiki/Config/ApacheProxy Galaxy's ApacheProxy documentation], here's how to edit Apache's httpd.conf to proxy Galaxy on port 80
<pre>
<Proxy http://localhost:8080>
        Order deny,allow
        Allow from all
</Proxy>
 
RewriteEngine on
RewriteRule ^/galaxy$ /galaxy/ [R]
RewriteRule ^/galaxy/static/style/(.*) /Users/pnm/project/galaxy_dist/static/june_2007_style/blue/$1 [L]
RewriteRule ^/galaxy/static/scripts/(.*) /Users/pnm/project/galaxy_dist/static/scripts/packed/$1 [L]
RewriteRule ^/galaxy/static/(.*) /Users/pnm/project/galaxy_dist/static/$1 [L]
RewriteRule ^/galaxy/favicon.ico /Users/pnm/project/galaxy_dist/static/favicon.ico [L]
RewriteRule ^/galaxy/robots.txt /Users/pnm/project/galaxy_dist/static/robots.txt [L]
RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P]
 
</pre>
* Change the path to where you have cloned/installed Galaxy
* http://localhost/galaxy should bring up Galaxy on port 80 now
 
Note: if you are using CentOS/RedHat as your platform, you may need to adjust or disable SELinux in order to allow Apache to access parts of the file system that are not part of the shipped default Directory directives of the Apache config.  You will see unexplained "permission denied" errors in the errors.log if you are bitten by this.
 
 
In production mode it is recommended that something other than SQLlite and the python web server be used, preferably postgres and apache. When setting up apache to proxy requests to the python web server on CentOS it is critical that the default CentOS security policy be overridden to allow proxying as shown below.
<pre>
setsebool -P httpd_can_network_relay=1
</pre>
 
Additionally, redirects to the file system are blocked by selinux security policy. The current workaround for that is to running:
<pre>
setenforce 0
</pre>
A better solution will be to allow access to the required directories using chcon like this:
<pre>
chcon -R -t httpd_user_content_t galaxy_dist/
</pre>
This has been done and appears to work, allowing selinux to be turned back on.
 
Additionally, postgres ident privileges should be changed. One workaround is to set postgres to trust all local users as shown below in /var/lib/pgsql/data/pg_hba.conf
<pre>
local  all        all                              trust
</pre>
 
SMTP Configuration
In lib/galaxy/config.py and universe_wsgi.ini
<pre>
self.smtp_server = kwargs.get( 'smtp_server', "vera.dpo.uab.edu" )
and
#smtp_server = vera.dpo.uab.edu
</pre>
 
 
=== Shibboleth Installation ===
The instructions for the base rpm install on the SP (your galaxy box) are here:
 
https://spaces.internet2.edu/display/SHIB2/NativeSPLinuxInstall
 
When doing a yum install, it will install both 32 bit and 64 bit giving this error:
 
<pre>Cannot load /usr/lib/shibboleth/mod_shib_22.so into server: /usr/lib/shibboleth/mod_shib_22.so: wrong ELF class: ELFCLASS32 </pre>
 
This can be fixed by doing:
<pre>
I have found that if you edit "/etc/httpd/conf.d/shib.conf" and alter the LoadModule line and change
 
"LoadModule mod_shib /usr/lib/shibboleth/mod_shib_22.so"
 
to
 
"LoadModule mod_shib /usr/lib64/shibboleth/mod_shib_22.so"
 
shibboleth sp will then start correctly.
</pre>
 
=== Python ===
Galaxy makes some assumptions about python being > 2.4 as of February 2011. A completely new galaxy specific install of python 2.6.6 was done on the CentOS 5 galaxy VM. Some directions are found here:
http://bda.ath.cx/blog/2009/04/08/installing-python-26-in-centos-5-or-rhel5/
 
=== RPy, R, Numpy,Nose, Atlas, BLAS, LAPACK, gFortran or g77 ===
RPy requires R. R must be entirely rebuild with shared library option set on. Not a major problem. R can also be installed without shared libs using yum install R.
 
Rpy strongly suggests that numerical python (Numpy) be installed, which is dependent on Atlas, BLAS, LAPACK and needs Nose for testing. These packages can build with either the GNU of g77 fortran compiler, but they must use all the same one (I choose GNU) and renamed g77 to diable_g77 as numpy searches for g77 first. Numpy has problems installing, originally manually copied the directory into the python lib directories as the INSTALL instructions seem to suggest that no install step is required and will be handled by setup. This is not the case, "python setup install" must be run. It is worth checking that numpy can be imported.
 
CentOS using yum install, can install atlas, blast and lapack but it cannot build rpy with it do to fortran incomaptibilies.
 
Atlas and atlas-dev installs won't be compatible with numpy - but work easily with yum install.
 
Additionally, steps on installing mercurial can be found here:
http://jake.murzy.com/post/2992010793/installing-and-setting-up-mercurial-rhel-centos-apache
 
Current issue is that rpy is old, didn't recognize the version of R (since it used 2 digits). Fixed rpy code to handle this, but rpy still looks for non-existent files in numpy. Probably these files existed in an earlier version of numpy.
 
Another workaround from: http://www.mail-archive.com/rpy-list@lists.sourceforge.net/msg01573.html
 
Just substitute line 77 of src/RPy.h
#include <Rdevice.h> /* must follow Graphics.h */
with
#include <Rembedded.h> /* must follow Graphics.h */
it builds and seems to work
 
Also worked around the lack of finding arrayobjects.h by manually copying across the numpy include directory into:
/usr/local/galaxy-python/lib/python2.6/numpy/core/numpy/include
 
The proper correct solution is to update galaxy so that it can use rpy2...
 
 
 
=== NextGen Mapping Setup (bowtie, bwa, lastz, megablast, srma ===
 
Instructions here:
https://bitbucket.org/galaxy/galaxy-central/wiki/NGSLocalSetup
Requires fastx and RPy to work.
 
Placing fastx binary files into /usr/local/bin as suggested, rather than trying to do something more advanced at this point. This does overlap with the EMBOSS binaries, probably not a good idea in the future.
Placed links to bowtie, bwa and lastz in /usr/local/bin as well.<br>
Downloaded and copied srma-0.1.14.jar to ~/galaxy_dist/tool-data/shared/jars/ <br>
megablast from NCBI was downloaded and run as an RPM, no problems. Stuck everything in /usr/bin <BR>
 
=== Post Processing ===
SAML Tools are required, though it can't convert megablast output. Problem building:
<pre>
gcc -g -Wall -O2  -o samtools bam_tview.o bam_maqcns.o bam_plcmd.o sam_view.o bam_rmdup.o bam_rmdupse.o bam_mate.o bam_stat.o bam_color.o bamtk.o kaln.o bam2bcf.o bam2bcf_indel.o errmod.o sample.o libbam.a -lm  -lcurses  -lz -Lbcftools -lbcf
libbam.a(bam_import.o): In function `__bam_get_lines':
/usr/local/src/samtools-0.1.12a/bam_import.c:76: undefined reference to `gzopen64'
libbam.a(bam_import.o): In function `sam_open':
/usr/local/src/samtools-0.1.12a/bam_import.c:442: undefined reference to `gzopen64'
libbam.a(bam_import.o): In function `sam_header_read2':
/usr/local/src/samtools-0.1.12a/bam_import.c:126: undefined reference to `gzopen64'
collect2: ld returned 1 exit status
make[1]: *** [samtools] Error 1
make[1]: Leaving directory `/usr/local/src/samtools-0.1.12a'
make: *** [all-recur] Error 1
</pre>
Looks like both 32 bit and 64 bit zlib were on the system, and it was preferentially using the 32 bit open. The 32 bit has 64 packages dependent on it, so I did:
<pre>
rpm -e --nodeps zlib-1.2.3-3.i386
rpm -e --nodeps zlib-devel-1.2.3-3.i386
</pre>
Unfortunately this failed to fix the build, as the problem may be due to the lack of a gzopen function in the CentOS version of zlib. Apparently the authors of zlib are developing on ubuntu now, another reason to switch. For more information see the post here: <BR>
http://forums.fedoraforum.org/showthread.php?t=228945
<BR>
However I did manage to find a version of libz in /usr/local/lib which based on:
<PRE> objdump -T /usr/local/lib/libz.so | grep gzopen</PRE>
Indicated that it contained gzopen64 unlike the CentOS standard version in /usr/lib64
<BR>
Unfortunately, make continued to fail after I linked in the gzopen64 containing zlib library due to the inability of the make script to find libbc as specified as the gcc parameter -lbc. However this turned out to be a typo on my part, the actual fix was to adjust the CFLAGS setting in the Makefile to:
<PRE>
CFLAGS=        -g -Wall -O2 -L/usr/local/lib #-m64 #-arch ppc
</PRE>
I then re-installed the 32 bit zlib version as it has a number of CentOS dependencies I don't want to break:
<PRE>
yum install zlib-1.2.3-3.i386
yum install zlib-devel-1.2.3-3.i386
</PRE>
Also, samtools needs the version of the zlib library to run, so I set the galaxy LD_LIBRARY_PATH to check /usr/local/lib first in order to load the correct library.
 
 
=== GNUPlot ===
Is required for some plotting, including seeing fastq quality statistics. I did:
<PRE>yum install gnuplot</PRE>
However this gives an error as shown below
<PRE>line 0: undefined variable inside</PRE>
and also described here:
http://cell-innovation.nig.ac.jp/wiki/tiki-view_forum_thread.php?topics_offset=1&forumId=7&comments_parentId=103
 
But it still works!
I tried upgrading to gnuplot 4.4.1 and 4.4.2 but there are other dependency issues on libg, newer versions of Cairo and pango that make it not worthwhile to try on this VM.
 
=== SNP Analysis ===
Apparently there is a program called SNPeff (http://snpeff.sourceforge.net/)  which is integrated with galaxy but doesn't come set up with the base install of galaxy. It requires java, which was installed easily with:
<Pre>
yum install java
</PRE>
SNPeff should go in /usr/local/snpeff
 
=== Errors/Bugs ===
As per galaxy-dev email I sent the file:
lib/galaxy/model/migrate/versions/0068_rename_sequencer_to_external_services.py
Contains SQL errors as it tries to rename a SEQUENCE with the incorrect sytax. ALTER TABLE should be used, not ALTER SEQUENCE. This file has errors in 2 locations.
The problem remains as of February the 7th, 2011.


= Support =
In order to facilitate interaction among UAB Galaxy users, share experience, and provide peer-support UAB IT Research Computing has established a galaxy-users group. To join this group and participate in email discussions please subscribe to the [http://vo.uabgrid.uab.edu/sympa/subscribe/galaxy-user galaxy-user group]. On-line archives of these discussions are available [https://vo.uabgrid.uab.edu/sympa/arc/galaxy-user here]. Please note, the email discussions are a public forum. You are advised to only post information you are authorized to share and comfortable with being public.




Revision as of 22:32, 26 May 2011

Overview

Galaxy is an easy-to-use, open-source, scalable framework for tool and data integration. Galaxy provides access to tools (mainly comparative genomics) through an interface (e.g., a web-based interface). The Galaxy framework is implemented in the Python programming language.

Galaxy@UAB

The UAB Galaxy instance can be accessed at http://galaxy.uabgrid.uab.edu using BlazerID credentials. We are using Galaxy revision 50e249442c5a on campus installation.

Hardware

Behind the scenes the Galaxy server at UAB is powered by Cheaha cluster.

Available Tools

Software Version Information
bwa 0.5.9 Further information
bowtie 0.12.7
lastz 1.02.00 Further information
samtools 0.1.12a Further information
Legacy blast (megablast) 2.2.25 Further information
srma 0.1.15 Further information
velvet 1.1.03 Further information
Top Hat 1.2.0 Further information
Cuff Links 1.0.1 Further information
Lift Over 26-Apr-2011 18:26 2.6M Further information
R R-2.13.0 Further information
RPy 1.0.3 Further information
ps2pdf ?? Further information
MACS 1.4.0rc2 Further information
taxonomy2tree r3 Further information
sputnik NA Further information
beam2 Unknown Further information
addscores NA Further information
clustalw 2.1 Further information
gmaj NA Further information
gpass NA Further information
HYPHY 2.0020110330 beta Further information
laj NA Further information
pass2 NA Further information
twoBitToFa NA Further information
Perl revision 5 version 8 subversion 8 Further information
perM 3.3 Further information
GNUPlot 4.4.3 Further information
Numpy 1.6.0 Further information
numexpr 1.4.2 Further information
hdf5 1.8.7 Further information
Cython 0.14.1 Further information
Python Tables (tables) 2.2.1 Further information
FastX Toolkit 0.0.13 Further information

Available datasets

Public instance

A public instance of Galaxy maintained by Penn State University is at http://usegalaxy.org/

Support

In order to facilitate interaction among UAB Galaxy users, share experience, and provide peer-support UAB IT Research Computing has established a galaxy-users group. To join this group and participate in email discussions please subscribe to the galaxy-user group. On-line archives of these discussions are available here. Please note, the email discussions are a public forum. You are advised to only post information you are authorized to share and comfortable with being public.


References

Retrieved from "https://docs.uabgrid.uab.edu/w/index.php?title=Galaxy&oldid=2804"