UAB Condor Pilot - Revision history

Jpr@uab.edu: Added link to PDF version

2012-06-21T16:33:43Z

Added link to PDF version

← Older revision		Revision as of 16:33, 21 June 2012
Line 1:		Line 1:
	The '''UAB Condor Pilot''' explored the utility of the cloud computing paradigm to research applications using aggregated, unused compute cycles harvested from many computers. The pilot established a demonstration spare-cycle compute fabric using the [[wikipedia:Condor_High-Throughput_Computing_System\|Condor scheduler]] and compared the performance of a [[wikipedia:Docking_(molecular)\|molecular docking]] workflow on this fabric and several larger production Condor fabrics to the performance of the same workflow running on our campus compute cluster [[Cheaha]].		The '''UAB Condor Pilot''' explored the utility of the cloud computing paradigm to research applications using aggregated, unused compute cycles harvested from many computers. The pilot established a demonstration spare-cycle compute fabric using the [[wikipedia:Condor_High-Throughput_Computing_System\|Condor scheduler]] and compared the performance of a [[wikipedia:Docking_(molecular)\|molecular docking]] workflow on this fabric and several larger production Condor fabrics to the performance of the same workflow running on our campus compute cluster [[Cheaha]].

	The UAB Condor Pilot successfully demonstrated the value of harvested unused compute cycles to the molecular docking workflow. The results suggest that similar applications, especially those that can scale by repeating the same task on distinct data sets, will likewise benefit from the abundant compute resources that can be harvested via a Condor compute fabric. The UAB Condor Pilot ended in May 2012 with the presentation of [http://research.cs.wisc.edu/condor/CondorWeek2012/presentations/robinson-uab.pdf our results (pdf)] at [http://research.cs.wisc.edu/condor/CondorWeek2012/ Condor Week 2012].		The UAB Condor Pilot successfully demonstrated the value of harvested unused compute cycles to the molecular docking workflow. The results suggest that similar applications, especially those that can scale by repeating the same task on distinct data sets, will likewise benefit from the abundant compute resources that can be harvested via a Condor compute fabric. The UAB Condor Pilot ended in May 2012 with the presentation of [http://research.cs.wisc.edu/condor/CondorWeek2012/presentations/robinson-uab.pdf our results (pdf)] at [http://research.cs.wisc.edu/condor/CondorWeek2012/ Condor Week 2012]. This page is [[Media:UAB_Condor_Pilot-20120619-1012.pdf\|available as a PDF]].

	== Background ==		== Background ==

Jpr@uab.edu: Emphasize cloud computing paradigm explored during pilot

2012-06-19T15:12:11Z

Emphasize cloud computing paradigm explored during pilot

← Older revision		Revision as of 15:12, 19 June 2012
Line 1:		Line 1:
	The '''UAB Condor Pilot''' explored the utility to research applications of aggregated unused compute cycles harvested from many computers. The pilot established a demonstration spare-cycle compute fabric using the [[wikipedia:Condor_High-Throughput_Computing_System\|Condor scheduler]] and compared the performance of a [[wikipedia:Docking_(molecular)\|molecular docking]] workflow on this fabric and several larger production Condor fabrics to the performance of the same workflow running on our campus compute cluster [[Cheaha]].		The '''UAB Condor Pilot''' explored the utility of the cloud computing paradigm to research applications using aggregated, unused compute cycles harvested from many computers. The pilot established a demonstration spare-cycle compute fabric using the [[wikipedia:Condor_High-Throughput_Computing_System\|Condor scheduler]] and compared the performance of a [[wikipedia:Docking_(molecular)\|molecular docking]] workflow on this fabric and several larger production Condor fabrics to the performance of the same workflow running on our campus compute cluster [[Cheaha]].

	The UAB Condor Pilot successfully demonstrated the value of harvested unused compute cycles to the molecular docking workflow. The results suggest that similar applications, especially those that can scale by repeating the same task on distinct data sets, will likewise benefit from the abundant compute resources that can be harvested via a Condor compute fabric. The UAB Condor Pilot ended in May 2012 with the presentation of [http://research.cs.wisc.edu/condor/CondorWeek2012/presentations/robinson-uab.pdf our results (pdf)] at [http://research.cs.wisc.edu/condor/CondorWeek2012/ Condor Week 2012].		The UAB Condor Pilot successfully demonstrated the value of harvested unused compute cycles to the molecular docking workflow. The results suggest that similar applications, especially those that can scale by repeating the same task on distinct data sets, will likewise benefit from the abundant compute resources that can be harvested via a Condor compute fabric. The UAB Condor Pilot ended in May 2012 with the presentation of [http://research.cs.wisc.edu/condor/CondorWeek2012/presentations/robinson-uab.pdf our results (pdf)] at [http://research.cs.wisc.edu/condor/CondorWeek2012/ Condor Week 2012].

Jpr@uab.edu: /* Condor */ Expand on the description of Condor's operation and capabilities

2012-06-06T21:01:54Z

Condor: Expand on the description of Condor's operation and capabilities

← Older revision		Revision as of 21:01, 6 June 2012
Line 8:		Line 8:

	Condor is a resource allocation and management system designed to simplify harvesting idle compute cycles from under-utilized computers. Condor is a production-quality software system developed by researchers at the University of Wisconsin. It is deployed in a wide range of environments from lab or departmental compute pools with 10's of processors to global compute fabrics such as the [https://opensciencegrid.org/ Open Science Grid (OSG)] that averages between 80,000 to 100,000 processors.		Condor is a resource allocation and management system designed to simplify harvesting idle compute cycles from under-utilized computers. Condor is a production-quality software system developed by researchers at the University of Wisconsin. It is deployed in a wide range of environments from lab or departmental compute pools with 10's of processors to global compute fabrics such as the [https://opensciencegrid.org/ Open Science Grid (OSG)] that averages between 80,000 to 100,000 processors.

			Condor organizes computers offering their spare cycles into resource collections called "pools". A Condor pool provides compute cycles to applications in the context of jobs by matching criteria expressed by the applications to capabilities expressed by the resources; a process descriptively entitled "match making". A special focus of Condor is to avoid inconveniencing resource providers who offer their spare cycles to applications. Condor can quickly evacuate a job from an assigned resource, e.g. an end user's desktop computer, should activity on the mouse or keyboard be detected. Condor also provides mechanisms to the job owner to checkpoint work and avoid loss of forward progress in this event. Condor is a flexible system that can harness loosely-coupled systems as well as traditional tightly-coupled cluster systems. The focus of this pilot was on harvesting compute cycles of loosely coupled systems.

	There is an active user and developer community around Condor. Condor is supported on Linux, Mac and Windows ensuring utility to a broad spectrum of applications, from scientific computations to large scale statistical analyses. There are personal instances to support individuals migrating or developing their own workflows. The loosely-coupled nature of Condor resource collections make it straight forward to dynamically scale out on popular cloud computing fabrics such as [http://aws.amazon.com/ec2/ Amazon EC2].		There is an active user and developer community around Condor. Condor is supported on Linux, Mac and Windows ensuring utility to a broad spectrum of applications, from scientific computations to large scale statistical analyses. There are personal instances to support individuals migrating or developing their own workflows. The loosely-coupled nature of Condor resource collections make it straight forward to dynamically scale out on popular cloud computing fabrics such as [http://aws.amazon.com/ec2/ Amazon EC2].

Jpr@uab.edu: /* References */ Fix formatting of bullets and provide links to papers

2012-06-01T19:21:22Z

References: Fix formatting of bullets and provide links to papers

← Older revision		Revision as of 19:21, 1 June 2012
Line 83:		Line 83:
	Performance improvements to molecular docking workflows using Autodock are a common topic in the literature. The following papers address various approaches that have been taken to improve molecular docking workflow performance on a wide variety of hardware platforms, from BlueGene computers to Hadoop clusters. These papers used a reference docking set to compare performance, while our pilot preferred a data set representative of existing workflows on our cluster.		Performance improvements to molecular docking workflows using Autodock are a common topic in the literature. The following papers address various approaches that have been taken to improve molecular docking workflow performance on a wide variety of hardware platforms, from BlueGene computers to Hadoop clusters. These papers used a reference docking set to compare performance, while our pilot preferred a data set representative of existing workflows on our cluster.

	* Ellingson S. et. al. High-Throughput Virtual Molecular Docking: Hadoop Implementation of Autdock4 on a Private Cloud (2011)		* Ellingson S. et. al. [http://dl.acm.org/citation.cfm?id=1996028 High-Throughput Virtual Molecular Docking: Hadoop Implementation of Autdock4 on a Private Cloud] (2011)
	• Norgan A., et. al. Multilevel Parallelization of Autodock 4.2 (2011)		* Norgan A., et. al. [http://www.ncbi.nlm.nih.gov/pubmed/21527034 Multilevel Parallelization of Autodock 4.2] (2011)
	• Collingnon B., et. al. ~~Task-‐Parallel~~ Message Passing Interface Implementation of Autodock4 for Docking of Very Large Databases of Compounds Using High-Performance Super-Computers (2010)		* Collingnon B., et. al. [http://www.ncbi.nlm.nih.gov/pubmed/21387347 Task‐Parallel Message Passing Interface Implementation of Autodock4 for Docking of Very Large Databases of Compounds Using High-Performance Super-Computers] (2010)
	• Huang N., et. al. Benchmarking Sets for Molecular Docking (2006)		* Huang N., et. al. [http://www.ncbi.nlm.nih.gov/pubmed/17154509 Benchmarking Sets for Molecular Docking] (2006)

Jpr@uab.edu: Add references section and include Dr. Aller in participants

2012-06-01T19:17:18Z

Add references section and include Dr. Aller in participants

← Older revision		Revision as of 19:17, 1 June 2012
Line 43:		Line 43:
	=== Participants ===		=== Participants ===

	The UAB Condor Pilot was a collaboration between multiple organizations on campus. It was sparked by an inquiry from [http://www.uab.edu/nutrition/about/faculty-and-staff/primary-faculty/45-primary-faculty-/43-charles-w-prince Dr. Charles Prince], Assistant Vice President for Research, seeking to understand if idle desktops at UAB could contribute effectively to research computing demand. The resources to build the demonstration campus Condor pool were provided by the [http://www.cis.uab.edu Department of Computer and Information Sciences (CIS)] in the College of Arts and Sciences and by [http://www.uab.edu/it UAB IT Desktop Support Services]. Project management and the application development were provided by the [http://www.uab.edu/it/home/research-scholarship UAB IT Research Computing group].		The UAB Condor Pilot was a collaboration between multiple organizations on campus. It was sparked by an inquiry from [http://www.uab.edu/nutrition/about/faculty-and-staff/primary-faculty/45-primary-faculty-/43-charles-w-prince Dr. Charles Prince], Assistant Vice President for Research, seeking to understand if idle desktops at UAB could contribute effectively to research computing demand. The molecular docking workflow and data sets were provided by [http://medicine.uab.edu/pharmacology/about/70538/ Dr. Stephen Aller] of the Department of Pharmacology and Toxicology in UAB's School of Medicine. The resources to build the demonstration campus Condor pool were provided by the [http://www.cis.uab.edu Department of Computer and Information Sciences (CIS)] in the College of Arts and Sciences and by [http://www.uab.edu/it UAB IT Desktop Support Services]. Project management and the application development were provided by the [http://www.uab.edu/it/home/research-scholarship UAB IT Research Computing group].

	== Performance Comparison ==		== Performance Comparison ==
Line 78:		Line 78:

	The ability to move this same workflow, with very little adjustment, across a pilot Condor pool, production campus pool and national grid fabric demonstrates the advantage Condor has in portability. One of the key barriers limiting research adoption of new platforms (even more powerful ones) is the effort to port the workflow to the new resource environment. With Condor, local compute resources can be used to develop a workflow (including a simple personal deployment) and then this workflow can easily be move to larger and larger compute resources as demand grows.		The ability to move this same workflow, with very little adjustment, across a pilot Condor pool, production campus pool and national grid fabric demonstrates the advantage Condor has in portability. One of the key barriers limiting research adoption of new platforms (even more powerful ones) is the effort to port the workflow to the new resource environment. With Condor, local compute resources can be used to develop a workflow (including a simple personal deployment) and then this workflow can easily be move to larger and larger compute resources as demand grows.

			== References ==

			Performance improvements to molecular docking workflows using Autodock are a common topic in the literature. The following papers address various approaches that have been taken to improve molecular docking workflow performance on a wide variety of hardware platforms, from BlueGene computers to Hadoop clusters. These papers used a reference docking set to compare performance, while our pilot preferred a data set representative of existing workflows on our cluster.

			* Ellingson S. et. al. High-Throughput Virtual Molecular Docking: Hadoop Implementation of Autdock4 on a Private Cloud (2011)
			• Norgan A., et. al. Multilevel Parallelization of Autodock 4.2 (2011)
			• Collingnon B., et. al. Task-‐Parallel Message Passing Interface Implementation of Autodock4 for Docking of Very Large Databases of Compounds Using High-Performance Super-Computers (2010)
			• Huang N., et. al. Benchmarking Sets for Molecular Docking (2006)

Jpr@uab.edu: Reviewed entire article for spelling corections and wording improvements

2012-06-01T19:08:15Z

Reviewed entire article for spelling corections and wording improvements

Show changes

Jpr@uab.edu: /* Conclusions */ Enhanced conclusions to cover points in presentation

2012-06-01T15:18:34Z

Conclusions: Enhanced conclusions to cover points in presentation

← Older revision		Revision as of 15:18, 1 June 2012
Line 68:		Line 68:

	The primary conclusion discussed in the Performance Analysis above is that idle compute cycles harnessed by the Condor resource scheduler are capable of significant contribution to research computing needs. As such, UAB IT recommends the adoption and support of Condor computing resources at UAB.		The primary conclusion discussed in the Performance Analysis above is that idle compute cycles harnessed by the Condor resource scheduler are capable of significant contribution to research computing needs. As such, UAB IT recommends the adoption and support of Condor computing resources at UAB.

			The performance summary above also demonstrates a key characteristic of high-throughput computing jobs: they scale close to linearly with the addition of compute resources. Since each task in the queue is independent of other tasks the resource scheduler can easily assign tasks as resources become available. When combined with a flexible resource scheduler like Condor, which can harness many different compute platforms across domains, large compute pools can be assembled to address demanding workflow requirements.

			The static job configuration in this test could stand improvement. While it facilitated a quick conversion of the workflow from SGE to Condor, it did lead to imbalances in effort which can be seen as the long tails at the end of the workflow. These resulted from longer running jobs being scheduled late in the overall workflow. Allowing a granularity of 1-to-1 receptor-ligand pairs could improve the balance across compute resources. The trade-off with this approach is that some docking searches are very quick and would have more scheduling overhead than actual runtime. Additional frameworks that layer on top of Condor an address this by harnessing compute resources for longer cycles and repeatedly assigning tasks as they complete work.

			Comparing the utility of Linux and Windows resources, our pilot with molecular docking and AutoDock also showed that Linux resources are easier to use for this science domain than Windows workstations. This is largely a function of the application for different platform. Autodock is supported on Linux and Windows, but the supporting job scripts written in Bash require additional support on Windows. This was easily resolved with UAB IT Desktop Support resources by added the Cygwin framework to the Windows workstations. A test run with this configuration found the workflow adapted easily to Cygwin but that the 32bit architecture of the workstation slowed AutoDock perfomance. It is expected that a production deployment would find more current hardware and that 64bit Windows systems would be available.

			The ability to move this same workflow, with very little adjustment, across a pilot Condor pool, production campus deployment and national grid fabric demonstrates the advantage Condor has in portability. One of the key barriers that often limits research use of new platforms is the effort to port the workflow to the new scheduling environment. With Condor, local compute resources can be used to develop a workflow (including a simple personal deployment) and this workflow can then easily be move to larger and larger compute resources as demand grows.

Jpr@uab.edu: /* Conclusions */ Initial draft of primary conclusion

2012-06-01T14:11:55Z

Conclusions: Initial draft of primary conclusion

← Older revision		Revision as of 14:11, 1 June 2012
Line 66:		Line 66:

	== Conclusions ==		== Conclusions ==

			The primary conclusion discussed in the Performance Analysis above is that idle compute cycles harnessed by the Condor resource scheduler are capable of significant contribution to research computing needs. As such, UAB IT recommends the adoption and support of Condor computing resources at UAB.

Jpr@uab.edu: /* Performance Comparison */ Initial permance analysis

2012-06-01T13:23:07Z

Performance Comparison: Initial permance analysis

← Older revision		Revision as of 13:23, 1 June 2012
Line 60:		Line 60:

	While the overall runtime of Cheaha, the Wisconsin pool, and OSG are roughly similar, they clearly demonstrate that idle compute cycles harnessed by Condor can perform as well as, or even out perform, UAB's dedicated compute cluster for this class of compute jobs.		While the overall runtime of Cheaha, the Wisconsin pool, and OSG are roughly similar, they clearly demonstrate that idle compute cycles harnessed by Condor can perform as well as, or even out perform, UAB's dedicated compute cluster for this class of compute jobs.

			[[File:Combined_run_out_1.png\|600px]]

			The original data and methods to reproduce these experimental runs and more details analysis are published on the [http://projects.uabgrid.uab.edu/condor UAB Condor project site].

	== Conclusions ==		== Conclusions ==

Jpr@uab.edu: /* Performance Comparison */ first draft performance

2012-06-01T13:14:10Z

Performance Comparison: first draft performance

← Older revision		Revision as of 13:14, 1 June 2012
Line 44:		Line 44:

	== Performance Comparison ==		== Performance Comparison ==

			In order to assess the utility of idle compute cycles, the performance of the molecular docking workflow was compared on four compute fabrics: the UAB 888 core campus compute cluster in use for most large-scale computational workflows today, including this molecular docking workflow; the pilot UAB Condor pool of 40 Linux workstations; the University of Wisconsin's production Condor pool; and the Condor pool composed of OSG resources made available by the Engage VO.

			The molecular docking workflow analyzed 4 receptor proteins against a database of 5440 ligands. This data set was divided into a simple subset of jobs that each included 1 receptor and 5 ligands for a total of 1088 independent analysis jobs to distribute across the available compute resources of each of the 4 compute platforms above.

			The following graphic highlights a simple comparison of run-time performance of each fabric. The upper graph describes the number of jobs running at any one point in time on the target compute fabric. This graph illustrates the variable nature of available compute resources, especially for the dynamic harvesting of idle cycles on the Condor fabrics. The lower graph records the rate of completion of jobs and illustrates the linear relationship between the completion rate and the number of available resources.

			The UAB campus cluster Cheaha (blue) had approximately 300 cores available for the duration of this workflow on the day that it was run. This resulted in steady linear progress toward workflow completion. The curved slopes at the beginning and end, represent job submission rates and work flow drain times as the remaining jobs complete in different lengths of time. The overall runtime was approximately 3.5 hours.

			The UAB Condor pilot pool (red) had only 40 cores available for the duration of this workflow. While the overall run-time was significantly affected by this small resource pool, it should be noted that the progress remained linear, though at a much more gentle slope. This illustrates the adaptability of the workflow structure of serial jobs.

			The Wisconsin production Condor pool (green) had two periods of resources availability during the run. The first period had approximately 150 idle cores available for the first hour. This jumped to close to 400 cores during the second hour. The increase in the slope of the completion graph should be noted. As more idle resources became available, the molecular docking workflow performance increased. This is an important characteristic of serial, or pleasantly parallel workflows. There performance is very responsive to available resources. The workflow completed in approximately 3.75 hours on the Wisconsin pool during this time.

			The OSG workflow run had a variable number of resources available during the run, peaking at approximately 500 cores. Nonetheless, there was steady progress toward completion and in fact the workflow completed in the shortest runtime on OSG, approximately 3 hours.

			While the overall runtime of Cheaha, the Wisconsin pool, and OSG are roughly similar, they clearly demonstrate that idle compute cycles harnessed by Condor can perform as well as, or even out perform, UAB's dedicated compute cluster for this class of compute jobs.

	== Conclusions ==		== Conclusions ==