Welcome - Revision history

Test admin: Update url for new docs site and use consistent text for obsolete reference.

2022-08-31T20:11:03Z

Update url for new docs site and use consistent text for obsolete reference.

← Older revision		Revision as of 20:11, 31 August 2022
Line 1:		Line 1:
	The introduction to UAB Research Computing resources has been moved to https://~~uabrc~~.~~github~~.io		The introduction to UAB Research Computing resources has been moved to https://docs.rc.uab.edu .

	The obsolete content of ~~this~~ page can be found at [[Obsolete: Welcome]] for historical reference.		The obsolete content of the original page can be found at [[Obsolete: Welcome]] for historical reference.

Test admin: Move content for Welcome page to an obsolete page

2022-08-30T17:22:07Z

Move content for Welcome page to an obsolete page

Show changes

Test admin: Removing {{Main_Banner}}

2022-08-04T18:49:41Z

Removing {{Main_Banner}}

← Older revision		Revision as of 18:49, 4 August 2022
Line 1:		Line 1:
	~~{{Main_Banner}}~~
	Welcome to the '''Research Computing System'''		Welcome to the '''Research Computing System'''

Jpr@uab.edu: /* Personnel */ update system admins

2021-03-19T20:13:14Z

Personnel: update system admins

← Older revision		Revision as of 20:13, 19 March 2021
Line 92:		Line 92:
	==== Personnel ====		==== Personnel ====

	UAB IT Research Computing currently maintains a support staff of 10 lead by the Assistant Vice President for Research Computing and includes an HPC Architect-Manager, four Software developers, two Scientists, a system ~~administrator~~ and a project coordinator.		UAB IT Research Computing currently maintains a support staff of 10 lead by the Assistant Vice President for Research Computing and includes an HPC Architect-Manager, four Software developers, two Scientists, two system administrators and a project coordinator.

	=== Acknowledgment in Publications ===		=== Acknowledgment in Publications ===

	{{Grant_Ack}}		{{Grant_Ack}}

Jpr@uab.edu: /* Cloud Resources */ fix markup

2021-03-19T20:10:26Z

Cloud Resources: fix markup

← Older revision		Revision as of 20:10, 19 March 2021
Line 72:		Line 72:
	In addition UAB researchers also have access to regional and national HPC resources such as Alabama Supercomputer Authority (ASA), XSEDE and Open Science Grid (OSG).		In addition UAB researchers also have access to regional and national HPC resources such as Alabama Supercomputer Authority (ASA), XSEDE and Open Science Grid (OSG).

	==== Cloud Resources =====		==== Cloud Resources ====

	Research Computing has operated a development OpenStack cloud resource since 2019. This platform has been used to support application development and DevOps processes to research labs across campus. In 2021 a production implementation of this cloud platform will be made available to researchers on campus. This fabric is composed of five Dell R640 48 core 192G RAM compute nodes for 240 cores and 960GB of standard cloud compute resources. In addition the fabric will feature four NVIDIA DGX A100 nodes that include 8 A100 GPUs and 1TB of RAM each. All of these resources will be available to the research community for provisioning on demand via the OpenStack services (Ussuri release). The production implementation will further support researchers making their hosted services available beyond campus while adhering to standard campus network security practices. This off-campus access feature has not been available via the development cloud.		Research Computing has operated a development OpenStack cloud resource since 2019. This platform has been used to support application development and DevOps processes to research labs across campus. In 2021 a production implementation of this cloud platform will be made available to researchers on campus. This fabric is composed of five Dell R640 48 core 192G RAM compute nodes for 240 cores and 960GB of standard cloud compute resources. In addition the fabric will feature four NVIDIA DGX A100 nodes that include 8 A100 GPUs and 1TB of RAM each. All of these resources will be available to the research community for provisioning on demand via the OpenStack services (Ussuri release). The production implementation will further support researchers making their hosted services available beyond campus while adhering to standard campus network security practices. This off-campus access feature has not been available via the development cloud.

Jpr@uab.edu: /* Description of Cheaha for Grants (Detailed) */ Update resource descriptions for new compute, network and storage capacity. Add cloud section.

2021-03-19T20:09:44Z

Description of Cheaha for Grants (Detailed): Update resource descriptions for new compute, network and storage capacity. Add cloud section.

← Older revision		Revision as of 20:09, 19 March 2021
Line 64:		Line 64:
	==== Cheaha HPC system ====		==== Cheaha HPC system ====

	Cheaha is a campus HPC resource dedicated to enhancing research computing productivity at UAB. Cheaha is managed by UAB Information Technology's Research Computing group (RC) and is available to members of the UAB community in need of increased computational capacity. Cheaha supports high performance computing (HPC) and high throughput computing (HTC) paradigms. Cheaha is composed of resources that span data centers located in ~~the~~ UAB IT ~~Data Centers~~ in the 936 Building and the RUST Computer Center. Research Computing in open collaboration with the campus research community is leading the design and development of these resources.		Cheaha is a campus HPC resource dedicated to enhancing research computing productivity at UAB. Cheaha is managed by UAB Information Technology's Research Computing group (RC) and is available to members of the UAB community in need of increased computational capacity. Cheaha supports high performance computing (HPC) and high throughput computing (HTC) paradigms. Cheaha is composed of resources that span data centers located in two UAB campus IT data centers, in the 936 Building and the RUST Computer Center, and a commercial data center at DC BLOX in Birmingham. Research Computing, in open collaboration with the campus research community, is leading the design and development of these resources.

	==== Compute Resources ====		==== Compute Resources ====

	Cheaha provides users with both a web based interface and a traditional command-line interactive environment ~~with~~ access to many scientific tools that can leverage ~~its~~ dedicated pool of local compute resources. The local compute pool provides access to ~~three~~ generations of compute hardware based on the x86 64-bit architecture. It includes 96 nodes: 2x12 core (2304 cores total) 2.5 GHz Intel Xeon E5-2680 v3 compute nodes with FDR InfiniBand interconnect. ~~Out of~~ the 96 compute nodes, 36 nodes have 128 GB RAM, 38 nodes have 256 GB RAM, and 14 nodes have 384 GB RAM. There are also four compute nodes with the Intel Xeon Phi 7210 accelerator cards and four compute nodes with the NVIDIA K80 GPUs. ~~The second generation~~ is composed of 18 nodes: 2x14 core (504 cores total) 2.4GHz Intel Xeon E5-2680 v4 compute nodes with 256GB RAM, four NVIDIA Tesla P100 16GB GPUs per node, and EDR InfiniBand interconnect. ~~The third generation~~ is composed of 35 nodes with EDR ~~Infinband~~ interconnect: 2x12 core (840 cores total) 2.60GHz Intel Xeon Gold 6126 compute nodes with 21 compute nodes at 192GB RAM, 10 nodes at 768GB RAM and 4 nodes at 1.5TB of RAM. The compute nodes combine to provide over ~~528~~ TFLOP/s of dedicated computing power.		Cheaha provides users with both a web based interface, via open OnDemand, and a traditional command-line interactive environment, via SSH. These interfaces provide access to many scientific tools that can leverage a dedicated pool of local compute resources via the SLURM batch scheduler. The local compute pool provides access to five generations of compute hardware based on the x86 64-bit architecture. Gen6 (2015-2016) includes 96 nodes: 2x12 core (2304 cores total) 2.5 GHz Intel Xeon E5-2680 v3 compute nodes with an FDR InfiniBand interconnect. Of the 96 compute nodes, 36 nodes have 128 GB RAM, 38 nodes have 256 GB RAM, and 14 nodes have 384 GB RAM. There are also four compute nodes with the Intel Xeon Phi 7210 accelerator cards and four compute nodes with the NVIDIA K80 GPUs. Gen7 (2017) is composed of 18 nodes: 2x14 core (504 cores total) 2.4GHz Intel Xeon E5-2680 v4 compute nodes with 256GB RAM, four NVIDIA Tesla P100 16GB GPUs per node, and an EDR InfiniBand interconnect. Gen8 (2019) is composed of 35 nodes with EDR InfiniBand interconnect: 2x12 core (840 cores total) 2.60GHz Intel Xeon Gold 6126 compute nodes with 21 compute nodes at 192GB RAM, 10 nodes at 768GB RAM and 4 nodes at 1.5TB of RAM. Gen9 (available Q2 2021) is composed of 52 nodes with EDR InfiniBand interconnect: 2x24 core (2496 cores total) 3.0GHz Intel Xeon Gold 6248R compute nodes each with 192GB RAM. The compute nodes combine to provide over 600 TFLOP/s of dedicated computing power.

	In addition UAB researchers also have access to regional and national HPC resources such as Alabama Supercomputer Authority (ASA), XSEDE and Open Science Grid (OSG).		In addition UAB researchers also have access to regional and national HPC resources such as Alabama Supercomputer Authority (ASA), XSEDE and Open Science Grid (OSG).

			==== Cloud Resources =====

			Research Computing has operated a development OpenStack cloud resource since 2019. This platform has been used to support application development and DevOps processes to research labs across campus. In 2021 a production implementation of this cloud platform will be made available to researchers on campus. This fabric is composed of five Dell R640 48 core 192G RAM compute nodes for 240 cores and 960GB of standard cloud compute resources. In addition the fabric will feature four NVIDIA DGX A100 nodes that include 8 A100 GPUs and 1TB of RAM each. All of these resources will be available to the research community for provisioning on demand via the OpenStack services (Ussuri release). The production implementation will further support researchers making their hosted services available beyond campus while adhering to standard campus network security practices. This off-campus access feature has not been available via the development cloud.

	==== Storage Resources ====		==== Storage Resources ====

	The compute nodes on Cheaha are backed by high performance, 6.6PB raw ~~GPFS~~ storage on DDN SFA14KX hardware connected via an InfiniBand fabric. The non-scratch files on the GPFS cluster are replicated to 6.~~6PB~~ raw storage on a DDN SFA12KX located in ~~another building~~ to provide site redundancy. An additional ~~20TB~~ of traditional SAN storage is also available for home directories.		The compute nodes on Cheaha are backed by high performance, 6.6PB GPFS raw storage on DDN SFA14KX hardware connected via an EDR /FDR InfiniBand fabric. The non-scratch files on the GPFS cluster are replicated to 6.0PB raw storage on a DDN SFA12KX located in the RUST data center to provide site redundancy. An additional 10TB of traditional SAN storage is also available for home directories.

			Three new storage fabrics will come on line in 2021. All three storage fabrics are based on Ceph with different hardware configurations to address different usage scenarios. The fabrics are a 6.9PB archive storage fabric built using 12 Dell DSS7500 nodes, an expanded 1.3PB nearline storage fabric built with 14 Dell 740xd nodes, and a 248TB SSD cache storage fabric built with 8 Dell 840 nodes.

	==== Network Resources ====		==== Network Resources ====

	The UAB Research Network is currently a dedicated 40GE optical connection between the UAB Shared HPC Facility and the RUST Campus Data Center to create a multi-site facility housing the Research Computing System~~, which leverages~~ the network for connecting storage and compute hosting resources. The network supports direct connection to high-bandwidth regional networks ~~and~~ the capability to connect data intensive research facilities directly with the high performance computing services of the Research Computing System. This network can support very high speed secure connectivity between nodes connected to it for high speed file transfer of very large data sets without the concerns of interfering with other traffic on the campus backbone, ensuring predictable latencies. ~~In addition, the network also consist of a secure~~ Science DMZ with ~~data transfer nodes~~ (DTNs), Perfsonar measurement nodes, and a Bro security node connected directly to the border router that provide a "friction-free" pathway to access external data repositories as well as computational resources.		The UAB Research Network is currently a dedicated 40GE optical connection between the UAB Shared HPC Facility in 936 and the RUST Campus Data Center to create a multi-site facility housing the Research Computing System (RCS). This network is being upgraded in 2021 to replace aging equipment and extend service to the DC BLOX data center. The new network provides a 200Gbs Ethernet backbone for East-West traffic for connecting storage and compute hosting resources. The network supports direct connection to campus and high-bandwidth regional networks via 40Gbps Globus Data Transfer Nodes (DTNs) providing the capability to connect data intensive research facilities directly with the high performance computing and storage services of the Research Computing System. This network can support very high speed secure connectivity between nodes connected to it for high speed file transfer of very large data sets without the concerns of interfering with other traffic on the campus backbone, ensuring predictable latencies. The Science DMZ interface with (DTNs) includes Perfsonar measurement nodes and a Bro security node connected directly to the border router that provide a "friction-free" pathway to access external data repositories as well as computational resources.

	The campus network backbone is based on a 40 gigabit redundant Ethernet network with 480 gigabit/second back-planes on the core L2/L3 Switch/Routers. For efficient management, a collapsed backbone design is used. Each campus building is connected using 10 Gigabit Ethernet links over single mode optical fiber. Desktops are connected at 1 gigabits/second speed. The campus wireless network blankets classrooms, common areas and most academic office buildings.		The campus network backbone is based on a 40 gigabit redundant Ethernet network with 480 gigabit/second back-planes on the core L2/L3 Switch/Routers. For efficient management, a collapsed backbone design is used. Each campus building is connected using 10 Gigabit Ethernet links over single mode optical fiber. Desktops are connected at 1 gigabits/second speed. The campus wireless network blankets classrooms, common areas and most academic office buildings.

Mhanby@uab.edu at 17:47, 26 May 2020

2020-05-26T17:47:55Z

← Older revision		Revision as of 17:47, 26 May 2020
Line 56:		Line 56:
	=== Description of Cheaha for Grants (short)===		=== Description of Cheaha for Grants (short)===

	UAB IT Research Computing maintains high performance compute (HPC) and storage resources for investigators. The Cheaha compute cluster provides over 3744 conventional INTEL CPU cores and 80 accelerators (including 72 NVIDIA P100 GPUS's) interconnected via an EDR InfiniBand network and provides 528 TFLOP/s of aggregate theoretical peak performance. A high performance, 6.6PB raw GPFS storage on a DDN SFA14KX cluster with site replication to a DDN SFA12KX cluster, is also connected to the compute nodes via an ~~Infiniband~~ fabric. An additional 20TB of traditional SAN storage is also available for home directories. This general access compute fabric is available to all UAB investigators.		UAB IT Research Computing maintains high performance compute (HPC) and storage resources for investigators. The Cheaha compute cluster provides over 3744 conventional INTEL CPU cores and 80 accelerators (including 72 NVIDIA P100 GPUS's) interconnected via an EDR InfiniBand network and provides 528 TFLOP/s of aggregate theoretical peak performance. A high performance, 6.6PB raw GPFS storage on a DDN SFA14KX cluster with site replication to a DDN SFA12KX cluster, is also connected to the compute nodes via an InfiniBand fabric. An additional 20TB of traditional SAN storage is also available for home directories. This general access compute fabric is available to all UAB investigators.

	=== Description of Cheaha for Grants (Detailed) ===		=== Description of Cheaha for Grants (Detailed) ===
Line 64:		Line 64:
	==== Cheaha HPC system ====		==== Cheaha HPC system ====

	Cheaha is a campus HPC resource dedicated to enhancing research computing productivity at UAB. Cheaha is managed by UAB Information Technology's Research Computing group (RC) and is available to members of the UAB community in need of increased computational capacity. Cheaha supports high-performance computing (HPC) and high throughput computing (HTC) paradigms. Cheaha is composed of resources that span data centers located in the UAB IT Data Centers in the 936 Building and the RUST Computer Center. Research Computing in open collaboration with the campus research community is leading the design and development of these resources.		Cheaha is a campus HPC resource dedicated to enhancing research computing productivity at UAB. Cheaha is managed by UAB Information Technology's Research Computing group (RC) and is available to members of the UAB community in need of increased computational capacity. Cheaha supports high performance computing (HPC) and high throughput computing (HTC) paradigms. Cheaha is composed of resources that span data centers located in the UAB IT Data Centers in the 936 Building and the RUST Computer Center. Research Computing in open collaboration with the campus research community is leading the design and development of these resources.

	==== Compute Resources ====		==== Compute Resources ====
Line 74:		Line 74:
	==== Storage Resources ====		==== Storage Resources ====

	The compute nodes on Cheaha are backed by high-performance, 6.6PB raw GPFS storage on DDN SFA14KX hardware connected via an ~~Infiniband~~ fabric. The non-scratch files on the GPFS cluster are replicated to 6.6PB raw storage on a DDN SFA12KX located in another building to provide site redundancy. An additional 20TB of traditional SAN storage is also available for home directories.		The compute nodes on Cheaha are backed by high performance, 6.6PB raw GPFS storage on DDN SFA14KX hardware connected via an InfiniBand fabric. The non-scratch files on the GPFS cluster are replicated to 6.6PB raw storage on a DDN SFA12KX located in another building to provide site redundancy. An additional 20TB of traditional SAN storage is also available for home directories.

	==== Network Resources ====		==== Network Resources ====

Mhanby@uab.edu: /* Description of Cheaha for Grants (short) */

2020-05-26T17:44:45Z

Description of Cheaha for Grants (short)

← Older revision		Revision as of 17:44, 26 May 2020
Line 56:		Line 56:
	=== Description of Cheaha for Grants (short)===		=== Description of Cheaha for Grants (short)===

	UAB IT Research Computing maintains high performance compute and storage resources for investigators. The Cheaha compute cluster provides over ~~2900~~ conventional INTEL CPU cores and 80 accelerators (including 72 NVIDIA P100 GPUS's) interconnected via an EDR InfiniBand network and provides ~~468~~ TFLOP/s of aggregate theoretical peak performance. A high-performance, 6.6PB raw GPFS storage on DDN SFA12KX ~~hardware~~ is also connected to ~~these~~ compute nodes via ~~the~~ Infiniband fabric. An additional 20TB of traditional SAN storage is also available for home directories. This general access compute fabric is available to all UAB investigators.		UAB IT Research Computing maintains high performance compute (HPC) and storage resources for investigators. The Cheaha compute cluster provides over 3744 conventional INTEL CPU cores and 80 accelerators (including 72 NVIDIA P100 GPUS's) interconnected via an EDR InfiniBand network and provides 528 TFLOP/s of aggregate theoretical peak performance. A high performance, 6.6PB raw GPFS storage on a DDN SFA14KX cluster with site replication to a DDN SFA12KX cluster, is also connected to the compute nodes via an Infiniband fabric. An additional 20TB of traditional SAN storage is also available for home directories. This general access compute fabric is available to all UAB investigators.

	=== Description of Cheaha for Grants (Detailed) ===		=== Description of Cheaha for Grants (Detailed) ===

Mhanby@uab.edu: /* Compute Resources */

2020-05-26T17:40:55Z

Compute Resources

← Older revision		Revision as of 17:40, 26 May 2020
Line 68:		Line 68:
	==== Compute Resources ====		==== Compute Resources ====

	Cheaha provides users with both a web based interface and a traditional command-line interactive environment with access to many scientific tools that can leverage its dedicated pool of local compute resources. The local compute pool provides access to three generations of compute hardware based on the x86 64-bit architecture. It includes 96 nodes: 2x12 core (2304 cores total) 2.5 GHz Intel Xeon E5-2680 v3 compute nodes with FDR InfiniBand interconnect. Out of the 96 compute nodes, 36 nodes have 128 GB RAM, 38 nodes have 256 GB RAM, and 14 nodes have 384 GB RAM. There are also four compute nodes with the Intel Xeon Phi 7210 accelerator cards and four compute nodes with the NVIDIA K80 GPUs. The second generation is composed of 18 nodes: 2x14 core (504 cores total) 2.4GHz Intel Xeon E5-2680 v4 compute nodes with 256GB RAM, four NVIDIA Tesla P100 16GB GPUs per node, and EDR InfiniBand interconnect. The third generation is composed of 35 nodes: 2x12 core (840 cores total) 2.60GHz Intel Xeon Gold 6126 compute nodes with 21 compute nodes at 192GB RAM, 10 nodes at 768GB RAM and 4 nodes at 1.5TB of RAM. The compute nodes combine to provide over 528 TFLOP/s of dedicated computing power.		Cheaha provides users with both a web based interface and a traditional command-line interactive environment with access to many scientific tools that can leverage its dedicated pool of local compute resources. The local compute pool provides access to three generations of compute hardware based on the x86 64-bit architecture. It includes 96 nodes: 2x12 core (2304 cores total) 2.5 GHz Intel Xeon E5-2680 v3 compute nodes with FDR InfiniBand interconnect. Out of the 96 compute nodes, 36 nodes have 128 GB RAM, 38 nodes have 256 GB RAM, and 14 nodes have 384 GB RAM. There are also four compute nodes with the Intel Xeon Phi 7210 accelerator cards and four compute nodes with the NVIDIA K80 GPUs. The second generation is composed of 18 nodes: 2x14 core (504 cores total) 2.4GHz Intel Xeon E5-2680 v4 compute nodes with 256GB RAM, four NVIDIA Tesla P100 16GB GPUs per node, and EDR InfiniBand interconnect. The third generation is composed of 35 nodes with EDR Infinband interconnect: 2x12 core (840 cores total) 2.60GHz Intel Xeon Gold 6126 compute nodes with 21 compute nodes at 192GB RAM, 10 nodes at 768GB RAM and 4 nodes at 1.5TB of RAM. The compute nodes combine to provide over 528 TFLOP/s of dedicated computing power.

	In addition UAB researchers also have access to regional and national HPC resources such as Alabama Supercomputer Authority (ASA), XSEDE and Open Science Grid (OSG).		In addition UAB researchers also have access to regional and national HPC resources such as Alabama Supercomputer Authority (ASA), XSEDE and Open Science Grid (OSG).

Mhanby@uab.edu: /* Storage Resources */

2020-05-26T17:40:00Z

Storage Resources

← Older revision		Revision as of 17:40, 26 May 2020
Line 74:		Line 74:
	==== Storage Resources ====		==== Storage Resources ====

	The compute nodes on Cheaha are backed by high-performance, 6.6PB raw GPFS storage on DDN ~~SFA12KX~~ hardware connected via ~~the~~ Infiniband fabric. ~~An expansion of~~ the GPFS ~~fabric will double the capacity and is scheduled~~ to be on~~-line Fall 2018~~. An additional 20TB of traditional SAN storage is also available for home directories.		The compute nodes on Cheaha are backed by high-performance, 6.6PB raw GPFS storage on DDN SFA14KX hardware connected via an Infiniband fabric. The non-scratch files on the GPFS cluster are replicated to 6.6PB raw storage on a DDN SFA12KX located in another building to provide site redundancy. An additional 20TB of traditional SAN storage is also available for home directories.

	==== Network Resources ====		==== Network Resources ====