Cheaha - User contributions [en]

Jupyter

2020-08-28T19:03:02Z

Ozborn@uab.edu: More information on custom building conda environments on cheahea

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [https://rc.uab.edu On Demand]. To access.

== 1. Click [https://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =

See [https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf Conda Cheet Sheet] for some helpful Conda documentation. Additional info (some specific to cheaha) is below.

== Preparing to build a conda env on cheaha==
Do NOT build your new conda env on the head node, use an interactive job to create the new environment by using the command below.
<pre>
srun --ntasks=1 --cpus-per-task=2 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
</pre>

== Set up your cheaha .condarc file (Cheahea) ==
This tells conda where to look for the installed environments. This file should be in your home directory on Cheaha (/data/user/username/.condarc). Make sure to include that '.' in the beginning of the filename.
<pre>
channels:
- defaults
envs_dirs:
- /data/user/username/python/env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large.
The file will be hidden and not visible in Jupyter environment. To view or edit it, start terminal and run 'nano ~/.condarc' or 'vi ~/.condarc'.

== Build your conda environment ==
Once you are off the root node, you can start building your conda environment.
conda create --name mynlp python=3.7

To update an existing environment:
<pre>
conda env update --prefix ./scibert --file scibert.yml --prune
</pre>

== Add ipykernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks. Insert these lines in your .yml file. You can edit the file in Jupyter environment.
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>

== Add other packages of interest ==
Add additional package of interest to your conda environment, for example "torch", "transformers", etc.. Pip can be used as well.
<pre>
conda install torch
</pre>

You can see your conda environment packages using
<pre>
conda list
</pre>

== (Optional) Export YAML file containing your environment to cheaha ==
If 8you already have a functional conda environment elsewhere (like your laptop) you can export it to cheaha. The command below exports the "scibert" environment built on another machine. You can create any filename, but keep the extension .yml
<pre>
conda env export --no-builds > scibert.yml
</pre>
Copy the file to Cheaha. You can use "Upload" button in Jupyter

To replicate the new conda environment on cheaha execute the following commands:
<pre>
conda env create --file scibert.yml --prefix /data/user/username/python/env/scibert
</pre>
If you use --name argument instead to specify the environment name, conda will create a subdirectory with same name as the yml filename. That is confusing, so using --prefix makes the environment path cleaner

== Load environment on Jupyter Notebook ==
[[File:JupyterCustomEnv.jpeg|pics500x]]

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

UAB Openstack Tutorial

2020-04-21T16:41:51Z

Ozborn@uab.edu: /* Create Application Credentials */

= UAB Openstack Tutorial =

This tutorial is meant as a guide for deploying VMs in UAB's OpenStack environment.

== Overview ==
1. Send email to support@listserv.uab.edu and request an account

2. Log into ruffner.rc.uab.edu to get application credentials

3. Use terraform to create and deploy a VM to UAB OpenStack

4. Provision your VM with ansible

== Steps ==

=== Create Application Credentials ===
See https://gitlab.rc.uab.edu/rrand11/terraform-openstack

1. In Openstack, go to '''Identity''' -> '''Application Credentials'''

2. Click '''Create Application Credential'''

3. Name the credential, add a description, and check the box making it unrestricted (Leave the rest blank. It is important not to add an expiration date.)

4. Download the credentials as an RC file.

5. Save credentials RC file in your terraform-first-instance directory (echo $TERRAFORM_DIR).

=== Using Terraform to Create and Deploy VM ===
1. May be easiest to do from a cheaha node?

2. Download Terraform https://www.terraform.io/downloads.html

3. See https://gitlab.rc.uab.edu/jelaiw/ccts-bmi-incubator for some templates. May want to git clone.

4. See /Users/ozborn/code/repo/ccts-bmi-incubator/openstack/mice.tf for an example, shown below:

# Test web server for CCTS Informatics
resource "openstack_compute_instance_v2" "mice" {
name = "mice"
image_name = "CentOS-7-x86_64-GenericCloud-1905"
flavor_name = "m1.medium"
key_pair = var.admin_key_pair
security_groups = ["default", "web"]
# Work around race condition.
# See https://github.com/terraform-providers/terraform-provider-openstack/issues/775.
network {
uuid = openstack_networking_subnet_v2.foo_subnet.network_id
}
}

resource "openstack_compute_floatingip_associate_v2" "mice_fip" {
floating_ip = var.mice_floating_ip
instance_id = openstack_compute_instance_v2.mice.id
}

# See https://www.terraform.io/docs/providers/openstack/r/networking_secgroup_rule_v2.html.
resource "openstack_networking_secgroup_v2" "web" {
name = "web"
description = "A security group for managing rules and access to a standard HTTP web server."
}

resource "openstack_networking_secgroup_rule_v2" "web_std_port" {
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 80
port_range_max = 80
remote_ip_prefix = "0.0.0.0/0"
security_group_id = openstack_networking_secgroup_v2.web.id
}

=== Using Ansible to Provision VM ===
1. Take a look at /Users/ozborn/code/repo/ccts-bmi-incubator/openstack/setup-httpd.yml for Apache web server provisioning

2. ansible-configure after doing ansible-lint to verify setup

== Current Bugs Feb 2020 (per JPR) ==
There are two bugs that we are working on in the web ui:

1. launching instances fails unless "Create new volume" is set to "no" on the source tab

2. attaching volumes to instances fails in both the ui and cli.

== Networking Setup (per JPR) ==
The assigned IP addresses (floating public) will be in the range 192.168.16.128-250. These should be mapped to 164.111.161.x where x repeats the last octet of the assigned 192 number. The networking is set up to allow ingres from on-campus only. The instances can go out to anywhere though.

== References (per JPR) ==
We don't have official getting started docs for now We pretty much just follows the Bright Cluster Manager docs for testing the openstack API from the cli. (Section 1.4 of https://support.brightcomputing.com/manuals/8.2/openstack-deployment-manual.pdf).

Louis has created some notes in a readme that may be useful as well:

https://gitlab.rc.uab.edu/louistw/cluster-installation-note/blob/master/openstack.md

UAB Openstack Tutorial

2020-02-28T17:29:54Z

Ozborn@uab.edu: Add info on getting creds

= UAB Openstack Tutorial =

This tutorial is meant as a guide for deploying VMs in UAB's OpenStack environment.

== Overview ==
1. Send email to support@listserv.uab.edu and request an account

2. Log into ruffner.rc.uab.edu to get application credentials

3. Use terraform to create and deploy a VM to UAB OpenStack

4. Provision your VM with ansible

== Steps ==

=== Create Application Credentials ===
See https://gitlab.rc.uab.edu/rrand11/terraform-openstack

1. In Openstack, go to '''Identity''' -> '''Application Credentials'''

2. Click '''Create Application Credential'''

3. Name the credential, add a description, and check the box making it unrestricted (Leave the rest blank. It is important not to add an expiration date.)

4. Download the credentials as an RC file.

5. Save credentials RC file in your terraform-first-instance directory.

=== Using Terraform to Create and Deploy VM ===
1. May be easiest to do from a cheaha node?

2. Download Terraform https://www.terraform.io/downloads.html

3. See https://gitlab.rc.uab.edu/jelaiw/ccts-bmi-incubator for some templates. May want to git clone.

4. See /Users/ozborn/code/repo/ccts-bmi-incubator/openstack/mice.tf for an example, shown below:

# Test web server for CCTS Informatics
resource "openstack_compute_instance_v2" "mice" {
name = "mice"
image_name = "CentOS-7-x86_64-GenericCloud-1905"
flavor_name = "m1.medium"
key_pair = var.admin_key_pair
security_groups = ["default", "web"]
# Work around race condition.
# See https://github.com/terraform-providers/terraform-provider-openstack/issues/775.
network {
uuid = openstack_networking_subnet_v2.foo_subnet.network_id
}
}

resource "openstack_compute_floatingip_associate_v2" "mice_fip" {
floating_ip = var.mice_floating_ip
instance_id = openstack_compute_instance_v2.mice.id
}

# See https://www.terraform.io/docs/providers/openstack/r/networking_secgroup_rule_v2.html.
resource "openstack_networking_secgroup_v2" "web" {
name = "web"
description = "A security group for managing rules and access to a standard HTTP web server."
}

resource "openstack_networking_secgroup_rule_v2" "web_std_port" {
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 80
port_range_max = 80
remote_ip_prefix = "0.0.0.0/0"
security_group_id = openstack_networking_secgroup_v2.web.id
}

=== Using Ansible to Provision VM ===
1. Take a look at /Users/ozborn/code/repo/ccts-bmi-incubator/openstack/setup-httpd.yml for Apache web server provisioning

2. ansible-configure after doing ansible-lint to verify setup

== Current Bugs Feb 2020 (per JPR) ==
There are two bugs that we are working on in the web ui:

1. launching instances fails unless "Create new volume" is set to "no" on the source tab

2. attaching volumes to instances fails in both the ui and cli.

== Networking Setup (per JPR) ==
The assigned IP addresses (floating public) will be in the range 192.168.16.128-250. These should be mapped to 164.111.161.x where x repeats the last octet of the assigned 192 number. The networking is set up to allow ingres from on-campus only. The instances can go out to anywhere though.

== References (per JPR) ==
We don't have official getting started docs for now We pretty much just follows the Bright Cluster Manager docs for testing the openstack API from the cli. (Section 1.4 of https://support.brightcomputing.com/manuals/8.2/openstack-deployment-manual.pdf).

Louis has created some notes in a readme that may be useful as well:

https://gitlab.rc.uab.edu/louistw/cluster-installation-note/blob/master/openstack.md

UAB Openstack Tutorial

2020-02-28T17:17:28Z

Ozborn@uab.edu: Initial Page Creation

= UAB Openstack Tutorial =

This tutorial is meant as a guide for deploying VMs in UAB's OpenStack environment.

== Overview ==
1. Send email to support@listserv.uab.edu and request an account

2. Log into ruffner.rc.uab.edu to get application credentials

3. Use terraform to create and deploy a VM to UAB OpenStack

4. Provision your VM with ansible

== Using Terraform to Create and Deploy VM ==
1. May be easiest to do from a cheaha node
2. Download Terraform https://www.terraform.io/downloads.html
3. See https://gitlab.rc.uab.edu/jelaiw/ccts-bmi-incubator for some templates. May want to git clone.
4. See /Users/ozborn/code/repo/ccts-bmi-incubator/openstack/mice.tf for an example

# Test web server for CCTS Informatics
resource "openstack_compute_instance_v2" "mice" {
name = "mice"
image_name = "CentOS-7-x86_64-GenericCloud-1905"
flavor_name = "m1.medium"
key_pair = var.admin_key_pair
security_groups = ["default", "web"]
# Work around race condition.
# See https://github.com/terraform-providers/terraform-provider-openstack/issues/775.
network {
uuid = openstack_networking_subnet_v2.foo_subnet.network_id
}
}

resource "openstack_compute_floatingip_associate_v2" "mice_fip" {
floating_ip = var.mice_floating_ip
instance_id = openstack_compute_instance_v2.mice.id
}

# See https://www.terraform.io/docs/providers/openstack/r/networking_secgroup_rule_v2.html.
resource "openstack_networking_secgroup_v2" "web" {
name = "web"
description = "A security group for managing rules and access to a standard HTTP web server."
}

resource "openstack_networking_secgroup_rule_v2" "web_std_port" {
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 80
port_range_max = 80
remote_ip_prefix = "0.0.0.0/0"
security_group_id = openstack_networking_secgroup_v2.web.id
}

== Using Ansible to Provision VM ==
1. Take a look at /Users/ozborn/code/repo/ccts-bmi-incubator/openstack/setup-httpd.yml for Apache web server provisioning

2. ansible-configure after doing ansible-lint to verify setup

== Current Bugs Feb 2020 (per JPR) ==
There are two bugs that we are working on in the web ui:

1. launching instances fails unless "Create new volume" is set to "no" on the source tab

2. attaching volumes to instances fails in both the ui and cli.

== Networking Setup (per JPR) ==
The assigned IP addresses (floating public) will be in the range 192.168.16.128-250. These should be mapped to 164.111.161.x where x repeats the last octet of the assigned 192 number. The networking is set up to allow ingres from on-campus only. The instances can go out to anywhere though.

== References (per JPR) ==
We don't have official getting started docs for now We pretty much just follows the Bright Cluster Manager docs for testing the openstack API from the cli. (Section 1.4 of https://support.brightcomputing.com/manuals/8.2/openstack-deployment-manual.pdf).

Louis has created some notes in a readme that may be useful as well:

https://gitlab.rc.uab.edu/louistw/cluster-installation-note/blob/master/openstack.md

Jupyter

2020-02-08T01:19:40Z

Ozborn@uab.edu: /* Build your conda environment */ How to update

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [http://rc.uab.edu On Demand]. To access.

== 1. Click [http://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =
== Export YAML file containing your environment to cheaha ==
Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server. You can create any filename, but keep the extension .yml
<pre>
conda env export --no-builds > scibert.yml
</pre>

== Set up your cheaha .condarc file ==
This file should be in your home directory on cheaha.
<pre>
channels:
- defaults
envs_dirs:
- /data/user/ozborn/Conda_Env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large

== Add ipykernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>

== Build your conda environment ==
Do NOT run it on the head node, use an interactive job to create the new environment.
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
</pre>

To build a new environment:
<pre>
conda env create --file scibert.yml --name scibert
</pre>

TO update an existing environment:
<pre>
conda env update --prefix ./scibert --file scibert.yml --prune
</pre>

== Load environment on Jupyter Notebook ==
[[File:JupyterCustomEnv.jpeg|pics500x]]

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

Jupyter

2020-02-07T02:33:32Z

Ozborn@uab.edu: --no-builds option

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [http://rc.uab.edu On Demand]. To access.

== 1. Click [http://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =
== Export YAML file containing your environment to cheaha ==
Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server. You can create any filename, but keep the extension .yml
<pre>
conda env export --no-builds > scibert.yml
</pre>

== Set up your cheaha .condarc file ==
This file should be in your home directory on cheaha.
<pre>
channels:
- defaults
envs_dirs:
- /data/user/ozborn/Conda_Env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large

== Add ipykernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>

== Build your conda environment ==
Do NOT run it on the head node, use an interactive job to create the new environment.
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
</pre>

<pre>
conda env create --file scibert.yml --name scibert
</pre>

== Load environment on Jupyter Notebook ==
[[File:JupyterCustomEnv.jpeg|pics500x]]

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

Jupyter

2020-01-15T03:26:45Z

Ozborn@uab.edu: /* Build your conda environment */

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [http://rc.uab.edu On Demand]. To access.

== 1. Click [http://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =
== Export YAML file containing your environment to cheaha ==
Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server.
<pre>
conda env export > scibert.yml
</pre>

== Set up your cheaha .condarc file ==
This file should be in your home directory on cheaha.
<pre>
channels:
- defaults
envs_dirs:
- /data/user/ozborn/Conda_Env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large

== Add ipykernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>

== Build your conda environment ==
Do NOT run it on the head node, use an interactive job to create the new environment.
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
</pre>

<pre>
conda env create --file scibert.yml --name scibert
</pre>

== Load environment on Jupyter Notebook ==
[[File:JupyterCustomEnv.jpeg|pics500x]]

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

Jupyter

2020-01-15T03:25:04Z

Ozborn@uab.edu: /* Build your conda environment */

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [http://rc.uab.edu On Demand]. To access.

== 1. Click [http://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =
== Export YAML file containing your environment to cheaha ==
Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server.
<pre>
conda env export > scibert.yml
</pre>

== Set up your cheaha .condarc file ==
This file should be in your home directory on cheaha.
<pre>
channels:
- defaults
envs_dirs:
- /data/user/ozborn/Conda_Env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large

== Add ipykernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>

== Build your conda environment ==
Do NOT run it on the head node, use an interactive job to create the new environment.
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
</pre>

== Load environment on Jupyter Notebook ==
[[File:JupyterCustomEnv.jpeg|pics500x]]

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

File:JupyterCustomEnv.jpeg

2020-01-15T03:22:35Z

Ozborn@uab.edu: File showing what a custom conda environment looks like in Jupyter On Demand

File showing what a custom conda environment looks like in Jupyter On Demand

Jupyter

2020-01-15T03:21:49Z

Ozborn@uab.edu: /* Set up your cheaha .condarc file */

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [http://rc.uab.edu On Demand]. To access.

== 1. Click [http://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =
== Export YAML file containing your environment to cheaha ==
Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server.
<pre>
conda env export > scibert.yml
</pre>

== Set up your cheaha .condarc file ==
This file should be in your home directory on cheaha.
<pre>
channels:
- defaults
envs_dirs:
- /data/user/ozborn/Conda_Env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large

== Add ipykernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>

== Build your conda environment ==
Do NOT run it on the head node, use an interactive job to create the new environment.
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
</pre>

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

Jupyter

2020-01-15T03:21:10Z

Ozborn@uab.edu: /* Add ipkernel to your conda environment */

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [http://rc.uab.edu On Demand]. To access.

== 1. Click [http://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =
== Export YAML file containing your environment to cheaha ==
Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server.
<pre>
conda env export > scibert.yml
</pre>

== Set up your cheaha .condarc file ==
<pre>
channels:
- defaults
envs_dirs:
- /data/user/ozborn/Conda_Env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large
== Add ipykernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>

== Build your conda environment ==
Do NOT run it on the head node, use an interactive job to create the new environment.
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
</pre>

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

Jupyter

2020-01-15T03:20:47Z

Ozborn@uab.edu: /* Export YAML file containing your environment to cheaha */

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [http://rc.uab.edu On Demand]. To access.

== 1. Click [http://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =
== Export YAML file containing your environment to cheaha ==
Wherever your working environment is, export it to cheaha. Below exports the scibert environment which I set up on a different server.
<pre>
conda env export > scibert.yml
</pre>

== Set up your cheaha .condarc file ==
<pre>
channels:
- defaults
envs_dirs:
- /data/user/ozborn/Conda_Env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large
== Add ipkernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>
== Build your conda environment ==
Do NOT run it on the head node, use an interactive job to create the new environment.
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
</pre>

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

Jupyter

2020-01-15T03:20:27Z

Ozborn@uab.edu: /* Export YAML file containing your environment to cheaha */

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [http://rc.uab.edu On Demand]. To access.

== 1. Click [http://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =
== Export YAML file containing your environment to cheaha ==
Wherever your working environment is, export it to cheaha. Below exports the scibert environment.
<pre>
conda env export > scibert.yml
</pre>

== Set up your cheaha .condarc file ==
<pre>
channels:
- defaults
envs_dirs:
- /data/user/ozborn/Conda_Env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large
== Add ipkernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>
== Build your conda environment ==
Do NOT run it on the head node, use an interactive job to create the new environment.
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
</pre>

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

Jupyter

2020-01-15T03:19:02Z

Ozborn@uab.edu: /* Adding Custom Conda Environments to Jupyter */

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [http://rc.uab.edu On Demand]. To access.

== 1. Click [http://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =
== Export YAML file containing your environment to cheaha ==
Wherever your working environment is, export it to cheaha.
== Set up your cheaha .condarc file ==
<pre>
channels:
- defaults
envs_dirs:
- /data/user/ozborn/Conda_Env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large
== Add ipkernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>
== Build your conda environment ==
Do NOT run it on the head node, use an interactive job to create the new environment.
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
</pre>

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

Jupyter

2020-01-15T03:17:20Z

Ozborn@uab.edu: Added Custom Jupyter Conda Environments

[http://jupyter.org/ Jupyter Notebook] is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. For more information on jupyter notebook, click [http://jupyter.org/documentation here].

= Jupyter On Demand =

As of 2019, UAB Research Computing allows access to cheaha via [http://rc.uab.edu On Demand]. To access.

== 1. Click [http://rc.uab.edu On Demand] ==

== 2. Select Interactive App and pick Jupyter Notebook ==

[[File:JupyterNotebookStart.png|500px]]

== 3. Load in Anaconda ==
'''
module load Anaconda3/5.3.1
'''
The following should also work for an updated version of Anaconda.
'''
module load Anaconda3
'''

== 4. If you require running on a '''''GPU''''', please add the following to your environment. ==

'''
module load cuda92/toolkit/9.2.88
module load CUDA/9.2.88-GCC-7.3.0-2.30
'''

Additionally, you will need to request a GPU as shown below by
including the pascalenodes argument:

[[File:PascalNodes.png|500px]]

== 5. Click Launch ==

[[File:LaunchJupyter.png|500px]]

Wait until you receive an email or get a blue Launch button. This can happen in about 10-20 seconds or may take much longer depending on the resources (CPU count and memory requested).

== 6. Connect to Jupyter Notebook ==

== 7. Test Pytorch with a new notebook ==

[[File:TestPytorch.png|500px]]

= Adding Custom Conda Environments to Jupyter =
== Export YAML file containing your environment to cheaha ==
Wherever your working environment is, export it to cheaha.
== Set up your cheaha .condarc file ==
<pre>
channels:
- defaults
envs_dirs:
- /data/user/ozborn/Conda_Env
</pre>
Use the data directory rather than the home directory for your conda environment as it can get quite large
== Add ipkernel to your conda environment ==
Add this conda module is what gets your environment to show up in Jupyter notebooks
<pre>
name: scibert
channels:
- defaults
dependencies:
- ipykernel=5.1.2
- _libgcc_mutex=0.1=main
- alabaster=0.7.12=py37_0
</pre>

= Jupyter by Proxy =
'''''(not longer required as of August 2019, use OnDemand option instead and this only as a fallback option)'''''

The cheaha cluster supports Jupyter notebooks for data analysis, but such jobs should be running using the SLURM job submission system to avoid overloading the head node. To run a Jupyter Notebook on cheaha, login to cheaha from your client machine and start an [https://docs.uabgrid.uab.edu/wiki/Slurm#Interactive_Job interactive job].

One important note is that cheaha only supports openssh, you should be able to use native ssh from Mac or Linux machines. Windows 10 supports openssh as well, but it is not enabled by default. On updated Windows 10 machines, '''a Developers Command Prompt''' (available via searching from the Start Menu) is able to run openssh via the ssh command similar to Mac and Linux users. Another option for Windows machines is the installation of Cygwin. Putty has been [[Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems|tested]], but does not work reliably on cheaha for proxying connections.

The Jupyter notebooks is built with [[Anaconda]],a free and open source distribution of python and R for scientific computing. If you need additional packages, you can create your own [[Python_Virtual_Environment]] just for that purpose.

== 1. Start the Jupyter Notebook ==
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=4096 --time=08:00:00 --partition=medium --job-name=JOB_NAME --pty /bin/bash
module load Anaconda3/5.2.0
unset XDG_RUNTIME_DIR
jupyter notebook --no-browser --ip=$host
</pre>
A headless Jupyter notebook should now be running on a compute node. The next step is to proxy this connection to your local machine.

== 2. Proxy Connection Locally ==
Now, start up a '''new''' tab/terminal/window on your client machine and relogin to cheaha, using
<pre>
ssh -L 88XX:c00XX:88XX BLAZERID@cheaha.rc.uab.edu
</pre>
'''Note:'''
* '''c00XX''' is the compute node where you started the jupyter notebook, for example c0047
* '''88XX''' is the port that the notebook is running, for example 8888
* For windows users, you can find instructions for port forwarding, [https://docs.uabgrid.uab.edu/wiki/Setting_Up_VNC_Session#Port-forwarding_from_Windows_Systems here]

== 3. Copy notebook URL ==
After running the jupyter notebook command the server should start running in headless mode and provide you with a URL including a port # (typically but not always 8888) and a compute node on cheaha (for example C0047) that looks something like this:
<pre>
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://c0047:8888/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
Copy the URL shown below into you clipboard/buffer for pasting into the browser as shown in step 4).

== 4. Access Notebook through Local Browser via Proxy Connection ==
Now access the link on your client machine browser locally using the link generated by jupyter notebook by '''substituting in localhost instead of c00XX'''. Make sure you have the correct port as well.
<pre>
http://localhost:88XX/?token=73da89e0eabdeb9d6dc1241a55754634d4e169357f60626c&token=73da89e0eabdeb7d6dc1241a55754634d4e169357f60626c
</pre>
A Jupyter notebook should then open in your browser connected to the compute node.

== Jupyter Options ==

=== DeepNLP option (development in progress) ===
For the use of additional libraries (pytorch, spacy) related to Deep Learning and/or NLP after loading Anaconda3/5.2.0 run:
<pre>
conda activate /share/apps/rc/software/Anaconda3/5.2.0/envs/DeepNLP
</pre>

=== Heavy Data IO option ===
Additionally, if anticipating large IO data transfer adjust the run command to set a higher data rate limit as shown below:
<pre>
jupyter notebook --no-browser --ip=$host --NotebookApp.iopub_data_rate_limit=1.0e10
</pre>

=== Memory Heavy option ===
<pre>
srun --ntasks=1 --cpus-per-task=4 --mem-per-cpu=16384 --time=08:00:00 --partition=medium --job-name=POSTag --pty /bin/bash
</pre>

=== GPU Option ===
Finally, if your job requires a GPU then add the [https://docs.uabgrid.uab.edu/wiki/Slurm#Requesting_for_GPUs gres and partition arguments] as shown below:
<pre>
srun --ntasks=1 --cpus-per-task=1 --mem-per-cpu=4096 --time=08:00:00 --partition=pascalnodes --job-name=JOB_NAME --gres=gpu:1 --pty /bin/bash
</pre>

Jupyter

2019-10-28T02:08:41Z

Ozborn@uab.edu: /* Jupyter On Demand */

File:LaunchJupyter.png

2019-10-28T02:07:11Z

Ozborn@uab.edu: Button to press.

Button to press.

Jupyter

2019-10-28T02:02:24Z

Ozborn@uab.edu: Adding numbering

Jupyter

2019-10-28T01:57:22Z

Ozborn@uab.edu: /* Jupyter On Demand */

Jupyter

2019-10-28T00:05:30Z

Ozborn@uab.edu: Updated GPU instructions

File:PascalNodes.png

2019-10-28T00:03:36Z

Ozborn@uab.edu: Arguments for getting a GPU

Arguments for getting a GPU

File:TestPytorch.png

2019-10-28T00:03:18Z

Ozborn@uab.edu: Shows successful pytorch run on pascalnodes

Shows successful pytorch run on pascalnodes

File:PascalNodes.pdf

2019-10-27T23:57:35Z

Ozborn@uab.edu: Shows arguments for starting a job using PascaleNodes to get a GPU

Shows arguments for starting a job using PascaleNodes to get a GPU

File:TestPytorch.pdf

2019-10-27T23:57:03Z

Ozborn@uab.edu: Shows success test of PyTorch on Jupter Notebook Oct 2019

Shows success test of PyTorch on Jupter Notebook Oct 2019

Jupyter

2019-10-24T16:19:56Z

Ozborn@uab.edu: /* Jupyter On Demand */

Jupyter

2019-10-24T16:17:27Z

Ozborn@uab.edu: /* Jupyter by Proxy */ Warning added

Jupyter

2019-10-24T16:16:28Z

Ozborn@uab.edu: /* Jupyter On Demand */ Cleanup

Jupyter

2019-10-24T16:14:41Z

Ozborn@uab.edu: /* Jupyter On Demand */

Jupyter

2019-10-22T16:31:02Z

Ozborn@uab.edu: /* Jupyter On Demand */

Jupyter

2019-10-22T16:26:33Z

Ozborn@uab.edu: /* Jupyter On Demand */

Jupyter

2019-10-22T16:15:08Z

Ozborn@uab.edu: Updated Anaconda environment

Jupyter

2019-09-17T15:31:11Z

Ozborn@uab.edu: GPU environment description

File:JupyterNotebookGPUEnvironment.png

2019-09-17T15:29:21Z

Ozborn@uab.edu: Arguments for using GPUs

Arguments for using GPUs

Jupyter

2019-09-17T15:29:00Z

Ozborn@uab.edu: /* Jupyter On Demand */

Jupyter

2019-09-17T15:24:11Z

Ozborn@uab.edu: /* Jupyter On Demand */ Cleanup starting Jupyter

File:JupyterNotebookStart.png

2019-09-17T15:15:56Z

Ozborn@uab.edu: Image of starting Jupyter Notebook on cheaha

Image of starting Jupyter Notebook on cheaha

Jupyter

2019-09-17T15:13:02Z

Ozborn@uab.edu: /* Jupyter On Demand */ - environment arguments

Jupyter

2019-08-31T22:21:47Z

Ozborn@uab.edu: On Demand added

PheDRS

2019-01-30T17:38:14Z

Ozborn@uab.edu: /* NLP Discovered Patients */

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
TODO

Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide to PheDRS Data Loading Criteria ===

==== Data Loading Vocabulary ====
* '''Code''' - Structured data in the ''REGISTRY_PATIENT_CODES'' table including ICD-10-CM codes, DRG codes, MULTUM medication codes, etc...
* '''NLP''' - UMLS CUIs stored (currently) in the medics schema
* '''Encounter''' - Encounter data stored in ''REGISTRY_ENCOUNTER''
* '''Detection''' - Indicates that detected patient will be added as candidates to a registry
* '''Intervention''' - Like detection, but the Intervention property will be set on the patient regardless of status
* '''Collection''' - The data type will be copied to the ''REGISTRY_PATIENT_CODES'' table or ''REGISTRY_ENCOUNTER'' from the appropriate ETL table

==== Data Loading Process ====
This table details how the PheDRS Data Loading Criteria ontology is used to by backend clients (controlled by loadAll.bsh) to set patient status and review status. Like other registry configuration criteria, the configuration information is stored in the cvterm and cvtermprop for that registry.

These scripts detect patients for the registry (updating status_id) and flag patients for review (updating review_status_id) when registry specific events occur based on registry configuration.

The server side process will never set the patient registry status to 'Under Review" or "Reviewed"

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY _PATIENT _CVTERM !! Output Patient Registry Status Change !! Output Patient Review Status !! Output REGISTRY _PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT is set to a non-zero value || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || ||
|-
| Encounter Intervention || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP (which contains WHERE clause SQL) are loaded into REGISTRY_ENCOUNTERS || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| |||| ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Document Loading (deprecated) || Existence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Evidence Code Ontology ==
The evidence code ontology is used to describe how a patient's status in the registry (ACCEPTED, REJECTED, CANDIDATE OR PROVISIONAL) has been decided.
{| class="wikitable"
|+ Evidence Code
|-
! Name
! Short form
! Interpretation
|-
| Referred
| REF
| Current patient registry status has been assigned manually
|-
| Algorithm Discovered Patient
| ALG
| Current patient registry status assigned by an algorithm
|-
| NLP Discovered Patient
| NLP
| Current patient registry status has been assigned on the basis of NLP processed text.
|-
| Code discovered patient
| DIAG
| Current patient registry status has been assigned on the basis of a structured code, ex) ICD-10-CM
|-
| Encounter Discovered Patient
| ENC
| Current patient registry status has been assigned on the basis of an encounter
|-
|}

=== Registry Cvtermprop Data Loading Values for Evidence Updating ===
==== Code Discovered Patients ====
* Registry cvtermprop must have property in cvtermprop starting with "Code" and ending with "Detection"
* Registry cvtermprop must have a type_id with appropriate ontology and a value that ends with "diagnosis criteria" or "inclusion criteria"
==== Encounter Discovered Patients ====
* Registry cvtermprop value must have criteria for "Encounter Intervention" or "Encounter Detection" and non-null SQL clause in the value field
==== NLP Discovered Patients ====
* Registry cvtermprop must have property in cvtermprop starting with "NLP" AND ending with ("Detection" or "Intervention" or "Inclusion")
* Registry cvtermprop must have a type_id with appropriate ontology AND a value that ends with ("detection NLP criteria" or "inclusion NLP criteria" or "intervention NLP criteria")
TODO - make separate CV for this

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-30T17:31:06Z

Ozborn@uab.edu: /* Evidence Code Ontology */ Updating NLP

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
TODO

Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide to PheDRS Data Loading Criteria ===

==== Data Loading Vocabulary ====
* '''Code''' - Structured data in the ''REGISTRY_PATIENT_CODES'' table including ICD-10-CM codes, DRG codes, MULTUM medication codes, etc...
* '''NLP''' - UMLS CUIs stored (currently) in the medics schema
* '''Encounter''' - Encounter data stored in ''REGISTRY_ENCOUNTER''
* '''Detection''' - Indicates that detected patient will be added as candidates to a registry
* '''Intervention''' - Like detection, but the Intervention property will be set on the patient regardless of status
* '''Collection''' - The data type will be copied to the ''REGISTRY_PATIENT_CODES'' table or ''REGISTRY_ENCOUNTER'' from the appropriate ETL table

==== Data Loading Process ====
This table details how the PheDRS Data Loading Criteria ontology is used to by backend clients (controlled by loadAll.bsh) to set patient status and review status. Like other registry configuration criteria, the configuration information is stored in the cvterm and cvtermprop for that registry.

These scripts detect patients for the registry (updating status_id) and flag patients for review (updating review_status_id) when registry specific events occur based on registry configuration.

The server side process will never set the patient registry status to 'Under Review" or "Reviewed"

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY _PATIENT _CVTERM !! Output Patient Registry Status Change !! Output Patient Review Status !! Output REGISTRY _PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT is set to a non-zero value || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || ||
|-
| Encounter Intervention || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP (which contains WHERE clause SQL) are loaded into REGISTRY_ENCOUNTERS || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| |||| ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Document Loading (deprecated) || Existence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Evidence Code Ontology ==
The evidence code ontology is used to describe how a patient's status in the registry (ACCEPTED, REJECTED, CANDIDATE OR PROVISIONAL) has been decided.
{| class="wikitable"
|+ Evidence Code
|-
! Name
! Short form
! Interpretation
|-
| Referred
| REF
| Current patient registry status has been assigned manually
|-
| Algorithm Discovered Patient
| ALG
| Current patient registry status assigned by an algorithm
|-
| NLP Discovered Patient
| NLP
| Current patient registry status has been assigned on the basis of NLP processed text.
|-
| Code discovered patient
| DIAG
| Current patient registry status has been assigned on the basis of a structured code, ex) ICD-10-CM
|-
| Encounter Discovered Patient
| ENC
| Current patient registry status has been assigned on the basis of an encounter
|-
|}

=== Registry Cvtermprop Data Loading Values for Evidence Updating ===
==== Code Discovered Patients ====
* Registry cvtermprop must have property in cvtermprop starting with "Code" and ending with "Detection"
* Registry cvtermprop must have a type_id with appropriate ontology and a value that ends with "diagnosis criteria" or "inclusion criteria"
==== Encounter Discovered Patients ====
* Registry cvtermprop value must have criteria for "Encounter Intervention" or "Encounter Detection" and non-null SQL clause in the value field
==== NLP Discovered Patients ====
* Registry cvtermprop must have property in cvtermprop starting with "NLP" AND ending with ("Detection" or "Intervention")
* Registry cvtermprop must have a type_id with appropriate ontology AND a value that ends with ("NLP criteria" or "inclusion NLP criteria" or "intervention NLP criteria")
TODO - make separate CV for this

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-30T17:19:45Z

Ozborn@uab.edu: /* Evidence Code Ontology */

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
TODO

Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide to PheDRS Data Loading Criteria ===

==== Data Loading Vocabulary ====
* '''Code''' - Structured data in the ''REGISTRY_PATIENT_CODES'' table including ICD-10-CM codes, DRG codes, MULTUM medication codes, etc...
* '''NLP''' - UMLS CUIs stored (currently) in the medics schema
* '''Encounter''' - Encounter data stored in ''REGISTRY_ENCOUNTER''
* '''Detection''' - Indicates that detected patient will be added as candidates to a registry
* '''Intervention''' - Like detection, but the Intervention property will be set on the patient regardless of status
* '''Collection''' - The data type will be copied to the ''REGISTRY_PATIENT_CODES'' table or ''REGISTRY_ENCOUNTER'' from the appropriate ETL table

==== Data Loading Process ====
This table details how the PheDRS Data Loading Criteria ontology is used to by backend clients (controlled by loadAll.bsh) to set patient status and review status. Like other registry configuration criteria, the configuration information is stored in the cvterm and cvtermprop for that registry.

These scripts detect patients for the registry (updating status_id) and flag patients for review (updating review_status_id) when registry specific events occur based on registry configuration.

The server side process will never set the patient registry status to 'Under Review" or "Reviewed"

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY _PATIENT _CVTERM !! Output Patient Registry Status Change !! Output Patient Review Status !! Output REGISTRY _PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT is set to a non-zero value || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || ||
|-
| Encounter Intervention || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP (which contains WHERE clause SQL) are loaded into REGISTRY_ENCOUNTERS || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| |||| ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Document Loading (deprecated) || Existence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Evidence Code Ontology ==
The evidence code ontology is used to describe how a patient's status in the registry (ACCEPTED, REJECTED, CANDIDATE OR PROVISIONAL) has been decided.
{| class="wikitable"
|+ Evidence Code
|-
! Name
! Short form
! Interpretation
|-
| Referred
| REF
| Current patient registry status has been assigned manually
|-
| Algorithm Discovered Patient
| ALG
| Current patient registry status assigned by an algorithm
|-
| NLP Discovered Patient
| NLP
| Current patient registry status has been assigned on the basis of NLP processed text.
|-
| Code discovered patient
| DIAG
| Current patient registry status has been assigned on the basis of a structured code, ex) ICD-10-CM
|-
| Encounter Discovered Patient
| ENC
| Current patient registry status has been assigned on the basis of an encounter
|-
|}

=== Registry Cvtermprop Data Loading Values for Evidence Updating ===
==== Code Discovered Patients ====
* Registry cvtermprop value must end with "diagnosis criteria" or "inclusion criteria"
==== Encounter Discovered Patients ====
* Registry cvtermprop value must have criteria for "Encounter Intervention" or "Encounter Detection" and non-null SQL clause in the value field
==== NLP Discovered Patients ====
* Registry cvtermprop value must end with "NLP criteria" or "inclusion NLP criteria"
TODO - make separate CV for this

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-30T16:59:05Z

Ozborn@uab.edu: /* Developer Guide to PheDRS Data Loading Criteria */ Updating vocabulary meaning.

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
TODO

Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide to PheDRS Data Loading Criteria ===

==== Data Loading Vocabulary ====
* '''Code''' - Structured data in the ''REGISTRY_PATIENT_CODES'' table including ICD-10-CM codes, DRG codes, MULTUM medication codes, etc...
* '''NLP''' - UMLS CUIs stored (currently) in the medics schema
* '''Encounter''' - Encounter data stored in ''REGISTRY_ENCOUNTER''
* '''Detection''' - Indicates that detected patient will be added as candidates to a registry
* '''Intervention''' - Like detection, but the Intervention property will be set on the patient regardless of status
* '''Collection''' - The data type will be copied to the ''REGISTRY_PATIENT_CODES'' table or ''REGISTRY_ENCOUNTER'' from the appropriate ETL table

==== Data Loading Process ====
This table details how the PheDRS Data Loading Criteria ontology is used to by backend clients (controlled by loadAll.bsh) to set patient status and review status. Like other registry configuration criteria, the configuration information is stored in the cvterm and cvtermprop for that registry.

These scripts detect patients for the registry (updating status_id) and flag patients for review (updating review_status_id) when registry specific events occur based on registry configuration.

The server side process will never set the patient registry status to 'Under Review" or "Reviewed"

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY _PATIENT _CVTERM !! Output Patient Registry Status Change !! Output Patient Review Status !! Output REGISTRY _PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT is set to a non-zero value || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || ||
|-
| Encounter Intervention || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP (which contains WHERE clause SQL) are loaded into REGISTRY_ENCOUNTERS || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| |||| ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Document Loading (deprecated) || Existence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Evidence Code Ontology ==
The evidence code ontology is used to describe how a patient's status in the registry (ACCEPTED, REJECTED, CANDIDATE OR PROVISIONAL) has been decided.
{| class="wikitable"
|+ Evidence Code
|-
! Name
! Short form
! Interpretation
|-
| Referred
| REF
| Current patient registry status has been assigned manually
|-
| Algorithm Discovered Patient
| ALG
| Current patient registry status assigned by an algorithm
|-
| NLP Discovered Patient
| NLP
| Current patient registry status has been assigned on the basis of NLP processed text.
|-
| Code discovered patient
| DIAG
| Current patient registry status has been assigned on the basis of a structured code, ex) ICD-10-CM
|-
| Encounter Discovered Patient
| ENC
| Current patient registry status has been assigned on the basis of an encounter
|-
|}

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-24T20:39:55Z

Ozborn@uab.edu: /* Phenotype Detection Registry System (PheDRS) User Documentation */ Evidence Codes

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
TODO

Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide to PheDRS Data Loading Criteria ===
This table details how the PheDRS Data Loading Criteria ontology is used to by backend clients (controlled by loadAll.bsh) to set patient status and review status. Like other registry configuration criteria, the configuration information is stored in the cvterm and cvtermprop for that registry.

These scripts detect patients for the registry (updating status_id) and flag patients for review (updating review_status_id) when registry specific events occur based on registry configuration.

The server side process will never set the patient registry status to 'Under Review" or "Reviewed"

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY _PATIENT _CVTERM !! Output Patient Registry Status Change !! Output Patient Review Status !! Output REGISTRY _PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT is set to a non-zero value || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || ||
|-
| Encounter Intervention || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP (which contains WHERE clause SQL) are loaded into REGISTRY_ENCOUNTERS || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| |||| ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Document Loading (deprecated) || Existence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Evidence Code Ontology ==
The evidence code ontology is used to describe how a patient's status in the registry (ACCEPTED, REJECTED, CANDIDATE OR PROVISIONAL) has been decided.
{| class="wikitable"
|+ Evidence Code
|-
! Name
! Short form
! Interpretation
|-
| Referred
| REF
| Current patient registry status has been assigned manually
|-
| Algorithm Discovered Patient
| ALG
| Current patient registry status assigned by an algorithm
|-
| NLP Discovered Patient
| NLP
| Current patient registry status has been assigned on the basis of NLP processed text.
|-
| Code discovered patient
| DIAG
| Current patient registry status has been assigned on the basis of a structured code, ex) ICD-10-CM
|-
| Encounter Discovered Patient
| ENC
| Current patient registry status has been assigned on the basis of an encounter
|-
|}

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-24T20:28:59Z

Ozborn@uab.edu: /* Installation / Setup */ TODO

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
TODO

Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide to PheDRS Data Loading Criteria ===
This table details how the PheDRS Data Loading Criteria ontology is used to by backend clients (controlled by loadAll.bsh) to set patient status and review status. Like other registry configuration criteria, the configuration information is stored in the cvterm and cvtermprop for that registry.

These scripts detect patients for the registry (updating status_id) and flag patients for review (updating review_status_id) when registry specific events occur based on registry configuration.

The server side process will never set the patient registry status to 'Under Review" or "Reviewed"

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY _PATIENT _CVTERM !! Output Patient Registry Status Change !! Output Patient Review Status !! Output REGISTRY _PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT is set to a non-zero value || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || ||
|-
| Encounter Intervention || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP (which contains WHERE clause SQL) are loaded into REGISTRY_ENCOUNTERS || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| |||| ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Document Loading (deprecated) || Existence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-18T20:19:00Z

Ozborn@uab.edu: /* Developer Guide to PheDRS Data Loading Criteria */

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide to PheDRS Data Loading Criteria ===
This table details how the PheDRS Data Loading Criteria ontology is used to by backend clients (controlled by loadAll.bsh) to set patient status and review status. Like other registry configuration criteria, the configuration information is stored in the cvterm and cvtermprop for that registry.

These scripts detect patients for the registry (updating status_id) and flag patients for review (updating review_status_id) when registry specific events occur based on registry configuration.

The server side process will never set the patient registry status to 'Under Review" or "Reviewed"

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY _PATIENT _CVTERM !! Output Patient Registry Status Change !! Output Patient Review Status !! Output REGISTRY _PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT is set to a non-zero value || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || ||
|-
| Encounter Intervention || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP (which contains WHERE clause SQL) are loaded into REGISTRY_ENCOUNTERS || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| |||| ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || ||
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Document Loading (deprecated) || Existence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-18T19:54:48Z

Ozborn@uab.edu: /* Developer Guide to PheDRS Data Loading Criteria */

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide to PheDRS Data Loading Criteria ===
This table details how the PheDRS Data Loading Criteria ontology is used to by backend clients (controlled by loadAll.bsh) to set patient status and review status. Like other registry configuration criteria, the configuration information is stored in the cvterm and cvtermprop for that registry.

These scripts detect patients for the registry (updating status_id) and flag patients for review (updating review_status_id) when registry specific events occur based on registry configuration.

The server side process will never set the patient registry status to 'Under Review" or "Reviewed"

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY _PATIENT _CVTERM !! Output Patient Registry Status Change !! Output Patient Review Status !! Output REGISTRY _PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT ||
|-
| || || || || || || || || is set to a non-zero value ||
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || ||
|-
| Encounter Intervention || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| |||| ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || (which contains WHERE clause SQL) are loaded into ||
|-
| || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || REGISTRY_ENCOUNTERS ||
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Document Loading (deprecated) || Existence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-18T19:51:27Z

Ozborn@uab.edu: /* Developer Guide for Registry Configuration Properties */

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide to PheDRS Data Loading Criteria ===
This table details how the PheDRS Data Loading Criteria ontology is used to by backend clients (controlled by loadAll.bsh) to set patient status and review status. Like other registry configuration criteria, the configuration information is stored in the cvterm and cvtermprop for that registry.

These scripts detect patients for the registry (updating status_id) and flag patients for review (updating review_status_id) when registry specific events occur based on registry configuration.

The server side process will never set the patient registry status to 'Under Review" or "Reviewed"

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY_PATIENT _CVTERM !! Output Patient Registry Status Change !! Output Patient Review Status !! Output REGISTRY_PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT ||
|-
| || || || || || || || || is set to a non-zero value ||
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || ||
|-
| Encounter Intervention || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| |||| ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || (which contains WHERE clause SQL) are loaded into ||
|-
| || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || REGISTRY_ENCOUNTERS ||
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Document Loading || Existence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-18T19:44:47Z

Ozborn@uab.edu: /* Developer Guide for Registry Configuration Properties */

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide for Registry Configuration Properties ===
This table details the relationship between the registry PHEDRS back end processing script and registry specific information in the PHEDRS schema (encoded mostly in cvterm and cvtermprop tables)
The goal is to have the back-end script reading the database to get registry specific parameters to detect patients and flag events that require registry review.
These scripts detect patients for the registry (updating status_id) and flag patients for review (updating review_status_id) when registry specific events occur based on registry configuration.
The server side process will never set the patient registry status to 'Under Review" or "Reviewed"

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY_PATIENT _CVTERM !! Output Patient Registry Status Change !! Output Patient Review Status !! Output REGISTRY_PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT ||
|-
| || || || || || || || || is set to a non-zero value ||
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || ||
|-
| Encounter Intervention || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| |||| ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || (which contains WHERE clause SQL) are loaded into ||
|-
| || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || REGISTRY_ENCOUNTERS ||
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Document Loading || Existence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-18T19:38:32Z

Ozborn@uab.edu: /* Developer Guide for Registry Configuration Properties */

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide for Registry Configuration Properties ===

{| class="wikitable"
|+ Developer Guide for Registry Configuration Properties
|-
! CRITERIA NAME
! EVENT DESCRIPTION
! Input Patient Registry Status
! Input Patient Review Status
! Input Required Registry Patient Cvterm/s
! Outputput Patient Registry Status
! Output Patient Review Status
! Output Change to Registry Patient Cvterm/s
! Action Description
! Implementation Notes
|-
|Diagnosis (Code_Detection)
|PATIENT_CODE entry that matches diagnosis criteria in CVTERMPROP
|NULL
|NULL
|NULL
|CANDIDATE
|NEVER REVIEWED
|ANY
|Add candidate
|CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|}

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY_PATIENT _CVTERM !! Output Patient Registry Status !! Output Patient Review Status !! Output REGISTRY_PATIENT _CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry hat matches diagnosis criteria in CVTERMPROP || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate t || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || ||
|-
| NLP_Intervention || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || PATIENT_CODE entry that matches inclusion criteria in CVTERMPROP|| All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || || || || || || || ||
|-
| || || || || || || || || ||
|-
| Eligibility || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT ||
|-
| || || || || || || || || is set to a non-zero value ||
|-
| || || || || || || || || ||
|-
| Code_Collection || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || ||
|-
| Encounter Collection || COPD,CRCP(?) || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || || ||
|-
| Encounter Intervention || COPD || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| || || || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || (which contains WHERE clause SQL) are loaded into ||
|-
| || || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || REGISTRY_ENCOUNTERS ||
|-
| || || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || || ||
|-
| Document Loading || COPD || Exisitence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==

PheDRS

2019-01-18T19:33:32Z

Ozborn@uab.edu: /* Developer Guide for Registry Configuration Properties */

= Phenotype Detection Registry System (PheDRS) User Documentation =

== Overview ==

The Phenotype Detection Registry System (PheDRS) is a tool at UAB used to assist researchers and clinicians identify populations of interest at UAB matching clinical criteria from both structured data (billing codes, medications, labs) and unstructured documents using Natural Language Processing. The user documentation described here is intended for managers and users of the PheDRS system.

Additional documentation for PheDRS is found in the following locations:

1. Project specific information is found in the appropriate git repo wikis and markdown text

2. UAB specific developer documentation is found in the UAB_BioMedInformatics\PHEDRS Box directory

== Installation / Setup ==
Currently this is very site specific and depends on various backend systems, EHR vendors, etc... UAB specific site setup can be found in the UAB_BioMedInformatics\PHEDRS Box directory

== Roles ==
PheDRS supports the following user roles:

=== Manager ===
The manager of a registry. Full read and write access to the registry. Can add new users as registrars or managers to the registry.
=== Registrar ===
A user with read and write access to the registry.
=== Administrator ===
System administrator responsible for maintaining the entire PheDRS system including all registries.
=== Viewer ===
Read only access to the registry.
=== Inactive ===
Former registrars no longer active.
=== De-ID Viewer ===
Viewer allowed to view de-identified data (18 HIPAA Safe Harbor items removed) data.

== Registry Configuration ==
The registry is configured by the registry manager, often in conjunction with the system administrator.

=== Registry Property Descriptions and Examples ===
{| class="wikitable"
|+ Registry Configuration: Cvtermprop parameters
|-
! CV NAME
! CV ID
! Example of cvterm name for TYPE_ID
! Meaning and Use of VALUE field for this type
! Example of Value Field
|-
| UAB ICD-10-CM Codes
| 9
| Panlobular emphysema
| If patient has this billing code, assign as candidate to registry
| UAB COPD Registry ICD-10-CM diagnosis criteria

|-
| UAB MS DRG Codes
| 12
| COPD w CC
| If patient has this DRG code, then add anchor cvterm property to this code
| DRG Codes COPD Anchor Encounter criteria

|-
|
|
| COPD w MCC
| If patient has this DRG code, then add patient to the registry as "accepted"
| UAB COPD Registry DRG Codes inclusion criteria

|-
| PHEDRS Data Loading Criteria
| 23
| Default NLP Pipeline Collection
| This is the ID# of the default pipeline for displaying documents in this registry. Unenforced Foreign Key to db_id=14
| 2

|-
|
|
| Encounter Collection
| Encounters matching the appended SQL criteria are added to the registry
| AND (SRC_ADMIT_DT_TM >= (SYSDATE-730)) AND FORMATTED_MEDICAL_RECORD_NBR IN (SELECT FORMATTED_UAB_MRN FROM REGISTRY_PATIENT WHERE REGISTRY_ID=2861)

|-
|
|
| Encounter Intervention
| Encounters matching the appended SQL criteria are added to the registry AND candidate is flagged for review
| AND SRC_ADMIT_DT_TM >= (SYSDATE-30) AND ( (reason_for_visit_txt LIKE '%COPD%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHITIS%') OR (UPPER(reason_for_visit_txt) LIKE '%BRONCHIECTASIS%') OR (UPPER(reason_for_visit_txt) LIKE '%EMPHYSEMA%') )

|-
| PHEDRS MetaData CV
| 24
| Patient Review Cutoff Date
| Do not retrieve any data (codes, encounters, documents) from source system before this date
| 1/1/17

|-
|
|
| Registry Application Title
| Use this text here for the name of the registry
| COPD Registry Control Panel

|-
|
|
| Display Codes Database Identifier
| DB_ID permissable to display in "Diagnoses and Drgs" Panel
| 8

|-
| PHEDRS Tab CV
| 25
| REGISTRY_INPATIENT_REVIEW
| Indicates that this tab should be displayed in this registry. Value indicates display order from LEFT to RIGHT (1 for leftmost tab)
| 1

|-
| Optum UAB APR DRG Codes 2017
| 26
|
|
|

|-
| UAB WH CLN REF Codeset 69
| 27
|
|
|
|}

=== Developer Guide for Registry Configuration Properties ===

{| class="wikitable"
|+ Developer Guide for Registry Configuration Properties
|-
! CRITERIA NAME
! EVENT DESCRIPTION
! Input Patient Registry Status
! Input Patient Review Status
! Input Required Registry Patient Cvterm/s
! Outputput Patient Registry Status
! Output Patient Review Status
! Output Change to Registry Patient Cvterm/s
! Action Description
! Implementation Notes
|-
|Diagnosis (Code_Detection)
|PATIENT_CODE entry that matches diagnosis criteria in CVTERMPROP
|NULL
|NULL
|NULL
|CANDIDATE
|NEVER REVIEWED
|ANY
|Add candidate
|CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|}

{| class="wikitable"
! Criteria Name !! Event Description !! Input Patient Registry Status !! Input Patient Review Status !! Input REGISTRY_PATIENT _CVTERM !! Output Patient Registry Status !! Output Patient Review Status !! Output REGISTRY_PATIENT_CVTERM !! Action Description !! Implementation Notes
|-
| Diagnosis (Code_Detection) || PATIENT_CODE entry || NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate that matches diagnosis criteria in CVTERMPROP || CVTERMs corresponding to codes have a CVTERMPROP set on them of type registry_name (from cv_id 11) with a value containing text "diagnosis criteria"
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Plan is to swap CVTERMs such that CVTERM_ID is the registry and the TYPE_ID is the code
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore diagnosis criteria on accepted patients ||
|-
| || || || || || || || || ||
|-
| NLP_Detection || NLP_HITS_EXTENDED contains CUI with detection criteria|| NULL || NULL || NULL || CANDIDATE || NEVER REVIEWED || ANY || Add candidate || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || Rejected|Provisional || ANY || ANY || NO CHANGE || NEEDS REVIEW || ANY || Flag rejected and provisional candidates for review || Value Field Example: nlp_detection|C98764323,C0012345,C0654321
|-
| || || Accepted|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || ANY || Ignore detection criteria on accepted patients || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || || || || || || || ||
|-
| NLP_Intervention || None || NLP_HITS_EXTENDED || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || CVTERMS corresponding to registries have a CVTERMPROP set on them with cvterm_id (registry_id), type_id (cvterm in NLP Pipeline Parameter Ontology) with value containing a UMLS CUI
|-
| || || contains CUI with || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Value Field Example: nlp_intervention|C98764323,C0012345,C0654321
|-
| || || intervention criteria || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || || Multiple CUIs are supported as additional rows in CVTERMPROP. Contextual CUIs are supported via commas
|-
| || || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || || Compare date of code/encounter with the end(if not null) or start_date of the REGISTRY_PATIENT_CVTERM entry
|-
| || || || || || || || || || ||
|-
| Inclusion (Code Inclusion) || COPD || PATIENT_CODE entry || All except Accepted || ANY || ANY || ACCEPTED || NEEDS REVIEW || || DRG Codes generate automatic registry inclusion ||
|-
| || || that matches inclusion || Accepted || ANY || ANY || NO CHANGE || NEEDS REVIEW || || ||
|-
| || || criteria in CVTERMPROP || || || || || || || ||
|-
| || || || || || || || || || ||
|-
| Eligibility || CRCP || Custom || ANY || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Machine learning of patient eligibility || Custom registry specific implementation that should result in a publication… :)
|-
| || || || || || || || || || MACHINE_ACCEPTANCE_SCORE in REGISTRY_PATIENT ||
|-
| || || || || || || || || || is set to a non-zero value ||
|-
| || || || || || || || || || ||
|-
| Code_Collection || CRCP, MM || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Specify data to collect for the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (data source) and value which is a regular expression specifying code_values to collect from REGISTRY_PATIENT_CODES
|-
| || || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Do not collect data unless patient is in registry ||
|-
| || || || || || || || || || ||
|-
| Encounter Collection || COPD,CRCP(?) || None (daily) || Rejected|Candidate || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Collect specified encounter data for people in the registry || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in the format below
|-
| || || || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || || REGISTRY_ENCOUNTER COLUMN_NAME|REGEX||REGISTRY_ENCOUNTER_COLUMN_NAME|REGEX….
|-
| || || || || || || || || || ||
|-
| Encounter Intervention || COPD || Encounter || ANY || ANY || Requires Intervention || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Encounters matching criteria specified in CVTERM_PROP || CVTERMPROP has cvterm_id (registry_id) and type_id (sourcesystem_cd) and value in SQL format
|-
| || || || ANY || ANY || Intervention Complete < Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || (which contains WHERE clause SQL) are loaded into ||
|-
| || || || ANY || ANY || Any other current valid cvterms || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || Requires Intervention || REGISTRY_ENCOUNTERS ||
|-
| || || || ANY || ANY || Intervention Complete > Encounter Date || NO CHANGE (CANDIDATE IF NULL) || NO CHANGE || NO CHANGE || Flag patients for intervention ||
|-
| Encounter Detection || None || Encounter || || || || || || || ||
|-
| || || || || || || || || || ||
|-
| Document Loading || COPD || Exisitence of documents || Provisional|Accepted || ANY || ANY || NO CHANGE || NO CHANGE || NO CHANGE || Load NLP_DOCS from source systems for registry patients || Should be a default analysis type that is used for CVTERMPROP
|}

== Back End Workflow / Architecture ==

== Web Service Documentation ==
=== RegistryWS ===
=== UDAS ===

== Paper References ==