PublicDatasetsNgs - Revision history

Curtish@uab.edu at 20:19, 18 December 2012

2012-12-18T20:19:03Z

@@ Line 1: / Line 1: @@
 = Public Genomes and indexes for NGS =
@@ Line 77: / Line 79: @@
 * Embl needs chr
 * UCSC needs gtf + refgene then my script to get good GTF

Curtish@uab.edu: /* Directory Structure */

2012-11-12T21:17:47Z

Directory Structure

← Older revision		Revision as of 21:17, 12 November 2012
Line 11:		Line 11:
	= Directory Structure =		= Directory Structure =
	* GIT:/??/scripts		* GIT:/??/scripts
	* /lustre/~~projects~~/public_dataset/ngs		* /lustre/project/public_dataset/ngs
	* curator: ngs-ccts group		* curator: ngs-ccts group

Curtish@uab.edu: /* Directory Structure */

2012-10-25T18:22:10Z

Directory Structure

← Older revision		Revision as of 18:22, 25 October 2012
Line 11:		Line 11:
	= Directory Structure =		= Directory Structure =
	* GIT:/??/scripts		* GIT:/??/scripts
	* /lustre/projects/~~public_datasets~~/ngs		* /lustre/projects/public_dataset/ngs
	* curator: ngs-ccts group		* curator: ngs-ccts group

Curtish@uab.edu: /* Public Genomes and indexes for NGS */

2012-10-25T18:21:57Z

Public Genomes and indexes for NGS

← Older revision		Revision as of 18:21, 25 October 2012
Line 1:		Line 1:
	= Public Genomes and indexes for NGS =		= Public Genomes and indexes for NGS =

	cheaha:/lustre/~~projects~~/public_datasets/ngs		cheaha:/lustre/project/public_datasets/ngs

	CCTS BMI in conjunection with anyone who wishes to participate, maintains a public directory of genomic sequences and indexes built from them for use with a variety of NGS alignment and analysis programs, for use both from the command-line and within the Galaxy environment		CCTS BMI in conjunection with anyone who wishes to participate, maintains a public directory of genomic sequences and indexes built from them for use with a variety of NGS alignment and analysis programs, for use both from the command-line and within the Galaxy environment

Curtish@uab.edu: /* Notes on what CCTS BMI wants to populate */

2012-10-25T18:20:10Z

Notes on what CCTS BMI wants to populate

← Older revision		Revision as of 18:20, 25 October 2012
Line 61:		Line 61:
	*** [dkcrossm] For humans, if you use UCSC's genome, then mitochondria goes first, then numerically (chrMt, chr1, chr2...chrX, chrY). If you use another human genome (from 1000 genome project), then it goes in numerical order with mitochondria after the chrY (chr1, chr2, chr3...chrY, chrMt).		*** [dkcrossm] For humans, if you use UCSC's genome, then mitochondria goes first, then numerically (chrMt, chr1, chr2...chrX, chrY). If you use another human genome (from 1000 genome project), then it goes in numerical order with mitochondria after the chrY (chr1, chr2, chr3...chrY, chrMt).
	**** For non-humans, it doesn’t seem to matter. I've tried it both ways. In a C. elegans study, I had mitochondria first. In a yeast study, I had mitochondria last, and they both worked. So, I guess non-humans chromosome order isn't critical.		**** For non-humans, it doesn’t seem to matter. I've tried it both ways. In a C. elegans study, I had mitochondria first. In a yeast study, I had mitochondria last, and they both worked. So, I guess non-humans chromosome order isn't critical.

	*** [dkcrossm] It seems that GATK has reversed their policy again on the chromosome order for humans (it doesn’t say anything about the other species chromosome orders, so that is still left up in the air). I’ve copied their section on human chromosome order for your reference:		*** [dkcrossm] It seems that GATK has reversed their policy again on the chromosome order for humans (it doesn’t say anything about the other species chromosome orders, so that is still left up in the air). I’ve copied their section on human chromosome order for your reference:
	*** If you are using human data, your reads must be aligned to one of the official b3x (e.g. b36, b37) or hg1x (e.g. hg18, hg19) references. The contig ordering in the reference you used must exactly match that of one of the official references canonical orderings. These are defined by historical karotyping of largest to smallest chromosomes, followed by the X, Y, and MT. The order is thus 1, 2, 3, ..., 10, 11, 12, ... 20, 21, 22, X, Y, MT. The GATK will detect misordered contigs (for example, lexicographically sorted) and throw an error. This draconian approach, though unnecessary technically, ensures that all supplementary data provided with the GATK works correctly. You can use ReorderSam to fix a BAM file aligned to a missorted reference sequence.		*** If you are using human data, your reads must be aligned to one of the official b3x (e.g. b36, b37) or hg1x (e.g. hg18, hg19) references. The contig ordering in the reference you used must exactly match that of one of the official references canonical orderings. These are defined by historical karotyping of largest to smallest chromosomes, followed by the X, Y, and MT. The order is thus 1, 2, 3, ..., 10, 11, 12, ... 20, 21, 22, X, Y, MT. The GATK will detect misordered contigs (for example, lexicographically sorted) and throw an error. This draconian approach, though unnecessary technically, ensures that all supplementary data provided with the GATK works correctly. You can use ReorderSam to fix a BAM file aligned to a missorted reference sequence.

Curtish@uab.edu: /* Notes on what CCTS BMI wants to populate */

2012-10-25T18:19:47Z

Notes on what CCTS BMI wants to populate

← Older revision		Revision as of 18:19, 25 October 2012
Line 59:		Line 59:
	* GATK Chrome order		* GATK Chrome order
	** "new" GATK order		** "new" GATK order
			*** [dkcrossm] For humans, if you use UCSC's genome, then mitochondria goes first, then numerically (chrMt, chr1, chr2...chrX, chrY). If you use another human genome (from 1000 genome project), then it goes in numerical order with mitochondria after the chrY (chr1, chr2, chr3...chrY, chrMt).
			**** For non-humans, it doesn’t seem to matter. I've tried it both ways. In a C. elegans study, I had mitochondria first. In a yeast study, I had mitochondria last, and they both worked. So, I guess non-humans chromosome order isn't critical.

	*** [dkcrossm] It seems that GATK has reversed their policy again on the chromosome order for humans (it doesn’t say anything about the other species chromosome orders, so that is still left up in the air). I’ve copied their section on human chromosome order for your reference:		*** [dkcrossm] It seems that GATK has reversed their policy again on the chromosome order for humans (it doesn’t say anything about the other species chromosome orders, so that is still left up in the air). I’ve copied their section on human chromosome order for your reference:
	*** If you are using human data, your reads must be aligned to one of the official b3x (e.g. b36, b37) or hg1x (e.g. hg18, hg19) references. The contig ordering in the reference you used must exactly match that of one of the official references canonical orderings. These are defined by historical karotyping of largest to smallest chromosomes, followed by the X, Y, and MT. The order is thus 1, 2, 3, ..., 10, 11, 12, ... 20, 21, 22, X, Y, MT. The GATK will detect misordered contigs (for example, lexicographically sorted) and throw an error. This draconian approach, though unnecessary technically, ensures that all supplementary data provided with the GATK works correctly. You can use ReorderSam to fix a BAM file aligned to a missorted reference sequence.		*** If you are using human data, your reads must be aligned to one of the official b3x (e.g. b36, b37) or hg1x (e.g. hg18, hg19) references. The contig ordering in the reference you used must exactly match that of one of the official references canonical orderings. These are defined by historical karotyping of largest to smallest chromosomes, followed by the X, Y, and MT. The order is thus 1, 2, 3, ..., 10, 11, 12, ... 20, 21, 22, X, Y, MT. The GATK will detect misordered contigs (for example, lexicographically sorted) and throw an error. This draconian approach, though unnecessary technically, ensures that all supplementary data provided with the GATK works correctly. You can use ReorderSam to fix a BAM file aligned to a missorted reference sequence.

Curtish@uab.edu: /* Notes on what CCTS BMI wants to populate */

2012-10-04T21:06:31Z

Notes on what CCTS BMI wants to populate

← Older revision		Revision as of 21:06, 4 October 2012
Line 21:		Line 21:
	# list available dbkeys		# list available dbkeys
	rsync --list-only --dirs -lptgoDzvP datacache.g2.bx.psu.edu::indexes		rsync --list-only --dirs -lptgoDzvP datacache.g2.bx.psu.edu::indexes
			# list available dbkeys for "microbes"
			rsync --list-only --dirs -lptgoDzvP datacache.g2.bx.psu.edu::indexes/microbes/
	# list available .loc files		# list available .loc files
	rsync --list-only --dirs -lptgoDzvP datacache.g2.bx.psu.edu::location		rsync --list-only --dirs -lptgoDzvP datacache.g2.bx.psu.edu::location

Curtish@uab.edu: /* Public Genomes and indexes for NGS */

2012-10-04T17:32:08Z

Public Genomes and indexes for NGS

← Older revision		Revision as of 17:32, 4 October 2012
Line 1:		Line 1:
	= Public Genomes and indexes for NGS =		= Public Genomes and indexes for NGS =

	CCTS BMI in conjunection with anyone who wishes to participate, maintains a public directory of genomic sequences and indexes built from them for use with a variety of NGS alignment and analysis programs, for use both from the command-line and Galaxy.		cheaha:/lustre/projects/public_datasets/ngs

	~~cheaha~~:~~/lustre/projects/public_datasets~~		CCTS BMI in conjunection with anyone who wishes to participate, maintains a public directory of genomic sequences and indexes built from them for use with a variety of NGS alignment and analysis programs, for use both from the command-line and within the Galaxy environment

			See also [[NgsCcts]] for programs that build and use these indices and databases.

			WARNING: these datasets are currently under construction. Contact curtish A uab.edu with questions.

	= Directory Structure =		= Directory Structure =

Curtish@uab.edu: /* Notes on what CCTS BMI wants to populate */

2012-10-03T15:35:15Z

Notes on what CCTS BMI wants to populate

← Older revision		Revision as of 15:35, 3 October 2012
Line 22:		Line 22:
	# list mm9 dir/file tree		# list mm9 dir/file tree
	rsync --list-only -azvP rsync://datacache.g2.bx.psu.edu/indexes/mm9 /lustre/project/public_datasets/ngs/galaxy-datacache/indexes		rsync --list-only -azvP rsync://datacache.g2.bx.psu.edu/indexes/mm9 /lustre/project/public_datasets/ngs/galaxy-datacache/indexes
	# pull down mm9		# pull down mm9 (all)
	rsync -azvP rsync://datacache.g2.bx.psu.edu/indexes/mm9 /lustre/project/public_datasets/ngs/galaxy-datacache/indexes		rsync -azvP rsync://datacache.g2.bx.psu.edu/indexes/mm9 /lustre/project/public_datasets/ngs/galaxy-datacache/indexes
			# pull down mm9 w/o alignment files
			rsync --exclude align/ -azvP rsync://datacache.g2.bx.psu.edu/indexes/mm9 /lustre/project/public_datasets/ngs/galaxy-datacache/indexes

	# pull down all loc files		# pull down all loc files
	rsync -azvP rsync://datacache.g2.bx.psu.edu/location/ /lustre/project/public_datasets/ngs/galaxy-datacache/location/		rsync -azvP rsync://datacache.g2.bx.psu.edu/location/ /lustre/project/public_datasets/ngs/galaxy-datacache/location/

Curtish@uab.edu: /* Notes on what CCTS BMI wants to populate */

2012-10-02T18:58:14Z

Notes on what CCTS BMI wants to populate

← Older revision		Revision as of 18:58, 2 October 2012
Line 17:		Line 17:
	# list available dbkeys		# list available dbkeys
	rsync --list-only --dirs -lptgoDzvP datacache.g2.bx.psu.edu::indexes		rsync --list-only --dirs -lptgoDzvP datacache.g2.bx.psu.edu::indexes
			# list available .loc files
			rsync --list-only --dirs -lptgoDzvP datacache.g2.bx.psu.edu::location

			# list mm9 dir/file tree
			rsync --list-only -azvP rsync://datacache.g2.bx.psu.edu/indexes/mm9 /lustre/project/public_datasets/ngs/galaxy-datacache/indexes
	# pull down mm9		# pull down mm9
	rsync -azvP rsync://datacache.g2.bx.psu.edu/indexes/mm9 /lustre/project/public_datasets/ngs/galaxy-datacache		rsync -azvP rsync://datacache.g2.bx.psu.edu/indexes/mm9 /lustre/project/public_datasets/ngs/galaxy-datacache/indexes

			# pull down all loc files
			rsync -azvP rsync://datacache.g2.bx.psu.edu/location/ /lustre/project/public_datasets/ngs/galaxy-datacache/location/