GenPlay File Formats

From GenPlay, Einstein Genome Analyzer

Revision as of 17:52, 23 November 2010 by Julien (talk | contribs) (→‎Usage)
Jump to: navigation, search

.2bit format

Description

A .2bit file stores multiple DNA sequences (up to 4 Gb total) in a compact randomly-accessible format. The file contains masking information as well as the DNA itself.

The file begins with a 16-byte header containing the following fields:

  1. signature - the number 0x1A412743 in the architecture of the machine that created the file
  2. version - zero for now. Readers should abort if they see a version number higher than 0.
  3. sequenceCount - the number of sequences in the file.
  4. reserved - always zero for now

All fields are 32 bits unless noted. If the signature value is not as given, the reader program should byte-swap the signature and check if the swapped version matches. If so, all multiple-byte entities in the file will have to be byte-swapped. This enables these binary files to be used unchanged on different architectures.

The header is followed by a file index, which contains one entry for each sequence. Each index entry contains three fields:

  1. nameSize - a byte containing the length of the name field
  2. name - the sequence name itself, of variable length depending on nameSize
  3. offset - the 32-bit offset of the sequence data relative to the start of the file

The index is followed by the sequence records, which contain nine fields:

  1. dnaSize - number of bases of DNA in the sequence
  2. nBlockCount - the number of blocks of Ns in the file (representing unknown sequence)
  3. nBlockStarts - the starting position for each block of Ns
  4. nBlockSizes - the size of each block of Ns
  5. maskBlockCount - the number of masked (lower-case) blocks
  6. maskBlockStarts - the starting position for each masked block
  7. maskBlockSizes - the size of each masked block
  8. reserved - always zero for now
  9. packedDna - the DNA packed to two bits per base, represented as so: T - 00, C - 01, A - 10, G - 11.

The first base is in the most significant 2-bit byte; the last base is in the least significant 2 bits. For example, the sequence TCAG is represented as 00011011. The packedDna field is padded with 0 bits as necessary to take an even multiple of 32 bits in the file, which improves I/O performance on some machines.

Usage

The .2bit files can be loaded as sequence tracks.

BAM / SAM format

Description

SAM (Sequence Alignment/Map) format is a generic format for storing large nucleotide sequence alignments.

BAM is a Binary version of the Sequence Alignment / Map (SAM) format.

More information about these file formats can be found here

Usage

SAM files can generate fixed window track. BAM can't be used directly with GenPlay but can be easily converted in a SAM file using tools available on the Internet.

BED format

Description

BED format has three necessary fields and nine additional optional fields. The number of fields per line have to be constant throughout any single set of data in an annotation track. The order of the optional fields requires that lower-numbered fields be filled if higher-numbered fields are used.

  • The first three required BED fields are:
  1. chromosome - The name of the chromosome (e.g. chr1, chrX etc.) or scaffold (e.g. scaffold10461).
  2. Start - The beginning position of the feature in the chromosome or scaffold. The first base in a chromosome is numbered 0.
  3. End - The ending position of the feature in the chromosome or scaffold.
  • The 9 additional optional BED fields are:
  1. name – This is the name of the gene.
  2. score - A score between 0 and 1000.
  3. strand - Defines the strand as either '+' (red) or '-' (blue).
  4. thickStart
  5. thickEnd
  6. itemRgb
  7. blockCount
  8. blockSizes
  9. blockStarts

Example

Here is an example of a bed file. 

track type=bed
searchURL="http://genome.ucsc.edu/cgi-bin/hgGene?org=Human&hgg_chrom=none&db=hg19&hgg_gene="
chr1	11873	14409	uc001aaa.3	0	+	11873	11873	0	3	354,109,1189,	0,739,1347,
chr1	11873	14409	uc010nxq.1	0	+	12189	13639	0	3	354,127,1007,	0,721,1529,
chr1	11873	14409	uc010nxr.1	        0	+	11873	11873	0	3	354,52,1189,	0,772,1347,
chr1	14362	16765	uc009vis.2	        0	-	14362	14362	0	4	467,69,147,159,	0,607,1433,2244,

Note: The search URL "http://genome.ucsc.edu/cgi-bin/hgGene?org=Human&hgg_chrom=none&db=hg19&hgg_gene=" is included at the beginning of each BED file. It provides the URL that contains the description of the genes. One can form the complete URL by appending the name of the gene at the end of the search URL after the equal to sign. For example, "http://genome.ucsc.edu/cgi-bin/hgGene?org=Human&hgg_chrom=none&db=hg19&hgg_gene=uc001aaa.3" will provide information on WDR 78.

Usage

A Bed file with the 3 required fields can generate stripes. If the 2 first required fields are specified, the file can be loaded as a fixed windows, a variable windows or a repeat track. If all the fields are specified the file can be loaded as a gene track.

BedGraph format

Description

The BedGraph format is a really simple format useful to visualize windows on the genome. This windows can have a score. The fields in a BedGraph file are the followings:

  1. Chromosome
  2. Window start position
  3. Window stop position
  4. Score

Example

track type =bedgraph
chr1	18598	19673	1
chr1	124987	125426	3
chr1	317653	318092	15
chr1	427014	428027	8

Usage

BedGraph files can be used to load fixed windows and variable windows track.

They can also be loaded as stripes

GFF format

Description

GFF stands for General Feature Format. They have 9 compulsory fields which are tab delimited. They are as follows:

  1. seqname - The name of the sequence. Must be a chromosome.
  2. source - The source code that was responsible for generation of this feature.
  3. feature - The name of this type of feature.
  4. start - The starting position of the feature in the sequence. The first base are 1 indexed.
  5. end - The ending position of the feature (inclusive).
  6. score - A score between 0 and 1000.
  7. strand - Valid entries include '+', '-', or '.' (for don't know/don't care).
  8. frame – Is a number between 0 and 2, if the feature is a coding exon and if the feature is not a coding exon, it is ‘.’.
  9. group - Lines belonging to the same group are linked together into a single item.

More information about this format can be found on Sanger website

Usage

GFF files can be used to load repeats, variable and fixed windows track as well as stripes. A GFF file needs to have a gff extension or have the following first line '##GFF '

GTF format

GTF stands for Gene Transfer Format and is a stricter version of GFF. The first 8 GTF fields are the same as GFF. The group field has been expanded into a list of attributes. Each attribute is a type/value pair. Attributes must be terminated by a semi-colon. The gap between any two attributes must be exactly one space. The attribute list must begin with the two required attributes:

  • gene_id value - A GUID for the genomic source of the sequence
  • transcript_id value - A GUID for the predicted transcript

MAF format

Description

MAF stands for Multiple Alignment Format. It stores a sequence of numerous alignments in an easily parse-able and readable format. Each multiple alignment ends with a blank line. Each sequence in a alignment is on a single line with words delimited by white space. Comments begin with ‘#’ and lines beginning with ‘##’ contain meta-data.

The file is divided into paragraphs that terminate in a blank line. Within a paragraph, the first word of a line indicates its type. Each multiple alignment is in a separate paragraph that begins with an "a" line and contains an "s" line for each sequence in the multiple alignment. Some MAF files may contain other optional line types:

  • an "i" line containing information about what is in the aligned species DNA before and after the immediately preceding "s" line
  • an "e" line containing information about the size of the gap between the alignments that span the current block
  • a "q" line indicating the quality of each aligned base for the species

Example

Here is a simple example of a three alignment blocks derived from five starting sequences. The first track line is necessary for custom tracks, but should be removed otherwise. Repeats are shown as lowercase, and each block may have a subset of the input sequences. All sequence columns and rows must contain at least one nucleotide (no columns or rows that contain only insertions). 

track name=euArc visibility=pack
##maf version=1 scoring=tba.v8 
# tba.v8 (((human chimp) baboon) (mouse rat)) 
                  
a score=23262.0     
s hg18.chr7    27578828 38 + 158545518 AAA-GGGAATGTTAACCAAATGA---ATTGTCTCTTACGGTG
s panTro1.chr6 28741140 38 + 161576975 AAA-GGGAATGTTAACCAAATGA---ATTGTCTCTTACGGTG
s baboon         116834 38 +   4622798 AAA-GGGAATGTTAACCAAATGA---GTTGTCTCTTATGGTG
s mm4.chr6     53215344 38 + 151104725 -AATGGGAATGTTAAGCAAACGA---ATTGTCTCTCAGTGTG
s rn3.chr4     81344243 40 + 187371129 -AA-GGGGATGCTAAGCCAATGAGTTGTTGTCTCTCAATGTG
a score=5062.0                    
s hg18.chr7    27699739 6 + 158545518 TAAAGA
s panTro1.chr6 28862317 6 + 161576975 TAAAGA
s baboon         241163 6 +   4622798 TAAAGA 
s mm4.chr6     53303881 6 + 151104725 TAAAGA
s rn3.chr4     81444246 6 + 187371129 taagga
a score=6636.0
s hg18.chr7    27707221 13 + 158545518 gcagctgaaaaca
s panTro1.chr6 28869787 13 + 161576975 gcagctgaaaaca
s baboon         249182 13 +   4622798 gcagctgaaaaca
s mm4.chr6     53310102 13 + 151104725 ACAGCTGAAAATA

Microarray format

The datasets for the built-in microarray tracks in the Genome Browser are stored in BED15 format, an extension of BED format that includes three additional fields: expCount, expIds, and expScores. To display correctly in the Genome Browser, microarray tracks require the setting of several attributes in the trackDb file associated with the track's genome assembly. Each microarray track set must also have an associated microarrayGroups.ra configuration file that contains additional information about the data in each of the arrays. User-created microarray custom tracks are similar in format to BED custom tracks with the addition of three required track line parameters in the header--expNames, expScale, and expStep--that mimic the trackDb and microarrayGroups.ra settings of built-in microarray tracks.

PSL format

PSL lines represent alignments, and are typically taken from files generated by BLAT or psLayout. All of the following fields are required on each data line within a PSL file: 

1.	matches - Number of bases that match that aren’t repeats
2.	mismatches - Number of bases that don't match
3.	repeats - Number of bases that match but are part of repeats
4.	nbasecount - Number of 'N' bases
5.	numofinsertsquery - Number of inserts in query
6.	numofbaseinsertsquery - Number of bases inserted in query
7.	numofinsertstarget - Number of inserts in target
8.	numofbaseinsertsquery - Number of bases inserted in target
9.	strand - '+' or '-' for query strand. For translated alignments, second '+'or '-' is for genomic strand
10.	queryseqname - Query sequence name
11.	queryseqize - Query sequence size
12.	querystart - Alignment start position in query
13.	queryend - Alignment end position in query
14.	targetseqname - Target sequence name
15.	targetseqsize - Target sequence size
16.	targetstart - Alignment start position in target
17.	targetend - Alignment end position in target
18.	blockcount - Number of blocks in the alignment (a block contains no gaps)
19.	blocksizes - Comma-separated list of sizes of each block
20.	querystarts - Comma-separated list of starting positions of each block in query
21.	targetstarts - Comma-separated list of starting positions of each block in target

WIG format

Wiggle format (WIG) allows the display of continuous-valued data in a track format.

Retrieved from "http://genplay.net/wiki/index.php?title=GenPlay_File_Formats&oldid=188"