Intro

Data Security
Upload uses java skript and, in case you login via the public account, a cookie. The latter stores your 32-character session key which is only transmitted after a secure SSL connection has been established, thus never travelling the internet unencrypted. It serves to reliably prevent anyone from accessing your data. Like with any decent key, this will include yourself in case you loose it. Since cookies may get deleted, e.g. in the course of clearing your browsing history, it is advisable to store your key: Copy/paste your session key from here or find out how to view your browser's cookies, look for one set by bioinf.fz-borstel.de (Name: PHYRESSID) and copy-paste its content (i.e. your session key, looks like c29027d90d4fc80f589473695932a64c). Now you can also access your data from another browser and/or another computer by using your key here. This will create above cookie on the new computer/browser such that you only have to "unlock" once. On the other hand, like with any other key it should not be left lying around. To prevent others from using it, make shure to delete the cookie after accessing your data from a public computer. Data Expiry
NGS data are large and our storage capacities limited. We can hold your data only for a limited time. Effective expiry periods for data and sessions are displayed on the upload page. They may need to be adapted to available storage and recent workloads from time to time.

Handling

Before upload

• It is advisable not to trim the reads. If repetitions need to be pooled, one way to preserve the paired-end information is (given the read IDs are unique):

  zcat 123-13_r001_R1.fastq.gz 123-13_r002_R1.fastq.gz 123-13_r003_R1.fastq.gz | gzip > 123-13_r001to3_R1.fastq.gz 
  zcat 123-13_r001_R2.fastq.gz 123-13_r002_R2.fastq.gz 123-13_r003_R2.fastq.gz | gzip > 123-13_r001to3_R2.fastq.gz

• File extension(s) need to be preceeded by _R1, _R2 (as in example filenames above), or _R0 for single end reads. Only fastq and GNU-zipped (.fastq, fastq.gz) formats are supported (latter encouraged).
• Please also select your platform (Illumina or Ion Torrent), beforehand, because data stemming from Ion Torrent will not even upload (failing the FASTQ validation) as Illumina data (default). All the differences in data handling between the platforms are detailed here.

Upload
You can drag & drop files from your desktop on this webpage with Google Chrome, Mozilla Firefox and Apple Safari. Alternatively, will open a window for selecting local files. Upload uses the jQuery file upload plugin supporting multiple file selection, progress bars, chunked and resumable uploads. Maximum file size for uploads was set to 2.1 GB. Files of this size yield mean coverages of more than 1,000x. If for any reason you need to process larger files, please contact us.

Initiate uploading with the blue "start upload" button. When the waitbar almost reaches the right end, the system will take extra time checking the entire file content for proper fastq format. Do not switch off your computer nor your browser until all files are uploaded and checked.

Processing
Start processing all uploaded files by Process files (below ). Or automatically after upload by checking the little box beforehand (before starting upload of a multi-file selection). Steps such as mapping or variant calling take several minutes. Meanwhile, a progress page (updated every few seconds) shows all remaining steps in grey while linking already computed ones in black and red. Processing many large files may take hours to overnight (maybe days, hard to provide any estimate, depends on the traffic). Just drop by later to get an idea or in the morning to collect your results. It is safe both to close your browser and switch off your computer meanwhile (as long as you record your session key). Every result is computed only once, do not hesitate to klick Process files again after coming back.

Computational Steps

(Warning: For programmers only. In case you wonder whether Unix is a Peruvian freshwater toadfish you may find this part uninformative.)

Multiple-file upload by jQuery-File-Upload (server-side PHP handler), each file may be sized up to 2.1 GB.

File check by fastQValidator (using LibStatGen).

Sessions are mediated by cookies (encrypted transfer via HTTPS) using CGI::Session with ip-match switched on. In case this causes problems (if your IP changes during your browsing time), please let us know.

Expired data are deleted by a nightly cronjob starting at 00:01 CET.

Versions
jQuery JavaScript Library v1.9.1
jQuery UI - v1.10.3
libStatGen v1.0.10
FastQValidator v0.1.1
Blast v2.2.26
Ref. Genome: NC_000962.3