go to the NCBI SRA database site, click the record you are interested in and then click the tab ‘data access’ and copy-paste the https URL into a text file
place this text file of your SRA URLs into the directory where you wish to download the data (no spaces in the list or any trailing characters, etc)
enter the command
for i in `cat srr.list` ; do wget $i ; done
et voila, go have a snack
When I’m not sure which adaptors were used for the construction of a sequencing library, but I know they were Illumina, I take the top ~100k reads and run trimmomatic using more-or-less default settings against the 2 different Illumina truseq adaptors that ‘ship’ with that software. Then compare how the two trim-logs look — the one that trimmed off more crap is the one to proceed with. Something like this (these are set for small RNA libraries where the inserts are very short!):
java -jar trimmomatic-0.36.jar SE -threads 5 -trimlog trimlog_test test.fastq out.test.fastq ILLUMINACLIP:TruSeq2-SE.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15 MINLEN:1
awk ‘$6>0 {print}’ trimlog_test | wc
when my lab departed for the John Innes Centre about a year ago, there was a large quantity of raw computer parts left behind for the salvage heap. I wanted a dedicated place to back up my laptop and home computers remotely, and had heard about freely-available network attached storage (NAS) systems, but that was where my knowledge ended. Our lab had been using the somewhat pricey synology NAS system, and I wanted to learn if there was an open source alternative.
FreeBSD-based FreeNAS fit the bill. There is a TON of info and support for how FreeNAS works and how to get it going. What I envisioned was taking some old HDDs laying around, reformatting them, and using them as the storage core of the system. I would boot to a usb drive on which I’ve installed FreeNAS, and format things from there. The amazing thing is… it basically worked and now I have a server for a lot of previously unsecure stuff. Booting off of a USB loaded with the OS is actually the recommended way to go: see this simple instructional video from FreeNAS
One detail that is worth noting is that FreeNAS storage is memory intensive. I am not 100% clear on why, but I believe it has to do with the ZFS-based architecture (NB: zfs does not stand for anything!). For each TB of hard drive, they recommend 1GB of RAM, starting with at least 4GB RAM as a base. Luckily I had a bunch of DDR3 RAM laying around and a capable Asus X99 motherboard with an i7 cpu. Once I got the thing running, it had 64GB RAM supporting 8TB of hard storage; I’ve gone about 2TB into this thing after about a year.
this makes it possible to remotely view your data on any integrated genome viewer (IGV) session with an internet connection. Make sure your computer’s web server enabled — for me on ubuntu 14 and 16, it meant installing the so-called LAMP stack (I used the slick little installer package called tacksel as outlined here)
<?xml version="1.0" encoding="UTF-8"?>
<Global name="bsseq_sex_cells" infolink="http://ip_addr/igv/" version="1">
<Category name="bsseq_sperm">
<Category name="wt.sp.f4">
<Resource name="wt.sp.CG.f4"
color="0,100,0"
path="http://ip_addr/igvdata/bsseq.sperm/wt.f4.sp.all.cg.gff.clean.CG-methyl.tdf" />
<Resource name="wt.sp.CHG.f4"
color="0,100,0"
path="http://ip_addr/igvdata/bsseq.sperm/wt.f4.sp.all.cg.gff.clean.CHG-methyl.tdf" />
<Resource name="wt.sp.CHH.f4"
color="0,100,0"
path="http://ip_addr/igvdata/bsseq.sperm/wt.f4.sp.all.cg.gff.clean.CHH-methyl.tdf" />
</Category>
here I have specified the “global name” as bsseq_sex_cells, so this will be an expandable menu item in IGV, with the CG, CHG, and CHH tracks available as items within that menu.
As you can see above in the xml snippet, under ‘Resource’ the path shows my tracks for this set of experiments are all within a sub-directory in ./igvdata called bsseq.sperm — it seemed a good way for me to organize my data. Also cool is that each sub-directory corresponds to a different menu heading, like so
6. to enable this, set your IGV session to look for your server at your IP address:
if you want to split your file on unique identifiers in say col. 2, and your identifiers are, for ex., CGA, CGC, CGG, and CGT, running this one-liner will generate 4 files, named as your identifiers:
awk ‘{print >> $2 }’ input (don’t specify output)
perhaps I would have finished my last paper a month earlier if I had known about this simple trick: when running a simple for loop in bash, take advantage of a multicore processor and use an “&” rather than “;” to close the loop. for instance:
just discovered bashhub – after years of using the built-in tool ‘history’ then piping through grep to find past commands. I don’t get all the options available, but the basic bashhub features allow you to do this sort of search in one step, or enter interactive mode (sorta like htop but for your command history).