Choose computing environment¶
There are two ways to follow this tutorial:
A) Use the pangeo binder. This is a good option if you don't have access to a Unix/Linux computer.
Open the binder environment in a new tab (i.e., by typing Ctrl and clicking link). It will take a few minutes to build:
Click on the "Terminal" button to launch it:
Then, follow the set up instructions below starting at step 2. The Snakefile and necessary conda environment files are already installed in the binder.
When you close the binder, it does NOT save your work so download any files you want to keep.
To download files, right-click the file you want to save, and select "Download":
To close the binder, go to "File" and click "Shut Down":
We also have an Rstudio version of the binder environment. The only difference compared to the terminal binder environment is the layout of Rstudio panels. The template Snakefile and software are pre-loaded with the binder.
B) Use a Unix or Linux environment (e.g., with a Mac or High Performing Computing environment).
- For this option, please follow all the set up instructions below.
Set up computing environment¶
We will use conda to create an computer software for this tutorial. If you don't have conda installed, please see the Set up computing environment with conda on MacOS tutorial.
Please refer to the conda command cheat sheet for commonly used conda commands!
Step 1: Download tutorial files:¶
First, create a new directory for this tutorial in the
(base) conda environment, e.g.,:
Rename the "Snakefile.py" to "Snakefile". There should be no file extension (we just added it so you'd be able to download the file!).
If you want to use
wget to download them to a remote computer, these commands should work on most Linux systems:
wget https://training.nih-cfde.org/en/latest/General-Tools/Snakemake/snakemake_tutorial_docs/environment.yml wget https://training.nih-cfde.org/en/latest/General-Tools/Snakemake/snakemake_tutorial_docs/Snakefile.py mv Snakefile.py Snakefile
Step 2: Create new conda environment:¶
conda is a software installation system that we will not cover in any detail today, but it is how we install all of our software for this lesson! Please see our Introduction to Conda for more information.
environment.yml file tells conda 1) where to look for the software installations under "channels" and 2) what software to install under "dependencies". You can also specify specific software versions, otherwise conda will download the most up-to-date version. Here are the specifications we'll use for this tutorial, as described in
channels (where the software will be installed from):
dependencies (which software will be installed):
If you're using the pangeo binder, the steps below are already done for you; if you are on a laptop or other computer, you can install all the necessary software by running the following command:
conda env create -n snaketest -f environment.yml
This creates a new environment called
snaketest, which you can then activate.
Step 3: Activate conda environment:¶
To use the software you just installed, execute:
conda activate snaketest
Your terminal command prompt should now look like
(snaketest) $ instead of
(base) $. This indicates that we can use all of the software installed by conda for this lesson.
Step 4: Test that your environment is ready to go¶
You should have several software packages installed in your
snaketest environment now. Check it out!
samtools 1.10 Using htslib 1.10.2 Copyright (C) 2019 Genome Research Ltd.
If you get an error, the software installation may have failed. You can check the software that is installed in your conda environment:
To leave the conda environment, type:
Later in the tutorial, we'll use
wget to download data. Installing
wget on MacOS can be achieved with
conda install. This step will take a few minutes and the installation should be done in the
base conda environment:
Go back to base environment:
conda install -c conda-forge wget
GNU Wget 1.20.1 built on darwin14.5.0. -cares +digest -gpgme +https +ipv6 -iri +large-file -metalink -nls +ntlm +opie -psl +ssl/openssl Wgetrc: /Users/amanda/miniconda3/envs/snaketest/etc/wgetrc (system) Compile: x86_64-apple-darwin13.4.0-clang -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/Users/amanda/miniconda3/envs/snaketest/etc/wgetrc" -DLOCALEDIR="/Users/amanda/miniconda3/envs/snaketest/share/locale" -I. -I../lib -I../lib -D_FORTIFY_SOURCE=2 -mmacosx-version-min=10.9 -I/Users/amanda/miniconda3/envs/snaketest/include -DHAVE_LIBSSL -I/Users/amanda/miniconda3/envs/snaketest/include -DNDEBUG -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -I/Users/amanda/miniconda3/envs/snaketest/include -fdebug-prefix-map=/opt/concourse/worker/volumes/live/c2196a4d-0070-4d98-71cc-9baa5fb53ac9/volume/wget_1551978005217/ work=/usr/local/src/conda/wget-1.20.1 -fdebug-prefix-map=/Users/amanda/miniconda3/envs/snaketest=/usr/local/src/conda-prefix Link: x86_64-apple-darwin13.4.0-clang -I/Users/amanda/miniconda3/envs/snaketest/include -DHAVE_LIBSSL -I/Users/amanda/miniconda3/envs/snaketest/include -DNDEBUG -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -I/Users/amanda/miniconda3/envs/snaketest/include -fdebug-prefix-map=/opt/concourse/worker/volumes/live/c2196a4d-0070-4d98-71cc-9baa5fb53ac9/volume/wget_1551978005217/work=/usr/local/src/conda/wget-1.20.1 -fdebug-prefix-map=/Users/amanda/miniconda3/envs/snaketest=/usr/local/src/conda-prefix -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/amanda/miniconda3/envs/snaketest/lib -L/Users/amanda/miniconda3/envs/snaketest/lib -L/Users/amanda/miniconda3/envs/snaketest/lib -lssl -lcrypto -L/Users/amanda/miniconda3/envs/snaketest/lib -lz ftp-opie.o openssl.o http-ntlm.o ../lib/libgnu.a -lcrypto Copyright (C) 2015 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://www.gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Originally written by Hrvoje Niksic <[email protected]>. Please send bug reports and questions to <[email protected]>.