UNIX commands are like sentences that can be very simple or complex. The simplest commands consist of only the command name. Many require the name of a file or directory and allow specially formatted arguments, known as flags or options, which modify the default behavior of the program. The grammar of a shell allows you to combine existing tools into powerful pipelines and handle large volumes of data automatically. Sequences of commands can be written into a script, improving the reproducibility of workflows. The ease of getting things done via the shell will increase with your exposure to the program.
We should note that folders are called directories at the command line. For all intents and purposes, they can be used interchangeably, but if you would like more information please read about "the folder metaphor".
This Binder comes preloaded with data provided by your instructors. If you want to do these exercises locally, on your own computer, you can download the data here.
ls are two simple commands that can be used to answer the two commonly asked questions "where am I?" and "what files are here?". We will use these frequently through the next sections.
To answer the question "where am I?", we can use the print working directory or
pwd command to see what directory we are currently located in.
This will print the absolute path to the directory where we are located. An absolute path shows the complete series of directories you need to locate either a directory or a file starting from the root directory of your computer. The absolute path to the root directory is
/. A useful way to start thinking about directories and files is through levels. At the highest level of your computer, you have the root directory. Everything that is contained in your computer is located in directories below your root directory.
The home directory is typically two levels down. For many Mac users, the home directory is
/Users/USERNAME. If you are using the Binder provided for this workshop, the home directory is
/home/jovyan. Because the absolute path to the home directory is different for every user, you can refer to the home directory with the tilde symbol
The list or
ls command is a simple yet powerful command that is used to list the contents of your computer. It can be executed with or without optional flags and directories or files. Let's look at the contents in our working directory by using the
We can see the following files:
books CFDE-logo.png GTEx images MiSeq README.md rstudio-console.png rstudio-terminal.png seattle
If we want more information about the files, such as the date they were created and their file size, we can add "flags"
-l for long listing format.
total 520 drwxr-xr-x 2 jovyan jovyan 4096 Jun 7 04:37 books -rw-r--r-- 1 jovyan jovyan 71154 Jun 7 04:36 CFDE-logo.png drwxr-xr-x 1 jovyan jovyan 4096 Jun 7 04:36 GTEx drwxr-xr-x 2 jovyan jovyan 4096 Jun 7 04:37 images drwxr-xr-x 2 jovyan jovyan 4096 Jun 7 04:37 MiSeq -rw-r--r-- 1 jovyan jovyan 2679 Jun 7 04:37 README.md -rw-r--r-- 1 jovyan jovyan 239942 Jun 7 04:37 rstudio-console.png -rw-r--r-- 1 jovyan jovyan 188705 Jun 7 04:37 rstudio-terminal.png drwxr-xr-x 2 jovyan jovyan 4096 Jun 7 04:37 seattle
Flags (sometimes called options) allow us to finely control the behavior of the command. But how did we know to add
-l after ls? The
ls manual describes the command and all its options in detail. Like most commands, you can type the command followed
--help to view the manual in your terminal.
You can use multiple flags, wildcards, and specify directories to modify the behavior of a command. What does the command
ls do when used with the following
-aflag will list hidden files and directories.
-Fflag will class the file types by appending an identifier. This works best if there are directories present.
- We can combine
-aFto use both options.
Now we have seen how to list the contents of folders on our computers and what is located in the directory we are presently in. But some of the beauty of the shell is that we can perform activities in locations that we are not currently in. To do this we can either use an absolute path or a relative path. A relative path is the path to another directory from the one you are currently in. An absolute path starts from the root and ends in the appropriate subdirectory.
To move from one directory to the other, we use the
cd command to change directories. We can use the
ls commands to confirm that we did indeed change directories. Because you can change directories using either the relative or absolute path, there are multiple ways to successfully move up or down in the directory hierarchy.
Let's return to our home directory using the
cd command and a relative path, then print the working directory to confirm.
Let's practice using the
ls commands to explore files in different directories.
books/ is in our working directory, we can navigate there with a relative path. What files are in the
books directory and how large are they?
cd books/ pwd ls -lh
We can see the following files:
-rw-r--r-- 1 jovyan jovyan 171K Jan 18 21:13 Alice_in_wonderland.txt -rw-r--r-- 1 jovyan jovyan 789K Jan 18 21:13 A-tale-of-two-cities.txt -rw-r--r-- 1 jovyan jovyan 789K Jan 18 21:13 book.txt -rw-r--r-- 1 jovyan jovyan 282K Jan 18 21:13 PeterPan.txt -rw-r--r-- 1 jovyan jovyan 1.1K Jan 18 21:13 README.md -rw-r--r-- 1 jovyan jovyan 80K Jan 18 21:13 WizardOfOz.txt.gz -rw-r--r-- 1 jovyan jovyan 12M Jan 18 21:13 yeast.fasta
books, which of the following commands could Jovyan use to navigate to the
- No, MiSeq does not exist in the current working directory.
- No, MiSeq does not exist in the current working directory.
- Yes, MiSeq is in the home directory.
- Yes, this is the full path to MiSeq.
- Yes, MiSeq is in the directory one level above.
- No, MiSeq is not in the directory two levels above.
- No, MiSeq is not in the root directory.
Most, but not all of the files in the MiSeq directory are .fastq files. Which .fastq files are the largest? We can use the wildcard
* to list only files that end in .fastq. We can use the
-S option to sort by size.
cd ~/MiSeq pwd ls -lhS *.fastq
-rwxr-xr-x 1 jovyan jovyan 11M Jun 7 04:37 F3D2_S190_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 11M Jun 7 04:37 F3D2_S190_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 9.2M Jun 7 04:37 F3D147_S213_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 9.2M Jun 7 04:37 F3D147_S213_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 7.1M Jun 7 04:37 F3D149_S215_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 7.0M Jun 7 04:37 F3D149_S215_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 6.7M Jun 7 04:37 F3D148_S214_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 6.7M Jun 7 04:37 F3D148_S214_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 4.3M Jun 7 04:37 F3D6_S194_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 4.3M Jun 7 04:37 F3D6_S194_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 4.2M Jun 7 04:36 F3D0_S188_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 4.2M Jun 7 04:37 F3D0_S188_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 4.0M Jun 7 04:37 F3D145_S211_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 4.0M Jun 7 04:37 F3D145_S211_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 3.8M Jun 7 04:37 F3D9_S197_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 3.8M Jun 7 04:37 F3D9_S197_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 3.7M Jun 7 04:37 F3D3_S191_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 3.7M Jun 7 04:37 F3D3_S191_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 3.2M Jun 7 04:37 F3D141_S207_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 3.2M Jun 7 04:37 F3D141_S207_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 3.2M Jun 7 04:37 F3D1_S189_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 3.2M Jun 7 04:37 F3D1_S189_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 3.0M Jun 7 04:37 F3D150_S216_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 3.0M Jun 7 04:37 F3D150_S216_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.9M Jun 7 04:37 F3D8_S196_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.9M Jun 7 04:37 F3D8_S196_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.8M Jun 7 04:37 F3D7_S195_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.8M Jun 7 04:37 F3D7_S195_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.7M Jun 7 04:37 F3D146_S212_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.7M Jun 7 04:37 F3D146_S212_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.6M Jun 7 04:37 F3D144_S210_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.6M Jun 7 04:37 F3D144_S210_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.6M Jun 7 04:37 Mock_S280_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.6M Jun 7 04:37 Mock_S280_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.4M Jun 7 04:37 F3D5_S193_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 2.4M Jun 7 04:37 F3D5_S193_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 1.8M Jun 7 04:37 F3D142_S208_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 1.8M Jun 7 04:37 F3D142_S208_L001_R2_001.fastq -rwxr-xr-x 1 jovyan jovyan 1.8M Jun 7 04:37 F3D143_S209_L001_R1_001.fastq -rwxr-xr-x 1 jovyan jovyan 1.7M Jun 7 04:37 F3D143_S209_L001_R2_001.fastq
||print name of current/working directory|
||list directory contents|
||change the working directory|
||current or working directory|
||directory one level up|