diff --git a/00-introduction.md b/00-introduction.md
new file mode 100644
index 000000000..e3bea9ba4
--- /dev/null
+++ b/00-introduction.md
@@ -0,0 +1,549 @@
+---
+title: Introducing R and RStudio IDE
+teaching: 30
+exercises: 15
+source: Rmd
+---
+
+::::::::::::::::::::::::::::::::::::::: objectives
+
+- Know advantages of analyzing data in R
+- Know advantages of using RStudio
+- Create an RStudio project, and know the benefits of working within a project
+- Be able to customize the RStudio layout
+- Be able to locate and change the current working directory with `getwd()` and `setwd()`
+- Compose an R script file containing comments and commands
+- Understand what an R function is
+- Locate help for an R function using `?`, `??`, and `args()`
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::::::::::::::::::: questions
+
+- Why use R?
+- Why use RStudio and how does it differ from R?
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+
+## Getting ready to use R for the first time
+
+In this lesson we will take you through the very first things you need to get
+R working.
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: This lesson works best on the cloud
+
+Remember, these lessons assume we are using the pre-configured virtual machine
+instances provided to you at a genomics workshop. Much of this work could be
+done on your laptop, but we use instances to simplify workshop setup
+requirements, and to get you familiar with using the cloud (a common
+requirement for working with big data).
+Visit the [Genomics Workshop setup page](https://www.datacarpentry.org/genomics-workshop/setup.html)
+for details on getting this instance running on your own, or for the info you
+need to do this on your own computer.
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## A Brief History of R
+
+[The programming language R](https://en.wikipedia.org/wiki/R_\(programming_language\)) has been around
+since 1995, and was created by Ross Ihaka and Robert Gentleman at the University
+of Auckland, New Zealand. R is based off the S programming language developed
+at Bell Labs and was developed to teach intro statistics. See this [slide deck](https://www.stat.auckland.ac.nz/~ihaka/downloads/Massey.pdf)
+by Ross Ihaka for more info on the subject.
+
+## Advantages of using R
+
+At more than 20 years old, R is fairly mature and [growing in popularity](https://www.tiobe.com/tiobe-index/r/). However, programming isn't a popularity contest. Here are key advantages of
+analyzing data in R:
+
+- **R is [open source](https://en.wikipedia.org/wiki/Open-source_software)**.
+ This means R is free - an advantage if you are at an institution where you
+ have to pay for your own MATLAB or SAS license. Open source, is important to
+ your colleagues in parts of the world where expensive software in
+ inaccessible. It also means that R is actively developed by a community (see
+ [r-project.org](https://www.r-project.org/)),
+ and there are regular updates.
+- **R is widely used**. Ok, maybe programming is a popularity contest. Because,
+ R is used in many areas (not just bioinformatics), you are more likely to
+ find help online when you need it. Chances are, almost any error message you
+ run into, someone else has already experienced.
+- **R is powerful**. R runs on multiple platforms (Windows/MacOS/Linux). It can
+ work with much larger datasets than popular spreadsheet programs like
+ Microsoft Excel, and because of its scripting capabilities is far more
+ reproducible. Also, there are thousands of available software packages for
+ science, including genomics and other areas of life science.
+
+:::::::::::::::::::::::::::::::::::::: discussion
+
+## Discussion: Your experience
+
+What has motivated you to learn R? Have you had a research question for which
+spreadsheet programs such as Excel have proven difficult to use, or where the
+size of the data set created issues?
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Introducing RStudio Server
+
+In these lessons, we will be making use of a software called [RStudio](https://www.rstudio.com/products/RStudio/),
+an [Integrated Development Environment (IDE)](https://en.wikipedia.org/wiki/Integrated_development_environment).
+RStudio, like most IDEs, provides a graphical interface to R, making it more
+user-friendly, and providing dozens of useful features. We will introduce
+additional benefits of using RStudio as you cover the lessons. In this case,
+we are specifically using [RStudio Server](https://www.rstudio.com/products/RStudio/#Server),
+a version of RStudio that can be accessed in your web browser. RStudio Server
+has the same features of the Desktop version of RStudio you could download as
+standalone software.
+
+## Log on to RStudio Server
+
+Open a web browser and enter the URL you used to log in at the terminal
+(provided by your instructors), followed by
+`:8787`. For example, if your URL was ec2.12.2.45.678.compute-1.amazonaws.com, you should enter:
+
+```
+http://ec2.12.2.45.678.compute-1.amazonaws.com:8787
+```
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Make sure there are no spaces before or after your URL.
+
+If your URL has spaces, your web browser may interpret it as a search query.
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+You should now be looking at a page that will allow you to login to the RStudio
+server:
+
+
+
+Enter your user credentials and click Sign In. The credentials for
+the genomics Data Carpentry instances will be provided by your instructors.
+
+You should now see the RStudio interface:
+
+
+
+## Create an RStudio project
+
+One of the first benefits we will take advantage of in RStudio is something
+called an **RStudio Project**. An RStudio project allows you to more easily:
+
+- Save data, files, variables, packages, etc. related to a specific
+ analysis project
+- Restart work where you left off
+- Collaborate, especially if you are using version control such as [git](https://swcarpentry.github.io/git-novice/).
+
+1. To create a project, go to the File menu, and click New Project....
+
+
+
+2. In the window that opens select **New Directory**, then **New Project**. For
+ "Directory name:" enter **dc\_genomics\_r**. For "Create project as subdirectory of", click Browse... and then click Choose which will select your home directory "~".
+
+3. Finally click Create Project. In the "Files" tab of your output
+ pane (more about the RStudio layout in a moment), you should see an RStudio
+ project file, **dc\_genomics\_r.Rproj**. All RStudio projects end with the
+ "**.Rproj**" file extension.
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Make your project more reproducible with renv
+
+One of the most wonderful and also frustrating aspects of working with R is
+managing packages. We will talk more about them, but packages (e.g. ggplot2)
+are add-ons that extend what you can do with R. Unfortunately it is very
+common that you may run into versions of R and/or R packages that are not
+compatible. This may make it difficult for someone to run your R script using
+their version of R or a given R package, and/or make it more difficult to run
+their scripts on your machine. [renv](https://rstudio.github.io/renv/)
+is an RStudio add-on that will associate your packages and project so that
+your work is more portable and reproducible. To turn on renv click on
+the Tools menu and select Project Options. Under
+**Enviornments** check off "**Use renv with this project**" and follow any
+installation instructions.
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Creating your first R script
+
+Now that we are ready to start exploring R, we will want to keep a record of the
+commands we are using. To do this we can create an R script:
+
+Click the File menu and select New File and then
+R Script. Before we go any further, save your script by clicking the
+save/disk icon that is in the bar above the first line in the script editor, or
+click the File menu and select save. In the "Save File"
+window that opens, name your file **"genomics\_r\_basics"**. The new script
+**genomics\_r\_basics.R** should appear under "files" in the output pane. By
+convention, R scripts end with the file extension **.R**.
+
+## Overview and customization of the RStudio layout
+
+Here are the major windows (or panes) of the RStudio
+environment:
+
+
+
+- **Source**: This pane is where you will write/view R scripts. Some outputs
+ (such as if you view a dataset using `View()`) will appear as a tab here.
+- **Console/Terminal/Jobs**: This is actually where you see the execution of
+ commands. This is the same display you would see if you were using R at the
+ command line without RStudio. You can work interactively (i.e. enter R
+ commands here), but for the most part we will run a script (or lines in a
+ script) in the source pane and watch their execution and output here. The
+ "Terminal" tab give you access to the BASH terminal (the Linux operating
+ system, unrelated to R). RStudio also allows you to run jobs (analyses) in the background. This is useful if some analysis will take a while to run. You can see the status of those jobs in the background.
+- **Environment/History**: Here, RStudio will show you what datasets and
+ objects (variables) you have created and which are defined in memory.
+ You can also see some properties of objects/datasets such as their type
+ and dimensions. The "History" tab contains a history of the R commands you've executed R.
+- **Files/Plots/Packages/Help/Viewer**: This multipurpose pane will show you the
+ contents of directories on your computer. You can also use the "Files" tab to
+ navigate and set the working directory. The "Plots" tab will show the output
+ of any plots generated. In "Packages" you will see what packages are actively
+ loaded, or you can attach installed packages. "Help" will display help files
+ for R functions and packages. "Viewer" will allow you to view local web
+ content (e.g. HTML outputs).
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Uploads and downloads in the cloud
+
+In the "Files" tab you can select a file and download it from your cloud
+instance (click the "more" button) to your local computer.
+Uploads are also possible.
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+All of the panes in RStudio have configuration options. For example, you can
+minimize/maximize a pane, or by moving your mouse in the space between
+panes you can resize as needed. The most important customization options for
+pane layout are in the View menu. Other options such as font sizes,
+colors/themes, and more are in the Tools menu under
+Global Options.
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## You are working with R
+
+Although we won't be working with R at the terminal, there are lots of reasons
+to. For example, once you have written an RScript, you can run it at any Linux
+or Windows terminal without the need to start up RStudio. We don't want
+you to get confused - RStudio runs R, but R is not RStudio. For more on
+running an R Script at the terminal see this [Software Carpentry lesson](https://swcarpentry.github.io/r-novice-inflammation/05-cmdline/).
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Getting to work with R: navigating directories
+
+Now that we have covered the more aesthetic aspects of RStudio, we can get to
+work using some commands. We will write, execute, and save the commands we
+learn in our **genomics\_r\_basics.R** script that is loaded in the Source pane.
+First, lets see what directory we are in. To do so, type the following command
+into the script:
+
+
+```r
+getwd()
+```
+
+To execute this command, make sure your cursor is on the same line the command
+is written. Then click the Run button that is just above the first
+line of your script in the header of the Source pane.
+
+In the console, we expect to see the following output\*:
+
+```output
+[1] "/home/dcuser/dc_genomics_r"
+```
+
+\* Notice, at the Console, you will also see the instruction you executed
+above the output in blue.
+
+Since we will be learning several commands, we may already want to keep some
+short notes in our script to explain the purpose of the command. Entering a `#`
+before any line in an R script turns that line into a comment, which R will
+not try to interpret as code. Edit your script to include a comment on the
+purpose of commands you are learning, e.g.:
+
+
+```r
+# this command shows the current working directory
+getwd()
+```
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise: Work interactively in R
+
+What happens when you try to enter the `getwd()` command in the Console pane?
+
+::::::::::::::: solution
+
+## Solution
+
+You will get the same output you did as when you ran `getwd()` from the
+source. You can run any command in the Console, however, executing it from
+the source script will make it easier for us to record what we have done,
+and ultimately run an entire script, instead of entering commands one-by-one.
+
+
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+For the purposes of this exercise we want you to be in the directory `"/home/dcuser/R_data"`.
+What if you weren't? You can set your home directory using the `setwd()`
+command. Enter this command in your script, but *don't run* this yet.
+
+
+```r
+# This sets the working directory
+setwd()
+```
+
+You may have guessed, you need to tell the `setwd()` command
+what directory you want to set as your working directory. To do so, inside of
+the parentheses, open a set of quotes. Inside the quotes enter a `/` which is
+the root directory for Linux. Next, use the Tab key, to take
+advantage of RStudio's Tab-autocompletion method, to select `home`, `dcuser`,
+and `dc_genomics_r` directory. The path in your script should look like this:
+
+
+```r
+# This sets the working directory
+setwd("/home/dcuser/dc_genomics_r")
+```
+
+When you run this command, the console repeats the command, but gives you no
+output. Instead, you see the blank R prompt: `>`. Congratulations! Although it
+seems small, knowing what your working directory is and being able to set your
+working directory is the first step to analyzing your data.
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Never use `setwd()`
+
+Wait, what was the last 2 minutes about? Well, setting your working directory
+is something you need to do, you need to be very careful about using this as
+a step in your script. For example, what if your script is being on a computer
+that has a different directory structure? The top-level path in a Unix file
+system is root `/`, but on Windows it is likely `C:\`. This is one of several
+ways you might cause a script to break because a file path is configured
+differently than your script anticipates. R packages like ['here'](https://cran.r-project.org/package=here)
+and ['file.path'](https://www.rdocumentation.org/packages/base/versions/3.4.3/topics/file.path)
+allow you to specify file paths in a way that is more operating system
+independent. See Jenny Bryan's [blog post](https://www.tidyverse.org/articles/2017/12/workflow-vs-script/) for this
+and other R tips.
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Using functions in R, without needing to master them
+
+A function in R (or any computing language) is a short
+program that takes some input and returns some output. Functions may seem like an advanced topic (and they are), but you have already
+used at least one function in R. `getwd()` is a function! The next sections will help you understand what is happening in
+any R script.
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise: What do these functions do?
+
+Try the following functions by writing them in your script. See if you can
+guess what they do, and make sure to add comments to your script about your
+assumed purpose.
+
+- `dir()`
+- `sessionInfo()`
+- `date()`
+- `Sys.time()`
+
+::::::::::::::: solution
+
+## Solution
+
+- `dir()` # Lists files in the working directory
+- `sessionInfo()` # Gives the version of R and additional info including
+ on attached packages
+- `date()` # Gives the current date
+- `Sys.time()` # Gives the current time
+
+*Notice*: Commands are case sensitive!
+
+
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+You have hopefully noticed a pattern - an R
+function has three key properties:
+
+- Functions have a name (e.g. `dir`, `getwd`); note that functions are case
+ sensitive!
+- Following the name, functions have a pair of `()`
+- Inside the parentheses, a function may take 0 or more arguments
+
+An argument may be a specific input for your function and/or may modify the
+function's behavior. For example the function `round()` will round a number
+with a decimal:
+
+
+```r
+# This will round a number to the nearest integer
+round(3.14)
+```
+
+```{.output}
+[1] 3
+```
+
+## Getting help with function arguments
+
+What if you wanted to round to one significant digit? `round()` can
+do this, but you may first need to read the help to find out how. To see the help
+(In R sometimes also called a "vignette") enter a `?` in front of the function
+name:
+
+
+```r
+?round()
+```
+
+The "Help" tab will show you information (often, too much information). You
+will slowly learn how to read and make sense of help files. Checking the "Usage" or "Examples"
+headings is often a good place to look first. If you look under "Arguments," we
+also see what arguments we can pass to this function to modify its behavior.
+You can also see a function's argument using the `args()` function:
+
+
+```r
+args(round)
+```
+
+```{.output}
+function (x, digits = 0)
+NULL
+```
+
+`round()` takes two arguments, `x`, which is the number to be
+rounded, and a
+`digits` argument. The `=` sign indicates that a default (in this case 0) is
+already set. Since `x` is not set, `round()` requires we provide it, in contrast
+to `digits` where R will use the default value 0 unless you explicitly provide
+a different value. We can explicitly set the digits parameter when we call the
+function:
+
+
+```r
+round(3.14159, digits = 2)
+```
+
+```{.output}
+[1] 3.14
+```
+
+Or, R accepts what we call "positional arguments", if you pass a function
+arguments separated by commas, R assumes that they are in the order you saw
+when we used `args()`. In the case below that means that `x` is 3.14159 and
+digits is 2.
+
+
+```r
+round(3.14159, 2)
+```
+
+```{.output}
+[1] 3.14
+```
+
+Finally, what if you are using `?` to get help for a function in a package not installed on your system, such as when you are running a script which has dependencies.
+
+
+```r
+?geom_point()
+```
+
+will return an error:
+
+```error
+Error in .helpForCall(topicExpr, parent.frame()) :
+ no methods for ‘geom_point' and no documentation for it as a function
+```
+
+Use two question marks (i.e. `??geom_point()`) and R will return
+results from a search of the documentation for packages you have installed on your computer
+in the "Help" tab. Finally, if you think there
+should be a function, for example a statistical test, but you aren't
+sure what it is called in R, or what functions may be available, use
+the `help.search()` function.
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise: Searching for R functions
+
+Use `help.search()` to find R functions for the following statistical
+functions. Remember to put your search query in quotes inside the function's
+parentheses.
+
+- Chi-Squared test
+- Student t-test
+- mixed linear model
+
+::::::::::::::: solution
+
+## Solution
+
+While your search results may return several tests, we list a few you might
+find:
+
+- Chi-Squared test: `stats::Chisquare`
+- Student t-test: `stats::t.test`
+- mixed linear model: `stats::lm.glm`
+
+
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+We will discuss more on where to look for the libraries and packages that contain functions you want to use. For now, be aware that two important ones are [CRAN](https://cran.r-project.org/) - the main repository for R, and [Bioconductor](https://bioconductor.org/) - a popular repository for bioinformatics-related R packages.
+
+## RStudio contextual help
+
+Here is one last bonus we will mention about RStudio. It's difficult to
+remember all of the arguments and definitions associated with a given function.
+When you start typing the name of a function and hit the Tab key,
+RStudio will display functions and associated help:
+
+
+
+Once you type a function, hitting the Tab inside the parentheses
+will show you the function's arguments and provide additional help
+for each of these arguments.
+
+
+
+:::::::::::::::::::::::::::::::::::::::: keypoints
+
+- R is a powerful, popular open-source scripting language
+- You can customize the layout of RStudio, and use the project feature to manage the files and packages used in your analysis
+- RStudio allows you to run R in an easy-to-use interface and makes it easy to find help
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
diff --git a/01-r-basics.md b/01-r-basics.md
new file mode 100644
index 000000000..7c3495ad4
--- /dev/null
+++ b/01-r-basics.md
@@ -0,0 +1,1054 @@
+---
+title: R Basics
+teaching: 60
+exercises: 20
+source: Rmd
+---
+
+::::::::::::::::::::::::::::::::::::::: objectives
+
+- Be able to create the most common R objects including vectors
+- Understand that vectors have modes, which correspond to the type of data they contain
+- Be able to use arithmetic operators on R objects
+- Be able to retrieve (subset), name, or replace, values from a vector
+- Be able to use logical operators in a subsetting operation
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::::::::::::::::::: questions
+
+- What will these lessons not cover?
+- What are the basic features of the R language?
+- What are the most common objects in R?
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+
+## "The fantastic world of R awaits you" OR "Nobody wants to learn how to use R"
+
+Before we begin this lesson, we want you to be clear on the goal of the workshop
+and these lessons. We believe that every learner can **achieve competency
+with R**. You have reached competency when you find that you are able to
+**use R to handle common analysis challenges in a reasonable amount of time**
+(which includes time needed to look at learning materials, search for answers
+online, and ask colleagues for help). As you spend more time using R (there is
+no substitute for regular use and practice) you will find yourself gaining
+competency and even expertise. The more familiar you get, the more
+complex the analyses you will be able to carry out, with less frustration, and
+in less time - the fantastic world of R awaits you!
+
+## What these lessons will not teach you
+
+Nobody wants to learn how to use R. People want to learn how to use R to analyze
+their own research questions! Ok, maybe some folks learn R for R's sake, but
+these lessons assume that you want to start analyzing genomic data as soon as
+possible. Given this, there are many valuable pieces of information about R
+that we simply won't have time to cover. Hopefully, we will clear the hurdle of
+giving you just enough knowledge to be dangerous, which can be a high bar
+in R! We suggest you look into the additional learning materials in the tip box
+below.
+
+**Here are some R skills we will *not* cover in these lessons**
+
+- How to create and work with R matrices
+- How to create and work with loops and conditional statements, and the "apply"
+ family of functions (which are super useful, read [this blog post to learn more about these functions](https://www.r-bloggers.com/r-tutorial-on-the-apply-family-of-functions/))
+- How to do basic string manipulations (e.g. finding patterns in text using grep, replacing text)
+- How to plot using the default R graphic tools (we *will* cover plot creation, but will do so using the popular plotting package `ggplot2`)
+- How to use advanced R statistical functions
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Where to learn more
+
+The following are good resources for learning more about R. Some of them
+can be quite technical, but if you are a regular R user you may ultimately
+need this technical knowledge.
+
+- [R for Beginners](https://cran.r-project.org/doc/contrib/Paradis-rdebuts_en.pdf):
+ By Emmanuel Paradis and a great starting point
+- [The R Manuals](https://cran.r-project.org/manuals.html): Maintained by the
+ R project
+- [R contributed documentation](https://cran.r-project.org/other-docs.html):
+ Also linked to the R project; importantly there are materials available in
+ several languages
+- [R for Data Science](https://r4ds.had.co.nz/): A wonderful collection by
+ noted R educators and developers Garrett Grolemund and Hadley Wickham
+- [Practical Data Science for Stats](https://peerj.com/collections/50-practicaldatascistats/):
+ Not exclusively about R usage, but a nice collection of pre-prints on data science
+ and applications for R
+- [Programming in R Software Carpentry lesson](https://software-carpentry.org/lessons/):
+ There are several Software Carpentry lessons in R to choose from
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Creating objects in R
+
+:::::::::::::::::::::::::::::::::::::::::: prereq
+
+## Reminder
+
+At this point you should be coding along in the "**genomics\_r\_basics.R**"
+script we created in the last episode. Writing your commands in the script
+(and commenting it) will make it easier to record what you did and why.
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+What might be called a variable in many languages is called an **object**
+in R.
+
+**To create an object you need:**
+
+- a name (e.g. 'first_value')
+- a value (e.g. '1')
+- the assignment operator ('\<-')
+
+In your script, "**genomics\_r\_basics.R**", using the R assignment operator '\<-',
+assign '1' to the object 'first_value' as shown. Remember to leave a comment in the line
+above (using the '#') to explain what you are doing:
+
+
+```r
+# this line creates the object 'first_value' and assigns it the value '1'
+
+first_value <- 1
+```
+
+Next, run this line of code in your script. You can run a line of code
+by hitting the Run button that is just above the first line of your
+script in the header of the Source pane or you can use the appropriate shortcut:
+
+- Windows execution shortcut: Ctrl\+Enter
+- Mac execution shortcut: Cmd(⌘)\+Enter
+
+To run multiple lines of code, you can highlight all the line you wish to run
+and then hit Run or use the shortcut key combo listed above.
+
+In the RStudio 'Console' you should see:
+
+```output
+first_value <- 1
+>
+```
+
+The 'Console' will display lines of code run from a script and any outputs or
+status/warning/error messages (usually in red).
+
+In the 'Environment' window you will also get a table:
+
+| Values | |
+| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| first_value | 1 |
+
+The 'Environment' window allows you to keep track of the objects you have
+created in R.
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise: Create some objects in R
+
+Create the following objects; give each object an appropriate name
+(your best guess at what name to use is fine):
+
+1. Create an object that has the value of number of pairs of human chromosomes
+2. Create an object that has a value of your favorite gene name
+3. Create an object that has this URL as its value: "[ftp://ftp.ensemblgenomes.org/pub/bacteria/release-39/fasta/bacteria\_5\_collection/escherichia\_coli\_b\_str\_rel606/](ftp://ftp.ensemblgenomes.org/pub/bacteria/release-39/fasta/bacteria_5_collection/escherichia_coli_b_str_rel606/)"
+4. Create an object that has the value of the number of chromosomes in a
+ diploid human cell
+
+::::::::::::::: solution
+
+## Solution
+
+Here as some possible answers to the challenge:
+
+
+```r
+human_chr_number <- 23
+gene_name <- 'pten'
+ensemble_url <- 'ftp://ftp.ensemblgenomes.org/pub/bacteria/release-39/fasta/bacteria_5_collection/escherichia_coli_b_str_rel606/'
+human_diploid_chr_num <- 2 * human_chr_number
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Naming objects in R
+
+Here are some important details about naming objects in R.
+
+- **Avoid spaces and special characters**: Object names cannot contain spaces or the minus sign (`-`). You can use '\_' to make names more readable. You should avoid
+ using special characters in your object name (e.g. ! @ # . , etc.). Also,
+ object names cannot begin with a number.
+- **Use short, easy-to-understand names**: You should avoid naming your objects
+ using single letters (e.g. 'n', 'p', etc.). This is mostly to encourage you
+ to use names that would make sense to anyone reading your code (a colleague,
+ or even yourself a year from now). Also, avoiding excessively long names will
+ make your code more readable.
+- **Avoid commonly used names**: There are several names that may already have a
+ definition in the R language (e.g. 'mean', 'min', 'max'). One clue that a name
+ already has meaning is that if you start typing a name in RStudio and it gets
+ a colored highlight or RStudio gives you a suggested autocompletion you have
+ chosen a name that has a reserved meaning.
+- **Use the recommended assignment operator**: In R, we use '\<- ' as the
+ preferred assignment operator. '=' works too, but is most commonly used in
+ passing arguments to functions (more on functions later). There is a shortcut
+ for the R assignment operator:
+ - Windows execution shortcut: Alt\+\-
+ - Mac execution shortcut: Option\+\-
+
+There are a few more suggestions about naming and style you may want to learn
+more about as you write more R code. There are several "style guides" that
+have advice. One of the more widely used is the [tidyverse R style guide](https://style.tidyverse.org/index.html),
+but there is also a [Google R style guide](https://google.github.io/styleguide/Rguide.html), and
+[Jean Fan's R style guide](https://jef.works/R-style-guide/), among others.
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Pay attention to warnings in the script console
+
+If you enter a line of code in your script that contains an error, RStudio
+may give you an error message and underline this mistake. Sometimes these
+messages are easy to understand, but often the messages may need some figuring
+out. Paying attention to these warnings will help you avoid mistakes. In the example below, our object name has a space, which
+is not allowed in R. The error message does not say this directly,
+but R is "not sure"
+about how to assign the name to "human\_ chr\_number" when the object name we
+want is "human\_chr\_number".
+
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Reassigning object names or deleting objects
+
+Once an object has a value, you can change that value by overwriting it. R will
+not give you a warning or error if you overwriting an object, which
+may or may not be a good thing
+depending on how you look at it.
+
+
+```r
+# gene_name has the value 'pten' or whatever value you used in the challenge.
+# We will now assign the new value 'tp53'
+gene_name <- 'tp53'
+```
+
+You can also remove an object from R's memory entirely. The `rm()` function
+will delete the object.
+
+
+```r
+# delete the object 'gene_name'
+rm(gene_name)
+```
+
+If you run a line of code that has only an object name, R will normally display
+the contents of that object. In this case, we are told the object no
+longer exists.
+
+```error
+Error: object 'gene_name' not found
+```
+
+## Understanding object data types (modes)
+
+In R, **every object has two properties**:
+
+- **Length**: How many distinct values are held in that object
+- **Mode**: What is the classification (type) of that object.
+
+We will get to the "length" property later in the lesson. The **"mode" property**
+**corresponds to the type of data an object represents**. The most common modes
+you will encounter in R are:
+
+| Mode (abbreviation) | Type of data |
+| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Numeric (num) | Numbers such floating point/decimals (1.0, 0.5, 3.14), there are also more specific numeric types (dbl - Double, int - Integer). These differences are not relevant for most beginners and pertain to how these values are stored in memory |
+| Character (chr) | A sequence of letters/numbers in single '' or double " " quotes |
+| Logical | Boolean values - TRUE or FALSE |
+
+There are a few other modes (i.e. "complex", "raw" etc.) but these
+are the three we will work with in this lesson.
+
+Data types are familiar in many programming languages, but also in natural
+language where we refer to them as the parts of speech, e.g. nouns, verbs,
+adverbs, etc. Once you know if a word - perhaps an unfamiliar one - is a noun,
+you can probably guess you can count it and make it plural if there is more than
+one (e.g. 1 [Tuatara](https://en.wikipedia.org/wiki/Tuatara), or 2 Tuataras). If
+something is a adjective, you can usually change it into an adverb by adding
+"-ly" (e.g. [jejune](https://www.merriam-webster.com/dictionary/jejune) vs.
+jejunely). Depending on the context, you may need to decide if a word is in one
+category or another (e.g "cut" may be a noun when it's on your finger, or a verb
+when you are preparing vegetables). These concepts have important analogies when
+working with R objects.
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise: Create objects and check their modes
+
+Create the following objects in R, then use the `mode()` function to verify
+their modes. Try to guess what the mode will be before you look at the solution
+
+1. `chromosome_name <- 'chr02'`
+2. `od_600_value <- 0.47`
+3. `chr_position <- '1001701'`
+4. `spock <- TRUE`
+5. `pilot <- Earhart`
+
+::::::::::::::: solution
+
+## Solution
+
+
+```{.error}
+Error in eval(expr, envir, enclos): object 'Earhart' not found
+```
+
+
+```r
+mode(chromosome_name)
+```
+
+```{.output}
+[1] "character"
+```
+
+```r
+mode(od_600_value)
+```
+
+```{.output}
+[1] "numeric"
+```
+
+```r
+mode(chr_position)
+```
+
+```{.output}
+[1] "character"
+```
+
+```r
+mode(spock)
+```
+
+```{.output}
+[1] "logical"
+```
+
+```r
+mode(pilot)
+```
+
+```{.error}
+Error in eval(expr, envir, enclos): object 'pilot' not found
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+Notice from the solution that even if a series of numbers is given as a value
+R will consider them to be in the "character" mode if they are enclosed as
+single or double quotes. Also, notice that you cannot take a string of alphanumeric
+characters (e.g. Earhart) and assign as a value for an object. In this case,
+R looks for an object named `Earhart` but since there is no object, no assignment can
+be made. If `Earhart` did exist, then the mode of `pilot` would be whatever
+the mode of `Earhart` was originally. If we want to create an object
+called `pilot` that was the **name** "Earhart", we need to enclose
+`Earhart` in quotation marks.
+
+
+```r
+pilot <- "Earhart"
+mode(pilot)
+```
+
+```{.output}
+[1] "character"
+```
+
+## Mathematical and functional operations on objects
+
+Once an object exists (which by definition also means it has a mode), R can
+appropriately manipulate that object. For example, objects of the numeric modes
+can be added, multiplied, divided, etc. R provides several mathematical
+(arithmetic) operators including:
+
+| Operator | Description |
+| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| \+ | addition |
+| \- | subtraction |
+| \* | multiplication |
+| / | division |
+| ^ or \*\* | exponentiation |
+| a%/%b | integer division (division where the remainder is discarded) |
+| a%%b | modulus (returns the remainder after division) |
+
+These can be used with literal numbers:
+
+
+```r
+(1 + (5 ** 0.5))/2
+```
+
+```{.output}
+[1] 1.618034
+```
+
+and importantly, can be used on any object that evaluates to (i.e. interpreted
+by R) a numeric object:
+
+
+
+
+```r
+# multiply the object 'human_chr_number' by 2
+
+human_chr_number * 2
+```
+
+```{.output}
+[1] 46
+```
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise: Compute the golden ratio
+
+One approximation of the golden ratio (φ) can be found by taking the sum of 1
+and the square root of 5, and dividing by 2 as in the example above. Compute
+the golden ratio to 3 digits of precision using the `sqrt()` and `round()`
+functions. Hint: remember the `round()` function can take 2 arguments.
+
+::::::::::::::: solution
+
+## Solution
+
+
+```r
+round((1 + sqrt(5))/2, digits = 3)
+```
+
+```{.output}
+[1] 1.618
+```
+
+Notice that you can place one function inside of another.
+
+
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Vectors
+
+Vectors are probably the
+most used commonly used object type in R.
+**A vector is a collection of values that are all of the same type (numbers, characters, etc.)**.
+One of the most common
+ways to create a vector is to use the `c()` function - the "concatenate" or
+"combine" function. Inside the function you may enter one or more values; for
+multiple values, separate each value with a comma:
+
+
+```r
+# Create the SNP gene name vector
+
+snp_genes <- c("OXTR", "ACTN3", "AR", "OPRM1")
+```
+
+Vectors always have a **mode** and a **length**.
+You can check these with the `mode()` and `length()` functions respectively.
+Another useful function that gives both of these pieces of information is the
+`str()` (structure) function.
+
+
+```r
+# Check the mode, length, and structure of 'snp_genes'
+mode(snp_genes)
+```
+
+```{.output}
+[1] "character"
+```
+
+```r
+length(snp_genes)
+```
+
+```{.output}
+[1] 4
+```
+
+```r
+str(snp_genes)
+```
+
+```{.output}
+ chr [1:4] "OXTR" "ACTN3" "AR" "OPRM1"
+```
+
+Vectors are quite important in R. Another data type that we will
+work with later in this lesson, data frames, are collections of
+vectors. What we learn here about vectors will pay off even more
+when we start working with data frames.
+
+## Creating and subsetting vectors
+
+Let's create a few more vectors to play around with:
+
+
+```r
+# Some interesting human SNPs
+# while accuracy is important, typos in the data won't hurt you here
+
+snps <- c("rs53576", "rs1815739", "rs6152", "rs1799971")
+snp_chromosomes <- c("3", "11", "X", "6")
+snp_positions <- c(8762685, 66560624, 67545785, 154039662)
+```
+
+Once we have vectors, one thing we may want to do is specifically retrieve one
+or more values from our vector. To do so, we use **bracket notation**. We type
+the name of the vector followed by square brackets. In those square brackets
+we place the index (e.g. a number) in that bracket as follows:
+
+
+```r
+# get the 3rd value in the snp vector
+snps[3]
+```
+
+```{.output}
+[1] "rs6152"
+```
+
+In R, every item your vector is indexed, starting from the first item (1)
+through to the final number of items in your vector. You can also retrieve a
+range of numbers:
+
+
+```r
+# get the 1st through 3rd value in the snp vector
+
+snps[1:3]
+```
+
+```{.output}
+[1] "rs53576" "rs1815739" "rs6152"
+```
+
+If you want to retrieve several (but not necessarily sequential) items from
+a vector, you pass a **vector of indices**; a vector that has the numbered
+positions you wish to retrieve.
+
+
+```r
+# get the 1st, 3rd, and 4th value in the snp vector
+
+snps[c(1, 3, 4)]
+```
+
+```{.output}
+[1] "rs53576" "rs6152" "rs1799971"
+```
+
+There are additional (and perhaps less commonly used) ways of subsetting a
+vector (see [these
+examples](https://thomasleeper.com/Rcourse/Tutorials/vectorindexing.html)).
+Also, several of these subsetting expressions can be combined:
+
+
+```r
+# get the 1st through the 3rd value, and 4th value in the snp vector
+# yes, this is a little silly in a vector of only 4 values.
+snps[c(1:3,4)]
+```
+
+```{.output}
+[1] "rs53576" "rs1815739" "rs6152" "rs1799971"
+```
+
+## Adding to, removing, or replacing values in existing vectors
+
+Once you have an existing vector, you may want to add a new item to it. To do
+so, you can use the `c()` function again to add your new value:
+
+
+```r
+# add the gene "CYP1A1" and "APOA5" to our list of snp genes
+# this overwrites our existing vector
+snp_genes <- c(snp_genes, "CYP1A1", "APOA5")
+```
+
+We can verify that "snp\_genes" contains the new gene entry
+
+
+```r
+snp_genes
+```
+
+```{.output}
+[1] "OXTR" "ACTN3" "AR" "OPRM1" "CYP1A1" "APOA5"
+```
+
+Using a negative index will return a version of a vector with that index's
+value removed:
+
+
+```r
+snp_genes[-6]
+```
+
+```{.output}
+[1] "OXTR" "ACTN3" "AR" "OPRM1" "CYP1A1"
+```
+
+We can remove that value from our vector by overwriting it with this expression:
+
+
+```r
+snp_genes <- snp_genes[-6]
+snp_genes
+```
+
+```{.output}
+[1] "OXTR" "ACTN3" "AR" "OPRM1" "CYP1A1"
+```
+
+We can also explicitly rename or add a value to our index using double bracket notation:
+
+
+```r
+snp_genes[6]<- "APOA5"
+snp_genes
+```
+
+```{.output}
+[1] "OXTR" "ACTN3" "AR" "OPRM1" "CYP1A1" "APOA5"
+```
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise: Examining and subsetting vectors
+
+Answer the following questions to test your knowledge of vectors
+
+Which of the following are true of vectors in R?
+A) All vectors have a mode **or** a length
+B) All vectors have a mode **and** a length
+C) Vectors may have different lengths
+D) Items within a vector may be of different modes
+E) You can use the `c()` to add one or more items to an existing vector
+F) You can use the `c()` to add a vector to an existing vector
+
+::::::::::::::: solution
+
+## Solution
+
+A) False - Vectors have both of these properties
+B) True
+C) True
+D) False - Vectors have only one mode (e.g. numeric, character); all items in
+a vector must be of this mode.
+E) True
+F) True
+
+
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Logical Subsetting
+
+There is one last set of cool subsetting capabilities we want to introduce. It is possible within R to retrieve items in a vector based on a logical evaluation or numerical comparison. For example, let's say we wanted get all of the SNPs in our vector of SNP positions that were greater than 100,000,000. We could index using the '>' (greater than) logical operator:
+
+
+```r
+snp_positions[snp_positions > 100000000]
+```
+
+```{.output}
+[1] 154039662
+```
+
+In the square brackets you place the name of the vector followed by the comparison operator and (in this case) a numeric value. Some of the most common logical operators you will use in R are:
+
+| Operator | Description |
+| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| \< | less than |
+| \<= | less than or equal to |
+| \> | greater than |
+| \>= | greater than or equal to |
+| \== | exactly equal to |
+| != | not equal to |
+| !x | not x |
+| a \| b | a or b |
+| a \& b | a and b |
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## The magic of programming
+
+The reason why the expression `snp_positions[snp_positions > 100000000]` works
+can be better understood if you examine what the expression "snp\_positions > 100000000"
+evaluates to:
+
+
+```r
+snp_positions > 100000000
+```
+
+```{.output}
+[1] FALSE FALSE FALSE TRUE
+```
+
+The output above is a logical vector, the 4th element of which is TRUE. When
+you pass a logical vector as an index, R will return the true values:
+
+
+```r
+snp_positions[c(FALSE, FALSE, FALSE, TRUE)]
+```
+
+```{.output}
+[1] 154039662
+```
+
+If you have never coded before, this type of situation starts to expose the
+"magic" of programming. We mentioned before that in the bracket notation you
+take your named vector followed by brackets which contain an index:
+**named\_vector[index]**. The "magic" is that the index needs to *evaluate to*
+a number. So, even if it does not appear to be an integer (e.g. 1, 2, 3), as
+long as R can evaluate it, we will get a result. That our expression
+`snp_positions[snp_positions > 100000000]` evaluates to a number can be seen
+in the following situation. If you wanted to know which **index** (1, 2, 3, or
+4\) in our vector of SNP positions was the one that was greater than
+100,000,000?
+
+We can use the `which()` function to return the indices of any item that
+evaluates as TRUE in our comparison:
+
+
+```r
+which(snp_positions > 100000000)
+```
+
+```{.output}
+[1] 4
+```
+
+**Why this is important**
+
+Often in programming we will not know what inputs
+and values will be used when our code is executed. Rather than put in a
+pre-determined value (e.g 100000000) we can use an object that can take on
+whatever value we need. So for example:
+
+
+```r
+snp_marker_cutoff <- 100000000
+snp_positions[snp_positions > snp_marker_cutoff]
+```
+
+```{.output}
+[1] 154039662
+```
+
+Ultimately, it's putting together flexible, reusable code like this that gets
+at the "magic" of programming!
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## A few final vector tricks
+
+Finally, there are a few other common retrieve or replace operations you may
+want to know about. First, you can check to see if any of the values of your
+vector are missing (i.e. are `NA`, that stands for `not avaliable`). Missing data will get a more detailed treatment later,
+but the `is.NA()` function will return a logical vector, with TRUE for any NA
+value:
+
+
+```r
+# current value of 'snp_genes':
+# chr [1:7] "OXTR" "ACTN3" "AR" "OPRM1" "CYP1A1" NA "APOA5"
+
+is.na(snp_genes)
+```
+
+```{.output}
+[1] FALSE FALSE FALSE FALSE FALSE FALSE
+```
+
+Sometimes, you may wish to find out if a specific value (or several values) is
+present a vector. You can do this using the comparison operator `%in%`, which
+will return TRUE for any value in your collection that is in
+the vector you are searching:
+
+
+```r
+# current value of 'snp_genes':
+# chr [1:7] "OXTR" "ACTN3" "AR" "OPRM1" "CYP1A1" NA "APOA5"
+
+# test to see if "ACTN3" or "APO5A" is in the snp_genes vector
+# if you are looking for more than one value, you must pass this as a vector
+
+c("ACTN3","APOA5") %in% snp_genes
+```
+
+```{.output}
+[1] TRUE TRUE
+```
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Review Exercise 1
+
+What data modes are the following vectors?
+a. `snps`
+b. `snp_chromosomes`
+c. `snp_positions`
+
+::::::::::::::: solution
+
+## Solution
+
+
+```r
+mode(snps)
+```
+
+```{.output}
+[1] "character"
+```
+
+```r
+mode(snp_chromosomes)
+```
+
+```{.output}
+[1] "character"
+```
+
+```r
+mode(snp_positions)
+```
+
+```{.output}
+[1] "numeric"
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Review Exercise 2
+
+Add the following values to the specified vectors:
+a. To the `snps` vector add: "rs662799"
+b. To the `snp_chromosomes` vector add: 11
+c. To the `snp_positions` vector add: 116792991
+
+::::::::::::::: solution
+
+## Solution
+
+
+```r
+snps <- c(snps, "rs662799")
+snps
+```
+
+```{.output}
+[1] "rs53576" "rs1815739" "rs6152" "rs1799971" "rs662799"
+```
+
+```r
+snp_chromosomes <- c(snp_chromosomes, "11") # did you use quotes?
+snp_chromosomes
+```
+
+```{.output}
+[1] "3" "11" "X" "6" "11"
+```
+
+```r
+snp_positions <- c(snp_positions, 116792991)
+snp_positions
+```
+
+```{.output}
+[1] 8762685 66560624 67545785 154039662 116792991
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Review Exercise 3
+
+Make the following change to the `snp_genes` vector:
+
+Hint: Your vector should look like this in 'Environment':
+`chr [1:7] "OXTR" "ACTN3" "AR" "OPRM1" "CYP1A1" NA "APOA5"`. If not
+recreate the vector by running this expression:
+`snp_genes <- c("OXTR", "ACTN3", "AR", "OPRM1", "CYP1A1", NA, "APOA5")`
+
+a. Create a new version of `snp_genes` that does not contain CYP1A1 and then
+b. Add 2 NA values to the end of `snp_genes`
+
+::::::::::::::: solution
+
+## Solution
+
+
+```r
+snp_genes <- snp_genes[-5]
+snp_genes <- c(snp_genes, NA, NA)
+snp_genes
+```
+
+```{.output}
+[1] "OXTR" "ACTN3" "AR" "OPRM1" "APOA5" NA NA
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Review Exercise 4
+
+Using indexing, create a new vector named `combined` that contains:
+
+- The the 1st value in `snp_genes`
+- The 1st value in `snps`
+- The 1st value in `snp_chromosomes`
+- The 1st value in `snp_positions`
+
+::::::::::::::: solution
+
+## Solution
+
+
+```r
+combined <- c(snp_genes[1], snps[1], snp_chromosomes[1], snp_positions[1])
+combined
+```
+
+```{.output}
+[1] "OXTR" "rs53576" "3" "8762685"
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Review Exercise 5
+
+What type of data is `combined`?
+
+::::::::::::::: solution
+
+## Solution
+
+
+```r
+typeof(combined)
+```
+
+```{.output}
+[1] "character"
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Lists
+
+Lists are quite useful in R, but we won't be using them in the genomics lessons.
+That said, you may come across lists in the way that some bioinformatics
+programs may store and/or return data to you. One of the key attributes of a
+list is that, unlike a vector, a list may contain data of more than one mode.
+Learn more about creating and using lists using this [nice
+tutorial](https://r4ds.had.co.nz/vectors.html#lists). In this one example, we will create
+a named list and show you how to retrieve items from the list.
+
+
+```r
+# Create a named list using the 'list' function and our SNP examples
+# Note, for easy reading we have placed each item in the list on a separate line
+# Nothing special about this, you can do this for any multiline commands
+# To run this command, make sure the entire command (all 4 lines) are highlighted
+# before running
+# Note also, as we are doing all this inside the list() function use of the
+# '=' sign is good style
+snp_data <- list(genes = snp_genes,
+ refference_snp = snps,
+ chromosome = snp_chromosomes,
+ position = snp_positions)
+# Examine the structure of the list
+str(snp_data)
+```
+
+```{.output}
+List of 4
+ $ genes : chr [1:7] "OXTR" "ACTN3" "AR" "OPRM1" ...
+ $ refference_snp: chr [1:5] "rs53576" "rs1815739" "rs6152" "rs1799971" ...
+ $ chromosome : chr [1:5] "3" "11" "X" "6" ...
+ $ position : num [1:5] 8.76e+06 6.66e+07 6.75e+07 1.54e+08 1.17e+08
+```
+
+To get all the values for the `position` object in the list, we use the `$` notation:
+
+
+```r
+# return all the values of position object
+
+snp_data$position
+```
+
+```{.output}
+[1] 8762685 66560624 67545785 154039662 116792991
+```
+
+To get the first value in the `position` object, use the `[]` notation to index:
+
+
+```r
+# return first value of the position object
+
+snp_data$position[1]
+```
+
+```{.output}
+[1] 8762685
+```
+:::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::::::::::::::::::: keypoints
+
+- Effectively using R is a journey of months or years. Still you don't have to be an expert to use R and you can start using and analyzing your data with with about a day's worth of training
+- It is important to understand how data are organized by R in a given object type and how the mode of that type (e.g. numeric, character, logical, etc.) will determine how R will operate on that data.
+- Working with vectors effectively prepares you for understanding how data are organized in R.
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
diff --git a/02-data-prelude.md b/02-data-prelude.md
new file mode 100644
index 000000000..6f4abb79d
--- /dev/null
+++ b/02-data-prelude.md
@@ -0,0 +1,173 @@
+---
+title: Introduction to the example dataset and file type
+teaching: 15
+exercises: 0
+source: Rmd
+---
+
+::::::::::::::::::::::::::::::::::::::: objectives
+
+- Know what the example dataset represents
+- Know the concepts of how VCF files are generated
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::::::::::::::::::: questions
+
+- What data are we using in the lesson?
+- What are VCF files?
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+
+## Preface
+
+The Intro to R and RStudio for Genomics is a part of the Genomics Data Carpentry lessons. In this lesson we will learn the necessary skill sets for R and RStudio and apply them directly to a real next-generation sequencing (NGS) data in the variant calling format (VCF) file type. Previous Genomics Data Carpentry lessons teach learners how to generate a VCF file from FASTQ files downloaded from NCBI Sequence Read Archive (SRA), so we won't cover that here. Instead, in this episode we will give a brief overview of the data and a what VCF file types are for those who wish to teach the Intro to R and RStudio for Genomics lesson independently of the Genomics Data Carpentry lessons.
+
+This dataset was selected for several reasons, including:
+
+- Simple, but iconic NGS-problem: Examine a population where we want to characterize changes in sequence *a priori*
+- Dataset publicly available - in this case through the NCBI SRA ([http://www.ncbi.nlm.nih.gov/sra](https://www.ncbi.nlm.nih.gov/sra))
+
+## Introduction to the dataset
+
+Microbes are ideal organisms for exploring 'Long-term Evolution Experiments' (LTEEs) - thousands of generations can be generated and stored in a way that would be virtually impossible for more complex eukaryotic systems. In [Tenaillon et al 2016](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4988878/), 12 populations of *Escherichia coli* were propagated for more than 50,000 generations in a glucose-limited minimal medium. This medium was supplemented with citrate which *E. coli* cannot metabolize in the aerobic conditions of the experiment. Sequencing of the populations at regular time points reveals that spontaneous citrate-using mutants (Cit+) appeared in a population of *E.coli* (designated Ara-3) at around 31,000 generations. It should be noted that spontaneous Cit+ mutants are extraordinarily rare - inability to metabolize citrate is one of the defining characters of the *E. coli* species. Eventually, Cit+ mutants became the dominant population as the experimental growth medium contained a high concentration of citrate relative to glucose. Around the same time that this mutation emerged, another phenotype become prominent in the Ara-3 population. Many *E. coli* began to develop excessive numbers of mutations, meaning they became hypermutable.
+
+Strains from generation 0 to generation 50,000 were sequenced, including ones that were both Cit+ and Cit- and hypermutable in later generations.
+
+For the purposes of this workshop we're going to be working with 3 of the sequence reads from this experiment.
+
+| SRA Run Number | Clone | Generation | Cit | Hypermutable | Read Length | Sequencing Depth |
+| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ------- | ------------ | ----------- | ---------------- |
+| SRR2589044 | REL2181A | 5,000 | Unknown | None | 150 | 60\.2 |
+| SRR2584863 | REL7179B | 15,000 | Unknown | None | 150 | 88 |
+| SRR2584866 | REL11365 | 50,000 | Cit+ | plus | 150 | 138\.3 |
+
+We want to be able to look at differences in mutation rates between hypermutable and non-hypermutable strains. We also want to analyze the sequences to figure out what changes occurred in genomes to make the strains Cit+. Ultimately, we will use R to answer the questions:
+
+- How many base pair changes are there between the Cit+ and Cit- strains?
+- What are the base pair changes between strains?
+
+## How VCF files are generated
+
+Publicly accessible sequencing files in FASTQ formats can be downloaded from NCBI SRA. However, at FASTQ files contain unaligned sequences of varying quality, and requires clean up and alignment steps for variants to be called from the reference genome.
+
+Five steps are taken to transform FASTQ files to variant calls contained in VCF files and at each step, specialized non-R based bioinformatics tools that are used:
+
+
+
+
+
+## How variant calls are stored in VCF files
+
+VCF files contain variants that were called against a reference genome. These files are slightly more complicated than regular tables you can open using programs like Excel and contain two sections: header and records.
+
+Below you will see the header (which describes the format), the time and date the file was created, the version of bcftools that was used, the command line parameters used, and some additional information:
+
+```
+##fileformat=VCFv4.2
+##FILTER=
+##bcftoolsVersion=1.8+htslib-1.8
+##bcftoolsCommand=mpileup -O b -o results/bcf/SRR2584866_raw.bcf -f data/ref_genome/ecoli_rel606.fasta results/bam/SRR2584866.aligned.sorted.bam
+##reference=file://data/ref_genome/ecoli_rel606.fasta
+##contig=
+##ALT=
+##INFO=
+##INFO=
+##INFO=
+##INFO=
+##INFO=
+##INFO=
+##INFO=
+##INFO=
+##INFO=
+##INFO=
+##FORMAT=
+##FORMAT=
+##INFO=
+##INFO=
+##INFO=
+##INFO=
+##INFO=
+##INFO=
+##bcftools_callVersion=1.8+htslib-1.8
+##bcftools_callCommand=call --ploidy 1 -m -v -o results/bcf/SRR2584866_variants.vcf results/bcf/SRR2584866_raw.bcf; Date=Tue Oct 9 18:48:10 2018
+```
+
+Followed by information on each of the variations observed:
+
+```
+#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT results/bam/SRR2584866.aligned.sorted.bam
+CP000819.1 1521 . C T 207 . DP=9;VDB=0.993024;SGB=-0.662043;MQSB=0.974597;MQ0F=0;AC=1;AN=1;DP4=0,0,4,5;MQ=60
+CP000819.1 1612 . A G 225 . DP=13;VDB=0.52194;SGB=-0.676189;MQSB=0.950952;MQ0F=0;AC=1;AN=1;DP4=0,0,6,5;MQ=60
+CP000819.1 9092 . A G 225 . DP=14;VDB=0.717543;SGB=-0.670168;MQSB=0.916482;MQ0F=0;AC=1;AN=1;DP4=0,0,7,3;MQ=60
+CP000819.1 9972 . T G 214 . DP=10;VDB=0.022095;SGB=-0.670168;MQSB=1;MQ0F=0;AC=1;AN=1;DP4=0,0,2,8;MQ=60 GT:PL
+CP000819.1 10563 . G A 225 . DP=11;VDB=0.958658;SGB=-0.670168;MQSB=0.952347;MQ0F=0;AC=1;AN=1;DP4=0,0,5,5;MQ=60
+CP000819.1 22257 . C T 127 . DP=5;VDB=0.0765947;SGB=-0.590765;MQSB=1;MQ0F=0;AC=1;AN=1;DP4=0,0,2,3;MQ=60 GT:PL
+CP000819.1 38971 . A G 225 . DP=14;VDB=0.872139;SGB=-0.680642;MQSB=1;MQ0F=0;AC=1;AN=1;DP4=0,0,4,8;MQ=60 GT:PL
+CP000819.1 42306 . A G 225 . DP=15;VDB=0.969686;SGB=-0.686358;MQSB=1;MQ0F=0;AC=1;AN=1;DP4=0,0,5,9;MQ=60 GT:PL
+CP000819.1 45277 . A G 225 . DP=15;VDB=0.470998;SGB=-0.680642;MQSB=0.95494;MQ0F=0;AC=1;AN=1;DP4=0,0,7,5;MQ=60
+CP000819.1 56613 . C G 183 . DP=12;VDB=0.879703;SGB=-0.676189;MQSB=1;MQ0F=0;AC=1;AN=1;DP4=0,0,8,3;MQ=60 GT:PL
+CP000819.1 62118 . A G 225 . DP=19;VDB=0.414981;SGB=-0.691153;MQSB=0.906029;MQ0F=0;AC=1;AN=1;DP4=0,0,8,10;MQ=59
+CP000819.1 64042 . G A 225 . DP=18;VDB=0.451328;SGB=-0.689466;MQSB=1;MQ0F=0;AC=1;AN=1;DP4=0,0,7,9;MQ=60 GT:PL
+```
+
+The first few columns represent the information we have about a predicted variation.
+
+| column | info |
+| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| CHROM | contig location where the variation occurs |
+| POS | position within the contig where the variation occurs |
+| ID | a `.` until we add annotation information |
+| REF | reference genotype (forward strand) |
+| ALT | sample genotype (forward strand) |
+| QUAL | Phred-scaled probability that the observed variant exists at this site (higher is better) |
+| FILTER | a `.` if no quality filters have been applied, PASS if a filter is passed, or the name of the filters this variant failed |
+
+In an ideal world, the information in the `QUAL` column would be all we needed to filter out bad variant calls.
+However, in reality we need to filter on multiple other metrics.
+
+The last two columns contain the genotypes and can be tricky to decode.
+
+| column | info |
+| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| FORMAT | lists in order the metrics presented in the final column |
+| results | lists the values associated with those metrics in order |
+
+For our file, the metrics presented are GT:PL:GQ.
+
+| metric | definition |
+| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| AD, DP | the depth per allele by sample and coverage |
+| GT | the genotype for the sample at this loci. For a diploid organism, the GT field indicates the two alleles carried by the sample, encoded by a 0 for the REF allele, 1 for the first ALT allele, 2 for the second ALT allele, etc. A 0/0 means homozygous reference, 0/1 is heterozygous, and 1/1 is homozygous for the alternate allele. |
+| PL | the likelihoods of the given genotypes |
+| GQ | the Phred-scaled confidence for the genotype |
+
+For more information on VCF files visit The Broad Institute's [VCF guide](https://gatk.broadinstitute.org/hc/en-us/articles/360035531692-VCF-Variant-Call-Format).
+
+## References
+
+Tenaillon O, Barrick JE, Ribeck N, Deatherage DE, Blanchard JL, Dasgupta A, Wu GC, Wielgoss S, Cruveiller S, Médigue C, Schneider D, Lenski RE.
+Tempo and mode of genome
+evolution in a 50,000-generation experiment (2016) Nature. 536(7615): 165–170.
+[Paper](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4988878/), [Supplemental materials](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4988878/#)
+Data on NCBI SRA: [https://trace.ncbi.nlm.nih.gov/Traces/sra/?study=SRP064605](https://trace.ncbi.nlm.nih.gov/Traces/sra/?study=SRP064605)
+Data on EMBL-EBI ENA: [https://www.ebi.ac.uk/ena/data/view/PRJNA295606](https://www.ebi.ac.uk/ena/data/view/PRJNA295606)
+
+This episode was adapted from the Data Carpentry Genomic lessons:
+
+- [Project organization and management for Genomics](https://datacarpentry.org/organization-genomics/data/)
+- [Data wrangling and processing for genomics](https://datacarpentry.org/wrangling-genomics/04-variant_calling/index.html)
+
+:::::::::::::::::::::::::::::::::::::::: keypoints
+
+- The dataset comes from a real world experiment in E. coli.
+- Publicly available FASTQ files can be downloaded from NCBI SRA.
+- Several steps are taken outside of R/RStudio to create VCF files from FASTQ files.
+- VCF files store variant calls in a special format.
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
diff --git a/03-basics-factors-dataframes.md b/03-basics-factors-dataframes.md
new file mode 100644
index 000000000..3b47fbbf5
--- /dev/null
+++ b/03-basics-factors-dataframes.md
@@ -0,0 +1,1400 @@
+---
+title: R Basics continued - factors and data frames
+teaching: 60
+exercises: 30
+source: Rmd
+---
+
+::::::::::::::::::::::::::::::::::::::: objectives
+
+- Explain the basic principle of tidy datasets
+- Be able to load a tabular dataset using base R functions
+- Be able to determine the structure of a data frame including its dimensions and the datatypes of variables
+- Be able to subset/retrieve values from a data frame
+- Understand how R may coerce data into different modes
+- Be able to change the mode of an object
+- Understand that R uses factors to store and manipulate categorical data
+- Be able to manipulate a factor, including subsetting and reordering
+- Be able to apply an arithmetic function to a data frame
+- Be able to coerce the class of an object (including variables in a data frame)
+- Be able to import data from Excel
+- Be able to save a data frame as a delimited file
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::::::::::::::::::: questions
+
+- How do I get started with tabular data (e.g. spreadsheets) in R?
+- What are some best practices for reading data into R?
+- How do I save tabular data generated in R?
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+
+## Working with spreadsheets (tabular data)
+
+A substantial amount of the data we work with in genomics will be tabular data,
+this is data arranged in rows and columns - also known as spreadsheets. We could
+write a whole lesson on how to work with spreadsheets effectively ([actually we did](https://datacarpentry.org/organization-genomics/)). For our
+purposes, we want to remind you of a few principles before we work with our
+first set of example data:
+
+**1\) Keep raw data separate from analyzed data**
+
+This is principle number one because if you can't tell which files are the
+original raw data, you risk making some serious mistakes (e.g. drawing conclusion
+from data which have been manipulated in some unknown way).
+
+**2\) Keep spreadsheet data Tidy**
+
+The simplest principle of **Tidy data** is that we have one row in our
+spreadsheet for each observation or sample, and one column for every variable
+that we measure or report on. As simple as this sounds, it's very easily
+violated. Most data scientists agree that significant amounts of their time is
+spent tidying data for analysis. Read more about data organization in
+[our lesson](https://datacarpentry.org/organization-genomics/) and
+in [this paper](https://www.jstatsoft.org/article/view/v059i10).
+
+**3\) Trust but verify**
+
+Finally, while you don't need to be paranoid about data, you should have a plan
+for how you will prepare it for analysis. **This a focus of this lesson.**
+You probably already have a lot of intuition, expectations, assumptions about
+your data - the range of values you expect, how many values should have
+been recorded, etc. Of course, as the data get larger our human ability to
+keep track will start to fail (and yes, it can fail for small data sets too).
+R will help you to examine your data so that you can have greater confidence
+in your analysis, and its reproducibility.
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Keep your raw data separate
+
+When you work with data in R, you are not changing the original file you
+loaded that data from. This is different than (for example) working with
+a spreadsheet program where changing the value of the cell leaves you one
+"save"-click away from overwriting the original file. You have to purposely
+use a writing function (e.g. `write.csv()`) to save data loaded into R. In
+that case, be sure to save the manipulated data into a new file. More on this
+later in the lesson.
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Importing tabular data into R
+
+There are several ways to import data into R. For our purpose here, we will
+focus on using the tools every R installation comes with (so called "base" R) to
+import a comma-delimited file containing the results of our variant calling workflow.
+We will need to load the sheet using a function called `read.csv()`.
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise: Review the arguments of the `read.csv()` function
+
+**Before using the `read.csv()` function, use R's help feature to answer the
+following questions**.
+
+*Hint*: Entering '?' before the function name and then running that line will
+bring up the help documentation. Also, when reading this particular help
+be careful to pay attention to the 'read.csv' expression under the 'Usage'
+heading. Other answers will be in the 'Arguments' heading.
+
+A) What is the default parameter for 'header' in the `read.csv()` function?
+
+B) What argument would you have to change to read a file that was delimited
+by semicolons (;) rather than commas?
+
+C) What argument would you have to change to read file in which numbers
+used commas for decimal separation (i.e. 1,00)?
+
+D) What argument would you have to change to read in only the first 10,000 rows
+of a very large file?
+
+::::::::::::::: solution
+
+## Solution
+
+A) The `read.csv()` function has the argument 'header' set to TRUE by default,
+this means the function always assumes the first row is header information,
+(i.e. column names)
+
+B) The `read.csv()` function has the argument 'sep' set to ",". This means
+the function assumes commas are used as delimiters, as you would expect.
+Changing this parameter (e.g. `sep=";"`) would now interpret semicolons as
+delimiters.
+
+C) Although it is not listed in the `read.csv()` usage, `read.csv()` is
+a "version" of the function `read.table()` and accepts all its arguments.
+If you set `dec=","` you could change the decimal operator. We'd probably
+assume the delimiter is some other character.
+
+D) You can set `nrow` to a numeric value (e.g. `nrow=10000`) to choose how
+many rows of a file you read in. This may be useful for very large files
+where not all the data is needed to test some data cleaning steps you are
+applying.
+
+Hopefully, this exercise gets you thinking about using the provided help
+documentation in R. There are many arguments that exist, but which we wont
+have time to cover. Look here to get familiar with functions you use
+frequently, you may be surprised at what you find they can do.
+
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Why does ?read.csv open the documentations to read.table?
+
+The reason for this is because `read.csv` is actually a short cut
+for `read.table("file.csv", sep = ",")`. You can see in the help
+documentation that there are several additional variations of
+`read.table`, such as `read.csv2` to read tables separated by `;`
+and `read.delim` to read in tables separated by `\t` (tabs). If you know how your table is separated, you can use one of the provided short cuts,
+but case you run into an unconventional separator you are now equipt with the knowledge to define it in the `sep = ` arugument of `read.table`!
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+Now, let's read in the file `combined_tidy_vcf.csv` which will be located in
+`/home/dcuser/r_data/`. Call this data `variants`. The
+first argument to pass to our `read.csv()` function is the file path for our
+data. The file path must be in quotes and now is a good time to remember to
+use tab autocompletion. **If you use tab autocompletion you avoid typos and
+errors in file paths.** Use it!
+
+
+```r
+## read in a CSV file and save it as 'variants'
+
+variants <- read.csv("/home/dcuser/r_data/combined_tidy_vcf.csv")
+```
+
+
+
+One of the first things you should notice is that in the Environment window,
+you have the `variants` object, listed as 801 obs. (observations/rows)
+of 29 variables (columns). Double-clicking on the name of the object will open
+a view of the data in a new tab.
+
+
+
+## Summarizing, subsetting, and determining the structure of a data frame.
+
+A **data frame is the standard way in R to store tabular data**. A data fame
+could also be thought of as a collection of vectors, all of which have the same
+length. Using only two functions, we can learn a lot about out data frame
+including some summary statistics as well as well as the "structure" of the data
+frame. Let's examine what each of these functions can tell us:
+
+
+```r
+## get summary statistics on a data frame
+
+summary(variants)
+```
+
+```{.output}
+ sample_id CHROM POS ID
+ Length:801 Length:801 Min. : 1521 Mode:logical
+ Class :character Class :character 1st Qu.:1115970 NA's:801
+ Mode :character Mode :character Median :2290361
+ Mean :2243682
+ 3rd Qu.:3317082
+ Max. :4629225
+
+ REF ALT QUAL FILTER
+ Length:801 Length:801 Min. : 4.385 Mode:logical
+ Class :character Class :character 1st Qu.:139.000 NA's:801
+ Mode :character Mode :character Median :195.000
+ Mean :172.276
+ 3rd Qu.:225.000
+ Max. :228.000
+
+ INDEL IDV IMF DP
+ Mode :logical Min. : 2.000 Min. :0.5714 Min. : 2.00
+ FALSE:700 1st Qu.: 7.000 1st Qu.:0.8824 1st Qu.: 7.00
+ TRUE :101 Median : 9.000 Median :1.0000 Median :10.00
+ Mean : 9.396 Mean :0.9219 Mean :10.57
+ 3rd Qu.:11.000 3rd Qu.:1.0000 3rd Qu.:13.00
+ Max. :20.000 Max. :1.0000 Max. :79.00
+ NA's :700 NA's :700
+ VDB RPB MQB BQB
+ Min. :0.0005387 Min. :0.0000 Min. :0.0000 Min. :0.1153
+ 1st Qu.:0.2180410 1st Qu.:0.3776 1st Qu.:0.1070 1st Qu.:0.6963
+ Median :0.4827410 Median :0.8663 Median :0.2872 Median :0.8615
+ Mean :0.4926291 Mean :0.6970 Mean :0.5330 Mean :0.7784
+ 3rd Qu.:0.7598940 3rd Qu.:1.0000 3rd Qu.:1.0000 3rd Qu.:1.0000
+ Max. :0.9997130 Max. :1.0000 Max. :1.0000 Max. :1.0000
+ NA's :773 NA's :773 NA's :773
+ MQSB SGB MQ0F ICB
+ Min. :0.01348 Min. :-0.6931 Min. :0.00000 Mode:logical
+ 1st Qu.:0.95494 1st Qu.:-0.6762 1st Qu.:0.00000 NA's:801
+ Median :1.00000 Median :-0.6620 Median :0.00000
+ Mean :0.96428 Mean :-0.6444 Mean :0.01127
+ 3rd Qu.:1.00000 3rd Qu.:-0.6364 3rd Qu.:0.00000
+ Max. :1.01283 Max. :-0.4536 Max. :0.66667
+ NA's :48
+ HOB AC AN DP4 MQ
+ Mode:logical Min. :1 Min. :1 Length:801 Min. :10.00
+ NA's:801 1st Qu.:1 1st Qu.:1 Class :character 1st Qu.:60.00
+ Median :1 Median :1 Mode :character Median :60.00
+ Mean :1 Mean :1 Mean :58.19
+ 3rd Qu.:1 3rd Qu.:1 3rd Qu.:60.00
+ Max. :1 Max. :1 Max. :60.00
+
+ Indiv gt_PL gt_GT gt_GT_alleles
+ Length:801 Length:801 Min. :1 Length:801
+ Class :character Class :character 1st Qu.:1 Class :character
+ Mode :character Mode :character Median :1 Mode :character
+ Mean :1
+ 3rd Qu.:1
+ Max. :1
+
+```
+
+Our data frame had 29 variables, so we get 29 fields that summarize the data.
+The `QUAL`, `IMF`, and `VDB` variables (and several others) are
+numerical data and so you get summary statistics on the min and max values for
+these columns, as well as mean, median, and interquartile ranges. Many of the
+other variables (e.g. `sample_id`) are treated as characters data (more on this
+in a bit).
+
+There is a lot to work with, so we will subset the first three columns into a
+new data frame using the `data.frame()` function.
+
+
+```r
+## put the first three columns of variants into a new data frame called subset
+
+subset<-data.frame(variants[,c(1:3,6)])
+```
+
+Now, let's use the `str()` (structure) function to look a little more closely
+at how data frames work:
+
+
+```r
+## get the structure of a data frame
+
+str(subset)
+```
+
+```{.output}
+'data.frame': 801 obs. of 4 variables:
+ $ sample_id: chr "SRR2584863" "SRR2584863" "SRR2584863" "SRR2584863" ...
+ $ CHROM : chr "CP000819.1" "CP000819.1" "CP000819.1" "CP000819.1" ...
+ $ POS : int 9972 263235 281923 433359 473901 648692 1331794 1733343 2103887 2333538 ...
+ $ ALT : chr "G" "T" "T" "CTTTTTTTT" ...
+```
+
+Ok, thats a lot up unpack! Some things to notice.
+
+- the object type `data.frame` is displayed in the first row along with its
+ dimensions, in this case 801 observations (rows) and 4 variables (columns)
+- Each variable (column) has a name (e.g. `sample_id`). This is followed
+ by the object mode (e.g. chr, int, etc.). Notice that before each
+ variable name there is a `$` - this will be important later.
+
+## Introducing Factors
+
+Factors are the final major data structure we will introduce in our R genomics
+lessons. Factors can be thought of as vectors which are specialized for
+categorical data. Given R's specialization for statistics, this make sense since
+categorial and continuous variables are usually treated differently. Sometimes
+you may want to have data treated as a factor, but in other cases, this may be
+undesirable.
+
+Let's see the value of treating some of which are categorical in nature as
+factors. Let's take a look at just the alternate alleles
+
+
+```r
+## extract the "ALT" column to a new object
+
+alt_alleles <- subset$ALT
+```
+
+Let's look at the first few items in our factor using `head()`:
+
+
+```r
+head(alt_alleles)
+```
+
+```{.output}
+[1] "G" "T" "T" "CTTTTTTTT" "CCGCGC" "T"
+```
+
+There are 801 alleles (one for each row). To simplify, lets look at just the
+single-nuleotide alleles (SNPs). We can use some of the vector indexing skills
+from the last episode.
+
+
+```r
+snps <- c(alt_alleles[alt_alleles=="A"],
+ alt_alleles[alt_alleles=="T"],
+ alt_alleles[alt_alleles=="G"],
+ alt_alleles[alt_alleles=="C"])
+```
+
+This leaves us with a vector of the 701 alternative alleles which were single
+nucleotides. Right now, they are being treated a characters, but we could treat
+them as categories of SNP. Doing this will enable some nice features. For
+example, we can try to generate a plot of this character vector as it is right
+now:
+
+
+```r
+plot(snps)
+```
+
+```{.warning}
+Warning in xy.coords(x, y, xlabel, ylabel, log): NAs introduced by coercion
+```
+
+```{.warning}
+Warning in min(x): no non-missing arguments to min; returning Inf
+```
+
+```{.warning}
+Warning in max(x): no non-missing arguments to max; returning -Inf
+```
+
+```{.error}
+Error in plot.window(...): need finite 'ylim' values
+```
+
+Whoops! Though the `plot()` function will do its best to give us a quick plot,
+it is unable to do so here. One way to fix this it to tell R to treat the SNPs
+as categories (i.e. a factor vector); we will create a new object to avoid
+confusion using the `factor()` function:
+
+
+```r
+factor_snps <- factor(snps)
+```
+
+Let's learn a little more about this new type of vector:
+
+
+```r
+str(factor_snps)
+```
+
+```{.output}
+ Factor w/ 4 levels "A","C","G","T": 1 1 1 1 1 1 1 1 1 1 ...
+```
+
+What we get back are the categories ("A","C","G","T") in our factor; these are
+called "Levels". **Levels are the different categories contained in a factor**.
+By default, R will organize the levels in a factor in alphabetical order. So the
+first level in this factor is "A".
+
+For the sake of efficiency, R stores the content of a factor as a vector of
+integers, which an integer is assigned to each of the possible levels. Recall
+levels are assigned in alphabetical order. In this case, the first item in our
+`factor_snps` object is "A", which happens to be the 1st level of our factor,
+ordered alphabetically. This explains the sequence of "1"s ("Factor w/ 4 levels
+"A","C","G","T": 1 1 1 1 1 1 1 1 1 1 ..."), since "A" is the first level, and
+the first few items in our factor are all "A"s.
+
+We can see how many items in our vector fall into each category:
+
+
+```r
+summary(factor_snps)
+```
+
+```{.output}
+ A C G T
+211 139 154 203
+```
+
+As you can imagine, this is already useful when you want to generate a tally.
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: treating objects as categories without changing their mode
+
+You don't have to make an object a factor to get the benefits of treating
+an object as a factor. See what happens when you use the `as.factor()`
+function on `factor_snps`. To generate a tally, you can sometimes also use
+the `table()` function; though sometimes you may need to combine both (i.e.
+`table(as.factor(object))`)
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Plotting and ordering factors
+
+One of the most common uses for factors will be when you plot categorical
+values. For example, suppose we want to know how many of our variants had each
+possible SNP we could generate a plot:
+
+
+```r
+plot(factor_snps)
+```
+
+
+
+This isn't a particularly pretty example of a plot but it works. We'll be
+learning much more about creating nice, publication-quality graphics later in
+this lesson.
+
+If you recall, factors are ordered alphabetically. That might make sense, but
+categories (e.g., "red", "blue", "green") often do not have an intrinsic order.
+What if we wanted to order our plot according to the numerical value (i.e.,
+in descending order of SNP frequency)? We can enforce an order on our factors:
+
+
+```r
+ordered_factor_snps <- factor(factor_snps, levels = names(sort(table(factor_snps))))
+```
+
+Let's deconstruct this from the inside out (you can try each of these commands
+to see why this works):
+
+1. We create a table of `factor_snps` to get the frequency of each SNP:
+ `table(factor_snps)`
+2. We sort this table: `sort(table(factor_snps))`; use the `decreasing =`
+ parameter for this function if you wanted to change from the default of
+ FALSE
+3. Using the `names` function gives us just the character names of the table
+ sorted by frequencies:`names(sort(table(factor_snps)))`
+4. The `factor` function is what allows us to create a factor. We give it the
+ `factor_snps` object as input, and use the `levels=` parameter to enforce the
+ ordering of the levels.
+
+Now we see our plot has be reordered:
+
+
+```r
+plot(ordered_factor_snps)
+```
+
+
+
+Factors come in handy in many places when using R. Even using more
+sophisticated plotting packages such as ggplot2 will sometimes require you
+to understand how to manipulate factors.
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Packages in R -- what are they and why do we use them?
+
+Packages are simply collections of functions and/or data that can be used to extend the
+capabilities of R beyond the core functionality that comes with it by default. There are
+useful R packages available that span all types of statistical analysis, data visualization,
+and more. The main place that R packages are installed from is a website called
+[CRAN](https://cran.r-project.org/) (the Comprehensive R Archive Network). Many thousands
+of R packages are available there, and when you use the built-in R function `install.packages()`,
+it will look for a CRAN repository to install from. So, for example, to install
+[tidyverse](https://www.tidyverse.org) packages such as `dplyr` and `ggplot2`
+(which you'll do in the next few lessons), you would use the following command:
+
+
+```r
+# install a package from CRAN
+install.packages("ggplot2")
+```
+
+```{.output}
+The following package(s) will be installed:
+- ggplot2 [3.4.4]
+These packages will be installed into "~/work/genomics-r-intro/genomics-r-intro/renv/profiles/lesson-requirements/renv/library/R-4.3/x86_64-pc-linux-gnu".
+
+# Installing packages --------------------------------------------------------
+- Installing ggplot2 ... OK [linked from cache]
+Successfully installed 1 package in 5.2 milliseconds.
+```
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Subsetting data frames
+
+Next, we are going to talk about how you can get specific values from data frames, and where necessary, change the mode of a column of values.
+
+The first thing to remember is that a data frame is two-dimensional (rows and
+columns). Therefore, to select a specific value we will will once again use
+`[]` (bracket) notation, but we will specify more than one value (except in some cases
+where we are taking a range).
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise: Subsetting a data frame
+
+**Try the following indices and functions and try to figure out what they return**
+
+a. `variants[1,1]`
+
+b. `variants[2,4]`
+
+c. `variants[801,29]`
+
+d. `variants[2, ]`
+
+e. `variants[-1, ]`
+
+f. `variants[1:4,1]`
+
+g. `variants[1:10,c("REF","ALT")]`
+
+h. `variants[,c("sample_id")]`
+
+i. `head(variants)`
+
+j. `tail(variants)`
+
+k. `variants$sample_id`
+
+l. `variants[variants$REF == "A",]`
+
+::::::::::::::: solution
+
+## Solution
+
+a.
+
+
+```r
+variants[1,1]
+```
+
+```{.output}
+[1] "SRR2584863"
+```
+
+b.
+
+
+```r
+variants[2,4]
+```
+
+```{.output}
+[1] NA
+```
+
+c.
+
+
+```r
+variants[801,29]
+```
+
+```{.output}
+[1] "T"
+```
+
+d.
+
+
+```r
+variants[2, ]
+```
+
+```{.output}
+ sample_id CHROM POS ID REF ALT QUAL FILTER INDEL IDV IMF DP VDB
+2 SRR2584863 CP000819.1 263235 NA G T 85 NA FALSE NA NA 6 0.096133
+ RPB MQB BQB MQSB SGB MQ0F ICB HOB AC AN DP4 MQ
+2 1 1 1 NA -0.590765 0.166667 NA NA 1 1 0,1,0,5 33
+ Indiv gt_PL
+2 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 112,0
+ gt_GT gt_GT_alleles
+2 1 T
+```
+
+e.
+
+
+```r
+variants[-1, ]
+```
+
+
+```{.output}
+ sample_id CHROM POS ID REF ALT QUAL FILTER INDEL IDV IMF
+2 SRR2584863 CP000819.1 263235 NA G T 85 NA FALSE NA NA
+3 SRR2584863 CP000819.1 281923 NA G T 217 NA FALSE NA NA
+4 SRR2584863 CP000819.1 433359 NA CTTTTTTT CTTTTTTTT 64 NA TRUE 12 1.0
+5 SRR2584863 CP000819.1 473901 NA CCGC CCGCGC 228 NA TRUE 9 0.9
+6 SRR2584863 CP000819.1 648692 NA C T 210 NA FALSE NA NA
+7 SRR2584863 CP000819.1 1331794 NA C A 178 NA FALSE NA NA
+ DP VDB RPB MQB BQB MQSB SGB MQ0F ICB HOB AC AN DP4 MQ
+2 6 0.096133 1 1 1 NA -0.590765 0.166667 NA NA 1 1 0,1,0,5 33
+3 10 0.774083 NA NA NA 0.974597 -0.662043 0.000000 NA NA 1 1 0,0,4,5 60
+4 12 0.477704 NA NA NA 1.000000 -0.676189 0.000000 NA NA 1 1 0,1,3,8 60
+5 10 0.659505 NA NA NA 0.916482 -0.662043 0.000000 NA NA 1 1 1,0,2,7 60
+6 10 0.268014 NA NA NA 0.916482 -0.670168 0.000000 NA NA 1 1 0,0,7,3 60
+7 8 0.624078 NA NA NA 0.900802 -0.651104 0.000000 NA NA 1 1 0,0,3,5 60
+ Indiv gt_PL
+2 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 112,0
+3 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 247,0
+4 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 91,0
+5 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 255,0
+6 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 240,0
+7 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 208,0
+ gt_GT gt_GT_alleles
+2 1 T
+3 1 T
+4 1 CTTTTTTTT
+5 1 CCGCGC
+6 1 T
+7 1 A
+```
+
+f.
+
+
+```r
+variants[1:4,1]
+```
+
+```{.output}
+[1] "SRR2584863" "SRR2584863" "SRR2584863" "SRR2584863"
+```
+
+g.
+
+
+```r
+variants[1:10,c("REF","ALT")]
+```
+
+```{.output}
+ REF
+1 T
+2 G
+3 G
+4 CTTTTTTT
+5 CCGC
+6 C
+7 C
+8 G
+9 ACAGCCAGCCAGCCAGCCAGCCAGCCAGCCAG
+10 AT
+ ALT
+1 G
+2 T
+3 T
+4 CTTTTTTTT
+5 CCGCGC
+6 T
+7 A
+8 A
+9 ACAGCCAGCCAGCCAGCCAGCCAGCCAGCCAGCCAGCCAGCCAGCCAGCCAGCCAG
+10 ATT
+```
+
+h.
+
+
+```r
+variants[,c("sample_id")]
+```
+
+
+```{.output}
+[1] "SRR2584863" "SRR2584863" "SRR2584863" "SRR2584863" "SRR2584863"
+[6] "SRR2584863"
+```
+
+i.
+
+
+```r
+head(variants)
+```
+
+```{.output}
+ sample_id CHROM POS ID REF ALT QUAL FILTER INDEL IDV IMF
+1 SRR2584863 CP000819.1 9972 NA T G 91 NA FALSE NA NA
+2 SRR2584863 CP000819.1 263235 NA G T 85 NA FALSE NA NA
+3 SRR2584863 CP000819.1 281923 NA G T 217 NA FALSE NA NA
+4 SRR2584863 CP000819.1 433359 NA CTTTTTTT CTTTTTTTT 64 NA TRUE 12 1.0
+5 SRR2584863 CP000819.1 473901 NA CCGC CCGCGC 228 NA TRUE 9 0.9
+6 SRR2584863 CP000819.1 648692 NA C T 210 NA FALSE NA NA
+ DP VDB RPB MQB BQB MQSB SGB MQ0F ICB HOB AC AN DP4 MQ
+1 4 0.0257451 NA NA NA NA -0.556411 0.000000 NA NA 1 1 0,0,0,4 60
+2 6 0.0961330 1 1 1 NA -0.590765 0.166667 NA NA 1 1 0,1,0,5 33
+3 10 0.7740830 NA NA NA 0.974597 -0.662043 0.000000 NA NA 1 1 0,0,4,5 60
+4 12 0.4777040 NA NA NA 1.000000 -0.676189 0.000000 NA NA 1 1 0,1,3,8 60
+5 10 0.6595050 NA NA NA 0.916482 -0.662043 0.000000 NA NA 1 1 1,0,2,7 60
+6 10 0.2680140 NA NA NA 0.916482 -0.670168 0.000000 NA NA 1 1 0,0,7,3 60
+ Indiv gt_PL
+1 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 121,0
+2 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 112,0
+3 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 247,0
+4 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 91,0
+5 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 255,0
+6 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 240,0
+ gt_GT gt_GT_alleles
+1 1 G
+2 1 T
+3 1 T
+4 1 CTTTTTTTT
+5 1 CCGCGC
+6 1 T
+```
+
+j.
+
+
+```r
+tail(variants)
+```
+
+```{.output}
+ sample_id CHROM POS ID REF ALT QUAL FILTER INDEL IDV IMF DP
+796 SRR2589044 CP000819.1 3444175 NA G T 184 NA FALSE NA NA 9
+797 SRR2589044 CP000819.1 3481820 NA A G 225 NA FALSE NA NA 12
+798 SRR2589044 CP000819.1 3893550 NA AG AGG 101 NA TRUE 4 1 4
+799 SRR2589044 CP000819.1 3901455 NA A AC 70 NA TRUE 3 1 3
+800 SRR2589044 CP000819.1 4100183 NA A G 177 NA FALSE NA NA 8
+801 SRR2589044 CP000819.1 4431393 NA TGG T 225 NA TRUE 10 1 10
+ VDB RPB MQB BQB MQSB SGB MQ0F ICB HOB AC AN DP4 MQ
+796 0.4714620 NA NA NA 0.992367 -0.651104 0 NA NA 1 1 0,0,4,4 60
+797 0.8707240 NA NA NA 1.000000 -0.680642 0 NA NA 1 1 0,0,4,8 60
+798 0.9182970 NA NA NA 1.000000 -0.556411 0 NA NA 1 1 0,0,3,1 52
+799 0.0221621 NA NA NA NA -0.511536 0 NA NA 1 1 0,0,3,0 60
+800 0.9272700 NA NA NA 0.900802 -0.651104 0 NA NA 1 1 0,0,3,5 60
+801 0.7488140 NA NA NA 1.007750 -0.670168 0 NA NA 1 1 0,0,4,6 60
+ Indiv gt_PL
+796 /home/dcuser/dc_workshop/results/bam/SRR2589044.aligned.sorted.bam 214,0
+797 /home/dcuser/dc_workshop/results/bam/SRR2589044.aligned.sorted.bam 255,0
+798 /home/dcuser/dc_workshop/results/bam/SRR2589044.aligned.sorted.bam 131,0
+799 /home/dcuser/dc_workshop/results/bam/SRR2589044.aligned.sorted.bam 100,0
+800 /home/dcuser/dc_workshop/results/bam/SRR2589044.aligned.sorted.bam 207,0
+801 /home/dcuser/dc_workshop/results/bam/SRR2589044.aligned.sorted.bam 255,0
+ gt_GT gt_GT_alleles
+796 1 T
+797 1 G
+798 1 AGG
+799 1 AC
+800 1 G
+801 1 T
+```
+
+k.
+
+
+```r
+variants$sample_id
+```
+
+
+```{.output}
+[1] "SRR2584863" "SRR2584863" "SRR2584863" "SRR2584863" "SRR2584863"
+[6] "SRR2584863"
+```
+
+l.
+
+
+```r
+variants[variants$REF == "A",]
+```
+
+
+```{.output}
+ sample_id CHROM POS ID REF ALT QUAL FILTER INDEL IDV IMF DP
+11 SRR2584863 CP000819.1 2407766 NA A C 104 NA FALSE NA NA 9
+12 SRR2584863 CP000819.1 2446984 NA A C 225 NA FALSE NA NA 20
+14 SRR2584863 CP000819.1 2665639 NA A T 225 NA FALSE NA NA 19
+16 SRR2584863 CP000819.1 3339313 NA A C 211 NA FALSE NA NA 10
+18 SRR2584863 CP000819.1 3481820 NA A G 200 NA FALSE NA NA 9
+19 SRR2584863 CP000819.1 3488669 NA A C 225 NA FALSE NA NA 13
+ VDB RPB MQB BQB MQSB SGB MQ0F ICB HOB AC
+11 0.0230738 0.900802 0.150134 0.750668 0.500000 -0.590765 0.333333 NA NA 1
+12 0.0714027 NA NA NA 1.000000 -0.689466 0.000000 NA NA 1
+14 0.9960390 NA NA NA 1.000000 -0.690438 0.000000 NA NA 1
+16 0.4059360 NA NA NA 1.007750 -0.670168 0.000000 NA NA 1
+18 0.1070810 NA NA NA 0.974597 -0.662043 0.000000 NA NA 1
+19 0.0162706 NA NA NA 1.000000 -0.680642 0.000000 NA NA 1
+ AN DP4 MQ
+11 1 3,0,3,2 25
+12 1 0,0,10,6 60
+14 1 0,0,12,5 60
+16 1 0,0,4,6 60
+18 1 0,0,4,5 60
+19 1 0,0,8,4 60
+ Indiv gt_PL
+11 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 131,0
+12 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 255,0
+14 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 255,0
+16 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 241,0
+18 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 230,0
+19 /home/dcuser/dc_workshop/results/bam/SRR2584863.aligned.sorted.bam 255,0
+ gt_GT gt_GT_alleles
+11 1 C
+12 1 C
+14 1 T
+16 1 C
+18 1 G
+19 1 C
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+The subsetting notation is very similar to what we learned for
+vectors. The key differences include:
+
+- Typically provide two values separated by commas: data.frame[row, column]
+- In cases where you are taking a continuous range of numbers use a colon
+ between the numbers (start:stop, inclusive)
+- For a non continuous set of numbers, pass a vector using `c()`
+- Index using the name of a column(s) by passing them as vectors using `c()`
+
+Finally, in all of the subsetting exercises above, we printed values to
+the screen. You can create a new data frame object by assigning
+them to a new object name:
+
+
+```r
+# create a new data frame containing only observations from SRR2584863
+
+SRR2584863_variants <- variants[variants$sample_id == "SRR2584863",]
+
+# check the dimension of the data frame
+
+dim(SRR2584863_variants)
+```
+
+```{.output}
+[1] 25 29
+```
+
+```r
+# get a summary of the data frame
+
+summary(SRR2584863_variants)
+```
+
+```{.output}
+ sample_id CHROM POS ID
+ Length:25 Length:25 Min. : 9972 Mode:logical
+ Class :character Class :character 1st Qu.:1331794 NA's:25
+ Mode :character Mode :character Median :2618472
+ Mean :2464989
+ 3rd Qu.:3488669
+ Max. :4616538
+
+ REF ALT QUAL FILTER
+ Length:25 Length:25 Min. : 31.89 Mode:logical
+ Class :character Class :character 1st Qu.:104.00 NA's:25
+ Mode :character Mode :character Median :211.00
+ Mean :172.97
+ 3rd Qu.:225.00
+ Max. :228.00
+
+ INDEL IDV IMF DP
+ Mode :logical Min. : 2.00 Min. :0.6667 Min. : 2.0
+ FALSE:19 1st Qu.: 3.25 1st Qu.:0.9250 1st Qu.: 9.0
+ TRUE :6 Median : 8.00 Median :1.0000 Median :10.0
+ Mean : 7.00 Mean :0.9278 Mean :10.4
+ 3rd Qu.: 9.75 3rd Qu.:1.0000 3rd Qu.:12.0
+ Max. :12.00 Max. :1.0000 Max. :20.0
+ NA's :19 NA's :19
+ VDB RPB MQB BQB
+ Min. :0.01627 Min. :0.9008 Min. :0.04979 Min. :0.7507
+ 1st Qu.:0.07140 1st Qu.:0.9275 1st Qu.:0.09996 1st Qu.:0.7627
+ Median :0.37674 Median :0.9542 Median :0.15013 Median :0.7748
+ Mean :0.40429 Mean :0.9517 Mean :0.39997 Mean :0.8418
+ 3rd Qu.:0.65951 3rd Qu.:0.9771 3rd Qu.:0.57507 3rd Qu.:0.8874
+ Max. :0.99604 Max. :1.0000 Max. :1.00000 Max. :1.0000
+ NA's :22 NA's :22 NA's :22
+ MQSB SGB MQ0F ICB
+ Min. :0.5000 Min. :-0.6904 Min. :0.00000 Mode:logical
+ 1st Qu.:0.9599 1st Qu.:-0.6762 1st Qu.:0.00000 NA's:25
+ Median :0.9962 Median :-0.6620 Median :0.00000
+ Mean :0.9442 Mean :-0.6341 Mean :0.04667
+ 3rd Qu.:1.0000 3rd Qu.:-0.6168 3rd Qu.:0.00000
+ Max. :1.0128 Max. :-0.4536 Max. :0.66667
+ NA's :3
+ HOB AC AN DP4 MQ
+ Mode:logical Min. :1 Min. :1 Length:25 Min. :10.00
+ NA's:25 1st Qu.:1 1st Qu.:1 Class :character 1st Qu.:60.00
+ Median :1 Median :1 Mode :character Median :60.00
+ Mean :1 Mean :1 Mean :55.52
+ 3rd Qu.:1 3rd Qu.:1 3rd Qu.:60.00
+ Max. :1 Max. :1 Max. :60.00
+
+ Indiv gt_PL gt_GT gt_GT_alleles
+ Length:25 Length:25 Min. :1 Length:25
+ Class :character Class :character 1st Qu.:1 Class :character
+ Mode :character Mode :character Median :1 Mode :character
+ Mean :1
+ 3rd Qu.:1
+ Max. :1
+
+```
+
+## Coercing values in data frames
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: coercion isn't limited to data frames
+
+While we are going to address coercion in the context of data frames
+most of these methods apply to other data structures, such as vectors
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+Sometimes, it is possible that R will misinterpret the type of data represented
+in a data frame, or store that data in a mode which prevents you from
+operating on the data the way you wish. For example, a long list of gene names
+isn't usually thought of as a categorical variable, the way that your
+experimental condition (e.g. control, treatment) might be. More importantly,
+some R packages you use to analyze your data may expect characters as input,
+not factors. At other times (such as plotting or some statistical analyses) a
+factor may be more appropriate. Ultimately, you should know how to change the
+mode of an object.
+
+First, its very important to recognize that coercion happens in R all the time.
+This can be a good thing when R gets it right, or a bad thing when the result
+is not what you expect. Consider:
+
+
+```r
+snp_chromosomes <- c('3', '11', 'X', '6')
+typeof(snp_chromosomes)
+```
+
+```{.output}
+[1] "character"
+```
+
+Although there are several numbers in our vector, they are all in quotes, so
+we have explicitly told R to consider them as characters. However, even if we removed
+the quotes from the numbers, R would coerce everything into a character:
+
+
+```r
+snp_chromosomes_2 <- c(3, 11, 'X', 6)
+typeof(snp_chromosomes_2)
+```
+
+```{.output}
+[1] "character"
+```
+
+```r
+snp_chromosomes_2[1]
+```
+
+```{.output}
+[1] "3"
+```
+
+We can use the `as.` functions to explicitly coerce values from one form into
+another. Consider the following vector of characters, which all happen to be
+valid numbers:
+
+
+```r
+snp_positions_2 <- c("8762685", "66560624", "67545785", "154039662")
+typeof(snp_positions_2)
+```
+
+```{.output}
+[1] "character"
+```
+
+```r
+snp_positions_2[1]
+```
+
+```{.output}
+[1] "8762685"
+```
+
+Now we can coerce `snp_positions_2` into a numeric type using `as.numeric()`:
+
+
+```r
+snp_positions_2 <- as.numeric(snp_positions_2)
+typeof(snp_positions_2)
+```
+
+```{.output}
+[1] "double"
+```
+
+```r
+snp_positions_2[1]
+```
+
+```{.output}
+[1] 8762685
+```
+
+Sometimes coercion is straight forward, but what would happen if we tried
+using `as.numeric()` on `snp_chromosomes_2`
+
+
+```r
+snp_chromosomes_2 <- as.numeric(snp_chromosomes_2)
+```
+
+```{.warning}
+Warning: NAs introduced by coercion
+```
+
+If we check, we will see that an `NA` value (R's default value for missing
+data) has been introduced.
+
+
+```r
+snp_chromosomes_2
+```
+
+```{.output}
+[1] 3 11 NA 6
+```
+
+Trouble can really start when we try to coerce a factor. For example, when we
+try to coerce the `factor_snps` vector into a numeric mode
+look at the result:
+
+
+```r
+as.numeric(factor_snps)
+```
+
+```{.output}
+ [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
+ [38] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
+ [75] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
+[112] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
+[149] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
+[186] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 4 4 4 4 4 4 4 4 4 4 4
+[223] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
+[260] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
+[297] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
+[334] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
+[371] 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
+[408] 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
+[445] 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
+[482] 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
+[519] 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
+[556] 3 3 3 3 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
+[593] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
+[630] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
+[667] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
+[704] 2 2 2 2
+```
+
+Strangely, it works! Almost. Instead of giving an error message, R returns
+numeric values, which in this case are the integers assigned to the levels in
+this factor. This kind of behavior can lead to hard-to-find bugs, for example
+when we do have numbers in a factor, and we get numbers from a coercion. If
+we don't look carefully, we may not notice a problem.
+
+If you need to coerce an entire column you can overwrite it using an expression
+like this one:
+
+
+```r
+# make the 'REF' column a character type column
+
+variants$REF <- as.character(variants$REF)
+
+# check the type of the column
+typeof(variants$REF)
+```
+
+```{.output}
+[1] "character"
+```
+
+## StringsAsFactors = ?
+
+Lets summarize this section on coercion with a few take home messages.
+
+- When you explicitly coerce one data type into another (this is known as
+ **explicit coercion**), be careful to check the result. Ideally, you should try to see if its possible to avoid steps in your analysis that force you to
+ coerce.
+- R will sometimes coerce without you asking for it. This is called
+ (appropriately) **implicit coercion**. For example when we tried to create
+ a vector with multiple data types, R chose one type through implicit
+ coercion.
+- Check the structure (`str()`) of your data frames before working with them!
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: coercion isn't limited to data frames
+
+Prior to R 4.0 when importing a data frame using any one of the `read.table()`
+functions such as `read.csv()` , the argument `StringsAsFactors` was by
+default
+set to true TRUE. Setting it to FALSE will treat any non-numeric column to
+a character type. `read.csv()` documentation, you will also see you can
+explicitly type your columns using the `colClasses` argument. Other R packages
+(such as the Tidyverse "readr") don't have this particular conversion issue,
+but many packages will still try to guess a data type.
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Data frame bonus material: math, sorting, renaming
+
+Here are a few operations that don't need much explanation, but which are good
+to know.
+
+There are lots of arithmetic functions you may want to apply to your data
+frame, covering those would be a course in itself (there is some starting
+material in [the _Loops in R_ episode of one of the Software Carpentry lessons](https://swcarpentry.github.io/r-novice-inflammation/15-supp-loops-in-depth.html)). Our lessons will cover some additional summary statistical functions in
+a subsequent lesson, but overall we will focus on data cleaning and
+visualization.
+
+You can use functions like `mean()`, `min()`, `max()` on an
+individual column. Let's look at the "DP" or filtered depth. This value shows the number of filtered
+reads that support each of the reported variants.
+
+
+```r
+max(variants$DP)
+```
+
+```{.output}
+[1] 79
+```
+
+You can sort a data frame using the `order()` function:
+
+
+```r
+sorted_by_DP <- variants[order(variants$DP), ]
+head(sorted_by_DP$DP)
+```
+
+```{.output}
+[1] 2 2 2 2 2 2
+```
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise
+
+The `order()` function lists values in increasing order by default. Look at
+the documentation for this function and change `sorted_by_DP` to start with
+variants with the greatest filtered depth ("DP").
+
+::::::::::::::: solution
+
+## Solution
+
+
+```r
+ sorted_by_DP <- variants[order(variants$DP, decreasing = TRUE), ]
+ head(sorted_by_DP$DP)
+```
+
+```{.output}
+[1] 79 46 41 29 29 27
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+You can rename columns:
+
+
+```r
+colnames(variants)[colnames(variants) == "sample_id"] <- "strain"
+
+# check the column name (hint names are returned as a vector)
+colnames(variants)
+```
+
+```{.output}
+ [1] "strain" "CHROM" "POS" "ID"
+ [5] "REF" "ALT" "QUAL" "FILTER"
+ [9] "INDEL" "IDV" "IMF" "DP"
+[13] "VDB" "RPB" "MQB" "BQB"
+[17] "MQSB" "SGB" "MQ0F" "ICB"
+[21] "HOB" "AC" "AN" "DP4"
+[25] "MQ" "Indiv" "gt_PL" "gt_GT"
+[29] "gt_GT_alleles"
+```
+
+## Saving your data frame to a file
+
+We can save data to a file. We will save our `SRR2584863_variants` object
+to a .csv file using the `write.csv()` function:
+
+
+```r
+write.csv(SRR2584863_variants, file = "data/SRR2584863_variants.csv")
+```
+
+The `write.csv()` function has some additional arguments listed in the help, but
+at a minimum you need to tell it what data frame to write to file, and give a
+path to a file name in quotes (if you only provide a file name, the file will
+be written in the current working directory).
+
+## Importing data from Excel
+
+Excel is one of the most common formats, so we need to discuss how to make
+these files play nicely with R. The simplest way to import data from Excel is
+to **save your Excel file in .csv format**\*. You can then import into R right
+away. Sometimes you may not be able to do this (imagine you have data in 300
+Excel files, are you going to open and export all of them?).
+
+One common R package (a set of code with features you can download and add to
+your R installation) is the [readxl package](https://CRAN.R-project.org/package=readxl) which can open and import Excel
+files. Rather than addressing package installation this second (we'll discuss this soon!), we can take
+advantage of RStudio's import feature which integrates this package. (Note:
+this feature is available only in the latest versions of RStudio such as is
+installed on our cloud instance).
+
+First, in the RStudio menu go to **File**, select **Import Dataset**, and
+choose **From Excel...** (notice there are several other options you can
+explore).
+
+
+
+Next, under **File/Url:** click the Browse button and navigate to the **Ecoli\_metadata.xlsx** file located at `/home/dcuser/dc_sample_data/R`.
+You should now see a preview of the data to be imported:
+
+
+
+Notice that you have the option to change the data type of each variable by
+clicking arrow (drop-down menu) next to each column title. Under **Import
+Options** you may also rename the data, choose a different sheet to import, and
+choose how you will handle headers and skipped rows. Under **Code Preview** you
+can see the code that will be used to import this file. We could have written
+this code and imported the Excel file without the RStudio import function, but
+now you can choose your preference.
+
+In this exercise, we will leave the title of the data frame as
+**Ecoli\_metadata**, and there are no other options we need to adjust. Click the
+Import button to import the data.
+
+Finally, let's check the first few lines of the `Ecoli_metadata` data
+frame:
+
+
+
+
+```r
+head(Ecoli_metadata)
+```
+
+```{.output}
+# A tibble: 6 × 7
+ sample generation clade strain cit run genome_size
+
+1 REL606 0 NA REL606 unknown 4.62
+2 REL1166A 2000 unknown REL606 unknown SRR098028 4.63
+3 ZDB409 5000 unknown REL606 unknown SRR098281 4.6
+4 ZDB429 10000 UC REL606 unknown SRR098282 4.59
+5 ZDB446 15000 UC REL606 unknown SRR098283 4.66
+6 ZDB458 20000 (C1,C2) REL606 unknown SRR098284 4.63
+```
+
+The type of this object is 'tibble', a type of data
+frame we will talk more about in the 'dplyr' section. If you needed
+a true R data frame you could coerce with `as.data.frame()`.
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Exercise: Putting it all together - data frames
+
+**Using the `Ecoli_metadata` data frame created above, answer the following questions**
+
+A) What are the dimensions (# rows, # columns) of the data frame?
+
+B) What are categories are there in the `cit` column? *hint*: treat column as factor
+
+C) How many of each of the `cit` categories are there?
+
+D) What is the genome size for the 7th observation in this data set?
+
+E) What is the median value of the variable `genome_size`
+
+F) Rename the column `sample` to `sample_id`
+
+G) Create a new column (name genome\_size\_bp) and set it equal to the genome\_size multiplied by 1,000,000
+
+H) Save the edited Ecoli\_metadata data frame as "exercise\_solution.csv" in your current working directory.
+
+::::::::::::::: solution
+
+## Solution
+
+
+```r
+dim(Ecoli_metadata)
+```
+
+```{.output}
+[1] 30 7
+```
+
+```r
+levels(as.factor(Ecoli_metadata$cit))
+```
+
+```{.output}
+[1] "minus" "plus" "unknown"
+```
+
+```r
+table(as.factor(Ecoli_metadata$cit))
+```
+
+```{.output}
+
+ minus plus unknown
+ 9 9 12
+```
+
+```r
+Ecoli_metadata[7,7]
+```
+
+```{.output}
+# A tibble: 1 × 1
+ genome_size
+
+1 4.62
+```
+
+```r
+median(Ecoli_metadata$genome_size)
+```
+
+```{.output}
+[1] 4.625
+```
+
+```r
+colnames(Ecoli_metadata)[colnames(Ecoli_metadata) == "sample"] <- "sample_id"
+Ecoli_metadata$genome_size_bp <- Ecoli_metadata$genome_size * 1000000
+write.csv(Ecoli_metadata, file = "exercise_solution.csv")
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::::::::::::::::::: keypoints
+
+- It is easy to import data into R from tabular formats including Excel. However, you still need to check that R has imported and interpreted your data correctly
+- There are best practices for organizing your data (keeping it tidy) and R is great for this
+- Base R has many useful functions for manipulating your data, but all of R's capabilities are greatly enhanced by software packages developed by the community
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
diff --git a/04-bioconductor-vcfr.md b/04-bioconductor-vcfr.md
new file mode 100644
index 000000000..8e290e5dd
--- /dev/null
+++ b/04-bioconductor-vcfr.md
@@ -0,0 +1,148 @@
+---
+title: Using packages from Bioconductor
+teaching: 10
+exercises: 3
+source: Rmd
+---
+
+::::::::::::::::::::::::::::::::::::::: objectives
+
+- Describe what the Bioconductor repository is and what it is used for
+- Describe how Bioconductor differs from CRAN
+- Search Bioconductor for relevant packages
+- Install a package from Bioconductor
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::::::::::::::::::: questions
+
+- How do I use packages from the Bioconductor repository?
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+
+## Installing packages from somewhere else besides CRAN?
+
+So far we have told you about using packages that are included in the base installation of R (this is what comes with R 'out of the box'), and packages that you can install from [CRAN](https://cran.r-project.org/) (the Comprehensive R Archive Network), which is the primary place many people look for supplemental R packages to install. However, not all R packages are available on CRAN. For bioinformatics-related packages in particular, there is another repository that has many powerful packages that you can install. It is called [Bioconductor](https://bioconductor.org/) and it is a repository specifically focused on bioinformatics packages. [Bioconductor](https://bioconductor.org/) has a mission of "promot[ing] the statistical analysis and comprehension of current and emerging high-throughput biological assays." This means that many if not all of the packages available on Bioconductor are focused on the analysis of biological data, and that it can be a great place to look for tools to help you analyze your -omics datasets!
+
+## So how do I use it?
+
+Since access to the [Bioconductor](https://bioconductor.org/) repository is not built in to base R 'out of the box', there are a couple steps needed to install packages from this alternative source. We will work through the steps (only 2!) to install a package to help with the VCF analysis we are working on, but you can use the same approach to install any of the many thousands of available packages.
+
+![](fig/bioconductor_website_screenshot.jpg){alt='screenshot of bioconductor homepage'}
+
+## First, install the `BiocManager` package
+
+The first step is to install a package that *is* on CRAN, `BiocManager`. This package will allow us to use it to install packages from Bioconductor. You can think of Bioconductor kind of like an alternative app store for your phone, except instead of apps you are installing packages, and instead of your phone it's your local R package library.
+
+
+```r
+# install the BiocManager from CRAN using the base R install.packages() function
+install.packages("BiocManager")
+```
+
+To check if this worked (and also so you can make a note of the version for reproducibility purposes), you can run `BiocManager::version()` and it should give you the version number.
+
+
+```r
+# to make sure it worked, check the version
+BiocManager::version()
+```
+
+## Second, install the vcfR package from Bioconductor using `BiocManager`
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Head's Up: Installing vcfR may take a while due to numerous dependencies
+
+Just be aware that installing packages that have many dependencies can take a while.
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+```r
+# install the vcfR package from bioconductor using BiocManager::install()
+BiocManager::install("vcfR")
+```
+
+Depending on your particular system, you may need to also allow it to install some dependencies or update installed packages in order to successfully complete the process.
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Note: Installing packages from Bioconductor vs from CRAN
+
+Some packages begin by being available only on Bioconductor, and then later
+move to CRAN. `vcfR` is one such package, which originally was only available
+from Bioconductor, but is currently available from CRAN. The other thing to
+know is that `BiocManager::install()` will also install packages from CRAN (it
+is a wrapper around `install.packages()` that adds some extra features). There
+are [other benefits to using `BiocManager::install()` for Bioconductor
+packages](https://www.bioconductor.org/install/).
+In short, Bioconductor packages
+have a release cycle that is different from CRAN and the `install()` function
+is aware of that difference, so it helps to keep package versions in line with
+one another in a way that doesn't generally happen with the base R
+`install.packages()`.
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Search for Bioconductor packages based on your analysis needs
+
+While we are only focusing in this workshop on VCF analyses, there are hundreds or thousands of different types of data and analyses that bioinformaticians may want to work with. Sometimes you may get a new dataset and not know exactly where to start with analyzing or visualizing it. The Bioconductor package search view can be a great way to browse through the packages that are available.
+
+![](fig/bioconductor_search.jpg){alt='screenshot of bioconductor search'}
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Searching for packages on the Bioconductor website
+
+There are several thousand packages available through the Bioconductor website.
+It can be a bit of a challenge to find what you want, but one helpful resource
+is the [package search page](https://bioconductor.org/packages/release/BiocViews.html#___Software).
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+In bioinformatics, there are often many different tools that can be used in a
+particular instance. The authors of `vcfR` have [compiled some of
+them](https://github.com/knausb/vcfR#software-that-produce-vcf-files). One of
+those packages that [is available from
+Bioconductor](https://bioconductor.org/packages/release/bioc/html/VariantAnnotation.html)
+is called `VariantAnnotation` and may also be of interest to those working with
+vcf files in R.
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge
+
+- Use the `BiocManager::available()` function to see what packages are available matching a search term.
+- Use the [biocViews](https://bioconductor.org/packages/release/BiocViews.html#___Software) interface to search for packages of interest.
+
+You may or may not want to try installing the package, since not all dependencies always install easily. However, this will at least let you see what is available.
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Refreshing the RStudio package view after installing
+
+If you install a package from Bioconductor, you may need to refresh the RStudio package view to see it in your list. You can do this by clicking the "Refresh" button in the Packages pane of RStudio.
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Resources
+
+- [Bioconductor](https://bioconductor.org/)
+- [Bioconductor package search](https://bioconductor.org/packages/release/BiocViews.html#___Software)
+- [CRAN](https://cran.r-project.org/)
+
+:::::::::::::::::::::::::::::::::::::::: keypoints
+
+- Bioconductor is an alternative package repository for bioinformatics packages.
+- Installing packages from Bioconductor requires a new method, since it is not compatible with the `install.packages()` function used for CRAN.
+- Check Bioconductor to see if there is a package relevant to your analysis before writing code yourself.
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
diff --git a/05-dplyr.md b/05-dplyr.md
new file mode 100644
index 000000000..571c6982a
--- /dev/null
+++ b/05-dplyr.md
@@ -0,0 +1,988 @@
+---
+title: Data Wrangling and Analyses with Tidyverse
+teaching: 40
+exercises: 15
+source: Rmd
+---
+
+::::::::::::::::::::::::::::::::::::::: objectives
+
+- Describe what the `dplyr` package in R is used for.
+- Apply common `dplyr` functions to manipulate data in R.
+- Employ the ‘pipe’ operator to link together a sequence of functions.
+- Employ the ‘mutate’ function to apply other chosen functions to existing columns and create new columns of data.
+- Employ the ‘split-apply-combine’ concept to split the data into groups, apply analysis to each group, and combine the results.
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::::::::::::::::::: questions
+
+- How can I manipulate data frames without repeating myself?
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+
+Bracket subsetting is handy, but it can be cumbersome and difficult to read, especially for complicated operations.
+
+Luckily, the [`dplyr`](https://cran.r-project.org/package=dplyr)
+package provides a number of very useful functions for manipulating data frames
+in a way that will reduce repetition, reduce the probability of making
+errors, and probably even save you some typing. As an added bonus, you might
+even find the `dplyr` grammar easier to read.
+
+Here we're going to cover some of the most commonly used functions as well as using
+pipes (`%>%`) to combine them:
+
+1. `glimpse()`
+2. `select()`
+3. `filter()`
+4. `group_by()`
+5. `summarize()`
+6. `mutate()`
+7. `pivot_longer` and `pivot_wider`
+
+Packages in R are sets of additional functions that let you do more
+stuff in R. The functions we've been using, like `str()`, come built into R;
+packages give you access to more functions. You need to install a package and
+then load it to be able to use it.
+
+
+```r
+install.packages("dplyr") ## installs dplyr package
+install.packages("tidyr") ## installs tidyr package
+install.packages("ggplot2") ## installs ggplot2 package
+install.packages("readr") ## install readr package
+```
+
+You might get asked to choose a CRAN mirror -- this is asking you to
+choose a site to download the package from. The choice doesn't matter too much; I'd recommend choosing the RStudio mirror.
+
+
+```r
+library("dplyr") ## loads in dplyr package to use
+library("tidyr") ## loads in tidyr package to use
+library("ggplot2") ## loads in ggplot2 package to use
+library("readr") ## load in readr package to use
+```
+
+You only need to install a package once per computer, but you need to load it
+every time you open a new R session and want to use that package.
+
+::::::::::::::::::::::::::::::::::::::::: callout
+
+## Tip: Installing packages
+
+It may be temping to install the `tidyverse` package, as it contains many
+useful collection of packages for this lesson and beyond. However, when
+teaching or following this lesson, we advise that participants install
+`dplyr`, `readr`, `ggplot2`, and `tidyr` individually as shown above.
+Otherwise, a substaial amount of the lesson will be spend waiting for the
+installation to complete.
+
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+## What is dplyr?
+
+The package `dplyr` is a fairly new (2014) package that tries to provide easy
+tools for the most common data manipulation tasks. This package is also included in the [`tidyverse` package](https://www.tidyverse.org/), which is a collection of eight different packages (`dplyr`, `ggplot2`, `tibble`, `tidyr`, `readr`, `purrr`, `stringr`, and `forcats`). It is built to work directly
+with data frames. The thinking behind it was largely inspired by the package
+`plyr` which has been in use for some time but suffered from being slow in some
+cases.` dplyr` addresses this by porting much of the computation to C++. An
+additional feature is the ability to work with data stored directly in an
+external database. The benefits of doing this are that the data can be managed
+natively in a relational database, queries can be conducted on that database,
+and only the results of the query returned.
+
+This addresses a common problem with R in that all operations are conducted in
+memory and thus the amount of data you can work with is limited by available
+memory. The database connections essentially remove that limitation in that you
+can have a database that is over 100s of GB, conduct queries on it directly and pull
+back just what you need for analysis in R.
+
+### Loading .csv files in tidy style
+
+The Tidyverse's `readr` package provides its own unique way of loading .csv files in to R using `read_csv()`, which is similar to `read.csv()`. `read_csv()` allows users to load in their data faster, doesn't create row names, and allows you to access non-standard variable names (ie. variables that start with numbers of contain spaces), and outputs your data on the R console in a tidier way. In short, it's a much friendlier way of loading in potentially messy data.
+
+Now let's load our vcf .csv file using `read_csv()`:
+
+
+
+### Taking a quick look at data frames
+
+Similar to `str()`, which comes built into R, `glimpse()` is a `dplyr` function that (as the name suggests) gives a glimpse of the data frame.
+
+
+```{.output}
+Rows: 801
+Columns: 29
+$ sample_id "SRR2584863", "SRR2584863", "SRR2584863", "SRR2584863", …
+$ CHROM "CP000819.1", "CP000819.1", "CP000819.1", "CP000819.1", …
+$ POS 9972, 263235, 281923, 433359, 473901, 648692, 1331794, 1…
+$ ID NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, …
+$ REF "T", "G", "G", "CTTTTTTT", "CCGC", "C", "C", "G", "ACAGC…
+$ ALT "G", "T", "T", "CTTTTTTTT", "CCGCGC", "T", "A", "A", "AC…
+$ QUAL 91.0000, 85.0000, 217.0000, 64.0000, 228.0000, 210.0000,…
+$ FILTER NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, …
+$ INDEL FALSE, FALSE, FALSE, TRUE, TRUE, FALSE, FALSE, FALSE, TR…
+$ IDV NA, NA, NA, 12, 9, NA, NA, NA, 2, 7, NA, NA, NA, NA, NA,…
+$ IMF NA, NA, NA, 1.000000, 0.900000, NA, NA, NA, 0.666667, 1.…
+$ DP 4, 6, 10, 12, 10, 10, 8, 11, 3, 7, 9, 20, 12, 19, 15, 10…
+$ VDB 0.0257451, 0.0961330, 0.7740830, 0.4777040, 0.6595050, 0…
+$ RPB NA, 1.000000, NA, NA, NA, NA, NA, NA, NA, NA, 0.900802, …
+$ MQB NA, 1.0000000, NA, NA, NA, NA, NA, NA, NA, NA, 0.1501340…
+$ BQB NA, 1.000000, NA, NA, NA, NA, NA, NA, NA, NA, 0.750668, …
+$ MQSB NA, NA, 0.974597, 1.000000, 0.916482, 0.916482, 0.900802…
+$ SGB -0.556411, -0.590765, -0.662043, -0.676189, -0.662043, -…
+$ MQ0F 0.000000, 0.166667, 0.000000, 0.000000, 0.000000, 0.0000…
+$ ICB NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, …
+$ HOB NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, …
+$ AC 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,…
+$ AN 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,…
+$ DP4 "0,0,0,4", "0,1,0,5", "0,0,4,5", "0,1,3,8", "1,0,2,7", "…
+$ MQ 60, 33, 60, 60, 60, 60, 60, 60, 60, 60, 25, 60, 10, 60, …
+$ Indiv "/home/dcuser/dc_workshop/results/bam/SRR2584863.aligned…
+$ gt_PL 1210, 1120, 2470, 910, 2550, 2400, 2080, 2550, 11128, 19…
+$ gt_GT 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,…
+$ gt_GT_alleles "G", "T", "T", "CTTTTTTTT", "CCGCGC", "T", "A", "A", "AC…
+```
+
+In the above output, we can already gather some information about `variants`, such as the number of rows and columns, column names, type of vector in the columns, and the first few entries of each column. Although what we see is similar to outputs of `str()`, this method gives a cleaner visual output.
+
+### Selecting columns and filtering rows
+
+To select columns of a data frame, use `select()`. The first argument to this function is the data frame (`variants`), and the subsequent arguments are the columns to keep.
+
+
+```r
+select(variants, sample_id, REF, ALT, DP)
+```
+
+```{.output}
+# A tibble: 801 × 4
+ sample_id REF ALT DP
+
+ 1 SRR2584863 T G 4
+ 2 SRR2584863 G T 6
+ 3 SRR2584863 G T 10
+ 4 SRR2584863 CTTTTTTT CTTTTTTTT 12
+ 5 SRR2584863 CCGC CCGCGC 10
+ 6 SRR2584863 C T 10
+ 7 SRR2584863 C A 8
+ 8 SRR2584863 G A 11
+ 9 SRR2584863 ACAGCCAGCCAGCCAGCCAGCCAGCCAGCCAG ACAGCCAGCCAGCCAGCCAGCCAGCC… 3
+10 SRR2584863 AT ATT 7
+# ℹ 791 more rows
+```
+
+To select all columns *except* certain ones, put a "-" in front of
+the variable to exclude it.
+
+
+```r
+select(variants, -CHROM)
+```
+
+```{.output}
+# A tibble: 801 × 28
+ sample_id POS ID REF ALT QUAL FILTER INDEL IDV IMF DP
+
+ 1 SRR2584863 9972 NA T G 91 NA FALSE NA NA 4
+ 2 SRR2584863 263235 NA G T 85 NA FALSE NA NA 6
+ 3 SRR2584863 281923 NA G T 217 NA FALSE NA NA 10
+ 4 SRR2584863 433359 NA CTTTTTTT CTTT… 64 NA TRUE 12 1 12
+ 5 SRR2584863 473901 NA CCGC CCGC… 228 NA TRUE 9 0.9 10
+ 6 SRR2584863 648692 NA C T 210 NA FALSE NA NA 10
+ 7 SRR2584863 1331794 NA C A 178 NA FALSE NA NA 8
+ 8 SRR2584863 1733343 NA G A 225 NA FALSE NA NA 11
+ 9 SRR2584863 2103887 NA ACAGCCA… ACAG… 56 NA TRUE 2 0.667 3
+10 SRR2584863 2333538 NA AT ATT 167 NA TRUE 7 1 7
+# ℹ 791 more rows
+# ℹ 17 more variables: VDB , RPB , MQB , BQB , MQSB ,
+# SGB , MQ0F , ICB , HOB , AC , AN , DP4 ,
+# MQ , Indiv , gt_PL , gt_GT , gt_GT_alleles
+```
+
+`dplyr` also provides useful functions to select columns based on their names. For instance, `ends_with()` allows you to select columns that ends with specific letters. For instance, if you wanted to select columns that end with the letter "B":
+
+
+```r
+select(variants, ends_with("B"))
+```
+
+```{.output}
+# A tibble: 801 × 8
+ VDB RPB MQB BQB MQSB SGB ICB HOB
+
+ 1 0.0257 NA NA NA NA -0.556 NA NA
+ 2 0.0961 1 1 1 NA -0.591 NA NA
+ 3 0.774 NA NA NA 0.975 -0.662 NA NA
+ 4 0.478 NA NA NA 1 -0.676 NA NA
+ 5 0.660 NA NA NA 0.916 -0.662 NA NA
+ 6 0.268 NA NA NA 0.916 -0.670 NA NA
+ 7 0.624 NA NA NA 0.901 -0.651 NA NA
+ 8 0.992 NA NA NA 1.01 -0.670 NA NA
+ 9 0.902 NA NA NA 1 -0.454 NA NA
+10 0.568 NA NA NA 1.01 -0.617 NA NA
+# ℹ 791 more rows
+```
+
+::::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge
+
+Create a table that contains all the columns with the letter "i" and column "POS",
+without columns "Indiv" and "FILTER".
+Hint: look at for a function called `contains()`, which can be found in the help documentation for ends with we just covered (`?ends_with`). Note that contains() is not case sensistive.
+
+::::::::::::::: solution
+
+## Solution
+
+
+```r
+# First, we select "POS" and all columns with letter "i". This will contain columns Indiv and FILTER.
+variants_subset <- select(variants, POS, contains("i"))
+# Next, we remove columns Indiv and FILTER
+variants_result <- select(variants_subset, -Indiv, -FILTER)
+variants_result
+```
+
+```{.output}
+# A tibble: 801 × 7
+ POS sample_id ID INDEL IDV IMF ICB
+
+ 1 9972 SRR2584863 NA FALSE NA NA NA
+ 2 263235 SRR2584863 NA FALSE NA NA NA
+ 3 281923 SRR2584863 NA FALSE NA NA NA
+ 4 433359 SRR2584863 NA TRUE 12 1 NA
+ 5 473901 SRR2584863 NA TRUE 9 0.9 NA
+ 6 648692 SRR2584863 NA FALSE NA NA NA
+ 7 1331794 SRR2584863 NA FALSE NA NA NA
+ 8 1733343 SRR2584863 NA FALSE NA NA NA
+ 9 2103887 SRR2584863 NA TRUE 2 0.667 NA
+10 2333538 SRR2584863 NA TRUE 7 1 NA
+# ℹ 791 more rows
+```
+
+:::::::::::::::::::::::::
+
+We can also get to `variants_result` in one line of code:
+
+::::::::::::::: solution
+
+## Alternative solution
+
+
+```r
+variants_result <- select(variants, POS, contains("i"), -Indiv, -FILTER)
+variants_result
+```
+
+```{.output}
+# A tibble: 801 × 7
+ POS sample_id ID INDEL IDV IMF ICB
+
+ 1 9972 SRR2584863 NA FALSE NA NA NA
+ 2 263235 SRR2584863 NA FALSE NA NA NA
+ 3 281923 SRR2584863 NA FALSE NA NA NA
+ 4 433359 SRR2584863 NA TRUE 12 1 NA
+ 5 473901 SRR2584863 NA TRUE 9 0.9 NA
+ 6 648692 SRR2584863 NA FALSE NA NA NA
+ 7 1331794 SRR2584863 NA FALSE NA NA NA
+ 8 1733343 SRR2584863 NA FALSE NA NA NA
+ 9 2103887 SRR2584863 NA TRUE 2 0.667 NA
+10 2333538 SRR2584863 NA TRUE 7 1 NA
+# ℹ 791 more rows
+```
+
+:::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::::
+
+To choose rows, use `filter()`:
+
+
+```r
+filter(variants, sample_id == "SRR2584863")
+```
+
+```{.output}
+# A tibble: 25 × 29
+ sample_id CHROM POS ID REF ALT QUAL FILTER INDEL IDV IMF
+
+ 1 SRR2584863 CP000819… 9.97e3 NA T G 91 NA FALSE NA NA
+ 2 SRR2584863 CP000819… 2.63e5 NA G T 85 NA FALSE NA NA
+ 3 SRR2584863 CP000819… 2.82e5 NA G T 217 NA FALSE NA NA
+ 4 SRR2584863 CP000819… 4.33e5 NA CTTT… CTTT… 64 NA TRUE 12 1
+ 5 SRR2584863 CP000819… 4.74e5 NA CCGC CCGC… 228 NA TRUE 9 0.9
+ 6 SRR2584863 CP000819… 6.49e5 NA C T 210 NA FALSE NA NA
+ 7 SRR2584863 CP000819… 1.33e6 NA C A 178 NA FALSE NA NA
+ 8 SRR2584863 CP000819… 1.73e6 NA G A 225 NA FALSE NA NA
+ 9 SRR2584863 CP000819… 2.10e6 NA ACAG… ACAG… 56 NA TRUE 2 0.667
+10 SRR2584863 CP000819… 2.33e6 NA AT ATT 167 NA TRUE 7 1
+# ℹ 15 more rows
+# ℹ 18 more variables: DP , VDB , RPB , MQB , BQB ,
+# MQSB , SGB , MQ0F , ICB , HOB , AC ,
+# AN , DP4 , MQ , Indiv , gt_PL , gt_GT ,
+# gt_GT_alleles
+```
+
+`filter()` will keep all the rows that match the conditions that are provided.
+Here are a few examples:
+
+
+```r
+# rows for which the reference genome has T or G
+filter(variants, REF %in% c("T", "G"))
+```
+
+```{.output}
+# A tibble: 340 × 29
+ sample_id CHROM POS ID REF ALT QUAL FILTER INDEL IDV IMF DP
+
+ 1 SRR25848… CP00… 9.97e3 NA T G 91 NA FALSE NA NA 4
+ 2 SRR25848… CP00… 2.63e5 NA G T 85 NA FALSE NA NA 6
+ 3 SRR25848… CP00… 2.82e5 NA G T 217 NA FALSE NA NA 10
+ 4 SRR25848… CP00… 1.73e6 NA G A 225 NA FALSE NA NA 11
+ 5 SRR25848… CP00… 2.62e6 NA G T 31.9 NA FALSE NA NA 12
+ 6 SRR25848… CP00… 3.00e6 NA G A 225 NA FALSE NA NA 15
+ 7 SRR25848… CP00… 3.91e6 NA G T 225 NA FALSE NA NA 10
+ 8 SRR25848… CP00… 9.97e3 NA T G 214 NA FALSE NA NA 10
+ 9 SRR25848… CP00… 1.06e4 NA G A 225 NA FALSE NA NA 11
+10 SRR25848… CP00… 6.40e4 NA G A 225 NA FALSE NA NA 18
+# ℹ 330 more rows
+# ℹ 17 more variables: VDB , RPB , MQB , BQB , MQSB ,
+# SGB , MQ0F , ICB , HOB , AC , AN , DP4 ,
+# MQ , Indiv , gt_PL , gt_GT , gt_GT_alleles
+```
+
+```r
+# rows that have TRUE in the column INDEL
+filter(variants, INDEL)
+```
+
+```{.output}
+# A tibble: 101 × 29
+ sample_id CHROM POS ID REF ALT QUAL FILTER INDEL IDV IMF DP
+
+ 1 SRR25848… CP00… 4.33e5 NA CTTT… CTTT… 64 NA TRUE 12 1 12
+ 2 SRR25848… CP00… 4.74e5 NA CCGC CCGC… 228 NA TRUE 9 0.9 10
+ 3 SRR25848… CP00… 2.10e6 NA ACAG… ACAG… 56 NA TRUE 2 0.667 3
+ 4 SRR25848… CP00… 2.33e6 NA AT ATT 167 NA TRUE 7 1 7
+ 5 SRR25848… CP00… 3.90e6 NA A AC 43.4 NA TRUE 2 1 2
+ 6 SRR25848… CP00… 4.43e6 NA TGG T 228 NA TRUE 10 1 10
+ 7 SRR25848… CP00… 1.48e5 NA AGGGG AGGG… 122 NA TRUE 8 1 8
+ 8 SRR25848… CP00… 1.58e5 NA GTTT… GTTT… 19.5 NA TRUE 6 1 6
+ 9 SRR25848… CP00… 1.73e5 NA CAA CA 180 NA TRUE 11 1 11
+10 SRR25848… CP00… 1.75e5 NA GAA GA 194 NA TRUE 10 1 10
+# ℹ 91 more rows
+# ℹ 17 more variables: VDB , RPB , MQB , BQB , MQSB ,
+# SGB , MQ0F , ICB , HOB , AC , AN , DP4 ,
+# MQ , Indiv , gt_PL , gt_GT , gt_GT_alleles
+```
+
+```r
+# rows that don't have missing data in the IDV column
+filter(variants, !is.na(IDV))
+```
+
+```{.output}
+# A tibble: 101 × 29
+ sample_id CHROM POS ID REF ALT QUAL FILTER INDEL IDV IMF DP
+
+ 1 SRR25848… CP00… 4.33e5 NA CTTT… CTTT… 64 NA TRUE 12 1 12
+ 2 SRR25848… CP00… 4.74e5 NA CCGC CCGC… 228 NA TRUE 9 0.9 10
+ 3 SRR25848… CP00… 2.10e6 NA ACAG… ACAG… 56 NA TRUE 2 0.667 3
+ 4 SRR25848… CP00… 2.33e6 NA AT ATT 167 NA TRUE 7 1 7
+ 5 SRR25848… CP00… 3.90e6 NA A AC 43.4 NA TRUE 2 1 2
+ 6 SRR25848… CP00… 4.43e6 NA TGG T 228 NA TRUE 10 1 10
+ 7 SRR25848… CP00… 1.48e5 NA AGGGG AGGG… 122 NA TRUE 8 1 8
+ 8 SRR25848… CP00… 1.58e5 NA GTTT… GTTT… 19.5 NA TRUE 6 1 6
+ 9 SRR25848… CP00… 1.73e5 NA CAA CA 180 NA TRUE 11 1 11
+10 SRR25848… CP00… 1.75e5 NA GAA GA 194 NA TRUE 10 1 10
+# ℹ 91 more rows
+# ℹ 17 more variables: VDB , RPB , MQB , BQB , MQSB ,
+# SGB , MQ0F , ICB , HOB , AC , AN , DP4 ,
+# MQ , Indiv , gt_PL , gt_GT , gt_GT_alleles
+```
+
+We have a column titled "QUAL". This is a Phred-scaled confidence
+score that a polymorphism exists at this position given the sequencing
+data. Lower QUAL scores indicate low probability of a polymorphism
+existing at that site. `filter()` can be useful for selecting mutations that
+have a QUAL score above a certain threshold:
+
+
+```r
+# rows with QUAL values greater than or equal to 100
+filter(variants, QUAL >= 100)
+```
+
+```{.output}
+# A tibble: 666 × 29
+ sample_id CHROM POS ID REF ALT QUAL FILTER INDEL IDV IMF DP
+
+ 1 SRR25848… CP00… 2.82e5 NA G T 217 NA FALSE NA NA 10
+ 2 SRR25848… CP00… 4.74e5 NA CCGC CCGC… 228 NA TRUE 9 0.9 10
+ 3 SRR25848… CP00… 6.49e5 NA C T 210 NA FALSE NA NA 10
+ 4 SRR25848… CP00… 1.33e6 NA C A 178 NA FALSE NA NA 8
+ 5 SRR25848… CP00… 1.73e6 NA G A 225 NA FALSE NA NA 11
+ 6 SRR25848… CP00… 2.33e6 NA AT ATT 167 NA TRUE 7 1 7
+ 7 SRR25848… CP00… 2.41e6 NA A C 104 NA FALSE NA NA 9
+ 8 SRR25848… CP00… 2.45e6 NA A C 225 NA FALSE NA NA 20
+ 9 SRR25848… CP00… 2.67e6 NA A T 225 NA FALSE NA NA 19
+10 SRR25848… CP00… 3.00e6 NA G A 225 NA FALSE NA NA 15
+# ℹ 656 more rows
+# ℹ 17 more variables: VDB , RPB , MQB , BQB , MQSB ,
+# SGB , MQ0F , ICB , HOB , AC , AN , DP4 ,
+# MQ