worksheet_wrangling/worksheet_wrangling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "8474d060f4cbdef2ac14512fcf3f85dc",
     "grade": false,
     "grade_id": "cell-a3e4ac7f29e7ee20",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "# Worksheet 3: Cleaning and Wrangling Data\n",
    "\n",
    "\n",
    "### Lecture and Tutorial Learning Goals:\n",
    "\n",
    "After completing this week's lecture and tutorial work, you will be able to:\n",
    "\n",
    "* distinguish vectors and data frames in R, and how they relate to each other\n",
    "* define the term \"tidy data\"\n",
    "* discuss the advantages and disadvantages of storing data in a tidy data format\n",
    "* recall and use the following tidyverse functions and operators for their intended data wrangling tasks:\n",
    "    - `select`\n",
    "    - `filter`\n",
    "    - `|>`\n",
    "    - `map`\n",
    "    - `mutate`\n",
    "    - `summarize`\n",
    "    - `group_by`\n",
    "    - `pivot_longer`\n",
    "    - `separate`\n",
    "    - `%in%`\n",
    "\n",
    "This worksheet covers parts of [the Wrangling chapter](https://datasciencebook.ca/wrangling.html) of the online textbook. You should read this chapter before attempting the worksheet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "207bf88059639d30a09e0e5cd0558c48",
     "grade": false,
     "grade_id": "cell-9ad6a603a70e118a",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "### Run this cell before continuing. \n",
    "library(tidyverse)\n",
    "library(repr)\n",
    "source(\"tests.R\")\n",
    "source(\"cleanup.R\")\n",
    "options(repr.matrix.max.rows = 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "32ea0009671c5410cda436e494f880bb",
     "grade": false,
     "grade_id": "cell-0d59d96414adc1b8",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 0.0** Multiple Choice: \n",
    "<br> {points: 1}\n",
    "\n",
    "Which statement below is incorrect about vectors and data frames in R?\n",
    "\n",
    "A. the columns of data frames are vectors\n",
    "\n",
    "B. data frames can have columns of different types (e.g., a column of numeric data, and a column of character data)\n",
    "\n",
    "C. vectors can have elements of different types (e.g., element one can be numeric, and element 2 can be a character)\n",
    "\n",
    "D. data frames are a special kind of list\n",
    "\n",
    "\n",
    "*Assign your answer to an object called `answer0.0`. Make sure your answer is an uppercase letter and is surrounded by quotation marks (e.g. `\"F\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "73c7403b671a429a0e2414467fba10ec",
     "grade": false,
     "grade_id": "cell-cffd5df5830ecafd",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "0314d2ff90143157acd951c48a4f6f4a",
     "grade": true,
     "grade_id": "cell-0bea903e0f37148c",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_0.0()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "aa15ea35e2065a6be1a7393c9bd15d7b",
     "grade": false,
     "grade_id": "cell-77d758b913b7fb7d",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 0.1** Multiple Choice: \n",
    "<br> {points: 1}\n",
    "\n",
    "Which of the following does **_not_** characterize a tidy dataset?\n",
    "\n",
    "A. each row is a single observation\n",
    "\n",
    "B. each value should not be in a single cell\n",
    "\n",
    "C. each column is a single variable\n",
    "\n",
    "D. each value is a single cell\n",
    "\n",
    "\n",
    "*Assign your answer to an object called `answer0.1`. Make sure your answer is an uppercase letter and is surrounded by quotation marks (e.g. `\"F\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "fdad5116018a86871d0cbfd227b1bcdf",
     "grade": false,
     "grade_id": "cell-875bec2db6814c4e",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "a254df2e51db780c622dad1ac7a6de30",
     "grade": true,
     "grade_id": "cell-7bc5804c8cd5900d",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_0.1()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "b9e60ff519c442b408227bba4d5f294e",
     "grade": false,
     "grade_id": "cell-122d814c0cfa7294",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 0.2** Multiple Choice: \n",
    "<br> {points: 1}\n",
    "\n",
    "For which scenario would using one of the `group_by()` + `summarize()` be appropriate?\n",
    "\n",
    "A. To apply the same function to every row. \n",
    "\n",
    "B. To apply the same function to every column.\n",
    "\n",
    "C. To apply the same function to groups of rows. \n",
    "\n",
    "D. To apply the same function to groups of columns.\n",
    "\n",
    "*Assign your answer to an object called `answer0.2`.  Make sure your answer is an uppercase letter and is surrounded by quotation marks (e.g. `\"F\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "e143827671391169e0a27129d6f3246c",
     "grade": false,
     "grade_id": "cell-85fb659000512dcd",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "337dd1f69cce775e0bcbfd01ee75c50b",
     "grade": true,
     "grade_id": "cell-386ada4b41ae9cae",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_0.2()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "2bb97b950e612b69f314882d5b30be09",
     "grade": false,
     "grade_id": "cell-a181b4e3f1a49cf7",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 0.3** Multiple Choice: \n",
    "<br> {points: 1}\n",
    "\n",
    "For which scenario would using one of the `purrr` `map_*` functions be appropriate?\n",
    "\n",
    "A. To apply the same function to groups of rows.\n",
    "\n",
    "B. To apply the same function to every column.\n",
    "\n",
    "C. To apply the same function to groups of columns. \n",
    "\n",
    "D. All of the above.\n",
    "\n",
    "*Assign your answer to an object called `answer0.3`. Make sure your answer is an uppercase letter and is surrounded by quotation marks (e.g. `\"F\"`).**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "4adf7048c61347f737d36efd15fbff2b",
     "grade": false,
     "grade_id": "cell-32255c3e7f7a4e6c",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "2bd034b561fd53812e487104e3f31b0d",
     "grade": true,
     "grade_id": "cell-efb75338147fc001",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_0.3()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "8ce794daf58240c87dc3816f2d6868f2",
     "grade": false,
     "grade_id": "cell-fa871a669adbf899",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "## 1. Assessing avocado prices to inform restaurant menu planning\n",
    "\n",
    "It is a well known that millennials LOVE avocado toast (joking...well mostly 😉), and so many restaurants will offer menu items that centre around this delicious food! Like many food items, avocado prices fluctuate. So a restaurant who wants to maximize profits on avocado-containing dishes might ask if there are times when the price of avocados are less expensive to purchase? If such times exist, this is when the restaurant should put avocado-containing dishes on the menu to maximize their profits for those dishes. \n",
    "\n",
    "<img align=\"left\" src=\"https://www.averiecooks.com/wp-content/uploads/2017/07/egghole-2.jpg\" width=\"150\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "6f879185b3739704219c1e2c00a8c06c",
     "grade": false,
     "grade_id": "cell-f911d0be7d9dc7f7",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "*Source: https://www.averiecooks.com/egg-hole-avocado-toast/*\n",
    "\n",
    "To answer this question we will analyze a data set of avocado sales from multiple US markets. This data was downloaded from the [Hass Avocado Board website](http://www.hassavocadoboard.com/) in May of 2018 & compiled into a single CSV. Each row in the data set contains weekly sales data for a region. The data set spans the year 2015-2018.\n",
    "\n",
    "Some relevant columns in the dataset:\n",
    "\n",
    "- `Date` - The date in year-month-day format\n",
    "- `average_price` - The average price of a single avocado\n",
    "- `type` - conventional or organic\n",
    "- `yr` - The year\n",
    "- `region` - The city or region of the observation\n",
    "- `small_hass_volume` in pounds (lbs)\t\n",
    "- `large_hass_volume` in pounds (lbs)\t\t\n",
    "- `extra_l_hass_volume`\tin pounds (lbs)\t\n",
    "- `wk` - integer number for the calendar week in the year (e.g., first week of January is 1, and last week of December is 52).\n",
    "\n",
    "To answer our question of whether there are times in the year when avocados are typically less expensive (and thus we can make more profitable menu items with them at a restaurant) we will want to create a scatter plot of `average_price` (y-axis) versus `Date` (x-axis)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "b70f48237dcfff91c82d6e063849d737",
     "grade": false,
     "grade_id": "cell-acb1e485a76f043b",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 1.1** Multiple Choice:\n",
    "<br> {points: 1}\n",
    "\n",
    "Which of the following is not included in the `csv` file?\n",
    "\n",
    "A. Average price of a single avocado.\n",
    "\n",
    "B. The farming practice (production with/without the use of chemicals). \n",
    "\n",
    "C. Average price of a bag of avocados.\n",
    "\n",
    "D. All options are included in the data set.\n",
    "\n",
    "*Assign your answer to an object called `answer1.1`. Make sure your answer is an uppercase letter and is surrounded by quotation marks (e.g. `\"F\"`).** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "82bc54b1241b3e44d1d0e6dd19e6258b",
     "grade": false,
     "grade_id": "cell-39161480e71d9b43",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "cfc95fa0c256fc1f9614d5a60d4162de",
     "grade": true,
     "grade_id": "cell-1c278f180e20468f",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.1()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "c81f6ac7523cc54fa7354c0bf3038fdb",
     "grade": false,
     "grade_id": "cell-56154f20b1c3af0b",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 1.2** Multiple Choice:\n",
    "<br> {points: 1}\n",
    "\n",
    "The rows in the data frame represent:\n",
    "\n",
    "A. daily avocado sales data for a region\n",
    "\n",
    "B. weekly avocado sales data for a region\n",
    "\n",
    "C. bi-weekly avocado sales data for a region\n",
    "\n",
    "D. yearly avocado sales data for a region\n",
    "\n",
    "*Assign your answer to an object called `answer1.2`. Make sure your answer is an uppercase letter and is surrounded by quotation marks (e.g. `\"F\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "655ad083ef09a5ba2923d3a9ba268c32",
     "grade": false,
     "grade_id": "cell-1d614695a9d4a4f9",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "b0d60f3d88b72cb86dc789ded1dfb09d",
     "grade": true,
     "grade_id": "cell-6adfe52857aa9333",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.2()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "1f6891412bdb99ff7256f7043f4d7bea",
     "grade": false,
     "grade_id": "cell-7e263f37387da63d",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 1.3** \n",
    "<br> {points: 1}\n",
    "\n",
    "The first step to plotting total volume against average price is to read the file `avocado_prices.csv` using the shortest relative path. The data file was given to you along with this worksheet, but you will have to look to see where it is in the `worksheet_03` directory to correctly load it. When you do this, you should also preview the file to help you choose an appropriate `read_*` function to read the data.\n",
    "\n",
    "*Assign your answer to an object called `avocado`.* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "aa4426491b18313dfae5ed796740b61a",
     "grade": false,
     "grade_id": "cell-434656ab0b99d8bd",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "#... <- ...(\"...\")\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "avocado "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "289eb66f735ebdd930c5e5f3a065553e",
     "grade": true,
     "grade_id": "cell-a968fbd8b038ba4b",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.3()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "72d9feec9842c0518744b2fd5de9e35f",
     "grade": false,
     "grade_id": "cell-8813edd076a1a198",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 1.4** Multiple Choice:\n",
    "<br> {points: 1}\n",
    "\n",
    "Why are the 2nd to 5th columns `col_double` instead of `col_integer`?\n",
    "\n",
    "A. They aren't \"real\" numbers. \n",
    "\n",
    "B. They contain decimals. \n",
    "\n",
    "C. They are numbers created using text/letters. \n",
    "\n",
    "D. They are `col_integer`... \n",
    "\n",
    "*Assign your answer to an object called `answer1.4`. Make sure your answer is an uppercase letter and is surrounded by quotation marks (e.g. `\"F\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "3bc7e3d54005917c40a0ed28dfb37a10",
     "grade": false,
     "grade_id": "cell-7b981c1250a92624",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Make sure the correct answer is an uppercase letter. \n",
    "# Surround your answer with quotation marks.\n",
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "ab0a8c96ef5e1e95118e961390a2fe39",
     "grade": true,
     "grade_id": "cell-962d5be15864a23e",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.4()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "782e74484a91ec0cffa051cf88e20533",
     "grade": false,
     "grade_id": "cell-31ff875d27e26cf4",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Before we get started doing our analysis, let's learn about the pipe operator, `|>`, as it can be very helpful when doing data analysis in R!\n",
    "\n",
    "### Pipe Operators: `|>`\n",
    "Pipe operators allow you to chain together different functions - it takes the output of one statement and makes it the input of the next statement. Having a chain of processing functions is known as a *pipeline*. \n",
    "\n",
    "If we wanted to subset the avocado data to obtain just the average prices for organic avocados, we would need to first filter the `type` column using the function: `filter()` for the rows where the type is organic. Then we would need to use the `select()` function to get just the average price column.\n",
    "\n",
    "Below we illustrate how to do this using the pipe operator, `|>`, instead of creating an intermediate object as we have in past worksheets: \n",
    "\n",
    "> *Note: the indentation on the second line of the pipeline is not required, but added for readability.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run this cell\n",
    "filter(avocado, type == \"organic\") |> \n",
    "    select(average_price)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "c09b63ea9d49b015be709c37a8bfba71",
     "grade": false,
     "grade_id": "cell-c0f8ea7d3833df9f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We can even start off a pipeline by passing the data frame into the first function. This is convenient and aids in readability. You will see this being used often in this course going forward. Below we show an example of this doing the same task we just completed above (subsetting the average price data for organic avocados)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "avocado |> \n",
    "    filter( type == \"organic\") |> \n",
    "    select(average_price)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "22c6811eecf6db2aa0423c328fff9fce",
     "grade": false,
     "grade_id": "cell-80ee505836e01bce",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 1.5**\n",
    "\n",
    "{points: 1}\n",
    "\n",
    "To answer our question, let's now create the scatter plot where we plot `average_price` on the y-axis versus `Date` on the x-axis. Fill in the ... in the cell below. Copy and paste your finished answer in place of `fail()`. Assign your answer to an object called `avocado_plot`. Don't forget to create proper English axis labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "e801dd9ee2c0c4e2c1c9d1def7894c6a",
     "grade": false,
     "grade_id": "cell-fd8156be131e6ab6",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "options(repr.plot.width = 14, repr.plot.height = 7) # Modifies the size of the plots\n",
    "#... <- ... |>\n",
    "#    ggplot(aes(x = ..., y = ...)) + \n",
    "#        geom_...() +\n",
    "#        xlab(\"...\") + \n",
    "#        ylab(\"...\") + \n",
    "#        theme(text = element_text(size=20))\n",
    "\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "avocado_plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "e5c1740c0568b65e8a58ac724f906962",
     "grade": true,
     "grade_id": "cell-89b7338558e28dbc",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.5()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "9465b3984d66440694cf1351ca05455e",
     "grade": false,
     "grade_id": "cell-f5bf4e53775781f8",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We might be able to squint and start to see some pattern in the data above, but really what we see in the plot above is not very informative. Why? Because there is a lot of overplotting (data points sitting on top of other data points). What can we do? One solution is to reduce/aggregate the data in a meaningful way to help anwer our question. Remember that we are interested in determining if there are times when the price of avocados are less expensive so that we can recommend when restaurants should put dishes on the menu that contain avocado to maximize their profits for those dishes.\n",
    "\n",
    "In the data we plotted above, each row is the total sales for avocados for that region for each year. Lets use `group_by` + `summarize` calculate the average price for each week across years and region. We can then plot that aggregated price against the week and perhaps get a clearer picture."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "99aef19dabfde6cd8f9179246c5f7e20",
     "grade": false,
     "grade_id": "cell-07909555b45c6c28",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 1.6**\n",
    "<br> {points: 1}\n",
    "\n",
    "Create a reduced/aggregated version of the `avocado` data set and name it `avocado_aggregate`. To do this you will want to `group_by` the `wk` column and then use `summarize` to calculate the average price (name that column `average_price`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "87a613361322e879ccca3ca2422a4501",
     "grade": false,
     "grade_id": "cell-25fc8f43f1d193db",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "#... <- ... |> \n",
    "#    group_by(...) |> \n",
    "#    summarize(... = mean(average_price, na.rm = TRUE))\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "avocado_aggregate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "e57fd588df1b21a2b87b6af27f115eff",
     "grade": true,
     "grade_id": "cell-81ec3e479caeb7d7",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.6()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "b26867888ffbb3769f20b5f5ce836682",
     "grade": false,
     "grade_id": "cell-aa0422ac0aade558",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 1.7**\n",
    "<br> {points: 1}\n",
    "\n",
    "Now let's take the `avocado_aggregate` data frame and use it to create a scatter plot where we plot `average_price` on the y-axis versus `wk` on the x-axis. Assign your answer to an object called `avocado_aggregate_plot`. Don't forget to create proper English axis labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d3beb3c289346dfa5f21b94370748bbd",
     "grade": false,
     "grade_id": "cell-e081bf3fd0e40162",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "#... <- ... |>\n",
    "#    ggplot(aes(x = ..., y = ...)) + \n",
    "#        ...() +\n",
    "#        ...(\"...\") + \n",
    "#        ...(\"...\") +\n",
    "#        theme(text = element_text(size=20))\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "avocado_aggregate_plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "f1b56442de638acdc91d986963850030",
     "grade": true,
     "grade_id": "cell-d70b07b4c2dc0202",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.7()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "536f294b841952b053e6f231262c2319",
     "grade": false,
     "grade_id": "cell-b8dfe798ae35428d",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We can now see that the prices of avocados does indeed fluctuate throughout the year. And we could use this information to recommend to restaurants that if they want to maximize profit from menu items that contain avocados, they should only offer them on the menu roughly between December and May. \n",
    "\n",
    "Why might this happen? Perhaps price has something to do with supply? We can also use this data set to get some insight into that question by plotting total avocado volume (y-axis) versus week. To do this, we will first have to create a column called `total_volume` whose value is the sum of the small, large and extra large-sized avocado volumes. To do this we will have to go back to the original `avocado` data frame we loaded."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "ad8ac615f9133ddadd61ad23a41ea6d1",
     "grade": false,
     "grade_id": "cell-11acd47d959662ee",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 1.8**\n",
    "<br> {points: 1}\n",
    "\n",
    "Our next step to plotting `total_volume` per week against week is to use `mutate` to create a new column in the `avocado` data frame called `total_volume` which is equal to the sum of all three volume columns:\n",
    "\n",
    "Fill in the `...` in the cell below. Copy and paste your finished answer and replace the `fail()`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "5274b14679b865aa0fde74d44afd48e2",
     "grade": false,
     "grade_id": "cell-20ec4476bb9540db",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "#... <- ... |>\n",
    "#     mutate(... = ... + ... + ...) \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "avocado"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "f88967c9f4b2f910dc354d4aa86ceadc",
     "grade": true,
     "grade_id": "cell-1b331febb2ce27b5",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.8()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "6e703f4172d907804a701ef63416c05c",
     "grade": false,
     "grade_id": "cell-97fc42a8c2d802d8",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 1.9** \n",
    "<br> {points: 1}\n",
    "\n",
    "Now, create another reduced/aggregated version of the `avocado` data frame and name it `avocado_aggregate_2`. To do this you will want to `group_by` the `wk` column and then use `summarize` to calculate the average total volume (name that column `total_volume`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "75e39d0f2120f42f83874b8f5558ae68",
     "grade": false,
     "grade_id": "cell-d3d0314b483daf3c",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "#... <- ... |> \n",
    "#    group_by(...) |> \n",
    "#    summarize(...)\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "avocado_aggregate_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "2113ab67114548ca65314f85259d12af",
     "grade": true,
     "grade_id": "cell-975338ad4661f5af",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.9()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "59114583ff055f715682d0748985136a",
     "grade": false,
     "grade_id": "cell-ea1a11950f66b4f5",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 1.10** \n",
    "<br> {points: 1}\n",
    "\n",
    "Now let's take the `avocado_aggregate_2` data frame and use it to create a scatter plot where we plot average `total_volume` (in pounds, lbs) on the y-axis versus `wk` on the x-axis. Assign your answer to an object called `avocado_aggregate_plot_2`. Don't forget to create proper English axis labels.\n",
    "\n",
    "> Hint: don't forget to include the units for volume in your data visualization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "a43a84f71030854f3b88155a5580b04d",
     "grade": false,
     "grade_id": "cell-11af7abfd81fb0f2",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "#... <- ... |>\n",
    "#    ggplot(aes(x = ..., y = ...)) + \n",
    "#        ...() +\n",
    "#        ...(\"...\") + \n",
    "#        ...(\"...\") +\n",
    "#        theme(text = element_text(size=20))\n",
    "\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "avocado_aggregate_plot_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "4a088bd0899940592c1240d866798b4a",
     "grade": true,
     "grade_id": "cell-7a0b7fca31c9c8ec",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_1.10()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "0e53d03902f6638ab153df8dfd95379d",
     "grade": false,
     "grade_id": "cell-83ce58105c97d477",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We can see from the above plot of the average total volume versus the week that there are more avocados sold (and perhaps this reflects what is available for sale) roughly between January to May. This time period of increased volume corresponds with the lower avocado prices. We can *hypothesize* (but not conclude, of course) that the lower prices may be due to an increased availability of avocados during this time period."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "df304d903814167320ad048736e36c94",
     "grade": false,
     "grade_id": "cell-15b71c3a5274adf6",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "## 2. Sea Surface Temperatures in Departure Bay\n",
    "The next data set that we will be looking at contains environmental data from 1914 to 2018. The data was collected by the DFO (Canada's Department of Fisheries and Oceans) at the Pacific Biological Station (Departure Bay). Daily sea surface temperature (in degrees Celsius) and salinity (in practical salinity units, PSU) observations have been carried out at several locations on the coast of British Columbia. The number of stations reporting at any given time has varied as sampling has been discontinued at some stations, and started or resumed at others.\n",
    "\n",
    "Presently termed the British Columbia Shore Station Oceanographic Program (BCSOP), there are 12 participating stations; most of these are staffed by Fisheries and Oceans Canada. You can look at data from other stations at http://www.pac.dfo-mpo.gc.ca/science/oceans/data-donnees/lightstations-phares/index-eng.html \n",
    "\n",
    "Further information from the Government of Canada's website indicates: \n",
    ">  Observations are made daily using seawater collected in a bucket lowered into the surface water at or near the daytime high tide. This sampling method was designed long ago by Dr. John P. Tully and has not been changed in the interests of a homogeneous data set. This means, for example, that if an observer starts sampling one day at 6 a.m., and continues to sample at the daytime high tide on the second day the sample will be taken at about 06:50 the next day, 07:40 the day after etc. When the daytime high-tide gets close to 6 p.m. the observer will then begin again to sample early in the morning, and the cycle continues. Since there is a day/night variation in the sea surface temperatures the daily time series will show a signal that varies with the14-day tidal cycle. This artifact does not affect the monthly sea surface temperature data.\n",
    "\n",
    "In this worksheet, we want to see if the sea surface temperature has been changing over time. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "f13331a2b99db0ac5fd1b4ceed05f238",
     "grade": false,
     "grade_id": "cell-8f4dc9433ea2a7e9",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 2.1** True or False:\n",
    "<br> {points: 1}\n",
    "\n",
    "The sampling of surface water occurs at the same time each day. \n",
    "\n",
    "*Assign your answer to an object called `answer2.1`. Make sure your answer is lowercase \"true\" or lowercase \"false\".* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "8ec97a92c46e8c64c2b16d1eeda1a06d",
     "grade": false,
     "grade_id": "cell-4eb1407dda8e1fe2",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "ba18db3e1ade149dae228da8b22cee45",
     "grade": true,
     "grade_id": "cell-aef90db69249870d",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.1()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "ba7a38ce80cead306bafd5c1eece5391",
     "grade": false,
     "grade_id": "cell-0155ff05e1bac66f",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 2.2** Multiple Choice:\n",
    "<br> {points: 1}\n",
    "\n",
    "If high tide occurred at 9am today, what time would the scientist collect data tomorrow?\n",
    "\n",
    "A. 11:10 am \n",
    "\n",
    "B. 9:50 am \n",
    "\n",
    "C. 10:00 pm \n",
    "\n",
    "D. Trick question... you skip days when collecting data. \n",
    "\n",
    "*Assign your answer to an object called `answer2.2`. Make sure your answer is an uppercase letter and is surrounded by quotation marks (e.g. `\"F\"`).* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "528f666f5284cc657d8125de2ed994b1",
     "grade": false,
     "grade_id": "cell-c93cda405137fbfd",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "354e1bec7b5e370d97f914f24e15fc7f",
     "grade": true,
     "grade_id": "cell-957f2b6edf976bfd",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.2()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "1b2717ffc63af7cd52a02706b52bc753",
     "grade": false,
     "grade_id": "cell-e1bf20ed85ae3d0d",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 2.3**\n",
    "<br> {points: 1}\n",
    "\n",
    "To begin working with this data, read the file `departure_bay_temperature.csv` using a relative path. Note, this file (just like the avocado data set) is found within the `worksheet_03` directory. \n",
    "\n",
    "*Assign your answer to an object called `sea_surface`.* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "cfc6b010459183ec8fd3b62ce030eaf3",
     "grade": false,
     "grade_id": "cell-b10c1a879331aa37",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "sea_surface"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "22675910c6b63be6fed9c9f0c2480641",
     "grade": true,
     "grade_id": "cell-09a9fd7ca9f44ada",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.3()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "b2943f4ae6816a7575bb5d137e7270ee",
     "grade": false,
     "grade_id": "cell-e755c156454253ba",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Question 2.3.1**\n",
    "<br> {points: 1}\n",
    "\n",
    "The data above in Question 2.3 is not tidy, which reasons listed below explain why?\n",
    "\n",
    "A. There are NA's in the data set\n",
    "\n",
    "B. The variable temperature is split across more than one column\n",
    "\n",
    "C. Values for the variable month are stored as column names\n",
    "\n",
    "D. A and C\n",
    "\n",
    "E. B and C\n",
    "\n",
    "F. All of the above\n",
    "\n",
    "Assign your answer to an object called `answer2.3.1`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7233a6a7a91152c7fd65d0dfa7cfb189",
     "grade": false,
     "grade_id": "cell-6e1789eb7032fc30",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "6273260afec61c22bc5e2b61ca7945a9",
     "grade": true,
     "grade_id": "cell-f449a87635bac905",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.3.1()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "a5a8e73f0ac6f2949ffc9afcf5d929ae",
     "grade": false,
     "grade_id": "cell-0b23c682c8caa306",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 2.4**\n",
    "<br> {points: 1}\n",
    "\n",
    "Given `ggplot` expects tidy data, we need to convert our data into that format. To do this we will use the `pivot_longer()` function. We would like our data to end up looking like this:\n",
    "\n",
    "| Year | Month | Temperature |\n",
    "|------|-------|-------------|\n",
    "| 1914 | Jan   | 7.2         |\n",
    "| 1914 | Feb   | NA          |\n",
    "| 1914 | Mar   | NA          |\n",
    "| ...  | ...   | ...         |\n",
    "| 2018 | Oct   | NA          |\n",
    "| 2018 | Nov   | NA          |\n",
    "| 2018 | Dec   | NA          |\n",
    "\n",
    "\n",
    "Fill in the `...` in the cell below. Copy and paste your finished answer and replace the `fail()`.\n",
    "\n",
    "*Assign your answer to an object called `tidy_temp`.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "246cd4dcbd0d5b667736eb5ac878fca7",
     "grade": false,
     "grade_id": "cell-422718995704f040",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "#... <- sea_surface |>\n",
    "#    ...(cols = Jan:Dec, \n",
    "#                 names_to = \"...\", \n",
    "#                 values_to = \"Temperature\")\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "tidy_temp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "f335d09d319494be0674f71d55e9beee",
     "grade": true,
     "grade_id": "cell-afb070ca8361d0a7",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.4()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "dd1bbc7a7b00bf348099703f993ebfef",
     "grade": false,
     "grade_id": "cell-4d3484d80d9f0854",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 2.5**\n",
    "<br> {points: 1}\n",
    "\n",
    "Now that we have our data in a tidy format, we can create our plot that compares the average monthly sea surface temperatures (in degrees Celsius) to the year they were recorded. To make our plots more informative, we should plot each month separately. We can use `filter` to do this before we pipe our data into the `ggplot` function. Let's start out by just plotting the data for the month of November. As usual, use proper English to label your axes :)\n",
    "\n",
    "*Assign your answer to an object called `nov_temp_plot`.*\n",
    "\n",
    "> Hint: don't forget to include the units for temperature in your data visualization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "49fca5ffb2061324fe2504e831e3d010",
     "grade": false,
     "grade_id": "cell-b1ac578751969b78",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "options(repr.plot.width = 12, repr.plot.height = 7)\n",
    "#... <- ... |> \n",
    "#    filter(... == ...) |> \n",
    "#    ggplot(aes(x = ..., y = ...)) + \n",
    "#    geom_point() + \n",
    "#    xlab(...) + \n",
    "#    ylab(...) +\n",
    "#    theme(text = element_text(size=20))\n",
    "\n",
    "\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "nov_temp_plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "633720c4a671cf10090f747fce20d584",
     "grade": true,
     "grade_id": "cell-8ddfbb3c8b82e695",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.5()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "e2d6e996639ebb98c1dc6a91d6860978",
     "grade": false,
     "grade_id": "cell-9de2b3c001bbe02d",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We can see that there may be a small decrease in colder temperatures in recent years, and/or the temperatures in recent years look less variable compared to years before 1975. What about other months? Let's plot them! \n",
    "\n",
    "Instead of repeating the code above for the 11 other months, we'll take advantage of a `ggplot2` function that we haven't met yet, `facet_wrap`. This function is used to create many plots side-by-side, and wrapped around to new lines if there are too many plots. You tell `ggplot2` how to split up the plots by specifying the argument `facets = vars(...)`, where `...` represents the variable that is used to split the plots. We will learn more about this function next week, this week we will give you the code for it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "bcd8994e43fe728824738e23aba981fa",
     "grade": false,
     "grade_id": "cell-27e2c5089e6710ef",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 2.6**\n",
    "<br> {points: 1}\n",
    "\n",
    "Fill in the missing code below to plot the average monthly sea surface temperatures to the year they were recorded for all months. Assign your answer to an object called `all_temp_plot`.\n",
    "\n",
    "> Hint: don't forget to include the units for temperature in your data visualization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "b5c514bade0d6cfcc40cf3f5d5348b2d",
     "grade": false,
     "grade_id": "cell-6a5d54a80ebdc73c",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "options(repr.plot.width = 14, repr.plot.height = 8)\n",
    "#... <- ... |> \n",
    "#    ggplot(aes(x = ..., y = ...)) + \n",
    "#    geom_point() + \n",
    "#    facet_wrap(facets = vars(factor(Month, levels = c(\"Jan\",\"Feb\",\"Mar\",\"Apr\",\"May\",\"Jun\",\n",
    "#                                          \"Jul\",\"Aug\",\"Sep\",\"Oct\",\"Nov\",\"Dec\")))) +\n",
    "#    xlab(...) + \n",
    "#    ylab(...) +\n",
    "#    theme(text = element_text(size=20))\n",
    "\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "all_temp_plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "964923f22dc559f667a23f4217efa1b7",
     "grade": true,
     "grade_id": "cell-4529c8b1eb657878",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_2.6()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "ae37273c57e7f7f4b8619bfe745f2ff5",
     "grade": false,
     "grade_id": "cell-5406840f482b0738",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We can see above that some months show a small, but general increase in temperatures, whereas others don't. And some months show a change in variability and others do not. From this it is clear to us that if we are trying to understand temperature changes over time, we best keep data from different months separate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "d8218a3c7217edb9483fa20667e9d437",
     "grade": false,
     "grade_id": "cell-4fed26cd837beb2b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "## 3. Pollution in Madrid\n",
    "We're working with a data set from Kaggle once again! [This data](https://www.kaggle.com/decide-soluciones/air-quality-madrid) was collected under the instructions from Madrid's City Council and is publicly available on their website. In recent years, high levels of pollution during certain dry periods has forced the authorities to take measures against the use of cars and act as a reasoning to propose certain regulations. This data includes daily and hourly measurements of air quality from 2001 to 2008. Pollutants are categorized based on their chemical properties.\n",
    "\n",
    "There are a number of stations set up around Madrid and each station's data frame contains all particle measurements that such station has registered from 01/2001 - 04/2008. Not every station has the same equipment, therefore each station can measure only a certain subset of particles. The complete list of possible measurements and their explanations are given by the website:\n",
    "\n",
    "- `SO_2`: sulphur dioxide level measured in μg/m³. High levels can produce irritation in the skin and membranes, and worsen asthma or heart diseases in sensitive groups.\n",
    "- `CO`: carbon monoxide level measured in mg/m³. Carbon monoxide poisoning involves headaches, dizziness and confusion in short exposures and can result in loss of consciousness, arrhythmias, seizures or even death.\n",
    "- `NO_2`: nitrogen dioxide level measured in μg/m³. Long-term exposure is a cause of chronic lung diseases, and are harmful for the vegetation.\n",
    "- `PM10`: particles smaller than 10 μm. Even though they cannot penetrate the alveolus, they can still penetrate through the lungs and affect other organs. Long term exposure can result in lung cancer and cardiovascular complications.\n",
    "- `NOx`: nitrous oxides level measured in μg/m³. Affect the human respiratory system worsening asthma or other diseases, and are responsible of the yellowish-brown color of photochemical smog.\n",
    "- `O_3`: ozone level measured in μg/m³. High levels can produce asthma, bronchytis or other chronic pulmonary diseases in sensitive groups or outdoor workers.\n",
    "- `TOL`: toluene (methylbenzene) level measured in μg/m³. Long-term exposure to this substance (present in tobacco smoke as well) can result in kidney complications or permanent brain damage.\n",
    "- `BEN`: benzene level measured in μg/m³. Benzene is a eye and skin irritant, and long exposures may result in several types of cancer, leukaemia and anaemias. Benzene is considered a group 1 carcinogenic to humans.\n",
    "- `EBE`: ethylbenzene level measured in μg/m³. Long term exposure can cause hearing or kidney problems and the IARC has concluded that long-term exposure can produce cancer.\n",
    "- `MXY`: m-xylene level measured in μg/m³. Xylenes can affect not only air but also water and soil, and a long exposure to high levels of xylenes can result in diseases affecting the liver, kidney and nervous system.\n",
    "- `PXY`: p-xylene level measured in μg/m³. See MXY for xylene exposure effects on health.\n",
    "- `OXY`: o-xylene level measured in μg/m³. See MXY for xylene exposure effects on health.\n",
    "- `TCH`: total hydrocarbons level measured in mg/m³. This group of substances can be responsible of different blood, immune system, liver, spleen, kidneys or lung diseases.\n",
    "- `NMHC`: non-methane hydrocarbons (volatile organic compounds) level measured in mg/m³. Long exposure to some of these substances can result in damage to the liver, kidney, and central nervous system. Some of them are suspected to cause cancer in humans."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "7ae6419d92c15b3cdd292ac8c76868bf",
     "grade": false,
     "grade_id": "cell-892f43f18911d0ba",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "The goal of this assignment is to see if pollutants are decreasing (is air quality improving) and also compare which pollutant has decreased the most over the span of 5 years (2001 - 2006). \n",
    "1. First do a plot of one of the pollutants (EBE). \n",
    "2. Next, group it by month and year; calculate the maximum value and plot it (to see the trend through time). \n",
    "3. Now we will look at which pollutant decreased the most. Repeat the same thing for every column - to speed up the process, use the `map()` function. First we will look at pollution in 2001 (get the maximum value for each of the pollutants). And then do the same for 2006. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "e311d83737d4b352f7c251cba753bea2",
     "grade": false,
     "grade_id": "cell-a99486a2f16469ae",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 3.1** Multiple Choice: \n",
    "<br> {points: 1}\n",
    "\n",
    "What big picture question are we trying to answer?\n",
    "\n",
    "A. Did EBE decrease in Madrid between 2001 and 2006?\n",
    "\n",
    "B. Of all the pollutants, which decreased the most between 2001 and 2006? \n",
    "\n",
    "C. Of all the pollutants, which decreased the least between 2001 and 2006?\n",
    "\n",
    "D. Did EBE increase in Madrid between 2001 and 2006?\n",
    "\n",
    "*Assign your answer to an object called `answer3.1`. Make sure your answer is an uppercase letter and is surrounded by quotation marks (e.g. `\"F\"`).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "ef0cd09d143de2057de811dd39b491bc",
     "grade": false,
     "grade_id": "cell-fffa52c5bf768a43",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "52f65805107a8a48fc3640016a77a212",
     "grade": true,
     "grade_id": "cell-d67db6d2cd3971aa",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_3.1()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "e719027bfac0e2844277d98fd28f8616",
     "grade": false,
     "grade_id": "cell-beefc03e5d6c203d",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 3.2** \n",
    "<br> {points: 1}\n",
    "\n",
    "To begin working with this data, read the file `madrid_pollution.csv`. Note, this file (just like the avocado and sea surface data set) is found in the `worksheet_wrangling` directory. \n",
    "\n",
    "*Assign your answer to an object called `madrid`.* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "decaa8c52f915536511884a665d29eeb",
     "grade": false,
     "grade_id": "cell-7f7939bcc9bebde2",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "madrid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "73a7b9e47f4fa038015d58838df36ba0",
     "grade": true,
     "grade_id": "cell-902507dc58ec5428",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_3.2()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "d8bafda006ad3f30c5961db98a120a68",
     "grade": false,
     "grade_id": "cell-da06d5b257b928f0",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 3.3**\n",
    "<br> {points: 1}\n",
    "\n",
    "Now that the data is loaded in R, create a scatter plot that compares ethylbenzene (`EBE`) values against the date they were recorded. This graph will showcase the concentration of ethylbenzene in Madrid over time. As usual, label your axes: \n",
    "\n",
    "- x = Date\n",
    "- y = Ethylbenzene (μg/m³)\n",
    "\n",
    "*Assign your answer to an object called `EBE_pollution`.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7664b92c901435f283b0579b4423a413",
     "grade": false,
     "grade_id": "cell-14614015f46ccfff",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "options(repr.plot.width = 13, repr.plot.height = 7)\n",
    "\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "EBE_pollution\n",
    "\n",
    "# Are levels increasing or decreasing?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "0fb7263a512a25f8a83b1bfc6f7d29a5",
     "grade": true,
     "grade_id": "cell-4de75a47d9cc2dca",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_3.3()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "b658b04f5daef6e4549c08a50bdfd2f3",
     "grade": false,
     "grade_id": "cell-112c1b3ac6b6c720",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We can see from this plot that over time, there are less and less high (> 25 μg/m³) EBE values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "702b212febb4a1763174a2d68d4d9776",
     "grade": false,
     "grade_id": "cell-648260c1e625b576",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 3.4**\n",
    "<br> {points: 1}\n",
    "\n",
    "The question above asks you to write out code that allows visualization of all EBE recordings - which are taken every single hour of every day. Consequently the graph consists of many points and appears densely plotted. In this question, we are going to clean up the graph and focus on max EBE readings from each month. To further investigate if this trend is changing over time, we will use `group_by` and `summarize` to create a new data set.\n",
    "\n",
    "Fill in the `...` in the cell below. Copy and paste your finished answer and replace the `fail()`.\n",
    "\n",
    "*Assign your answer to an object called `madrid_pollution`.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "0aeab1ca4da030976b0d400a18e5772d",
     "grade": false,
     "grade_id": "cell-dbd1c81ead522ff6",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# ... <- ... |>\n",
    "#     group_by(year, ...) |>\n",
    "#     ...(max_ebe = max(EBE, na.rm = TRUE))\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "madrid_pollution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "941edf34a49f1655913fabc91b70c74d",
     "grade": true,
     "grade_id": "cell-d04ca4acf0f5f6bc",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_3.4()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "c43479b5983b707594d3eb0d6ac3a15c",
     "grade": false,
     "grade_id": "cell-7275e7512b1bd674",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 3.5**\n",
    "<br> {points: 1}\n",
    "\n",
    "Plot the new maximum EBE values versus the month they were recorded, split into side-by-side plots for each year. Again, we will use facetting (this time with `facet_grid`, more on this next week) to plot each year side-by-side. We will also use the `theme` function to rotate the axis labels to make them more readable (more on this is coming next week too!).\n",
    "\n",
    "*Assign your answer to an object called `madrid_plot`. Remember to label your axes.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "e1fa5e92f1a5a3dfefc6fba73705b3b5",
     "grade": false,
     "grade_id": "cell-a406232e78b0e3ab",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "#... <- ... |>\n",
    "#    ggplot(aes(x = ..., y = ...)) + \n",
    "#    geom_point() +\n",
    "#    xlab(...) + \n",
    "#    ylab(...) +\n",
    "#    facet_grid(~ year) +\n",
    "#    theme(axis.text.x = element_text(angle = 90, hjust = 1)) +\n",
    "#    theme(text = element_text(size=20))\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "madrid_plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d7bc589a1e2bad980403b568ad7dc65f",
     "grade": true,
     "grade_id": "cell-b12cea24ac607772",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_3.5()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "aed445f91369da7e6cd460d230988562",
     "grade": false,
     "grade_id": "cell-f8e81164975da6e9",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 3.6**\n",
    "<br> {points: 1}\n",
    "\n",
    "Now we want to see which of the pollutants has decreased the most. Therefore, we must repeat the same thing that we did in the questions above but for every pollutant (using the original data set)! This is where `purrr`'s `map*` functions can be really helpful! \n",
    "\n",
    "First we will look at Madrid pollution in 2001 (filter for this year). Next we have to select the columns that should be excluded (such as the date). Lastly, use the `map_df()` function to create max values for all columns.\n",
    "\n",
    "Fill in the `...` in the cell below. Copy and paste your finished answer and replace the `fail()`.\n",
    "\n",
    "*Assign your answer to an object called `pollution_2001`.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "55b1298fba220ac8763dff5548459330",
     "grade": false,
     "grade_id": "cell-be1574acc60b53e9",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# ... <- madrid |>\n",
    "#     ...(year == 2001) |>\n",
    "#     select(-..., -year, -mnth) |>\n",
    "#     map_df(..., na.rm  = TRUE))\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "pollution_2001"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "015f95b93acd2ff7ff3f27698bcdb503",
     "grade": true,
     "grade_id": "cell-23ecb8f3102f3435",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_3.6()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "4e7a96c4f21bfc5fb9e81ae4c20ade55",
     "grade": false,
     "grade_id": "cell-e39870b13e1cfdbc",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 3.7**\n",
    "<br> {points: 1}\n",
    "\n",
    "Now repeat what you did for Question 3.6, but filter for 2006 instead. \n",
    "\n",
    "*Assign your answer to an object called `pollution_2006`.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7890c496897278378c4cd9f166aba4b5",
     "grade": false,
     "grade_id": "cell-f32f902d017d6f0f",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "pollution_2006"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "528a8598f5b71a61e0549f37705f98db",
     "grade": true,
     "grade_id": "cell-df023e23794302c6",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_3.7()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "265298ababb03aa0e03006df13fbafed",
     "grade": false,
     "grade_id": "cell-a16e6e9379986fdd",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 3.8** \n",
    "<br> {points: 1}\n",
    "\n",
    "Which pollutant decreased by the greatest magnitude between 2001 and 2006? Given that your the two objects you just created, `pollution_2001` and `pollution_2006` are data frames with the same columns you should be able to subtract the two objects to find which pollutant decreased by the greatest magnitude between the two years. \n",
    "\n",
    "*Assign your answer to an object called `answer3.8`. Make sure to write the answer exactly as it is given in the data set.* Example: \n",
    "\n",
    "```\n",
    "answer3.8 <- \"BEN\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "f988bed96177149121b3142818b78cc7",
     "grade": false,
     "grade_id": "cell-37d1d8a36a3010c8",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# Replace the fail() with your answer. \n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "1941973e6663ebbd1e1bece63ac9daff",
     "grade": true,
     "grade_id": "cell-5ed11e7cbe1ac843",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_3.8()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "e0977947c8e05625898097c1f754b8a9",
     "grade": false,
     "grade_id": "cell-39c7d54ce32a3cf3",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 3.9**\n",
    "<br> {points: 1}\n",
    "\n",
    "Given that there were only 14 columns in the data frame above, you could use your eyes to pick out which pollutant decreased by the greatest magnitude between 2001 and 2006. But what would you do if you had 100 columns? Or 1000 columns? It would take A LONG TIME for your human eyeballs to find the biggest difference. Maybe you could use the min funcion:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d3c50bcb4f2ec48ecc00289a8195e2ba",
     "grade": false,
     "grade_id": "cell-3e5f51fc465018c3",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# run this cell\n",
    "pollution_2006 - pollution_2001\n",
    "min(pollution_2006 - pollution_2001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "ed0e9881b9f8fb72cbaace4a1908a946",
     "grade": false,
     "grade_id": "cell-6281bc171c2d9074",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "This is a step in the right direction, but you get the value and not the column name... What are we to do? Tidy our data! Our data is not in tidy format, and so it's difficult to access the values for the variable pollutant because they are stuck as column headers. Let's use `pivot_longer` to tidy our data and make it look like this:\n",
    "\n",
    "| pollutant | value  |\n",
    "|-----------|--------|\n",
    "| BEN       | -33.04 |\n",
    "| CO        | -6.91  |\n",
    "| ...       | ...    |\n",
    "\n",
    "To answer this question, fill in the `...` in the cell below. Copy and paste your finished answer and replace the `fail()`.\n",
    "\n",
    "*Assign your answer to an object called `pollution_diff` and ensure it has the same column names as the table pictured above.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "3dd4d3fbb05fd6310f9a0adc1782765a",
     "grade": false,
     "grade_id": "cell-96e5081abb9ab341",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "pollution_diff  <- pollution_2006 - pollution_2001\n",
    "#pollution_diff  <- ... |> \n",
    "#    pivot_longer(cols = everything(), \n",
    "#           names_to = ..., \n",
    "#           values_to = ...)\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "pollution_diff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "8c4a22672acf690132ee16da17e3d735",
     "grade": true,
     "grade_id": "cell-434094b036007273",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_3.9()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "bf25b2aa5cf65fde40419967d7b5df8f",
     "grade": false,
     "grade_id": "cell-8933eb3ea15101f3",
     "locked": true,
     "schema_version": 3,
     "solution": false
    }
   },
   "source": [
    "**Question 3.10**\n",
    "<br> {points: 1}\n",
    "\n",
    "Now that you have tidy data, you can use `arrange` and `desc` to order the data in descending order. Each element of the `value` column corresponds to an amount of decrease in a pollutant; so the *largest decrease* in pollutant should be *most negative entry*, i.e., the last row in the resulting dataframe. Therefore, we can take the sorted dataframe and pipe it to `tail` (with the argument `n = 1`) to return only the last row of the data frame.\n",
    "\n",
    "(the function `tail` is just like `head`, except it returns the last rows of the dataframe instead of the first rows.)\n",
    "\n",
    "To answer this question, fill in the `...` in the cell below. Copy and paste your finished answer and replace the `fail()`.\n",
    "\n",
    "*Assign your answer to an object called `max_pollution_diff`.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d5565a68ebb8ed926c3255df69ab8c26",
     "grade": false,
     "grade_id": "cell-b25d71fbf02fa70e",
     "locked": false,
     "schema_version": 3,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "#... <- ... |> arrange(desc(...)) |> \n",
    "#    tail(n = 1)\n",
    "\n",
    "# your code here\n",
    "fail() # No Answer - remove if you provide an answer\n",
    "max_pollution_diff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "6dd5ca67c29760fec448b7009e54790b",
     "grade": true,
     "grade_id": "cell-f9cfd97235c900d5",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "test_3.10()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "3a90443569aa389278a39db2c8f5df4a",
     "grade": false,
     "grade_id": "cell-cc19d9a2d3663e09",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "At the end of this data wrangling worksheet, we'll leave you with a couple quotes to ponder:\n",
    "\n",
    "> “Happy families are all alike; every unhappy family is unhappy in its own way.” –– Leo Tolstoy\n",
    "\n",
    "> “Tidy datasets are all alike, but every messy dataset is messy in its own way.” –– Hadley Wickham\n",
    "\n",
    "*source: [Tidy data chapter](https://r4ds.had.co.nz/tidy-data.html) from R for Data Science by Garrett Grolemund & Hadley Wickham*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "410e38bd9f145a89ad826cdcea776d45",
     "grade": false,
     "grade_id": "cell-b4e61ca9bc9efa93",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "source(\"cleanup.R\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "4.3.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}