LMB_Python_Basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python for Beginners"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This course is intended to be run with [Jupyter Notebook](https://jupyter.org/).\n",
    "\n",
    "For ease of installation, especially under Mac and Windows, we recommend installing Python 3 using the [Anaconda](https://www.anaconda.com/distribution/) bundle, which will include many commonly-used scientific libraries.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic code structure"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A very simple Python program is illustrated below. Note that all the commands start at the beginning of their line, i.e., they have no spaces or indentation before them. Indentation is a very important part of the Python language syntax and it is assumed to be consistent in each block of code; that’s how blocks are defined, as illustrated later. Spaces that are internal to each line, such as those next to the equals sign here are much less critical; indeed you could have more or no spaces here. The program consists of the lines:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mass = 3.4\n",
    "volume = 1.8\n",
    "density = mass/volume       # Division operation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What this tells the python interpreter to do is to assign a value of 3.4 to the label called `mass` and assign 1.8 to `volume`. The `density` is a new value that is calculated from the other values: one divided by the other, as indicated by the `/` symbol. Note that there are comments, from the `#` symbol onwards, which is only for a human to read and will not be interpreted as Python. Each labeled item, which is more correctly called a *variable*, is a way of attaching a name to a piece of data, and as the term suggests the actual value that is being referred to can change while the remainder of the program remains the same, i.e. we could calculate the density in the same way for different masses and volumes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "density = round(density, 3) # 3 : decimal places\n",
    "print(density)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next the density is re-assigned to a new value which is its old value rounded to three decimal places: the old value and the number of decimal places are stated in the parenthesis of `round()`. Finally we use the `print()` function, which has the effect of displaying the underlying numeric value of the density to screen.\n",
    "\n",
    "Note that we are free to choose any name for our variables within certain rules: names can contain numbers, letters and underscore (`_`) only but may not start with a number. The use of `round()` and `print()` are examples of *functions*; named tasks performing a specific set of operations on input data, specified inside the parenthesis, and which often generate some output. Many functions are inbuilt into Python (see [python.org](https://www.python.org) for full documentation), but it is possible to write new functions inside a Python program. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simple Data Types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are several basic data types in Python. The simplest of these include numbers, as already illustrated, values for true or false and a special `None` value:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data type\t|Description\t|Example\t|Converter\n",
    ":---|:---|:---|:---\n",
    "Integer\t|Whole numbers\t|x = 128|int()\n",
    "Floating point\t|Numbers with decimal points and/or power of ten exponent (scientific notation)\t|x = 5e-3<br>y = 12.00 \t|float()\n",
    "Complex\t|Numbers with real and imaginary parts\t|x = 1.0-2j<br>y = 1j\t|complex()\n",
    "Boolean\t|Truth values `True` and `False`\t|x = True<br>y = False\t|bool()\n",
    "None\t|A special value None for nothingness/undefined\t|x = None|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Numbers\n",
    "\n",
    "Integer numbers are specified without any decimal point or exponent and represent the exact whole number. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = -7\n",
    "b = 123\n",
    "print(a, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Floating point numbers represent a number of significant digits and an exponent, though the latter is often implicit. An explicit `e` suffix states which power of ten the digits operate at."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "c = 3.1415926536\n",
    "d = 2.0e7 # Twenty million: two times ten to the seven\n",
    "print(c)\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The scientific `e` notation is often optional, and a given number might be written in many different ways, e.g `0.001`, `1e-3` and `0.1e-2` are the same value. Floating point numbers have limited precision, i.e. they cannot represent all fractions precisely, as one might expect with ⅓ etc. However, some floating point calculations create an error in the least significant digits (caused by the underlying system being binary), which can occasionally cause problems for the programmer if not expected:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(3.0 * 7.1)  # Not exactly 21.3 !"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculations with integers often give integer results, though division of two integers gives a floating point result (this was different in Python 2). Calculations involving any floating point number tend to give a floating point result.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 2     # Integer\n",
    "y = 5     # Integer\n",
    "z = 3.0   # Floating point\n",
    "\n",
    "print(x * y) # 10  - Integer\n",
    "print(x / y) # 0.4 - Floating point due to division\n",
    "print(x + z) # 5.0 - Floating point"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As shown below, data type can be inspected directly using the `type()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t1 = type(1)  # Get a value's data type\n",
    "t2 = type(2.9)\n",
    "print(t1) # 'int' - integer number\n",
    "print(t2) # 'float' - floating point number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(type(x * y - z))  # float"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Objects can be converted between one data type and another with the relevant conversion function, though naturally this can cause rounding when creating an integer from a float, e.g. `int(7.3)` gives integer 7. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 1.3\n",
    "y = 3.8\n",
    "z = 7\n",
    "\n",
    "print(int(x))    #  Remove non-integer part (integer floor)\n",
    "print(int(y))\n",
    "print(float(z))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each entity in Python is an 'object' of a given type; it is a rich description of the data rather than just the plain underlying value. The object-oriented nature of Python is revealed whenever the dot syntax is used. This accesses values and functions (i.e. named operations run with `()` parentheses) that belong to an object, i.e. as `my_object.value` or `my_object.function()`.\n",
    "\n",
    "For example, rather than using the usual `+` symbol, here we perform addition of two numbers using `x.__add__()`; the addition function belongs to the integer object we have called `x`. The double underscores are a hint this is a special inbuilt Pythonfunction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 7            \n",
    "y = x + 5         # Addition\n",
    "z = x.__add__(2)  # Also, addition\n",
    "print(y, z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly, Python functions are aso Python objects. For example, here documentation text is accessed for the integer creation/conversion function `int()` using `.__doc__`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(int.__doc__) # Print Python documentation for the int() function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mathematical operations\n",
    "\n",
    "\n",
    "Standard arithmetic operations can be done on numeric data with the appropriate symbol: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 5.7\n",
    "y = 1.2\n",
    "print(x + y)  # Addition\n",
    "print(x - y)  # Subtraction\n",
    "print(x * y)  # Multiplication\n",
    "print(x / y)  # Division"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly using double asterisks `**` means raise to the power:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(2 ** 3)   # Two cubed\n",
    "print(3 ** 0.5) # Square root three"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The modulus operation: the remainder after division, is performed using the `%` symbol."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(19 % 12)     # The remainder after division by 12\n",
    "print(378 % 360)\n",
    "print(537 % 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When multiple opersations are used the usual `/\\*+-` precidence is applied. Round brackets can be used to specify the operation order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 2\n",
    "y = 7\n",
    "print(x + y * 5)    # Multiplication first\n",
    "print((x + y) * y)  # Addition first"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When performing mathematical operations, all the usual rules apply, e.g:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(x / 0.7)  #  No problem\n",
    "print(x / 0.0)  #  Gives an error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interestingly, Python actually does have the concept of (plus and minus) infinity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = float('inf')\n",
    "print(x)\n",
    "print(x + 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The symbols used above are just some of the inbuilt mathematical symbols (operators) in Python. The full complement of these is:\n",
    "\n",
    "|Operation\t|Description\t|Example |\n",
    "|:----------|:--------------|:-------|\n",
    "|x + y \t|Addition\t|`revenue = profit + expenses`|\n",
    "|x - y \t|Subtraction\t|`income = profit - taxes`|\n",
    "|x * y \t|Multiplication\t|`area = volume * height`|\n",
    "|x / y \t|Division \t|mean = (x + y + z) / 3.0|\n",
    "|x // y |Floor division: divide and round down.\t|`a = 13.0 // 5.0`<br>*a is 2.0*|\n",
    "|x % y \t|Modulus: remainder after division.\t|`a = 13 % 5`<br>*a is 3*|\n",
    "|-x \t|Negate the value of x\t|`a = -5 * 3`<br>*a is -15*|\n",
    "|x ** y |Raise to the power, i.e. x<sup>y</sup>.|`a = 2 ** 3`<br>*a is 8*|\n",
    "\n",
    "Note that there are also related operators that modify the value of a variable, in-place. For example `x *= 3` means x is assigned to a new value which is triple its old value: equivalent to `x = x * 3`. Similarly `x += 3` means add 3 to x and `x -=3` means subtract 3 from x."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 11\n",
    "x = x * 3  # Set x to be triple the previous value\n",
    "print(x)   # 33\n",
    "\n",
    "x *= 3     # Also, set x to be triple the previous value\n",
    "print(x)   # 99\n",
    "\n",
    "x += 3     # Add three to x\n",
    "print(x)   # 102"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 1: Taylor series terms</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>Calculate an approximation to $e^x$ using the first few terms of the Taylor series:<br></font>\n",
    "\n",
    "$1 + x + \\dfrac{x^{2}}{2} + \\dfrac{x^{3}}{6} + \\dfrac{x^{4}}{24} + \\dfrac{x^{5}}{120} + \\dfrac{x^{6}}{720} + \\dfrac{x^{7}}{5040}$\n",
    "\n",
    "<font color=purple><br>Print estimates for $e$, $e^\\sqrt{2}$ and $e^{i\\pi}$ by using values of x=1, x=$\\sqrt{2}$ and x=3.14159265$i$ with the above equation. \n",
    "</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code\n",
    "pi = 3.141592653589793"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Booleans\n",
    "\n",
    "Boolean values represent truth or falsehood, for example as used in logical operations. The Boolean data type can be created directly by using the special words `True` and `False` or by using the `bool()` function. This treats zeros, empty containers (see below) and the `None` object as false, and everything else as true."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = True\n",
    "print(x)\n",
    "print(bool(0.0)) # False\n",
    "print(bool(7))   # True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many `True` and `False` values in Python arise as a result of a comparison operation. For example, when testing if one number is larger or smaller than another:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "x = 2\n",
    "print(x > 5) # False\n",
    "y = x < 3\n",
    "print(y)     # True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Boolean operations `and`, `or` and `not` can be used to combine multiple comparisons. Note, as shown below, that testing equality uses **double equals** signs `==`, which is easily confused with the single sign used for assignment, and that `!=` is the test for not equals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 1\n",
    "y = -1\n",
    "print(x > y and y > 1)   # False - second fails\n",
    "print(x != 2 or y == 1)  # True - first succeeds\n",
    "print(not (y > x))       # True - not False is True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The complement of general comparison operators in Python is specified in the below table. In Python 3 inequality comparisons can only be made on values of a comparable type (Python 2 is different in this regard; you could compare anything, whether it was meaningful or not).  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Operator|Description|Example\n",
    "---|---|---\n",
    "`==`|Tests whether two values are equal.|`x = 3`<br>`x == 3.0`<br>*True*\n",
    "`!=`|Tests whether two values are not equal.|`x = 3`<br>`x != 3`<br>*False*\n",
    "`>`|Tests whether the value of the first value is greater in than the second.|`x = 2**10`<br>`x > 1024`<br>*False; <br>2<sup>10</sup> equal not more*\n",
    "`<`|Tests whether the value of the first value is smaller in than the second.   |`x = 2**10`<br>`x < 1025`<br>*True*\n",
    "`>=`|Tests whether the value of the first value is greater or equal in than the second|`x = 2**10`<br>`x >= 1024`<br>*True*\n",
    "`<=`|Tests whether the value of the first value is greater or equal in than the second.|`x = 2**10`<br>`x <= 512`<br>*False*\n",
    "`is`|Tests whether two values represent the same Python object.|`3 == 3.0`<br>*True*<br>`3 is 3.0`<br>*False*\n",
    "`is not`|Tests whether two values represent different Python objects.|`3 is not 3.0`<br>*True*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The keyword operators `is` and `is not` are of note because they compare whether two items are the same Python object, not whether their values are the same. As we illustrate in the table, an integer and floating point number can be equal in **value** but they are two different **objects**, with different data types. The `is` comparison is often used with `None`, to detect if something is defined whether or not it has a zero value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 2: XOR</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>\n",
    "Write a combination of Python logical operations for two input variables that are equivalent to a single *exclusive or* (XOR) operation. The truth table for XOR is:</font>\n",
    "    \n",
    "In A| In B|Out\n",
    "---|---|---\n",
    "False|False|False\n",
    "True|False|True\n",
    "False|True|True\n",
    "True|True|False\n",
    "\n",
    "<font color=purple>Test your code on various inputs.\n",
    "</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The None object\n",
    "\n",
    "The `None` object is special built-in value which can be thought of as representing **nothingness** or that something is **undefined**. For example, it can be used to indicate that a variable exists, but has not yet been set to anything specific."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = None\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Text strings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Text in Python is represented by the `string` data type (i.e. a string of characters) and can be specified in code using single or double quotes, which distinguishes the characters from unquoted parts of Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 'Hello'\n",
    "y = \"world\"\n",
    "print(x,y)  # Print both values to screen"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is also a triple-quote syntax which allows text strings to flow over multiple lines:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = '''This method lets text flow from one line\n",
    "to the next line inside triple quotes and is\n",
    "very handy for adding documentation.\n",
    "'''\n",
    "print(x)       "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Text strings can be created from any value with `str()`, like when using `print()` and where appropriate they may be converted to other data types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = str(1.23)             # x is text '1.23'\n",
    "y = int('     007    ' )  # y is the integer 7\n",
    "print(x, type(x))\n",
    "print(y, type(y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some symbolic operators work with strings, though naturally in a textual way, even if the characters happen to represent digits:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = '1' + '99'  # concatenation\n",
    "y = 'abc' * 3   # repeat concatenation\n",
    "print(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many operations are performed on strings using inbuilt functions that belong to the Python object, as accessed using the dot notation, noting that the original data remains unaffected:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 'abc'      # x is a string object\n",
    "y = x.upper()  # x not changed \n",
    "print(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Strings can be considered to be arrays of characters, and accordingly have a length which can be found with the inbuilt `len()` function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 'abcde'\n",
    "print(len(x)) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some elements of a textual data are not printable as normal character glyphs, such as new lines or tab stops. These are represented in Python strings using special escape codes that start with `‘\\’`, for example `‘\\n’` means new line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Hello\\nworld') # \\n means new line"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The full list of character escape codes is listed in the following table. It is notable that Python has a concept or raw strings where these codes are not used: raw strings have an `‘r’` before their quotes, e.g. `r'Hello\\nworld'` has actual `‘\\’` and `‘n’` characters in the middle, not a new line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Code|Description|Example|Notes\n",
    "---|---|---|---\n",
    "\\\\ |A backslash character, which needs to be forced when the following character would otherwise form an escape code|text = '\\\\title'|Text is '\\title' and does not have a tab code (\\t)  inside.\n",
    "\\' |A single quote, which may need to be escaped in situations where it should not be considered as the start or end of a string.|text = 'Don\\'t do that!'|Not required when a string is defined with double quotes.\n",
    "\\\" |A double quote, which may need to be escaped in situations where it should not be considered as the start or end of a string.|text = \"Shout \\\"Help!\\\"loudly.\"|Not required when a string is defined with single quotes.\n",
    "\\n |A newline (linefeed) control character. Used to separate lines of text in UNIX and Linux based computers|text = 'Line A\\nLine B\\n'|Text value is split into two lines on Linux and UNIX machines\n",
    "\\r |A carriage return control character. Used in combination with \\n on Windows based systems to separate lines of text.|text = \"Line A\\r\\nLine B\\r\\n\"|Text is split into two lines on Windows machines.\n",
    "\\t |A tab character, providing indentation with white space to pre-set stop points.|text = 'Col 1\\tCol 2\\tCol 3\\n'|Tabs indent to form three columns.\n",
    "\\u···· |Specifies a Unicode character using a 16-bit hexadecimal value.|text = u'\\u03b1-helix'|Creates '\\uf061-helix', e.g. for graphical displays.\n",
    "\\x··|Specifies a character using a hexadecimal value.|text = '\\x48\\x65\\x6C\\x6C\\x6f'|Text is hexadecimal code for 'Hello'."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 3: Text escape codes</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>\n",
    "Create a single string variable that represents a table of one- and three-letter DNA codes (i.e. `T` - `Thy`, `A` - `Ade` etc.) so that each base goes on a different row. Separate the elements of each pair with a tap-stop and each row with a new line. Finally print the string variable.\n",
    "</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### String methods\n",
    "\n",
    "There are many inbuilt functions (methods) that are associated with strings, as listed in the full [Python documentation](https://docs.python.org/3/library/stdtypes.html#string-methods), and many of these will be used in later examples. A simple list of all the inbuilt values and functions for strings (and indeed any kind of Python object) can be accessed using `dir()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 'ABC'\n",
    "print(dir(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This reveals the `.lower` function, amonst many, which as you might guess creates a lowercase version of a string. We can get a description of this using `help()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(help(x.lower))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(x.lower())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many other handy inbuilt string functions, such as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = '  Hello\\n'\n",
    "y = x.strip()        # Remove edge whitespace characters\n",
    "print(y)\n",
    "print(x.count('l'))  # Count characters/sub-string"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A very useful string function, especially when reading data from file, is `split()`, which chops a string on a sub-string (or by whitespace by default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 'Val Gly Lys'\n",
    "y = x.split()  # Make a list of separate words\n",
    "z = x.split('Gly')\n",
    "print(y)\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the most important of these functions is `format()`, which allows variable values to be inserted inside a text string. The values are inserted into the string at the `{}` positions, replacing the bracket section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "name = 'Lisa'\n",
    "t = 'Hello {}.'.format(name)    # name inserted into brackets\n",
    "print(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "name1 = 'Homer'\n",
    "name2 = 'Marge'\n",
    "t = 'Hello {} and {}.'.format(name1, name2)    # inserting two names\n",
    "print(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In more recent versions of Python the <b>f-straings</b> syntax can be used to do the same kind of formatting in a very neat way where the variable names are used directly in the curly braces, taking spacia note that the string is prefixed with an `f` before the first quote."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "name = 'Lisa'\n",
    "t = f'Hello {name}.'    # \"f\" prefixes the string; variable can be inserted into brackets\n",
    "print(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "name1 = 'Homer'\n",
    "name2 = 'Marge'\n",
    "t = f'Hello {name1} and {name2}.'    # inserting two names\n",
    "print(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The brackets can optionally have a number, specifying which item to insert where, and also a format specification after `:` that specifies how to represent the item. For example here the names are padded to 12 characters width using spaces:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t = f'Hello \"{name1:12}\" and \"{name2:>12}\".'    # Padded to 12 characters wide, left and right (using >)\n",
    "print(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many special formatting codes, as described in the [`format()` documentation](https://docs.python.org/3.4/library/string.html#formatspec).\n",
    "This system is especially useful for specifying decimal places and scientific/exponent formatting for numbers.\n",
    "In the next example the first number is formatted as floating-point with five decimal places using the code `.5f` and the second number is in exponent format using the code `.2e`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 0.12\n",
    "y = 34121.0\n",
    "s = f'X {x:.5f} Y {y:.2e}'   # 5 dp float, 2 dp sci\n",
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is very common to combine both an overall character width and a number of decimal places. Here two numbers are formatted to be nine characters wide and use four decimal places, with code `9.4f`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 2 ** 10\n",
    "y = 2 ** 0.5\n",
    "t = f'A;{x:9.4f} B;{y:9.4f}'   #  Pad to 9 characters wid and use 4 d.p.\n",
    "print(t) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 4: Formatted text</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>Create and print a single, formatted text string indicating the masses of the Earth ($5.97237×10^{24} kg$) and the Moon ($7.342×10^{22} kg$) next to the ratio of their masses (Earth/Moon). Display the masses in yottagrams ($10^{21}$ kg), padded to a field-width of eight characters and to two decimal places. Format the ratio to 5 decimal places.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Indices and sub-strings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Text strings can be thought of as lists of individual characters. As such, each character has a position within the string. This position is accessed using square brackets and an index number. It is notable that these index numbers **start at zero** (like many other computing languages)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 'abcde'\n",
    "print(x[0])  # First character\n",
    "print(x[2])  # Third character"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Negative indices count back from the end of the straing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(x[-1]) # Last character\n",
    "print(x[-2]) # Second last character"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Internal substrings can also be accessed using square brackets, by specifying an appropriate index range using a `start:limit` notation. Note that the limit index is **not included** in the range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 'abcdefghijklmnopqrstuvwxyz'\n",
    "print(x[1:3])   # index 1, up to, but not including 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(x[2:-2])  # Omit the first and last two characters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unspecified range indices default to the start and end of the string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(x[2:])   # From idex to to end\n",
    "print(x[:7])   # From start to < index 7\n",
    "print(x[:])    # All characters (a copy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A step size may also be included in the index range; specified after the start and limit using a second `:`. For example the following starting at `b` and going to the end of the string, selects every other character (i.e. step = 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(x[1::2]) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Membership"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given that text strings are sequences of characters, they can be tested for the presence or absence (i.e. membership) of characters using the `in` keyword:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 'Bananarama'\n",
    "print('a' in x)\n",
    "print('b' in x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This can also be used to test the presence of sub-strings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Banana' in x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `index()` method of strings is handy to find where a character, or sub string, is first found:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(x.index('r'))\n",
    "print(x.index('an'))   #  Smallest index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Collections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Collections are Python objects that can contain other Python objects; containers for organizing data. For example, a list of various objects, specified with square parenthesis:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = True\n",
    "y = ['GCAT', 4.9, x] # a list of 3 items, which includes the value of x\n",
    "z = []               # an empty list\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = False\n",
    "print(y)    # Unaltered"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Containers can contain the simple data types already discussed and also other containers (within certain rules) and they can also be empty. The basic inbuilt types of container in Python are:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Type|Description|Example|Converter\n",
    "----|-----------|-------|---------\n",
    "List|A modifiable, ordered list of items|`x = ['cat', 'dog', 'pig']`<br>*A list of three strings*|`list()`\n",
    "Tuple|An unmodifiable, ordered list of items|`x = (0.742, 0.159)`<br>*A tuple of two floats*|`tuple()`\n",
    "Set|A modifiable, unordered collection of unique items|`x = {9, 1, 7, 2, 5}`<br>*A set of five numbers*|`set()`\n",
    "Frozenset|An unmodifiable, unordered collection of items|`x = frozenset([1,2,3])`|`frozenset()`\n",
    "Dictionary|A modifiable, unordered collection of items accessed by unique keys|`x = {'A':1, 'C':3, 'E':5}`<br>*A dictionary with three key:value pairs*|`dict()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lists\n",
    "\n",
    "Lists are specified with square brackets and can be accessed by numeric indices and ranges, like strings, with the first item being at index `0`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = ['a', 'b', 'c', 'd']\n",
    "print(x[2])   # index 2 : third position\n",
    "print(x[1:])  # from index 1 to the end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The value of a specific index or range can also be set, and items in the list can be removed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x[0] = 'z'    # index 0 is set to 'z'\n",
    "print(x)\n",
    "del x[2]      # deletes item at index 2\n",
    "print(x)      "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lists can be added together (concatenated) using the `+` operation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = [1,4,9,16]\n",
    "y = [25]       # A list with one item\n",
    "z = x + y\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, you cannot add non-list items in this manner:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = x + 9   # Gives an error\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "z = x + [9] # Works: added a list with one item\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A list can contain internal sub-lists, the elements of which can be accessed using multiple bracketed indices:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = [[4,7], [9,6]] # A list containing two other lists\n",
    "print(y[1])        # sub-list at y index 1\n",
    "print(y[1][0])     # item 0 from sub-list 1  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Collections can be made from other collections using the appropriate creation function, e.g. `list()` to make list and `set()` to make a set (see below). More generally however, they can be created from Python objects that are iterable: things that can generate a sequence of items. For example, as shown below, `range()` generates a sequence of integer numbers up to a specified limit (a start number and step could also be specified),  which is then used to make a list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = list('PQRST')     # Create list of characters from text string\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = list(range(10))   # A list of the range from 0 up to <10 \n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Collections can be unpacked into individual variables if the number of items being assigned matches the collection size:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = [4, 1, 0]  # A list containing three items \n",
    "a, b, c = x    # Assign to each sub-element\n",
    "print(a)\n",
    "print(b)\n",
    "print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As with all the main Python collections, lists have a size, accessible with `len()`, and membership tests are performed using the `in` operator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = ['a', 'b', 'c', 'd']\n",
    "print(len(x))     # number of items in list\n",
    "print('c' in x)   # string 'c' is in the list\n",
    "print(2 in x)     # number 2 is not in list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tuples\n",
    "\n",
    "Tuples, specified with round brackets, are similar to lists and have indexed items etc. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = (9, 3, 1, 0)  # Create a tuple\n",
    "y = (2,)          # Tuple with one item (needs trailing comma)\n",
    "z = ()            # Empty tuple\n",
    "print(x)          # Print whole tuple\n",
    "print(x[-1])      # Print the last item"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, they cannot be modified: their contents are defined when they are created. This is useful because tuples can be used as keys in a dictionary (see below) while lists cannot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x[0] = 6          # Does not work! Tuples cannot be changed!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although tuples cannot be modified, they are easily converted to lists if needed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w = list(x)       # Create equivalent list from a tuple\n",
    "w[0] = 6          # List copy can be changed\n",
    "print(w)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 5: Splitting and joining lists</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>Convert the string of-one letter amino acid codes given below into a list containing every third letter. Use the string `string.join(list)` function combine the list's characters with commas into a single string.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code\n",
    "a = 'MVHLTPEEKSAVTALWGKVNVDEVGGEALGRLLVVYPWTQRFFESFGDLSTPDAVMGNPK'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sets\n",
    "\n",
    "Sets are specified with curly brackets. They do not contain repeated values and the items do not have an order, and so cannot be accessed by index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = {1,2,3,4,3,2,1}  # duplicates ignored\n",
    "y = set([3,4,5,6])   # created from a list\n",
    "print(y)             # print a whole set\n",
    "print(len(x))        # size of the set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sets have helpful associated set operations (intersection, union, difference, disjoint test, subset test etc.), and allow for membership tests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(1 in y)        # 1 is not in y\n",
    "print(x & y)         # intersection, items in both\n",
    "print(x | y)         # union, items in either"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dictionaries\n",
    "\n",
    "Dictionaries are like lookup tables containing `key:value` pairs, they are also specified with curly brackets but can be distinguished from sets by `:` joining the pairs. The values in a dictionary are accessed by key, not by index, and each key is used only once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "d = {\"G\":329.21, \"C\":289.18, \"A\":313.21, \"T\":314.19}\n",
    "print(d['A'])      # value associated with 'A'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dictionary keys are typically strings or numbers, but more generally they are restricted to items that are not modifiable (so you cannot use lists or other dictionaries as keys) but the values being referred to can be any data type."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The length of a dictionary is the number of key-value pairs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(len(d))      # number of key:value pairs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dictionaries have various inbuilt functions to access thier component parts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "keys = d.keys()     # Just keys\n",
    "values = d.values() # Just values\n",
    "items = d.items()   # Pairs\n",
    "print(keys)\n",
    "print(items)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These give special iterable objects, which may be converted to lists etc. as required: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(list(keys))  # A list of keys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If a key is already present in the dictionary then a simple assignment of the form `dict[key] = value` is used to change the value associated with that key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "d = {\"X\":99, \"Y\":121, \"Z\":14}\n",
    "print('Original:', d)\n",
    "d['Y'] = 201     # Change the value of an existing item\n",
    "print('Updated:', d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, if a key was not already present this kind of assignment will add a new key:value pair. Existing keys cannot be changed directly but it is possible to remove a key:value pair using `del` and add the same value back again with a different key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "d['W'] = 256    # Add a new key:value pair\n",
    "print('Bigger:', d)\n",
    "del d['Z']        # Delete a key and its value from the dictionary\n",
    "print('Smaller:', d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Empty collections\n",
    "\n",
    "As already hinted at above, empty collections (with size zero) are specified using the correct kind of brackets with no contents, or using the conversion function with no inputs. However, a slight complication here is that the curly parentheses `{}` make an empy dictionary, not a set. Although both these collections use the same kind of brackets, dictionaries were introduced into Python first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = []\n",
    "b = ()\n",
    "c = {}\n",
    "d = dict()\n",
    "e = set()\n",
    "\n",
    "print(len(e))\n",
    "print(bool(a))   # Logically false\n",
    "print(type(a), type(b), type(c), type(d), type(e))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collection operations\n",
    "\n",
    "The collection types have a number of inbuilt functions (methods) that are accessed with the dot syntax, some of which are described below for lists and sets. Naturally the functions available to a given collection are appropriate to its type, e.g. sets do not have functions that refer to positional indices. All of the inbuilt functions and attributes of a Python object may be listed with `dir()`, noting that those with `__` around the name are generally only for internal use. For example to see what is inbuilt for lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(dir([]))   #  Any list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `append()` and `extend()` methods are commonly used to add items to the **end** of lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = ['Mon', 'Tue', 'Wed'] # A list of strings\n",
    "y = ['Fri', 'Sat', 'Sun'] # And another\n",
    "print(x)\n",
    "x.append('Thu')    # Add a single new item to end\n",
    "x.extend(y)\t       # Extend with items from another collection \n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `insert()` method is used to add items at any position:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.insert(0, 'Sun') # Insert an item at an index\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The position (first occurrence) of an item can be found using `index()` and the number of occurences found with `count()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(x.index('Sat'))     # Positional index of an item\n",
    "print(x.count('Sun'))     # Number of items"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Items can be removed (first occurrence) with `remove()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.remove('Sun')    # Remove an item\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another handy function is to sort a list internally:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.sort()           # Sort contents alphabetically\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly, sets can have their contents changed. Note here that there is no positional (index) information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = {'G', 'C', 'A', 'T'}   # A set with 4 strings\n",
    "t = {'N', 'R', 'Y'}\n",
    "s.add('U')          # Add a single item (if not present) \n",
    "print(s)\n",
    "s.update(t)         # Add any new items from another collection\n",
    "print(s)\n",
    "s.remove('N')\n",
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 6: Dictionary construction</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>Construct a dictionary containing containing counts for the different DNA letters in the sequence given below. Use one-letters code as the dictionary key, and the counts as the corresponding value. Using the dictionary calculate the percentage G+C content of the sequence.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code\n",
    "seq = \"\"\"ATTAATTAATTCTGAGAGCTGCTGAGTTGTGTTTACTGAGAGATTGTGTATCTGCGAGAGAAGTCTGTAGCAAGTAGCTAGACTGTGCTTGACCTAGGAACATATACAGTAGATTGCTAAAATGTCTCACTTGGGGAATTTTAGACTAAACAGTAGAGCATGTATAAAAATACTCTAGTC\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Control code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lines of Python code are generally executed in sequential order, one after the other. However there are situations where we wish to deviate from this, for example to repeat a section of code several times in a loop, or to only execute a block of code under certain conditions. Accordingly, there are a set of keyword commands including `if`, `for`, `while`, `try` and `def` that are used to control the execution of a subsequent code block, which is indented relative to the keyword and comes **after a colon**.\n",
    "\n",
    "## Conditional code\n",
    "\n",
    "The `if` statement is used to only perform operations if a particular condition is met (if the value of an expression is logically true)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 3\n",
    "if x < -1 or x > 1:    # Run indented lines when true. ** Note colon at end ** \n",
    "    x *= 2             # Start of indented lines   \n",
    "    print('Value was doubled')  \n",
    "\n",
    "print('Value is:', x)  # Always executed, not in indented block"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It should be noted that any number of spaces can be used for indentation of the controlled block, but it must be consistent. Four spaces are generally recommended, though sometimes two may be used in cramped situations.\n",
    "\n",
    "The `if` statement can also have a number of `elif` clauses and a final `else` clause, each with their own block: `elif` does further conditional tests if all preceding ones failed, and `else` marks the block that is run if no condition is met."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if x > 0:\n",
    "    print('Positive')\n",
    "elif x < 0:            # Checked if first condition was false\n",
    "    print('Negative')\n",
    "else:                  # If all fails\n",
    "    print('Zero')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When testing whether the expression after an `if` statement is true it is obvious what happens in situations like `x < 10` or `y == 5`, given that these comparisons generate `True` or `False` (Boolean) objects. However, any object in Python can be tested for truth. A few specific objects are deemed to be logically false: `False`, `None`, `0`, `0.0`, empty string:`''` and empty collections, while almost everything else is deemed to be true."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 'abc'\n",
    "if x:              # Test innate truth of x\n",
    "    print('true')  # This prints; any non-empty string is true\n",
    "else:\n",
    "    print('false')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loops\n",
    "\n",
    "Repetitive loops can be created with a `for` statement or a `while` statement. The `for` loop extracts items from a collection (or other iterable object, like a string of characters) and assigns a loop variable to each value in turn, repeating execution of the indented block of code each time. Here a list of numbers `data` is defined and `x` is repeatedly assigned to the value of each item in the list before printing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = [1,4,9,25,36]\n",
    "for x in data:        # x is first 1, then 4, then 9 etc.\n",
    "    print(x)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The exmple is next modified so that the value of `x` is added to a running total every cycle. This total was was initially defined as zero before the loop:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "total = 0             # Starting value\n",
    "data = [1,4,9,25,36]\n",
    "for x in data:        \n",
    "    total += x        # Add current value of x to total\n",
    "    print(x, total)   # Current values in this cycle\n",
    "    \n",
    "print('Final:', total)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is often convenient to use the `enumerate()` function with a `for` loop. This allows the loop to iterate over both item numbers (usually the positional indices) and their actual values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = 'AGCAGTAGACGAACAT'     # String of characters\n",
    "for pair in enumerate(text):  # (index, item) pairs\n",
    "    print(pair)               "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As illustrated below, it is common for the number and value (`i` and `x` respectively) to be stated as separate variables in the `for` line, using the unpacking syntax illustrated previously:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = 'AGCAGTAGACGAACAT'     # String of characters\n",
    "for i, x in enumerate(text):  # separate index and item\n",
    "    print(i, x)               "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A `while` loop repeats a block of code while a certain condition evaluates to be true, and so it is important to make sure that the condition is eventually false (on the command-line Ctrl+C keys can be used to stop an ‘infinite’ loop)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 1\n",
    "while x < 1000:   # Repeat the indented block while this is true\n",
    "    print(x) \n",
    "    x *= 2        # Double the value\n",
    "\n",
    "print('Final:', x)  # final value stopped the loop: not less than 1000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 7: Taylor series reprised</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>Using a `for` loop, repeat Exercise 1, summing the Taylor series of $e^x$, i.e:<br></font>\n",
    "\n",
    "$1 + x + x^2/2! + x^3/3! + x^4/4! + x^5/5! + x^6/6! + x^7/7!$ + ...\n",
    "\n",
    "<font color=purple><br>Print estimates for $e$ and $e^2$, do this for both eight and 16 terms. Hint: use the previous factorial to make a new factorial each time.\n",
    "</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Loops can appear inside other loops, as long as each uses a different iterating variable. Here the inner loop is indented relative to the first, hence the inner-most `print()` is indented twice, relative to the start."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(3):       # i is first 0, then 1, then 2\n",
    "    for j in range(3):   # for each value of i, is 0, then 1, then 2 \n",
    "        print(i, j, i*j)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Loops (both `while` and `for`) can be skipped, for the remainder of their block, using `continue` and stopped entirely with `break`. In this example an inner `if` statement is used to trigger these in specific circumstances. Note the `print()` is in the block of the `for` statement and so is indented once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = [3, -1, 2, -5, 0, 9, -2]\n",
    "for x in data:  \n",
    "   if x < 0:\n",
    "       continue      # Skip the remainder of 'for' loop\n",
    "   elif x == 0:\n",
    "       break         # Quit entirely\n",
    "     \n",
    "   print(x)          # Only prints positive values before zero"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is another kind of loop that does not have an indented syntax. It is not a general purpose loop like the ones described above, rather it is a means of constructing a collection (list, set, dictionary). In essence, a kind of `for` loop is specified inside the collection’s brackets.  Considering the following example for constructing a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "squares = []\n",
    "for x in range(1, 10):\n",
    "    squares.append(x*x)\n",
    "print(squares)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This could be equivalently written as a list comprehension, effectively building a list from the inside, where the item that enters the list (here `x*x`) appears before the `for`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "squares = [x*x for x in range(1,10)]\n",
    "print(squares)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Changing the bracket type changes the type of collection constructed. Curly brackets `{}` can be used to construct a set or a dictionary, depending on whether a `key:value` dictionary specification is made."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = {x*x for x in range(1,10)}              # Make a set\n",
    "d = {i:x for i,x in enumerate('ABCDEF')}    # Make a dictionary\n",
    "print('Set:', s)\n",
    "print('Dict:', d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is also an option to add a conditional filter when making a collection. Here an internal `if` comes after the `for` section(s). In the following example a list is made using only odd values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "odd_sq = [x*x for x in range(1,10) if x % 2 == 1]    # Only odd x, then squared\n",
    "print(odd_sq)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example: Calculate sequence identity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following example calculates the percentage of sequence identity for two input sequences `seq1` and `seq2`. The sequences should be aligned and carry ‘-’ characters to represent alignment gaps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq1 = 'ALIGDPVENTS'\n",
    "seq2 = 'ALIGN-MENTS'\n",
    "n = len(seq1)\n",
    "\n",
    "count = 0.0                   # Starting identity count is zero\n",
    "\n",
    "for i in range(n):            # Loop through  position indices\n",
    "    a = seq1[i]               # Letter at index i for first seq\n",
    "    b = seq2[i]               # Letter at index i for second seq\n",
    "    if a == b and a != '-':   # Test if letters are same and not a gap\n",
    "      count += 1.0            # .. if they are increase count by one\n",
    "\n",
    "ident = 100.0 * count/n      # Calc. and send back identity as % total \n",
    "\n",
    "print('Identity {:.2f}%'.format(ident)) # Format to 2 dp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 8: Dictionary counting</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>Using the protein sequence given below, count the number of occurrences of each amino acid letter using a dictionary and a `for` loop. Use an `if` statement inside the loop to initialise the count for letters that have not been seen before i.e. `if letter not in count_dict:`. Use a second loop to print each letter and its corresponding count.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code\n",
    "seq = \"MYGKIIFVLLLSEIVSISASSTTGVAMHTSTSSSVTKSYISSQTNDTHKRDTYAATPRAHEVSEISVRTVYPPEEETGERVQLAHHFSEPEITLIIFGVMAGVIGTILLISYGIRRLIKKSPSDVKPLPSPDTDVPLSSVEIENPETSDQ\"\n",
    "count_dict = {}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Catching errors\n",
    "\n",
    "A `try`, `except` block is used to catch and deal with illegal circumstances. The code in a `try` block is run and if a problem occurs an `except` block of code may be run if a particular kind of error (a type of `Exception` object) is detected. Consider the following error-generating code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 1\n",
    "y = 0\n",
    "w = x/y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can prevent the program from failing and sensibly handle an error. In the jargon we catch a particular *exception*: detect error objects at the `except` keywords. Here the error object is a `ZeroDivisionError` which is present in standard Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "x = 1\n",
    "y = 0\n",
    "try:                 # Run the following block and check for failure\n",
    "    w = x / y\n",
    "\n",
    "except ZeroDivisionError as err:        # \n",
    "   print('divided by zero, continuing') # warn, but otherwise ignore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An error of any kind may be detected using the general `Exception` object. Also, \n",
    "the original error can be re-triggered using `raise()` if required. For example, here we extend the above with a second `except` to catch any additional errors, which we then fail at."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 1\n",
    "y = '0'\n",
    "try:                 # Run the following block and check for failure\n",
    "    w = x / y\n",
    "\n",
    "except ZeroDivisionError as err:        # This specific error is OK\n",
    "   print('divided by zero, continuing')\n",
    "\n",
    "except Exception as err:   # Any other error is not OK\n",
    "    raise(err)             # Trigger the error, do not continue"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions\n",
    "\n",
    "The keyword `def` is used to define functions, i.e. user created subroutines. In essence these are a specification for a named bit of code. Defining a function is distinct from running (or calling) a function, but once defined a function can be called into action any number of times; achieved by using its name with parentheses (which convey any input data). Naturally the general idea is that functions represent reusable code, performing the same operation in many different places, albeit for different input data. \n",
    "\n",
    "Here is a very simple function that prints some text, noting that when defining the function no actual print operation is done:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def demo_func():\n",
    "    print('Hello')\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function only works, after it is defined, by invoking its name with brackets. In the jargon the function is *called*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "demo_func()    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next this function is redefined, so that it takes a single input, which we call `text`. This input is referred to as an *argument* and is a label that will only be filled with a definite value when the function is run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def demo_func(text):\n",
    "    print(text)\n",
    "\n",
    "demo_func('abcdefg')\n",
    "demo_func('GCAT')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the `text` variable is internal to the function and not defined outside:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following function called `my_calc`, takes two input values (two arguments) which are labelled as `x` and `y` inside the function and which do not have any specific values inside the definition. The definition involves specifying a calculation, the result of which, `z` is passed back from the function at the `return` statement. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_calc(x, y):      # Two inputs\n",
    "    z = x * x - y * y   # The operations to perform\n",
    "    return z            # Send back the result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the definition exists the name `my_calc` can be used on two input values, which fill the `x` and `y` slots in the function, generating a value for `z`, which is then what is output from the function and, in this case, printed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = my_calc(4,5)    # internal x set to 4, y to 5\n",
    "print(r)            # the result\n",
    "print(my_calc(3,2)) # run on different values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is significant flexibility with the input arguments of Python functions. They can have defaults, for when they are not explicitly specified, and there is freedom to use named or unnamed arguments. Named input arguments can appear in any order though they must be stated after any unnamed ones, which fill slots in order, as shown above. The next example is a modification of the previous function which uses a default value of `1` for `y` on the `def` line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_calc(x, y=1):     # x is mandatory, y defaults to 1 if not given\n",
    "    z = x * x - y * y\n",
    "    return z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calling the function is then illustrated with and without specifying an explicit value for `y`, using the named and unnamed conventions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = my_calc(7)           # x is 7, y defaults to 1  \n",
    "b = my_calc(x=2, y=2)    # name both arguments\n",
    "c = my_calc(y=9, x=-1)   # name arguments in a different order\n",
    "d = my_calc(3, y=-2)     # unnamed arguments (x is 3) come first\n",
    "print(a, b, c, d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Decorators are a relatively recent addition to Python and allow a modifying statement to be added to the start of a function with a ‘@’ syntax. In essence this wraps one function with another, which can modify and inspect both its input and output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@decorator_func \n",
    "def my_calc(x, y=1):\n",
    "    z = x * x - y * y\n",
    "    return z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the sake of brevity, creating decorator functions will not be discussed; however decorator use will be shown in later examples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 9: Reverse-complement function</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>Write a function to find the reverse complement of a DNA sequence, pairing G with C, A with T and their corresponding reciprocals (this mapping can be stored as a Python dictionary). The function should accept an input sequence seq and give back (return) a reverse complement sequence. The sequence can be assumed to be an iterable collection of characters like a list or a string. After the defining the function, call the function with some test data to check it works properly.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code\n",
    "def rev_complement(seq):\n",
    "    rc_seq = ''\n",
    "    \n",
    "    # fill function contents\n",
    "\n",
    "test_seq = 'AGCATAAGAATAGCAGCAGCGCGA'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modules"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some functions, like `len()` or `int()`, are available at any time in Python. However, functions must often be imported into a program from a separate module before they can be used. There are three basic sources of modules: those that automatically come as part of every Python installation (the Standard Library), those that require a separate installation (such as NumPy or BioPython) and those that are specific to the user. If a module is accessible to Python it can be used via the `import` keyword and the various components of the module are referred to with dot syntax:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math           # Import the inbuilt mathematics module\n",
    "print(math.e)         # An attribute representing e\n",
    "print(math.exp(2.0))  # Use the exponent function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to locally use a different name for the module using the `import .. as ..` syntax:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math as m      # Import as a different name\n",
    "print(m.log(2.0))     # Use the logarithm function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, specific components of a module may be imported using a `from .. import ..` syntax:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import sqrt, cos   # Import named module components\n",
    "x = sqrt(3/2)                # Use the square root function\n",
    "print(cos(x))                # Use the cosine function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `math` module used above contains a several commonly used mathematical constants and functions. There are various other commonly used libraries in the standard set, some of which we illustrate below. However, for a full module listing see the documentation at [python.org](https://docs.python.org/3/py-modindex.html). The `sys` module relates to the run-time Python environment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "print(sys.argv) # List of words typed at the command line after \"python\"\n",
    "print(sys.path) # Directory search path used to locate Python modules \n",
    "sys.exit()      # Quit the Python program"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `time` module has some simple functions for handling time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "time.sleep(5)       # Pause for 5 seconds\n",
    "print(time.time())  # Number of seconds since the start of Unix epoch (00:00:00, 1 January 1971)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, it is the `datetime` module that has more comple functionality that allows you to work with calendar days."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "now = datetime.date.today()\n",
    "print(now)\n",
    "print(now.month)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is especially useful when subtracting dates to calculate timespans:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "c20 = datetime.date(1900,1,1)\n",
    "delta = now - c20\n",
    "print(delta.days)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `random` module is use to generate pseudo-random numbers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "random.seed(3)                   # Set the random number seed\n",
    "print(random.randint(1,10))      # A random integer from 1 to 10 inc.\n",
    "print(random.uniform(0.0, 2.0))  # A random float between 0.0 and 2.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = [1,2,3,4,5]\n",
    "random.shuffle(data)             # Shuffle list order\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `os` module is for things that depend on which particular operating system (e.g. Windows, OSX, Linux) is running. Much of this relates to use of file systems, i.e. dealing with file paths, directories, permissions etc. Note that locations in the file system (directories and file names) are simpy specified as a text string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.chdir('/home/user/')        # Change the current working directory\n",
    "dir_name = 'temp'\n",
    "os.mkdir(dir_name)             # Make a new directory\n",
    "os.listdir(dir_name)           # Get a list of directory contents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_path = '/home/user/test.py'\n",
    "os.remove(file_path)            # Delete a file \n",
    "os.rename(file_path, new_path)  # Move a file to a new location"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is notable that `os` doesn’t handle copying files; this is done with the `shutil` module. Within the `os` module of particular importance is the `os.path` submodule which handles the text strings that represent locations within the file system.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_path = 'LMB_Python_Basics.ipynb'\n",
    "print(os.path.exists(file_path))   # True if the file path exists, else False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_path = os.path.abspath(file_path)\n",
    "print(file_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(os.path.split(file_path))    # Split into [leading, end/file] parts                               "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(os.path.isdir(file_path))    # True if a directory, else False\n",
    "print(os.path.join('usr','local')) # Join with directory separator 'usr/local' \n",
    "print(os.path.splitext(file_path)) # Chop the file extension ['folder/file', '.txt']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `re` module is used for regular expressions; pattern matching in text strings. Often the functions generate a special match object that can be interrogated, for example to find where the pattern was found and what the actual substring was."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "pattern = re.compile('\\d+') # Make a regular expression object (one or more digits)\n",
    "text = 'A 123 B 456'        # A string to look in \n",
    "\n",
    "match_obj = pattern.search(text)  # Match inside string\n",
    "print(match_obj)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(match_obj.group())          # '123' – matching substring\n",
    "print(match_obj.start())          # 2 – position of match"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_2 = pattern.sub('**', text)  # Substitute all matches with '**'\n",
    "print(text_2)                     # 'A ** B **' "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hits = pattern.findall(text)      # List of matching sub-strings\n",
    "print(hits)                       # ['123', '456']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 10: Using the `math` module</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>Using a loop construct a list of sine and cosine pairs for every ten degrees in the range zero to 180. Hint: use `math.radians()` to convert from degrees.</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code\n",
    "import math\n",
    "p = []\n",
    "for a in range(0, 181, 10):\n",
    "    r = math.radians(a)\n",
    "    s = math.sin(r)\n",
    "    c = math.cos(r)\n",
    "    p.append((s,c))\n",
    "print(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A table listing a selection of commonly used modules from the Python Standard Library is given below. Full documentation of these and the other modules mentioned above can be found at [python.org](https://docs.python.org/3/py-modindex.html).\n",
    "\n",
    "Module|Description\n",
    "------|------------\n",
    "argparse|A module that helps interpret command-line options/arguments, i.e. information typed after the name of a program, as available in sys.argv.\n",
    "copy|Create a new Python object by copying an existing object. Can create shallow or deep copies; whether any object contained by an object is itself also copied.\n",
    "datetime|Provides date, time, timedelta and datetime objects to represent temporal information. Deals with daylight savings, date formatting, time string interpretation etc.\n",
    "glob|Provides file name fetching using UNIX-like wild cards, i.e. patterns that include \\u201c*\\u201d and \\u201c?\\u201d rather than regular expressions.\n",
    "ftplib|Used to send and receive files using the File Transfer Protocol. \n",
    "gzip, bz2, zipfile, tarfile|Modules that deal with creating and extracting compressed and/or archived files. \n",
    "http |Used to send and receive information across the Internet using the Hypertext Transport Protocol. A lower level library than urllib.\n",
    "multiprocessing|Run Python code as separate, parallel, processes/jobs on multiple core/processor systems.\n",
    "pickle|Converts Python object data to and from a text string (serialization) which may be saved to or loaded from a file system.\n",
    "platform|Used to get information about the current computer and its architecture.\n",
    "shutil|Performs higher level file operations, such as copying files, copying trees of files and finding executable files.\n",
    "sqlite3|Interaction with SQLite : a lightweight file-based SQL database.\n",
    "string|Provides some functions not directly available to string objects. Useful for accessing particular sets of characters such as whitespace, punctuation, digits etc. \n",
    "subprocess|Run an external program as a separate job/process and connect any input/output data streams.\n",
    "time|Basic time related functions using numbers and strings. Can be used to time program execution and to pause execution (time.sleep()) .\n",
    "threading|Run Python code in separate threads, which will not run concurrently on multiprocessor systems (use multiprocessing for that). Can be useful to process intermittent data streams.\n",
    "urllib |Used to send and receive information across the Internet. Higher level and often more convenient than httplib. Handles web proxies, redirection, passwords, cookies etc. Often used to interact with web services and databases.\n",
    "zlib|Used to compress data into more compact representation, using the zlib algorithm. Can be useful for caches and undo functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Writing custom modules is easily achieved in Python: in general a normal file containing Python code can be imported as a module. There are a few caveats to this, but most importantly the Python system needs to know where to look in the file system to find a module. Modules are found by looking in a series of directories, the *search path*, for file names that match the attempted `import`. Some of these will be in standard locations for the Python installation, but search directories can be added at any time via `sys.path`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "print(sys.path)                        # Current module search path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sys.path.append('/home/tim/my_modules')\n",
    "print(sys.path)                        # Extended search path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Considering a file called `my_module.py` that resides in the current working directory and which has the following contents:<br>\n",
    "`\n",
    "CONSTANT = 1.0545718e−34       # Example constant\n",
    "def my_calc(x, y):             # Example function\n",
    "    return x*x - 2*x*y - y*y`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once, the directory containing the Python file is present in `sys.path` we can access it with `import my_module` in a different Python program and use its variables and functions. It is notable that the file extension (`.py` here) is omitted from the module name when it is imported."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sys.path.append(os.getcwd())    # Directory with my_module.py\n",
    "import my_module                # Import custom module (no .py)\n",
    "\n",
    "print(my_module.CONSTANT)       # Use variable from other Python file\n",
    "print(my_module.my_calc(9,-1))  # Use function from other Python file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is often convenient to more permanently add the locations of custom Python modules to the search path by adding them to the PYTHONPATH environment variable (generally set in the operating system), so that `sys.path` automatically contains the required locations whenever any Python program is run. If instead the module was located inside a sub-directory of the one in `sys.path`, such as `‘examples/’`, relative to the current working directory, then the module could be imported as follows, using dot syntax."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import examples.demo_module     # Import from sub-directory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(Before Python version 3.3, to make thid sub-directory import work, a file called `__init__.py`, which is usually blank or contains only `pass`, must be present in the `my_examples` directory.("
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When a module is imported its contents are run as Python. While this is usually no problem for defining constants and functions etc. sometimes the file may contain code that should only be run when the file is used directly and not when it is imported. To overcome this, the special internal Python `__name__` variable can be inspected. This will be the string `__main__` if the code is run as the main program, but will otherwise be the name of the module. Hence this can simply be checked, to make sure the code is not imported as a module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_calc(x, y):             # Example function\n",
    "    return x*x - 2*x*y - y*y\n",
    "\n",
    "if __name__ == '__main__':     # Is the code run as a main program?\n",
    "    print(my_calc(2,3))        # Run test code only when it is main\n",
    "    print(my_calc(-1,0))       # Module imports do not run this block"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Though the standard libraries are extensive and custom modules allow great flexibility there are a large number of external modules that are really useful for molecular biology, and which mean you can build programs using pre-written and well-tested code. A small selection of popular general purpose external modules useful for molecular biology is given in the below table."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Module|Description\n",
    "------|-----------\n",
    "NumPy|Numeric Python with a highly-functional multidimensional array object and associated functionality for linear algebra, pseudo-random numbers, Fourier transforms etc.\n",
    "BioPython|A collection of tools for computational biology. Provides modules to work with and manipulate many common bioinformatics format files relating to sequences, alignments, phylogentics, molecular structures etc. \n",
    "PySam|The Python interface to SAMtools, which allows reading, writing and manipulation of BAM and SAM format files used in high-throughput DNA sequence mapping. Also deals with variant call VCF/BCF files.\n",
    "HTSeq|A module for the analysis of high-throughput DNA sequencing data. Deals with the common informatics formats including FASTQ, SAM, BAM, BED and GFF. Has specialized data structures to handle large genomic array and genomic interval data.\n",
    "PIL|The Python Imaging Library which handles loading and saving image data in a large number of different file formats. Provides many functions to manipulate image data: enhance, filer and mask etc.\n",
    "Pandas|Manipulation and analysis of data tables with an organization similar to spreadsheets and relational databases. Provides many advanced functions for data manipulation and file access (including Excel, CSV and general text).\n",
    "Pymol, UCSF Chimera|Popular molecular structure viewers that may be imported as Python modules to produce graphics for protein structures etc.\n",
    "SciPy|Extensive scientific an engineering library, building upon NumPy with modules for statistics, integration, optimization, signal processing etc.\n",
    "matplotlib|A plotting library to create many different types of charts and graphs from numeric data, with customizable graphical styles.\n",
    "scikit-learn|A library for machine learning, clustering, regression, dimensionality reduction etc. that is built upon SciPy and NumPy.\n",
    "keras|Deep neural networks using a TensorFlow or Theano back-end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Stored data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A key operation in most programs is reading and saving data to and from disk, or other storage filesystem. In Python we can choose to handle all of the writing explicitly in a program. However, if there is a module which handles a particular data format then that is often a good choice instead. For example, we can use BioPython modules to read most common bioinformatics formats and use PANDAS to read and write CSV files and Excel spreadsheets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "File operations revolve around a file object. This is typically generated using the inbuilt `open()` function, given the location of the data on the file system. Once the file object is created various functions can be used to read and write data; the object is the Python interface to the stored data. In the next example a file object is created, and all of its data read. The use of `'r'` is to specify that the file is opened for reading, though this is the default when not specified. The data may be read in its entirety using `read()`: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = 'examples/demo_file.txt'   # Location of data in file system\n",
    "file_obj = open(path, 'r')        # Create file object for reading\n",
    "data = file_obj.read()            # Read all the data (as a string)\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the data is re-read then nothing results because the file object remembers the last read point and this refers to the end of the data after the first complete read."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = file_obj.read()            # Nothing remaining\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to change this we can use `seek(`) to go to a particular position (in bytes with `0` being the start). When the file is no longer needed it is closed for further use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(file_obj.read())   # Empty, nothing to read at end of file\n",
    "file_obj.seek(0)         # Point at start of data      \n",
    "print(file_obj.read())   # Read everything\n",
    "file_obj.close()         # Close file, no further operations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recently it has become common practice to use the with.. as.. syntax to deal with file objects. The with block creates a managed context, which in this case means that the file object is only open within the block and closed automatically after."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(path) as file_obj:\n",
    "    print(file_obj.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are other functions to read the file in terms of lines, separated with \\n (or \\r if universal read mode 'rU' mode is used). All lines can be read at once using readlines() or readline() can fetch a single line. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(path, 'rU') as file_obj:  # NOTE: 'U' mode will become obsolete in Python 3\n",
    "    print(file_obj.readline())      # First line\n",
    "    print(file_obj.readline())      # Second line\n",
    "    print(file_obj.readlines())     # All remaining lines (as a list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Subsequent readline() calls give each line in turn; the pointer to the file data picks up at the end of the previous line. However, it is often more convenient to treat the file object as an iterator and loop through it, i.e. as if it were a list of lines:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_obj = open(path, 'r')\n",
    "for line in file_obj:       # Loop through all lines in file\n",
    "    print(line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many files that are commonly read by Python are essentially stored as text. Consequently, any numbers must be properly interpreted as such, appropriately creating proper `int` or `float` objects, if we want to use the data for calculations etc. It is fairly common to have file data formatted so that each line represents a different item and each line contains multiple columns, separated by commas, spaces or tabs, such as:\n",
    "\n",
    "`\n",
    "chr  length          prot_genes\n",
    "1\t248956422\t2058\n",
    "2\t242193529\t1309\n",
    "3\t198295559\t1078`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code will read this kind of file, given file name, and put the data into a list. Note how the first header line (which contains no numeric data) is read before the `for` loop and that the lines are split (in this instance on the default white-space) to make sub-lists.\n",
    "."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_path = 'examples/chromo_stats.tsv'\n",
    "\n",
    "data = []\n",
    "with open(file_path) as file_obj:\n",
    "    head = file_obj.readline()    # Read first header line; not used\n",
    "    for line in file_obj:         # Go through each remaining line\n",
    "        row = line.split()        # Split line at whitespace into a list\n",
    "        data.append(row)\n",
    "\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=purple>Exercise 11: </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=purple>Extend the above example that reads a tab separated value (TSV) file line-by-line. Extract the numerical value from the second column and calculate the sum over the whole file, i.e. the total of chromosome lengths. </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise code\n",
    "file_path = 'examples/chromo_stats.tsv'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For writing to a file mode `'w'` or mode `'a'` must be used: opening with `'w'` mode initially writes a blank file (deleting any previous data) while `‘a’` appends to the end of a file. To do the actual writing, data is simply passed to the `write()` function, in one or more parts. If we want to save the data as lines the appropriate newline characters (`'\\n'` etc.) need to be added."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_path = 'demo_out.txt'\n",
    "file_obj = open(file_path, 'w')  # Open file object in writing mode\n",
    "\n",
    "x = 1\n",
    "while x < 1000:               # A loop which will generate many lines\n",
    "  line = '{}\\n'.format(x)     # Create the string for each line\n",
    "  file_obj.write(line)        # Write each line\n",
    "  x *= 2\n",
    "\n",
    "file_obj.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When dealing with files it is common practice to accept the name of the file to use at the time when the program is run. On the command line this is easily achieved by specifying the name of the file after the Python script name. What was entered at the command line is then accessed using the `sys.argv` list. For example, if the following is typed at the operating system command prompt (>):\n",
    "\n",
    "> python programFile.py data/inputFile.txt\n",
    "\n",
    "then the name of the files can be captured in the following way, noting that the first item in the sys.argv list is the Python script itself, so it is the second item (index 1) that we usually want."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys                 # Access the sys module\n",
    "py_script = sys.argv[0]    # 'programFile.py'   \n",
    "data_file = sys.argv[1]    # 'data/inputFile.txt'\n",
    "\n",
    "data = open(data_file).read()  # Make file object and read its data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When dealing with common data file formats there may be a module that already deals with file access; to handle all of the reading and writing for you and interpret the data properly. For example the BioPython module can read and write many bioinformatics formats. In the next example we import the SeqIO module. This can read from an open file object using its `parse()` function to generate sequence record objects (named protein below) from which data is accessed via the dot syntax. In this case the data format is specified as FASTA, common for protein or nucleic acid sequences:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Bio import SeqIO    # Load BioPython module; must be installed\n",
    " \n",
    "file_name = \"examples/demo_sequences.fasta\"   # Location of data\n",
    "file_obj = open(file_name)\n",
    "\n",
    "for protein in SeqIO.parse(file_obj, 'fasta'): # Go through each entry\n",
    "  print(protein.id)                            # The ID of seq record\n",
    "  print(protein.seq)                           # The actual sequence\n",
    "\n",
    "file_obj.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next the Pandas module is demonstrated to show its ability to read character-separated value files (CSV) using the `read_csv()` function. Similar `read_excel()` and `read_sql_table()` functions, for Excel spreadsheets and SQL databases, also exist. The Pandas module stores data in DataFrame objects, which print nicely."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas\n",
    "file_path = 'examples/chromo_stats.tsv'\n",
    "data_set = pandas.read_csv(file_path, sep='\\t', header=0)\n",
    "\n",
    "print(type(data_set))    \n",
    "print(data_set)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here the `DataFrame` columns are accessed in a similar way to dictionaries:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for value in data_set['chr']:\n",
    "      print(value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Download an informatics file from a URL"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following function illustrates how data can be fetched over the Internet using the urlib module which does the hard work of finding the requested file and downloading its data. The url variable contains most of the Internet address location but needs a specific database entry identifier (db_id) to be inserted into the address (using format()) before the connection can be opened to the correct file. There is a slight complication in this function as the data is downloaded in its raw binary form and we have to specifically convert it to a Python string object via decode() which interprets the data using a standard (UTF-8) character encoding scheme."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def download_db_id(db_id, url, file_name=None): # Save file is optional\n",
    "\n",
    "  from urllib.request import urlopen     # Import urlopen() function  \n",
    "  response = urlopen(url.format(db_id))  # Makes object to handle link\n",
    "  data = response.read()                 # Read data from URL\n",
    "  data = data.decode('utf-8')            # Interpret data as plain text\n",
    "  \n",
    "  if file_name:                       # If save file was specified...\n",
    "    file_obj = open(file_name, 'w')   # Create file object for writing\n",
    "    file_obj.write(data)              # Save all data to file\n",
    "    file_obj.close()\n",
    "\n",
    "  return data                         # Hand back data from function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function is tested by downloading a protein structure from the PDB. The URL for these databases are given below and include ‘{}’ indicating where the identifier code for the specific database entry will be added. It is notable that these variables use the uppercase convention as they are acting as constants and lie outside the functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "PDB_URL = 'https://files.rcsb.org/download/{}.pdb'\n",
    "\n",
    "data = download_db_id('1AFO', PDB_URL, '1AF0.pdb')  # Save .pdb file\n",
    "print(data)                                   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next example combines the file download in the above example with use of BioPython. The SeqIO module from BioPython will be used to handle the FASTA format data obtained from the UniProt database, which saves us from having to interpret the data format ourselves. The function takes a database entry identifier db_id and downloads the FASTA format data, as illustrated above, before automatically parsing the data and sending back the (first or only) sequence as a sequence record object created by BioPython. This object has handy attributes that can be accessed via the dot syntax to get at the actual sequence etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "UNIPROT_FASTA_URL = 'http://www.uniprot.org/uniprot/{}.fasta'\n",
    "\n",
    "def get_uniprot_seq_record(db_id):\n",
    "\n",
    "   from Bio import SeqIO\n",
    "   \n",
    "   file_name = db_id + '.fasta'  # Add extension to ID to make file name\n",
    "   \n",
    "   if not os.path.exists(file_name):\n",
    "     download_db_id(db_id, UNIPROT_FASTA_URL, file_name)\n",
    "   \n",
    "   file_obj = open(file_name)    # Open file for reading\n",
    "   \n",
    "   for seq_record in SeqIO.parse(file_obj, 'fasta'):\n",
    "     return seq_record           # Give back first record encountered\n",
    "\n",
    "sr = get_uniprot_seq_record('P18754')\n",
    "print(sr.id, sr.seq)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Object classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All the Python objects (items of data) used thus far have been of the standard types. However it is possible to create custom Python objects using the class keyword. This creates a named prototype which connects data values and bound functions (methods) together in an organized way. A class can often be thought of as equivalent to a table in a database. For reasons of brevity this aspect of Python will not be discussed in great detail in this chapter. However, a simple example is provided below which illustrates a rudimentary Person object. The class block contains the definitions of two functions. It defines the `__init__()` method which, because it has a particular name, is run any time an object of that type is created: here its task is to associate the input name and age with `self`; a special variable that represents the run-time object. A second method `get_first_name()` is defined which extracts and passes back the first part of the full name, which was stored as `self.name`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Person():         # Next indented block is in the class definition\n",
    "\n",
    "  def __init__(self, name, age):  # Values specified when object is made\n",
    "    self.name = name              # Link input values to the object\n",
    "    self.age = age\n",
    "\n",
    "  def get_first_name(self):       # A second, custom function\n",
    "    names = self.name.split()     # self refers to the run-time object\n",
    "    return names[0]               # Give back first word\n",
    "\n",
    "p1 = Person('Lisa Simpson', 8)    # Make object of Person class\n",
    "p2 = Person('Bart Simpson', 10)   # Make another\n",
    "print(p1.age, p2.age)             # Values linked to objects - 8, 10 \n",
    "print(p1.get_first_name())        # Run a linked function - gives 'Lisa'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As illustrated above, two objects were made using the Person prototype, thus creating two different instances of that class: stored as variables `p1` and `p2`. The methods and simple attributes that were associated with the `self` are available to each instance using the dot syntax, e.g. `p1.name` and `p1.get_first_name()`. Here, the `self` value stated in class definition represents the object stated before the dot, i.e. `p1`, and is not passed via the parentheses."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}