Chapter2_Patterns_for_cleaner_python.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "substantial-offense",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "authorized-packet",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "WindowsPath('/Users/olli/Desktop/PythonNotebooks/datasets')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p = Path('/Users/olli/Desktop/PythonNotebooks') \n",
    "sub_dir = 'datasets'\n",
    "data_dir = p/sub_dir\n",
    "data_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "novel-hamilton",
   "metadata": {},
   "source": [
    "# Chapter 2 - Patterns for Cleaner Python\n",
    "___"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "tracked-column",
   "metadata": {},
   "source": [
    "## 2.1 Assertions\n",
    "\n",
    "### SUMMARY\n",
    "\n",
    "- Python’s assert statement is a debugging aid that tests a condition as an internal self-check in your program.\n",
    "- Asserts should only be used to **help developers identify bugs**.\n",
    "  They’re not a mechanism for handling run-time errors.\n",
    "- Asserts can be globally disabled with an interpreter setting. Hence they are vulnerable to security breaches. \n",
    "\n",
    "### What are they?\n",
    "\n",
    "- Internal-self checks for your program\n",
    "- They keep track of impossible conditions in your code\n",
    "\n",
    "### Syntax\n",
    "\n",
    "```python\n",
    "assert_stmt ::= \"assert\" expression1 [\",\" expression2]\n",
    "\n",
    "# At run time it gets converted to:\n",
    "if __debug__:\n",
    "    if not expression1:\n",
    "        raise AssertionError(expression2)\n",
    "\n",
    "```\n",
    "\n",
    "### Example\n",
    "\n",
    "```python\n",
    "assert 0 <= price <= product['price']\n",
    "```\n",
    "\n",
    "### Reasons for using assertions\n",
    "\n",
    "- Speed up debugging efforts\n",
    "- Makes code more maintainable\n",
    "- exception stacktrace points out the _exact line of code containing the failed assertion_\n",
    "\n",
    "### What not to use them for?\n",
    "\n",
    "- You shouldn't use them as a way to handle _run-time errors_\n",
    "- _Don't use them for data validation_ as they can be globally disabled. Use  conditional logic and raise validation exceptions\n",
    "- They can introduce security risks and bugs into your applications if you are not careful\n",
    "\n",
    "### Common pitfalls\n",
    "\n",
    "1. Introducing security risks\n",
    "\n",
    "```python\n",
    "# if assertions are disabled the security checks will not take place\n",
    "# also the product id may be invalid which opens the door to denial of service attacks\n",
    "\n",
    "#### INCORRECT WAY ####\n",
    "def delete_product(prod_id, user):\n",
    "    assert user.is_admin(), 'Must be admin'\n",
    "    assert store.has_product(prod_id), 'Unknown product'\n",
    "    store.get_product(prod_id).delete()\n",
    "\n",
    "\n",
    "#### CORRECT WAY ####\n",
    "def delete_product(product_id, user):\n",
    "    if not user.is_admin():\n",
    "        raise AuthError('Must be admin to delete')\n",
    "    if not store.has_product(product_id):\n",
    "        raise ValueError('Unknown product id')\n",
    "    store.get_product(product_id).delete()\n",
    "\n",
    "```\n",
    "\n",
    "2. Asserts that never fail\n",
    "\n",
    "It's suprisingly easy to write Python assert statements that always evaluate to true. This has to do with non-empty tuples always being truthy in Python. If you pass a tuple to an assert statement, it leads to the assert condition always being true.\n",
    "\n",
    "```python\n",
    "\n",
    "#### EXAMPLE OF ASSERTIONS THAT ALWAYS EVALUATES TO TRUE ####\n",
    "\n",
    "assert(1 == 2, 'This should fail')\n",
    "\n",
    "\n",
    "assert (\n",
    "    counter == 10,\n",
    "    'It should have counted all the items'\n",
    ")\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "suitable-sessions",
   "metadata": {},
   "outputs": [],
   "source": [
    "# example assertion use\n",
    "def apply_discount(product, discount):\n",
    "    price = int(product['price'] * (1.0 - discount))\n",
    "    assert 0 <= price <= product['price'], \"invalid price\"\n",
    "    return price\n",
    "\n",
    "\n",
    "# price in cents\n",
    "shoes = {\n",
    "    'name': 'Fancy Shoes',\n",
    "    'price': 14900\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "mineral-summary",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11175\n"
     ]
    }
   ],
   "source": [
    "# valid input\n",
    "print(apply_discount(shoes, 0.25))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "smooth-marking",
   "metadata": {},
   "outputs": [
    {
     "ename": "AssertionError",
     "evalue": "invalid price",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mAssertionError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-7-045bbc4f3f59>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[1;31m# invalid input\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mapply_discount\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mshoes\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m2\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m<ipython-input-5-2515ec168cde>\u001b[0m in \u001b[0;36mapply_discount\u001b[1;34m(product, discount)\u001b[0m\n\u001b[0;32m      2\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mapply_discount\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mproduct\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdiscount\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m     \u001b[0mprice\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mproduct\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'price'\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m*\u001b[0m \u001b[1;33m(\u001b[0m\u001b[1;36m1.0\u001b[0m \u001b[1;33m-\u001b[0m \u001b[0mdiscount\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 4\u001b[1;33m     \u001b[1;32massert\u001b[0m \u001b[1;36m0\u001b[0m \u001b[1;33m<=\u001b[0m \u001b[0mprice\u001b[0m \u001b[1;33m<=\u001b[0m \u001b[0mproduct\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'price'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"invalid price\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      5\u001b[0m     \u001b[1;32mreturn\u001b[0m \u001b[0mprice\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      6\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mAssertionError\u001b[0m: invalid price"
     ]
    }
   ],
   "source": [
    "# invalid input\n",
    "print(apply_discount(shoes, 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "breeding-lyric",
   "metadata": {},
   "source": [
    "___\n",
    "## 2.2 Context Managers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "lovely-sharp",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.\n"
     ]
    }
   ],
   "source": [
    "# read with context handler\n",
    "# context managers abstractify management of files \n",
    "# they also can account for an exception unlike the try finally structure\n",
    "with open(data_dir/'lorem_ipsum.txt') as file:\n",
    "    print(file.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "latest-danger",
   "metadata": {},
   "outputs": [],
   "source": [
    "# we can add this type of context management to our own classes\n",
    "class ManagedFile:\n",
    "    def __init__(self,name):\n",
    "        self.name = name\n",
    "    \n",
    "    def __enter__(self):\n",
    "        print('opening file')\n",
    "        self.file = open(self.name,'r')\n",
    "        return self.file\n",
    "    \n",
    "    def __exit__(self,exc_type,exc_val,exc_tb):\n",
    "        print('closing file')\n",
    "        if self.file:\n",
    "            self.file.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "sweet-phrase",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "opening file\n",
      "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.\n",
      "closing file\n"
     ]
    }
   ],
   "source": [
    "with ManagedFile(data_dir/'lorem_ipsum.txt') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "coral-budapest",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "opening file\n",
      "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.\n",
      "closing file\n"
     ]
    }
   ],
   "source": [
    "# here is another way to do this with decorators \n",
    "from contextlib import contextmanager\n",
    "\n",
    "@contextmanager\n",
    "def managed_file(name):\n",
    "    try:\n",
    "        print('opening file')\n",
    "        f = open(name, 'r')\n",
    "        yield f\n",
    "    finally:\n",
    "        print('closing file')\n",
    "        f.close()\n",
    "\n",
    "\n",
    "with managed_file(data_dir/'lorem_ipsum.txt') as f:\n",
    "    print(f.read())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "abstract-weekend",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.\n"
     ]
    }
   ],
   "source": [
    "# alternative using try finally \n",
    "# this will fail to account for exceptions\n",
    "\n",
    "f = open(data_dir/'lorem_ipsum.txt')\n",
    "try:\n",
    "    print(f.read())\n",
    "finally:\n",
    "    f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "regulation-extent",
   "metadata": {},
   "source": [
    "___\n",
    "## 2.3 Underscores and their uses"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "scenic-proposition",
   "metadata": {},
   "source": [
    "### Uses with classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "assured-present",
   "metadata": {},
   "outputs": [],
   "source": [
    "# this will set a Test class variable\n",
    "_Test__mangled = 23\n",
    "\n",
    "class Test:\n",
    "    def __init__(self):\n",
    "        self.foo = 21\n",
    "        self._bar = 17\n",
    "        self.__set_car(13)\n",
    "    \n",
    "    def __set_car(self,value):\n",
    "        if value <= 0:\n",
    "            raise ValueError('Invalid value')\n",
    "        else:\n",
    "            self.__car = value\n",
    "    \n",
    "    def get_car(self):\n",
    "        return self.__car\n",
    "    \n",
    "    def mangled_test(self):\n",
    "        return __mangled"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "handy-semiconductor",
   "metadata": {},
   "outputs": [],
   "source": [
    "t = Test()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "proper-entrepreneur",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.foo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "turkish-journalism",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t._bar"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "advised-person",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.get_car()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "informed-patio",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "23"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.mangled_test()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "technological-mortality",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['_Test__car', '_Test__set_car']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[x for x in dir(t) if 'Test' in x]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "above-button",
   "metadata": {},
   "source": [
    "### Placeholders"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "broke-contact",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hello\n",
      "hello\n",
      "hello\n",
      "hello\n",
      "hello\n"
     ]
    }
   ],
   "source": [
    "for _ in range(5):\n",
    "    print('hello')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "alert-signature",
   "metadata": {},
   "source": [
    "### Import caveats\n",
    "\n",
    "```python\n",
    "# my_module.py:\n",
    "def external_func():\n",
    "    return 23\n",
    "def _internal_func():\n",
    "    return 42  \n",
    "```\n",
    "\n",
    "```python\n",
    "# main.py \n",
    "from my_module import * \n",
    "external_func() \n",
    "_internal_func() # this will not work \n",
    "```\n",
    "\n",
    "```python \n",
    "# main.py\n",
    "import my_module \n",
    "my_module.external_func() \n",
    "my_module._internal_func() # this will work\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "public-coordinator",
   "metadata": {},
   "source": [
    "### Name Mangling and Inheritance "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "ecological-dating",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Test: \n",
    "    def __init__(self):\n",
    "        self.foo = 11\n",
    "        self._bar = 23 \n",
    "        self.__baz = 42\n",
    "        \n",
    "class ExtendedTest(Test):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.foo = 'overidden'\n",
    "        self._bar = 'overidden'\n",
    "        self.__baz = 'overidden'\n",
    "        \n",
    "    def __set_baz(self,value):\n",
    "        if value not in ['red', 'blue', 'green']:\n",
    "            raise ValueError('Please enter red, blue or green')\n",
    "        self.__baz = value\n",
    "    \n",
    "    def get_baz(self):\n",
    "        return self.__baz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "loving-killing",
   "metadata": {},
   "outputs": [],
   "source": [
    "t = ExtendedTest()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "related-tooth",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['_ExtendedTest__baz',\n",
       " '_ExtendedTest__set_baz',\n",
       " '_Test__baz',\n",
       " '__class__',\n",
       " '__delattr__',\n",
       " '__dict__',\n",
       " '__dir__',\n",
       " '__doc__',\n",
       " '__eq__',\n",
       " '__format__',\n",
       " '__ge__',\n",
       " '__getattribute__',\n",
       " '__gt__',\n",
       " '__hash__',\n",
       " '__init__',\n",
       " '__init_subclass__',\n",
       " '__le__',\n",
       " '__lt__',\n",
       " '__module__',\n",
       " '__ne__',\n",
       " '__new__',\n",
       " '__reduce__',\n",
       " '__reduce_ex__',\n",
       " '__repr__',\n",
       " '__setattr__',\n",
       " '__sizeof__',\n",
       " '__str__',\n",
       " '__subclasshook__',\n",
       " '__weakref__',\n",
       " '_bar',\n",
       " 'foo',\n",
       " 'get_baz']"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dir(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "moderate-binding",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'overidden'"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# overidden as expected \n",
    "t.foo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "executive-privilege",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'overidden'"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# overidden as expected\n",
    "t._bar"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "effective-overview",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# name mangling for parent class\n",
    "t._Test__baz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "asian-middle",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'overidden'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# name mangling for child class\n",
    "t._ExtendedTest__baz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "related-detection",
   "metadata": {},
   "outputs": [],
   "source": [
    "# this will access the pseudo-private set_baz method\n",
    "t._ExtendedTest__set_baz('red')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "bizarre-monroe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'red'"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we can see this class instance has baz set to red\n",
    "t.get_baz()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "later-sword",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# and the mangled parent variable is untouced\n",
    "t._Test__baz"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fiscal-haiti",
   "metadata": {},
   "source": [
    "___\n",
    "## 2.5 String Formatting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "corresponding-friendly",
   "metadata": {},
   "outputs": [],
   "source": [
    "num = 50159747054\n",
    "name = 'Bob'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "basic-colombia",
   "metadata": {},
   "source": [
    "### Old Style "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "tracked-street",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello, badc0ffee'"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# x placeholder in string converts number to hexadecimal\n",
    "'Hello, %x' % num"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "personal-chase",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hey Bob, there is a 0xbadc0ffee error!'"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'Hey %(name)s, there is a 0x%(num)x error!'% {\n",
    "    'name' : name, 'num' : num\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "completed-diary",
   "metadata": {},
   "source": [
    "### New Style"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "christian-promotion",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello, Bob'"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'Hello, {}'.format(name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "suburban-hybrid",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hey Bob, there is a 0xbadc0ffee error!'"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'Hey {name}, there is a 0x{num:x} error!'.format(name=name, num=num)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "involved-embassy",
   "metadata": {},
   "source": [
    "### Literal String Interpolation\n",
    "\n",
    "Reference: https://zetcode.com/python/fstring/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "conscious-nursery",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello, Bob!'"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f'Hello, {name}!'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "golden-beach",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Five plus ten is 15 and not 30.'"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# evaluate expression inline\n",
    "a = 5\n",
    "b = 10\n",
    "f'Five plus ten is {a + b} and not {2 * (a + b)}.'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "clean-directory",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Hey Bob, there's a 5.02e+10 error!\""
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f\"Hey {name}, there's a {num:.2e} error!\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fleet-accommodation",
   "metadata": {},
   "source": [
    "### Template Strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "fossil-albuquerque",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hey, Bob!'"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from string import Template\n",
    "\n",
    "t = Template('Hey, $name!')\n",
    "t.substitute(name = name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "roman-roulette",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hey Bob, there is a 0xbadc0ffee error!'"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "templ_string = 'Hey $name, there is a $num error!'\n",
    "Template(templ_string).substitute(name=name, num=hex(num))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "driving-abuse",
   "metadata": {},
   "source": [
    "### Attack vector of format string\n",
    "\n",
    "Highlights the important use case of Template strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "experienced-stand",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'this-is-a-secret'"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# SECRET CAN BE ACCESSED WITH FORMAT STRING\n",
    "SECRET = 'this-is-a-secret'\n",
    "\n",
    "class Error:\n",
    "    def __init__(self):\n",
    "        pass\n",
    "\n",
    "# user can trigger error and pass input    \n",
    "err = Error()\n",
    "user_input = '{error.__init__.__globals__[SECRET]}'\n",
    "# Uh-oh...\n",
    "user_input.format(error=err)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "expensive-municipality",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Invalid placeholder in string: line 1, col 1",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-42-374ff8c60fca>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      2\u001b[0m \u001b[1;31m# ie. you cannot use the error\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[0muser_input\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m'${error.__init__.__globals__[SECRET]}'\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 4\u001b[1;33m \u001b[0mTemplate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0muser_input\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msubstitute\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0merror\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0merr\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m~\\anaconda3\\envs\\Keras-workspace\\lib\\string.py\u001b[0m in \u001b[0;36msubstitute\u001b[1;34m(self, mapping, **kws)\u001b[0m\n\u001b[0;32m    124\u001b[0m             raise ValueError('Unrecognized named group in pattern',\n\u001b[0;32m    125\u001b[0m                              self.pattern)\n\u001b[1;32m--> 126\u001b[1;33m         \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpattern\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msub\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mconvert\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtemplate\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    127\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    128\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0msafe_substitute\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmapping\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0m_sentinel_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m/\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkws\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\anaconda3\\envs\\Keras-workspace\\lib\\string.py\u001b[0m in \u001b[0;36mconvert\u001b[1;34m(mo)\u001b[0m\n\u001b[0;32m    121\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdelimiter\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    122\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mmo\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mgroup\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'invalid'\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 123\u001b[1;33m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_invalid\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmo\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    124\u001b[0m             raise ValueError('Unrecognized named group in pattern',\n\u001b[0;32m    125\u001b[0m                              self.pattern)\n",
      "\u001b[1;32m~\\anaconda3\\envs\\Keras-workspace\\lib\\string.py\u001b[0m in \u001b[0;36m_invalid\u001b[1;34m(self, mo)\u001b[0m\n\u001b[0;32m    104\u001b[0m             \u001b[0mcolno\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mi\u001b[0m \u001b[1;33m-\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m''\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlines\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    105\u001b[0m             \u001b[0mlineno\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlines\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 106\u001b[1;33m         raise ValueError('Invalid placeholder in string: line %d, col %d' %\n\u001b[0m\u001b[0;32m    107\u001b[0m                          (lineno, colno))\n\u001b[0;32m    108\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: Invalid placeholder in string: line 1, col 1"
     ]
    }
   ],
   "source": [
    "# the template string closes the attack vector\n",
    "# ie. you cannot use the error \n",
    "user_input = '${error.__init__.__globals__[SECRET]}'\n",
    "Template(user_input).substitute(error=err)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}