keras_tgs_salt_tpu.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TGS Salt Identification Challenge\n",
    "\n",
    "Several areas of Earth with large accumulations of oil and gas also have huge deposits of salt below the surface.\n",
    "\n",
    "But unfortunately, knowing where large salt deposits are precisely is very difficult. Professional seismic imaging still requires expert human interpretation of salt bodies. This leads to very subjective, highly variable renderings. More alarmingly, it leads to potentially dangerous situations for oil and gas company drillers.\n",
    "\n",
    "To create the most accurate seismic images and 3D renderings, TGS (the world’s leading geoscience data company) is hoping Kaggle’s machine learning community will be able to build an algorithm that automatically and accurately identifies if a subsurface target is salt or not.\n",
    "\n",
    "You can find more infomation about this challenge at [kaggle](https://www.kaggle.com/c/tgs-salt-identification-challenge)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 457
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 14029,
     "status": "ok",
     "timestamp": 1539440918306,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "HrwYWcoMY00C",
    "outputId": "1addccae-c2c6-42fe-de3f-b5a5d8d90cc0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (4.26.0)\n",
      "Collecting kaggle\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/c6/78/832b9a9ec6b3baf8ec566e1f0a695f2fd08d2c94a6797257a106304bfc3c/kaggle-1.4.7.1.tar.gz (52kB)\n",
      "\u001b[K    100% |████████████████████████████████| 61kB 7.0MB/s \n",
      "\u001b[?25hRequirement already satisfied: urllib3<1.23.0,>=1.15 in /usr/local/lib/python3.6/dist-packages (from kaggle) (1.22)\n",
      "Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.6/dist-packages (from kaggle) (1.11.0)\n",
      "Requirement already satisfied: certifi in /usr/local/lib/python3.6/dist-packages (from kaggle) (2018.8.24)\n",
      "Requirement already satisfied: python-dateutil in /usr/local/lib/python3.6/dist-packages (from kaggle) (2.5.3)\n",
      "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from kaggle) (2.18.4)\n",
      "Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from kaggle) (4.26.0)\n",
      "Collecting python-slugify (from kaggle)\n",
      "  Downloading https://files.pythonhosted.org/packages/00/ad/c778a6df614b6217c30fe80045b365bfa08b5dd3cb02e8b37a6d25126781/python-slugify-1.2.6.tar.gz\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->kaggle) (3.0.4)\n",
      "Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->kaggle) (2.6)\n",
      "Collecting Unidecode>=0.04.16 (from python-slugify->kaggle)\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/59/ef/67085e30e8bbcdd76e2f0a4ad8151c13a2c5bce77c85f8cad6e1f16fb141/Unidecode-1.0.22-py2.py3-none-any.whl (235kB)\n",
      "\u001b[K    100% |████████████████████████████████| 235kB 10.6MB/s \n",
      "\u001b[?25hBuilding wheels for collected packages: kaggle, python-slugify\n",
      "  Running setup.py bdist_wheel for kaggle ... \u001b[?25l-\b \bdone\n",
      "\u001b[?25h  Stored in directory: /root/.cache/pip/wheels/44/2c/df/22a6eeb780c36c28190faef6252b739fdc47145fd87a6642d4\n",
      "  Running setup.py bdist_wheel for python-slugify ... \u001b[?25l-\b \bdone\n",
      "\u001b[?25h  Stored in directory: /root/.cache/pip/wheels/e3/65/da/2045deea3098ed7471eca0e2460cfbd3fdfe8c1d6fa6fcac92\n",
      "Successfully built kaggle python-slugify\n",
      "Installing collected packages: Unidecode, python-slugify, kaggle\n",
      "Successfully installed Unidecode-1.0.22 kaggle-1.4.7.1 python-slugify-1.2.6\n"
     ]
    }
   ],
   "source": [
    "# Install the dependencies we need\n",
    "!pip install tqdm\n",
    "!pip install kaggle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "J_XHzgYuZHMN"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import pprint\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from random import randint\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from skimage.transform import resize\n",
    "\n",
    "import tensorflow as tf\n",
    "\n",
    "from tensorflow.keras.preprocessing.image import load_img\n",
    "from tensorflow.keras import Model\n",
    "from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau\n",
    "from tensorflow.keras.models import load_model\n",
    "from tensorflow.keras.optimizers import Adam\n",
    "from tensorflow.keras.layers import Input, Conv2D, Conv2DTranspose, MaxPooling2D, concatenate, Dropout\n",
    "\n",
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 252
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1308,
     "status": "ok",
     "timestamp": 1539440930172,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "j6YTX_mHH_q4",
    "outputId": "978aaf1c-7128-4ed5-ac07-724185388688"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found TPU at: grpc://10.17.106.138:8470\n",
      "TPU devices:\n",
      "[_DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:CPU:0, CPU, -1, 3788293867414123363),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 14218655969421810422),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:XLA_GPU:0, XLA_GPU, 17179869184, 5812111958003439941),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:0, TPU, 17179869184, 8454891999339572241),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:1, TPU, 17179869184, 15413725777721296706),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:2, TPU, 17179869184, 1794556755405902195),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:3, TPU, 17179869184, 8722551663338303265),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:4, TPU, 17179869184, 10265242285712045992),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:5, TPU, 17179869184, 83959706521883974),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:6, TPU, 17179869184, 6395746279842848549),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:7, TPU, 17179869184, 17317425135332706070),\n",
      " _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 17179869184, 13109444320199115483)]\n"
     ]
    }
   ],
   "source": [
    "# Check the TPU we are using\n",
    "try:\n",
    "  device_name = os.environ['COLAB_TPU_ADDR']\n",
    "  TPU_ADDRESS = 'grpc://' + device_name\n",
    "  print('Found TPU at: {}'.format(TPU_ADDRESS))\n",
    "  \n",
    "  with tf.Session(TPU_ADDRESS) as session:\n",
    "    devices = session.list_devices()\n",
    "    \n",
    "  print('TPU devices:')\n",
    "  pprint.pprint(devices)\n",
    "\n",
    "except KeyError:\n",
    "  print('TPU not found')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 90,
     "resources": {
      "http://localhost:8080/nbextensions/google.colab/files.js": {
       "data": "Ly8gQ29weXJpZ2h0IDIwMTcgR29vZ2xlIExMQwovLwovLyBMaWNlbnNlZCB1bmRlciB0aGUgQXBhY2hlIExpY2Vuc2UsIFZlcnNpb24gMi4wICh0aGUgIkxpY2Vuc2UiKTsKLy8geW91IG1heSBub3QgdXNlIHRoaXMgZmlsZSBleGNlcHQgaW4gY29tcGxpYW5jZSB3aXRoIHRoZSBMaWNlbnNlLgovLyBZb3UgbWF5IG9idGFpbiBhIGNvcHkgb2YgdGhlIExpY2Vuc2UgYXQKLy8KLy8gICAgICBodHRwOi8vd3d3LmFwYWNoZS5vcmcvbGljZW5zZXMvTElDRU5TRS0yLjAKLy8KLy8gVW5sZXNzIHJlcXVpcmVkIGJ5IGFwcGxpY2FibGUgbGF3IG9yIGFncmVlZCB0byBpbiB3cml0aW5nLCBzb2Z0d2FyZQovLyBkaXN0cmlidXRlZCB1bmRlciB0aGUgTGljZW5zZSBpcyBkaXN0cmlidXRlZCBvbiBhbiAiQVMgSVMiIEJBU0lTLAovLyBXSVRIT1VUIFdBUlJBTlRJRVMgT1IgQ09ORElUSU9OUyBPRiBBTlkgS0lORCwgZWl0aGVyIGV4cHJlc3Mgb3IgaW1wbGllZC4KLy8gU2VlIHRoZSBMaWNlbnNlIGZvciB0aGUgc3BlY2lmaWMgbGFuZ3VhZ2UgZ292ZXJuaW5nIHBlcm1pc3Npb25zIGFuZAovLyBsaW1pdGF0aW9ucyB1bmRlciB0aGUgTGljZW5zZS4KCi8qKgogKiBAZmlsZW92ZXJ2aWV3IEhlbHBlcnMgZm9yIGdvb2dsZS5jb2xhYiBQeXRob24gbW9kdWxlLgogKi8KKGZ1bmN0aW9uKHNjb3BlKSB7CmZ1bmN0aW9uIHNwYW4odGV4dCwgc3R5bGVBdHRyaWJ1dGVzID0ge30pIHsKICBjb25zdCBlbGVtZW50ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnc3BhbicpOwogIGVsZW1lbnQudGV4dENvbnRlbnQgPSB0ZXh0OwogIGZvciAoY29uc3Qga2V5IG9mIE9iamVjdC5rZXlzKHN0eWxlQXR0cmlidXRlcykpIHsKICAgIGVsZW1lbnQuc3R5bGVba2V5XSA9IHN0eWxlQXR0cmlidXRlc1trZXldOwogIH0KICByZXR1cm4gZWxlbWVudDsKfQoKLy8gTWF4IG51bWJlciBvZiBieXRlcyB3aGljaCB3aWxsIGJlIHVwbG9hZGVkIGF0IGEgdGltZS4KY29uc3QgTUFYX1BBWUxPQURfU0laRSA9IDEwMCAqIDEwMjQ7Ci8vIE1heCBhbW91bnQgb2YgdGltZSB0byBibG9jayB3YWl0aW5nIGZvciB0aGUgdXNlci4KY29uc3QgRklMRV9DSEFOR0VfVElNRU9VVF9NUyA9IDMwICogMTAwMDsKCmZ1bmN0aW9uIF91cGxvYWRGaWxlcyhpbnB1dElkLCBvdXRwdXRJZCkgewogIGNvbnN0IHN0ZXBzID0gdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKTsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIC8vIENhY2hlIHN0ZXBzIG9uIHRoZSBvdXRwdXRFbGVtZW50IHRvIG1ha2UgaXQgYXZhaWxhYmxlIGZvciB0aGUgbmV4dCBjYWxsCiAgLy8gdG8gdXBsb2FkRmlsZXNDb250aW51ZSBmcm9tIFB5dGhvbi4KICBvdXRwdXRFbGVtZW50LnN0ZXBzID0gc3RlcHM7CgogIHJldHVybiBfdXBsb2FkRmlsZXNDb250aW51ZShvdXRwdXRJZCk7Cn0KCi8vIFRoaXMgaXMgcm91Z2hseSBhbiBhc3luYyBnZW5lcmF0b3IgKG5vdCBzdXBwb3J0ZWQgaW4gdGhlIGJyb3dzZXIgeWV0KSwKLy8gd2hlcmUgdGhlcmUgYXJlIG11bHRpcGxlIGFzeW5jaHJvbm91cyBzdGVwcyBhbmQgdGhlIFB5dGhvbiBzaWRlIGlzIGdvaW5nCi8vIHRvIHBvbGwgZm9yIGNvbXBsZXRpb24gb2YgZWFjaCBzdGVwLgovLyBUaGlzIHVzZXMgYSBQcm9taXNlIHRvIGJsb2NrIHRoZSBweXRob24gc2lkZSBvbiBjb21wbGV0aW9uIG9mIGVhY2ggc3RlcCwKLy8gdGhlbiBwYXNzZXMgdGhlIHJlc3VsdCBvZiB0aGUgcHJldmlvdXMgc3RlcCBhcyB0aGUgaW5wdXQgdG8gdGhlIG5leHQgc3RlcC4KZnVuY3Rpb24gX3VwbG9hZEZpbGVzQ29udGludWUob3V0cHV0SWQpIHsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIGNvbnN0IHN0ZXBzID0gb3V0cHV0RWxlbWVudC5zdGVwczsKCiAgY29uc3QgbmV4dCA9IHN0ZXBzLm5leHQob3V0cHV0RWxlbWVudC5sYXN0UHJvbWlzZVZhbHVlKTsKICByZXR1cm4gUHJvbWlzZS5yZXNvbHZlKG5leHQudmFsdWUucHJvbWlzZSkudGhlbigodmFsdWUpID0+IHsKICAgIC8vIENhY2hlIHRoZSBsYXN0IHByb21pc2UgdmFsdWUgdG8gbWFrZSBpdCBhdmFpbGFibGUgdG8gdGhlIG5leHQKICAgIC8vIHN0ZXAgb2YgdGhlIGdlbmVyYXRvci4KICAgIG91dHB1dEVsZW1lbnQubGFzdFByb21pc2VWYWx1ZSA9IHZhbHVlOwogICAgcmV0dXJuIG5leHQudmFsdWUucmVzcG9uc2U7CiAgfSk7Cn0KCi8qKgogKiBHZW5lcmF0b3IgZnVuY3Rpb24gd2hpY2ggaXMgY2FsbGVkIGJldHdlZW4gZWFjaCBhc3luYyBzdGVwIG9mIHRoZSB1cGxvYWQKICogcHJvY2Vzcy4KICogQHBhcmFtIHtzdHJpbmd9IGlucHV0SWQgRWxlbWVudCBJRCBvZiB0aGUgaW5wdXQgZmlsZSBwaWNrZXIgZWxlbWVudC4KICogQHBhcmFtIHtzdHJpbmd9IG91dHB1dElkIEVsZW1lbnQgSUQgb2YgdGhlIG91dHB1dCBkaXNwbGF5LgogKiBAcmV0dXJuIHshSXRlcmFibGU8IU9iamVjdD59IEl0ZXJhYmxlIG9mIG5leHQgc3RlcHMuCiAqLwpmdW5jdGlvbiogdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKSB7CiAgY29uc3QgaW5wdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoaW5wdXRJZCk7CiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gZmFsc2U7CgogIGNvbnN0IG91dHB1dEVsZW1lbnQgPSBkb2N1bWVudC5nZXRFbGVtZW50QnlJZChvdXRwdXRJZCk7CiAgb3V0cHV0RWxlbWVudC5pbm5lckhUTUwgPSAnJzsKCiAgY29uc3QgcGlja2VkUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBpbnB1dEVsZW1lbnQuYWRkRXZlbnRMaXN0ZW5lcignY2hhbmdlJywgKGUpID0+IHsKICAgICAgcmVzb2x2ZShlLnRhcmdldC5maWxlcyk7CiAgICB9KTsKICB9KTsKCiAgY29uc3QgY2FuY2VsID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnYnV0dG9uJyk7CiAgaW5wdXRFbGVtZW50LnBhcmVudEVsZW1lbnQuYXBwZW5kQ2hpbGQoY2FuY2VsKTsKICBjYW5jZWwudGV4dENvbnRlbnQgPSAnQ2FuY2VsIHVwbG9hZCc7CiAgY29uc3QgY2FuY2VsUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBjYW5jZWwub25jbGljayA9ICgpID0+IHsKICAgICAgcmVzb2x2ZShudWxsKTsKICAgIH07CiAgfSk7CgogIC8vIENhbmNlbCB1cGxvYWQgaWYgdXNlciBoYXNuJ3QgcGlja2VkIGFueXRoaW5nIGluIHRpbWVvdXQuCiAgY29uc3QgdGltZW91dFByb21pc2UgPSBuZXcgUHJvbWlzZSgocmVzb2x2ZSkgPT4gewogICAgc2V0VGltZW91dCgoKSA9PiB7CiAgICAgIHJlc29sdmUobnVsbCk7CiAgICB9LCBGSUxFX0NIQU5HRV9USU1FT1VUX01TKTsKICB9KTsKCiAgLy8gV2FpdCBmb3IgdGhlIHVzZXIgdG8gcGljayB0aGUgZmlsZXMuCiAgY29uc3QgZmlsZXMgPSB5aWVsZCB7CiAgICBwcm9taXNlOiBQcm9taXNlLnJhY2UoW3BpY2tlZFByb21pc2UsIHRpbWVvdXRQcm9taXNlLCBjYW5jZWxQcm9taXNlXSksCiAgICByZXNwb25zZTogewogICAgICBhY3Rpb246ICdzdGFydGluZycsCiAgICB9CiAgfTsKCiAgaWYgKCFmaWxlcykgewogICAgcmV0dXJuIHsKICAgICAgcmVzcG9uc2U6IHsKICAgICAgICBhY3Rpb246ICdjb21wbGV0ZScsCiAgICAgIH0KICAgIH07CiAgfQoKICBjYW5jZWwucmVtb3ZlKCk7CgogIC8vIERpc2FibGUgdGhlIGlucHV0IGVsZW1lbnQgc2luY2UgZnVydGhlciBwaWNrcyBhcmUgbm90IGFsbG93ZWQuCiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gdHJ1ZTsKCiAgZm9yIChjb25zdCBmaWxlIG9mIGZpbGVzKSB7CiAgICBjb25zdCBsaSA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2xpJyk7CiAgICBsaS5hcHBlbmQoc3BhbihmaWxlLm5hbWUsIHtmb250V2VpZ2h0OiAnYm9sZCd9KSk7CiAgICBsaS5hcHBlbmQoc3BhbigKICAgICAgICBgKCR7ZmlsZS50eXBlIHx8ICduL2EnfSkgLSAke2ZpbGUuc2l6ZX0gYnl0ZXMsIGAgKwogICAgICAgIGBsYXN0IG1vZGlmaWVkOiAkewogICAgICAgICAgICBmaWxlLmxhc3RNb2RpZmllZERhdGUgPyBmaWxlLmxhc3RNb2RpZmllZERhdGUudG9Mb2NhbGVEYXRlU3RyaW5nKCkgOgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAnbi9hJ30gLSBgKSk7CiAgICBjb25zdCBwZXJjZW50ID0gc3BhbignMCUgZG9uZScpOwogICAgbGkuYXBwZW5kQ2hpbGQocGVyY2VudCk7CgogICAgb3V0cHV0RWxlbWVudC5hcHBlbmRDaGlsZChsaSk7CgogICAgY29uc3QgZmlsZURhdGFQcm9taXNlID0gbmV3IFByb21pc2UoKHJlc29sdmUpID0+IHsKICAgICAgY29uc3QgcmVhZGVyID0gbmV3IEZpbGVSZWFkZXIoKTsKICAgICAgcmVhZGVyLm9ubG9hZCA9IChlKSA9PiB7CiAgICAgICAgcmVzb2x2ZShlLnRhcmdldC5yZXN1bHQpOwogICAgICB9OwogICAgICByZWFkZXIucmVhZEFzQXJyYXlCdWZmZXIoZmlsZSk7CiAgICB9KTsKICAgIC8vIFdhaXQgZm9yIHRoZSBkYXRhIHRvIGJlIHJlYWR5LgogICAgbGV0IGZpbGVEYXRhID0geWllbGQgewogICAgICBwcm9taXNlOiBmaWxlRGF0YVByb21pc2UsCiAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgYWN0aW9uOiAnY29udGludWUnLAogICAgICB9CiAgICB9OwoKICAgIC8vIFVzZSBhIGNodW5rZWQgc2VuZGluZyB0byBhdm9pZCBtZXNzYWdlIHNpemUgbGltaXRzLiBTZWUgYi82MjExNTY2MC4KICAgIGxldCBwb3NpdGlvbiA9IDA7CiAgICB3aGlsZSAocG9zaXRpb24gPCBmaWxlRGF0YS5ieXRlTGVuZ3RoKSB7CiAgICAgIGNvbnN0IGxlbmd0aCA9IE1hdGgubWluKGZpbGVEYXRhLmJ5dGVMZW5ndGggLSBwb3NpdGlvbiwgTUFYX1BBWUxPQURfU0laRSk7CiAgICAgIGNvbnN0IGNodW5rID0gbmV3IFVpbnQ4QXJyYXkoZmlsZURhdGEsIHBvc2l0aW9uLCBsZW5ndGgpOwogICAgICBwb3NpdGlvbiArPSBsZW5ndGg7CgogICAgICBjb25zdCBiYXNlNjQgPSBidG9hKFN0cmluZy5mcm9tQ2hhckNvZGUuYXBwbHkobnVsbCwgY2h1bmspKTsKICAgICAgeWllbGQgewogICAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgICBhY3Rpb246ICdhcHBlbmQnLAogICAgICAgICAgZmlsZTogZmlsZS5uYW1lLAogICAgICAgICAgZGF0YTogYmFzZTY0LAogICAgICAgIH0sCiAgICAgIH07CiAgICAgIHBlcmNlbnQudGV4dENvbnRlbnQgPQogICAgICAgICAgYCR7TWF0aC5yb3VuZCgocG9zaXRpb24gLyBmaWxlRGF0YS5ieXRlTGVuZ3RoKSAqIDEwMCl9JSBkb25lYDsKICAgIH0KICB9CgogIC8vIEFsbCBkb25lLgogIHlpZWxkIHsKICAgIHJlc3BvbnNlOiB7CiAgICAgIGFjdGlvbjogJ2NvbXBsZXRlJywKICAgIH0KICB9Owp9CgpzY29wZS5nb29nbGUgPSBzY29wZS5nb29nbGUgfHwge307CnNjb3BlLmdvb2dsZS5jb2xhYiA9IHNjb3BlLmdvb2dsZS5jb2xhYiB8fCB7fTsKc2NvcGUuZ29vZ2xlLmNvbGFiLl9maWxlcyA9IHsKICBfdXBsb2FkRmlsZXMsCiAgX3VwbG9hZEZpbGVzQ29udGludWUsCn07Cn0pKHNlbGYpOwo=",
       "headers": [
        [
         "content-type",
         "application/javascript"
        ]
       ],
       "ok": true,
       "status": 200,
       "status_text": ""
      }
     }
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 10427,
     "status": "ok",
     "timestamp": 1539440943692,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "9WtCXg_0aHaA",
    "outputId": "c315dc41-1a03-4891-9545-e11911443a99"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "     <input type=\"file\" id=\"files-cc921683-dfcb-4a59-83a1-e170150dde0d\" name=\"files[]\" multiple disabled />\n",
       "     <output id=\"result-cc921683-dfcb-4a59-83a1-e170150dde0d\">\n",
       "      Upload widget is only available when the cell has been executed in the\n",
       "      current browser session. Please rerun this cell to enable.\n",
       "      </output>\n",
       "      <script src=\"/nbextensions/google.colab/files.js\"></script> "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving kaggle.json to kaggle.json\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'kaggle.json': b'{\"username\":\"mankitpong\",\"key\":\"d34f86cca67e2fa0f21031dbf47aa780\"}'}"
      ]
     },
     "execution_count": 4,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Upload our Kaggle API token\n",
    "from google.colab import files\n",
    "files.upload()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 5249,
     "status": "ok",
     "timestamp": 1539440951433,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "2z1Q0Mnpc7YE",
    "outputId": "ebaecb05-bf08-4b51-c402-1ead9a0771b0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kaggle.json  sample_data\n"
     ]
    }
   ],
   "source": [
    "# Make sure we have uploaded the file successfully\n",
    "!ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "s8kUvUsFdHK2"
   },
   "outputs": [],
   "source": [
    "# The Kaggle API client expects this file to be in ~/.kaggle,\n",
    "# so lets move it there.\n",
    "!mkdir -p ~/.kaggle\n",
    "!cp kaggle.json ~/.kaggle/\n",
    "\n",
    "# This permissions change avoids a warning on Kaggle tool startup.\n",
    "!chmod 600 ~/.kaggle/kaggle.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 269
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 14397,
     "status": "ok",
     "timestamp": 1539440984343,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "2GRpe9mGda9y",
    "outputId": "240d8c92-e297-4acc-c09f-78e3a2044f1d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading depths.csv to /content\n",
      "\r",
      "  0% 0.00/322k [00:00<?, ?B/s]\n",
      "100% 322k/322k [00:00<00:00, 64.8MB/s]\n",
      "Downloading sample_submission.csv to /content\n",
      "  0% 0.00/264k [00:00<?, ?B/s]\n",
      "100% 264k/264k [00:00<00:00, 86.3MB/s]\n",
      "Downloading train.csv to /content\n",
      "  0% 0.00/922k [00:00<?, ?B/s]\n",
      "100% 922k/922k [00:00<00:00, 123MB/s]\n",
      "Downloading test.zip to /content\n",
      " 99% 161M/163M [00:04<00:00, 58.5MB/s]\n",
      "100% 163M/163M [00:04<00:00, 41.5MB/s]\n",
      "Downloading train.zip to /content\n",
      " 87% 33.0M/37.9M [00:00<00:00, 15.7MB/s]\n",
      "100% 37.9M/37.9M [00:00<00:00, 48.1MB/s]\n"
     ]
    }
   ],
   "source": [
    "# Let's now download our dataset\n",
    "!kaggle competitions download -c tgs-salt-identification-challenge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 50
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 19800,
     "status": "ok",
     "timestamp": 1539441005424,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "InnJQ1W9eT7s",
    "outputId": "c2974c65-847e-47a8-c096-86022eae9979"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "depths.csv   sample_data\t    test      train\t train.zip\n",
      "kaggle.json  sample_submission.csv  test.zip  train.csv\n"
     ]
    }
   ],
   "source": [
    "# And we'll need those training images unzipped\n",
    "!mkdir train\n",
    "!mkdir test\n",
    "!unzip -q train.zip -d train\n",
    "!unzip -q test.zip -d test\n",
    "!ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7jKIdlH3gaFU"
   },
   "outputs": [],
   "source": [
    "# Helper functions for data processing\n",
    "img_size_ori = 101\n",
    "img_size_target = 128\n",
    "\n",
    "def upsample(img):\n",
    "    if img_size_ori == img_size_target:\n",
    "        return img\n",
    "    return resize(img, (img_size_target, img_size_target), mode='constant', preserve_range=True)\n",
    "\n",
    "\n",
    "def downsample(img):\n",
    "    if img_size_ori == img_size_target:\n",
    "        return img\n",
    "    return resize(img, (img_size_ori, img_size_ori), mode='constant', preserve_range=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GKgpMbL3gbp_"
   },
   "outputs": [],
   "source": [
    "# Data preprocessing\n",
    "train_df = pd.read_csv(\"./train.csv\", index_col=\"id\", usecols=[0])\n",
    "depths_df = pd.read_csv(\"./depths.csv\", index_col=\"id\")\n",
    "train_df = train_df.join(depths_df)\n",
    "test_df = depths_df[~depths_df.index.isin(train_df.index)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 4092,
     "status": "ok",
     "timestamp": 1539441029024,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "h5Rs074Hi9NB",
    "outputId": "f904e863-36b0-4a96-ab09-a19e34cec0ea"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 4000/4000 [00:03<00:00, 1279.68it/s]\n"
     ]
    }
   ],
   "source": [
    "train_df[\"images\"] = [np.array(load_img(\"./train/images/{}.png\".format(idx), color_mode = \"grayscale\")) / 255 for idx in tqdm(train_df.index)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2437,
     "status": "ok",
     "timestamp": 1539441033149,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "NSCh6OUHkbm5",
    "outputId": "1233a6cc-b8ff-4386-bf23-ea6b96d902d7"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 4000/4000 [00:01<00:00, 2432.68it/s]\n"
     ]
    }
   ],
   "source": [
    "train_df[\"masks\"] = [np.array(load_img(\"./train/masks/{}.png\".format(idx), color_mode = \"grayscale\")) / 255 for idx in tqdm(train_df.index)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JyLfDfQMknvS"
   },
   "outputs": [],
   "source": [
    "train_df[\"coverage\"] = train_df.masks.map(np.sum) / pow(img_size_ori, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bKSfmStTlG4v"
   },
   "outputs": [],
   "source": [
    "def cov_to_class(val):    \n",
    "    for i in range(0, 11):\n",
    "        if val * 10 <= i :\n",
    "            return i\n",
    "        \n",
    "train_df[\"coverage_class\"] = train_df.coverage.map(cov_to_class)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4bUsdtjWlKr0"
   },
   "outputs": [],
   "source": [
    "# Split data into train and test sets\n",
    "ids_train, ids_valid, x_train, x_valid, y_train, y_valid, cov_train, cov_test, depth_train, depth_test = train_test_split(\n",
    "    train_df.index.values,\n",
    "    np.array(train_df.images.map(upsample).tolist()).reshape(-1, img_size_target, img_size_target, 1), \n",
    "    np.array(train_df.masks.map(upsample).tolist()).reshape(-1, img_size_target, img_size_target, 1), \n",
    "    train_df.coverage.values,\n",
    "    train_df.z.values,\n",
    "    test_size=0.2, stratify=train_df.coverage_class, random_state=1337)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ojwNsiAcJMA3"
   },
   "outputs": [],
   "source": [
    "# Data augmentation by flipping the image\n",
    "x_train = np.append(x_train, [np.fliplr(x) for x in x_train], axis=0)\n",
    "y_train = np.append(y_train, [np.fliplr(x) for x in y_train], axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "n3il94PclT_z"
   },
   "outputs": [],
   "source": [
    "# Buid the U-net model\n",
    "def build_model(input_layer, start_neurons):\n",
    "    # 128 -> 64\n",
    "    conv1 = Conv2D(start_neurons * 1, (3, 3), activation=\"relu\", padding=\"same\")(input_layer)\n",
    "    conv1 = Conv2D(start_neurons * 1, (3, 3), activation=\"relu\", padding=\"same\")(conv1)\n",
    "    pool1 = MaxPooling2D((2, 2))(conv1)\n",
    "    pool1 = Dropout(0.25)(pool1)\n",
    "\n",
    "    # 64 -> 32\n",
    "    conv2 = Conv2D(start_neurons * 2, (3, 3), activation=\"relu\", padding=\"same\")(pool1)\n",
    "    conv2 = Conv2D(start_neurons * 2, (3, 3), activation=\"relu\", padding=\"same\")(conv2)\n",
    "    pool2 = MaxPooling2D((2, 2))(conv2)\n",
    "    pool2 = Dropout(0.5)(pool2)\n",
    "\n",
    "    # 32 -> 16\n",
    "    conv3 = Conv2D(start_neurons * 4, (3, 3), activation=\"relu\", padding=\"same\")(pool2)\n",
    "    conv3 = Conv2D(start_neurons * 4, (3, 3), activation=\"relu\", padding=\"same\")(conv3)\n",
    "    pool3 = MaxPooling2D((2, 2))(conv3)\n",
    "    pool3 = Dropout(0.5)(pool3)\n",
    "\n",
    "    # 16 -> 8\n",
    "    conv4 = Conv2D(start_neurons * 8, (3, 3), activation=\"relu\", padding=\"same\")(pool3)\n",
    "    conv4 = Conv2D(start_neurons * 8, (3, 3), activation=\"relu\", padding=\"same\")(conv4)\n",
    "    pool4 = MaxPooling2D((2, 2))(conv4)\n",
    "    pool4 = Dropout(0.5)(pool4)\n",
    "\n",
    "    # Middle\n",
    "    convm = Conv2D(start_neurons * 16, (3, 3), activation=\"relu\", padding=\"same\")(pool4)\n",
    "    convm = Conv2D(start_neurons * 16, (3, 3), activation=\"relu\", padding=\"same\")(convm)\n",
    "\n",
    "    # 8 -> 16\n",
    "    deconv4 = Conv2DTranspose(start_neurons * 8, (3, 3), strides=(2, 2), padding=\"same\")(convm)\n",
    "    uconv4 = concatenate([deconv4, conv4])\n",
    "    uconv4 = Dropout(0.5)(uconv4)\n",
    "    uconv4 = Conv2D(start_neurons * 8, (3, 3), activation=\"relu\", padding=\"same\")(uconv4)\n",
    "    uconv4 = Conv2D(start_neurons * 8, (3, 3), activation=\"relu\", padding=\"same\")(uconv4)\n",
    "\n",
    "    # 16 -> 32\n",
    "    deconv3 = Conv2DTranspose(start_neurons * 4, (3, 3), strides=(2, 2), padding=\"same\")(uconv4)\n",
    "    uconv3 = concatenate([deconv3, conv3])\n",
    "    uconv3 = Dropout(0.5)(uconv3)\n",
    "    uconv3 = Conv2D(start_neurons * 4, (3, 3), activation=\"relu\", padding=\"same\")(uconv3)\n",
    "    uconv3 = Conv2D(start_neurons * 4, (3, 3), activation=\"relu\", padding=\"same\")(uconv3)\n",
    "\n",
    "    # 32 -> 64\n",
    "    deconv2 = Conv2DTranspose(start_neurons * 2, (3, 3), strides=(2, 2), padding=\"same\")(uconv3)\n",
    "    uconv2 = concatenate([deconv2, conv2])\n",
    "    uconv2 = Dropout(0.5)(uconv2)\n",
    "    uconv2 = Conv2D(start_neurons * 2, (3, 3), activation=\"relu\", padding=\"same\")(uconv2)\n",
    "    uconv2 = Conv2D(start_neurons * 2, (3, 3), activation=\"relu\", padding=\"same\")(uconv2)\n",
    "\n",
    "    # 64 -> 128\n",
    "    deconv1 = Conv2DTranspose(start_neurons * 1, (3, 3), strides=(2, 2), padding=\"same\")(uconv2)\n",
    "    uconv1 = concatenate([deconv1, conv1])\n",
    "    uconv1 = Dropout(0.5)(uconv1)\n",
    "    uconv1 = Conv2D(start_neurons * 1, (3, 3), activation=\"relu\", padding=\"same\")(uconv1)\n",
    "    uconv1 = Conv2D(start_neurons * 1, (3, 3), activation=\"relu\", padding=\"same\")(uconv1)\n",
    "\n",
    "    #uconv1 = Dropout(0.5)(uconv1)\n",
    "    output_layer = Conv2D(1, (1,1), padding=\"same\", activation=\"sigmoid\")(uconv1)\n",
    "    \n",
    "    return output_layer\n",
    "\n",
    "input_layer = Input((img_size_target, img_size_target, 1))\n",
    "output_layer = build_model(input_layer, 16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "qQ_qgtrml0E7"
   },
   "outputs": [],
   "source": [
    "model = Model(input_layer, output_layer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 356
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 16196,
     "status": "ok",
     "timestamp": 1539441092978,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "jAKDMuwRmCLh",
    "outputId": "65df2985-273a-4423-c7c6-a7d89dc99666"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Querying Tensorflow master (b'grpc://10.17.106.138:8470') for TPU system metadata.\n",
      "INFO:tensorflow:Found TPU system:\n",
      "INFO:tensorflow:*** Num TPU Cores: 8\n",
      "INFO:tensorflow:*** Num TPU Workers: 1\n",
      "INFO:tensorflow:*** Num TPU Cores Per Worker: 8\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, -1, 3788293867414123363)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 14218655969421810422)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_GPU:0, XLA_GPU, 17179869184, 5812111958003439941)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 17179869184, 8454891999339572241)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 17179869184, 15413725777721296706)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 17179869184, 1794556755405902195)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 17179869184, 8722551663338303265)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 17179869184, 10265242285712045992)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 17179869184, 83959706521883974)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 17179869184, 6395746279842848549)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 17179869184, 17317425135332706070)\n",
      "INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 17179869184, 13109444320199115483)\n",
      "WARNING:tensorflow:tpu_model (from tensorflow.contrib.tpu.python.tpu.keras_support) is experimental and may change or be removed at any time, and without warning.\n",
      "INFO:tensorflow:Connecting to: b'grpc://10.17.106.138:8470'\n"
     ]
    }
   ],
   "source": [
    "# Transfer the model on TPU\n",
    "tpu_model = tf.contrib.tpu.keras_to_tpu_model(\n",
    "    model,\n",
    "    strategy=tf.contrib.tpu.TPUDistributionStrategy(\n",
    "        tf.contrib.cluster_resolver.TPUClusterResolver(tpu=TPU_ADDRESS)\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pAHeniYCmhuP"
   },
   "outputs": [],
   "source": [
    "tpu_model.compile(loss=\"binary_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1546
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1388,
     "status": "ok",
     "timestamp": 1539441104346,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "wHi9-1jympz9",
    "outputId": "51a85e2a-de7f-4cd6-de95-bc52fc78751f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_1 (InputLayer)            (None, 128, 128, 1)  0                                            \n",
      "__________________________________________________________________________________________________\n",
      "conv2d (Conv2D)                 (None, 128, 128, 16) 160         input_1[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_1 (Conv2D)               (None, 128, 128, 16) 2320        conv2d[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "max_pooling2d (MaxPooling2D)    (None, 64, 64, 16)   0           conv2d_1[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "dropout (Dropout)               (None, 64, 64, 16)   0           max_pooling2d[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_2 (Conv2D)               (None, 64, 64, 32)   4640        dropout[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_3 (Conv2D)               (None, 64, 64, 32)   9248        conv2d_2[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2D)  (None, 32, 32, 32)   0           conv2d_3[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "dropout_1 (Dropout)             (None, 32, 32, 32)   0           max_pooling2d_1[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_4 (Conv2D)               (None, 32, 32, 64)   18496       dropout_1[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_5 (Conv2D)               (None, 32, 32, 64)   36928       conv2d_4[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2D)  (None, 16, 16, 64)   0           conv2d_5[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "dropout_2 (Dropout)             (None, 16, 16, 64)   0           max_pooling2d_2[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_6 (Conv2D)               (None, 16, 16, 128)  73856       dropout_2[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_7 (Conv2D)               (None, 16, 16, 128)  147584      conv2d_6[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "max_pooling2d_3 (MaxPooling2D)  (None, 8, 8, 128)    0           conv2d_7[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "dropout_3 (Dropout)             (None, 8, 8, 128)    0           max_pooling2d_3[0][0]            \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_8 (Conv2D)               (None, 8, 8, 256)    295168      dropout_3[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_9 (Conv2D)               (None, 8, 8, 256)    590080      conv2d_8[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_transpose (Conv2DTranspo (None, 16, 16, 128)  295040      conv2d_9[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "concatenate (Concatenate)       (None, 16, 16, 256)  0           conv2d_transpose[0][0]           \n",
      "                                                                 conv2d_7[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "dropout_4 (Dropout)             (None, 16, 16, 256)  0           concatenate[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_10 (Conv2D)              (None, 16, 16, 128)  295040      dropout_4[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_11 (Conv2D)              (None, 16, 16, 128)  147584      conv2d_10[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_transpose_1 (Conv2DTrans (None, 32, 32, 64)   73792       conv2d_11[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_1 (Concatenate)     (None, 32, 32, 128)  0           conv2d_transpose_1[0][0]         \n",
      "                                                                 conv2d_5[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "dropout_5 (Dropout)             (None, 32, 32, 128)  0           concatenate_1[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_12 (Conv2D)              (None, 32, 32, 64)   73792       dropout_5[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_13 (Conv2D)              (None, 32, 32, 64)   36928       conv2d_12[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_transpose_2 (Conv2DTrans (None, 64, 64, 32)   18464       conv2d_13[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_2 (Concatenate)     (None, 64, 64, 64)   0           conv2d_transpose_2[0][0]         \n",
      "                                                                 conv2d_3[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "dropout_6 (Dropout)             (None, 64, 64, 64)   0           concatenate_2[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_14 (Conv2D)              (None, 64, 64, 32)   18464       dropout_6[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_15 (Conv2D)              (None, 64, 64, 32)   9248        conv2d_14[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_transpose_3 (Conv2DTrans (None, 128, 128, 16) 4624        conv2d_15[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_3 (Concatenate)     (None, 128, 128, 32) 0           conv2d_transpose_3[0][0]         \n",
      "                                                                 conv2d_1[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "dropout_7 (Dropout)             (None, 128, 128, 32) 0           concatenate_3[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_16 (Conv2D)              (None, 128, 128, 16) 4624        dropout_7[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_17 (Conv2D)              (None, 128, 128, 16) 2320        conv2d_16[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_18 (Conv2D)              (None, 128, 128, 1)  17          conv2d_17[0][0]                  \n",
      "==================================================================================================\n",
      "Total params: 2,158,417\n",
      "Trainable params: 2,158,417\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "tpu_model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 2187
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 559288,
     "status": "ok",
     "timestamp": 1539441667933,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "YbyhLPndm6fv",
    "outputId": "ca2af071-9362-42c8-bd88-f9ef94708009"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 6400 samples, validate on 800 samples\n",
      "Epoch 1/200\n",
      "INFO:tensorflow:New input shapes; (re-)compiling: mode=train, [TensorSpec(shape=(4, 128, 128, 1), dtype=tf.float32, name='input_10'), TensorSpec(shape=(4, 128, 128, 1), dtype=tf.float32, name='conv2d_18_target0')]\n",
      "INFO:tensorflow:Overriding default placeholder.\n",
      "INFO:tensorflow:Remapping placeholder for input_1\n",
      "INFO:tensorflow:Cloning Adam {'lr': 0.0010000000474974513, 'beta_1': 0.8999999761581421, 'beta_2': 0.9990000128746033, 'decay': 0.0, 'epsilon': 1e-07, 'amsgrad': False}\n",
      "INFO:tensorflow:Get updates: Tensor(\"loss/mul:0\", shape=(), dtype=float32)\n",
      "INFO:tensorflow:Started compiling\n",
      "INFO:tensorflow:Finished compiling. Time elapsed: 7.537999868392944 secs\n",
      "INFO:tensorflow:Setting weights on TPU model.\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.5485 - acc: 0.7390INFO:tensorflow:New input shapes; (re-)compiling: mode=eval, [TensorSpec(shape=(4, 128, 128, 1), dtype=tf.float32, name='input_10'), TensorSpec(shape=(4, 128, 128, 1), dtype=tf.float32, name='conv2d_18_target0')]\n",
      "INFO:tensorflow:Overriding default placeholder.\n",
      "INFO:tensorflow:Remapping placeholder for input_1\n",
      "INFO:tensorflow:Cloning Adam {'lr': 0.0010000000474974513, 'beta_1': 0.8999999761581421, 'beta_2': 0.9990000128746033, 'decay': 0.0, 'epsilon': 1e-07, 'amsgrad': False}\n",
      "INFO:tensorflow:Started compiling\n",
      "INFO:tensorflow:Finished compiling. Time elapsed: 3.2120699882507324 secs\n",
      "\n",
      "Epoch 00001: val_loss improved from inf to 0.33374, saving model to ./keras.model\n",
      "INFO:tensorflow:Copying TPU weights to the CPU\n",
      "6400/6400 [==============================] - 43s 7ms/step - loss: 0.5468 - acc: 0.7401 - val_loss: 0.3337 - val_acc: 0.8716\n",
      "Epoch 2/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.3920 - acc: 0.8411\n",
      "Epoch 00002: val_loss improved from 0.33374 to 0.30989, saving model to ./keras.model\n",
      "INFO:tensorflow:Copying TPU weights to the CPU\n",
      "6400/6400 [==============================] - 22s 3ms/step - loss: 0.3917 - acc: 0.8413 - val_loss: 0.3099 - val_acc: 0.8941\n",
      "Epoch 3/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.3225 - acc: 0.8678\n",
      "Epoch 00003: val_loss improved from 0.30989 to 0.26749, saving model to ./keras.model\n",
      "INFO:tensorflow:Copying TPU weights to the CPU\n",
      "6400/6400 [==============================] - 22s 3ms/step - loss: 0.3224 - acc: 0.8677 - val_loss: 0.2675 - val_acc: 0.8942\n",
      "Epoch 4/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.2898 - acc: 0.8875\n",
      "Epoch 00004: val_loss improved from 0.26749 to 0.21080, saving model to ./keras.model\n",
      "INFO:tensorflow:Copying TPU weights to the CPU\n",
      "6400/6400 [==============================] - 22s 3ms/step - loss: 0.2902 - acc: 0.8872 - val_loss: 0.2108 - val_acc: 0.9214\n",
      "Epoch 5/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.3034 - acc: 0.8792\n",
      "Epoch 00005: val_loss did not improve from 0.21080\n",
      "6400/6400 [==============================] - 22s 3ms/step - loss: 0.3024 - acc: 0.8796 - val_loss: 0.2376 - val_acc: 0.9111\n",
      "Epoch 6/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.2773 - acc: 0.8896\n",
      "Epoch 00006: val_loss did not improve from 0.21080\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.2767 - acc: 0.8898 - val_loss: 0.2142 - val_acc: 0.9159\n",
      "Epoch 7/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.2599 - acc: 0.8938\n",
      "Epoch 00007: val_loss did not improve from 0.21080\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.2592 - acc: 0.8942 - val_loss: 0.2256 - val_acc: 0.9097\n",
      "Epoch 8/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.2590 - acc: 0.8936\n",
      "Epoch 00008: val_loss improved from 0.21080 to 0.19760, saving model to ./keras.model\n",
      "INFO:tensorflow:Copying TPU weights to the CPU\n",
      "6400/6400 [==============================] - 22s 3ms/step - loss: 0.2582 - acc: 0.8940 - val_loss: 0.1976 - val_acc: 0.9237\n",
      "Epoch 9/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.2746 - acc: 0.8913\n",
      "Epoch 00009: val_loss improved from 0.19760 to 0.19404, saving model to ./keras.model\n",
      "INFO:tensorflow:Copying TPU weights to the CPU\n",
      "6400/6400 [==============================] - 22s 3ms/step - loss: 0.2737 - acc: 0.8918 - val_loss: 0.1940 - val_acc: 0.9213\n",
      "Epoch 10/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.2292 - acc: 0.9043\n",
      "Epoch 00010: val_loss did not improve from 0.19404\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.2311 - acc: 0.9035 - val_loss: 0.1961 - val_acc: 0.9221\n",
      "Epoch 11/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.2241 - acc: 0.9062\n",
      "Epoch 00011: val_loss did not improve from 0.19404\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.2235 - acc: 0.9065 - val_loss: 0.2027 - val_acc: 0.9217\n",
      "Epoch 12/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.2360 - acc: 0.9043\n",
      "Epoch 00012: val_loss did not improve from 0.19404\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.2352 - acc: 0.9047 - val_loss: 0.1975 - val_acc: 0.9253\n",
      "Epoch 13/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1879 - acc: 0.9198\n",
      "Epoch 00013: val_loss improved from 0.19404 to 0.18777, saving model to ./keras.model\n",
      "INFO:tensorflow:Copying TPU weights to the CPU\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.1877 - acc: 0.9199 - val_loss: 0.1878 - val_acc: 0.9148\n",
      "Epoch 14/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.2241 - acc: 0.9078\n",
      "Epoch 00014: val_loss did not improve from 0.18777\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.2239 - acc: 0.9079 - val_loss: 0.2074 - val_acc: 0.9133\n",
      "Epoch 15/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.2095 - acc: 0.9106\n",
      "Epoch 00015: val_loss improved from 0.18777 to 0.17698, saving model to ./keras.model\n",
      "INFO:tensorflow:Copying TPU weights to the CPU\n",
      "6400/6400 [==============================] - 22s 3ms/step - loss: 0.2097 - acc: 0.9102 - val_loss: 0.1770 - val_acc: 0.9281\n",
      "Epoch 16/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1691 - acc: 0.9298\n",
      "Epoch 00016: val_loss did not improve from 0.17698\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.1700 - acc: 0.9294 - val_loss: 0.2004 - val_acc: 0.9269\n",
      "Epoch 17/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1820 - acc: 0.9187\n",
      "Epoch 00017: val_loss did not improve from 0.17698\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.1826 - acc: 0.9182 - val_loss: 0.2574 - val_acc: 0.9050\n",
      "Epoch 18/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1746 - acc: 0.9265\n",
      "Epoch 00018: val_loss did not improve from 0.17698\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.1743 - acc: 0.9268 - val_loss: 0.1899 - val_acc: 0.9266\n",
      "Epoch 19/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1607 - acc: 0.9311\n",
      "Epoch 00019: val_loss did not improve from 0.17698\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.1607 - acc: 0.9310 - val_loss: 0.1971 - val_acc: 0.9272\n",
      "Epoch 20/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1552 - acc: 0.9309\n",
      "Epoch 00020: val_loss did not improve from 0.17698\n",
      "\n",
      "Epoch 00020: ReduceLROnPlateau reducing learning rate to 0.00010000000474974513.\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.1567 - acc: 0.9302 - val_loss: 0.2123 - val_acc: 0.9206\n",
      "Epoch 21/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1678 - acc: 0.9277\n",
      "Epoch 00021: val_loss did not improve from 0.17698\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.1676 - acc: 0.9278 - val_loss: 0.2959 - val_acc: 0.8977\n",
      "Epoch 22/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1617 - acc: 0.9318\n",
      "Epoch 00022: val_loss did not improve from 0.17698\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.1624 - acc: 0.9312 - val_loss: 0.2351 - val_acc: 0.9191\n",
      "Epoch 23/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1723 - acc: 0.9230\n",
      "Epoch 00023: val_loss did not improve from 0.17698\n",
      "6400/6400 [==============================] - 22s 3ms/step - loss: 0.1724 - acc: 0.9230 - val_loss: 0.2549 - val_acc: 0.8764\n",
      "Epoch 24/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1533 - acc: 0.9334\n",
      "Epoch 00024: val_loss did not improve from 0.17698\n",
      "6400/6400 [==============================] - 22s 3ms/step - loss: 0.1530 - acc: 0.9336 - val_loss: 0.2476 - val_acc: 0.9091\n",
      "Epoch 25/200\n",
      "6368/6400 [============================>.] - ETA: 0s - loss: 0.1519 - acc: 0.9350\n",
      "Epoch 00025: val_loss did not improve from 0.17698\n",
      "\n",
      "Epoch 00025: ReduceLROnPlateau reducing learning rate to 1.0000000474974514e-05.\n",
      "6400/6400 [==============================] - 21s 3ms/step - loss: 0.1515 - acc: 0.9352 - val_loss: 0.2582 - val_acc: 0.9153\n",
      "Epoch 00025: early stopping\n"
     ]
    }
   ],
   "source": [
    "# Start our training\n",
    "early_stopping = EarlyStopping(patience=10, verbose=1)\n",
    "model_checkpoint = ModelCheckpoint(\"./keras.model\", save_best_only=True, verbose=1)\n",
    "reduce_lr = ReduceLROnPlateau(factor=0.1, patience=5, min_lr=0.00001, verbose=1)\n",
    "\n",
    "epochs = 200\n",
    "batch_size = 32\n",
    "\n",
    "history = tpu_model.fit(x_train, y_train,\n",
    "                        validation_data=[x_valid, y_valid], \n",
    "                        epochs=epochs,\n",
    "                        batch_size=batch_size,\n",
    "                        callbacks=[early_stopping, model_checkpoint, reduce_lr])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "46MNAtBiu9_4"
   },
   "outputs": [],
   "source": [
    "model = load_model(\"./keras.model\")\n",
    "preds_valid = model.predict(x_valid).reshape(-1, img_size_target, img_size_target)\n",
    "preds_valid = np.array([downsample(x) for x in preds_valid])\n",
    "y_valid_ori = np.array([train_df.loc[idx].masks for idx in ids_valid])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "fO5GhT5wyyhl"
   },
   "outputs": [],
   "source": [
    "# src: https://www.kaggle.com/aglotero/another-iou-metric\n",
    "def iou_metric(y_true_in, y_pred_in, print_table=False):\n",
    "    labels = y_true_in\n",
    "    y_pred = y_pred_in\n",
    "    \n",
    "    true_objects = 2\n",
    "    pred_objects = 2\n",
    "\n",
    "    intersection = np.histogram2d(labels.flatten(), y_pred.flatten(), bins=(true_objects, pred_objects))[0]\n",
    "\n",
    "    # Compute areas (needed for finding the union between all objects)\n",
    "    area_true = np.histogram(labels, bins = true_objects)[0]\n",
    "    area_pred = np.histogram(y_pred, bins = pred_objects)[0]\n",
    "    area_true = np.expand_dims(area_true, -1)\n",
    "    area_pred = np.expand_dims(area_pred, 0)\n",
    "\n",
    "    # Compute union\n",
    "    union = area_true + area_pred - intersection\n",
    "\n",
    "    # Exclude background from the analysis\n",
    "    intersection = intersection[1:,1:]\n",
    "    union = union[1:,1:]\n",
    "    union[union == 0] = 1e-9\n",
    "\n",
    "    # Compute the intersection over union\n",
    "    iou = intersection / union\n",
    "\n",
    "    # Precision helper function\n",
    "    def precision_at(threshold, iou):\n",
    "        matches = iou > threshold\n",
    "        true_positives = np.sum(matches, axis=1) == 1   # Correct objects\n",
    "        false_positives = np.sum(matches, axis=0) == 0  # Missed objects\n",
    "        false_negatives = np.sum(matches, axis=1) == 0  # Extra objects\n",
    "        tp, fp, fn = np.sum(true_positives), np.sum(false_positives), np.sum(false_negatives)\n",
    "        return tp, fp, fn\n",
    "\n",
    "    # Loop over IoU thresholds\n",
    "    prec = []\n",
    "    if print_table:\n",
    "        print(\"Thresh\\tTP\\tFP\\tFN\\tPrec.\")\n",
    "    for t in np.arange(0.5, 1.0, 0.05):\n",
    "        tp, fp, fn = precision_at(t, iou)\n",
    "        if (tp + fp + fn) > 0:\n",
    "            p = tp / (tp + fp + fn)\n",
    "        else:\n",
    "            p = 0\n",
    "        if print_table:\n",
    "            print(\"{:1.3f}\\t{}\\t{}\\t{}\\t{:1.3f}\".format(t, tp, fp, fn, p))\n",
    "        prec.append(p)\n",
    "    \n",
    "    if print_table:\n",
    "        print(\"AP\\t-\\t-\\t-\\t{:1.3f}\".format(np.mean(prec)))\n",
    "    return np.mean(prec)\n",
    "\n",
    "def iou_metric_batch(y_true_in, y_pred_in):\n",
    "    batch_size = y_true_in.shape[0]\n",
    "    metric = []\n",
    "    for batch in range(batch_size):\n",
    "        value = iou_metric(y_true_in[batch], y_pred_in[batch])\n",
    "        metric.append(value)\n",
    "    return np.mean(metric)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 65142,
     "status": "ok",
     "timestamp": 1539441819142,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "R6uU9vIWzjcx",
    "outputId": "2c3e07f0-2e7f-473a-a558-e9958616ac70"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 50/50 [01:04<00:00,  1.27s/it]\n"
     ]
    }
   ],
   "source": [
    "# Find the best threshold\n",
    "thresholds = np.linspace(0, 1, 50)\n",
    "ious = np.array([iou_metric_batch(y_valid_ori, np.int32(preds_valid > threshold)) for threshold in tqdm(thresholds)])\n",
    "threshold_best_index = np.argmax(ious[9:-10]) + 9\n",
    "iou_best = ious[threshold_best_index]\n",
    "threshold_best = thresholds[threshold_best_index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 310
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2019,
     "status": "ok",
     "timestamp": 1539441822518,
     "user": {
      "displayName": "ManKit Pong",
      "photoUrl": "",
      "userId": "08505139514039014912"
     },
     "user_tz": -480
    },
    "id": "PAyw5PCszsPs",
    "outputId": "f50c1eb9-9323-4944-d0ed-f5b19aa373a2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f80be3b2320>"
      ]
     },
     "execution_count": 27,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEVCAYAAAAPRfkLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4HOW1+PHvrqrVLNmSLRfZcj22\n3LCNG2AwJXQIDk5CSINACgFC+N2bXNLDTS4kXEqA9HATSKGHjkMNxcYGF8AF28cgWS6SbVWrWW21\n+/tjRmItq1nW7kra83keP5Zmp5x3dzVn3jLveAKBAMYYYwyAN9IBGGOM6T8sKRhjjGljScEYY0wb\nSwrGGGPaWFIwxhjTxpKCMcaYNrGRDmCgE5HfAae7v04CioF69/cFwLPAfar69xAce5m778nHuF0A\nyFHVfe2WfwG4WlWX9VmQzn7vBz5S1Z93s14h8AVVXR20LNfdtsPvqoicANwPLAGygP8DxgO1wH+o\n6msdbPM6MBE4HLT4TGAc8Jd2q08C5qnqFhEZAfwDmNDRey4iXmAtsF1Vr3CXnQzcCaS5x7tRVd90\nX/se8GUgAGwHvqmqB9z36xygKmj3XwIOAC+1O+w44LOq+qyIpAB/cH/v7P16HMhs/YxFZDrwO2Ak\n4AN+oqpPuK9dCXwH8AD7gGtVdaeI/BS4DigL2vX3gGeAD9odchTwQ1W9V0TigF8A/48Ovn/uMW8H\nVqhqbkfxt1v3DOB2IAXYDVzZyT5Tcb4Xi3E+gx+o6j/d1/4L5zNIAR7H+c4E3NeuAb7r7uYl4DpV\nbe7sPXO/qx8C+UGHXwdcAbwB3Kyqr3RXrkizpHCcVPWa1p87OalFIKro4J6E/w58VVXrReSPwPOq\n+is3WbwgIhNUtb6Dzb+kqq+3W1YETAva/yLgXmCriAzD+cP+FzChk5CuwTlRbHe3TwCeBj6tqq+J\nyPnAQ8AYEfkE8BXgRFWtEpFf4JzgvuDu63uqen8HxwiObzzwMtB6olkDPNdJbIjIBcCJQGHQ4seA\nO1T1LyIyC1grIq/inMz/F5ijqkUi8g3gz8Ap7na/VtWfdhNfKvAezskW971Y30V8c4BLOnu93brJ\nwMPAuar6roh8C/g9cGEHq98J7Me5WJgK/F5EngY+AVwNLATqcN7LLwB/E5FTcJLXQqAS+CtwMvA6\nnb9nAEWq2vYeBMX7FeAVEZnWyfex37CkEB4T3KvTKcCbwOdV1e9esX8f50oiDxCcK5BRQCPOlc8G\n9wrwbzh/cAnAq8A3W3cuIj/A+TLH41zpvyYiicCvcGoxfmAl8F1VbQnazgvcA1yMcxX6RvvA3XWK\ngAtVdaO77Ns4V11XdxSXqjZ39kb0JK5jsAIoV9W1IjIUOAO4FEBV3xeRPcAynBN5b9yNe+XoflaX\n4Hw2F7dfUURGAdcDdwFz3cVxwNeCaiurgdEikg7MAjaoamtt4N/AbccY323Az4JOMl/HOfl9t/2K\nIpKEc5L/Kc73DRGJAX4G/BPArQ014SS9icCHqloUFN8vjjG+HwIPqOp+9/efuZ/VjzuIz4vz3f9h\nD49zBlCgqu+6v/8ZuF1EUlW1Jmi/CcDngIluDUBxa/ZuYn5SVSvd338DXI7znb4S+IOqlrq7utxd\np6v37FBnwarqhyLyNs7fzL09KF/EWJ9CeCwDzsM56Z+Oc8XRyqOqgtOE8BTwV1WdCnwDeFpEYnGq\nt4dUdTrOlY4PmOFuPxbY4r7W+kcF8G0gx11vHrAU548j2LnA2TgJ6TTg1PaBq6ofeJIjT4TLgUe7\niaszPYmrp1a4sQFMBkpVtS7o9XyCrlzb+X8i8p6IbBKRq9u/6F5V16vqKgBVrVRV7SKWXwE3E9Tk\no6q1rU0xrvOAnap6COeK8yQRGet+xstxrlRbXS4i60Vkm4h8X0Q87eKbifP+/SPoeGu7iO8nOCe7\nwqD1W1T1EVX1uftc5L60E3gbmCQiM91jX9ouvrNEZI2IqIjc4Z58g+PLBL6Ik1h7Et/XgS3ucXti\nKkHNNKpaC5TjfA+CTcFpzr3CfS/XichZ7msBICZo3dqg7ecAKSKyyi3jLSIS0817BpAmIk+JyA4R\necFtamr1BPCZHpYvYiwphMc/VbXe/eJ+iHMib9Va3Z8GjMC54kFV3wJKgZOAEmCJiJwNxKjqNar6\nvrtdtao+4/78XtC+LwD+qKo+90ryHzgJINipOM0tte46j3YS/+O4ScH9Y5+Dc4XfVVyd6UlcPbWQ\nj5sjkoCGdq/XA8kdbPc8Tj/EPJwrwFtE5LR263wXpzmnWyJyLpChqg91sc5snFrE1wHcK9wHcE7S\n5Tifxa3u6m8Aj+DUxs7B6U/4Yrtdfgf4lZu0u4tvlrufTssjIjnAg8D1qnpYVYtxarHvAxXAtcBN\n7urv4iTj03H6chYC/9Vul9cD/1DV6h7El41zsXBTd+sG6ennne7+a1DVPOBHwONuc+DLwGfdxJwE\nfBVIDNruFOB8nIu4C3FqD8FxH/GeATXu79/GudB6mY8v7ADeARa1T/D9jTUfhUfwH0YLR16dVLj/\np+N80bcH9UOkAcNV9TH3S/wzYJqI/B2nvbOrfWfhtIW2qsRJOsGG4XSMB6/TkTdw2sHHAWfhJJIG\noMO4VLWxk/10F5efoy9UYtxydWQETmICp004sd3rSThXf0dQ1f8N+vUDEXkYJ1m9ASAiY4GZwAtd\nlAN33SE4J9tO28JF5CSchHt1az+GiFzsHnMkznfg+zj9I+eranBn9163r+RCnHbt1iaRS4D/7EF8\nHuC3OCeu5o76uMRZuBK4VVX/4S6bC/wAp9lljziDEJ4RkZlBFyEAjSJyF84J/b+Dll8OfLa7+Fx3\nAf+tqpVuM2BP9PTzrsL5Dv0OQFVfdJsVF6vqShG5B6dPphLnSj4naLuH3KaoGnE6/88G7oOO3zNV\nLcfpgMdd507gxzi1mm0439U4IIOP/+77Hasp9B/FOFf904L+jVbVJwFU9Q+qugjnCmQ+ztVjVw4C\nw4N+H+4uC1YJBP8RZnW0I7e9/yngIpyT0aNBr/VlXAeA3HbrTwX2dLKv4Cuuj4BMt/+l1RScP8Y2\nIhLjdmgGiwWC+0EuAF7uYT/HfJza2WoROYDTXPJZEXnePd5snI7Jz6nqyqDtzgZeUNVyt637EZwm\nPNwmm+DmmPbxLcMZ4VRK93JwanaPufE9gdNstdk91hic5HeTqt4XtN2ZwBpVbX3vH8H5jDNFZLKI\npHUWn3vCTMGpufbEhcAdbnzrgRwROdC+SaqdHQQ1FbnJJAOnJh5sr/t/atCyFvcfqnqb+7e2BOd7\nuMVdZzdH/m20bdPZeyYiGSLSfhBCDEd+dv2eJYX+YzewT0RWgNNMIyIPiUiyiPxInNELuB1/u3Da\nQ7vyHHCVexJMxml+eL7dOmuBc0Qkya0+f7qL/T2OkxQW4l5BhyCu3+G09Y9w9z8Sp2P0jk72VYKb\nyNxmipeBb7nbng5k00HnOfCciHzaXS8H+BRHvjdzcEcQdUdVV6tquqpmq2o2cAPwiKpe4F6lP4DT\n+b6q/abAme77Dk4i2ur+/MegcmTgJNrexrdHVdOC4vsUzsl+trvK73CaoR7rIL6TRKQ1gZ+Pk7TL\ncGoEt4iIR5yBA1/vIL4dbrLrSYypQfEtAPa6v3dV43wNGC/OKCGAG4Hn2vUp4fbfvIhbq3L7AHKB\n9SKyTEReE5F4cUZK3YjzeYGTBL8qIkPd2uAX+HiUV2fv2QLg3yLSenH1VZwLmgL39yycBNFph3R/\nYM1H/YQ7wuUynOFyP8dpSrlTVetE5G/AX8QZUx3AaZv8G057bmfuxRlB8oG7zWPuv2DP4pyMFOcP\nfiUddDa7/o3TXvqvoD/WzuLqSqdxqerf3ETxijhj2puB36vq7zvZ1zqcP8Q17u/fAB5wE1U1zlDQ\nRgBxhgx+R53hi58C7nXf52accetrgvY7FtgUfCARuQhn9E4SkC0iO3CGH57ZRVkXA7OBX4rIL4OW\nX44zfFKAzSLSgvP+t7ZZfwn4g4h8Fefq9G84Q1mD4zvQLr55OJ9PHBDjxod2MDwyaJvROIl+mjhj\n8lt9R537HubjDLcM8PH7GRBn9NkfcTpXW3C+N8GJu6P4RnJkgn5dRHzAmfrxCKeOYrwOGKmqPwpe\nrs4Q5MuA37jfmY/4eFTVGOBFVZ3prn4V8FdxhoxX4dzHUSEiq9wyfIjz93ZXa/Oeqj4iIjNwEnU9\nznDa+3vwnv0WeEtE/Dij9i4NqnEuAtb1pB8okjz2PAUzULknha+p6hmRjsWEhnsS/g9V/Y9Ix3K8\nRORBnKTwq0jH0hVrPjID2WPAKBFZEOlATMiMAv4U6SCOl4hMxBl+3e/LYjUFM6C5zSb3ASdrP79T\n1EQncW7Mew24RVVfjHQ83bGkYIwxpo01HxljjGkz4EcflZbW9Lqqk5GRRGXl4e5XHESszNHByjz4\nHW95s7JSO7yzOqprCrGxMd2vNMhYmaODlXnwC1V5ozopGGOMOZIlBWOMMW0sKRhjjGkT0o5md/bE\nxTjTGdygquvd5WMImgceZ9qDm3BuRrof5wlJLTgPmSnAGGNMWISspiDO/PRT3NkHr8J5whfgTJ6m\nqsvUeU7sWTiTRj2DMyfMIVU9BfgfPp5f3hhjTBiEsvnoTJzpllHV7UBGu+l2W12B8xCaWneb1idp\nvcKRTygzxhgTYqFsPsoGNgb9Xuoua/8kpqv5+Mlb2e56qPsMYxGJV9Wmzg6SkZF0XEOzsrJSu19p\nkLEyRwcr8+AXivKG8+a1o26UEJElOPOud/bIvm4fW3ecN29QWlrT/YqDiJV54Gn2+Skuq2NvSS17\nS2rxeGDcyBTGjUgle3gSsTFHV/gHepl7I9rKfLzl7SyhhDIpFONc+bcaDexvt86FfPzgiuBtNrnz\n6Xu6qiX0V/v3F/OlL12GyDQ8Hg9NTU1885s3MGfOCce0n9Wr32DRopOIi4s7Yvnrr7/KsmVnsnLl\nsxQU5HPddd/uk7hXrLiIv/71EZKSknq17ltvreL111/lBz/4aZ/EMxg1NPnIL6pm595DfLjvELsP\n1hAfG0NqUhwpQ+JITYonJSmO5MQ4Kqob2HOwlv3ldbT4O75xPzbGw+jMZMaNSGXimDROlBGkDInr\ncF1jeiKUSeEl4Gach4XMA4rd550GWwA83G6bT+M8KekinJkFB6Rx48bz61//EYD333+XBx64jzvv\n/PUx7ePhh//BvHkLjkgK+/cX88orL7JsWVfPdjH9RbOvhR17DvHBrgonCRyoxe9OQukBsocn0dIS\noLy6kX2ldUdtHx/nJTc7lZyRqeSMSCFnRAqBQIA9B2vZW1LD3pJa9pXWsedgLau37OfBl3dywuRM\nzjtlIjnDhnRYizCmKyFLCqq6RkQ2isganKcaXSsiVwBVrc8dxpkrvSRos0eAT4jIaqAR90lKA11F\nRQWZmc4T+srKSrn11p/h8zXj9Xr5r//6EdnZ2fzqV//Ljh3baWlpYfnyFXi9XrZt28p//ue3uPvu\n37Ulhjvv/CXbt3/AX/7yJ0aOzKasrJQf/OA7FBbu4nOf+yIXXvhJLrtsOYsXn0xGRgYXXHDxEce7\n7bZfEBeXetTxzj//IgD++c9Hefvtt2hpaeHOO+8lPj6B2277H4qLi2hqauLqq7/BwoWL28qWn/8R\nP//5j0lLG8ro0WPD/+b2QxXVDWzOL2dzfjnbdlfQ1Ow8aCvG62Hi6DSm5Axl6th0powdSlLixwnf\n1+Kntr6ZmsPN1NY3k54Sz8iMJLzeo1tRp4xNb/u5xe/nQEU9W/LLeWvLfjZoKRu0lLSkOBbPyOak\nmdnkjEjB4+m2NdaY0PYpqOpN7RZtavf6rHa/t/DxIwkHtD17dnPddV+jqamJsrJS7rjjXgD+9Kff\ncdlln2fBgkWsXbuaBx64j2uuuZ41a1bz6KNP4/P5WLnyWS6+eDn33fd7br/9niNqCp/73Bd54olH\nufLKr7Jy5bMUFxfxu9/9H0VFe/nxj7/PhRd+Ep/Px+LFJ7F48Unceut/H3G83/72t1x55TeOOl6r\niRMn8cUvXsFPfvJ9NmxYT11dLfHx8fz613+krKyU6677Og8//ETb+vfffx9f+crXWLp0Gbfffis+\nX/je40gJBALUNfioqG7gUG0jlTUf/9u1v4Z9pbVt644ansTsScOZNXE4k8YMJSGu80ERsTFe0lMS\nSE/p6nn1R4vxehmTmcyYzGTOWZjD7oM1vPtROa9v3MdL6/fy0vq9DE2JJ2/8MPJyM8jLHUZG6rEd\nw0SPAT9Lan8V3Hy0e3chP/rRf/HnP/+DrVs3s2fPbh544P/w+/2kp2eQljaUnJzx3HTT/+P008/i\n3HMv6PFxZsyYRUxMDJmZI6ir+/hklJc3A+Co440cmdXl8WbPdvo9srKc/aluZ+7c+QBkZmYRHx9H\ndXVV2/qFhQXMnDkHgLlz5/P228GPOh5cDlYcZtXm/azZup9DtR13dcXGeJk5cRhzJmUya9JwRqQP\nCWuMHo+H3Ow0Fswaw8VLxrPpo3I2aAnbCitY+8EB1n7gPDp51PAk8nKHITnpTM1JJy05Pqxxmv7L\nkkIYjB+fS0JCAiUlB4mNjeNnP/slmZmZR6xzxx33oLqDl19+gRdeeJ677vpNj/YdE/PxlWfwA5Ni\nY+Pa/g8+XuuIhc6Od/T+PEfst7m5GY/HG7QObc0bfn+/fh55rzQ2tbBBS1i1qZid+5xkmJQQy9wp\nmWSkJpCR6lzZD0tNID01geFpicR3URsIp9gYL/Mli/mShT8QoKi0jm2FFWwrrET3VvLqxn28unEf\nANnDkpiaM5QpY9ORnHSGD0205qYoZUkhDKqrqygvLycrawR5eTNZtep1li9fwcaN6ykvL2fWrNms\nXv0mn/60M2LpK1/5AgAej5eWlpYj9uX1Hr2sK+2P19xcx/jxUzs8XkemT8/j3Xc3cNZZ53Dw4AG8\nXi+pqR8PZRs3bjw7dmxn0aIlvPvuxk73MxD4WvyUHqrnYEU9BysPs6+klo07S2loct7v6eMzWDpn\nFPOnZhE3wKZp9no8bR3V5ywch6/FT35RFTv3VfHh3kN8WFTFm5v28+YmZ4DghFFpXHxyLrMnDbfk\nEGUsKYRIa58CQFNTEzfe+B3i4uK46qqvccstN/PKKy/i8Xj4/vd/QmZmFlu3buLVV18iLi6OCy64\nGIC5c+fxzW9exb33/pH0dKdjcfz4Caju4J577mDy5KndxtH+eLfffhsxMckdHq8jZ555Nu+9t5Hr\nr/86Pl8z3/nO9494/ctfvopbbrmZxx57iNGjx+DzNff2LeuxQCCAryVAjNfTYSdsdxqbWyguq6Oo\ntI6islqKyuo4WHGYsqoG2j+dNiM1gU+cmMMps0eRFeamoFCKjfEi4zKQcRmA01m9t6SWnXur2FZY\nweb8cu5+fDPjR6Zy8cm5nDAls0+Tgz8QwGvJpl8a8M9oPp4nr0XbzS4wcMpcfbiJdz44yMadpdTW\nN9PU3EJTcwuNPj9NzS1tJ+/4OC9D4mNJjI8hMSGWIfExxMfF4PV48HicET9DhsTT3OyjsamForI6\nSivraf+lSUuKY+SwJEZmJDFy2BCy3Z9HZyb3KvFE2vF+zvtKanlubSHrt5cQAHJGpHDRSbnMmjic\nipoGyqsbqKhupKyqgfKqBuobfQxJiCU5MZakxFiSE+NISowlIS6GyppGyqudbcqrnP8PN/hYOmc0\nl581pc+GzQ6U73Zf6YOb1zr8YltSiKIvEfTvMvta/Gz6qJw1W/ezOb+cFn8AjweSE+NIiPMSHxdD\nfGwM8XFe4mO9tPgD1De10NDoo6GphfomX9vwz84kJ8YyNiuFMVnJbf+PyUw+YmjoYNBXn3NxWR3P\nrS3knW0Hj6pF9UZsjIdhaYnuvRkNTBuXzjeXz+qTG+7683c7FEKVFKz5yERUi99PflE163eU8M62\ng9TWO81P40akcNKsUSzOG3lMI2Na/H6afX78fqeJwu8PkDEsmdLSGmJjvKQmxVkb+TEYnZnM1y6a\nwcUnT+CFd3ZTVtXAsLREMtMSGT40kWHu/8mJsdQ3+Khr8HG4odn5v9FHY3ML6SlOB/zwtARSk+Px\nejw0NrXwp+e28e7OUn7+1w3csGI2o4YnR7q4BqspRNWVBfSPMjc0+fhgVwXvfVjG5vzytkQQfLPV\nuJF9N9FXfyhzuA2EMvsDAZ58s4Dn1+5mSEIs31w+kxm5w3q9v4FQ5r5kNQUzoB1u8LFxZwkbdpSy\nfXcFvhYnl6enxLNs7hhOmJxJXm6GTcsQRbweD5eeNolRw5O4/187uOuRTXz+E1M4fZ7dGR9JlhRM\nyPha/GwtcG6aev+jMpp9Tnv/2KwU5k7J5IQpmeRmp1pzTpQ7aeYoRqQnce8Tm/nbSzv5qKiaFcsm\n2V3XEWJJwfSpQCBAQXE1az44wPrtJW1NQ6OGJ7F4RjaL8kaG/S5f0/9NHjuUH33pRH795BbWfnCA\njTtLOG/ReM5dOI6E+IF1T8hAZ0nB9InquibWbD3A6i37KS5zZvtMS47n7AU5LJmRzbiRNiGb6Vpm\n+hB+/OUFrNpczJOrdvH06l288X4Ry0+dyMkzRw3IocEDkSUF02stfqd5aNXm/Wz6qIwWf4DYGA8L\np4/glFmjmJ6bQYzX+ghMz3m9Hk47YQwLp4/kX+/s4aV1e/jLyh28smEfnzlj8nF1RJuesaRgeuVA\nxWF+9egmSg7VA87NTUtnj2LxjGx7yIs5bkMSYvnUqRNZdsJonnizgDVbD3DHw++Tl5vBpadNYsKo\njh73bvqCJQVzzIrK6rj9ofeoqmti6exRnD5vDONHWoex6XvD0hK5+sI8PnFiDo+/kc8HuyrYVriB\nE6eNYPnSCXZvQwhYUjDHZG9JLbc//B41h5v5/CemcuZ8Gz5oQm98dir/8dkT2F5YweNvFLBhRwnv\naimnzB7F+UvGkzokjsbmFnwt/rYpTuwipXcsKZgeKzxQzR0Pv8/hBh9fOldYdsKYSIdkosz03GH8\ncHwG7+4s5Yk3C3hzUzFvbirucN3E+BguOjmXcxaOs8n3joElBdMj+UVV3PnoJhqafHzlgumcPGtU\npEMyUcrj8TBfRnDClEzWbHHugWnxB4iNi6GhoRm/P4A/4Mzb9Nhr+eieQ1x1wXRSk+xBQj1hScF0\na+feQ9z12Caam/189aI8FudlRzokY4jxelk6ZzRL54wGjp72obquiT89t43N+eX89C/r+frFM5ia\nk97Z7ozLxguaLhWV1nLno+/j8/m55pIZlhDMgJGWHM+Nn5nDpadNpKq2idsefI/n1hTiH+DzvYWa\nJQXTpcdfz6fJrSHMlxGRDseYY+L1eLhgSS7fvXwuQ1PieeLNAu565H2qD3f8jG1jScF04aN9VWzK\nL2dqTjoLpllCMAPX1Jx0fnrlAmZPGs4HhZX88h/vUlXbGOmw+iVLCqZDgUCAf76RD8CK0ybZ8D4z\n4KUmxfOtFbM5Z2EO+8sPc9tD73HIEsNRLCmYDn2wqwLde4g5k4YzeezQSIdjTJ/wejx85vTJnLto\nnJMYHrTE0F5IRx+JyF3AYiAA3KCq64NeywEeAuKBd1X1GyKyDHgM+MBdbYuqXh/KGM3RnFpCAQDL\nT50Y4WiM6Vsej4dPL5sEwAvv7OG2B9/ju5fPJT3FpuqGENYUROQ0YIqqLgGuAu5pt8odwB2quhBo\nEZFx7vI3VHWZ+88SQgRs1FJ2H6xhUd7IPn0CmjH9RWtiOG/ROA5UWI0hWCibj84EngJQ1e1Ahoik\nAYiIF1gKPOO+fq2q7glhLKaHWvx+nnizAK/HwyVLJ0Q6HGNCxuPxsGLZJM5b7CSGX1piAELbfJQN\nbAz6vdRdVg1kATXAXSIyD1ilqt9z18sTkWeAYcDNqvpyVwfJyEgiNrb3D+HIyoq+K+GuyvzyO7s5\nUHGYcxaPZ+bUkWGMKrTsc44OvSnzNStOIGlIPP987SN+9fhm/vf6pSQlDoyZfkPxGYfzjmZPu5/H\nAHcDhcDzInIB8D5wM/AoMBF4TUQmq2qng4orKw/3OqBoe9A3dF3mZp+fv7+wndgYL2fPHzto3hv7\nnKPD8ZT5/IU5HKpu4NWN+7jlz+9w/aWz+/1DfY73M+4soYSy+agYp2bQajSw3/25DNitqvmq2gK8\nCsxQ1SJVfURVA6qaDxzASR4mDF5/r4iK6kbOnD/Gno9roorH4+GyMyczIzeDTfnlbcOxo1Eok8JL\nwAoAt4moWFVrAFTVBxSIyBR33fmAisjnReQ/3W2ygZFAUQhjNK76Rh/PrS0kMT6G8xePj3Q4xoRd\njNfLNy6ZychhSfzrnT2s2bq/+40GoZAlBVVdA2wUkTU4I4+uFZErRGS5u8q3gb+4r1cBz+J0PJ8m\nIquAp4Frumo6Mn3njfeLqTnczLkLx9lskiZqJSfGccOK2SQlxHL/v3aQX1QV6ZDCLqR9Cqp6U7tF\nm4Je+wg4pd3rNcBFoYzJdOydbQeJ8Xo4wx6aY6Jc9rAkvnHJDO56dBP3PrGFH3/5RIalJUY6rLCx\nO5oNJYfq2X2whum5GfZ8ZWOAmROGc9mZU6iua+Kef26msakl0iGFjSUFw8YdJQAssFlQjWlz1vyx\nnDpnNHsO1vLnldsJRMmU25YUDOt3lBDj9TB3alakQzGm3/B4PHzh7KlMGTuU9TtKWO9ePA12lhSi\nXOmhegoP1DB9vDUdGdNebIyXqy6YTlyslwdf3kldQ3OkQwo5SwpRboM6Vz8n2vMSjOnQiIwkPnnK\nBKoPN/PYax9FOpyQs6QQ5TbsKMXr8TB3SmakQzGm3zp7QQ5js1J4c9N+dE9lpMMJKUsKUaysqp5d\n+6uZPj7d7k0wpguxMV6uOG8aHuCBF5Rmnz/SIYWMJYUotmFHKWBNR8b0xMTRaZwxfywHKg7z/NrC\nSIcTMpYUotgGLXGajmzUkTE98qlTJ5KRmsDza3dTXFYX6XBCwpJClCqvaqCguBoZl06aNR0Z0yND\nEmL5wiem0uIP8NcXduAfhPcuWFKIUhvdUUcLrOnImGMyd2oW86dmsXNfFas2FUc6nD5nSSFKrdcS\nPB6YZ01Hxhyzyz8xlSEJMTwntlRgAAAd5UlEQVT6Wj7VdYNrzk5LClGoorqB/KJqJCedtGRrOjLm\nWGWkJrB86cS2KecHE0sKUWiDOqOOrOnImN5bNncMmUMT3YdTNUQ6nD5jSSEKbdjhNh3ZBHjG9Fps\njJeLT56AryXAs2sKIx1On7GkEGXKq+r5qKgKyUlnqDUdGXNclswcSfawJFZv3k/JcTwvvj+xpBBl\n3trsjJawG9aMOX4xXi+XLJ1Aiz/A06t3RTqcPmFJIcqsfr8YDzbqyJi+cuK0EeSMSOHtDw5SNAhu\naLOkEEUqqhvYXliBjEsnPSUh0uEYMyh4PR6WL51IAHhqVUGkwzlulhSiSOtDQhZMHxnhSIwZXOZM\nHs7E0Wls1FJ2H6iJdDjHxZJCFFm3vQSv18N8saYjY/qSx+Nh+akTAXjizYFdW7CkECVKDznTZM+e\nnGlzHRkTAnnjM5g2Lp0tBeV8uO9QpMPpNUsKUaK16WjpCWMiHIkxg5PH4+FTp04C4Ik3CggM0Mny\nLClEiXXbDxLj9bBk1qhIh2LMoDV57FBmTxqO7j3Ett0D8wltsaHcuYjcBSwGAsANqro+6LUc4CEg\nHnhXVb/R3Tamdw5WHGbPwVpmTxpOalI8DXWNkQ7JmEHrkqUT2JxfznNvFTIjd1ikwzlmIaspiMhp\nwBRVXQJcBdzTbpU7gDtUdSHQIiLjerCN6YV12w8CsHC63bBmTKjlZqcxc+IwdO+hAdm3EMrmozOB\npwBUdTuQISJpACLiBZYCz7ivX6uqe7raxvTeuu0lxMZ4mTvFRh0ZEw4XLskF4Pm1uyMbSC+Esvko\nG9gY9Hupu6wayAJqgLtEZB6wSlW/1802HcrISCI2NqbXQWZlpfZ624Fg94FqisrqWDwzm3FjM4DB\nX+aOWJmjQ38pc1ZWKjPW7mZzfjnVjS1MGpsesuP0tZD2KbTjaffzGOBuoBB4XkQu6GabDlUexyRU\nWVmplJYO7BtNuvPiW858LHMmDqe0tCYqytyelTk69Lcyn3PiWD4oKOfvK7fxzeWz+nz/x1vezhJK\nKJuPinGu8luNBva7P5cBu1U1X1VbgFeBGd1sY45RIBBg3Y4S4mO9zJk8PNLhGBNVZkwYxvjsVDZq\nKfvLB86cSKFMCi8BKwDcJqJiVa0BUFUfUCAiU9x15wPa1Tbm2O0tqeVgxWFmT84kMT6clUJjjMfj\n4cIl4wkAKwdQ30LIkoKqrgE2isganFFE14rIFSKy3F3l28Bf3NergGc72iZU8UWDddudG9YW2agj\nYyJi7tQsRg1PYu0HByk7VB/pcHokpJePqnpTu0Wbgl77CDilB9uYXggEAqzbfpCE+BhmTbSmI2Mi\nwevxcMGS8dz33Hb+tW4PXzxbIh1St+yO5kGq8EANZVUNzJ2SSXxc70dnGWOOz6K8kWQOTWTVpv0c\nqu3/N45aUhik2m5Ym2bTZBsTSTFeL+cvHo+vxc9L6/dGOpxuWVIYhJp9Lbyz7SBDEmKZMWHg3WZv\nzGBz8qxshqbE89p7RdTWN0c6nC5ZUhiEXlq/l0O1TZx2wmjiYu0jNibS4mJjOHfhOBqbWvj3xn2R\nDqdLdsYYZKrrmnh+7W5ShsRx4ZLxkQ7HGOM67YTRJMTH8NbW/f16Wm1LCoPMU6t30dDUwidPmUBS\nYlykwzHGuBLjY5k7JZPSQw0U7O905p6Is6QwiBSV1vLG+0WMGp7EaSeMjnQ4xph2FrnPR39n28EI\nR9I5SwqDyKOv5RMIwKdPn0xsjH20xvQ3MyYMIzkxlvXbS/D7+2cTkp05BomtBeVsKShn+vgM5kyy\nm9WM6Y9iY7wsmDaCqromduzpn09ms6QwCLT4/Tzy74/wAJ89YzIeT7eTyxpjImRRXv9uQrKkMAis\n2ryforI6Tp49inEj+8d88saYjk3JSScjNYGNWkqzzx/pcI5iSWGAq2/08dSbBSTExbB86cRIh2OM\n6YbX42HBtBEcbvSxdVd5pMM5iiWFAW7l27upPtzMeYvHkZGaEOlwjDE90J+bkCwpDGA1h5t4af1e\nMlITOGfhuEiHY4zpodzsVEZkDOH9j8poaPJFOpwjWFIYwLYWVNDs83Pm/LEk2EyoxgwYHo+HxXkj\naWr28/6HZZEO5wiWFAawLW575Gx7XoIxA05/bUKypDBA+QMBthZUkJ4Sz5is5EiHY4w5RqOGJzNu\nRApbd1X0q5lTLSkMULsP1FBb38zMCcPtvgRjBqhFeSNp8QfYqCWRDqWNJYUBamuB03Q0c6I9L8GY\ngWphP5wLyZLCALV1VwUeD+TlWlIwZqAaPjSRyWOHonsOUVnTPx7VaUlhADrc0Ex+UTUTR6eRMsSm\nxzZmIFucN5IAsH5H/2hCsqQwAG0rrMQfCDBzgo06MmagO3HaCLweT9tz1SPNksIA1HprvPUnGDPw\npSXFM2XsUHYVV/eLUUixXb0oIqe2WxQAqoAtqto/JwMf5AKBAFsKKkhOjGVCdlqkwzHG9IG83Ax0\n7yF27K7kxGkjIhpLl0kB+J8Olo0E6kTkk6q6p6uNReQuYDFOMrlBVdcHvVYI7AVa3EWfB6YAjwEf\nuMu2qOr13cQYVYrL6qisaWTh9BF4vTYU1ZjBIC93GE+u2sW2/p4UVHVpR8tF5ALgTmBFZ9uKyGnA\nFFVdIiLTgT8DS9qtdp6q1gZtMwV4Q1U73W+021JQAcAsu4vZmEEjd1QqQxJi2FZYEelQetenoKrP\nA901aJ8JPOWuvx3IEBFr7zhOH7j9CTMmWH+CMYNFjNfLtHEZlFTWU3aoPqKxdNd81JXuxkJmAxuD\nfi91l1UHLfu9iOQCq4HvucvyROQZnKRzs6q+3NVBMjKSiI3t/WRwWVkD56E0DY0+dG8VE0anMWVC\nZq/3M5DK3FeszNFhIJd54cxRvPdhGXsr6pk+pWdNSKEob3cdzR3VJIYBXwO2H+Ox2jeA/xh4AajA\nqVFcCqwFbgYeBSYCr4nIZFVt6mynlZWHjzGMj2VlpVJaWtPr7cNtc34ZvhY/08al9zrugVbmvmBl\njg4DvczjMpMAeGdLMXN7MLLweMvbWULprqbgw+kkho9P6jXA48AN3WxbjFMzaDUa2N/6i6r+tfVn\nEVkJzFLVx4FH3MX5InIAGAPs6uZYUaGtP8HuTzBm0MkelkRGagLbdzv3IXkjNKdZdx3Nx3Mfw0s4\nV/1/EJF5QLGq1gCIyFCc2sBFbi3gNOBxEfk8MEpVbxeRbJyRTkXHEcOgsrWgnIT4GCaPHRrpUIwx\nfczj8ZA3PoO3th5gX0ltxJ633qM+BRGJAS4HFuDUHN5W1Ye62kZV14jIRhFZA/iBa0XkCqBKVZ90\nawdvi0g98B5O7SMFeFBEPgnEA9d01XQUTUoqD3Owsp65UzKJjbF7Do0ZjPJyh/HW1gNsK6zs30kB\nuBfIAl7HaUb6jIgsVtUum5BU9aZ2izYFvXY3cHe712uAi3oYU1TZustpOpppQ1GNGbSm52YAsG13\nBecuiswjdnuaFGao6mlBv/9aRFaFIiDTsa1uf8JMG4pqzKCVnpLAmMxkdu45RLPPT1xs+FsFenrE\n+OCRSG5z0vEMZzXHwNfiZ/vuSkYOSyIrfUikwzHGhND03AyafH7yi6oicvyentifB9aLyBvu76cD\nD4cmJNOe7j1EY3MLs6yWYMygl5c7jFc27GPb7gqmjc8I+/F7VFNQ1Z8D1wG7gULg66r6yxDGZYK8\n+I4zxVSk50QxxoSe5KTj9XjYVlgZkeMfy81r77j/2l5TVX+oAjOO/KIqtu6qYNq4dKbmpEc6HGNM\niA1JiGXimDTyi6o43NBMUmJ4H6TVXU3BBzR38K91uQmxp99y7tv75CkTIhyJMSZc8sZnEAjAjj2H\nwn7sUN68Zo5TfnEVWwucWoKMC3/bojEmMmZMGMYzbxWyrbCCeVOzwnpsO+n3Y8+sLgTg4pOtlmBM\nNJkwKo2E+JiI9CtYUuinCoqr2VJQjuSkR2QEgjEmcmJjvEzLSedAxWEqqhvCemxLCv3UM25fwsXW\nl2BMVMrLdYagh7u2YEmhH9q1v5rN+eVMzUln2jgbcWRMNMoLmvKiVSAQoL7Rx8GKwxSX1na26XGx\nu5L7oadXuyOOTs7FE6Hpc40xkTU6M5mhyfG892EZ//O3DVTVNlFd10ST7+M7AX529SLGZCb36XGj\nNikUldVR1djC0ITeP7UtFNpqCWOHWl+CMVHM4/Fw4rQRvLpxH4X7a0hNimPU8GSGpsSTlhzPxLHp\njAjBtDdRmxTuX7mdqrombrvmpEiHcoRnVn/cl2C1BGOi2+VnTeGSpRMYkhB71EN3QvWkuajtU4jx\neqiobsDvD3S/cpgUHqhmU345U8YOZbrVEoyJeh6Ph+TEuLA+hS1qk0JKUjz+ABxu9EU6lDavbtgH\nOPclWC3BGBMJ0ZsUhjgtZ7X1/WO2Dl+Ln/c/KiMjNaHtQRvGGBNuUZwU4gGoPdw/koLuPURdg495\nU7Mi9sBuY4yJ4qTgzDzYX2oKG7UUgPlhnufEGGOCRX1SqKlvinAk4PcHeHdnKSlD4piSMzTS4Rhj\noljUJ4W6+sh3NOcXV1Fd18S8qZnEeKP2IzHG9ANRewZKSeo/NYXWpqN5U+3JasaYyIrepNBWU4hs\nn0IgEGCjljIkIcbuTTDGRFzUJ4WaCI8+2nOwlvLqBuZMziQuNmo/DmNMPxHSaS5E5C5gMRAAblDV\n9UGvFQJ7gRZ30edVtairbfpSUmIsXk/kawobd5YANurIGNM/hCwpiMhpwBRVXSIi04E/A0varXae\nqtYe4zZ9wuvxkJIUT02kk4KWEh/rZeaE4RGNwxhjILTNR2cCTwGo6nYgQ0TSQrBNr6UmxUf0PoXi\nsjr2lx9m5sThJMT3r9lajTHRKZTNR9nAxqDfS91l1UHLfi8iucBq4Hs93OYIGRlJxMb27oSalhzP\n/vI6hg9PwesN/13E/95UDMDpJ+aQlZUatuOG81j9hZU5OkRbmUNR3nBOnd3+rPtj4AWgAqd2cGkP\ntjlKZeXhXgeUlhyP3x9gT1ElyYlxvd5Pb7357j5ivB4mjEgOyRS4HQnVdLv9mZU5OkRbmY+3vJ0l\nlFAmhWKcq/xWo4H9rb+o6l9bfxaRlcCs7rbpa2nJ7vxH9c1hTwqlh+rZc7CWmROHkRSBhGSMMR0J\nZZ/CS8AKABGZBxSrao37+1AReVFE4t11TwO2drVNKKQmRW5SvHd3OjesnSh2w5oxpv8IWU1BVdeI\nyEYRWQP4gWtF5AqgSlWfdGsHb4tIPfAe8LiqBtpvE6r44MiaQrht3FmKxwMnTM4M+7GNMaYzIe1T\nUNWb2i3aFPTa3cDdPdgmZFIjlBQO1TaSv6+KqTnpbYnJGGP6g6i+hTZSNYX3dpYSAOaJ3bBmjOlf\nLCkQ/qSwwZ6dYIzpp6I6KbR1NIcxKVTVNbFjTyWTRqcxLC0xbMc1xpieiOqk0FZTCOPoo41aQiAA\nC6aPDNsxjTGmp6I6KaQkxeMhvDWFddtL8AALptlQVGNM/xPVSSHG6yEpMTZsSaGyppEP9x5iytih\nZKQmhOWYxhhzLKI6KYDzXIVwJYUNO0oIYE1Hxpj+y5JCkpMUAoFAyI+1bsdBPB440ZqOjDH9lCWF\nxDha/AEamlq6X/k4lFc1kF9UzbRxGQy1G9aMMf2UJYUk97GcIW5CWr/DecLagulWSzDG9F+WFNxn\nNYd6WOq67Qfxejx2w5oxpl+zpNCaFEJYUyipPEzhgRrycjPabpgzxpj+yJJCW1JoCtkxrOnIGDNQ\nWFIY0jrVhS9kx3hnWwkxXms6Msb0f5YUhjizh4eqprC/vI59pbXMmjjcnrBmjOn3LCkkhbamsG67\nNR0ZYwYOSwpto4/6vqYQCARYt/0gcbFee8KaMWZAiPqkkJzY2nzU96OPikrr2F9+mNkThzMkIaQP\nuTPGmD4R9UkhNsbLkITQTIq3bsdBwJqOjDEDR9QnBYDUEEyKV1XXxNqtB4mP8zJnkjUdGWMGBmvT\nAJKHxFFR0kAgEMDj8Rz3/gqKq/nNk1uorGnk7AU5JMTH9EGUxhgTepYUgNSkOHwtARqbW0iMP763\nZNWmYv72ktLiD/DpZZM4d9G4PorSGGNCz5ICkJz48fxHvU0KvhY/D73yIa+9V0RyYixf/+QMZk4Y\n3pdhGmNMyFlSwKkpANQ2NJPJkGPevqq2kd88tZWP9lUxNiuF6y6dxYj0Y9+PMcZEWkiTgojcBSwG\nAsANqrq+g3VuBZao6jIRWQY8BnzgvrxFVa8PZYxwfDOllh2q59Z/vEtlTSMLp4/gyvOmWx+CMWbA\nCllSEJHTgCmqukREpgN/Bpa0WycPOBUIPhu/oaorQhVXR3o7U2qzr4XfPLWVyppGPnXqRC5YMr5P\nOqqNMSZSQjkk9UzgKQBV3Q5kiEhau3XuAH4Qwhh6pDUpHOuDdv7x8ofsPlDDKbNHceFJuZYQjDED\nXiibj7KBjUG/l7rLqgFE5ArgDaCw3XZ5IvIMMAy4WVVf7uogGRlJxMb2vrkmKyuVsVWNAPg9HrKy\nUnu03SvrdvPmpmImjhnKty+fT0LcwGky6mkZBxMrc3SItjKHorzh7Ghuu4wWkWHAlcBZwJigdT4E\nbgYeBSYCr4nIZFXtdGKiysrDvQ4oKyuV0tIafE1ODaGkvI7S0pput9t9oIbf/nMzSQmxfO2iPKoP\n9T6GcGstczSxMkeHaCvz8Za3s4QSyqRQjFMzaDUa2O/+fAaQBawCEoBJInKXqt4IPOKuky8iB3CS\nxq4QxnlMHc11Dc385sktNPv8XHPJTBtlZIwZVELZp/ASsAJAROYBxapaA6Cqj6tqnqouBpYD76rq\njSLyeRH5T3ebbGAkUBTCGIGedzT7AwHue3YbZVUNXHjSeJv51Bgz6ISspqCqa0Rko4isAfzAtW4/\nQpWqPtnJZs8AD4rIJ4F44Jqumo76SmyMl8T4mG6Twsq1u9mUX05ebgaXnDIx1GEZY0zYhbRPQVVv\nardoUwfrFALL3J9rgItCGVNnUrqZFC+/qIonVxUwLC2Br108A6/XRhoZYwYfmyXV1V1S2LizlEAA\nvnC2kOY+rc0YYwYbSwqulKQ4mn1+GptbOny9oLgaDyA56eENzBhjwsiSgqurEUgtfj+FB6oZnZls\nT1AzxgxqlhRcXY1AKiqto6nZz8TR7W/INsaYwcWSgqurpFCwvxrAkoIxZtCzpOBK7SopFLcmhaFh\njckYY8LNkoIruYuksKu4moS4GMZkJoc7LGOMCStLCq7Oagr1jT6Ky+rIzU61exOMMYOeJQVXciej\njwr3VxPA+hOMMdHBkoIr1b0hrbbhyKRgnczGmGhiScGVMsS5/6D28JFTLVknszEmmlhScMXFxpAQ\nF3PE09cCgQAFxdVkpCaQkZoQweiMMSY8LCkESRkSS11QUqiobqSqromJo6zpyBgTHSwpBEkZEn9E\nTcH6E4wx0caSQpCUpDiamv00uZPiFRRXAZYUjDHRw5JCkPZTXRQUV+PxwPjs6HoYuDEmellSCBKc\nFHwtfnYfqGFMZgqJ8TYzqjEmOlhSCBKcFIpK62jy2cyoxpjoYpfAQYKTwsHKesD6E4wx0cWSQpDg\npLDLRh4ZY6KQNR8FSUn6OCkUFFeTEB/D6OE2M6oxJnpYUgiSkugkhdLKeg6UH2aCzYxqjIkylhSC\npLo1ha27KtyZUW2+I2NMdLGkEKR1+uyqOmdSPOtPMMZEm5B2NIvIXcBiIADcoKrrO1jnVmCJqi7r\n6TahkhAXQ3yslyafH7CkYIyJPiGrKYjIacAUVV0CXAXc08E6ecCpx7JNqLXWFoalJZCeYjOjGmOi\nSyibj84EngJQ1e1Ahoi0v/S+A/jBMW4TUq2P5bSZUY0x0SiUzUfZwMag30vdZdUAInIF8AZQ2NNt\nOpKRkURsbEyvg8zKOnJeo4yhiewpqWXWlBFHvTZYDNZydcXKHB2ircyhKG84b15rG9spIsOAK4Gz\ngDE92aYzlZWHex1QVlYqpaU1RyxLiHUqTyOHJhz12mDQUZkHOytzdIi2Mh9veTtLKKFMCsU4V/mt\nRgP73Z/PALKAVUACMMntYO5qm7A4ZfYoEuJirJPZGBOVQtmn8BKwAkBE5gHFqloDoKqPq2qeqi4G\nlgPvquqNXW0TLjMnDOfK86cTG2OjdY0x0SdkZz5VXQNsFJE1OKOIrhWRK0Rk+bFsE6r4jDHGHC2k\nfQqqelO7RZs6WKcQWNbFNsYYY8LE2kiMMca0saRgjDGmjSUFY4wxbSwpGGOMaWNJwRhjTBtLCsYY\nY9p4AoFApGMwxhjTT1hNwRhjTBtLCsYYY9pYUjDGGNPGkoIxxpg2lhSMMca0saRgjDGmjSUFY4wx\nbcL5OM6IcZ/qthgIADeo6vqg184CbgFagJWq+rPIRNm3uinz6cCtOGVW4GpV9Uck0D7UVZmD1rkV\nWKKqy8IcXkh08znnAA8B8TgPsvpGZKLsW92U+VrgCzjf7Q2q+u3IRNm3RGQm8DRwl6r+ut1rfXoO\nG/Q1BRE5DZiiqkuAq3Ae3hPsHuBS4GTgbBHJC3OIfa4HZf4jsEJVTwZSgXPDHGKf60GZcT/bU8Md\nW6j0oMx3AHeo6kKgRUTGhTvGvtZVmUUkDfgOsFRVTwHyRGRxZCLtOyKSDNwLvNrJKn16Dhv0SQE4\nE3gKQFW3AxnulwcRmQhUqOpe90p5pbv+QNdpmV3zVXWf+3MpMDzM8YVCd2UG5yT5g3AHFkJdfbe9\nwFLgGff1a1V1T6QC7UNdfc5N7r8UEYkFkoCKiETZtxqB83GeYX+EUJzDoiEpZOOc+FqVuss6eq0E\nGBWmuEKpqzKjqtUAIjIKOBvnizTQdVlmEbkCeAMoDGtUodVVmbOAGuAuEVntNpsNBp2WWVUbgJuB\nAmA38I6q7gx7hH1MVX2qWt/Jy31+DouGpNCep5evDWRHlUtERgDPAt9U1fLwhxRybWUWkWHAlTg1\nhcHM0+7nMcDdwGnAXBG5ICJRhVbw55wGfB+YCkwAFonInEgFFiHHfQ6LhqRQTNAVIzAa2N/Ja2Po\noIo2AHVV5tY/nn8BP1TVl8IcW6h0VeYzcK6cVwFPAvPczsqBrqsylwG7VTVfVVtw2qNnhDm+UOiq\nzNOBAlUtU9UmnM97fpjjC7c+P4dFQ1J4CVgBICLzgGJVrQFQ1UIgTURy3TbIC931B7pOy+y6A2cU\nwwuRCC5EuvqcH1fVPFVdDCzHGYlzY+RC7TNdldkHFIjIFHfd+TgjzQa6rr7bhcB0ERni/n4i8GHY\nIwyjUJzDomLqbBH5Bc6oEz9wLTAXqFLVJ0XkVOCX7qr/VNXbIxRmn+qszMCLQCWwNmj1B1X1j2EP\nso919TkHrZML3D+IhqR29d2eDNyPc/G3BbhmkAw97qrMX8dpKvQBa1T1u5GLtG+IyHycC7lcoBko\nwhlAsCsU57CoSArGGGN6Jhqaj4wxxvSQJQVjjDFtLCkYY4xpY0nBGGNMG0sKxhhj2kTFLKnGdERE\nbgMWAok4wxpbh+nmAA+p6g/7+HgBIM69h6An678O/FxVX2m3/O/AK6p6f1/GZwxYUjBRrHUMu3vv\nwurWexdE5KfY34aJUvbFN6ZjY0XkcWAa8LqqXudOqnchkAHcCawBfo8zhcZQnGmqH3SfV/EL4DBO\nLeRbQXP+f0tELgJGApep6mYRWYRzc1IzzjMCrlPVba2BuDOe/h8wC2eit+TQFt1EM+tTMKZjk4HL\ncKZK+LKItE4vfgJwvqo+D/wceEFVz8C5w/a/RSQL+DZwp6qeDlzBkbNWbnOXPwh81V32V+BGd/md\nwG/axXIWTnJaAHwRiLZJ3kwYWU3BmI6tdtv+fSJSDqS7y99V1Ub359OBBSLyZff3ZpzZOR8EbhGR\nhcDTqvpM0H5fd//fB4iIpAMjg2oSrwMPt4tlFs6UDQHgsIi80yclNKYDlhSM6Vj7zuDWKYmbgpY1\n4kw9vqHduutE5EWcZ1X8WETWqer3O9ivB6e5qP1xOloWPGdRTA/iN6ZXrPnImN5bDXwGQESGiMhv\nRSRWRG4GYlT1UeAGYElnO1DVKmC/268ATlPR2+1W2wYsFhGPiKQCizAmRKymYEzv/RS4T0RWAwnA\nH1XVJyIfAi+LSCXOVf1PutnPl4A7RaQF5+Hr17R7/UXg88A7OB3NazEmRGyWVGOMMW2s+cgYY0wb\nSwrGGGPaWFIwxhjTxpKCMcaYNpYUjDHGtLGkYIwxpo0lBWOMMW3+PweudSfJ/9wFAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f80becac048>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(thresholds, ious)\n",
    "plt.plot(threshold_best, iou_best, \"xr\", label=\"Best threshold\")\n",
    "plt.xlabel(\"Threshold\")\n",
    "plt.ylabel(\"IoU\")\n",
    "plt.title(\"Threshold vs IoU ({}, {})\".format(threshold_best, iou_best))\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7_d1m1lK0PEW"
   },
   "outputs": [],
   "source": [
    "# Source https://www.kaggle.com/bguberfain/unet-with-depth\n",
    "def RLenc(img, order='F', format=True):\n",
    "    \"\"\"\n",
    "    img is binary mask image, shape (r,c)\n",
    "    order is down-then-right, i.e. Fortran\n",
    "    format determines if the order needs to be preformatted (according to submission rules) or not\n",
    "\n",
    "    returns run length as an array or string (if format is True)\n",
    "    \"\"\"\n",
    "    bytes = img.reshape(img.shape[0] * img.shape[1], order=order)\n",
    "    runs = []  ## list of run lengths\n",
    "    r = 0  ## the current run length\n",
    "    pos = 1  ## count starts from 1 per WK\n",
    "    for c in bytes:\n",
    "        if (c == 0):\n",
    "            if r != 0:\n",
    "                runs.append((pos, r))\n",
    "                pos += r\n",
    "                r = 0\n",
    "            pos += 1\n",
    "        else:\n",
    "            r += 1\n",
    "\n",
    "    # if last run is unsaved (i.e. data ends with 1)\n",
    "    if r != 0:\n",
    "        runs.append((pos, r))\n",
    "        pos += r\n",
    "        r = 0\n",
    "\n",
    "    if format:\n",
    "        z = ''\n",
    "\n",
    "        for rr in runs:\n",
    "            z += '{} {} '.format(rr[0], rr[1])\n",
    "        return z[:-1]\n",
    "    else:\n",
    "        return runs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "m_gCEu020QFz"
   },
   "outputs": [],
   "source": [
    "x_test = np.array([upsample(np.array(load_img(\"./test/images/{}.png\".format(idx), grayscale=True))) / 255 for idx in tqdm(test_df.index)]).reshape(-1, img_size_target, img_size_target, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "KtjwyHHl0sET"
   },
   "outputs": [],
   "source": [
    "preds_test = model.predict(x_test)\n",
    "pred_dict = {idx: RLenc(np.round(downsample(preds_test[i]) > threshold_best)) for i, idx in enumerate(tqdm(test_df.index.values))}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wjWTdUGR1RBM"
   },
   "outputs": [],
   "source": [
    "sub = pd.DataFrame.from_dict(pred_dict,orient='index')\n",
    "sub.index.names = ['id']\n",
    "sub.columns = ['rle_mask']\n",
    "sub.to_csv('submission.csv')"
   ]
  }
 ],
 "metadata": {
  "accelerator": "TPU",
  "colab": {
   "collapsed_sections": [],
   "name": "keras_tgs_salt.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}