TF_2_0_and_cloud_functions.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "TF 2.0 and cloud functions.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/sayakpaul/TF-2.0-Hacks/blob/master/TF_2_0_and_cloud_functions.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LDc_vepL86LA",
        "colab_type": "text"
      },
      "source": [
        "The purpose of this notebook is to show how easy it is to serve a machine learning model via [Cloud Functions](https://console.cloud.google.com/functions/) on the Google Cloud Platform. It is absolutely possible to do this via Colab. In this notebook, we will be\n",
        "- building a simple neural network model to classify the apparels as listed in the FashionMNIST dataset\n",
        "- serializing the model weights in a way that is compatible with the Cloud Functions' ecosystem\n",
        "- using the `gcloud` CLI to deploy our model on GCP via Cloud Functions\n",
        "\n",
        "So, let's get started. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xW-MxeH4kTld",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# install `tensorflow` 2.0 latest\n",
        "pip install tensorflow==2.0.0b1"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U5-crVLxkiPN",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "d2f8eefc-834c-4e3f-f041-a0e329ba3cc8"
      },
      "source": [
        "# import tensorflow and verify the version\n",
        "import tensorflow as tf\n",
        "\n",
        "print(tf.__version__)"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.0.0-beta1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3qPW76p9k5nq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# all the imports we care about in this notebook\n",
        "import matplotlib.pyplot as plt\n",
        "from tensorflow.keras.layers import *\n",
        "from tensorflow.keras.models import *"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZbcZvU-PlLef",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# load and prepare our data\n",
        "fashion_mnist = mnist = tf.keras.datasets.fashion_mnist\n",
        "\n",
        "(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()\n",
        "x_train, x_test = x_train / 255.0, x_test / 255.0"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e4qwoUwcmbMG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# the humble model\n",
        "model = tf.keras.models.Sequential([\n",
        "  tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
        "  tf.keras.layers.Dense(128, activation='relu'),\n",
        "  tf.keras.layers.Dropout(0.2),\n",
        "  tf.keras.layers.Dense(10, activation='softmax')\n",
        "])\n",
        "\n",
        "model.compile(optimizer='adam',\n",
        "              loss='sparse_categorical_crossentropy',\n",
        "              metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ShG3bxScmfdn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 341
        },
        "outputId": "3fe5fd1c-bc12-4a3e-d43d-97a26e26e55c"
      },
      "source": [
        "# kickstart the training\n",
        "model.fit(x_train, y_train, validation_data=(x_test, y_test), \n",
        "              epochs=5, batch_size=128,\n",
        "              verbose=1)\n"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING: Logging before flag parsing goes to stderr.\n",
            "W0711 15:16:04.927937 140034585638784 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:1250: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Train on 60000 samples, validate on 10000 samples\n",
            "Epoch 1/5\n",
            "60000/60000 [==============================] - 3s 42us/sample - loss: 0.5935 - accuracy: 0.7950 - val_loss: 0.4510 - val_accuracy: 0.8406\n",
            "Epoch 2/5\n",
            "60000/60000 [==============================] - 2s 37us/sample - loss: 0.4178 - accuracy: 0.8515 - val_loss: 0.4079 - val_accuracy: 0.8559\n",
            "Epoch 3/5\n",
            "60000/60000 [==============================] - 2s 37us/sample - loss: 0.3810 - accuracy: 0.8625 - val_loss: 0.3864 - val_accuracy: 0.8592\n",
            "Epoch 4/5\n",
            "60000/60000 [==============================] - 2s 36us/sample - loss: 0.3578 - accuracy: 0.8703 - val_loss: 0.3728 - val_accuracy: 0.8656\n",
            "Epoch 5/5\n",
            "60000/60000 [==============================] - 2s 37us/sample - loss: 0.3388 - accuracy: 0.8773 - val_loss: 0.3660 - val_accuracy: 0.8660\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7f5c35db07b8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4UAxuKOzmxCM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# save the weights\n",
        "model.save_weights('fashion_mnist_weights')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e1JbsSZZAPXy",
        "colab_type": "text"
      },
      "source": [
        "This will give birth to two files:\n",
        "- fashion_mnist_weights.data-00000-of-00001\n",
        "- fashion_mnist_weights.index"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "egqd7uhvx5Wd",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "d8b551d1-a7d3-48f6-827a-65180fb1ee55"
      },
      "source": [
        "# sample prediction\n",
        "class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
        "               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']\n",
        "test_img = plt.imread('test.png')\n",
        "prob = model.predict(test_img.reshape(1, 28, 28))\n",
        "print(class_names[prob.argmax()])"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Trouser\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CVtyUV5u-BSw",
        "colab_type": "text"
      },
      "source": [
        "The test image looks like the following, by the way:\n",
        "\n",
        "![](https://storage.googleapis.com/gweb-cloudblog-publish/images/Cloud_Storage_test_image.max-100x100.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wInfsoDw-NJP",
        "colab_type": "text"
      },
      "source": [
        "Once the model weights are saved we need to create a `.py` file named `main.py` as required by Cloud Functions. The `main.py` file should look like so:\n",
        "\n",
        "```python\n",
        "import numpy\n",
        "import tensorflow as tf\n",
        "\n",
        "from google.cloud import storage\n",
        "from tensorflow.keras.layers import *\n",
        "from tensorflow.keras.models import *\n",
        "from PIL import Image\n",
        "\n",
        "\n",
        "# we keep model as global variable so we don't have to reload it \n",
        "# in case of warm invocations\n",
        "model = None\n",
        "\n",
        "def get_me_the_model():\n",
        "    model = tf.keras.models.Sequential([\n",
        "      tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
        "      tf.keras.layers.Dense(128, activation='relu'),\n",
        "      tf.keras.layers.Dropout(0.2),\n",
        "      tf.keras.layers.Dense(10, activation='softmax')\n",
        "])\n",
        "\n",
        "    model.compile(optimizer='adam',\n",
        "              loss='sparse_categorical_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "    return model\n",
        "\n",
        "def download_blob(bucket_name, source_blob_name, destination_file_name):\n",
        "    \"\"\"downloads a blob from the bucket.\"\"\"\n",
        "    storage_client = storage.Client()\n",
        "    bucket = storage_client.get_bucket(bucket_name)\n",
        "    blob = bucket.blob(source_blob_name)\n",
        "\n",
        "    blob.download_to_filename(destination_file_name)\n",
        "\n",
        "    print('Blob {} downloaded to {}.'.format(\n",
        "        source_blob_name,\n",
        "        destination_file_name))\n",
        "\n",
        "def handler(request):\n",
        "    global model\n",
        "    class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
        "               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']\n",
        "\n",
        "    # model load which only happens during cold invocations\n",
        "    if model is None:\n",
        "        download_blob('<your_gs_buckets_name>', 'tensorflow/fashion_mnist_weights.index', '/tmp/fashion_mnist_weights.index')\n",
        "        download_blob('<your_gs_buckets_name>', 'tensorflow/fashion_mnist_weights.data-00000-of-00001', '/tmp/fashion_mnist_weights.data-00000-of-00001')\n",
        "        model = get_me_the_model()\n",
        "        model.load_weights('/tmp/fashion_mnist_weights')\n",
        "    \n",
        "    download_blob('<your_gs_buckets_name>', 'tensorflow/test.png', '/tmp/test.png')\n",
        "    image = numpy.array(Image.open('/tmp/test.png'))\n",
        "    input_np = (numpy.array(Image.open('/tmp/test.png'))/255)\n",
        "    input_np = input_np.reshape(1, 28, 28)\n",
        "    predictions = model.predict(input_np)\n",
        "    print(predictions)\n",
        "    print(\"Image is \"+class_names[numpy.argmax(predictions)])\n",
        "    \n",
        "    return class_names[numpy.argmax(predictions)]\n",
        "  ```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kxzhkJAK_w3A",
        "colab_type": "text"
      },
      "source": [
        "**Note** that in place of `<your_gs_buckets_name>` enter the bucket's name (without `gs://`) in which you have stored the model weights. Also note that, I have stored them in a folder named **tensorflow**. When the model is deployed as a cloud function, `main.py`will download the model from the storage bucket and will store it into **tmp** folder. \n",
        "\n",
        "Now to get started with the deployment process, first authenticate yourself. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LRTGladR4GV0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!gcloud auth login"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qKPeeELHBLj7",
        "colab_type": "text"
      },
      "source": [
        "Set the GCP project (preferably billing enabled)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bkdGl7wf4eJz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!gcloud config set project fast-ai-exploration"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WJ65TFI5BTlr",
        "colab_type": "text"
      },
      "source": [
        "And deploy!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ilzIcfKu4BRC",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 341
        },
        "outputId": "0db88533-9776-4512-b607-196b53858bdb"
      },
      "source": [
        "!gcloud functions deploy handler --runtime python37 --trigger-http --memory 2048\n",
        "!gcloud functions call handler"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "availableMemoryMb: 2048\n",
            "entryPoint: handler\n",
            "httpsTrigger:\n",
            "  url: https://us-central1-fast-ai-exploration.cloudfunctions.net/handler\n",
            "labels:\n",
            "  deployment-tool: cli-gcloud\n",
            "name: projects/fast-ai-exploration/locations/us-central1/functions/handler\n",
            "runtime: python37\n",
            "serviceAccountEmail: fast-ai-exploration@appspot.gserviceaccount.com\n",
            "sourceUploadUrl: https://storage.googleapis.com/gcf-upload-us-central1-9df9b855-cdbd-46fb-8ddd-787a7047f525/e641be16-f3ca-4d53-b188-8108a000fa07.zip?GoogleAccessId=service-29880397572@gcf-admin-robot.iam.gserviceaccount.com&Expires=1562864691&Signature=n%2F%2F4sB55%2FYRNY%2FpmXQ7PH1wrABZKwpz6XcOlDSBcsauPdciHebBywXv%2FKTvvV9O04tUT%2BrvjXvdfyweMSMcKtHyPYfyaBWIxxlPW89NHEZtzvgQ1l5dGHQv7BNdOqMkbnykFoNAyCbxr8oK5COJaOAXTRbrWR%2FRfVuo9FA6aJnNNqvQAwGbzmrlesyGqcZ0bYNMOOZf2MySjJ1cypDr8UNwuxls2HntAo0yLjyCpDvTynfHKyVlMwn1SegvxOixYZ14lpH%2B8rHfnjWCY9eyZ2cfjcVsx%2FBHZQNgXVT5tST0zA28V0b13Fh1Zkr%2B9EmwCzzVrLeugqmC%2BCZWuLIK%2B%2Fg%3D%3D\n",
            "status: ACTIVE\n",
            "timeout: 60s\n",
            "updateTime: '2019-07-11T16:37:58Z'\n",
            "versionId: '1'\n",
            "executionId: gpn5hqhg1rab\n",
            "result: Trouser\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AyB6f1h3BW52",
        "colab_type": "text"
      },
      "source": [
        "**Notice** that the function `handler()` in `main.py` internally calls the test image, so you don't need to worry about it. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gc9tgcm8BmJj",
        "colab_type": "text"
      },
      "source": [
        "**Reference**:\n",
        "- https://cloud.google.com/blog/products/ai-machine-learning/how-to-serve-deep-learning-models-using-tensorflow-2-0-with-cloud-functions"
      ]
    }
  ]
}