Metadata.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Metadata"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yUuWJAzeQPK9"
   },
   "source": [
    " <a href=\"https://colab.research.google.com/github/joerg84/Graph_Powered_ML_Workshop/blob/master/Metadata.ipynb\">\n",
    "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/> \n",
    "</a>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "or4HsszsQPK_"
   },
   "source": [
    "##### Copyright 2019 The TensorFlow Authors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pAHJZoiBQPLA"
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zapkQrLnQPLW"
   },
   "source": [
    "# ArangoML Pipeline Cloud\n",
    "The purpose of this notebook is to show how easy it is to drop in ArangoML Pipeline to your pre-existing Machine Learning workflows.\n",
    "\n",
    "We took the simplest existing example of TensorFlow, their beginner's notebook, and simply dropped in our pipeline to capture and store metadata.\n",
    "\n",
    "If you would like to continue learning about ArangoML and the managed metadata pipeline read our release post https://www.arangodb.com/2020/01/arangoml-pipeline-cloud-manage-machine-learning-metadata/\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9E5w9666QPLX"
   },
   "source": [
    "# TensorFlow 2 quickstart for beginners"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "eOAQLYOyQPLX"
   },
   "source": [
    "This short introduction uses [Keras](https://www.tensorflow.org/guide/keras/overview) to:\n",
    "\n",
    "1. Build a neural network that classifies images.\n",
    "2. Train this neural network.\n",
    "3. And, finally, evaluate the accuracy of the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AaAI7GObQPLY"
   },
   "source": [
    "This is a [Google Colaboratory](https://colab.research.google.com/notebooks/welcome.ipynb) notebook file. Python programs are run directly in the browser—a great way to learn and use TensorFlow. To follow this tutorial, run the notebook in Google Colab by clicking the button at the top of this page.\n",
    "\n",
    "1. In Colab, connect to a Python runtime: At the top-right of the menu bar, select *CONNECT*.\n",
    "2. Run all the notebook code cells: Select *Runtime* > *Run all*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "I6OW0iDhQPLY"
   },
   "source": [
    "# Install Required Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WzaldCfiQPLZ"
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "!pip3 install python-arango\n",
    "!pip3 install arangopipe==0.0.6.9.3\n",
    "!pip3 install pandas PyYAML==5.1.1 sklearn2\n",
    "!pip3 install json-tricks "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1dsEKJH-QPLc"
   },
   "source": [
    "Download and install the TensorFlow 2 package. Import TensorFlow into your program:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7L40moFHQPLc"
   },
   "outputs": [],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "\n",
    "# Install TensorFlow\n",
    "try:\n",
    "  # %tensorflow_version only exists in Colab.\n",
    "  %tensorflow_version 2.x\n",
    "except Exception:\n",
    "  pass\n",
    "\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mqP9osowQPLf"
   },
   "source": [
    "# Initial Connection to a Managed Service ArangoPipe Database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7I3ve7wVQPLg"
   },
   "outputs": [],
   "source": [
    "from arangopipe.arangopipe_storage.arangopipe_api import ArangoPipe\n",
    "from arangopipe.arangopipe_storage.arangopipe_admin_api import ArangoPipeAdmin\n",
    "from arangopipe.arangopipe_storage.arangopipe_config import ArangoPipeConfig\n",
    "from arangopipe.arangopipe_storage.managed_service_conn_parameters import ManagedServiceConnParam\n",
    "mdb_config = ArangoPipeConfig()\n",
    "msc = ManagedServiceConnParam()\n",
    "conn_params = { msc.DB_SERVICE_HOST : \"arangoml.arangodb.cloud\", \\\n",
    "                        msc.DB_SERVICE_END_POINT : \"createDB\",\\\n",
    "                        msc.DB_SERVICE_NAME : \"createDB\",\\\n",
    "                        msc.DB_SERVICE_PORT : 8529,\n",
    "                        msc.DB_CONN_PROTOCOL : 'https'}\n",
    "        \n",
    "mdb_config = mdb_config.create_connection_config(conn_params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vxDDXEXYQPLj"
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "admin = ArangoPipeAdmin(reuse_connection = False, config = mdb_config)\n",
    "ap_config = admin.get_config()\n",
    "ap = ArangoPipe(config = ap_config)\n",
    "# Error indicating \"heart beat check was not found\" is expected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OwF9yYJYQPLm"
   },
   "source": [
    "Load and prepare the [MNIST dataset](http://yann.lecun.com/exdb/mnist/). Convert the samples from integers to floating-point numbers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tv6GqMuRQPLm"
   },
   "outputs": [],
   "source": [
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "x_train, x_test = x_train / 255.0, x_test / 255.0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "q5ksf8uQQPLq"
   },
   "source": [
    "Set identifying metadata information for this project. \\\n",
    "This includes project name, dataset, featureset, and model information.\\\n",
    "This information is then registered and stored."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "p4gQlUCQQPLq"
   },
   "outputs": [],
   "source": [
    "proj_info = {\"name\": \"MNIST Handwriting Analysis\"}\n",
    "proj_reg = admin.register_project(proj_info)\n",
    "\n",
    "ds_info = {\"name\" : \"MNIST dataset\",\\\n",
    "           \"description\": \"Classification task pertaining to classifiying the digit in an iamge\" }\n",
    "ds_reg = ap.register_dataset(ds_info)\n",
    "\n",
    "featureset = {'name': 'MNIST digits',\n",
    "              'description': '28 x 28 pixel images with a label'}\n",
    "fs_reg = ap.register_featureset(featureset, ds_reg[\"_key\"])\n",
    "\n",
    "model_info = {\"name\": \"Neural Network\",\\\n",
    "              \"type\": \"Neural network with Linear layer, ReLU activation, Dropout Layer (20%) and Softmax output layer\"}\n",
    "model_reg = ap.register_model(model_info, project = \"MNIST Handwriting Analysis\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fNxA4_J2QPLt"
   },
   "source": [
    "Build the `tf.keras.Sequential` model by stacking layers. Choose an optimizer and loss function for training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wUK1niqIQPLt"
   },
   "outputs": [],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "  tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dropout(0.2),\n",
    "  tf.keras.layers.Dense(10, activation='softmax')\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "i3q5J0x3QPLv"
   },
   "source": [
    "Train and evaluate the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "C969ta1rQPLw"
   },
   "outputs": [],
   "source": [
    "import uuid #used as run id\n",
    "from datetime import datetime\n",
    "model.fit(x_train, y_train, epochs=5)\n",
    "\n",
    "\n",
    "# Values for any important model parameters and to store performance results.\n",
    "ruuid = uuid.uuid4()\n",
    "\n",
    "# current date and time\n",
    "now = datetime.now()\n",
    "timestamp = datetime.timestamp(now)\n",
    "\n",
    "model_params = {\"run_id\": str(ruuid)}\n",
    "(loss), (accuracy) = model.evaluate(x_test,  y_test, verbose=2)\n",
    "print(\"model loss %.2f , model accuracy %.2f\" % (loss, accuracy))\n",
    "model_perf = {\"loss\": str(loss),\n",
    "              \"accuracy\": str(accuracy),\n",
    "              \"run_id\": str(ruuid),\n",
    "              \"timestamp\": timestamp}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ltD6O5StQPL0"
   },
   "outputs": [],
   "source": [
    "model_json = model.to_json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vj5qvfD4QPL3"
   },
   "outputs": [],
   "source": [
    "from json_tricks import dumps\n",
    "weights = model.get_weights()\n",
    "json_weights = dumps(weights)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "L9gzZgs0QPL5"
   },
   "outputs": [],
   "source": [
    "model_params['json_weights'] = json_weights\n",
    "model_params['model_json'] = model_json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Zo_KgqceQPL7"
   },
   "outputs": [],
   "source": [
    "  run_info = {\"dataset\" : ds_reg[\"_key\"],\\\n",
    "              \"featureset\": fs_reg[\"_key\"],\\\n",
    "              \"run_id\": str(ruuid),\\\n",
    "              \"model\": model_reg[\"_key\"],\\\n",
    "              \"model-params\": model_params,\\\n",
    "              \"model-perf\": model_perf,\\\n",
    "              \"pipeline\" : \"Handwriting-Analysis-Pipeline\",\\\n",
    "              \"tag\": \"MNIST_model_params_saved\",\\\n",
    "              \"project\": \"MNIST Handwriting Analysis\"}\n",
    "\n",
    "  ap.log_run(run_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zacH9rLeQPL9"
   },
   "source": [
    "# Reusing the Previous Connection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cTI_-RRcQPL-"
   },
   "source": [
    "In a subsequent session you can reuse the connection you created previously using the snippet shown below. Note that you are not providing connection information during this interaction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1iSziJR3QPL-"
   },
   "outputs": [],
   "source": [
    "admin = ArangoPipeAdmin()  \n",
    "ap_config = admin.get_config()\n",
    "ap = ArangoPipe(config = ap_config)\n",
    "# Error indicating \"heart beat check was not found\" is expected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MDUYdQtWQPMB"
   },
   "source": [
    "Look up the model you stored in the database with the previous connection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "FdEo_FQVQPMB"
   },
   "outputs": [],
   "source": [
    "ap.lookup_model(\"Neural Network\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bA93OpkaQPMD"
   },
   "source": [
    "The image classifier is now trained to ~98% accuracy on this dataset. To learn more, read the [TensorFlow tutorials](https://www.tensorflow.org/tutorials/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "SBrFp_sVQPME"
   },
   "source": [
    "## Recreate a model from persisted state"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VhintBMFQPME"
   },
   "outputs": [],
   "source": [
    "saved_model_params = ap.lookup_modelparams(tag_value = \"MNIST_model_params_saved\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "as7nJKIjQPMH"
   },
   "outputs": [],
   "source": [
    "saved_model = saved_model_params['model_json']\n",
    "saved_model_weights = saved_model_params['json_weights']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yC1P7dJ1QPMJ"
   },
   "outputs": [],
   "source": [
    "mdb_config.cfg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DlbttnwCQPMK"
   },
   "outputs": [],
   "source": [
    "from json_tricks import loads\n",
    "remat_weight = loads(saved_model_weights)\n",
    "reinitialized_model = tf.keras.models.model_from_json(saved_model)\n",
    "reinitialized_model.set_weights(remat_weight)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WSYgWDE9QPMM"
   },
   "source": [
    "## Compare the predictions of the old and new model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "SiLrLmqXQPMN"
   },
   "outputs": [],
   "source": [
    "new_predictions = reinitialized_model.predict(x_test)\n",
    "old_predictions = model.predict(x_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5cJjuOOKQPMP"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "np.array_equal(new_predictions, old_predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DyM3Nd8GQPMT"
   },
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "name": "Arangopipe_with_TensorFlow_Beginner_Guide.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}