frameworks/tensorflow/Tensorflow_Quickstart_Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensorflow\n",
    "\n",
    "Tensorflow 為 Google 開發，採用資料流程圖，用於數值運算的開源深度學習框架。相較於其他深度學習框架，tensorflow 主要有高度靈活性、可攜性與多語言支援等特性。\n",
    "\n",
    "* 高度靈活性: 使用者可以封裝自訂的上層函式庫，亦可以撰寫 C++ 程式來豐富底層操作。\n",
    "* 可攜性: 可以在 CPU 與 GPU 裝置下混和調用來進行深度學習外，可以在不同的平台、雲端、容器或開發版(如 Raspberry Pi 等)等環境下執行。\n",
    "* 多語言支援: Tensorflow 提供多種程式語言 API，包含 Python、C++、Java、GO 或 Javascript 等。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reference\n",
    "\n",
    "* Official Website: https://www.tensorflow.org/\n",
    "* Github: https://github.com/tensorflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## NVidia Installation\n",
    "\n",
    "Tensorflow 可以自動調用 NVIDIA 顯示卡 (GPU) 來加速深度學習，底下是必要安裝組件。GPU 可用於加速深度學習，不僅是減少訓練時間，在測試與佈署產品時都能有效提升效率，但並非為深度學習所必備，相關內容已有相當多的資源可以參考，但最有趣的還是一影片可以讓你了解 CPU 與 GPU 在圖像上的計算差異，參考網址 https://www.youtube.com/watch?v=-P28LKWTzrI 。\n",
    "\n",
    "### 安裝組件\n",
    "\n",
    "1. 於 NVIDIA 官網 (http://www.nvidia.com.tw/Download/index.aspx) 下載目前最新的驅動程式。\n",
    "2. 下載並安裝 CUDA SDK (https://developer.nvidia.com/cuda-downloads)，CUDA 為 NVIDIA 提供的 SDK 可對顯示卡進行程式設計。\n",
    "3. 下載並安裝 cuDNN (https://developer.nvidia.com/rdp/cudnn-archive)，cuDNN 為一使用 CUDA 的深度神經網路函式庫。\n",
    "4. 設置底下環境變數\n",
    "\n",
    "```shell\n",
    "# windows example\n",
    "setx PATH \"C:\\Program Files\\NVIDIA Corporation\\NVSMI;%PATH%\"\n",
    "setx PATH \"C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v9.0\\bin;%PATH%\"\n",
    "setx PATH \"C:\\local\\cudnn-9.0-v7.0\\cuda\\bin;%PATH%\"\n",
    "```\n",
    "\n",
    "### 限制\n",
    "\n",
    "* 顯卡限制: 需要注意 Tensorflow 有最低支援的顯示卡要求，需要 compute capability >= 3.0 的顯卡，可以至官網 https://developer.nvidia.com/cuda-gpus 查詢 (此限制與其他深度學習框架，如 CNTK 相同)。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 安裝環境\n",
    "\n",
    "\n",
    "### 安裝 Anaconda\n",
    "\n",
    "Anaconda 為基於 Python 用於資料科學、機器學習等領域使用的 IDE，不僅提供如 jupyter notebook 線上互動開發環境、spyder 大量程式碼開發等環境，更已包含許多常用的工具 (如 pip) 與函式庫 (numpy, scipy) 等，相當適合想要進入人工智慧、資料科學等領域的人使用。可以至網頁 https://www.anaconda.com/download/ 下載安裝。\n",
    "\n",
    "建議安裝 python 3.5 以上的環境，此環境可以與其他深度學習框架併用，如 CNTK 等。可以透過底下方式檢驗是否已安裝 anaconda，\n",
    "\n",
    "```shell\n",
    "# 於 CLI 下，輸入 python 或 pip --version 等\n",
    "# 應出現如 Python 3.5.4 |Anaconda custom (64-bit) 或 pip 9.0.2 等資訊\n",
    "$ python\n",
    "$ pip --version\n",
    "```\n",
    "\n",
    "底下的操作皆以 Anaconda 及其相關工具作為主要 IDE 進行開發。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 安裝其他建議套件\n",
    "\n",
    "Tensorflow 會使用如 numpy, scipy, matplotlib, jupyter, scikit-image, librosa, nltk, keras, tflearn 或 pandas 等套件，需要確認是否皆已安裝，如下範例:\n",
    "\n",
    "| package_name | description |\n",
    "| -- | -- |\n",
    "| numpy | 用來儲存與處理大型矩陣的科學計算套件 |\n",
    "| scipy | 包含大量的數學工具包，如最佳化、線性代數、積分、訊號處理或圖像處理等 |\n",
    "| matplotlib | 繪圖函式庫 |\n",
    "| jupyter | Ipython 升級版，於瀏覽器中建立和共用程式，jupyter 本身便是一個 web 應用程式，使用 MQ 進行訊息管理 |\n",
    "| scikit-image (skimage, imported name) | 圖像處理演算法，利於影像前處理 |\n",
    "| librosa | 音訊特徵分析 |\n",
    "| nltk | 自然語言處理工作包 |\n",
    "| keras | tensorflow 中高階封裝的架構，利於快速建立模型 |\n",
    "| tflearn | 另一 tensorflow 中高階封裝的架構，利於快速建立模型 |\n",
    "| pandas | 提供高效資料結構與資料分析的函式庫 |\n",
    "\n",
    "可以透過指令 `pip list` 來確認是否有安裝上述函式庫，若無可以透過 `pip install <package> --upgrade` 指令來安裝。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Optional] 準備沙盒環境\n",
    "\n",
    "安裝 tensorflow 前，可以先透過 `virtualenv` 來建立獨立運行的 python 執行環境，之後可在沙盒裡建立 tensorflow，並可以依需求預設許多執行環境設定 (如同環境變數)，之後可以簡易透過不同沙盒來達成轉換執行環境，且可以確保不會影響外部 python 環境。\n",
    "\n",
    "可以先透過 pip 安裝 virtualenv，\n",
    "\n",
    "```shell\n",
    "$ pip install virtualenv --upgrade\n",
    "```\n",
    "\n",
    "可以於想要的位置建立沙盒環境，\n",
    "\n",
    "```shell\n",
    "$ cd ~\n",
    "$ virtualenv --system-site-package ~/tensorflow\n",
    "```\n",
    "\n",
    "可以進入該目錄，並啟動沙盒，\n",
    "\n",
    "```shell\n",
    "$ cd ~/tensorflow\n",
    "$ source bin/activate\n",
    "```\n",
    "\n",
    "之後便可以接續底下的 tensorflow 安裝，**需要注意的是此步驟並非絕對需要**，視個人狀況而定。另外亦須注意，**若將 tensorflow 建立於沙盒中，則之後每次啟動皆須先啟動沙盒**。\n",
    "\n",
    "### 安裝 Tensorflow\n",
    "\n",
    "可以參考網頁 https://www.tensorflow.org/install/ 來下載並安裝 tensorflow，須注意底下安裝為針對伺服器、筆記型電腦、桌上型電腦等安裝方法，CPU 架構屬於 x86_64, amd64 等硬體，而若屬於開發版如 ARM 架構等硬體，需從 Source Code 進行編譯與安裝，若甚至於下載非官方優化的 tensorflow 套件來安裝等。Tensorflow 與其他深度學習的框架相同，可分成 CPU 與 GPU 兩種函式庫，此須視硬體規格來決定，而 GPU 版則可以透過如 `tf.device(\"/cpu:0\")` 方式來切換使用 CPU 進行訓練。\n",
    "\n",
    "此外，若有需要安裝其他語言的 tensorflow 函式庫，如 Java (https://www.tensorflow.org/install/install_java)、Go (https://www.tensorflow.org/install/install_go) 等亦須參考其他網址來安裝。\n",
    "\n",
    "* [Optional] 創建 Python 3.x 的環境\n",
    "\n",
    "可以直接下載 python 3.5.x 的 anaconda 來安裝，或是直接下載原生 python 3.5.x 來安裝皆可使用。\n",
    "\n",
    "```shell\n",
    "# create a python 3.5 environment\n",
    "$ conda create -n tensorflow python=3.5\n",
    "```\n",
    "\n",
    "* 安裝 CPU 版本的核心函式庫\n",
    "\n",
    "可以透過 `conda` 或 `pip` 來安裝，安裝方式分別如下:\n",
    "\n",
    "```shell\n",
    "# install the tensorflow over conda\n",
    "$ conda install -c conda-forge tensorflow\n",
    "\n",
    "# install the tensorflow over pip\n",
    "$ pip install tensorflow\n",
    "```\n",
    "\n",
    "* 安裝 GPU 版本的核心函式庫，需要先安裝 CUDA SDK 與 cuDNN 套件\n",
    "\n",
    "```shell\n",
    "$ pip install tensorflow-gpu\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 更新 Tensorflow\n",
    "\n",
    "可以透過底下指令更新 tensorflow 函式庫\n",
    "\n",
    "```shell\n",
    "# CPU 版本\n",
    "$ pip install --ignore-installed --upgrade tensorflow \n",
    "\n",
    "# GPU 版本\n",
    "$ pip install --ignore-installed --upgrade tensorflow-gpu \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 安裝於其他環境\n",
    "\n",
    "### 安裝於 Java (透過 JDK 方式使用 Tensorflow)\n",
    "\n",
    "#### 安裝於 Linux 或 MacOS\n",
    "\n",
    "* 下載 **libtensorflow.jar** (https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-1.7.0.jar)。\n",
    "* 依需求下載 CPU 或 GPU 版本之 Java Native Interface (JNI)，如下以 CPU 為範例：\n",
    "\n",
    "```shell\n",
    "TF_TYPE=\"cpu\" # Default processor is CPU. If you want GPU, set to \"gpu\"\n",
    "OS=$(uname -s | tr '[:upper:]' '[:lower:]')\n",
    "mkdir -p ./jni\n",
    "curl -L \\\n",
    "    \"https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow_jni-${TF_TYPE}-${OS}-x86_64-1.7.0.tar.gz\" |\n",
    "    tar -xz -C ./jni\n",
    "```\n",
    "\n",
    "#### 安裝於 Windows\n",
    "\n",
    "* 下載 **libtensorflow.jar** (https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-1.7.0.jar)。\n",
    "* 下載 Java Native Interface (JNI): https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow_jni-cpu-windows-x86_64-1.7.0.zip ，並解壓縮出 `.jni`檔案。\n",
    "\n",
    "#### 驗證是否安裝成功\n",
    "\n",
    "* 將下列內容存成 `HelloTF.java` 檔案\n",
    "\n",
    "```java\n",
    "import org.tensorflow.Graph;\n",
    "import org.tensorflow.Session;\n",
    "import org.tensorflow.Tensor;\n",
    "import org.tensorflow.TensorFlow;\n",
    "\n",
    "public class HelloTF {\n",
    "  public static void main(String[] args) throws Exception {\n",
    "    try (Graph g = new Graph()) {\n",
    "      final String value = \"Hello from \" + TensorFlow.version();\n",
    "\n",
    "      // Construct the computation graph with a single operation, a constant\n",
    "      // named \"MyConst\" with a value \"value\".\n",
    "      try (Tensor t = Tensor.create(value.getBytes(\"UTF-8\"))) {\n",
    "        // The Java API doesn't yet include convenience functions for adding operations.\n",
    "        g.opBuilder(\"Const\", \"MyConst\").setAttr(\"dtype\", t.dataType()).setAttr(\"value\", t).build();\n",
    "      }\n",
    "\n",
    "      // Execute the \"MyConst\" operation in a Session.\n",
    "      try (Session s = new Session(g);\n",
    "           Tensor output = s.runner().fetch(\"MyConst\").run().get(0)) {\n",
    "        System.out.println(new String(output.bytesValue(), \"UTF-8\"));\n",
    "      }\n",
    "    }\n",
    "  }\n",
    "}\n",
    "```\n",
    "\n",
    "* 編譯此檔案\n",
    "\n",
    "```shell\n",
    "javac -cp libtensorflow-1.7.0.jar HelloTF.java\n",
    "```\n",
    "\n",
    "* 執行此編譯後檔案\n",
    "\n",
    "```shell\n",
    "# linux/macOS\n",
    "java -cp libtensorflow-1.7.0.jar:. -Djava.library.path=./jni HelloTF\n",
    "\n",
    "# windows\n",
    "# notice the jni represents the file path of tensorflow_jni.dll\n",
    "# java -cp libtensorflow-1.7.0.jar;. -Djava.library.path=jni HelloTF\n",
    "cd /path/to/jni\n",
    "java -cp libtensorflow-1.7.0.jar;. -Djava.library.path=.\\ HelloTF\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Tensorflow 快速導覽\n",
    "\n",
    "### 驗證 Tensorflow 安裝與版本"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.7.0\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 偵測 CPU/GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[name: \"/device:CPU:0\"\n",
       " device_type: \"CPU\"\n",
       " memory_limit: 268435456\n",
       " locality {\n",
       " }\n",
       " incarnation: 3003868865581207446]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check supported devices\n",
    "from tensorflow.python.client import device_lib\n",
    "device_lib.list_local_devices()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 第一個簡單範例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'Hello, Tensorflow!'\n"
     ]
    }
   ],
   "source": [
    "hello = tf.constant('Hello, Tensorflow!')\n",
    "sess = tf.Session()\n",
    "print(sess.run(hello))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensorflow 快速導覽\n",
    "\n",
    "底下為建立資料集 iris 的深度學習分類器的範例。\n",
    "\n",
    "### 資料內容\n",
    "\n",
    "第一列為 (總筆數),(資料特徵維度),(類別0:setosa),(類別1:versicolor),(類別2:virginica)\n",
    "\n",
    "第二列後 (sepal length),(sepal width),(petal length),(petal width),species (label)\n",
    "\n",
    "* 訓練資料集\n",
    "\n",
    "```text\n",
    "120,4,setosa,versicolor,virginica\n",
    "6.4,2.8,5.6,2.2,2\n",
    "5.0,2.3,3.3,1.0,1\n",
    "4.9,2.5,4.5,1.7,2\n",
    "4.9,3.1,1.5,0.1,0\n",
    "5.7,3.8,1.7,0.3,0\n",
    "4.4,3.2,1.3,0.2,0\n",
    "5.4,3.4,1.5,0.4,0\n",
    "```\n",
    "\n",
    "* 測試資料集\n",
    "\n",
    "```text\n",
    "30,4,setosa,versicolor,virginica\n",
    "5.9,3.0,4.2,1.5,1\n",
    "6.9,3.1,5.4,2.1,2\n",
    "5.1,3.3,1.7,0.5,0\n",
    "6.0,3.4,4.5,1.6,1\n",
    "5.5,2.5,4.0,1.3,1\n",
    "6.2,2.9,4.3,1.3,1\n",
    "5.5,4.2,1.4,0.2,0\n",
    "6.3,2.8,5.1,1.5,2\n",
    "5.6,3.0,4.1,1.3,1\n",
    "```\n",
    "\n",
    "### 資料準備"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import tensorflow as tf\n",
    "\n",
    "TRAIN_URL = \"http://download.tensorflow.org/data/iris_training.csv\"\n",
    "TEST_URL = \"http://download.tensorflow.org/data/iris_test.csv\"\n",
    "\n",
    "CSV_COLUMN_NAMES = ['SepalLength', 'SepalWidth',\n",
    "                    'PetalLength', 'PetalWidth', 'Species']\n",
    "SPECIES = ['Setosa', 'Versicolor', 'Virginica']\n",
    "\n",
    "def maybe_download():\n",
    "    train_path = tf.keras.utils.get_file(TRAIN_URL.split('/')[-1], TRAIN_URL)\n",
    "    test_path = tf.keras.utils.get_file(TEST_URL.split('/')[-1], TEST_URL)\n",
    "\n",
    "    return train_path, test_path\n",
    "\n",
    "def load_data(y_name='Species'):\n",
    "    \"\"\"Returns the iris dataset as (train_x, train_y), (test_x, test_y).\"\"\"\n",
    "    train_path, test_path = maybe_download()\n",
    "\n",
    "    train = pd.read_csv(train_path, names=CSV_COLUMN_NAMES, header=0)\n",
    "    train_x, train_y = train, train.pop(y_name)\n",
    "\n",
    "    test = pd.read_csv(test_path, names=CSV_COLUMN_NAMES, header=0)\n",
    "    test_x, test_y = test, test.pop(y_name)\n",
    "\n",
    "    return (train_x, train_y), (test_x, test_y)\n",
    "\n",
    "\n",
    "def train_input_fn(features, labels, batch_size):\n",
    "    \"\"\"An input function for training\"\"\"\n",
    "    # Convert the inputs to a Dataset.\n",
    "    dataset = tf.data.Dataset.from_tensor_slices((dict(features), labels))\n",
    "\n",
    "    # Shuffle, repeat, and batch the examples.\n",
    "    dataset = dataset.shuffle(1000).repeat().batch(batch_size)\n",
    "\n",
    "    # Return the dataset.\n",
    "    return dataset\n",
    "\n",
    "\n",
    "def eval_input_fn(features, labels, batch_size):\n",
    "    \"\"\"An input function for evaluation or prediction\"\"\"\n",
    "    features=dict(features)\n",
    "    if labels is None:\n",
    "        # No labels, use only features.\n",
    "        inputs = features\n",
    "    else:\n",
    "        inputs = (features, labels)\n",
    "\n",
    "    # Convert the inputs to a Dataset.\n",
    "    dataset = tf.data.Dataset.from_tensor_slices(inputs)\n",
    "\n",
    "    # Batch the examples\n",
    "    assert batch_size is not None, \"batch_size must not be None\"\n",
    "    dataset = dataset.batch(batch_size)\n",
    "\n",
    "    # Return the dataset.\n",
    "    return dataset\n",
    "\n",
    "\n",
    "# The remainder of this file contains a simple example of a csv parser,\n",
    "#     implemented using a the `Dataset` class.\n",
    "\n",
    "# `tf.parse_csv` sets the types of the outputs to match the examples given in\n",
    "#     the `record_defaults` argument.\n",
    "CSV_TYPES = [[0.0], [0.0], [0.0], [0.0], [0]]\n",
    "\n",
    "def _parse_line(line):\n",
    "    # Decode the line into its fields\n",
    "    fields = tf.decode_csv(line, record_defaults=CSV_TYPES)\n",
    "\n",
    "    # Pack the result into a dictionary\n",
    "    features = dict(zip(CSV_COLUMN_NAMES, fields))\n",
    "\n",
    "    # Separate the label from the features\n",
    "    label = features.pop('Species')\n",
    "\n",
    "    return features, label\n",
    "\n",
    "\n",
    "def csv_input_fn(csv_path, batch_size):\n",
    "    # Create a dataset containing the text lines.\n",
    "    dataset = tf.data.TextLineDataset(csv_path).skip(1)\n",
    "\n",
    "    # Parse each line.\n",
    "    dataset = dataset.map(_parse_line)\n",
    "\n",
    "    # Shuffle, repeat, and batch the examples.\n",
    "    dataset = dataset.shuffle(1000).repeat().batch(batch_size)\n",
    "\n",
    "    # Return the dataset.\n",
    "    return dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensorflow 模型\n",
    "\n",
    "建立一個 DNN 網路由 4 層組成，第 1 層為輸入層，第 2 與 3 層為隱藏層，第 4 層為輸出層，隱藏層各有 10 節點，輸出層輸出 3 個類別，分別代表 setosa, versicolor, virginica。每次用 100 `batch_size` 資料來做訓練，共進行 1000 次循環 (epochs, `train_steps`)。透過 `train` 函式對子資料進行訓練，透過 evaluate 函式對測試資料進行驗證。產生 3 筆各 4 維特徵的資料，並透過剛訓練的分類器進行預測。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using default config.\n",
      "WARNING:tensorflow:Using temporary folder as model directory: C:\\Users\\ACER47~1\\AppData\\Local\\Temp\\tmpi90xy9wu\n",
      "INFO:tensorflow:Using config: {'_session_config': None, '_keep_checkpoint_max': 5, '_is_chief': True, '_save_checkpoints_secs': 600, '_task_id': 0, '_save_checkpoints_steps': None, '_model_dir': 'C:\\\\Users\\\\ACER47~1\\\\AppData\\\\Local\\\\Temp\\\\tmpi90xy9wu', '_num_ps_replicas': 0, '_keep_checkpoint_every_n_hours': 10000, '_save_summary_steps': 100, '_tf_random_seed': None, '_service': None, '_evaluation_master': '', '_master': '', '_num_worker_replicas': 1, '_task_type': 'worker', '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x0000021A1F5EFE48>, '_global_id_in_cluster': 0, '_log_step_count_steps': 100}\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 1 into C:\\Users\\ACER47~1\\AppData\\Local\\Temp\\tmpi90xy9wu\\model.ckpt.\n",
      "INFO:tensorflow:step = 1, loss = 173.6498\n",
      "INFO:tensorflow:global_step/sec: 426.178\n",
      "INFO:tensorflow:step = 101, loss = 7.374618 (0.235 sec)\n",
      "INFO:tensorflow:global_step/sec: 697.929\n",
      "INFO:tensorflow:step = 201, loss = 11.638523 (0.143 sec)\n",
      "INFO:tensorflow:global_step/sec: 670.489\n",
      "INFO:tensorflow:step = 301, loss = 6.9025927 (0.149 sec)\n",
      "INFO:tensorflow:global_step/sec: 680.985\n",
      "INFO:tensorflow:step = 401, loss = 6.911475 (0.147 sec)\n",
      "INFO:tensorflow:global_step/sec: 697.494\n",
      "INFO:tensorflow:step = 501, loss = 2.9322293 (0.147 sec)\n",
      "INFO:tensorflow:global_step/sec: 396.808\n",
      "INFO:tensorflow:step = 601, loss = 6.8326483 (0.252 sec)\n",
      "INFO:tensorflow:global_step/sec: 694.027\n",
      "INFO:tensorflow:step = 701, loss = 6.7185106 (0.140 sec)\n",
      "INFO:tensorflow:global_step/sec: 697.171\n",
      "INFO:tensorflow:step = 801, loss = 7.591189 (0.153 sec)\n",
      "INFO:tensorflow:global_step/sec: 658.597\n",
      "INFO:tensorflow:step = 901, loss = 6.139774 (0.142 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 1000 into C:\\Users\\ACER47~1\\AppData\\Local\\Temp\\tmpi90xy9wu\\model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 3.931043.\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-04-19-06:33:14\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from C:\\Users\\ACER47~1\\AppData\\Local\\Temp\\tmpi90xy9wu\\model.ckpt-1000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Finished evaluation at 2018-04-19-06:33:14\n",
      "INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.93333334, average_loss = 0.05827907, global_step = 1000, loss = 1.7483721\n",
      "\n",
      "Test set accuracy: 0.933\n",
      "\n",
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from C:\\Users\\ACER47~1\\AppData\\Local\\Temp\\tmpi90xy9wu\\model.ckpt-1000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "\n",
      "Prediction is \"Setosa\" (99.9%), expected \"Setosa\"\n",
      "\n",
      "Prediction is \"Versicolor\" (99.8%), expected \"Versicolor\"\n",
      "\n",
      "Prediction is \"Virginica\" (98.8%), expected \"Virginica\"\n"
     ]
    }
   ],
   "source": [
    "\"\"\"An Example of a DNNClassifier for the Iris dataset.\"\"\"\n",
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "\n",
    "import argparse\n",
    "import tensorflow as tf\n",
    "\n",
    "# save the above code into the file\n",
    "# and put it on the same directory with this notebook\n",
    "import iris_data\n",
    "\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('--batch_size', default=100, type=int, help='batch size')\n",
    "parser.add_argument('--train_steps', default=1000, type=int, help='number of training steps')\n",
    "\n",
    "def main(argv):\n",
    "    args = parser.parse_args(argv[1:])\n",
    "\n",
    "    # Fetch the data\n",
    "    (train_x, train_y), (test_x, test_y) = iris_data.load_data()\n",
    "\n",
    "    # Feature columns describe how to use the input.\n",
    "    my_feature_columns = []\n",
    "    for key in train_x.keys():\n",
    "        my_feature_columns.append(tf.feature_column.numeric_column(key=key))\n",
    "\n",
    "    # Build 2 hidden layer DNN with 10, 10 units respectively.\n",
    "    classifier = tf.estimator.DNNClassifier(\n",
    "        feature_columns=my_feature_columns,\n",
    "        # Two hidden layers of 10 nodes each.\n",
    "        hidden_units=[10, 10],\n",
    "        # The model must choose between 3 classes.\n",
    "        n_classes=3)\n",
    "\n",
    "    # Train the Model.\n",
    "    classifier.train(\n",
    "        input_fn=lambda:iris_data.train_input_fn(train_x, train_y, args.batch_size),\n",
    "        steps=args.train_steps)\n",
    "\n",
    "    # Evaluate the model.\n",
    "    eval_result = classifier.evaluate(\n",
    "        input_fn=lambda:iris_data.eval_input_fn(test_x, test_y, args.batch_size))\n",
    "\n",
    "    print('\\nTest set accuracy: {accuracy:0.3f}\\n'.format(**eval_result))\n",
    "\n",
    "    # Generate predictions from the model\n",
    "    expected = ['Setosa', 'Versicolor', 'Virginica']\n",
    "    predict_x = {\n",
    "        'SepalLength': [5.1, 5.9, 6.9],\n",
    "        'SepalWidth': [3.3, 3.0, 3.1],\n",
    "        'PetalLength': [1.7, 4.2, 5.4],\n",
    "        'PetalWidth': [0.5, 1.5, 2.1],\n",
    "    }\n",
    "\n",
    "    predictions = classifier.predict(\n",
    "        input_fn=lambda:iris_data.eval_input_fn(predict_x,\n",
    "                                                labels=None,\n",
    "                                                batch_size=args.batch_size))\n",
    "\n",
    "    template = ('\\nPrediction is \"{}\" ({:.1f}%), expected \"{}\"')\n",
    "\n",
    "    for pred_dict, expec in zip(predictions, expected):\n",
    "        class_id = pred_dict['class_ids'][0]\n",
    "        probability = pred_dict['probabilities'][class_id]\n",
    "\n",
    "        print(template.format(iris_data.SPECIES[class_id], 100 * probability, expec))\n",
    "\n",
    "# run the code        \n",
    "main([])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}