assignment7-04.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "assignment7.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyOTYTkX4jgfZzfuxNTBBL1l",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/dlguswn3659/class-MachineLearning/blob/master/assignment7-04.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pMcbWdbh98uY",
        "colab_type": "text"
      },
      "source": [
        "# **Machine Learning Assignment07** #\n",
        "\n",
        "Name : 이현주\n",
        "\n",
        "Student # : 20184060"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SmK4D4Gm-Owm",
        "colab_type": "text"
      },
      "source": [
        "# **1. Plot the training data [1pt]**\n",
        "\n",
        "- plot the training data points (x,y) with their labels / in colors (blue for label 0 and red for label 1)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b6EhbyxXNp1e",
        "colab_type": "code",
        "colab": {
          "resources": {
            "http://localhost:8080/nbextensions/google.colab/files.js": {
              "data": "Ly8gQ29weXJpZ2h0IDIwMTcgR29vZ2xlIExMQwovLwovLyBMaWNlbnNlZCB1bmRlciB0aGUgQXBhY2hlIExpY2Vuc2UsIFZlcnNpb24gMi4wICh0aGUgIkxpY2Vuc2UiKTsKLy8geW91IG1heSBub3QgdXNlIHRoaXMgZmlsZSBleGNlcHQgaW4gY29tcGxpYW5jZSB3aXRoIHRoZSBMaWNlbnNlLgovLyBZb3UgbWF5IG9idGFpbiBhIGNvcHkgb2YgdGhlIExpY2Vuc2UgYXQKLy8KLy8gICAgICBodHRwOi8vd3d3LmFwYWNoZS5vcmcvbGljZW5zZXMvTElDRU5TRS0yLjAKLy8KLy8gVW5sZXNzIHJlcXVpcmVkIGJ5IGFwcGxpY2FibGUgbGF3IG9yIGFncmVlZCB0byBpbiB3cml0aW5nLCBzb2Z0d2FyZQovLyBkaXN0cmlidXRlZCB1bmRlciB0aGUgTGljZW5zZSBpcyBkaXN0cmlidXRlZCBvbiBhbiAiQVMgSVMiIEJBU0lTLAovLyBXSVRIT1VUIFdBUlJBTlRJRVMgT1IgQ09ORElUSU9OUyBPRiBBTlkgS0lORCwgZWl0aGVyIGV4cHJlc3Mgb3IgaW1wbGllZC4KLy8gU2VlIHRoZSBMaWNlbnNlIGZvciB0aGUgc3BlY2lmaWMgbGFuZ3VhZ2UgZ292ZXJuaW5nIHBlcm1pc3Npb25zIGFuZAovLyBsaW1pdGF0aW9ucyB1bmRlciB0aGUgTGljZW5zZS4KCi8qKgogKiBAZmlsZW92ZXJ2aWV3IEhlbHBlcnMgZm9yIGdvb2dsZS5jb2xhYiBQeXRob24gbW9kdWxlLgogKi8KKGZ1bmN0aW9uKHNjb3BlKSB7CmZ1bmN0aW9uIHNwYW4odGV4dCwgc3R5bGVBdHRyaWJ1dGVzID0ge30pIHsKICBjb25zdCBlbGVtZW50ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnc3BhbicpOwogIGVsZW1lbnQudGV4dENvbnRlbnQgPSB0ZXh0OwogIGZvciAoY29uc3Qga2V5IG9mIE9iamVjdC5rZXlzKHN0eWxlQXR0cmlidXRlcykpIHsKICAgIGVsZW1lbnQuc3R5bGVba2V5XSA9IHN0eWxlQXR0cmlidXRlc1trZXldOwogIH0KICByZXR1cm4gZWxlbWVudDsKfQoKLy8gTWF4IG51bWJlciBvZiBieXRlcyB3aGljaCB3aWxsIGJlIHVwbG9hZGVkIGF0IGEgdGltZS4KY29uc3QgTUFYX1BBWUxPQURfU0laRSA9IDEwMCAqIDEwMjQ7Ci8vIE1heCBhbW91bnQgb2YgdGltZSB0byBibG9jayB3YWl0aW5nIGZvciB0aGUgdXNlci4KY29uc3QgRklMRV9DSEFOR0VfVElNRU9VVF9NUyA9IDMwICogMTAwMDsKCmZ1bmN0aW9uIF91cGxvYWRGaWxlcyhpbnB1dElkLCBvdXRwdXRJZCkgewogIGNvbnN0IHN0ZXBzID0gdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKTsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIC8vIENhY2hlIHN0ZXBzIG9uIHRoZSBvdXRwdXRFbGVtZW50IHRvIG1ha2UgaXQgYXZhaWxhYmxlIGZvciB0aGUgbmV4dCBjYWxsCiAgLy8gdG8gdXBsb2FkRmlsZXNDb250aW51ZSBmcm9tIFB5dGhvbi4KICBvdXRwdXRFbGVtZW50LnN0ZXBzID0gc3RlcHM7CgogIHJldHVybiBfdXBsb2FkRmlsZXNDb250aW51ZShvdXRwdXRJZCk7Cn0KCi8vIFRoaXMgaXMgcm91Z2hseSBhbiBhc3luYyBnZW5lcmF0b3IgKG5vdCBzdXBwb3J0ZWQgaW4gdGhlIGJyb3dzZXIgeWV0KSwKLy8gd2hlcmUgdGhlcmUgYXJlIG11bHRpcGxlIGFzeW5jaHJvbm91cyBzdGVwcyBhbmQgdGhlIFB5dGhvbiBzaWRlIGlzIGdvaW5nCi8vIHRvIHBvbGwgZm9yIGNvbXBsZXRpb24gb2YgZWFjaCBzdGVwLgovLyBUaGlzIHVzZXMgYSBQcm9taXNlIHRvIGJsb2NrIHRoZSBweXRob24gc2lkZSBvbiBjb21wbGV0aW9uIG9mIGVhY2ggc3RlcCwKLy8gdGhlbiBwYXNzZXMgdGhlIHJlc3VsdCBvZiB0aGUgcHJldmlvdXMgc3RlcCBhcyB0aGUgaW5wdXQgdG8gdGhlIG5leHQgc3RlcC4KZnVuY3Rpb24gX3VwbG9hZEZpbGVzQ29udGludWUob3V0cHV0SWQpIHsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIGNvbnN0IHN0ZXBzID0gb3V0cHV0RWxlbWVudC5zdGVwczsKCiAgY29uc3QgbmV4dCA9IHN0ZXBzLm5leHQob3V0cHV0RWxlbWVudC5sYXN0UHJvbWlzZVZhbHVlKTsKICByZXR1cm4gUHJvbWlzZS5yZXNvbHZlKG5leHQudmFsdWUucHJvbWlzZSkudGhlbigodmFsdWUpID0+IHsKICAgIC8vIENhY2hlIHRoZSBsYXN0IHByb21pc2UgdmFsdWUgdG8gbWFrZSBpdCBhdmFpbGFibGUgdG8gdGhlIG5leHQKICAgIC8vIHN0ZXAgb2YgdGhlIGdlbmVyYXRvci4KICAgIG91dHB1dEVsZW1lbnQubGFzdFByb21pc2VWYWx1ZSA9IHZhbHVlOwogICAgcmV0dXJuIG5leHQudmFsdWUucmVzcG9uc2U7CiAgfSk7Cn0KCi8qKgogKiBHZW5lcmF0b3IgZnVuY3Rpb24gd2hpY2ggaXMgY2FsbGVkIGJldHdlZW4gZWFjaCBhc3luYyBzdGVwIG9mIHRoZSB1cGxvYWQKICogcHJvY2Vzcy4KICogQHBhcmFtIHtzdHJpbmd9IGlucHV0SWQgRWxlbWVudCBJRCBvZiB0aGUgaW5wdXQgZmlsZSBwaWNrZXIgZWxlbWVudC4KICogQHBhcmFtIHtzdHJpbmd9IG91dHB1dElkIEVsZW1lbnQgSUQgb2YgdGhlIG91dHB1dCBkaXNwbGF5LgogKiBAcmV0dXJuIHshSXRlcmFibGU8IU9iamVjdD59IEl0ZXJhYmxlIG9mIG5leHQgc3RlcHMuCiAqLwpmdW5jdGlvbiogdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKSB7CiAgY29uc3QgaW5wdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoaW5wdXRJZCk7CiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gZmFsc2U7CgogIGNvbnN0IG91dHB1dEVsZW1lbnQgPSBkb2N1bWVudC5nZXRFbGVtZW50QnlJZChvdXRwdXRJZCk7CiAgb3V0cHV0RWxlbWVudC5pbm5lckhUTUwgPSAnJzsKCiAgY29uc3QgcGlja2VkUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBpbnB1dEVsZW1lbnQuYWRkRXZlbnRMaXN0ZW5lcignY2hhbmdlJywgKGUpID0+IHsKICAgICAgcmVzb2x2ZShlLnRhcmdldC5maWxlcyk7CiAgICB9KTsKICB9KTsKCiAgY29uc3QgY2FuY2VsID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnYnV0dG9uJyk7CiAgaW5wdXRFbGVtZW50LnBhcmVudEVsZW1lbnQuYXBwZW5kQ2hpbGQoY2FuY2VsKTsKICBjYW5jZWwudGV4dENvbnRlbnQgPSAnQ2FuY2VsIHVwbG9hZCc7CiAgY29uc3QgY2FuY2VsUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBjYW5jZWwub25jbGljayA9ICgpID0+IHsKICAgICAgcmVzb2x2ZShudWxsKTsKICAgIH07CiAgfSk7CgogIC8vIENhbmNlbCB1cGxvYWQgaWYgdXNlciBoYXNuJ3QgcGlja2VkIGFueXRoaW5nIGluIHRpbWVvdXQuCiAgY29uc3QgdGltZW91dFByb21pc2UgPSBuZXcgUHJvbWlzZSgocmVzb2x2ZSkgPT4gewogICAgc2V0VGltZW91dCgoKSA9PiB7CiAgICAgIHJlc29sdmUobnVsbCk7CiAgICB9LCBGSUxFX0NIQU5HRV9USU1FT1VUX01TKTsKICB9KTsKCiAgLy8gV2FpdCBmb3IgdGhlIHVzZXIgdG8gcGljayB0aGUgZmlsZXMuCiAgY29uc3QgZmlsZXMgPSB5aWVsZCB7CiAgICBwcm9taXNlOiBQcm9taXNlLnJhY2UoW3BpY2tlZFByb21pc2UsIHRpbWVvdXRQcm9taXNlLCBjYW5jZWxQcm9taXNlXSksCiAgICByZXNwb25zZTogewogICAgICBhY3Rpb246ICdzdGFydGluZycsCiAgICB9CiAgfTsKCiAgaWYgKCFmaWxlcykgewogICAgcmV0dXJuIHsKICAgICAgcmVzcG9uc2U6IHsKICAgICAgICBhY3Rpb246ICdjb21wbGV0ZScsCiAgICAgIH0KICAgIH07CiAgfQoKICBjYW5jZWwucmVtb3ZlKCk7CgogIC8vIERpc2FibGUgdGhlIGlucHV0IGVsZW1lbnQgc2luY2UgZnVydGhlciBwaWNrcyBhcmUgbm90IGFsbG93ZWQuCiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gdHJ1ZTsKCiAgZm9yIChjb25zdCBmaWxlIG9mIGZpbGVzKSB7CiAgICBjb25zdCBsaSA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2xpJyk7CiAgICBsaS5hcHBlbmQoc3BhbihmaWxlLm5hbWUsIHtmb250V2VpZ2h0OiAnYm9sZCd9KSk7CiAgICBsaS5hcHBlbmQoc3BhbigKICAgICAgICBgKCR7ZmlsZS50eXBlIHx8ICduL2EnfSkgLSAke2ZpbGUuc2l6ZX0gYnl0ZXMsIGAgKwogICAgICAgIGBsYXN0IG1vZGlmaWVkOiAkewogICAgICAgICAgICBmaWxlLmxhc3RNb2RpZmllZERhdGUgPyBmaWxlLmxhc3RNb2RpZmllZERhdGUudG9Mb2NhbGVEYXRlU3RyaW5nKCkgOgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAnbi9hJ30gLSBgKSk7CiAgICBjb25zdCBwZXJjZW50ID0gc3BhbignMCUgZG9uZScpOwogICAgbGkuYXBwZW5kQ2hpbGQocGVyY2VudCk7CgogICAgb3V0cHV0RWxlbWVudC5hcHBlbmRDaGlsZChsaSk7CgogICAgY29uc3QgZmlsZURhdGFQcm9taXNlID0gbmV3IFByb21pc2UoKHJlc29sdmUpID0+IHsKICAgICAgY29uc3QgcmVhZGVyID0gbmV3IEZpbGVSZWFkZXIoKTsKICAgICAgcmVhZGVyLm9ubG9hZCA9IChlKSA9PiB7CiAgICAgICAgcmVzb2x2ZShlLnRhcmdldC5yZXN1bHQpOwogICAgICB9OwogICAgICByZWFkZXIucmVhZEFzQXJyYXlCdWZmZXIoZmlsZSk7CiAgICB9KTsKICAgIC8vIFdhaXQgZm9yIHRoZSBkYXRhIHRvIGJlIHJlYWR5LgogICAgbGV0IGZpbGVEYXRhID0geWllbGQgewogICAgICBwcm9taXNlOiBmaWxlRGF0YVByb21pc2UsCiAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgYWN0aW9uOiAnY29udGludWUnLAogICAgICB9CiAgICB9OwoKICAgIC8vIFVzZSBhIGNodW5rZWQgc2VuZGluZyB0byBhdm9pZCBtZXNzYWdlIHNpemUgbGltaXRzLiBTZWUgYi82MjExNTY2MC4KICAgIGxldCBwb3NpdGlvbiA9IDA7CiAgICB3aGlsZSAocG9zaXRpb24gPCBmaWxlRGF0YS5ieXRlTGVuZ3RoKSB7CiAgICAgIGNvbnN0IGxlbmd0aCA9IE1hdGgubWluKGZpbGVEYXRhLmJ5dGVMZW5ndGggLSBwb3NpdGlvbiwgTUFYX1BBWUxPQURfU0laRSk7CiAgICAgIGNvbnN0IGNodW5rID0gbmV3IFVpbnQ4QXJyYXkoZmlsZURhdGEsIHBvc2l0aW9uLCBsZW5ndGgpOwogICAgICBwb3NpdGlvbiArPSBsZW5ndGg7CgogICAgICBjb25zdCBiYXNlNjQgPSBidG9hKFN0cmluZy5mcm9tQ2hhckNvZGUuYXBwbHkobnVsbCwgY2h1bmspKTsKICAgICAgeWllbGQgewogICAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgICBhY3Rpb246ICdhcHBlbmQnLAogICAgICAgICAgZmlsZTogZmlsZS5uYW1lLAogICAgICAgICAgZGF0YTogYmFzZTY0LAogICAgICAgIH0sCiAgICAgIH07CiAgICAgIHBlcmNlbnQudGV4dENvbnRlbnQgPQogICAgICAgICAgYCR7TWF0aC5yb3VuZCgocG9zaXRpb24gLyBmaWxlRGF0YS5ieXRlTGVuZ3RoKSAqIDEwMCl9JSBkb25lYDsKICAgIH0KICB9CgogIC8vIEFsbCBkb25lLgogIHlpZWxkIHsKICAgIHJlc3BvbnNlOiB7CiAgICAgIGFjdGlvbjogJ2NvbXBsZXRlJywKICAgIH0KICB9Owp9CgpzY29wZS5nb29nbGUgPSBzY29wZS5nb29nbGUgfHwge307CnNjb3BlLmdvb2dsZS5jb2xhYiA9IHNjb3BlLmdvb2dsZS5jb2xhYiB8fCB7fTsKc2NvcGUuZ29vZ2xlLmNvbGFiLl9maWxlcyA9IHsKICBfdXBsb2FkRmlsZXMsCiAgX3VwbG9hZEZpbGVzQ29udGludWUsCn07Cn0pKHNlbGYpOwo=",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/javascript"
                ]
              ],
              "status": 200,
              "status_text": ""
            }
          },
          "base_uri": "https://localhost:8080/",
          "height": 396
        },
        "outputId": "4f808875-79a9-4c17-b806-dee5f51a8404"
      },
      "source": [
        "%matplotlib inline\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import matplotlib as mpl\n",
        "import seaborn as sns\n",
        "import matplotlib.animation as animation\n",
        "from scipy import stats \n",
        "from mpl_toolkits.mplot3d import Axes3D\n",
        "from sklearn.datasets.samples_generator import make_regression \n",
        "import csv\n",
        "import math\n",
        "from google.colab import files\n",
        "\n",
        "uploaded = files.upload()\n",
        "\n",
        "data    = np.genfromtxt(\"data-nonlinear.txt\", delimiter=',')\n",
        "\n",
        "pointX  = data[:, 0]\n",
        "pointY  = data[:, 1]\n",
        "label   = data[:, 2]\n",
        "\n",
        "pointX0 = pointX[label == 0]\n",
        "pointY0 = pointY[label == 0]\n",
        "\n",
        "pointX1 = pointX[label == 1]\n",
        "pointY1 = pointY[label == 1]\n",
        "\n",
        "plt.figure()\n",
        "plt.scatter(pointX0, pointY0, c='b')\n",
        "plt.scatter(pointX1, pointY1, c='r')\n",
        "plt.tight_layout()\n",
        "plt.gca().set_aspect('equal', adjustable='box')\n",
        "plt.show()"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-5565e715-94c1-4b0f-9aa1-4be6dc1327a3\" name=\"files[]\" multiple disabled />\n",
              "     <output id=\"result-5565e715-94c1-4b0f-9aa1-4be6dc1327a3\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script src=\"/nbextensions/google.colab/files.js\"></script> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "error",
          "ename": "KeyboardInterrupt",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-2-cd4086f82d16>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     12\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mgoogle\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolab\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mfiles\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     13\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 14\u001b[0;31m \u001b[0muploaded\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfiles\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupload\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     15\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     16\u001b[0m \u001b[0mdata\u001b[0m    \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgenfromtxt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"data-nonlinear.txt\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdelimiter\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m','\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/google/colab/files.py\u001b[0m in \u001b[0;36mupload\u001b[0;34m()\u001b[0m\n\u001b[1;32m     62\u001b[0m   result = _output.eval_js(\n\u001b[1;32m     63\u001b[0m       'google.colab._files._uploadFiles(\"{input_id}\", \"{output_id}\")'.format(\n\u001b[0;32m---> 64\u001b[0;31m           input_id=input_id, output_id=output_id))\n\u001b[0m\u001b[1;32m     65\u001b[0m   \u001b[0mfiles\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_collections\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdefaultdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0m_six\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbinary_type\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     66\u001b[0m   \u001b[0;31m# Mapping from original filename to filename as saved locally.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/google/colab/output/_js.py\u001b[0m in \u001b[0;36meval_js\u001b[0;34m(script, ignore_result)\u001b[0m\n\u001b[1;32m     37\u001b[0m   \u001b[0;32mif\u001b[0m \u001b[0mignore_result\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     38\u001b[0m     \u001b[0;32mreturn\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 39\u001b[0;31m   \u001b[0;32mreturn\u001b[0m \u001b[0m_message\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_reply_from_input\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrequest_id\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     40\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     41\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/google/colab/_message.py\u001b[0m in \u001b[0;36mread_reply_from_input\u001b[0;34m(message_id, timeout_sec)\u001b[0m\n\u001b[1;32m     99\u001b[0m     \u001b[0mreply\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_read_next_input_message\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    100\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mreply\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0m_NOT_READY\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreply\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdict\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 101\u001b[0;31m       \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msleep\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0.025\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    102\u001b[0m       \u001b[0;32mcontinue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    103\u001b[0m     if (reply.get('type') == 'colab_reply' and\n",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZqqLOKHkCI_o",
        "colab_type": "text"
      },
      "source": [
        "# **2. Plot the training error with varying regularization parameters[9pt]**\n",
        "\n",
        "- choose a value for λ1 in such a way that over-fitting is demonstrated and plot the training error J(θ) at every iteration of gradient descent until convergence (in red color) [3pt]\n",
        "- choose a value for λ2 in such a way that just-right is demonstrated and plot the training error J(θ) at every iteration of gradient descent until convergence (in green color) [3pt]\n",
        "- choose a value for λ3​ in such a way that under-fitting is demonstrated and plot the training error J(θ) at every iteration of gradient descent until convergence (in blue color) [3pt]\n",
        "- the above three curves should be presented all together in a single figure"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NukW1MTrRyyF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X = pointX #전체 x\n",
        "y = np.array(pointY,dtype=float)\n",
        "label = np.array(label,dtype=float)\n",
        "# print(len(y))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Py4hDf40R0ZC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def g_function(X, y, theta, it, i):\n",
        "  result = 0\n",
        "  for m in range(0, 10): \n",
        "    for n in range(0, 10):\n",
        "      result = result + theta[m*10 + n][it]*(X[i]**m)*(y[i]**n)\n",
        "\n",
        "  return result\n",
        "  # return  (theta[0][it] + theta[1][it]*X[i] + theta[2][it]*y[i] + theta[3][it]*X[i]*X[i] +\n",
        "  #         theta[4][it]*X[i]*y[i] + theta[5][it]*y[i]*y[i] + theta[6][it]*X[i]*X[i]*X[i] + theta[7][it]*X[i]*X[i]*y[i] +\n",
        "  #         theta[8][it]*X[i]*y[i]*y[i] + theta[9][it]*y[i]*y[i]*y[i] + theta[10][it]*X[i]*X[i]*X[i]*X[i] + theta[11][it]*X[i]*X[i]*X[i]*y[i] +\n",
        "  #         theta[12][it]*X[i]*X[i]*y[i]*y[i] + theta[13][it]*X[i]*y[i]*y[i]*y[i] + theta[14][it]*y[i]*y[i]*y[i]*y[i] \n",
        "  #         #  + theta[15][it]*X[i]*X[i]*X[i]*X[i]*X[i]\n",
        "  #          )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jWLGVuO3R3jm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def gradient_descent(X,y,label,theta,learning_rate,iterations, controller):\n",
        "    \n",
        "    m = len(y)\n",
        "\n",
        "    cost_history = np.zeros(iterations)\n",
        "    theta_history = np.zeros((iterations,100))\n",
        "    \n",
        "    differential_g_function = np.zeros(100)\n",
        "\n",
        "    for it in range(iterations):\n",
        "        print(it)\n",
        "        pred = 0\n",
        "        preds = np.zeros(100)\n",
        "        sum_theta = 0\n",
        "\n",
        "        for i in range(0, m):\n",
        "          for p in range(0, 10): \n",
        "            for n in range(0, 10):\n",
        "              differential_g_function[p*10 + n] = (X[i]**p)*(y[i]**n)\n",
        "          # print(i)\n",
        "          #cost 함수\n",
        "          # print('100바퀴 이거 오래걸리나?')\n",
        "          pred = pred + ((-1)*label[i]*math.log(1/(1 + math.exp((-1)*g_function(X, y, theta, it, i))))-((1-label[i])*math.log(1 + (-1)*(1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))))))\n",
        "          \n",
        "          #  σ(z)   =    (1/(1 + math.exp((-1)*(theta[0][it] + theta[1][it]*X[i] + theta[2][it]*y[i]))))\n",
        "\n",
        "          #theta함수\n",
        "          for j in range(0, 100):\n",
        "            preds[j] = preds[j] + ((1/(1 + math.exp((-1)*g_function(X, y, theta, it, i)))) - label[i])*differential_g_function[j]\n",
        "            # (y[i]**(j%10))\n",
        "        \n",
        "        for i in range(0, 100):\n",
        "          theta[i][it + 1] = theta[i][it] - learning_rate*((1/m)*preds[i] + controller*theta[i][it])\n",
        "          # print(theta[i][it])\n",
        "        # print('100바퀴 이거 오래걸리나?')\n",
        "\n",
        "        for i in range(0, 100):\n",
        "          theta_history[it][i] = theta[i][it]\n",
        "\n",
        "        for p in range(0, 10): \n",
        "          for n in range(0, 10):\n",
        "            sum_theta = sum_theta + (theta[p*10 + n][it]**2)\n",
        "            # print(theta[p*10 + n][it]**2)\n",
        "\n",
        "        # print(controller)\n",
        "        # print(sum_theta)\n",
        "        # print(pred)\n",
        "        cost_history[it]  = (1/m) * pred + (controller/2)*sum_theta\n",
        "        print(cost_history[it])\n",
        "        \n",
        "    return theta, cost_history, theta_history"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OrrdDO_pR5su",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "lr =0.1\n",
        "n_iter = 100\n",
        "\n",
        "theta = np.zeros((100, n_iter + 1))\n",
        "theta2 = np.zeros((100, n_iter + 1))  \n",
        "theta3 = np.zeros((100, n_iter + 1))  \n",
        "\n",
        "X_b = np.c_[np.ones((len(X),0)),X]\n",
        "X_b = np.array(X_b,dtype=float)\n",
        "# print(X_b.shape)\n",
        "# print(X_b.dtype)\n",
        "# print(y.dtype)\n",
        "control_lambda1 = 0.00000001\n",
        "theta,cost_history,theta_history = gradient_descent(X_b,y,label,theta,lr,n_iter, control_lambda1)\n",
        "\n",
        "control_lambda2 = 1\n",
        "theta2,cost_history2,theta_history2 = gradient_descent(X_b,y,label,theta,lr,n_iter, control_lambda2)\n",
        "\n",
        "lr =0.0001\n",
        "control_lambda3 = 1000\n",
        "theta3,cost_history3,theta_history3 = gradient_descent(X_b,y,label,theta,lr,n_iter, control_lambda3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WfyioTj5R65B",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# # fig,ax = plt.subplots(figsize=(12,8))\n",
        "\n",
        "# theta0_history = []\n",
        "# theta1_history = []\n",
        "# theta2_history = []\n",
        "# theta3_history = []\n",
        "# theta4_history = []\n",
        "# theta5_history = []\n",
        "# theta6_history = []\n",
        "# theta7_history = []\n",
        "# theta8_history = []\n",
        "# theta9_history = []\n",
        "# theta10_history = []\n",
        "# theta11_history = []\n",
        "# theta12_history = []\n",
        "# theta13_history = []\n",
        "# theta14_history = []\n",
        "# # theta15_history = []\n",
        "\n",
        "# for it in range(n_iter):\n",
        "#   theta0_history.append(theta_history[it][0])\n",
        "#   theta1_history.append(theta_history[it][1])\n",
        "#   theta2_history.append(theta_history[it][2])\n",
        "#   theta3_history.append(theta_history[it][3])\n",
        "#   theta4_history.append(theta_history[it][4])\n",
        "#   theta5_history.append(theta_history[it][5])\n",
        "#   theta6_history.append(theta_history[it][6])\n",
        "#   theta7_history.append(theta_history[it][7])\n",
        "#   theta8_history.append(theta_history[it][8])\n",
        "#   theta9_history.append(theta_history[it][9])\n",
        "#   theta10_history.append(theta_history[it][10])\n",
        "#   theta11_history.append(theta_history[it][11])\n",
        "#   theta12_history.append(theta_history[it][12])\n",
        "#   theta13_history.append(theta_history[it][13])\n",
        "#   theta14_history.append(theta_history[it][14])\n",
        "#   # theta15_history.append(theta_history[it][15])\n",
        "\n",
        "# # ax.set_ylabel('Theta')\n",
        "# # ax.set_xlabel('Iterations')\n",
        "# # _=ax.plot(range(n_iter),theta0_history,'b.', color = 'red')\n",
        "# # _=ax.plot(range(n_iter),theta1_history,'b.', color = 'green')\n",
        "# # _=ax.plot(range(n_iter),theta2_history,'b.', color = 'blue')\n",
        "\n",
        "# # for it in range(0, 15):\n",
        "# #   print(theta_history[-1][it])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "we9XQdQYR9ZF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "fig,ax = plt.subplots(figsize=(12,8))\n",
        "\n",
        "print(cost_history[1])\n",
        "\n",
        "ax.set_ylabel('J(Theta)')\n",
        "ax.set_xlabel('Iterations')\n",
        "_=ax.plot(range(n_iter),cost_history,'b.', color = 'red')\n",
        "_=ax.plot(range(n_iter),cost_history2,'b.', color = 'green')\n",
        "_=ax.plot(range(n_iter),cost_history3,'b.', color = 'blue')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mhOvyBZLcFk9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%pylab inline \n",
        "import numpy as np \n",
        "import pandas as pd \n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d979NoCtcLYL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def sigmoid(z): \n",
        "  g = 1 / (1 + np.exp(-z)) \n",
        "  \n",
        "  return g \n",
        "  \n",
        "# print('sigmoid(10) = %s, sigmoid(-10) = %s, sigmoid(0) = %s' % (sigmoid(10), sigmoid(-10), sigmoid(0)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xUYV0C5NcL56",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def mapFeature(X1, X2): \n",
        "  degree =  9\n",
        "  #print(X1.shape) \n",
        "  m = np.size(X1) \n",
        "  out = np.ones(m) \n",
        "  res = [] \n",
        "  \n",
        "  res.append(out) \n",
        "  \n",
        "  for i in np.arange(0, degree + 1): \n",
        "    for j in np.arange(0, degree + 1): \n",
        "      out1 = (X1 ** i) * (X2 ** j) \n",
        "      res.append(out1) \n",
        "      \n",
        "  return np.array(res).T \n",
        "\n",
        "data = matrix(data)\n",
        "\n",
        "X = data[:, 0:2] \n",
        "y = data[:, 2] \n",
        "\n",
        "X = np.array(X)\n",
        "y = np.array(y)\n",
        "      \n",
        "X_reg = mapFeature(X[:, 0], X[:, 1]) \n",
        "# print(X_reg[0:2, :])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vy1SkrH9cNsS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import scipy.optimize as op \n",
        "\n",
        "def myCostReg(theta, X, y, s_lambda): \n",
        "  \n",
        "  X = matrix(X) \n",
        "\n",
        "  J = 0 \n",
        "  grad = np.zeros(np.size(theta)) \n",
        "  m, n = X.shape \n",
        "  y = y.reshape((m, 1)) \n",
        "  theta = theta.reshape((n, 1)) \n",
        "\n",
        "  theta_0 = theta[0] \n",
        "  theta_j = theta[1:] \n",
        "\n",
        "  theta = np.matrix(theta) \n",
        "  sum_theta_square = np.sum(theta_j ** 2) \n",
        "  predictions = X * theta \n",
        "  sig_term = sigmoid(predictions)\n",
        "\n",
        "  J = 1/m * sum(np.multiply(-y, log(sig_term)) - np.multiply((1 - y), log(1 - sig_term))) + s_lambda / (2 * m) * sum_theta_square \n",
        "  \n",
        "  return J \n",
        "  \n",
        "def myGradientReg(theta, X, y, s_lambda): \n",
        " \n",
        "  X = matrix(X) \n",
        "\n",
        "  J = 0 \n",
        "  grad = np.zeros(np.size(theta)) \n",
        "  m, n = X.shape \n",
        "  y = y.reshape((m, 1)) \n",
        "  theta = theta.reshape((n, 1)) \n",
        "\n",
        "  theta_0 = theta[0] \n",
        "  theta_j = theta[1:] \n",
        " \n",
        "  theta = np.matrix(theta) \n",
        "\n",
        "  predictions = X * theta \n",
        "  sig_term = sigmoid(predictions) \n",
        "\n",
        "  errors = (sig_term - y) \n",
        "\n",
        "  delta_0 = 1/m * errors.T * X[:, 0] \n",
        "\n",
        "  delta = 1/m * errors.T * X[:, 1:]\n",
        "\n",
        "  regular = s_lambda / m * theta_j \n",
        "  delta_j = delta.T + regular \n",
        "  grad[0] = delta_0 \n",
        "  grad[1:] = delta_j.T \n",
        "   \n",
        "  return grad.flatten() \n",
        "   \n",
        "initial_theta = np.zeros((X_reg.shape[1], 1)) \n",
        "\n",
        "s_lambda = 0.01 \n",
        "Result = op.minimize(fun = myCostReg, x0 = initial_theta, args = (X_reg, y, s_lambda), method = 'TNC', jac = myGradientReg); \n",
        "cost = Result.fun \n",
        "theta = Result.x \n",
        "\n",
        "s_lambda2 = 1 \n",
        "Result2 = op.minimize(fun = myCostReg, x0 = initial_theta, args = (X_reg, y, s_lambda2), method = 'TNC', jac = myGradientReg); \n",
        "cost2 = Result2.fun \n",
        "theta2 = Result2.x \n",
        "\n",
        "s_lambda3 = 100 \n",
        "Result3 = op.minimize(fun = myCostReg, x0 = initial_theta, args = (X_reg, y, s_lambda3), method = 'TNC', jac = myGradientReg); \n",
        "cost3 = Result3.fun \n",
        "theta3 = Result3.x \n",
        "\n",
        "# theta[0] = theta_history[1500][0]\n",
        "# theta[1] = theta_history[1500][1]\n",
        "# theta[2] = theta_history[1500][2]\n",
        "# theta[3] = theta_history[1500][3]\n",
        "# theta[4] = theta_history[1500][4]\n",
        "# theta[5] = theta_history[1500][5]\n",
        "# theta[6] = theta_history[1500][6]\n",
        "# theta[7] = theta_history[1500][7]\n",
        "# theta[8] = theta_history[1500][8]\n",
        "# theta[9] = theta_history[1500][9]\n",
        "# theta[10] = theta_history[1500][10]\n",
        "# theta[11] = theta_history[1500][11]\n",
        "# theta[12] = theta_history[1500][12]\n",
        "# theta[13] = theta_history[1500][13]\n",
        "# theta[14] = theta_history[1500][14]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "InuWtnRQcO5l",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def predict(theta, X): \n",
        "  m = matrix(X) \n",
        "  m, n = X.shape \n",
        "  theta = theta.reshape((n, 1)) \n",
        "  theta = matrix(theta) \n",
        "  p = np.zeros((m, 1)) \n",
        "  p = sigmoid(X * theta) >= 0.5; \n",
        "  \n",
        "  return p "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iNdR62YqcQPg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "p = predict(theta, X_reg) \n",
        "p2 = predict(theta2, X_reg) \n",
        "p3 = predict(theta3, X_reg) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0DiidmcWcRIi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def plotBoundary(theta, X, y): \n",
        "  \n",
        "  if np.size(X, 1) <= 3: \n",
        "    plot_x = np.array([np.min(X[:, 1]) - 2, np.max(X[:, 1] + 2)]) \n",
        "    plot_y = (-1 / theta[2]) * (theta[1] * plot_x + theta[0]) \n",
        "    plot(plot_x, plot_y) \n",
        "    plt.legend(('Admitted', 'Not admitted'), loc='upper right') \n",
        "  \n",
        "  else: \n",
        "\n",
        "    u = linspace(-1, 1.5, 800) \n",
        "    v = linspace(-1, 1.5, 800) \n",
        "    z = np.zeros((np.size(u), np.size(v))) \n",
        "\n",
        "    for i in np.arange(np.size(u)): \n",
        "      for j in np.arange(np.size(v)): \n",
        "        z[i, j] = np.sum(mapFeature(u[i], v[j]) * theta) \n",
        "        \n",
        "        z = z.T \n",
        "\n",
        "    # plt.figure()\n",
        "    # plt.scatter(pointX0, pointY0, c='b')\n",
        "    # plt.scatter(pointX1, pointY1, c='r')\n",
        "    # plt.tight_layout()\n",
        "    # plt.gca().set_aspect('equal', adjustable='box')\n",
        "    plt.contour(u, v, z) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mJXO5HtIcSbj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# plotBoundary(theta, X_reg, y)\n",
        "# plotBoundary(theta2, X_reg, y)\n",
        "# plotBoundary(theta3, X_reg, y)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SoKpG5oscYnA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "accuracy_history = np.zeros(1500)\n",
        "\n",
        "for i in range(1500):\n",
        "  p = predict(theta_history[i], X_reg) \n",
        "  accuracy_history[i] = np.mean(double(p == y))\n",
        "\n",
        "accuracy_history2 = np.zeros(1500)\n",
        "\n",
        "for i in range(1500):\n",
        "  p2 = predict(theta_history2[i], X_reg) \n",
        "  accuracy_history2[i] = np.mean(double(p2 == y))\n",
        "\n",
        "accuracy_history3 = np.zeros(1500)\n",
        "\n",
        "for i in range(1500):\n",
        "  p3 = predict(theta_history3[i], X_reg) \n",
        "  accuracy_history3[i] = np.mean(double(p3 == y))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CBj-WlzDcZ42",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# fig,ax = plt.subplots(figsize=(12,8))\n",
        "\n",
        "# ax.set_ylabel('accuracy')\n",
        "# ax.set_xlabel('Iterations')\n",
        "# _=ax.plot(range(1500),accuracy_history,'r.')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mujVAKAmCNjP",
        "colab_type": "text"
      },
      "source": [
        "# **3. Display the values of the chosen regularization parameters[3pt]**\n",
        "\n",
        "- display the value of the chosen λ1 for the demonstration of over-fitting (in red color) [1pt]\n",
        "- display the value of the chosen λ2​ for the demonstration of just-right (in green color) [1pt]\n",
        "- display the value of the chosen λ3 for the demonstration of under-fitting (in blue color) [1pt]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W0_y4h3lCOEW",
        "colab_type": "text"
      },
      "source": [
        "# **4. Plot the training accuracy with varying regularization parameters[9pt]**\n",
        "\n",
        "- plot the training accuracy with the chosen λ1 for over-fitting at every iteration of gradient descent until convergence (in red color) [3pt]\n",
        "- plot the training accuracy with the chosen λ2​ for just-right at every iteration of gradient descent until convergence (in green color) [3pt]\n",
        "- plot the training accuracy with the chosen λ3​ for under-fitting at every iteration of gradient descent until convergence (in blue color) [3pt]\n",
        "- the above three curves should be presented all together in a single figure"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oaEwX9Jl57mi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "fig,ax = plt.subplots(figsize=(12,8))\n",
        "\n",
        "ax.set_ylabel('accuracy')\n",
        "ax.set_xlabel('Iterations')\n",
        "_=ax.plot(range(1500),accuracy_history,'r.', color = 'red')\n",
        "_=ax.plot(range(1500),accuracy_history2,'r.', color = 'green')\n",
        "_=ax.plot(range(1500),accuracy_history3,'r.', color = 'blue')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nv_zAWupCOVI",
        "colab_type": "text"
      },
      "source": [
        "# **5. Display the final training accuracy with varying regularization parameters[3pt]**\n",
        "\n",
        "- display the final training accuracy obtained with the chosen λ1​ for over-fitting in number (%) at convergence (in red color) [1pt]\n",
        "- display the final training accuracy obtained with the chosen λ2​ for just-right in number (%) at convergence (in green color) [1pt]\n",
        "- display the final training accuracy obtained with the chosen λ3​ for under-fitting in number (%) at convergence (in blue color) [1pt]"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZGxyAHZm6DBa",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(\"Train Accuracy:  \", end=' ')\n",
        "print(accuracy_history[-1]*100, end='%')\n",
        "print(\"Train Accuracy:  \", end=' ')\n",
        "print(accuracy_history2[-1]*100, end='%')\n",
        "print(\"Train Accuracy:  \", end=' ')\n",
        "print(accuracy_history3[-1]*100, end='%')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MokllgPaCOmA",
        "colab_type": "text"
      },
      "source": [
        "# **6. Plot the optimal classifier with varying regularization parameters superimposed on the training data[9pt]**\n",
        "\n",
        "- plot the boundary of the optimal classifier with the chosen λ1​ for over-fitting at convergence (in red color) [3pt]\n",
        "- plot the boundary of the optimal classifier with the chosen λ2 for just-right at convergence (in green color) [3pt]\n",
        "- plot the boundary of the optimal classifier with the chosen λ3​ for under-fitting at convergence (in blue color) [3pt]\n",
        "- the boundary of the classifier is defined by {(x,y)∣σ(g(x,y;θ))=0.5}={(x,y)∣g(x,y;θ)=0}\n",
        "- the boundaries of the classifiers with different regularization parameters should be presented with the training data points (x,y) with their labels lll in colors (blue for label 0 and red for label 1)\n",
        "- you can use contour function in python3"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "jRZdvhoye2gw"
      },
      "source": [
        "# **Machine Learning Assignment07** #\n",
        "\n",
        "Name : 이현주\n",
        "\n",
        "Student # : 20184060"
      ]
    }
  ]
}