MNIST_NNs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Topic\n",
    "Option 2: Implement a neural network from scratch.\n",
    "\n",
    "Write a `NeuralNetwork` class in Python that implements a neural network model with dense layers and ReLU activation. \n",
    "\n",
    "This class should contain the following methods:\n",
    "\n",
    "1. `__init__`: Declares a neural network with a list of integers indicating the size of each layer. \n",
    "2. `fit`: Trains the neural network model using the Stochastic Gradient Descent method.\n",
    "3. `predict`: Calculates the output values for a list of input data.\n",
    "\n",
    "It is up to you to decide the details, but your implementation cannot use neural network classes from any existing library. Please refer to the slides on neural networks and this online textbook for the formulas and algorithms you should use. After completing the implementation, apply the model on a dataset (for example, the Iris dataset) to see whether it gives reasonable results.\n",
    "\n",
    "Instructions: All code should be executed, and results should be displayed in the notebook. Writing is as important as coding. Clearly describe every step that you take."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Libraries importation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np # for calculus\n",
    "from pandas import read_csv  # for data loading\n",
    "import matplotlib.pyplot as plt # plotting\n",
    "import os # os system managment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Description\n",
    "__Full neural network construction with only math and numpy__\n",
    "\n",
    "For this short project, I will create a fully connected two layer neural network to predict the numbers associated with the images in the MNIST data set. The network will take in the 784 pixels in an image as input. The first hidden layer will contain 10 neurons with ReLU activation, followed by a 10 neuron output layer with sigmoid activation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, to make it possible, i'll use a trainning and testing dataset form `kaggle` from __`mnist-in-csv`__."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test & Train data loading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "script_dir = os.getcwd() # get the current working directory\n",
    "# define the custom name and extension of train an test dataset\n",
    "# in our case ext mime is csv.\n",
    "train_file = \"mnist_train.csv\"  \n",
    "test_file = \"mnist_test.csv\"\n",
    "# compute the absolute files path\n",
    "abs_train_file_path = os.path.join(script_dir, train_file)\n",
    "abs_test_file_path = os.path.join(script_dir, test_file)\n",
    "# load each fiile\n",
    "train = read_csv(abs_train_file_path)\n",
    "test = read_csv(abs_test_file_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Less exploration of data to pull out somes details we said before"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> show head data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>1x1</th>\n",
       "      <th>1x2</th>\n",
       "      <th>1x3</th>\n",
       "      <th>1x4</th>\n",
       "      <th>1x5</th>\n",
       "      <th>1x6</th>\n",
       "      <th>1x7</th>\n",
       "      <th>1x8</th>\n",
       "      <th>1x9</th>\n",
       "      <th>...</th>\n",
       "      <th>28x19</th>\n",
       "      <th>28x20</th>\n",
       "      <th>28x21</th>\n",
       "      <th>28x22</th>\n",
       "      <th>28x23</th>\n",
       "      <th>28x24</th>\n",
       "      <th>28x25</th>\n",
       "      <th>28x26</th>\n",
       "      <th>28x27</th>\n",
       "      <th>28x28</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 785 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   label  1x1  1x2  1x3  1x4  1x5  1x6  1x7  1x8  1x9  ...  28x19  28x20  \\\n",
       "0      5    0    0    0    0    0    0    0    0    0  ...      0      0   \n",
       "1      0    0    0    0    0    0    0    0    0    0  ...      0      0   \n",
       "2      4    0    0    0    0    0    0    0    0    0  ...      0      0   \n",
       "3      1    0    0    0    0    0    0    0    0    0  ...      0      0   \n",
       "4      9    0    0    0    0    0    0    0    0    0  ...      0      0   \n",
       "\n",
       "   28x21  28x22  28x23  28x24  28x25  28x26  28x27  28x28  \n",
       "0      0      0      0      0      0      0      0      0  \n",
       "1      0      0      0      0      0      0      0      0  \n",
       "2      0      0      0      0      0      0      0      0  \n",
       "3      0      0      0      0      0      0      0      0  \n",
       "4      0      0      0      0      0      0      0      0  \n",
       "\n",
       "[5 rows x 785 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "it's there a data set of 784 pixels of different images showing a number between `0` to `9`. each row content a label which is the number show by the image and 784 pixels's value belong to it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> show train information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 60000 entries, 0 to 59999\n",
      "Columns: 785 entries, label to 28x28\n",
      "dtypes: int64(785)\n",
      "memory usage: 359.3 MB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "the dataset `don't have missing values or noise values`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train and Test data pretreatment\n",
    "in the goal of building the NN, we need to get each pixel to image as input of our first or entry layer of our NN, thus we will change the type data from `DataFrame` to `ndarray`, next transpose each dataset as modeling to be trust by the model nn. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dtype changing\n",
    "train = np.array(train)\n",
    "test = np.array(test)\n",
    "# split train label and features before transpose the array\n",
    "X_train = train[:, 1:].T\n",
    "Y_train = train[:, 0].T\n",
    "# split test label and features before transpose the array\n",
    "X_test = test[:, 1:].T\n",
    "Y_test = test[:, 0].T\n",
    "# normalize each feature value to involve image computation\n",
    "X_train = X_train / 255.\n",
    "X_test = X_test / 255.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our NN will have a simple two-layer architecture. Input layer $a^{[0]}$ will have 784 units corresponding to the 784 pixels in each 28x28 input image. A hidden layer $a^{[1]}$ will have 10 units with ReLU activation, and finally our output layer $a^{[2]}$ will have 10 units corresponding to the ten digit classes with softmax activation.\n",
    "\n",
    "**Forward propagation**\n",
    "\n",
    "$$Z^{[1]} = W^{[1]} X + b^{[1]}$$\n",
    "$$A^{[1]} = g_{\\text{ReLU}}(Z^{[1]}))$$\n",
    "$$Z^{[2]} = W^{[2]} A^{[1]} + b^{[2]}$$\n",
    "$$A^{[2]} = g_{\\text{softmax}}(Z^{[2]})$$\n",
    "\n",
    "**Backward propagation**\n",
    "\n",
    "$$dZ^{[2]} = A^{[2]} - Y$$\n",
    "$$dW^{[2]} = \\frac{1}{m} dZ^{[2]} A^{[1]T}$$\n",
    "$$dB^{[2]} = \\frac{1}{m} \\Sigma {dZ^{[2]}}$$\n",
    "$$dZ^{[1]} = W^{[2]T} dZ^{[2]} .* g^{[1]\\prime} (z^{[1]})$$\n",
    "$$dW^{[1]} = \\frac{1}{m} dZ^{[1]} A^{[0]T}$$\n",
    "$$dB^{[1]} = \\frac{1}{m} \\Sigma {dZ^{[1]}}$$\n",
    "\n",
    "**Parameter updates**\n",
    "\n",
    "$$W^{[2]} := W^{[2]} - \\alpha dW^{[2]}$$\n",
    "$$b^{[2]} := b^{[2]} - \\alpha db^{[2]}$$\n",
    "$$W^{[1]} := W^{[1]} - \\alpha dW^{[1]}$$\n",
    "$$b^{[1]} := b^{[1]} - \\alpha db^{[1]}$$\n",
    "\n",
    "**Vars and shapes**\n",
    "\n",
    "Forward prop\n",
    "\n",
    "- $A^{[0]} = X$: 784 x m\n",
    "- $Z^{[1]} \\sim A^{[1]}$: 10 x m\n",
    "- $W^{[1]}$: 10 x 784 (as $W^{[1]} A^{[0]} \\sim Z^{[1]}$)\n",
    "- $B^{[1]}$: 10 x 1\n",
    "- $Z^{[2]} \\sim A^{[2]}$: 10 x m\n",
    "- $W^{[1]}$: 10 x 10 (as $W^{[2]} A^{[1]} \\sim Z^{[2]}$)\n",
    "- $B^{[2]}$: 10 x 1\n",
    "\n",
    "Backprop\n",
    "\n",
    "- $dZ^{[2]}$: 10 x m ($~A^{[2]}$)\n",
    "- $dW^{[2]}$: 10 x 10\n",
    "- $dB^{[2]}$: 10 x 1\n",
    "- $dZ^{[1]}$: 10 x m ($~A^{[1]}$)\n",
    "- $dW^{[1]}$: 10 x 10\n",
    "- $dB^{[1]}$: 10 x 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Activation:\n",
    "    def __init__(self, name):\n",
    "        self.name = name\n",
    "    def __str__(self):\n",
    "        return \"{} activation function\".format(self.name)\n",
    "        \n",
    "class ReLU(Activation):\n",
    "    def __init__(self):\n",
    "        super().__init__(self,\"Relu\")\n",
    "    def __call__(self, Z):\n",
    "        \"\"\"\n",
    "        ReLU method\n",
    "        \n",
    "        Args:\n",
    "            Z (ndarray): there is a computed features\n",
    "            \n",
    "        Return:\n",
    "            Z (ndarray) max between Zi and 0\n",
    "        \"\"\"\n",
    "        return np.maximum(Z,0)\n",
    "    \n",
    "    def derivative_ReLU(self, Z):\n",
    "        \"\"\"\n",
    "        derivative_ReLU method\n",
    "        \n",
    "        Args:\n",
    "            Z (ndarray): there is a computed features\n",
    "            \n",
    "        Return:\n",
    "            Z (ndarray) with true if zi > 0 and false else\n",
    "        \"\"\"\n",
    "        return Z > 0\n",
    "    \n",
    "class SoftMax(Activation):\n",
    "    def __init__(self):\n",
    "        super().__init__(self,\"SoftMax\")\n",
    "    def __call__(self, Z):\n",
    "        \"\"\"\n",
    "        softmax method\n",
    "        \n",
    "        Args:\n",
    "            Z (ndarray): there is a computed features\n",
    "            \n",
    "        Return:\n",
    "            (ndarray) computed softmax values for each sets of scores in Z.\n",
    "        \"\"\"\n",
    "        exp = np.exp(Z - np.max(Z))\n",
    "        return exp / exp.sum(axis=0)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Layer():\n",
    "    def __init__(self, name):\n",
    "        self.name = name\n",
    "\n",
    "class Affine(Layer):\n",
    "    def __init__(self, nin, nout, activation):\n",
    "        super().__init__(self,\"Affine Layer\")\n",
    "        self.input_size = nin\n",
    "        self.hidden_nodes = nout\n",
    "        self.W = np.random.normal(size=(nout, nin)) * np.sqrt(1./(nin))\n",
    "        self.b = np.random.normal(size=(nout, 1)) * np.sqrt(1./nout)\n",
    "        self.activator = activation\n",
    "\n",
    "    def forward_propagation(self,X):\n",
    "        Z1 = self.W.dot(X) + self.b  # nout, nin\n",
    "        A1 = self.activator(Z1)  # nout, nin\n",
    "        return Z1, A1\n",
    "        \n",
    "    def update_params(self, alpha, dW, db):\n",
    "        self.W -= alpha * dW\n",
    "        self.b -= alpha * np.reshape(db, (self.hidden_nodes, 1))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Optimizer():\n",
    "    def __init__(self, name):\n",
    "        self.name = name\n",
    "\n",
    "\n",
    "class GradientDescent(Optimizer):\n",
    "    def __init__(self, alpha, iterations):\n",
    "        super().__init__(self, \"Gradient Descent Optimizer\")\n",
    "        self.alpha = alpha\n",
    "        self.iterations = iterations\n",
    "\n",
    "    def __call__(X, Y, layers, fprop, bprop, uparams):\n",
    "        W1, b1, W2, b2 = init_params(size)\n",
    "        for i in range(iterations):\n",
    "            Z1, A1, Z2, A2 = forward_propagation(X, W1, b1, W2, b2)\n",
    "            dW1, db1, dW2, db2 = backward_propagation(X, Y, A1, A2, W2, Z1, m)\n",
    "\n",
    "            W1, b1, W2, b2 = update_params(alpha, W1, b1, W2, b2, dW1, db1, dW2, db2)   \n",
    "\n",
    "            if (i+1) % int(iterations/10) == 0:\n",
    "                print(f\"Iteration: {i+1} / {iterations}\")\n",
    "                prediction = get_predictions(A2)\n",
    "                print(f'Training accuracy: {get_accuracy(prediction, Y):.3%}')\n",
    "        return W1, b1, W2, b2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "class mnist_nn:\n",
    "    \"\"\" \n",
    "    mnist neural network build from scratch within numpy, pandas and matplotlib\n",
    "    \n",
    "    Args:\n",
    "        features (ndarray): the features content all examples features values based for train ....\n",
    "        labels (ndarray): the labels content all real labels of all examples features...\n",
    "        alpha (Numeric): the alpha is an update coeficient...\n",
    "        iterations (Numeric): the iterations is the number of epochs of training stage...\n",
    "        \n",
    "    Attributes:\n",
    "        features (ndarray): where store the features content all examples features values based for train ....\n",
    "        labels (ndarray): where store the labels content all real labels of all examples features...\n",
    "        alpha (Numeric): where store the alpha is an update coeficient...\n",
    "        iterations (Numeric): where store the iterations is the number of epochs of training stage...\n",
    "        w1 (ndarray): the normally randomly initialized weights from the input layer to the first hidden layer...\n",
    "        b1 (ndarray): the biais from the input layers to the first hidden layer...\n",
    "        w2 (ndarray): the normally randomly initialized weights from the first hidden layer to the second hidden layer...\n",
    "        b2 (ndarray): the biais from the first hidden layers to the second hidden layer...\n",
    "        \n",
    "    \"\"\"\n",
    "    def __init__(self, features, labels, alpha, iterations):\n",
    "        ''' Return an instance of mnist_nn with all attributes and methods '''\n",
    "        self.layers = []\n",
    "\n",
    "    def forward_propagation(X):\n",
    "        fvar = []\n",
    "        for i in range(len(self.layers)):\n",
    "            if i == 0:\n",
    "                Z, A = layers[i].forward_propagation(X)\n",
    "                fvar.append({'Z':Z,'A':A})\n",
    "            else:\n",
    "                Z, A = layers[i].forward_propagation(fvar[i-1]['A'])\n",
    "                fvar.append({'Z': Z, 'A': A})\n",
    "        ''' Z1 = W1.dot(X) + b1 #10, m\n",
    "        A1 = ReLU(Z1) # 10,m\n",
    "        Z2 = W2.dot(A1) + b2 #10,m\n",
    "        A2 = softmax(Z2) #10,m '''\n",
    "        return fvar\n",
    "\n",
    "    def one_hot(Y):\n",
    "        ''' return an 0 vector with 1 only in the position correspondind to the value in Y'''\n",
    "        one_hot_Y = np.zeros((Y.max()+1,Y.size))\n",
    "        one_hot_Y[Y,np.arange(Y.size)] = 1\n",
    "        return one_hot_Y\n",
    "\n",
    "    def backward_propagation(X, Y, fvar):\n",
    "        one_hot_Y = one_hot(Y)\n",
    "        pvar = []\n",
    "        for i in range(len(self.layers)-1,-1,-1):\n",
    "            if i == len(self.layers)-1:\n",
    "                dZ = 2*(fvar[i]['A'] - one_hot_Y) #10,m\n",
    "                dW = 1/m * (dZ2.dot(fvar[i-1]['A'].T)) # 10 , 10\n",
    "                db = 1/m * np.sum(dZ2,1) # 10, 1\n",
    "                pvar.append({'dZ':dZ,'dW':dW,'db':db})\n",
    "            elif i == 0:\n",
    "                dZ1 = W2.T.dot(dZ2)*derivative_ReLU(Z1)  # 10, m\n",
    "                dW1 = 1/m * (dZ1.dot(X.T))  # 10, 784\n",
    "                db1 = 1/m * np.sum(dZ1, 1)  # 10, 1\n",
    "            else:\n",
    "                \n",
    "        dZ2 = 2*(A2 - one_hot_Y) #10,m\n",
    "        dW2 = 1/m * (dZ2.dot(A1.T)) # 10 , 10\n",
    "        db2 = 1/m * np.sum(dZ2,1) # 10, 1\n",
    "        dZ1 = W2.T.dot(dZ2)*derivative_ReLU(Z1) # 10, m\n",
    "        dW1 = 1/m * (dZ1.dot(X.T)) #10, 784\n",
    "        db1 = 1/m * np.sum(dZ1,1) # 10, 1\n",
    "\n",
    "        return dW1, db1, dW2, db2\n",
    "\n",
    "    def update_params(alpha, W1, b1, W2, b2, dW1, db1, dW2, db2):\n",
    "        W1 -= alpha * dW1\n",
    "        b1 -= alpha * np.reshape(db1, (10,1))\n",
    "        W2 -= alpha * dW2\n",
    "        b2 -= alpha * np.reshape(db2, (10,1))\n",
    "\n",
    "        return W1, b1, W2, b2\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on class mnist_nn in module __main__:\n",
      "\n",
      "class mnist_nn(builtins.object)\n",
      " |  mnist_nn(features, labels, alpha, iterations)\n",
      " |  \n",
      " |  mnist neural network build from scratch within numpy, pandas and matplotlib\n",
      " |  \n",
      " |  Args:\n",
      " |      features (ndarray): the features content all examples features values based for train ....\n",
      " |      labels (ndarray): the labels content all real labels of all examples features...\n",
      " |      alpha (Numeric): the alpha is an update coeficient...\n",
      " |      iterations (Numeric): the iterations is the number of epochs of training stage...\n",
      " |      \n",
      " |  Attributes:\n",
      " |      features (ndarray): where store the features content all examples features values based for train ....\n",
      " |      labels (ndarray): where store the labels content all real labels of all examples features...\n",
      " |      alpha (Numeric): where store the alpha is an update coeficient...\n",
      " |      iterations (Numeric): where store the iterations is the number of epochs of training stage...\n",
      " |      w1 (ndarray): the weights from the input layer to the first hidden layer\n",
      " |      b1 (ndarray): the biais from the input layers to the first hidden layer\n",
      " |      w2 (ndarray): the weights from the first hidden layer to the second hidden layer\n",
      " |      b2 (ndarray): the biais from the first hidden layers to the second hidden layer\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, features, labels, alpha, iterations)\n",
      " |      Return an instance of mnist_nn with all attributes and methods\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors defined here:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(mnist_nn)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1"
  },
  "kernelspec": {
   "display_name": "Python 3.9.2 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}