NN_Quantizer.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the Dataset\n",
    "MNIST is a dataset of 70,000 handwritten images. Keras provides a function to directly download the data.\n",
    "<br>The dataset is then split into 60K training images and 10K test images.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data() #load and split data "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Normalize the images\n",
    "The input are 28 pixels by 28 pixels images. They are represented in grayscale, which means each pixel is a single number between 0 and 255. 0 is a black pixel and 255 is a white pixel.\n",
    "<br>Before training, it is a good practice to normalize the data. In our case the min-max scaler is used to get values between 0 and 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert datasets to float\n",
    "x_train = x_train.astype('float32')\n",
    "x_test  = x_test.astype('float32')\n",
    "\n",
    "# Normalize the pixel data\n",
    "x_train /= 255.0\n",
    "x_test  /= 255.0"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Construct the NN model\n",
    "\n",
    "Our NN will have the following structure\n",
    "1. **AveragePooling:** This layer turns a 28x28 image to 14x14. It takes the input image and transforms each 4x4 sub-matrix by replacing it with its average value.\n",
    "2. **Flatten:** This layer converts a 14x14 matrix to a 196x1 1D vector. This conversion is needed for the next layers.\n",
    "3. **Dense layer 1:** In a dense layer (also known as a fully connected layer), the output of each neuron is calculated by a weighted sum of the inputs from all neurons in the preceding layer. This sum is passed through an activation function before being propagated to the next layer.\n",
    "4. **Dense layer 2:** The output layer has 10 neurons: one for each of the possible digit. The neuron with the largest value corresponds to the recognized digit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_15\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " average_pooling2d_15 (Avera  (None, 14, 14, 1)        0         \n",
      " gePooling2D)                                                    \n",
      "                                                                 \n",
      " flatten_15 (Flatten)        (None, 196)               0         \n",
      "                                                                 \n",
      " dense_30 (Dense)            (None, 32)                6304      \n",
      "                                                                 \n",
      " dense_31 (Dense)            (None, 10)                330       \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 6,634\n",
      "Trainable params: 6,634\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Importing Keras model and layers\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Flatten, AveragePooling2D\n",
    "\n",
    "# Construct the NN by stacking all required layers\n",
    "model_s_nn = Sequential() # Sequential: the layers will be connected to one another\n",
    "model_s_nn.add(AveragePooling2D(pool_size=(2, 2), input_shape=(28, 28, 1)))\n",
    "model_s_nn.add(Flatten()) # Flattening the 2D arrays for fully connected layers\n",
    "model_s_nn.add(Dense(32, activation=tf.nn.relu))\n",
    "model_s_nn.add(Dense(10,activation=tf.nn.softmax))\n",
    "model_s_nn.summary() # Print NN summary"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the Model\n",
    "\n",
    "The model can now be trained using the training dataset. For each image x_train(i), the model will try to find the associated digit y_train(i). It will gradually adjust the values for the different weights and biases of the dense layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "1875/1875 [==============================] - 3s 1ms/step - loss: 0.5000 - accuracy: 0.8675\n",
      "Epoch 2/10\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.2609 - accuracy: 0.9257\n",
      "Epoch 3/10\n",
      "1875/1875 [==============================] - 3s 1ms/step - loss: 0.2184 - accuracy: 0.9379\n",
      "Epoch 4/10\n",
      "1875/1875 [==============================] - 3s 2ms/step - loss: 0.1887 - accuracy: 0.9460\n",
      "Epoch 5/10\n",
      "1875/1875 [==============================] - 3s 1ms/step - loss: 0.1668 - accuracy: 0.9516\n",
      "Epoch 6/10\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.1502 - accuracy: 0.9560\n",
      "Epoch 7/10\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.1374 - accuracy: 0.9595\n",
      "Epoch 8/10\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.1270 - accuracy: 0.9622\n",
      "Epoch 9/10\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.1188 - accuracy: 0.9648\n",
      "Epoch 10/10\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.1120 - accuracy: 0.9667\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x25542bf6b00>"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Train the model\n",
    "model_s_nn.compile(optimizer='adam', \n",
    "              loss='sparse_categorical_crossentropy', \n",
    "              metrics=['accuracy']) # Train on accuracy\n",
    "model_s_nn.fit(x=x_train,y=y_train, epochs=10)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test the model\n",
    "The accuracy of the model on the training set is approx 97%, which is pretty good. \n",
    "<br>To ensure that the model will actually perform well on new data and has not just 'learned' the training set, we need to test it on new data.\n",
    "Note that 'learning' the dataset is known as overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 [==============================] - 0s 1ms/step - loss: 0.1287 - accuracy: 0.9621\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.12868091464042664, 0.9621000289916992]"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Test the trained model\n",
    "model_s_nn.evaluate(x_test, y_test)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, our model performs well on unseen data. We will now turn the biases and weights to Verilog vectors."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore the weights used by the model and Quantize\n",
    "The trained model has a set of weights and biases for each neuron. The output of a neuron is being calculated as: out(n)=Sum(wn(k)*in(k))+b(n).\n",
    "<br>A 5 neurons layer which takes into input a vector of 20 elements has:\n",
    "* 5 biases, one per neuron\n",
    "* 20*5 weights, 20 per neuron\n",
    "<br><br>\n",
    "Those biases and neurons are coded as floats, which would take too much space. We will reduce these to 8 bits so that it can be stored into our board.\n",
    "<br> The verilog arrays generated are indexed as follows:  array_layer[neuron_num][weight_num]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define some helper functions that will quantize an array to N bits / print the result\n",
    "\n",
    "# Returns a quantized array\n",
    "# Arguments:     \n",
    "# use_scale       If non-zero it will use this instead of auto-computing the scaling factor.\n",
    "# Returns:\n",
    "# out             The quantized data\n",
    "# out_int         The quantized data, but scaled to an int value in the range +/- (2**n_bit-1)-1\n",
    "# scale           The scaling factor used between out and out_int\n",
    "\n",
    "def quantize_nbit(data, n_bit, use_scale=0, verbose=0):\n",
    "    max_bit_val = (2**(n_bit-1))-1\n",
    "    max_val     = np.max(np.abs(data))\n",
    "    if use_scale > 0:\n",
    "        scale = use_scale\n",
    "    else :\n",
    "        scale   = max_bit_val / max_val \n",
    "    if verbose:\n",
    "        print('Quantizing to +/- {}, scaling by {}'.format(max_bit_val, scale))\n",
    "        \n",
    "    out_int = np.around(data * scale)\n",
    "    out = out_int /  scale\n",
    "    \n",
    "    return out, out_int, scale\n",
    "\n",
    "import IPython.display as dp\n",
    "def print_nowrap(s):\n",
    "    display(dp.HTML(\"<style>.nowrap{white-space:nowrap;}</style><span class='nowrap'>\" +s+ \"</span>\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[<keras.layers.pooling.average_pooling2d.AveragePooling2D object at 0x0000025542BD3250>, <keras.layers.reshaping.flatten.Flatten object at 0x00000255737FE500>, <keras.layers.core.dense.Dense object at 0x0000025542B8BDC0>, <keras.layers.core.dense.Dense object at 0x0000025542B8B9D0>]\n",
      "Layer 2 - Array Shape/Range: Weights = (196, 32), Biases = (32,)\n",
      "Quantizing to +/- 127, scaling by 55.62836738975581\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>.nowrap{white-space:nowrap;}</style><span class='nowrap'>localparam signed [7:0] B_ARRAY_L2 [0:31] = '{ 12, 4, -6, 20, 6, 13, -4, 7, 6, 8, 8, 19, 18, 10, -7, 18, 6, -13, 4, 10, -2, -11, 14, 1, -6, 3, -18, 12, 23, 10, -8, -1 };</span>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>.nowrap{white-space:nowrap;}</style><span class='nowrap'>localparam signed [7:0] W_ARRAY_L2 [0:31] [0:195] = '{<br>{ 3, -1, 7, 24, 19, 29, 12, 1, 22, 31, 15, 6, -3, 7, -8, 9, 21, 52, 56, 44, 24, -2, -14, 10, 19, 4, 20, 18, 7, 36, 33, 9, 12, 8, 17, -2, 1, 3, -11, -8, -10, -11, 18, 23, 24, 1, -6, 7, 9, 2, -11, -10, -2, -2, 24, 31, 27, 15, 2, -5, -17, -26, -8, 6, -20, -15, -4, 2, 9, 10, 11, 3, -16, -36, -35, -14, 30, 1, -19, -8, -15, -35, 7, 43, -2, 11, 2, 5, 20, 37, 20, -21, -18, -19, -28, -41, -44, 14, 16, 30, 9, 9, 25, 2, -14, -17, -20, -9, -1, 6, 26, 22, 12, 42, -9, -46, -43, -25, -27, -6, 3, 11, 14, 22, 22, 28, 23, 33, -18, -21, -30, -29, -9, 13, 24, 13, 8, 14, -4, 1, 5, 24, -7, 10, 21, 14, 26, 28, 9, 12, 0, 22, 13, 32, 1, 49, 27, 7, 29, 32, 15, 9, -7, 3, -9, 13, -15, 22, 7, 26, 12, 17, 16, 14, 9, 13, 3, -6, -4, 4, 20, 18, -3, -1, -26, -19, -20, 9, 1, -29, -25, 0, 7, 18, -1, 5 },<br>{ 1, 6, 13, 11, 26, 46, 43, 21, 22, 34, 12, 16, -8, 6, -8, 6, -10, -4, 12, -7, -4, 3, 10, -1, -10, 10, 0, -4, 4, -37, -34, -16, 18, -4, -8, -5, 2, 1, 4, -14, 5, 20, 12, -32, -21, 2, 13, 4, 7, -19, -14, -15, -4, 14, 2, 4, 23, 13, 4, 15, 10, -3, 10, -8, -5, 2, -2, 16, 9, 29, 31, -25, 0, 23, 23, 14, 8, -11, -7, 5, 3, 13, 5, 2, 7, 15, 5, 15, 14, -15, -11, -10, -14, -8, 7, 9, 55, -4, 14, 26, 2, -23, -59, -29, 11, 8, -7, -15, -22, -9, 24, 26, 8, 12, -43, -73, -63, 27, 41, 12, -4, -8, -17, -12, 26, 4, -2, -31, -82, -50, -1, 47, 43, -2, -1, 2, -11, -8, 17, 15, 14, -21, -40, -1, -2, 24, 29, 5, 3, 10, -5, -7, -5, 4, 2, -11, 0, -10, -6, 3, 21, 15, -5, 0, 1, -7, -7, -22, 5, -29, -20, -12, -9, 0, 9, 6, 2, 1, -12, -18, 9, 5, -9, -12, -20, -6, 2, -6, -25, -23, -37, -23, -16, 17, 5, -8 },<br>{ 5, -4, -7, -4, -24, -47, -37, -1, 19, -31, -22, 1, -9, 3, 3, 0, 12, 5, 32, 33, 34, 20, 16, 4, -2, -23, 8, -1, 11, 25, 24, 24, 16, 26, 20, 28, 28, 26, 5, -17, -11, 3, 23, 1, -1, -16, 1, -1, 21, 20, 16, 11, 4, -18, -52, -14, -13, 15, -2, -5, -7, -14, -20, -14, 11, 18, 5, -9, -127, -12, -24, 24, -7, -16, -32, -25, -36, -12, 5, 28, 18, 3, -71, -6, -37, 16, -24, -26, -2, -2, 8, 10, -7, 10, 21, -25, -30, 21, 6, 13, 4, 11, -11, -22, 6, 4, 0, 5, 4, -12, -1, 27, 28, 41, 33, -2, -30, -26, -15, -12, 2, 7, 3, 0, 6, 11, -20, 29, 24, 8, -12, 3, -4, 4, 17, 8, 3, -14, -8, 20, -12, 38, 14, 15, 11, 1, -2, 1, 20, 13, 3, -20, 1, 15, 7, 48, -8, 7, 2, 18, 9, 6, 6, 13, 7, 19, 22, 6, 5, 6, 6, 16, 7, 13, 16, 13, 17, 11, 24, 34, 43, 6, -7, 13, 23, -7, -4, 9, 18, 17, 18, 17, 31, 17, 8, 8 },<br>{ 4, -5, 2, -3, -13, -8, -10, 10, -13, -22, -3, 0, 6, -4, -8, 13, 2, 6, -27, -31, 5, 3, -2, 0, 8, 3, -29, -7, 0, 14, 22, 5, -23, -2, 8, 11, 7, -9, -14, -1, 16, -6, 5, -14, -7, -20, -10, 4, 4, -1, -5, -4, -2, 0, 10, 32, -36, -12, -27, -21, 0, 7, 10, -3, -3, 6, 13, 1, 34, 35, 1, -35, -38, -27, 4, 0, 9, -3, -17, -2, 0, 32, 61, 46, -13, -34, -19, -23, 0, 19, 39, 10, 1, -6, -12, -26, -13, 2, 6, -16, -32, -58, -4, 24, 41, 26, 5, -2, -15, -30, -36, -35, 20, -1, -19, -61, -81, -27, 12, 20, 8, -7, -9, -39, -45, -19, -20, 15, 17, -14, -48, -52, -30, 3, 9, -4, -6, -41, -19, 13, 13, -6, 22, 24, 10, -3, -16, 5, 10, -2, -20, -13, -11, 8, 7, 17, 26, 11, 7, 14, 1, -1, -9, -19, -34, -39, -15, 27, 6, 36, 19, 1, 23, 24, 6, 2, -6, -36, -10, 13, 8, -4, 8, 5, 17, -2, -21, 9, -15, -3, 10, 18, 26, 18, -7, 7 },<br>{ -7, -8, -7, 0, 7, 4, 2, 27, 33, 0, -8, -1, 6, 0, -8, 13, 6, -24, 15, 17, 33, 26, 19, -12, -13, -4, 5, 3, 0, 9, -33, -10, -2, 9, 15, 21, 28, 19, 4, -16, -12, 11, 4, -16, -23, -28, -27, 6, -2, 6, 27, 40, 30, 22, -25, -18, 13, 5, -12, -28, -15, -11, 2, 7, 39, 33, 25, 38, 34, 7, 8, -8, -35, -3, -9, -10, -6, 1, -14, -1, 17, 27, 29, 5, 1, -15, -15, 4, 5, 8, 12, -24, -42, -12, -3, 11, 9, 1, -22, -7, 25, 16, 3, 13, 16, -19, -33, -24, -4, 10, 7, 28, -19, -3, 25, 17, 8, 8, -32, -21, 2, -32, -20, 1, 21, 26, 24, -23, 0, 11, 17, -8, -27, -5, -5, -27, -13, 14, 19, 20, 23, 17, 33, 20, 10, 2, 7, -3, 8, -15, 0, 21, -17, -1, -2, 43, 9, 22, 22, 22, 17, 5, 7, 8, -6, 27, 6, -23, 3, 2, -8, -3, 27, 31, 32, 16, 22, 21, 21, 16, 20, -4, -5, -3, -17, -16, 15, 28, 11, -9, -8, -1, -17, 15, -8, 9 },<br>{ 9, 6, 4, -11, -25, -24, -27, 3, -63, -34, -37, -2, -6, 2, -7, 1, -5, -24, -36, -36, -22, -5, 7, -4, 7, -16, -3, -12, 5, 9, -13, -1, -17, 4, -5, -6, -11, -12, -8, -2, 18, 7, 14, 26, 0, 7, 1, 3, 15, 8, 15, 8, 4, 13, 21, 22, 9, -6, -10, 16, 2, 7, 20, 30, 23, 22, 23, 24, 58, 36, 17, 8, 5, 19, 6, 26, 37, 35, 16, 18, 8, 8, 50, 72, 18, 22, 6, 13, 0, -2, 11, 12, 5, 8, -7, -36, -29, 29, 5, -14, 0, -3, -9, -7, -30, -14, -5, 2, -2, -5, -6, -44, 2, -17, -3, 0, 0, -15, -29, -8, -1, 3, 2, -1, -3, -27, 25, 0, 5, 10, 9, 2, 3, 1, -1, 4, 5, 4, 4, -30, -5, -27, -9, 6, 3, -8, -4, -14, -1, -11, -10, -4, 3, -29, 5, -30, -7, -4, -1, 3, -8, 2, 1, -19, -9, -10, -6, -25, -6, 34, 26, 17, 13, 20, 16, 13, 16, 12, 5, 8, 4, -18, 7, 0, -17, 1, 46, 53, 60, 63, 70, 69, 48, 27, -6, 7 },<br>{ 7, -2, 3, 3, 12, 33, 8, -2, -28, 20, 25, 29, -5, 7, 1, -7, -23, -1, -10, 0, -57, -56, -41, -10, 16, 8, 9, -1, 11, 20, 6, 3, -5, -18, -51, -21, 3, 14, 3, 0, -18, -20, 24, 19, 5, 5, 4, -10, -37, -11, 17, 20, 14, -3, -9, -3, 37, 17, -1, 4, 3, 1, -51, 1, 34, 18, -7, -21, -40, -5, 39, 37, 9, -11, 1, -48, -56, 34, 30, -3, -4, -10, -21, 4, 26, 9, 1, -16, -36, -17, 24, 22, 10, 8, 0, -16, -40, -18, 14, -14, 7, -34, -3, 20, 24, 8, 10, 4, 21, -41, -52, -22, 34, -14, 2, -5, 5, 20, 36, 5, 3, 33, 0, -50, -60, 5, 23, 15, 4, -17, -12, -2, 13, -5, 10, 1, -23, -52, -66, 10, 13, 3, 6, 4, 4, -9, 2, 5, -7, -24, -10, -35, -39, 17, 4, -11, 26, 21, 3, 3, 0, 4, -8, -22, -10, -7, -28, 27, 7, 22, 17, 5, -15, -7, 6, -18, -15, -7, -26, -21, -25, 4, -9, -14, -29, -28, -40, -9, -46, -55, -29, -41, -42, -28, 6, -2 },<br>{ -6, 2, 1, 28, 28, 45, 19, 2, 58, 28, 36, 23, -6, -3, -3, 28, 22, 45, 65, 65, 39, 25, 16, 38, 24, 51, 41, 13, -8, 12, 9, 4, 0, 15, 4, 9, 16, 16, 10, -15, 9, 44, -9, -18, -2, -10, 1, 14, 6, 6, 7, -6, -15, -19, 14, 6, -40, -10, 8, 8, 11, -2, -1, -1, -8, -14, -8, -11, -12, 3, -35, -1, 27, 18, -5, -14, -3, 11, -14, 2, 9, 22, 40, 14, -32, -4, 26, 0, -18, -27, 28, 24, 1, -4, 1, 15, 30, 18, -25, 7, 5, 3, -21, 8, 37, 14, 7, 10, 9, -16, 11, 33, -11, 9, 17, 0, -4, 14, 20, -3, 3, 11, -5, -2, 11, 22, -23, -16, -3, 25, 7, 11, 2, 5, 24, 23, -5, 16, 24, 6, -15, 13, 13, 21, 22, 19, 16, 22, 15, 15, 3, 19, 11, -14, 3, 47, 1, -17, 2, 8, 24, 0, 4, -1, 6, 13, 11, -30, 5, -7, -15, -7, -8, -2, 0, -3, 14, 19, 20, 8, 25, 1, -2, 18, 25, 1, 7, 19, 25, 15, 4, 10, 32, -3, 12, 3 },<br>{ 3, 3, -6, 12, 2, 0, 4, 15, 42, 11, 8, 4, 7, 5, 2, 2, 7, 17, 45, 31, 27, 9, -7, 16, 2, -13, 8, 10, -9, -2, -19, 4, 14, 16, 10, 14, 8, -1, 1, 15, -10, 4, 16, -5, -26, -1, 8, 18, 26, 40, 33, 25, 15, 18, 9, 6, 12, -11, -13, -19, 2, 10, 15, 36, 38, 25, 0, 10, -7, 6, 25, 2, -25, -7, -11, -22, -42, -49, -35, -23, -22, -19, -15, 34, 14, -6, -27, -25, -15, -21, -28, -34, -35, -13, -23, -37, -37, 16, -1, -3, -27, 5, -6, -2, 10, -1, -15, -28, -18, -40, -13, 29, -18, 2, -2, -16, 2, 6, 11, -7, -17, -33, -15, -22, 2, 42, 9, -7, -5, -16, -13, -17, -7, -15, 4, 4, 9, -5, 23, 41, 12, 40, 5, -5, 6, 8, 31, 25, 26, 30, 35, 23, 28, 48, -7, 38, 18, 4, 1, 27, 21, 29, 28, 31, 20, 35, 22, 4, 6, 0, -11, 4, 26, 26, 36, 34, 20, 33, 13, 8, 35, 15, 8, -5, -15, -17, 3, 13, 16, 21, 39, 21, -11, -24, 16, -1 },<br>{ -3, -3, 10, -9, 10, 4, 9, 19, -31, 14, 1, -17, -8, -2, -7, 1, -15, -43, -29, -46, -17, -12, 1, -5, -20, 17, -9, -7, -4, -16, -13, 13, -11, -19, -3, -5, -8, -13, -10, -13, 3, 3, -21, 2, -6, -10, 6, 6, 5, 14, 10, 3, 9, -9, -3, -8, -5, 4, -9, 16, 18, -2, 3, 14, -7, 8, -8, -8, -7, 7, 13, -15, 0, 9, 3, 13, 22, 29, -17, 2, 3, 6, -21, -6, 19, 24, 22, 13, 4, 2, -1, 20, 12, 4, 17, 29, -17, -25, -6, 12, 0, -18, -12, 10, 2, 11, 3, 7, -11, -29, -45, -54, -2, -22, -33, -25, 14, -1, 0, 22, 16, -7, -12, -27, -26, -12, 16, -60, -74, -24, 9, -1, 5, 8, -1, -5, -6, -38, -17, 13, -5, -57, -26, -14, -10, -17, -2, -1, -9, -13, -13, -32, -4, -8, -5, -49, 0, 4, -1, -3, 5, 4, 8, -2, -16, -11, -7, -32, 1, -20, -8, -4, -1, 1, -6, 4, -2, 8, 6, -11, -13, -9, -1, 9, 5, 24, 37, 16, 12, 38, 14, 25, 23, 20, 3, -9 },<br>{ 9, 9, 1, -6, 27, 0, 12, -7, -26, -10, -24, -24, 7, -8, -7, -2, 8, -1, -9, 8, -5, 14, 6, 7, 20, 28, -4, 14, -6, 2, -28, -25, -2, 12, -5, -13, -5, -4, 11, 15, 53, 20, -6, -33, -31, -33, -1, -9, -16, 7, -2, -7, 6, 14, 49, 73, -14, -22, -23, 2, 10, -16, -15, -9, -2, 2, 15, 8, 66, 26, -31, -31, -6, 18, 32, 5, -45, -41, -27, -18, -18, -3, 44, 22, -21, -20, -7, 18, 30, 32, 2, -13, -12, 6, -7, -16, -23, -39, -1, -3, 3, 15, 19, 24, 13, 6, 20, 4, -21, -24, -30, -14, -7, -3, 2, 8, 15, 14, 34, 1, 16, -9, -13, -18, -7, -25, -8, -6, -3, 25, 7, 16, 19, 4, -5, -3, -9, -13, -1, -20, -14, 8, -41, -29, -14, -12, 1, 1, 8, 3, 9, 25, 14, -7, -3, 12, -34, -59, -23, -5, 7, 7, 5, 10, 26, 46, 24, -3, -7, 5, -8, -12, -14, -1, -3, 11, 23, 24, 32, 47, 21, -12, 7, 11, 21, 15, 4, 12, 13, 2, -7, -23, -1, -5, -12, 7 },<br>{ 9, -6, 3, 11, 23, 30, 30, -16, 11, 34, 29, 22, -1, 6, 5, -1, 13, 5, 30, 11, -18, -6, -8, 5, 18, 13, 17, 19, -3, -5, -2, -24, -22, -25, -24, -16, -11, -1, -3, 3, 1, 1, -11, 15, 8, -12, -8, -36, -19, -5, -8, -9, -8, -2, 11, 19, 2, -11, -20, -6, -19, -23, -4, 0, -6, -10, -3, -18, 14, -21, 9, -10, -30, -7, -12, 2, 12, -3, -1, -11, -20, 1, 23, 1, 0, -42, -34, 3, 2, 32, 35, -6, -2, -3, -15, -13, 2, -12, 17, 9, 4, 21, 15, 41, 13, -3, 24, 18, -14, 16, 38, 1, -21, 12, 6, 15, 15, 20, -9, 15, 46, 30, 16, 17, 10, -9, -19, 10, 12, 15, 9, 7, 2, 28, 28, 15, 6, -8, 7, -18, 17, 24, -4, -4, 4, 5, 8, -8, -11, -14, 3, -5, -18, 4, 2, 16, 2, -15, -8, -7, -14, -32, -17, -28, -20, -21, -7, 16, -1, 3, -7, -22, -26, -32, -36, -24, -23, -6, -11, 5, 17, -1, 5, -19, -13, -38, -69, -61, -40, -55, -29, -27, -31, -12, -16, -8 },<br>{ 4, -5, -1, -3, -8, -21, -14, -13, -31, -20, -33, -8, 0, 0, -3, -5, -8, -18, -28, -38, -61, -67, -61, -27, -23, -24, -26, -3, 0, -11, -5, -10, -14, -41, -18, -26, -24, -19, 2, -3, 4, -23, -26, 8, 1, -14, -29, -7, 4, 5, 16, 18, 28, 17, -4, 23, -18, -16, 5, -7, -19, -17, 19, 16, -4, 1, 6, -2, 10, 2, -5, -16, -22, -25, -16, 3, 32, 25, 5, -28, -19, -10, 11, 11, 11, 5, -9, -2, 0, 13, -9, 24, 21, -24, -9, -15, 14, 12, 1, 5, 1, -17, -8, -2, -9, 28, 27, -12, -8, 6, -8, -31, 8, -15, -10, 8, 1, 5, 7, 19, -11, -34, -1, 6, -7, 8, 20, -26, -25, -12, -9, 8, 17, -18, -44, -28, -23, -29, 2, 31, -7, -28, -23, -16, 3, 2, 2, -14, -5, -29, -23, -26, 26, -6, -3, -33, 1, -3, 13, 11, 14, 11, 8, -11, 1, 9, -6, 5, -5, -23, -18, -14, -17, -12, -13, 1, -13, 6, 2, 15, 15, 7, 1, 10, 5, 14, 32, -6, 1, 18, 4, 11, 11, 21, 8, 5 },<br>{ 6, -6, -9, -20, -25, -36, -28, -31, -19, -28, -28, -19, 1, -7, 0, -4, -7, -55, -74, -76, -112, -79, -29, 7, 8, 5, 8, -7, 10, 13, -10, -4, -18, -18, -15, -30, -10, 0, -4, 10, 23, 5, 19, 18, 7, 8, 15, 17, 13, 14, 18, 12, 10, 7, 9, 18, -5, 16, 18, 18, 26, 11, -2, 3, 3, 9, 16, 13, 21, 11, -23, 19, 12, 19, 23, 3, -26, -35, -9, -9, 1, 2, 6, 23, 7, 7, 24, 15, 20, 24, 3, 2, 0, -3, 21, 13, -2, 9, 12, 29, 3, 15, 31, 27, 30, 17, 19, 12, 5, 2, -30, -15, 13, -12, -25, -18, 3, 13, 35, 17, 23, -3, 1, 7, -30, -22, -18, 10, -26, -38, -64, -39, -28, -9, -9, -15, -2, -18, -36, 0, 8, 0, -14, -37, -30, -41, -14, -14, -12, -16, 8, -2, -5, -13, -10, 20, 6, 10, 7, 5, 1, -2, -19, 6, -5, 9, -8, 5, 7, 28, 30, 32, 22, 49, 30, 15, 19, 7, 9, 20, -33, -24, 4, -2, 31, 42, 47, 56, 51, 40, 44, 17, 8, 17, -1, 6 },<br>{ 2, -6, 10, -4, 9, 6, 27, 24, -9, 21, 6, -1, -8, -3, 5, 6, -13, -9, -18, 2, -9, 1, 11, 0, -18, 10, -5, -14, -1, 0, -7, -6, -4, -6, -2, 5, 11, 21, 19, 8, -14, -41, -9, -3, 5, 0, -20, 12, 14, 1, 7, 24, 20, 6, -28, -67, 13, 3, 8, 12, 0, -2, 2, -9, 6, 32, 28, 2, -15, -61, 15, 15, 5, 0, 6, -11, -17, -50, 3, 11, 20, 10, -15, -73, -15, 8, 14, 21, 1, -16, -46, -52, -22, -14, 1, 19, 15, -41, 10, -6, 19, 22, 16, -26, -30, -26, -29, -5, 21, 17, 13, -12, 2, -8, 9, 30, 30, 0, -5, -14, -9, 21, 23, 16, -11, -10, 14, 5, -6, 12, 33, 31, -12, -14, 22, 22, 18, -3, -6, -13, 13, 18, 14, 14, 11, 19, 4, 1, 24, 10, 4, -1, 8, 2, -10, -19, 5, 19, 7, 6, 10, 20, 13, 4, 8, 10, -2, -18, -4, 12, 9, 2, -6, 6, -5, -9, -16, -27, 0, 13, -13, 8, -8, -4, 25, 26, -3, -35, -17, -30, -73, -30, 6, -7, 5, 3 },<br>{ -3, -2, -7, -10, -12, 0, -10, -3, -6, -34, -35, -1, 8, -3, 4, 10, -4, -34, -16, -36, -37, -30, -21, -34, -17, -23, -3, 24, 0, -13, -29, -34, -33, -26, -30, -46, -49, -36, -34, -31, -8, -3, 1, 30, -27, -14, -4, -2, 13, 11, 15, 8, 3, -7, -4, 11, 35, -1, -18, -21, -4, 20, 28, 31, 28, 35, 12, 22, 42, 18, 21, 12, 0, 14, 16, 19, 28, 16, 14, 11, 19, 13, 57, 67, 20, 30, 14, 5, -2, 11, 10, -7, -20, -14, -21, -24, -34, 56, 8, 0, 7, 6, -1, 1, -7, -27, -10, 7, 5, -4, 26, 21, -19, -13, 14, 1, -12, 7, -6, -11, 23, 10, 1, -3, 12, 23, 29, -30, 3, 5, 2, 0, -1, -3, -4, 6, 4, 18, 6, 4, 20, -27, 2, -19, -17, -23, -5, -20, -9, 0, -7, -1, 10, -4, -5, 10, 0, 4, -8, -3, -11, -21, -18, -29, -29, -1, 25, 5, -6, 28, 33, -1, -3, 1, 2, 12, 13, 12, 14, 16, 19, -12, 1, 4, -5, 5, 22, 47, 59, 71, 66, 59, 66, 41, 10, 3 },<br>{ 9, -3, 7, 1, 2, 2, -2, 15, 18, 37, 34, 15, 6, -9, -2, 8, 20, 21, -1, -9, 10, 6, 35, 21, 25, 20, 4, -1, 3, 29, 7, 8, -10, 3, -4, 19, 18, 37, 41, 31, 28, 31, 3, -4, 2, 2, -8, -11, 3, 14, 0, 11, 5, 34, 46, 0, -28, 4, -13, -1, 12, 1, 11, 7, 6, -2, -2, 10, 14, 14, -17, -6, -18, -7, -3, -4, 28, 20, -3, 1, -15, 12, 35, 16, -35, -17, -29, -4, -14, -3, 45, 13, -4, -1, -21, -22, 7, -7, 5, -8, -28, -11, -11, -13, 22, 7, -20, -25, -12, -32, 1, -39, 21, 6, 1, -11, -9, 3, 2, -26, -18, -10, -9, 11, -10, -56, -24, 20, 14, 13, -5, -8, -23, -24, 1, 12, 11, 12, -18, -39, 6, -2, 10, 23, 19, 10, 16, 13, 24, 39, 28, 11, -17, -18, 4, 20, 7, 2, 8, 25, 25, 27, 23, 19, 6, -7, -34, 20, 7, 24, 2, 9, 18, 14, 28, 11, 10, -12, -19, -22, -37, -13, 7, 3, 7, -38, -27, -14, -21, -23, -4, -11, -16, -12, -8, 1 },<br>{ -5, -3, -5, -19, -23, -26, -25, -10, 10, -11, -17, 4, -9, -7, -4, 0, 9, -14, 22, 11, 2, -27, -31, 0, 14, 19, 17, 18, 3, 22, -29, -5, 3, 23, 18, 17, 31, 17, 20, 3, 22, 26, 10, -4, 10, 11, 19, 17, 5, 15, 11, 10, 6, 7, 9, 8, -28, 8, 15, 21, 20, 9, 13, -5, 2, 0, -3, 8, 32, 27, -41, 4, 28, 12, 13, 4, 5, -16, -13, 7, -7, -18, 8, 24, -25, 13, 2, -6, -20, 0, -7, -11, -8, 22, 9, -9, 7, 30, -2, -20, -50, -37, -10, -6, 3, -12, -41, -8, -2, -11, 18, 9, -10, 6, -20, -34, -11, 0, 13, -17, -11, 5, 15, 20, 18, 2, -5, 4, 30, 8, -7, 3, -14, -20, 4, 4, 18, 29, 0, 2, 6, 7, 10, 17, 3, -2, 9, 9, 15, 0, 7, 23, -1, -1, -10, 32, -10, 16, 9, 19, 27, 37, 27, 1, 10, 21, 17, -5, 6, 5, -8, 16, 30, 32, 44, 41, 34, 24, -4, 13, 2, -16, -6, -10, -5, -14, -42, 13, -6, -31, -2, -12, -14, -4, 2, 9 },<br>{ -8, 5, 9, 29, 43, 32, 23, -22, 43, 56, 29, 21, -3, 4, 3, -21, 7, 24, 36, 42, 12, 7, 2, -2, 9, 9, 3, 3, 6, -17, 12, 8, 16, 5, 8, -4, 11, 6, 0, 12, -4, -13, 9, 27, 15, 8, 6, 3, -1, 3, -4, -9, -3, 1, 2, -18, 18, 24, 9, -21, -4, 11, -4, -6, -19, -8, 4, 14, -14, -32, 16, 0, 4, -16, -9, 10, -4, -24, -7, -3, 13, 12, -46, -63, 16, -8, -15, 1, 6, 17, -14, -22, 6, 6, 10, 50, 49, -23, 5, -14, 4, 17, 18, 13, 9, 18, 8, 21, 20, 46, 26, 44, 2, -12, -6, 14, 27, 20, 17, 18, 11, 14, 33, 17, 33, 21, 7, 4, 2, -6, 5, 13, 14, 2, 3, -6, 6, 7, 0, 14, 11, 12, 1, -9, 5, 19, 8, 7, -6, -21, -22, -17, -38, 0, -4, -2, -9, 5, -3, 6, 9, 15, -10, -27, -31, -25, -16, -11, 1, -5, 4, 8, 0, 5, 0, 14, -6, -55, -50, -32, -15, 3, 3, -22, -21, -10, -2, -12, -26, -22, -27, -45, -27, -4, -9, 1 },<br>{ -6, 2, -2, 20, -9, -9, -17, -1, 14, 8, 4, 16, -5, 8, 8, -4, 13, 11, -8, -22, -39, -54, -50, -30, -21, -16, -9, 2, 5, 15, 33, -3, -9, -30, -29, -30, -18, -5, -10, -6, -33, -9, 11, 25, 6, 12, 16, 4, 7, 6, 1, -5, -6, -15, -53, -22, 16, 20, 27, 30, 40, 37, 51, 23, -20, -16, 8, 15, -48, -8, 18, 21, 45, 42, 41, 41, 26, 7, 5, 7, -4, -2, -33, 9, 18, -12, -10, -17, -30, -46, -63, -16, 14, 8, -19, -20, 2, 41, 11, -19, -50, -47, -53, -14, -12, -1, 5, 11, 11, -3, 3, 28, -1, -20, -52, -25, -8, -10, -1, -1, -3, 9, 0, -1, 15, 32, 26, -10, -6, -7, -7, -1, -5, 9, 1, -4, -1, -7, -4, 42, 9, 5, 1, -2, -1, 0, 1, 6, -6, -8, -13, -6, 24, 27, -3, -23, -3, 25, 9, -2, -2, 7, 11, -6, -9, 2, 19, -15, -8, -9, 33, 21, -3, -19, -12, -14, 9, -1, 2, -13, 20, 6, 7, -15, -8, 13, 43, 19, 23, 35, 15, 41, 24, 22, -1, -1 },<br>{ 2, -3, 0, -6, 29, 0, 14, 5, -4, 18, -11, -14, -3, -7, 5, -3, 8, 16, -21, -8, 11, 24, 31, 18, 29, -8, 4, 5, 2, 11, 13, 13, 14, 6, 2, 23, 12, 21, 6, 3, 24, -15, 31, 15, 12, 14, 11, -19, -22, -12, -6, -4, -1, 5, 22, 47, 39, 5, 15, 16, -5, -22, 18, -5, -24, -2, 6, 14, 31, 7, 46, 22, -1, -6, -4, 15, 44, 9, 2, 4, 9, 6, 15, 8, 40, 14, 3, 11, 28, 33, 5, 6, 23, 5, -14, -16, -22, 2, 16, 0, 12, 16, 17, -15, -33, 6, 9, 9, 9, 22, -2, -30, 16, 12, 12, 18, -4, -31, -32, -7, -5, 8, 16, 20, -9, -42, 21, 29, 17, 8, 16, -18, -16, 10, 9, 10, 18, -1, -21, -27, 5, -9, -3, 7, 3, 7, -6, 19, 19, 14, 2, -16, -35, -23, -2, -18, -25, -21, -29, -14, -3, 30, 31, 23, -14, -45, -51, 19, 0, -3, 13, 5, -1, -3, 0, 20, 4, -15, -24, -44, -27, 21, 0, -13, -23, -6, 9, 6, 7, -8, 17, 1, -1, 24, -19, 0 },<br>{ -9, 3, 1, 34, 34, 49, 62, -22, -8, 37, 25, 22, -5, 2, 0, 4, 10, 2, 0, 25, 9, -5, -9, 26, 33, 65, 9, 9, -2, -36, -33, -10, 0, -1, 11, 2, 11, 16, 14, 17, 36, 29, -39, -31, -20, -12, -5, 2, 18, 25, 28, 10, 3, -1, 32, 1, -36, 2, -13, 1, 6, 5, 1, 4, -4, 3, -2, -22, 26, -4, -41, -37, 2, 18, 19, 10, 9, -20, -11, -6, -5, 7, 15, -18, -31, -16, -7, 17, 10, 10, 19, 3, 9, 11, 24, 38, 4, -53, -11, -45, -20, 0, 8, 28, 6, 5, 16, 31, 3, 6, -14, -43, -23, -35, -32, 10, 25, 32, -2, -14, 28, 7, 2, -7, -13, -42, -17, -37, 2, 5, 0, 19, -9, -2, 12, 0, -5, -14, -38, -29, 14, -39, -3, 8, 1, -4, 18, 19, -9, -12, -21, -3, -44, -23, -17, 1, -19, -7, 0, 20, 27, 22, 9, 2, -9, 0, -17, -20, 8, 0, -7, -12, -13, 7, 10, 16, 11, 13, -2, -10, -21, -26, -5, 4, 26, 5, -14, -4, -2, -9, 0, -15, -7, -26, 4, 8 },<br>{ -2, 7, -10, -25, -29, -23, -35, -34, -45, -25, -17, -7, 9, -4, -4, -25, -14, -44, -52, -23, -36, -27, -2, -20, 1, -19, -29, 0, -13, -14, -21, -13, -3, 9, -2, 3, 0, -5, 4, -6, -7, -23, -14, 16, 11, 8, 9, 5, -3, -21, -12, 10, -8, 13, -9, 10, 23, 14, 16, 3, 8, 8, 2, -33, 15, 16, -8, 4, 11, -12, 46, 5, 0, 1, 13, 18, 7, -61, 14, 24, 1, -12, -24, -16, 31, 6, -5, 10, 7, 29, 19, -67, 1, 23, 14, -4, -17, 10, -9, -3, 7, 22, 16, 24, -18, -41, 12, 19, 0, -5, -14, 3, -22, -13, 19, 11, 4, -9, -54, -6, 24, 11, 2, -15, -23, -21, 13, -5, 22, 1, -18, -46, -27, 16, 15, 6, -11, -14, -16, 13, -5, 11, -9, -20, -27, -9, 13, -1, 4, 1, -9, -7, -1, 21, -2, -5, -11, -10, -4, 11, 11, 5, 4, 3, 9, 9, 24, 5, 9, -10, -27, -8, -4, -2, 7, 3, 12, -4, 1, -8, -17, 3, 2, -5, -29, -23, -22, -18, -12, -22, -13, -10, -29, -6, -18, -1 },<br>{ 1, -7, -9, 5, -17, 1, -20, 12, 19, -2, 4, 4, 7, 8, -4, 21, 17, 33, 23, 10, 11, 5, 12, 19, 22, -17, 0, 2, 1, 27, 41, 16, 5, 0, 8, 7, 7, 3, 1, 9, 6, -12, 26, 34, 11, 2, 17, 22, 9, -2, 6, 8, 1, 1, -11, 11, 11, 3, 4, 7, 4, 10, 8, 1, 3, 0, -5, 10, -26, 5, -1, -4, -10, -3, -22, -32, -52, 1, -12, -20, -27, -18, -25, 7, 12, -17, -67, -83, -68, -60, -21, 16, 2, -3, -15, -28, 14, 44, -1, -23, -38, -23, -23, 2, 23, 30, 14, 19, 24, 20, 39, 32, 13, 22, 13, 26, 15, -1, 28, 5, 7, 19, 24, 20, 39, 49, -7, 40, 35, 31, 29, 10, 2, 10, 4, 4, 2, -6, 34, 45, -15, 44, 7, 17, 17, 0, 7, 16, 7, -2, -9, -3, 21, 14, 14, 10, 27, -5, -15, -4, -14, -10, 0, -7, -14, -9, -5, 35, 9, 33, 18, 7, 10, -1, 2, -1, -7, -29, -12, -21, 0, 13, 9, -4, -19, 9, 18, 21, -2, 17, 5, 9, 1, -29, 0, 1 },<br>{ 7, 0, 3, 18, 31, 49, 56, 47, 73, 34, 7, 17, -9, -7, 5, 18, 10, -6, -3, 4, 20, 23, 14, 39, 44, 24, 34, -2, -5, -13, -39, -16, -12, 16, 10, 14, 18, 25, 22, 38, 42, 39, 0, -17, -28, -1, 15, 21, 28, 31, 16, 14, 8, -9, 25, 31, 7, -19, -21, -7, 15, 32, 9, -24, -39, -57, -63, -77, -49, 18, -26, -15, -1, 17, 11, 3, -29, -24, 13, -15, -31, -49, -71, 3, -21, 2, 7, 13, -1, -12, -25, -7, 14, 28, 28, 32, -20, -23, -2, -21, -3, -2, -2, -2, -24, -17, 4, 8, 20, 20, 12, -39, -17, -4, -5, -15, -12, 0, -17, -18, 9, 17, 6, -2, 3, -47, -20, 9, 3, -4, -12, 0, -35, -2, 13, 0, 4, -2, 25, -38, -24, 3, 13, -2, 4, 5, 0, 16, 14, 2, 2, 5, 32, -17, -4, -7, 3, -7, 4, 5, 7, 14, 8, -3, 0, 6, 18, 16, -8, 4, 5, 2, -7, -9, 1, -6, 1, 4, 28, 25, 11, -12, 6, -5, -2, -10, 20, -14, 4, 19, 24, 32, 34, 20, 18, 4 },<br>{ -5, 4, 4, 19, 38, 42, 39, 23, 24, 46, 34, 19, 7, -8, -1, 13, 13, 35, 41, 37, 43, 33, 11, 12, 18, 35, -12, -7, -3, 2, 0, -5, -9, -9, -8, -10, -17, -17, -36, -35, -49, -34, -7, -30, -30, -14, -17, -7, 4, 18, 5, -13, -24, -47, -63, -67, 7, -10, -15, 0, 7, 4, 13, 42, -3, -20, -17, -27, -56, -35, 1, 14, 3, 17, 9, 10, 19, 11, -23, 9, 17, 9, -10, -39, -11, 24, 8, 11, 7, 4, 0, -8, -3, 4, 5, 21, 6, -42, -11, -16, 8, 9, 15, 16, 3, -6, -12, -6, 13, 9, 15, -41, 5, -26, 3, -5, 3, 31, -1, -12, -10, 10, 2, -4, 4, -10, 24, -21, -35, -28, -6, 6, 2, 11, -5, 11, 11, -1, 3, -3, 9, -18, -16, -21, -15, -3, 30, 21, -3, -4, -6, -17, 12, 14, 5, -12, -24, -34, -19, -9, 1, -7, -11, -27, -17, -1, -9, 5, -7, -7, 5, -16, 6, -6, -2, -16, -1, -2, 6, 6, 14, 14, 0, 12, 10, 15, 33, 35, 15, 33, 32, 17, 35, 11, 13, 0 },<br>{ -1, 9, -8, -9, -13, -19, -8, -8, 19, -46, -34, -17, 0, -9, 1, -6, -12, -38, -33, -21, -2, 5, 4, -15, -23, -11, 3, -2, -1, 0, -9, -1, -10, 4, 20, 28, 24, 15, 2, -20, 3, 3, 30, 25, 0, -5, 0, 16, 21, 24, 23, 17, 8, -1, -5, 1, 22, 18, -9, 5, 5, -2, -11, 2, 11, 32, 17, 15, -3, 9, 15, 32, 1, 2, -7, -34, -21, -6, -3, 28, 25, 9, -25, -7, 13, 17, 3, -11, -18, -16, -5, -13, -3, 25, 29, -7, -2, 17, 20, 0, -5, -2, -10, -1, 1, 9, 8, 17, 7, -5, -5, 24, 19, 9, -6, -8, 6, 2, -1, 0, 16, -6, -21, 6, -3, 21, 25, 12, 4, -5, 5, 2, -19, -2, 1, -8, -12, -12, -14, -13, 18, 8, 19, 22, 3, -14, -8, -5, -9, -15, -18, -20, -15, -4, -1, 4, -13, 22, 10, 5, 8, -1, -5, -7, -19, 2, 12, -20, 4, 14, 16, 30, 7, 21, 14, 18, 17, 5, -6, -20, 9, 6, 9, -9, -13, 27, 54, 31, 16, 17, 23, 39, 18, 2, -1, -1 },<br>{ -7, -8, -11, -14, -25, -30, -46, -13, -11, -30, -17, -7, -5, 0, -2, 4, -5, -4, 5, 3, 0, -27, -27, -63, -59, -43, -26, -9, 3, 38, 9, 9, 4, 23, 22, 6, -13, -26, -43, -35, -21, -17, 21, 43, 13, -1, -1, 8, 30, 25, 19, -12, -34, -61, -61, -5, 9, 16, 7, -11, 0, 1, -4, 41, 40, 13, -44, -75, -99, -5, 10, 39, 6, -10, -16, -13, -10, 28, 55, 26, -29, -69, -49, 11, 2, 27, -15, -16, -10, -7, 1, 28, 27, 9, -34, -46, 15, 43, 9, 28, -13, -10, 0, -11, -5, 15, 20, -8, -25, -6, 0, 41, 13, 13, -16, 0, 1, -14, 12, 17, 15, -10, 1, 20, 17, 57, 22, 16, -5, -10, -15, -4, -5, -8, -3, 1, 11, 7, 2, 35, 16, 22, -12, -14, -9, -8, -18, -18, -7, 3, 6, 1, 18, 16, 13, 27, 10, -4, -10, 3, -9, -10, -13, -3, 5, 6, 19, -11, -9, 2, 32, 13, -15, -10, -9, 6, 0, -7, 22, -2, 11, 26, 9, 9, 14, 22, 67, 38, 25, 29, 31, 33, 16, 11, 7, -7 },<br>{ -7, 0, -6, -2, 18, 7, -4, 15, -17, 26, -1, -12, 4, 8, -4, -7, -10, -7, -10, -9, -4, 16, 0, -8, -12, 2, -8, 7, 3, -2, -1, -15, -29, -18, -9, 8, 8, 12, 11, 9, 1, -4, -16, -19, -20, -44, -37, -17, -16, -2, -5, -3, 1, 6, 21, 22, -32, -28, -32, -26, -29, 0, 7, -7, -6, -11, -7, -16, -11, -9, -21, -49, -27, -13, 0, 6, -10, -5, -3, -23, -19, -50, -45, -8, -15, -20, -7, 9, 15, 16, 10, -1, 0, -7, -8, -17, -33, -17, -17, 16, 23, 15, 8, 0, 16, 30, 2, -25, -19, -25, -8, 36, -22, 21, 20, 11, 16, 5, 11, 26, -6, -19, -3, -9, 8, 41, -24, 17, 2, 4, 8, -13, 11, 16, 2, 5, 12, 7, 45, 25, -8, 19, 9, 9, 4, -8, -21, 6, 32, 36, 35, 24, 29, 9, -5, 6, 17, -2, -3, -10, -17, -4, 15, 33, 43, 8, -7, 11, -4, -32, -22, -15, -21, -11, -28, -23, -27, -17, 3, 10, -4, 25, -3, -13, -18, -38, -35, -25, -40, -40, -25, -33, -28, -30, -7, -7 },<br>{ 9, 3, -7, -12, -12, -11, 1, 17, -4, -30, -31, -18, -4, -1, -5, -16, -8, 8, -5, -2, 10, 21, 11, -19, -17, 4, 6, -4, 0, -6, 2, 2, -7, 2, 19, 17, 18, 22, 18, 6, 4, -3, -9, -23, 6, -2, 2, -6, -2, -3, -10, -3, -9, 0, -9, -11, -26, -3, -5, -8, -1, 6, 6, -11, -15, -8, 1, -8, -33, -11, -28, -6, 1, 2, -2, -4, -2, -15, 6, 1, -17, -27, -32, -31, -13, -16, -11, -7, -2, 3, 3, 2, 10, 5, 18, 11, 25, 5, -8, 7, -7, 2, 10, 16, 18, 20, 16, -4, -1, -6, 5, 62, -9, 17, -2, 2, 27, 19, 27, 34, 4, -7, 2, -4, 17, 38, -20, 13, 9, 6, 11, -12, -4, 17, 9, -1, 7, 20, 30, 34, -2, 29, 19, 23, 6, -7, -10, 16, 13, 13, 23, 29, 7, 15, 2, 11, 23, 13, 12, 0, -9, 13, 12, 18, 24, 24, -4, 1, -3, -45, -60, -44, -23, -10, -16, 3, 0, 9, 5, 0, -2, 12, 7, -17, 2, -23, -47, -38, -61, -72, -66, -69, -74, -36, -3, 1 },<br>{ 0, 5, -12, -14, 8, 7, 15, 27, 13, -33, -37, -7, -8, 9, -5, 11, -4, 13, 26, 50, 44, 36, 24, 17, -16, -9, 0, -7, 1, 12, -26, -5, -43, -67, -52, -28, -10, 15, 2, -14, -21, 0, -5, 2, -23, -39, -96, -86, -29, 17, 29, 36, 25, -9, -30, -12, -23, -6, -41, -68, -39, -4, -1, -8, 5, 9, 6, 23, -16, -6, -13, -16, -37, -22, 5, -1, -36, -31, 2, 10, 11, 3, -7, 6, -30, -10, -23, -5, 3, -1, -24, 23, 17, 26, 17, 6, 13, 19, -16, -25, 1, -2, -4, 5, 14, 22, 20, 2, -8, -28, -11, 24, 23, -13, 4, 14, 10, 21, 22, -1, 0, -12, -28, -24, 1, 10, 8, -1, -4, -5, 10, 2, -1, 2, -2, 3, 6, 16, 18, 21, 2, 9, 26, 2, -13, -19, 3, 2, -2, -1, -16, 11, 19, -22, 3, 13, 6, -1, -10, -24, -21, -10, -14, -16, -17, -10, -23, -12, -7, -11, 10, 4, 6, -12, 0, 6, 3, 0, 5, 3, -27, -3, 6, 14, 14, 34, 25, 28, 29, 58, 56, 50, 70, 6, -22, 2 },<br>{ -5, -6, 6, 27, 32, 46, 29, -14, 3, 40, 39, 29, 2, 9, -6, 5, 16, 31, 16, 21, 23, 21, 3, 17, 25, 6, -3, -4, -1, 9, 20, 20, 13, 0, -13, -15, -21, -15, -15, -28, -19, -39, 6, 13, 10, 7, 20, 1, 7, 10, 9, -18, -15, -5, -13, 10, -14, 24, 5, 3, -1, -20, -1, 3, -6, -15, -11, -9, -30, -15, -5, 12, -19, -4, -15, -10, 5, -5, 4, 9, 4, -24, -61, -38, 3, -3, -29, 8, 2, 20, 11, 11, 25, 28, 25, 5, -38, -37, -4, 7, -11, -8, 5, 23, 14, 1, 28, 21, 23, -8, -61, -41, 25, -19, -50, -32, 2, 25, 0, 12, 15, 1, 1, -2, -41, -29, -15, 25, -8, -28, -43, -1, 10, 7, 6, -8, -19, -28, -51, 15, 6, 37, -16, -11, -20, -18, 3, 2, 3, -18, -30, -41, -17, -10, -5, 15, -9, -4, 9, 5, -7, -10, -11, -12, -27, -18, -17, 18, 1, 29, 33, 28, 20, 11, -4, -13, -6, -3, 0, -17, -22, 1, -1, 15, 44, 46, 38, 23, 9, 10, 18, 14, 0, -3, 6, 5 }<br>};<br></span>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Layer 3 - Array Shape/Range: Weights = (32, 10), Biases = (10,)\n",
      "Quantizing to +/- 127, scaling by 45.52645992341783\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>.nowrap{white-space:nowrap;}</style><span class='nowrap'>localparam signed [7:0] B_ARRAY_L3 [0:9] = '{ -18, 10, 6, -9, -2, 16, -1, 3, -3, -8 };</span>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>.nowrap{white-space:nowrap;}</style><span class='nowrap'>localparam signed [7:0] W_ARRAY_L3 [0:9] [0:31] = '{<br>{ -17, -7, -22, -31, 44, -6, 6, -15, 22, -16, -26, 29, -29, -57, 35, 7, 5, 17, 13, -27, -11, 10, 5, -73, 0, 30, 16, 17, -51, -15, -14, 24 },<br>{ 27, 17, -17, 13, -12, -7, 30, -2, 8, 17, -61, -52, 26, -36, -2, -53, 15, -39, -52, -14, 26, -55, 17, 19, 15, 47, -28, 9, 37, 23, 33, -32 },<br>{ 32, 28, 30, -100, 24, -31, -61, 20, 32, -31, -6, 35, -14, -31, -18, 2, -12, 18, 20, 50, -49, -22, 22, 18, -55, -12, 13, 21, 29, 19, 21, -15 },<br>{ 43, -44, 35, 17, -2, 8, 9, 6, 7, -31, -11, -18, -73, 32, -35, -1, 16, 16, -11, 22, 32, -16, -72, 20, -45, 9, 21, 19, -29, -7, 19, 0 },<br>{ -35, 0, -30, 26, -55, -6, 9, -34, -127, -4, 31, 14, 18, 36, 23, -18, -16, -16, 19, -56, 10, 9, 47, -21, -57, -45, -9, 6, 22, 26, -52, 24 },<br>{ 29, -17, -18, 37, 4, 28, -19, 6, 25, -29, 33, 24, 7, 15, -43, 38, 7, 28, -27, -34, 9, -5, -23, 32, 63, -24, -39, -76, -5, -19, -42, -45 },<br>{ 9, 22, -49, -30, -6, -39, 24, 19, 12, -12, 19, 23, -12, -59, 7, -59, 5, -17, 33, -54, 1, 11, -27, -17, 2, 23, -44, -74, 22, 11, -92, 41 },<br>{ -6, 8, -19, -16, -2, 22, 23, -45, 12, 13, -41, -29, 22, 21, -4, 16, -68, -21, 17, 34, 45, -19, 44, 42, -1, -10, 19, 19, -44, -19, -35, 12 },<br>{ -79, 30, -12, 6, 13, 5, -49, 30, -25, 19, 24, -41, -4, -1, -58, -4, 18, 25, 5, -46, -6, 27, -34, -27, -43, -41, 20, -6, -5, 21, 14, -57 },<br>{ -64, -78, 19, 28, -7, 9, -73, 7, -51, 14, 19, -34, 22, 24, 23, 15, -11, -14, -26, -15, -18, 16, -15, -81, 28, 3, 16, 34, -20, -16, 55, 24 }<br>};<br></span>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Quantize and Export the weights\n",
    "n_bit               = 8            # Number of bits to quantize to\n",
    "quantized_wb        = [0,0,0,0]\n",
    "quantized_wb_scale  = [0,0,0,0]\n",
    "\n",
    "print(model_s_nn.layers)\n",
    "\n",
    "for l,layer in enumerate(model_s_nn.layers):\n",
    "    if len(layer.weights)>0:\n",
    "        w,b = layer.weights\n",
    "        w,b = w.numpy(), b.numpy()\n",
    "        print('Layer {} - Array Shape/Range: Weights = {}, Biases = {}'.format(l, w.shape, b.shape))\n",
    "        w_q, w_qi, scale = quantize_nbit(w, n_bit, verbose=1)\n",
    "        b_q, b_qi, _     = quantize_nbit(b, n_bit, use_scale=scale)\n",
    "        \n",
    "        # Print the scaled values\n",
    "        num_w, num_neuron = w.shape\n",
    "        \n",
    "        # Print the Biases as an SV parameter array\n",
    "        s = \"localparam signed [{}:0] B_ARRAY_L{} [0:{}] = '{{ {} }};\".format(n_bit-1, l, num_neuron-1, ', '.join(str(int(e)) for e in b_qi))\n",
    "        print_nowrap(s)\n",
    "        \n",
    "        # Print Weights as an SV parameter array\n",
    "        s = \"localparam signed [{}:0] W_ARRAY_L{} [0:{}] [0:{}] = '{{<br>\".format(n_bit-1, l, num_neuron-1, len(w_qi[:,0])-1)\n",
    "        for n in range(num_neuron) :\n",
    "            # Note: you can change\n",
    "            #s += \"Layer {}, Neuron {}, Bias = {}, Weights = {}<br>\".format(l, n, int(b_qi[n]), ', '.join(str(int(e)) for e in w_qi[:,n]))\n",
    "            #s += \"bias_l{}[{}] = {}; weight_l{}[{}] = {{ {} }}<br>\".format(l, n, int(b_qi[n]), l, n, ', '.join(str(int(e)) for e in w_qi[:,n]))\n",
    "            s += \"{{ {} }},<br>\".format(', '.join(str(int(e)) for e in w_qi[:,n]))\n",
    "        s = s[0:-5] # remove last comma\n",
    "        s += \"<br>};<br>\"\n",
    "        print_nowrap(s)\n",
    "        \n",
    "            \n",
    "        # Save the quantized weights/bias for use later\n",
    "        quantized_wb[l]       = (w_qi, b_qi)\n",
    "        quantized_wb_scale[l] = (w_q,  b_q)\n",
    "        "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The weights and biases are available above and can be added to the Verilog code. "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creation of test vectors"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function transform images in test vectors to be used in the Verilog code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write data out in the form\n",
    "# logic signed [7:0] data [0:N-1] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 37, 4, 0, 0, 0, 0, 53, 0, 0, 0, 0, 0, 0, 0, 82, 9, 0, 0, 0, 0, 95, 0, 0, 0, 0, 0, 0, 26, 98, 0, 0, 0, 0, 20, 102, 0, 0, 0, 0, 0, 0, 56, 73, 0, 0, 0, 0, 39, 86, 0, 0, 0, 0, 1, 32, 114, 40, 0, 0, 0, 0, 38, 103, 51, 63, 83, 91, 89, 55, 121, 4, 0, 0, 0, 0, 0, 36, 44, 44, 19, 0, 0, 33, 107, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 42, 77, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 42, 87, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 42, 101, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 50, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};\n",
    "def test_data_to_verilog(n_bit, data, show_img=0, suffix=''):\n",
    "\n",
    "    # Input data - Flatten into 1-d vector\n",
    "    m_output = data.reshape(1, np.prod(data.shape)) \n",
    "    m_output = m_output[0]\n",
    "\n",
    "    # check the output shape is 1D\n",
    "    if m_output.shape[0] != m_output.size:\n",
    "        print('Error: Model output is not 1D. Check the model and layer requested')\n",
    "    \n",
    "    # Show the image if requested\n",
    "    if show_img:\n",
    "        plt.subplot(111)\n",
    "        dim=int(np.sqrt(m_output.size))\n",
    "        plt.imshow(m_output.reshape(dim,dim), cmap='Greys')\n",
    "        plt.show()\n",
    "    \n",
    "    # Quantize\n",
    "    data_q, data_qi, scale = quantize_nbit(m_output, n_bit)\n",
    "    \n",
    "    # Print this arry to verilog\n",
    "    s = \"logic signed [{}:0] test_data{} [0:{}] = '{{ {} }};\".format(n_bit-1, suffix, data_qi.size-1, ', '.join(str(int(e)) for e in data_qi))\n",
    "    print_nowrap(s)\n",
    "    \n",
    "    return data_q, data_qi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAam0lEQVR4nO3dfUyV9/3/8dfxhuMdHIcIByY61FZdVcycMmL1awdDWea8+0Nrl2hnNDpsqs61cWu1uiVsNukaDdMs2bRN6s1MqqZmM7NYMN3QRapxpi0Tw6oOwdUNDmJFI5/fH8az31GsXngObw4+H8mVyDnXh+vda1d47vIcDz7nnBMAAB2sm/UAAIDHEwECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmelgPcLfW1lbV1tYqMTFRPp/PehwAgEfOOTU1NSkjI0Pdut3/PqfTBai2tlaZmZnWYwAAHtGFCxc0aNCg+z7f6QKUmJgo6fbgSUlJxtMAALwKhULKzMwM/zy/n5gFqKSkRK+//rrq6uqUnZ2tLVu2aOLEiQ9cd+ev3ZKSkggQAMSxB72MEpM3IezZs0erV6/W+vXr9dFHHyk7O1vTpk3T5cuXY3E4AEAcikmA3njjDS1ZskTPP/+8vv71r2vbtm3q06ePfv/738ficACAOBT1AN24cUOVlZXKz8//30G6dVN+fr4qKiru2b+lpUWhUChiAwB0fVEP0Oeff65bt24pLS0t4vG0tDTV1dXds39xcbECgUB44x1wAPB4MP+HqGvXrlVjY2N4u3DhgvVIAIAOEPV3waWkpKh79+6qr6+PeLy+vl7BYPCe/f1+v/x+f7THAAB0clG/A0pISND48eNVWloafqy1tVWlpaXKzc2N9uEAAHEqJv8OaPXq1Vq4cKG++c1vauLEiXrzzTfV3Nys559/PhaHAwDEoZgEaN68efr3v/+tdevWqa6uTuPGjdOhQ4fueWMCAODx5XPOOesh/n+hUEiBQECNjY18EgIAxKGH/Tlu/i44AMDjiQABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDRw3oAAPDik08+8bwmPz+/Xcc6deqU5zUDBw5s17EeR9wBAQBMECAAgImoB+i1116Tz+eL2EaOHBntwwAA4lxMXgN66qmn9P777//vID14qQkAECkmZejRo4eCwWAsvjUAoIuIyWtAZ8+eVUZGhoYOHarnnntO58+fv+++LS0tCoVCERsAoOuLeoBycnK0Y8cOHTp0SFu3blVNTY0mT56spqamNvcvLi5WIBAIb5mZmdEeCQDQCfmccy6WB2hoaNCQIUP0xhtvaPHixfc839LSopaWlvDXoVBImZmZamxsVFJSUixHAxCH+HdAnV8oFFIgEHjgz/GYvzugf//+evLJJ1VdXd3m836/X36/P9ZjAAA6mZj/O6CrV6/q3LlzSk9Pj/WhAABxJOoBWrNmjcrLy/XPf/5Tf/3rXzV79mx1795dzz77bLQPBQCIY1H/K7iLFy/q2Wef1ZUrVzRw4EA9/fTTOnbsGH8vCgCIEPUA7d69O9rfsks4e/as5zX//e9/Pa+ZOHGi5zVAPDl+/LjnNXl5eTGYBI+Kz4IDAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEzE/BfS4bbS0lLPaz799FPPa/gwUsST9vxC5vZ8sO8//vEPz2sQe9wBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwASfht1BNm/e7HlNQUFBDCYBOo+rV696XlNcXOx5zYsvvuh5jSQNHDiwXevwcLgDAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBM8GGkHeTWrVvWIwCdzrJlyzrkOKNGjeqQ48Ab7oAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABN8GGk71NbWel7zr3/9KwaTAPHtP//5T4cc5zvf+U6HHAfecAcEADBBgAAAJjwH6OjRo5oxY4YyMjLk8/m0f//+iOedc1q3bp3S09PVu3dv5efn6+zZs9GaFwDQRXgOUHNzs7Kzs1VSUtLm85s2bdLmzZu1bds2HT9+XH379tW0adN0/fr1Rx4WANB1eH4TQmFhoQoLC9t8zjmnN998U6+88opmzpwpSXr77beVlpam/fv3a/78+Y82LQCgy4jqa0A1NTWqq6tTfn5++LFAIKCcnBxVVFS0uaalpUWhUChiAwB0fVENUF1dnSQpLS0t4vG0tLTwc3crLi5WIBAIb5mZmdEcCQDQSZm/C27t2rVqbGwMbxcuXLAeCQDQAaIaoGAwKEmqr6+PeLy+vj783N38fr+SkpIiNgBA1xfVAGVlZSkYDKq0tDT8WCgU0vHjx5WbmxvNQwEA4pznd8FdvXpV1dXV4a9ramp06tQpJScna/DgwVq5cqV+8Ytf6IknnlBWVpZeffVVZWRkaNasWdGcGwAQ5zwH6MSJE3rmmWfCX69evVqStHDhQu3YsUMvvfSSmpubtXTpUjU0NOjpp5/WoUOH1KtXr+hNDQCIe54DNHXqVDnn7vu8z+fTxo0btXHjxkcarDP785//7HnNtWvXYjAJ0Hk0Nzd7XvP3v/89BpPca8CAAR1yHHhj/i44AMDjiQABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACY8fxo2pDNnznTIccaNG9chxwGi4Wc/+5nnNbW1tZ7XjB071vOahIQEz2sQe9wBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAm+DDSTiwnJ8d6BHQiLS0tntdUVla261i//e1vPa/Zs2dPu47l1ebNmz2v6dWrVwwmwaPiDggAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMMGHkXZiDQ0N1iNEXW1trec1ra2tnteUl5d7XiNJNTU1ntfcuHHD85otW7Z4XnPr1i3Pa/r27et5jSQVFBR4XtOeD/y8efOm5zWjRo3yvAadE3dAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJPoy0Hfr06eN5jc/n87zm+9//vuc1I0aM8LymI1VUVHhe45zzvKZHj/Zd2v369fO8Jicnx/OaNWvWeF4zefJkz2vGjRvneY3Uvg8xzczM9LymubnZ85qBAwd6XoPOiTsgAIAJAgQAMOE5QEePHtWMGTOUkZEhn8+n/fv3Rzy/aNEi+Xy+iG369OnRmhcA0EV4DlBzc7Oys7NVUlJy332mT5+uS5cuhbddu3Y90pAAgK7H8yu1hYWFKiws/NJ9/H6/gsFgu4cCAHR9MXkNqKysTKmpqRoxYoSWL1+uK1eu3HfflpYWhUKhiA0A0PVFPUDTp0/X22+/rdLSUv3qV79SeXm5CgsL7/v77IuLixUIBMJbe97KCQCIP1H/d0Dz588P/3nMmDEaO3ashg0bprKyMuXl5d2z/9q1a7V69erw16FQiAgBwGMg5m/DHjp0qFJSUlRdXd3m836/X0lJSREbAKDri3mALl68qCtXrig9PT3WhwIAxBHPfwV39erViLuZmpoanTp1SsnJyUpOTtaGDRs0d+5cBYNBnTt3Ti+99JKGDx+uadOmRXVwAEB88xygEydO6Jlnngl/fef1m4ULF2rr1q06ffq03nrrLTU0NCgjI0MFBQX6+c9/Lr/fH72pAQBxz+fa80mPMRQKhRQIBNTY2NilXg966623PK8pKyuL/iBxaMGCBZ7XDB8+vF3HysrKate6ruaPf/yj5zXf+973PK8ZOXKk5zUff/yx5zXoWA/7c5zPggMAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAICJqP9KbrRt4cKFHbIGiIaDBw92yHF++MMfdshx0DlxBwQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmODDSAGYmTNnjvUIMMQdEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADARA/rAQB0Dc45z2s+++wzz2uGDh3qeQ06J+6AAAAmCBAAwISnABUXF2vChAlKTExUamqqZs2apaqqqoh9rl+/rqKiIg0YMED9+vXT3LlzVV9fH9WhAQDxz1OAysvLVVRUpGPHjunw4cO6efOmCgoK1NzcHN5n1apVeu+997R3716Vl5ertrZWc+bMifrgAID45ulNCIcOHYr4eseOHUpNTVVlZaWmTJmixsZG/e53v9POnTv17W9/W5K0fft2jRo1SseOHdO3vvWt6E0OAIhrj/QaUGNjoyQpOTlZklRZWambN28qPz8/vM/IkSM1ePBgVVRUtPk9WlpaFAqFIjYAQNfX7gC1trZq5cqVmjRpkkaPHi1JqqurU0JCgvr37x+xb1pamurq6tr8PsXFxQoEAuEtMzOzvSMBAOJIuwNUVFSkM2fOaPfu3Y80wNq1a9XY2BjeLly48EjfDwAQH9r1D1FXrFihgwcP6ujRoxo0aFD48WAwqBs3bqihoSHiLqi+vl7BYLDN7+X3++X3+9szBgAgjnm6A3LOacWKFdq3b5+OHDmirKysiOfHjx+vnj17qrS0NPxYVVWVzp8/r9zc3OhMDADoEjzdARUVFWnnzp06cOCAEhMTw6/rBAIB9e7dW4FAQIsXL9bq1auVnJyspKQkvfDCC8rNzeUdcACACJ4CtHXrVknS1KlTIx7fvn27Fi1aJEn69a9/rW7dumnu3LlqaWnRtGnT9Jvf/CYqwwIAug5PAXqYDxvs1auXSkpKVFJS0u6hAMQfn8/neU1ra2sMJkG84LPgAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYKJdvxEVAKLhyJEjntfk5eXFYBJY4A4IAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDBh5ECiArnnPUIiDPcAQEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJvgwUgD3mDt3ruc127Zti8Ek6Mq4AwIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATPBhpADukZeX53lNa2trDCZBV8YdEADABAECAJjwFKDi4mJNmDBBiYmJSk1N1axZs1RVVRWxz9SpU+Xz+SK2ZcuWRXVoAED88xSg8vJyFRUV6dixYzp8+LBu3rypgoICNTc3R+y3ZMkSXbp0Kbxt2rQpqkMDAOKfpzchHDp0KOLrHTt2KDU1VZWVlZoyZUr48T59+igYDEZnQgBAl/RIrwE1NjZKkpKTkyMef+edd5SSkqLRo0dr7dq1unbt2n2/R0tLi0KhUMQGAOj62v027NbWVq1cuVKTJk3S6NGjw48vWLBAQ4YMUUZGhk6fPq2XX35ZVVVVevfdd9v8PsXFxdqwYUN7xwAAxCmfc861Z+Hy5cv1pz/9SR9++KEGDRp03/2OHDmivLw8VVdXa9iwYfc839LSopaWlvDXoVBImZmZamxsVFJSUntGAwAYCoVCCgQCD/w53q47oBUrVujgwYM6evTol8ZHknJyciTpvgHy+/3y+/3tGQMAEMc8Bcg5pxdeeEH79u1TWVmZsrKyHrjm1KlTkqT09PR2DQgA6Jo8BaioqEg7d+7UgQMHlJiYqLq6OklSIBBQ7969de7cOe3cuVPf/e53NWDAAJ0+fVqrVq3SlClTNHbs2Jj8BwAA4pOn14B8Pl+bj2/fvl2LFi3ShQsX9IMf/EBnzpxRc3OzMjMzNXv2bL3yyisP/XrOw/7dIQCgc4rJa0APalVmZqbKy8u9fEsAwGOKz4IDAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJjoYT3A3ZxzkqRQKGQ8CQCgPe78/L7z8/x+Ol2AmpqaJEmZmZnGkwAAHkVTU5MCgcB9n/e5ByWqg7W2tqq2tlaJiYny+XwRz4VCIWVmZurChQtKSkoymtAe5+E2zsNtnIfbOA+3dYbz4JxTU1OTMjIy1K3b/V/p6XR3QN26ddOgQYO+dJ+kpKTH+gK7g/NwG+fhNs7DbZyH26zPw5fd+dzBmxAAACYIEADARFwFyO/3a/369fL7/dajmOI83MZ5uI3zcBvn4bZ4Og+d7k0IAIDHQ1zdAQEAug4CBAAwQYAAACYIEADARNwEqKSkRF/72tfUq1cv5eTk6G9/+5v1SB3utddek8/ni9hGjhxpPVbMHT16VDNmzFBGRoZ8Pp/2798f8bxzTuvWrVN6erp69+6t/Px8nT171mbYGHrQeVi0aNE918f06dNtho2R4uJiTZgwQYmJiUpNTdWsWbNUVVUVsc/169dVVFSkAQMGqF+/fpo7d67q6+uNJo6NhzkPU6dOved6WLZsmdHEbYuLAO3Zs0erV6/W+vXr9dFHHyk7O1vTpk3T5cuXrUfrcE899ZQuXboU3j788EPrkWKuublZ2dnZKikpafP5TZs2afPmzdq2bZuOHz+uvn37atq0abp+/XoHTxpbDzoPkjR9+vSI62PXrl0dOGHslZeXq6ioSMeOHdPhw4d18+ZNFRQUqLm5ObzPqlWr9N5772nv3r0qLy9XbW2t5syZYzh19D3MeZCkJUuWRFwPmzZtMpr4PlwcmDhxoisqKgp/fevWLZeRkeGKi4sNp+p469evd9nZ2dZjmJLk9u3bF/66tbXVBYNB9/rrr4cfa2hocH6/3+3atctgwo5x93lwzrmFCxe6mTNnmsxj5fLly06SKy8vd87d/t++Z8+ebu/eveF9PvnkEyfJVVRUWI0Zc3efB+ec+7//+z/34osv2g31EDr9HdCNGzdUWVmp/Pz88GPdunVTfn6+KioqDCezcfbsWWVkZGjo0KF67rnndP78eeuRTNXU1Kiuri7i+ggEAsrJyXksr4+ysjKlpqZqxIgRWr58ua5cuWI9Ukw1NjZKkpKTkyVJlZWVunnzZsT1MHLkSA0ePLhLXw93n4c73nnnHaWkpGj06NFau3atrl27ZjHefXW6DyO92+eff65bt24pLS0t4vG0tDR9+umnRlPZyMnJ0Y4dOzRixAhdunRJGzZs0OTJk3XmzBklJiZaj2eirq5Oktq8Pu4897iYPn265syZo6ysLJ07d04//elPVVhYqIqKCnXv3t16vKhrbW3VypUrNWnSJI0ePVrS7eshISFB/fv3j9i3K18PbZ0HSVqwYIGGDBmijIwMnT59Wi+//LKqqqr07rvvGk4bqdMHCP9TWFgY/vPYsWOVk5OjIUOG6A9/+IMWL15sOBk6g/nz54f/PGbMGI0dO1bDhg1TWVmZ8vLyDCeLjaKiIp05c+axeB30y9zvPCxdujT85zFjxig9PV15eXk6d+6chg0b1tFjtqnT/xVcSkqKunfvfs+7WOrr6xUMBo2m6hz69++vJ598UtXV1dajmLlzDXB93Gvo0KFKSUnpktfHihUrdPDgQX3wwQcRv74lGAzqxo0bamhoiNi/q14P9zsPbcnJyZGkTnU9dPoAJSQkaPz48SotLQ0/1traqtLSUuXm5hpOZu/q1as6d+6c0tPTrUcxk5WVpWAwGHF9hEIhHT9+/LG/Pi5evKgrV650qevDOacVK1Zo3759OnLkiLKysiKeHz9+vHr27BlxPVRVVen8+fNd6np40Hloy6lTpySpc10P1u+CeBi7d+92fr/f7dixw3388cdu6dKlrn///q6urs56tA714x//2JWVlbmamhr3l7/8xeXn57uUlBR3+fJl69FiqqmpyZ08edKdPHnSSXJvvPGGO3nypPvss8+cc8798pe/dP3793cHDhxwp0+fdjNnznRZWVnuiy++MJ48ur7sPDQ1Nbk1a9a4iooKV1NT495//333jW98wz3xxBPu+vXr1qNHzfLly10gEHBlZWXu0qVL4e3atWvhfZYtW+YGDx7sjhw54k6cOOFyc3Ndbm6u4dTR96DzUF1d7TZu3OhOnDjhampq3IEDB9zQoUPdlClTjCePFBcBcs65LVu2uMGDB7uEhAQ3ceJEd+zYMeuROty8efNcenq6S0hIcF/96lfdvHnzXHV1tfVYMffBBx84SfdsCxcudM7dfiv2q6++6tLS0pzf73d5eXmuqqrKdugY+LLzcO3aNVdQUOAGDhzoevbs6YYMGeKWLFnS5f5PWlv//ZLc9u3bw/t88cUX7kc/+pH7yle+4vr06eNmz57tLl26ZDd0DDzoPJw/f95NmTLFJScnO7/f74YPH+5+8pOfuMbGRtvB78KvYwAAmOj0rwEBALomAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMDE/wMXNIyPwE63lgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>.nowrap{white-space:nowrap;}</style><span class='nowrap'>logic signed [7:0] test_data [0:783] = '{ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 33, 116, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31, 40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 60, 90, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 76, 105, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 110, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 127, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 111, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 91, 127, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 23, 122, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 99, 127, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 60, 127, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 115, 127, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 79, 127, 60, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 81, 127, 108, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 79, 127, 33, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 43, 89, 124, 127, 45, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 79, 127, 42, 0, 0, 0, 23, 24, 58, 72, 75, 120, 121, 117, 89, 120, 126, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 75, 126, 118, 103, 103, 103, 126, 127, 125, 120, 99, 71, 45, 14, 2, 116, 125, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 59, 88, 88, 88, 88, 88, 49, 28, 0, 0, 0, 0, 0, 51, 127, 110, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 84, 127, 68, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 84, 127, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 84, 127, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 84, 127, 47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 84, 127, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 84, 127, 76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 84, 127, 76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 48, 127, 76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };</span>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "test_data, test_data_int = test_data_to_verilog(8,x_train[2],show_img=1)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  },
  "vscode": {
   "interpreter": {
    "hash": "12fa2b87cac659b457be56b1f9fd9f19d0e5d67b8aa1ffabf71ca4ccf3ec44db"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}