00_tensorflow_fundamentals.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7e5f6426",
   "metadata": {},
   "source": [
    "## Training on the most fundamental concepts of tensors using TensorFlow.\n",
    "\n",
    "More specifically, I will work on :    \n",
    "\n",
    "    Introduction to tensors\n",
    "    Getting informations from tensors\n",
    "    Manipulating tensors\n",
    "    Tensors & Numpy\n",
    "    Using @tf.function (a way to speed up the regular Python functions)\n",
    "    Using GPUs with TensorFlow (or CPUs)\n",
    "    \n",
    "    \n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9cbbb7ac",
   "metadata": {},
   "source": [
    "### Introduction to Tensors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "56199bfd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.10.0\n"
     ]
    }
   ],
   "source": [
    "# Import TensorFlow\n",
    "import tensorflow as tf\n",
    "\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d0c95aa2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow_probability as tfp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "dfc26ad1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=int32, numpy=7>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating tensors with tf.constant()\n",
    "\n",
    "scalar = tf.constant(7)\n",
    "scalar"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7601f9b4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# checking the number of dimensions of a tensor\n",
    "scalar.ndim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "98e0ca79",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "3d78fb3f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([10, 15])>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a vector\n",
    "vector = tf.constant([10,15])\n",
    "vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7b816928",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking the dimension of the vector\n",
    "vector.ndim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ca7ecb8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# -------------------------- "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "30cbe09c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
       "array([[ 5,  3],\n",
       "       [12, 15],\n",
       "       [10,  5]])>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a matrix (has more than one dimension)\n",
    "matrix = tf.constant([[5,3],[12,15],[10,5]])\n",
    "matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "091aad82",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking the dimension of the matrix\n",
    "matrix.ndim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "45009ea1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ----------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "165682a9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 2), dtype=float16, numpy=\n",
       "array([[ 1.,  5.],\n",
       "       [15., 18.],\n",
       "       [ 9., 15.]], dtype=float16)>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a matrix while specifing its data's type\n",
    "matrix2 = tf.constant([[1,5.],[15.,18.],[9.,15]], dtype=tf.float16)\n",
    "matrix2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "76938167",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking the dimensions of the matrix\n",
    "matrix2.ndim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "fda3594a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "854c2d35",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(4, 2, 3), dtype=float32, numpy=\n",
       "array([[[ 1.,  5., 16.],\n",
       "        [12., 20.,  9.]],\n",
       "\n",
       "       [[13., 19., 12.],\n",
       "        [24., 31., 15.]],\n",
       "\n",
       "       [[ 9., 11.,  8.],\n",
       "        [19.,  8.,  5.]],\n",
       "\n",
       "       [[ 7., 16.,  3.],\n",
       "        [98., 12., 42.]]], dtype=float32)>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a tensor\n",
    "tensor = tf.constant([\n",
    "    [ [1,5,16],[12,20,9] ],\n",
    "    [ [13,19,12], [24,31,15]],\n",
    "    [ [9,11,8], [19,8,5] ],\n",
    "    [ [7,16,3], [98,12,42] ]\n",
    "],dtype=tf.float32)\n",
    "tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "8c93f3e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor.ndim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "4306dce8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# --------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44671e70",
   "metadata": {},
   "source": [
    "What we have created so far :  \n",
    "    \n",
    "    scalar : a single number\n",
    "    vector : a number with direction (e.g. wind speed and direction)\n",
    "    matrix : a 2-dimensional array of numbers\n",
    "    tensor : a n-dimensional array of numbers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b74016ef",
   "metadata": {},
   "source": [
    "### Creating Tensor with `tf.Variable`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1fe96fdf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensorflow.python.ops.variables.Variable"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "bd9f5f7e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.framework.constant_op.constant(value, dtype=None, shape=None, name='Const')>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.constant"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "e7505195",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([10,  7])>,\n",
       " <tf.Tensor: shape=(2,), dtype=int32, numpy=array([10,  7])>)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "changeable_tensor = tf.Variable([10,7])\n",
    "unchangeable_tensor = tf.constant([10,7])\n",
    "changeable_tensor, unchangeable_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "cbcfe2ea",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<tf.Tensor: shape=(), dtype=int32, numpy=10>,\n",
       " <tf.Tensor: shape=(), dtype=int32, numpy=7>)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Getting the elements in the changeable_tensor\n",
    "changeable_tensor[0], changeable_tensor[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "a1a3bf29",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([23,  7])>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Getting one of the elements in the changeable_tensor\n",
    "changeable_tensor[0].assign(23)\n",
    "changeable_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8fb72781",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "d8dd68a0",
   "metadata": {},
   "source": [
    "### Creating random tensors    \n",
    "\n",
    "Random tensors are tensors of some arbitrary size containing random numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "f27e6b2d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
       "array([[-0.7565803 , -0.06854702],\n",
       "       [ 0.07595026, -1.2573844 ],\n",
       "       [-0.23193763, -1.8107855 ]], dtype=float32)>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#creating two random (but the same) tensors\n",
    "random_1 = tf.random.Generator.from_seed(42) #set seed for reproducibility\n",
    "random_1 = random_1.normal(shape=(3,2))\n",
    "random_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c0fa6a4",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "0ccf7a47",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
       "array([[-0.7565803 , -0.06854702],\n",
       "       [ 0.07595026, -1.2573844 ],\n",
       "       [-0.23193763, -1.8107855 ]], dtype=float32)>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random_2 = tf.random.Generator.from_seed(42)\n",
    "random_2 = random_2.normal(shape=(3,2))\n",
    "random_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "cf64edbf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
       " array([[-0.7565803 , -0.06854702],\n",
       "        [ 0.07595026, -1.2573844 ],\n",
       "        [-0.23193763, -1.8107855 ]], dtype=float32)>,\n",
       " <tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
       " array([[-0.7565803 , -0.06854702],\n",
       "        [ 0.07595026, -1.2573844 ],\n",
       "        [-0.23193763, -1.8107855 ]], dtype=float32)>,\n",
       " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
       " array([[ True,  True],\n",
       "        [ True,  True],\n",
       "        [ True,  True]])>)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# are random_1 and random_2 equal ?\n",
    "\n",
    "random_1, random_2, random_1==random_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eef97500",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "64fbe843",
   "metadata": {},
   "source": [
    "### Shuffling the order of elements in a tensor\n",
    "\n",
    "    It is valuable for when you want to shuffle the data so the inherent order doesn't affect learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "44331308",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "not_shufled = tf.constant([ [10,8], [12,5], [2,5] ])\n",
    "not_shufled.ndim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "e07521fb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
       "array([[10,  8],\n",
       "       [12,  5],\n",
       "       [ 2,  5]])>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "not_shufled"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "6916b787",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
       "array([[ 2,  5],\n",
       "       [12,  5],\n",
       "       [10,  8]])>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# shuffle the non-shuffled tensor\n",
    "tf.random.shuffle(not_shufled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "05e6dec7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
       "array([[10,  8],\n",
       "       [12,  5],\n",
       "       [ 2,  5]])>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# shuffle the non-shuffled tensor\n",
    "tf.random.set_seed(42)  #global level random seed\n",
    "tf.random.shuffle(not_shufled,seed=42) #operation level random seed"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a993e47e",
   "metadata": {},
   "source": [
    "https://www.tensorflow.org/api_docs/python/tf/random/set_seed  \n",
    "\n",
    "Il looks like if we want our shuffled tensors to be in the same order (everytime they are shuffled), we've got to use the global level random as well as the operation level random see.\n",
    "\n",
    "> Rule 4 : \"If both the global and the operation seed are set : both seeds are used in conjunction to determmine the random sequence\"."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62ae6eaf",
   "metadata": {},
   "source": [
    "### Others ways to make tensors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "9cadacf0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(4, 3), dtype=float16, numpy=\n",
       "array([[1., 1., 1.],\n",
       "       [1., 1., 1.],\n",
       "       [1., 1., 1.],\n",
       "       [1., 1., 1.]], dtype=float16)>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.ones([4,3],dtype=tf.float16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "41b00d3f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(5, 2), dtype=float32, numpy=\n",
       "array([[0., 0.],\n",
       "       [0., 0.],\n",
       "       [0., 0.],\n",
       "       [0., 0.],\n",
       "       [0., 0.]], dtype=float32)>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.zeros([5,2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "3ac91f4e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(5, 2), dtype=float32, numpy=\n",
       "array([[0., 0.],\n",
       "       [0., 0.],\n",
       "       [0., 0.],\n",
       "       [0., 0.],\n",
       "       [0., 0.]], dtype=float32)>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.zeros(shape=(5,2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1989133c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "12489cfa",
   "metadata": {},
   "source": [
    "### Turning Numpy arrays into tensors   \n",
    "\n",
    "The main difference bteween Numpy arrays and TensorFlow tensors is that tensors can be run on a GPU (much faster for numerical computing)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "63eff967",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
       "       18, 19, 20, 21, 22, 23, 24], dtype=int16)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numpy_A = np.arange(1,25,dtype=np.int16)\n",
    "numpy_A\n",
    "# X = tf.constant(some_matrix)  #capital for matrix or tensor\n",
    "# y = tf.constant(vector) #non-capital for vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "5326fd60",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(24,), dtype=int16, numpy=\n",
       "array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
       "       18, 19, 20, 21, 22, 23, 24], dtype=int16)>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = tf.constant(numpy_A)\n",
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "d7b0f21a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "24"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "2*3*4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "ca16073f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3, 4), dtype=int16, numpy=\n",
       "array([[[ 1,  2,  3,  4],\n",
       "        [ 5,  6,  7,  8],\n",
       "        [ 9, 10, 11, 12]],\n",
       "\n",
       "       [[13, 14, 15, 16],\n",
       "        [17, 18, 19, 20],\n",
       "        [21, 22, 23, 24]]], dtype=int16)>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A2 = tf.constant(numpy_A, shape=(2,3,4))\n",
    "A2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1eb79458",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "cd761c6a",
   "metadata": {},
   "source": [
    "### Getting informations from tensors\n",
    "\n",
    "* Shape : The length of each of the dimensions of a tensor\n",
    "* Rank : The number of tensor dimensions. A scalar has rank 0, a vector has rank 1, a matrix is rand 2, a tensor has rank n\n",
    "* Axis or dimension : A particular dimension of a tensor.\n",
    "* Size :  The total number of items in the tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "20be0654",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3, 4, 5), dtype=float32, numpy=\n",
       "array([[[[0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.]],\n",
       "\n",
       "        [[0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.]],\n",
       "\n",
       "        [[0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.]]],\n",
       "\n",
       "\n",
       "       [[[0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.]],\n",
       "\n",
       "        [[0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.]],\n",
       "\n",
       "        [[0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.],\n",
       "         [0., 0., 0., 0., 0.]]]], dtype=float32)>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a rank 4 tensor (4 dimensions)\n",
    "rank_4_tensor = tf.zeros([2,3,4,5])\n",
    "rank_4_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "c507420f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 4, 5), dtype=float32, numpy=\n",
       "array([[[0., 0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0., 0.]],\n",
       "\n",
       "       [[0., 0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0., 0.]],\n",
       "\n",
       "       [[0., 0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0., 0.],\n",
       "        [0., 0., 0., 0., 0.]]], dtype=float32)>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rank_4_tensor[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "7df8575b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(TensorShape([2, 3, 4, 5]), 4, <tf.Tensor: shape=(), dtype=int32, numpy=120>)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rank_4_tensor.shape, rank_4_tensor.ndim, tf.size(rank_4_tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "cc1c8326",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Datatype of every element :  <dtype: 'float32'>\n",
      "Number of dimensions (rank) :  4\n",
      "Shape of tensor :  (2, 3, 4, 5)\n",
      "Elements along the 0 axis :  2\n",
      "Elements along the last axis :  5\n",
      "Total number of elements in our tensor :  tf.Tensor(120, shape=(), dtype=int32)\n",
      "Total number of elements in our tensor (2) :  120\n"
     ]
    }
   ],
   "source": [
    "# Geting various attributes of our tensor\n",
    "print(\"Datatype of every element : \", rank_4_tensor.dtype)\n",
    "print(\"Number of dimensions (rank) : \", rank_4_tensor.ndim)\n",
    "print(\"Shape of tensor : \", rank_4_tensor.shape)\n",
    "print(\"Elements along the 0 axis : \", rank_4_tensor.shape[0])\n",
    "print(\"Elements along the last axis : \", rank_4_tensor.shape[-1])\n",
    "print(\"Total number of elements in our tensor : \", tf.size(rank_4_tensor))\n",
    "print(\"Total number of elements in our tensor (2) : \", tf.size(rank_4_tensor).numpy())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6d8a97a",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "b1cbd1c4",
   "metadata": {},
   "source": [
    "### Indexing tensors\n",
    "\n",
    "Tensors can be indexed just like Python lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "eb76d7ca",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3, 4, 5), dtype=float32, numpy=\n",
       "array([[[[1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.]],\n",
       "\n",
       "        [[1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.]],\n",
       "\n",
       "        [[1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.]]],\n",
       "\n",
       "\n",
       "       [[[1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.]],\n",
       "\n",
       "        [[1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.]],\n",
       "\n",
       "        [[1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.],\n",
       "         [1., 1., 1., 1., 1.]]]], dtype=float32)>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a rank 4 tensor (4 dimensions)\n",
    "rank_4_tensor2 = tf.ones([2,3,4,5])\n",
    "rank_4_tensor2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "6728f12c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 2, 2, 2), dtype=float32, numpy=\n",
       "array([[[[1., 1.],\n",
       "         [1., 1.]],\n",
       "\n",
       "        [[1., 1.],\n",
       "         [1., 1.]]],\n",
       "\n",
       "\n",
       "       [[[1., 1.],\n",
       "         [1., 1.]],\n",
       "\n",
       "        [[1., 1.],\n",
       "         [1., 1.]]]], dtype=float32)>"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Getting the first 2 elements of each dimension\n",
    "rank_4_tensor2[:2,:2,:2,:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "0305adcb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(1, 1, 1, 5), dtype=float32, numpy=array([[[[1., 1., 1., 1., 1.]]]], dtype=float32)>"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Get the first element of each dimension from each index except the final one\n",
    "rank_4_tensor2[:1,:1,:1,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "a724e404",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(1, 1, 4, 1), dtype=float32, numpy=\n",
       "array([[[[1.],\n",
       "         [1.],\n",
       "         [1.],\n",
       "         [1.]]]], dtype=float32)>"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Get the first element of each dimension from each index except for the second last one\n",
    "rank_4_tensor2[:1,:1,:,:1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca614615",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "301459f3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=float32, numpy=\n",
       "array([[ 5.,  2.,  3.],\n",
       "       [19.,  5.,  8.]], dtype=float32)>"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a rank 2 tensor (2 dimensions)\n",
    "rank_2_tensor = tf.constant([[5,2,3], [19,5,8.]])\n",
    "rank_2_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "e2950ff5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(TensorShape([2, 3]), 2)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rank_2_tensor.shape, rank_2_tensor.ndim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "21eef55d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2,), dtype=float32, numpy=array([3., 8.], dtype=float32)>"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Getting the last item of each dimension of our rank 2 tensor\n",
    "rank_2_tensor[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "59a06136",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3, 1), dtype=float32, numpy=\n",
       "array([[[ 5.],\n",
       "        [ 2.],\n",
       "        [ 3.]],\n",
       "\n",
       "       [[19.],\n",
       "        [ 5.],\n",
       "        [ 8.]]], dtype=float32)>"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Adding in extra dimension to our rank 2 tensor (we will not change the informations, only changing the shape)\n",
    "rank_3_tensor = rank_2_tensor[...,tf.newaxis] # the same as rank_2_tensor[:,:,tf.newaxis]\n",
    "rank_3_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "7c0c0b0c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3, 1), dtype=float32, numpy=\n",
       "array([[[ 5.],\n",
       "        [ 2.],\n",
       "        [ 3.]],\n",
       "\n",
       "       [[19.],\n",
       "        [ 5.],\n",
       "        [ 8.]]], dtype=float32)>"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Alternative to tf.newaxis\n",
    "rank_3_tensor2 = tf.expand_dims(rank_2_tensor, axis=-1) # \"-1\" means expand the final axis\n",
    "rank_3_tensor2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "b43091e4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(1, 2, 3), dtype=float32, numpy=\n",
       "array([[[ 5.,  2.,  3.],\n",
       "        [19.,  5.,  8.]]], dtype=float32)>"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.expand_dims(rank_2_tensor, axis=0) # expand the 0 axis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d10527a",
   "metadata": {},
   "source": [
    "tf.expand_dims(rank_2_tensor, axis=1) # expand the 1 axis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc17b7c3",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "b884bc0d",
   "metadata": {},
   "source": [
    "### Manipulating tensors (tensor operations)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8b77bde",
   "metadata": {},
   "source": [
    "**Basic operations** : \n",
    "`+`,`-`,`*`,`/`,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "29cf2305",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[10,  7,  4],\n",
       "       [12,  1,  9]])>"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor = tf.constant([ [10,7,4],[12,1,9] ])\n",
    "tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "dfa3e1d0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[20, 17, 14],\n",
       "       [22, 11, 19]])>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Addition\n",
    "tensor+10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "69e95564",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[10,  7,  4],\n",
       "       [12,  1,  9]])>"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Notice the original tensor is unchanged\n",
    "tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "db06f44d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[ 8,  5,  2],\n",
       "       [10, -1,  7]])>"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor-2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "5a331f2d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[20, 14,  8],\n",
       "       [24,  2, 18]])>"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor*2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "0d14c556",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=float64, numpy=\n",
       "array([[5. , 3.5, 2. ],\n",
       "       [6. , 0.5, 4.5]])>"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor/2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "035324d2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[5, 3, 2],\n",
       "       [6, 0, 4]])>"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor//2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e802e0a7",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "8f783b6c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[10,  7,  4],\n",
       "       [12,  1,  9]])>"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "c51d1ce5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[20, 14,  8],\n",
       "       [24,  2, 18]])>"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can use the TensorFlow built-in function\n",
    "tf.multiply(tensor,2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "708870ae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[20, 17, 14],\n",
       "       [22, 11, 19]])>"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.add(tensor,10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "5e9b50ae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[30, 27, 24],\n",
       "       [32, 21, 29]])>"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.math.add(tensor,20)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f053766",
   "metadata": {},
   "source": [
    "**Matrix Multiplication**\n",
    "\n",
    "There are two rules our tensors (or matrices) need to fulful if we are going to matrxi multiply them:\n",
    "* The inner dimensions of the two matrices must match\n",
    "* The resulting matrix has the shape of the inner dimensions\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "4e259cf1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
       "array([[10,  7],\n",
       "       [ 3,  4]])>"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor = tf.constant([ [10,7],[3,4] ])\n",
    "tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "d27c6d2d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
       "array([[121,  98],\n",
       "       [ 42,  37]])>"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.matmul(tensor,tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "edccc800",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
       "array([[100,  49],\n",
       "       [  9,  16]])>"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Wrong way to do matrix multiplication with Python operator\n",
    "tensor*tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "7dc73b41",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
       "array([[121,  98],\n",
       "       [ 42,  37]])>"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Matrix multiplication with Python operator \"@\"\n",
    "tensor @ tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "597d9b54",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "f1c88b3e",
   "metadata": {},
   "source": [
    "**Matrix multiplication with matrices having different shapes**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "a5c55c7a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
       " array([[1, 2],\n",
       "        [3, 4],\n",
       "        [5, 6]])>,\n",
       " <tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
       " array([[ 7,  8],\n",
       "        [ 9, 10],\n",
       "        [11, 12]])>)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Creating a tensor of shape (3,2)\n",
    "X = tf.constant([ [1,2],[3,4],[5,6] ])\n",
    "Y = tf.constant([ [7,8],[9,10],[11,12] ])\n",
    "X,Y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "534c1cc1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(TensorShape([3, 2]), TensorShape([3, 2]))"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape, Y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "760141ae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[ 7,  8,  9],\n",
       "       [10, 11, 12]])>"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Reshaping Y\n",
    "tf.reshape(Y,shape=(2,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "66c1d15b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(TensorShape([3, 2]), TensorShape([2, 3]))"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape, tf.reshape(Y,shape=(2,3)).shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "682ad433",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
       "array([[ 27,  30,  33],\n",
       "       [ 61,  68,  75],\n",
       "       [ 95, 106, 117]])>"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Matrix multiplying X by reshaped Y\n",
    "X @ tf.reshape(Y,shape=(2,3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "0228ca0e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
       "array([[ 27,  30,  33],\n",
       "       [ 61,  68,  75],\n",
       "       [ 95, 106, 117]])>"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.matmul(X, tf.reshape(Y,shape=(2,3)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "b9cc732c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
       "array([[ 58,  64],\n",
       "       [139, 154]])>"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Matrix multiplying reshaped X by Y\n",
    "tf.matmul(tf.reshape(X,shape=(2,3)),Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "96a24bef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
       "array([[1, 2],\n",
       "       [3, 4],\n",
       "       [5, 6]])>"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "0b720929",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
       "array([[1, 3, 5],\n",
       "       [2, 4, 6]])>"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.transpose(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b82bc77",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "c32788f4",
   "metadata": {},
   "source": [
    "**The dot product**\n",
    "\n",
    "Matrix multiplication is also referred to as the dot product.\n",
    "One can perform matrix multiplication using:\n",
    "* `tf.matmul()`\n",
    "* `tf.tensordot()`\n",
    "* `@`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "e33516bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
       " array([[1, 2],\n",
       "        [3, 4],\n",
       "        [5, 6]])>,\n",
       " <tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
       " array([[ 7,  8],\n",
       "        [ 9, 10],\n",
       "        [11, 12]])>)"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X,Y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "377d422a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
       "array([[ 89,  98],\n",
       "       [116, 128]])>"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Performing dot product on X and Y (requires X or Y to be transposed)\n",
    "tf.tensordot(tf.transpose(X),Y, axes=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "49425caa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
       "array([[ 58,  64],\n",
       "       [139, 154]])>"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.tensordot( tf.reshape(X,shape=(2,3)), Y, axes=1 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "adfe109c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Normal Y: \n",
      "tf.Tensor(\n",
      "[[ 7  8]\n",
      " [ 9 10]\n",
      " [11 12]], shape=(3, 2), dtype=int32) \n",
      "\n",
      "Y reshaped to (2,3): \n",
      "tf.Tensor(\n",
      "[[ 7  8  9]\n",
      " [10 11 12]], shape=(2, 3), dtype=int32) \n",
      "\n",
      "Y transposed: \n",
      "tf.Tensor(\n",
      "[[ 7  9 11]\n",
      " [ 8 10 12]], shape=(2, 3), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "# Checking the values of Y, reshape Y, and transposed Y\n",
    "print(\"Normal Y: \")\n",
    "print(Y, \"\\n\")\n",
    "\n",
    "print(\"Y reshaped to (2,3): \")\n",
    "print(tf.reshape(Y, (2,3)), \"\\n\")\n",
    "\n",
    "print(\"Y transposed: \")\n",
    "print(tf.transpose(Y))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4c83b22",
   "metadata": {},
   "source": [
    "*Generally, when performing matrix multiplication on two tensors and one of the axes doesn't line up, you will transpose (rather than reshape) one of the tensors to satisfy matrix multiplication rules.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9ba6659",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "1066fdba",
   "metadata": {},
   "source": [
    "### Changing the datatype of a tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "791e8e74",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.10.0'"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "16f2da8b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
       "array([[1., 1.],\n",
       "       [1., 1.],\n",
       "       [1., 1.]], dtype=float32)>"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a new tensor with default datatype (float32)\n",
    "tensor = tf.ones(shape=(3,2))\n",
    "tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "0b4c1d65",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tf.float32"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B = tf.constant([1.7, 1.2])\n",
    "B.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "41d82a29",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tf.int32"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "C = tf.constant([1, 5])\n",
    "C.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "e3b0d486",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tf.float16"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Change from float32 to float16 (reduce precision)\n",
    "D = tf.cast(B, dtype=tf.float16)\n",
    "D.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "006bba73",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tf.float32"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Change from int32 to float32\n",
    "E = tf.cast(C,dtype=tf.float32)\n",
    "E.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e8a338a8",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "41ff22ec",
   "metadata": {},
   "source": [
    "### Aggregating tensors\n",
    "\n",
    "Aggreating tensors = Condensing them from multiple values down to a smaller amount of values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "2d1c3aba",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ -7, -10])>"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "D = tf.constant([-7, -10])\n",
    "D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "01e9fe65",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ 7, 10])>"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the absolute values\n",
    "tf.abs(D)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8040b5a",
   "metadata": {},
   "source": [
    "Let's go through the following forms of aggregation :\n",
    "* Get the minimum of a tensor\n",
    "* Get the maximum of a tensor\n",
    "* Get the mean of a tensor\n",
    "* Get the sum of a tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "075f25a2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(50,), dtype=int32, numpy=\n",
       "array([35, 24,  4, 31, 10,  8, 34, 23, 91, 28, 59, 28, 59, 12, 66, 60, 32,\n",
       "       53, 36,  8, 87, 54, 59, 60, 23, 55, 49, 54, 55, 44, 81, 53, 44, 41,\n",
       "       11, 79, 32, 44, 86, 89,  3, 14, 19, 52, 38,  3, 89, 41, 34,  0])>"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a random tensor with values between 0 and 100 of size 50\n",
    "E = tf.constant(np.random.randint(0,100, size=50))\n",
    "E"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "3655aaf5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<tf.Tensor: shape=(), dtype=int32, numpy=50>, TensorShape([50]), 1)"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.size(E), E.shape, E.ndim, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "233cbc8c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=int32, numpy=0>"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the minimum\n",
    "tf.reduce_min(E)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "475d9bba",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=int32, numpy=91>"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the maximum\n",
    "tf.reduce_max(E)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "baebaab2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=int32, numpy=41>"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the mean\n",
    "tf.reduce_mean(E)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "45899750",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=int32, numpy=2094>"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the sum\n",
    "tf.reduce_sum(E)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "f39c435e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=630.78564>"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the variance\n",
    "tf.math.reduce_variance(tf.cast(E,dtype=tf.float32))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "55d63bd0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=int32, numpy=631>"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the variance \n",
    "tfp.stats.variance(E)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "2980562f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=25.115446>"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the standard deviation\n",
    "tf.math.reduce_std( tf.cast(E, dtype=tf.float32) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "2876ca9a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=25.115446>"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the standard deviation\n",
    "tfp.stats.stddev( tf.cast(E, dtype=tf.float32) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5b896ba",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "37848208",
   "metadata": {},
   "source": [
    "### Finding the positional maximum and minimum"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "6d77fe55",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
       "array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
       "       0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
       "       0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
       "       0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
       "       0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
       "       0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
       "       0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
       "       0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
       "       0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
       "       0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
       "      dtype=float32)>"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating a new tensor for finding potitional minimum and maximum\n",
    "tf.random.set_seed(42)\n",
    "F = tf.random.uniform(shape=[50])\n",
    "F"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "0a2e1a5d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=int64, numpy=42>"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the positional maximum (Index of the largest value position)\n",
    "tf.argmax(F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "1763c0b3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.argmax(F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "92436f3d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Value of the positional maximum\n",
    "F[ tf.argmax(F) ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "id": "3cd5e27a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the max value of F\n",
    "tf.reduce_max(F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "id": "95c69a72",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=bool, numpy=True>"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check for equality\n",
    "F[ tf.argmax(F) ] == tf.reduce_max(F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "2bc61f86",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=int64, numpy=16>"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the positional minimum (Index of the lower value position)\n",
    "tf.argmin(F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "id": "92c4b9a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the minimum using the positional minimum index\n",
    "F[ tf.argmin(F) ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "id": "8a0d2ae2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.reduce_min(F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0873e5c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "aa2c2a8f",
   "metadata": {},
   "source": [
    "### Squeezing a tensor (removing all single dimensions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "id": "adc06cf6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(1, 1, 1, 1, 50), dtype=float32, numpy=\n",
       "array([[[[[0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
       "           0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
       "           0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
       "           0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
       "           0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
       "           0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
       "           0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
       "           0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
       "           0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
       "           0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043]]]]],\n",
       "      dtype=float32)>"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.random.set_seed(42)\n",
    "G = tf.constant(tf.random.uniform(shape=[50]), shape=(1,1,1,1,50))\n",
    "G"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "id": "ac6b4274",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorShape([1, 1, 1, 1, 50])"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "G.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "id": "d5d57666",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
       "array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
       "       0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
       "       0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
       "       0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
       "       0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
       "       0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
       "       0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
       "       0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
       "       0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
       "       0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
       "      dtype=float32)>"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "G_squeezed = tf.squeeze(G)\n",
    "G_squeezed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b20e13f2",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "adbb48c3",
   "metadata": {},
   "source": [
    "### One-hot encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45c03c6f",
   "metadata": {},
   "source": [
    "**One-hot encode a list**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "id": "a7b93be0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(4, 4), dtype=float32, numpy=\n",
       "array([[1., 0., 0., 0.],\n",
       "       [0., 1., 0., 0.],\n",
       "       [0., 0., 1., 0.],\n",
       "       [0., 0., 0., 1.]], dtype=float32)>"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "some_list = [0,1,2,3]\n",
    "\n",
    "tf.one_hot( some_list, depth= 4 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "id": "8787f449",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(4, 5), dtype=float32, numpy=\n",
       "array([[1., 0., 0., 0., 0.],\n",
       "       [0., 1., 0., 0., 0.],\n",
       "       [0., 0., 1., 0., 0.],\n",
       "       [0., 0., 0., 1., 0.]], dtype=float32)>"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "some_list2 = [0,1,2,3]\n",
    "\n",
    "tf.one_hot( some_list2, depth= 5 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "id": "ff0dbe0b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(4, 7), dtype=float32, numpy=\n",
       "array([[0., 1., 0., 0., 0., 0., 0.],\n",
       "       [0., 0., 1., 0., 0., 0., 0.],\n",
       "       [1., 0., 0., 0., 0., 0., 0.],\n",
       "       [0., 0., 0., 0., 0., 0., 1.]], dtype=float32)>"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "some_list3 = [1,2,0,6]\n",
    "\n",
    "tf.one_hot( some_list3, depth= 7 )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0608439",
   "metadata": {},
   "source": [
    "**Specify custom value for one-hot encoding**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "id": "543bdef9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(4, 4), dtype=string, numpy=\n",
       "array([[b'yo I love deep learning', b'I also like to dance',\n",
       "        b'I also like to dance', b'I also like to dance'],\n",
       "       [b'I also like to dance', b'yo I love deep learning',\n",
       "        b'I also like to dance', b'I also like to dance'],\n",
       "       [b'I also like to dance', b'I also like to dance',\n",
       "        b'yo I love deep learning', b'I also like to dance'],\n",
       "       [b'I also like to dance', b'I also like to dance',\n",
       "        b'I also like to dance', b'yo I love deep learning']],\n",
       "      dtype=object)>"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Specify custom value for one-hot encoding\n",
    "tf.one_hot(some_list, depth=4, on_value=\"yo I love deep learning\", off_value=\"I also like to dance\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c489f50",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "b29dbf6d",
   "metadata": {},
   "source": [
    "### Squaring, log, square root"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "id": "8a6f5b3f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([1, 2, 3, 4, 5, 6, 7, 8, 9])>"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a new tensor\n",
    "H = tf.range(1,10)\n",
    "H"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "id": "00f8987e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([ 1,  4,  9, 16, 25, 36, 49, 64, 81])>"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Square a tensor\n",
    "tf.square(H)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "id": "4d29a09a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
       "array([0.       , 0.6931472, 1.0986123, 1.3862944, 1.609438 , 1.7917595,\n",
       "       1.9459102, 2.0794415, 2.1972246], dtype=float32)>"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the log of a tensor\n",
    "tf.math.log( tf.cast(H, dtype=tf.float32) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "id": "fafb2c0f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
       "array([1.       , 1.4142135, 1.7320508, 2.       , 2.236068 , 2.4494898,\n",
       "       2.6457512, 2.828427 , 3.       ], dtype=float32)>"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the square root of a tensor\n",
    "tf.sqrt( tf.cast(H, dtype=tf.float32) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9904d94c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "ab0e41b3",
   "metadata": {},
   "source": [
    "### TensorFlow and NumPy's compatibility\n",
    "\n",
    "TensorFlow interacts beautiffuly with NumPy arrays.   \n",
    "One of the main differences between a TensorFlow tensor and a NumPy array, though, is that a TensorFlow tensor can be run on a GPU or TPU (for faster numercial processing).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "id": "b7701b9b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(10,), dtype=int32, numpy=array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])>"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a tensor directly from a NumPy array\n",
    "H = tf.constant( np.arange(0,10) )\n",
    "H"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "id": "a5ded0d0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([ 3.,  7., 10.])>"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a tensor directly from a NumPy array\n",
    "J = tf.constant( np.array( [3.,7,10] ) )\n",
    "J"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "id": "04fb1fa4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3.,  7., 10.])"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Convert a tensor to NumPy array\n",
    "np.array(J)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "id": "50af4aca",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(np.array(J))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "id": "1de6e888",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3.,  7., 10.])"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Convert tensor to NumPy array\n",
    "J.numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "id": "cf2a4880",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(J.numpy())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "id": "ba2f2a8e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([ 3.,  7., 10.])>"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "J"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "id": "11fa6b00",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.0"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "J.numpy()[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "id": "cabd1b42",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10.0"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "J.numpy()[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "id": "7c29410a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tf.float64, tf.float32)"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The default types of each are slightly different\n",
    "J1 = tf.constant( np.array([3.,7,10.]) )\n",
    "J2 = tf.constant( [3.,7,10.] )\n",
    "# Check the datatype of each\n",
    "J1.dtype, J2.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9cdfa317",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "6e7c5cf9",
   "metadata": {},
   "source": [
    "### Findind access to GPUs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "id": "a503a833",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.framework.test_util.is_gpu_available(cuda_only=False, min_cuda_compute_capability=None)>"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.test.is_gpu_available"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "id": "1ca5d2ae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.config.list_physical_devices()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "id": "66e51df1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.config.list_physical_devices(\"CPU\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "id": "04f71500",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.config.list_physical_devices(\"GPU\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "id": "a199e257",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'nvidia-smi' n'est pas reconnu en tant que commande interne\n",
      "ou externe, un programme ex‚cutable ou un fichier de commandes.\n"
     ]
    }
   ],
   "source": [
    "# Check what type of GPU is being used\n",
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32445778",
   "metadata": {},
   "source": [
    "> **Note:** If there is access to a CUDA-enabled GPU, TensorFlow will automatically use it whenever possible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bed8529f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c1c67580",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}