10_Introduction_to_GNNs_with_PyTorch_Geometric.ipynb

{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "title: 11 Introduction to GNNs\n",
    "description: Introduction to Graph Neural Networks. Applies basic GCN to Cora dataset for node classification.\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/drive/1d0jLDwgNBtjBVQOFe8lO_1WrqTVeVZx9?usp=sharing\" target=\"_blank\"><img align=\"left\" alt=\"Colab\" title=\"Open in Colab\" src=\"https://colab.research.google.com/assets/colab-badge.svg\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sjX6CPJFA52R"
   },
   "source": [
    "## Introduction to GNNs with PyTorch Geometric\n",
    "\n",
    "In this short notebook, the goal is to provide a introductory guide to get started with Graph Neural Networks using the popular library called [PyTorch Geometric](https://pytorch-geometric.readthedocs.io/en/latest/index.html). PyTorch Geometric is a PyTorch based libary hence we will be using PyTorch in this tutorial. \n",
    "\n",
    "The code used in this tutorial has been adapted from their official [examples](https://pytorch-geometric.readthedocs.io/en/latest/notes/introduction.html). I have incorporated a bit more beginner-friendly guidance and kept it minimal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mwTz9zaHC7YA",
    "outputId": "ce24d6a4-907f-4094-eb98-bc6ea0520e34"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11.1\n"
     ]
    }
   ],
   "source": [
    "# Find the CUDA version PyTorch was installed with\n",
    "!python -c \"import torch; print(torch.version.cuda)\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2jo0YpV0DLDW",
    "outputId": "238637c0-e60b-42fc-e7de-86f47f39ec4f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.10.0+cu111\n"
     ]
    }
   ],
   "source": [
    "# PyTorch version\n",
    "!python -c \"import torch; print(torch.__version__)\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "P-VLTfxzEmLu"
   },
   "source": [
    "Install the follow packages but make sure to install the right version below. Find more instructions [here](https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html) if you get lost. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fLbSOIkaDRe4",
    "outputId": "b196161e-d1bf-4595-ccaa-49747f3ec00c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in links: https://data.pyg.org/whl/torch-1.10.0+cu111.html\n",
      "Collecting torch-scatter\n",
      "  Downloading https://data.pyg.org/whl/torch-1.10.0%2Bcu113/torch_scatter-2.0.9-cp37-cp37m-linux_x86_64.whl (7.9 MB)\n",
      "\u001b[K     |████████████████████████████████| 7.9 MB 2.5 MB/s \n",
      "\u001b[?25hInstalling collected packages: torch-scatter\n",
      "Successfully installed torch-scatter-2.0.9\n"
     ]
    }
   ],
   "source": [
    "!pip install torch-scatter -f https://data.pyg.org/whl/torch-1.10.0+cu111.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Q-wRLXE_DkZF",
    "outputId": "cb249940-3c85-4572-eb41-3d3ef2a5407d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in links: https://data.pyg.org/whl/torch-1.10.0+cu111.html\n",
      "Collecting torch-sparse\n",
      "  Downloading https://data.pyg.org/whl/torch-1.10.0%2Bcu113/torch_sparse-0.6.12-cp37-cp37m-linux_x86_64.whl (3.5 MB)\n",
      "\u001b[K     |████████████████████████████████| 3.5 MB 2.8 MB/s \n",
      "\u001b[?25hRequirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from torch-sparse) (1.4.1)\n",
      "Requirement already satisfied: numpy>=1.13.3 in /usr/local/lib/python3.7/dist-packages (from scipy->torch-sparse) (1.19.5)\n",
      "Installing collected packages: torch-sparse\n",
      "Successfully installed torch-sparse-0.6.12\n"
     ]
    }
   ],
   "source": [
    "!pip install torch-sparse -f https://data.pyg.org/whl/torch-1.10.0+cu111.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lAobCDu6Dppo",
    "outputId": "d7675ad2-6b5f-4162-caa8-13fa658d1793"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting torch-geometric\n",
      "  Downloading torch_geometric-2.0.3.tar.gz (370 kB)\n",
      "\u001b[K     |████████████████████████████████| 370 kB 5.3 MB/s \n",
      "\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (1.19.5)\n",
      "Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (4.62.3)\n",
      "Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (1.4.1)\n",
      "Requirement already satisfied: networkx in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (2.6.3)\n",
      "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (1.0.2)\n",
      "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (2.23.0)\n",
      "Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (1.3.5)\n",
      "Collecting rdflib\n",
      "  Downloading rdflib-6.1.1-py3-none-any.whl (482 kB)\n",
      "\u001b[K     |████████████████████████████████| 482 kB 48.3 MB/s \n",
      "\u001b[?25hRequirement already satisfied: googledrivedownloader in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (0.4)\n",
      "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (2.11.3)\n",
      "Requirement already satisfied: pyparsing in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (3.0.7)\n",
      "Collecting yacs\n",
      "  Downloading yacs-0.1.8-py3-none-any.whl (14 kB)\n",
      "Requirement already satisfied: PyYAML in /usr/local/lib/python3.7/dist-packages (from torch-geometric) (3.13)\n",
      "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->torch-geometric) (2.0.1)\n",
      "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->torch-geometric) (2018.9)\n",
      "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas->torch-geometric) (2.8.2)\n",
      "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas->torch-geometric) (1.15.0)\n",
      "Collecting isodate\n",
      "  Downloading isodate-0.6.1-py2.py3-none-any.whl (41 kB)\n",
      "\u001b[K     |████████████████████████████████| 41 kB 486 kB/s \n",
      "\u001b[?25hRequirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from rdflib->torch-geometric) (4.10.1)\n",
      "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from rdflib->torch-geometric) (57.4.0)\n",
      "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->rdflib->torch-geometric) (3.7.0)\n",
      "Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->rdflib->torch-geometric) (3.10.0.2)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->torch-geometric) (2021.10.8)\n",
      "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->torch-geometric) (2.10)\n",
      "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->torch-geometric) (3.0.4)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->torch-geometric) (1.24.3)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->torch-geometric) (3.1.0)\n",
      "Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->torch-geometric) (1.1.0)\n",
      "Building wheels for collected packages: torch-geometric\n",
      "  Building wheel for torch-geometric (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for torch-geometric: filename=torch_geometric-2.0.3-py3-none-any.whl size=581968 sha256=eee568026f004ea2d960222f33768328b638194617a05429f1e10e5c019857d0\n",
      "  Stored in directory: /root/.cache/pip/wheels/c3/2a/58/87ce0508964d4def1aafb92750c4f3ac77038efd1b9a89dcf5\n",
      "Successfully built torch-geometric\n",
      "Installing collected packages: isodate, yacs, rdflib, torch-geometric\n",
      "Successfully installed isodate-0.6.1 rdflib-6.1.1 torch-geometric-2.0.3 yacs-0.1.8\n"
     ]
    }
   ],
   "source": [
    "!pip install torch-geometric"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GNfiSdoUFaoF"
   },
   "source": [
    "## Getting Started"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "D6LCvEr7CGF9"
   },
   "source": [
    "Import PyTorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "pqrlyFN1AtXI",
    "outputId": "850011fd-9eb1-4afc-8ba2-1416f4801c05"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.10.0+cu111\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "# print torch version\n",
    "print(torch.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V0xfcmMfCFIH"
   },
   "source": [
    "The great thing about PyTorch Geometric is that it contain useful functionalities to import and load graph related data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gfIc5j5YB2_a"
   },
   "outputs": [],
   "source": [
    "from torch_geometric.data import Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4wuBs1NHEFqn"
   },
   "source": [
    "Now let's create an unweighted and undirected graph with three nodes and four total edges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "-nLnUIObCTjK",
    "outputId": "e4ac0c52-5b42-40be-8f35-091f04fd7a9c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data(x=[3, 1], edge_index=[2, 4])\n"
     ]
    }
   ],
   "source": [
    "# define edge list\n",
    "edge_index = torch.tensor([[0, 1, 1, 2], [1, 0, 2, 1]], dtype=torch.long)\n",
    "\n",
    "# define node features\n",
    "x = torch.tensor([[-1], [0], [1]])\n",
    "\n",
    "# create graph data object\n",
    "data = Data(x=x, edge_index=edge_index)\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zV7bQ6tAEQ7H"
   },
   "source": [
    "Our data object `Data` has many useful utility functions to check the properties of the graph. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "idbfqPzoEOPC",
    "outputId": "f22248ff-27e6-4cec-ec15-a0c2bbea463c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n"
     ]
    }
   ],
   "source": [
    "# check number of edges of the graph\n",
    "print(data.num_edges)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "r0lcSstME0MP",
    "outputId": "c88ad32a-22aa-4507-8824-ac9961cad74e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n"
     ]
    }
   ],
   "source": [
    "# check number of nodes of the graph\n",
    "print(data.num_nodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "yiegwHTyE2AO",
    "outputId": "182f22cc-0a52-48b8-d1b6-19c0d926a35f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n"
     ]
    }
   ],
   "source": [
    "# check number of features of the graph\n",
    "print(data.num_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "JNWjhaSeE3Yk",
    "outputId": "5a08aebe-a590-4a93-f58f-a0d45d70a2b2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n"
     ]
    }
   ],
   "source": [
    "# check if graph is directed\n",
    "print(data.is_directed())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HF-jGPyhFeO6"
   },
   "source": [
    "## Loading Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KBVX6mZfFBxE"
   },
   "source": [
    "Find more fun functions related to graph data [here](https://pytorch-geometric.readthedocs.io/en/latest/modules/data.html#torch_geometric.data.Data). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LHvE23mwFJ-3"
   },
   "source": [
    "One of the cool things about the PyTorch Geometric library is that it contains out-of-the-box benchmark datasets that are ready to use and explore. A popular dataset is the Cora dataset that is used for supervised graph node classification. (We will talk about these applications in an upcoming tutorial but for now we will focus on the data itself).\n",
    "\n",
    "\"The Cora dataset consists of 2708 scientific publications classified into one of seven classes. The citation network consists of 5429 links. Each publication in the dataset is described by a 0/1-valued word vector indicating the absence/presence of the corresponding word from the dictionary. The dictionary consists of 1433 unique words.\" - [Papers with Code](https://paperswithcode.com/dataset/cora)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_8ganBm_FiaQ"
   },
   "source": [
    "Let's load the Cora dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "K3bXwOpoE75M",
    "outputId": "ede1ca54-0336-4642-c82f-c1b9c7ed5f3b"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.x\n",
      "Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.tx\n",
      "Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.allx\n",
      "Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.y\n",
      "Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.ty\n",
      "Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.ally\n",
      "Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.graph\n",
      "Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.test.index\n",
      "Processing...\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "from torch_geometric.datasets import Planetoid\n",
    "\n",
    "dataset = Planetoid(root='tmp/Cora', name='Cora')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rQTFnBcuFpLv"
   },
   "source": [
    "Let's check some of the properties of the Cora dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "q9jvXbRXFlGG",
    "outputId": "5d22903d-65dd-4e67-b5d4-ccfe682c1157"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of graphs:  1\n",
      "Number of features:  1433\n",
      "Number of classes:  7\n"
     ]
    }
   ],
   "source": [
    "# number of graphs\n",
    "print(\"Number of graphs: \", len(dataset))\n",
    "\n",
    "# number of features\n",
    "print(\"Number of features: \", dataset.num_features)\n",
    "\n",
    "# number of classes\n",
    "print(\"Number of classes: \", dataset.num_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "57SX-idAF02R"
   },
   "source": [
    "We can see that this particular graph dataset only contains one graph. Graph data can be very complex and can include multiple graphs depending on the type of data and application. Let's check more feature of the Cora dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8aOt8HwfFrH_",
    "outputId": "b7043e70-aafe-4b7d-c877-83feed65c12b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of nodes:  2708\n",
      "Number of edges:  10556\n",
      "Is directed:  False\n"
     ]
    }
   ],
   "source": [
    "# select the first graph\n",
    "data = dataset[0]\n",
    "\n",
    "# number of nodes\n",
    "print(\"Number of nodes: \", data.num_nodes)\n",
    "\n",
    "# number of edges\n",
    "print(\"Number of edges: \", data.num_edges)\n",
    "\n",
    "# check if directed\n",
    "print(\"Is directed: \", data.is_directed())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3XX2MRY4GEQS"
   },
   "source": [
    "You can sample nodes from the graph this way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "qGJKbv-4GAtY",
    "outputId": "8c2a45dc-0b4d-4027-b252-5e48ef787cd6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of sample nodes:  torch.Size([5, 1433])\n"
     ]
    }
   ],
   "source": [
    "# sample nodes from the graph\n",
    "print(\"Shape of sample nodes: \", data.x[:5].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wV1yBQSvGM9q"
   },
   "source": [
    "We extracted 5 nodes from the graph and checked its shape. You will see that each node has `1433` features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oXMY2lU0GWQL"
   },
   "source": [
    "Another great advantage of using PyTorch Geometric to load the Cora data is that it comes pre-processed and ready to use. It also has the splits for training, validation and test which we can directly use for training a GNN.\n",
    "\n",
    "Let's check some stats for the partitions of the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MzITbLkpGIUP",
    "outputId": "7a95cc3c-67dc-4050-e22a-556928cefae8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# of nodes to train on:  140\n",
      "# of nodes to test on:  1000\n",
      "# of nodes to validate on:  500\n"
     ]
    }
   ],
   "source": [
    "# check training nodes\n",
    "print(\"# of nodes to train on: \", data.train_mask.sum().item())\n",
    "\n",
    "# check test nodes\n",
    "print(\"# of nodes to test on: \", data.test_mask.sum().item())\n",
    "\n",
    "# check validation nodes\n",
    "print(\"# of nodes to validate on: \", data.val_mask.sum().item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7IGOoSfwHjeD"
   },
   "source": [
    "That information is important as it will indicate to our model which nodes to train against and which to test against, and so on.\n",
    "\n",
    "When training neural networks we train them using batches of data. PyTorch Geometric provides efficient processes to load batches of data.\n",
    "\n",
    "PyTorch Geometric contains a data loader which is a very popular feature in PyTorch to efficiently load data when training neural networks. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_SUZUnXzH1zN"
   },
   "source": [
    "So let's try to load the data using the built in `DataLoader`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9tdHl4oZGw_y"
   },
   "outputs": [],
   "source": [
    "from torch_geometric.datasets import Planetoid\n",
    "from torch_geometric.loader import DataLoader\n",
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xtJAp4QqIMWw",
    "outputId": "8df69cee-89ab-441b-9526-6029b41b6bb8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cpu\n"
     ]
    }
   ],
   "source": [
    "print(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "06xbeTJcH8C-"
   },
   "outputs": [],
   "source": [
    "dataset = Planetoid(root='tmp/Cora', name='Cora')\n",
    "data = dataset[0].to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8MH0lQsYIV0V"
   },
   "source": [
    "Print some quick statistics about the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gK7-K6uYH_Iu",
    "outputId": "0613a450-caf0-4137-c00f-a45ed8db412f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X shape:  torch.Size([2708, 1433])\n",
      "Edge shape:  torch.Size([2, 10556])\n",
      "Y shape:  torch.Size([2708])\n"
     ]
    }
   ],
   "source": [
    "print(\"X shape: \", data.x.shape)\n",
    "print(\"Edge shape: \", data.edge_index.shape)\n",
    "print(\"Y shape: \", data.y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TUGNxhBOIuYe"
   },
   "source": [
    "## Model and Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v1pz24blIwnR"
   },
   "source": [
    "Finally, let's define a standard GCN to train on the Cora dataset. The aim is to train a model that gets better at predicting the class of the node."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dhdMehneI2bn"
   },
   "source": [
    "To keep thins simple we will use the same model definition as used in the [tutorial](https://pytorch-geometric.readthedocs.io/en/latest/notes/introduction.html) we adpated the code from. Note that we are using the built-in `GCNConv` model but you could easily implement your own (something we will cover in a future tutorial). \n",
    "\n",
    "The model below uses two `GCNConv` layers. The first layer is followed by a non-linearity `ReLU` and `Dropout`. The result is fed to the second layer on top of which we apply `Softmax` to get distribution over the number of classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "q0AiwyWrJGhj"
   },
   "outputs": [],
   "source": [
    "import torch.nn.functional as F\n",
    "from torch_geometric.nn import GCNConv\n",
    "\n",
    "class GCN(torch.nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        \"\"\" GCNConv layers \"\"\"\n",
    "        self.conv1 = GCNConv(data.num_features, 16)\n",
    "        self.conv2 = GCNConv(16, dataset.num_classes)\n",
    "\n",
    "    def forward(self, data):\n",
    "        x, edge_index = data.x, data.edge_index\n",
    "        x = self.conv1(x, edge_index)\n",
    "        x = F.relu(x)\n",
    "        x = F.dropout(x, training=self.training)\n",
    "        x = self.conv2(x, edge_index)\n",
    "\n",
    "        return F.log_softmax(x, dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Mfob8LS2KezY"
   },
   "source": [
    "Initial model and optimizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GSZL4HS5Kd55"
   },
   "outputs": [],
   "source": [
    "model = GCN().to(device)\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.01, weight_decay=5e-4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qpgxKMhmKitV"
   },
   "source": [
    "Define axcuracy function for evaluating performance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "abnE-XTmKl92"
   },
   "outputs": [],
   "source": [
    "# useful function for computing accuracy\n",
    "def compute_accuracy(pred_y, y):\n",
    "    return (pred_y == y).sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "26W7sxVsKrGM"
   },
   "source": [
    "And finally we train the model on the trainin nodes for 200 epochs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "m39ZbE6RKyim",
    "outputId": "2e7a3a02-e654-4c11-9c50-d40eab4e6c45"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 10, Loss: 0.8235, Training Acc: 0.9357\n",
      "Epoch: 20, Loss: 0.2665, Training Acc: 0.9786\n",
      "Epoch: 30, Loss: 0.1056, Training Acc: 0.9857\n",
      "Epoch: 40, Loss: 0.0583, Training Acc: 1.0000\n",
      "Epoch: 50, Loss: 0.0461, Training Acc: 1.0000\n",
      "Epoch: 60, Loss: 0.0388, Training Acc: 0.9929\n",
      "Epoch: 70, Loss: 0.0406, Training Acc: 1.0000\n",
      "Epoch: 80, Loss: 0.0447, Training Acc: 1.0000\n",
      "Epoch: 90, Loss: 0.0571, Training Acc: 0.9929\n",
      "Epoch: 100, Loss: 0.0304, Training Acc: 1.0000\n",
      "Epoch: 110, Loss: 0.0373, Training Acc: 1.0000\n",
      "Epoch: 120, Loss: 0.0268, Training Acc: 1.0000\n",
      "Epoch: 130, Loss: 0.0504, Training Acc: 0.9857\n",
      "Epoch: 140, Loss: 0.0245, Training Acc: 1.0000\n",
      "Epoch: 150, Loss: 0.0294, Training Acc: 1.0000\n",
      "Epoch: 160, Loss: 0.0378, Training Acc: 0.9929\n",
      "Epoch: 170, Loss: 0.0441, Training Acc: 1.0000\n",
      "Epoch: 180, Loss: 0.0223, Training Acc: 1.0000\n",
      "Epoch: 190, Loss: 0.0370, Training Acc: 0.9929\n",
      "Epoch: 200, Loss: 0.0224, Training Acc: 1.0000\n"
     ]
    }
   ],
   "source": [
    "# train the model\n",
    "model.train()\n",
    "losses = []\n",
    "accuracies = []\n",
    "for epoch in range(200):\n",
    "    optimizer.zero_grad()\n",
    "    out = model(data)\n",
    "\n",
    "    loss = F.nll_loss(out[data.train_mask], data.y[data.train_mask])\n",
    "    correct = compute_accuracy(out.argmax(dim=1)[data.train_mask], data.y[data.train_mask])\n",
    "    acc = int(correct) / int(data.train_mask.sum())\n",
    "    losses.append(loss.item())\n",
    "    accuracies.append(acc)\n",
    "\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    if (epoch+1) % 10 == 0:\n",
    "        print('Epoch: {}, Loss: {:.4f}, Training Acc: {:.4f}'.format(epoch+1, loss.item(), acc))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 265
    },
    "id": "bUtwTwemLJGs",
    "outputId": "d351364f-f04a-4a75-b3b3-80bd517fe569"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xUZb748c83kzLpHRIIIaGHEoqhrKjIVRF0FV11xcV63Yuua7nrNvzdu67r9uJdV9druXZd0V1srKKuKIqolCBNCCXUJCQkJKTXyTy/P84kDJCQSTJpM9/365VXZk79njNnvueZ5zznPGKMQSmllO8K6OsAlFJK9SxN9Eop5eM00SullI/TRK+UUj5OE71SSvm4wL4OoC0JCQkmLS2tr8NQSqkBY9OmTceMMYltjeuXiT4tLY3s7Oy+DkMppQYMETnU3jitulFKKR/XYaIXkWEislpEdorIDhG5p41pREQeEZFcEdkmItPcxt0kIntdfzd5ewOUUkqdmSdVNw7gh8aYr0QkEtgkIh8aY3a6TbMAGO36mwk8DswUkTjg50AWYFzzrjDGHPfqViillGpXh4neGFMIFLpeV4lIDjAUcE/0C4EXjfU8hXUiEiMiycD5wIfGmDIAEfkQmA8s8+pWKKUGjKamJvLz86mvr+/rUAYku91OSkoKQUFBHs/TqYuxIpIGTAXWnzJqKJDn9j7fNay94W0tewmwBCA1NbUzYSmlBpD8/HwiIyNJS0tDRPo6nAHFGENpaSn5+fmkp6d7PJ/HF2NFJAJ4HfhPY0xlF2I8I2PMU8aYLGNMVmJimy2ElFI+oL6+nvj4eE3yXSAixMfHd/rXkEeJXkSCsJL834wxb7QxSQEwzO19imtYe8OVUn5Mk3zXdWXfedLqRoBngBxjzP+0M9kK4EZX65tZQIWrbv8DYJ6IxIpILDDPNczrHM1OHludy6d7Snpi8UopNWB5Ukc/G7gB2C4iW1zD/h+QCmCMeQJYCVwC5AK1wC2ucWUi8ktgo2u+B1suzHqbLUB4as1+vpmZzJwxWvWjlGpfREQE1dXVfR1Gr/Gk1c1a4Iy/FVytbb7fzrhngWe7FF0niAjpCeEcOFbT06tSSqkBxafujB2hiV4p1UVbtmxh1qxZZGZmcuWVV3L8uHW7zyOPPML48ePJzMxk0aJFAHz66adMmTKFKVOmMHXqVKqqqvoy9A71y2fddFV6QjhvbC6gttFBWLBPbZpSPukX/9zBziPebcQ3fkgUP79sQqfnu/HGG3n00UeZM2cO999/P7/4xS94+OGH+d3vfseBAwcICQmhvLwcgD/96U889thjzJ49m+rqaux2u1e3wdt8qkSfnhgOwMFjtX0ciVJqIKmoqKC8vJw5c+YAcNNNN7FmzRoAMjMzWbx4MS+//DKBgVYBcvbs2dx777088sgjlJeXtw7vr/p3dJ2UnmAl+gPHahg/JKqPo1FKdaQrJe/e9u6777JmzRr++c9/8utf/5rt27ezdOlSLr30UlauXMns2bP54IMPGDduXF+H2i6fKtGnxbckev+5mq6U6r7o6GhiY2P57LPPAHjppZeYM2cOTqeTvLw85s6dy+9//3sqKiqorq5m3759TJo0iZ/+9KdMnz6dXbt29fEWnJlPlejDQwJJirKzXy/IKqXOoLa2lpSUlNb39957Ly+88AK33347tbW1jBgxgueee47m5mauv/56KioqMMZw9913ExMTw89+9jNWr15NQEAAEyZMYMGCBX24NR3zqUQPaBNLpVSHnE5nm8PXrVt32rC1a9eeNuzRRx/1ekw9yaeqbsC6IKuJXimlTvC9RB8fTnltE+W1jX0dilJK9Qs+l+iHxoYCUFBe18eRKKVU/+BziX5IjJXoC8u1UwOllAJfTPTR1h1qRyq0RK+UUuCDiT4hIoQgm3BES/RKKQX4YKIPCBCSou0UaoleKXUGb731FiLS72928gafS/QAydGhWkevlDqjZcuWcc4557Bs2bIeW0dzc3OPLbszfDLRD4m2a6sbpVS7qqurWbt2Lc888wyvvvoqYCXlH/3oR0ycOJHMzMzWm6I2btzI2WefzeTJk5kxYwZVVVU8//zz3Hnnna3L++Y3v8knn3wCWJ2a/PCHP2Ty5Ml8+eWXPPjgg0yfPp2JEyeyZMkSrO47IDc3lwsvvJDJkyczbdo09u3bx4033shbb73VutzFixfz9ttvd3t7O7wzVkSeBb4JFBtjJrYx/sfAYrflZQCJrt6lDgJVQDPgMMZkdTtiDwyJCeVoZSHNToMtQPumVKrfem8pFG337jKTJsGC351xkrfffpv58+czZswY4uPj2bRpExs2bODgwYNs2bKFwMBAysrKaGxs5Nprr+W1115j+vTpVFZWEhoaesZl19TUMHPmTB566CEAxo8fz/333w/ADTfcwDvvvMNll13G4sWLWbp0KVdeeSX19fU4nU5uvfVW/vznP3PFFVdQUVHBF198wQsvvNDtXeJJif55YH57I40xfzTGTDHGTAHuAz49pbvAua7xvZLkAZJjQnE4DceqG3prlUqpAWTZsmWtnYgsWrSIZcuWsWrVKm677bbWRw7HxcWxe/dukpOTmT59OgBRUVEdPpLYZrNx1VVXtb5fvXo1M2fOZNKkSXz88cfs2LGDqqoqCgoKuPLKKwGw2+2EhYUxZ84c9u7dS0lJCcuWLeOqq67yyiOQPelKcI2IpHm4vOuAnqvw8lBLE8uC8joGR/XvDgGU8msdlLx7QllZGR9//DHbt29HRGhubkZEWpO5JwIDA096Xk59/Ylrgna7HZvN1jr8jjvuIDs7m2HDhvHAAw+cNG1bbrzxRl5++WVeffVVnnvuuU5uXdu8VkcvImFYJf/X3QYb4F8isklElnQw/xIRyRaR7JKSkm7FkhytN00ppdq2fPlybrjhBg4dOsTBgwfJy8sjPT2dyZMn8+STT+JwOADrhDB27FgKCwvZuHEjAFVVVTgcDtLS0tiyZUvrY4w3bNjQ5rpaknpCQgLV1dUsX74cgMjISFJSUlrr4xsaGqittTpMuvnmm3n44YcBq9rHG7x5MfYy4PNTqm3OMcZMAxYA3xeR89qb2RjzlDEmyxiTlZiY2K1AhrbcHatNLJVSp1i2bFlrlUmLq666isLCQlJTU8nMzGTy5Mm88sorBAcH89prr3HXXXcxefJkLrroIurr65k9ezbp6emMHz+eu+++m2nTprW5rpiYGP7jP/6DiRMncvHFF5/0q+Gll17ikUceITMzk7PPPpuioiIABg8eTEZGBrfccovXtllargCfcSKr6uadti7Guk3zJvAPY8wr7Yx/AKg2xvypo/VlZWWZ7OzsDuNqjzGGCT//gGunDxsQPdgo5U9ycnLIyMjo6zD6rdraWiZNmsRXX31FdHR0m9O0tQ9FZFN710K9UqIXkWhgDvC227BwEYlseQ3MA772xvo8iIfkaDtFFVp1o5QaOFatWkVGRgZ33XVXu0m+KzxpXrkMOB9IEJF84OdAEIAx5gnXZFcC/zLGuD8IfjDwpoi0rOcVY8z7Xou8A0nRdooqNdErpQaOCy+8kEOHDnl9uZ60urnOg2mex2qG6T5sPzC5q4F1V1JUKF/sO9ZXq1dKnYExBlchUHWSJ9Xtp/LJO2MBkqPtFFc10Ozs/E5RSvUcu91OaWlplxKWvzPGUFpait3euWbjPtdnbIukaDvNrpumtC29Uv1HSkoK+fn5dLcZtb+y2+0ndWzuCd9N9K7kXlhRr4leqX4kKCiI9PT0vg7Dr/hs1U2S6+7YIm1Lr5Tycz6b6JOjT5TolVLKn/lsoo8LDybYFqBNLJVSfs9nE72IMDg6RG+aUkr5PZ9N9ADJUaFadaOU8ns+neiTou0c1aobpZSf8/lEX1hRrzdmKKX8mm8n+ig7jQ4nx2ub+joUpZTqMz6d6JNb29Jr9Y1Syn/5dKJvvWmqUm+aUkr5L79I9NryRinlz3w60SdGhBAgWnWjlPJvPp3oA20BDIrUnqaUUv7NpxM9aE9TSinVYaIXkWdFpFhE2uzvVUTOF5EKEdni+rvfbdx8EdktIrkistSbgXsqKcqudfRKKb/mSYn+eWB+B9N8ZoyZ4vp7EEBEbMBjwAJgPHCdiIzvTrBdkRRt56gmeqWUH+sw0Rtj1gBlXVj2DCDXGLPfGNMIvAos7MJyuiU52k5Vg4Oqer1pSinln7xVR/8NEdkqIu+JyATXsKFAnts0+a5hbRKRJSKSLSLZ3uxirKWJpT7zRinlr7yR6L8ChhtjJgOPAm91ZSHGmKeMMVnGmKzExEQvhGVp6VKwqKLBa8tUSqmBpNuJ3hhTaYypdr1eCQSJSAJQAAxzmzTFNaxXJUeHAlCoXQoqpfxUtxO9iCSJiLhez3AtsxTYCIwWkXQRCQYWASu6u77OGhQVAuhNU0op/xXY0QQisgw4H0gQkXzg50AQgDHmCeBq4Hsi4gDqgEXGei6wQ0TuBD4AbMCzxpgdPbIVZ2APshEXHkyh1tErpfxUh4neGHNdB+P/Cvy1nXErgZVdC817BkWGUFypdfRKKf/k83fGAsRHBHO8trGvw1BKqT7hF4k+NiyYshpN9Eop/+QXiT4+PJjSaq26UUr5J79I9HHhIVTWO2hqdvZ1KEop1ev8I9FHBANoPb1Syi/5R6IPsxK91tMrpfyRfyT6cFeir9ZEr5TyP36R6ONdVTelWqJXSvkhv0j0rSV6TfRKKT/kF4k+JjQI0ESvlPJPfpHoA20BxIQFaaJXSvklv0j0YFXfaKJXSvkjv0n08eHBlNbo3bFKKf/jN4leS/RKKX/lZ4leOwhXSvkfv0r0x2sbcTpNX4eilFK9qsNELyLPikixiHzdzvjFIrJNRLaLyBciMtlt3EHX8C0iku3NwDsrLjyEZqehsl5L9Uop/+JJif55YP4Zxh8A5hhjJgG/BJ46ZfxcY8wUY0xW10L0jvhwvTtWKeWfOkz0xpg1QNkZxn9hjDnuersOSPFSbF6lnYQrpfyVt+vobwXec3tvgH+JyCYRWXKmGUVkiYhki0h2SUmJl8OC4fHhABwsrfH6spVSqj/rsHNwT4nIXKxEf47b4HOMMQUiMgj4UER2uX4hnMYY8xSuap+srCyvXzFNjrITHBjA4dJaby9aKaX6Na+U6EUkE3gaWGiMKW0ZbowpcP0vBt4EZnhjfV0RECAMiw3lkCZ6pZSf6XaiF5FU4A3gBmPMHrfh4SIS2fIamAe02XKntwyPD+dQmSZ6pZR/6bDqRkSWAecDCSKSD/wcCAIwxjwB3A/EA/8rIgAOVwubwcCbrmGBwCvGmPd7YBs8lhoXxrr9pRhjcMWllFI+r8NEb4y5roPx3wW+28bw/cDk0+foO2nxYdQ2NnOsupHEyJC+DkcppXqF39wZCyda3hwu05Y3Sin/4VeJPjU+DEAvyCql/IpfJfqU2FBE4KAmeqWUH/GrRB8SaGNIdCiH9aYppZQf8atEDzA8PkybWCql/IpfJnq9O1Yp5U/8LtGnxoVTWtNIlT6uWCnlJ/wu0adpyxullJ/xu0Tf0sTysNbTK6X8hN8l+pabprREr5TyF36X6CNCAokPD+aQNrFUSvkJv0v04GpiqSV6pZSf8NNEH6519Eopv+GXiT41LowjFXU0OJr7OhSllOpxfpnoh8eHYQzkldX1dShKKdXj/DLRpyW4Ogo/phdklVK+zy8T/ciECAD2lVT3cSRKKdXzPEr0IvKsiBSLSJt9vorlERHJFZFtIjLNbdxNIrLX9XeTtwLvjuiwIBIiQjTRK6X8QoddCbo8D/wVeLGd8QuA0a6/mcDjwEwRicPqYzYLMMAmEVlhjDnenaC9YWRiOPtK+knVjdMJAV34cdVUB04HBEeACBhj/QUEWP8b3U5kgaFgC3QNr7GmDw4/eXmNNWCcJ5bX4frrwel6ZpDYINi66xhHIzQ3dG5bOlrnqdvTWbZgCHR1H9my39wFhXfuM2ioxjqkAVsIBAafvmz3dTqbIcB2+nIcDdDcaL2WgNM/E/f1tCcgCILsJ947ndBUAwGBEBR6+vq78/m0tez2tqdFoB1sQdbrlmMMTt5v7XE/xrrq1PU4nda2uG+Pu/a2rS2NNdaxGRJxyvBaMB009nBfT+t+k9OX5QUeJXpjzBoRSTvDJAuBF40xBlgnIjEikozVqfiHxpgyABH5EJgPLOtO0N4wclAE724r7LmOwkv2WF+s+JGnjzMGDq+DwROgcAv8/UaYshjm/co6+GqOwaEvrKSx+SUo+AqmfAfSzoGQSEifAxuehA/vt5LKsFlw5ePw5vegqRa+/QK8cy/sX31inaGxMPUGOLDGWifAWbfApQ9Zwz57CA5+Zg0fNB5Gz4Otr1rrm3kbhMVDzgrYtRLGXWJ9ebf9/eSDedSFEDMctvwNHPWd21/xo2Dm7RAxyHpfeQTWPwHNDsi8BnLegdK9nVumu4AgmLwIastg97unj48cArNut+Lf97G1bcPPhsnXWduy8Wk4fhCmXg/5GyFv/Yl57dHw7ZesYZ/89kQiC7TDwsesz+TdH53YbzvehJH/BuEJ1j52T4xp51qfk2mGTc+fvJ72iA0mXQNj50NVEax7HMoPWcMvehASRsM/boHUWRCXDptf7vznEzcSxl9u7ZfKAkBg/EI494eQnAml++Dzv8DWZacn+pAoa5sKsk/enkC7ddyXH4aDa2HS1VB3HHJXwfgrrOXsfOvE/uyqQLu1/tl3WyfTl75l7d+My9y2x43Y4KJfwKw7rM9q7Z+hqtCK9eBnULYfplwPBZsgb501z6RrYOH/WsM+ewhyP/QstrGXQHjiif0WPgh+3I3jvB1i5WYPJrQS/TvGmIltjHsH+J0xZq3r/UfAT7ESvd0Y8yvX8J8BdcaYP7WxjCXAEoDU1NSzDh061IXN8dwzaw/wy3d2sum/LyQ+wksdhW9fbh2w5YesZIFYCTNqiDXeFgQTvgW73oEv/2olCEeDdSDWl1uJMjQOcv4JDleLoMhkGDbTmqelpBiRBNVFMGY+JE+2DkSnwyohBAS6vsQCs++BsDjrxHLoC9jzHsSNgMnfgYo8+OqFE8uKSIJpN1ol0O3/gJJdMOJ8qC2Fou3WeoMjYczFsOcD64syZTHEDrfG1ZbBVy9a25F5LQzK8Hy/OR3w9esn1tNiaJYVz6HPIWmSte9aSoadVZoLW5ZZJahpN544oYCVSPb8Cw6ttd7bQmDcpdYJsPaYNSxuBCRmwO6VEJ1iJfyW0vfmv8Gx3dZyxi+ElOnW8F3vWid0DCRlQtkBa7+NvcRKZk111sknYbQ1fX2ldWKvKrTeRw87eT3tKT9sJe8m170hQ8+CjMutde95z0pu8aOtz7LLn88bULQNUs92nVCOWrE2VFrHYNH2EyfTlu1pkb8Rdq44fb8V74Jtr1nfgxFzrJN5kB1GXWTtZ7HBtBtOfH+6qmU9GGtdzU3WcVu0/cT2iNuvuYOfW/ut5buRMBZi02DvBxCbbhWE3I+DuuNWoaRl+rB46xgLiz9zXDUlsOkF63ObfJ2134JCYfp3u7SZIrLJGJPV5rj+kujdZWVlmezsbI/i6qpPdhdz83Mb+ftt32BGelzXFlJXDm/dYSWh4WfDiwvBHgX2GKsE0dxoldhaSk+N1Se+jFOvt77YDZVw1bOw8f8g+zlr3KgL4KybrZ/L8SOtZFddAjXFVsJa97j1RV3wR6s6Zt/H8OHPrVJIUBis/DHM+SlkfPPkeGuOWSX7lp/wXzwKW1+DGd+1DrTWaganlRQiEq2TxLG91s/n6GHW9rX8/A6JPHn5TfXWtobGdH5fGmNtW0tp0BZibbsIVBdbpZ7u/vKqr7ROhC1VTKcq228l38hk6wTZVGcNa0mUtkDrhBYSefIJp7YMlv+7dRxc+IsTVUCNtfD2HVa12WUPWwmzZb811lr71B59cgyOBqt0DNYX39MTW125VTJ132/OZnjvp9ZJ/VtPWeO68/nUlJx8gqwrt37ptPxC+cb3ITKp7fnb2m8tywi0Wwne/fNpqMKr1RgV+fDFX60S+cLHrJPTqdvTwtlsfYeO7oCz74Sxl1qfac0x67vd1nGw6XlY/5SV4Kfd2P4xdqr2joMu6I1E/yTwiTFmmev9bqwkfz5wvjHmtrama09vJPq8slrO/cNqfnPlJL4zM/XkkRUF1k/glLNOHl5dbJXSWqorNr1glXLA+hLFpMJta9r/kBtrrJKXMVZ1SE9UGSml/NKZEr2nF2M7sgK4U0RexboYW2GMKRSRD4DfiEisa7p5wH1eWme3DI0JJSQwoO2WN+/9BPatturKirbDjresKot/3mNVy7QICofFr1ulhI1PW6WmM53Jg8OtBK+UUr3Io0QvIsuwSucJIpKP1ZImCMAY8wSwErgEyAVqgVtc48pE5JfARteiHmy5MNvXAgKEEYkR5Ba7En1dufWTLTjcqgppqoXd78GXj8GRr2D949bPtptXnqiDDA63/kZfCP/2312vP1ZKqR7kaaub6zoYb4DvtzPuWeDZzofW89Liw9hTWGHVb2/4P6te9uLfWElebLD6N1C2D87/f9ZFktHzYNC4themSV4p1U95q+pmQBoaE0r97lXw+cOQfp7VyuKdH1hVMlMXw4anrAuis75nXYRUSqkByC8fgdBiaGwoU8wujATAoldgxFyrOd3IuVbTQbBao2iSV0oNYH5foh8ju6mLyyAsJBIuuN8q1Y+/AoZMgWtesEr6Sik1gPl3oo8OJD0gl+KYa0gDGDoN7s050bZ2whV9GJ1SSnmHX1fdDG/YT5g0sM8+6cTAyMHavl0p5VP8OtGHF1utPrdIOy1plFLKB/hn1U3ZAXjuEqT2GEUyiF213n9anFJK9Rf+mejzs6HqCExZzLKCDAqOa5eCSinf5Z9VN8cPWv8v+ROlQ8+noFwTvVLKd/lnoi8/CBGDITiMoTFhVNQ1Ud3g6HA2pZQaiPwz0R8/ZD1fGuumKYAjWqpXSvko/030MVaHGUNjrESfV1bblxEppVSP8b9E39wElfmtJfpRiVaLm91Hq/owKKWU6jn+l+gr8qxeflxd4EWHBTE0JpScQk30Sinf5H+JvqXFjatED5CRHEVOYWWfhKOUUj3NfxO9q44eYHxyJPtLqqlvau6bmJRSqgd5lOhFZL6I7BaRXBFZ2sb4P4vIFtffHhEpdxvX7DZuhTeD75TqYlhxFxxca/VW79azfEZyFE4De7SeXinlgzq8M1ZEbMBjwEVAPrBRRFYYY3a2TGOM+YHb9HcBU90WUWeMmeK9kLsoZwV89aL1Om4kBNhaR2UkW8+b33mkksyUmL6ITimleownJfoZQK4xZr8xphF4FVh4humvA5Z5IzivKs6xeo4KT4RBGSeNSo0LIzzYpvX0Simf5MmzboYCeW7v84GZbU0oIsOBdOBjt8F2EckGHMDvjDFvtTPvEmAJQGpqqgdhddLRnZA0Ca5bdlJpHqyOwscmRWrLG6WUT/L2xdhFwHJjjPtVzeHGmCzgO8DDIjKyrRmNMU8ZY7KMMVmJiYnejcoYKN4Bg8dbHYDbo0+bJCM5ipyiSqx+zpVSynd4kugLgGFu71Ncw9qyiFOqbYwxBa7/+4FPOLn+vndUHoH6Chg0vt1JMpKjqKp3kK9PslRK+RhPEv1GYLSIpItIMFYyP631jIiMA2KBL92GxYpIiOt1AjAb2HnqvD2uOMf630GiB7SeXinlczpM9MYYB3An8AGQA/zdGLNDRB4UkcvdJl0EvGpOrvvIALJFZCuwGquOvg8S/Q7r/+D2E/24pEhE0Hp6pZTP8ajjEWPMSmDlKcPuP+X9A23M9wUw6dThvSZvI6x/wnrsQeQQCI1td9LwkECGx4VpiV4p5XN8t4epA2vglUXQVAsYGHVRh7OMHxLFjiOa6JVSvsV3E/3bd1p3v97wBhxef8ZqmxYZSVGs3F5EVX0TkfagXghSKaV6nu8m+upimH4rxKRafx5ouSC7u6iKrLS4noxOKaV6jW8+1Ky5CRx1bbaXP5OxSZEA7Dla3RNRKaVUn/DNRN/gajkTEtmp2YbEhBJkEw5rb1NKKR/im4m+vsL6HxLVqdlsAUJKbJh2K6iU8im+mei7WKIHGBYXpiV6pZRP8dFE72oiae9ciR4gNS5UE71Syqf4aKLveok+NS6MiromKmqbvByUUkr1Dd9M9PWuEn1I51rdAKTGhQNoqV4p5TN8M9F3q+omDNBEr5TyHb6d6Lt0MTYU0ESvlPIdPproq6wOwAPtnZ410h5EXHiwJnqllM/wzURfX2mV5kW6NHtqnLalV0r5Dt9M9A1VXaqfbzE8Poxt+eVsPFjmxaCUUqpv+Giir+xS/XyL2+eMJDosiG8/+SWrdxV7MTCllOp9Pproq7rUtLJFRnIU799zHvHhIbyxub3ucZVSamDwKNGLyHwR2S0iuSKytI3xN4tIiYhscf19123cTSKy1/V3kzeDb1d990r0YPU4dd6YBD7bW0Kz03Q8g1JK9VMdJnoRsQGPAQuA8cB1ItJWLx6vGWOmuP6eds0bB/wcmAnMAH4uIu335+ctDZXdqqNvcf7YQZTXNrEtv9wLQSmlVN/wpEQ/A8g1xuw3xjQCrwILPVz+xcCHxpgyY8xx4ENgftdC7YSGyk4/ubIt545KIEDgk90lXghKKaX6hieJfiiQ5/Y+3zXsVFeJyDYRWS4iwzo5LyKyRESyRSS7pKQbidUYVx1996puAGLDg5k8LIZP9miiV0oNXN66GPtPIM0Yk4lVan+hswswxjxljMkyxmQlJiZ2PZKmOnA6vFJ1AzBvfBJb88r5uqDCK8tTSqne5kmiLwCGub1PcQ1rZYwpNcY0uN4+DZzl6bxe140nV7Zl8axUokODeHjVHq8sTymlepsniX4jMFpE0kUkGFgErHCfQESS3d5eDuS4Xn8AzBORWNdF2HmuYT2noetPrmxLlD2IJeeNYFVOMVvz9KKsUmrg6TDRG2McwJ1YCToH+LsxZoeIPCgil7smu1tEdojIVuBu4GbXvGXAL7FOFhuBB13Dek43HmjWnpvOTiM4MIB3txd6bZlKKdVbAj2ZyBizEkLHhg8AABfqSURBVFh5yrD73V7fB9zXzrzPAs92I8bOqe/6I4rbExESyMjECPYcrfLaMpVSqrf43p2xXq6jbzF2cAR7j1Z7dZlKKdUbfDDRt1TdeK9EDzAmKZKC8jqq6rWLQaXUwOKDib5nSvRjBlnL26OleqXUAKOJ3kNjk1oSvdbTK6UGFt9M9IF2sAV5dbFDY0IJDbJpoldKDTi+l+gbq71emgcICBDGDNaWN0qpgcf3En1DFQRH9MiixwyOZOeRSnKLNdkrpQYOH0z0PVOiB5g/MYnKegcX/s8a/r4xr+MZlFKqH/DBRO+dJ1e25YKMwXx537+RGhfGv3Ye7ZF1KKWUt/leom/suaobgEGRdiYPi2FXUWWPrUMppbzJ9xJ9D5boW4xLiiT/eB2VevOUUmoA8MFEXw0hPVeiBxifbN11u6tQL8oqpfo/H0z0vVCiT7aWr9U3SqmBwLcSfbMDHHUQ3LOJPinKTkxYEDmFmuiVUv2fbyX6xpbHH/Rs1Y2IMC4pkhytulFKDQC+legbXA8c6+GqG4CM5Ch2F1VR39Tc4+tSSqnu8CjRi8h8EdktIrkisrSN8feKyE4R2SYiH4nIcLdxzSKyxfW34tR5varRleh7sHlli7ljB1HvaOb6p9dzvKaxx9enlFJd1WGiFxEb8BiwABgPXCci40+ZbDOQZYzJBJYDf3AbV2eMmeL6u5ye1PrkSu8+i74t541J5K/XTWNbQQX3r9jR4+tTSqmu8qREPwPINcbsN8Y0Aq8CC90nMMasNsbUut6uA1K8G6aHGnqnjr7FpZnJ3DBrOO9tL6S4qr5X1qmUUp3lSaIfCrg/2CXfNaw9twLvub23i0i2iKwTkSvam0lElrimyy4pKfEgrDb00LPoz+Q7M1NxOI0++0Yp1W959WKsiFwPZAF/dBs83BiTBXwHeFhERrY1rzHmKWNMljEmKzExsWsB9GIdfYuRiRGcPTKeV9YfpsGhF2aVUv2PJ4m+ABjm9j7FNewkInIh8F/A5caYhpbhxpgC1//9wCfA1G7Ee2Z9UKIH+I9zR3Ckop4bn9lARa0+FkEp1b94kug3AqNFJF1EgoFFwEmtZ0RkKvAkVpIvdhseKyIhrtcJwGxgp7eCP01D75foAeaOG8RfFk3hq8PHmfXbj/jxP7Zqs0ulVL8R2NEExhiHiNwJfADYgGeNMTtE5EEg2xizAquqJgL4h4gAHHa1sMkAnhQRJ9ZJ5XfGmB5M9JVgC4HA4B5bRXsWThnKyMQInvv8IP/YlM+csYl8M3NIr8ehlFKn6jDRAxhjVgIrTxl2v9vrC9uZ7wtgUncC7JQe6kbQUxOHRvOHqzP5eNdRPsop1kSvlOoXfO/O2F5qWtkeW4Awd+wgVu8uptlp+jQWpZQCn0v0VT3+QDNPXJAxmPLaJtbmHuOjnKOa8JVSfcq3En0fV920OHdMAoEBwi3PbeDWF7J57vMDfR2SUsqP+Vaib6js86obgCh7EFdOHcq01Fiyhsfyl4/2Ulrd0PGMSinVA3ws0fePEj3AH6+ZzPLvnc3vrsqktrGZX72bgzFahaOU6n0+luh7tmPwrhg1KILvzx3Fm5sL+H9vbsfZTn29MYbaRkcvR6eU8ge+lej7SR39qX5w4WjunDuKZRvyeHLNfipqm3hgxQ72HrXu5DXGcM+rW5jzx0+oadBkr5TyLo/a0Q8Y33oKYtP6OorTiAg/nDeG3OJq/rxqD+/vKGJrXjmrco6y4s5zeHfbEVZsPQLAG5sLuGHW8A6WqJRSnpP+WG+clZVlsrOz+zoMryupamDenz/leG0Td5w/kqfXHkCABoeTOWMSKa1poKHJyb9+cB6uO4yVUsojIrLJ9QDJ0/hWib6fS4wM4blbZlBUUc/8iUnMSI/j/a+LGDM4kmuyUnj/6yJ+vHwbn+wuYe64QX0drlLKR2iJvh+pb2pm3p/XUFxVz4OXT+SCjEHER4T0dVhKqQFAS/QDhD3IxuvfO5vbX97ET17fBlitdhZMTOLGb6Txee4x3tl2hKULMhg1qO3WRcYYdhZWMnZwJIE237rWrpTqGi3R90NNzU42Hijj6yMVfLK7hHX7SwkQweE02AKEsCAb912SwaWZyQTZhKfW7Odv6w/zo3ljOHCslic+3ceFGYN49LpphAbb+npzBjRjDP/IzmfuuEEkRvb/X1fGGL2+46fOVKLXRD8A7C+p5sUvDzEiMZy5Ywdx5ytfsTW/4qRpUuPCOFxmdds7Mz2ODQfLmJ4Wx8u3ziQ48ETJvqK2iQOlNUwaGo0tQDDGkFdWx+DoEEICT5wU6puasQfZyCur5S8f7WVqagwXT0gi4QxVSXuOVnG0sp5zR3veQ1iDo5nfvJtDSJCNO84fSUxYMMYYdh+tYlCknYq6JrbmlXPxhKRunbRqGhz85PVtZCRF8v25o7jn1S2IwEPXTD7jL5+1e49x/TPrufqsFP50zWQAiivrOVrZwKSU6Nbpmp2GAOG0JLvpUBnBNttJ03bE0ezkn9uOMHVYLGkJ4R7Pt+nQcf7ztc38ZdFUpqXGAtb+bXA4ibIHebwcNTBpovcxxhi25lfwee4xjDFMGx7LjLQ4/vLRXhxOw4/njeXtrQX84LWtXD8rlRnp8RRX1pMWH879b3/NkYp6BkeFMCQmlMLyeooq6xmXFMnvr8rk0z0l/HPrEQ6V1fLwtVN48tN9bCuowBgItgWwcMoQrj4rhUCb8N72IrbkldPkNKTFh/HOtkKanYZ7LhjNf144GoDP9h7DAFH2QA6V1hIabGNQZAhNzYZj1Q288MVB1h8oI0AgKjSIV5fMYlteRWvVVYsFE5P4xcIJvPzlIdISwslIjqKwoo5xSVEMiQk9bR/tL6nm832lnDsqgdrGZn729tdsOnQcEatHsKfW7AfguhnD+M6M4azZW8KHO49yTVYK35mR2pqwb3p2A5/uKSHIJrx3z3n84f1drMo5itPALxdO4IZvpJFXVsvip9eTmRLNI4umEuA6gT6z9gC/XplDSGAAz98yg1kj4k+KsanZyXtfF5FXVos9yMbcsYnsL6nhLx/tZXtBBaFBNn6xcALfzhpGTYODjQfLGJkYQUps6GknlKZmJ5c+8hl7jlaTmRLNW3fMpqrBwaKn1lFSVc8b35tNanzYGY+rTYeOMyw2lEFRdo+PRafTsDb3GGcNjyU8pHM1wY0OJz9ZvpWM5ChumzOS9ftLiY8IZtSg7t0Lk1tcxZf7Slk8czgBAdZ+eu7zA7y7rZCymkZevHUG9U3NPLxqL/dfNp5Bkadv758+2M2Gg2U8uHAC45KiOlxndYODbfnlTBkWQ1hw39SIa6L3U798ZyfPrD35gWpDou3cMXcUn+ceo7rBQUxYMBnJkTy+eh9Vrpu1ZqbHUd3gYMeRSgAeXzyNEYkRvLzuEMs35VPn6j0rODCAzKFWSfXrIxVcljkEp4HXv8pnzOAIwkMC2Xy4/IwxBgcG8MerMxmbFMkNz2wgIiSQY9UNjB0cyfyJSYQE2ThW1cBfPtpLSGAADQ7nacuYkR7HXf82irjwYHKLq9l8uJxX1h+msfnEtMG2AH595UQe+tceiirrmTQ0mrNHxvOkK+EDDIsLJa+sjlGDIkiOtjNlWAyPfpzLt7NSWL4pn9AgG/UOJ7fPGcHuoipW5RRzYcZgdh+t5GhlA40OJ7eek87dF4zmN+/m8Fp2HvPGD+bAsRqOlNfxqysncsWUoQCs3l3Mb1buIre4+rTtSYgI4ccXj+GtzUf4cn8pP5k/lg93Hm3dlwkRwWQNj+O756aTlRaHMYY/r9rLIx/t5cqpQ3lzcwG3zE5j8+FydhypwB5kIzEyhEXTh9HUbDDGMGFoNLPS4wkNtlFcVc+v383h7S1HiAgJ5IfzxrBoemrrL6iq+iYKyusoqqhn3f4ywoJtXDl1KCmxofzq3RyeWXuApCg7Dy6cwLwJSYB1Aqioa6Kk2moyPCTGTnxECMYYvtxfSpQ9iOWb8nn+i4MAnD82kU92l5AYGcKKO2fz63dzsAfZ+O23JhEggtMYgtr45VVYUcfL6w5xyaRkJgyJ5qvDx7n52Q1U1jtYPDOVX10xkeWb8vnx8m1MGBLF/pIavjEynpKqBrYXVHDdjGHcd0kGr6w/zNRhMWSlxZFTWMllf12LYD12/H++PYXLJg8hr6yWhIiQ035ZHjhWw3df2Mi+khpCg2xcNjmZG7+RxpjBkZTXNnKkop7ModGtJ50WBeV15JXVMjjKTnonfrm1RxO9n2pqdvLSl4cYlxxJWnw4W/LKmZke12ZLntziKlZuL+KyyUNITwinqr6JH7y2hTGDI/nJ/HGt09U0OPhoVzFOp+HC8YOJcJXiWuqGjTG8/lUBL3xxkNLqBu6+YDTpCeFU1jtIiw+jvslJSXU9QbYA4sNDGBYXSqSrWuHLfaUsfnodwYEBvH/Pea3VFsYYHlixg5zCKn55xURqGh0UHK9jcJSdjQfL+Nu6QxypqG+N0RYgXJaZzHfPHcHGg2WEhwQyd6xVx/5RzlGWvrGdZ27KYuKQaDYeLGuNbWRiBH/bcJiPc45SWFHPrqIq7EEBfLn0Av777a9Zub2Qh6+dwsIpQ2l0OPnD+7v4MOcodY3NPHVjFss35fHyusOtcdw5dxT3XjSGY9UN3PbyJjYfLmdYnPXrI6+sjmFxofz3peOZMyaR4soGPt1TzPD4cGaOiCMk0Iaj2cmdr2zm/R1F2AKEBxdOwGlgy+FyPt1TzLHqRiYNjSY6NIi1ucf4ZmYyjyyayrce/4IteeWEBdv449WTSYgI5t+f30hN48ndW4YEBpCZEs3W/AqMMSw5bwTb8iv4bO8xIu2BzEyPJzBA+Hh3MY2uE2yQzbpWZAykxIaSf7yOK6YMYc/RanYWVvKjeWPYcaSSVTlHaWo+ObekJ4RjD7KRU1jZOuzms9Morqpn5fYiLpmUxKqcYoJtAVS7Ch3fGBHPvpJqahubuXhCktVFp8DklGh2FVXx7rZCGhxOQoNsXJqZzIqtR0iOtnPOqAT+tv4wYwZHcKi0lrOGx/Liv8/guc8P8uuVOQBkpkTzdUEFY5OiWmMaGhNKaLCNsppG3vje2fxk+TY25x1nzphEVuUUMzgqhNmjEliz5xjx4cHEhQez8WAZkfZAli4Yx5a8ct7cXEB9kxMRaEmvZw2PZemCcWQNj6Wp2fDHD3bxf58daD1evzdnJJNSojEG5k9M6vjL3YZuJ3oRmQ/8BasrwaeNMb87ZXwI8CJwFlAKXGuMOegadx9wK9AM3G2M+aCj9Wmi91/vf11ISJCNuWM9v4+gwdHMe9uLCLQJYwZbJzX36xKn8vSC5c4jlTicTjJTYqhucHCotIYJQ06va29ZntNp+Cz3GBsOlJKZYl3TaNHsNLyy4TDr9pfS0NTMgonJXDZ5yBnjBKt64/fv72JmelxraRmgttHBK+sP897XRewrqeaO80dy6zkjsAUI1Q0OyqobGRobis1ViqxvasbhNAQGWKXj7IPH+XRPCesPlDJpaDS3nTeStIRwjDFsOFDGa9l57CiopKKuiYsnDGZGejyx4UFMGRbD8dom3tl6hPUHyhg9OIKfXjyOxmYndy3bzIc7jxIaZOPa6cMYHh9GYmQIQbYADpfW8lnuMYor67lldhoBIhwpr+eOuSMB2F1UxYQhUby07hD3v72DBxdOoK6xmd++t4sZ6XGkxITy4c6jxEcE0+BwUlhRT0xYEPPGD+a6Gak8sGIH2wsquPqsFH588TgSIoJ5/ouDfLqnhNrGZv538TQSIkJwNDtZ/PR6hseHsXRBBnP+sJq6pmYe+vZkRISXvjzIxoPH+cPVmXw7axiV9U1c++Q69hyt4uaz09iSV05OYSVzxw2iqt5BcWU9c8YmcsOs4aTEWlVjx2sa+WhXMXlltUTaAwkODODhVXspq2kkPjyYmkYH9U1OFs9MZf7EJN7cXMAbXxUAEB8ezKafXdThsdmWbiV6EbEBe4CLgHyszsKvc+/7VUTuADKNMbeLyCLgSmPMtSIyHlgGzACGAKuAMcaYM/acrYleqYHH0ezkzc0FnD0qgaFtXDfxVEVtE9Fh1q+8kqoGEiKCTzoxG2MoqWogPiKk9UTW6HBSVtNIUnTH1xfcT/QbXNeHstLiWseXVDWc1MKqpsFBWU0jw+LCTpvfU1X1TXy48yif55YSGxbE7NEJJxVm9h6tar1o3tG1lPZ0N9F/A3jAGHOx6/19AMaY37pN84Frmi9FJBAoAhKBpe7Tuk93pnVqoldKqc45U6L35I6aoUCe2/t817A2pzHGOIAKIN7DeVuCXCIi2SKSXVJS4kFYSimlPNFvbp00xjxljMkyxmQlJnreDlsppdSZeZLoC4Bhbu9TXMPanMZVdRONdVHWk3mVUkr1IE8S/UZgtIiki0gwsAhYcco0K4CbXK+vBj42VuX/CmCRiISISDowGtjgndCVUkp5osNbuIwxDhG5E/gAq3nls8aYHSLyIJBtjFkBPAO8JCK5QBnWyQDXdH8HdgIO4PsdtbhRSinlXXrDlFJK+YDutrpRSik1gGmiV0opH9cvq25EpAQ41MXZE4BjXgzHWzSuzuuvsWlcnaNxdV5XYhtujGmzbXq/TPTdISLZ7dVT9SWNq/P6a2waV+doXJ3n7di06kYppXycJnqllPJxvpjon+rrANqhcXVef41N4+ocjavzvBqbz9XRK6WUOpkvluiVUkq50USvlFI+zmcSvYjMF5HdIpIrIkv7MI5hIrJaRHaKyA4Rucc1/AERKRCRLa6/S/oovoMist0VQ7ZrWJyIfCgie13/Y3s5prFu+2WLiFSKyH/2xT4TkWdFpFhEvnYb1ub+EcsjrmNum4hM64PY/igiu1zrf1NEYlzD00Skzm3fPdHLcbX72YnIfa59tltELu7luF5zi+mgiGxxDe/N/dVejui548wYM+D/sB62tg8YAQQDW4HxfRRLMjDN9ToSqxvG8cADwI/6wb46CCScMuwPwFLX66XA7/v4sywChvfFPgPOA6YBX3e0f4BLgPcAAWYB6/sgtnlAoOv1791iS3Ofrg/iavOzc30XtgIhQLrre2vrrbhOGf8QcH8f7K/2ckSPHWe+UqKfAeQaY/YbYxqBV4GFfRGIMabQGPOV63UVkEM7vWr1IwuBF1yvXwCu6MNYLgD2GWO6emd0txhj1mA9gdVde/tnIfCisawDYkQkuTdjM8b8y1i9ugGsw+rzoVe1s8/asxB41RjTYIw5AORifX97NS4REeDbWH1a96oz5IgeO858JdF73GVhbxKRNGAqsN416E7XT69ne7t6xI0B/iUim0RkiWvYYGNMoet1ETC4b0IDrEdcu3/5+sM+a2//9Lfj7t+xSn4t0kVks4h8KiLn9kE8bX12/WWfnQscNcbsdRvW6/vrlBzRY8eZryT6fkdEIoDXgf80xlQCjwMjgSlAIdbPxr5wjjFmGrAA+L6InOc+0li/Ffukza1YHdtcDvzDNai/7LNWfbl/zkRE/gurz4e/uQYVAqnGmKnAvcArIhLViyH1u8/uFNdxcoGi1/dXGzmilbePM19J9P2qy0IRCcL6AP9mjHkDwBhz1BjTbIxxAv9HD/1c7YgxpsD1vxh40xXH0Zafgq7/xX0RG9bJ5ytjzFFXjP1in9H+/ukXx52I3Ax8E1jsShC4qkZKXa83YdWFj+mtmM7w2fX5PhOru9NvAa+1DOvt/dVWjqAHjzNfSfSedHfYK1x1f88AOcaY/3Eb7l6ndiXw9anz9kJs4SIS2fIa60Le15zcFeRNwNu9HZvLSaWs/rDPXNrbPyuAG12tImYBFW4/vXuFiMwHfgJcboypdRueKCI21+sRWN147u/FuNr77PpD96IXAruMMfktA3pzf7WXI+jJ46w3rjL3xh/Wlek9WGfi/+rDOM7B+sm1Ddji+rsEeAnY7hq+Akjug9hGYLV42ArsaNlPQDzwEbAXWAXE9UFs4Vgdyke7Dev1fYZ1oikEmrDqQm9tb/9gtYJ4zHXMbQey+iC2XKz625Zj7QnXtFe5PuMtwFfAZb0cV7ufHfBfrn22G1jQm3G5hj8P3H7KtL25v9rLET12nOkjEJRSysf5StWNUkqpdmiiV0opH6eJXimlfJwmeqWU8nGa6JVSysdpoldKKR+niV4ppXzc/werzw3QOTy/pgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the loss and accuracy\n",
    "import matplotlib.pyplot as plt\n",
    "plt.plot(losses)\n",
    "plt.plot(accuracies)\n",
    "plt.legend(['Loss', 'Accuracy'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "30cq1fmqK-my"
   },
   "source": [
    "It looks like the model achieves a very high accuracy and small loss on the training dataset. To see how well it generalizes, let's test on the testing nodes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_6Q354OSLVkG",
    "outputId": "8a86891a-bd2f-4d43-cb4f-d4773c8b6f00"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.7900\n"
     ]
    }
   ],
   "source": [
    "# evaluate the model on test set\n",
    "model.eval()\n",
    "pred = model(data).argmax(dim=1)\n",
    "correct = compute_accuracy(pred[data.test_mask], data.y[data.test_mask])\n",
    "acc = int(correct) / int(data.test_mask.sum())\n",
    "print(f'Accuracy: {acc:.4f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iwOLBEpgLW3p"
   },
   "source": [
    "Very cool! It seems we got a very nice accuracy for the test as well. Our model is doing okay. There are many ways you can go about trying to improve this model, but we will keep that for another time. Hopefully, with this tutorial you got a glimpse of graph data and how to use PyTorch Geometric to train GNNs on a very popular dataset. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NapVlc1wH_1Y"
   },
   "source": [
    "Note that I haven't tested if this code works with GPUs. I will leave that as an exercise for the learner. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZtRNFO0mL2_b"
   },
   "source": [
    "If you are interested in the full tutorial and more examples, visit the [PyTorch Geomtric documentation](https://pytorch-geometric.readthedocs.io/en/latest/notes/introduction.html) where I adapted the code from. \n",
    "\n",
    "Feel free to reach out on [Twitter](https://twitter.com/omarsar0) if you have any further questions."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "Introduction to GNNs with PyTorch Geometric.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}