Colab0.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# CS224W - Colab 0\n",
        "\n",
        "Colab 0 可以帮助你熟悉图挖掘和图神经网络的基本概念。本次实验中将介绍两个库，[NetworkX](https://networkx.org/documentation/stable/) 和 [PyTorch Geometric](https://pytorch-geometric.readthedocs.io/en/latest/).\n",
        "\n",
        "你可以对实验中 PyTorch Geometric 部分不熟。图神经网络的概念和实现将在后续课程和实验中介绍。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8gzsP50bF6Gb"
      },
      "source": [
        "## NetworkX 教程\n",
        "\n",
        "`NetworkX` 是 `python` 中构建、操纵以及挖掘图时最常用的库。本部分教程主要使用来自[官方教程](https://colab.research.google.com/github/jdwittenauer/ipython-notebooks/blob/master/notebooks/libraries/NetworkX.ipynb#scrollTo=zA1OO6huHeV6)的内容。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Nwwq0nSdmsOL"
      },
      "source": [
        "### 配置"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QeqN7MHvH1OA"
      },
      "outputs": [],
      "source": [
        "# Import the NetworkX package\n",
        "import networkx as nx"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MCIeGAfLfAMK"
      },
      "source": [
        "### 图\n",
        "\n",
        "NetworkX 提供了用于存储各种图的类，比如有向图、无向图等。同时也提供了能够创建多重图的类。更多信息请参考[NetworkX 图类型](https://networkx.org/documentation/stable/reference/classes/index.html)。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qSI_n6P-e0PW"
      },
      "outputs": [],
      "source": [
        "# Create an undirected graph G\n",
        "G = nx.Graph()\n",
        "print(G.is_directed())\n",
        "\n",
        "# Create a directed graph H\n",
        "H = nx.DiGraph()\n",
        "print(H.is_directed())\n",
        "\n",
        "# Add graph level attribute\n",
        "G.graph[\"Name\"] = \"Bar\"\n",
        "print(G.graph)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x0pLs0-Ka9j8"
      },
      "source": [
        "### 结点\n",
        "\n",
        "结点（带属性）可以很轻松地加入到 NetworkX 的图中。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jQ8ApAL5H1GB"
      },
      "outputs": [],
      "source": [
        "# Add one node with node level attributes\n",
        "G.add_node(0, feature=0, label=0)\n",
        "\n",
        "# Get attributes of the node 0\n",
        "print(f\"Node 0 has the attributes {G.nodes[0]}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "btOdMYqJaKia"
      },
      "outputs": [],
      "source": [
        "# Add multiple nodes with attributes\n",
        "G.add_nodes_from([\n",
        "  (1, {\"feature\": 1, \"label\": 1}),\n",
        "  (2, {\"feature\": 2, \"label\": 2})\n",
        "])\n",
        "\n",
        "# Loop through all the nodes\n",
        "# Set data=True will return node attributes\n",
        "for node in G.nodes(data=True):\n",
        "  print(node)\n",
        "\n",
        "# Get number of nodes\n",
        "print(f\"G has {G.number_of_nodes()} nodes\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0AdoaZPgbRis"
      },
      "source": [
        "### 边\n",
        "\n",
        "与结点类似，边（带属性）也可以很轻松地加入到 NetworkX 图中。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0szH5F6EH079"
      },
      "outputs": [],
      "source": [
        "# Add one edge with edge weight 0.5\n",
        "G.add_edge(0, 1, weight=0.5)\n",
        "\n",
        "# Get attributes of the edge (0, 1)\n",
        "print(f\"Edge (0, 1) has the attributes {G.edges[(0, 1)]}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QRTmi4EUaf_I"
      },
      "outputs": [],
      "source": [
        "# Add multiple edges with edge weights\n",
        "G.add_edges_from([\n",
        "  (1, 2, {\"weight\": 0.3}),\n",
        "  (2, 0, {\"weight\": 0.1})\n",
        "])\n",
        "\n",
        "# Loop through all the edges\n",
        "# Here there is no data=True, so only the edge will be returned\n",
        "for edge in G.edges():\n",
        "  print(edge)\n",
        "\n",
        "# Get number of edges\n",
        "print(f\"G has {G.number_of_edges()} edges\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9u1Utjn4bc7k"
      },
      "source": [
        "### 可视化"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lykPzFOEahuP"
      },
      "outputs": [],
      "source": [
        "# Draw the graph\n",
        "nx.draw(G, with_labels = True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_Q6YTP2FDbOS"
      },
      "source": [
        "### 结点的度和邻居"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GFA3B6Z_DE3q"
      },
      "outputs": [],
      "source": [
        "node_id = 1\n",
        "\n",
        "# Degree of node 1\n",
        "print(f\"Node {node_id} has degree {G.degree[node_id]}\")\n",
        "\n",
        "# Get neighbor of node 1\n",
        "for neighbor in G.neighbors(node_id):\n",
        "  print(f\"Node {node_id} has neighbor {neighbor}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4gVRVckZeSdA"
      },
      "source": [
        "### 其他功能\n",
        "\n",
        "NetworkX 也提供了很多其他用来学习图的方法。\n",
        "\n",
        "这里是结点 [PageRank](https://networkx.org/documentation/stable/reference/algorithms/generated/networkx.algorithms.link_analysis.pagerank_alg.pagerank.html#networkx.algorithms.link_analysis.pagerank_alg.pagerank) 的示例（PageRank 原理可以参考 Lecture 内容）。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_gZfQ82Wiuvv"
      },
      "outputs": [],
      "source": [
        "num_nodes = 4\n",
        "# Create a new path like graph and change it to a directed graph\n",
        "G = nx.DiGraph(nx.path_graph(num_nodes))\n",
        "nx.draw(G, with_labels = True)\n",
        "\n",
        "# Get the PageRank\n",
        "pr = nx.pagerank(G, alpha=0.8)\n",
        "print(pr)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RrIFCJnlvGkg"
      },
      "source": [
        "### 文档\n",
        "\n",
        "你可以查阅 [NetworkX的文档](https://networkx.org/documentation/stable/) 以获取更全面的了解。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MDcCjrW3JWzG"
      },
      "source": [
        "## PyTorch Geometric 教程\n",
        "\n",
        "PyTorch Geometric (PyG) 是PyTorch库的拓展。它提供了用来构建图深度学习模型的基础模块，包括各种图神经网络层和大量的测试数据集。\n",
        "\n",
        "如果还不理解 `GCNConv` 的话请不用担心 —— 后续的 lecture 会涵盖这部分内容。\n",
        "\n",
        "本教程主要采用了来自[Matthias Fey](https://rusty1s.github.io/#/)的[教程](https://colab.research.google.com/drive/1h3-vJGRVloF5zStxL5I0rSy4ZUPNsjy8?usp=sharing#scrollTo=ci-LpZWhRJoI)。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WU7NCZtQ6msa"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "print(f\"PyTorch has version {torch.__version__}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n7gMjMCT677n"
      },
      "source": [
        "### 配置\n",
        "\n",
        "请按照[官方指示](https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html)，安装`PyG`库。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HLxnaKsN8GVf"
      },
      "source": [
        "### 可视化"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qywlcjyr8USw"
      },
      "outputs": [],
      "source": [
        "# Helper function for visualization.\n",
        "%matplotlib inline\n",
        "import torch\n",
        "import networkx as nx\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# Visualization function for NX graph or PyTorch tensor\n",
        "def visualize(h, color, epoch=None, loss=None):\n",
        "    plt.figure(figsize=(7,7))\n",
        "    plt.xticks([])\n",
        "    plt.yticks([])\n",
        "\n",
        "    if torch.is_tensor(h):\n",
        "        h = h.detach().cpu().numpy()\n",
        "        plt.scatter(h[:, 0], h[:, 1], s=140, c=color, cmap=\"Set2\")\n",
        "        if epoch is not None and loss is not None:\n",
        "            plt.xlabel(f'Epoch: {epoch}, Loss: {loss.item():.4f}', fontsize=16)\n",
        "    else:\n",
        "        nx.draw_networkx(G, pos=nx.spring_layout(G, seed=42), with_labels=False,\n",
        "                         node_color=color, cmap=\"Set2\")\n",
        "    plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bbny-iTO7NQN"
      },
      "source": [
        "### 介绍\n",
        "\n",
        "最近几年，图深度学习逐渐成为深度学习社区中最火热的研究领域之一。图神经网络GNN旨在将经典深度学习概念推广到不规则结构的数据上（相较于图片或文本），并让神经网络能够对结果进行解释。\n",
        "\n",
        "本教程会通过基于[`PyG`](https://github.com/rusty1s/pytorch_geometric)库的模型，给你引入一些关于图深度学习的基本概念。\n",
        "\n",
        "跟随[Kipf et al. (2017)](https://arxiv.org/abs/1609.02907)，让我们深入GNN的世界看一下简单的图结构样例，即著名的[Zachary's karate club network](https://en.wikipedia.org/wiki/Zachary%27s_karate_club)。这个图是一个社交网络，共有空手道成员34名，并记录了这些人中哪两个人之间在空手道俱乐部之外相互互动。此处，我们想要根据成员间互动检测生成的小团体。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g3uPffzbyqn9"
      },
      "source": [
        "### Datasets 对象\n",
        "\n",
        "`PyG`提供了访问数据集的简单方法，即用[`torch_geometric.datasets`](https://pytorch-geometric.readthedocs.io/en/latest/modules/datasets.html#torch_geometric.datasets)的子模块。在初始化[`KarateClub`](https://pytorch-geometric.readthedocs.io/en/latest/modules/datasets.html#torch_geometric.datasets.KarateClub)数据集后，我们可以先看看他的一些属性。\n",
        "\n",
        "比如，我们可以看到这个数据集只有**一张图**，并且数据集中的每个结点都被分配了**34维特征向量**，可以唯一地描述空手道俱乐部中地成员。\n",
        "\n",
        "除此之外，图中还有4个分类，代表了成员所属的小团体。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YrpL9CtS7nx2"
      },
      "outputs": [],
      "source": [
        "from torch_geometric.datasets import KarateClub\n",
        "\n",
        "dataset = KarateClub()\n",
        "print(f'Dataset: {dataset}:')\n",
        "print('======================')\n",
        "print(f'Number of graphs: {len(dataset)}')\n",
        "print(f'Number of features: {dataset.num_features}')\n",
        "print(f'Number of classes: {dataset.num_classes}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I5zhmKIH72Rf"
      },
      "source": [
        "### Data 对象\n",
        "\n",
        "PyTorch Geometric 中的每张图都由一个[`Data`](https://pytorch-geometric.readthedocs.io/en/latest/modules/data.html#torch_geometric.data.Data)对象表示，该对象包含了用于描述这张图所有信息。我们可以打印该对象`print(data)`以获得这张图的概述。\n",
        "\n",
        "``` python\n",
        "Data(edge_index=[2, 156], x=[34, 34], y=[34], train_mask=[34])\n",
        "```\n",
        "\n",
        "可以看到，`Data`对象有4个属性：\n",
        "\n",
        "1. `edge_index`属性有关于**图连通性**的所有信息，即每条边的源节点和终结点。\n",
        "2. `x`是**结点特征**，该例中每个结点都有一个34维的特征向量。\n",
        "3. `y`是**结点的标签**，该例中为结点所属的类。\n",
        "4. `train_mask`描述了我们已知分类的结点，该例中每个团体只有一个，共4个结点。（即我们知道了34个结点中的4个结点属于哪个团体。）\n",
        "\n",
        "`Data`对象还提供了一些**工具函数**来计算图的一些基础属性。比如，我们可以很轻松的推测出图中是否存在孤立结点，图是否有自连边，亦或图是否有向等。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sTLapYhP7uCn"
      },
      "outputs": [],
      "source": [
        "data = dataset[0]  # Get the first graph object.\n",
        "\n",
        "print(data)\n",
        "print('==============================================================')\n",
        "\n",
        "# Gather some statistics about the graph.\n",
        "print(f'Number of nodes: {data.num_nodes}')\n",
        "print(f'Number of edges: {data.num_edges}')\n",
        "print(f'Average node degree: {data.num_edges / data.num_nodes:.2f}')\n",
        "print(f'Number of training nodes: {data.train_mask.sum()}')\n",
        "print(f'Training node label rate: {int(data.train_mask.sum()) / data.num_nodes:.2f}')\n",
        "print(f'Contains isolated nodes: {data.has_isolated_nodes()}')\n",
        "print(f'Contains self-loops: {data.has_self_loops()}')\n",
        "print(f'Is undirected: {data.is_undirected()}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wQJyi9OB8dh_"
      },
      "source": [
        "### Edge Index 对象\n",
        "\n",
        "通过打印`edge_index`，我们可以更深入地理解`PyG`在内部是如何表示图的连通性的。可以看到对于每条边，`edge_index`都有一个包含两个结点序号的元组与之对应，其中第一个结点序号是源结点的索引，第二个是终结点的索引。这种表示称为**COO(coordinate format)格式**，通常用来表示稀疏矩阵。`PyG`选择使用**COO格式**，而没有选择用邻接矩阵形式的稠密表征。\n",
        "\n",
        "在将`Data`对象转化为`networkx`库形式后，我们可以进行可视化。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iFA6Xi4O79r0"
      },
      "outputs": [],
      "source": [
        "edge_index = data.edge_index\n",
        "print(edge_index.t())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3KfJHtlV8h3W"
      },
      "outputs": [],
      "source": [
        "from torch_geometric.utils import to_networkx\n",
        "\n",
        "G = to_networkx(data, to_undirected=True)\n",
        "visualize(G, color=data.y)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NUdHZY2u8vn3"
      },
      "source": [
        "### 实现图神经网络\n",
        "\n",
        "在学习了`PyG`的数据操作之后，是时候搭建我们第一个图神经网络了！\n",
        "\n",
        "我们将使用最简单的GNN算子，即**GCN层** ([Kipf et al. (2017)](https://arxiv.org/abs/1609.02907))。\n",
        "\n",
        "`PyG`通过[`GCNConv`](https://pytorch-geometric.readthedocs.io/en/latest/modules/nn.html#torch_geometric.nn.conv.GCNConv)实现了**GCN层**，可以通过传入结点表征`x`以及COO形式的图连通性表征`edge_index`。\n",
        "\n",
        "有了`GCNConv`，我们就可以通过在一个`torch.nn.Module`类中定义网络结构，以创建第一个图神经网络。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6tQGQF8r8zIr"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "from torch.nn import Linear\n",
        "from torch_geometric.nn import GCNConv\n",
        "\n",
        "\n",
        "class GCN(torch.nn.Module):\n",
        "    def __init__(self):\n",
        "        super(GCN, self).__init__()\n",
        "        torch.manual_seed(12345)\n",
        "        self.conv1 = GCNConv(dataset.num_features, 4)\n",
        "        self.conv2 = GCNConv(4, 4)\n",
        "        self.conv3 = GCNConv(4, 2)\n",
        "        self.classifier = Linear(2, dataset.num_classes)\n",
        "\n",
        "    def forward(self, x, edge_index):\n",
        "        h = self.conv1(x, edge_index)\n",
        "        h = h.tanh()\n",
        "        h = self.conv2(h, edge_index)\n",
        "        h = h.tanh()\n",
        "        h = self.conv3(h, edge_index)\n",
        "        h = h.tanh()  # Final GNN embedding space.\n",
        "        \n",
        "        # Apply a final (linear) classifier.\n",
        "        out = self.classifier(h)\n",
        "\n",
        "        return out, h\n",
        "\n",
        "model = GCN()\n",
        "print(model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5zgbaD5P8_M_"
      },
      "source": [
        "此处我们在`__init__`方法中初始化了我们所有的层，并且在`forward`中定义了我们网络的计算流。\n",
        "\n",
        "首先定义并堆叠了三层`GCNConv`，即可以聚集每个结点3跳领域内的信息。除此之外，`GCNConv`层还将结点特征维度降到了2，且每个`GCNConv`层都由[`tanh`](https://pytorch.org/docs/stable/generated/torch.nn.Tanh.html?highlight=tanh#torch.nn.Tanh)激活。\n",
        "\n",
        "然后，我们施加了一个线性变换，用来将结点映射到4个社区中的1个。\n",
        "\n",
        "最终返回分类器的分类结果和最后一层`GCNConv`之后的结点表征。\n",
        "\n",
        "接下来我们通过`GCN()`来实例化模型，然后打印模型来输出该模型所使用子模块的概述。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "48uhs_0j9AMX"
      },
      "outputs": [],
      "source": [
        "model = GCN()\n",
        "\n",
        "_, h = model(data.x, data.edge_index)\n",
        "print(f'Embedding shape: {list(h.shape)}')\n",
        "\n",
        "visualize(h, color=data.y)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "看起来相当好，尽管模型的权重是完全随机初始化，且我们还没有进行任何训练，同种颜色的结点（同个团体）在嵌入空间中还是紧密地聚合在一起了。我们可以从中得出结论：GNN可以让图中相近结点的表征也变得相似。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nDtJ9Zjw9I_Y"
      },
      "source": [
        "### 在Karate Club数据集上训练网络\n",
        "\n",
        "既然我们模型已经参数化且可微，我们可以加一些标签，训练模型并看看嵌入如何变化。此处我们将使用半监督学习或直推式学习的方法：允许使用输入的所有图数据，但是只用给出的标签。\n",
        "\n",
        "训练我们的模型和训练其他PyTorch模型类似。除了定义网络结构外，我们还要定义一个损失函数（此处使用[`CrossEntropyLoss`](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html)）并初始化一个优化器（此处使用[`Adam`](https://pytorch.org/docs/stable/optim.html?highlight=adam#torch.optim.Adam)）。\n",
        "\n",
        "然后，我们进行了多轮前播反播优化。\n",
        "\n",
        "如果你熟悉PyTorch，你应该很熟悉这个过程。如果不熟悉，PyTorch文档可以给你介绍[如何用PyTorch训练一个神经网络](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#define-a-loss-function-and-optimizer)。\n",
        "\n",
        "注意，我们的半监督学习是通过下面这行代码实现的：\n",
        "\n",
        "```\n",
        "loss = criterion(out[data.train_mask], data.y[data.train_mask])\n",
        "```\n",
        "\n",
        "即我们虽然计算了所有结点的嵌入，但是只使用训练结点（数据集中标注的结点）来计算损失。这里是通过用`train_mask`来过滤分类结果`out`以及ground-truth`data.y`来实现的。\n",
        "\n",
        "让我们开始训练来看看节点嵌入如何随着训练演变。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FI3DETGi9ND6"
      },
      "outputs": [],
      "source": [
        "import time\n",
        "\n",
        "model = GCN()\n",
        "criterion = torch.nn.CrossEntropyLoss()  # Define loss criterion.\n",
        "optimizer = torch.optim.Adam(model.parameters(), lr=0.01)  # Define optimizer.\n",
        "\n",
        "def train(data):\n",
        "    optimizer.zero_grad()  # Clear gradients.\n",
        "    out, h = model(data.x, data.edge_index)  # Perform a single forward pass.\n",
        "    loss = criterion(out[data.train_mask], data.y[data.train_mask])  # Compute the loss solely based on the training nodes.\n",
        "    loss.backward()  # Derive gradients.\n",
        "    optimizer.step()  # Update parameters based on gradients.\n",
        "    return loss, h\n",
        "\n",
        "for epoch in range(401):\n",
        "    loss, h = train(data)\n",
        "    # Visualize the node embeddings every 10 epochs\n",
        "    if epoch % 10 == 0:\n",
        "        visualize(h, color=data.y, epoch=epoch, loss=loss)\n",
        "        time.sleep(0.3)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F2B3X6tf9YpS"
      },
      "source": [
        "如你所见，3层GCN模型可以将结点线性分类，且大部分结点都分类正确。\n",
        "\n",
        "除此之外，我们只用了几行代码就实现以上，感谢PyTorch Geometric库帮我们处理数据以及实现GNN。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OmqyWVNObNcK"
      },
      "source": [
        "### 文档\n",
        "\n",
        "你可以阅读[PyG文档](https://pytorch-geometric.readthedocs.io/en/latest/)来了解更多。"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "machine_shape": "hm",
      "name": "CS224W - Colab 0.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.9.12 ('cs224w')",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.12"
    },
    "vscode": {
      "interpreter": {
        "hash": "3d401f214986802b882e8a583f41ee2435f6c7e7677ce5b0132581f351a1fd17"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}