Package_Test.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "view-in-github"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/kshitijrajsharma/fAIr-utilities/blob/master/Package_Test.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "RkOrDpW7-lwb"
      },
      "source": [
        "#[fAIr](https://github.com/hotosm/fAIr): AI-assisted  Mapping Developer Setup\n",
        "\n",
        "### fAIr is an open AI-assisted mapping service developed by the Humanitarian OpenStreetMap Team (HOT) that aims to improve the efficiency and accuracy of mapping efforts for humanitarian purposes. The service uses AI models, specifically computer vision techniques, to detect objects such as buildings, roads, waterways, and trees from satellite and UAV imagery.\n",
        "\n",
        "\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "-xQtyOpe_G8q"
      },
      "source": [
        "Before you Continue Make sure you have docker container running with jupyter notebook and You connected your collab with your local runtime, Learn more https://github.com/hotosm/fAIr-utilities"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "DIJeAZaA-vKe"
      },
      "source": [
        "## Setup fAIr Basemodel\n",
        "1. Currently [ramp](https://rampml.global/) is supported  , Clone Ramp Scripts"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hwMuPLgj-ooQ",
        "outputId": "9b10c7c5-c56c-4bac-abf4-916506c9c186"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Cloning into 'ramp-code'...\n",
            "remote: Enumerating objects: 762, done.\u001b[K\n",
            "remote: Counting objects: 100% (762/762), done.\u001b[K\n",
            "remote: Compressing objects: 100% (572/572), done.\u001b[K\n",
            "remote: Total 762 (delta 172), reused 750 (delta 163), pack-reused 0\u001b[K\n",
            "Receiving objects: 100% (762/762), 32.86 MiB | 5.94 MiB/s, done.\n",
            "Resolving deltas: 100% (172/172), done.\n"
          ]
        }
      ],
      "source": [
        "import os\n",
        "\n",
        "if not os.path.exists('ramp-code'):\n",
        "    !git clone https://github.com/kshitijrajsharma/ramp-code-fAIr.git ramp-code\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "oC7VbM97_Tnh"
      },
      "source": [
        "2. Download Ramp Basemodel checkpoint"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9C3W-VC7-y1i",
        "outputId": "41c39f54-31fe-4195-bae8-d8846fcf08eb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting gdown\n",
            "  Downloading gdown-4.7.1-py3-none-any.whl (15 kB)\n",
            "Collecting filelock\n",
            "  Downloading filelock-3.12.2-py3-none-any.whl (10 kB)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.8/dist-packages (from gdown) (4.62.3)\n",
            "Requirement already satisfied: requests[socks] in /usr/local/lib/python3.8/dist-packages (from gdown) (2.27.1)\n",
            "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.8/dist-packages (from gdown) (4.11.1)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.8/dist-packages (from gdown) (1.16.0)\n",
            "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.8/dist-packages (from beautifulsoup4->gdown) (2.3.2.post1)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests[socks]->gdown) (2021.10.8)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests[socks]->gdown) (1.26.7)\n",
            "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.8/dist-packages (from requests[socks]->gdown) (2.0.4)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests[socks]->gdown) (3.3)\n",
            "Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /usr/local/lib/python3.8/dist-packages (from requests[socks]->gdown) (1.7.1)\n",
            "Installing collected packages: filelock, gdown\n",
            "Successfully installed filelock-3.12.2 gdown-4.7.1\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n",
            "\u001b[33mWARNING: You are using pip version 21.2.4; however, version 23.1.2 is available.\n",
            "You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.\u001b[0m\n",
            "Downloading...\n",
            "From (uriginal): https://drive.google.com/uc?id=1wvJhkiOrSlHmmvJ0avkAdu9sslFf5_I0\n",
            "From (redirected): https://drive.google.com/uc?id=1wvJhkiOrSlHmmvJ0avkAdu9sslFf5_I0&confirm=t&uuid=d206ac41-c51b-4da3-b430-3ffbf8734b28\n",
            "To: /tf/checkpoint.tf.zip\n",
            "100%|████████████████████████████████████████| 113M/113M [00:16<00:00, 6.76MB/s]\n",
            "Archive:  checkpoint.tf.zip\n",
            "   creating: ramp-code/ramp/checkpoint.tf/\n",
            "  inflating: ramp-code/ramp/__MACOSX/._checkpoint.tf  \n",
            "  inflating: ramp-code/ramp/checkpoint.tf/.DS_Store  \n",
            "  inflating: ramp-code/ramp/__MACOSX/checkpoint.tf/._.DS_Store  \n",
            "  inflating: ramp-code/ramp/checkpoint.tf/keras_metadata.pb  \n",
            "  inflating: ramp-code/ramp/checkpoint.tf/README.md  \n",
            "   creating: ramp-code/ramp/checkpoint.tf/variables/\n",
            "  inflating: ramp-code/ramp/checkpoint.tf/desktop.ini  \n",
            "  inflating: ramp-code/ramp/checkpoint.tf/saved_model.pb  \n",
            "  inflating: ramp-code/ramp/checkpoint.tf/variables/variables.data-00000-of-00001  \n",
            "  inflating: ramp-code/ramp/checkpoint.tf/variables/variables.index  \n",
            "  inflating: ramp-code/ramp/checkpoint.tf/variables/desktop.ini  \n"
          ]
        }
      ],
      "source": [
        "if not os.path.exists('ramp-code/checkpoint.tf'):\n",
        "  !pip install gdown\n",
        "  !gdown --fuzzy https://drive.google.com/file/d/1YQsY61S_rGfJ_f6kLQq4ouYE2l3iRe1k/view?usp=sharing\n",
        "  !unzip checkpoint.tf.zip -d ramp-code/ramp"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "br6vfZU-dBkR"
      },
      "source": [
        "## Set your Env variable\n",
        "Specify ramp home , Where you have it Installed , Remember ramp home is parent dir where ramp-code is"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fwSUqY31_n3W",
        "outputId": "d7df0d51-ceef-4af3-f34f-98348a59ff51"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "/tf\n"
          ]
        }
      ],
      "source": [
        "import os\n",
        "os.environ.update(os.environ)\n",
        "# Add a new environment variable to the operating system for ramp home\n",
        "os.environ[\"RAMP_HOME\"] = os.path.join(os.getcwd())\n",
        "print(os.environ[\"RAMP_HOME\"])"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "Q3VgJPrbuy1I"
      },
      "source": [
        "# Prepare your Input Data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1Xeuyx3WuseX",
        "outputId": "a1e084ec-8a53-4bd1-bda6-60f3717ac99b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Segmentation Models: using `keras` framework.\n"
          ]
        }
      ],
      "source": [
        "import ramp.utils\n",
        "import hot_fair_utilities\n",
        "base_path = f\"{os.getcwd()}/ramp-data/sample_2\""
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "p8KpvGw0vX_A"
      },
      "source": [
        "## Preprocess input\n",
        "Plain tile img 2 Georefrenced raster tif"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EiojXUfXSkkA",
        "outputId": "83b3af51-97c1-4370-b019-563032094e76"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Georeferencing for input: 100%|██████████| 147/147 [00:01<00:00, 73.55it/s]\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:   0%|          | 0/147 [00:00<?, ?it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:   1%|▏         | 2/147 [00:00<00:08, 16.48it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:   3%|▎         | 4/147 [00:00<00:08, 16.73it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:   4%|▍         | 6/147 [00:00<00:08, 16.46it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:   5%|▌         | 8/147 [00:00<00:08, 16.64it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:   7%|▋         | 10/147 [00:00<00:08, 16.43it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:   8%|▊         | 12/147 [00:00<00:08, 16.15it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  10%|▉         | 14/147 [00:00<00:08, 16.27it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  11%|█         | 16/147 [00:00<00:07, 16.57it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  12%|█▏        | 18/147 [00:01<00:08, 15.37it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  14%|█▎        | 20/147 [00:01<00:08, 14.90it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  15%|█▍        | 22/147 [00:01<00:08, 14.54it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  16%|█▋        | 24/147 [00:01<00:08, 14.88it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  18%|█▊        | 26/147 [00:01<00:07, 15.59it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  19%|█▉        | 28/147 [00:01<00:07, 15.99it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  20%|██        | 30/147 [00:01<00:07, 16.01it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  22%|██▏       | 32/147 [00:02<00:07, 15.94it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  23%|██▎       | 34/147 [00:02<00:07, 16.03it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  24%|██▍       | 36/147 [00:02<00:06, 16.00it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  26%|██▌       | 38/147 [00:02<00:06, 15.96it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  27%|██▋       | 40/147 [00:02<00:06, 15.91it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  29%|██▊       | 42/147 [00:02<00:06, 15.25it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  30%|██▉       | 44/147 [00:02<00:06, 15.47it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  31%|███▏      | 46/147 [00:02<00:06, 15.57it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  33%|███▎      | 48/147 [00:03<00:06, 16.04it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  34%|███▍      | 50/147 [00:03<00:05, 16.29it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  35%|███▌      | 52/147 [00:03<00:05, 16.47it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  37%|███▋      | 54/147 [00:03<00:05, 16.59it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  38%|███▊      | 56/147 [00:03<00:05, 16.83it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  39%|███▉      | 58/147 [00:03<00:05, 16.53it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  41%|████      | 60/147 [00:03<00:05, 16.86it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  42%|████▏     | 62/147 [00:03<00:05, 16.91it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  44%|████▎     | 64/147 [00:03<00:04, 17.08it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  45%|████▍     | 66/147 [00:04<00:04, 17.07it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  46%|████▋     | 68/147 [00:04<00:04, 16.68it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  48%|████▊     | 70/147 [00:04<00:04, 16.78it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  49%|████▉     | 72/147 [00:04<00:04, 16.95it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  50%|█████     | 74/147 [00:04<00:04, 16.60it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  52%|█████▏    | 76/147 [00:04<00:04, 16.38it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  53%|█████▎    | 78/147 [00:04<00:04, 16.56it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  54%|█████▍    | 80/147 [00:04<00:04, 16.71it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  56%|█████▌    | 82/147 [00:05<00:03, 16.41it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  57%|█████▋    | 84/147 [00:05<00:03, 16.37it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  59%|█████▊    | 86/147 [00:05<00:03, 16.40it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  60%|█████▉    | 88/147 [00:05<00:03, 16.83it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  61%|██████    | 90/147 [00:05<00:03, 17.22it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  63%|██████▎   | 92/147 [00:05<00:03, 17.61it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  64%|██████▍   | 94/147 [00:05<00:03, 17.34it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  65%|██████▌   | 96/147 [00:05<00:02, 17.21it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  67%|██████▋   | 98/147 [00:06<00:02, 16.75it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  68%|██████▊   | 100/147 [00:06<00:02, 17.06it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  69%|██████▉   | 102/147 [00:06<00:02, 16.14it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  71%|███████   | 104/147 [00:06<00:02, 14.80it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  72%|███████▏  | 106/147 [00:06<00:02, 14.87it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  73%|███████▎  | 108/147 [00:06<00:02, 15.39it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  75%|███████▍  | 110/147 [00:06<00:02, 15.51it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  76%|███████▌  | 112/147 [00:06<00:02, 16.12it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  78%|███████▊  | 114/147 [00:07<00:02, 16.12it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  79%|███████▉  | 116/147 [00:07<00:01, 16.29it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  80%|████████  | 118/147 [00:07<00:01, 16.76it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  82%|████████▏ | 120/147 [00:07<00:01, 16.88it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  83%|████████▎ | 122/147 [00:07<00:01, 16.97it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  84%|████████▍ | 124/147 [00:07<00:01, 16.42it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  86%|████████▌ | 126/147 [00:07<00:01, 16.26it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  87%|████████▋ | 128/147 [00:07<00:01, 15.32it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  88%|████████▊ | 130/147 [00:08<00:01, 15.95it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  90%|████████▉ | 132/147 [00:08<00:00, 16.36it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  91%|█████████ | 134/147 [00:08<00:00, 16.56it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  93%|█████████▎| 136/147 [00:08<00:00, 16.71it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  94%|█████████▍| 138/147 [00:08<00:00, 16.89it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  95%|█████████▌| 140/147 [00:08<00:00, 16.82it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  97%|█████████▋| 142/147 [00:08<00:00, 17.24it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  98%|█████████▊| 144/147 [00:08<00:00, 17.13it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input:  99%|█████████▉| 146/147 [00:08<00:00, 16.98it/s]/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Clipping labels for input: 100%|██████████| 147/147 [00:09<00:00, 16.31it/s]\n"
          ]
        }
      ],
      "source": [
        "from hot_fair_utilities import preprocess\n",
        "model_input_image_path = f\"{base_path}/input\"\n",
        "preprocess_output=f\"/{base_path}/preprocessed\"\n",
        "preprocess(\n",
        "            input_path = model_input_image_path,\n",
        "            output_path = preprocess_output,\n",
        "            rasterize=True,\n",
        "            rasterize_options=[\"binary\"],\n",
        "            georeference_images=True,\n",
        "        )"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "7tTPCK7Svn9a"
      },
      "source": [
        "# Train Model\n",
        "Using preprocessed data run fine tuning or training itself"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8JV7LitOvmDD",
        "outputId": "64e980cb-1fa7-4043-ace1-63475fb37e58"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Starting to prepare data for training\n",
            "Data is ready for training\n",
            "Metric constructor function: get_sparse_categorical_accuracy_fn\n",
            "Model: importing saved model /tf/ramp-code/ramp/checkpoint.tf\n",
            "<keras.engine.functional.Functional object at 0x7f5f6bf5b580>\n",
            "Starting Training with 2 epochs , 2 batch size , 62 steps per epoch , 11 validation steps......\n",
            "Epoch 1/2\n",
            "62/62 [==============================] - ETA: 0s - loss: 0.6998 - sparse_categorical_accuracy: 0.8185"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 92). These functions will not be directly callable after loading.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: /tf/ramp-data/sample_2/train/model-checkpts/20230621-071045/model_20230621-071045_001_0.862.tf/assets\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: /tf/ramp-data/sample_2/train/model-checkpts/20230621-071045/model_20230621-071045_001_0.862.tf/assets\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 1s 1s/step\n",
            "62/62 [==============================] - 64s 719ms/step - loss: 0.6998 - sparse_categorical_accuracy: 0.8185 - val_loss: 0.3552 - val_sparse_categorical_accuracy: 0.8619\n",
            "Epoch 2/2\n",
            "62/62 [==============================] - ETA: 0s - loss: 0.3742 - sparse_categorical_accuracy: 0.8578"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 92). These functions will not be directly callable after loading.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: /tf/ramp-data/sample_2/train/model-checkpts/20230621-071045/model_20230621-071045_002_0.873.tf/assets\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: /tf/ramp-data/sample_2/train/model-checkpts/20230621-071045/model_20230621-071045_002_0.873.tf/assets\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 43ms/step\n",
            "62/62 [==============================] - 40s 599ms/step - loss: 0.3742 - sparse_categorical_accuracy: 0.8578 - val_loss: 0.2700 - val_sparse_categorical_accuracy: 0.8727\n",
            "Training Finished , Time taken to train : 104.51257558499856 seconds\n",
            "Generating graphs ....\n",
            "Graph generated at : /tf/ramp-data/sample_2/train/graphs\n",
            "extracting highest accuracy model\n",
            "/tf/ramp-data/sample_2/train/model-checkpts/20230621-071045\n",
            "model_20230621-071045_002_0.873.tf\n"
          ]
        }
      ],
      "source": [
        "from hot_fair_utilities import train\n",
        "train_output = f\"{base_path}/train\"\n",
        "final_accuracy, final_model_path = train(\n",
        "    input_path=preprocess_output,\n",
        "    output_path=train_output,\n",
        "    epoch_size=2,\n",
        "    batch_size=2,\n",
        "    model=\"ramp\",\n",
        "    model_home=os.environ[\"RAMP_HOME\"],\n",
        ")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "jPWr38uUvzDX"
      },
      "source": [
        "# Model Output : Checkpoints\n",
        "Visulize your training accuracy and newly created training checkpoint"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tufyLrt_v5dU",
        "outputId": "821b1978-91f2-46ac-ebbf-6477b0d4f6b1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "87.3 /tf/ramp-data/sample_2/train/model-checkpts/20230621-071045/model_20230621-071045_002_0.873.tf\n"
          ]
        }
      ],
      "source": [
        "print(final_accuracy,final_model_path)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "XLIk0VCQv8X9"
      },
      "source": [
        "# Inference\n",
        "Run predictions on newly created local model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xgc3fpaGv7Cb",
        "outputId": "16a78f2d-964a-4721-f2e0-454e216b99a4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Using : /tf/ramp-data/sample_2/train/model-checkpts/20230621-071045/model_20230621-071045_002_0.873.tf\n",
            "It took 12 sec to load model\n",
            "1/1 [==============================] - 1s 1s/step\n",
            "It took 2 sec to predict with 0.5 Confidence Threshold\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Georeferencing for output: 100%|██████████| 2/2 [00:00<00:00, 63.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "It took 0 sec to georeference\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        }
      ],
      "source": [
        "from hot_fair_utilities import predict\n",
        "prediction_output = f\"{base_path}/prediction/output\"\n",
        "predict(\n",
        "    checkpoint_path=final_model_path,\n",
        "    input_path=f\"{base_path}/prediction/input\",\n",
        "    prediction_path=prediction_output,\n",
        ")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "2U2t7sZgwL_n"
      },
      "source": [
        "# Post process Predictions\n",
        "Since inference will give the detected feature mask , Convert them to Geojson"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lzhsaLIqwTiA",
        "outputId": "88d73119-d494-43e9-84b2-1d17f91f2774"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 2/2 [00:00<00:00, 30.18mask/s]\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n",
            "Building graph: 100%|██████████| 85/85 [00:07<00:00, 11.11shapes/s]\n",
            "Merging components: 100%|██████████| 77/77 [00:14<00:00,  5.33component/s]\n",
            "/usr/local/lib/python3.8/dist-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.\n",
            "  pd.Int64Index,\n"
          ]
        }
      ],
      "source": [
        "from hot_fair_utilities import polygonize\n",
        "geojson_output= f\"{prediction_output}/prediction.geojson\"\n",
        "polygonize(\n",
        "    input_path=prediction_output,\n",
        "    output_path=geojson_output,\n",
        "    remove_inputs = True,\n",
        ")"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "authorship_tag": "ABX9TyM7Fo4AAPK9QVnkNKsVwBfQ",
      "include_colab_link": true,
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}