performance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Performance comparision between FastAssert and the OpenAI API\n",
    "\n",
    "In this notebook, we are going to quantify the gains in term of latency of using FastAssert to generate structured JSON outputs compared to OpenAI function calling. We will also check that we keep the same level of quality in term of JSON structure and JSON content.\n",
    "\n",
    "Cost are not covered as they depend on the setup to run FastAssert, but one can agree running FastAssert is cheaper than the OpenAI API by several orders of magnitude\n",
    "\n",
    "For the purpose of this demonstration, we will use OpenAI's own [function calling example](https://platform.openai.com/docs/api-reference/chat/create) (select `Function`on the right) : Text a user prompt asking for the current weather in a city, respond with a structured output containing the name of the city, and the guessed temperature unit in this zone."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This demo will be runned on 14 examples\n",
      "Prompts are variations of this one: What's the weather like in Paris today? Temperature unit can only be celsius or fahrenheit. For example, a valid response is {\\\"location\\\": \\\"Paris\\\",\\\"unit\\\":\\\"celsius\\\"}. Respond with the city name and the temperature unit used in this zone in this format.\n"
     ]
    }
   ],
   "source": [
    "# The structure we are expecting\n",
    "json_obj = {\n",
    "  \"type\": \"object\",\n",
    "  \"properties\": {\n",
    "    \"location\": {\n",
    "      \"type\": \"string\",\n",
    "      \"description\": \"The city and state, e.g., San Francisco, CA\"\n",
    "    },\n",
    "    \"unit\": {\n",
    "      \"type\": \"string\",\n",
    "      \"enum\": [\"celsius\", \"fahrenheit\"],\n",
    "      \"description\": \"The temperature unit\"\n",
    "    }\n",
    "  },\n",
    "  \"required\": [\"location\", \"unit\"],\n",
    "  \"additionalProperties\": False\n",
    "}\n",
    "\n",
    "# A list of inputs to test the project, that will be formated in text prompts before being passed to the constrained LLMs\n",
    "# For notebook visibility purposes, we put them in a compact format\n",
    "inputs = [{\"location\": \"Paris\", \"unit\": \"celsius\"},{\"location\": \"Toronto\", \"unit\": \"fahrenheit\"}, {\"location\": \"Sydney\", \"unit\": \"celsius\"}, {\"location\": \"Berlin\", \"unit\": \"celsius\"}, {\"location\": \"Moscow\", \"unit\": \"celsius\"}, {\"location\": \"Mumbai\", \"unit\": \"celsius\"}, {\"location\": \"New York, NY\", \"unit\": \"fahrenheit\"}, {\"location\": \"London\", \"unit\": \"celsius\"}, {\"location\": \"Tokyo\", \"unit\": \"celsius\"}, {\"location\": \"Cairo\", \"unit\": \"celsius\"}, {\"location\": \"Los Angeles\", \"unit\": \"fahrenheit\"}, {\"location\": \"Beijing\", \"unit\": \"celsius\"}, {\"location\": \"Cape Town\", \"unit\": \"celsius\"}, {\"location\": \"Mexico City\", \"unit\": \"celsius\"}]\n",
    "print(f\"This demo will be runned on {len(inputs)} examples\")\n",
    "\n",
    "# Build the prompts using these inputs\n",
    "# Prepare one example for the model\n",
    "example = \"For example, a valid response is \" + '{\\\\\"location\\\\\": \\\\\"Paris\\\\\",\\\\\"unit\\\\\":\\\\\"celsius\\\\\"}.'\n",
    "prompts = []\n",
    "for input in inputs:\n",
    "    prompts.append(f\"What's the weather like in {input.get('location')} today? Temperature unit can only be celsius or fahrenheit. \" + example + \" Respond with the city name and the temperature unit used in this zone in this format.\")\n",
    "\n",
    "print(f\"Prompts are variations of this one: {prompts[0]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation and Setup\n",
    "\n",
    "Make sure you are using the right kernel (`notebookenv`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: requests in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (2.31.0)\n",
      "Requirement already satisfied: openai in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (1.12.0)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from requests) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from requests) (3.6)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from requests) (2.2.0)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from requests) (2024.2.2)\n",
      "Requirement already satisfied: anyio<5,>=3.5.0 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from openai) (4.2.0)\n",
      "Requirement already satisfied: distro<2,>=1.7.0 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from openai) (1.9.0)\n",
      "Requirement already satisfied: httpx<1,>=0.23.0 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from openai) (0.26.0)\n",
      "Requirement already satisfied: pydantic<3,>=1.9.0 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from openai) (2.6.1)\n",
      "Requirement already satisfied: sniffio in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from openai) (1.3.0)\n",
      "Requirement already satisfied: tqdm>4 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from openai) (4.66.2)\n",
      "Requirement already satisfied: typing-extensions<5,>=4.7 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from openai) (4.9.0)\n",
      "Requirement already satisfied: httpcore==1.* in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from httpx<1,>=0.23.0->openai) (1.0.3)\n",
      "Requirement already satisfied: h11<0.15,>=0.13 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai) (0.14.0)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from pydantic<3,>=1.9.0->openai) (0.6.0)\n",
      "Requirement already satisfied: pydantic-core==2.16.2 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from pydantic<3,>=1.9.0->openai) (2.16.2)\n"
     ]
    }
   ],
   "source": [
    "!pip install requests openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Setup the OpenAI client\n",
    "from openai import OpenAI\n",
    "\n",
    "client = OpenAI(\n",
    "    # This is the default and can be omitted\n",
    "    api_key=os.environ.get(\"OPENAI_API_KEY\"),\n",
    ")\n",
    "\n",
    "# Create the function to generate structured output using OpenAI\n",
    "def openai_constrained_generation(prompt: str, json_schema, model_name=\"gpt-3.5-turbo\"):\n",
    "\n",
    "    tools = [\n",
    "      {\n",
    "        \"type\": \"function\",\n",
    "        \"function\": {\n",
    "          \"name\": \"get_current_weather\",\n",
    "          \"description\": \"Get the current weather in a given location\",\n",
    "          \"parameters\": json_obj,\n",
    "        }\n",
    "      }\n",
    "    ]\n",
    "    messages = [{\"role\": \"user\", \"content\": prompt}]\n",
    "    completion = client.chat.completions.create(\n",
    "      model=model_name,\n",
    "      messages=messages,\n",
    "      tools=tools,\n",
    "      tool_choice={\"type\": \"function\", \"function\": {\"name\": \"get_current_weather\"}},\n",
    "      temperature=0\n",
    "    )\n",
    "\n",
    "    return completion.choices[0].message.tool_calls[0].function.arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'{\\n  \"location\": \"Boston, MA\",\\n  \"unit\": \"celsius\"\\n}'"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's try it out\n",
    "openai_constrained_generation(\"What's the weather like in Boston today? Respond with the city name and the temperature unit used in this zone. Temperature unit can only be celsius or fahrenheit.\" + example, json_obj)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect and test our FastAssert server\n",
    "Make sure the docker container is running and can serve requests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Server is up and running!\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import time\n",
    "\n",
    "BASE_URL = \"http://127.0.0.1:8000\"\n",
    "\n",
    "# Ping the server\n",
    "ping_response = requests.get(f\"{BASE_URL}/health\")\n",
    "\n",
    "# Check if the request was successful\n",
    "if ping_response.status_code == 200:\n",
    "    print(\"Server is up and running!\")\n",
    "else:\n",
    "    raise ValueError(f\"Failed to reach the server, status code: {ping_response.status_code}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "# Create the json LLM generation\n",
    "def constrained_generation(prompt: str, json_schema):\n",
    "\n",
    "    url = f\"{BASE_URL}/generate\"\n",
    "    payload = {\n",
    "        \"prompt\": prompt,\n",
    "        \"schema\": json_schema,\n",
    "        \"temperature\": 0\n",
    "    }\n",
    "\n",
    "    response = requests.post(url, json=payload)\n",
    "    parsed_response = json.loads(response.text)\n",
    "\n",
    "    prompt_and_llm_answer = parsed_response['text'][0]\n",
    "\n",
    "    # Remove the input text, as it is returned completed with the answer\n",
    "    llm_answer = prompt_and_llm_answer[len(prompt):]\n",
    "\n",
    "    return llm_answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'{\"location\": \"Boston\",\"unit\":\"fahrenheit\"}'"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's try our FastAssert server\n",
    "constrained_generation(\"What's the weather like in Boston today? Respond with the city name and the temperature unit used in this zone. Temperature unit can only be celsius or fahrenheit.\" + instruction, json_obj)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's benchmark the two methods\n",
    "\n",
    "We will run on the input list 3 constrained generation models :\n",
    "- OpenAI `gpt-3.5-turbo` with function call\n",
    "- OpenAI `gpt-4` with function call\n",
    "- our FastAssert server running a constrained `Mistral-7B-v0.2`\n",
    "\n",
    "### Running the models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "openai_gpt3_5_times = []\n",
    "openai_gpt4_times = []\n",
    "fastassert_times = []\n",
    "\n",
    "openai_gpt3_5_answers = []\n",
    "openai_gpt4_answers = []\n",
    "fastassert_answers = []\n",
    "\n",
    "# We use the exact same prompt for each model\n",
    "for prompt in prompts:\n",
    "    # Time OpenAI gpt-3.5-turbo\n",
    "    start_time = time.time()\n",
    "    answer = openai_constrained_generation(prompt, json_obj)\n",
    "    end_time = time.time()\n",
    "    openai_gpt3_5_answers.append(answer)\n",
    "    openai_gpt3_5_times.append(end_time - start_time)\n",
    "\n",
    "    # Time OpenAI gpt-4\n",
    "    start_time = time.time()\n",
    "    answer = openai_constrained_generation(prompt, json_obj, model_name=\"gpt-4\")\n",
    "    end_time = time.time()\n",
    "    openai_gpt4_answers.append(answer)\n",
    "    openai_gpt4_times.append(end_time - start_time)\n",
    "\n",
    "    # Time FastAssert\n",
    "    start_time = time.time()\n",
    "    answer = constrained_generation(prompt + instruction, json_obj)\n",
    "    end_time = time.time()\n",
    "    fastassert_answers.append(answer)\n",
    "    fastassert_times.append(end_time - start_time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "## DEBUG\n",
    "\n",
    "fastassert_times = []\n",
    "\n",
    "fastassert_answers = []\n",
    "\n",
    "# We use the exact same prompt for each model\n",
    "for prompt in prompts:\n",
    "\n",
    "    # Time FastAssert\n",
    "    start_time = time.time()\n",
    "    answer = constrained_generation(prompt + instruction, json_obj)\n",
    "    end_time = time.time()\n",
    "    fastassert_answers.append(answer)\n",
    "    fastassert_times.append(end_time - start_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Assessing the results\n",
    "\n",
    "#### Latency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting matplotlib\n",
      "  Downloading matplotlib-3.8.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)\n",
      "Collecting numpy\n",
      "  Downloading numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m61.0/61.0 kB\u001b[0m \u001b[31m5.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting contourpy>=1.0.1 (from matplotlib)\n",
      "  Downloading contourpy-1.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)\n",
      "Collecting cycler>=0.10 (from matplotlib)\n",
      "  Downloading cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)\n",
      "Collecting fonttools>=4.22.0 (from matplotlib)\n",
      "  Downloading fonttools-4.49.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (159 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m159.1/159.1 kB\u001b[0m \u001b[31m19.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting kiwisolver>=1.3.1 (from matplotlib)\n",
      "  Downloading kiwisolver-1.4.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.4 kB)\n",
      "Requirement already satisfied: packaging>=20.0 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from matplotlib) (23.2)\n",
      "Collecting pillow>=8 (from matplotlib)\n",
      "  Downloading pillow-10.2.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (9.7 kB)\n",
      "Collecting pyparsing>=2.3.1 (from matplotlib)\n",
      "  Downloading pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from matplotlib) (2.8.2)\n",
      "Requirement already satisfied: six>=1.5 in /opt/conda/envs/notebookenv/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)\n",
      "Downloading matplotlib-3.8.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.6/11.6 MB\u001b[0m \u001b[31m103.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m0:01\u001b[0m\n",
      "\u001b[?25hDownloading numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.3 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m18.3/18.3 MB\u001b[0m \u001b[31m117.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
      "\u001b[?25hDownloading contourpy-1.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (313 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m313.4/313.4 kB\u001b[0m \u001b[31m46.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading cycler-0.12.1-py3-none-any.whl (8.3 kB)\n",
      "Downloading fonttools-4.49.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.9 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.9/4.9 MB\u001b[0m \u001b[31m130.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m\n",
      "\u001b[?25hDownloading kiwisolver-1.4.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.4/1.4 MB\u001b[0m \u001b[31m111.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading pillow-10.2.0-cp311-cp311-manylinux_2_28_x86_64.whl (4.5 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.5/4.5 MB\u001b[0m \u001b[31m117.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m\n",
      "\u001b[?25hDownloading pyparsing-3.1.1-py3-none-any.whl (103 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m103.1/103.1 kB\u001b[0m \u001b[31m21.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hInstalling collected packages: pyparsing, pillow, numpy, kiwisolver, fonttools, cycler, contourpy, matplotlib\n",
      "Successfully installed contourpy-1.2.0 cycler-0.12.1 fonttools-4.49.0 kiwisolver-1.4.5 matplotlib-3.8.3 numpy-1.26.4 pillow-10.2.0 pyparsing-3.1.1\n"
     ]
    }
   ],
   "source": [
    "!pip install matplotlib numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Completion times:\n",
      "OpenAI GPT-3.5 Mean: 0.8534583875111171 s, Standard Deviation: 0.18086406864731974\n",
      "OpenAI GPT-4 Mean: 1.1603457927703857 s, Standard Deviation: 0.2482123538585998\n",
      "FastAssert Mean: 0.2983722516468593 s, Standard Deviation: 0.01182277109689749\n"
     ]
    }
   ],
   "source": [
    "import statistics\n",
    "\n",
    "# Assuming inputs are defined and the above loop code has been executed\n",
    "\n",
    "# Calculate mean and standard deviation for OpenAI GPT-3.5 times\n",
    "mean_gpt3_5 = statistics.mean(openai_gpt3_5_times)\n",
    "std_dev_gpt3_5 = statistics.stdev(openai_gpt3_5_times)\n",
    "\n",
    "# Calculate mean and standard deviation for OpenAI GPT-4 times\n",
    "mean_gpt4 = statistics.mean(openai_gpt4_times)\n",
    "std_dev_gpt4 = statistics.stdev(openai_gpt4_times)\n",
    "\n",
    "# Calculate mean and standard deviation for FastAssert times\n",
    "mean_fastassert = statistics.mean(fastassert_times)\n",
    "std_dev_fastassert = statistics.stdev(fastassert_times)\n",
    "\n",
    "# Print the results\n",
    "print(\"Completion times:\")\n",
    "print(f\"OpenAI GPT-3.5 Mean: {mean_gpt3_5} s, Standard Deviation: {std_dev_gpt3_5}\")\n",
    "print(f\"OpenAI GPT-4 Mean: {mean_gpt4} s, Standard Deviation: {std_dev_gpt4}\")\n",
    "print(f\"FastAssert Mean: {mean_fastassert} s, Standard Deviation: {std_dev_fastassert}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fc687a2e210>"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAGzCAYAAADT4Tb9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAABDwUlEQVR4nO3de1yUZf7/8fdwPglkykFF8ax4ArVY1L5qombkobTINJHUrLRMslYqT9mKWpgVptlqhy3TPOTuYz2m6ZrlrqXStm15PqWAmAmCCsrcvz/8MesEKqPgwO3r+XjMY2eu+7ru+3PPDNvb+77ueyyGYRgCAAAwCRdnFwAAAFCeCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDeAyVgsFk2ePLlc1zl06FCFh4eX6zpvVGWs6VqGDh0qPz8/Z5dR4W7ks+nSpYu6dOlSrvXg1kO4AUqxf/9+jRw5Ug0aNJCXl5f8/f3VsWNHvfnmmzp37pyzy6sQx48f1+TJk5Wenu7sUmwqY00AKj83ZxcAVDarVq3Sgw8+KE9PTw0ZMkQtW7ZUYWGhtm7dqueff14//vij5s+f7+wyy93x48c1ZcoUhYeHKzIy0m7Ze++9J6vVSk0AqgTCDXCZgwcP6uGHH1a9evX05ZdfKjQ01LZs1KhR2rdvn1atWuXECp3D3d3d2SWUUBlrAlA5cFoKuMzMmTOVl5enBQsW2AWbYo0aNdKYMWNsry9evKipU6eqYcOG8vT0VHh4uF588UUVFBTYjQsPD9d9992nzZs3q3379vL29larVq20efNmSdKKFSvUqlUreXl5qV27dtq1a5fd+OK5GgcOHFDPnj3l6+urWrVq6ZVXXpFhGNfcr2PHjumxxx5TcHCwPD091aJFCy1cuNC2fPPmzbrjjjskSYmJibJYLLJYLPrggw9s2//9HIr8/Hw999xzCgsLk6enp5o2barXX3+9RD0Wi0WjR4/WypUr1bJlS9v2165de9WaHa3p0KFDslgsev311zVnzhw1aNBAPj4+6tGjh44ePSrDMDR16lTVqVNH3t7e6tu3r06dOlViu2vWrNFdd90lX19fVatWTXFxcfrxxx/t+mRmZioxMVF16tSRp6enQkND1bdvXx06dOiq+1Tsap+jYRgKDw9X3759S4w7f/68AgICNHLkyKuuv/g9X7p0qSIiIuTt7a2YmBj98MMPkqR3331XjRo1kpeXl7p06VJq3UuXLlW7du3k7e2tGjVqaPDgwTp27FiJfsWfq5eXl1q2bKnPP/+81JqsVqtmz56tFi1ayMvLS8HBwRo5cqR+++23a71dgOMMADa1a9c2GjRoUOb+CQkJhiRjwIABxpw5c4whQ4YYkox+/frZ9atXr57RtGlTIzQ01Jg8ebLxxhtvGLVr1zb8/PyMjz/+2Khbt64xffp0Y/r06UZAQIDRqFEjo6ioyG47Xl5eRuPGjY1HH33USEtLM+677z5DkjFhwgS7bUkyJk2aZHudmZlp1KlTxwgLCzNeeeUVY+7cuUafPn0MScYbb7xh6/PKK68YkozHH3/c+Mtf/mL85S9/Mfbv32/bfr169WzrtFqtxt13321YLBZj+PDhRlpamtG7d29DkvHss8+WqKdNmzZGaGioMXXqVGP27NlGgwYNDB8fH+PkyZNXfG8drengwYOGJCMyMtKIiIgwZs2aZbz88suGh4eH8Yc//MF48cUXjQ4dOhhvvfWW8cwzzxgWi8VITEy02+ZHH31kWCwW45577jHefvttY8aMGUZ4eLgRGBhoHDx40NavQ4cORkBAgPHyyy8bf/7zn41p06YZXbt2Nf7xj39ccX8c+Rxfeuklw93d3fj111/txn/22WeGJGPLli1X3Y4ko3Xr1kZYWJjd96pu3bpGWlqaERERYaSmptren65du9qNf//99w1Jxh133GG88cYbxvjx4w1vb28jPDzc+O2332z91q1bZ7i4uBgtW7Y0Zs2aZbz00ktGQECA0aJFC7vPxjAMY/jw4Yabm5sxYsQIY968ecYf//hHw9fX17jjjjuMwsJCW7/OnTsbnTt3vur+AddCuAH+v5ycHEOS0bdv3zL1T09PNyQZw4cPt2sfN26cIcn48ssvbW316tUzJBnffPONrW3dunWGJMPb29s4fPiwrf3dd981JBmbNm2ytRWHqKefftrWZrVajbi4OMPDw8PIzs62tf8+3AwbNswIDQ0tESQefvhhIyAgwDh79qxhGIbx7bffGpKM999/v8S+/j5IrFy50pBkvPrqq3b9BgwYYFgsFmPfvn129Xh4eNi1ff/994Yk4+233y6xrcs5UlNxuKlZs6Zx+vRpW3tycrItYF24cMHWPnDgQMPDw8M4f/68YRiGcebMGSMwMNAYMWKE3XYyMzONgIAAW/tvv/1mSDJee+21q9ZemrJ+jrt37zYkGXPnzrUb36dPHyM8PNywWq1X3Y4kw9PT0y6QFX+vQkJCjNzcXFt78ftT3LewsNAICgoyWrZsaZw7d87W7+9//7shyZg4caKtLTIy0ggNDbV7v9evX29IsvtsvvrqK0OS8cknn9jVuXbt2hLthBuUB05LAf9fbm6uJKlatWpl6r969WpJUlJSkl37c889J0kl5uZEREQoJibG9jo6OlqSdPfdd6tu3bol2g8cOFBim6NHj7Y9Lz71UFhYqA0bNpRao2EYWr58uXr37i3DMHTy5Enbo2fPnsrJydHOnTvLtL+XW716tVxdXfXMM8/YtT/33HMyDENr1qyxa4+NjVXDhg1tr1u3bi1/f/9S9/FGPfjggwoICLC9Ln4/Bw8eLDc3N7v2wsJC26mWL774QqdPn9bAgQPt3idXV1dFR0dr06ZNkiRvb295eHho8+bN131K5VqfY5MmTRQdHa1PPvnE1u/UqVNas2aNBg0aJIvFcs1tdOvWze60XfH70L9/f7vv+O+/b999951OnDihp556Sl5eXrZ+cXFxatasme17nZGRofT0dCUkJNi93927d1dERIRdLUuXLlVAQIC6d+9u9962a9dOfn5+tvcWKC9MKAb+P39/f0nSmTNnytT/8OHDcnFxUaNGjezaQ0JCFBgYqMOHD9u1Xx5gJNn+gxAWFlZq++//w+ni4qIGDRrYtTVp0kSSrjjXIzs7W6dPn9b8+fOveIXXiRMnSm2/msOHD6tWrVolgmDz5s1tyy/3+32XpNtuu61C5ltc7/u8d+9eSZfCZmmKvx+enp6aMWOGnnvuOQUHB+sPf/iD7rvvPg0ZMkQhISHXrK+sn+OQIUM0evRoHT58WPXq1dPSpUt14cIFPfroo9fchnT970PxZ9e0adMS62zWrJm2bt1q169x48Yl+jVt2tQuNO/du1c5OTkKCgoqtdbr+Q4CV0O4Af4/f39/1apVS//5z38cGleWf0VLkqurq0PtRhkmCl9L8aXSgwcPVkJCQql9WrdufcPbuZaK3MeybutaNRS/V3/5y19KDSmXH/V59tln1bt3b61cuVLr1q3ThAkTlJKSoi+//FJRUVE3uguSpIcfflhjx47VJ598ohdffFEff/yx2rdvX2roKI0zvm9XYrVaFRQUZHck6nI1a9assG3j1kS4AS5z3333af78+dq2bZvdKaTS1KtXT1arVXv37rUdsZCkrKwsnT59WvXq1SvX2qxWqw4cOGD7V74k7dmzR5KueDfYmjVrqlq1aioqKlJsbOxV11/WkCZd2vcNGzbozJkzdkdvfv75Z9vy8uBITTeq+LRZUFDQNd+r4v7PPfecnnvuOe3du1eRkZFKTU3Vxx9/fNVxZf0cq1evrri4OH3yyScaNGiQvv76a82ePdvxHXNQ8We3e/fuEkexdu/ebVte/L/FR7x+3+9yDRs21IYNG9SxY0d5e3tXRNmAHebcAJd54YUX5Ovrq+HDhysrK6vE8v379+vNN9+UJN17772SVOI/OLNmzZJ0aY5CeUtLS7M9NwxDaWlpcnd3V7du3Urt7+rqqv79+2v58uWlHpHKzs62Pff19ZUknT59+pp13HvvvSoqKrKrR5LeeOMNWSwW9erVqyy7c02O1HSjevbsKX9/f02bNk0XLlwosbz4vTp79qzOnz9vt6xhw4aqVq1aiVsAXElZP8dHH31U//3vf/X888/L1dVVDz/8sKO75bD27dsrKChI8+bNs9ufNWvW6KeffrJ9r0NDQxUZGakPP/xQOTk5tn5ffPGF/vvf/9qt86GHHlJRUZGmTp1aYnsXL168KZ8vbi0cuQEu07BhQy1atEjx8fFq3ry53R2Kv/nmGy1dulRDhw6VJLVp00YJCQmaP3++Tp8+rc6dO2v79u368MMP1a9fP3Xt2rVca/Py8tLatWuVkJCg6OhorVmzRqtWrdKLL7541cP606dP16ZNmxQdHa0RI0YoIiJCp06d0s6dO7VhwwbbvV4aNmyowMBAzZs3T9WqVZOvr6+io6NVv379Euvs3bu3unbtqpdeekmHDh1SmzZttH79ev31r3/Vs88+azd5+EY4UtON8vf319y5c/Xoo4+qbdu2evjhh1WzZk0dOXJEq1atUseOHZWWlqY9e/aoW7dueuihhxQRESE3Nzd9/vnnysrKKlP4cORzjIuL0+23366lS5eqV69eV5yzUp7c3d01Y8YMJSYmqnPnzho4cKCysrL05ptvKjw8XGPHjrX1TUlJUVxcnDp16qTHHntMp06d0ttvv60WLVooLy/P1q9z584aOXKkUlJSlJ6erh49esjd3V179+7V0qVL9eabb2rAgAEVvm+4hTjtOi2gEtuzZ48xYsQIIzw83PDw8DCqVatmdOzY0Xj77bdtlw4bhmFcuHDBmDJlilG/fn3D3d3dCAsLM5KTk+36GMalS8Hj4uJKbEeSMWrUKLu24kuaL7/UOCEhwfD19TX2799v9OjRw/Dx8TGCg4ONSZMm2d0Pp3idl18KbhiGkZWVZYwaNcoICwsz3N3djZCQEKNbt27G/Pnz7fr99a9/NSIiIgw3Nze7S7B/f9m1YVy6dHrs2LFGrVq1DHd3d6Nx48bGa6+9VuIy5dL2sfg9SUhIKNH+e2WtqbT3zTAMY9OmTYYkY+nSpXbtxfdy+fbbb0v079mzpxEQEGB4eXkZDRs2NIYOHWp89913hmEYxsmTJ41Ro0YZzZo1M3x9fY2AgAAjOjra+Oyzz665L458jsWeeuopQ5KxaNGia66/WFm/V8X7W9r7s2TJEiMqKsrw9PQ0qlevbgwaNMj45ZdfSmxr+fLlRvPmzQ1PT08jIiLCWLFiRanfF8MwjPnz5xvt2rUzvL29jWrVqhmtWrUyXnjhBeP48eO2PlwKjvJgMYwKnEUGoFwMHTpUy5Yts/vXMG4NY8eO1YIFC5SZmSkfHx9nlwNUCcy5AYBK6vz58/r444/Vv39/gg3gAObcAEAlc+LECW3YsEHLli3Tr7/+avd7ZgCujXADAJXMf//7Xw0aNEhBQUF66623FBkZ6eySgCqFOTcAAMBUmHMDAABMhXADAABM5Zabc2O1WnX8+HFVq1btpt7aHQAAXD/DMHTmzBnVqlVLLi5XPzZzy4Wb48ePl/hVXAAAUDUcPXpUderUuWqfWy7cFP/I39GjR+Xv7+/kagAAQFnk5uYqLCzM7sd6r+SWCzfFp6L8/f0JNwAAVDFlmVLChGIAAGAqhBsAAGAqhBsAAGAqt9ycGwAAbpRhGLp48aKKioqcXYqpuLu7y9XV9YbXQ7gBAMABhYWFysjI0NmzZ51diulYLBbVqVNHfn5+N7Qewg0AAGVktVp18OBBubq6qlatWvLw8OCGsOXEMAxlZ2frl19+UePGjW/oCA7hBgCAMiosLJTValVYWJh8fHycXY7p1KxZU4cOHdKFCxduKNwwoRgAAAdd6/b/uD7ldRSMTwcAAJgK4QYAAJiKU+fcbNmyRa+99pp27NihjIwMff755+rXr1+Zxn799dfq3LmzWrZsqfT09AqtEwCAawkfv+qmbu/Q9Libur2qxKlHbvLz89WmTRvNmTPHoXGnT5/WkCFD1K1btwqqDAAAcxk6dKgsFoueeOKJEstGjRoli8WioUOH3vzCKoBTw02vXr306quv6v7773do3BNPPKFHHnlEMTExFVQZAADmExYWpsWLF+vcuXO2tvPnz2vRokWqW7euEysrX1XuUvD3339fBw4c0Mcff6xXX331mv0LCgpUUFBge52bm1uR5QG4CTIyMpSRkeHwuNDQUIWGhlZARUDV0LZtW+3fv18rVqzQoEGDJEkrVqxQ3bp1Vb9+fVs/q9WqGTNmaP78+crMzFSTJk00YcIEDRgwQJJUVFSkxx9/XF9++aUyMzNVt25dPfXUUxozZoxtHUOHDtXp06fVqVMnpaamqrCwUA8//LBmz54td3f3Ct3PKhVu9u7dq/Hjx+urr76Sm1vZSk9JSdGUKVMquDIAN9O77757XX/XkyZN0uTJk8u/IKAKeeyxx/T+++/bws3ChQuVmJiozZs32/qkpKTo448/1rx589S4cWNt2bJFgwcPVs2aNdW5c2dZrVbVqVNHS5cu1e23365vvvlGjz/+uEJDQ/XQQw/Z1rNp0yaFhoZq06ZN2rdvn+Lj4xUZGakRI0ZU6D5WmXBTVFSkRx55RFOmTFGTJk3KPC45OVlJSUm217m5uQoLC6uIEgHcJCNHjlSfPn3s2s6dO6dOnTpJkrZu3Spvb+8S4zhqA0iDBw9WcnKyDh8+LOnSBTqLFy+2hZuCggJNmzZNGzZssE3/aNCggbZu3ap3331XnTt3lru7u90/MOrXr69t27bps88+sws3t912m9LS0uTq6qpmzZopLi5OGzduJNwUO3PmjL777jvt2rVLo0ePlnTpsJlhGHJzc9P69et19913lxjn6ekpT0/Pm10ugApU2uml/Px82/PIyEj5+vre7LKAKqFmzZqKi4vTBx98IMMwFBcXpxo1atiW79u3T2fPnlX37t3txhUWFioqKsr2es6cOVq4cKGOHDmic+fOqbCwUJGRkXZjWrRoYXen4dDQUP3www8Vs2OXqTLhxt/fv8Qb8s477+jLL7/UsmXL7M4VAgCAK3vsscdsBwp+f8VyXl6eJGnVqlWqXbu23bLigwWLFy/WuHHjlJqaqpiYGFWrVk2vvfaa/vWvf9n1//3cGovFIqvVWq77Uhqnhpu8vDzt27fP9vrgwYNKT09X9erVVbduXSUnJ+vYsWP66KOP5OLiopYtW9qNDwoKkpeXV4l2AABwZffcc48KCwtlsVjUs2dPu2URERHy9PTUkSNH1Llz51LHf/311+rQoYOeeuopW9v+/fsrtGZHODXcfPfdd+ratavtdfHcmISEBH3wwQfKyMjQkSNHnFUeAACm5Orqqp9++sn2/HLVqlXTuHHjNHbsWFmtVnXq1Ek5OTn6+uuv5e/vr4SEBDVu3FgfffSR1q1bp/r16+svf/mLvv3220pzFsWp4aZLly4yDOOKyz/44IOrjp88eTJXPgAAKoWqdsdgf3//Ky6bOnWqatasqZSUFB04cECBgYFq27atXnzxRUmXJvXv2rVL8fHxslgsGjhwoJ566imtWbPmZpV/VRbjaunChHJzcxUQEKCcnJyrfrAAqpb8/Hz5+flJunTKmwnFqAjnz5/XwYMHVb9+fXl5eTm7HNO52vvryH+/+eFMAABgKoQbAABgKoQbAABgKoQbAABgKoQbAAAcdItdi3PTlNf7SrgBAKCMiu+4e/bsWSdXYk6FhYWSSt57x1FV5ucXAABwNldXVwUGBurEiROSJB8fH1ksFidXZQ5Wq1XZ2dny8fGRm9uNxRPCDQAADggJCZEkW8BB+XFxcVHdunVvODASbgAAcIDFYlFoaKiCgoJ04cIFZ5djKh4eHnJxufEZM4QbAACug6ur6w3PDUHFYEIxAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFcINAAAwFaeGmy1btqh3796qVauWLBaLVq5cedX+K1asUPfu3VWzZk35+/srJiZG69atuznFAgCAKsGp4SY/P19t2rTRnDlzytR/y5Yt6t69u1avXq0dO3aoa9eu6t27t3bt2lXBlQIAgKrCYhiG4ewiJMlisejzzz9Xv379HBrXokULxcfHa+LEiaUuLygoUEFBge11bm6uwsLClJOTI39//xspGUAlkp+fLz8/P0lSXl6efH19nVwRgPKUm5urgICAMv33u0rPubFarTpz5oyqV69+xT4pKSkKCAiwPcLCwm5ihQAA4Gar0uHm9ddfV15enh566KEr9klOTlZOTo7tcfTo0ZtYIQAAuNncnF3A9Vq0aJGmTJmiv/71rwoKCrpiP09PT3l6et7EygAAgDNVyXCzePFiDR8+XEuXLlVsbKyzywEAAJVIlTst9emnnyoxMVGffvqp4uLinF0OAACoZJx65CYvL0/79u2zvT548KDS09NVvXp11a1bV8nJyTp27Jg++ugjSZdORSUkJOjNN99UdHS0MjMzJUne3t4KCAhwyj4AAIDKxalHbr777jtFRUUpKipKkpSUlKSoqCjbZd0ZGRk6cuSIrf/8+fN18eJFjRo1SqGhobbHmDFjnFI/AACofCrNfW5uFkeukwdQdXCfG8Dcbpn73AAAAPwe4QYAAJhKlbwUHEDlFD5+ldO2bS08b3vefMJauXh4Oa0WSTo0nas5AWfhyA0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVwg0AADAVN2cXgOuTkZGhjIwMh8eFhoYqNDS0AioCAKByINxUUe+++66mTJni8LhJkyZp8uTJ5V8QAACVBOGmiho5cqT69Olj13bu3Dl16tRJkrR161Z5e3uXGMdRGwCA2RFuqqjSTi/l5+fbnkdGRsrX1/dmlwUAgNM5dULxli1b1Lt3b9WqVUsWi0UrV6685pjNmzerbdu28vT0VKNGjfTBBx9UeJ0AAKDqcGq4yc/PV5s2bTRnzpwy9T948KDi4uLUtWtXpaen69lnn9Xw4cO1bt26Cq4UAABUFU49LdWrVy/16tWrzP3nzZun+vXrKzU1VZLUvHlzbd26VW+88YZ69uxZUWUCAIAqpErd52bbtm2KjY21a+vZs6e2bdt2xTEFBQXKzc21ewAAAPOqUuEmMzNTwcHBdm3BwcHKzc3VuXPnSh2TkpKigIAA2yMsLOxmlAoAAJykSoWb65GcnKycnBzb4+jRo84uCQAAVKAqdSl4SEiIsrKy7NqysrLk7+9f6j1dJMnT01Oenp43ozwAAFAJVKkjNzExMdq4caNd2xdffKGYmBgnVQQAACobp4abvLw8paenKz09XdKlS73T09N15MgRSZdOKQ0ZMsTW/4knntCBAwf0wgsv6Oeff9Y777yjzz77TGPHjnVG+QAAoBJyarj57rvvFBUVpaioKElSUlKSoqKiNHHiREmXfhyyOOhIUv369bVq1Sp98cUXatOmjVJTU/XnP/+Zy8ABAICNU+fcdOnSRYZhXHF5aXcf7tKli3bt2lWBVQEAgKqsSs25AQAAuBbCDQAAMJUbDje5ublauXKlfvrpp/KoBwAA4IY4HG4eeughpaWlSZLOnTun9u3b66GHHlLr1q21fPnyci8QAADAEQ6Hmy1btuiuu+6SJH3++ecyDEOnT5/WW2+9pVdffbXcCwQAAHCEw+EmJydH1atXlyStXbtW/fv3l4+Pj+Li4rR3795yLxAAAMARDoebsLAwbdu2Tfn5+Vq7dq169OghSfrtt9/k5eVV7gUCAAA4wuH73Dz77LMaNGiQ/Pz8VK9ePXXp0kXSpdNVrVq1Ku/6AAAAHOJwuHnqqad055136ujRo+revbtcXC4d/GnQoAFzbgAAgNNd1x2K27dvr/bt29u1xcXFlUtBAAAAN6JM4SYpKanMK5w1a9Z1FwMAAHCjyhRufv9bTjt37tTFixfVtGlTSdKePXvk6uqqdu3alX+FAAAADihTuNm0aZPt+axZs1StWjV9+OGHuu222yRdulIqMTHRdv8bAKhIF/NOqSjvlF2bcaHQ9rww64As7h4lxrn6VZebX/UKrw+Aczk85yY1NVXr16+3BRtJuu222/Tqq6+qR48eeu6558q1QAD4vbz0Ncr5+tMrLs9a9EKp7QEdByqw06CKKgtAJeFwuMnNzVV2dnaJ9uzsbJ05c6ZcigKAq/GL7CXvRtEOj3PlqA1wS3A43Nx///1KTExUamqq7rzzTknSv/71Lz3//PN64IEHyr1AAPg9N04vAbgKh8PNvHnzNG7cOD3yyCO6cOHCpZW4uWnYsGF67bXXyr1AAAAARzgcbnx8fPTOO+/otdde0/79+yVJDRs2lK+vb7kXBwAA4KjruomfJPn6+qp169blWQsAAMANczjc5Ofna/r06dq4caNOnDghq9Vqt/zAgQPlVhwAAICjHA43w4cP1z/+8Q89+uijCg0NlcViqYi6AAAArovD4WbNmjVatWqVOnbsWBH1AAAA3BAXRwfcdtttql6dSzABAEDl5HC4mTp1qiZOnKizZ89WRD0AAAA35Lp+fmH//v0KDg5WeHi43N3d7Zbv3Lmz3IoDAABwlMPhpl+/fhVQBgAAQPlwONxMmjSpIuoAAAAoF9d9E78dO3bop59+kiS1aNFCUVFR5VYUAADA9XI43Jw4cUIPP/ywNm/erMDAQEnS6dOn1bVrVy1evFg1a9Ys7xoBAADKzOGrpZ5++mmdOXNGP/74o06dOqVTp07pP//5j3Jzc/XMM89URI0AAABl5vCRm7Vr12rDhg1q3ry5rS0iIkJz5sxRjx49yrW4qih8/CqnbdtaeN72vPmEtXLx8HJaLZJ0aHqcU7cPALg1OXzkxmq1lrj8W5Lc3d1L/M4UAADAzeZwuLn77rs1ZswYHT9+3NZ27NgxjR07Vt26dSvX4gAAABzlcLhJS0tTbm6uwsPD1bBhQzVs2FD169dXbm6u3n777YqoEQAAoMwcnnMTFhamnTt3asOGDfr5558lSc2bN1dsbGy5FwcAAOCo67rPjcViUffu3dW9e/fyrgcAAOCGOHxa6plnntFbb71Voj0tLU3PPvtsedQEAABw3RwON8uXL1fHjh1LtHfo0EHLli0rl6IAAACul8Ph5tdff1VAQECJdn9/f508ebJcigIAALheDoebRo0aae3atSXa16xZowYNGjhcwJw5cxQeHi4vLy9FR0dr+/btV+0/e/ZsNW3aVN7e3goLC9PYsWN1/vz5q44BAAC3DocnFCclJWn06NHKzs7W3XffLUnauHGjUlNTNXv2bIfWtWTJEiUlJWnevHmKjo7W7Nmz1bNnT+3evVtBQUEl+i9atEjjx4/XwoUL1aFDB+3Zs0dDhw6VxWLRrFmzHN0VAABgQg6Hm8cee0wFBQX605/+pKlTp0qSwsPDNXfuXA0ZMsShdc2aNUsjRoxQYmKiJGnevHlatWqVFi5cqPHjx5fo/80336hjx4565JFHbNsdOHCg/vWvfzm6GwAAwKQcPi0lSU8++aR++eUXZWVlKTc3VwcOHHA42BQWFmrHjh1298dxcXFRbGystm3bVuqYDh06aMeOHbZTVwcOHNDq1at17733XnE7BQUFys3NtXsAAADzuq5wc/HiRW3YsEErVqyQYRiSpOPHjysvL6/M6zh58qSKiooUHBxs1x4cHKzMzMxSxzzyyCN65ZVX1KlTJ7m7u6thw4bq0qWLXnzxxStuJyUlRQEBAbZHWFhYmWsEAABVj8Ph5vDhw2rVqpX69u2rUaNGKTs7W5I0Y8YMjRs3rtwLvNzmzZs1bdo0vfPOO9q5c6dWrFihVatW2U6PlSY5OVk5OTm2x9GjRyu0RgAA4FwOz7kZM2aM2rdvr++//1633367rf3+++/XiBEjyryeGjVqyNXVVVlZWXbtWVlZCgkJKXXMhAkT9Oijj2r48OGSpFatWik/P1+PP/64XnrpJbm4lMxqnp6e8vT0LHNdAACganP4yM1XX32ll19+WR4eHnbt4eHhOnbsWJnX4+HhoXbt2mnjxo22NqvVqo0bNyomJqbUMWfPni0RYFxdXSXJdnoMAADc2hw+cmO1WlVUVFSi/ZdfflG1atUcWldSUpISEhLUvn173XnnnZo9e7by8/NtV08NGTJEtWvXVkpKiiSpd+/emjVrlqKiohQdHa19+/ZpwoQJ6t27ty3kAACAW5vD4aZHjx6aPXu25s+fL+nSj2jm5eVp0qRJV71qqTTx8fHKzs7WxIkTlZmZqcjISK1du9Y2yfjIkSN2R2pefvllWSwWvfzyyzp27Jhq1qyp3r17609/+pOjuwEAAEzKYjh4PueXX35Rz549ZRiG9u7dq/bt22vv3r2qUaOGtmzZUurN9yqT3NxcBQQEKCcnR/7+/uW+/vDxq8p9nWVlLTyvo28MkCSFjV0mFw8vp9UiSYemxzl1+7j5nPn9r2z4/gPly5H/fjt85KZOnTr6/vvvtWTJEn3//ffKy8vTsGHDNGjQIHl7e1930QAAAOXB4XAjSW5ubho0aJAGDRpU3vUAAADcEIevlvrwww+1atX/Dj2/8MILCgwMVIcOHXT48OFyLQ4AAMBRDoebadOm2U4/bdu2TWlpaZo5c6Zq1KihsWPHlnuBAAAAjnD4tNTRo0fVqFEjSdLKlSs1YMAAPf744+rYsaO6dOlS3vUBAAA4xOEjN35+fvr1118lSevXr1f37t0lSV5eXjp37lz5VgcAAOAgh4/cdO/eXcOHD1dUVJT27Nlju7fNjz/+qPDw8PKuDwAAwCEOH7mZM2eOYmJilJ2dreXLl9t+X2rHjh0aOHBguRcIAADgCIeP3AQGBiotLa1E+5QpU8qlIAAAgBvh8JEbAACAyoxwAwAATIVwAwAATIVwAwAATIVwAwAATMXhcJOVlaVHH31UtWrVkpubm1xdXe0eAAAAzuTwpeBDhw7VkSNHNGHCBIWGhspisVREXQAAANfF4XCzdetWffXVV4qMjKyAcgAAAG6Mw6elwsLCZBhGRdQCAABwwxwON7Nnz9b48eN16NChCigHAADgxjh8Wio+Pl5nz55Vw4YN5ePjI3d3d7vlp06dKrfiAAAAHOVwuJk9e3YFlAEAAFA+HA43CQkJFVEHAABAuShTuMnNzZW/v7/t+dUU90PFuph3SkV59qcAjQuFtueFWQdkcfcoMc7Vr7rc/KpXeH0AADhLmcLNbbfdpoyMDAUFBSkwMLDUe9sYhiGLxaKioqJyLxIl5aWvUc7Xn15xedaiF0ptD+g4UIGdBlVUWQAAOF2Zws2XX36p6tUv/Wt/06ZNFVoQysYvspe8G0U7PM6VozYAAJMrU7jp3Llzqc/hPG6cXgIAoFT8cCYAADAVwg0AADAVwg0AADAVwg0AADAVh8PNuXPndPbsWdvrw4cPa/bs2Vq/fn25FgYAAHA9HA43ffv21UcffSRJOn36tKKjo5Wamqq+fftq7ty55V4gAACAIxwONzt37tRdd90lSVq2bJmCg4N1+PBhffTRR3rrrbfKvUAAAABHOBxuzp49q2rVqkmS1q9frwceeEAuLi76wx/+oMOHD5d7gQAAAI5wONw0atRIK1eu1NGjR7Vu3Tr16NFDknTixAl+VwoAADidw+Fm4sSJGjdunMLDw3XnnXcqJiZG0qWjOFFRUeVeIAAAgCPK9PMLlxswYIA6deqkjIwMtWnTxtberVs33X///eVaHAAAgKOu6z43ISEhqlatmr744gudO3dOknTHHXeoWbNm5VocAACAoxwON7/++qu6deumJk2a6N5771VGRoYkadiwYXruuefKvUAAAABHOBxuxo4dK3d3dx05ckQ+Pj629vj4eK1du7ZciwMAAHCUw+Fm/fr1mjFjhurUqWPX3rhx4+u6FHzOnDkKDw+Xl5eXoqOjtX379qv2P336tEaNGqXQ0FB5enqqSZMmWr16tcPbBQAA5uTwhOL8/Hy7IzbFTp06JU9PT4fWtWTJEiUlJWnevHmKjo7W7Nmz1bNnT+3evVtBQUEl+hcWFqp79+4KCgrSsmXLVLt2bR0+fFiBgYGO7gYAADAph4/c3HXXXbafX5Aki8Uiq9WqmTNnqmvXrg6ta9asWRoxYoQSExMVERGhefPmycfHRwsXLiy1/8KFC3Xq1CmtXLlSHTt2VHh4uDp37mx31dbvFRQUKDc31+4BAADMy+FwM3PmTM2fP1+9evVSYWGhXnjhBbVs2VJbtmzRjBkzyryewsJC7dixQ7Gxsf8rxsVFsbGx2rZtW6lj/va3vykmJkajRo1ScHCwWrZsqWnTpqmoqOiK20lJSVFAQIDtERYWVvadBQAAVY7D4aZly5bas2ePOnXqpL59+yo/P18PPPCAdu3apYYNG5Z5PSdPnlRRUZGCg4Pt2oODg5WZmVnqmAMHDmjZsmUqKirS6tWrNWHCBKWmpurVV1+94naSk5OVk5Njexw9erTMNQIAgKrH4Tk3khQQEKCXXnqpvGu5JqvVqqCgIM2fP1+urq5q166djh07ptdee02TJk0qdYynp6fDc4EAAEDVdV3h5vz58/r3v/+tEydOyGq12i3r06dPmdZRo0YNubq6Kisry649KytLISEhpY4JDQ2Vu7u7XF1dbW3NmzdXZmamCgsL5eHh4eCeAAAAs3E43Kxdu1ZDhgzRyZMnSyyzWCxXnf9yOQ8PD7Vr104bN25Uv379JF06MrNx40aNHj261DEdO3bUokWLZLVa5eJy6Yzanj17FBoaSrABAACSrmPOzdNPP60HH3xQGRkZslqtdo+yBptiSUlJeu+99/Thhx/qp59+0pNPPqn8/HwlJiZKkoYMGaLk5GRb/yeffFKnTp3SmDFjtGfPHq1atUrTpk3TqFGjHN0NAABgUg4fucnKylJSUlKJicDXIz4+XtnZ2Zo4caIyMzMVGRmptWvX2tZ95MgR2xEaSQoLC9O6des0duxYtW7dWrVr19aYMWP0xz/+8YZrAQAA5nBdvwq+efNmh66MuprRo0df8TTU5s2bS7TFxMTon//8Z7lsGwAAmI/D4SYtLU0PPvigvvrqK7Vq1Uru7u52y5955plyKw4AAMBRDoebTz/9VOvXr5eXl5c2b94si8ViW2axWAg3AADAqRwONy+99JKmTJmi8ePH282HAQAAqAwcTieFhYWKj48n2AAAgErJ4YSSkJCgJUuWVEQtAAAAN8zh01JFRUWaOXOm1q1bp9atW5eYUDxr1qxyKw4AAMBRDoebH374QVFRUZKk//znP3bLLp9cDAAA4AwOh5tNmzZVRB0AAADlglnBAADAVMp05OaBBx7QBx98IH9/fz3wwANX7btixYpyKQwAAOB6lCncBAQE2ObTBAQEVGhBAAAAN6JM4eb999/XK6+8onHjxun999+v6JoAAACuW5nn3EyZMkV5eXkVWQsAAMANK3O4MQyjIusAAAAoFw5dLcV9bAAAQGXn0H1umjRpcs2Ac+rUqRsqCAAA4EY4FG6mTJnC1VIAAKBScyjcPPzwwwoKCqqoWgAAAG5YmefcMN8GAABUBVwtBQAATKXMp6WsVmtF1gEAAFAu+OFMAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKoQbAABgKpUi3MyZM0fh4eHy8vJSdHS0tm/fXqZxixcvlsViUb9+/Sq2QAAAUGU4PdwsWbJESUlJmjRpknbu3Kk2bdqoZ8+eOnHixFXHHTp0SOPGjdNdd911kyoFAABVgdPDzaxZszRixAglJiYqIiJC8+bNk4+PjxYuXHjFMUVFRRo0aJCmTJmiBg0a3MRqAQBAZefUcFNYWKgdO3YoNjbW1ubi4qLY2Fht27btiuNeeeUVBQUFadiwYdfcRkFBgXJzc+0eAADAvJwabk6ePKmioiIFBwfbtQcHByszM7PUMVu3btWCBQv03nvvlWkbKSkpCggIsD3CwsJuuG4AAFB5Of20lCPOnDmjRx99VO+9955q1KhRpjHJycnKycmxPY4ePVrBVQIAAGdyc+bGa9SoIVdXV2VlZdm1Z2VlKSQkpET//fv369ChQ+rdu7etzWq1SpLc3Ny0e/duNWzY0G6Mp6enPD09K6B6AABQGTn1yI2Hh4fatWunjRs32tqsVqs2btyomJiYEv2bNWumH374Qenp6bZHnz591LVrV6Wnp3PKCQAAOPfIjSQlJSUpISFB7du315133qnZs2crPz9fiYmJkqQhQ4aodu3aSklJkZeXl1q2bGk3PjAwUJJKtAMAgFuT08NNfHy8srOzNXHiRGVmZioyMlJr1661TTI+cuSIXFyq1NQgAADgRE4PN5I0evRojR49utRlmzdvvurYDz74oPwLAgAAVRaHRAAAgKkQbgAAgKkQbgAAgKkQbgAAgKkQbgAAgKkQbgAAgKkQbgAAgKkQbgAAgKkQbgAAgKkQbgAAgKkQbgAAgKkQbgAAgKkQbgAAgKkQbgAAgKm4ObsAAABQdhkZGcrIyHB4XGhoqEJDQyugosqHcAMAQBXy7rvvasqUKQ6PmzRpkiZPnlz+BVVChBsAAKqQkSNHqk+fPnZt586dU6dOnSRJW7dulbe3d4lxt8pRG4lwAwBAlVLa6aX8/Hzb88jISPn6+t7ssioVwg0AAOUgfPwqp23bWnje9rz5hLVy8fByWi2SdGh6nFO3z9VSAADAVDhyAwBAFXIx75SK8k7ZtRkXCm3PC7MOyOLuUWKcq191uflVr/D6KgPCDQAAVUhe+hrlfP3pFZdnLXqh1PaAjgMV2GlQRZVVqRBuAACoQvwie8m7UbTD41xvkaM2EuEGAIAqxe0WOr10vZhQDAAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATIVwAwAATKVShJs5c+YoPDxcXl5eio6O1vbt26/Y97333tNdd92l2267TbfddptiY2Ov2h8AANxanB5ulixZoqSkJE2aNEk7d+5UmzZt1LNnT504caLU/ps3b9bAgQO1adMmbdu2TWFhYerRo4eOHTt2kysHAACVkdPDzaxZszRixAglJiYqIiJC8+bNk4+PjxYuXFhq/08++URPPfWUIiMj1axZM/35z3+W1WrVxo0bb3LlAACgMnJquCksLNSOHTsUGxtra3NxcVFsbKy2bdtWpnWcPXtWFy5cUPXq1UtdXlBQoNzcXLsHAAAwL6eGm5MnT6qoqEjBwcF27cHBwcrMzCzTOv74xz+qVq1adgHpcikpKQoICLA9wsLCbrhuAABQeTn9tNSNmD59uhYvXqzPP/9cXl5epfZJTk5WTk6O7XH06NGbXCUAALiZ3Jy58Ro1asjV1VVZWVl27VlZWQoJCbnq2Ndff13Tp0/Xhg0b1Lp16yv28/T0lKenZ7nUCwAAKj+nHrnx8PBQu3bt7CYDF08OjomJueK4mTNnaurUqVq7dq3at29/M0oFAABVhFOP3EhSUlKSEhIS1L59e915552aPXu28vPzlZiYKEkaMmSIateurZSUFEnSjBkzNHHiRC1atEjh4eG2uTl+fn7y8/Nz2n4AAIDKwenhJj4+XtnZ2Zo4caIyMzMVGRmptWvX2iYZHzlyRC4u/zvANHfuXBUWFmrAgAF265k0aZImT558M0sHAACVkNPDjSSNHj1ao0ePLnXZ5s2b7V4fOnSo4gsCAABVVpW+WgoAAOD3CDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUCDcAAMBUKkW4mTNnjsLDw+Xl5aXo6Ght3779qv2XLl2qZs2aycvLS61atdLq1atvUqUAAKCyc3q4WbJkiZKSkjRp0iTt3LlTbdq0Uc+ePXXixIlS+3/zzTcaOHCghg0bpl27dqlfv37q16+f/vOf/9zkygEAQGXk9HAza9YsjRgxQomJiYqIiNC8efPk4+OjhQsXltr/zTff1D333KPnn39ezZs319SpU9W2bVulpaXd5MoBAEBl5ObMjRcWFmrHjh1KTk62tbm4uCg2Nlbbtm0rdcy2bduUlJRk19azZ0+tXLmy1P4FBQUqKCiwvc7JyZEk5ebm3mD1pbMWnK2Q9VZFFfUeo/Li+/8/fP9vPXz//6civv/F6zQM45p9nRpuTp48qaKiIgUHB9u1BwcH6+effy51TGZmZqn9MzMzS+2fkpKiKVOmlGgPCwu7zqpRVgGznV0B4Dx8/3Erq8jv/5kzZxQQEHDVPk4NNzdDcnKy3ZEeq9WqU6dO6fbbb5fFYnFiZRUjNzdXYWFhOnr0qPz9/Z1dDnBT8f3Hrczs33/DMHTmzBnVqlXrmn2dGm5q1KghV1dXZWVl2bVnZWUpJCSk1DEhISEO9ff09JSnp6ddW2Bg4PUXXUX4+/ub8ssNlAXff9zKzPz9v9YRm2JOnVDs4eGhdu3aaePGjbY2q9WqjRs3KiYmptQxMTExdv0l6YsvvrhifwAAcGtx+mmppKQkJSQkqH379rrzzjs1e/Zs5efnKzExUZI0ZMgQ1a5dWykpKZKkMWPGqHPnzkpNTVVcXJwWL16s7777TvPnz3fmbgAAgErC6eEmPj5e2dnZmjhxojIzMxUZGam1a9faJg0fOXJELi7/O8DUoUMHLVq0SC+//LJefPFFNW7cWCtXrlTLli2dtQuViqenpyZNmlTiVBxwK+D7j1sZ3///sRhluaYKAACginD6TfwAAADKE+EGAACYCuEGAACYCuEGAACYCuEGAACYCuHGSTIzMzVmzBg1atRIXl5eCg4OVseOHTV37lydPXvpx9fCw8NlsVhksVjk6+urtm3baunSpSWWlfYYOnRoiW1u3bpVHTt21O233y5vb281a9ZMb7zxxlXrPHToUKnr/+c//1nu7wngjL+Ly3399ddyc3NTZGRkBe8pzGbo0KGlfuf27dt33evcvHmzLBaLTp8+Xerybdu2ydXVVXFxcde9jfLUpUsXPfvss84uQ1IluM/NrejAgQPq2LGjAgMDNW3aNLVq1Uqenp764YcfNH/+fNWuXVt9+vSRJL3yyisaMWKEcnNzlZqaqvj4eNWuXVvffvutioqKJEnffPON+vfvr927d9tuue3t7V1iu76+vho9erRat24tX19fbd26VSNHjpSvr68ef/zxq9a8YcMGtWjRwvb69ttvL6+3A5DkvL+LYqdPn9aQIUPUrVu3Ej/xApTFPffco/fff9+urWbNmhW2vQULFujpp5/WggULdPz48TL95lJFKCwslIeHh1O2fUUGbrqePXsaderUMfLy8kpdbrVaDcMwjHr16hlvvPGGrf3ChQuGj4+PMX78eLv+mzZtMiQZv/32m8O13H///cbgwYOvuPzgwYOGJGPXrl0OrxtwhLP/LuLj442XX37ZmDRpktGmTZvr2QXcwhISEoy+ffuWaE9NTTVatmxp+Pj4GHXq1DGefPJJ48yZM7blhw4dMu677z4jMDDQ8PHxMSIiIoxVq1bZ/r/38kdCQoJt3JkzZww/Pz/j559/NuLj440//elPdts9deqU8cgjjxg1atQwvLy8jEaNGhkLFy40DMMwCgoKjFGjRhkhISGGp6enUbduXWPatGm2sb/99psxbNgwo0aNGka1atWMrl27Gunp6bblxX8j7733nhEeHm5YLBYjISGhRL0HDx4snzf3OnBa6ib79ddftX79eo0aNUq+vr6l9rnSr5W7ubnJ3d1dhYWF5VLLrl279M0336hz587X7NunTx8FBQWpU6dO+tvf/lYu2weKOfvv4v3339eBAwc0adKk614HUBoXFxe99dZb+vHHH/Xhhx/qyy+/1AsvvGBbPmrUKBUUFGjLli364YcfNGPGDPn5+SksLEzLly+XJO3evVsZGRl68803beM+++wzNWvWTE2bNtXgwYO1cOFCGZfdk3fChAn673//qzVr1uinn37S3LlzVaNGDUnSW2+9pb/97W/67LPPtHv3bn3yyScKDw+3jX3wwQd14sQJrVmzRjt27FDbtm3VrVs3nTp1ytZn3759Wr58uVasWKH09HS9+eabiomJ0YgRI5SRkaGMjAyFhYVV1Nt6TZyWusn27dsnwzDUtGlTu/YaNWro/Pnzki592WfMmGG3vLCwUKmpqcrJydHdd999QzXUqVNH2dnZunjxoiZPnqzhw4dfsa+fn59SU1PVsWNHubi4aPny5erXr59WrlxpO0UA3Chn/l3s3btX48eP11dffSU3N/4vEdfv73//u/z8/Gyve/XqZZsPJl2aE/bqq6/qiSee0DvvvCPp0k8M9e/fX61atZIkNWjQwNa/evXqkqSgoCAFBgbabWvBggUaPHiwpEunw3JycvSPf/xDXbp0sa03KipK7du3t2272JEjR9S4cWN16tRJFotF9erVsy3bunWrtm/frhMnTth+xuH111/XypUrtWzZMtsUhsLCQn300Ud2p908PDzk4+OjkJAQx9+8csZfciWxfft2Wa1WDRo0SAUFBbb2P/7xj3r55Zd1/vx5+fn5afr06WWaPHb5H9jgwYM1b9482+uvvvpKeXl5+uc//6nx48erUaNGGjhwYKnrqVGjhpKSkmyv77jjDh0/flyvvfYa4QYVrqL/LubMmaNHHnlEU6ZMUZMmTSpkH3Dr6Nq1q+bOnWt77evrqw0bNiglJUU///yzcnNzdfHiRZ0/f15nz56Vj4+PnnnmGT355JNav369YmNj1b9/f7Vu3fqq29m9e7e2b9+uzz//XNKlo5fx8fFasGCBLdw8+eST6t+/v3bu3KkePXqoX79+6tChg6RLk5+7d++upk2b6p577tF9992nHj16SJK+//575eXllZhXee7cOe3fv9/2ul69ehU6n+hGEW5uskaNGslisWj37t127cVp/fcTHp9//nkNHTpUfn5+Cg4OvuKh+d9LT0+3PS+eTFmsfv36kqRWrVopKytLkydPvmK4KU10dLS++OKLMvcHrsVZfxdnzpzRd999p127dmn06NGSJKvVKsMw5ObmpvXr19/wkVLcOnx9fdWoUSPb60OHDum+++7Tk08+qT/96U+qXr26tm7dqmHDhqmwsFA+Pj4aPny4evbsqVWrVmn9+vVKSUlRamqqnn766StuZ8GCBbp48aLdBGLDMOTp6am0tDQFBASoV69eOnz4sFavXq0vvvhC3bp106hRo/T666+rbdu2OnjwoNasWaMNGzbooYceUmxsrJYtW6a8vDyFhoZq8+bNJbZ7+dGjK50+riwINzfZ7bffru7duystLU1PP/30Nb8gNWrUsPtjKauyjrFarXb/Ii6L9PR0hYaGOlwTcCXO+ruwWq364Ycf7Nreeecdffnll1q2bJntHwLA9dixY4esVqtSU1Pl4nJpiutnn31Wol9YWJieeOIJPfHEE0pOTtZ7772np59+2nYFUvEVgJJ08eJFffTRR0pNTbUdbSnWr18/ffrpp3riiSckXbpSKyEhQQkJCbrrrrv0/PPP6/XXX5d0KdzHx8crPj5eAwYM0D333KNTp06pbdu2yszMlJubm92prLLw8PCwq9WZCDdO8M4776hjx45q3769Jk+erNatW8vFxUXffvutfv75Z7Vr165CtjtnzhzVrVtXzZo1kyRt2bJFr7/+up555hlbn7S0NH3++efauHGjJOnDDz+Uh4eHoqKiJEkrVqzQwoUL9ec//7lCasStyxl/Fy4uLmrZsqVdW1BQkLy8vEq0A45q1KiRLly4oLffflu9e/fW119/bTdFQJKeffZZ9erVS02aNNFvv/2mTZs2qXnz5pIunfqxWCz6+9//rnvvvVfe3t7asGGDfvvtNw0bNkwBAQF26+rfv78WLFigJ554QhMnTlS7du3UokULFRQU6O9//7ttvbNmzVJoaKiioqLk4uKipUuXKiQkRIGBgYqNjVVMTIz69eunmTNnqkmTJjp+/LhWrVql+++/3zaHpzTh4eH617/+pUOHDsnPz0/Vq1e3hbqbjXDjBA0bNtSuXbs0bdo0JScn65dffpGnp6ciIiI0btw4PfXUUxWyXavVquTkZB08eFBubm5q2LChZsyYoZEjR9r6nDx50u68qiRNnTpVhw8flpubm5o1a6YlS5ZowIABFVIjbl3O+rsAKkqbNm00a9YszZgxQ8nJyfq///s/paSkaMiQIbY+RUVFGjVqlH755Rf5+/vrnnvusd1ctXbt2poyZYrGjx+vxMREDRkyRL/++qtiY2NLBBvpUriZOXOm/v3vf8vDw0PJyck6dOiQvL29ddddd2nx4sWSpGrVqmnmzJnau3evXF1ddccdd2j16tW2ILJ69Wq99NJLSkxMVHZ2tkJCQvR///d/Cg4Ovur+jhs3TgkJCYqIiNC5c+d08OBBh4/+lBeLcfm1YwAAAFUc97kBAACmQrgBAACmQrgBAACmQrgBAACmQrgBAACmQrgBAACmQrgBAACmQrgBAACmQrgBAACmQrgBAACmQrgBAACm8v8AkE4swSqMwdMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# Data\n",
    "labels = ['GPT-3.5', 'GPT-4', 'FastAssert']\n",
    "means = [np.mean(openai_gpt3_5_times), np.mean(openai_gpt4_times), np.mean(fastassert_times)]\n",
    "std_devs = [np.std(openai_gpt3_5_times), np.std(openai_gpt4_times), np.std(fastassert_times)]\n",
    "\n",
    "x = np.arange(len(labels))  # the label locations\n",
    "width = 0.35  # the width of the bars\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "rects1 = ax.bar(x, means, width, label='Mean', yerr=std_devs, capsize=5)\n",
    "\n",
    "# Add some text for labels, title and custom x-axis tick labels, etc.\n",
    "ax.set_ylabel('Times in seconds')\n",
    "ax.set_title('Completion times by model')\n",
    "ax.set_xticks(x)\n",
    "ax.set_xticklabels(labels)\n",
    "ax.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Validation errors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "from jsonschema import validate, ValidationError\n",
    "\n",
    "def validate_openai_json(text, schema):\n",
    "    # Step 1: Unescape the string\n",
    "    # Decode the escaped string to get the actual JSON string\n",
    "    json_string = text.replace(r'\\n', '')\n",
    "    \n",
    "    # Step 2: Parse the JSON string into a Python dictionary\n",
    "    json_obj = json.loads(json_string)\n",
    "\n",
    "    # Step 3: Validate the JSON object against the schema\n",
    "    try:\n",
    "        validate(instance=json_obj, schema=schema)\n",
    "        return True\n",
    "    except ValidationError as e:\n",
    "        return False\n",
    "    \n",
    "def validate_fastassert_json(text, schema):\n",
    "    try:\n",
    "        # Load the JSON\n",
    "        json_obj = json.loads(text)\n",
    "        # Validate the JSON object against the schema\n",
    "        validate(instance=json_obj, schema=schema)\n",
    "        return True\n",
    "    except Exception as e:\n",
    "        return False\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenAI GPT-3.5 validation error rate: 0.0\n",
      "OpenAI GPT-4 validation error rate: 0.0\n",
      "Fastassert validation error rate: 0.0\n"
     ]
    }
   ],
   "source": [
    "openai_gpt3_5_validations = []\n",
    "openai_gpt4_validations = []\n",
    "fastassert_validations = []\n",
    "\n",
    "for answer in openai_gpt3_5_answers:\n",
    "    if validate_openai_json(answer, json_obj):\n",
    "        openai_gpt3_5_validations.append(1)\n",
    "    else:\n",
    "        openai_gpt3_5_validations.append(0)\n",
    "\n",
    "for answer in openai_gpt4_answers:\n",
    "    if validate_openai_json(answer, json_obj):\n",
    "        openai_gpt4_validations.append(1)\n",
    "    else:\n",
    "        openai_gpt4_validations.append(0)\n",
    "\n",
    "for answer in fastassert_answers:\n",
    "    if validate_fastassert_json(answer, json_obj):\n",
    "        fastassert_validations.append(1)\n",
    "    else:\n",
    "        fastassert_validations.append(0)\n",
    "\n",
    "print(f\"OpenAI GPT-3.5 validation error rate: {1- sum(openai_gpt3_5_validations)/len(openai_gpt3_5_validations)}\")\n",
    "print(f\"OpenAI GPT-4 validation error rate: {1- sum(openai_gpt4_validations)/len(openai_gpt4_validations)}\")\n",
    "print(f\"Fastassert validation error rate: {1- sum(fastassert_validations)/len(fastassert_validations)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check the extracted data\n",
    "\n",
    "The hardest part of the task is infering the commonly used temperature unit. We will check the success accuracy of the several models on this task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_accuracy(inputs, answers):\n",
    "\n",
    "    results = []\n",
    "\n",
    "    for i in range(0, len(inputs)):\n",
    "        input = inputs[i]\n",
    "        answer = answers[i]\n",
    "\n",
    "        answer = answer.replace(r'\\n', '')\n",
    "\n",
    "        #try:\n",
    "        # Load the json object\n",
    "        json_obj = json.loads(answer)\n",
    "\n",
    "        # Check the \"unit\" key against the input label\n",
    "        if input[\"unit\"] == json_obj[\"unit\"]:\n",
    "            results.append(1)\n",
    "        else:\n",
    "            results.append(0)\n",
    "        #except Exception as e:\n",
    "        #    results.append(0)\n",
    "\n",
    "    return sum(results)/len(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPT3-5 turbo accuracy 0.7857142857142857\n",
      "GPT4 accuracy 0.9285714285714286\n",
      "FastAssert accuracy 0.7857142857142857\n",
      "\n",
      "FastAssert achieved the same accuracy as GPT3-5 turbo on this extraction task!\n"
     ]
    }
   ],
   "source": [
    "compute_accuracy(inputs, fastassert_answers)\n",
    "\n",
    "print(f\"GPT3-5 turbo accuracy {compute_accuracy(inputs, openai_gpt3_5_answers)}\")\n",
    "print(f\"GPT4 accuracy {compute_accuracy(inputs, openai_gpt4_answers)}\")\n",
    "print(f\"FastAssert accuracy {compute_accuracy(inputs, fastassert_answers)}\")\n",
    "print(\"\")\n",
    "\n",
    "if compute_accuracy(inputs, openai_gpt3_5_answers) <= compute_accuracy(inputs, fastassert_answers):\n",
    "    print(\"FastAssert achieved the same accuracy as GPT3-5 turbo on this extraction task!\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (notebookenv)",
   "language": "python",
   "name": "notebookenv"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}