diff --git a/docs/api/utilities/metrics/correlation.md b/docs/api/utilities/metrics/correlation.md index 1437195..11f0b34 100644 --- a/docs/api/utilities/metrics/correlation.md +++ b/docs/api/utilities/metrics/correlation.md @@ -1,11 +1,3 @@ # Correlation ::: exponax.metrics.correlation - ---- - -::: exponax.metrics.mean_correlation - ---- - -::: exponax.metrics._correlation \ No newline at end of file diff --git a/docs/api/utilities/metrics/derivative.md b/docs/api/utilities/metrics/derivative.md new file mode 100644 index 0000000..a1e30de --- /dev/null +++ b/docs/api/utilities/metrics/derivative.md @@ -0,0 +1,25 @@ +# Derivative-based Metrics + +Related to Sobolev Norms + +::: exponax.metrics.H1_MSE + +--- + +::: exponax.metrics.H1_MAE + +--- + +::: exponax.metrics.H1_RMSE + +--- + +::: exponax.metrics.H1_nMSE + +--- + +::: exponax.metrics.H1_nMAE + +--- + +::: exponax.metrics.H1_nRMSE diff --git a/docs/api/utilities/metrics/fourier.md b/docs/api/utilities/metrics/fourier.md new file mode 100644 index 0000000..2694640 --- /dev/null +++ b/docs/api/utilities/metrics/fourier.md @@ -0,0 +1,31 @@ +# Fourier-based + +::: exponax.metrics.fourier_MSE + +--- + +::: exponax.metrics.fourier_MAE + +--- + +::: exponax.metrics.fourier_RMSE + +--- + +::: exponax.metrics.fourier_nMSE + +--- + +::: exponax.metrics.fourier_nMAE + +--- + +::: exponax.metrics.fourier_nRMSE + +--- + +::: exponax.metrics.fourier_norm + +--- + +::: exponax.metrics.fourier_aggregator \ No newline at end of file diff --git a/docs/api/utilities/metrics/fourier_nrmse.md b/docs/api/utilities/metrics/fourier_nrmse.md deleted file mode 100644 index 548d6c0..0000000 --- a/docs/api/utilities/metrics/fourier_nrmse.md +++ /dev/null @@ -1,11 +0,0 @@ -# Fourier nRMSE - -::: exponax.metrics.fourier_nRMSE - ---- - -::: exponax.metrics.mean_fourier_nRMSE - ---- - -::: exponax.metrics._fourier_nRMSE \ No newline at end of file diff --git a/docs/api/utilities/metrics/mse_based.md b/docs/api/utilities/metrics/mse_based.md deleted file mode 100644 index 68d319e..0000000 --- a/docs/api/utilities/metrics/mse_based.md +++ /dev/null @@ -1,19 +0,0 @@ -# MSE-based metrics - -::: exponax.metrics.MSE - ---- - -::: exponax.metrics.nMSE - ---- - -::: exponax.metrics.mean_MSE - ---- - -::: exponax.metrics.mean_nMSE - ---- - -::: exponax.metrics._MSE \ No newline at end of file diff --git a/docs/api/utilities/metrics/rmse_based.md b/docs/api/utilities/metrics/rmse_based.md deleted file mode 100644 index fbf7dfd..0000000 --- a/docs/api/utilities/metrics/rmse_based.md +++ /dev/null @@ -1,19 +0,0 @@ -# RMSE-bsed metrics - -::: exponax.metrics.RMSE - ---- - -::: exponax.metrics.nRMSE - ---- - -::: exponax.metrics.mean_RMSE - ---- - -::: exponax.metrics.mean_nRMSE - ---- - -::: exponax.metrics._RMSE \ No newline at end of file diff --git a/docs/api/utilities/metrics/spatial.md b/docs/api/utilities/metrics/spatial.md new file mode 100644 index 0000000..53776d9 --- /dev/null +++ b/docs/api/utilities/metrics/spatial.md @@ -0,0 +1,31 @@ +# Spatial-based + +::: exponax.metrics.MSE + +--- + +::: exponax.metrics.MAE + +--- + +::: exponax.metrics.RMSE + +--- + +::: exponax.metrics.nMSE + +--- + +::: exponax.metrics.nMAE + +--- + +::: exponax.metrics.nRMSE + +--- + +::: exponax.metrics.spatial_norm + +--- + +::: exponax.metrics.spatial_aggregator diff --git a/docs/api/utilities/metrics/utils.md b/docs/api/utilities/metrics/utils.md new file mode 100644 index 0000000..1d6ba5d --- /dev/null +++ b/docs/api/utilities/metrics/utils.md @@ -0,0 +1,3 @@ +# Utilities for Metric Computation + +::: exponax.metrics.mean_metric \ No newline at end of file diff --git a/docs/examples/on_metrics_advanced.ipynb b/docs/examples/on_metrics_advanced.ipynb new file mode 100644 index 0000000..6df1f69 --- /dev/null +++ b/docs/examples/on_metrics_advanced.ipynb @@ -0,0 +1,294 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# On metrics -- Advanced Notebook\n", + "\n", + "This will get mathematical, be warned!\n", + "\n", + " ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ This notebook is a WIP, it will come with future release of Exponax ⚠️ ⚠️ ⚠️ ⚠️ ⚠️\n", + "\n", + "At the moment it is a dump of ideas on metric consistency with functional norms, connection to Parseval's theorem, and the relation to the Fourier transform." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import jax\n", + "import jax.numpy as jnp" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import exponax as ex" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Consistency of the metrics computation\n", + "\n", + "The discretized states in exponax $u_h \\in \\mathbb{R}^{C \\times N}$ represent\n", + "continuous functions sampled at an equidistant interval $\\Delta x = L/N$ where\n", + "$L$ is the length of the domain and $N$ is the number of discretization points.\n", + "Since we only work with **periodic boundary conditions**, we employ the\n", + "convention that the left point of the domain is considered a degree of freedom\n", + "and the right point is not. Hence, $u_0$ refers to the value of the continuous\n", + "function at $u(0)$ and $u_{N-1}$ refers to the value of the continuous function\n", + "at $u(\\frac{L}{N} (N-1))$.\n", + "\n", + "Now assume, we wanted to compute the squared $L^2$ norm of the function $u(x)$ over the\n", + "domain $\\Omega = (0, L)$\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)}^2 = \\int_{\\Omega} |u(x)|^2 \\; \\mathrm{d}x\n", + "$$\n", + "\n", + "A way to numerically approximate any integral with points given at equidistant\n", + "samples is via the trapezoidal rule. Assume we wanted to evaluate the following\n", + "integral\n", + "\n", + "$$\n", + "I = \\int_{0}^{L} f(x) \\; \\mathrm{d}x\n", + "$$\n", + "\n", + "The trapezoidal rule states that\n", + "\n", + "$$\n", + "I = \\Delta x \\left( \\frac{f(0) + f(L)}{2} + \\sum_{i=1}^{M-1} f(i \\Delta x) \\right) + \\mathcal{O}(\\Delta x^2)\n", + "$$\n", + "\n", + "where $\\Delta x = L/(M-1)$ is the distance between two consecutive points. In\n", + "contrast to our discretization on periodic grids, the trapezoidal rule also\n", + "accounts for the point on the right end of the domain. However, since the right\n", + "end of the domain must be equal to the value on the left end of the domain, we\n", + "have that $f(0) = f(L)$ and the trapezoidal rule simplifies to\n", + "\n", + "$$\n", + "I = \\Delta x \\sum_{i=0}^{M-1} f(i \\Delta x) + \\mathcal{O}(\\Delta x^2)\n", + "$$\n", + "\n", + "Or if we had $f(x)$ discretized as $f_h \\in \\mathbb{R}^N$ where $f_h = f(i\n", + "\\Delta x)$ with the periodic convention, we get\n", + "\n", + "$$\n", + "I = \\Delta x \\sum_i f_i + \\mathcal{O}(\\Delta x^2)\n", + "$$\n", + "\n", + "or expressed in terms of $L$ and $N$\n", + "\n", + "$$\n", + "I = \\frac{L}{N} \\sum_i f_i + \\mathcal{O}\\left(N^{-2}\\right)\n", + "$$\n", + "\n", + "This is exactly as scaled mean\n", + "\n", + "$$\n", + "I = L \\; \\text{mean}(f) + \\mathcal{O}\\left(N^{-2}\\right)\n", + "$$\n", + "\n", + "Since we actually wanted to evaluate the integral over the square absolute\n", + "function, we have that\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)}^2 = \\frac{L}{N} \\sum_i |u_i|^2 + \\mathcal{O}\\left(N^{-2}\\right)\n", + "$$\n", + "\n", + "or again in terms of the mean\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)}^2 = L \\; \\text{mean}(|u_h|^2) + \\mathcal{O}\\left(N^{-2}\\right)\n", + "$$\n", + "\n", + "Taking the mean over the element-wise squared is nothing else than the MSE (mean\n", + "squared error)\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)}^2 = L \\; \\text{MSE}(u_h) + \\mathcal{O}\\left(N^{-2}\\right)\n", + "$$\n", + "\n", + "Hence, the consistent counterpart to the squared (functional) $L^2$ norm is the\n", + "**scaled** MSE.\n", + "\n", + "For the regular $L^2$ norm, we have that\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)} = \\sqrt{\\int_{\\Omega} |u(x)|^2 \\; \\mathrm{d}x}\n", + "$$\n", + "\n", + "As such, we get a consistent counterpart\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)} = \\sqrt{L \\; \\text{MSE}(u_h) + \\mathcal{O}\\left(N^{-2}\\right)}\n", + "$$\n", + "\n", + "(TODO: check this). Roughly, we can say that\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)} \\approx \\sqrt{L \\; \\text{MSE}(u_h)} + \\mathcal{O}\\left(N^{-1}\\right)\n", + "$$\n", + "\n", + "Adn we can identify the RMSE as the consistent counterpart to the $L^2$ norm.\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)} \\approx \\sqrt{L} \\text{RMSE}(u_h) + \\mathcal{O}\\left(N^{-1}\\right)\n", + "$$\n", + "\n", + "It is scaled by the square root of the length of the domain.\n", + "\n", + "### Requirements\n", + "\n", + "The quadratic convergence on the MSE is only valid if the function is at least\n", + "twice continuously differentiable. In order to be that it must be that it also\n", + "periodic. In such a case, the estimate (might) even converges exponentially\n", + "(https://en.wikipedia.org/wiki/Trapezoidal_rule#Periodic_and_peak_functions)\n", + "fast!\n", + "\n", + "As a consequence, a bandlimited discrete function representation (might) not\n", + "evem have a discretization error at all!\n", + "\n", + "On the other hand, if the function is not periodic, the estimate likely not\n", + "converges quadratically. It converges linearly\n", + "(https://en.wikipedia.org/wiki/Riemann_sum#Left_rule) if it is continuous which\n", + "is guaranteed by the periodicity assumption.\n", + "\n", + "### Conclusion\n", + "\n", + "Assuming we are on the periodic domain, we have:\n", + "\n", + "- A bandlimited function is exactly integrated\n", + "- A non-bandlimited, but periodically continuous function converges\n", + " exponentially linear (similar to how the spectral derivative converges)\n", + "- A discontinuous function converges linearly\n", + "\n", + "Due the special case how periodic grids are layed out, we will never have the\n", + "case of quadratic convergence." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Mean-Average Error (MAE)\n", + "\n", + "Is consistent with the L1 norm???" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Higher dimensions\n", + "\n", + "In higher dimensions with a domain $\\Omega = (0, L)^D$ with $D$ being the\n", + "number of spatial dimensions with the same convention for periodic boundary\n", + "conditions, we have that\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)}^2 = \\frac{L^D}{N^D} \\sum_i |u_i|^2 + \\mathcal{O}\\left(N^{-2}\\right)\n", + "$$\n", + "\n", + "Assuming the $\\text{mean}$ function takes the mean over the flattened axes with\n", + "$N^D$ elements, we have that\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)}^2 = L^D \\; \\text{mean}(|u_h|^2) + \\mathcal{O}\\left(N^{-2}\\right)\n", + "$$\n", + "\n", + "or in terms of the MSE\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)}^2 = L^D \\; \\text{MSE}(u_h) + \\mathcal{O}\\left(N^{-2}\\right)\n", + "$$\n", + "\n", + "Correspondingly, the RMSE is the consistent counterpart to the $L^2$ norm in\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)} \\approx \\sqrt{L^D} \\; \\text{RMSE}(u_h) + \\mathcal{O}\\left(N^{-1}\\right)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Multiple Channels\n", + "\n", + "If the underlying function is a vector-valued function $u(x) \\in \\mathbb{R}^C$,\n", + "we can compute the $L^2$ norm of the function as\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)}^2 = \\int_{\\Omega} u(x)^T u(x) \\; \\mathrm{d}x\n", + "$$\n", + " \n", + "Hence, the consistent MSE reads\n", + "\n", + "$$\n", + "\\|u\\|_{L^2(\\Omega)}^2 = L^D \\; \\text{MSE}(u_h^T u_h) + \\mathcal{O}\\left(N^{-2}\\right)\n", + "$$\n", + "\n", + "with the inner product being understand only over the leading channel axis." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Differences between $p_2$, $l_2$, and $L_2$ norms and their relation to commonly used metrics\n", + "\n", + "https://mathworld.wolfram.com/L2-Norm.html" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Parseval's Identity: Spatial and Fourier aggregator\n", + "\n", + "Conceptually both do the same, but the Fourier aggregator can do more it that it\n", + "also allows filtering and taking derivatives. The latter gives rise to\n", + "Sobolev-based losses.\n", + "\n", + "However, they are only identical if the function is bandlimited." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "exponax_dev", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.4" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/docs/examples/on_metrics_simple.ipynb b/docs/examples/on_metrics_simple.ipynb new file mode 100644 index 0000000..fbbfcd1 --- /dev/null +++ b/docs/examples/on_metrics_simple.ipynb @@ -0,0 +1,237 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# The metrics of comparing fields in `Exponax`\n", + "\n", + "There are four major classes of metrics:\n", + "\n", + "1. Spatial-based (that work in physical space)\n", + "2. Fourier-based (that work in the coefficient space)\n", + "3. Correlation-based\n", + "4. Derivative-based (which sugarcoat the functionalities to Fourier-based\n", + " approaches to achieve Sobolev-like norms)\n", + "\n", + "Class 1., 2., and 4. can be further divided into:\n", + "1. Absolute metrics (i.e., related to the MAE)\n", + "2. Absolute squared metrics (i.e., related to the MSE)\n", + "3. Rooted metrics (i.e., related to the RMSE)\n", + "\n", + "Then for each of the three, there is both the absolute version and a\n", + "relative/normalized version\n", + "\n", + "All metrics computation work on single state arrays, i.e., arrays with a leading channel axis and one, two, or three subsequent spatial axes. **The arrays shall not have leading batch axes.** To work with batched arrays use `jax.vmap` and then reduce, e.g., by `jnp.mean`. Alternatively, use the convinience wrapper [`exponax.metrics.mean_metric`][].\n", + "\n", + " ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ This notebook is a WIP, it will come with future release of Exponax ⚠️ ⚠️ ⚠️ ⚠️ ⚠️" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import jax\n", + "import jax.numpy as jnp" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import exponax as ex" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The Standard Candidates: MAE, MSE, RMSE" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Absolute Metrics" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Normalized/Relative Metrics" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Why it needs the domain size?" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Correlation" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Fourier-based Metrics" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Wait? Isn't that my MSE? A quick intro Parseval's theorem" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Filtering and Scale-Specific Metrics" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Metrics with derivatives" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Sobolev-like Metrics" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Wait? Who is Sobolev?" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "u_strongly_diffused = ex.ic.DiffusedNoise(1, intensity=0.001)(\n", + " 100, key=jax.random.PRNGKey(0)\n", + ")\n", + "u_less_diffused = ex.ic.DiffusedNoise(1, intensity=0.0003)(\n", + " 100, key=jax.random.PRNGKey(0)\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAksAAAG2CAYAAABvWcJYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAACNYklEQVR4nOzdd3iTVfvA8W+SppPuXSiUvfcoRVSUDSooDhBFEeFV4XXgxJ/jVVTELYriQsEFLhyoQNkiZVNmKXt1UkrpbtPm+f3xNIFCGzoymvb+XFevpHmenNw5tOHuOee5j0ZRFAUhhBBCCFEhraMDEEIIIYSoyyRZEkIIIYSwQJIlIYQQQggLJFkSQgghhLBAkiUhhBBCCAskWRJCCCGEsECSJSGEEEIICyRZEkIIIYSwQJIlIYQQQggLJFkSQgghhLDAqZKl9evXc+ONNxIREYFGo+HXX3+94nPWrl1Ljx49cHNzo1WrVnz11VeXnTN37lyioqJwd3cnOjqaLVu2WD94IYQQQjglp0qW8vLy6Nq1K3Pnzq3S+ceOHWPkyJFcd911xMfH8+ijj3L//fezfPly8zmLFy9m+vTpvPjii+zYsYOuXbsydOhQ0tPTbfU2hBBCCOFENM66ka5Go2HJkiWMHj260nOefvpp/vzzT/bu3Wt+bOzYsWRlZbFs2TIAoqOj6d27Nx9++CEARqORyMhI/vvf//LMM8/Y9D0IIYQQou5zcXQAthQXF8egQYPKPTZ06FAeffRRAIqLi9m+fTszZswwH9dqtQwaNIi4uLhK2y0qKqKoqMj8vdFoJDMzk8DAQDQajXXfhBBCCCFsQlEUcnJyiIiIQKutfLKtXidLqamphIaGlnssNDSU7OxsCgoKOHfuHKWlpRWec+DAgUrbnTVrFi+99JJNYhZCCCGEfZ06dYomTZpUerxeJ0u2MmPGDKZPn27+/vz58zRt2pRjx47h7e1ttdcxGAysWbOG6667Dr1eb7V2RXnSz/YjfW0f0s/2If1sH7bs55ycHJo3b37F/7vrdbIUFhZGWlpaucfS0tLw8fHBw8MDnU6HTqer8JywsLBK23Vzc8PNze2yxwMCAvDx8bFO8Kg/IJ6engQGBsovog1JP9uP9LV9SD/bh/Szfdiyn03tXWkJjVNdDVddMTExrFq1qtxjsbGxxMTEAODq6krPnj3LnWM0Glm1apX5HCGEEEI0bE6VLOXm5hIfH098fDyglgaIj4/n5MmTgDo9NmHCBPP5DzzwAEePHuWpp57iwIEDfPTRR/zwww889thj5nOmT5/OZ599xoIFC0hISODBBx8kLy+PiRMn2vW9CSGEEKJucqppuG3btnHdddeZvzetG7rnnnv46quvSElJMSdOAM2bN+fPP//kscce4/3336dJkyZ8/vnnDB061HzOHXfcwZkzZ3jhhRdITU2lW7duLFu27LJF30IIIYRomJwqWRowYACWykJVVJ17wIAB7Ny502K706ZNY9q0abUNTwghhBD1kFNNwwkhhBBC2JskS0IIIYQQFkiyJIQQQghhgSRLQgghhBAWSLIkhBBCCGGBJEtCCCGEEBZIsiSEEEIIYYEkS0IIIYQQFkiyJIQQQghhgSRLQgghhBAWSLIkhBBCCGGBJEtCCCGEEBZIsiSEEEIIYYEkS0IIIYQQFkiyJIQQQghhgSRLQgghhBAWSLIkhBBCCGGBJEtCCCGEEBZIsiSEEEIIYYEkS0IIIYQQFkiyJIQQQghhgSRLQgghhBAWSLIkhBBCCGGBJEtCCCGEEBZIsiSEEEIIYYEkS0IIIYQQFkiyJIQQQghhgSRLQgghhBAWSLIkhBBCCGGBJEtCCCGEEBZIsiSEEEIIYYEkS0IIIYQQFkiyJIQQQghhgSRLQgghhBAWSLIkhBBCCGGB0yVLc+fOJSoqCnd3d6Kjo9myZUul5w4YMACNRnPZ18iRI83n3HvvvZcdHzZsmD3eihBCCCGcgIujA6iOxYsXM336dObNm0d0dDTvvfceQ4cOJTExkZCQkMvO/+WXXyguLjZ/f/bsWbp27cptt91W7rxhw4bx5Zdfmr93c3Oz3ZsQQgghhFNxqpGld955h8mTJzNx4kQ6dOjAvHnz8PT0ZP78+RWeHxAQQFhYmPkrNjYWT0/Py5IlNze3cuf5+/vb4+0IIYQQwgk4zchScXEx27dvZ8aMGebHtFotgwYNIi4urkptfPHFF4wdOxYvL69yj69du5aQkBD8/f25/vrreeWVVwgMDKy0naKiIoqKiszfZ2dnA2AwGDAYDNV5WxaZ2rJmm+Jy0s/2I31tH9LP9iH9bB+27OeqtqlRFEWx+qvbQHJyMo0bN2bjxo3ExMSYH3/qqadYt24dmzdvtvj8LVu2EB0dzebNm+nTp4/58UWLFuHp6Unz5s05cuQIzz77LI0aNSIuLg6dTldhW//73/946aWXLnv8u+++w9PTs4bvUAghhBD2lJ+fz5133sn58+fx8fGp9DynGVmqrS+++ILOnTuXS5QAxo4da77fuXNnunTpQsuWLVm7di0DBw6ssK0ZM2Ywffp08/fZ2dlERkYyZMgQi51dXQaDgdjYWAYPHoxer7dau6I86Wf7kb62D+ln+5B+tg9b9rNpZuhKnCZZCgoKQqfTkZaWVu7xtLQ0wsLCLD43Ly+PRYsW8fLLL1/xdVq0aEFQUBCHDx+uNFlyc3OrcBG4Xq+3yS+MrdoV5Uk/24/0tX1IP9uH9LN92KKfq9qe0yzwdnV1pWfPnqxatcr8mNFoZNWqVeWm5Sry448/UlRUxF133XXF1zl9+jRnz54lPDy81jELIYQQwvk5TbIEMH36dD777DMWLFhAQkICDz74IHl5eUycOBGACRMmlFsAbvLFF18wevToyxZt5+bm8uSTT7Jp0yaOHz/OqlWrGDVqFK1atWLo0KF2eU9CCCGEqNucZhoO4I477uDMmTO88MILpKam0q1bN5YtW0ZoaCgAJ0+eRKstn/8lJiayYcMGVqxYcVl7Op2O3bt3s2DBArKysoiIiGDIkCHMnDlTai0JIYQQAnCyZAlg2rRpTJs2rcJja9euveyxtm3bUtkFfx4eHixfvtya4QkhhBCinnGqaTghhBBCCHuTZEkIIYQQwgJJloQQQgghLJBkSQghhBDCAkmWhBBCCCEskGRJCCGEEMICSZaEEEIIISyQZEkIIYQQwgJJloQQQgghLJBkSQghhBDCAkmWhBBCQMpu2PAe5KQ6OhIh6hyn2xtOCCGElWUnw8KboOAcrJ0Fve6Dqx4F71BHRyZEnSAjS0II0ZAZS+HnyWqipPeEkkLY9BG83xWW/x/kZzo6QiEcTpIlIYRoyP55B05sAL0XPLAB7voFmvSGkgKI+xC+HwuK4ugohXAoSZaEEKKhOrlJnXYDGPk2BLaEVgNhUiyM/xlc3OHUZji1xbFxCuFgkiwJIURDVHAOfr4flFLocgd0G3fhmEYDrQdB51vV77d84pgYhagjJFkSQoiGRlHg94fh/Cnwb66OKlWkzxT1dv9vkJ1iv/iEqGMkWRJCiIbmyCpI+B20LnDrfHDzrvi88K4Q2ReMJbD9S/vGKEQdIsmSEEI0NDu/UW973QeNe1g+N7psdGnbl1BSbNu4hKijJFkSQoiGpOAcHPhTvd/9riuf3/4m8A6HvHR1Ok6IBkiSJSGEaEj2/gylxRDaCcK6XPl8nV4dgQJZ6C0aLEmWhBCiIYn/Tr3tdqd61VtV9LwXdK5weisk7bBZaELUVZIsCSFEQ5F+AJK2qwu7O99e9ec1CoGON6v3t3xqm9iEqMMkWRJCiIZiV9moUush0Ci4es81lRHY+zPknrFuXELUcZIsCSFEQ1BaArsWqfe73Vn95zfpBRHd1fVO+3+1amhC1HWSLAkhRENwZDXkpoFnILQeWrM22t+k3h6KtV5cQjgBSZaEEKIhiP9Wve18O7i41qyN1kPU22PrwVBgnbiEcAKSLAkhRH2XnwmJf6n3azIFZxLaEXwaQ0kBHN9gndiEcAKSLAkhRH1nrq3UGcKrUFupMhoNtB6s3j+0wjqxCeEEJFkSQoj6bvdi9bY2o0ompvVOB5erG/IK0QBIsiSEEPXZ+SS1mCQa6HRL7dtrfo1aoDLrBGQcqn17QjgBSZaEEKI+S/hDvY2MBu+w2rfn1gii+qv3Dy2vfXtCOAFJloQQoj4zJUsdbrJem6ar4mTdkmggJFkSQoj6Ku8MnNyo3m9/o/XaNSVLJ+KgMNt67QpRR0myJIQQ9ZQm8S9QjGrlbb+m1ms4sCUEtASjAY6utV67QtRRTpcszZ07l6ioKNzd3YmOjmbLli2VnvvVV1+h0WjKfbm7u5c7R1EUXnjhBcLDw/Hw8GDQoEEcOiSLFoUQzk+buFS9Y81RJROZihMNiFMlS4sXL2b69Om8+OKL7Nixg65duzJ06FDS09MrfY6Pjw8pKSnmrxMnTpQ7/sYbbzBnzhzmzZvH5s2b8fLyYujQoRQWFtr67QghhM3oS/LQHP9H/ab9KOu/QBtTshQrJQREvedUydI777zD5MmTmThxIh06dGDevHl4enoyf/78Sp+j0WgICwszf4WGhpqPKYrCe++9x3PPPceoUaPo0qULCxcuJDk5mV9//dUO70gIIWwj7PxONMYSCOkAQa2s/wLNrgK9F+SmQupu67cvRB3iNMlScXEx27dvZ9CgQebHtFotgwYNIi4urtLn5ebm0qxZMyIjIxk1ahT79u0zHzt27Bipqanl2vT19SU6Otpim0IIUdeFn9+q3ml/5avgcotK+Hn7aWYu3c/7Kw/xddxxlu5OZuPhDHIKDRU/ycUNWgxQ78tUnKjnXBwdQFVlZGRQWlpabmQIIDQ0lAMHDlT4nLZt2zJ//ny6dOnC+fPneeutt+jXrx/79u2jSZMmpKammtu4tE3TsYoUFRVRVFRk/j47W70axGAwYDBU8sFSA6a2rNmmuJz0s/1IX9uHIfccIdl71fttRkAF/V1SauTfI2f5bVcKsQnpFBqMFbbl7e7CxJhm3BPTFB8PfbljmhbX45L4J8bE5ZTGPGr191HXyc+zfdiyn6vaptMkSzURExNDTEyM+ft+/frRvn17PvnkE2bOnFnjdmfNmsVLL7102eMrVqzA09Ozxu1WJjY21uptistJP9uP9LVtRZzbTG/FQK5bKKu2HgPN8XLHE89r+O6wlqxijfmxYHeF9n4KBiPklUCeQUNmEZwrLGHOmiN8tv4w14YrXBtuxLPsfw73Yh1DAU3SNlb+tohivY/93mQdIj/P9mGLfs7Pz6/SeU6TLAUFBaHT6UhLSyv3eFpaGmFhVatKq9fr6d69O4cPHwYwPy8tLY3w8PBybXbr1q3SdmbMmMH06dPN32dnZxMZGcmQIUPw8bHeh4XBYCA2NpbBgwej1+uv/ARRI9LP9iN9bR+an34CwK3bbYwYNNL8uNGoMG/9MeZtOoxRAX9PPTd0DuOmruF0beKLRqMp147RqLB8fxofrDnCofQ8lp3WsDHDlbdu7cx1bYMBUDK+QJO2h8HNQekywn5vsg6Qn2f7sGU/m2aGrsRpkiVXV1d69uzJqlWrGD16NABGo5FVq1Yxbdq0KrVRWlrKnj17GDFC/YVu3rw5YWFhrFq1ypwcZWdns3nzZh588MFK23Fzc8PNze2yx/V6vU1+YWzVrihP+tl+pK9tyFCAcnQVAJoON+FS1s9Z+cU8tjieNYlnALi9VxNeHtUJd73OYnM3dY/khq5NWLYvlfdWHuRgWi4PfLuTF2/syD39oqDtcEjbg8uRWOh5t03fWl0lP8/2YYt+rmp7TpMsAUyfPp177rmHXr160adPH9577z3y8vKYOHEiABMmTKBx48bMmjULgJdffpm+ffvSqlUrsrKyePPNNzlx4gT3338/oF4p9+ijj/LKK6/QunVrmjdvzvPPP09ERIQ5IRNCCKdyeBUaQx75+gD04d0B2Jt0nv98vZ2krALcXLTMHNWJ23tHVrlJrVbDiM7hDGofynO/7uGHbad58fd9HD+bx3PdhqFb/wYcXg0lxeDiaqt3JoTDOFWydMcdd3DmzBleeOEFUlNT6datG8uWLTMv0D558iRa7YUL/M6dO8fkyZNJTU3F39+fnj17snHjRjp06GA+56mnniIvL48pU6aQlZVF//79WbZs2WXFK4UQwins+RGAJP9oojQaDqblMP7zzZwvMNAs0JOPxvegY4RvjZp2ddEye0wXmgV68ebyRL789zinzwbxiVco2rw0OPEvtLzOmu9GiDrBqZIlgGnTplU67bZ27dpy37/77ru8++67FtvTaDS8/PLLvPzyy9YKUQghHKMoBw4uAyDJPwZ9VgETvtjK+QIDPZr68eXEPvh61G4aQ6PRMPW6VjQN8OTxH3cReyCD1X7dGMRy9bUlWRL1kNPUWRJCCHEFB/6EkkKUwFYkuTRj4lfbSc0upE1oI+bf27vWidLFbuwawfeT++LroWdxdkcAlMS/pZq3qJckWRJCiPqibAqusO3NfHLAhWNn82ns58GC+/rg52n9tUQ9m/nz+T292KLtQpGiR5N1AuVMxXXvhHBmkiwJIUR9kJcBR9YA8H9H2nMyT4O/p54F9/Uh3NfDZi/bOyqAWXf0ZaNRXQu6adl3NnstIRxFkiUhhKgP9i0BpZQkz3b8csIdV63CZ3f3oFVII5u/9IjO4bh1UEuy6A4v59edSTZ/TSHsSZIlIYSoD/aohSjnn+8FwITWRro2qdlVbzXRb/h4AHpqDvLqTxvYejzTbq8thK1JsiSEEM7u3Ak4tQmjouGP0hgm94+ic4CdF1r7RaKEdkSnUbhKiWfadzvIyC268vOEcAKSLAkhhJMz7FZHlTYZ2xMV1ZLpg1o5JA5Nm+EAjPbcTVp2EY8s2kmpUa6OE85PkiUhhHByGRu/AWCV/ho+uLM7LjoHfbS3VZOlq7W78NEr/Hv4LO+vPOiYWISwIkmWhBDCiS1fvZrwoqMUKzoGj5lCqI8Ddx+I6AFeweiKc/i4v7qb+wdrDrM2Md1xMQlhBZIsCSGEkzp6JpdjaxcCcDrwKvp2bOnYgLRa6DAKgKsylzA+uimKAo8tjic5q8CxsQlRC5IsCSGch6JAyi7Y/pVaV6gBKy4x8tj327lB8w8AUQPucXBEZaIfUG8T/+aFfm50auzDuXwD077bQUmp0bGxCVFDkiwJIeq2UgMcWQ1/PgHvdoJProE/HoHlzzo6Mod6d+VBAlL/oYkmA6O7P9r2Ix0dkiqoNbQZBii4bfuUj+7sibebCztOZvHhmsOOjk6IGpFkSQhRt301Er6+GbZ+BtmnQVu2v9mhFWAsdWxsDhJ35Czz1h3hTt0qALTd7gS97ap0V1vfh9Tb+G9p6lHIKzd3AmDOqkNsPyH1l4TzkWRJCFF3ZZ2CU5tBo4Xud8O4RfDUUXD1hoJz6pRcA5OVX8z0H+IJU84yUBevPtjzXkeGdLnm10BoZzDkw/avGNWtMaO7RWBU4NHF8eQUGhwdoRDVIsmSEKLuOhmn3oZ3g1Efqpemu/uo/xkDHF3jsNAcQVEUnl2yh5TzhfzH+1+0GKFZfwhu4+jQytNoIGaqen/Lp1BSzMujO9HYz4NTmQX87/f9jo1PiGqSZEkIUXed+Fe9bdav/OMtr1NvjzSsZOnHbaf5a08qblojd+rXqg/2mujQmCrVaQw0CoOcFNj/Kz7uet69oxtaDfy84zR/7Ep2dIRCVJkkS0KIuutE2cjSZcnS9ertyU1QnGffmBzkyJlcXvx9HwDv9kjDNT8VPAOh/Y0OjqwSLq7Q5371ftyHoCj0aR7A1OvU6uL/t2SPlBMQTkOSJSFE3ZSXARmJ6v2mMeWPBbQA36ZgNMDxf+0fm50VlZTy8Pc7KTCU0q9lIMMLl6kHut0JLm6ODc6SnveBi4e6tqxslPDhga3pGulHdmEJT/y4C6NshyKcgCRLQoi6ybReKbg9eAaUP6bRXJiKawDrlt5clsi+5Gz8PfW8PzQAzeGV6oGedXQKzsQrELqNU++veQ0Mheh1Wt69vSvuei0bj5xlQdxxh4YoRFVIsiSEqJtObFRvL52CM2kg65bWJqbz+YZjAMwe04XgQ4sBRV3kHujgit1VETMNdG7qyNK3t0JhNi2CG/F/I9oD8PrfBzicnuPgIIWwTJIlIUTddKVkqfm1gAbOJEB2/VwsfCaniCd+VMsj3N23GUPaBcLOr9WDve5zYGTVENgS7vpJLfdw/B9YcCPkZXBX32Zc0yaYohIjjy3ehUGqe4s6TJIlIUTdU5gNqbvV+5UlS54BENFdvX90rV3CsqdSo8LjP+4iI7eYtqHe/N/I9nDgT8hNA69gaFtHKnZXRfNr4N4/1AXpKfEwfxia86d589Yu+Hro2ZN0ng9WS3VvUXdJsiSEqHtObQHFCP5R4BNR+XnmqbjVdgnLnt5fdYj1B8/g5qJlzrjuuOt1as0igB4T1KvNnElEd7hvOfhGwtlD8PlAQtc+xTed44nR7uO7NTuIP5Xl6CiFqJCLowMQQojLnCybgmtayaiSScvr4Z+31ZElo1Hd9b4eWHMgnTmrDgHw2s2daRvmDal71HU/Gh30muTgCGsoqDXctwy+vkW90nHHAjoD35flfce+bIphwCT03ceBd6hDQxXiYvXjk0UIUb9cab2SSZM+oPeCvDOQttf2cdnBybP5PLJoJwB39W3KmJ5N1AObP1FvO9wEvo0dFJ0V+DaByavh1vlw9RPQdiSlflEANDeeRL/6RXinPXw3Fg78BYqUFhCOJ8mSEKJuMRRC0nb1/pWSJRdXiOqv3q8HJQQKDaU88M12sgtL6Bbpx/M3dFAP5GfCnh/V+9EPOC5Aa3FrpFb4Hvg8jPsO3aO72HjrdmYYJrHD2AqUUjj4Nywap44cCuFgkiwJIeqWpO1QWgyNQtXik1dST9YtKYrC87/uZX9KNgFernw0vgduLjr14I4FUFIIYV0gMtqxgdpIv06t0Pe5j1uKX+Z2/fsU9Sir/r3m1XpfHkLUfZIsCSHqlpMXTcFpNFc+37T1yYk4MDjv9hlf/nucH7efRquBD8Z1J8LPQz1QWgJbPlfvRz9QtT5xUjOGt6dFkBdbcoJ5Mu8u6H63utD/50lw/rSjwxMNmCRLQoi65UQVF3ebBLUBrxAoLYJU51y39PeeFGb+uR+AZ4a346pWQRcOJv4J2afVy+47jXFQhPbh4arjnTu6odNq+H1XMn9GTldH0/LPwg/3QEmxo0MUDZQkS0KIuqO0RC0bAFder2Si0UBYJ/V++j7bxGVD209k8ujieBRFLTw5+epLph43l5UL6Hkv6N3tHp+9dYv0M2+2++wfhzkz/DNw94WkbbDi/xwcnWioJFkSQtQdqbuhOFf9zzGkQ9WfZzo3zbmSpaNncrl/wTaKSowMah/K/27qiObiabbUPXBig3OXC6iB/17fis6NfTlfYGD6yvMYR5cljFs+hd0/ODY40SBJsiSEqDtObVZvI/tWr2ZSaNnIUtp+68dkI2dyirjnyy2cyzfQNdKPD8Z1R6e9ZD1SfSkXUE16nZZ37+iGu17LP4cyWHC2LVzzpHrw76egOM+xAYoGR5IlIUTdkV6W7IR3rd7zQstGltL32b8uT3YyrH4FkuOr/JRzecXc++UWTmUW0DTAky/u6YWHq+6SdlNg92L1fp//WC9eJ9Eq5MJmu7P+PsDB9tPUiu4F52DX944NTjQ4kiwJIeqOMwfV2+C21XteUFt1qqrgHOSkWj+uiigKxH8Pc/vC+jdh4SjIPHbFp53NLWLcZ5vYl5xNoJcrX03sTVAjt8tP3PSRWkIhMhqa9rXBG6j77urbjAFtgykuMfLoD3sw9CmrMRX3kVqxXQg7kWRJCFE3KAqcOaDer26ypHdXd7cH+6xbykmD78fBrw9A0XnQuUFhFiwab3GK6EyOmigdSM0h2NuNRVP60iK40eUnFpyDbfPV+/2n1+tyAZZoNBreuLULAV6u7E/J5v2zfcDNFzKPwKHljg5PNCCSLAkh6oa8M2rCgQYCW1X/+aEd1VtbXxF3cAV8FK1WmNbqYeAL8N9tavmC9H3w29QKpwLTswsZ+2kcB9NyCfVRE6XWod4Vv8aWz9WF7iEdoc1Q276fOi7E251Zt3QGYO6/qSS3ukM9EDfXgVGJhsbpkqW5c+cSFRWFu7s70dHRbNmypdJzP/vsM66++mr8/f3x9/dn0KBBl51/7733otFoyn0NGzbM1m9DCHGpM4nqrX8U6D2q//yQsmTJlou8Sw2wZIo68hPWBf6zDq5+HPyawu0LQesC+5bAv++Ve9qpzHzu+HQTR87kEeHrzuIpMbSsaEQJoDgfNn+s3u//WIMdVbrY0I5h3NErEkWBBw/1QtG6wPF/qrVOTIjacKpkafHixUyfPp0XX3yRHTt20LVrV4YOHUp6enqF569du5Zx48axZs0a4uLiiIyMZMiQISQlJZU7b9iwYaSkpJi/vv9eFg8KYXc1nYIzuXiRt60cXacmSl4hcP+qC6NZAM1iYPhs9f7Kl+DwSgA2Hz3LqLn/ciwjj8Z+Hiz+TwxRQV6Vv8aOhWoRRv8o6Hiz7d6Lk3nhxg40C/RkV3Yjtnldqz646SPHBiUaDKdKlt555x0mT57MxIkT6dChA/PmzcPT05P58+dXeP63337LQw89RLdu3WjXrh2ff/45RqORVatWlTvPzc2NsLAw85e/v7893o4Q4mIZZYu7g9rU7PmmWktnEtURIFvYv0S9bX+juonvpXpNgh4TAAV+uo/f12/hri82k5lXTKfGPvz0YAyRAZ6Vt19SDBs/UO/3exh0LlZ/C87Ky82F98d2x0Wr4eWMsv0A9/6sXo0ohI05zW9icXEx27dvZ8aMGebHtFotgwYNIi4urkpt5OfnYzAYCAgIKPf42rVrCQkJwd/fn+uvv55XXnmFwMDAStspKiqiqKjI/H12djYABoMBg8F6H9KmtqzZpric9LP9WOprXfoBtEBJQGuUmvxbNIrAxdULTXEehrTEmo9QVabUgMuBP9EAJW1vqDzGwbPQpexFm7IDn9gnMJQ+xfCOYcy+pRMerjqLP2eaXYtwyT6N4hVCSafboYY/k/X1Z7pjmBePDmzFW7EKW5X29DYmULppHsbrnndIPPW1n+saW/ZzVdt0mmQpIyOD0tJSQkNDyz0eGhrKgQMHqtTG008/TUREBIMGDTI/NmzYMG655RaaN2/OkSNHePbZZxk+fDhxcXHodLoK25k1axYvvfTSZY+vWLECT08LfzXWUGxsrNXbFJeTfrafivp6SNIePICNBzM4l/RXjdq92iWMgOIjxK/4lmR/615uH5y9m34F5yh08WH5vvOwv+IYzxXB+oyxfKzsYYBuFy/5rsPXuz9rViZVeL6ZYuT6hNfwBvb7XsfhFatrHXN9/JlurEBrHy2f5g6nt2sCpZs/Y0VuR0p1jtsKpj72c11ki37Oz8+v0nlOkyzV1uuvv86iRYtYu3Yt7u4XfqnGjh1rvt+5c2e6dOlCy5YtWbt2LQMHDqywrRkzZjB9+nTz99nZ2eb1UD4+PlaL2WAwEBsby+DBg9Hr9VZrV5Qn/Ww/lfZ1YTb6necAiLnxHnCv2e+RTomF+CP0aOxOtwEjrBHyhbaXqpeq67vcwojhN1x2XFEUftuVwtt/HiCnsAkfut7K45rvmVCymJJrHgbvMIvta7d+hi4+BcXdlzZ3vk4bt0qulKuC+v4z3evqQkZ94MIxYyjNSWN4xHmMPW+xexz1vZ/rClv2s2lm6EqcJlkKCgpCp9ORlpZW7vG0tDTCwix/CL311lu8/vrrrFy5ki5dulg8t0WLFgQFBXH48OFKkyU3Nzfc3C4vIqfX623yC2OrdkV50s/2c1lfpx5Vb73D0XtXPgV+ReGdIR50GQfQWfPfstQAB9WRJF2nWy5r+2xuEc8u2cPyfernU/emftw85nX4NQFNSjz65U/D2G8rv7Lt2D8Q+xwAmmufQd8ooOLzqqm+/kxHBuqZdVt3vvluEM9rvyV/y0K8+05xWDz1tZ/rGlv0c1Xbc5oF3q6urvTs2bPc4mzTYu2YmJhKn/fGG28wc+ZMli1bRq9eva74OqdPn+bs2bOEh4dbJW4hRBVklJUNqO06I9PVaWl7a9fOpY6tV6+C8wyCZleZHy4pNfL9lpMMfW89y/eloddpeHJoW378TwwtQv1g9EdqLabEP2HfLxW3fe4E/DABlFLofDv0fdC6sddTQzuGoe02lmJFh3fmHs4e2e7okEQ95jTJEsD06dP57LPPWLBgAQkJCTz44IPk5eUxceJEACZMmFBuAfjs2bN5/vnnmT9/PlFRUaSmppKamkpubi4Aubm5PPnkk2zatInjx4+zatUqRo0aRatWrRg6tGEXghPCrkxlA4JqmSyZrojLOglFObVr62L7f1Vv298IOhcURWHFvlSGvf8PM37ZQ0ZuMW1Dvfl16lVMva4VLrqyj9bQjnDNE+r9v56EvIzy7RbnqVW/CzIhvBvcNEfqKlXD46OvYrNrNABbfplDqdHO+wKKBsNppuEA7rjjDs6cOcMLL7xAamoq3bp1Y9myZeZF3ydPnkR70U7lH3/8McXFxdx6663l2nnxxRf53//+h06nY/fu3SxYsICsrCwiIiIYMmQIM2fOrHCaTQhhI+Y94WpYNsDEMwC8wyEnBdITILJP7WMrLYGEpQAY249m0+EM3l15kK3H1TVW/p56pl3fmrv6NsXNpYKLQvpPh/2/q/WfPoqBtsOh3Uhofg38+hCk7QGvYHWaribFOBswd72OlkMehD830jd3JR+v3M+0IR2v/EQhqsmpkiWAadOmMW3atAqPrV27ttz3x48ft9iWh4cHy5fL/kJCOJx5Gq5d7dsK6aAmS2n7rJMsHf8HCjIp0Psx8pcSjmZuBsBdr+W+q5rzwICW+LhbWPfg4go3z4NvboG8dNixQP3Suaob5Wr1cPvX4Nuk9rE2QBE9R5K/Kgz/wlQS131PXMvHiGlZi3VvQlTAqabhhBD1kKFAXbcDtZ+GgwuVvGuxoW5JqZHdp7OYv+EYa5d8BsCSgh4czSyikZsLd/VtytonruOpYe0sJ0om4V3gsf1w1y/Q+37wjlATJYARb6rVv0XNaHV49rkbgNu0a3lk0U4ycouu8CQhqsfpRpaEEPVMxiFAAQ9/8AqqfXuhnQAoSt7DqfRcSoxGDCUKBqPRvKbFtM9tqVHhfEExGbnFnM0t5mxeEYfScok/lUWBoRQdpWxxWw8aOBQ0iDf7d2Fkl3A8XWvw0eniCq0Gql8j3oKUeDVRbNav9u+5oes2Hta/SX/dXvQ5p3lscTwLJvZBq5X1X8I6JFkSQjiWeZuTtjVe3HzybD7/HskgISWbgpMG3gQKTu9h0DtrgZq16ePuwviQ0wSm51DiHsCL//2P9bYf0Wggort12hIQ0ByaX4P22HrGuv7D24eC+WD1YR4Z1NrRkYl6QpIlIYRjnal52YBDaTnMXXOY33clY7oQyhVvZrlp8dPk0do9h3MuQeh1WvQ6LTqtpnzqpAE/Dz2BjdwIauRKoJcbjf096NHUn9YhjdCuehHSwaXdcNmnra7rPgGOrec+r428Uzya91YdpFtTP65tE+zoyEQ9IL/9QgjHMpUNqEaytC/5PB+uPsyyfanmKbU+zQPoFulH+3BvSte3xOXcIWLHB0HrwTWP7dg/6m3za2rehrCP9jeAuy9eBck81z6dmQlhPLpoJ0sfvprGfnKVYZ1ValDnxSvamLoOkWRJCOFYF0/DXYGiKHyw+jDvxB40PzasYxjTrm9Fp8a+F0483BnOHVIXedc0WSrMVtcVAURdXbM2hP3oPdSinls/416Pf1jSeBJ7k7KZ+u0OfvhPDK4ucj1TnZGbDodWQOLfcGQNuHnDAxugUd0dBZSfHiFqIuskLL4bVs10dCTOrbQEzh5R719hZKmk1MizS/aaE6WRXcJZ/ug1zLu7Z/lECS66Iq4WlbxPxoFihIAW4Nu45u0I++kxAQDdgaV8elMYvh564k9l8eqf+x0cmADU3/UvhsBbbeC3qXBgKRjyIDcV/nnL0dFZJMmSENV1dB18ci0k/K7+gqcfcHREzuvcMTAaQO9lsc5QQXEpD3yzne+3nESjgZmjOjL3zh60Datks1nT4unT22oe2/GyKbio/jVvQ9hXeBd1FNBoICLhS969oysAC+JO8Ft8koODE2z+BE5tBhS1Yv2AGXDDu+qxrV9cKCFSB0myJERVKQps/AC+Hq1uT6Epq9a8bb5Dw3Jq5m1OWld6JVxmXjHjPtvEyoR03Fy0fDy+J3fHRFlut3HZPpDnjkHumZrFZlqvFCXrlZxK/8fU2+1fcX1TPdOuawXA0z/vZn9y1XaYFzagKHCorAj0rV/Cf9bBgGeg133QYoD6R9Pa1x0aoiWSLAlRFcV58PMkWPGcOjXT9U6442v12K5F6nFRfVe4Ei6vqIRxn24i/lQWvh56vr0/mmGdwq7croffhWrgp7dWP66CLEjdrd6XkSXn0vJ6COuiTu9s+YzHBrfhmjbBFBqM/OebbWTlFzs6wobp7BE4d1ytWN96SPljA19Qb3d9D2l1c8pUkiUhqmL1q7D3Z9C6qAUFR38EbYaDf3MoOg97K9lRXlhmWtxdQbKkKArP/bqXxLQcQrzd+PnBGHpFBVS97Sa91dvTW6ofl2m9UmAr8Amv/vOF42g0F0aXNs9DV5LPnLHdaBrgyanMAv77/U7ZcNcRDq1Qb6OuArdG5Y817gntbwIUWP2K3UOrCkmWhKiKw7Hq7ai50Gey+oGs1UKvierjMhVXM+ZpuMuTpR+2nWLJziS0Gvjwzh60CqlkfVJlzMlSDdYtmafg5Co4p9RhlPqHTEEm7FiIn6crn9zdEw+9jn8OZfDWikRHR9jwmJKlS0eVTK5/HjRaSPwTTtVgNNjGJFkS4kryMy+MgLS65DL0buPVDVGTd0DyTvvH5syMxrKtTrhsZCkhJZsXflP3dnt8SFv6NK/GiJKJaRPdpO3qVXfVIYu7nZtWB1c9ot7f+CGUFNM+3IfZt3YB4OO1R/hrT4oDA2xginLhxL/q/Us/Q02C26ifpwAr/3dhT6I6QpIlIa7ENDIR2Bq8LtnN3CtI/SsWZHSpurJPgyFfXcPg39z8cG5RCVO/3UFRiZFr2wTz4LUta9Z+UFtw81VfI70am+rmZ0LqHvW+jCw5r67joFGo+nO29ycAbuoawf391Z+1J37cRWJqjiMjbDiOrVc3jvZrpl7MUZkBz4DODU5sgMOr7BdfFUiyJMSVnNqs3ppGKi7V6z71ds9PUHjePjHVB6bF3UGtzVuJKIrCs7/s4WhGHmE+7rx7R7eab4aq1UKTnur9U9VYt3RiI6BAUBvwDq3ZawvH07tD34fU+xveU0cygWeGt6Nfy0Dyi0uZvFAWfNvFxVNwlvZ/9G2iLnMA2PCu7eOqBkmWhLgS0wLhypKlpjEQ3F4dwdi12H5xObsKtjn5NT6J33clo9Nq+ODO7gR41XILhCZl/2bVuSLu+Ab1VkaVnF+v+9TRxYxE2L0IABedlg/v7EETfw9OZubLgm9bUxQ4VLbms7L1SheLmaqWZTmxAVJrUVTWyiRZEsKS0hI4vV29Hxld8TkazYXRpW3z69xce51lTpbUS/xzi0p47S/1sUcHtqZ3da58q0xk2SLv6owsmdYrNZdkyem5+0C//6r3l043rysM8HLl07t74a7X8s+hDN5YLoVlbSY9QZ0KdXGv2hpAnwhof6N6f+tnto2tGiRZEsKS9H1qvRY3X8t7l3W9A/SecCZBvexcXNklNZY+WH2IMzlFRAV6MuXaFtZ5jeoWp8zPvLBFSjNZ3F0vXD1dXVRcUgCLxkNOGgAdInx481a1wvcn647y+65kR0ZZf5lLBlwNrp5Ve06fKert7h+g4Jxt4qomSZaEsMQ0ItGkl7oGpjLuvhcWeif+bfu4nJ2iXJQsteNYRh7zNxwD4PkbOuDmorPO63j4XUhyqzIVZ5qCC25fpzf1FNWg1cGtX6gXaGQnwQ93Q0kRADd2jeCBsgsInvppF/uSZc2h1VVnCs6kWT8I6agubYj/zjZxVZMkS0JYYkqWKpuCu5hpiLk2+5E1FDmpUJStrk0IaMkrS/djKFW4tk0w17cLse5rmabiqpQsScmAesndF8YtUkeIT22GP6ebp8ufHNqWa8sqfD/wzXZZ8G1NhecvjLS3rqRkQEU0mgsLvbd8phaIdTBJloSwxHwlXO8rn2taTJy8E0oNtoupHtBklI0qBbZk7ZEsVh1Ix0Wr4fkbOqCxdLVMTVR1kbeiwNG16n1Zr1T/BLWC2+arhQ93fqNWii44h06rYc7Y7uYK3w8vipcF39ZyZA0opeqoXkDzK59/sS63q0nuuWNojqy2TXzVIMmSEJXJSYWsE4DmwtoXSwJbgbufujbCVKdHVMiULBmD2vLyUnUvqHv6RdEqpJGlp9VMVYtTnt6qFh91cYfmsnluvdRqEAx+Wb3/z1vwVhv4YQK+p1byyZ1daKQ3cuDgQb5Z8ps6fbT/9/JfR1ZXv8BpQ1aTKTgTVy/ofjcA2m2fWzGomnGpyZNKSkpYu3YtR44c4c4778Tb25vk5GR8fHxo1MgGH3ZCOIJpCi60o3pVzZVoterapsMr1am4xj1sG58zK0uWdhWGcvRMHoFerjw80EKxutowFacsOq8u2A/vWvF5W79QbzuNAQ9/28QiHC9mmvof8ZbP1Z+H/b/B/t9o7+LOXl0h6IA9ZV8ViegBN8+rdPNnUUZR1M9CqN4U3MV63Qdxc9EcWYVXh6HWi60Gqj2ydOLECTp37syoUaOYOnUqZ86oV5jMnj2bJ554wuoBNnTuxWcdHULDdaX6ShUxT/nUYPPWBkRTtn3M98fVq2OeHNoWXw+9bV6sKsUp887CvrLNkHtPsk0com4wlfp4aCM8sEFNnrxCoKQQACNa0hQ/EpQoCsJ6Q2TfC19uvurWRvOuhn/ngLHUwW+mDsvLgNxUQFO1NZ8VCWwJrQejQSHqzEqrhldd1R5ZeuSRR+jVqxe7du0iMPDC1g8333wzkydPtmpwDVrheXRLHuT6Q6tg4GAIiHR0RA2P+Uq4aiRLNanr09AoCpqyGkt7isJpHdKI23rZ+Oe7SR91CuX01gsLRy+282t1O4aI7uoO6KJhCOusfg16SS0v4e5HqZsfD8/fxuZjmbTM8+L3af3xciv7rzI7GX5/WN1YO/Z5dAl/4OV9q2PfQ11lWpfo17TqJQMq0mcKHFpBs8x/oDgP9H5WCa+6qj2y9M8///Dcc8/h6lq+sm5UVBRJSUlWC6zBc20EOSnojYXoVr3g6GganpKiCxvjVmdkqXFPQKOudcpNt0lozs6tJBtNYRZGNBxVwnlkUGt0Nd3SpKqaWEhijaUX9vXrJaNKDZLORd12p1Ewer2eueN7EObjzpEzeTz3614UU6FZnwgY/yPc9AG4eqM9vYX+h16FgiyHhl8nmTYfr+10ZcuBKP7N0ZUWoalOJX4rq3ayZDQaKS29fOjx9OnTeHt7WyUoAWh1lA57AwUN2n2/wNF1jo6oYUnZrY40eAZBQDUKJLr7mitSV2uLjQbEu1D9o+qkMYSmIQGM6BRu+xc1TcOdOwZp+8sfO7xKTW7dfdX1SqLBC2rkxpxx3dFpNSzZmcSP205fOKjRQI8J8FAcSkBL3EvOo/3nTccFW1edKUuWgtrUrh2tltKbPiK20zsoLQbUOqwah1HdJwwZMoT33nvP/L1GoyE3N5cXX3yRESNGWDM2Ed6NY0ED1ft/PQElUv/Dbi7ePLe6l7LLVJxFbvlqpeRDSmMeGdS65hvlVoeHP5g+aL+9FbJOXji2rWxhd7e7ajddIOqVPs0DmD5Y/Y/+hd/3cjAtp/wJfpGUDnsDKLta69IkvKEzTcPVNlkClCa9KdQ79qKLaidLb7/9Nv/++y8dOnSgsLCQO++80zwFN3v2bFvE2KAlhI9B8QpWhzTjPnR0OA3HxclSdZmmfKQ4ZYVyM9WRpbMeze0zqmQyZr56ZVx2EiwcrU6TnjsBB5erx037+wlR5sFrW3JNWcHKh77dQX5x+bIBSvNrSfbthUYphWVPy76QFztjpWm4OqLayVKTJk3YtWsXzz77LI899hjdu3fn9ddfZ+fOnYSEWLnyrqDExYvSgS+p36x7o/xfxMI2FKV6lbsvZS5OuUNqslwiu8CAe34KAG0797LPqJKJVyDcvQR8m0LmEfj6Fvj3fUBRR52CWtkvFuEUtFoN79zelVAfNw6n5/L8r/suO2dv43EoLu5wbL1ahkBAUa66eS5YZWSpLqhRUUoXFxfuuusu3njjDT766CPuv/9+PDw8rB2bKKN0ug2aXaUWO1w2w9Hh1H/ZSeolr1oX9eqo6gpqo65/MeRf2JRVALAg7iQtNerIUtfufe0fgG9jmPCreql42p4LU3C977d/LMIpBDVyY87Y7mg18POO0yzZebrc8QK3YIx9p6nfrHgOivMdEGUdc/aQeusVDJ4Bjo3FSqpdOmDhwoUWj0+YMKHGwYhKaDQw8m2Y1x8OLFU3am073NFR1V+mqzgCWoK+Bn8EaLVqxe8jq9RF3hHdrBqeszpfYOCXuL1M16iblWqDHfQXZ2BLuPsX+HKkWqjSOwLayO+TqFx0i0AeHdSGd2IP8sJv+4huHkiE34XPBmO/h9HtWQznT8G/78F1zzou2LrAWou765Aa1Vm6mMFgID8/H1dXVzw9PSVZspWQ9tD3Idg4B36aBLcvhNaDHB1V/XT2iHob2LLmbTTpfSFZqqiuTwO0YONxwopOgBsovpFo3BxY7T+sM9z1E/z9FPSdql46LoQFDw1oyZrEdHaezOKpn3az8L6L1jPqPWHIK/DjPbDhPeh2J/hHOSpUx7Pi4u66otrTcOfOnSv3lZubS2JiIv379+f777+3RYzCZMAMdW2FIQ++ux12fO3oiOqnzGPqbXVKBlyqOjvdNwD5xSV8+e8xWmvVKTglqA4s+ozsA1PWQpfbHB2JcAIuOi1v39YVd72WDYcz+GbzifIndBgFUVdDaRFsbOAX45wpS5bqyeJusNJGuq1bt+b111+/bNRJWJmrJ9z5I3QZq+7k/Ps0WPu6XIFhbZlH1dvq7pJ9MdPGu5lH1bL/DdwPW09xLt9Ad/c0AJR69BenaDhaBDfimWFqHbXX/krg+Nm8Cwc1Grh6unp/zw9gKHBAhHVERv2bhrNKsgTqou/k5GRrNVepuXPnEhUVhbu7O9HR0WzZYrmWzY8//ki7du1wd3enc+fO/PXXX+WOK4rCCy+8QHh4OB4eHgwaNIhDhw7Z8i3Ujouruolj/7JfyrWz4Pf/QqnBsXHVJ+ZkqRYjSx5+6mXq0OBHlwylRj77Rx2t6++rJo51YmRJiBqYEBPFVa0CKTQYefLnvZRe/Ldq8wHq9h6F5xvulXGlhgufoQ15ZOn3338v9/Xbb78xb9487rrrLq666ipbxGi2ePFipk+fzosvvsiOHTvo2rUrQ4cOJT294m0lNm7cyLhx45g0aRI7d+5k9OjRjB49mr17L1yh9MYbbzBnzhzmzZvH5s2b8fLyYujQoRQWFtr0vVRFoaGSTRo1Ghj0orroW6NV97VaOFpGMKzBWKpWeQZ1gXdtyFQcAH/sSiYpq4CgRm6EFZdNXdSjD1HRsGi1Gt68tSvebi7EnzrPqiTNxQehe9m63e0LHBOgo2UeA2MJ6L3Ap7HVmjU6eAKl2smSKeEwfd1yyy3873//o0uXLsyfP98WMZq98847TJ48mYkTJ9KhQwfmzZuHp6dnpa/7/vvvM2zYMJ588knat2/PzJkz6dGjBx9+qM4nK4rCe++9x3PPPceoUaPo0qULCxcuJDk5mV9//dWm76Uqnvx5L7N36fhswzFSzlcwpNv7fhj7Pbh6w4kN8OkAdZsOUXPZSeo2J1o9+DapXVumeksNuJK30agwb526YP6BvoFoc9UaS0pg/RmeFw1PhJ8HL97UEYBlp7Ucy7hoOq77ePWP2JMbL1wV1pCYF3e3rv7uBxc5n29gzYF03lqeyF3zt/LsVh0FxZUMINhBtS8BMRqNtojjioqLi9m+fTszZlyoM6TVahk0aBBxcXEVPicuLo7p06eXe2zo0KHmROjYsWOkpqYyaNCFq8p8fX2Jjo4mLi6OsWPHVthuUVERRUVF5u+zs7MB9cpAg8E602FFhlI2HM4gt0jDG8sP8eaKQ/SJ8uemLuHc0CUMT9eyf7oWA2Hiclx+vAtN5lGUL4ZQeuMclA43WyWOhsD0b2YwGNCkH8IFUPybUVJqhNJa/LyHdUcPKEk7KCkqBK3OKvE6k1UH0jmYlksjNxdui8wFoEDvj6LzACv9rojLXfwzLWzjps4h/LYzgH8OZ/LC7/tZOLEXGo0GPILRtRqM9tBySrd/hdFUVLiB0KYloAOMga0prcHP39bj53j5zwMcSL1kexk07Dxxln6trVv8uqq/I05zvWxGRgalpaWEhoaWezw0NJQDBw5U+JzU1NQKz09NTTUfNz1W2TkVmTVrFi+9dPkvwIoVK/D0tN7eUs92hvhMDdvPaDmSo2HzsXNsPnaO95fvY0q7UoIvKgHk0vhJehk+JjRnNy5LJrNv80oOh460WiwNQWxsLFEZq+kKpBkasfmS9W3VphgZqXHFxZDHul+/Is/djlt71AGKAu/t1QEaogOLOfrvcroDOe4RxMXGOjq8BiFW+tmmBnjDJo2OTcfOMXPhMnoFq3NFYaXtiWY5JdsWsrygJ4rWaf6rrbUex9cQCRw4q3Comp+hOzM0fH1YS6mijkgFuys0977wlXlwG39ZeUlxfn7ViohW6V/w0tEZS955550qn+usZsyYUa5PsrOziYyMZMiQIfj4+FjtdQwGA16xsfzvrutJzyth6e5Uvt50krScIj5IdOfDsd3o2+Ki6qjGWyhd+wq6uA/okPwDba8fh9Ksv9Xiqa8MBgOxsbEMHjwYt3Wb4BQEt4tmxODabwytTe8MydsZ0C4ApUPD2mh66/FzHN+0FVcXLS/fdS1hG9fCSchxb8LgwYPR6/WODrHeuvhnWvrZdgwGA9szVvLXKR1/p3rw6O1X4eOhB+MQlA8W4ZabyoiWCkr7hvO7r/vibQDa9LuR1u2q9r4VRWH+xhN8FadOWw5uH8LMm9oT2MgNsO3Ps2lm6EqqlCzt3LmzSo1pajE/eSVBQUHodDrS0tLKPZ6WlkZYWFiFzwkLC7N4vuk2LS2N8PDwcud069at0ljc3Nxwc3O77HG9Xm+TDya9Xk9UsCfTBvpwe5+mTFm4nfhTWUxcsJ2Zozsxrk9T05kw9BUozEKz82tcfnsQHvhX3RNLXJFer0eXpS5A1gW1RmeNf8vwLpC8HZcz+0F/e+3bcyKfbTgOwK09m9A4oBEkbwcg06slTW30uyLKs9VnkrhgYITCgQJPjmbk8/6ao7w8qhOgh+53wT9v4RL/DXS51dFh2oeiwNnDALiEdYAq/OyVGhVmLt3PVxuPA3Bvvyiev6EDugr2jbTFz3NV26vSAu81a9ZU6Wv16tW1CtoSV1dXevbsyapVq8yPGY1GVq1aRUxMTIXPiYmJKXc+qMPSpvObN29OWFhYuXOys7PZvHlzpW06Woi3O4um9OWmrhGUGBVm/LKHl//Yj/HiSwWGz1brW+SkwG9TpQ5TdVijbMDFwruotw1s4f3+5GzWJJ5Bq4EpV7dQa86k7gHgnJdsWCvqDxct/O/G9gB8vekEu09nqQe636XeHl0D5447JDa7y05SiyZrXapUp05RFB5dHG9OlJ4b2Z4Xb6w4UXI0q9VZsofp06fz2WefsWDBAhISEnjwwQfJy8tj4sSJgLov3cULwB955BGWLVvG22+/zYEDB/jf//7Htm3bmDZN3fRQo9Hw6KOP8sorr/D777+zZ88eJkyYQEREBKNHj3bEW6wSd72O98d2Y/pg9Yqi+f8e460ViRdOcPWCW+eDzhUO/g1bPnVQpE5GMV5UNsBKyVJYWbJUlig0FKYr4EZ0DicqyAtSdoGxBMUrhAK9jHSK+iWmRSCju0WgKPB/S/ZSalTUZKHFAPWEnd84ND67MVXuDmgBuiuP2PyyI4k/diXjqtMy984e3H91C5vOUNVGjVadbdu2jR9++IGTJ09SXFxc7tgvv/xilcAqcscdd3DmzBleeOEFUlNT6datG8uWLTMv0D558iRa7YX8r1+/fnz33Xc899xzPPvss7Ru3Zpff/2VTp06mc956qmnyMvLY8qUKWRlZdG/f3+WLVuGu7u7zd6HNWg0Gh4e2JowH3ee+nk3H609QquQRtzSo+xy97DO6l5Ffz+l7oTdNObCKIeoWE4KlBSqfxX5RlqnzZAO6mXEeemQkwreFU8Z1ycnzuaxdLdaoPbBAWW1qspqTSmNe9XqcmIh6qr/G9mBVQfS2ZN0nu+3nOSuvs2gxz1wdK2aLA2YUf+viK1G5e70nEJeXrofgEcHt2Zkl7p9AUy1R5YWLVpEv379SEhIYMmSJRgMBvbt28fq1avx9fW1RYzlTJs2jRMnTlBUVMTmzZuJjo42H1u7di1fffVVufNvu+02EhMTKSoqYu/evYwYUX7BmUaj4eWXXyY1NZXCwkJWrlxJmzbOUwPm9t6RPFT2H9IzP+9h2/HMCwf7TFF3Uy8thp/ug+KqrfpvqDSmKTi/ZtbbWNXVEwJbq/cbyOjSJ+uPYlTg2jbBdIwo+0w4vQ0ApXFPB0YmhO0Ee7vxeNlo/3srD5JbVALtRoKHv/qH2NG1jg3QHqqxJ9yLv+3jfIGBTo191Kn6Oq7aydJrr73Gu+++yx9//IGrqyvvv/8+Bw4c4Pbbb6dp06ZXbkBY3RND2jKsYxjFpUb+8/V2TmWWJUUaDYyaC97hcPYQ7Fjo2EDrOlOyFFjLyt2XCuus3qbW/3VL6dmF/LTtNIA5iQckWRINwvi+zWge5EVGbjGfrjsCLm7QaYx6cNcixwZnDxll1/VfYTujv/ek8PfeVFy0Gt4Y0xUXXd1fEVTtCI8cOcLIkWr9HldXV/Ly8tBoNDz22GN8+qmsjXEErVbDO3d0pWOED2fzirl/wTZyCssKbXkFwjVPqvc3faRu5yEqpLH2eiWTBrTI+4t/j1FcaqRHUz/6NC8ra5GdDNmnQaNFCe/m0PiEsCW9TsvTw9RE4bN/jpGWXQhdx6kHE/6AoksLLdYzF1fvrsS5vGKe/20foE7Td4iwXrkdW6p2suTv709OjvoP3rhxY/M+a1lZWVUu7iSsz9PVhc/v6UWItxuJaTk8/sMuFNNVcF3HgUcAZJ1Qf2FFhWyWLJlHlur3NNz5AgPfbjoJwEMDWl1YqFk2qkRIR3Bt5KDohLCPoR3D6NnMnwJDKe+sOAiNe6pT8SUF9Xtz3fxMyDuj3rewZmnm0v1k5BbROqQR0653nitjq5wsmZKia665xlwV9rbbbuORRx5h8uTJjBs3joEDB9omSlEl4b4efDahF646LSv2p7Fo6yn1gKunuo8cQNyHjguwjtNYawPdS5muiMs8Uq//svxm0wlyi0poG+rN9e0u2pLAtJFwk16OCUwIO9JoNDw7oh0AP24/RWJaLnQt2zqrPk/FmRZ3+zQBt4r/KPrn0Bl+2ZmERgOzb+2Cm4vzLHivcrLUpUsXoqOj6dy5M7fddhsA//d//8f06dNJS0tjzJgxfPHFFzYLVFRN10g/nhiqZvUv/7H/wgaPve9XSwmc3gonNzswwjpKMaq7ZUOV6oNUi1cQeEeo99P2WbftOqKguJT5G9T+e2BAC7QX10kxjSw16e2AyISwv57NAhjeKQyjAq//nQBd7lAPHP8Hsk46NjhbMS/urnhUSVEU3o1VE6p7YqLo0dTfXpFZRZWTpXXr1tGxY0dmzZpF+/btueeee/j333955pln+P3333n77bfx93euN19f3d+/BTEtAikwlPLoop0YSo3gHQpdyipIx33g2ADrIHdDFpqSAtDowM8GFyqYpuLq6bqlxVtPcjavmCb+HtzYJeLCgVIDJJftACDJkmhAnhrWDhethjWJZ9iY4QFRV6sHdi92bGC2kqnWViOw4qm1zccy2XEyC1cXbfmLP5xElZOlq6++mvnz55OSksIHH3zA8ePHufbaa2nTpg2zZ8+2uPGssC+tVsPbt3fFx92FXafP88GqsisUYtRinCQsvXDllwDAqyhdvePXtErF1KrNtMi7Hl4RV2go5eOyIpT/ubZl+Stb0vapazXcfSv9EBWiPmoe5MX4aPUPr9f+TkC5eCquPu6qYBox84+q8PDcNeo2KLf3akKIT92uY1iRai/w9vLyYuLEiaxbt46DBw9y2223MXfuXJo2bcpNN91kixhFDUT4efDaLepoxodrDqv1l0LaQ6tBgAKbPnZsgHWMV3HZHoLWLhtgUo/LByzeeoq07CLCfd25vVeT8gdN65Ua9wRt3b88WAhrenhga7xcdexNyma1pi/oPdW905K2Ozo06zun7quJX7PLDu05fZ5/DmWg02r4zzXON6oEtdzupFWrVjz77LM899xzeHt78+eff1orLmEFN3SJ4JYejTEq8NgP8Wo5AdPo0s5v1KsXBACNCstGRq9wJZzRqLAv+TzzNxxj6rc7mPHLHtYfPKNOdVpiWuSdnqBOTdUThYZSPlqr/sX40HWtLl+wKeuVRAMW2MiNe/pFAfDO+hSU9jeoB+K/c1xQtpJlSpYuX8Zg+oy4qWsEkQGe9ozKampcpnj9+vXMnz+fn3/+Ga1Wy+23386kSZOsGZuwgpdu6siWY5mcyizg9b8P8OroARDaCdL2wvYv4erHHR1ineBVXDYNV0mytO14JvPWHWXLsbNkF5aUO/b9lpP4e+oZ1imMG7pE0K9l4OX7G/k1AzcfKMpWF0KGdaI+WLTlJGnZRURUNKoEkCTJkmjY7r+6BQs2HmdfcjY7Og2nJz/A3p9h2Cy1aGV9UJQL+WfV+/7lR5YOp+eybJ/6x+iDTrhWyaRaI0vJycm89tprtGnThgEDBnD48GHmzJlDcnIyn332GX379rVVnKKGvN31vHlrVwC+3XySjUfOQsxU9eCOhfVz7rwGvIrKpuEuKRtgNCrMW3eEOz7dxMqENLILS/By1XFtm2CeHNqW8dFNCfRy5Vy+ge+3nGL855t5/IddFJdcMtKk1apJKtSbekvqqJK6VqnCUaX8THXKAdRpOCEaoAAvVyaUjS79b08Ainc4FGbBwWUOjcuqTOuV3P3U9YkXmbfuCIoCgzuE0ibU2/6xWUmVR5aGDx/OypUrCQoKYsKECdx33320bXvl/V+E48W0DOSuvk35ZtNJnvp5N8sfGoGXizucO64uwK0noxw1pigXJUsXRpay8ot5/IddrDqgjjqN6hbBfVc1p2OET7lFzC/d1JHNxzJZujuZH7ad5pedSaRmFzLv7p74uF+0WDy8C5zcWJYsjbPHO7OpRVtOkp6jjirdVuGoUtm6jMBW4Blg3+CEqEMml40u7UnJ43i3G2l+4FPY+gV0GOXo0Kyjkim4pKwCft2ZBOCUV8BdrMojS3q9np9++onTp08ze/ZsSZSczDPD29PYz4PT5wp4Y/VpaFlWQFQqekNuGi7GIhSN1vzLHn8qi5FzNrDqQDquLlpm3dKZ9+7oRtdIv8v2MXLRabmqVRCzbunCF/f0wstVx8YjZ7nt4ziSswounFiPFnlfcVQJLipGKVNwomEL8HI1r116ObWv+llzbB2k7XdsYNZivhKu/BTcZ+uPUmJUuKpVIN2drK7SpaqcLP3++++MGjUKnc55Km6KCxq5uTB7jLrIeEHcCQ4HXa8eSPjdgVHVDebK3b6R4OLKpqNnuf2TOJKyCmgW6MmSh/oxrk/Ty9chVWBA2xB+eCDGvO3M6Ln/sjfpvHow7KLyAU4+/XnFUSWQyt1CXGTy1S3wdNWxJtWdM40HqQ9unufYoKylgivhzhcYWLT1wvZHzk6u5W1A+rcOYlyfSAAe2RmKonWB9P2QcdjBkTlYWbKkBLQgISWbyQu2UVxi5Lq2wfzx3/50jPC9QgPldYzwZcnUq2gT2oj0nCLGf76Zk2fzIbgdaPVQeN6pq/jmF5dceVSppBhOl03DyciSEOrapZgoAN7NKRvZ3/1D/bgqOevyZOnXnUkUGoy0DfWmX8tABwVmPZIsNTAzRrQn3NedfZlajjTqoT54oGFPxWnKCnTmekZyz/wt5BSV0CcqgI/vumTNUTU09vPgpwf70TXSj/MFBv7zzXYKjDoIUfeMcuZF3p//c4z0nCIa+3lUPqp0cBkUnYdGoeoGukIIJl/dHA+9ju/TmpDj114t2LpjgaPDqj3TyFLZNJyiKHy/Rf2D8M7oqo3K13WSLDUwPu56ZpUVq5x/tmwNTQNft2SahvvmoI70nCLahnrz2T29cNfXbsrZx13PvLt6ENTIlYSUbJ5dsgfFvO3JrtqG7RDp2YXMK6vW/fTwdpVvhLnza/W26zjQ1bhCiRD1SmAjNybENAM0LDQOVx/c8jmUllh8Xp1nGikvG1naeSqLA6k5uLloGd29sQMDsx5JlhqgAW1DGNs7ktjSnhjRqFctnT/tuIAURb0yZNciMBTa/+XLNtDdkRtAhK87X93XG18P62x5Eu7rwQfjeqDTaliyM4ktheo0qLMmS+/EHiS/uJRukX7c2CW84pOyk+HwSvV+97vtF5wQTmBS/+a46rTMSe+CwT0Qsk/DgaWODqvmCs6po8hgvkDmu81q8nRDlwirfZY6miRLDdT/jWyPq18424xlO0QnOPCXNfFv+HM6LPkPvNsBVr5kt+TNWGqkOF1ds5XpGsHCSX0I9/Ww6mvEtAxkxnB1+u2tPWXVa1Pirfoa9nAgNZsftp0C4Pkb2lc+tB7/LShGaNoPgpx/YacQ1hTi484tPRpThCvL3MtGl5x5obdpCs4rGFw9OV9gYOnuZECdgqsvJFlqoNRilV1YXqouvs3a8bPjgtlVVvpf56pWgd3wDrzXBX6aZPORpvkrt+Oh5APw9NghtAqxTdG0Sf2bc2PXCPaUNqUULeSmQXaKTV7LVl776wBGBUZ0DqNns0rqJhmN6lY6AD1kVEmIiky+pgUaDcxMjVEvtDkZB8nxjg6rZi6Zgvst/sLC7h5N/RwXl5VJstSA9WsVhGe3mwHwTt9Gztlk+weRnwmJZZVs718Ft38NUVeDUgp7f1K3ZLGRNQfS+XPdRgDOa/3p0bKSaSUr0Gg0zB7TmWahQRw2RgCgJO+02etZ27qDZ1h/8Ax6nYanh7Wr/MQTG9Rip67e9afgnhBW1jK4EUM6hJKOPzu9B6gPOuvoUtaFxd2Kopin4Mb1iawXC7tNJFlq4B4cNYAD2pboMLLsZ9slJpXa+zMYDWoNovAu0OEmuHcpDHxBPW5a+2JlxzPyeHjRTiJRK3cbPENs8joX83R1Ye74HuxHrRK+e+s6m7+mNZQaFWb9lQDAhJgomgV6VX7yjrKF3Z3HgKuF84Ro4B64Vq1o/drZa9UH9vwEWaccGFENnbtQvfvihd03d6/kSlknJclSA+fp6kKjstGl4FPLWbbXzlNDu75Xb7tesv1H2xHq7fENYCjAmnKLSpjy9TZyCkuI8c8GIM/V9skSQKuQRoS3V/dQzDy8hWMZeXZ53dpYvPUUB1Jz8PXQ89/rLaxBKsi6UOS0+wS7xCaEs+re1J8+zQPYVtKSE9491D8a/3nL0WFV30XTcN9fvLDbs34s7DaRZEnQJOYOAPpp9/LyT3FqAUV7OHNQvRJPo4POt5U/FtwOvCOgpBBO/Gu1l1QUhSd/3MXBtFxCvN0Y3awYgHy3UKu9xpX0iVGrp3fgKNN/iKek1HiFZzhOWnYhs/5WR5UeHdQaP0/Xyk/e86P67xXSARr3sFOEQjivB8tGl57LHq0+sPMbKLs612mUTcPleTbhD/PC7khHRmQTkiwJCG6DEtQWV00pPYp3MPW7HRSVlNr+dU2jSq0HQ6Pg8sc0GmhVVuX28GqrveT7qw7x995U9DoNH9/VA49c9S+hPDf7jCwBaCO6oGi0hGqyOH3ymLluUV2jKArP/7qXnMISukb6masPV8pUW6n73eq/nxDCogFtg2kb6s0/Ra046d8XjCWw3olGlxTFPLIUm+xKocFIm9BG9HDyfeAqIsmSAEDTSt2r6HrXfexJOs+svw7Y9gWNRti9WL1/6RSciTlZss66pT92JfPeykMAzBzVSb2iq6x6t72m4QBw9UITpJZs6Kw9xnsrD13YP64O+XtvKiv2p+GiVRen67QWEqCUXeqXVg9d7rBfkEI4MY1Gw3+uVdcwPp9ddkHEru/hbN38A+oyeWfAkA9oWJigjpCP7V0/KnZfSpIloWp5HQAjPA8ACl9tPM5fe2y4fun4eshOAndfaDOs4nNaDACNFjISa73wcdepLJ74US0EeX//5ozt0xSK89RL+LHvyBIA4d0AuDn0DCVGhccWx1NosMNoXhVl5Rfzwm97AXX/t3ZhPpWfnLILvi9LeNuNBC/n3wdKCHu5sWsEEb7urMtrRnLINeqVwGtfd3RYVVM2qmTwCmNHUj56nabeVOy+lCRLQtWsH+hccc9PZka0ui7l6Z92c+KsjRYg71qk3nYaA3r3is/x8L+wCeuRVTV+qZTzBUxeuI2iss1xZ4xorx44dxwAxd0Pg0ujGrdfIxHdABgakEpQIzcOpefy+t82Hs2rqryzfPnTr+TnZtMqpBFTr2tZ+bkJS2H+MDXxDWwNQ2baL04h6gG9Tsu9V0UB8Eq+erENe36E9DryeWBJ2WdoikZd83l9uxACvCysa3RikiwJlasXREYDcH/EcXo18yenqITJC7dxPt9g3dcqyoX9ZVdNVTYFZ9KydlNx+cXqezDt+TZnXPcL00llCykV/6gatV0rZSNLrum7eeu2LgB8tfE4aw6k2z+Wi+35idJ3OvDY0cnsd7+Pv0r+g9t3t8BfT8K2+XByMxRmq2sV/nkHFo9Xh+FbXg/3rzRvdyCEqLqxfZrSyM2FvzJCOdNkMKDAOicYXSpb3L0nTx15vq1n/VvYbSLJkrigxQAAdMfW8cGd3QnxduNgWi6TFmyloNiKU0QJf4AhDwJaXhg5qkzZWiqOroPS6iVthYZSHvp2B3uTsgn0cuXze3rh7X7R5axlG+jiiGQprDOggZwUBkQoTCz7y/LJn3ZxJqfI/vEYSyH2Bfh5ErrSQnIUdcsX1/wUOLoWtnwKSx+D+UPg9Uh4qw2sekl9bp8pcOeP4OFn/7iFqAd83PXc0VtNNN4pHqM+uG8JpO51YFRVUDYNd9gQRFAjV65tG3yFJzgvSZbEBWXrlji2nvBGehZO6oOPuwvbTpzjoW+3Y7DWJe57y7ZW6TruyldNRXRTp+OKsuH0tiq/RKGhlP98vZ21iWdw12v55O6eRAZ4lj/JNLLk17wawVuJWyMIaq3eT4nn6WHtaBvqTUZuMU/9tAtFUewXS8E5+PY2+Pd9AD4uuZEbPL8m79HDMCkWRs2Ffv9VR/m8y6qc56WrJR9GvAUj3gSdi/3iFaIemnhVFDqthu9P+nC+xQ3qg/++59CYrqisIOUpJZibuzdGr6u/KUX9fWei+sK7gbufmpgk76BdmA/z7+2Nu17LmsQzPPXTbozGWv4nXmqAE+oWI7QdfuXztTp1igeqvG6p0FDK5IXbWHfwDB56HfPv7U2vqAr2MjvnwGk4ME/FkRyPu17HnHHdcXVR+3ph3An7xJB5DD67Ho6swqB1Y1rxf3mP8Xx0dx+8/IIhsg90vwuGvAJ3/wKPH4CnjsG9f8HDO6DPZPvEKUQ918Tfk+GdwgD4QrlJfXDfkjq9h2Rp5nEATivBjOlZvyp2X0qSJXGBVgfNr1HvH1kDQK+oAD4a3wOdVsOSnUm8+ldC7UY9UnarU3DufmrxwqowTcVVYd1SQXEp9y/Yxj+HMvDQ6/hyYm/6tQyq+ORMB07DgXmRNynqVXptw7x5dri679qrfyWQkJJt+xiWPwuZRynwbMxNBf9jqTGG18d0pmOEb+XP8QyAqKsc129C1FOTr1bLCHx80JviiD5q3aVtXzg4qkoYjXBevUrZM6Sl5Stm6wFJlkR5pqm4o2vND13fLpQ3b1UXIX+x4RhP/7y75kUrT2xQb5v1A20Vf/xMI0vJ8ZCXUelp5/KKmbRgKxsOZ+DpquOrib3p26KSy9hLDeb5dsXfAdNwcGFkKSXe/NA9/aIY0DaY4hIj9y/YRnpOoe1eP20/JP6FgoY78p4gQWnGvf2i6t2eTkI4i66RfvSJCsBQqvCX12j1wW1fgsGGnwM1lZOCTinBoOi4rncXR0djc5IsifJalCVLp7dAUY754Vt6NGHmqI5oNfDDttPc8ckm0rJr8AtsmoJrdlXVn+MdBqGdAcU84nWpuCNnGf7+P2w8chYvVx0L7utDdGWJEqh/ESmloHNT23eE8C6ARr3sPvcMoBape/f2bjQP8iIpq4BJX20jv7jENq+/4V31xiWG3UWh9I7y51lTWQUhhEPcf7X6x9tLh5pj9GkM+Rmw9ycHR3W5Y4f3AZBKIDd2r/9XwTpNspSZmcn48ePx8fHBz8+PSZMmkZuba/H8//73v7Rt2xYPDw+aNm3Kww8/zPnz5SslazSay74WLVpk67dTdwU0V6dXjCVwvPyebHfHRPHVxD74euiJP5XFjR9sYMfJc1Vv21gKJ+LU+1HVSJYAWpWNLl0yFWcoNfLm8gPc+fkmUrMLaRHsxeL/xNC7ojVKFzNNwQU0VwtfOoKbNwSWbUx70eiSv5crX97bG39PPXuSzvPw9/GU1nat2KXOHUcpW2g/O28EId5uzL2zB64uTvORIES9NKh9KM2DvDhXqLAj5Fb1wU3z1HIddUj8nt0A5Hs2trxnZD3hNJ+M48ePZ9++fcTGxrJ06VLWr1/PlClTKj0/OTmZ5ORk3nrrLfbu3ctXX33FsmXLmDRp0mXnfvnll6SkpJi/Ro8ebcN34gRMo0tHLx/FuaZNML9Pu4o2oY1Izyli7Ceb+Pyfo1WblkvbC0XnwdW7bKSoGkz1lo6tMz90OD2H2z+JY+6aIygK3NErkqX/7U+nxhbW25iYywY4aArOxLRuKTm+3MNRQV58fk8vXF20rExIY+bS/VZ92aJ176FRSllf2pnjrm2Yd3dPQnwqKQ4qhLAbrVbDpP7q59Lzp3uiuHhA2h6rbiheW0UlpaSeOAiAd3grB0djH06RLCUkJLBs2TI+//xzoqOj6d+/Px988AGLFi0iOTm5wud06tSJn3/+mRtvvJGWLVty/fXX8+qrr/LHH39QUlJ+WsPPz4+wsDDzl7t7A/9Po6zeUmVTXs0CvfjloasY2jGU4lIjr/yZwMC31/HT9tOWR0BMU3BN+1b/UvMmvdQRoJwU9hw4wJSF2xj0znp2nszC292FD+/szuxbu+DpWsV2Lx5ZcqQK1i2Z9GwWwLu3q8e/2nicz/85apWXzEw7iSb+G7Vd3Ri+vT+6Xm58KYSzurVnEwK9XEnIcuFEkxvVBzd97NigLrJiXxpBBvUqvdCmbRwcjX04RXGUuLg4/Pz86NWrl/mxQYMGodVq2bx5MzfffHOV2jl//jw+Pj64uJR/21OnTuX++++nRYsWPPDAA0ycONHiRoBFRUUUFV0oHJidrV61ZDAYMBisV+3a1JY126ySyH64oEGTkYjh7AnwibjsFDctzLm9C4u3n+bDNUc5fa6AJ37cxcdrD/PI9S25vm0wbnpduefojv2DFiiNjMFYzfeUb9Ci9W6Jd/Yh5ixcTKxR/VkY3D6E/xvRlsZ+HtXqJ93Zo2osPk0d18+AJqQTLoCSvJOSCl5/SPsgnhzSmjdXHOKVPxM4eiaHZ4e3w62G02Wp2YVs+PxFxmFgF2147L4JtAvzstt7d2RfNyTSz/Zhq37WAXf3bcp7qw7zeuY1zOMHlMS/KDlzpE5Uyf9u8wke0arrLBXfJjb/ObPlz3NV23SKZCk1NZWQkPIbnbq4uBAQEEBqamqV2sjIyGDmzJmXTd29/PLLXH/99Xh6erJixQoeeughcnNzefjhhytta9asWbz00kuXPb5ixQo8PT0reEbtxMbGWr3NK7nGszn++UfZ89sHnAq8utLzfIEn28M/qRpWJmk5ciaPhxfvRq9VaO2j0MFfoYOfQqCbwrDD63AD/k1SOPfXXxZfv6gUkvPh4HkNiVlajufCLF0Et7scopv2CNmBPbg+wkiYZzK7Niazq5rvb8DJPfgCWw6fIf2M2r+O6GeX0gJGoEGTncTqX7+m0PXyRemNFRjWRMOy0zq+23Kaf/adYmKbUgKrMQCqKBCfqWHl8QL+1vwNGkhtMpLCnRs4utOKb6iKHNHXDZH0s33Yop9DDOCq1bEsPYDjfh2JKtzHiR//j32Nr7BFlI2lF0DcURfedFOTpY37kzl30vLnubXYop/z8/OrdJ5Dk6VnnnmG2bNnWzwnISGh1q+TnZ3NyJEj6dChA//73//KHXv++efN97t3705eXh5vvvmmxWRpxowZTJ8+vVz7kZGRDBkyBB8f69WaMBgMxMbGMnjwYPR6/ZWfYEVaj52w8V26+Zyj84gRVzx/NJBTaOCLf0/w044k0rKL2J+lYX+WeryneyqjyKVI48YylwH4unqiKFBiVCg1KhhKjZzJKSIpq5CkrALOVbAf3Qn3tlCyjvtaZDHl7mE1f3OKgsveBwDoNeQ2DD7NHNbPAMqZj9GkxDOwpRtK54r7eiSw/lAGT/y0h1N5Bt5LcOeNMZ0Y2C6kwvMvlpiaw8y/DrD52Dmm6WLx1hdQHNCO6yc8e+UK6lbmyJ/phkT62T5s3c8HXA6wcNNJfvK8nScKX6Tl+X9pNvFzcHGz+mtV1RvLD+LCYSI0mQDEDL/9QmV/G7FlP5tmhq7EocnS448/zr333mvxnBYtWhAWFkZ6evkNRktKSsjMzCQszPJl3zk5OQwbNgxvb2+WLFlyxY6Ojo5m5syZFBUV4eZW8Q+km5tbhcf0er1NfmFs1a5FbQbBxnfRHl2LVqerUk2kAL2eJ4e154mh7TiQmsPaxDOsSUxn+4lztDfsAT1sLWnFdzvSqhRCgJcrfVsEcFWrIPq3CqJpYTh89ikeZ3aDi0vN/6PPSVU3f9Vo0Qe1BEVtxyH9DBDVH1LicTm9CXqMr/S0gR3C+fNhP6Z+t4OdJ7N44Nt4ukb6MahdCIM6hNIuzBuNRoOiKKTnFHE4PZdle1P5dvMJjAo0cinlIfdYKAHX654EV8ddweKwvm5gpJ/tw1b9PPmalny75RRzk1vyiH8w+oIz6FO2XVhXamfFJUZ+2ZlMuOYsWoygc0Pv16TqNfNqyRb9XNX2HJosBQcHExx85Y33YmJiyMrKYvv27fTs2ROA1atXYzQaiY6OrvR52dnZDB06FDc3N37//fcqLdyOj4/H39+/0kSpwYiMVq9ay8+AlJ3QuGeVn6rRaGgf7kP7cB8eHNCSvKISSn/8Hg6Db/sBPBrSmnN5xei0Wlx0GrQaDS5aDYGNXGni70kTfw8a+3vg437JD3FJR9Dq1b3Mzh2v+eJs0+Junybg4gqOXtcRdTXEfXhZqYaKRPh5sHhKDK//fYAvNx5j16ksdp3K4u3YgzT28yDY240j6bnkFJW/iGFk53BeancCzz+y1L8CO4y2zXsRQlhNZIAnN3QJ57f4ZLbre9G34G84uMJhyVLs/jTO5hUT7XUeSlHXT9kpUXI0p1iz1L59e4YNG8bkyZOZN28eBoOBadOmMXbsWCIi1MXHSUlJDBw4kIULF9KnTx+ys7MZMmQI+fn5fPPNN2RnZ5uH24KDg9HpdPzxxx+kpaXRt29f3N3diY2N5bXXXuOJJ55w5NutG3R6aHEtHFgKh1ZWK1m6lJerDlI3A9C530g6R9Xw6gkXNwjrBMk7IXlHzZOlc3XkSjiTpn0BDWQeUUe9rlAk09VFyws3duCBa1uw6kA6qxLS+OdQBklZBSRlFQCg1UBUoBdtQr2Z0K+ZuuXLTx+qDXS8RTa+FcJJTLmmBb/FJ7PwbFv66v+GQytg2GsOieX7LequB6OiDHAE8G/mkDgcwWk+Mb/99lumTZvGwIED0Wq1jBkzhjlz5piPGwwGEhMTzYu1duzYwebN6n/QrVqVrwNx7NgxoqKi0Ov1zJ07l8ceewxFUWjVqhXvvPMOkyfL5qAAtB6iJkuHY2HA0zVvJ/Mo5Kap1bJrkXQBENFDTZaSdkCnMTWMp44lSx5+ENYZUnfD8Q3Q+dYqPS3Ex51xfZoyrk9TCopLiTuaQUGxkVYhjYgK8sTN5aKrEYvzIPFv9X7nGvabEMLuOkb4cnXrIP451IlSvQ7d2UPqZ2pAC7vGceJsHhsOZ6DRQN+AXDVZqgNX5tmL0yRLAQEBfPfdd5Uej4qKKrfB64ABA6644euwYcMYNqwWC4XrO9MGtqe3Qd5Z8LKwfYglx8v2g2vSC/S1rGHVuIe6sWRyLS7hqisFKS8W1b/aydLFPFx1XN8utPITEv9W12n5N1cTTiGE03jg2paMP5TBNqUt0Zr9cCgWov9j1xgWbVU3zb2mdTC+hWX1Df0azshSw5hsFDXj2xhCOqLuyba65u2YKs8261f7mEz/0afsUrdPqYm6NrIEF/bKs1WV3rKtTeg0xu5XwAkhaqdfy0C6RvqxsqSb+sDB5XZ9/eISIz9uU5OlcX2amjchb0jTcJIsCctaD1ZvD9eivkVNNs+tTHBb0HtCcS5kHKpZG5lllbDr0shSs36ABjIOQm76FU+vloJz6l+iUPOpSyGEw2g0Gh4f3IY1xm4AKMc3qFPrdrIyIY2M3GKCvd0Y2D4Ezp1QD8jIkhBlzMnSSjAaq//8cyfg/CnQukBkn9rHo9VBeFf1fvKO6j+/8DwUqPVB6tTIkmcAhHZU71t7dClhKRgNENIBQjtYt20hhF1c3ToI/6adOGUMRlNaBMfW2+V1FUXhk3VHABjbOxK9sQhyy4pBS7IkRJnIaHDzgfyzNVsndGiFetu4F7h6WScm01RcUg2SJdNolFcwuHlbJx5rMY28VaGEQLWYp+BusW67Qgi70Wg0TB/Szjy6lLvXPlWzNxzOYNfp87jrtdzbLwqy1Ok4XBupf+Q1EJIsCct0+gs1PWoyFZfwu3rb/karhUTjsmSpJiNLpgQrorv14rGWKFOytMF6beamw7F16n2ZghPCqcW0DCQ19BoASg4sU/cxsrEPVh8G1LVKgY3cLqxX8mvWoNY/SrIkrsw0FWcaJaqqvLMXRkna32C9eEyJTuoeKCmu3nOTtqu3tS1hYAumkaUzCWrfWcP+30AxqqNxdr7UWAhhfYNGjKFQ0eNnSCf5UA3+YKyGrccz2XIsE71Ow5Rryj4/so6rtw1ocTdIsiSqwlRCIGkH5GVU/XkH/walFMK6gH+U9eIJaAHuflBaDOn7q/fcupwseQVBcDv1vrXWLe35Sb2tQTkCIUTd06NlYxI91T8Yd6xcbNPX+rBsVOnWnpGE+3qoD5oXdzecGksgyZKoCp8ICO1MtUsIJPyh3ra/ybrxaDQXRpeqMxVXkAVny9Ys1dVaQ1H91VtrJEtZp+DUJkADHW+ufXtCiDohqLs6Uh+cuo7D6bk2eY09p8+z7uAZdFoND17b8sKBi6fhGhBJlkTVtC4bXarqVFxRzoXEyprrlUxMyVJ1FnmbFqj7R9W8wKatWXORt2lhd7Or1IRXCFEvNO49CoCemoO8+dumKxZgrokP16h/WN7UNYKmgZ4XDmSVjSzJNJwQFWhlKiGwqmrFIA+tUKfJAlurtZGszbzIuxpX6NXlKTgTU7KUthfyM2vejqJA/LfqfZmCE6J+8Y+i2L81LhojLsfW8fOOJKs2fzAth+X70gB4aEDL8gcbYI0lkGRJVFVkH3DzVWsUndpy5fPNU3A32uaKCdM0WnoCFOdX7TmmUai6nCx5h6oJJgqcjKt5O6c2qwUu9Z5yFZwQ9ZBr2yEA9NPuY+bS/ZzJKbJa2x+tUdcqDesYRuvQi0qsFOVcqFMna5aEqIBOD+1Gqvf/fc/yuYZCOFg2XWeLKThQp5UahaoLyFN3X/l8RYGkber9urpeyaT51ertgT9r3saOr9XbjjeDu0/tYxJC1C1lRX77uh3jfIGB//2+zyrNxh05y2+71L3fpl5XfhN683olD/8G97kiyZKouqsfB40WDi67MKVVkaNrwJAHPk1sV89Io1ELXULV6hJlJ0NuGmh0EN7FNjFZS5c71Nt9S6Awu/rPL8yGfb+o93tMsF5cQoi6o4n6+dfCeAIvbTF/7klh+b7UWjV5Pt/A4z/EoyhwR69IOjfxLX9CA52CA0mWRHUEtbrwH/na1ys/z9ZTcCamRecHl135XFNyF9LBepXEbSUyGoLagCH/wiLt6tj3i/rcwNZqW0KI+senMTQKQ6OU8mw3dQru+V/3cr7AUKPmFEXhud/2kny+kKhAT164sYKtkbIaZtkAkGRJVNc1T6qjM4dWwKmtlx8vNUBiWRl+W03BmbQZpt6e3nblzWfNi7vr+BQcqAmmaURox8LqP9/0nB4TGlSFXSEaFI3GvP7y9vA0WgR5kZ5TxCtL99fo6rhf45P4Y1cyOq2Gd+/ohpeby+UnnWuYV8KBJEuiugJbQtdx6v21sy4/fuJfdZd7zyBo2te2sfhEQHg3QIGDyy2f6wxXwl2s6zjQ6tU6Uql7qv68tP3qe9W6XPh3EkLUT03UzzN9yg5m36ouL/hx+2leX3agWgnTqcx8XvhVXfP0yMDWdG/qX/GJDbTGEkiyJGrimifU/4yPrCp/ZVxRDmz6WL3fbiRodbaPpe0I9Tbx78rPMRohOV697yzJklfQhQX11Rld2lm2sLvtcGgUbP24hBB1h2ndZtJ2ekcF8MIN6tTZJ+uO8vxvezEar5wwlRoVpv8QT05RCT2b+V9eKuBi5hpLUbUM3PlIsiSqL6A5dLtTvb/mNfVKsz0/wYe9L6wfsteoRtvh6u2R1WAoqPics4egOEe9jN60nYgzME3F7V5c+Xu7WEkR7Ppevd9dFnYLUe9FdAc0cP4U5KRxX//mzLqlMxoNfLPpJE/8tIuSUmOlTz+Vmc+kBVvZevwcjdxcePf2brjoKkkLFKXBbnUCkiyJmrq6bHTp6Br47Dr4eRLkpIB/cxj/MzSLsU8cYZ3Vq+5KCuDouorPMU3BhXcDXQXz8HVVi+vAtykUnr+waN6SA3+qU6DeEdBqoO3jE0I4lrvPhaK/ZZ9z4/o05b07uqHTavhlRxL//X4nqecLyz3NUGrk47VHGPzuOtYmnkGv0/D6mM7lK3VfquCc+kcnNMhkyYn+5xB1in8z6H4XbP9KraLt4q4mUP3+C3p3+8Wh0aijS1s/UzfubTvs8nOcaXH3xbRatY/XvqZOxXW53fL5pum67uPtMwUqhHC8xr3gzAG1jlw7dVnCqG6Ncdfr+O93O/l7byp/700lxNuNLk186RDhy7K9KRxMU/eU69sigFdGd6ZVSCPLr2OagmsUCnoPW76jOklGlkTNXfu0Ogzc/iaYuhmufdK+iZKJaSoucZm6PulSzra4+2Ldx6u1rY7/A2ePVH7e/t/UUT6AbuPtE5sQwvHKFnlzelu5h4d2DOPLib3pEO6DVgPpOUWsTEhnzqpDHEzLJcDLlbdv68r3k/teOVGCBl1jCWRkSdSGTwRMWevoKCCqP7h6Q24qpOwsnxQZCiF1r3rfGZMl3ybQapBaqmHHQhj80uXnpOyCJQ+o9/tOVdeUCSEaBtMi7+Sd6h+L2gtjIFe1CuKvR64mv7iE/cnZ7D59nr1J5wn2cePBa1vi5+la9ddpwDWWQJIlUR+4uEGr69XRlcS/yydFaXvBaFBLGTjrL3mPCWqytO1Ltchk2VA7ADlp8P2dahHKlgNh8MuOi1MIYX8hHcDFA4qy1YtZKti43NPVhV5RAfSKCqj565jKBjTAGksg03CivqishMDFU3DOWqCxzTD1r8ei87BoHPw2Vd3SxFAIi8dD9mm1Wvet851rAbsQovZ0LhDRTb1/yVScVTXwaThJlkT90HqIurYnba/6F1BpibqZ7IZ31ePOOAVnotPDxL+g38OABnZ+A/Ough/vgdNbwd0P7lwMHn4ODlQI4RCmz7ckGyZLMg0nRD3gGQBNY9QK4itfUtfxnD2kHvONVK8qc2YubjBkprqYfcl/1IQw66S69cxtX6mV1YUQDVOTC8UpbUJRZBrO0QEIYTWmq+L2/qQmSh4BMORVmLYNfBs7NjZradYPHtyormPSe8HIt6HldY6OSgjhSKaRpbR9VStgW1256VBSqI7e+0Zav30nICNLov7oMArWvq7ej5kGMVPVom31jZs33PQB3PB+uStfhBANlG8keIVAXro6qm7tfTlNU3A+jdVlAQ2QJEui/vBrCg+XFcisj0nSpSRREkKAevFKk16Q+Je6yNvayVID3ubERD5tRf3SKKRhJEpCCHEx0w4Ftli3lHVcvW2gV8KBJEtCCCGE8zMVp7TFFXEZZRfLNOALSSRZEkIIIZydaWQp6yTkZVi37TOJ6m0FBS8bCkmWhBBCCGfn7guBrdT7yfHWa9dovDCyFNzOeu06GUmWhBBCiPogvJt6m7zTem1mnwZDHmj14N9w952UZEkIIYSoDyK6q7cp8dZr88xB9TawZYPeTslpkqXMzEzGjx+Pj48Pfn5+TJo0idzcXIvPGTBgABqNptzXAw88UO6ckydPMnLkSDw9PQkJCeHJJ5+kpKTElm9FCCGEsD7THnHWHFnKKFuvFNTGem06IadJE8ePH09KSgqxsbEYDAYmTpzIlClT+O677yw+b/Lkybz88oWd2D09Pc33S0tLGTlyJGFhYWzcuJGUlBQmTJiAXq/ntddes9l7EUIIIawurAuggewktep2o5Dat3nmgHrbgNcrgZOMLCUkJLBs2TI+//xzoqOj6d+/Px988AGLFi0iOTnZ4nM9PT0JCwszf/n4XKjBs2LFCvbv388333xDt27dGD58ODNnzmTu3LkUFxfb+m0JIYQQ1uPuY/1F3qZpuAZ8JRw4ychSXFwcfn5+9OrVy/zYoEGD0Gq1bN68mZtvvrnS53777bd88803hIWFceONN/L888+bR5fi4uLo3LkzoaGh5vOHDh3Kgw8+yL59++jevXuFbRYVFVFUVGT+Pjs7GwCDwYDBYKjVe72YqS1rtikuJ/1sP9LX9iH9bB91sZ914V3Rnj1E6entGJvXct9IRcElIxENYPBrAQ56n7bs56q26RTJUmpqKiEh5YcTXVxcCAgIIDU1tdLn3XnnnTRr1oyIiAh2797N008/TWJiIr/88ou53YsTJcD8vaV2Z82axUsvvXTZ4ytWrCg3zWctsbGxVm9TXE762X6kr+1D+tk+6lI/tzjnSmcgfddytuR0qFVbroZshhecQ0HDsm1HMGpPWSfIGrJFP+fn51fpPIcmS8888wyzZ8+2eE5CQkKN258yZYr5fufOnQkPD2fgwIEcOXKEli1rXol0xowZTJ8+3fx9dnY2kZGRDBkypNw0X20ZDAZiY2MZPHgwen3D3LzQHqSf7Uf62j6kn+2jLvaz5qQ/fP0dYcZURowYUbu2TmyAvYBfM4bdMNoq8dWELfvZNDN0JQ5Nlh5//HHuvfdei+e0aNGCsLAw0tPTyz1eUlJCZmYmYWFhVX696OhoAA4fPkzLli0JCwtjy5Yt5c5JS0sDsNium5sbbm5ulz2u1+tt8gtjq3ZFedLP9iN9bR/Sz/ZRp/q5SQ9AgyYnBX1hJniHXvEplTp3BABNcNs68f5s0c9Vbc+hyVJwcDDBwcFXPC8mJoasrCy2b99Oz549AVi9ejVGo9GcAFVFfHw8AOHh4eZ2X331VdLT083TfLGxsfj4+NChQ+2GL4UQQgi7c2ukXuafkajWW/IeWvO2zIu7G3bZAHCSq+Hat2/PsGHDmDx5Mlu2bOHff/9l2rRpjB07loiICACSkpJo166deaToyJEjzJw5k+3bt3P8+HF+//13JkyYwDXXXEOXLl0AGDJkCB06dODuu+9m165dLF++nOeee46pU6dWOHIkhBBC1HnmekvxtWvHVGOpgZcNACdJlkC9qq1du3YMHDiQESNG0L9/fz799FPzcYPBQGJionmxlqurKytXrmTIkCG0a9eOxx9/nDFjxvDHH3+Yn6PT6Vi6dCk6nY6YmBjuuusuJkyYUK4ukxBCCOFUTJW8a1uc0rSBblDDLhsATnI1HEBAQIDFApRRUVEoimL+PjIyknXr1l2x3WbNmvHXX39ZJUYhhBDC4Ux7xNVm25PC85CTot6XaTjnGVkSQgghRBWEdQaNVk12ciovg2NRxiH11jsc3H2tF5uTkmRJCCGEqE9Mi7yh5uuWTNucNPA94UwkWRJCCCHqm9quWzKtV2rg25yYSLIkhBBC1De1XbeUUVY2QEaWAEmWhBBCiPrHaiNLUjYAJFkSQggh6h/TIu/cNMhOqd5zDQVw7rh6X6bhAEmWhBBCiPrH1fPCqFB1R5fOHgYUcPcDryvvstEQSLIkhBBC1EemdUtJ26v3vIun4DQaq4bkrCRZEkIIIeqjZjHq7dG11XtehuwJdylJloQQQoj6qNUg9TZpO+SdrfrzzDWWZL2SiSRLQgghRH3kEwEhHQEFjq6p+vPOmEaW5Eo4E0mWhBBCiPqq1UD19vDKqp1fnF+2wBuZhruIJEtCCCFEfdV6sHp7eCUYjVc+/8hqMBrAtyn4Rto2NiciyZIQQghRX0X2BddGkHcGUndf+fwDf6q37W+QK+EuIsmSEEIIUV+5uELza9X7V5qKKy2Bg3+r99uNtG1cTkaSJSGEEKI+q+q6pRP/QsE58AyEpjG2j8uJSLIkhBBC1GemEgKntkBBVuXnHViq3rYdDlqdzcNyJpIsCSGEEPWZfzMIagNKaeUFKhXlwnqldjfaLTRnIcmSEEIIUd+ZRpcqm4pL3gnZSaD3ghYD7BaWs5BkSQghhKjvzMnSKnUU6VKmKbjWg0Dvbr+4nIQkS0IIIUR91+wqcPGAnGRI33/58YSyZEmm4CokyZIQQghR3+ndIaq/ev/SqbiMQ5CRCFo9tBli/9icgCRLQgghRENwcTXvi5mm4JpfA+6+9o3JSUiyJIQQQjQEpnVLx9bDj/dC2j71e/MUnBSirIwkS0IIIURDENgS+vxHvb9vCXzcD74bC0nbAI0kSxZIsiSEEEI0FCPegAf+hQ6jAc2F7U2a9AbvMEdGVqdJsiSEEEI0JGGd4PYF8FAcdLoV3Hyg3zRHR1WnuTg6ACGEEEI4QEh7uPULR0fhFGRkSQghhBDCAkmWhBBCCCEskGRJCCGEEMICSZaEEEIIISyQZEkIIYQQwgJJloQQQgghLJBkSQghhBDCAqdJljIzMxk/fjw+Pj74+fkxadIkcnNzKz3/+PHjaDSaCr9+/PFH83kVHV+0aJE93pIQQgghnIDTFKUcP348KSkpxMbGYjAYmDhxIlOmTOG7776r8PzIyEhSUlLKPfbpp5/y5ptvMnz48HKPf/nllwwbNsz8vZ+fn9XjF0IIIYRzcopkKSEhgWXLlrF161Z69eoFwAcffMCIESN46623iIiIuOw5Op2OsLDy+9wsWbKE22+/nUaNGpV73M/P77JzhRBCCCHASabh4uLi8PPzMydKAIMGDUKr1bJ58+YqtbF9+3bi4+OZNGnSZcemTp1KUFAQffr0Yf78+SiKYrXYhRBCCOHcnGJkKTU1lZCQkHKPubi4EBAQQGpqapXa+OKLL2jfvj39+vUr9/jLL7/M9ddfj6enJytWrOChhx4iNzeXhx9+uNK2ioqKKCoqMn+fnZ0NgMFgwGAwVPVtXZGpLWu2KS4n/Ww/0tf2If1sH9LP9mHLfq5qmw5Nlp555hlmz55t8ZyEhIRav05BQQHfffcdzz///GXHLn6se/fu5OXl8eabb1pMlmbNmsVLL7102eMrVqzA09Oz1vFeKjY21uptistJP9uP9LV9SD/bh/Szfdiin/Pz86t0nkZx4JzTmTNnOHv2rMVzWrRowTfffMPjjz/OuXPnzI+XlJTg7u7Ojz/+yM0332yxja+//ppJkyaRlJREcHCwxXP//PNPbrjhBgoLC3Fzc6vwnIpGliIjI8nIyMDHx8di+9VhMBiIjY1l8ODB6PV6q7UrypN+th/pa/uQfrYP6Wf7sGU/Z2dnExQUxPnz5y3+/+3QkaXg4OArJi8AMTExZGVlsX37dnr27AnA6tWrMRqNREdHX/H5X3zxBTfddFOVXis+Ph5/f/9KEyUANze3Co/r9Xqb/MLYql1RnvSz/Uhf24f0s31IP9uHLfq5qu05xZql9u3bM2zYMCZPnsy8efMwGAxMmzaNsWPHmq+ES0pKYuDAgSxcuJA+ffqYn3v48GHWr1/PX3/9dVm7f/zxB2lpafTt2xd3d3diY2N57bXXeOKJJ+z23oQQQghRtzlFsgTw7bffMm3aNAYOHIhWq2XMmDHMmTPHfNxgMJCYmHjZ/OP8+fNp0qQJQ4YMuaxNvV7P3Llzeeyxx1AUhVatWvHOO+8wefJkm78fIYQQQjgHp0mWAgICKi1ACRAVFVXhJf+vvfYar732WoXPGTZsWLlilEIIIYQQl3KKOktCCCGEEI4iyZIQQgghhAWSLAkhhBBCWCDJkhBCCCGEBZIsCSGEEEJYIMmSEEIIIYQFkiwJIYQQQlggyZIQQgghhAWSLAkhhBBCWCDJkhBCCCGEBZIsCSGEEEJYIMmSEEIIIYQFkiwJIYQQQlggyZIQQgghhAWSLAkhhBBCWCDJkhBCCCGEBZIsCSGEEEJYIMmSEEIIIYQFkiwJIYQQQlggyZIQQgghhAWSLAkhhBBCWCDJkhBCCCGEBZIsCSGEEEJYIMmSEEIIIYQFkiwJIYQQQlggyZIQQgghhAWSLAkhhBBCWCDJkhBCCCGEBZIsCSGEEEJYIMmSEEIIIYQFkiwJIYQQQlggyZIQQgghhAWSLAkhhBBCWCDJkhBCCCGEBZIsCSGEEEJYIMmSEEIIIYQFTpMsvfrqq/Tr1w9PT0/8/Pyq9BxFUXjhhRcIDw/Hw8ODQYMGcejQoXLnZGZmMn78eHx8fPDz82PSpEnk5uba4B0IIYQQwhk5TbJUXFzMbbfdxoMPPljl57zxxhvMmTOHefPmsXnzZry8vBg6dCiFhYXmc8aPH8++ffuIjY1l6dKlrF+/nilTptjiLQghhBDCCbk4OoCqeumllwD46quvqnS+oii89957PPfcc4waNQqAhQsXEhoayq+//srYsWNJSEhg2bJlbN26lV69egHwwQcfMGLECN566y0iIiJs8l6EEEII4TycJlmqrmPHjpGamsqgQYPMj/n6+hIdHU1cXBxjx44lLi4OPz8/c6IEMGjQILRaLZs3b+bmm2+usO2ioiKKiorM358/fx5Qp/QMBoPV3oPBYCA/P5+zZ8+i1+ut1q4oT/rZfqSv7UP62T6kn+3Dlv2ck5MDqAMsltTbZCk1NRWA0NDQco+Hhoaaj6WmphISElLuuIuLCwEBAeZzKjJr1izzSNfFmjdvXtuwhRBCCGFnOTk5+Pr6VnrcocnSM888w+zZsy2ek5CQQLt27ewUUdXMmDGD6dOnm783Go1kZmYSGBiIRqOx2utkZ2cTGRnJqVOn8PHxsVq7ojzpZ/uRvrYP6Wf7kH62D1v2s6Io5OTkXHHZjUOTpccff5x7773X4jktWrSoUdthYWEApKWlER4ebn48LS2Nbt26mc9JT08v97ySkhIyMzPNz6+Im5sbbm5u5R6r6hV6NeHj4yO/iHYg/Ww/0tf2If1sH9LP9mGrfrY0omTi0GQpODiY4OBgm7TdvHlzwsLCWLVqlTk5ys7OZvPmzeYr6mJiYsjKymL79u307NkTgNWrV2M0GomOjrZJXEIIIYRwLk5TOuDkyZPEx8dz8uRJSktLiY+PJz4+vlxNpHbt2rFkyRIANBoNjz76KK+88gq///47e/bsYcKECURERDB69GgA2rdvz7Bhw5g8eTJbtmzh33//Zdq0aYwdO1auhBNCCCEE4EQLvF944QUWLFhg/r579+4ArFmzhgEDBgCQmJhovjIN4KmnniIvL48pU6aQlZVF//79WbZsGe7u7uZzvv32W6ZNm8bAgQPRarWMGTOGOXPm2OdNXYGbmxsvvvjiZVN+wrqkn+1H+to+pJ/tQ/rZPupCP2uUK10vJ4QQQgjRgDnNNJwQQgghhCNIsiSEEEIIYYEkS0IIIYQQFkiyJIQQQghhgSRLddjcuXOJiorC3d2d6OhotmzZ4uiQnNqsWbPo3bs33t7ehISEMHr0aBITE8udU1hYyNSpUwkMDKRRo0aMGTOGtLQ0B0Xs/F5//XVzGQ8T6WPrSUpK4q677iIwMBAPDw86d+7Mtm3bzMcVReGFF14gPDwcDw8PBg0axKFDhxwYsfMpLS3l+eefp3nz5nh4eNCyZUtmzpxZbi8x6efqW79+PTfeeCMRERFoNBp+/fXXcser0qeZmZmMHz8eHx8f/Pz8mDRpUrlyQtYkyVIdtXjxYqZPn86LL77Ijh076Nq1K0OHDr2s4riounXr1jF16lQ2bdpEbGwsBoOBIUOGkJeXZz7nscce448//uDHH39k3bp1JCcnc8sttzgwaue1detWPvnkE7p06VLucelj6zh37hxXXXUVer2ev//+m/379/P222/j7+9vPueNN95gzpw5zJs3j82bN+Pl5cXQoUMpLCx0YOTOZfbs2Xz88cd8+OGHJCQkMHv2bN544w0++OAD8znSz9WXl5dH165dmTt3boXHq9Kn48ePZ9++fcTGxrJ06VLWr1/PlClTbBOwIuqkPn36KFOnTjV/X1paqkRERCizZs1yYFT1S3p6ugIo69atUxRFUbKyshS9Xq/8+OOP5nMSEhIUQImLi3NUmE4pJydHad26tRIbG6tce+21yiOPPKIoivSxNT399NNK//79Kz1uNBqVsLAw5c033zQ/lpWVpbi5uSnff/+9PUKsF0aOHKncd9995R675ZZblPHjxyuKIv1sDYCyZMkS8/dV6dP9+/crgLJ161bzOX///bei0WiUpKQkq8coI0t1UHFxMdu3b2fQoEHmx7RaLYMGDSIuLs6BkdUvpgKmAQEBAGzfvh2DwVCu39u1a0fTpk2l36tp6tSpjBw5slxfgvSxNf3+++/06tWL2267jZCQELp3785nn31mPn7s2DFSU1PL9bWvry/R0dHS19XQr18/Vq1axcGDBwHYtWsXGzZsYPjw4YD0sy1UpU/j4uLw8/OjV69e5nMGDRqEVqtl8+bNVo/JaSp4NyQZGRmUlpYSGhpa7vHQ0FAOHDjgoKjqF6PRyKOPPspVV11Fp06dAEhNTcXV1fWyTZFDQ0NJTU11QJTOadGiRezYsYOtW7dedkz62HqOHj3Kxx9/zPTp03n22WfZunUrDz/8MK6urtxzzz3m/qzoc0T6uuqeeeYZsrOzadeuHTqdjtLSUl599VXGjx8PIP1sA1Xp09TUVEJCQsodd3FxISAgwCb9LsmSaJCmTp3K3r172bBhg6NDqVdOnTrFI488QmxsbLlthYT1GY1GevXqxWuvvQaoW0Dt3buXefPmcc899zg4uvrjhx9+4Ntvv+W7776jY8eOxMfH8+ijjxIRESH93IDINFwdFBQUhE6nu+wKobS0NMLCwhwUVf0xbdo0li5dypo1a2jSpIn58bCwMIqLi8nKyip3vvR71W3fvp309HR69OiBi4sLLi4urFu3jjlz5uDi4kJoaKj0sZWEh4fToUOHco+1b9+ekydPApj7Uz5HaufJJ5/kmWeeYezYsXTu3Jm7776bxx57jFmzZgHSz7ZQlT4NCwu77IKnkpISMjMzbdLvkizVQa6urvTs2ZNVq1aZHzMajaxatYqYmBgHRubcFEVh2rRpLFmyhNWrV9O8efNyx3v27Ilery/X74mJiZw8eVL6vYoGDhzInj17iI+PN3/16tWL8ePHm+9LH1vHVVdddVnpi4MHD9KsWTMAmjdvTlhYWLm+zs7OZvPmzdLX1ZCfn49WW/6/Sp1Oh9FoBKSfbaEqfRoTE0NWVhbbt283n7N69WqMRiPR0dHWD8rqS8aFVSxatEhxc3NTvvrqK2X//v3KlClTFD8/PyU1NdXRoTmtBx98UPH19VXWrl2rpKSkmL/y8/PN5zzwwANK06ZNldWrVyvbtm1TYmJilJiYGAdG7fwuvhpOUaSPrWXLli2Ki4uL8uqrryqHDh1Svv32W8XT01P55ptvzOe8/vrrip+fn/Lbb78pu3fvVkaNGqU0b95cKSgocGDkzuWee+5RGjdurCxdulQ5duyY8ssvvyhBQUHKU089ZT5H+rn6cnJylJ07dyo7d+5UAOWdd95Rdu7cqZw4cUJRlKr16bBhw5Tu3bsrmzdvVjZs2KC0bt1aGTdunE3ilWSpDvvggw+Upk2bKq6urkqfPn2UTZs2OTokpwZU+PXll1+azykoKFAeeughxd/fX/H09FRuvvlmJSUlxXFB1wOXJkvSx9bzxx9/KJ06dVLc3NyUdu3aKZ9++mm540ajUXn++eeV0NBQxc3NTRk4cKCSmJjooGidU3Z2tvLII48oTZs2Vdzd3ZUWLVoo//d//6cUFRWZz5F+rr41a9ZU+Hl8zz33KIpStT49e/asMm7cOKVRo0aKj4+PMnHiRCUnJ8cm8WoU5aIypEIIIYQQohxZsySEEEIIYYEkS0IIIYQQFkiyJIQQQghhgSRLQgghhBAWSLIkhBBCCGGBJEtCCCGEEBZIsiSEEEIIYYEkS0IIIYQQFkiyJISod86cOcODDz5I06ZNcXNzIywsjKFDh/Lvv/86OjQhhBNycXQAQghhbWPGjKG4uJgFCxbQokUL0tLSWLVqFWfPnnV0aEIIJyQjS0KIeiUrK4t//vmH2bNnc91119GsWTP69OnDjBkzuOmmmwDQaDR8/PHHDB8+HA8PD1q0aMFPP/1Urp2nn36aNm3a4OnpSYsWLXj++ecxGAzlzvnjjz/o3bs37u7uBAUFcfPNN5uPFRUV8cQTT9C4cWO8vLyIjo5m7dq1Nn//Qgjrk2RJCFGvNGrUiEaNGvHrr79SVFRU6XnPP/88Y8aMYdeuXYwfP56xY8eSkJBgPu7t7c1XX33F/v3/3979g6S3hnEA/4b0lxMONVhLYlnUcMiDSVbQYJLUUBaSIEREQUNrEOQvaWmIRBqqIYKGgiMhQmCbUEtKGIXBXaKhMzQUbf0jkvcOFw7Ir+vl3pv3x63vBxz0ed7z+pzp4eE9nN+wtraGra0tRCIRPZ5IJOD1ejEwMIDz83Mkk0k4HA49Pjs7i1QqBVVVkc1m4fP54PF4cHV1VZzCiaho+CJdIvpyYrEYpqen8fLyAkVR0NvbC7/fD1mWAfwxWZqZmcHm5qa+prOzE4qiYGNj48Nrrq6uQlVVZDIZAEBXVxcsFgt2d3d/ytU0DRaLBZqmob6+Xv+9r68PDocDy8vLn1kuERUZJ0tE9OWMjo7i9vYWBwcH8Hg8ODo6gqIo2NnZ0XOcTmfeGqfTmTdZikaj6O7uhslkgiRJCAaD0DRNj19cXMDlcn24/+XlJXK5HJqbm/VJlyRJOD4+xvX19ecWS0RFxwPeRPQlVVRUwO12w+1248ePH5iamkIoFMLExMRfrk2lUggEAlhaWkJ/fz+MRiNUVUU4HNZzKisr/3T94+MjDAYDzs7OYDAY8mKSJP3jmojo1+BkiYi+hba2Njw9Penf0+l0XjydTqO1tRUAcHJygoaGBiwsLMBut8NqteLm5iYvX5ZlJJPJD/ey2WzI5XK4u7tDU1NT3sdkMn1yZURUbJwsEdGX8vDwAJ/Ph8nJSciyjOrqamQyGaysrGBoaEjP29/fh91uR09PD/b29nB6eort7W0AgNVqhaZpUFUVHR0dSCQSiMfjefuEQiG4XC40NjbC7/fj/f0dh4eH+lN0gUAA4+PjCIfDsNlsuL+/RzKZhCzLGBwc/E/vCRH9S4KI6At5fX0V8/PzQlEUYTQaRVVVlWhpaRHBYFA8Pz8LIYQAINbX14Xb7Rbl5eXCbDaLaDSad525uTlRU1MjJEkSY2NjIhKJCKPRmJcTi8VEe3u7KCsrE7W1tWJkZESPvb29icXFRWE2m0Vpaamoq6sTXq9XZLPZot8DIvpcfBqOiL6dkpISxONxDA8P/+q/QkT/AzyzRERERFQAmyUiIiKiAnjAm4i+HZ4+IKK/g5MlIiIiogLYLBEREREVwGaJiIiIqAA2S0REREQFsFkiIiIiKoDNEhEREVEBbJaIiIiICmCzRERERFQAmyUiIiKiAn4Hprq+OB853+sAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "ex.viz.plot_state_1d(jnp.concatenate([u_strongly_diffused, u_less_diffused]))" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Array(0.37669653, dtype=float32)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "ex.metrics.nRMSE(u_strongly_diffused, u_less_diffused)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Array(1.1598254, dtype=float32)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "ex.metrics.H1_nRMSE(u_strongly_diffused, u_less_diffused)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Application: Detecting Blurry Predictions of Neural Emulators" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "exponax_dev", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.4" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/exponax/__init__.py b/exponax/__init__.py index b31fd5a..27ab6cf 100644 --- a/exponax/__init__.py +++ b/exponax/__init__.py @@ -1,7 +1,6 @@ -from . import _metrics as metrics from . import _poisson as poisson from . import _spectral as spectral -from . import etdrk, ic, nonlin_fun, stepper, viz +from . import etdrk, ic, metrics, nonlin_fun, stepper, viz from ._base_stepper import BaseStepper from ._forced_stepper import ForcedStepper from ._interpolation import FourierInterpolator, map_between_resolutions diff --git a/exponax/_metrics.py b/exponax/_metrics.py deleted file mode 100644 index ff17eae..0000000 --- a/exponax/_metrics.py +++ /dev/null @@ -1,609 +0,0 @@ -from typing import Optional - -import jax -import jax.numpy as jnp -from jaxtyping import Array, Float - -from ._spectral import fft, low_pass_filter_mask - - -def _MSE( - u_pred: Float[Array, "... N"], - u_ref: Optional[Float[Array, "... N"]] = None, - domain_extent: float = 1.0, - *, - num_spatial_dims: Optional[int] = None, -) -> float: - """ - Low-level function to compute the mean squared error (MSE) correctly scaled - for states representing physical fields on uniform Cartesian grids. - - MSE = 1/L^D * 1/N * sum_i (u_pred_i - u_ref_i)^2 - - Note that by default (`num_spatial_dims=None`), the number of spatial - dimensions is inferred from the shape of the input fields. Please adjust - this argument if you call this function with an array that also contains - channels (even for arrays with singleton channels. - - Providing correct information regarding the scaling (i.e. providing - `domain_extent` and `num_spatial_dims`) is not necessary if the result is - used to compute a normalized error (e.g. nMSE) if the normalization is - computed similarly. - - **Arguments**: - - `u_pred` (array): The first field to be used in the loss - - `u_ref` (array, optional): The second field to be used in the error - computation. If `None`, the error will be computed with respect to - zero. - - `domain_extent` (float, optional): The extent of the domain in which - the fields are defined. This is used to scale the error to be - independent of the domain size. Default is 1.0. - - `num_spatial_dims` (int, optional): The number of spatial dimensions - in the field. If `None`, it will be inferred from the shape of the - input fields and then is the number of axes present. Default is - `None`. - - **Returns**: - - `mse` (float): The (correctly scaled) mean squared error between the - fields. - """ - if u_ref is None: - diff = u_pred - else: - diff = u_pred - u_ref - - if num_spatial_dims is None: - # Assuming that we only have spatial dimensions - num_spatial_dims = len(u_pred.shape) - - scale = 1 / (domain_extent**num_spatial_dims) - - mse = scale * jnp.mean(jnp.square(diff)) - - return mse - - -def MSE( - u_pred: Float[Array, "C ... N"], - u_ref: Optional[Float[Array, "C ... N"]] = None, - domain_extent: float = 1.0, -): - """ - Compute the mean squared error (MSE) between two fields. - - This function assumes that the arrays have one leading channel axis and an - arbitrary number of following spatial dimensions! For batched operation use - `jax.vmap` on this function or use the [`exponax.metrics.mean_MSE`][] function. - - **Arguments**: - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array, optional): The second field to be used in the error - computation. If `None`, the error will be computed with respect to - zero. - - `domain_extent` (float, optional): The extent of the domain in which - the fields are defined. This is used to scale the error to be - independent of the domain size. Default is 1.0. - - **Returns**: - - `mse` (float): The (correctly scaled) mean squared error between the - fields. - """ - - num_spatial_dims = len(u_pred.shape) - 1 - - mse = _MSE(u_pred, u_ref, domain_extent, num_spatial_dims=num_spatial_dims) - - return mse - - -def nMSE( - u_pred: Float[Array, "C ... N"], - u_ref: Float[Array, "C ... N"], -) -> float: - """ - Compute the normalized mean squared error (nMSE) between two fields. - - In contrast to [`exponax.metrics.MSE`][], no `domain_extent` is required, because of the - normalization. - - **Arguments**: - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array): The second field to be used in the error computation. - This is also used to normalize the error. - - **Returns**: - - `nmse` (float): The normalized mean squared error between the fields - """ - - num_spatial_dims = len(u_pred.shape) - 1 - - # Do not have to supply the domain_extent, because we will normalize with - # the ref_mse - diff_mse = _MSE(u_pred, u_ref, num_spatial_dims=num_spatial_dims) - ref_mse = _MSE(u_ref, num_spatial_dims=num_spatial_dims) - - nmse = diff_mse / ref_mse - - return nmse - - -def mean_MSE( - u_pred: Float[Array, "B C ... N"], - u_ref: Float[Array, "B C ... N"], - domain_extent: float = 1.0, -) -> float: - """ - Compute the mean MSE between two fields. Use this function to correctly - operate on arrays with a batch axis. - - **Arguments**: - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array): The second field to be used in the error computation. - - `domain_extent` (float, optional): The extent of the domain in which - the fields are defined. This is used to scale the error to be - independent of the domain size. Default is 1.0. - - **Returns**: - - `mean_mse` (float): The mean mean squared error between the fields - """ - batch_wise_mse = jax.vmap(MSE, in_axes=(0, 0, None))(u_pred, u_ref, domain_extent) - mean_mse = jnp.mean(batch_wise_mse) - return mean_mse - - -def mean_nMSE( - u_pred: Float[Array, "B C ... N"], - u_ref: Float[Array, "B C ... N"], -): - """ - Compute the mean nMSE between two fields. Use this function to correctly - operate on arrays with a batch axis. - - **Arguments**: - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array): The second field to be used in the error computation. - - **Returns**: - - `mean_nmse` (float): The mean normalized mean squared error between - """ - batch_wise_nmse = jax.vmap(nMSE)(u_pred, u_ref) - mean_nmse = jnp.mean(batch_wise_nmse) - return mean_nmse - - -def _RMSE( - u_pred: Float[Array, "... N"], - u_ref: Optional[Float[Array, "... N"]] = None, - domain_extent: float = 1.0, - *, - num_spatial_dims: Optional[int] = None, -) -> float: - """ - Low-level function to compute the root mean squared error (RMSE) correctly - scaled for states representing physical fields on uniform Cartesian grids. - - RMSE = sqrt(1/L^D * 1/N * sum_i (u_pred_i - u_ref_i)^2) - - Note that by default (`num_spatial_dims=None`), the number of spatial - dimensions is inferred from the shape of the input fields. Please adjust - this argument if you call this function with an array that also contains - channels (even for arrays with singleton channels!). - - Providing correct information regarding the scaling (i.e. providing - `domain_extent` and `num_spatial_dims`) is not necessary if the result is - used to compute a normalized error (e.g. nRMSE) if the normalization is - computed similarly. - - **Arguments**: - - `u_pred` (array): The first field to be used in the loss - - `u_ref` (array, optional): The second field to be used in the error - computation. If `None`, the error will be computed with respect to - zero. - - `domain_extent` (float, optional): The extent of the domain in which - the fields are defined. This is used to scale the error to be - independent of the domain size. Default is 1.0. - - `num_spatial_dims` (int, optional): The number of spatial dimensions - in the field. If `None`, it will be inferred from the shape of the - input fields and then is the number of axes present. Default is - `None`. - - **Returns**: - - `rmse` (float): The (correctly scaled) root mean squared error between - the fields. - """ - if u_ref is None: - diff = u_pred - else: - diff = u_pred - u_ref - - if num_spatial_dims is None: - # Assuming that we only have spatial dimensions - num_spatial_dims = len(u_pred.shape) - - # Todo: Check if we have to divide by 1/L or by 1/L^D for D dimensions - scale = 1 / (domain_extent**num_spatial_dims) - - rmse = jnp.sqrt(scale * jnp.mean(jnp.square(diff))) - return rmse - - -def RMSE( - u_pred: Float[Array, "C ... N"], - u_ref: Optional[Float[Array, "C ... N"]] = None, - domain_extent: float = 1.0, -) -> float: - """ - Compute the root mean squared error (RMSE) between two fields. - - This function assumes that the arrays have one leading channel axis and an - arbitrary number of following spatial dimensions! For batched operation use - `jax.vmap` on this function or use the [`exponax.metrics.mean_RMSE`][] function. - - **Arguments**: - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array, optional): The second field to be used in the error - computation. If `None`, the error will be computed with respect to - zero. - - `domain_extent` (float, optional): The extent of the domain in which - the fields are defined. This is used to scale the error to be - independent of the domain size. Default is 1.0. - - **Returns**: - - `rmse` (float): The (correctly scaled) root mean squared error between - the fields. - """ - - num_spatial_dims = len(u_pred.shape) - 1 - - rmse = _RMSE(u_pred, u_ref, domain_extent, num_spatial_dims=num_spatial_dims) - - return rmse - - -def nRMSE( - u_pred: Float[Array, "C ... N"], - u_ref: Float[Array, "C ... N"], -) -> float: - """ - Compute the normalized root mean squared error (nRMSE) between two fields. - - In contrast to [`exponax.metrics.RMSE`][], no `domain_extent` is required, because of - the normalization. - - **Arguments**: - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array): The second field to be used in the error computation. - - **Returns**: - - `nrmse` (float): The normalized root mean squared error between the - fields - """ - - num_spatial_dims = len(u_pred.shape) - 1 - - # Do not have to supply the domain_extent, because we will normalize with - # the ref_rmse - diff_rmse = _RMSE(u_pred, u_ref, num_spatial_dims=num_spatial_dims) - ref_rmse = _RMSE(u_ref, num_spatial_dims=num_spatial_dims) - - nrmse = diff_rmse / ref_rmse - - return nrmse - - -def mean_RMSE( - u_pred: Float[Array, "B C ... N"], - u_ref: Float[Array, "B C ... N"], - domain_extent: float = 1.0, -) -> float: - """ - Compute the mean RMSE between two fields. Use this function to correctly - operate on arrays with a batch axis. - - **Arguments**: - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array): The second field to be used in the error computation. - - `domain_extent` (float, optional): The extent of the domain in which - - **Returns**: - - `mean_rmse` (float): The mean root mean squared error between the - fields - """ - batch_wise_rmse = jax.vmap(RMSE, in_axes=(0, 0, None))(u_pred, u_ref, domain_extent) - mean_rmse = jnp.mean(batch_wise_rmse) - return mean_rmse - - -def mean_nRMSE( - u_pred: Float[Array, "B C ... N"], - u_ref: Float[Array, "B C ... N"], -): - """ - Compute the mean nRMSE between two fields. Use this function to correctly - operate on arrays with a batch axis. - - **Arguments**: - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array): The second field to be used in the error computation. - - **Returns**: - - `mean_nrmse` (float): The mean normalized root mean squared error - """ - batch_wise_nrmse = jax.vmap(nRMSE)(u_pred, u_ref) - mean_nrmse = jnp.mean(batch_wise_nrmse) - return mean_nrmse - - -def _correlation( - u_pred: Float[Array, "... N"], - u_ref: Float[Array, "... N"], -) -> float: - """ - Low-level function to compute the correlation between two fields. - - This function assumes field without channel axes. Even for singleton channel - axes, use `correlation` for correct operation. - - **Arguments**: - - - `u_pred` (array): The first field to be used in the loss - - `u_ref` (array): The second field to be used in the error computation - - **Returns**: - - - `correlation` (float): The correlation between the fields - """ - u_pred_normalized = u_pred / jnp.linalg.norm(u_pred) - u_ref_normalized = u_ref / jnp.linalg.norm(u_ref) - - correlation = jnp.dot(u_pred_normalized.flatten(), u_ref_normalized.flatten()) - - return correlation - - -def correlation( - u_pred: Float[Array, "C ... N"], - u_ref: Float[Array, "C ... N"], -) -> float: - """ - Compute the correlation between two fields. Average over all channels. - - This function assumes that the arrays have one leading channel axis and an - arbitrary number of following spatial axes. For operation on batched arrays - use `mean_correlation`. - - **Arguments**: - - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array): The second field to be used in the error computation. - - **Returns**: - - - `correlation` (float): The correlation between the fields, averaged over - all channels. - """ - channel_wise_correlation = jax.vmap(_correlation)(u_pred, u_ref) - correlation = jnp.mean(channel_wise_correlation) - return correlation - - -def mean_correlation( - u_pred: Float[Array, "B C ... N"], - u_ref: Float[Array, "B C ... N"], -) -> float: - """ - Compute the mean correlation between multiple samples of two fields. - - This function assumes that the arrays have one leading batch axis, followed - by a channel axis and an arbitrary number of following spatial axes. - - If you want to apply this function on two trajectories of fields, you can - use `jax.vmap` to transform it, use `jax.vmap(mean_correlation, in_axes=I)` - with `I` being the index of the time axis (e.g. `I=0` for time axis at the - beginning of the array, or `I=1` for time axis at the second position, - depending on the convention). - - **Arguments**: - - - `u_pred` (array): The first tensor of fields to be used in the error - computation. - - `u_ref` (array): The second tensor of fields to be used in the error - computation. - - **Returns**: - - - `mean_correlation` (float): The mean correlation between the fields - """ - batch_wise_correlation = jax.vmap(correlation)(u_pred, u_ref) - mean_correlation = jnp.mean(batch_wise_correlation) - return mean_correlation - - -# # Below seems to produce the same resuls as `correlation` -# def pearson_correlation( -# u_pred: Float[Array, "... N"], -# u_ref: Float[Array, "... N"], -# ) -> float: -# """ -# Based on -# https://github.com/pdearena/pdearena/blob/22360a766387c3995220b4a1265a936ab9a81b88/pdearena/modules/loss.py#L39 -# """ - -# u_pred_mean = jnp.mean(u_pred) -# u_ref_mean = jnp.mean(u_ref) - -# u_pred_centered = u_pred - u_pred_mean -# u_ref_centered = u_ref - u_ref_mean - -# # u_pred_std = jnp.sqrt(jnp.mean(u_pred_centered ** 2)) -# # u_ref_std = jnp.sqrt(jnp.mean(u_ref_centered ** 2)) - -# u_pred_std = jnp.std(u_pred) -# u_ref_std = jnp.std(u_ref) - -# # numerator = jnp.sum(u_pred_centered * u_ref_centered) -# # denominator = jnp.sqrt(jnp.sum(u_pred_centered ** 2) * jnp.sum(u_ref_centered ** 2)) - -# # correlation = numerator / denominator - -# correlation = jnp.mean(u_pred_centered * u_ref_centered) / (u_pred_std * u_ref_std) - -# return correlation - - -def _fourier_nRMSE( - u_pred: Float[Array, "... N"], - u_ref: Float[Array, "... N"], - *, - low: Optional[int] = None, - high: Optional[int] = None, - num_spatial_dims: Optional[int] = None, - eps: float = 1e-5, -) -> float: - """ - Low-level function to compute the normalized root mean squared error (nRMSE) - between two fields in Fourier space. - - If `num_spatial_dims` is not provided, it will be inferred from the shape of - the input fields. Please adjust this argument if you call this function with - an array that also contains channels (even for arrays with singleton - channels). - - **Arguments**: - - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array): The second field to be used in the error computation. - - `low` (int, optional): The low-pass filter cutoff. Default is 0. - - `high` (int, optional): The high-pass filter cutoff. Default is the - Nyquist frequency. - - `num_spatial_dims` (int, optional): The number of spatial dimensions in - the field. If `None`, it will be inferred from the shape of the input - fields and then is the number of axes present. Default is `None`. - - `eps` (float, optional): Small value to avoid division by zero and to - remove numerical rounding artiacts from the FFT. Default is 1e-5. - """ - if num_spatial_dims is None: - num_spatial_dims = len(u_pred.shape) - # Assumes we have the same N for all dimensions - num_points = u_pred.shape[-1] - - if low is None: - low = 0 - if high is None: - high = (num_points // 2) + 1 - - low_mask = low_pass_filter_mask( - num_spatial_dims, - num_points, - cutoff=low - 1, # Need to subtract 1 because the cutoff is inclusive - ) - high_mask = low_pass_filter_mask( - num_spatial_dims, - num_points, - cutoff=high, - ) - - mask = jnp.invert(low_mask) & high_mask - - u_pred_fft = fft(u_pred, num_spatial_dims=num_spatial_dims) - u_ref_fft = fft(u_ref, num_spatial_dims=num_spatial_dims) - - # The FFT incurse rounding errors around the machine precision that can be - # noticeable in the nRMSE. We will zero out the values that are smaller than - # the epsilon to avoid this. - u_pred_fft = jnp.where( - jnp.abs(u_pred_fft) < eps, - jnp.zeros_like(u_pred_fft), - u_pred_fft, - ) - u_ref_fft = jnp.where( - jnp.abs(u_ref_fft) < eps, - jnp.zeros_like(u_ref_fft), - u_ref_fft, - ) - - u_pred_fft_masked = u_pred_fft * mask - u_ref_fft_masked = u_ref_fft * mask - - diff_fft_masked = u_pred_fft_masked - u_ref_fft_masked - - # Need to use vdot to correctly operate with complex numbers - diff_norm_unscaled = jnp.sqrt( - jnp.vdot(diff_fft_masked.flatten(), diff_fft_masked.flatten()) - ).real - ref_norm_unscaled = jnp.sqrt( - jnp.vdot(u_ref_fft_masked.flatten(), u_ref_fft_masked.flatten()) - ).real - - nrmse = diff_norm_unscaled / (ref_norm_unscaled + eps) - - return nrmse - - -def fourier_nRMSE( - u_pred: Float[Array, "C ... N"], - u_ref: Float[Array, "C ... N"], - *, - low: Optional[int] = None, - high: Optional[int] = None, - eps: float = 1e-5, -) -> float: - """ - Compute the normalized root mean squared error (nRMSE) between two fields - in Fourier space. - - **Arguments**: - - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array): The second field to be used in the error computation. - - `low` (int, optional): The low-pass filter cutoff. Default is 0. - - `high` (int, optional): The high-pass filter cutoff. Default is the Nyquist - frequency. - - `eps` (float, optional): Small value to avoid division by zero and to - remove numerical rounding artiacts from the FFT. Default is 1e-5. - - **Returns**: - - - `nrmse` (float): The normalized root mean squared error between the fields - """ - num_spatial_dims = len(u_pred.shape) - 1 - - nrmse = _fourier_nRMSE( - u_pred, u_ref, low=low, high=high, num_spatial_dims=num_spatial_dims, eps=eps - ) - - return nrmse - - -def mean_fourier_nRMSE( - u_pred: Float[Array, "B C ... N"], - u_ref: Float[Array, "B C ... N"], - *, - low: Optional[int] = None, - high: Optional[int] = None, - eps: float = 1e-5, -) -> float: - """ - Compute the mean nRMSE between two fields in Fourier space. Use this function - to correctly operate on arrays with a batch axis. - - **Arguments**: - - - `u_pred` (array): The first field to be used in the error computation. - - `u_ref` (array): The second field to be used in the error computation. - - `low` (int, optional): The low-pass filter cutoff. Default is 0. - - `high` (int, optional): The high-pass filter cutoff. Default is the Nyquist - frequency. - - `eps` (float, optional): Small value to avoid division by zero and to - remove numerical rounding artiacts from the FFT. Default is 1e-5. - - **Returns**: - - - `mean_nrmse` (float): The mean normalized root mean squared error between the - fields - """ - batch_wise_nrmse = jax.vmap( - lambda pred, ref: fourier_nRMSE(pred, ref, low=low, high=high, eps=eps) - )(u_pred, u_ref) - mean_nrmse = jnp.mean(batch_wise_nrmse) - return mean_nrmse diff --git a/exponax/metrics/__init__.py b/exponax/metrics/__init__.py new file mode 100644 index 0000000..53520ea --- /dev/null +++ b/exponax/metrics/__init__.py @@ -0,0 +1,50 @@ +from ._correlation import correlation +from ._derivative import H1_MAE, H1_MSE, H1_RMSE, H1_nMAE, H1_nMSE, H1_nRMSE +from ._fourier import ( + fourier_aggregator, + fourier_MAE, + fourier_MSE, + fourier_nMAE, + fourier_nMSE, + fourier_norm, + fourier_nRMSE, + fourier_RMSE, +) +from ._spatial import ( + MAE, + MSE, + RMSE, + nMAE, + nMSE, + nRMSE, + spatial_aggregator, + spatial_norm, +) +from ._utils import mean_metric + +__all__ = [ + "spatial_aggregator", + "spatial_norm", + "MAE", + "MSE", + "RMSE", + "nMAE", + "nMSE", + "nRMSE", + "fourier_aggregator", + "fourier_norm", + "fourier_MAE", + "fourier_MSE", + "fourier_RMSE", + "fourier_nMAE", + "fourier_nMSE", + "fourier_nRMSE", + "correlation", + "H1_MAE", + "H1_nMAE", + "H1_MSE", + "H1_nMSE", + "H1_RMSE", + "H1_nRMSE", + "mean_metric", +] diff --git a/exponax/metrics/_correlation.py b/exponax/metrics/_correlation.py new file mode 100644 index 0000000..cf7a95b --- /dev/null +++ b/exponax/metrics/_correlation.py @@ -0,0 +1,60 @@ +import jax +import jax.numpy as jnp +from jaxtyping import Array, Float + + +def _correlation( + u_pred: Float[Array, "... N"], + u_ref: Float[Array, "... N"], +) -> float: + """ + Low-level function to compute the correlation between two fields. + + This function assumes field without channel axes. Even for singleton channel + axes, use `correlation` for correct operation. + + **Arguments**: + + - `u_pred` (array): The first field to be used in the loss + - `u_ref` (array): The second field to be used in the error computation + + **Returns**: + + - `correlation` (float): The correlation between the fields + """ + u_pred_normalized = u_pred / jnp.linalg.norm(u_pred) + u_ref_normalized = u_ref / jnp.linalg.norm(u_ref) + + correlation = jnp.dot(u_pred_normalized.flatten(), u_ref_normalized.flatten()) + + return correlation + + +def correlation( + u_pred: Float[Array, "C ... N"], + u_ref: Float[Array, "C ... N"], +) -> float: + """ + Compute the correlation between two fields. Average over all channels. + + This function assumes that the arrays have one leading channel axis and an + arbitrary number of following spatial axes. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + **Arguments**: + + - `u_pred`: The first field to be used in the error computation. + - `u_ref`: The second field to be used in the error computation. + + **Returns**: + + - `correlation`: The correlation between the fields, averaged over + all channels. + """ + channel_wise_correlation = jax.vmap(_correlation)(u_pred, u_ref) + correlation = jnp.mean(channel_wise_correlation) + return correlation diff --git a/exponax/metrics/_derivative.py b/exponax/metrics/_derivative.py new file mode 100644 index 0000000..8d4d28f --- /dev/null +++ b/exponax/metrics/_derivative.py @@ -0,0 +1,368 @@ +from typing import Optional + +from jaxtyping import Array, Float + +from ._fourier import ( + fourier_MAE, + fourier_MSE, + fourier_nMAE, + fourier_nMSE, + fourier_nRMSE, + fourier_RMSE, +) + + +def H1_MAE( + u_pred: Float[Array, "C ... N"], + u_ref: Optional[Float[Array, "C ... N"]] = None, + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, +) -> float: + """ + Compute the mean abolute error associated with the H1 norm, i.e., the MAE + across state and all its first derivatives. + + This is **not** consistent with the H1 norm because it uses a Fourier-based + approach. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + !!! warning + Not supplying `domain_extent` will have the result be orders of + magnitude different. + + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention + with a leading channel axis, and either one, two, or three subsequent + spatial axes. + - `u_ref`: The reference state array. Must have the same shape as + `u_pred`. If not specified, the MAE is computed against zero, i.e., the + norm of `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + """ + regular_mae = fourier_MAE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=None, + ) + first_derivative_mae = fourier_MAE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=1, + ) + return regular_mae + first_derivative_mae + + +def H1_nMAE( + u_pred: Float[Array, "C ... N"], + u_ref: Float[Array, "C ... N"], + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, +) -> float: + """ + Compute the normalized mean abolute error associated with the H1 norm, i.e., + the nMAE across state and all its first derivatives. + + This is **not** consistent with the H1 norm because it uses a Fourier-based + approach. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + !!! warning + Not supplying `domain_extent` will have the result be orders of + magnitude different. + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention + with a leading channel axis, and either one, two, or three subsequent + spatial axes. + - `u_ref`: The reference state array. Must have the same shape as + `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + """ + regular_nmae = fourier_nMAE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=None, + ) + first_derivative_nmae = fourier_nMAE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=1, + ) + return regular_nmae + first_derivative_nmae + + +def H1_MSE( + u_pred: Float[Array, "C ... N"], + u_ref: Optional[Float[Array, "C ... N"]] = None, + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, +) -> float: + """ + Compute the mean squared error associated with the H1 norm, i.e., the MSE + across state and all its first derivatives. + + Given the correct `domain_extent`, this is consistent with the squared norm + in the H1 Sobolev space H^1 = W^(1,2): + https://en.wikipedia.org/wiki/Sobolev_space#The_case_p_=_2 + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + !!! warning + Not supplying `domain_extent` will have the result be orders of + magnitude different. + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention + with a leading channel axis, and either one, two, or three subsequent + spatial axes. + - `u_ref`: The reference state array. Must have the same shape as + `u_pred`. If not specified, the MSE is computed against zero, i.e., the + norm of `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + """ + regular_mse = fourier_MSE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=None, + ) + first_derivative_mse = fourier_MSE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=1, + ) + return regular_mse + first_derivative_mse + + +def H1_nMSE( + u_pred: Float[Array, "C ... N"], + u_ref: Float[Array, "C ... N"], + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, +) -> float: + """ + Compute the normalized mean squared error associated with the H1 norm, i.e., + the nMSE across state and all its first derivatives. + + Given the correct `domain_extent`, this is consistent with the **relative** + squared norm in the H1 Sobolev space H^1 = W^(1,2): + https://en.wikipedia.org/wiki/Sobolev_space#The_case_p_=_2 + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + !!! warning + Not supplying `domain_extent` will have the result be orders of + magnitude different. + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention + with a leading channel axis, and either one, two, or three subsequent + spatial axes. + - `u_ref`: The reference state array. Must have the same shape as + `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + """ + regular_nmse = fourier_nMSE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=None, + ) + first_derivative_nmse = fourier_nMSE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=1, + ) + return regular_nmse + first_derivative_nmse + + +def H1_RMSE( + u_pred: Float[Array, "C ... N"], + u_ref: Optional[Float[Array, "C ... N"]] = None, + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, +) -> float: + """ + Compute the root mean squared error associated with the H1 norm, i.e., the + RMSE across state and all its first derivatives. + + Given the correct `domain_extent`, this is consistent with the norm in the + H1 Sobolev space H^1 = W^(1,2): + https://en.wikipedia.org/wiki/Sobolev_space#The_case_p_=_2 + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + !!! warning + Not supplying `domain_extent` will have the result be orders of + magnitude different. + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention + with a leading channel axis, and either one, two, or three subsequent + spatial axes. + - `u_ref`: The reference state array. Must have the same shape as + `u_pred`. If not specified, the RMSE is computed against zero, i.e., the + norm of `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + """ + regular_rmse = fourier_RMSE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=None, + ) + first_derivative_rmse = fourier_RMSE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=1, + ) + return regular_rmse + first_derivative_rmse + + +def H1_nRMSE( + u_pred: Float[Array, "C ... N"], + u_ref: Float[Array, "C ... N"], + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, +) -> float: + """ + Compute the normalized root mean squared error associated with the H1 norm, + i.e., the nRMSE across state and all its first derivatives. + + Given the correct `domain_extent`, this is consistent with the **relative** + norm in the H1 Sobolev space H^1 = W^(1,2): + https://en.wikipedia.org/wiki/Sobolev_space#The_case_p_=_2 + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + !!! warning + Not supplying `domain_extent` will have the result be orders of + magnitude different. + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention + with a leading channel axis, and either one, two, or three subsequent + spatial axes. + - `u_ref`: The reference state array. Must have the same shape as + `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + """ + regular_nrmse = fourier_nRMSE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=None, + ) + first_derivative_nrmse = fourier_nRMSE( + u_pred, + u_ref, + domain_extent=domain_extent, + low=low, + high=high, + derivative_order=1, + ) + return regular_nrmse + first_derivative_nrmse diff --git a/exponax/metrics/_fourier.py b/exponax/metrics/_fourier.py new file mode 100644 index 0000000..fc9fe54 --- /dev/null +++ b/exponax/metrics/_fourier.py @@ -0,0 +1,588 @@ +from typing import Literal, Optional + +import jax +import jax.numpy as jnp +from jaxtyping import Array, Float + +from .._spectral import ( + build_derivative_operator, + build_scaling_array, + fft, + low_pass_filter_mask, +) + + +def fourier_aggregator( + state_no_channel: Float[Array, "... N"], + *, + num_spatial_dims: Optional[int] = None, + domain_extent: float = 1.0, + num_points: Optional[int] = None, + inner_exponent: float = 2.0, + outer_exponent: Optional[float] = None, + low: Optional[int] = None, + high: Optional[int] = None, + derivative_order: Optional[float] = None, +) -> float: + """ + Aggregate over the spatial axes of a (channel-less) state array in Fourier + space. + + While conceptually similar to [`exponax.metrics.spatial_aggregator`][], this + function additionally allows filtering specific frequency ranges and to take + derivatives. In higher dimensions, the derivative contributions (i.e., the + entries of the gradient) are summed up. + + !!! info + The result of this function (under default settings) is (up to rounding + errors) identical to [`exponax.metrics.spatial_aggregator`][] for + `inner_exponent=1.0`. As such, it can be a consistent counterpart for + metrics based on the `L²(Ω)` functional norm. + + !!! tip + To apply this function to a state tensor with a leading channel axis, + use `jax.vmap`. + + **Arguments**: + + - `state_no_channel`: The state tensor **without a leading channel + dimension**. + - `num_spatial_dims`: The number of spatial dimensions. If not specified, + it is inferred from the number of axes in `state_no_channel`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `num_points`: The number of points `N` in each spatial dimension. If not + specified, it is inferred from the last axis of `state_no_channel`. + - `inner_exponent`: The exponent `p` each magnitude of a Fourier coefficient + is raised to before aggregation. + - `outer_exponent`: The exponent `q` the aggregated magnitudes are raised + to. If not specified, it is set to `1/p`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + - `derivative_order`: The order of the derivative to take. If not specified, + no derivative is taken. + """ + if num_spatial_dims is None: + num_spatial_dims = state_no_channel.ndim + if num_points is None: + num_points = state_no_channel.shape[-1] + + if outer_exponent is None: + outer_exponent = 1 / inner_exponent + + # Transform to Fourier space + state_no_channel_hat = fft(state_no_channel, num_spatial_dims=num_spatial_dims) + + # Remove small values that occured due to rounding errors, can become + # problematic for "normalized" norms + state_no_channel_hat = jnp.where( + jnp.abs(state_no_channel_hat) < 1e-5, + jnp.zeros_like(state_no_channel_hat), + state_no_channel_hat, + ) + + # Filtering out if desired + if low is not None or high is not None: + if low is None: + low = 0 + if high is None: + high = (num_points // 2) + 1 + + low_mask = low_pass_filter_mask( + num_spatial_dims, + num_points, + cutoff=low - 1, # Need to subtract 1 because the cutoff is inclusive + ) + high_mask = low_pass_filter_mask( + num_spatial_dims, + num_points, + cutoff=high, + ) + + mask = jnp.invert(low_mask) & high_mask + + state_no_channel_hat = state_no_channel_hat * mask + + # Taking derivatives if desired + if derivative_order is not None: + derivative_operator = build_derivative_operator( + num_spatial_dims, domain_extent, num_points + ) + state_with_derivative_channel_hat = ( + state_no_channel_hat * derivative_operator**derivative_order + ) + else: + # Add singleton derivative axis to have subsequent code work + state_with_derivative_channel_hat = state_no_channel_hat[None] + + # Scale coefficients to extract the correct form, this is needed because we + # use the rfft + scaling_array_recon = build_scaling_array( + num_spatial_dims, + num_points, + mode="reconstruction", + ) + + scale = (domain_extent / num_points) ** num_spatial_dims + + def aggregate(s): + scaled_coefficient_magnitude = ( + jnp.abs(s) ** inner_exponent / scaling_array_recon + ) + aggregated = jnp.sum(scaled_coefficient_magnitude) + return (scale * aggregated) ** outer_exponent + + aggregated_per_derivative = jax.vmap(aggregate)(state_with_derivative_channel_hat) + + return jnp.sum(aggregated_per_derivative) + + +def fourier_norm( + state: Float[Array, "C ... N"], + state_ref: Optional[Float[Array, "C ... N"]] = None, + *, + mode: Literal["absolute", "normalized"] = "absolute", + domain_extent: float = 1.0, + inner_exponent: float = 2.0, + outer_exponent: Optional[float] = None, + low: Optional[int] = None, + high: Optional[int] = None, + derivative_order: Optional[float] = None, +) -> float: + """ + Compute norms of states via aggregation in Fourier space. + + Each channel is treated separately and the results are summed up. + + While conceptually similar to [`exponax.metrics.spatial_norm`][], this + function additionally allows filtering specific frequency ranges and to take + derivatives. In higher dimensions, the derivative contributions (i.e., the + entries of the gradient) are summed up. + + !!! tip + To operate on states with a leading batch axis, use `jax.vmap`. Then the + batch axis can be reduced, e.g., by `jnp.mean`. As a helper for this, + [`exponax.metrics.mean_metric`][] is provided. + + If both `low` and `high` are `None`, the full spectrum is considered. In + this case, this function with `inner_exponent=2.0` (up to rounding errors) + produces the same result as [`exponax.metrics.spatial_norm`][] which is a + consequence of Parseval's theorem. + + + **Arguments**: + + - `state`: The state tensor. Must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `state_ref`: The reference state tensor. Must have the same shape as + `state`. If not specified, only the absolute norm of `state` is + computed. + - `mode`: The mode of the norm. Either `"absolute"` or `"normalized"`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `inner_exponent`: The exponent `p` each magnitude of a Fourier coefficient + is raised to before aggregation. + - `outer_exponent`: The exponent `q` the aggregated magnitudes are raised + to. If not specified, it is set to `1/p`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + - `derivative_order`: The order of the derivative to take. If not specified, + no derivative is taken. + """ + if state_ref is None: + if mode == "normalized": + raise ValueError("mode 'normalized' requires state_ref") + diff = state + else: + diff = state - state_ref + + diff_norm_per_channel = jax.vmap( + lambda s: fourier_aggregator( + s, + domain_extent=domain_extent, + inner_exponent=inner_exponent, + outer_exponent=outer_exponent, + low=low, + high=high, + derivative_order=derivative_order, + ), + )(diff) + + if mode == "normalized": + ref_norm_per_channel = jax.vmap( + lambda r: fourier_aggregator( + r, + domain_extent=domain_extent, + inner_exponent=inner_exponent, + outer_exponent=outer_exponent, + low=low, + high=high, + derivative_order=derivative_order, + ), + )(state_ref) + normalized_diff_per_channel = diff_norm_per_channel / ref_norm_per_channel + norm_per_channel = normalized_diff_per_channel + else: + norm_per_channel = diff_norm_per_channel + + return jnp.sum(norm_per_channel) + + +def fourier_MAE( + u_pred: Float[Array, "C ... N"], + u_ref: Optional[Float[Array, "C ... N"]] = None, + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, + derivative_order: Optional[float] = None, +) -> float: + """ + Compute the mean absolute error in Fourier space. + + ∑_(channels) ∑_(modi) (L/N)ᴰ |fft(uₕ - uₕʳ)| + + The channel axis is summed **after** the aggregation. + + While conceptually similar to [`exponax.metrics.MAE`][], this + function is **not** consistent with the `L¹(Ω)` functional norm. However, it + additionally allows filtering specific frequency ranges and to take + derivatives. In higher dimensions, the derivative contributions (i.e., the + entries of the gradient) are summed up. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention + with a leading channel axis, and either one, two, or three subsequent + spatial axes. + - `u_ref`: The reference state array. Must have the same shape as + `u_pred`. If not specified, the MAE is computed against zero, i.e., the + norm of `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + - `derivative_order`: The order of the derivative to take. If not specified, + no derivative is taken. + """ + return fourier_norm( + u_pred, + u_ref, + mode="absolute", + domain_extent=domain_extent, + inner_exponent=1.0, + outer_exponent=1.0, + low=low, + high=high, + derivative_order=derivative_order, + ) + + +def fourier_nMAE( + u_pred: Float[Array, "C ... N"], + u_ref: Float[Array, "C ... N"], + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, + derivative_order: Optional[float] = None, +) -> float: + """ + Compute the normalized mean absolute error in Fourier space. + + ∑_(channels) (∑_(modi) (L/N)ᴰ |fft(uₕ - uₕʳ)| / ∑_(modi) (L/N)ᴰ + |fft(uₕʳ)|) + + The channel axis is summed **after** the aggregation. + + While conceptually similar to [`exponax.metrics.nMAE`][], this + function is **not** consistent with the `L¹(Ω)` functional norm. However, it + additionally allows filtering specific frequency ranges and to take + derivatives. In higher dimensions, the derivative contributions (i.e., the + entries of the gradient) are summed up. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as + `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + - `derivative_order`: The order of the derivative to take. If not specified, + no derivative is taken. + """ + return fourier_norm( + u_pred, + u_ref, + mode="normalized", + domain_extent=domain_extent, + inner_exponent=1.0, + outer_exponent=1.0, + low=low, + high=high, + derivative_order=derivative_order, + ) + + +def fourier_MSE( + u_pred: Float[Array, "C ... N"], + u_ref: Optional[Float[Array, "C ... N"]] = None, + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, + derivative_order: Optional[float] = None, +) -> float: + """ + Compute the mean squared error in Fourier space. + + ∑_(channels) ∑_(modi) (L/N)ᴰ |fft(uₕ - uₕʳ)|² + + The channel axis is summed **after** the aggregation. + + Under default settings with correctly specific `domain_extent`, this + function (up to rounding errors) produces the identical result as + [`exponax.metrics.MSE`][] which is a consequence of Parseval's theorem. + However, it additionally allows filtering specific frequency ranges and to + take derivatives. In higher dimensions, the derivative contributions (i.e., + the entries of the gradient) are summed up. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as + `u_pred`. If not specified, the MSE is computed against zero, i.e., the + norm of `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + - `derivative_order`: The order of the derivative to take. If not specified, + no derivative is taken. + """ + return fourier_norm( + u_pred, + u_ref, + mode="absolute", + domain_extent=domain_extent, + inner_exponent=2.0, + outer_exponent=1.0, + low=low, + high=high, + derivative_order=derivative_order, + ) + + +def fourier_nMSE( + u_pred: Float[Array, "C ... N"], + u_ref: Float[Array, "C ... N"], + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, + derivative_order: Optional[float] = None, +) -> float: + """ + Compute the normalized mean squared error in Fourier space. + + ∑_(channels) (∑_(modi) (L/N)ᴰ |fft(uₕ - uₕʳ)|² / ∑_(modi) (L/N)ᴰ + |fft(uₕʳ)|²) + + The channel axis is summed **after** the aggregation. + + Under default settings with correctly specific `domain_extent`, this + function (up to rounding errors) produces the identical result as + [`exponax.metrics.nMSE`][] which is a consequence of Parseval's theorem. + However, it additionally allows filtering specific frequency ranges and to + take derivatives. In higher dimensions, the derivative contributions (i.e., + the entries of the gradient) are summed up. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + **Arguments:** + + - `u_pred`: The state array. Must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + - `derivative_order`: The order of the derivative to take. If not specified, + no derivative is taken. + """ + return fourier_norm( + u_pred, + u_ref, + mode="normalized", + domain_extent=domain_extent, + inner_exponent=2.0, + outer_exponent=1.0, + low=low, + high=high, + derivative_order=derivative_order, + ) + + +def fourier_RMSE( + u_pred: Float[Array, "C ... N"], + u_ref: Optional[Float[Array, "C ... N"]] = None, + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, + derivative_order: Optional[float] = None, +) -> float: + """ + Compute the root mean squared error in Fourier space. + + ∑_(channels) √(∑_(modi) (L/N)ᴰ |fft(uₕ - uₕʳ)|²) + + The channel axis is summed **after** the aggregation. + + Under default settings with correctly specific `domain_extent`, this + function (up to rounding errors) produces the identical result as + [`exponax.metrics.RMSE`][] which is a consequence of Parseval's theorem. + However, it additionally allows filtering specific frequency ranges and to + take derivatives. In higher dimensions, the derivative contributions (i.e., + the entries of the gradient) are summed up. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as `u_pred`. + If not specified, the RMSE is computed against zero, i.e., the norm of + `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + - `derivative_order`: The order of the derivative to take. If not specified, + no derivative is taken. + """ + return fourier_norm( + u_pred, + u_ref, + mode="absolute", + domain_extent=domain_extent, + inner_exponent=2.0, + outer_exponent=0.5, + low=low, + high=high, + derivative_order=derivative_order, + ) + + +def fourier_nRMSE( + u_pred: Float[Array, "C ... N"], + u_ref: Float[Array, "C ... N"], + *, + domain_extent: float = 1.0, + low: Optional[int] = None, + high: Optional[int] = None, + derivative_order: Optional[float] = None, +) -> float: + """ + Compute the normalized root mean squared error in Fourier space. + + ∑_(channels) (√(∑_(modi) (L/N)ᴰ |fft(uₕ - uₕʳ)|²) / √(∑_(modi) (L/N)ᴰ + |fft(uₕʳ)|²)) + + The channel axis is summed **after** the aggregation. + + Under default settings with correctly specific `domain_extent`, this + function (up to rounding errors) produces the identical result as + [`exponax.metrics.nRMSE`][] which is a consequence of Parseval's theorem. + However, it additionally allows filtering specific frequency ranges and to + take derivatives. In higher dimensions, the derivative contributions (i.e., + the entries of the gradient) are summed up. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + **Arguments**: + + - `u_pred`: The state array. Must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `low`: The lower cutoff (inclusive) frequency for filtering. If not + specified, it is set to `0`, meaning start it starts (including) the + mean/zero mode. + - `high`: The upper cutoff (inclusive) frequency for filtering. If not + specified, it is set to `N//2 + 1`, meaning it ends (including) at the + Nyquist mode. + - `derivative_order`: The order of the derivative to take. If not specified, + no derivative is taken. + """ + return fourier_norm( + u_pred, + u_ref, + mode="normalized", + domain_extent=domain_extent, + inner_exponent=2.0, + outer_exponent=0.5, + low=low, + high=high, + derivative_order=derivative_order, + ) diff --git a/exponax/metrics/_spatial.py b/exponax/metrics/_spatial.py new file mode 100644 index 0000000..af15ebb --- /dev/null +++ b/exponax/metrics/_spatial.py @@ -0,0 +1,446 @@ +from typing import Literal, Optional + +import jax +import jax.numpy as jnp +from jaxtyping import Array, Float + + +def spatial_aggregator( + state_no_channel: Float[Array, "... N"], + *, + num_spatial_dims: Optional[int] = None, + domain_extent: float = 1.0, + num_points: Optional[int] = None, + inner_exponent: float = 2.0, + outer_exponent: Optional[float] = None, +) -> float: + """ + Aggregate over the spatial axes of a (channel-less) state tensor to get a + *consistent* counterpart to a functional L^p norm in the continuous case. + + Assuming the `Exponax` convention that the domain is always the scaled + hypercube `Ω = (0, L)ᴰ` (with `L = domain_extent`) and each spatial + dimension being discretized uniformly into `N` points (i.e., there are `Nᴰ` + points in total), and the left boundary is considered a degree of freedom, + and the right is not, there is the following relation between a continuous + function `u(x)` and its discretely sampled counterpart `uₕ` + + ‖ u(x) ‖ᵖ_Lᵖ(Ω) = (∫_Ω |u(x)|ᵖ dx)^(1/p) = ( (L/N)ᴰ ∑ᵢ|uᵢ|ᵖ )^(1/p) + + where the summation `∑ᵢ` must be understood as a sum over all `Nᴰ` points + across all spatial dimensions. The `inner_exponent` corresponds to `p` in + the above formula. This function allows setting the outer exponent `q` + manually. If it is not specified, it is set to `1/q = 1/p` to get a valid + norm. + + !!! tip + To apply this function to a state tensor with a leading channel axis, + use `jax.vmap`. + + **Arguments:** + + - `state_no_channel`: The state tensor **without a leading channel + dimension**. + - `num_spatial_dims`: The number of spatial dimensions. If not specified, + it is inferred from the number of axes in `state_no_channel`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `num_points`: The number of points `N` in each spatial dimension. If not + specified, it is inferred from the last axis of `state_no_channel`. + - `inner_exponent`: The exponent `p` in the L^p norm. + - `outer_exponent`: The exponent `q` the result after aggregation is raised + to. If not specified, it is set to `q = 1/p`. + + !!! warning + To get a truly consistent counterpart to the continuous norm, the + `domain_extent` must be set. This is relevant to compare performance + across domain sizes. However, if this is just used as a training + objective, the `domain_extent` can be set to `1.0` since it only + contributes a multiplicative factor. + + !!! info + The approximation to the continuous integral is of the following form: + - **Exact** if the state is bandlimited. + - **Exponentially linearly convergent** if the state is smooth. It + is converged once the state becomes effectively bandlimited + under `num_points`. + - **Polynomially linear** in all other cases. + """ + if num_spatial_dims is None: + num_spatial_dims = state_no_channel.ndim + if num_points is None: + num_points = state_no_channel.shape[-1] + + if outer_exponent is None: + outer_exponent = 1 / inner_exponent + + scale = (domain_extent / num_points) ** num_spatial_dims + + aggregated = jnp.sum(jnp.abs(state_no_channel) ** inner_exponent) + + return (scale * aggregated) ** outer_exponent + + +def spatial_norm( + state: Float[Array, "C ... N"], + state_ref: Optional[Float[Array, "C ... N"]] = None, + *, + mode: Literal["absolute", "normalized"] = "absolute", + domain_extent: float = 1.0, + inner_exponent: float = 2.0, + outer_exponent: Optional[float] = None, +) -> float: + """ + Compute the conistent counterpart of the `Lᴾ` functional norm. + + See [`exponax.metrics.spatial_aggregator`][] for more details. This function + sums over the channel axis **after aggregation**. If you need more low-level + control, consider using [`exponax.metrics.spatial_aggregator`][] directly. + + This function allows providing a second state (`state_ref`) to compute + either the absolute or normalized difference. The `"absolute"` mode computes + + (‖|uₕ − uₕʳ|ᵖ ‖_L²(Ω))^q + + while the `"normalized"` mode computes + + (‖|uₕ − uₕʳ|ᵖ‖_ L²(Ω))^q / (‖|uₕʳ|ᵖ‖_ L²(Ω))^q + + In either way, the channels are summed **after** the aggregation. The + `inner_exponent` corresponds to `p` in the above formulas. The + `outer_exponent` corresponds to `q`. If it is not specified, it is set to `q + = 1/p` to get a valid norm. + + !!! tip + To operate on states with a leading batch axis, use `jax.vmap`. Then the + batch axis can be reduced, e.g., by `jnp.mean`. As a helper for this, + [`exponax.metrics.mean_metric`][] is provided. + + + **Arguments:** + + - `state`: The state tensor. Must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `state_ref`: The reference state tensor. Must have the same shape as + `state`. If not specified, only the absolute norm of `state` is + computed. + - `mode`: The mode of the norm. Either `"absolute"` or `"normalized"`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. + - `inner_exponent`: The exponent `p` in the L^p norm. + - `outer_exponent`: The exponent `q` the result after aggregation is raised + to. If not specified, it is set to `q = 1/p`. + """ + if state_ref is None: + if mode == "normalized": + raise ValueError("mode 'normalized' requires state_ref") + diff = state + else: + diff = state - state_ref + + diff_norm_per_channel = jax.vmap( + lambda s: spatial_aggregator( + s, + domain_extent=domain_extent, + inner_exponent=inner_exponent, + outer_exponent=outer_exponent, + ), + )(diff) + + if mode == "normalized": + ref_norm_per_channel = jax.vmap( + lambda r: spatial_aggregator( + r, + domain_extent=domain_extent, + inner_exponent=inner_exponent, + outer_exponent=outer_exponent, + ), + )(state_ref) + normalized_diff_per_channel = diff_norm_per_channel / ref_norm_per_channel + norm_per_channel = normalized_diff_per_channel + else: + norm_per_channel = diff_norm_per_channel + + return jnp.sum(norm_per_channel) + + +def MAE( + u_pred: Float[Array, "C ... N"], + u_ref: Optional[Float[Array, "C ... N"]] = None, + *, + domain_extent: float = 1.0, +) -> float: + """ + Compute the mean absolute error (MAE) between two states. + + ∑_(channels) ∑_(space) (L/N)ᴰ |uₕ - uₕʳ| + + Given the correct `domain_extent`, this is consistent to the following + functional norm: + + ‖ u - uʳ ‖_L¹(Ω) = ∫_Ω |u(x) - uʳ(x)| dx + + The channel axis is summed **after** the aggregation. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + + **Arguments:** + + - `u_pred`: The state array, must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as `u_pred`. + If not specified, the MAE is computed against zero, i.e., the norm of + `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. Must be + provide to get the correctly consistent norm. If this metric is used an + optimization objective, it can often be ignored since it only + contributes a multiplicative factor. + """ + return spatial_norm( + u_pred, + u_ref, + mode="absolute", + domain_extent=domain_extent, + inner_exponent=1.0, + outer_exponent=1.0, + ) + + +def nMAE( + u_pred: Float[Array, "C ... N"], + u_ref: Float[Array, "C ... N"], + *, + domain_extent: float = 1.0, +) -> float: + """ + Compute the normalized mean absolute error (nMAE) between two states. + + ∑_(channels) [∑_(space) (L/N)ᴰ |uₕ - uₕʳ| / ∑_(space) (L/N)ᴰ |uₕʳ|] + + Given the correct `domain_extent`, this is consistent to the following + functional norm: + + ‖ u - uʳ ‖_L¹(Ω) / ‖ uʳ ‖_L¹(Ω) = ∫_Ω |u(x) - uʳ(x)| dx / ∫_Ω |uʳ(x)| dx + + The channel axis is summed **after** the aggregation. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + + **Arguments:** + + - `u_pred`: The state array, must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. Must be + provide to get the correctly consistent norm. If this metric is used an + optimization objective, it can often be ignored since it only + contributes a multiplicative factor. + """ + return spatial_norm( + u_pred, + u_ref, + mode="normalized", + domain_extent=domain_extent, + inner_exponent=1.0, + outer_exponent=1.0, + ) + + +def MSE( + u_pred: Float[Array, "C ... N"], + u_ref: Optional[Float[Array, "C ... N"]] = None, + *, + domain_extent: float = 1.0, +) -> float: + """ + Compute the mean squared error (MSE) between two states. + + ∑_(channels) ∑_(space) (L/N)ᴰ |uₕ - uₕʳ|² + + Given the correct `domain_extent`, this is consistent to the following + functional norm: + + ‖ u - uʳ ‖²_L²(Ω) = ∫_Ω |u(x) - uʳ(x)|² dx + + The channel axis is summed **after** the aggregation. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + + **Arguments:** + + - `u_pred`: The state array, must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as `u_pred`. + If not specified, the MSE is computed against zero, i.e., the norm of + `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. Must be + provide to get the correctly consistent norm. If this metric is used an + optimization objective, it can often be ignored since it only + contributes a multiplicative factor. + """ + return spatial_norm( + u_pred, + u_ref, + mode="absolute", + domain_extent=domain_extent, + inner_exponent=2.0, + outer_exponent=1.0, + ) + + +def nMSE( + u_pred: Float[Array, "C ... N"], + u_ref: Float[Array, "C ... N"], + *, + domain_extent: float = 1.0, +) -> float: + """ + Compute the normalized mean squared error (nMSE) between two states. + + ∑_(channels) [∑_(space) (L/N)ᴰ |uₕ - uₕʳ|² / ∑_(space) (L/N)ᴰ |uₕʳ|²] + + Given the correct `domain_extent`, this is consistent to the following + functional norm: + + ‖ u - uʳ ‖²_L²(Ω) / ‖ uʳ ‖²_L²(Ω) = ∫_Ω |u(x) - uʳ(x)|² dx / ∫_Ω |uʳ(x)|² dx + + The channel axis is summed **after** the aggregation. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + + **Arguments:** + + - `u_pred`: The state array, must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. Must be + provide to get the correctly consistent norm. If this metric is used an + optimization objective, it can often be ignored since it only + contributes a multiplicative factor. + """ + return spatial_norm( + u_pred, + u_ref, + mode="normalized", + domain_extent=domain_extent, + inner_exponent=2.0, + outer_exponent=1.0, + ) + + +def RMSE( + u_pred: Float[Array, "C ... N"], + u_ref: Optional[Float[Array, "C ... N"]] = None, + *, + domain_extent: float = 1.0, +) -> float: + """ + Compute the root mean squared error (RMSE) between two states. + + (∑_(channels) √(∑_(space) (L/N)ᴰ |uₕ - uₕʳ|²)) + + Given the correct `domain_extent`, this is consistent to the following + functional norm: + + (‖ u - uʳ ‖_L²(Ω)) = (∫_Ω |u(x) - uʳ(x)|² dx) + + The channel axis is summed **after** the aggregation. Hence, it is also + summed **after** the square root. If you need the RMSE per channel, consider + using [`exponax.metrics.spatial_aggregator`][] directly. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + + **Arguments:** + + - `u_pred`: The state array, must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as `u_pred`. + If not specified, the RMSE is computed against zero, i.e., the norm of + `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. Must be + provide to get the correctly consistent norm. If this metric is used an + optimization objective, it can often be ignored since it only + contributes a multiplicative factor + """ + return spatial_norm( + u_pred, + u_ref, + mode="absolute", + domain_extent=domain_extent, + inner_exponent=2.0, + outer_exponent=0.5, + ) + + +def nRMSE( + u_pred: Float[Array, "C ... N"], + u_ref: Float[Array, "C ... N"], + *, + domain_extent: float = 1.0, +) -> float: + """ + Compute the normalized root mean squared error (nRMSE) between two states. + + ∑_(channels) [√(∑_(space) (L/N)ᴰ |uₕ - uₕʳ|²) / √(∑_(space) (L/N)ᴰ + |uₕʳ|²)] + + Given the correct `domain_extent`, this is consistent to the following + functional norm: + + (‖ u - uʳ ‖_L²(Ω) / ‖ uʳ ‖_L²(Ω)) = (∫_Ω |u(x) - uʳ(x)|² dx / ∫_Ω + |uʳ(x)|² dx + + The channel axis is summed **after** the aggregation. Hence, it is also + summed **after** the square root and after normalization. If you need more + fine-grained control, consider using + [`exponax.metrics.spatial_aggregator`][] directly. + + !!! tip + To apply this function to a state tensor with a leading batch axis, use + `jax.vmap`. Then the batch axis can be reduced, e.g., by `jnp.mean`. As + a helper for this, [`exponax.metrics.mean_metric`][] is provided. + + + **Arguments:** + + - `u_pred`: The state array, must follow the `Exponax` convention with a + leading channel axis, and either one, two, or three subsequent spatial + axes. + - `u_ref`: The reference state array. Must have the same shape as `u_pred`. + - `domain_extent`: The extent `L` of the domain `Ω = (0, L)ᴰ`. Must be + provide to get the correctly consistent norm. If this metric is used an + optimization objective, it can often be ignored since it only + contributes a multiplicative factor + """ + return spatial_norm( + u_pred, + u_ref, + mode="normalized", + domain_extent=domain_extent, + inner_exponent=2.0, + outer_exponent=0.5, + ) diff --git a/exponax/metrics/_utils.py b/exponax/metrics/_utils.py new file mode 100644 index 0000000..22a93a1 --- /dev/null +++ b/exponax/metrics/_utils.py @@ -0,0 +1,14 @@ +import jax +import jax.numpy as jnp + + +def mean_metric( + metric_fn, + *args, + **kwargs, +): + """ + 'meanifies' a metric function to operate on arrays with a leading batch axis + """ + wrapped_fn = lambda *a: metric_fn(*a, **kwargs) + return jnp.mean(jax.vmap(wrapped_fn)(*args)) diff --git a/mkdocs.yml b/mkdocs.yml index e7c4c0c..dd541da 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -89,12 +89,14 @@ nav: - 1D Advection: 'examples/simple_advection_example_1d.ipynb' - 1D Solver Showcase: 'examples/solver_showcase_1d.ipynb' - 1D Initial Condition Showcase: 'examples/initial_condition_showcase_1d.ipynb' + - Basics on Metrics: 'examples/on_metrics_simple.ipynb' - Understanding General and Normalized Stepper: 'examples/understanding_general_and_normalized_stepper.ipynb' - Subclassing a custom Solver: 'examples/creating_your_own_solvers_1d.ipynb' - 2D Advection: 'examples/simple_advection_example_2d.ipynb' - 2D Solver Showcase: 'examples/solver_showcase_2d.ipynb' - Advanced: - 1D Burgers Emulator Training: 'examples/learning_burgers_autoregressive_neural_operator.ipynb' + - More on Metrics: 'examples/on_metrics_advanced.ipynb' - Additional: - Nice Features: 'examples/additional_features.ipynb' - Performance Hints: 'examples/performance_hints.ipynb' @@ -165,10 +167,11 @@ nav: - Interpolation: 'api/utilities/interpolation.md' - Normalized & Difficulty: 'api/utilities/normalized_and_difficulty.md' - Metrics: - - MSE-based: 'api/utilities/metrics/mse_based.md' - - RMSE-based: 'api/utilities/metrics/rmse_based.md' + - Spatial: 'api/utilities/metrics/spatial.md' + - Fourier-based: 'api/utilities/metrics/fourier.md' + - Derivative-based: 'api/utilities/metrics/derivative.md' - Correlation: 'api/utilities/metrics/correlation.md' - - Fourier nRMSE: 'api/utilities/metrics/fourier_nrmse.md' + - Utilities: 'api/utilities/metrics/utils.md' - Visualization: - Plot States: 'api/utilities/visualization/plot_states.md' - Plot Spatio-Temporal: 'api/utilities/visualization/plot_spatio_temporal.md' diff --git a/tests/test_metrics.py b/tests/test_metrics.py new file mode 100644 index 0000000..6b93d9e --- /dev/null +++ b/tests/test_metrics.py @@ -0,0 +1,235 @@ +import jax +import jax.numpy as jnp +import pytest + +import exponax as ex + + +@pytest.mark.parametrize("num_spatial_dims", [1, 2, 3]) +def test_constant_offset(num_spatial_dims: int): + DOMAIN_EXTENT = 5.0 + NUM_POINTS = 40 + grid = ex.make_grid(num_spatial_dims, DOMAIN_EXTENT, NUM_POINTS) + + u_0 = 2.0 * jnp.ones_like(grid[0:1]) + u_1 = 4.0 * jnp.ones_like(grid[0:1]) + + assert ex.metrics.MSE(u_1, u_0, domain_extent=1.0) == pytest.approx(4.0) + assert ex.metrics.MSE(u_1, u_0, domain_extent=DOMAIN_EXTENT) == pytest.approx( + DOMAIN_EXTENT**num_spatial_dims * 4.0 + ) + + # MSE metric is symmetric + assert ex.metrics.MSE(u_0, u_1, domain_extent=1.0) == ex.metrics.MSE( + u_1, u_0, domain_extent=1.0 + ) + assert ex.metrics.MSE(u_0, u_1, domain_extent=DOMAIN_EXTENT) == ex.metrics.MSE( + u_1, u_0, domain_extent=DOMAIN_EXTENT + ) + + # == approx(1.0) + assert ex.metrics.nMSE(u_1, u_0) == pytest.approx((4.0 - 2.0) ** 2 / (2.0) ** 2) + assert ex.metrics.nMSE(u_1, u_0) == pytest.approx(1.0) + + # == approx (1/4) + assert ex.metrics.nMSE(u_0, u_1) == pytest.approx((2.0 - 4.0) ** 2 / (4.0) ** 2) + assert ex.metrics.nMSE(u_0, u_1) == pytest.approx(1 / 4) + + assert ex.metrics.RMSE(u_1, u_0, domain_extent=1.0) == pytest.approx(2.0) + assert ex.metrics.RMSE(u_1, u_0, domain_extent=DOMAIN_EXTENT) == pytest.approx( + jnp.sqrt(DOMAIN_EXTENT**num_spatial_dims * 4.0) + ) + + # RMSE is symmetric + assert ex.metrics.RMSE(u_0, u_1, domain_extent=1.0) == ex.metrics.RMSE( + u_1, u_0, domain_extent=1.0 + ) + assert ex.metrics.RMSE(u_0, u_1, domain_extent=DOMAIN_EXTENT) == ex.metrics.RMSE( + u_1, u_0, domain_extent=DOMAIN_EXTENT + ) + + # == approx(1.0) + assert ex.metrics.nRMSE(u_1, u_0) == pytest.approx( + jnp.sqrt((4.0 - 2.0) ** 2 / 2.0**2) + ) + assert ex.metrics.nRMSE(u_1, u_0) == pytest.approx(1.0) + + # == approx(sqrt(1/4)) == approx(0.5) + assert ex.metrics.nRMSE(u_0, u_1) == pytest.approx( + jnp.sqrt((2.0 - 4.0) ** 2 / 4.0**2) + ) + assert ex.metrics.nRMSE(u_0, u_1) == pytest.approx(0.5) + + # The Fourier nRMSE should be identical to the spatial nRMSE + # assert ex.metrics.fourier_nRMSE(u_1, u_0) == ex.metrics.nRMSE(u_1, u_0) + # assert ex.metrics.fourier_nRMSE(u_0, u_1) == ex.metrics.nRMSE(u_0, u_1) + + # The Fourier based losses must be similar to their spatial counterparts due + # to Parseval's identity + assert ex.metrics.fourier_MSE(u_1, u_0) == pytest.approx(ex.metrics.MSE(u_1, u_0)) + assert ex.metrics.fourier_MSE(u_0, u_1) == pytest.approx(ex.metrics.MSE(u_0, u_1)) + # This equivalence does not hold for the MAE + # assert ex.metrics.fourier_MAE(u_1, u_0) == pytest.approx(ex.metrics.MAE(u_1, u_0)) + # assert ex.metrics.fourier_MAE(u_0, u_1) == pytest.approx(ex.metrics.MAE(u_0, u_1)) + assert ex.metrics.fourier_RMSE(u_1, u_0) == pytest.approx(ex.metrics.RMSE(u_1, u_0)) + assert ex.metrics.fourier_RMSE(u_0, u_1) == pytest.approx(ex.metrics.RMSE(u_0, u_1)) + + +def test_fourier_losses(): + # Test specific features of Fourier-based losses like filtering and + # derivatives + pass + + +@pytest.mark.parametrize( + "num_spatial_dims,ic_gen", + [ + (num_spatial_dims, ic_gen) + for num_spatial_dims in [1, 2, 3] + for ic_gen in [ + ex.ic.RandomTruncatedFourierSeries(num_spatial_dims, offset_range=(-1, 1)), + ] + ], +) +def test_fourier_equals_spatial_aggregation(num_spatial_dims, ic_gen): + """ + Must be identical due to Parseval's identity + """ + NUM_POINTS = 40 + DOMAIN_EXTENT = 5.0 + + u_0 = ic_gen(NUM_POINTS, key=jax.random.PRNGKey(0)) + u_1 = ic_gen(NUM_POINTS, key=jax.random.PRNGKey(1)) + + assert ex.metrics.fourier_MSE( + u_1, u_0, domain_extent=DOMAIN_EXTENT + ) == pytest.approx(ex.metrics.MSE(u_1, u_0, domain_extent=DOMAIN_EXTENT)) + # # This equivalence does not hold for the MAE + # assert ex.metrics.fourier_MAE(u_1, u_0, domain_extent=DOMAIN_EXTENT) == pytest.approx( + # ex.metrics.MAE(u_1, u_0, domain_extent=DOMAIN_EXTENT) + # ) + assert ex.metrics.fourier_RMSE( + u_1, u_0, domain_extent=DOMAIN_EXTENT + ) == pytest.approx(ex.metrics.RMSE(u_1, u_0, domain_extent=DOMAIN_EXTENT)) + + +@pytest.mark.parametrize( + "num_spatial_dims,num_points", + [ + (num_spatial_dims, num_points) + for num_spatial_dims in [1, 2, 3] + for num_points in [40, 41] + ], +) +def test_fourier_metric_filtering(num_spatial_dims, num_points): + # It is sufficient only test one fourier_XXXXX metric because they all use + # exponax.metrics.fourier_aggegator to perform the filtering + DOMAIN_EXTENT = 2 * jnp.pi + grid = ex.make_grid(num_spatial_dims, DOMAIN_EXTENT, num_points) + + u = jnp.sin(4 * grid[0:1]) + if num_spatial_dims > 1: + u *= jnp.sin(4 * grid[1:2]) + if num_spatial_dims > 2: + u *= jnp.sin(4 * grid[2:3]) + + # If all modi are included, metric must be non-zero + assert float(ex.metrics.fourier_MSE(u)) != pytest.approx(0.0, abs=1e-6) + # If the lower bound is higher than the active modi, the metric must be zero + assert float(ex.metrics.fourier_MSE(u, low=8)) == pytest.approx(0.0, abs=1e-6) + # If the upper bound is higher than the active modi, the metric must be non-zero + assert float(ex.metrics.fourier_MSE(u, high=8)) != pytest.approx(0.0, abs=1e-6) + # If the upper bound is lower than the active modi, the metric must be zero + assert float(ex.metrics.fourier_MSE(u, high=2)) == pytest.approx(0.0, abs=1e-6) + # If the lower bound is lower than the active modi, the metric must be non-zero + assert float(ex.metrics.fourier_MSE(u, low=2)) != pytest.approx(0.0, abs=1e-6) + # If the selected frequency interval includes all active modi, the metric + # must be non-zero + assert float(ex.metrics.fourier_MSE(u, low=2, high=8)) != pytest.approx( + 0.0, abs=1e-6 + ) + # If the selected frequency interval only considers inactive modi, the metric + # must be zero + assert float(ex.metrics.fourier_MSE(u, low=8, high=16)) == pytest.approx( + 0.0, abs=1e-6 + ) + assert float(ex.metrics.fourier_MSE(u, low=0, high=2)) == pytest.approx( + 0.0, abs=1e-6 + ) + # If the active modi is on the lower end of frequency space, the metric must + # be non-zero + assert float(ex.metrics.fourier_MSE(u, low=4, high=8)) != pytest.approx( + 0.0, abs=1e-6 + ) + # If the active modi is on the upper end of frequency space, the metric must + # be non-zero + assert float(ex.metrics.fourier_MSE(u, low=0, high=4)) != pytest.approx( + 0.0, abs=1e-6 + ) + + +@pytest.mark.parametrize( + "num_spatial_dims,metric_fn_name", + [ + (num_spatial_dims, metric_fn_name) + for num_spatial_dims in [1, 2, 3] + for metric_fn_name in [ + "MAE", + "nMAE", + "MSE", + "nMSE", + "RMSE", + "nRMSE", + ] + ], +) +def test_sobolev_vs_manual(num_spatial_dims, metric_fn_name): + NUM_POINTS = 40 + DOMAIN_EXTENT = 5.0 + + ic_gen = ex.ic.RandomTruncatedFourierSeries(num_spatial_dims, offset_range=(-1, 1)) + u_0 = ic_gen(NUM_POINTS, key=jax.random.PRNGKey(0)) + u_1 = ic_gen(NUM_POINTS, key=jax.random.PRNGKey(1)) + + fourier_metric_fn = getattr(ex.metrics, "fourier_" + metric_fn_name) + sobolev_metric_fn = getattr(ex.metrics, "H1_" + metric_fn_name) + + correct_metric_value = fourier_metric_fn( + u_0, u_1, domain_extent=DOMAIN_EXTENT + ) + fourier_metric_fn(u_0, u_1, domain_extent=DOMAIN_EXTENT, derivative_order=1) + + assert sobolev_metric_fn(u_0, u_1, domain_extent=DOMAIN_EXTENT) == pytest.approx( + correct_metric_value + ) + + +# # Below always evaluates to 2 * pi no matter the values of k and l +# def analytical_L2_diff_norm(k: int, l:int): +# term1 = 2 * jnp.pi +# term2 = -jnp.sin(4 * k * jnp.pi) / (4 * k) +# term3 = (2 * (-(l * jnp.cos(2 * l * jnp.pi) * jnp.sin(2 * k * jnp.pi)) +# + k * jnp.cos(2 * k * jnp.pi) * jnp.sin(2 * l * jnp.pi))) / (k**2 - l**2) +# term4 = -jnp.sin(4 * l * jnp.pi) / (4 * l) + +# result = term1 + term2 + term3 + term4 +# return result + + +@pytest.mark.parametrize("wavenumber_k,wavenumber_l", [(1, 2), (2, 1), (3, 4)]) +def test_analytical_solution_1d(wavenumber_k: int, wavenumber_l: int): + NUM_POINTS = 100 + DOMAIN_EXTENT = 2 * jnp.pi + + grid = ex.make_grid(1, DOMAIN_EXTENT, NUM_POINTS) + + u_0 = jnp.sin(wavenumber_k * grid) + u_1 = jnp.sin(wavenumber_l * grid) + + # assert ex.metrics.MSE(u_1, u_0, domain_extent=DOMAIN_EXTENT) == pytest.approx( + # analytical_L2_diff_norm(k, l) + # ) + assert ex.metrics.MSE(u_1, u_0, domain_extent=DOMAIN_EXTENT) == pytest.approx( + 2 * jnp.pi + ) + assert ex.metrics.nMSE(u_1, u_0) == pytest.approx(2.0) + assert ex.metrics.nMSE(u_1, u_0, domain_extent=DOMAIN_EXTENT) == pytest.approx(2.0) diff --git a/validation/metric_convergence.ipynb b/validation/metric_convergence.ipynb new file mode 100644 index 0000000..c2ee118 --- /dev/null +++ b/validation/metric_convergence.ipynb @@ -0,0 +1,417 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Metric Convergence\n", + "\n", + "MAE, MSE, and RMSE are the consistent counterpart of the L1, squared L2, and L2\n", + "**functional** norm respectively. So they must converge against their integral\n", + "if the resolution is refined." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import jax\n", + "import jax.numpy as jnp\n", + "import matplotlib.pyplot as plt" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import exponax as ex" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "# Should also work without double precision, but with longer floats we see convergence for longer\n", + "# jax.config.update(\"jax_enable_x64\", True)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "def get_difference(\n", + " num_points, domain_extent, metric_fn, true_value, pred_fn, ref_fn=None\n", + "):\n", + " grid_1d = ex.make_grid(1, domain_extent, num_points)\n", + " u = pred_fn(grid_1d)\n", + " if ref_fn is not None:\n", + " u_ref = ref_fn(grid_1d)\n", + " metric_result = metric_fn(u, u_ref, domain_extent=domain_extent)\n", + " else:\n", + " metric_result = metric_fn(u, domain_extent=domain_extent)\n", + " return abs(metric_result - true_value)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "2024-09-23 11:07:19.164501: W external/xla/xla/service/gpu/nvptx_compiler.cc:836] The NVIDIA driver's CUDA version is 12.2 which is older than the PTX compiler version (12.6.68). Because the driver is older than the PTX compiler version, XLA is disabling parallel compilation, which may slow down compilation. You should update your NVIDIA driver or use the NVIDIA-provided CUDA forward compatibility packages.\n" + ] + } + ], + "source": [ + "num_points_range = 2 ** jnp.arange(4, 11)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Smooth Functions\n", + "\n", + "For smooth functions, convergence over resolution $N$ must be exponential." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### MSE\n", + "\n", + "https://www.wolframalpha.com/input?i=int_0%5E%282*pi%29+f%5E2+dx++with+f%28x%29+%3D+e%5E%28-100+*+%28x-1%29%5E2%29+*+sin%28x%29" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "true_value_smooth_mse = (\n", + " 0.0886137772936767597782376343064707483999019551036447917203998375\n", + ")\n", + "domain_extent_smooth_mse = 2 * jnp.pi\n", + "fn_smooth_mse = lambda x: jnp.exp(-100 * (x - 1) ** 2) * jnp.sin(x)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "error_range_smooth_mse = [\n", + " get_difference(\n", + " num_points,\n", + " domain_extent_smooth_mse,\n", + " ex.metrics.MSE,\n", + " true_value_smooth_mse,\n", + " fn_smooth_mse,\n", + " )\n", + " for num_points in num_points_range\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Array(0.08800498, dtype=float32),\n", + " Array(0.03868547, dtype=float32),\n", + " Array(0.00021834, dtype=float32),\n", + " Array(2.9802322e-08, dtype=float32),\n", + " Array(7.450581e-09, dtype=float32),\n", + " Array(7.450581e-09, dtype=float32),\n", + " Array(7.450581e-09, dtype=float32)]" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "error_range_smooth_mse" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[]" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGhCAYAAACphlRxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA/b0lEQVR4nO3de3xU9YH///fM5MYlCZeQhCQDiAoaAhkIudAVFYlFbWOpFWkrFelj3e+PIttuqlv4fvcL0t3W7rfW0pbZuqWPLla0RmzBWluqTdV4QUICg0C4iAYSyI0QyGUgt5n5/RGJYrhkkknOzMnr+XjMHzk5M+c904Z5e87n8zkWn8/nEwAAQIiwGh0AAADAH5QXAAAQUigvAAAgpFBeAABASKG8AACAkEJ5AQAAIYXyAgAAQkqY0QECzev1qqqqStHR0bJYLEbHAQAAveDz+dTc3KykpCRZrVc+t2K68lJVVSW73W50DAAA0AeVlZVKSUm54j6mKy/R0dGSut58TEyMwWkAAEBvNDU1yW63d3+PX4npysuFS0UxMTGUFwAAQkxvhnwwYBcAAIQUygsAAAgplBcAABBSKC8AACCkUF4AAEBIobwAAICQEpTl5ctf/rJGjx6te++91+goAAAgyARlefn2t7+t3/72t0bHAAAAQSgoy8utt97aqxX2AADA0ON3eSkqKlJeXp6SkpJksVi0bdu2Hvs4nU5NmjRJUVFRys7OVnFxcSCyGsrj9WnHh6f1kuukdnx4Wh6vz+hIAAAMSX7fHsDtdis9PV3f/OY3dc899/T4fUFBgfLz8/XUU08pOztb69ev14IFC3T48GHFx8dLkhwOhzo7O3s899VXX1VSUpJfedra2tTW1tb9c1NTk5/v6Oq276/WupfLVN3Y2r1tfGyU1ual6o608QE/HgAAuDyLz+fr8ykEi8WirVu3auHChd3bsrOzlZmZqQ0bNkiSvF6v7Ha7Vq5cqVWrVvX6td944w1t2LBBL7744hX3e+yxx7Ru3boe2xsbGwNyb6Pt+6u1fPNuffZDunDnhV8umUWBAQCgn5qamhQbG9ur7++Ajnlpb29XaWmpcnNzPzmA1arc3Fzt2LEjkIfqtnr1ajU2NnY/KisrA/baHq9P614u61FcJHVvW/dyGZeQAAAYRAEtL/X19fJ4PEpISLhoe0JCgmpqanr9Orm5uVq0aJH+/Oc/KyUl5YrFJzIyUjExMXrmmWeUk5Oj+fPn9zn/ZxWXN1x0qeizfJKqG1tVXH46YMcEAABX5veYl8Hwt7/9ze/nrFixQitWrOg+7RQIdc2XLy6f9r+eKdU/XBenjImjNWviaE1LilFkmC0gGQAAwMUCWl7i4uJks9lUW1t70fba2lolJiYG8lCDIj46qlf7NbV26i/7a/SX/V1nlyLCrJqeHNtVZiaM1qyJo3r9WgAA4MoCWl4iIiKUkZGhwsLC7kG8Xq9XhYWFevjhhwN5qB6cTqecTqc8Hk/AXjPrmjEaHxulmsbWS457sUhKiI3S+vsc2lN5VqXHz2hPxRmddrer9PgZlR4/072vfcwwZUwY3X12ZmpCtMJsQbnMDgAAQc3v2UYtLS06evSoJGnmzJl68sknNW/ePI0ZM0YTJkxQQUGBli5dqv/+7/9WVlaW1q9frxdeeEGHDh3qMRZmIPgzWrk3Lsw2knRRgbncbCOfz6fjp891lZeKM9p9/IwO1zbrs5/y8AibHPZR3WVmln20YoeH9zsvAAChyJ/vb7/LyxtvvKF58+b12L506VJt2rRJkrRhwwb9+Mc/Vk1NjRwOh37+858rOzvbn8P0WaDLi9T/dV6aWju09+MzM6XHz8hVcVbNbT3XubkufuSnzs6M0uS4kbJaLZd4RQAAzGVAy0uw+vRloyNHjgS0vEhd06aLyxtU19yq+OgoZV0zRrY+FguP16ejdS3dZWZPxRl9VO/usV/ssHDNmvDJ2Zn0lFEaERmUY6wBAOiXIVleLhiIMy+D4XRLm/ZUnFVpRVehef/EWbV2eC/ax2qRbhwfo4yJo7sHA6eMHiaLhbMzAIDQRnkJwfLyWR0erw5WN33q7MxZnTx7vsd+46IjLxoInJbMNG0AQOgZkuVloC8bBYPqxvPaffxs92DgsqpGdXgu/p8vwmZVWnJM95mZjImjFR/DNG0AQHAbkuXlArOceemN1g6P9p1s7D47s/t41zTtz0oZPeyiS003JDJNGwAQXCgvQ6S8fJbP51NFw7nuMlN6hWna6Smjumc1zZowWqOGRxgTGgAAUV6GbHm5lObWDu2tbOy+1LSn4oyaW3tO07523IiLzs5cO45p2gCAwTMky8tQGPMSCF6vT0dPtVx0qelS07RjosI0a+JoZUz4eJq2fZRG9nKadiCnlQMAhoYhWV4u4MyL/xrc7dpT8cmlpr2XmaZ9Q2JM96WmjAljZB/Tc5p2fxf0AwAMTZQXyku/dHi8OlTdrNLjDSqtOKvdx89ccpp23MhIZXw8ZiZj4mhVnT2vbz/v6nEfqMvdSgEAgAsoL5SXgKtpbNXuT52dOXCJadpXYpGUGBult793G5eQAAA9+PP9bZq15gfirtL4RGJslO6aPl53Te86c9La4dH+T03Tfu+j02q6xEDgC3ySqhtbVVzeoDnXjh2k1AAAM+LMCwLipT0n9e0C11X3+9lXHfqSI3ngAwEAQoo/39+sVIaA6O0qvvHRrPYLAOgfygsCIuuaMRofG6UrjWYZH9s1bRoAgP6gvCAgbFaL1ualStJlC8z//UIqg3UBAP1GeUHA3JE2Xr9cMkuJsRdfGrpQV443nBv8UAAA02G2EQLqjrTxuj018aIVdo83uLXq9/v05GuHNff6OKUlxxodEwAQwphthAHn8/m0fPNubT9Qo2vHjdCfVs7VsAib0bEAAEGE2UYIKhaLRY/fM13x0ZH68JRbj//loNGRAAAhjPKCQTF6RISeWJQuSfrtjuN6/VCdwYkAAKGK8oJBc/OUcVr2D5MkSY+++L5Ot7QZGwgAEJIoLxhU37vjBk1JGKn6ljat+sM+mWzIFQBgEFBeMKiiwm1av3imImxWvVZWq4JdlUZHAgCEGMoLBl1qUoweWTBFkrTu5TKV17sNTgQACCWmKS9Op1OpqanKzMw0Ogp64R9vmqw5k8fqfIdH3ylwqcPjNToSACBEsM4LDFN19rwWrC9Sc2unvj3/ev3L7VOMjgQAMAjrvCAkJI0aph98ebokacPrR1V6/IzBiQAAoYDyAkPdnZ6khY4kebw+5b/gUktbp9GRAABBjvICw31/YZqSRw3T8dPn9P2XDxgdBwAQ5CgvMFxMVLievC9dFov0QskJbd9fbXQkAEAQo7wgKGRPHqv/75ZrJUmr/rBPtU2tBicCAAQryguCxr/kTtG0pBidPdehR7bslddrqolwAIAAobwgaESEWfWzrzoUGWbVWx/U67c7jhkdCQAQhIKuvFRWVurWW29VamqqZsyYoS1bthgdCYPouvho/Z8v3ChJevwvh3SkttngRACAYBN05SUsLEzr169XWVmZXn31VX3nO9+R283y8UPJN3Im6pYp49TW6dV3nneprdNjdCQAQBAJuvIyfvx4ORwOSVJiYqLi4uLU0NBgbCgMKovFoh8vmqExIyJUVt2kJ187YnQkAEAQ8bu8FBUVKS8vT0lJSbJYLNq2bVuPfZxOpyZNmqSoqChlZ2eruLi4T+FKS0vl8Xhkt9v79HyErvjoKP3onq7Vd39V9JF2fHja4EQAgGDhd3lxu91KT0+X0+m85O8LCgqUn5+vtWvXavfu3UpPT9eCBQtUV1fXvY/D4VBaWlqPR1VVVfc+DQ0NeuCBB/SrX/2qD28LZvD5aYn6WpZdPp/03RdcajzXYXQkAEAQ6NeNGS0Wi7Zu3aqFCxd2b8vOzlZmZqY2bNggSfJ6vbLb7Vq5cqVWrVrVq9dta2vT7bffroceekjf+MY3rrpvW1tb989NTU2y2+3cmNEk3G2d+sLP39Kx0+d0d3qSfv61mUZHAgAMAMNuzNje3q7S0lLl5uZ+cgCrVbm5udqxY0evXsPn8+nBBx/UbbfddtXiIkmPP/64YmNjux9cYjKXEZFh+ulih2xWi/64t0ovuU4aHQkAYLCAlpf6+np5PB4lJCRctD0hIUE1NTW9eo133nlHBQUF2rZtmxwOhxwOh/bt23fZ/VevXq3GxsbuR2VlZb/eA4LPzAmj9c+3XS9J+rdt+3XizDmDEwEAjBRmdIDPuummm+T1enu9f2RkpCIjI+V0OuV0OuXxMK3WjFbMu1ZvHKnTnoqz+u4Le/XcQzmyWS1GxwIAGCCgZ17i4uJks9lUW1t70fba2lolJiYG8lA9rFixQmVlZdq1a9eAHgfGCLNZtX6xQ8MjbNpZ3qCNb31kdCQAgEECWl4iIiKUkZGhwsLC7m1er1eFhYWaM2dOIA/Vg9PpVGpqqjIzMwf0ODDOxLEj9FjeNEnST149rP0nGw1OBAAwgt/lpaWlRS6XSy6XS5JUXl4ul8uliooKSVJ+fr42btyop59+WgcPHtTy5cvldru1bNmygAb/LM68DA2LZqdowbQEdXh8+k6BS60dXCYEgKHG7zEvJSUlmjdvXvfP+fn5kqSlS5dq06ZNWrx4sU6dOqU1a9aopqZGDodD27dv7zGIF+gLi8Wix++ZoT0VRTpa16If/eWQHrt7mtGxAACDqF/rvASTTw/YPXLkCOu8mNybR05p6W+6Vm7etCxTt06NNzgRAKA//FnnxTTl5QJ/3jxC22N/PKBN7x7TuOhIbf/2XI0dGWl0JABAHxm2SB0wmFbdeYOujx+pU81tWv2HfTJZDwcAXIZpyguzjYaeqHCb1n/VoXCbRa+W1eqFEhYoBIChgMtGCHn//eaHevwvhzQ8wqY///NcTYobYXQkAICfuGyEIeUf505WzuQxOtfu0XcKXOr09H6FZgBA6KG8IOTZrBb95D6HoqPC5Ko8qw2vHzU6EgBgAJmmvDDmZWhLHjVM/7EwTZL0i78f1e6KMwYnAgAMFMa8wFS+/fweveSq0sSxw/Xnf56rEZFBd+9RAMAlMOYFQ9b3v5Sm5FHDdPz0Of37n8qMjgMAGACUF5hK7LBw/eS+dFks0vO7KrV9f43RkQAAAUZ5genkTB6r/3XztZKk1X94X3VNrQYnAgAEkmnKCwN28Wn5t09R6vgYnTnXoUdefJ/VdwHARBiwC9P6oLZZX/zF22rr9Grd3dO09HOTjI4EALgMBuwCkq5PiNb/vutGSdIP/3xQH9Q2G5wIABAIlBeY2gNzJuqWKePU1unVt593qb2T1XcBINRRXmBqFotFP753hkYPD1dZdZOefO2I0ZEAAP1EeYHpxcdE6UdfmSFJ+u+iD/XeR6cNTgQA6A/TlBdmG+FKFkxL1Fcz7fL5pO++sFeN5zuMjgQA6CNmG2HIcLd16q6fv6Xjp8/pS44k/eyrM42OBAD4GLONgEsYERmmny52yGa16CVXlV5ynTQ6EgCgDygvGFJmTRitlbddJ0n6t237dfLseYMTAQD8RXnBkPPwvOvksI9Sc2un8gtc8nhNdeUUAEyP8oIhJ8xm1frFDg2PsGlneYN+/dZHRkcCAPiB8oIhaVLcCK3NS5UkPfHqYR2oajQ4EQCgtygvGLLum23X51MT1OHx6TvPu9Ta4TE6EgCgF0xTXljnBf6yWCz60VdmaFx0pD6oa9GP/nLI6EgAgF5gnRcMeW8eOaWlvymWJD39zSzdMmWcwYkAYOhhnRfAD7dMGacHPzdJkvTIlr1qcLcbGwgAcEWUF0DSqjtv0HXxI3WquU2r//C+THZCEgBMhfICSIoKt2n9YofCbRb99UCttpScMDoSAOAyKC/Ax9KSY/Xdz0+VJD328gEdP+02OBEA4FIoL8CnPDR3srKvGaNz7R59p8ClTo/X6EgAgM+gvACfYrNa9ORih6KjwrSn4qycr39odCQAwGdQXoDPSB41TP+xME2S9PO/f6A9FWcMTgQA+LSgKy9nz57V7Nmz5XA4lJaWpo0bNxodCUPQlxzJujs9SR6vT/9S4JK7rdPoSACAjwXdInUej0dtbW0aPny43G630tLSVFJSorFjx/bq+SxSh0BpPN+hO9cXqaqxVV/Lsuvxe2YYHQkATCukF6mz2WwaPny4JKmtrU0+n481N2CI2GHh+sl9Dlks0u+KK/XqgRqjIwEA1IfyUlRUpLy8PCUlJclisWjbtm099nE6nZo0aZKioqKUnZ2t4uJiv45x9uxZpaenKyUlRY8++qji4uL8jQkExJxrx+qfbp4sSVr1h32qa241OBEAwO/y4na7lZ6eLqfTecnfFxQUKD8/X2vXrtXu3buVnp6uBQsWqK6urnufC+NZPvuoqqqSJI0aNUp79+5VeXm5nnvuOdXW1l42T1tbm5qami56AIGUf/sUpY6PUYO7XY9uYfVdADBav8a8WCwWbd26VQsXLuzelp2drczMTG3YsEGS5PV6ZbfbtXLlSq1atcrvY3zrW9/SbbfdpnvvvfeSv3/ssce0bt26HtsZ84JA+qC2WV/8xdtq6/Tq+1+apgfmTDI6EgCYimFjXtrb21VaWqrc3NxPDmC1Kjc3Vzt27OjVa9TW1qq5uVlSVwEpKirS1KlTL7v/6tWr1djY2P2orKzs35sALuH6hGitvvMGSdIPXjmoo3XNBicCgKEroOWlvr5eHo9HCQkJF21PSEhQTU3vBjseP35cc+fOVXp6uubOnauVK1dq+vTpl90/MjJSMTExeuaZZ5STk6P58+f36z0Al/PAnEm6eco4tXV69e3nXWrvZPVdADBCmNEBPisrK0sul8vv561YsUIrVqzoPu0EBJrVatGP752hO9YX6UBVk376tyP63h03GB0LAIacgJ55iYuLk81m6zHAtra2VomJiYE8FGCIhJio7vVennrzQ+386LTBiQBg6AloeYmIiFBGRoYKCwu7t3m9XhUWFmrOnDmBPFQPTqdTqampyszMHNDjAHekJWrxbLt8Pin/hb1qau0wOhIADCl+l5eWlha5XK7uSzvl5eVyuVyqqKiQJOXn52vjxo16+umndfDgQS1fvlxut1vLli0LaPDPWrFihcrKyrRr164BPQ4gSWvyUjVx7HCdPHtea186YHQcABhS/B7zUlJSonnz5nX/nJ+fL0launSpNm3apMWLF+vUqVNas2aNampq5HA4tH379h6DeIFQNiIyTD9d7NCip3Zo656TmndDvO5OTzI6FgAMCUF3b6O+cjqdcjqd8ng8OnLkCOu8YFD89LUj+lnhB4qOCtP279ys5FHDjI4EACHJn3VeTFNeLuDGjBhMHR6vFj21Q67Ks8qZPEbP/WOOrFaL0bEAIOSE9I0ZgVASbrPqp4sdGh5h03sfNejXb39kdCQAMD3TlBdmG8Eo18SN0JovpkqSfvzXwyqr4v5aADCQuGwEBIDP59M/PVOq18pqNSVhpP748E2KCrcZHQsAQgaXjYBBZrFY9J9fmaFx0ZE6Utui/9x+yOhIAGBalBcgQMaMiNCP7+1affd/3jmmoiOnDE4EAOZkmvLCmBcEg1unxmvpnImSpEe27NUZd7vBiQDAfBjzAgTY+XaP8ja8raN1LbpjWqJ+uWSWLBamTwPAlTDmBTDQsAib1i92KNxm0fYDNdpSesLoSABgKpQXYACkJccq//apkqR1fzyg46fdBicCAPMwTXlhzAuCzT/dPFlZ14yRu92jfylwqdPjNToSAJgCY16AAXTizDnduf4tNbd1Kv/2Kfrn+dcbHQkAghJjXoAgkTJ6uP59YZok6WeFH8hVedbYQABgApQXYIAtnJmsu9OT5PH69J3n98jd1ml0JAAIaZQXYBD8+5fSlBQbpWOnz+k/XjlodBwACGmUF2AQxA4P1xP3pctikX5XXKHXymqNjgQAIcs05YXZRgh2n7s2Tv80d7Ik6Xu/f191za0GJwKA0MRsI2AQtXV6tND5rg5WN2ne1HH6zYOZrL4LAGK2ERC0IsNs+tlXHYoIs+r1w6f02x3HtOPD03rJdVI7Pjwtj9dU/y0BAAOCMy+AAf7nnXKte7msx/bxsVFam5eqO9LGG5AKAIzDmRcgyCVER11ye01jq5Zv3q3t+6sHOREAhA7KCzDIPF6f/v2VnmddJOnCadB1L5dxCQkALoPyAgyy4vIGVTdefqaRT1J1Y6uKyxsGLxQAhBDKCzDIejtFmqnUAHBppikvrPOCUBF/mfEufd0PAIYa05SXFStWqKysTLt27TI6CnBFWdeM0fjYKF1udReLumYdZV0zZjBjAUDIME15AUKFzWrR2rxUSbpsgVmblyqblcXrAOBSKC+AAe5IG69fLpmlxNiel4b+zxdvZJ0XALiCMKMDAEPVHWnjdXtqoorLG1TX3Krndh7XzvIzOlbvNjoaAAQ1zrwABrJZLZpz7Vh9yZGsf54/RZK0dfdJtbR1GpwMAIIX5QUIEp+7dqwmx42Qu92jl1wnjY4DAEGL8gIECYvFoq9nT5AkbX6vQia77RgABAzlBQgi92akKDLMqoPVTdpdcdboOAAQlIK2vJw7d04TJ07UI488YnQUYNCMGh6hL85IkiQ9u/O4wWkAIDgFbXn5wQ9+oJycHKNjAINuSU7XpaM/vV+tM+52g9MAQPAJyvLywQcf6NChQ7rzzjuNjgIMOod9lKYlxai906sXS08YHQcAgo7f5aWoqEh5eXlKSkqSxWLRtm3beuzjdDo1adIkRUVFKTs7W8XFxX4d45FHHtHjjz/ubzTAFCwWi5bkTJTUdenI62XgLgB8mt/lxe12Kz09XU6n85K/LygoUH5+vtauXavdu3crPT1dCxYsUF1dXfc+DodDaWlpPR5VVVV66aWXNGXKFE2ZMqXv7woIcXenJyk6MkzHTp/TOx/WGx0HAIKKxdeP+ZgWi0Vbt27VwoULu7dlZ2crMzNTGzZskCR5vV7Z7XatXLlSq1atuuprrl69Wps3b5bNZlNLS4s6Ojr03e9+V2vWrLnk/m1tbWpra+v+uampSXa7XY2NjYqJienrWwMMt+al/frtjuO6Y1qinvpGhtFxAGBANTU1KTY2tlff3wEd89Le3q7S0lLl5uZ+cgCrVbm5udqxY0evXuPxxx9XZWWljh07pieeeEIPPfTQZYvLhf1jY2O7H3a7vd/vAwgGFy4dvXawVjWNrQanAYDgEdDyUl9fL4/Ho4SEhIu2JyQkqKamJpCH6rZ69Wo1NjZ2PyorKwfkOMBgm5IQraxJY+Tx+vT8rgqj4wBA0AjqGzM++OCDV90nMjJSkZGRcjqdcjqd8ng8Ax8MGCT350xQ8bEGPV9cqYfnXacwW1BOEASAQRXQfwnj4uJks9lUW1t70fba2lolJiYG8lA9rFixQmVlZdq1a9eAHgcYTHekJWrsiAjVNLWq8FDd1Z8AAENAQMtLRESEMjIyVFhY2L3N6/WqsLBQc+bMCeShenA6nUpNTVVmZuaAHgcYTJFhNi2a3TWOa/N7rLgLAFIfyktLS4tcLpdcLpckqby8XC6XSxUVXdfk8/PztXHjRj399NM6ePCgli9fLrfbrWXLlgU0+Gdx5gVmdX/2BFks0lsf1OtYvdvoOABgOL/HvJSUlGjevHndP+fn50uSli5dqk2bNmnx4sU6deqU1qxZo5qaGjkcDm3fvr3HIF4AvWMfM1y3TBmnNw6f0nPFFfrfd91odCQAMFS/1nkJJp8esHvkyBHWeYGp/K2sVv/42xKNHh6uHavnKyrcZnQkAAgow9Z5MRKXjWBm826IV1JslM6c69Bf9lcbHQcADGWa8gKYmc1q0deyuu42vfk91nwBMLSZprww2whmtzjLrjCrRaXHz+hgdZPRcQDAMKYpL1w2gtnFR0dpwbSu9ZKYNg1gKDNNeQGGgvuzuy4dbdtzUi1tnQanAQBjUF6AEDLn2rGaPG6E3O0ebdtz0ug4AGAI05QXxrxgKLBYLLo/u+tu05vfOy6TrHQAAH4xTXlhzAuGintnpSgq3KpDNc3aXXHG6DgAMOhMU16AoSJ2eLjyZiRJkp5l2jSAIYjyAoSg+3O6Lh39aV+1zrjbDU4DAIOL8gKEoPSUWKUlx6i906stpZVGxwGAQWWa8sKAXQwlFotFSz4euPvszgp5vQzcBTB0mKa8MGAXQ83djiRFR4bp+OlzevtovdFxAGDQmKa8AEPN8Igw3TMrWZL07E5W3AUwdFBegBB2YeDu3w7Wqaax1eA0ADA4KC9ACJuSEK2sa8bI4/Xpd8VMmwYwNFBegBC35OOzL8/vqlCHx2twGgAYeKYpL8w2wlC1YFqCxo6IUG1TmwoP1hkdBwAGnGnKC7ONMFRFhtl0X6ZdEgN3AQwNpikvwFD29awJsliktz6oV3m92+g4ADCgKC+ACdjHDNetU8ZJkp7j7AsAk6O8ACZxYeDultITau3wGJwGAAYO5QUwiVunxit51DCdPdehP++rNjoOAAwYygtgEjarRV/L6hq4u/k9Lh0BMC/KC2Ai92XaFWa1aHfFWZVVNRkdBwAGhGnKC+u8AFJ8dJQWpCVKkjYzcBeASZmmvLDOC9Dl/uwJkqSX9pxUS1unwWkAIPBMU14AdJkzeayuHTdC7naPtu45aXQcAAg4ygtgMhaLRfdnd02bfva94/L5fAYnAoDAorwAJvSVjBRFhVt1qKZZpcfPGB0HAAKK8gKYUOywcN2dniSJadMAzIfyApjUhUtHf95XowZ3u8FpACBwKC+ASaXbR2l6cqzaPV5tKak0Og4ABAzlBTCxJTld06afK66Q18vAXQDmEJTlZdKkSZoxY4YcDofmzZtndBwgZOWlJyk6KkzHT5/TW0frjY4DAAERlOVFkt599125XC69/vrrRkcBQtbwiDB9ZVaKpK5p0wBgBkFbXgAExoUVd/92sFbVjecNTgMA/ed3eSkqKlJeXp6SkpJksVi0bdu2Hvs4nU5NmjRJUVFRys7OVnFxsV/HsFgsuuWWW5SZmalnn33W34gAPuX6hGhlXzNGXp/0u2IG7gIIfWH+PsHtdis9PV3f/OY3dc899/T4fUFBgfLz8/XUU08pOztb69ev14IFC3T48GHFx8dLkhwOhzo7e95z5dVXX1VSUpLefvttJScnq7q6Wrm5uZo+fbpmzJhxyTxtbW1qa2vr/rmpiTvpAp+1JGeidpY36PniCq287TqF2zjpCiB0WXz9WDvcYrFo69atWrhwYfe27OxsZWZmasOGDZIkr9cru92ulStXatWqVX4f49FHH9W0adP04IMPXvL3jz32mNatW9dje2Njo2JiYvw+HmBG7Z1efe5Hhapvadcv75+lO6ePNzoSAFykqalJsbGxvfr+Duh/frW3t6u0tFS5ubmfHMBqVW5urnbs2NGr13C73WpubpYktbS06O9//7umTZt22f1Xr16txsbG7kdlJafFgc+KCLPqvtl2SdKzOysMTgMA/RPQ8lJfXy+Px6OEhISLtickJKimpqZXr1FbW6ubbrpJ6enpysnJ0QMPPKDMzMzL7h8ZGamYmBg988wzysnJ0fz58/v1HgCz+lrWBFks0ttH61Ve7zY6DgD0md9jXgba5MmTtXfvXr+ft2LFCq1YsaL7tBOAi9nHDNe8qfH6+6E6Pfvecf3bF1ONjgQAfRLQMy9xcXGy2Wyqra29aHttba0SExMDeSgAfXBhxd0tpSfU2uExOA0A9E1Ay0tERIQyMjJUWFjYvc3r9aqwsFBz5swJ5KF6cDqdSk1NveIlJmCou2VKvJJHDVPj+Q698n610XEAoE/8Li8tLS1yuVxyuVySpPLycrlcLlVUdA0CzM/P18aNG/X000/r4MGDWr58udxut5YtWxbQ4J+1YsUKlZWVadeuXQN6HCCU2awWff3jRes272TFXQChye8xLyUlJRfdbyg/P1+StHTpUm3atEmLFy/WqVOntGbNGtXU1MjhcGj79u09BvECMMZ9s+1a/7cj2lNxVgeqGjUtiTFiAEJLv9Z5CSZOp1NOp1Mej0dHjhxhnRfgCh5+brf+9H61vpY1QY/fM93oOABg3DovRuKyEdB7S3ImSpJecp1Uc2uHwWkAwD+mKS8Aei/7mjG6Ln6kzrV7tG3PSaPjAIBfTFNemG0E9J7FYum+2/Tm9ypkkqvHAIYI05QXLhsB/rlnVoqGhdt0uLZZJcfPGB0HAHrNNOUFgH9ih4Xr7vQkSdLm95g2DSB0UF6AIez+j1fc/cu+Gp1uaTM4DQD0jmnKC2NeAP/NSBmlGSmxavd4taX0hNFxAKBXTFNeGPMC9M2S7K5p08/trJDXy8BdAMHPNOUFQN/kpScpOipMFQ3nVPTBKaPjAMBVUV6AIW5YhE1fmZUiqWvaNAAEO9OUF8a8AH235OOBu38/VKuqs+cNTgMAV2aa8sKYF6DvrouPVs7kMfL6pOeLOfsCILiZprwA6J8L9zt6flelOjxeg9MAwOVRXgBIkj6fmqi4kZGqa27Ta2W1RscBgMuivACQJEWEWbU4s2vg7rM7WXEXQPCivADo9rWsCbJYpHeOntZHp1qMjgMAl2Sa8sJsI6D/UkYP121T4yVJz+5k4C6A4GSa8sJsIyAwLgzcfbH0hFo7PAanAYCeTFNeAATGzVPGKXnUMDWe79DLe6uMjgMAPVBeAFzEZrXo69ldi9Zx6QhAMKK8AOhhcaZd4TaLXJVntf9ko9FxAOAilBcAPcSNjNQdaeMlMW0aQPChvAC4pCUfXzratqdKTa0dBqcBgE9QXgBcUtY1Y3R9/Eid7/Bo256TRscBgG6mKS+s8wIElsVi0f0fn33Z/N5x+Xw+gxMBQBfTlBfWeQEC756MFA0Lt+lIbYt2HTtjdBwAkGSi8gIg8GKiwvUlR5KkrrMvABAMKC8Aruj+7K4Vd/+yv1r1LW0GpwEAyguAq5ieEqv0lFh1eHzaUnLC6DgAQHkBcHX3f3y/o+eKj8vrZeAuAGNRXgBcVd6MJMVEhamy4bze/OCU0XEADHGUFwBXNSzCpnsz7JKkZxm4C8BglBcAvXLhZo1/P1Snk2fPG5wGwFBGeQHQK9fFj9ScyWPl9UnPF3O3aQDGCcryUl5ernnz5ik1NVXTp0+X2+02OhIASUs+Hrj7/K5KdXi8BqcBMFQFZXl58MEH9f3vf19lZWV68803FRkZaXQkAJI+Py1B46Ijdaq5Ta8eqDU6DoAhKujKy4EDBxQeHq65c+dKksaMGaOwsDCDUwGQpHCbVYtndw3cZcVdAEbxu7wUFRUpLy9PSUlJslgs2rZtW499nE6nJk2apKioKGVnZ6u4uLjXr//BBx9o5MiRysvL06xZs/TDH/7Q34gABtDXsifIapF2fHRaR+tajI4DYAjyu7y43W6lp6fL6XRe8vcFBQXKz8/X2rVrtXv3bqWnp2vBggWqq6vr3sfhcCgtLa3Ho6qqSp2dnXrrrbf0X//1X9qxY4dee+01vfbaa5fN09bWpqamposeAAZO8qhhuu2GeEnSczsZuAtg8Pl9PebOO+/UnXfeednfP/nkk3rooYe0bNkySdJTTz2lV155Rb/5zW+0atUqSZLL5brs85OTkzV79mzZ7V2npu+66y65XC7dfvvtl9z/8ccf17p16/x9GwD64f6cifrbwTq9WFqpRxdM1bAIm9GRAAwhAR3z0t7ertLSUuXm5n5yAKtVubm52rFjR69eIzMzU3V1dTpz5oy8Xq+Kiop04403Xnb/1atXq7GxsftRWVnZ7/cB4MpuuX6cUkYPU1Nrp15+v8roOACGmICWl/r6enk8HiUkJFy0PSEhQTU1Nb16jbCwMP3whz/UzTffrBkzZuj666/XF7/4xcvuHxkZqZiYGD3zzDPKycnR/Pnz+/UeAFyd1WrpXrTuWS4dARhkQTfbSOq6NLVv3z7t379fTz75ZK+es2LFCpWVlWnXrl0DnA6AJN03265wm0V7K89q/8lGo+MAGEICWl7i4uJks9lUW3vx+g+1tbVKTEwM5KF6cDqdSk1NVWZm5oAeB0CXuJGRujNtvCSmTQMYXAEtLxEREcrIyFBhYWH3Nq/Xq8LCQs2ZMyeQh+qBMy/A4Luw4u5Lrio1tXYYnAbAUOF3eWlpaZHL5eqeMVReXi6Xy6WKiq7r3vn5+dq4caOefvppHTx4UMuXL5fb7e6efQTAPDInjdaUhJE63+HR1t0njY4DYIjwu7yUlJRo5syZmjlzpqSusjJz5kytWbNGkrR48WI98cQTWrNmjRwOh1wul7Zv395jEG+gcdkIGHwWi0X3Z3edfdn83nH5fD6DEwEYCiw+k/1r09TUpNjYWDU2NiomJsboOIDpNbV2KPsHhTrf4VHBP+Uoe/JYoyMBCEH+fH8H5WwjAKEjJipcC2cmSZI2M20awCAwTXnhshFgnAuXjrbvr9ap5jaD0wAwO9OUF2YbAcZJS45Vun2UOjw+bSlllWsAA8s05QWAsZZ8vOLuczsr5PGaaigdgCBDeQEQEHnpSYodFq4TZ86r6Mgpo+MAMDHTlBfGvADGigq36d6MFEmsuAtgYJmmvDDmBTDehZs1/v1wnU6cOWdwGgBmZZryAsB4144bqc9dO1Y+n/R8MQN3AQwMyguAgLpwv6Pnd1WqvdNrcBoAZkR5ARBQt6cmaFx0pOpb2vRqWY3RcQCYkGnKCwN2geAQbrPqq5l2SQzcBTAwTFNeGLALBI+vZU2Q1SK991GDjta1GB0HgMmYprwACB5Jo4bpthu67iT/7E7OvgAILMoLgAGxJKdr2vTvS0/ofLvH4DQAzITyAmBA3Hz9ONnHDFNTa6de3ltldBwAJkJ5ATAgrFaLvp7VNW2aS0cAAsk05YXZRkDwuW92iiJsVu090ah9JxqNjgPAJExTXphtBASfsSMjdef0RElMmwYQOKYpLwCC04UVd1/ae1KN5zsMTgPADCgvAAbU7ImjNSVhpFo7vPrD7hNGxwFgApQXAAPKYrF0n315dmeFfD6fwYkAhDrKC4AB9+WZyRoeYdPRuhbtLG8wOg6AEEd5ATDgoqPC9SVHsiQG7gLoP8oLgEFxf3bXirt/PVCjU81tBqcBEMpMU15Y5wUIbmnJsXLYR6nD49MLJZVGxwEQwkxTXljnBQh+FwbuPrezQh4vA3cB9I1pyguA4PfFGeMVOyxcJ8+e15tH6oyOAyBEUV4ADJqocJsWZaRIkja/V2FwGgChivICYFB9/eOBu68frlNlwzmD0wAIRZQXAINq8riR+ofrxsrnk57fxdkXAP6jvAAYdEuyuwbuFuyqVHun1+A0AEIN5QXAoMtNTVB8dKTqW9r11wM1RscBEGIoLwAGXbjNqq9m2iWx4i4A/wVdeTl8+LAcDkf3Y9iwYdq2bZvRsQAE2FezJshqkXaWN+hoXbPRcQCEkKArL1OnTpXL5ZLL5dLbb7+tESNG6Pbbbzc6FoAASxo1TPNvTJDEtGkA/gm68vJpf/zjHzV//nyNGDHC6CgABsCFFXd/v/uEzrV3GpwGQKjwu7wUFRUpLy9PSUlJslgsl7yk43Q6NWnSJEVFRSk7O1vFxcV9CvfCCy9o8eLFfXougOA397o4TRgzXM2tnXp5b5XRcQCECL/Li9vtVnp6upxO5yV/X1BQoPz8fK1du1a7d+9Wenq6FixYoLq6T5YCdzgcSktL6/GoqvrkH6+mpia9++67uuuuu66Yp62tTU1NTRc9AIQGq9XSvWgdl44A9JbF5/P1+e5oFotFW7du1cKFC7u3ZWdnKzMzUxs2bJAkeb1e2e12rVy5UqtWrer1az/zzDP661//qs2bN19xv8cee0zr1q3rsb2xsVExMTG9Ph4AY5xuadOcx/+udo9Xf3z4HzQjZZTRkQAYoKmpSbGxsb36/g7omJf29naVlpYqNzf3kwNYrcrNzdWOHTv8eq3eXjJavXq1Ghsbux+VlZV+5wZgnLEjI3XX9ERJTJsG0DsBLS/19fXyeDxKSEi4aHtCQoJqanq/EFVjY6OKi4u1YMGCq+4bGRmpmJgYPfPMM8rJydH8+fP9zg3AWBcG7v5xb5Uaz3UYnAZAsAvK2UaxsbGqra1VREREr5+zYsUKlZWVadeuXQOYDMBAyJg4WlMTotXa4dXvd58wOg6AIBfQ8hIXFyebzaba2tqLttfW1ioxMTGQhwJgIhaLRUtyugbuPrvzuPoxFA/AEBDQ8hIREaGMjAwVFhZ2b/N6vSosLNScOXMCeagenE6nUlNTlZmZOaDHATAwFs5M1vAImz485dZ7HzUYHQdAEPO7vLS0tHSvgCtJ5eXlcrlcqqjomuaYn5+vjRs36umnn9bBgwe1fPlyud1uLVu2LKDBP4vLRkBoi44K18KZyZKkzTsZuAvg8sL8fUJJSYnmzZvX/XN+fr4kaenSpdq0aZMWL16sU6dOac2aNaqpqZHD4dD27dt7DOIFgM9akj1Rz+2s0F/316iuuVXx0VFGRwIQhPq1zkswcTqdcjqd8ng8OnLkCOu8ACHqy//1jvZUnNUjn5+ih2+73ug4AAaJYeu8GInLRoA5LMnumjb9u+JKebym+G8rAAFmmvICwBy+MGO8Rg0P18mz5/XG4bqrPwHAkGOa8sJsI8AcosJtWpSRIokVdwFcmmnKC5eNAPP4+seXjt44ckqVDecMTgMg2JimvAAwj2viRuim6+Lk80m/K+Zu0wAuRnkBEJQurLj7Qkml2ju9BqcBEExMU14Y8wKYS+6NCUqIiVR9S7u2H+j9jV0BmJ9pygtjXgBzCbNZ9dXMrrMvDNwF8GmmKS8AzOerWXbZrBYVlzfoSG2z0XEABAnKC4CgNT52mObfEC9Jem4nA3cBdDFNeWHMC2BOS3K6pk3/vvSEzrV3GpwGQDAwzb2NLvDn3ggAgp/X69O8n7yh46fP6aG51ygtOVbx0VHKumaMbFaL0fEABIg/399+31UaAAaT1WpRxoTROn76nDa+Vd69fXxslNbmpeqOtPEGpgNgBNNcNgJgTtv3V2vrnpM9ttc0tmr55t3avr/agFQAjER5ARC0PF6f1r1cpktd276wbd3LZdx9GhhiKC8AglZxeYOqG1sv+3ufpOrGVhWXNwxeKACGM015YbYRYD51zZcvLhft19S7/QCYg2nKCyvsAuYTHx3Vq/3+318P6VdFH+pUc9sAJwIQDExTXgCYT9Y1YzQ+NkpXmxB98myrfvjnQ8p5vFD/+HSJ/nqgRh0ebuYImBXrvAAIatv3V2v55t2SdNHA3QuF5if3pau1w6stpZXaU3G2+/dxIyO00JGs+zLtmpIQPWh5AfSNP9/flBcAQW/7/mqte7nsosG7l1rn5Whds7aUnNDvd59Ufcsnl5DSU2K1aLZdeelJih0WPqjZAfQO5YXyApiOx+tTcXmD6ppbr7rCbofHqzcPn9KW0koVHqxT58dTqSPDrFowLVH3zbbrc9eOlZUVeoGgQXmhvAD4WH1Lm7btOaktJSd0+FN3pk4eNUxfyUjRoowU2ccMNzAhAInyQnkB0IPP59O+k416oaRSf3RVqan1k5s8zpk8Votmp+jOtPEaFmEzMCUwdFFeKC8ArqC1w6O/HqjRi6Un9PbRel34V3BkZJjy0sdr0Wy7ZtpHyWLhshIwWIZkeXE6nXI6nfJ4PDpy5AjlBUCvnDx7Xr8vPaEXS0+oouFc9/Zrx43Qotl23TMzWfExvVtvBkDfDcnycgFnXgD0hdfr087yBm0prdRf9tXofIdHkmSzWnTrlHFaNNuu226IV0QYy2MBA4HyQnkB0A/NrR165f1qbSk9odLjZ7q3jxkRoS/PTNai2Sm6IZF/X4BAorxQXgAEyNG6Fr1YekK/333iotsPTE+O1X2zU3R3erJih7N2DNBflBfKC4AA6/R4VfTBKb2w64QKD9Wqw9P1T2fEx2vHLMpI0T9cF3fZtWcAXBnlhfICYACdbmnTNleVtpRU6lDNJ2vHJMVG6SsZKbo3I0UTx44wMCEQeigvlBcAg8Dn8+lAVZNeKKnUS64qNZ7v6P5d9jVjtGi2XXdNT9TwiDADUwKhgfJCeQEwyFo7PHqtrFZbSk/orQ9Oda8dMyLCpi/OSNJ9mSmaNWE0a8cAl0F5obwAMFDV2fP6w+4T2lJ6QsdPf7J2zORxI7Qow657ZiUrgbVjgIuEfHn56U9/ql//+tfy+XzKzc3Vz372s17/1wrlBUCw8Pm6bia5pfSEXnm/unvtGKtFumXKON032675NyawdgygEC8vp06dUk5Ojg4cOKDw8HDdfPPNeuKJJzRnzpxePZ/yAiAYtbR16s/vV+uFkkqVfGrtmNHDw7VwZrIWZdiVmsS/WRi6/Pn+DspRZJ2dnWptbZUkdXR0KD4+3uBEANA/IyPDdF+mXfdl2vXRqRZtKT2hP+w+odqmNv3PO8f0P+8cU1pyjBZl2PUlR5JGDY8wOjIQtPw+V1lUVKS8vDwlJSXJYrFo27ZtPfZxOp2aNGmSoqKilJ2dreLi4l6//rhx4/TII49owoQJSkpKUm5urq699lp/YwJA0Jo8bqS+d8cNeud7t+l/HszUXdMTFW6zaP/JJq394wFl/aBQK57brTcO18njDaqT40BQ8PvMi9vtVnp6ur75zW/qnnvu6fH7goIC5efn66mnnlJ2drbWr1+vBQsW6PDhw91nUBwOhzo7O3s899VXX9WwYcP0pz/9SceOHdOwYcN05513qqioSDfffPMl87S1tamt7ZNVL5uamvx9SwBgiDCbVfNuiNe8G+LV4G7XS66TeqHkhA5WN+mV96v1yvvVGh8bpXtmdV1WmhTH2jGA1M8xLxaLRVu3btXChQu7t2VnZyszM1MbNmyQJHm9Xtntdq1cuVKrVq266mtu2bJFb7zxhpxOpyTpxz/+sXw+n/71X//1kvs/9thjWrduXY/tjHkBEKr2n2zUi6UntM11UmfPfbJ2TNakMVo0O0V3TR+vEZFBedUf6DN/xrwEdIh7e3u7SktLlZub+8kBrFbl5uZqx44dvXoNu92ud999V62trfJ4PHrjjTc0derUy+6/evVqNTY2dj8qKyv7/T4AwEhpybF67O5p2vm/58v59Vm6deo4WS1S8bEGPfri+8r8wd/06Ja92nWsQZf670+P16cdH57WS66T2vHhaS49XQWfV+8Fy2cV0OpeX18vj8ejhISEi7YnJCTo0KFDvXqNnJwc3XXXXZo5c6asVqvmz5+vu++++7L7R0ZGKjIyUk6nU06nUx6Pp1/vAQCCRWSYTV+YMV5fmDFeNY2t+v3uE9pSUqljp89pS2nXOjLXxI3QvRkp+sqsFCXGRmn7/mqte7lM1Y2t3a8zPjZKa/NSdUfaeAPfTXDi8+q9YPqsAnrZqKqqSsnJyXr33Xcvmtr8r//6r3rzzTe1c+fOfge+GqZKAzAzn8+nkuNn9MKuSr2yr1rn2j9ZO+bG8TE6UNVz3N+FVbJ+uWQWX8ifsn1/tZZv3q3PfgnyefU0GJ+VYVOl4+LiZLPZVFtbe9H22tpaJSYmBvJQPXDmBcBQYLFYlDlpjDInjdFjd0/TK/uq9WLJCRUfa7hkcZHU/YXzf7cd0KSxI7jztbouf/zbtv09vowlPq/PutpnZZG07uUy3Z6aOGif1YAM2M3KytIvfvELSV0DdidMmKCHH364VwN2+4szLwCGoq27T+hfXthrdAwMYb97KEdzrh3b5+cP6JmXlpYWHT16tPvn8vJyuVwujRkzRhMmTFB+fr6WLl2q2bNnKysrS+vXr5fb7dayZcv8fycAgF6x9vK/eEdE2hQZZhvgNMGvrdMjd9vVz9TzefX+s6prbr3qPoHid3kpKSnRvHnzun/Oz8+XJC1dulSbNm3S4sWLderUKa1Zs0Y1NTVyOBzavn17j0G8gcZlIwBDWXx07270+OsHMvv1X8dmsePD0/raxveuuh+fV+8/q97+fzAQgu7eRv3FZSMAQ5HH69NN//l31TS2XnJsgkVSYmyU3v7ebUN+DIfE5+WPwfqsDFvnBQBgDJvVorV5qZI+mQFywYWf1+alDvkv4gv4vHovGD8r05QXp9Op1NRUZWZmGh0FAAxxR9p4/XLJLCXGXnz6PjE2imm/l8Dn1XvB9llx2QgATMbj9am4vEF1za2Kj45S1jVjOINwBXxevTeQn5Vh67wAAIxns1qG/CBTf/B59V6wfFamuWwEAACGBtOUF8a8AAAwNDDmBQAAGI6p0gAAwLQoLwAAIKRQXgAAQEgxTXlhwC4AAEMDA3YBAIDhGLALAABMy3Qr7F44kdTU1GRwEgAA0FsXvrd7c0HIdOWlublZkmS32w1OAgAA/NXc3KzY2Ngr7mO6MS9er1dVVVWKjo6WxRJ6N9bKzMzUrl27jI7Ra0bmHehjB/r1A/F6fX2Nvjyvt89pamqS3W5XZWUl48x6ib/z4Dl2sP2d9+f5of537vP51NzcrKSkJFmtVx7VYrozL1arVSkpKUbH6DObzRZSXwBG5h3oYwf69QPxen19jb48z9/nxMTEhNT/d43E33nwHDvY/s7783wz/J1f7YzLBQzYDTIrVqwwOoJfjMw70McO9OsH4vX6+hp9eV6o/X8xlITaZ8vf+eC9Xn+eP5T+zk132QjA4GJ5AsD8gu3vnDMvAPolMjJSa9euVWRkpNFRAAyQYPs758wLAAAIKZx5AQAAIYXyAgAAQgrlBQAAhBTKCwAACCmUFwAAEFIoLwAGTGVlpW699ValpqZqxowZ2rJli9GRAATY2bNnNXv2bDkcDqWlpWnjxo0DfkymSgMYMNXV1aqtrZXD4VBNTY0yMjJ05MgRjRgxwuhoAALE4/Gora1Nw4cPl9vtVlpamkpKSjR27NgBO6bp7m0EIHiMHz9e48ePlyQlJiYqLi5ODQ0NlBfARGw2m4YPHy5Jamtrk8/n00CfF+GyEYDLKioqUl5enpKSkmSxWLRt27Ye+zidTk2aNElRUVHKzs5WcXHxJV+rtLRUHo9Hdrt9gFMD8Ecg/s7Pnj2r9PR0paSk6NFHH1VcXNyAZqa8ALgst9ut9PR0OZ3OS/6+oKBA+fn5Wrt2rXbv3q309HQtWLBAdXV1F+3X0NCgBx54QL/61a8GIzYAPwTi73zUqFHau3evysvL9dxzz6m2tnZAMzPmBUCvWCwWbd26VQsXLuzelp2drczMTG3YsEGS5PV6ZbfbtXLlSq1atUpS12nk22+/XQ899JC+8Y1vGBEdQC/19e/80771rW/ptttu07333jtgOTnzAqBP2tvbVVpaqtzc3O5tVqtVubm52rFjhyTJ5/PpwQcf1G233UZxAUJQb/7Oa2tr1dzcLElqbGxUUVGRpk6dOqC5KC8A+qS+vl4ej0cJCQkXbU9ISFBNTY0k6Z133lFBQYG2bdsmh8Mhh8Ohffv2GREXQB/05u/8+PHjmjt3rtLT0zV37lytXLlS06dPH9BczDYCMGBuuukmeb1eo2MAGEBZWVlyuVyDekzOvADok7i4ONlsth4D82pra5WYmGhQKgCBFKx/55QXAH0SERGhjIwMFRYWdm/zer0qLCzUnDlzDEwGIFCC9e+cy0YALqulpUVHjx7t/rm8vFwul0tjxozRhAkTlJ+fr6VLl2r27NnKysrS+vXr5Xa7tWzZMgNTA/BHSP6d+wDgMl5//XWfpB6PpUuXdu/zi1/8wjdhwgRfRESELysry/fee+8ZFxiA30Lx75x1XgAAQEhhzAsAAAgplBcAABBSKC8AACCkUF4AAEBIobwAAICQQnkBAAAhhfICAABCCuUFAACEFMoLAAAIKZQXAAAQUigvAAAgpFBeAABASPn/AZauc1EnaZ1CAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.loglog(num_points_range, error_range_smooth_mse, \"-o\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### MAE\n", + "\n", + "https://www.wolframalpha.com/input?i=int_0%5E%282*pi%29+abs%28f%29+dx++with+f%28x%29+%3D+e%5E%28-100+*+%28x-1%29%5E2%29+*+sin%28x%29" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "true_value_smooth_mae = (\n", + " 0.1487744473186810166740730299247331106126900314034849949126656085\n", + ")\n", + "domain_extent_smooth_mae = 2 * jnp.pi\n", + "fn_smooth_mae = lambda x: jnp.exp(-100 * (x - 1) ** 2) * jnp.sin(x)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "error_range_smooth_mae = [\n", + " get_difference(\n", + " num_points,\n", + " domain_extent_smooth_mae,\n", + " ex.metrics.MAE,\n", + " true_value_smooth_mae,\n", + " fn_smooth_mae,\n", + " )\n", + " for num_points in num_points_range\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[]" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGhCAYAAACphlRxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9H0lEQVR4nO3de3xU5b3v8e/MhCRckmAIJAQCUVAgBGYwJiGtKEgseImi1aIeK7L39rSYUt2pttDugvRsde9a3bRlaiun+2C1aNQt1FtRm0qjggSCg2C4iEQIhNy45AZJyMycP0KiGC6ZZJI1s+bzfr3mj6ysWc9vcMb5Zq3f8yyL1+v1CgAAIEhYjS4AAADAF4QXAAAQVAgvAAAgqBBeAABAUCG8AACAoEJ4AQAAQYXwAgAAgkqY0QX4m8fjUXl5uaKiomSxWIwuBwAAdIHX61V9fb0SExNltZ7/3Irpwkt5ebmSkpKMLgMAAHRDWVmZRo4ced59TBdeoqKiJLW9+OjoaIOrAQAAXVFXV6ekpKSO7/HzMV14ab9UFB0dTXgBACDIdKXlg4ZdAAAQVAgvAAAgqBBeAABAUCG8AACAoEJ4AQAAQcU04cXpdColJUXp6elGlwIAAHqRxev1eo0uwp/q6uoUExOj2tpapkoDABAkfPn+Ns2ZFwAAEBoILwAAIKiYboXd3uL2eFVUelRV9U0aFhWpjItjZbNy40cAAPoa4aUL1u04rGWvl+hwbVPHtuExkVqak6LZqcMNrAwAgNDDZaMLWLfjsBY8v/WM4CJJFbVNWvD8Vq3bcdigygAACE2El/Nwe7xa9nqJzjYdq33bstdL5PaYasIWAAABjfByHkWlRzudcfkqr6TDtU0qKj3ad0UBABDiAjK83HLLLbrooot02223GVpHVf25g0t39gMAAD0XkOHlgQce0J/+9Cejy9CwqMgu7XfK7enlSgAAQLuADC/Tp09XVFSU0WUo4+JYDY+J1IUmRP/4lU/00zXbVV3f3Cd1AQAQynwOL4WFhcrJyVFiYqIsFovWrl3baR+n06nk5GRFRkYqMzNTRUVF/qi1z9msFi3NSZGkTgGm/efJI2Pk8UqrNx3Q9Cfe028KPtOJltY+rRMAgFDic3hpbGyU3W6X0+k86+/z8/OVl5enpUuXauvWrbLb7Zo1a5aqqqo69nE4HEpNTe30KC8v7/4r6SWzU4fr6bsvV0LMmZeQEmIi9fu7L9drP7hS+f97quwjY9TY4tZT7+7RjF+t10uby5iFBABAL+jRjRktFovWrFmjOXPmdGzLzMxUenq6VqxYIUnyeDxKSkrSwoULtWjRoi4fe/369VqxYoVeeeWV8+7X3Nys5uYvL9fU1dUpKSnJ7zdmvNAKux6PV29sP6xfrtulg8dOSpLGJ0Rp8fUTdPVlQ/1WBwAAZmTYjRlbWlpUXFys7OzsLwewWpWdna2NGzf6c6gOjz/+uGJiYjoeSUlJvTKOzWpR1pghutkxQlljhnS6NYDVatFN9kQV/Ohq/ez6CYqODNOuinrN++8iffePm7TzcF2v1AUAQKjxa3ipqamR2+1WfHz8Gdvj4+NVUVHR5eNkZ2fr9ttv11tvvaWRI0eeN/gsXrxYtbW1HY+ysrJu1+8PEWE23XfVJSr88Qz985UXq5/Novc/q9H1v3lfD7+8TRXnWTcGAABcWEDe2+hvf/tbl/eNiIhQRESEnE6nnE6n3G53L1bWdYMHhOvnN6bonqzR+uXbu/XmJ4f1cvFBvf5Juf7lykv0/eljNCgiIP/5AQAIaH498xIXFyebzabKysoztldWViohIcGfQ3WSm5urkpISbd68uVfH8dXoIQPlvOtyrbn/G0pPvkhNpzxa8d5eTX/iPT330X61skYMAAA+8Wt4CQ8PV1pamgoKCjq2eTweFRQUKCsry59DdeJ0OpWSkqL09PReHae7poy6SC99L0u/vztNF8cNVE1Di36+dodmLS/UuyWV6kHfNAAAIcXn2UYNDQ3au3evJGnKlCl66qmnNGPGDMXGxmrUqFHKz8/XvHnz9Ic//EEZGRlavny5XnrpJe3atatTL0xv8KVb2Sin3B6t3nRAy/+2R8dOnJIkZV4cq5/dMEGTRw42tjgAAAzgy/e3z+Fl/fr1mjFjRqft8+bN06pVqyRJK1as0BNPPKGKigo5HA795je/UWZmpi/DdFswhJd2dU2n9PT6z/XHD0rV0tp2+ehmR6Ie+tY4JcUOMLg6AAD6Tq+Gl0D11YbdPXv2BEV4aXfo+Ek9+fZuvfrxIUlSeJhV87+RrPtnjFVM/34GVwcAQO8LyfDSLpjOvHzdjkO1euytndrw+RFJ0uAB/fTDay7V3VNHKzwsIG9DBQCAXxBegjS8SJLX69X63dV67K2d+qyqQZI0esgA/WT2eF2XmiCL5UK3iQQAIPiEZHgJ5stGZ9Pq9ujl4oN68p09qmlou/3B5aMG62c3TFDa6FiDqwMAwL9CMry0C/YzL1/X2NyqZwr36ZnCfTp5qm0BvutSE/ST2eOVHDfQ4OoAAPAPwouJwku7yrom/de7e/TSljJ5vFKY1aK7p47WD2deqtiB4UaXBwBAjxBeTBhe2u2uqNfjf92p9burJUlRkWHKnTFW934jWZH9bAZXBwBA94RkeDFbz8uFfPBZjR59a2fH3apHDO6vh2eN0032RFmtNPUCAIJLSIaXdmY/8/JVbo9Xaz4+pCff2a3Dp+9WPWlEjBZfP17fGBNncHUAAHQd4SVEwku7plNu/fGDUj29/nM1NLdKkmaOH6bF14/X2GFRBlcHAMCFEV5CLLy0q2lo1m8KPtOfNx2Q2+OVzWrR3PQk/Wv2ZRoaFWF0eQAAnBPhJUTDS7vPqxv0H3/dpXdLKiVJA8Nt+t7VY/Qv0y7WgPAwg6sDAKCzkAwvodaw2xWb9h3RY2/t1LaDtZKk+OgI/ehb4/Tty0fKRlMvACCAhGR4aceZlzN5PF69sf2wfrlulw4eOylJGp8QpcXXT9DVlw01uDoAANoQXggvnTS3uvWnDfv1279/prqmtqbeaZfG6afXT9CE4fw7AQCMRXghvJzT8RMt+u3f9+pPG7/QKbdXFot02+Uj9aNvjVNCTKTR5QEAQhThhfByQfuPNOqXb+/Wm58cliRF9rPqvmmX6HtXj9GgCJp6AQB9i/BCeOmyjw8c02Nv7dTmL45JkuIGheuB7Mt0Z3qSwmxWg6sDAISKkAwvzDbqPq/Xq7c/rdR/rtul0ppGSdKYoQO16LoJyp4wTBYLM5MAAL0rJMNLO868dN8pt0erNx3Q8r/t0bETpyRJmRfH6mc3TNDkkYONLQ4AYGqEF8JLj9Q1ndLT6z/XHz8oVUurR5J0syNRD31rnJJiBxhcHQDAjAgvhBe/OHT8pJ58e7de/fiQJCk8zKr530jW/TPGKqZ/P4OrAwCYCeGF8OJXOw7V6rG3dmrD50ckSYMH9NMPr7lUd08drfAwmnoBAD1HeCG8+J3X69X63dV67K2d+qyqQZI0esgA/WT2eF2XmnBGU6/b41VR6VFV1TdpWFSkMi6O5XYEAIDzIrwQXnpNq9ujl4sP6sl39qimoVmSdPmowfrZDROUNjpW63Yc1rLXS3S4tqnjOcNjIrU0J0WzU4cbVTYAIMARXggvva6xuVXPFO7TM4X7dPKUW5I0JWmwPi473mnf9nMuT999OQEGAHBWvnx/m6Zhwel0KiUlRenp6UaXEhIGRoTpX6+9TOsfnq470pNkkc4aXCSpPR0ve71Ebo+psjIAwACceYFfvLS5TD/+n08uuN8L901V1pghfVARACCYhOSZFxgrol/X3kpV9U0X3gkAgPMgvMAvhkV17Y7UXd0PAIBzIbzALzIujtXwmEida0K0RW2zjjIuju3LsgAAJkR4gV/YrBYtzUmRpHMGmKU5Kaz3AgDoMcIL/GZ26nA9ffflSog589JQZD8r06QBAH4TZnQBMJfZqcN1bUqCikqPylV2TP+5brdOtXqUNprLRQAA/+DMC/zOZrUoa8wQLZg+VlNGDZbbK71SfNDosgAAJkF4Qa+6M2OUJOnFzQfkYYE6AIAfBFx4KSsr0/Tp05WSkqLJkyfr5ZdfNrok9MCNk4crKiJM+4+c0MZ9R4wuBwBgAgEXXsLCwrR8+XKVlJTonXfe0YMPPqjGxkajy0I3DQgP05wpIyRJq4sOGFwNAMAMAi68DB8+XA6HQ5KUkJCguLg4HT161Nii0CPtl47e+bSi407UAAB0l8/hpbCwUDk5OUpMTJTFYtHatWs77eN0OpWcnKzIyEhlZmaqqKioW8UVFxfL7XYrKSmpW89HYEhJjJY9abBOub36Hxp3AQA95HN4aWxslN1ul9PpPOvv8/PzlZeXp6VLl2rr1q2y2+2aNWuWqqqqOvZxOBxKTU3t9CgvL+/Y5+jRo7rnnnv0zDPPnLee5uZm1dXVnfFA4Lkroy2AvlB0QCa7FygAoI/16K7SFotFa9as0Zw5czq2ZWZmKj09XStWrJAkeTweJSUlaeHChVq0aFGXjtvc3Kxrr71W9913n7773e+ed99HHnlEy5Yt67Sdu0oHlsbmVmU+VqCG5latvi9T3xgTZ3RJAIAAYthdpVtaWlRcXKzs7OwvB7BalZ2drY0bN3bpGF6vV/fee6+uueaaCwYXSVq8eLFqa2s7HmVlZd2uH71nYESYbnYkSpJeKOK/EQCg+/waXmpqauR2uxUfH3/G9vj4eFVUVHTpGB9++KHy8/O1du1aORwOORwObd++/Zz7R0REKDo6Ws8995ymTp2qmTNn9ug1oPe0N+6+vaNCR2jcBQB0U8DdHuDKK6+Ux+Px+Xm5ubnKzc3tOO2EwJM6IkaTR8bok4O1enXrId131SVGlwQACEJ+PfMSFxcnm82mysrKM7ZXVlYqISHBn0MhSLWffaFxFwDQXX4NL+Hh4UpLS1NBQUHHNo/Ho4KCAmVlZflzqE6cTqdSUlKUnp7eq+OgZ3LsiRoYbtO+mkZ9tI/1ewAAvvM5vDQ0NMjlcsnlckmSSktL5XK5dOBA2+qpeXl5WrlypZ599lnt3LlTCxYsUGNjo+bPn+/Xwr8uNzdXJSUl2rx5c6+Og54ZFBGmmxxtK+6+wIq7AIBu8LnnZcuWLZoxY0bHz3l5eZKkefPmadWqVZo7d66qq6u1ZMkSVVRUyOFwaN26dZ2aeP3N6XTK6XTK7Xb36jjouf+VOUovFB3Quh0VOtrYotiB4UaXBAAIIj1a5yUQ+TJPHMbJ+e0H2n6oVv92wwT9yzQadwEg1Bm2zgvQVe2Nu6tp3AUA+IjwAkPc5EjUgHCb9lU3qqiUxl0AQNeZJrww2yi4DDpjxV0adwEAXWea8MJso+DTfunorR0VOtbYYnA1AIBgYZrwguAzaUSMJiZGq6XVo1c/PmR0OQCAIGGa8MJlo+BjsVhYcRcA4DPThBcuGwWnmx2J6t/Ppr1VDdqy/5jR5QAAgoBpwguCU1RkP91kP924u4nGXQDAhRFeYLg7M9suHb2x/bCOn6BxFwBwfoQXGM4+MkYThp9u3N1K4y4A4PxME15o2A1eFotFd2UkSaJxFwBwYaYJLzTsBrebp4xQ/342fVbVoGIadwEA52Ga8ILgFh3ZTzn24ZLa7ncEAMC5EF4QMNrXfHnzk8OqPXHK4GoAAIGK8IKA4UgarPEJUWpu9WjNxweNLgcAEKAILwgYFotFd2W2r7hbRuMuAOCsTBNemG1kDjc7Riiyn1W7K+u19cBxo8sBAAQg04QXZhuZQ0z/frpx8ukVd2ncBQCchWnCC8yjvXH3jU/KVXuSxl0AwJkILwg4l48arHHxUWo65dFfXKy4CwA4E+EFAcdisejO0yvurt7EirsAgDMRXhCQbpkyUhFhVu2qqJer7LjR5QAAAgjhBQEpZkA/3TD59Iq7m2jcBQB8ifCCgHXX6cbd1z8pV10TjbsAgDaEFwSstNEX6dJhg9oadz+mcRcA0MY04YVF6sznqyvu/pnGXQDAaaYJLyxSZ063TBnR0bi77WCt0eUAAAKAacILzGnwgHDdMKmtcfcFGncBACK8IAjcefrS0WvbylVP4y4AhDzCCwLeFaMv0thhg3TylFt/cZUbXQ4AwGCEFwS8thV3286+sOIuAIDwgqBw65QRCg+zquRwnbYfonEXAEIZ4QVB4aKB4bo+NUGS9EIRjbsAEMoILwga7ZeO/uIqV0Nzq8HVAACMEnDh5fjx47riiivkcDiUmpqqlStXGl0SAkTGxbG6ZOhAnWhx6zUadwEgZAVceImKilJhYaFcLpc2bdqkxx57TEeOHDG6LAQAi8XScb+j1UX7Da4GAGCUgAsvNptNAwYMkCQ1NzfL6/UyuwQdbr18pMJtVu04VKftrLgLACHJ5/BSWFionJwcJSYmymKxaO3atZ32cTqdSk5OVmRkpDIzM1VUVOTTGMePH5fdbtfIkSP18MMPKy4uztcyYVKxA8M1+3Tj7moadwEgJPkcXhobG2W32+V0Os/6+/z8fOXl5Wnp0qXaunWr7Ha7Zs2apaqqqo592vtZvv4oL2/rYxg8eLC2bdum0tJSrV69WpWVleesp7m5WXV1dWc8YG7tjbuvuQ7RuAsAIcji7cE1GYvFojVr1mjOnDkd2zIzM5Wenq4VK1ZIkjwej5KSkrRw4UItWrTI5zHuv/9+XXPNNbrtttvO+vtHHnlEy5Yt67S9trZW0dHRPo+HwOf1ejXzyX9oX02jHr91UkeYAQAEr7q6OsXExHTp+9uvPS8tLS0qLi5Wdnb2lwNYrcrOztbGjRu7dIzKykrV19dLagsghYWFGjdu3Dn3X7x4sWprazseZWVlPXsRCHhfXXGXNV8AIPT4NbzU1NTI7XYrPj7+jO3x8fGqqKjo0jH279+vadOmyW63a9q0aVq4cKEmTZp0zv0jIiIUHR19xgPm9+20tsbdTw7Wagcr7gJASAkzuoCvy8jIkMvl8vl5TqdTTqdTbrfb/0Uh4MQODNes1AS9vq1cLxQd0KO3nDvgAgDMxa9nXuLi4mSz2To12FZWViohIcGfQ3WSm5urkpISbd68uVfHQeC4MyNJUtuKu4007gJAyPBreAkPD1daWpoKCgo6tnk8HhUUFCgrK8ufQ3XidDqVkpKi9PT0Xh0HgSPrkiFKHjJADc2teuMTVtwFgFDhc3hpaGiQy+XquLRTWloql8ulAwfaGifz8vK0cuVKPfvss9q5c6cWLFigxsZGzZ8/36+Ffx1nXkLPVxt3VxfRqA0AocLnnpctW7ZoxowZHT/n5eVJkubNm6dVq1Zp7ty5qq6u1pIlS1RRUSGHw6F169Z1auIF/OHbaSP1q3d2a1vZcX1aXquJiTFGlwQA6GU9WuclkHy1YXfPnj2s8xJCcldv1ZufHNZ3p47W/5mTanQ5AIBuMGydFyNx2Sh0td+sce3Hh3SihcZdADA704QXhK6sS4Zo9JABqm9u1RvbDhtdDgCgl5kmvDDbKHRZrRbdkd7euMuKuwBgdqYJL1w2Cm23pY1UmNUiV9lxlZRzc04AMDPThBeEtqFREZo1sW0hxBc3c/YFAMyM8ALTaF/zZc3WQzrZwm0iAMCsTBNe6HnBN8YM0ajY0427rLgLAKZlmvBCzwusVovuOH2/oxdo3AUA0zJNeAGkLxt3tx44rl0VNO4CgBkRXmAqw6IidW1K260oXuR+RwBgSoQXmE574+6rWw/SuAsAJmSa8ELDLtpdOTZOIy/qr7qmVr21nRV3AcBsTBNeaNhFO6vV0nH2hcZdADAf04QX4KtuTxspm9WiLfuPaU9lvdHlAAD8iPACUxoWHansCcMkSas3cfYFAMyE8ALT+mrjbtMpGncBwCwILzCtaZcO1YjBNO4CgNmYJrww2whfZ7NadEc6K+4CgNmYJrww2whn8530JNmsFm3+4pg+o3EXAEzBNOEFOJv46EjNHN/WuPsCK+4CgCkQXmB6d2a2Ne7+D427AGAKhBeY3lWnG3drT57Suh0VRpcDAOghwgtMz2a1aO7pxt3VNO4CQNAjvCAkfOeKJFktUlHpUe2tajC6HABADxBeEBISYiJ1zfh4SdKLnH0BgKBmmvDCOi+4kLsy2y4d0bgLAMHNNOGFdV5wIVdfNkzDYyJ17MQpvf0pjbsAEKxME16ACzmjcZebNQJA0CK8IKS0N+5uKj2qz6tp3AWAYER4QUhJHNxfM8a1rbhL4y4ABCfCC0LOnRltK+6+UnxQza007gJAsCG8IORMHzf0K427lUaXAwDwEeEFISfMZtV3rmhr3H2Bxl0ACDqEF4Sk76S3Ne5u3HdE+2jcBYCgErDh5cSJExo9erQeeugho0uBCY0Y3F/TTzfu5m8uM7gaAIAvAja8PProo5o6darRZcDE2ht3X6ZxFwCCSkCGl88++0y7du3SddddZ3QpMLEZ44YqPjpCRxtb9G4JjbsAECx8Di+FhYXKyclRYmKiLBaL1q5d22kfp9Op5ORkRUZGKjMzU0VFRT6N8dBDD+nxxx/3tTTAJ2E2q+a2N+6y5gsABA2fw0tjY6PsdrucTudZf5+fn6+8vDwtXbpUW7duld1u16xZs1RVVdWxj8PhUGpqaqdHeXm5/vKXv+iyyy7TZZdd1v1XBXTRd9KTZLFIH+49oi9qGo0uBwDQBRav1+vt9pMtFq1Zs0Zz5szp2JaZman09HStWLFCkuTxeJSUlKSFCxdq0aJFFzzm4sWL9fzzz8tms6mhoUGnTp3Sj370Iy1ZsuSs+zc3N6u5ubnj57q6OiUlJam2tlbR0dHdfWkIIff+vyKt312t7189RouuG290OQAQkurq6hQTE9Ol72+/9ry0tLSouLhY2dnZXw5gtSo7O1sbN27s0jEef/xxlZWV6YsvvtCvfvUr3XfffecMLu37x8TEdDySkpJ6/DoQWr5ccbdMLa0eg6sBAFyIX8NLTU2N3G634uPjz9geHx+viooKfw7VYfHixaqtre14lJUx7RW+uWb8MA2LilBNA427ABAMwowu4HzuvffeC+4TERGhiIgIOZ1OOZ1Oud1MeYVv+p1ecXfFe3v1QtEB3TB5uNElAQDOw69nXuLi4mSz2VRZeeZfr5WVlUpISPDnUJ3k5uaqpKREmzdv7tVxYE5zTzfufrC3RvuP0LgLAIHMr+ElPDxcaWlpKigo6Njm8XhUUFCgrKwsfw7VidPpVEpKitLT03t1HJhTUuwATbt0qCTpRVbcBYCA5nN4aWhokMvlksvlkiSVlpbK5XLpwIG2dTLy8vK0cuVKPfvss9q5c6cWLFigxsZGzZ8/36+Ffx1nXtBTd7WvuLuFxl0ACGQ+97xs2bJFM2bM6Pg5Ly9PkjRv3jytWrVKc+fOVXV1tZYsWaKKigo5HA6tW7euUxMvEGhmThimoVERqq5vVsHOSl03id4XAAhEPVrnJZB8tWF3z549rPOCbnni7V1yvve5pl0ap+f+OdPocgAgZBi2zouRuGwEf7gjve3S0fuf1ajs6AmDqwEAnI1pwgvgD22Nu3GSpBc3c78jAAhEpgkvzDaCv7Q37r605aBOuWncBYBAY5rwwmUj+Et2SrziBrU37lZd+AkAgD5lmvAC+Es/m1W3XzFSkvRCEZeOACDQEF6As7gjve0Gn4WfVdO4CwABxjThhZ4X+NPoIQN15dg4eb1SPivuAkBAMU14oecF/nZnR+NuGY27ABBATBNeAH+7NiVeQwaGq6q+WX/fReMuAAQKwgtwDuFhVt1G4y4ABBzCC3Aed55ecfcfe6p18BiNuwAQCEwTXmjYRW9Ijhuob44dIq9XeonGXQAICKYJLzTsore0N+7mbylTK427AGA404QXoLd8KyVBQwaGq7KuWe/trja6HAAIeYQX4ALCw6y6LY3GXQAIFIQXoAvmnl5xd/3uKh06ftLgagAgtBFegC64ZOggZV0yRB4adwHAcKYJL8w2Qm+7M/PLFXdp3AUA45gmvDDbCL1t1sR4XTSgnw7XNmk9jbsAYBjThBegt0WE2WjcBYAAQHgBfHDH6TVf3ttdpXIadwHAEIQXwAdjhg5S5sWxbY27W2jcBQAjEF4AH911unE3f3OZ3B6vwdUAQOghvAA+mjUxoaNx9x97qowuBwBCDuEF8FFkP5u+fXlb4+7qTVw6AoC+RngBuqG9cffvuypVUdtkcDUAEFpME15YpA59aeywQcqgcRcADGGa8MIidehrd2XQuAsARjBNeAH62uzUBMX076dDx0+q8DNW3AWAvkJ4Abrpq427L2xixV0A6CuEF6AH7sxIkiQV7KpSZR2NuwDQFwgvQA9cGh+l9OSL5PZ49dJmGncBoC8QXoAeuvN04+6LNO4CQJ8gvAA9dP2k4YqODNOh4yf1Po27ANDrCC9AD0X2s+nW9sbdIhp3AaC3BWR4SU5O1uTJk+VwODRjxgyjywEuqP3S0d92VqmKxl0A6FVhRhdwLhs2bNCgQYOMLgPoknEJUUobfZGK9x/Ty8UHlTtjrNElAYBpBeSZFyAYtZ99eaHogDw07gJAr/E5vBQWFionJ0eJiYmyWCxau3Ztp32cTqeSk5MVGRmpzMxMFRUV+TSGxWLR1VdfrfT0dP35z3/2tUTAEDdObmvcPXjspD7YW2N0OQBgWj5fNmpsbJTdbtc//dM/6dZbb+30+/z8fOXl5en3v/+9MjMztXz5cs2aNUu7d+/WsGHDJEkOh0Otra2dnvvOO+8oMTFRH3zwgUaMGKHDhw8rOztbkyZN0uTJk89aT3Nzs5qbmzt+rqur8/UlAX7R3ri7asMXeqHogK66bKjRJQGAKVm8Xm+3z29bLBatWbNGc+bM6diWmZmp9PR0rVixQpLk8XiUlJSkhQsXatGiRT6P8fDDD2vixIm69957z/r7Rx55RMuWLeu0vba2VtHR0T6PB/TEroo6zV7+vsKsFm1YfI2GRUUaXRIABIW6ujrFxMR06fvbrz0vLS0tKi4uVnZ29pcDWK3Kzs7Wxo0bu3SMxsZG1dfXS5IaGhr097//XRMnTjzn/osXL1ZtbW3Ho6yMVU5hnPEJ0bp81GC1erx6pfig0eUAgCn5NbzU1NTI7XYrPj7+jO3x8fGqqKjo0jEqKyt15ZVXym63a+rUqbrnnnuUnp5+zv0jIiIUHR2t5557TlOnTtXMmTN79BqAnupYcbeojMZdAOgFATdV+pJLLtG2bdt8fl5ubq5yc3M7TjsBRrlxcqJ+8UaJDhw9oQ2fH9GVl8YZXRIAmIpfz7zExcXJZrOpsrLyjO2VlZVKSEjw51BAwOofbtMtU0ZIklYX7Te4GgAwH7+Gl/DwcKWlpamgoKBjm8fjUUFBgbKysvw5VCdOp1MpKSnnvcQE9JU70tsuHb3zaaWq65svsDcAwBc+h5eGhga5XC65XC5JUmlpqVwulw4caLunS15enlauXKlnn31WO3fu1IIFC9TY2Kj58+f7tfCvy83NVUlJiTZv3tyr4wBdkZIYLUcSjbsA0Bt87nnZsmXLGfcbysvLkyTNmzdPq1at0ty5c1VdXa0lS5aooqJCDodD69at69TEC5jdXRmj5Co7rhc3H9D3rrpEVqvF6JIAwBR6tM5LIHE6nXI6nXK73dqzZw/rvMBwJ1palfFogRqaW/Xnf8nUN8fSuAsA52LYOi9G4rIRAs2A8DDNmZIoSVpddMDgagDAPEwTXoBA1L7myzufVqimgcZdAPAH04QXZhshEE1MjJE9abBOub36Hxp3AcAvTBNeuGyEQHVXRpIk6YWiAzJJixkAGMo04QUIVDdOTtSgiDB9ceSENu47YnQ5ABD0CC9ALxsYEaabHW2Nuy8UceNQAOgp04QXel4QyNobd9/eUaEjNO4CQI+YJrzQ84JAljoiRpNHxqjF7dGrWw8ZXQ4ABDXThBcg0LWffaFxFwB6hvAC9JEce6IGhtu0r6ZRH+07anQ5ABC0TBNe6HlBoBsUEaabHCMktZ19AQB0j2nCCz0vCAZ3nb50tG5HhY42thhcDQAEJ9OEFyAYTBoZo9QR0acbd1lxFwC6g/AC9LH2xt3VNO4CQLcQXoA+dpM9UQPCbdpX3aiiUhp3AcBXhBegj0VF9tNN9vYVd2ncBQBfmSa8MNsIweSuzLZLR2/tqNAxGncBwCcWr8kuutfV1SkmJka1tbWKjo42uhzgrLxer2787Qf6tLxO3506WlckX6RhUZHKuDhWNqvF6PIAoM/58v0d1kc1AfgKi8WiySNj9Gl5nZ77aL+e+2i/JGl4TKSW5qRodupwgysEgMBlmstGQDBZt+OwXjzLHaYrapu04PmtWrfjsAFVAUBwILwAfczt8WrZ6yU62/Xa9m3LXi+R22OqK7oA4DeEF6CPFZUe1eHapnP+3ivpcG0T06gB4BwIL0Afq6o/d3Dpzn4AEGoIL0AfGxYV6df9ACDUEF6APpZxcayGx0TqfBOih8e0TZsGAHRmmvDCInUIFjarRUtzUiTpnAHmx7PHs94LAJyDacJLbm6uSkpKtHnzZqNLAS5odupwPX335UqIOfPSkO10Xtmwt8aAqgAgOLBIHWCQ2anDdW1KgopKj6qqvqmtx8Xr1V1/3KSXiw9qxvhhun4Si9UBwNcRXgAD2awWZY0Zcsa2BVeP0e/Wf67Fr27XlFGDNTymv0HVAUBgMs1lI8AsHsy+TJNHxqj25Cn96KVt8rBYHQCcgfACBJjwMKuWz3Wofz+bNnx+RP/3g31GlwQAAYXwAgSgS4YO0pLTM5KeeHu3Pi2vNbgiAAgchBcgQN2RnqRrU+J1yu3VAy+6dLLFbXRJABAQCC9AgLJYLPrPb0/W0KgI7a1q0ON/3Wl0SQAQEAIyvJSWlmrGjBlKSUnRpEmT1NjYaHRJgCFiB4brydvtkqQ/bdyvv++qNLgiADBeQIaXe++9V7/4xS9UUlKif/zjH4qIiDC6JMAwV102VP/0zYslST9+5RNV1zcbXBEAGCvgwsunn36qfv36adq0aZKk2NhYhYWxHA1C249nj9P4hCjVNLTox69sk9fL9GkAocvn8FJYWKicnBwlJibKYrFo7dq1nfZxOp1KTk5WZGSkMjMzVVRU1OXjf/bZZxo0aJBycnJ0+eWX67HHHvO1RMB0IvvZtPwOh8LDrHpvd7We/2i/0SUBgGF8Di+NjY2y2+1yOp1n/X1+fr7y8vK0dOlSbd26VXa7XbNmzVJVVVXHPg6HQ6mpqZ0e5eXlam1t1fvvv6/f/e532rhxo9599129++6756ynublZdXV1ZzwAMxqfEK1Fs8dLkv79zZ36rLLe4IoAwBgWbw/OP1ssFq1Zs0Zz5szp2JaZman09HStWLFCkuTxeJSUlKSFCxdq0aJFFzzmxo0b9cgjj+jtt9+WJD3xxBOSpIcffvis+z/yyCNatmxZp+21tbWKjo729SUBAc3j8ereVZtVuKdaKcOjtSb3G4oIsxldFgD0WF1dnWJiYrr0/e3XnpeWlhYVFxcrOzv7ywGsVmVnZ2vjxo1dOkZ6erqqqqp07NgxeTweFRYWasKECefcf/Hixaqtre14lJWV9fh1AIHKarXoV7dNVuzAcJUcrtOT7+wxuiQA6HN+DS81NTVyu92Kj48/Y3t8fLwqKiq6dIywsDA99thjuuqqqzR58mRdeumluvHGG8+5f0REhKKjo894AGY2LDpS/3HrJEnSM4X79OHeGoMrAoC+FXCzjSTpuuuu0/bt27Vjxw499dRTXXqO0+lUSkqK0tPTe7k6wHjfmpigOzNGSZJ+9NI2HT/RYnBFANB3/Bpe4uLiZLPZVFl55kJalZWVSkhI8OdQneTm5qqkpESbN2/u1XGAQPHzGyfokriBqqhr0uJXtzN9GkDI8Gt4CQ8PV1pamgoKCjq2eTweFRQUKCsry59DdcKZF4SaAeFh+vUdUxRmteivOyr0cvFBo0sCgD7hc3hpaGiQy+WSy+WS1LaUv8vl0oEDByRJeXl5WrlypZ599lnt3LlTCxYsUGNjo+bPn+/Xwr+OMy8IRZNGxijvW5dJkpa99qm+qOFWGgDMz+ela7ds2aIZM2Z0/JyXlydJmjdvnlatWqW5c+equrpaS5YsUUVFhRwOh9atW9epiReAf3zvqjFav7taRaVH9WC+Sy9/P0v9bAHZzgYAftGjdV4CidPplNPplNvt1p49e1jnBSHl0PGTmr28UPVNrfrhzEuVd+1lRpcEAD7xZZ0X04SXdr68eMBMXttWrh++8LGsFuml72XpiuRYo0sCgC4zbJE6AMa5yZ6oW6eMkMcrPZjvUn3TKaNLAoBeYZrwwmwjQFp280SNvKi/Dh47qaV/+dTocgCgV3DZCDCZLV8c1Xf+sFEer/SbO6foJnui0SUBwAVx2QgIYVckx+oHM8ZKkn62ZrsOHT9pcEUA4F+EF8CEFs68VI6kwapvalVevktuj6lOsAIIcaYJL/S8AF/qZ7Nq+VyHBoTbtKn0qP5Q+LnRJQGA39DzApjYS1vK9ONXPlGY1aI1939Tk0bGGF0SAJwVPS8AJEm3p43UdakJavV49cCLH+tES6vRJQFAjxFeABOzWCx6/NZJSoiO1L6aRv37mzuNLgkAeozwApjc4AHhevI7dknS6k0H9M6nFQZXBAA9Y5rwQsMucG7fHBun/33VJZKkRa9uV1Vdk8EVAUD30bALhIjmVrducW5QyeE6XXXZUK26N11Wq8XosgBAEg27AM4iIsym39zpUESYVYV7qvXsxi+MLgkAuoXwAoSQscOi9G83TJAkPf7XXdpVUWdwRQDgO8ILEGLunjpa14wfppZWjx580aWmU26jSwIAnxBegBBjsVj0n9+erLhB4dpVUa9frtttdEkA4BPThBdmGwFdNzQqQk/c1jZ9+r8/LFXhnmqDKwKArmO2ERDClvxlh/60cb+GRkXo7QevUuzAcKNLAhCimG0EoEt+ev0EjR02SNX1zfrJ/3wik/0tA8CkCC9ACIvsZ9Ov73Con82id0sq9eLmMqNLAoALIrwAIW5iYowenjVOkvSL10u0r7rB4IoA4PwILwD0L1deom+MGaKTp9x6MN+lU26P0SUBwDkRXgDIarXoye/YFdO/nz45WKvlf9tjdEkAcE6EFwCSpOEx/fUft06SJP1u/efatO+IwRUBwNmZJrywzgvQc9dNGq7b00bK65XyXtqm2pOnjC4JADphnRcAZ2hobtUNv3lf+4+c0E32RP3mzilGlwQgBLDOC4BuGxQRpuVzHbJZLXptW7nWfnzI6JIA4AyEFwCdTBl1kR6Yeakk6edrd6js6AmDKwKALxFeAJzV/dPHKG30RapvbtW/5rvUyvRpAAGC8ALgrMJsVi2f69CgiDBt2X9MT6//3OiSAEAS4QXAeSTFDtAvbp4oSVpe8JlcZceNLQgARHgBcAG3TBmhHHui3B6vHnzxYzU2txpdEoAQR3gBcF4Wi0X/PidViTGR+uLICf3i9RKjSwIQ4ggvAC4opn8/PTXXIYtFyt9Spr9uP2x0SQBCWMCFl927d8vhcHQ8+vfvr7Vr1xpdFhDypl4yRN+/eowkadGr21VR22RwRQBCVUCvsNvQ0KDk5GTt379fAwcO7NJzWGEX6D0trR59++kN2n6oVt8cO0TP/VOmrFaL0WUBMAHTrLD72muvaebMmV0OLgB6V3iYVcvvcKh/P5s+3HtEf/yg1OiSAIQgn8NLYWGhcnJylJiYKIvFctZLOk6nU8nJyYqMjFRmZqaKioq6VdxLL72kuXPnduu5AHrHmKGD9PMbUyRJT7y9W5+W1xpcEYBQ43N4aWxslN1ul9PpPOvv8/PzlZeXp6VLl2rr1q2y2+2aNWuWqqqqOvZxOBxKTU3t9CgvL+/Yp66uThs2bND1119/3nqam5tVV1d3xgNA77ozI0nXpsSrxe3RAy+61HTKbXRJAEJIj3peLBaL1qxZozlz5nRsy8zMVHp6ulasWCFJ8ng8SkpK0sKFC7Vo0aIuH/u5557T22+/reeff/68+z3yyCNatmxZp+30vAC960hDs2b/+n1V1zdrXtZoLbs51eiSAAQxw3peWlpaVFxcrOzs7C8HsFqVnZ2tjRs3+nSsrl4yWrx4sWprazseZWVlPtcNwHdDBkXoV7fbJUnPbtyv93ZVXeAZAOAffg0vNTU1crvdio+PP2N7fHy8Kioqunyc2tpaFRUVadasWRfcNyIiQtHR0Xruuec0depUzZw50+e6AXTP1ZcN1fxvJkuSHn5lm2oamo0tCEBICMjZRjExMaqsrFR4eHiXn5Obm6uSkhJt3ry5FysD8HU/mT1e4+KjVNPQoh+/8okCePUFACbh1/ASFxcnm82mysrKM7ZXVlYqISHBn0MBCBCR/Wz69Z0OhYdZ9fddVXp+0wGjSwJgcn4NL+Hh4UpLS1NBQUHHNo/Ho4KCAmVlZflzqE6cTqdSUlKUnp7eq+MA6Gx8QrR+Mnu8JOnf3yjR3qp6gysCYGY+h5eGhga5XC65XC5JUmlpqVwulw4caPtrKy8vTytXrtSzzz6rnTt3asGCBWpsbNT8+fP9WvjXcdkIMNb8byRr2qVxam716IcvuNTcyvRpAL3D5/CyZcsWTZkyRVOmTJHUFlamTJmiJUuWSJLmzp2rX/3qV1qyZIkcDodcLpfWrVvXqYnX3zjzAhjLarXoydvtumhAP5UcrtNT7+wxuiQAJhXQ9zbqDu5tBBjr7U8r9L3nimWxSH/+50x9Y2yc0SUBCAKmubcRgOAza2KC7sxIktcr5b20TcdPtBhdEgCTIbwA8Luf35iiS+IGqqKuST9ds53p0wD8yjThhZ4XIHAMCA/T8jscCrNa9Nb2Cr1SfNDokgCYCD0vAHqN8729euLt3RoYbtNbD0zT6CEDjS4JQICi5wVAQPj+1WOUcXGsGlvcejDfpVa3x+iSAJiAacILl42AwGOzWvTUd+yKigzTxweO67d/32t0SQBMgMtGAHrdX1yH9MCLLlkt0svfz1La6FijSwIQYLhsBCCg3OwYoVumjJDHKz2Y71J90ymjSwIQxAgvAPrEspsnasTg/io7elJLX/vU6HIABDHCC4A+ER3ZT8vvcMhqkV7dekivbys3uiQAQco04YWGXSDwpSfHKnfGWEnSz9ZsV/nxkwZXBCAY0bALoE+dcnt02+83alvZcWVeHKvV902VzWoxuiwABqNhF0DA6mez6tdzHRoQbtOm0qN6pnCf0SUBCDKEFwB9LjluoB7JmShJeurd3dp+sNbgigAEE8ILAEPcfsVIzZ6YoFNurx7I/1gnW9xGlwQgSBBeABjCYrHo8VsnKT46QvuqG/Xvb5YYXRKAIGGa8MJsIyD4XDQwXE/e7pAk/XnTAb1bUmlsQQCCArONABju0TdLtPL9UsUODNe6B6dpWFSk0SUB6GPMNgIQVB6aNU4ThkfraGOLHn75E5nsbyoAfkZ4AWC4iDCbfn2HQxFhVv1jT7We3fCF0SUBCGCEFwAB4bL4KP30+gmSpMf+uku7K+oNrghAoCK8AAgY92SN1oxxQ9XS6tEDL36splNMnwbQGeEFQMCwWCz65W12DRkYrl0V9Xri7d1GlwQgABFeAASUoVER+uVtkyVJf/ygVO9/Vm1wRQACDeEFQMCZOSFed08dJUn60UvbdLSxxeCKAAQS04QXFqkDzOVn16dozNCBqqpv1uJXmT4N4EssUgcgYO04VKtbfvehTrm9euyWVF0cN0hV9U0aFhWpjItjZbNajC4RJuD2eFVUepT3Vhf05r+VL9/fhBcAAe0P//hcj/91V6ftw2MitTQnRbNThxtQFcxi3Y7DWvZ6iQ7XNnVs4711dr39b8UKuwBMI+miAWfdXlHbpAXPb9W6HYf7uCKYxbodh7Xg+a1nfBlLvLfOJtD+rcL6dDQA8IHb49X/Ocfdpr2SLJKWvV6ia1MSOM0Pn7g9Xi17vURnu/TQvu3naz9V8pCBIf/ecnu8+re1O875b2XE55DwAiBgFZUe7fSX3ld5JR2ubVJR6VFljRnSd4Uh6F3ovSVJ1Q3Nmv3r9/uoouBlxOeQ8AIgYFXVn//Lxdf9gHZdfc8MjLApIszWy9UEtuZWtxqbL7zadV9+DgkvAALWsKhIv+4HtOvqe+b/3pMe8mf1Nn5+RHeu/OiC+/Xl55CGXQABK+PiWA2PidS5rqJb1DbbIePi2L4sCybAe6vrAvHfKiDDy3/9139p4sSJSklJ0Q9/+EMWpwJClM1q0dKcFEnq9D/O9p+X5qSEfEMlfPfV99bX8d46UyB+DgMuvFRXV2vFihUqLi7W9u3bVVxcrI8+uvDpKgDmNDt1uJ6++3IlxJx5SjohJlJP3305a3Gg29rfWwPDz+xp4b3VWaB9DgOy56W1tVVNTW2NP6dOndKwYcMMrgiAkWanDte1KQma/sR7Kjt2Uj+9frz++cpL+KsYPTY7dbi2HjiuZwr3afq4ofreVWNYYfcc2j+HgbAasc9nXgoLC5WTk6PExERZLBatXbu20z5Op1PJycmKjIxUZmamioqKunz8oUOH6qGHHtKoUaOUmJio7OxsjRkzxtcyAZiMzWrRwIi2v7cmDI/mywV+Y7W0vZfGDh2krDFDeG+dh81qUdaYIbrZMcLQfyufw0tjY6PsdrucTudZf5+fn6+8vDwtXbpUW7duld1u16xZs1RVVdWxj8PhUGpqaqdHeXm5jh07pjfeeENffPGFDh06pA0bNqiwsPCc9TQ3N6uuru6MBwAAXeU96/JrCGQ+Xza67rrrdN11153z90899ZTuu+8+zZ8/X5L0+9//Xm+++ab++7//W4sWLZIkuVyucz7/5Zdf1tixYxUb29a1fMMNN+ijjz7SVVddddb9H3/8cS1btszXlwEgCFlO/4VMDz96g4UTLkHDrw27LS0tKi4uVnZ29pcDWK3Kzs7Wxo0bu3SMpKQkbdiwQU1NTXK73Vq/fr3GjRt3zv0XL16s2trajkdZWVmPXwcAIIQQhoOOXxt2a2pq5Ha7FR8ff8b2+Ph47drV+a6wZzN16lRdf/31mjJliqxWq2bOnKmbbrrpnPtHREQoIiJCTqdTTqdTbveFVwEEEJz4wxiAFKCzjR599FE9+uijPj0nNzdXubm5HbfUBmBe/KGM3mDhulHQ8Otlo7i4ONlsNlVWVp6xvbKyUgkJCf4cCgAAvyAMBx+/hpfw8HClpaWpoKCgY5vH41FBQYGysrL8OVQnTqdTKSkpSk9P79VxABiHP4zRm3h7BQ+fLxs1NDRo7969HT+XlpbK5XIpNjZWo0aNUl5enubNm6crrrhCGRkZWr58uRobGztmH/UWLhsBoYNbhsCfeD8FH5/Dy5YtWzRjxoyOn/Py8iRJ8+bN06pVqzR37lxVV1dryZIlqqiokMPh0Lp16zo18QKArzjzgl7F+yto+Bxepk+ffsGU+oMf/EA/+MEPul1Ud7TPNmptbZUkFqsDTOjUyUZ5mk+ooaFedXWRF34C0AVNJxrkaT6hphMNfHcYqP3fvitnwixek50vO3jwoJKSkowuAwAAdENZWZlGjhx53n1MF148Ho/Ky8sVFRUVlNPe0tPTtXnzZqPL6DIj6+3tsf19fH8cr7vH6M7zuvqcuro6JSUlqaysTNHR0T7XFor4nAfO2IH2Oe/J84P9c+71elVfX6/ExERZreefTxSQ67z0hNVqvWBiC2Q2my2ovgCMrLe3x/b38f1xvO4eozvP8/U50dHRQfXeNRKf88AZO9A+5z15vhk+512dcOPXqdLoudzcXKNL8ImR9fb22P4+vj+O191jdOd5wfZeDCbB9m/L57zvjteT54fS59x0l40A9K325Qlqa2uD6mwCgK4LtM85Z14A9EhERISWLl2qiIgIo0sB0EsC7XPOmRcAABBUOPMCAACCCuEFAAAEFcILAAAIKoQXAAAQVAgvAAAgqBBeAPSasrIyTZ8+XSkpKZo8ebJefvllo0sC4GfHjx/XFVdcIYfDodTUVK1cubLXx2SqNIBec/jwYVVWVsrhcKiiokJpaWnas2ePBg4caHRpAPzE7XarublZAwYMUGNjo1JTU7VlyxYNGTKk18Y03b2NAASO4cOHa/jw4ZKkhIQExcXF6ejRo4QXwERsNpsGDBggSWpubpbX61VvnxfhshGAcyosLFROTo4SExNlsVi0du3aTvs4nU4lJycrMjJSmZmZKioqOuuxiouL5Xa7lZSU1MtVA/CFPz7nx48fl91u18iRI/Xwww8rLi6uV2smvAA4p8bGRtntdjmdzrP+Pj8/X3l5eVq6dKm2bt0qu92uWbNmqaqq6oz9jh49qnvuuUfPPPNMX5QNwAf++JwPHjxY27ZtU2lpqVavXq3KysperZmeFwBdYrFYtGbNGs2ZM6djW2ZmptLT07VixQpJksfjUVJSkhYuXKhFixZJajuNfO211+q+++7Td7/7XSNKB9BF3f2cf9X999+va665Rrfddluv1cmZFwDd0tLSouLiYmVnZ3dss1qtys7O1saNGyVJXq9X9957r6655hqCCxCEuvI5r6ysVH19vSSptrZWhYWFGjduXK/WRXgB0C01NTVyu92Kj48/Y3t8fLwqKiokSR9++KHy8/O1du1aORwOORwObd++3YhyAXRDVz7n+/fv17Rp02S32zVt2jQtXLhQkyZN6tW6mG0EoNdceeWV8ng8RpcBoBdlZGTI5XL16ZiceQHQLXFxcbLZbJ0a8yorK5WQkGBQVQD8KVA/54QXAN0SHh6utLQ0FRQUdGzzeDwqKChQVlaWgZUB8JdA/Zxz2QjAOTU0NGjv3r0dP5eWlsrlcik2NlajRo1SXl6e5s2bpyuuuEIZGRlavny5GhsbNX/+fAOrBuCLoPycewHgHN577z2vpE6PefPmdezz29/+1jtq1ChveHi4NyMjw/vRRx8ZVzAAnwXj55x1XgAAQFCh5wUAAAQVwgsAAAgqhBcAABBUCC8AACCoEF4AAEBQIbwAAICgQngBAABBhfACAACCCuEFAAAEFcILAAAIKoQXAAAQVAgvAAAgqPx/iTHK6od4WnUAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.loglog(num_points_range, error_range_smooth_mae, \"-o\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Sobolev H1 error\n", + "\n", + "https://www.wolframalpha.com/input?i=int_0%5E%282*pi%29+f%5E2+%2B+%28derivative%28f%2C+x%29%29%5E2+dx++with+f%28x%29+%3D+e%5E%28-100+*+%28x-1%29%5E2%29+*+sin%28x%29" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "true_value_smooth_h1 = (\n", + " 9.0126572135271277501623951970738217197152721339833724216761316788\n", + ")\n", + "domain_extent_smooth_h1 = 2 * jnp.pi\n", + "fn_smooth_h1 = lambda x: jnp.exp(-100 * (x - 1) ** 2) * jnp.sin(x)" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "error_range_smooth_h1 = [\n", + " get_difference(\n", + " num_points,\n", + " domain_extent_smooth_h1,\n", + " ex.metrics.H1_MSE,\n", + " true_value_smooth_h1,\n", + " fn_smooth_h1,\n", + " )\n", + " for num_points in num_points_range\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[]" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGhCAYAAACphlRxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA+tUlEQVR4nO3dfXhU9Z3//9fMhCQEkmAIuYMgqKCGSAYxxLSggrFIayR27Zfub6uIe9kuTVnbtHahu1+R3a5+u7ZeapmvtnT9Ymvt0nZL8BZvUt2IooHAIBBuBCNEkkwIgUkykITMzO+P3EBMgNxMcmbOPB/XNVevnJw5551cDnn1fG7eFr/f7xcAAECIsBpdAAAAwEAQXgAAQEghvAAAgJBCeAEAACGF8AIAAEIK4QUAAIQUwgsAAAgpEUYXEGg+n0/V1dWKjY2VxWIxuhwAANAPfr9fTU1NSktLk9V68WcrpgkvDodDDodDbW1tOnz4sNHlAACAQaiqqtKkSZMueo7FbDvsut1ujRs3TlVVVYqLizO6HAAA0A+NjY1KT0/XqVOnFB8ff9FzTfPkpUvXUFFcXBzhBQCAENOfKR9M2AUAACGF8AIAAEIK4QUAAIQUwgsAAAgphBcAABBSCC8AACCkBGV4ueuuu3TZZZfp7rvvNroUAAAQZIIyvDz44IP67W9/a3QZAAAgCAVleLnlllsUGxtrdBkAACAIBTy8lJaWKj8/X2lpabJYLCouLu51jsPh0JQpUxQdHa2cnByVlZUFuoyA8/r82nr4hDY5j2nr4RPy+kzVVQEAgJAR8PYAHo9HWVlZuv/++/X1r3+91/c3bNigoqIiPfvss8rJydGTTz6phQsX6sCBA0pKSgp0OQGxeU+N1rxcoRp3S/ex1Phorc7P0O2ZqQZWBgBA+An4k5dFixbppz/9qe66664+v//EE0/ogQce0LJly5SRkaFnn31WMTExeu655wZ1v9bWVjU2NvZ4BdLmPTVa/sKOHsFFkmrdLVr+wg5t3lMT0PsBAICLG9E5L21tbSovL1deXt65AqxW5eXlaevWrYO65mOPPab4+PjuV3p6eqDKldfn15qXK9TXAFHXsTUvVzCEBADACBrR8FJfXy+v16vk5OQex5OTk1VbW9v9dV5enr7xjW/otdde06RJky4abFatWiW32939qqqqCli9ZZUNvZ64nM8vqcbdorLKhoDdEwAAXFzA57wEwttvv93vc6OiohQVFSWHwyGHwyGv1xuwOuqaLhxcBnMeAAAYuhF98pKYmCibzSaXy9XjuMvlUkpKypCuXVhYqIqKCm3btm1I1zlfUmx0v85ra/cF7J4AAODiRjS8REZGavbs2SopKek+5vP5VFJSotzc3CFd2+FwKCMjQ9nZ2UMts9ucqQlKjY+W5RLn/fjPH2vlf3+sukaewAAAMNwCHl6am5vldDrldDolSZWVlXI6nTp69KgkqaioSOvWrdPzzz+vffv2afny5fJ4PFq2bNmQ7jscT15sVotW52dIUq8A0/W1PX2c/JL+a1uVbn78XT3x1kF5WtsDVgMAAOjJ4vf7A7pU5t1339X8+fN7HV+6dKnWr18vSVq7dq0ef/xx1dbWym636+mnn1ZOTk5A7t/Y2Kj4+Hi53W7FxcUF5JqX2udl+2cNevS1fdpx9JQkKXFslL6fN03fzE5XhC0oNzEGACCoDOTvd8DDi1HOn7B78ODBgIYXqWPZdFllg+qaWpQUG605UxNks557HuP3+7V5T61+tnm/PjtxWpJ05YQxWrnoWuVdmySL5VKDTwAAhK+wDC9dhuPJy0C0tfv04kdH9FTJJzp5+qykjrkz//zVa5WVPm7E6wEAIBQQXgwML911tJzVM+8e1nNbKtXauRopPytND33lak0eH2NYXQAABKOwDC/DPWw0WNWnzujnbx7Qxp3H5PdLo2wW3Zs7RSsWXKVxMZFGlwcAQFAIy/DSJVievHzR3mq3/s/r+/XeJ/WSpLjoCH1vwVW6N3eKokfZDK4OAABjEV6CMLx0+Z+Dx/XYa/u0v7ZJkjRx3Gj9+ParlT8zTVYrk3oBAOGJ8BLE4UXqWLn03zs+1y/ePCBXY6sk6bqJ8Vr11Wv0pSsTDa4OAICRF5bhJVjnvFzMmTavnnu/Us+8e1jNnRvbLbgmSasWXaNpybEGVwcAwMgJy/DSJRSevHxRfXOrni75RC9+dFTtPr+sFmlJdrp+kDddSXH9668EAEAoI7yEWHjp8unxZv1s8369sbejcWVMpE0PzLtC377pCo2JCsoG4AAABAThJUTDS5dtne0Gdp7XbuAHt03TkhtoNwAAMCfCS4iHF6mj3cDrne0GjnS2G7gqaaxW3n6NbqXdAADAZMIyvITihN3+aGv36fcfHdHT57UbyJmaoH/+2rWaOWmcscUBABAgYRleupjlycsXuc90tht4v1Jt57Ub+PHCq5WeQLsBAEBoI7yYMLx0OXbqjH5xXruBSJtV9+Zeru/RbgAAEMIILyYOL132HOtoN7DlUEe7gfjRo/S9+Vfp3i9drqgI2g0AAEIL4SUMwovUMam3o93Afh1wdbQbmHTZaD20kHYDAIDQQngJk/DSxevz67/LP9cv3jrXbmDmpHitWnStcq8cb3B1AABcWliGF7OuNhqI023tem5LR7sBT5tXknTrNUlaSbsBAECQC8vw0iUcn7x8UX1zq556+xO9WHZU3u52A5P1g7xptBsAAAQlwkuYh5cuh48362ev79ebFbQbAAAEN8IL4aWHssqOdgPOqlOSpAmxUfpB3nT9rxsm0W4AABAUCC+El178fr9e3V2j/9h8QEcbzrUbWLXoGi24hnYDAABjEV4ILxfU1u7TCx8e0dN//USnOtsN3HhFgn7yVdoNAACMQ3ghvFyS+8xZ/d93D+n/vf9Zd7uBO7PS9BDtBgAABiC8EF767fOTp/XEmwf1l53HJHW0G1j6pcv1vfnTFB8zyuDqAADhIizDC/u8DM2eY2499vo+vX/ohKSOdgMrFlyle3JpNwAAGH5hGV668ORl8Px+v949eFz/h3YDAIARRnghvAyJ1+fXn8ur9Is3D6qu6Vy7gZ989VrdeAXtBgAAgUd4IbwExOm2dv3mvUr96n/OtRvIu7aj3cBVSbQbAAAEDuGF8BJQx5ta9VTJQf2hrEpen182q0VLstP1/bxpSoql3QAAYOgIL4SXYXGorlk/27xfb53XbuDbN12hB+b1bDfg9flVVtmguqYWJcVGa87UBNmYLwMAuAjCC+FlWH306Qk9+vp+7Tqv3UDRbdP1jdmT9PY+l9a8XKEad0v3+anx0Vqdn6HbM1MNqhgAEOwIL4SXYddXu4HUuGjVNLb0Orfrmcsz37qeAAMA6NNA/n7TlQ+DYrFYdMfMNL1VdJP+9x0Zih8d0WdwkaSudLzm5Qp5fabKygAAAxBeMCRRETb9/dyp+sU37Bc9zy+pxt2issqGEakLAGBeQRleXnnlFV199dWaNm2afvOb3xhdDvrB09ber/Pqmvp+OgMAQH9FXPqUkdXe3q6ioiK98847io+P1+zZs3XXXXdp/Hg2Rwtm/V0yzdJqAMBQBd2Tl7KyMs2YMUMTJ07U2LFjtWjRIr355ptGl4VLmDM1Qanx0brYguioCKuumDBmxGoCAJhTwMNLaWmp8vPzlZaWJovFouLi4l7nOBwOTZkyRdHR0crJyVFZWVn396qrqzVx4sTurydOnKhjx44FukwEmM1q0er8DEm6YIBpbffpjl9u0fuH6keuMACA6QQ8vHg8HmVlZcnhcPT5/Q0bNqioqEirV6/Wjh07lJWVpYULF6qurm5Q92ttbVVjY2OPF4xxe2aqnvnW9UqJ7zk0lBofrX+541pdlTRWx5ta9a3//EiPv7FfZ70+gyoFAISygM95WbRokRYtWnTB7z/xxBN64IEHtGzZMknSs88+q1dffVXPPfecVq5cqbS0tB5PWo4dO6Y5c+Zc8HqPPfaY1qxZE7gfAENye2aqbstI6XOH3b+bc7n+9ZW9+kNZlRzvHNbWwyf01DdnKT0hxuiyAQAhZETnvLS1tam8vFx5eXnnCrBalZeXp61bt0qS5syZoz179ujYsWNqbm7W66+/roULF17wmqtWrZLb7e5+VVVVDfvPgYuzWS3KvXK8FtsnKvfK8d2tAUZH2vTY12dq7f83S7HREdpx9JS++vR7evXjGoMrBgCEkhENL/X19fJ6vUpOTu5xPDk5WbW1tZKkiIgI/eIXv9D8+fNlt9v1wx/+8KIrjaKiohQXF6ff/e53uvHGG3XrrbcO68+AobtjZppe+8d5mjV5nJpa2lX44g6t+stunensXA0AwMUE3WojSbrzzjt18OBBHTp0SN/+9rf79Z7CwkJVVFRo27Ztw1wdAiE9IUZ//E6uvnvLlbJYpD+UHdVixxYdqG0yujQAQJAb0fCSmJgom80ml8vV47jL5VJKSspIloIgMMpm1Y9vv0a/uz9HE2KjdNDVrDvXbtELHx6RyVpuAQACaETDS2RkpGbPnq2SkpLuYz6fTyUlJcrNzR3StR0OhzIyMpSdnT3UMjHC5k5L1OsPztPN0yeotd2nfyneo+/+fofcp88aXRoAIAgFvKt0c3OzDh06JEmaNWuWnnjiCc2fP18JCQmaPHmyNmzYoKVLl+pXv/qV5syZoyeffFJ//OMftX///l5zYQaDrtKhy+fz6z+3VOo/3tivs16/Jo4brae+adcNUxKMLg0AMMwG8vc74OHl3Xff1fz583sdX7p0qdavXy9JWrt2rR5//HHV1tbKbrfr6aefVk5OzpDu63A45HA45PV6dfDgQcJLCPv481Na8YedOnLitGxWi36QN03Lb7mqe9USAMB8DA0vRuPJizk0tZzVvxTv0SZntSQp94rxevKbdiXH0RsJAMxoIH+/g3K1ERAbPUpPLrHr59/IUkykTVs/PaFFT72nv+53XfrNAABTI7wgaFksFt09e5JeXjFXGalxavC06f712/WvL1eotZ09YQAgXJkmvLDayLyunDBWGwu/pPu+NEWS9Nz7lfqbZz5QZb3H2MIAAIZgzgtCytsVLj305106efqsxkTa9G8Fmfr69ZOMLgsAMETMeYFp5WUk6/UHb1LO1AR52rwq+uMuFW1wqrm13ejSAAAjxDThhWGj8JESH60XH7hRRbdNl9Ui/WXnMd3x9Hva/bnb6NIAACOAYSOEtG2fNejBP+xUtbtFo2wW/dPt1+jv506VxcKeMAAQShg2QtjInpKg1x6cp69kJOus16+fvrpP96/fphPNrUaXBgAYJoQXhLxxMZH61T2z9W+LZygywqp3DhzXoqfe0weH6o0uDQAwDAgvMAWLxaJ7cqdoU+GXdVXSWNU1terv/vMjPf7GfrV7fUaXBwAIINOEFybsQpKuTY3TS9/7sr6ZnS6/X3K8c1hLfv2hPj952ujSAAABwoRdmNbLu6r1k7/sVlNru+KiI/Szv5mpRdelGl0WAKAPTNgFJOVnpem1B+fJnj5OjS3tWv77HfrJxt1qOUtrAQAIZYQXmFp6Qoz+9A+5Wn7LlbJYpBc/Oqo7127RQVeT0aUBAAaJ8ALTG2Wz6p9uv0a/vX+OEsdG6aCrWfm/3KLff3REJhs1BYCwQHhB2Jg3bYJef3Cebpo+Qa3tPv3zxj0qfHGH3GfOGl0aAGAATBNeWG2E/pgQG6X192XrJ1+9RhFWi17bXauvPvWeyo+cNLo0AEA/sdoIYWtX1Smt+MNOHW04LZvVoqLbpusfbr5SNiutBQBgpLHaCOiHrPRxevUf52qxPU1en1+Pv3FA9/znR3I1thhdGgDgIggvCGux0aP05BK7Hr97pkaPsumDwye06Kn39M7+OqNLAwBcAOEFYc9isegbN6Tr5RVzdW1qnBo8bVq2fpt++kqF2tppLQAAwYbwAnS6KmmsNn73S7rvS1MkSb/ZUqm/eeYDVdZ7jC0MANAD4QU4T/Qomx65c4bW3XuDxsWM0u5jbt3x9HvauPNzo0sDAHQivAB9uC0jWa8/OE9zpibI0+bVDzbsUtEfnfK0thtdGgCEPcILcAGp8aP1hwdu1A/ypstqkf6y45ju+OUW7TnmNro0AAhrpgkvbFKH4WCzWvRg3jT917dzlRofrcp6j77+fz/Qc1sqaS0AAAZhkzqgn06dbtOP//yx3qxwSZIWXJOkx++eqfFjowyuDABCH5vUAcNgXEykfnXPbP3r4hmKjLDqr/vrtOip9/TB4XqjSwOAsEJ4AQbAYrHo3twpKv7ul3XlhDGqa2rV3/3mI/3izQNq97InDACMBMILMAgZaXF6ecVcLbkhXX6/9Mu/HtI3f/2hjp06Y3RpAGB6hBdgkGIiI/Szu2fql387S7FREdp+5KQWPVmqzXtqjC4NAEyN8AIMUX5Wml79x3nKSh+nxpZ2/cMLO/TPG3er5azX6NIAwJQIL0AATB4foz//Q66+c/MVkqTff3RUi9e+r4OuJoMrAwDzIbwAATLKZtWqRdfqt/fPUeLYKB1wNenOtVv0h7Kj7AkDAAEUlOHlrrvu0mWXXaa7777b6FKAAbtp+gS9/uA8zZuWqJazPq36y25978Wdcp85a3RpAGAKQRleHnzwQf32t781ugxg0CbERun5ZXO0atE1irBa9OruGn31qfdUfuSk0aUBQMgLyvByyy23KDY21ugygCGxWi36zs1X6s/Lv6TJCTE6duqM/tevtsrxziH5fB3DSF6fX1sPn9Am5zFtPXxCXh/DSwBwKQMOL6WlpcrPz1daWposFouKi4t7neNwODRlyhRFR0crJydHZWVlgagVCEn29HF65R/n6s6sNHl9fj3+xgHd89xH2lB2VHN/9lf97boP9eB/OfW36z7U3J/9laXWAHAJAw4vHo9HWVlZcjgcfX5/w4YNKioq0urVq7Vjxw5lZWVp4cKFqqur6z7HbrcrMzOz16u6unrAP0Bra6saGxt7vIBgExc9Sk99067/uHumRo+y6f1DJ/RPf9mtGndLj/Nq3S1a/sIOAgwAXMSQGjNaLBZt3LhRBQUF3cdycnKUnZ2ttWvXSpJ8Pp/S09O1YsUKrVy5st/Xfvfdd7V27Vr9+c9/vuh5jzzyiNasWdPrOI0ZEawO1Dbpa0+/p/YLDBFZJKXER2vLPy2QzWoZ2eIAwCCGNWZsa2tTeXm58vLyzt3AalVeXp62bt0ayFt1W7Vqldxud/erqqpqWO4DBEqDp+2CwUWS/JJq3C0qq2wYuaIAIIREBPJi9fX18nq9Sk5O7nE8OTlZ+/fv7/d18vLytGvXLnk8Hk2aNEl/+tOflJub2+e5UVFRioqKksPhkMPhkNfLrqYIbnVNLZc+aQDnAUC4CWh4CZS33357wO8pLCxUYWFh92MnIFglxUYH9DwACDcBHTZKTEyUzWaTy+XqcdzlciklJSWQtwJC1pypCUqNj9aFZrNYJKXGR2vO1ISRLAsAQkZAw0tkZKRmz56tkpKS7mM+n08lJSUXHPYJFIfDoYyMDGVnZw/rfYChslktWp2fIUkXDDCr8zOYrAsAFzDg8NLc3Cyn0ymn0ylJqqyslNPp1NGjRyVJRUVFWrdunZ5//nnt27dPy5cvl8fj0bJlywJa+BcVFhaqoqJC27ZtG9b7AIFwe2aqnvnW9UqJ7zk0FBVh1TPful63Z6YaVBkABL8Bz3nZvn275s+f3/11UVGRJGnp0qVav369lixZouPHj+vhhx9WbW2t7Ha7Nm/e3GsSb6AxYReh5vbMVN2WkaKyygZVVLv1b6/uU1u7T9dffpnRpQFAUBvSPi/BaCDrxIFgUuB4X86qU3r4jgzdP3eq0eUAwIgybJ8XAINXYE+TJG1yHjO4EgAIboQXIEjckZUmm9WiXZ+79enxZqPLAYCgZZrwwmojhLrEsVGaNy1RklTsHHifLwAIF6YJL6w2ghkU2CdK6hg6Mtl0NAAIGNOEF8AMbstI1uhRNh05cVo7q04ZXQ4ABCXCCxBExkRFaOGMjm0FNu1k4i4A9MU04YU5LzCLxbM6ho5e+bhGZ70+g6sBgOBjmvDCnBeYxbyrEjV+TKROeNq05ZN6o8sBgKBjmvACmEWEzar8rI49X4rZ8wUAeiG8AEFoceeGdW/udcnT2m5wNQAQXEwTXpjzAjOxp4/T5eNjdOasV29VuIwuBwCCimnCC3NeYCYWi0WLO/d82ciqIwDowTThBTCbrl5HWw7Vq7651eBqACB4EF6AIHXFhLHKmhQvr8+vV3bRLgAAuhBegCDWPXREryMA6EZ4AYJYflen6apTqqz3GF0OAAQF04QXVhvBjCbERunLV3V0mt7Eni8AIMlE4YXVRjCrrom7xTvpNA0AkonCC2BWC2ekaPQomz47cVq7PncbXQ4AGI7wAgS5MVERui2jo9N0MXu+AADhBQgFBbM6ho5e+bha7XSaBhDmCC9ACJg3bYISxkSqvrlNWw7RaRpAeCO8ACFglM2qO2amSpI2secLgDBHeAFCRNeGdW/srdXpNjpNAwhfhBcgRFw/eZwmJ8TodBudpgGEN9OEFzapg9lZLJYee74AQLgyTXhhkzqEg8WzOoaOSj+p1wk6TQMIU6YJL0A4uHLCWF03sbPT9Mc1RpcDAIYgvAAhpqDz6UsxvY4AhCnCCxBi8rNSZbVIO4+e0pETdJoGEH4IL0CISYqNPq/TNHu+AAg/hBcgBBV07vlCp2kA4YjwAoSghZkpih5l1af1Hu0+RqdpAOGF8AKEoLFREcq7tqvTNENHAMJL0IWXqqoq3XLLLcrIyNDMmTP1pz/9yeiSgKDUNXT00i46TQMIL0EXXiIiIvTkk0+qoqJCb775pr7//e/L42FFBfBFN02foMtiRqm+uVUfHD5hdDkAMGKCLrykpqbKbrdLklJSUpSYmKiGhgZjiwKCUGSEVV/r7DTNni8AwsmAw0tpaany8/OVlpYmi8Wi4uLiXuc4HA5NmTJF0dHRysnJUVlZ2aCKKy8vl9frVXp6+qDeD5hd19DRG3tqdabNa3A1ADAyBhxePB6PsrKy5HA4+vz+hg0bVFRUpNWrV2vHjh3KysrSwoULVVdX132O3W5XZmZmr1d19bmJhw0NDbr33nv161//+qL1tLa2qrGxsccLCBezL79Mky4bLU+bV2/to9M0gPBg8Q9hkwiLxaKNGzeqoKCg+1hOTo6ys7O1du1aSZLP51N6erpWrFihlStX9uu6ra2tuu222/TAAw/onnvuuei5jzzyiNasWdPruNvtVlxcXP9/GCBE/fyNA1r7ziHdek2S/vM+uqoDCE2NjY2Kj4/v19/vgM55aWtrU3l5ufLy8s7dwGpVXl6etm7d2q9r+P1+3XfffVqwYMElg4skrVq1Sm63u/tVVVU16PqBUFQwK02S9D8Hj6vB02ZwNQAw/AIaXurr6+X1epWcnNzjeHJysmpra/t1jffff18bNmxQcXGx7Ha77Ha7du/efcHzo6KiFBcXp9/97ne68cYbdeuttw7pZwBCzVVJscqcGKd2n1+vfsyeLwDML8LoAr5o7ty58vkGvmdFYWGhCgsLux87AeGkwD5Re441qthZrXtypxhdDgAMq4A+eUlMTJTNZpPL1XPioMvlUkpKSiBvBeA8+Vlpslik8iMndfTEaaPLAYBhFdDwEhkZqdmzZ6ukpKT7mM/nU0lJiXJzcwN5q14cDocyMjKUnc2ERYSf5LhofenK8ZKkTez5AsDkBhxempub5XQ65XQ6JUmVlZVyOp06evSoJKmoqEjr1q3T888/r3379mn58uXyeDxatmxZQAv/osLCQlVUVGjbtm3Deh8gWHV3mnbSaRqAuQ14zsv27ds1f/787q+LiookSUuXLtX69eu1ZMkSHT9+XA8//LBqa2tlt9u1efPmXpN4AQTW7Zkp+pfiPTp83KO91Y3KnMjcLwDmNKR9XoKJw+GQw+GQ1+vVwYMH2ecFYanw9zv06u4a/f3cqfrfd2QYXQ4A9Jth+7wYiWEjQCqY1TF09PKuanl9pvj/JQDQi2nCCwDp5ukTNC5mlOqaWrWVTtMATMo04YXVRkBHp+mvXtfRaXrjTlYdATAn04QXho2ADnd1Dh29sbdWLWfpNA3AfEwTXgB0mD35Mk0cN1rNre16m07TAEyI8AKYjNVq0WJ7R7PG4p30OgJgPqYJL8x5Ac7pGjp690CdTtJpGoDJmCa8MOcFOGdacqwyUjs7Te+uMbocAAgo04QXAD0VzOoYOqLXEQCzIbwAJnVn1kRZLNK2z06qqoFO0wDMwzThhTkvQE8p8dHKvaKj0/RLu5i4C8A8TBNemPMC9NbVaXrjTjpNAzAP04QXAL3dfl2KIiOsOlTXrL3VjUaXAwABQXgBTCwuepTyrk2SxMRdAOZBeAFMbnHn0NFLdJoGYBKEF8Dkbrl6guKiI+RqbNVHn9JpGkDoM014YbUR0LeoCJu+NrNjzxc6TQMwA9OEF1YbARdW0NnraPMeOk0DCH2mCS8ALix7SoLS4qPV1Nquv+6vM7ocABgSwgsQBqxWi+48b88XAAhlhBcgTJzfafrUaTpNAwhdhBcgTFydEqtrUmJ11uvXa7trjS4HAAaN8AKEkYLOpy/FDB0BCGGEFyCM3JmVJotFKvusQZ+fpNM0gNBEeAHCSNq40cqZmiCJTtMAQpdpwgub1AH909VpuphO0wBClGnCC5vUAf2z6LpURdqsOuhq1r6aJqPLAYABM014AdA/8aNHacE1dJoGELoIL0AYKpjV0S5gk7NaPjpNAwgxhBcgDN1ydZLioiNU29iiDyvpNA0gtBBegDAUPcqmr16XKknatJNVRwBCC+EFCFOLO1cdvbanhk7TAEIK4QUIUzlTE5QaH62mlna9Q6dpACGE8AKEqY5O0x0Td4tZdQQghARdeDl16pRuuOEG2e12ZWZmat26dUaXBJhW14Z17+w/LvfpswZXAwD9E3ThJTY2VqWlpXI6nfroo4/06KOP6sQJVkMAw+Ha1DhdnRyrNq9Pr++pMbocAOiXoAsvNptNMTExkqTW1lb5/X62MAeGUVen6Y10mgYQIgYcXkpLS5Wfn6+0tDRZLBYVFxf3OsfhcGjKlCmKjo5WTk6OysrKBnSPU6dOKSsrS5MmTdJDDz2kxMTEgZYJoJ+65r18VNmg6lNnDK4GAC5twOHF4/EoKytLDoejz+9v2LBBRUVFWr16tXbs2KGsrCwtXLhQdXXnVjN0zWf54qu6umO/iXHjxmnXrl2qrKzUiy++KJfLdcF6Wltb1djY2OMFoP8mjhutOXSaBhBCLP4hjMlYLBZt3LhRBQUF3cdycnKUnZ2ttWvXSpJ8Pp/S09O1YsUKrVy5csD3+O53v6sFCxbo7rvv7vP7jzzyiNasWdPruNvtVlxc3IDvB4SjP5Qd1aq/7NY1KbHa/P2bjC4HQBhqbGxUfHx8v/5+B3TOS1tbm8rLy5WXl3fuBlar8vLytHXr1n5dw+Vyqampo9Ot2+1WaWmprr766guev2rVKrnd7u5XVVXV0H4IIAx9NbOj0/T+2ibtr+XpJYDgFtDwUl9fL6/Xq+Tk5B7Hk5OTVVtb269rHDlyRPPmzVNWVpbmzZunFStW6Lrrrrvg+VFRUYqLi+vxAjAw8TGjdMvVEyRJxbQLABDkIowu4IvmzJkjp9M54Pc5HA45HA55vWxzDgxGwayJerPCpZecx/TjhVfLarUYXRIA9CmgT14SExNls9l6TbB1uVxKSUkJ5K16KSwsVEVFhbZt2zas9wHMasE1SYqNilC1u0VlnzUYXQ4AXFBAw0tkZKRmz56tkpKS7mM+n08lJSXKzc0N5K16cTgcysjIUHZ29rDeBzCr6FE2Lbqu4/9kbKJdAIAgNuDw0tzcLKfT2T20U1lZKafTqaNHj0qSioqKtG7dOj3//PPat2+fli9fLo/Ho2XLlgW08C/iyQswdF3tAl79uEat7QzBAghOA57zsn37ds2fP7/766KiIknS0qVLtX79ei1ZskTHjx/Xww8/rNraWtntdm3evLnXJF4AwSfnivFKiYtWbWOL3tl/XLdnDu9wLwAMxpD2eQkm50/YPXjwIPu8AIP06Gv79OvST7UoM0XPfGu20eUACBOG7fNiJIaNgMBY3NkuoGR/ndxn6DQNIPiYJrwACIyM1DhNTx6rtnafNtNpGkAQMk14YbUREBgWi0WLOyfusmEdgGBkmvDCsBEQOF1DRx9WnlCNm07TAIKLacILgMCZdFmM5kxJkN8vveTk6QuA4EJ4AdCnxbM6nr4UE14ABBnThBfmvACB9bXrUjXKZtG+mkYdqG0yuhwA6Gaa8MKcFyCwxsVE6ubpSZKkYtoFAAgipgkvAALvrlkdq45eclbL5zPFfpYATIDwAuCCbr02SWOjInTs1BltP3LS6HIAQBLhBcBFRI+ydfc3YugIQLAwTXhhwi4wPLqGjl79uEZt7T6DqwEAE4UXJuwCw+PGK8YrKTZK7jNn9e6BOqPLAQDzhBcAw8NmtejOrI49Xzax5wuAIEB4AXBJBZ1DR2/tc6mxhU7TAIxFeAFwSTPS4nRVUlen6VqjywEQ5ggvAC7JYrGowN41dMSqIwDGMk14YbURMLwW2zuGjj44fEK17haDqwEQzkwTXlhtBAyv9IQY3XD5ZfL7pZd3MXEXgHFME14ADL/FnRN32bAOgJEILwD67WvXpSrCatHe6kZ94qLTNABjEF4A9FvCmEjdcvUESTx9AWAcwguAAemauLvJWS2/n07TAEYe4QXAgORdm6wxkTZ9fvKMyuk0DcAAhBcAAzI60qbbM1MlSRt3MnQEYOSZJrywzwswcgpmdWxY9+puOk0DGHmmCS/s8wKMnC9dmagJsVE6dfqsSg8eN7ocAGHGNOEFwMixWS3Kn9nx9GUjq44AjDDCC4BBuatzw7q3K1xqotM0gBFEeAEwKJkT43TFhDFqbffpjb0uo8sBEEYILwAGpaPTdNeeLwwdARg5hBcAg9YVXt4/VK+6RjpNAxgZhBcAgzZ5fIyunzxOPr/0Ep2mAYwQwguAISmYda5dAACMhKANL6dPn9bll1+uH/3oR0aXAuAiujpN7z7m1qG6ZqPLARAGgja8/Pu//7tuvPFGo8sAcAnjx0bppukdnaaZuAtgJARlePnkk0+0f/9+LVq0yOhSAPTDYnvHhnXFzmN0mgYw7AYcXkpLS5Wfn6+0tDRZLBYVFxf3OsfhcGjKlCmKjo5WTk6OysrKBnSPH/3oR3rssccGWhoAg3wlI0VjIm2qajijHUfpNA1geA04vHg8HmVlZcnhcPT5/Q0bNqioqEirV6/Wjh07lJWVpYULF6qurq77HLvdrszMzF6v6upqbdq0SdOnT9f06dMH/1MBGFGjI21aOCNFklS8k4m7AIaXxT+EZ7wWi0UbN25UQUFB97GcnBxlZ2dr7dq1kiSfz6f09HStWLFCK1euvOQ1V61apRdeeEE2m03Nzc06e/asfvjDH+rhhx/u8/zW1la1trZ2f93Y2Kj09HS53W7FxcUN9kcDMED/c/C4lj5XpoQxkfroJ7dqlC0oR6UBBKnGxkbFx8f36+93QP91aWtrU3l5ufLy8s7dwGpVXl6etm7d2q9rPPbYY6qqqtJnn32mn//853rggQcuGFy6zo+Pj+9+paenD/nnADBwX75yvBLHRqrB06b3PqHTNIDhE9DwUl9fL6/Xq+Tk5B7Hk5OTVVtbG8hbdVu1apXcbnf3q6qqaljuA+DiImxW5Wd1dppm6AjAMIowuoCLue+++y55TlRUlKKiouRwOORwOOT1eoe/MAB9KrBP1P97/zO9VVGr5tZ2jY0K6n9iAISogD55SUxMlM1mk8vVs8Osy+VSSkpKIG/VS2FhoSoqKrRt27ZhvQ+AC5s5KV5TE8eo5axPb+4dnqetABDQ8BIZGanZs2erpKSk+5jP51NJSYlyc3MDeateHA6HMjIylJ2dPaz3AXBh53ea3riTDesADI8Bh5fm5mY5nU45nU5JUmVlpZxOp44ePSpJKioq0rp16/T8889r3759Wr58uTwej5YtWxbQwr+IJy9AcOjasO79Q/Wqa6LTNIDAG/CA9Pbt2zV//vzur4uKiiRJS5cu1fr167VkyRIdP35cDz/8sGpra2W327V58+Zek3gBmNOUxDGyp4+Ts+qUXtlVo/vnTjW6JAAmM6R9XoLJ+RN2Dx48yD4vgIGe/+AzrX5pr2ZOitdL35trdDkAQoBh+7wYiWEjIHh8bWaqbFaLPv7crU+P02kaQGCZJrwACB6JY6M0b1qiJKnYyZ4vAALLNOGF1UZAcOladVS8k07TAALLNOGFYSMguHxlRrJiIm062nBaO6tOGV0OABMxTXgBEFxiIiP0lYyOVYab2PMFQAARXgAMm8WzOoaOXvm4Rme9PoOrAWAWpgkvzHkBgs+8qxI1fkykTnjatOWTeqPLAWASpgkvzHkBgs/5naaLnQwdAQgM04QXAMGpq13Am3td8rS2G1wNADMgvAAYVvb0cZoyPkZnznr1ZgWdpgEMHeEFwLCyWCxa3L3nCxvWARg604QXJuwCwaugc9XRlkP1Ot7UanA1AEKdacILE3aB4DU1cYyy0sfJ6/PrlY95+gJgaEwTXgAEtwJ716ojwguAoSG8ABgRd8xMk81q0a6qU6qs9xhdDoAQRngBMCImxEbpy1d1dJrexJ4vAIaA8AJgxNw1q3PoiE7TAIbANOGF1UZA8PtKRopGj7LpsxOntetzt9HlAAhRpgkvrDYCgt+YqAjd1tlpuphO0wAGyTThBUBouKu703S12uk0DWAQCC8ARtTcaYlKGBOp+uY2bTlEp2kAA0d4ATCiRtmsumNmqiRpE3u+ABgEwguAEdfVLuCNvbU63UanaQADQ3gBMOJmpY/T5eNjdLrNq7cqXEaXAyDEEF4AjDiLxaLFWef2fAGAgTBNeGGfFyC0LO4cOir9pF4nmuk0DaD/TBNe2OcFCC1XThirmZPiOztN1xhdDoAQYprwAiD0LLZ3PH0pptcRgAEgvAAwTH5WqqwWaefRUzpygk7TAPqH8ALAMEmx0d2dpot3sucLgP4hvAAwVEHn0NEmJ52mAfQP4QWAoRZmpih6lFWf1nu0+xidpgFcGuEFgKHGRkXotowUSdJG9nwB0A+EFwCGK7B3bFj38q4aOk0DuKQIowvoy5QpUxQXFyer1arLLrtM77zzjtElARhGN02foMtiRqm+uVUfHD6hm6ZPMLokAEEsKMOLJH3wwQcaO3as0WUAGAEdnabT9LsPj6jYeYzwAuCiGDYCEBQKZnUMHb2xp1Zn2rwGVwMgmA04vJSWlio/P19paWmyWCwqLi7udY7D4dCUKVMUHR2tnJwclZWVDegeFotFN998s7Kzs/X73/9+oCUCCEHXT75M6Qmj5Wnz6q19dJoGcGEDDi8ej0dZWVlyOBx9fn/Dhg0qKirS6tWrtWPHDmVlZWnhwoWqq6vrPsdutyszM7PXq7q6Y5OqLVu2qLy8XC+99JIeffRRffzxx4P88QCEio5O0517vrDqCMBFWPxD2BXKYrFo48aNKigo6D6Wk5Oj7OxsrV27VpLk8/mUnp6uFStWaOXKlQO+x0MPPaQZM2bovvvu6/P7ra2tam0915G2sbFR6enpcrvdiouLG/D9ABjnUF2T8p4oVYTVorJ/zlPCmEijSwIwQhobGxUfH9+vv98BnfPS1tam8vJy5eXlnbuB1aq8vDxt3bq1X9fweDxqamqSJDU3N+uvf/2rZsyYccHzH3vsMcXHx3e/0tPTh/ZDADDMVUmxypwYp3afX69+TLsAAH0LaHipr6+X1+tVcnJyj+PJycmqra3t1zVcLpfmzp2rrKws3Xjjjbr33nuVnZ19wfNXrVolt9vd/aqqqhrSzwDAWAXdnaYJLwD6FnRLpa+44grt2rWr3+dHRUUpKipKDodDDodDXi+rFIBQlp+Vpkdf26fyIyd19MRpTR4fY3RJAIJMQJ+8JCYmymazyeXquVLA5XIpJSUlkLfqpbCwUBUVFdq2bduw3gfA8EqOi9aXruzoNL3JycRdAL0FNLxERkZq9uzZKikp6T7m8/lUUlKi3NzcQN6qF4fDoYyMjIsOMQEIDYs72wUU02kaQB8GHF6am5vldDrldDolSZWVlXI6nTp69KgkqaioSOvWrdPzzz+vffv2afny5fJ4PFq2bFlAC/8inrwA5nF7ZoqiIqw6fNyjvdWNRpcDIMgMeM7L9u3bNX/+/O6vi4qKJElLly7V+vXrtWTJEh0/flwPP/ywamtrZbfbtXnz5l6TeAHgQmKjRykvI1mvflyjjTuPKXNivNElAQgiQ9rnJZicP2H34MGD7PMChLi3Klx64LfblRQbpa2rbpXNajG6JADDyLB9XozEsBFgLjdPn6BxMaNU19SqrYdPGF0OgCBimvACwFwiI6z62nWpkqSNtAsAcB7ThBdWGwHmUzCrY8O6N/bWquUsezgB6GCa8MKwEWA+sydfponjRqu5tV1v02kaQCfThBcA5mO1WlQwq3PPF4aOAHQivAAIal29jt49cFwnPW0GVwMgGJgmvDDnBTCnacmxykjt7DS9u8bocgAEAdOEF+a8AOZ1V+fE3d9u/UybnMe09fAJeX2m2KIKwCAEXVdpAPiisdEd/1QddDXrwf9ySpJS46O1Oj9Dt2emGlgZACOY5skLAHPavKdGP/nL7l7Ha90tWv7CDm3ew1ASEG4ILwCCltfn15qXK9TXAFHXsTUvVzCEBIQZ04QXJuwC5lNW2aAad8sFv++XVONu0V92fK52r2/kCgNgKNM0ZuwykMZOAILbJuex7jkulxIVYdU1qXHKTIvTjLR4ZU6M0/TkWEWPsg1vkQACYiB/v5mwCyBoJcVG9+u86AirWtp92lV1SruqTnUfj7BadFXSWM1Ii9eMtDhlTozXtamxio0eNUwVAxgJhBcAQWvO1ASlxker1t3S57wXi6SU+GiVPjRfVSdPa291o/ZUu1VR3ag9x9w6efqs9tc2aX9tk/57x7n3TRkfoxkTOwNNZ7AZPzZqpH4sAEPEsBGAoLZ5T42Wv9CRPM7/x8rS+b/PfOv6PpdL+/1+1bhbOgLNMbf2Vjeqotqt6gvMoUmNj9aMtDhlpMV3DD1NjFdafLQsFkuf5wMIrIH8/Sa8AAh6m/fUaM3LFT0m7w52n5cGT5v2Vru151ij9nY+pfm03tPnuZfFjOoecup6UjN1/BhZrQQaINDCMrw4HA45HA55vV4dPHiQ8AKYjNfnV1llg+qaWpQUG605UxNkC1CIaG5t176ac09o9lY36hNXk9r7WII9JtKma1PjegSaaUmxiowwzeJNwBBhGV668OQFQCC0nPXqE1ez9lS7tbe6I9Tsq2lUy9neS7IjbVZNTxmrGakdq5wy0jomBsdEMq0Q6C/CC+EFwDBo9/pUWe/pCDTHGrsnCDe1tPc612qRrpgwtsek4Blp8YqPYaUT0BfCC+EFwAjx+/36/OSZ84ac3NpT3ajjTa19nj/pstHnAs3Ejv9NiuvfknDAzAgvhBcABqtrbOkOM11PaKoazvR5buLYqM59aDo32EuLV3rC6EGvdBrO+UHAcCG8EF4ABCH3mbOqOD/QHHPr8PFm9dWaKTY6QhmpHRvrdW2wd0XiGEXYLj4xOJArs4CRRHghvAAIEWfavNpf26g9nfvQ7DnWqAO1TWrro1dTVIT13EqnPlogdO2J88V/1C+1Jw4QDAgvhBcAIeys16dDdc095tFUVDfK0+btdW5XC4SM1Di9vc+lxj4mD0vndiPe8k8LGEJCUArL3kbn7/MCAKFslK3jCcu1qXH6Rucxn8+vIw2newSavdWNavC0dbdAuJiuDtxllQ3KvXL8sP8MwHDiyQsAhCi/36/axhbtOdao/97xuTbvqb3kezLT4rTg2mRaICDohOWTFwAINxaLRanxo5UaP1pjoyL6FV72VHfMr+lCCwSEIsILAJhAfzpwJ4yJVOH8K1VR09TdAuHk6bPacqheWw7Vd58bE2lTBi0QEMQILwBgAjarRavzM7T8hR2yqO8O3P9+V2aP1Uat7V4drG3u3FjvXAuE021ebT9yUtuPnOw+lxYICCbMeQEAExnqPi9en1+fHm/u3oema3JwX6uYaIGAQGKpNOEFQBgL9A67XS0Q9nbuQ9O10qmuny0QZqTFKyk2ionBuCjCC+EFAIZdXVNnC4TuJzSNOtpwus9zu1ogdO0WPCMtTpMTYgg06EZ4IbwAgCEG0wKha7fgGWnxunLCpVsgwJwIL4QXAAgaA22BcE3nSqeueTRXp5xrgQDzCvnwUllZqfvvv18ul0s2m00ffvihxowZ06/3El4AIPgNpAWCzWrRtKSxyjgv0GSkxSk2monBZhLy4eXmm2/WT3/6U82bN08NDQ2Ki4tTRET/luMRXgAgNF2sBUJfpoyP6dhgr3PIaUZanBLHRg34voGe4Gxmw/m7Cukddvfu3atRo0Zp3rx5kqSEhASDKwIAjASr1aKpiWM0NXGM8rPSJPVsgdC12qmi2q1qd4s+O3Fan504rVd313RfIyUuusfmejPS4jRx3OgLTgwe6tLycBJMv6sBP3kpLS3V448/rvLyctXU1Gjjxo0qKCjocY7D4dDjjz+u2tpaZWVl6Ze//KXmzJnTr+sXFxdr/fr18nq9OnbsmO6++2795Cc/6Xd9PHkBAPNr8LT1mBRcUd2oyhMe9fUXbVzMqO45NBmde9FMTRyjtypqtfyFHb12JO6KOc9863oCTKfNe2qG/Xc1rE9ePB6PsrKydP/99+vrX/96r+9v2LBBRUVFevbZZ5WTk6Mnn3xSCxcu1IEDB5SUlCRJstvtam/vveHRm2++qfb2dr333ntyOp1KSkrS7bffruzsbN1222191tPa2qrW1nN7DTQ2NvZ5HgDAPBLGRGretAmaN21C97Hm1nbtq+lYur2nc+n2J64mnTp9Vu8fOqH3D53oPnf0KKvaff4+Wyl0HfvfxXs1ZfyYsB9C8vr8+pfiPRf8XVkkrXm5QrdlpIzY72pIc14sFkuvJy85OTnKzs7W2rVrJUk+n0/p6elasWKFVq5ceclrbt26VY888ojeeOMNSdLjjz8uSXrooYf6PP+RRx7RmjVreh3nyQsA4EItEFrO9l7phKH5wwM3KvfK8YN+v2FzXtra2lReXq5Vq1Z1H7NarcrLy9PWrVv7dY3s7GzV1dXp5MmTio+PV2lpqb7zne9c8PxVq1apqKio++vGxkalp6cP/ocAAJhGVIRN102K13WT4ruPeX1+/ea9T/XY6/sv+f4xUTZFRYT3Mu3Wdq88rb1XgX1RXVPLJc8JlICGl/r6enm9XiUnJ/c4npycrP37L/0fiSRFRETo0Ucf1U033SS/36+vfOUruuOOOy54flRUlKKiouRwOORwOOT1XvoXDAAIXzarRTMnjevXub+5N3tITxPMYOvhE/rbdR9e8ryk2OgRqKZD0K02kqRFixZp0aJFA3pPYWGhCgsLux87AQBwIXOmJig1Plq17pY+53JYJKXEdywFDnfB+LsK6B7MiYmJstlscrlcPY67XC6lpKQE8lYAAAyazWrR6vwMSedWzHTp+np1fkbYT9aVgvN3FdDwEhkZqdmzZ6ukpKT7mM/nU0lJiXJzcwN5q14cDocyMjKUnZ09rPcBAJjD7ZmpeuZb1yslvudwR0p8NMukvyDYflcDXm3U3NysQ4cOSZJmzZqlJ554QvPnz1dCQoImT56sDRs2aOnSpfrVr36lOXPm6Mknn9Qf//hH7d+/v9dcmOHAPi8AgIFgh93+C9kddrdv36758+d3f9210mfp0qVav369lixZouPHj+vhhx9WbW2t7Ha7Nm/ePOzBhQm7AIDBsFktYT8pt7+C5XcVlL2NhoInLwAAhJ6B/P0O6JwXAACA4UZ4AQAAIcU04YXVRgAAhAfmvAAAAMMx5wUAAJiWacILw0YAAIQHho0AAIDhGDYCAACmFZRdpYei60FSY2OjwZUAAID+6vq73Z8BIdOFl6amJklSenq6wZUAAICBampqUnx8/EXPMd2cF5/Pp+rqasXGxspiCb3GWtnZ2dq2bZvRZfSbkfUO970Dff1AXG+w1xjM+/r7nsbGRqWnp6uqqop5Zv3E5zx47h1sn/OhvD/UP+d+v19NTU1KS0uT1XrxWS2me/JitVo1adIko8sYNJvNFlJ/AIysd7jvHejrB+J6g73GYN430PfExcWF1H+7RuJzHjz3DrbP+VDeb4bP+aWeuHRhwm6QKSwsNLqEATGy3uG+d6CvH4jrDfYag3lfqP23GEpC7XfL53zkrjeU94fT59x0w0YARhbbEwDmF2yfc568ABiSqKgorV69WlFRUUaXAmCYBNvnnCcvAAAgpPDkBQAAhBTCCwAACCmEFwAAEFIILwAAIKQQXgAAQEghvAAYNlVVVbrllluUkZGhmTNn6k9/+pPRJQEIsFOnTumGG26Q3W5XZmam1q1bN+z3ZKk0gGFTU1Mjl8slu92u2tpazZ49WwcPHtSYMWOMLg1AgHi9XrW2tiomJkYej0eZmZnavn27xo8fP2z3NF1vIwDBIzU1VampqZKklJQUJSYmqqGhgfACmIjNZlNMTIwkqbW1VX6/X8P9XIRhIwAXVFpaqvz8fKWlpclisai4uLjXOQ6HQ1OmTFF0dLRycnJUVlbW57XKy8vl9XqVnp4+zFUDGIhAfM5PnTqlrKwsTZo0SQ899JASExOHtWbCC4AL8ng8ysrKksPh6PP7GzZsUFFRkVavXq0dO3YoKytLCxcuVF1dXY/zGhoadO+99+rXv/71SJQNYAAC8TkfN26cdu3apcrKSr344otyuVzDWjNzXgD0i8Vi0caNG1VQUNB9LCcnR9nZ2Vq7dq0kyefzKT09XStWrNDKlSsldTxGvu222/TAAw/onnvuMaJ0AP002M/5+b773e9qwYIFuvvuu4etTp68ABiUtrY2lZeXKy8vr/uY1WpVXl6etm7dKkny+/267777tGDBAoILEIL68zl3uVxqamqSJLndbpWWlurqq68e1roILwAGpb6+Xl6vV8nJyT2OJycnq7a2VpL0/vvva8OGDSouLpbdbpfdbtfu3buNKBfAIPTnc37kyBHNmzdPWVlZmjdvnlasWKHrrrtuWOtitRGAYTN37lz5fD6jywAwjObMmSOn0zmi9+TJC4BBSUxMlM1m6zUxz+VyKSUlxaCqAARSsH7OCS8ABiUyMlKzZ89WSUlJ9zGfz6eSkhLl5uYaWBmAQAnWzznDRgAuqLm5WYcOHer+urKyUk6nUwkJCZo8ebKKioq0dOlS3XDDDZozZ46efPJJeTweLVu2zMCqAQxESH7O/QBwAe+8845fUq/X0qVLu8/55S9/6Z88ebI/MjLSP2fOHP+HH35oXMEABiwUP+fs8wIAAEIKc14AAEBIIbwAAICQQngBAAAhhfACAABCCuEFAACEFMILAAAIKYQXAAAQUggvAAAgpBBeAABASCG8AACAkEJ4AQAAIYXwAgAAQsr/D1V2HYtK+H9IAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.loglog(num_points_range, error_range_smooth_h1, \"-o\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Non-Smooth\n", + "\n", + "For non-smooth functions, convergence over resolution $N$ is only polynomially linear." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "true_value_nonsmooth_mse = (\n", + " 0.0420412074249998476683389584173419174549303041521076984836046108\n", + ")\n", + "domain_extent_smooth_mse = 1.0 # Domain size that makes the function non-periodic!\n", + "fn_smooth_mse = lambda x: jnp.exp(-100 * (x - 1) ** 2) * jnp.sin(x)" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "error_range_nonsmooth_mse = [\n", + " get_difference(\n", + " num_points,\n", + " domain_extent_smooth_mse,\n", + " ex.metrics.MSE,\n", + " true_value_nonsmooth_mse,\n", + " fn_smooth_mse,\n", + " )\n", + " for num_points in num_points_range\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[]" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGhCAYAAACphlRxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA56UlEQVR4nO3deXhU9b3H8c+ZyTIkJAMhEAKERUAwBhKBJCzJLSIqogEu4loVUVEBCS3WFnvbIm3v9bb29hZIRMWquCCIKAqktpZqTcK+hMWALIY1kBAikwWyzcz9g8qtFTCBSc4s79fz5PHJ5DdzPijjfHK+ZzHcbrdbAAAAPsJidgAAAICmoLwAAACfQnkBAAA+hfICAAB8CuUFAAD4FMoLAADwKZQXAADgU4LMDuBpLpdLxcXFioiIkGEYZscBAACN4Ha7VVlZqU6dOsliufS+Fb8rL8XFxYqLizM7BgAAuAxHjhxRly5dLrnG78pLRESEpHN/+MjISJPTAACAxqioqFBcXNz5z/FL8bvy8vWoKDIykvICAICPacwhHxywCwAAfArlBQAA+BTKCwAA8CmUFwAA4FMoLwAAwKdQXgAAgE+hvAAAAJ9CeQEAAD6F8gIAAHyK311ht7k4XW5tLCpXaWWNOkTYlNIjSlYLN34EAKClUV4a4aNdxzVnZaGOO2rOPxZrt2l2RrxGJcSamAwAgMDD2Og7fLTruKa8ufUbxUWSTjhqNOXNrfpo13GTkgEAEJgoL5fgdLk1Z2Wh3Bf42dePzVlZKKfrQisAAEBzoLxcwsai8m/tcflnbknHHTXaWFTecqEAAAhwlJdLKK28eHG5nHUAAODKUV4uoUOEzaPrAADAlaO8XEJKjyjF2m261AnRwVZDXdq2arFMAAAEOsrLJVgthmZnxEvSRQtMvdOtjKw8fVxY0nLBAAAIYJSX7zAqIVYL7hugjvZvjoZi7Tb9amyC+nex6/SZek1+fbPmrPxctQ1Ok5ICABAYDLfb7Vfn+VZUVMhut8vhcCgyMtJjr3uxK+zWNbj0m4/26I95RZKkhM6RyrpngLpHh3ts2wAA+LumfH5TXjxkze4SPblsu06fqVfr0CD91/h+GpPYqcW2DwCAL2vK57ffjI2ys7MVHx+v5ORkU7Z/wzUx+tOMdKV0j1JVbYMy396mWct36GwdYyQAADyJPS8e1uB0ae6afcr6ZL/cbunqmNbKuneAro6JaPEsAAD4ioDc8+ItgqwWPXlTH735cKraR4Rqb0mVxmTlaemmw/KznggAgCkoL81kWK9o5WSmK713tGrqXfrJ8p2asaRAlTX1ZkcDAMCnUV6aUfuIUC2alKKfjOorq8XQh9uLddv8PO086jA7GgAAPovy0swsFkNThvfUO48NVuc2rXTo1BmNX5CvV/OLGCMBAHAZKC8tZGC3KK3OTNNN8TGqd7o1Z2WhHn1ji06fqTM7GgAAPoXy0oLahIXoxfsH6pmMeIVYLfq4sESj5+Zq88Fys6MBAOAzKC8tzDAMPTish96bOlTd24Wp2FGju15ar+xP9svlYowEAMB3obyYJKGzXasy0zUuqZOcLree+/MXmvjqRp2srDU7GgAAXo3yYqLWoUH637uS9NsJ/WULtih3X5lumZurvH1lZkcDAMBrUV5MZhiG7hwUp5VPpKlPTITKqmp1/ysb9Ls/f6EGp8vseAAAeB3Ki5foHROhD54YpntSusrtlrI+2a+7X1qv4tNnzY4GAIBXobx4EVuwVc+O76f591yn1qFB2nzoK42el6uPC0vMjgYAgNegvHihjMROWp2Zpv5d7Dp9pl6TX9+sOSs/V20Dd6gGAIDy4qW6tQvXu48P1cNpPSRJr+Yf1IQF63SwrNrkZAAAmIvy4sVCgiz6+W3xevmBQWoTFqydxxy6bX6ePtxebHY0AABMQ3nxASPjY5STma7k7m1VVdugzLe3adbyHTpbxxgJABB4KC8+olObVnp78mBNH9FLhiEt2XREY7PztLek0uxoAAC0KMqLDwmyWvTkTX305sOpim4dqr0lVRqTlaelmw5zh2oAQMCgvPigYb2i9acZ6UrvHa2aepd+snynZiwpUGVNvdnRAABodpQXH9U+IlSLJqXox6P6yGox9OH2YmXMz9POow6zowEA0KwoLz7MYjE0dXgvvfPYYHVu00oHT53R+AX5ejW/iDESAMBvUV78wMBuUVqdmaab4mNU73RrzspCPfrGFp0+U2d2NAAAPI7y4ifahIXoxfsH6pmMeIVYLfq4sESj5+Zq88Fys6MBAOBRlBc/YhiGHhzWQ+9NHaru7cJU7KjRXS+tV/Yn++VyMUYCAPgHyosfSuhs16rMdI1N6iSny63n/vyFJr66UScra82OBgDAFaO8+KnWoUH6w11J+u2E/rIFW5S7r0y3zM1V3r4ys6MBAHBFKC9+zDAM3TkoTiufSFOfmAiVVdXq/lc26Hd//kINTpfZ8QAAuCyUlwDQOyZCK6YN0z0pcXK7paxP9uvul9ar+PRZs6MBANBklJcA0SrEqmfH99f8e65T69AgbT70lUbPy9XHhSVmRwMAoEkoLwEmI7GTVmemqV9nu06fqdfk1zdrzsrPVdvAHaoBAL6B8hKAurUL17tThuihYT0kSa/mH9SEBet0sKza5GQAAHw3ykuACg2y6hcZ8Xr5gUFqExasncccum1+nj7cXmx2NAAALonyEuBGxscoJzNdyd3bqqq2QZlvb9Os5Tt0to4xEgDAO1FeoE5tWuntyYM1fUQvGYa0ZNMRjc3O096SSrOjAQDwLZQXSJKCrBY9eVMfvfFQqqJbh2pvSZXGZOVp6abD3KEaAOBVKC/4hrTe0frTjHSl945WTb1LP1m+UzOWFKiypt7saAAASKK84ALaR4Rq0aQU/XhUH1kthj7cXqyM+XnaedRhdjQAACgvuDCLxdDU4b209NHB6mS36eCpMxq/IF+v5hcxRgIAmIrygksa1D1KOTPSdWN8jOqdbs1ZWahH39ii02fqzI4GAAhQlBd8pzZhIXrp/oF6JiNeIVaLPi4s0ei5udp8sNzsaACAAER5QaMYhqEHh/XQe1OHqnu7MBU7anTXS+uV/cl+uVyMkQAALYfygiZJ6GzXqsx0jU3qJKfLref+/IUmvrpRJytrz69xutxad+CUPig4pnUHTslJuQEAeJDh9rOjLysqKmS32+VwOBQZGWl2HL/ldru1bPNR/eLDXaqpdym6daj+cFeSqmrrNWdloY47as6vjbXbNDsjXqMSYk1MDADwZk35/Ka84IrsLanUE4u3am9J1UXXGP/454L7BlBgAAAX1JTPb8ZGuCJXx0Tog2lpuiu5y0XXfN2O56wsZIQEALhilBdcsVYhVo1Lunh5kc4VmOOOGm0s4gwlAMCVobzAI0ora757URPWAQBwMZQXeESHCJtH1wEAcDGUF3hESo8oxdpt5w/OvZC2YcFK6RHVYpkAAP7J68rLkSNHNHz4cMXHx6t///5atmyZ2ZHQCFaLodkZ8ZJ00QLz1Zl6zVn5uWrqnS0XDADgd7zuVOnjx4+rpKRESUlJOnHihAYOHKi9e/cqPDy8Uc/nVGlzfbTr+AWv85LQ2a6PC0skSdd2ilTWvQPUI7px/00BAP7Pr67zkpiYqFWrVikuLq5R6ykv5nO63NpYVK7Syhp1iLAppUeUrBZDn3xRqiff2a7y6jqFh1j1X+P7aWxSZ7PjAgC8QLNe5+Wzzz5TRkaGOnXqJMMwtGLFim+tyc7OVvfu3WWz2ZSamqqNGzc2dTOSpC1btsjpdDa6uMA7WC2GhvRsp7FJnTWkZztZLecGSdf36aCczHSldI9SdZ1TM5YU6On3djBGAgA0SZPLS3V1tRITE5WdnX3Bny9dulQzZ87U7NmztXXrViUmJurmm29WaWnp+TVJSUlKSEj41ldxcfH5NeXl5XrggQf00ksvXTJPbW2tKioqvvEF79XRbtPiyamaPqKXDEN6e+MRjc3K1/7SSrOjAQB8xBWNjQzD0Pvvv69x48adfyw1NVXJycnKysqSJLlcLsXFxWn69OmaNWtWo163trZWN954oyZPnqz777//kmufeeYZzZkz51uPMzbyfnn7yvSDpQUqq6pVq2CrfjUuQRMGXvpidwAA/2Ta7QHq6uq0ZcsWjRw58v83YLFo5MiRWrduXaNew+1268EHH9SIESO+s7hI0tNPPy2Hw3H+68iRI5edHy0rrXe0cmakaWjPdjpb79SPlm3Xk+9s15m6BrOjAQC8mEfLS1lZmZxOp2JiYr7xeExMjE6cONGo18jPz9fSpUu1YsUKJSUlKSkpSTt37rzo+tDQUEVGRn7jC76jQ4RNbzycqpk3Xi2LIS3felQZ8/O05wTjPwDAhQWZHeBfpaWlyeVymR0DLchqMZR5Q2+l9IjSjCXbdOBktcZm5WvOmGt1V3KcDONSl74DAAQaj+55iY6OltVqVUlJyTceLykpUceOHT25KfihwVe1U05mur53dXvVNrg0672dmrGkQFW1jJEAAP/Po+UlJCREAwcO1Jo1a84/5nK5tGbNGg0ZMsSTm4Kfatc6VK8+mKyfjOorq8XQh9uLddu8XO065jA7GgDASzS5vFRVVamgoEAFBQWSpKKiIhUUFOjw4cOSpJkzZ2rhwoVatGiRdu/erSlTpqi6ulqTJk3yaHD4L4vF0JThPfXOY4PVyW7TwVNnNP75tXpj3UF5+TUVAQAtoMmnSn/66ae6/vrrv/X4xIkT9dprr0mSsrKy9Nxzz+nEiRNKSkrSvHnzlJqa6pHA34Ur7PqXr6rr9NS72/XX3eeuE3RLQkf99+39ZW8VbHIyAIAn+dXtAZqK8uJ/3G63/phXpN98tEf1Tre6tG2lrHsHKCmujdnRAAAeYtp1XsyUnZ2t+Ph4JScnmx0FHmYYhh5Jv0rLHh+qLm1b6ehXZ3XHC2v1cu6XjJEAIACx5wU+xXG2XrOW79Cfdp27btDIazrod3ckqk1YiMnJAABXIiD3vCAw2FsF6/nvD9Cvxl6rEKtFf91dqtFzc7XlULnZ0QAALYTyAp9jGIbuH9Jd700dqu7twlTsqNGdL67Xgk8PyOXyqx2JAIALoLzAZyV0tmtVZrrGJHaS0+XWbz7ao0mvbdKpqlqzowEAmhHlBT6tdWiQ5t6dpP8e30+hQRb9fe9JjZ6Xq/VfnjI7GgCgmVBe4PMMw9DdKV31wRPD1LN9uEoqanXvwvWat2afnIyRAMDvUF7gN/p2jNTK6Wm6fUAXudzS7z/eqwde2aDSyhqzowEAPIjyAr8SFhKk/7kzUb+7I1Gtgq3K339Ko+fmKW9fmdnRAAAe4jflhYvU4Z9NGNhFK6cPU5+YCJVV1er+Vzbof/7yhRqcLrOjAQCuEBepg1+rqXdqzsrP9fbGI5KklB5Rmnf3depot5mcDADwz7hIHfAPtmCrnh3fX3PvTlJ4iFUbi8o1el6uPvmi1OxoAIDLRHlBQBib1FmrMtMVHxup8uo6TXp1k579027VM0YCAJ9DeUHA6BEdrvemDtXEId0kSS/+/Uvd9eI6HTt91uRkAICmoLwgoNiCrZozNkELvj9AEbYgbT18WqPn5urjwhKzowEAGonygoB0S79Y5WSmK7GLXY6z9Zr8+mb9cmWh6hoYIwGAt6O8IGDFRYVp2eND9UhaD0nSK/lFmvDCWh0+dcbkZACAS6G8IKCFBFn0s9vi9fIDg2RvFawdRx26dV6ucnYeNzsaAOAiKC+ApJHxMcqZka6B3dqqsrZBU9/aqp+t2KmaeqfZ0QAA/4LyAvxD5zattOTRwZoyvKck6c31h/Xvz6/VlyerTE4GAPhnflNeuD0APCHYatFPRvXVa5OSFRUeot3HK5QxP08fFBwzOxoA4B+4PQBwESUVNcp8e5s2FJVLku4aFKdnxlyrViFWk5MBgP/h9gCAB8RE2vTWI6nKvKG3DENauvmIxmbnaV9JpdnRACCgUV6ASwiyWjTzxqv15sOpim4dqr0lVRqTla9lm4+YHQ0AAhblBWiEYb2i9acZ6UrrFa2z9U499e4OzXynQNW1DWZHA4CAQ3kBGql9RKgWPZSiH910tSyG9N7WYxqTlafdxyvMjgYAAYXyAjSB1WLoiRG9teTRIeoYadOBk9Ual52vxRsOy8+OfQcAr0V5AS5DSo8o5cxI1/A+7VXb4NJP39+pzCUFqqypNzsaAPg9ygtwmaLCQ/TKxGQ9fUtfBVkMrdxerIz5edp1zGF2NADwa5QX4ApYLIYe+15PLX1siDq3aaWDp85o/PNrtWjtQcZIANBMKC+ABwzs1larM9N0Y3yM6pwuzf7wc015c6scZxkjAYCnUV4AD2kTFqKX7h+oX9wWr2CroY8+P6Fb5+Wq4Mhps6MBgF/xm/LCvY3gDQzD0ENpPbR8ylB1jQrT0a/OasKCtXo590vGSADgIdzbCGgmFTX1mrV8h3J2npAk3dC3g353R6LahoeYnAwAvA/3NgK8QKQtWNn3DtCvxiUoJMiiNXtKNXperjYfLD+/xulya92BU/qg4JjWHTglp8uvfpcAgGbBnhegBXxe7NATi7epqKxaVouhmTderR7twvWr1YU67qg5vy7WbtPsjHiNSog1MS0AtLymfH5TXoAWUlXboP94f6c+KCi+6BrjH/9ccN8ACgyAgMLYCPBCrUOD9Ie7kvTf4/tddM3Xv0nMWVnICAkALoLyArQgwzDUrV34Jde4JR131GhjUfkl1wFAoKK8AC2stLLmuxc1YR0ABBrKC9DCOkTYPLoOAAIN5QVoYSk9ohRrt50/OPdCWocGaUDXNi0VCQB8CuUFaGFWi6HZGfGSdNECU1XboPtf2aiSCkZHAPCvKC+ACUYlxGrBfQPU0f7N0VCs3aaHhnVXeIhVG4vKdcvcXP1970mTUgKAd+I6L4CJnC63NhaVq7SyRh0ibErpESWrxVBRWbWmvbVVhccrJElThvfUkzderSArv28A8E9cpI7yAj9QU+/Uf67erTfWH5IkDerWVvPuuU6d2rQyORkAeB4XqQP8gC3Yql+NS1D2vQMUERqkzYe+0uh5ufrbnhKzowGAqfymvGRnZys+Pl7JyclmRwE86tb+sVqVmaZ+ne06faZeD722Wf+Vs1v1TpfZ0QDAFIyNAB9R2+DUszl79Nrag5Kk67q20fx7rlOXtmHmBgMAD2BsBPih0CCrnhlzrV64b6AibUHadvi0bp2Xp798fsLsaADQoigvgI8ZldBRqzPTlRjXRo6z9Xr0jS365cpC1TUwRgIQGCgvgA+KiwrTsseG6JG0HpKkV/KLdMcLa3Wk/IzJyQCg+VFeAB8VEmTRz26L18sPDJK9VbC2H3Vo9LxcfbTruNnRAKBZUV4AHzcyPkY5M9I1oGsbVdY06PE3t2r2B7tU2+A0OxoANAvKC+AHOrdppaWPDdFj37tKkrRo3SHdvmCtDpZVm5wMADyP8gL4iWCrRU/fco1efTBZbcOCtetYhW6bn6dVO4rNjgYAHkV5AfzM9X07KGdGupK7t1VVbYOeWLxN//H+TtXUM0YC4B8oL4AfirW30tuTB2va9T1lGNJbGw7r359fqy9PVpkdDQCuGOUF8FNBVoueurmvFk1KUbvwEO0+fm6MtGLbMbOjAcAVobwAfu7frm6vnBnpGnxVlM7UOfWDpQWatXyHztYxRgLgmygvQACIibTprUcGK/OG3jIMacmmIxqXna/9pZVmRwOAJqO8AAHCajE088ar9dbDqYpuHaovSiqVMT9fy7ccNTsaADQJ5QUIMEN7RStnRpqG9Wqns/VOPblsu360bLvO1DWYHQ0AGoXyAgSgDhE2vf5QqmbeeLUshvTulqMam5WvvSWMkQB4P8oLEKCsFkOZN/TW4smD1SEiVPtKqzQmK09LNx2W2+02Ox4AXJTflJfs7GzFx8crOTnZ7CiATxl8VTvlzEhXeu9o1dS79JPlO/XDpQWqrmWMBMA7GW4/+xWroqJCdrtdDodDkZGRZscBfIbL5dYLnx3Q//xlr5wut66KDlf29wfomljeRwCaX1M+v/1mzwuAK2OxGJo6vJeWPDpYsXabviyr1tjsfC3ewBgJgHehvAD4huTuUVqdma4RfTuorsGln76/U5lLClRZU292NACQRHkBcAFR4SF6+YFB+unovgqyGFq5vVgZ8/O065jD7GgAQHkBcGEWi6FH/62nlj42RJ3btNLBU2c0/vm1en3dQcZIAExFeQFwSQO7tdXqzDSNvCZGdU6XfvHB55q2eKsqGCMBMAnlBcB3ahMWooUPDNTPb4tXsNVQzs4TunVernYcPW12NAABiPICoFEMw9DDaT307uND1aVtKx0pP6vbF6zVK3lFjJEAtCjKC4AmSYxro9WZ6Rp1bUfVO9365apCPfbGFjnOMEYC0DIoLwCazN4qWAvuG6A5Y65ViNWivxSWaPS8XG07/JXZ0QAEAMoLgMtiGIYmDu2u5VOGqlu7MB07fVZ3vLBOCz/7kjESgGZFeQFwRfp1sWvl9DTd2j9WDS63/jNntx5ZtFlfVdeZHQ2An6K8ALhikbZgZd1znX49LkEhQRat2VOqW+flasuhcrOjAfBDlBcAHmEYhu4b3E3vTx2qHtHhKnbU6M4X12vBpwfkcjFGAuA5lBcAHnVtp3NjpLFJneR0ufWbj/booUWbdKqq1uxoAPwE5QWAx7UODdIf7krSf4/vp9Agiz794qRGz8vVhi9PmR0NgB+gvABoFoZh6O6UrvrgiWHq2T5cJRW1umfhemX9bR9jJABXhPICoFn17RipD59I0/gBneVyS7/7y15NfHWjTlYyRgJweSgvAJpdeGiQfn9nkp6b0F+tgq3K3Vem0fNytfZAmdnRAPggyguAFnPHoDh9+MQwXR3TWicra3Xfyxv0h7/ulZMxEoAmoLwAaFG9YyL0wbQ03Tmoi1xu6Q9/3af7/7hBpRU1ZkcD4CMoLwBaXKsQq347IVH/e1eiwkKsWnvglEbPy1XuvpNmRwPgAygvAEzz79d10YdPpKlvxwiVVdXpgVc26nd//kINTpfZ0QB4Mb8pL9nZ2YqPj1dycrLZUQA0Qa8OrbVi2jDdm9pVbreU9cl+3fvyBp1wMEYCcGGG289u/1pRUSG73S6Hw6HIyEiz4wBogg+3F+un7+1UVW2DosJD9Ps7EzW8TwdJktPl1saicpVW1qhDhE0pPaJktRgmJwbgKU35/Ka8APAqRWXVemLxVn1eXCFJmjK8pxI6RerXq3fr+D/tjYm12zQ7I16jEmLNigrAgygvlBfAp9XUO/VfObv1+rpDF13z9T6XBfcNoMAAfqApn99+c8wLAP9hC7bql2MTlHXPdbrYYOjr37rmrCzkOjFAgKG8APBa7VqH6lK1xC3puKNGG4vKWyoSAC9AeQHgtUorG3fGUWPXAfAPlBcAXqtDhM2j6wD4B8oLAK+V0iNKsXbbRY97kaTwEKuS4tq0VCQAXoDyAsBrWS2GZmfES9JFC0x1nVN3v7ROR8rPtFwwAKaivADwaqMSYrXgvgHqaP/maCjWbtOU7/WUvVWwth91aPS8XH2067hJKQG0JK7zAsAnXOwKu8dOn9X0xVu19fBpSdKDQ7vr6dF9FRpkNTcwgCbhInWUFyCg1Dtd+t1fvtCLf/9SktSvs11Z916nbu3CTU4GoLG4SB2AgBJstejpW67RKw8OUtuwYO085tBt8/K0egdjJMAfUV4A+I0RfWOUMyNdg7q1VWVtg6Yt3qqfr9ilmnqn2dEAeBDlBYBfibW30pJHB2vq8J6SpDfWH9L459eqqKza5GQAPIXyAsDvBFkt+vGovnptUrKiwkNUeLxCGfPz9OH2YrOjAfAAygsAvzW8TwflZKYrpUeUqmoblPn2Nv30/Z2MkQAfR3kB4Nc62m1a/Eiqpo/oJcOQFm84rHHZ+TpwssrsaAAuE+UFgN8Lslr05E199PpDKYpuHaI9JyqVMT9PK7YdMzsagMtAeQEQMNJ7t1dOZrqGXNVOZ+qc+sHSAs1avkNn6xgjAb6E8gIgoHSItOnNR1I144beMgxpyaYjGpedr/2llWZHA9BIlBcAAcdqMfTDG6/WWw+nKrp1qL4oqVTG/Hwt33LU7GgAGoHyAiBgDe0VrZwZaRrWq53O1jv15LLt+tGy7TpT12B2NACXQHkBENA6RNj0+kOpmnnj1bIY0rtbjmpsVr72ljBGArwV5QVAwLNaDGXe0FuLJw9Wh4hQ7Sut0pisPL2z+Yj87N61gF+gvADAPwy+qp1yZqQrvXe0aupd+vG7O/TkO9tVXcsYCfAmlBcA+CfRrUO1aFKKnrq5jyyG9N62YxqTlac9JyrMjgbgHygvAPAvLBZD067vpSWPDlHHSJsOnKzW2Kx8Ldl4mDES4AUoLwBwESk9opQzI13D+7RXbYNLs97bqR8sLVAVYyTAVJQXALiEqPAQvTIxWT8Z1VdWi6EPCoo1Zn6eCosZIwFmobwAwHewWAxNGd5TSx8drFi7TV+WVWvc8/l6c/0hxkiACSgvANBIg7pHKSczXTf07aC6Bpd+tmKXpr+9TZU19WZHAwKK35SX7OxsxcfHKzk52ewoAPxY2/AQvTxxkP5j9DUKshhateO4Mubnadcxh9nRgIBhuP1sn2dFRYXsdrscDociIyPNjgPAj209/JWmL96mY6fPKsRq0c9uu0b3D+4mwzDMjgb4nKZ8fvvNnhcAaGkDurbV6sw0jbwmRnVOl37xweeatnirKhgjAc2K8gIAV6BNWIgWPjBQP78tXsFWQzk7T+i2eXnacfS02dEAv0V5AYArZBiGHk7roXcfH6oubVvpcPkZ3b5grV7NL+JsJKAZUF4AwEMS49podWa6Rl3bUfVOt+asLNTjb26R4wxjJMCTKC8A4EH2VsFacN8APZMRrxCrRX/+vES3zs9VwZHTZkcD/AblBQA8zDAMPTish5ZPGaquUWE6+tVZTViwVi/nfskYCfAAygsANJN+XexalZmmW/vFqsHl1q9X79bk17fo9Jk6s6MBPo3yAgDNKNIWrKx7r9OvxiUoxGrRX3eX6NZ5edpy6CuzowE+i/ICAM3MMAzdP7ib3ps6VN3bhenY6bO668V1evHvB+RyMUYCmoryAgAtJKGzXSunpykjsZMaXG49+6c9euT1zSqvZowENAXlBQBaUIQtWPPuTtJ//Xs/hQRZ9Lc9pbp1Xq42Hyw3OxrgMygvANDCDMPQvaldtWLqMF0VHa7jjhrd9dJ6Pf/pfsZIQCNQXgDAJPGdIvXh9DSNS+okp8ut3370hSa9tkmnqmrNjgZ4NcoLAJiodWiQ/veuJP3m9n4KDbLo73tPavS8XG348pTZ0QCvRXkBAJMZhqG7krvqwyfS1LN9uEoqanXPwvXK+ts+xkjABVBeAMBL9OkYoQ+fSNP4AZ3lcku/+8teTXx1o8oYIwHfQHkBAC8SHhqk39+ZpOcm9Jct2KLcfWUaPTdX6w4wRgK+RnkBAC90x6A4rXwiTb07tFZpZa2+//J6zf3rPjkZIwGUFwDwVr1jzo2R7hzURS639L9/3av7/7hBpZU1ZkcDTEV5AQAv1irEqt9OSNTv70xUWIhVaw+c0ui5ecrfX2Z2NMA0lBcA8AHjB3TRh0+kqU9MhMqqanXfHzfo9x/vZYyEgER5AQAf0atDa33wxDDdkxInt1uat2afvv/yepVUMEZCYKG8AIAPsQVb9ez4/pp7d5LCQ6xa/2W5Rs/N1Wd7T5odDWgxlBcA8EFjkzpr5fQ0XRMbqVPVdZr46kb97s9fqMHpktPl1roDp/RBwTGtO3CK0RL8juF2u/3qb3VFRYXsdrscDociIyPNjgMAzaqm3qlfrSrUWxsOS5J6tW+tipp6lVb+/4XtYu02zc6I16iEWLNiAt+pKZ/f7HkBAB9mC7bqP/+9n+bfc51sQRbtP1n1jeIiSSccNZry5lZ9tOu4SSkBz6K8AIAfGN0vVhG24Av+7Ovd63NWFjJCgl+gvACAH9hYVK6Tl7gHklvScUeNNhaVt1wooJlQXgDADzT2qrtcnRf+gPICAH6gQ4StUevahYc0cxKg+VFeAMAPpPSIUqzdJuM71v3PX77QsdNnWyQT0FwoLwDgB6wWQ7Mz4iXpWwXm6+9twRZtO+LQ6Lm5+mthSYvmAzyJ8gIAfmJUQqwW3DdAHe3fHCF1tNv0wn0D9PEPv6fELnY5ztbrkdc369erClXX4DIpLXD5uEgdAPgZp8utjUXlKq2sUYcIm1J6RMlqObf/pa7Bpf/+0x69kl8kSUqMa6Ose65TXFSYmZGBJn1+U14AIAD95fMT+tGy7aqoaVCkLUjP3ZGom6/taHYsBDCusAsAuKSbru2onBnpSopro4qaBj32xhbNWfk5YyT4BMoLAASoLm3D9M5jQzQ5vYck6dX8g7rjhbU6Un7G5GTApVFeACCAhQRZ9B+3xuuPEwepTViwth91aPS8XO6DBK9GeQEA6IZrYrQ6M10Du7VVZU2DHn9zq2Z/sEu1DU6zowHfQnkBAEiSOrdppSWPDtZj37tKkrRo3SFNWLBOh05Vm5wM+CbKCwDgvGCrRU/fco1efTBZbcOCtfOYQ7fNy9PqHYyR4D0oLwCAb7m+bwflzEjXoG5tVVnboGmLt+rnK3appp4xEsxHeQEAXFCs/dwYaerwnpKkN9Yf0vjn16qojDESzEV5AQBcVJDVoh+P6qtFD6UoKjxEhccrlDE/Tx9uLzY7GgIY5QUA8J2+d3V75WSmK6VHlKpqG5T59jb99P2djJFgCsoLAKBROtptWvxIqqaP6CXDkBZvOKxx2fk6cLLK7GgIMJQXAECjBVktevKmPnr9oRS1Cw/RnhOVypifpxXbjpkdDQGE8gIAaLL03u31pxnpGnxVlM7UOfWDpQWatXwHYyS0CK8rL6dPn9agQYOUlJSkhIQELVy40OxIAIAL6BBp01uPDNaMG3rLMKQlm45obFa+9pcyRkLzMtxut9vsEP/M6XSqtrZWYWFhqq6uVkJCgjZv3qx27do16vlNuaU2AMAz8veXacaSApVV1SosxKpfj0vQ+AFdzI4FH9KUz2+v2/NitVoVFhYmSaqtrZXb7ZaX9SsAwL8Y1itaOTPSNLRnO52pc2rmO9v11LLtOlvHGAme1+Ty8tlnnykjI0OdOnWSYRhasWLFt9ZkZ2ere/fustlsSk1N1caNG5u0jdOnTysxMVFdunTRU089pejo6KbGBAC0sA4RNr3xcKp+OPJqWQxp2ZajGpudp30llWZHg59pcnmprq5WYmKisrOzL/jzpUuXaubMmZo9e7a2bt2qxMRE3XzzzSotLT2/5uvjWf71q7j43EWP2rRpo+3bt6uoqEiLFy9WSUnJZf7xAAAtyWoxNGNkb735SKraR4Rqb0mVxmTla9nmI2ZHgx+5omNeDMPQ+++/r3Hjxp1/LDU1VcnJycrKypIkuVwuxcXFafr06Zo1a1aTtzF16lSNGDFCEyZMuODPa2trVVtbe/77iooKxcXFccwLAJjsZGWtZr5ToNx9ZZKk2wd00a/GXauwkCCTk8EbmXbMS11dnbZs2aKRI0f+/wYsFo0cOVLr1q1r1GuUlJSosvLcLkaHw6HPPvtMffr0uej6Z599Vna7/fxXXFzclf0hAAAe0T4iVIsmpehHN50bIy3felRjsvL1xQnGSLgyHi0vZWVlcjqdiomJ+cbjMTExOnHiRKNe49ChQ0pPT1diYqLS09M1ffp09evX76Lrn376aTkcjvNfR46waxIAvIXFYuiJEb319uTBiokM1f7SKo3NztM7m45wMgYum9ftu0tJSVFBQUGj14eGhio0NLT5AgEArljqVe2Uk5muH76zXZ/tPakfL9+hdV+e0q/HJSg81Os+iuDlPLrnJTo6Wlar9VsH2JaUlKhjx46e3BQAwMe0ax2q1x5M1o9H9ZHVYuj9bceUkZWn3ccrzI4GH+PR8hISEqKBAwdqzZo15x9zuVxas2aNhgwZ4slNAQB8kMViaOrwXlry6GB1jLTpy5PVGpedr7c3HmaMhEZrcnmpqqpSQUHB+dFOUVGRCgoKdPjwYUnSzJkztXDhQi1atEi7d+/WlClTVF1drUmTJnk0OADAdyV3j1LOjHRd36e9ahtcevq9nZqxpEBVtQ1mR4MPaPKp0p9++qmuv/76bz0+ceJEvfbaa5KkrKwsPffcczpx4oSSkpI0b948paameiTwd+H2AADgO1wut17K/VLP/fkLOV1u9YgOV/a9AxTfif9/B5qmfH573b2NrhTlBQB8z5ZD5Zq+eJuKHTUKCbLoF7fF6/upXWUYhtnR0EJ8+t5Glys7O1vx8fFKTk42OwoAoIkGdovS6sx03dC3g+oaXPrZil2a/vY2VdbUmx0NXog9LwAAr+F2u/VybpF+89EeNbjc6t4uTFn3DlBCZ7vZ0dDMAnLPCwDA9xmGocn/dpXeeXyIOrdppYOnzmj882v1xrqDnI2E8ygvAACvM6BrW63OTNPIa2JU53Tp5x98rmmLt6qCMRJEeQEAeKk2YSFa+MBA/fy2eAVbDeXsPKHb5uVpx9HTZkeDySgvAACvZRiGHk7roWWPD1WXtq10uPyMbl+wVq/lFzFGCmCUFwCA10uKa6PVmem6+doY1TvdemZloaa8uVWOs4yRAhHlBQDgE+ytgvXCfQM1O+PcGOmjz0/o1nm5Kjhy2uxoaGGUFwCAzzAMQ5OG9dDyKUMVF9VKR786qzteWKs/5jFGCiSUFwCAz+nfpY1WTU/XLQkdVe9061erCvXoG1vkOMMYKRD4TXnhCrsAEFjsrYL1/PcH6Jdjr1WI1aKPC0s0el6uth3+yuxoaGZcYRcA4PN2HXNo2uKtOnTqjIIshmbd0lcPp/Xg3kg+hCvsAgACSkJnu1ZNT9Ot/WPV4HLr16t3a/Lrm3X6TJ3Z0dAMKC8AAL8QYQtW1j3X6dfjEhQSZNFfd5dq9NxcbTnEGMnfUF4AAH7DMAzdN7ib3p86VD2iw1XsqNFdL67Ti38/IJfLr46SCGiUFwCA37m2k10rp6dpTGInNbjcevZPe/Twok0qr2aM5A8oLwAAv9Q6NEhz707Ss+P7KTTIok++OKnRc3O16WC52dFwhSgvAAC/ZRiG7knpqhXThumq6HCdqKjR3S+t1/Of7pfL5ZbT5da6A6f0QcExrTtwSk5GSz6BU6UBAAGhurZBP1uxS+9vOyZJio+NVFlVrUora8+vibXbNDsjXqMSYs2KGbA4VRoAgH8RHhqk39+ZqN/e3l9BFkOFxyu+UVwk6YSjRlPe3KqPdh03KSUag/ICAAgYhmHo9oFd1CYs+II//3oUMWdlISMkL+Y35YXbAwAAGmNjUbnKqi5+1pFb0nFHjTYWcWCvt/Kb8jJt2jQVFhZq06ZNZkcBAHix0soaj65Dy/Ob8gIAQGN0iLA1al1069BmToLLRXkBAASUlB5RirXb9F23bMz+ZJ9O/ssBvfAOlBcAQECxWgzNzoiXpG8VmK+/D7FatPZAuW6Zm6u1+8taNB++G+UFABBwRiXEasF9A9TR/s0RUke7TS/cN0A5M9LUJyZCZVW1+v4fN+h/P97L2UdehIvUAQACltPl1saicpVW1qhDhE0pPaJktZzb/3K2zqk5Kz/Xkk1HJElDrmqnufckNfqYGTRNUz6/KS8AAFzCim3H9NP3d+pMnVPRrUP0h7uuU1rvaLNj+R2usAsAgIeMu66zVk5PU9+OESqrqtP9r2zQ7//yBWMkE1FeAAD4Dj3bt9aKacN0T0pXud3SvL/t170L16ukgmvBmIHyAgBAI9iCrXp2fD/NvTtJ4SFWbSgq1+i5ufps70mzowUcygsAAE0wNuncGOma2Eidqq7TxFc36rk/71GD02V2tIDhN+WFexsBAFrKVe1b6/2pQ/X91HNjpOxPDujehRt0wsEYqSVwthEAAFdg1Y5izVq+U1W1DYoKD9Hv70zU8D4dzI7lczjbCACAFnJb/05aNT1N13aKVHl1nR58dZN+8xFjpOZEeQEA4Ap1jw7X8ilD9cCQbpKkBZ8e0N0vrVfx6bMmJ/NPlBcAADzAFmzVL8cm6PnvD1BEaJA2H/pKt87L1Sd7Ss2O5ncoLwAAeNDofrFalZmmfp3t+upMvSa9tknP5uxWPWMkj6G8AADgYd3ahevdKUP04NDukqQXP/tSd724TscYI3kE5QUAgGYQGmTVM2Ou1Qv3DVCELUhbD5/W6Lm5+mthidnRfB7lBQCAZjQqIVY5melK7GKX42y9Hnl9s/5zdSFjpCtAeQEAoJnFRYVp2eND9dCwHpKkhblFuuOFdTr61RmTk/kmygsAAC0gJMiiX2TE68X7ByrSFqSCI+fGSH/5/ITZ0XwO5QUAgBZ087UdtTozXUlxbVRR06BH39iiX64sVF0DY6TGorwAANDC4qLC9M5jQzQ5/dwY6ZX8It3xwlodKWeM1BiUFwAATBASZNF/3BqvhQ8Mkr1VsLYfdWj0vFx9tIsx0nehvAAAYKIb42O0OjNN13Vto8qaBj3+5hY98+Hnqm1wmh3Na/lNecnOzlZ8fLySk5PNjgIAQJN0aXtujPTYv10lSXpt7UFNWLBOh08xRroQw+12u80O4UlNuaU2AADe5m97SjTzne06faZeEaFB+s2E/hrdL9bsWM2uKZ/ffrPnBQAAfzCib4xyMtM1sFtbVdY2aOpbWzX7g12Mkf4J5QUAAC/TqU0rLXl0sB7/Xk9J0qJ1h3T7grU6WFZtcjLvQHkBAMALBVstmnVLX706KVltw4K161iFbpufp1U7is2OZjrKCwAAXuz6Ph2UMyNdyd3bqqq2QU8s3qafrdipmvrAHSNRXgAA8HKx9lZ6e/JgTbv+3BjpzfWHNf75tSoK0DES5QUAAB8QZLXoqZv7atFDKYoKD1Hh8QrdNi9XH24PvDES5QUAAB/yvavbKyczXSk9olRd51Tm29v09HuBNUaivAAA4GM62m1a/Eiqpo/oJcOQ3t54WOOy83XgZJXZ0VoE5QUAAB8UZLXoyZv66PWHUhTdOkR7TlQqY36eVmw7Zna0Zkd5AQDAh6X3PjdGGnxVlM7UOfWDpQWatXyHX4+RKC8AAPi4DpE2vfXIYM24obcMQ1qy6YjGZuVrf6l/jpEoLwAA+AGrxdAPb7xabz6cqujWofqipFJjsvL03tajZkfzOMoLAAB+ZFivaOXMSNPQnu10ps6pme9s11PLtutsnf+MkSgvAAD4mQ4RNr3xcKp+OPJqWQxp2ZajGpudp30llWZH8wjKCwAAfshqMTRjZG+9+Uiq2keEam9JlcZk5WvZ5iNmR7tilBcAAPzY0J7RyslMV3rvaJ2td+qpd3foyXe260xdg9nRLpvflJfs7GzFx8crOTnZ7CgAAHiV9hGhWjQpRT+66dwYafnWoxqTla+9PjpGMtxut9vsEJ5UUVEhu90uh8OhyMhIs+MAAOBV1n95SjOWbFNJRa1swRb9ckyC7hjURYZhmJqrKZ/ffrPnBQAAfLfBV7VTTma6/u3q9qqpd+nHy8+NkaprfWeMRHkBACDAtGsdqtceTNZTN/eR1WLovW3HNCYrT3tOVJgdrVEoLwAABCCLxdC063tpyaOD1THSpgMnqzU2K19LNh6Wtx9RQnkBACCAJXePUs6MdA3v0161DS7Nem+nfrC0QFVePEaivAAAEOCiwkP0ysRk/WRUX1kthj4oKNaY+XkqLPbOMRLlBQAAyGIxNGV4Ty19dLBi7TZ9WVatcc/na/GGc2Mkp8utdQdO6YOCY1p34JScLvNGS5wqDQAAvuGr6jo9uWy7/ranVJI0qFtbHfnqjEoqas+vibXbNDsjXqMSYj2yTU6VBgAAl61teIhefmCQfjq6ryyGtPnQV98oLpJ0wlGjKW9u1Ue7jrd4PsoLAAD4FovF0MNpV6ltWMgFf/712GbOysIWHyFRXgAAwAVtLCrXqeq6i/7cLem4o0Ybi8pbLpQoLwAA4CJKK2s8us5TKC8AAOCCOkTYPLrOUygvAADgglJ6RCnWbtPFbtlo6NxZRyk9oloyFuUFAABcmNViaHZGvCR9q8B8/f3sjHhZLS17R2rKCwAAuKhRCbFacN8AdbR/czTU0W7TgvsGeOw6L00R1OJbBAAAPmVUQqxujO+ojUXlKq2sUYeIc6Oilt7j8jXKCwAA+E5Wi6EhPduZHUMSYyMAAOBjKC8AAMCnUF4AAIBPobwAAACf4jflJTs7W/Hx8UpOTjY7CgAAaEaG2+1u2VtBNrOKigrZ7XY5HA5FRkaaHQcAADRCUz6//WbPCwAACAyUFwAA4FMoLwAAwKf43RV2vz6Ep6KiwuQkAACgsb7+3G7Mobh+V14qKyslSXFxcSYnAQAATVVZWSm73X7JNX53tpHL5VJxcbEiIiJkGObcMOpKJCcna9OmTWbHaDQz8zb3tj39+p54vct9jct5XmOfU1FRobi4OB05coQz/BqB97h3bd/b3udX8vymPrcp61vife52u1VZWalOnTrJYrn0US1+t+fFYrGoS5cuZse4bFar1ac+AMzM29zb9vTre+L1Lvc1Lud5TX1OZGSkT/3dNQvvce/avre9z6/k+U197uVsq7nf59+1x+VrHLDrZaZNm2Z2hCYxM29zb9vTr++J17vc17ic5/na30Vf4Wv/Xs3OG2jv8yt5flOfa/Z/2yvhd2MjAC2LC0MC/s/b3ufseQFwRUJDQzV79myFhoaaHQVAM/G29zl7XgAAgE9hzwsAAPAplBcAAOBTKC8AAMCnUF4AAIBPobwAAACfQnkB0GyOHDmi4cOHKz4+Xv3799eyZcvMjgTAg06fPq1BgwYpKSlJCQkJWrhwYYtsl1OlATSb48ePq6SkRElJSTpx4oQGDhyovXv3Kjw83OxoADzA6XSqtrZWYWFhqq6uVkJCgjZv3qx27do163b97t5GALxHbGysYmNjJUkdO3ZUdHS0ysvLKS+An7BarQoLC5Mk1dbWyu12qyX2iTA2AnBRn332mTIyMtSpUycZhqEVK1Z8a012dra6d+8um82m1NRUbdy48YKvtWXLFjmdTsXFxTVzagCN5Yn3+OnTp5WYmKguXbroqaeeUnR0dLPnprwAuKjq6molJiYqOzv7gj9funSpZs6cqdmzZ2vr1q1KTEzUzTffrNLS0m+sKy8v1wMPPKCXXnqpJWIDaCRPvMfbtGmj7du3q6ioSIsXL1ZJSUmz5+aYFwCNYhiG3n//fY0bN+78Y6mpqUpOTlZWVpYkyeVyKS4uTtOnT9esWbMknduVfOONN2ry5Mm6//77zYgOoBEu9z3+z6ZOnaoRI0ZowoQJzZqVPS8ALktdXZ22bNmikSNHnn/MYrFo5MiRWrdunSTJ7XbrwQcf1IgRIygugI9pzHu8pKRElZWVkiSHw6HPPvtMffr0afZslBcAl6WsrExOp1MxMTHfeDwmJkYnTpyQJOXn52vp0qVasWKFkpKSlJSUpJ07d5oRF0ATNeY9fujQIaWnpysxMVHp6emaPn26+vXr1+zZONsIQLNJS0uTy+UyOwaAZpKSkqKCgoIW3y57XgBclujoaFmt1m8dnFdSUqKOHTualAqAp3jze5zyAuCyhISEaODAgVqzZs35x1wul9asWaMhQ4aYmAyAJ3jze5yxEYCLqqqq0v79+89/X1RUpIKCAkVFRalr166aOXOmJk6cqEGDBiklJUV/+MMfVF1drUmTJpmYGkBj+ex73A0AF/HJJ5+4JX3ra+LEiefXzJ8/3921a1d3SEiIOyUlxb1+/XrzAgNoEl99j3OdFwAA4FM45gUAAPgUygsAAPAplBcAAOBTKC8AAMCnUF4AAIBPobwAAACfQnkBAAA+hfICAAB8CuUFAAD4FMoLAADwKZQXAADgUygvAADAp/wft/kDWIMWqE8AAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.loglog(num_points_range, error_range_nonsmooth_mse, \"-o\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "exponax_dev", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.14" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}