yadunut/nus
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
75272fb86e
nus/cs2109s/labs/ps6/ps6.ipynb

1965 lines
328 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"cells": [
{
"cell_type": "markdown",
"id": "9d313ffdcee5542c",
"metadata": {
"collapsed": false
},
"source": [
"### Problem Set 6: Introduction to PyTorch\n",
"\n",
"**Release Date:** 19 March 2024\n",
"\n",
"**Due Date:** 23:59, 06 April 2024\n",
"\n",
"In the real world, while fundamentals are welcomed and appreciated, implementing algorithms from scratch is time consuming, especially when it comes to Deep Learning (DL) models like neural networks with many layers. Backpropagating manually or by hand is often tedious and erroneous. Which is why, it is absolutely critical to learn **at least one** Machine Learning library, either to get jobs or build projects in this field. As such, in *Problem Set 6*, we will introduce you to **PyTorch**.\n",
"\n",
"<img src=\"imgs/img_logo.png\" width=\"600\">\n",
"\n",
"`PyTorch` is one of the largest DL libaries widely used around the globe. It offers a very Pythonic API to build layers and compose them together. In fact, data processing is also made easy using the multitude of tools and wrappers that are at your disposal – it is the complete workbench. Of course, there are other popular libraries such as `TensorFlow`, but they require you to understand how to understand \"computation graphs,\" and thus we feel are less accessible for beginners. Hence, we decided to use PyTorch for CS2109S. \n",
"\n",
"In *Problem Set 6*, we will attempt to help you learn the `PyTorch` API by having you build a simple deep neural network and training it locally on your system via backpropagation and stochastic gradient descent. Subsequently, you will also learn how to build data processing pipelines to prepare your data before ingestion into your model(s)."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "afedfcd3",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T04:57:17.202366Z",
"start_time": "2024-04-02T04:57:16.227482Z"
}
},
"outputs": [],
"source": [
"# RUN THIS CELL FIRST\n",
"import math\n",
"from collections import OrderedDict\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import torch\n",
"import torch.nn as nn\n",
"import numpy as np\n",
"from numpy import allclose, isclose"
]
},
{
"cell_type": "markdown",
"id": "a5482ebb",
"metadata": {},
"source": [
"# 1 Tensors in PyTorch\n",
"\n",
"### 1.1 Concept - What are Tensors?\n",
"\n",
"In Linear Algebra, you've learned about vectors – they are 1-dimensional (1D) serial arrays (like `[1, 2, 3, 4, 23, 18]`) containing a column (or row) of information. You've also learned about matrices – they are \"rectangles\" (i.e., 2D) that also capture elements.\n",
"\n",
"**Tensors** generalise the concept of matrices: they are $n$-dimensional arrays that contain or represent information. In *PyTorch*, everything is defined as a `tensor`. It's analogous to `np.array(...)` from *NumPy*. A `tensor` object in *PyTorch* looks like this:\n",
"\n",
"<img src=\"imgs/img_tensors.png\" width=\"600\">\n",
"\n",
"\n",
"---\n",
"The following are some mappings of useful functions between Numpy and Pytorch, in fact, they are so similar that there is a function `torch.from_numpy(ndarray)` which transforms a numpy array into a pytorch tensor! The main difference in the functions in the table below is that Numpy and Pytorch functions takes as input and gives as output numpy array or torch tensors respectively. PyTorch tensors also have additional functionality for GPU acceleration. Refer to this [website](https://pytorch-for-numpy-users.wkentaro.com/) for more information.\n",
"\n",
"<img src=\"imgs/img_numpy_pytorch.png\" width=\"600\">"
]
},
{
"cell_type": "markdown",
"id": "bcc457e7",
"metadata": {},
"source": [
"### 1.1.1 Demo - Tensor functions\n",
"\n",
"Notice that tensors have a `.grad` attribute. This is used for automatic gradient computation. \n",
"To create tensors, you can use the `torch.tensor(...)` constructor: \n",
"\n",
"A 0-dimensional tensor: `torch.tensor(5.0)` \n",
"A 1-dimensional tensor: `torch.tensor([1.0, 2.0, 3.0])` \n",
"A 2-dimensional tensor: `torch.tensor([[.4, .3], [.1, .2]])` \n",
"\n",
"If automatic gradient computation is required, then the equivalent constructors will be: \n",
"`torch.tensor(5.0, requires_grad=True)` \n",
"`torch.tensor([1.0, 2.0, 3.0], requires_grad=True)` \n",
"`torch.tensor([[.4, .3], [.1, .2]], requires_grad=True)` \n",
"\n",
"We can call detach() on these tensors to stop them from being traced for gradient computation, returning us the tensors without requires_grad=True.\n",
"\n",
"We can call item() on our tensors to return the value of our tensor as a standard python number:\n",
"\n",
"`>>> torch.tensor([1.0]).item()\n",
"1.0`\n",
"\n",
"The following code block shows how we can make use of all these functions introduced."
]
},
{
"cell_type": "code",
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Gradient of y with respect to x: tensor([6.])\n",
"Gradient of x after detachment: None\n",
"Value of x as a Python number: 2.0\n"
]
}
],
"source": [
"# Create a tensor with requires_grad set to True\n",
"x = torch.tensor([2.0], requires_grad=True)\n",
"\n",
"# Compute the gradient of a simple expression using backward\n",
"y = x**2 + 2 * x\n",
"y.backward()\n",
"\n",
"# Print the derivative value of y i.e dy/dx = 2x + 2 = 6.0.\n",
"print(\"Gradient of y with respect to x:\", x.grad)\n",
"\n",
"# Detach the gradient of x\n",
"x = x.detach()\n",
"\n",
"# Print the gradient of x after detachment\n",
"print(\"Gradient of x after detachment:\", x.grad)\n",
"\n",
"# Extract the scalar value of a tensor as a Python number\n",
"x_value = x.item()\n",
"print(\"Value of x as a Python number:\", x_value)"
],
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T01:51:48.246718Z",
"start_time": "2024-04-02T01:51:48.237176Z"
}
},
"id": "a937e9bf",
"execution_count": 2
},
{
"cell_type": "markdown",
"source": [
"### 1.1.2 Demo - Working with Tensors\n",
"\n",
"Here, we use `torch.linspace` to create a `torch.tensor`. In PyTorch, and Machine Learning in general, tensors form the basis of all operations.\n",
"\n",
"We then make use of the built-in *PyTorch* function `torch.sin` to create the corresponding y-values of a sine function, and plot the points using *Matplotlib*."
],
"metadata": {},
"id": "3a9b1300"
},
{
"cell_type": "code",
"execution_count": 3,
"id": "be36fbc0",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T01:51:50.250837Z",
"start_time": "2024-04-02T01:51:50.149722Z"
}
},
"outputs": [
{
"data": {
"text/plain": "<Figure size 640x480 with 1 Axes>",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGxCAYAAABfrt1aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABD3klEQVR4nO3dd3RUZeLG8WcyyaQXQkhCSKEXqaFXEVQUKyggFgQFFQVcF3dVZFewoq69gKgUK2BD7MpKFULvIJ0QSiChJSGkztzfH5HsLwIxgYR3knw/58w55s6de59cSebJO++912ZZliUAAAADPEwHAAAAVRdFBAAAGEMRAQAAxlBEAACAMRQRAABgDEUEAAAYQxEBAADGUEQAAIAxFBEAAGAMRQQoxrJly9S/f3/VrFlTDodDkZGR6tevnxISEkq1nfHjx8tms51XhgULFshms2nBggXn9fqSuuyyy3TZZZf95Xq5ubkaPny4atasKbvdrlatWpVrrr8yceJETZ8+/YzliYmJstlsZ32uvP3www8aP358uWz7zTffVP369eVwOGSz2XTixAkNGTJEtWvXLrLec889p6+//rpcMgBlygJwVm+88Ybl4eFhdezY0frwww+thQsXWh999JHVsWNHy8PDw3rzzTdLvK19+/ZZCQkJ55UjLS3NSkhIsNLS0s7r9SXVvXt3q3v37n+53muvvWZJst58801r6dKl1oYNG8o1119p2rTpWXNnZ2dbCQkJVkpKykXPNGLECKs8fr2uXbvWkmQNGzbMWrx4sZWQkGDl5+dbO3futNasWVNkXX9/f2vw4MFlngEoa56GexDglpYsWaKHHnpI11xzjWbPni1Pz//9qAwcOFB9+/bV3/72N8XHx6tLly7n3M6pU6fk5+en6OhoRUdHn1eWoKAgdezY8bxeWx42bdokX19fjRw50nSUYnl7e7vVcSsLmzdvliTdc889at++feHyevXqmYoEXDjTTQhwR9dee61lt9utffv2nfX5pKQky263W9ddd13hsnHjxlmSrNWrV1s333yzFRISYkVGRhZ57v/Lzs62Ro8ebUVERFi+vr5Wt27drFWrVllxcXFF/pKdP3++JcmaP39+4bLBgwdb/v7+1o4dO6zevXtb/v7+VnR0tDV69GgrOzu7yH7Gjx9vtW/f3qpWrZoVGBhoxcfHW++//77lcrmKrFeSERFJZzymTZtm7dmzp/C/z/aacePGnXGcNm3aZA0cONAKCgqywsPDrbvuuss6ceJEkdc6nU7rjTfesFq2bGn5+PhYwcHBVocOHaw5c+ZYlmVZcXFxZ+SJi4uzLMs6Z6bFixdbPXv2tAICAixfX1+rU6dO1nfffVdknWnTplmSrHnz5lnDhw+3qlevboWGhlp9+/a1Dhw4UOwxGjx48FmP0549eyzLsqysrCzrscces2rXrm15eXlZUVFR1gMPPGAdP3682O127979jG2e/ncyePDgwu/79DH/86Mko12ACYyIAH/idDo1f/58tW3b9pyjGDExMWrTpo3mzZsnp9Mpu91e+NxNN92kgQMHavjw4crMzDznfu666y7NmjVLjzzyiHr27KktW7aob9++Sk9PL1HOvLw83XDDDRo6dKgefvhhLVq0SE8//bSCg4P1xBNPFK6XmJio++67T7GxsZIK5r2MGjVKBw4cKLJeSSQkJOjpp5/W/PnzNW/ePEkFf40X932ey80336xbbrlFQ4cO1caNGzVmzBhJ0tSpUwvXGTJkiD7++GMNHTpUTz31lBwOh9asWaPExERJ0uzZs9WvXz8FBwdr4sSJkgpGQs5l4cKFuvLKK9WiRQtNmTJF3t7emjhxoq6//nrNmDFDt9xyS5H1hw0bpmuvvVaffvqp9u3bp3/+85+64447Cr/3s/n3v/+tzMxMffHFF0XmEtWsWVOWZalPnz769ddfNWbMGHXr1k0bNmzQuHHjlJCQoISEhHPmnzhxombMmKFnnnlG06ZNU+PGjVWjRo2zrpuQkKCePXuqR48e+ve//y2pYGQNcEummxDgbg4dOmRJsgYOHFjserfccoslyTp8+LBlWf/7S/+JJ544Y90/j4hs3rzZkmQ9+uijRdabMWNGkb90LevcIyKSrM8++6zI66+55hqrUaNG58zsdDqtvLw866mnnrKqV69eZFSkpHNETo/G/H/nMyLy4osvFlnvgQcesHx8fAozLVq0yJJkjR07ttg855ojcrZMHTt2tMLDw62MjIzCZfn5+VazZs2s6Ojown2fHhF54IEHimzzxRdftCRZycnJxWY61xyRn3766azf+6xZsyxJ1rvvvlvsdk/nWrlyZZHlfx4RsSzmiKDi4KwZ4DxZliVJZ5wNc/PNN//laxcuXChJGjBgQJHl/fr1KzIfpTg2m03XX399kWUtWrTQ3r17iyybN2+errjiCgUHB8tut8vLy0tPPPGEjh49qpSUlBLtqzzccMMNRb5u0aKFsrOzCzP9+OOPkqQRI0aUyf4yMzO1fPly9evXTwEBAYXL7Xa7Bg0apP3792vbtm1/mVHSGce4pE6PpAwZMqTI8v79+8vf31+//vrreW0XqMgoIsCfhIWFyc/PT3v27Cl2vcTERPn5+Sk0NLTI8po1a/7lPo4ePSpJioiIKLLc09NT1atXL1FOPz8/+fj4FFnm7e2t7Ozswq9XrFihXr16SZLee+89LVmyRCtXrtTYsWMlSVlZWSXaV3n48/d5+iOJ05lSU1Nlt9sVGRlZJvs7fvy4LMs66/+fqKgoSf/7/1LSjKV19OhReXp6nvGRis1mU2Rk5Bn7B6oCigjwJ3a7XT169NCqVau0f//+s66zf/9+rV69Wj179iwyP0Q6c4TkbE6/wR0+fLjI8vz8/DJ9M5o5c6a8vLz03XffacCAAercubPatm1bZts/7XQhysnJKbL8Qr6XGjVqyOl06tChQxeU7bRq1arJw8NDycnJZzx38OBBSQUltDxVr15d+fn5Sk1NLbLcsiwdOnSo3PcPuCOKCHAWY8aMkWVZeuCBB+R0Oos853Q6df/998uyrMIJlqV16aWXSpJmzZpVZPkXX3yh/Pz88wt9FjabTZ6enkXKUlZWlj766KMy24dUMLLj4+OjDRs2FFk+Z86c895m7969JUmTJk0qdj1vb+8SjVD4+/urQ4cO+uqrr4qs73K59PHHHys6OloNGzY877x/ziSdOXJy+eWXS5I+/vjjIsu//PJLZWZmFj5fVhlMjngBJcVZM8BZdOnSRa+99poeeughde3aVSNHjlRsbKySkpL09ttva/ny5XrttdfUuXPn89p+06ZNdeutt+rll1+W3W5Xz549tXnzZr388ssKDg6Wh0fZ/I1w7bXX6pVXXtFtt92me++9V0ePHtVLL71U7Jkl58Nms+mOO+7Q1KlTVa9ePbVs2VIrVqzQp59+et7b7NatmwYNGqRnnnlGhw8f1nXXXSdvb2+tXbtWfn5+GjVqlCSpefPmmjlzpmbNmqW6devKx8dHzZs3P+s2J0yYoCuvvFI9evTQP/7xDzkcDk2cOFGbNm3SjBkzzvvqt392ev8vvPCCevfuLbvdrhYtWujKK6/UVVddpUcffVTp6enq0qVL4Vkz8fHxGjRoUJns/3SGBQsW6Ntvv1XNmjUVGBioRo0aldn2gbJCEQHOYdSoUWrXrp1efvllPfzwwzp69KhCQ0PVtWtX/fbbb+rUqdMFbX/atGmqWbOmpkyZoldffVWtWrXSZ599pquvvlohISFl8j307NlTU6dO1QsvvKDrr79etWrV0j333KPw8HANHTq0TPZx2ssvvyxJevHFF3Xy5En17NlT33333RmXHi+N6dOnq3Xr1poyZYqmT58uX19fXXLJJXr88ccL13nyySeVnJyse+65RxkZGYqLiys8vffPunfvrnnz5mncuHEaMmSIXC6XWrZsqW+++UbXXXfdeef8s9tuu01LlizRxIkT9dRTT8myLO3Zs0e1a9fW119/rfHjx2vatGl69tlnFRYWpkGDBum5554r04L4+uuva8SIERo4cKBOnTql7t27l/ttAoDzYbNOT/0HYNzSpUvVpUsXffLJJ7rttttMxwGAckcRAQyZO3euEhIS1KZNG/n6+mr9+vV6/vnnFRwcrA0bNpxxRgwAVEZ8NAMYEhQUpF9++UWvvfaaMjIyFBYWpt69e2vChAmUEABVBiMiAADAGE7fBQAAxlBEAACAMRQRAABgjFtPVnW5XDp48KACAwPL7EJDAACgfFmWpYyMDEVFRf3lBRrduogcPHhQMTExpmMAAIDzsG/fPkVHRxe7jlsXkcDAQEkF30hQUJDhNAAAoCTS09MVExNT+D5eHLcuIqc/jgkKCqKIAABQwZRkWgWTVQEAgDEUEQAAYAxFBAAAGOPWc0QAALgQlmUpPz9fTqfTdJRKx8vLS3a7/YK3QxEBAFRKubm5Sk5O1qlTp0xHqZRsNpuio6MVEBBwQduhiAAAKh2Xy6U9e/bIbrcrKipKDoeDC2OWIcuylJqaqv3796tBgwYXNDJCEQEAVDq5ublyuVyKiYmRn5+f6TiVUo0aNZSYmKi8vLwLKiJMVgUAVFp/dXlxnL+yGmHi/xAAADCGIgIAAIyhiAAAUEEMGTJEffr0KfXr5s2bp8aNG8vlcpVo/Y0bNyo6OlqZmZml3ldpUUQAAKggXn/9dU2fPr3Ur3vkkUc0duzYEs+Zad68udq3b69XX3211PsqLYoIAAAVRHBwsEJCQkr1mqVLl2rHjh3q379/qV531113adKkSeV+MTiKCACgSrAsS6dy8408LMsqVdYvvvhCzZs3l6+vr6pXr64rrrhCmZmZZ3w0c9lll+nBBx/UI488otDQUEVGRmr8+PFFtjVz5kz16tVLPj4+hcfhiiuu0NVXX12Y68SJE4qNjdXYsWMLX3fVVVfp6NGjWrhw4fkd8BLiOiIAgCohK8+pS5742ci+tzx1lfwcJXvLTU5O1q233qoXX3xRffv2VUZGhhYvXnzOMvPBBx9o9OjRWr58uRISEjRkyBB16dJFV155pSRp0aJFuvXWWwvXt9ls+uCDD9S8eXO98cYb+tvf/qbhw4crIiKiSIlxOBxq2bKlFi9erJ49e57/N/8XKCIAALiR5ORk5efn66abblJcXJykgjkb59KiRQuNGzdOktSgQQO99dZb+vXXXwuLSGJioqKiooq8platWpo8ebIGDRqkw4cP69tvv9XatWvl5eV1xnqJiYll+N2diSICAKgSfL3s2vLUVcb2XVItW7bU5ZdfrubNm+uqq65Sr1691K9fP1WrVu2s67do0aLI1zVr1lRKSkrh11lZWYUfy/x//fv31+zZszVhwgRNmjRJDRs2PDO3r2+536uHIgIAqBJsNluJPx4xyW63a+7cuVq6dKl++eUXvfnmmxo7dqyWL19+1vX/PIphs9mKnKYbFham48ePn/G6U6dOafXq1bLb7dqxY8dZt33s2DHVq1fvAr6bv8ZkVQAA3IzNZlOXLl305JNPau3atXI4HJo9e/Z5bSs+Pl5btmw5Y/nDDz8sDw8P/fjjj3rjjTc0b968M9bZtGmT4uPjz2u/JUURAQDAjSxfvlzPPfecVq1apaSkJH311VdKTU1VkyZNzmt7V111lX777bciy77//ntNnTpVn3zyia688ko99thjGjx4cJGRk8TERB04cEBXXHHFBX0/f4UiAgCAGwkKCtKiRYt0zTXXqGHDhvrXv/6ll19+Wb179z6v7d1xxx3asmWLtm3bJklKTU3V0KFDNX78eLVu3VqSNG7cOEVFRWn48OGFr5sxY4Z69epVOGG2vNis0p7cfBGlp6crODhYaWlpCgoKMh0HAFBBZGdna8+ePapTp85ZJ2pWNY888ojS0tI0efLkEq2fk5OjBg0aaMaMGerSpctZ1ynuGJfm/ZsREQAAKrmxY8cqLi6uxFdJ3bt3r8aOHXvOElKW3H/6MAAAuCDBwcF6/PHHS7x+w4YNz3o6b3lgRAQAABhDEQEAAMZQRAAAlZYbn49R4ZXVsaWIAAAqndNXGy3vy5NXZbm5uZIKrgR7IZisCgCodOx2u0JCQgrvueLn5yebzWY4VeXhcrmUmpoqPz8/eXpeWJWgiAAAKqXIyEhJKnIDOJQdDw8PxcbGXnDBo4gAAColm82mmjVrKjw8XHl5eabjVDoOh0MeHhc+w4MiAgCo1Ox2+wXPY0D5YbIqAAAwhiICAACMKdciMmHCBLVr106BgYEKDw9Xnz59Cu/+BwAAUK5FZOHChRoxYoSWLVumuXPnKj8/X7169VJmZmZ57hYAAFQQNusiXnYuNTVV4eHhWrhwoS699NK/XL80txEGAADuoTTv3xf1rJm0tDRJUmho6Fmfz8nJUU5OTuHX6enpFyUXAAAw46JNVrUsS6NHj1bXrl3VrFmzs64zYcIEBQcHFz5iYmIuVjwAAGDARftoZsSIEfr+++/122+/KTo6+qzrnG1EJCYmho9mAACoQNzuo5lRo0bpm2++0aJFi85ZQiTJ29tb3t7eFyMSAABwA+VaRCzL0qhRozR79mwtWLBAderUKc/dAQCACqZci8iIESP06aefas6cOQoMDNShQ4ckScHBwfL19S3PXQMAgAqgXOeInOuOfNOmTdOQIUP+8vWcvgsAQMXjNnNELuIlSgAAQAXEvWYAAIAxFBEAAGAMRQQAABhDEQEAAMZQRAAAgDEUEQAAYAxFBAAAGEMRAQAAxlBEAACAMRQRAABgDEUEAAAYQxEBAADGUEQAAIAxFBEAAGAMRQQAABhDEQEAAMZQRAAAgDEUEQAAYAxFBAAAGEMRAQAAxlBEAACAMRQRAABgDEUEAAAYQxEBAADGUEQAAIAxFBEAAGAMRQQAABhDEQEAAMZQRAAAgDEUEQAAYAxFBAAAGEMRAQAAxlBEAACAMRQRAABgDEUEAAAYQxEBAADGUEQAAIAxFBEAAGAMRQQAABhDEQEAAMZQRAAAgDEUEQAAYAxFBAAAGEMRAQAAxlBEAACAMRQRAABgDEUEAAAYQxEBAADGUEQAAIAxFBEAAGAMRQQAABhDEQEAAMZQRAAAgDEUEQAAYAxFBAAAGEMRAQAAxlBEAACAMRQRAABgDEUEAAAYQxEBAADGUEQAAIAx5VpEFi1apOuvv15RUVGy2Wz6+uuvy3N3AACgginXIpKZmamWLVvqrbfeKs/dAACACsqzPDfeu3dv9e7duzx3AQAoZ1m5TqVl5SkjO0/p2XlKz86Xy2XJsiRLkmVZ8nXYFeDtWfDw8VSov0PennbT0VEBlGsRKa2cnBzl5OQUfp2enm4wDQBUHWmn8rT1ULq2H87QzpSTOnAiSwdOZOvgiSylZeWd1zbDAhyKCPJRzWAfxVX3V4PwADWICFT98AAF+3qV8XeAisqtisiECRP05JNPmo4BAJVaTr5Tmw6ka/XeY1qVeFzr95/Q4fScYl9j97ApyMdTQb5eCvD2lKeHTbLZ5GEreD4r16mTOfkFj+x85bssHTmZqyMnc7X54Jl/VEZX81Xr2GqKjw1RfGw1NY0Kkped8yeqIptlWdZF2ZHNptmzZ6tPnz7nXOdsIyIxMTFKS0tTUFDQRUgJAJWPZVnakXJSC7alaOH2VK1MPK7cfNcZ69UK8VWjyEA1iAhQTDU/1armq1ohvooM9lGgt6dsNluJ93f8VJ6S07J0OD1bB05ka09qpnakFIy2JKdln/Eaf4ddnepVV7cGNXRpwxqqXd2vxPuD+0lPT1dwcHCJ3r/dakTE29tb3t7epmMAQIXncllatfe4vt9wUHO3HNbBP735V/d3qHVcNbWNq6bWcdXUODJQgT5l83GJzWZTqL9Dof4ONY0KPuP59Ow8bdiXprVJx7V23wmtSTquE6fy9N/fU/Tf31MkSXHV/dS7WU1d27ymmtUKopRUYm5VRAAA58+yLG3Yn6av1x3QDxuTi3zc4u3poY51q6t7w4IRh3o1/I29uQf5eKlrgzB1bRAmqaA0bUlO1+IdR7Roe6pW7T2mvUdP6Z2Fu/TOwl2Kruar61pEqV+baNUPDzCSGeWnXD+aOXnypHbu3ClJio+P1yuvvKIePXooNDRUsbGxf/n60gztAEBVlXYqT1+vO6CZK/fp9+T/zccI9PFUr0sidW2LSHWuFyYfr4pxFktmTr7mb0vRjxsPad7WFGXlOQufax0bov5tY3Rdi5plNoKDslea9+9yLSILFixQjx49zlg+ePBgTZ8+/S9fTxEBgHPbdCBNU5fs0fcbkpXzx5wPh6eHrm4aqRtaRqlbw7AKfwptVq5T87el6Ks1+zV/W6qcroK3LH+HXf3aRGtIlzqqE+ZvOCX+zG2KyIWiiABAUS6XpfnbUvTe4t1atvtY4fJGEYEa2D5GfeNrKcTPYTBh+UlJz9bstQc0a9U+7U7NlCTZbFKPRuG6u0sddalfnbkkboIiAgCVTJ7TpdlrD2jywl3a9cebsN3Dputa1NTgzrUVHxNSZd6ELcvSkp1HNXXJHs3bmlK4vFmtII3q2UBXNomQh0fVOBbuiiICAJVE/h8F5M15O5V07JQkKdDbU7d2iNWQzrUVFeJrOKFZu1NP6oOlifps1f7CuSSNIwP14OUNdHXTSAqJIRQRAKjgnC5L36w/oNf/u0OJRwsKSHV/h+69tK5u6xDLRM0/OZaZqym/7dYHS/fqZE6+pIKPqx7t3Ug9GoVXmdEid0ERAYAK7LcdR/TsD78XngET+kcBubNTnPwcXHWhOCdO5WrqkkRNW7JHGdkFhaRDnVA9fk0TtYwJMRuuCqGIAEAFtDMlQ8/9sLVw3kOgj6eGd6+nIZ1ry9+bAlIaaafyNHHhTk1bklh4FdlrW9TUY1c3Vkyon+F0lR9FBAAqkLSsPL3yyzZ9vDxJTpclTw+b7ugYpwcvb6BQ/8p5BszFcuBEll75Zbu+WrtfllVwYbf7L6un4d3rVZjrqlREFBEAqAAsy9LX6w7o2e9/15GTuZKkKy+J0JjejVW3BlcQLUtbDqbrme+3aOmuo5Kk2FA/jbv+El3eJMJwssqJIgIAbm5nykn9++tNSthd8MZYt4a/nr6xmbrUDzOcrPKyLEvfbUjWs9//rkPpBffeuaJJuMZd35SPa8oYRQQA3FROvlNvz9upSQt3Kc9pydvTQ6N61tc9l9at8FdBrSgyc/L1xrwdmrJ4j/Jdlvwcdj1yVSPd2ak2p/uWEYoIALihDftP6J+fb9C2wxmSpJ6Nw/XkDfw1bsrOlAw9/tUmrUgsuEJtu9rV9MLNLfhYrAxQRADAjeTkO/XGrzv0zsLdcrosVfd36Ok+zdS7WSTXtzDM5bL0yfK9ev7HrcrMdcrh6aG/X9FQ93SrI0+7h+l4FRZFBADcxMb9aXr483XafvikJOm6FjX15A1NVT3A23Ay/H/7j5/S47M3adH2VEkFd/l99ZZWiqvODfXOB0UEAAxzuSxNXrRbL/+yTfkuS2EBDj19YzP1bl7TdDScg2VZ+nLNAT357WZlZOfL32HXuBuaqn+baEauSokiAgAGHUrL1ujP1hWeKtq7WaSe7duca4JUEPuPn9Loz9ZrxZ6CuSO9m0Xqub7NVY3/fyVGEQEAQ37efEiPfrlBJ07lydfLrvE3XKIBbWP4i7qCcbosTV60S6/8sl35LksRQd56ZUArTq8uIYoIAFxk2XlOPfXdFn26PElSwS3pXx8Yr3qcgVGhbdyfpr/NWqvdqZmy2aRRPRvob5c3kJ3TfItVmvdvpgQDwAXaezRTN01cWlhC7ru0rr66vwslpBJoHh2s70Z11a3tY2RZ0hu/7tCdU5crNSPHdLRKgxERALgAv2w+pIc/X6+M7HyF+jv02i2tdGnDGqZjoRx8vfaAHp+9UadynaoR6K03b41Xx7rVTcdyS4yIAEA5y3e69PyPW3XvR6uVkZ2v1rEh+v7BrpSQSqxPfC19M7KLGoQHKDUjR7e9t0xvz98pl8tt/56vECgiAFBKKRnZumPKcr2zcJck6a4utTXz3k6qGexrOBnKW/3wQM0Z2UU3xdeSy5L+8/M23ffxamVk55mOVmFRRACgFNbtO6Hr3vhNy3Yfk7/Drrdui9e465vK4cmv06rCz+Gplwe01PM3NZfD7qG5Ww6r78Sl2nMk03S0ComfHAAoodlr92vA5ASlZOSoQXiA5ozsqutaRJmOBQNsNpsGto/VrPs6KiLIWztTTuqGt37Tgm0ppqNVOBQRAPgLTpelCT/8rr/PWq/cfJeuaBKh2SO6qH44Z8VUdfGx1fTtyK5qHRuijOx83TV9pSYt2CU3Pg/E7VBEAKAY6dl5GvbBSk1etFuSNLJHfb07qI0CvD0NJ4O7CA/y0Yx7Oxae4vvCT1s1asZaZeU6TUerEPhJAoBz2HMkU8M+WKldqZny9vTQf/q31A0t+SgGZ/L2tOu5vs11SVSwnvxms77bkKx9x07pvcFtFR7oYzqeW2NEBADOYumuI+rz9hLtSs1UZJCPPh/eiRKCYtlsNg3qGKdPhnVQiJ+X1u9PU9+3l2rboQzT0dwaRQQA/uTL1fs1eOoKpWXlqVVMiL4Z2UUtokNMx0IF0aFudc1+oIvqhPnrwIks3TxpqRZuTzUdy21RRADgD5Zl6fX/7tDDn69XntPStS1qaua9HRUexNA6SqdOmL9mP9BZHeqE6mROvu6evlIfLdtrOpZboogAgKTcfJf++cUGvfrf7ZKk+7rX1ZsD4+XjZTecDBVViJ9DHw3toJtbR8vpsvTvrzfp6e+2yMmVWIugiACo8tKz83T39JX6YvV+edikZ/o005jeTeTBHVZxgRyeHnqpfwv986pGkqQpv+3RA5+sVnYeZ9ScRhEBUKUdOJGlfpOW6redR+TnsGvK4Ha6o2Oc6VioRGw2m0b0qK+3bouXw+6hnzcf1qApy5V2isvCSxQRAFXY78np6vv2Em0/fFLhgd767L5O6tE43HQsVFLXtYjSh0PbK9DHUysTj6v/5KU6eCLLdCzjKCIAqqQVe44VXq69YUSAZo/ooma1gk3HQiXXsW51fT68kyKDfLT98EndNHGpth+u2qf3UkQAVDlztxQMjWdk56td7Wr6/L7OqhXCnXNxcTSODNKXD3RW/fAAHUrPVr9JS7VizzHTsYyhiACoUj5btU/DP16tnHyXrmgSro+GdlCwn5fpWKhiaoX46ovhndQmrprSs/N1x5Tl+mnTIdOxjKCIAKgy3lm4S498sUFOl6V+baL1zh1tOD0XxoT4OfTJsA66okmEcvNdeuCT1ZqxIsl0rIuOIgKg0nO5LD37/RY9/+NWSQXXCPlPvxbytPMrEGb5eNn1zh2tdWv7GLksacxXG/Xuol2mY11U3PQOQKWW53Tp0S836Ks1ByRJY69ponsurWs4FfA/nnYPPde3uYJ9HXpn4S4998NWpWfl6+FeDWWzVf5r2VBEAFRaOflOjfx0reZuOSy7h00v3txCN7eJNh0LOIPNZtNjvRsryNdTL/60TW/N36n07DyNv75ppb+wHuOSACqlrFynhn2wSnO3HJbD00PvDmpDCYHbe+Cy+nq6TzPZbNKHCXv18Ofrle90mY5VrigiACqdjOw8DZ66Qot3FFwtdfqQdrq8SYTpWECJDOoYp1cHtJLdw6bZaw/o/k/WVOpLwlNEAFQqJ07l6o73l2tF4jEF+njqo6Ed1Ll+mOlYQKn0ia+lyXe0kcPTQ3O3HNbd01fqZE6+6VjlgiICoNJIzcjRwHeXaf3+NIX6OzTjno5qE1fNdCzgvFxxSYQ+uKu9/B12Ld11VHe8XznvT0MRAVApHDyRpVsmJ2jroQyFB3pr1r0duWQ7KrxO9arr03s6KsTPS+v2ndBt7y/T8cxc07HKFEUEQIW392im+r+ToN1HMlUrxFefD++kBhGBpmMBZaJlTIhm3ttR1f0d2nwwXbe+t0xHTuaYjlVmKCIAKrSdKRkaMDlBB05kqU6Yvz4f3klx1f1NxwLKVOPIIM26r6PCA7219VCGbpmcoMPp2aZjlQmKCIAKa8vBdA2YvEyH03PUKCJQs+7rqChuXodKqn54oGbd10k1g320KzVTt0xO0METWaZjXTCKCIAKadOBNN32/jIdy8xVi+hgzby3o8IDfUzHAspVnTB/fXZfJ0VX81Xi0VMaMDlB+46dMh3rglBEAFQ4G/af0G3vLdOJU3mKjw3Rx8M6qJq/w3Qs4KKICfXTrPs6Ka66n/YfL5iknXgk03Ss80YRAVChrNt3Qre/v1zp2flqE1dNH97dXkE+XqZjARdVrRBffXZfJ9Wr4a+DadkaMDlBO1NOmo51XigiACqMNUnHNej95crIzle72tX0wd3tFUgJQRUVEeSjmfd2UqOIQKVk5GjguwnadijDdKxSo4gAqBBW7z2mO6esUEZOvtrXCdX0u9orwJv7dqJqqxHorRn3dtQlNYN05GSuBr6boE0H0kzHKhWKCAC3tzKxoISczMlXp7rVNf2udvKnhACSpFB/hz69p4NaRgfr+Kk83f7+8gpVRigiANza8t1HNXjqCmXmOtWlfnVNHdJOfg5KCPD/hfg59NGwDoqPDVFaVsUqIxQRAG4rYddRDZm2UqdynerWIExTBreTr8NuOhbgloJ8vPTh3e2LlJGN+92/jFBEALilJTuP6K7pK5SV51T3hjX03p1t5eNFCQGKE/hHGWldWEaWacP+E6ZjFYsiAsDtLNqeqrunr1R2nks9GtXQ5EFtKCFACQX6eOmDu9urTVw1pWfn6473l7t1GaGIAHArC7alaNiHq5ST79IVTcL1DiUEKLXTZaTtH2Xk9veXa/2+E6ZjnRVFBIDbmL81Rfd+uFq5+S5deUmEJt7eRt6elBDgfAR4e2r63e3VrnY1ZWTn644py7XODcsIRQSAW/jvlsO696NVynW6dFXTCL19W2s5PPkVBVyIAG9PTburvdrXDlVGdr4Gvb9ca5OOm45VBD/lAIybu+Ww7v9ktfKclq5pHqm3KCFAmSkoI+3Uvk6oMnLydeeUFVrjRmXkovykT5w4UXXq1JGPj4/atGmjxYsXX4zdAqgAftl8SA/8UUKubVFTrw+Ml5edEgKUJX9vT00b4p5lpNx/2mfNmqWHHnpIY8eO1dq1a9WtWzf17t1bSUlJ5b1rAG7ul82HNOLTNcpzWrquRU29fksrSghQTvy9PTX9rnbqWDdUJ/8oI6v3mi8jNsuyrPLcQYcOHdS6dWtNmjSpcFmTJk3Up08fTZgwoci6OTk5ysnJKfw6PT1dMTExSktLU1BQUJllysl36sEZa3VLuxj1bBxRZtsFUHI/bz6kEZ+sUb7L0vUto/TqgJbypIQA5e5Ubr6GTl+lhN1H5e+w68Oh7dUmLrRM95Genq7g4OASvX+X6099bm6uVq9erV69ehVZ3qtXLy1duvSM9SdMmKDg4ODCR0xMTLnkmr4kUT9vPqz7Plqt/245XC77AHBuP236Xwm5gRICXFR+Dk9NHdJOnepWV2auUw/OWKfcfJexPOX6k3/kyBE5nU5FRBQddYiIiNChQ4fOWH/MmDFKS0srfOzbt69cct3dtY6uaR6pPKel+z9ZrbmUEeCi+WlTskZ+WlBCbmwVpVcoIcBF5+uwa+qQdrqmeaQmD2pjdHL4RdmzzWYr8rVlWWcskyRvb28FBQUVeZQHL7uHXh8Yr2tb1FSe09IDn6zWz5vPLEYAytaPG5M18tO1hSXk5f6UEMAUX4ddE29vo2a1go3mKNffAGFhYbLb7WeMfqSkpJwxSnKxedk99PotrXR9yyjlOS2N+GSNftqUbDQTUJn9uDFZI2cUlJA+raL0yoBWlBAA5VtEHA6H2rRpo7lz5xZZPnfuXHXu3Lk8d10innYPvTqgpW5sFaV8l6URn67VDxspI0BZ++GPEuJ0WeobX0svD2glu8eZo6IAqh7P8t7B6NGjNWjQILVt21adOnXSu+++q6SkJA0fPry8d10innYPvTKglTxsNs1ee0CjZqyVy7J0XYso09GASuGHjcka9UcJuSm+lv7TvyUlBEChci8it9xyi44ePaqnnnpKycnJatasmX744QfFxcWV965LzO5h00v9W8pmk75ac0B/m7lOLku6oSVlBLgQ329I1oMz/yghrWvpP/0oIQCKKvfriFyI0pyHXBacLkuPfrlBX6zeLw+b9OotrXRjq1rlvl+gMvpuw0H9beY6OV2Wbm4drRf7taCEAFWE21xHpKKxe9j04s0tNKBttFyW9PdZ6zR77X7TsYAK59v1/ysh/dpQQgCcG0XkTzw8bHr+phYa2C5GLksa/dl6fbmaMgKU1LfrD+qhWQUlpH+baL1wMyUEwLmV+xyRisjDw6bn+jaXzWbTjBVJ+scX62VJ6tcm2nQ0wK19s/6gHpq5Vi5LhSXEgxICoBiMiJyDh4dNz/Zppjs6xsqypH9+sV6frSyfK70ClcGcdQcoIQBKjSJSDA8Pm56+sZnu7BQny5Ie+XKDZq7grsHAn32xer/+PqvgbLMBbSkhAEqOIvIXbDabnryhqYZ0ri1Jeuyrjfp0OWUEOG3GiiT984v1clnSre1j9PxNlBAAJUcRKQGbzaZx11+iu7rUliQ9PnujPl6212wowA18mJCoMV9tlGVJgzvF6bm+zSkhAEqFyaolZLPZ9MR1l8jDZtOU3/boX19vkmVZGtSptulogBHvL96tZ77/XZJ0T7c6evyaJme9mSUAFIciUgo2m03/uraJ7B42vbtot/49Z7PyXZbu6lLHdDTgonp7/k795+dtkqQRPerpH70aUUIAnBeKSCnZbDaN6d1YNps0eeFuPfntFp3KdWpEj/qmowHlzrIsvfbfHXr91x2SpNFXNtSDlzcwnApARcYckfNgs9n02NWN9bc/fgH/5+dteunnbXLjq+UDF8yyLL3487bCEvLo1Y0pIQAuGEXkPNlsNv39yoYa07uxJOmt+Tv19He/U0ZQKVmWpWe+/12TFuySJP3r2ia6/7J6hlMBqAwoIhfovu719PSNTSVJU5fs0eOzN8rpooyg8nC5LI37ZrOm/LZHkvT0jU01rFtdw6kAVBbMESkDgzrVlo+XXY9+uUEzVuxTVq5TL/VvKU87PQ8VW77TpUe/3Kgv1+yXzSY9f1Nz3dIu1nQsAJUIRaSM9G8bIx8vu/4+a52+XndQ2XkuvXFrvByelBFUTDn5Tj04Y61+3nxYdg+bXurfQn3jud8SgLLFu2QZur5llN65o40cdg/9tPmQ7v1olbLznKZjAaV2Kjdfwz5YpZ83H5bD7qGJt7emhAAoFxSRMnbFJRGaOqSdfL3sWrAtVUOmrdDJnHzTsYASS8vK0x3vL9fiHUfk57Br2l3tdFXTSNOxAFRSFJFy0LVBmD4c2l4B3p5atvuYbn9vmY5l5pqOBfyl1IwcDXx3mdYknVCQj6c+HtZBXeqHmY4FoBKjiJSTdrVD9ek9HVTNz0vr96ep3ztLdeBElulYwDkdOJGlAZMT9HtyusICvDXrvk5qHVvNdCwAlRxFpBy1iA7R58M7KyrYR7tTM9Vv0lLtTMkwHQs4w67Uk+o/aan2HMlUrRBffTG8k5rUDDIdC0AVQBEpZ/XDA/TlA51VPzxAyWnZ6vdOgtYmHTcdCyi0ft8JDXgnQQfTslWvhr++uL+Taof5m44FoIqgiFwENYN99fl9ndQqJkQnTuXptveWa+H2VNOxAC3YlqJb31umo5m5al4rWJ/d10k1g31NxwJQhVBELpJq/g59MqyDLm1YQ1l5Tg2dvlJz1h0wHQtV2Fdr9mvYB6t0Ktepbg3CNOPejqoe4G06FoAqhiJyEfl7e+r9O9vqhpZRyndZemjWOk1fssd0LFQxlmXpnYW7NPqz9cp3WerTKkpTBrdTgDfXNwRw8VFELjKHp4deu6WVBneKk2VJ47/dogk//C4X96fBReByWXrquy16/setkqR7L62rVwa04grAAIzht48BHh42jb+hqf7Rq6EkafKi3Ro1cy1XYUW5ysl3atTMtZq2JFFSwR10H7+miTw8bGaDAajSKCKG2Gw2jezZQK/e0lJedpu+35CsO95fruNc+Azl4MSpXN05ZYW+35AsL7tNrw9sxR10AbgFiohhfeOj9cHd7RXo46lVe4/rpklLtfdopulYqEQSj2TqpolLtXzPMQV4e2rqkHa6sVUt07EAQBJFxC10rhemL+/vrFohvtpzJFN9Jy7VGq41gjKwMvGY+k5cot2nL1R2fyd1a1DDdCwAKEQRcRMNIwI1+4HOalYrSMcyc3Xru8v006Zk07FQgX299oBuf2+5jp/KU8voYM0e0VmNI7laKgD3QhFxI+FBPpp1byf1bByunHyXhn+8Rm/+ukOWxRk1KDnLsvTaf7froVnrlOt06eqmkZp5byeFB/qYjgYAZ6CIuBl/b0+9O6iNhnSuLUl6ee52jZqxVlm5nFGDv5ad59Toz9brtf/ukCTdd2ldTby9tXwddsPJAODsuIKRG/K0e2j8DU3VKDJQ//56k77bkKy9R0/p3TvbcPltnFNyWpbu+2i1NuxPk93Dpmf6NNOt7WNNxwKAYjEi4sZubR+rT4Z1UKi/QxsPpOn6N5cwiRVntSrxmK5/c4k27E9TNT8vfXR3e0oIgAqBIuLmOtStrjkjuqhxZKCOnMzRwMnL9OXq/aZjwY18snyvbn1vmY6czFHjyEB9M7KrOtcPMx0LAEqEIlIBxIT66cv7O+uqphHKdbr08OfrNW7OJuXmu0xHg0G5+S6Nnb1RY2dvUp7T0rXNa+qrBzorJtTPdDQAKDGKSAXh7+2pSbe30YOXN5AkfZCwVwMmJ+jgiSzDyWBCclqWbntvmT5ZniSbTfrnVY301m3x8nMw7QtAxUIRqUA8PGwafWVDTR3SVsG+Xlq374SufWOxFu9INR0NF9Gi7am69o3ftGrvcQV6e2rK4LYa0aO+bDbuGQOg4qGIVEA9G0fou1Fd1axWkI6fytOdU1fojV93cAffSs7psvTK3O0aPG2FjmXmqmlUkL57sKt6No4wHQ0AzhtFpIKKCfXTF8M769b2MbIs6ZW523X3Byt19GSO6WgoB6kZObpz6nK98esOWZZ0W4dYfXl/Z8VV9zcdDQAuiM1y48t2pqenKzg4WGlpaQoK4tLU5/L5qn3619eblJPvUo1Ab73cv6Uubcj9RCqLJTuP6O+z1iklI0e+XnY9d1Mz9Y2PNh0LAM6pNO/fFJFK4vfkdD04Y612pJyUJN17aV39o1cjOTwZ9KqocvKdeunnbXpv8R5JUoPwAE28vbUaRAQaTgYAxaOIVFFZuU49+8MWfbwsSZLUrFaQXh8Yr3o1AgwnQ2ltP5yhv81cp9+T0yVJt3eI1b+uvYRLtQOoECgiVdwvmw/pkS836MSpPPl62TX22ia6vUMsZ1VUAJZl6cOEvXruh9+Vk+9SqL9DL97cQldcwoRUABUHRQQ6lJat0Z+t09JdRyVJXepX1ws3t1B0NS525a72Hz+lMV9t1OIdRyRJ3RvW0H/6t+CuuQAqHIoIJEkul6XpSxP14s9blZ3nkr/DrsevbaLb2jM64k5cLkufLN+r53/cqsxcp7w9PTSmd2MN7lyb/08AKiSKCIrYcyRT//x8vVbtLbhhXtf6YZpwU3MuBe4GEo9k6pEvN2jFnmOSpHa1q+mFm1uoLvN6AFRgFBGcwfnH6Mh//hgd8fHy0KieDXRPt7qcWWNAbr5LU37bo9d/3a7sPJf8HHY9enVjDeoYJw8PRkEAVGwUEZzT7tSTGjt7kxJ2F8wdqVfDX8/0aa5O9aobTlZ1LNl5RE/M2aRdqZmSCubvPH9TC0aoAFQaFBEUy7Isfb3ugJ79/ncdOZkrSbopvpYeu6YxEyPL0aG0bD3z/RZ9tyFZkhQW4NBjvZvo5ta1mAsCoFKhiKBE0k7l6cWft+rTFUmyLMnPYdfw7vV0T7e6XK+iDGXnOTXltz2aOH+nMnOd8rBJgzrGaXSvRgr29TIdDwDKHEUEpbI26bie/HaL1u07IUmKDPLRP65qpJviazFf4QI4XZa+WrNfr8zdruS0bElS69gQPd2nmZpGBRtOBwDlhyKCUrMsS99uSNYLP27VgRNZkqSmUUH6x1WNdFnDGnx0UAqWZWnB9lS98ONWbT2UIUmqFeKrf1zVUDe2pNwBqPwoIjhv2XlOTV+aqLfn7VRGTr4kqVVMiB66ooG6U0iKdbqAvP7fHYWjS0E+nhrZs77u7FRbPl583AWgaqCI4IIdPZmjdxbu0kfL9io7zyVJio8N0YOXN2CE5E8sy9K8rSl649cdWr8/TZLk7emhOzvFaUSP+grxcxhOCAAXF0UEZSY1I0eT/ygkOfkFhaRBeICGdaujG1vVqtJ/5WfnOTVn3QFN/S1R2w4XfATj4+WhQR3jdM+ldTkDCUCVRRFBmUvJyNa7C3drxookZeY6JRWcfjqoY23d2j5G4UFV5003JT1bHy/bq4+XJ+lYZsHpz34Oe2EBCQvwNpwQAMyiiKDcpGfnaeaKJE1bklh4Jojdw6aejcM1sF2MujesIU975btSa77TpYXbUzVr5T7N25qifFfBj02tEF8N7hynW9rGKtiPU3EBQKKI4CLIc7r046ZD+mBpolb/cQ8bSYoI8lbf+Ghd16KmmkYFVei5JJZlafPBdH23IVlfrdmvlIycwufaxFXT3V3q6KqmEZWyeAHAhaCI4KLacThDs1bu01drDxR+VCFJcdX9dG3zmrq6WaSaRQVXiNNWXS5LGw+k6cdNh/TDxmQlHTtV+Fyov0M3xddS/7YxahQZaDAlALg3tykizz77rL7//nutW7dODodDJ06cKNXrKSIVS06+U7/+nqJv1x/UvK0phZNbJam6v0OXNqyh7g1rqEv9MNUIdJ95FKkZOVq8I1WLtqdq8Y4jOvr/ypSPl4d6NArXDS2jdHmTCG4QCAAl4DZFZNy4cQoJCdH+/fs1ZcoUikgVkpmTr3lbU/T9hmQt3pFaOMH1tNrV/dQ6tppax1VTq5gQ1Q8PuChn4GTnObUz5aTW7juhtXuPa03ScSUePVVkHX+HXd0b1dA1zWuqR6Nw+Xt7lnsuAKhM3KaInDZ9+nQ99NBDFJEqKjffpTVJx7Vwe6oWbkvV74fS9ed/dR42qXaYvxqGB6p+eICiQnwVFeKjqBBfRQT5KNDbs0Qf7bhcljKy83UoPVsH07J0KC1bB45naUdKhnYcPqnEo5lyneVffLNaQbq0QQ1d2rCGWsdWY+QDAC5Aad6/3epPvZycHOXk/G9CYHp6usE0KCsOTw91rFtdHetW16NXN1ZaVp7WJh3Xmr3HtWrvcW0+mK60rDztTs3U7tRMafPZtxPg7alAH0/5Oezy+GMSrM0muayCEZiM7Hyd/ONqsMUJ8vFUy5gQxcdWU+vYEMXHVOOMFwAwxK2KyIQJE/Tkk0+ajoFyFuzrpcsaheuyRuGSCs5OScnI0fbDGdp2KEN7jmQqOS1bB09kKTktW2lZeZKkkzklKxqSFOLnpcigghGVyGAf1asRoIYRAWoYEajwQO8KfTYPAFQmpS4i48eP/8uysHLlSrVt27bUYcaMGaPRo0cXfp2enq6YmJhSbwcVi81mU0SQjyKCfNStQY0zns/JdyojO/+PR54ycwrmm1gq+IzFJpsCvD0V4FMwYhLg7Vmlr/gKABVJqYvIyJEjNXDgwGLXqV279nmF8fb2lre3+5xNAffg7WmXd4CdK5YCQCVU6iISFhamsLCw8sgCAACqmHKdI5KUlKRjx44pKSlJTqdT69atkyTVr19fAQEB5blrAABQAZRrEXniiSf0wQcfFH4dHx8vSZo/f74uu+yy8tw1AACoALjEOwAAKFOlef/mqk0AAMAYiggAADCGIgIAAIyhiAAAAGMoIgAAwBiKCAAAMIYiAgAAjKGIAAAAYygiAADAGIoIAAAwhiICAACMoYgAAABjKCIAAMAYiggAADCGIgIAAIyhiAAAAGMoIgAAwBiKCAAAMIYiAgAAjKGIAAAAYygiAADAGIoIAAAwhiICAACMoYgAAABjKCIAAMAYiggAADCGIgIAAIyhiAAAAGMoIgAAwBiKCAAAMIYiAgAAjKGIAAAAYygiAADAGIoIAAAwhiICAACMoYgAAABjKCIAAMAYiggAADCGIgIAAIyhiAAAAGMoIgAAwBiKCAAAMIYiAgAAjKGIAAAAYygiAADAGIoIAAAwhiICAACMoYgAAABjKCIAAMAYiggAADCGIgIAAIyhiAAAAGMoIgAAwBiKCAAAMIYiAgAAjKGIAAAAYygiAADAGIoIAAAwhiICAACMoYgAAABjKCIAAMCYcisiiYmJGjp0qOrUqSNfX1/Vq1dP48aNU25ubnntEgAAVDCe5bXhrVu3yuVyafLkyapfv742bdqke+65R5mZmXrppZfKa7cAAKACsVmWZV2snf3nP//RpEmTtHv37hKtn56eruDgYKWlpSkoKKic0wEAgLJQmvfvchsROZu0tDSFhoae8/mcnBzl5OQUfp2enn4xYgEAAEMu2mTVXbt26c0339Tw4cPPuc6ECRMUHBxc+IiJiblY8QAAgAGlLiLjx4+XzWYr9rFq1aoirzl48KCuvvpq9e/fX8OGDTvntseMGaO0tLTCx759+0r/HQEAgAqj1HNEjhw5oiNHjhS7Tu3ateXj4yOpoIT06NFDHTp00PTp0+XhUfLuwxwRAAAqnnKdIxIWFqawsLASrXvgwAH16NFDbdq00bRp00pVQgAAQOVXbpNVDx48qMsuu0yxsbF66aWXlJqaWvhcZGRkee0WAABUIOVWRH755Rft3LlTO3fuVHR0dJHnLuIZwwAAwI2V22clQ4YMkWVZZ30AAABI3GsGAAAYRBEBAADGUEQAAIAxFBEAAGAMRQQAABhDEQEAAMZQRAAAgDEUEQAAYAxFBAAAGEMRAQAAxlBEAACAMRQRAABgDEUEAAAYQxEBAADGUEQAAIAxFBEAAGAMRQQAABhDEQEAAMZQRAAAgDEUEQAAYAxFBAAAGEMRAQAAxlBEAACAMRQRAABgDEUEAAAYQxEBAADGUEQAAIAxFBEAAGAMRQQAABhDEQEAAMZQRAAAgDEUEQAAYAxFBAAAGONpOkBxLMuSJKWnpxtOAgAASur0+/bp9/HiuHURycjIkCTFxMQYTgIAAEorIyNDwcHBxa5js0pSVwxxuVw6ePCgAgMDZbPZjGRIT09XTEyM9u3bp6CgICMZ3BnH59w4NsXj+BSP41M8jk/xTB8fy7KUkZGhqKgoeXgUPwvErUdEPDw8FB0dbTqGJCkoKIh/7MXg+Jwbx6Z4HJ/icXyKx/Epnsnj81cjIacxWRUAABhDEQEAAMZQRP6Ct7e3xo0bJ29vb9NR3BLH59w4NsXj+BSP41M8jk/xKtLxcevJqgAAoHJjRAQAABhDEQEAAMZQRAAAgDEUEQAAYAxFBAAAGEMRKYUbbrhBsbGx8vHxUc2aNTVo0CAdPHjQdCy3kJiYqKFDh6pOnTry9fVVvXr1NG7cOOXm5pqO5jaeffZZde7cWX5+fgoJCTEdx7iJEyeqTp068vHxUZs2bbR48WLTkdzCokWLdP311ysqKko2m01ff/216UhuY8KECWrXrp0CAwMVHh6uPn36aNu2baZjuY1JkyapRYsWhVdT7dSpk3788UfTsf4SRaQUevTooc8++0zbtm3Tl19+qV27dqlfv36mY7mFrVu3yuVyafLkydq8ebNeffVVvfPOO3r88cdNR3Mbubm56t+/v+6//37TUYybNWuWHnroIY0dO1Zr165Vt27d1Lt3byUlJZmOZlxmZqZatmypt956y3QUt7Nw4UKNGDFCy5Yt09y5c5Wfn69evXopMzPTdDS3EB0dreeff16rVq3SqlWr1LNnT914443avHmz6WjFs3De5syZY9lsNis3N9d0FLf04osvWnXq1DEdw+1MmzbNCg4ONh3DqPbt21vDhw8vsqxx48bWY489ZiiRe5JkzZ4923QMt5WSkmJJshYuXGg6ituqVq2a9f7775uOUSxGRM7TsWPH9Mknn6hz587y8vIyHcctpaWlKTQ01HQMuJnc3FytXr1avXr1KrK8V69eWrp0qaFUqIjS0tIkid8zZ+F0OjVz5kxlZmaqU6dOpuMUiyJSSo8++qj8/f1VvXp1JSUlac6cOaYjuaVdu3bpzTff1PDhw01HgZs5cuSInE6nIiIiiiyPiIjQoUOHDKVCRWNZlkaPHq2uXbuqWbNmpuO4jY0bNyogIEDe3t4aPny4Zs+erUsuucR0rGJV+SIyfvx42Wy2Yh+rVq0qXP+f//yn1q5dq19++UV2u1133nmnrEp8lfzSHh9JOnjwoK6++mr1799fw4YNM5T84jif44MCNputyNeWZZ2xDDiXkSNHasOGDZoxY4bpKG6lUaNGWrdunZYtW6b7779fgwcP1pYtW0zHKpan6QCmjRw5UgMHDix2ndq1axf+d1hYmMLCwtSwYUM1adJEMTExWrZsmdsPfZ2v0h6fgwcPqkePHurUqZPefffdck5nXmmPDwp+hux2+xmjHykpKWeMkgBnM2rUKH3zzTdatGiRoqOjTcdxKw6HQ/Xr15cktW3bVitXrtTrr7+uyZMnG052blW+iJwuFufj9EhITk5OWUZyK6U5PgcOHFCPHj3Upk0bTZs2TR4elX/A7UL+/VRVDodDbdq00dy5c9W3b9/C5XPnztWNN95oMBncnWVZGjVqlGbPnq0FCxaoTp06piO5Pcuy3P49qsoXkZJasWKFVqxYoa5du6patWravXu3nnjiCdWrV6/SjoaUxsGDB3XZZZcpNjZWL730klJTUwufi4yMNJjMfSQlJenYsWNKSkqS0+nUunXrJEn169dXQECA2XAX2ejRozVo0CC1bdu2cPQsKSmJOUWSTp48qZ07dxZ+vWfPHq1bt06hoaGKjY01mMy8ESNG6NNPP9WcOXMUGBhYOKoWHBwsX19fw+nMe/zxx9W7d2/FxMQoIyNDM2fO1IIFC/TTTz+ZjlY8k6fsVCQbNmywevToYYWGhlre3t5W7dq1reHDh1v79+83Hc0tTJs2zZJ01gcKDB48+KzHZ/78+aajGfH2229bcXFxlsPhsFq3bs0pmH+YP3/+Wf+dDB482HQ04871O2batGmmo7mFu+++u/BnqkaNGtbll19u/fLLL6Zj/SWbZVXimZYAAMCtVf4P8QEAgNuiiAAAAGMoIgAAwBiKCAAAMIYiAgAAjKGIAAAAYygiAADAGIoIAAAwhiICAACMoYgAAABjKCIAAMCY/wOwdaU1C/HBiwAAAABJRU5ErkJggg=="
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# This is a demonstration: You just need to run this cell without editing.\n",
"\n",
"x = torch.linspace(-math.pi, math.pi, 1000) # Task 1.1: What is torch.linspace?\n",
"y_true = torch.sin(x)\n",
"\n",
"plt.plot(x, y_true, linestyle='solid', label='sin(x)')\n",
"plt.axis('equal')\n",
"plt.title('Original function to fit')\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "23acad3b",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(True)"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Run this cell to explore what the FIRST 10 VALUES of x has been assigned to.\n",
"# By default, each cell will always print the output of the last expression in the cell\n",
"# You can explore what x is by modifying the expression e.g. x.max(), x.shape\n",
"x.shape\n",
"x.min() == -math.pi"
]
},
{
"cell_type": "markdown",
"id": "ece5d5e2",
"metadata": {},
"source": [
"### Task 1.1 - What is `torch.linspace`?\n",
"\n",
"From the example above, answer the following questions:\n",
"\n",
"1. What does `x = torch.linspace(-math.pi, math.pi, 1000)` do? \n",
"2. How many values are stored in `x`? \n",
"3. What are the minimum and maximum values in `x`? "
]
},
{
"cell_type": "markdown",
"id": "ba5928f9",
"metadata": {},
"source": [
"### 1.2.1 Demo - Using Tensors for linear regression\n",
"\n",
"For this example, we fit a **degree 3 polynomial** to the sine function, using a learning rate of 1e-6 and 5000 iterations."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "828fdba4",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-01T09:19:53.795438Z",
"start_time": "2024-04-01T09:19:53.435227Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"iter\tloss\t\n",
"----\t----\n",
"1000\t0.4506033957004547\n",
"2000\t0.4224050045013428\n",
"3000\t0.4060209095478058\n",
"4000\t0.396359920501709\n",
"5000\t0.39052513241767883\n"
]
},
{
"data": {
"text/plain": "<Figure size 640x480 with 1 Axes>",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGxCAYAAABfrt1aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABsX0lEQVR4nO3dd3QU1d8G8Gd3k930RnqnBBJKQu8t9CJNKQJSBJVQRMRXESyADRR+KqIgIk1EmlSlSe+9Q+ikF5IQ0nv2vn8s2bCkQ5JJeT7n7El29s7MdyebzJOZe2dkQggBIiIiIgnIpS6AiIiIqi8GESIiIpIMgwgRERFJhkGEiIiIJMMgQkRERJJhECEiIiLJMIgQERGRZBhEiIiISDIMIkRERCQZBpFq4sqVK+jbty9cXV1haGgIKysrtGnTBn/++edLL/vIkSOQyWQ4cuTIC80/Z84cyGSyl66jugkMDIRMJsPq1atLbZkHDx5E8+bNYWxsDJlMhu3bt2P16tWQyWQIDAzUtvvrr7/w448/5pk/JSUFc+bMeeHPQmGK+zk7deoU5syZg7i4uFKvoShjx46Fu7t7ua83R1xcHKytrbFhwwbttJzfL7lcjocPH+aZJzk5GWZmZpDJZBg7dqzOayEhIZg0aRLq1q2r/bvRqFEjvP322wgJCcmzjoIez3528tO5c2d07tz5Zd76S1mxYgWcnJyQnJwsWQ3VmZ7UBVD5iIuLg4uLC4YPH679hVu3bh1GjRqFwMBAfPrpp1KXSBITQmDo0KGoW7cudu7cCWNjY9SrVw9ZWVk4ffo0HBwctG3/+usv3LhxA9OmTdNZRkpKCubOnQsAku1YTp06hblz52Ls2LGwsLAo13V/9tlneO+998p1nc+aO3cuHB0dMWzYsDyvmZiYYNWqVfjyyy91pm/evBmZmZnQ19fXmR4aGoqmTZvCwsICH3zwAerVq4f4+Hj4+/tj06ZNePjwIVxcXHTm2bt3L8zNzfOs+9nPTkU0ZswYfPvtt/juu++0n18qPwwi1UR+/3G88sorCAgIwG+//VZkEElNTYWhoWEZVlj+MjMzIZPJoKfHXwMACA8PR2xsLAYNGoSuXbvqvGZjYyNRVZVL7dq1JVt3bGwsli1bhh9++CHfI4zDhg3DmjVrMHfuXMjluQfDV6xYgUGDBmHnzp067ZcvX46YmBicO3cONWvW1E4fOHAgZs2aBbVanWcdzZo1g7W1dSm+q/Khp6eHCRMm4Msvv8SMGTNgZGQkdUnVCk/NVHPW1tZ5dsTu7u545ZVXsHXrVjRp0gQGBgba/xJu376NXr16wcjICNbW1vDz80NiYmKx17dr1y40btwYKpUKNWvWxMKFC/NtJ4TAkiVL0LhxYxgaGsLS0hKDBw/Oc2hZCIFvvvkGbm5uMDAwQPPmzbF///48wSvnsP7atWvxwQcfwMnJCSqVCvfv3wcAHDhwAF27doWZmRmMjIzQrl07HDx4ME9d9+7dw4gRI2BrawuVSgUvLy/88ssvxXrvMpkMU6ZMwbJly1C3bl2oVCrUr19f5zB6jhs3bmDAgAGwtLSEgYEBGjdujDVr1hS6/OPHj0Mmk2H9+vV5Xvvjjz8gk8lw/vz5fOedM2cOnJ2dAQAzZsyATCbTnmJ4/tRM586dsWvXLgQFBeU5/J4TWObOnaud/uzh/uJuvxf9nM2ZMwcffvghAKBmzZraGnJO56jVanz33Xfw9PSESqWCra0tRo8ejdDQ0CKXHR0djXfeeQcuLi5QqVSwsbFBu3btcODAAW2b/E7N5Pzc165dCy8vLxgZGcHHxwf//vtvnnW8zOdr9erVyMrKyvdoCACMGzcOISEh2L9/v3ba3bt3ceLECYwbNy5P+8ePH0Mul8PW1jbf5T0bZspCbGwsJk2aBCcnJyiVStSqVQuffPIJ0tPTddpt3rwZrVq1grm5OYyMjFCrVi2d96NWq/HVV1+hXr16MDQ0hIWFBby9vbFo0SKd5YwcORIJCQn5/j5SGRNUrWRnZ4vMzEwRFRUlfvnlF6Gnpyd+/fVXnTZubm7CwcFB1KpVS6xcuVIcPnxYnDt3TkRGRgpbW1vh5OQkVq1aJXbv3i1GjhwpXF1dBQBx+PDhQtd94MABoVAoRPv27cXWrVvF5s2bRYsWLbTzP+vtt98W+vr64oMPPhB79+4Vf/31l/D09BR2dnYiMjJS227mzJkCgHjnnXfE3r17xfLly4Wrq6twcHAQnTp10rY7fPiwACCcnJzE4MGDxc6dO8W///4rHj9+LNauXStkMpkYOHCg2Lp1q/jnn3/EK6+8IhQKhThw4IB2GTdv3hTm5uaiUaNG4o8//hD//fef+OCDD4RcLhdz5swpctsDEC4uLqJ+/fpi/fr1YufOnaJXr14CgNi8ebO23e3bt4WpqamoXbu2+OOPP8SuXbvE8OHDBQDx7bffatsFBAQIAGLVqlXaaU2aNBHt2rXLs+4WLVqIFi1aFFhbSEiI2Lp1qwAg3n33XXH69Glx6dIlIYQQq1atEgBEQECAdju0a9dO2Nvbi9OnT2sfaWlpYu/evQKAGD9+vHb6/fv3S7T9XuZzFhISIt59910BQGzdulVbQ3x8vBBCiHfeeUcAEFOmTBF79+4Vv/76q7CxsREuLi4iOjq64B+eEKJnz57CxsZG/Pbbb+LIkSNi+/bt4vPPPxcbNmzQthkzZoxwc3PTmQ+AcHd3Fy1bthSbNm0Su3fvFp07dxZ6enriwYMH2nYv+/nq0qWLaNmyZZ7ps2fPFgBEdHS06NChgxg6dKj2tRkzZgh3d3ehVquFsbGxGDNmjPa1P//8UwAQPXr0EHv37tVuw/zkrCMyMlJkZmbqPLKysoqsvVOnTjq/r6mpqcLb21sYGxuLhQsXiv/++0989tlnQk9PT/Tp00fb7tSpU0Imk4nXX39d7N69Wxw6dEisWrVKjBo1Sttm3rx5QqFQiNmzZ4uDBw+KvXv3ih9//DHfberl5SVeffXVIuul0sUgUs1MmDBBABAAhFKpFEuWLMnTxs3NTSgUCnHnzh2d6TNmzBAymUxcuXJFZ3r37t2LFURatWolHB0dRWpqqnZaQkKCsLKy0gkip0+fFgDE//73P535Q0JChKGhofjoo4+EEELExsYKlUolhg0bptMuZ/78gkjHjh112iYnJwsrKyvRr18/nenZ2dnCx8dH5w97z549hbOzc54/yFOmTBEGBgYiNja20PcPQBgaGuoEqaysLOHp6Snq1Kmjnfb6668LlUolgoODdebv3bu3MDIyEnFxcUKI/INITmi4fPmydtq5c+cEALFmzZpC68tZ3oIFC3SmPx9EhBCib9++eXa4QggRHR0tAIjZs2fnea242+9lP2cLFizIU68QQty6dUsAEJMmTdKZfvbsWQFAzJo1q9DlmpiYiGnTphXapqAgYmdnJxISErTTIiMjhVwuF/PmzdNOe9nPl5GRkfDz88sz/dkgsmrVKqFSqcTjx49FVlaWcHBw0O6Qnw8iarVaTJgwQcjlcgFAyGQy4eXlJd5///082zZnHfk9ateuXWjdQuQNIr/++qsAIDZt2qTT7ttvvxUAxH///SeEEGLhwoUCgPZ3Ij+vvPKKaNy4cZE1CCHEyJEjhZ2dXbHaUunhqZlqZtasWTh//jx27dqFcePGYcqUKfmeHvH29kbdunV1ph0+fBgNGjSAj4+PzvQRI0YUud7k5GScP38er776KgwMDLTTTU1N0a9fP522//77L2QyGd544w1kZWVpH/b29vDx8dEeZj9z5gzS09MxdOhQnflbt25d4MiF1157Tef5qVOnEBsbizFjxuisS61Wo1evXjh//jySk5ORlpaGgwcPYtCgQTAyMtJp26dPH6SlpeHMmTNFboeuXbvCzs5O+1yhUGDYsGG4f/++9vTAoUOH0LVr1zwdAceOHYuUlBScPn26wOUPHz4ctra2OofzFy9eDBsbmwIP2ZeHkmy/l/mcFebw4cMAkGdkSMuWLeHl5ZXvqbjn261evRpfffUVzpw5g8zMzGKv29fXF6amptrndnZ2sLW1RVBQEICSbZ/8xMXFISUlpcDTKDmGDBkCpVKJdevWYffu3YiMjMyzPXLIZDL8+uuvePjwIZYsWYI333wTmZmZ+OGHH9CgQQMcPXo0zzwHDhzA+fPndR7bt28vegM959ChQzA2NsbgwYN1pufUmvOzatGiBQBg6NCh2LRpE8LCwvIsq2XLlrh69SomTZqEffv2ISEhocD12traIioqCllZWSWumV4cg0g14+rqiubNm6NPnz5YunQp3nnnHcycORPR0dE67fLr5f748WPY29vnmZ7ftOc9efIEarW6WPM/evQIQgjY2dlBX19f53HmzBnExMRo6wGgs2PPkd+0/N7Xo0ePAACDBw/Os65vv/0WQgjExsbi8ePHyMrKwuLFi/O069OnDwBo6ypMYe8/5/08fvw43+3v6Oio0y4/KpUKEyZMwF9//YW4uDhER0dj06ZNeOutt6BSqYqsr6yUZPu9zOesqBqA/D/bjo6OhW5XANi4cSPGjBmD33//HW3atIGVlRVGjx6NyMjIItddo0aNPNNUKhVSU1O1tb3M5ytnOc+G/PwYGxtj2LBhWLlyJVasWIFu3brBzc2t0Hnc3NwwceJErFixAvfu3cPGjRuRlpam7YvzLB8fHzRv3lzn0bBhw0KXn5+cz8DznW5tbW2hp6en/Vl17NgR27dvR1ZWFkaPHg1nZ2c0bNhQp5/UzJkzsXDhQpw5cwa9e/dGjRo10LVrV1y4cCHPeg0MDCCEQFpaWolrphfH4QLVXMuWLbX/9Tw7MiK/Xvc1atTI949ucf4QW1paQiaTFWt+a2tryGQyHD9+PN+dZ860nD/uOWHi+WXmd1Tk+feV08N/8eLFaN26db6129nZISsrCwqFAqNGjcLkyZPzbffsyIKCFPb+c95PjRo1EBERkaddeHi4Ts0FmThxIubPn4+VK1ciLS0NWVlZ8PPzK7K2smRpaVns7fcyn7PC5GzfiIgIbcfcHOHh4UVuV2tra/z444/48ccfERwcjJ07d+Ljjz9GVFQU9u7d+1K1lWT75CfnvcXGxha5rnHjxuH333/HtWvXsG7duhLXOnToUMybNw83btwo8bzFVaNGDZw9exZCCJ3f2ZyjFc/+rAYMGIABAwYgPT0dZ86cwbx58zBixAi4u7ujTZs20NPTw/Tp0zF9+nTExcXhwIEDmDVrFnr27ImQkBCdETKxsbFQqVQwMTEps/dGeTGIVHOHDx+GXC5HrVq1imzr6+uL7777DlevXtU5bP7XX38VOa+xsTFatmyJrVu3YsGCBdr/3BITE/HPP//otH3llVcwf/58hIWF5Tnt8qxWrVpBpVJh48aNePXVV7XTz5w5g6CgoGJdWKpdu3awsLCAv78/pkyZUmA7pVIJX19fXL58Gd7e3lAqlUUuOz8HDx7Eo0ePtEdssrOzsXHjRtSuXVu7c+zatSu2bduG8PBw7VEQQDPyxcjIqMDAlMPBwQFDhgzBkiVLkJGRgX79+sHV1fWF6i3Is//NPz8dQJ7XjIyMir39XuZzVlgNXbp0AQD8+eef2kP6AHD+/HncunULn3zySbGWD2iOLE6ZMgUHDx7EyZMniz1fQUqyffKTM6rkwYMHRbZt06YNxo0bh/j4eAwaNKjAdhEREfkePUpKSkJISIjOZ7O0de3aFZs2bcL27dt1avzjjz+0rz9PpVKhU6dOsLCwwL59+3D58mW0adNGp42FhQUGDx6MsLAwTJs2DYGBgahfv7729YcPH+o8p/LBIFJNvPPOOzAzM0PLli1hZ2eHmJgYbN68GRs3bsSHH35YrOtETJs2DStXrkTfvn3x1Vdfwc7ODuvWrcPt27eLVcOXX36JXr16oXv37vjggw+QnZ2Nb7/9FsbGxjr/ybVr1w7vvPMO3nzzTVy4cAEdO3aEsbExIiIicOLECTRq1AgTJ06ElZUVpk+fjnnz5sHS0hKDBg1CaGgo5s6dCwcHh2INLzQxMcHixYsxZswYxMbGYvDgwbC1tUV0dDSuXr2K6OhoLF26FACwaNEitG/fHh06dMDEiRPh7u6OxMRE3L9/H//88w8OHTpU5Pqsra3RpUsXfPbZZzA2NsaSJUtw+/ZtnSGDs2fPxr///gtfX198/vnnsLKywrp167Br1y589913+V4w6nnvvfceWrVqBQBYtWpVke1LqlGjRti6dSuWLl2KZs2aQS6Xo3nz5jA1NYWbmxt27NiBrl27wsrKCtbW1nB3dy/29nvZz1mjRo0AaH5eY8aMgb6+PurVq4d69erhnXfeweLFiyGXy9G7d28EBgbis88+g4uLC95///0ClxkfHw9fX1+MGDECnp6eMDU1xfnz57F3716dEPwyXvbz1blzZ+zZs6dY61qxYkWRbb7++mucPHkSw4YN0w6jDwgIwM8//4zHjx9jwYIFeea5ePFivp/P+vXrw8zMrFi1AcDo0aPxyy+/YMyYMQgMDESjRo1w4sQJfPPNN+jTpw+6desGAPj8888RGhqKrl27wtnZGXFxcVi0aBH09fXRqVMnAEC/fv3QsGFDNG/eHDY2NggKCsKPP/4INzc3eHh4aNepVqtx7tw5jB8/vth1UimRtq8slZeVK1eKDh06CGtra6GnpycsLCxEp06dxNq1a/O0dXNzE3379s13Of7+/qJ79+7CwMBAWFlZifHjx4sdO3YUazSDEELs3LlTeHt7C6VSKVxdXcX8+fO1Pe7zq7lVq1bC2NhYGBoaitq1a4vRo0eLCxcuaNuo1Wrx1VdfCWdnZ6FUKoW3t7f4999/hY+Pjxg0aJC2Xc6omWeHyT7r6NGjom/fvsLKykro6+sLJycn0bdv3zztAwICxLhx44STk5PQ19cXNjY2om3btuKrr74q8r0DEJMnTxZLliwRtWvXFvr6+sLT01OsW7cuT9vr16+Lfv36CXNzc6FUKoWPj4/O6JicWvDcqJlnubu7Cy8vryLren55xRk1ExsbKwYPHiwsLCyETCbT+fkdOHBANGnSRKhUKgFAZyRGcbffy37OZs6cKRwdHbUjPnLmyc7OFt9++62oW7eu0NfXF9bW1uKNN94QISEhhS4vLS1N+Pn5CW9vb2FmZiYMDQ1FvXr1xOzZs0VycrK2XUGjZiZPnpxnmW5ubjrbpiTbJz8HDx4UAMS5c+d0pj87aqYwz4+aOXPmjJg8ebLw8fERVlZWQqFQCBsbG9GrVy+xe/fufNdR0GP//v2Frvv5UTNCCPH48WPh5+cnHBwchJ6ennBzcxMzZ84UaWlp2jb//vuv6N27t3BychJKpVLY2tqKPn36iOPHj2vb/O9//xNt27YV1tbW2r8748ePF4GBgfluv4sXLxZaK5U+mRBClGfwISprAQEB8PT0xOzZszFr1iypy9GSyWSYPHkyfv755zJf17Vr1+Dj44NffvkFkyZNKvP1UcXg7e2Ndu3aaY/iUfGNGjUKDx8+LJVTbVQyPDVDldrVq1exfv16tG3bFmZmZrhz5w6+++47mJmZVctDrA8ePEBQUBBmzZoFBweHAodmUtX03XffYdCgQfjkk0/ydMilgj148AAbN24s1ulVKn0MIlSpGRsb48KFC1ixYgXi4uJgbm6Ozp074+uvvy5wCG9V9uWXX2ovJb5582beM6Oa6dWrFxYsWICAgAAGkRIIDg7Gzz//jPbt20tdSrXEUzNEREQkGV7QjIiIiCTDIEJERESSYRAhIiIiyVTozqpqtRrh4eEwNTXN95LjREREVPEIIZCYmAhHR8ciLy5ZoYNIeHh4njuQEhERUeUQEhJS5AiuCh1Ecm6bHRISUqLLAxMREZF0EhIS4OLiot2PF6ZCB5Gc0zFmZmYMIkRERJVMcbpVsLMqERERSYZBhIiIiCTDIEJERESSqdB9RKqj7OxsZGZmSl0GERFRofT19aFQKF56OQwiFUhSUhJCQ0PB2/8QEVFFJ5PJ4OzsDBMTk5daDoNIBZGdnY3Q0FAYGRnBxsaGF3AjIqIKSwiB6OhohIaGwsPD46WOjDCIVBCZmZkQQsDGxgaGhoZSl0NERFQoGxsbBAYGIjMz86WCCDurVjA8EkJERJVBae2vGESIiIhIMgwiREREJBkGESozY8eOxcCBA0s836FDh+Dp6Qm1Wl2s9tevX4ezszOSk5NLvC4iIpIWgwiVmUWLFmH16tUlnu+jjz7CJ598UuSto3M0atQILVu2xA8//FDidRERkbQYRKjMmJubw8LCokTznDp1Cvfu3cOQIUNKNN+bb76JpUuXIjs7u0TzERGRtBhEKighBFIysiR5lPSCan///TcaNWoEQ0ND1KhRA926dUNycnKeUzOdO3fG1KlT8dFHH8HKygr29vaYM2eOzrI2bNiAHj16wMDAQLsdunXrhl69emnriouLg6urKz755BPtfD179sTjx49x9OjRF9vgREQkCV5HpIJKzcxG/c/3SbJu/y96wkhZvI9GREQEhg8fju+++w6DBg1CYmIijh8/XmCYWbNmDaZPn46zZ8/i9OnTGDt2LNq1a4fu3bsDAI4dO4bhw4dr28tkMqxZswaNGjXCTz/9hPfeew9+fn6ws7PTCTFKpRI+Pj44fvw4unTp8uJvnoiIyhWDCL2UiIgIZGVl4dVXX4WbmxsATZ+Ngnh7e2P27NkAAA8PD/z88884ePCgNogEBgbC0dFRZx4nJycsW7YMo0aNwqNHj/DPP//g8uXL0NfXz9MuMDCwFN8dERGVNQaRCspQXwH/L3pKtu7i8vHxQdeuXdGoUSP07NkTPXr0wODBg2FpaZlve29vb53nDg4OiIqK0j5PTU3VnpZ51pAhQ7Bt2zbMmzcPS5cuRd26dfPWbWiIlJSUYtdORETSYxCpoGQyWbFPj0hJoVBg//79OHXqFP777z8sXrwYn3zyCc6ePZtv++ePYshkMp1hutbW1njy5Eme+VJSUnDx4kUoFArcu3cv32XHxsaidu3aL/FuiIiovLGzKr00mUyGdu3aYe7cubh8+TKUSiW2bdv2Qstq0qQJ/P3980z/4IMPIJfLsWfPHvz00084dOhQnjY3btxAkyZNXmi9REQkDQYReilnz57FN998gwsXLiA4OBhbt25FdHQ0vLy8Xmh5PXv2xIkTJ3Sm7dq1CytXrsS6devQvXt3fPzxxxgzZozOkZPAwECEhYWhW7duL/V+iIiofDGI0EsxMzPDsWPH0KdPH9StWxeffvop/ve//6F3794vtLw33ngD/v7+uHPnDgAgOjoa48ePx5w5c9C0aVMAwOzZs+Ho6Ag/Pz/tfOvXr0ePHj20HWaJiKhykImSXjSiHCUkJMDc3Bzx8fEwMzOTupwylZaWhoCAANSsWTPfzprVyUcffYT4+HgsW7asWO3T09Ph4eGB9evXo127dmVcHRERAYXvt0qy/+YREapwPvnkE7i5uRX7KqlBQUH45JNPGEKIiCqhij8sg6odc3NzzJo1q9jt69atm+9wXiIiqvh4RISIiIgkwyBCREREkmEQISIiIskwiBAREZFkGESIiIhIMgwiREREJBkGESIiIpIMgwgRERXLTz/9BJlMhrZt2yIlJUXqcqgETp8+DWNjY9ja2uLu3btSl6ODQYRKnRACc+bMgaOjIwwNDdG5c2fcvHmzyPm2bNmC+vXrQ6VSoX79+vnewXfJkiXaywk3a9YMx48fL/G609PT8e6778La2hrGxsbo378/QkNDddo8efIEo0aNgrm5OczNzTFq1CjExcWVfGOUUFHvLz9Hjx5Fs2bNYGBggFq1auHXX3/N06aobevu7g6ZTJbnMXnyZG2bOXPmwNPTE8bGxrC0tES3bt1w9uxZneU8ePAAgwYNgo2NDczMzDB06FA8evToBbdG+XqRn3l5ft6Cg4PRr18/GBsbw9raGlOnTkVGRoZOm+vXr6NTp04wNDSEk5MTvvjiC5TWXTzWrVuHDz/8ED/99BNiY2Px2muvITMzU6fN1atXMXz4cLi4uMDQ0BBeXl5YtGhRqay/MGX1N2fevHlo0aIFTE1NYWtri4EDB2rvg/Ui6xZCoHfv3pDJZNi+fXu+bdLT09G4cWPIZDJcuXKlWO+/KDdu3EDfvn0xbtw4dOjQAd27d8/zGQSACRMmoHbt2jA0NISNjQ0GDBiA27dvl0oNhRIVWHx8vAAg4uPjpS6lzKWmpgp/f3+RmpoqdSkvbf78+cLU1FRs2bJFXL9+XQwbNkw4ODiIhISEAuc5deqUUCgU4ptvvhG3bt0S33zzjdDT0xNnzpzRttmwYYPQ19cXy5cvF/7+/uK9994TxsbGIigoqETr9vPzE05OTmL//v3i0qVLwtfXV/j4+IisrCxtm169eomGDRuKU6dOiVOnTomGDRuKV155pUTbwc3NTRw+fLjY7Yvz/p738OFDYWRkJN577z3h7+8vli9fLvT19cXff/+tbVOcbRsVFSUiIiK0j/379wsAOvWvW7dO7N+/Xzx48EDcuHFDjB8/XpiZmYmoqCghhBBJSUmiVq1aYtCgQeLatWvi2rVrYsCAAaJFixYiOzu7+BuuhAICAkRp/Cl7kZ95eX3esrKyRMOGDYWvr6+4dOmS2L9/v3B0dBRTpkzRtomPjxd2dnbi9ddfF9evXxdbtmwRpqamYuHChS+9bXbt2iVMTEzEjh07hBBCPHr0SHh7e4vXX39d52e7YsUK8e6774ojR46IBw8eiLVr1wpDQ0OxePHiEq0PgAgICCh2+7L6m9OzZ0+xatUqcePGDXHlyhXRt29f4erqKpKSkl5o3d9//73o3bu3ACC2bduWb11Tp07Vtrl8+XKxt0FBAgIChKOjo5g5c6YQQvNZeuONN4SXl5eIiYnRabts2TJx9OhRERAQIC5evCj69esnXFxcdD6rzypsv1WS/XeZBpFvvvlGNG/eXJiYmAgbGxsxYMAAcfv27WLPzyAihEhPKviRUZK2KcVr+5LUarWwt7cX8+fP105LS0sT5ubm4tdffy1wvqFDh4pevXrpTOvZs6d4/fXXtc9btmwp/Pz8dNp4enqKjz/+uNjrjouLE/r6+mLDhg3aNmFhYUIul4u9e/cKIYTw9/cXAHT+IJ0+fVoAKNHnt6RBpKj3l5+PPvpIeHp66kybMGGCaN26tfZ5cbbt89577z1Ru3ZtoVarC2yT8/t54MABIYQQ+/btE3K5XOf3NTY2VgAQ+/fvz3cZUVFRws7OTnz99dfaaWfOnBH6+vpi3759Ba77WaURRF7kZ16en7fdu3cLuVwuwsLCtG3Wr18vVCqVdnsvWbJEmJubi7S0NG2befPmCUdHxwJ/jmvWrBHGxsbi7t272mlTpkwRHh4e2p3tiRMnhLW1tfjvv/905o2NjRUtW7YUkyZNynfZOSZNmiR8fX0LbfO8kgSRsvyb87yoqCgBQBw9erTE675y5YpwdnYWERERBQaR3bt3C09PT3Hz5s0ig8itW7eEoaGhWLdunXbali1bhEqlEteuXRNCaAKjh4eH+Oqrr3Tmzc7OFhMmTBAtW7YUiYmJBa7j6tWrAoC4f/9+vq9XiiBSnDRZGAYRIcRss4Iffw7WbfuVfcFtV/bRbfttzfzbvaQHDx4IAOLSpUs60/v37y9Gjx5d4HwuLi7i+++/15n2/fffC1dXVyGEEOnp6UKhUIitW7fqtJk6daro2LFjsdd98OBBAUDExsbqtPH29haff/65EELzX525uXmeGs3NzcXKlSsLfA/PK0kQKc77y0+HDh3E1KlTdaZt3bpV6OnpiYyMDCFE0ds2v1pq1KihEw7ya7NgwQJhbm4uoqOjhRBC7Ny5UygUCp0dYUpKipDL5WL27NkFLmvXrl1CX19fnD9/XiQmJoo6deqI9957r8D2zyuNIPIiP/Py/Lx99tlnwtvbW+f1nJB36NAhIYQQo0aNEv3799dpc+nSJQFAPHz4sIB3LsSQIUNEixYtRGZmptizZ4/Q19cX586dK7B9SY0cOVK89tprJZqnJEGkrP7m5OfevXsCgLh+/XqJ1p2cnCy8vLzE9u3bte/v+SASGRkpnJycxPnz57Wf6aKOiPzyyy/C3NxcBAYGirCwMGFlZSV++OGHQucprqSkJDFt2jRRs2ZNkZ6enm+b0goiZdpHZO/evRg7diwaNGgAHx8frFq1CsHBwbh48WJZrpYkFBkZCQCws7PTmW5nZ6d9raD5CpsnJiYG2dnZhbYpzrojIyOhVCphaWlZaBtbW9s8Ndra2hb6Hl5Gcd5ffgrabllZWYiJiSm0TUHL3b59O+Li4jB27Ng8r/37778wMTGBgYEBfvjhB+zfvx/W1tYAgNatW8PY2BgzZsxASkoKkpOT8eGHH0KtViMiIqLA99CnTx+8/fbbGDlyJPz8/GBgYID58+cX2L4svMjPvDw/b/n9DC0tLaFUKgttk/O8sM/QsmXLEBERgalTp2Ls2LGYPXs2WrRoUWD7kjh9+jQ2bdqECRMmlMry8lNWf3OeJ4TA9OnT0b59ezRs2LBE637//ffRtm1bDBgwoMBljx07Fn5+fmjevHmBNT9v0qRJaN++PUaNGoXRo0ejWbNmeO+994o9f36WLFkCExMTmJiYYO/evdi/fz+USuVLLbMo5Xr33fj4eACAlZVVvq+np6cjPT1d+zwhIaFc6qrQZoUX/JpMofv8w/uFtH0uc067/uI1PbVu3TqdPzB79uyBQqGpSSaT6bQVQuSZlqfEYsxTWm2e93yb/NoXtRw/Pz/8+eef2ucpKSno3bu3dpsAgL+/P1xdXQtcRmltt+enl2S5K1asQO/eveHo6JjnNV9fX1y5cgUxMTFYvnw5hg4dirNnz8LW1hY2NjbYvHkzJk6ciJ9++glyuRzDhw9H06ZNdbZBfhYuXIiGDRti06ZNuHDhAgwMDApt36BBAwQFBem8XxMTE+3rbm5uBXYYfP7nlJSUBODFfub5zVdWn7cXaZPfZ+F5lpaWWLFiBXr27Im2bdvi448/LrT24rp58yYGDBiAzz//HN27dy+0be/evfN0zG7QoIFO3Tk/p4KU1d+cHFOmTMG1a9dw4sSJEi1n586dOHToEC5fvlxgHYsXL0ZCQgJmzpxZaL35WblyJerWrQu5XI4bN24U+Z6LMnLkSHTv3h0RERFYuHAhhg4dipMnTxb5O/kyyi2I5Jcmnzdv3jzMnTu3vEqqHJTG0rctQP/+/dGqVSvtcycnJ+1/vpGRkXBwcNC+FhUVlee/hmfZ29vn+U/k2Xmsra2hUCgKbWNvb1/kuu3t7ZGRkYEnT57o/JcaFRWFtm3batvkN9IjOjq60PfwxRdf4P/+7/+0zzt37oxvv/1WZxvlt3Mv7vvLT0HbTU9PDzVq1Ci0TX7LDQoKwoEDB7B169Z812dsbIw6deqgTp06aN26NTw8PLBixQrtH9AePXrgwYMHiImJgZ6eHiwsLGBvb4+aNWsW+B4A4OHDhwgPD4darUZQUBC8vb0Lbb97927tiI2wsDB07txZZ4SBvr5+gfM+/3MCXuxnXp6fN3t7+zwjlJ48eYLMzEydNvn9nIG8/7E/79ixY1AoFAgPD0dycjLMzMwKbV8Uf39/dOnSBW+//TY+/fTTItv//vvvSE1N1T738PDA7t274eTkVOS8xfk5FDRfcX8v3n33XezcuRPHjh2Ds7NzidZ96NAhPHjwABYWFjrLfO2119ChQwccOXIEhw4dwpkzZ6BSqXTaNG/eHCNHjsSaNWsKfB9Xr15FcnIy5HI5IiMjC/wbU1w5I7c8PDzQunVrWFpaYtu2bRg+fPhLLbdQJThl9FImTZok3NzcREhISIFt0tLSRHx8vPYREhLCPiKVTE7nrW+//VY7LT09vVgdx3r37q0zrVevXnk6q06cOFGnjZeXV57OqoWtO6fz4MaNG7VtwsPD8+08ePbsWW2bM2fOlEtn1cLeX34++ugj4eXlpTPNz88vT2fVorZtjtmzZwt7e3uRmZlZrJpr165daP+PgwcPCplMVuh2S09PFz4+PmLMmDFi3rx5wsbGRkRGRhZr/UKUbmfVkvzMy/PzltNZNTw8XNtmw4YNeTqrWlhY6JzPnz9/fqGdVYUQ4uTJk0JfX1/s3r1beHt7F9qvojhu3LghbG1txYcffvjCy8ALdFYti785arVaTJ48WTg6Oup06C3JuiMiIsT169d1HgDEokWLtH13goKCdF7ft2+fACD+/vvvQveZjx8/Fo6OjmL27NliwoQJwtPTU6SkpBTYvqTS09OFoaGhWLVqVb6vV4rOqjmmTJkinJ2dC+0wlR92Vq2c5s+fL8zNzcXWrVvF9evXxfDhw/MMZxs1apTODvbkyZNCoVCI+fPni1u3bon58+cXOHx3xYoVwt/fX0ybNk0YGxuLwMDAEq3bz89PODs7iwMHDohLly6JLl265Duc0tvbW5w+fVqcPn1aNGrUqNyG7xb2/j7++GMxatQo7fOc4bvvv/++8Pf3FytWrMgzfLc421YITU96V1dXMWPGjDy1JSUliZkzZ4rTp0+LwMBAcfHiRTF+/HihUqnEjRs3tO1WrlwpTp8+Le7fvy/Wrl0rrKysxPTp0wt93//3f/8n3N3dRXx8vMjOzhYdO3YUffv2LfZ2K83hu0X9zOvVq6fTobi8Pm85w3e7du0qLl26JA4cOCCcnZ11hu/GxcUJOzs7MXz4cHH9+nWxdetWYWZmVujw3YSEBFGrVi3tz+jGjRvCwMBAbNq06YW24Y0bN4SNjY0YOXKkznDwnCHexVWSICJE2f3NmThxojA3NxdHjhzReT/P7uyLs+783l9Bw3eFEMXurDpkyBDRqlUrkZmZKZKTk0W9evWKHMVUkAcPHohvvvlGXLhwQQQFBYlTp06JAQMGCCsrK/Ho0aN856kUQaSoNFkUBpHKSa1Wa/+zVqlUomPHjtpe5jk6deokxowZozNt8+bNol69ekJfX194enqKLVu25Fn2L7/8Itzc3IRSqRRNmzbVDqMrybpTU1PFlClThJWVlTA0NBSvvPKKCA4O1mnz+PFjMXLkSGFqaipMTU3FyJEjxZMnT0q0HUoaRIrz/saMGSM6deqkM+3IkSOiSZMmQqlUCnd3d7F06dI8yy3Ots35L+zOnTt5XktNTRWDBg0Sjo6OQqlUCgcHB9G/f/88oytmzJgh7OzshL6+vvDw8BD/+9//Cv1v/PDhw0JPT08cP35cOy0oKEiYm5uLJUuWFDjfs0oriBTnZw5A57/D8vy8BQUFib59+wpDQ0NhZWUlpkyZojNCSQghrl27Jjp06CBUKpWwt7cXc+bMKXT7v/nmm6JRo0Y6y1m0aJGwsrISoaGhxdhqumbPni0A5Hm4ubmVaDklDSJl9Tcnv/fyIp+B/N7fywaR/IZeX7hwQSiVSrFr165C15+fsLAw0bt3b2Frayv09fWFs7OzGDFiRKFHM0sriMiEKKXL7uVj0qRJ+Ouvv7Bjxw7Uq1dPO93c3ByGhoZFzp+QkABzc3PEx8e/9DnLii4tLQ0BAQHaq2oSERFVZIXtt0qy/y7T4btLly5FfHw8OnfuDAcHB+1j48aNZblaIiIiqiTKdNRMGR5sISIioiqAN70jIiIiyTCIEBERkWQYRCoYns4iIqLKoLT2VwwiFUTOJbAzMjIkroSIiKhoOfurom7hUJRyvdcMFUxPTw9GRkaIjo6Gvr4+5HJmRCIiqpjUajWio6NhZGQEPb2XixIMIhWETCaDg4MDAgICtDfzIiIiqqjkcjlcXV1f+kZ7DCIViFKphIeHB0/PEBFRhadUKkvl6D2DSAUjl8t5ZVUiIqo22BGBiIiIJMMgQkRERJJhECEiIiLJMIgQERGRZBhEiIiISDIMIkRERCQZBhEiIiKSDIMIERERSYZBhIiIiCTDIEJERESSYRAhIiIiyTCIEBERkWQYRIiIiEgyDCJEREQkGQYRIiIikgyDCBEREUmGQYSIiIgkwyBCREREkmEQISIiIskwiBAREZFkGESIiIhIMgwiREREJBkGESIiIpIMgwgRERFJhkGEiIiIJMMgQkRERJJhECEiIiLJMIgQERGRZBhEiIiISDIMIkRERCQZBhEiIiKSDIMIERERSYZBhIiIiCTDIEJERESSYRAhIiIiyTCIEBERkWQYRIiIiEgyDCJEREQkGQYRIiIikgyDCBEREUmGQYSIiIgkwyBCREREkmEQISIiIskwiBAREZFkGESIiIhIMgwiREREJBkGESIiIpIMgwgRERFJhkGEiIiIJMMgQkRERJJhECEiIiLJMIgQERGRZBhEiIiISDJlGkSOHTuGfv36wdHRETKZDNu3by/L1REREVElU6ZBJDk5GT4+Pvj555/LcjVERERUSemV5cJ79+6N3r17l+UqiIiojKVmZCM+NROJaZlISMtEQloW1GoBIQABQAgBQ6UCJio9zcNAD1bGSqj0FFKXTpVAmQaRkkpPT0d6err2eUJCgoTVEBFVH/EpmbgdmYC7jxJxPyoJYXGpCItLQ3hcKuJTM19omdYmStiZGcDB3ABuNYzhYWsCDztT1LE1gbmhfim/A6qsKlQQmTdvHubOnSt1GUREVVp6VjZuhCXgYlAsLgQ+wdXQODxKSC90HoVcBjMDPZgZ6sNEpQc9uQyQySCXaV5PzchGUnqW5pGWhSy1QExSBmKSMnAzPO8/lc6WhmjqaokmrhZo4mqJBo5m0Fdw/ER1JBNCiHJZkUyGbdu2YeDAgQW2ye+IiIuLC+Lj42FmZlYOVRIRVT1CCNyLSsKRO1E4ejca5wOfICNLnaedk4Uh6tmbwsPOBC6WRnCyNISThSHszQ1gqtKDTCYr9vqepGQiIj4VjxLSEBaXhoDoZNyL0hxtiYhPyzOPsVKBNrVroIOHDTrWtYF7DaNir48qnoSEBJibmxdr/12hjoioVCqoVCqpyyAiqvTUaoELQU+w61o49vs/QvhzO/8axko0dbNEczdLNHWzhKe9KUwNSud0iUwmg5WxElbGSjRwNM/zekJaJq6FxONy8BNcDonDpeAniEvJxIFbUThwKwoA4FbDCL0bOqBvIwc0dDJjKKnCKlQQISKiFyeEwLXQeGy/Eobd1yN0Treo9ORoXasGOtXVHHGobWMs2c7dzEAf7T2s0d7DGoAmNPlHJOD4vRgcuxuNC0GxCHqcgl+PPsCvRx/A2dIQr3g7YnAzZ9SxNZGkZio7ZXpqJikpCffv3wcANGnSBN9//z18fX1hZWUFV1fXIucvyaEdIqLqKj4lE9uvhGHD+RDcisjtj2FqoIce9e3R19sebWtbw0C/coxiSU7PwuE7UdhzPRKHbkchNTNb+1pTVwsMae6CV7wdSu0IDpW+kuy/yzSIHDlyBL6+vnmmjxkzBqtXry5yfgYRIqKC3QiLx8qTAdh1LQLpT/t8KPXk6NXAHv19HNGhrnWlH0KbmpGNw3eisPVSKA7fiUa2WrPLMlYqMLiZM8a2q4ma1sYSV0nPqzBB5GUxiBAR6VKrBQ7ficLy4w9x5mGsdno9O1O83tIFg5o4wcJIKWGFZScqIQ3bLodh44UQPIxOBgDIZIBvPVuMa1cT7erUYF+SCoJBhIioisnMVmPb5TAsO/oAD57uhBVyGV7xdsCYtu5o4mJRbXbCQgicvP8YK08G4NDtKO30hk5meLeLB7p72UEurx7boqJiECEiqiKyngaQxYfuIzg2BQBgqtLD8FauGNvWHY4WhhJXKK2H0UlYcyoQmy6EavuSeNqbYmpXD/RqYM9AIhEGESKiSi5bLbDzahgWHbiHwMeaAFLDWIl3OtbCiFau7Kj5nNjkDKw48RBrTgUhKT0LgOZ01Yze9eBbz7baHC2qKBhEiIgqsRP3YvD17lvaETBWTwPI6DZuMFLyqguFiUvJwMqTgVh1MgCJaZpA0qqmFWb18YKPi4W0xVUjDCJERJXQ/ahEfLP7trbfg6mBHvw61cbYtu4wVjGAlER8SiaWHL2PVScDtVeR7evtgI97ecLFykji6qo+BhEiokokPjUT3/93B3+eDUa2WkBPLsMbrd0wtasHrIyr5giY8hIWl4rv/7uLrZdDIYTmwm4TO9eGX6falea6KpURgwgRUSUghMD2K2H4etctxCRlAAC617fDzN6eqGXDK4iWJv/wBHy1yx+nHjwGALhaGWF2v/ro6mUncWVVE4MIEVEFdz8qCZ9tv4HTDzU7xlo2xvhyQEO0q2MtcWVVlxAC/16LwNe7biEyQXPvnW5etpjdrwFP15QyBhEiogoqPSsbvxy6j6VHHyAzW0ClJ8e7Xerg7Y61Kv1VUCuL5PQs/HToHlYcD0CWWsBIqcBHPethdBt3DvctJQwiREQV0LXQOHy4+RruPEoEAHTxtMXc/vxvXCr3oxIxa+sNnAvUXKG2hbslvn3Nm6fFSgGDCBFRBZKelY2fDt7Dr0cfIlstUMNYiS8HNkTvhva8voXE1GqBdWeDMH/PbSRnZEOpJ8f73eri7Q41oaeQS11epcUgQkRUQVwPjccHm6/g7qMkAMAr3g6Y278BapioJK6MnhX6JAWztt3AsbvRADR3+f1hWGO41eAN9V4EgwgRkcTUaoFlxx7if//dQZZawNpEiS8HNETvRg5Sl0YFEEJgy6UwzP3nJhLTsmCsVGB2/wYY0syZR65KiEGEiEhCkfFpmL7pinaoaO+G9vh6UCNeE6SSCH2SgumbruJcgKbvSO+G9vhmUCNY8udXbAwiREQS2XczEjO2XENcSiYM9RWY078+hjZ34X/UlUy2WmDZsQf4/r+7yFIL2Jmp8P3QxhxeXUwMIkRE5SwtMxtf/OuPv84GA9Dckn7R601QmyMwKrXrofF4b+NlPIxOhkwGvNvFA+919YCCw3wLVZL9N7sEExG9pKDHyXh1ySltCJnQsRa2TmzHEFIFNHI2x7/vtsfwli4QAvjp4D2MXnkW0YnpUpf28iKuAlG3pK6CQYSI6GX8dzMSryw+Af+IBFgZK/HHuJaY2ccLSj3+ea0qjJR6mPeqN34c1hhGSgVO3n+MPj8dx5mnV8WtdFLjgN0fAr91BnZOBdRqScvhbwoR0QvIylZj/p7beGftRSSmZaGpqwV2TW2PjnVtpC6NysjAJk7YOaUdPGxNEJ2YjhHLz+CXw/ehVlfYHg752zEZOPcbINSAhQuQmSJpOewjQkRUQlGJaZi6/jLOPNSMqniznTtm9uZRkOoiJSMLn267ga2XwwBoblT4/VAfmBroS1xZIdRqQP708xl5Hdj6DtBrHlCrc5msjp1ViYjKyJWQOLzzxwVEJabDWKnAt4O98Yq3o9RlUTkTQmDj+RB8vuMmMrLVqGNrguWjm6OmdQW7AFp8KLB/NmBiB/T6Jne6EEAZjuRiZ1UiojKw7XIohi47jajEdHjYmmDHlPYMIdWUTCbD6y1dsXFCa9iZqXA/Kgn9fz6BI3eipC5NIyMFODIfWNwcuPE3cP53ICk69/UKNJycQYSIqAjZaoF5u2/h/Y1XkZGlRjcvO2yb3A51bDkqprpr4mqJf6a0R1NXCySmZeHN1eex9MgDSHayQa0Grm0Gfm4BHJkHZKUCrm2Bt/YDJhWz/xJPzRARFSIhLRPvrb+Mw3c0/01O8a2D6d3r8nbxpCM9Kxtzdt7E+nMhADT3FFow2AeGSkX5FfHIH9j2jqYPCACYuwDdvwAaDCr3IyAl2X/rlVNNRESVTkBMMt5acx4PopOh0pNjwRAf9PfhqRjKS6WnwDeDGqG+oznm7ryJf69FICQ2BcvHNIetqUH5FGFsA8QGACozoN1UoPVkQGlUPut+CTwiQkSUj1MPYjDxz0uIT82EvZkBfhvdDN7OFlKXRZXA2YePMeHPi4hLyYSThSFWjm2Bevampb+i2IfAze1Ah+m50+4fBBwaA8Y1Sn99JcDOqkREL2HLxVCMWXkO8amZaOxigZ1T2jGEULG1qlUD2ya1Q01rY4TFpeK1padw9G500TMWV3wYsOsDTT+Qg3OBh0dyX6vTVfIQUlIMIkRETwkhsOjAPXyw+SoyswX6ejtgwzutYWtWTofWqcqoaW2MbZPaolVNKySlZ2Hc6vNYeybo5RaaEAHs/gj4qbFmFIw6C6jTTTM0txLjqRkiIgAZWWrM2nYdf18MBQBM6FQLM3p6slMqvZSMLDVmbr2OLZc0n6vx7WtiVh+vkt00Lz0JOPQVcHEVkJWmmebaFvCdBdTsUAZVvzx2ViUiKoGEtExM+vMSTtyPgVwGfDGgId5o7SZ1WVQFKPXkWDjEG7VsjLFg3x2sOBGA0CcpWPR6ExjoF3NEjZ4BcG+fJoS4tAZ8ZwI1O1Woa4G8DB4RIaJqLSwuFW+uOoe7j5JgpFTglxFN4etpK3VZVAX9ey0c0zdeRUa2Gi3cLfH76BYwN8rnsvBPgoALK4DOswD9p6cF7x3QXKK9lm+lCCA8IkJEVAy3IhIwZuU5RCWmw9ZUhZVjW6Chk7nUZVEV9Yq3I6xNVHj7jws4H/gEQ5adwuo3W8LRwlDT4JE/cPJH4PrfgMgGLGsCzd/UvObRTbK6yxqDCBFVS+cCYjF+zXkkpmWhrp0JVr3ZEk45OwSiMtK6Vg1s9muDsSvP4+6jJLy65BQ29ZXD9eYy4O6e3Ia1OgO2XpLVWZ4YRIio2tnv/whT/rqE9KwiDpETlQFPezNsmdQWb604iTnxs+C67fbTV2RA/f5Au2mAU1MpSyxXHL5LRNXKpgsh8PvzItKz1OjmZYu141sxhFD5yUoHADhZGGL9xI7QNzBBhlBgk7oLjvXcAwz9o1qFEIBBhIiqkV+PPsBHf19DtlpgcDNn/PpGs+KPXCB6GY8fAHtmAAvrai5IBsDCSIkGb/6MWa7r8FHGWxi7MxbrzwVLXGj546kZIqry1GqBeXtuYfnxAACaa4R83MsTskow+oAqMbUaeHhYc/GxO3sAPB2keuNvoN17AACVgxfmj60H/R03sP5cCGZuvY7EtEy807G2dHWXMwYRIqrSMrPVmLHlGrZe0vwX+kkfL7zdsZbEVVGVlp6oCR8XVwNPAnOne/QAWk/UDMF9hp5Cjm8GNYK5oRK/Hn2Ab3bfRkJqFj7oUbdahGUGESKqstKzsjHlr8vY7/8ICrkM373mjdeaOUtdFlUHxxYCGUmAyhzweR1o+TZg7VFgc5lMho97e8LMUA/f7b2Dnw/fR0JaJub0a1Dlr+7LIEJEVVJqRjbeWXsBx+/FQKknx9KRTdHVq3Lfk4MqoMRHwPVNQOh5YMgazcXGVKZApxmAkRXQ4FVAaVTsxU3qXAemBvr4fMcN/HE6CIlpWVgw2Bt6iqrbpZNBhIiqnMS0TIxffQHnAmNhpFTg99HN0baOtdRlUVWRmQbc2Q1cXQ/cP6i5+BgAhF0EnJtrvm839YUXP6q1G0xVevhg81VsuxyGpPQsLB5egkvCVzIMIkRUpcSlZGDMynO4GhoPUwM9rH6zJZq5WUpdFlUF0XeBs0uBG1uAtPjc6c4tgcYjAJt6pbaqgU2cYKLSw6S/LmG//yOMW30ev41uDhNV1dttV91jPURU7UQnpuP1387gamg8rIyVWP92a4YQejnZWbnfxwUDF1ZqQoiZE9DhA2DKBeCt/ZpLsatMS3XV3erbYc2bLWGsVODUg8d44/eziE/JLNV1VAS86R0RVQnhcal44/ezeBiTDFtTFda91QoedqW7Y6Bq4kkgcGMrcHOr5i63Pb/WTFdnA7s/BLz6ATU7AvLyOVVyNSQOY1adQ1xKJho4muHP8a1gaawsl3W/qJLsvxlEiKjSC3qcjBHLzyIsLhVOFob46+1WcKthLHVZVJnEhwI3t2vCR9jF3OkWbsB7VyW/4+3tyASMXH4Wj5Mz4Glvij/fagVrE5WkNRWGQYSIqo37UYkY+ftZPEpIR01rY6x7q1Xu3UyJimPDSOD2v7nPZXLAvT3Q8DXAq79m9EsFcD8qESOWn0VUYjpq2xjjr7dbw87MQOqy8lWS/Tf7iBBRpeUfnoChy87gUUI66tmZYuOE1gwhVDB1NhB8Bjj4hW7fD1MHADLAtQ3QZyEw/TYw5h+g2dgKE0IAoI6tKTZOaAMHcwM8iE7GsGWnER6XKnVZL41HRIioUroRFo83VpxFXEomvJ3NsebNlhX+vDlJICVWc5n1+weBu/uAlBjN9LG7Afd2mu/jwwCFPmBiK12dJRASm4Lhy88g9EkqnC0Nsf7t1nCxKv61SspDSfbfVW8cEBFVeddC4/DG72eRkJaFJq4WWDOuJcwMeAddekbgCeDAHE1/D6HOnW5gDtTtpTvCxdyp3Mt7GS5WRtg4oQ1GLD+DoMcpGLbsNP56uzXcrStnvyiemiGiSuVKSBxGPg0hzdws8QdDCMUFA5fXASHnc6fpGWqudirUgI0X0PZdYPQO4MMHwKu/AQ7e0tVbCpwsDLFpQhvUtjFGeHwahi47jftRSVKX9UJ4aoaIKo1LwU8wZsU5JKZnoYW7JVa92bJKXuCJipAQDgQcBwKPab7GBWmmN3kDGPCL5nu1GriyDqjtC5hX3fsLRSem443fz+LOo0RYmyix7q3WqGcv/bB1jpohoirnYlAsxqw8j6T0LLSsaYVVY1vAmCGkeslIBpZ1BB7f150uUwBOTYH6A4G2UyQpTUqxyRl44/ez8I9IgKWRPtaOb4WGTuaS1sQ+IkRUpZwPjMXYleeQnJGNNrVqYMXY5jBS8s9XlZSVDkRcBULOAiHnNJ1IB6/UvKY01gytlckBBx/AvYPmwmKurUv9qqaViZWxEn+93Up7a4ORv5/FurekDyPFxSMiRFShnX34GG+uPo+UjGy0q1MDv49uAUNl1bz5V7V1dx8QeFwTPMIvA9kZua/pGwEfhwCKp8HzkT9g5ggYWkhSakWWkJaJMSvP4XJwHMwN9SUNIzw1Q0RVwukHjzFu9XmkZmajg4c1lo9uXmXvQFrlCQHEPtQEjZi7gO+s3NfWDQXu7ct9bmQNuLQCXFo+fbQqt8upV3aJaZkY/UwY+XN8KzRyLv8wwiBCRJXeyfsxGL/mPNIy1ehU1wbLRjVjCKlMom5pLh4WdQuI8gcirgHpz9yxdvptwMxB8/3FNZqAkhM+rGpJfkn1yizx6ZGRS8FxMDPQw59vtYK3s0W51sAgQkSV2rG70Xj7jwtIz1LDt54Nlr7BEFIhZSRrOo5G3daEjfbv554y2TsLOPOLbnuFCrBvCDg0BjpMr9KjWaSWmJaJsavO42LQE0nCCDurElGldeROFN5ZexEZWWp087LFLyObQqXHEFIhBBwHbmzRhI/HD4DEcN3X6/YE3NpqvndpCcTcAWy9NNfxcPAGbDw1nU+pzJka6GPNuJYYu/IcLgQ9wcjfz+LP8a3g42IhdWl58IgIEVUYh29HYcLai8jIVqN7fTv8MqIplHq87mKZEgJIitJcFCwu6OnXp48ngZqLfzk317Q9vwLYNV13fkNLTdCw9QJavAXY1S/3t0AFS0rPwpurzuF84BOYGuhh7fhWaFwOYYSnZoio0jng/wgT111EZrZAzwZ2WDycIeSlqdVAymMgMQJIjMz92mgwUKO2pk1+4eJZA5cCjUdovn90E7ixFahR5+mjdoW6KRzlLyk9C+NWnce5wFiYqvTwx/iWaOJqWabr5KkZIqpU9vs/wqSnIaRPI3sser0J9BUMIflSZwOpTzQBIzlG8zUlBqjZKTdc3N4F7JmhCR3qzLzLsKqZ29bCTXNdDjMnwNwFsHDVfdg3yp3ProHmQZWKiUoPq95sgTdXn8e5gFiMXnEOa8a3RNMyDiPFVS5BZMmSJViwYAEiIiLQoEED/Pjjj+jQoUN5rJqIKrj/bkZi8l+XkJkt0NfbAT8Oa1x9QkhaguZ0SFoCkBYPpCc883080Gho7j1Rbv0D7JyqCSHI50D2oGW54UKuD8SH5L5mbAOY2gOmjpqvZs/c5K1WJ+DTKPbdqOKMVXpYNVY3jPxRQcJImQeRjRs3Ytq0aViyZAnatWuHZcuWoXfv3vD394erq2tZr56IKrBnQ8grT0OInhQhRAggOxPITtd8zUrX9H3QN9C8nhSluQZGdgaQlfH0ayqQkQJkpmju5mrppmkbdEozHDUzGch8pk1miub7fj8CHt01be/sAba9U3Bdtg1yg4hCBaTG5r5mYAEYWwNGNTTX3TC2yX3NpSXw1kFN6DCxKzxkMIBUG8YqPax+swXGrT6PMw+fHhkZ1xLN3KQNI2XeR6RVq1Zo2rQpli5dqp3m5eWFgQMHYt68eTpt09PTkZ6ern2ekJAAFxeXUu8jkp6VjanrL2NYCxd08bQrteUSVTjqbM2ONWeHCmgO56cnPN3xZmp2quoszdfsTKBW59xrOISc1/zHLgQgsjXLE2rN90INNB4J6Kk0bR8cAiKva6ars/PO0+49wODp7/GtfxB0cR+O3XkEmchGzRqGaFPTAnIITfvuXwCmT383r20Cbm7PXa86O3f9ajXQ/6fcIwGX/gDO/aZZt067p98P/QNwbKJpe2655jbx2Rm6V/LMMXqHZlsAwIWVwL/vF7ydX/8L8Oyr+f7qxsLDxcBfgcbDNd/f2w9sn6TZLgbmgMpM9/uGrwJOzTRt0xKA+FBN+DC0ZICgF5aSkYXxqy/g9MPHMFYq8Mf4lmjmVrp9fSpMH5GMjAxcvHgRH3/8sc70Hj164NSpU3naz5s3D3Pnzi3LkgAAq08GYt/NRzh0OwpLRzZDt/oMIyQBITTXYchIAtITNTvDZ8+/39ymGbWQ8fQ/66w0IDNN85+4XE8zmiHHjslA4Inc1zPTnu5chabt549z2+6cCtzZVXBdn0YDekrN9+eWAdc3F9y2/sDcIOK/A7i4uuC2zd/UBpHAi/vhfn8tRuWMyo0HcOWZth3/LzeIRN8pvN70hNzvk6M1Yaggmam53wu1ZtvnR6YAsrNynxtaai6ypVDmPvQNnz6MAGPb3LaOjYHuXwJKI81r+kaae6TktLWqmdvWozvw4b2C632WgRlgwBEp9PKMlHpYOVZzZOT0w8eYuv4KDv9fZ8k6h5dpEImJiUF2djbs7HR39HZ2doiMjMzTfubMmZg+Pbf3ds4RkdI2rn1NXA2Nw+7rkZi47iKWjGyG7gwj9KLSkzSHzFOfAClPv+Y85HpA+2m5bdeP0FxBMiNJ8xDq3NeMbXV3Smd/A4LzBnYAgJ6hbhBJitKElvyoszShJ+coh9IIUJpq7t0h13+6Y9XTfJXra44c5LDx1NxYLOdGY3KFZied8738mT8hzi00AUiu0Kzr2XYyhWYnDGDvjQhsvmWHJrIBqGNnju4NHKFQ6GnmyWlrVCN3uZ59NBe+kj9dnkzxzPdyTWfLHA0GAfY+gFyed/1yBWBTL7et91BNEFCoNO9d75mQ8fzlxBsM0jyKw6ae7nqIKiBDpQIrx7bAB5uvYFLnOpKOUCuXzqqy5y7VK4TIMw0AVCoVVCpVmdejr5Bj0etNIJNdwa5rEZi07iJ+HtEUPRvYl/m6qZLISNEMdUyOBpIeaXb0Od/rGQC9v81tu9xXc++M/JjY6QaR1Cd5LwIlk2uCgcFzhy89umn+e9Y30pxa0TPM/ao00m3bbS7Q8UNNbfqGmqMUCpXm8L38uV/z134v/nbo+H+aR3E0eUPzKMSe6xF4d/1lZKkbw6RxH/gN8YGiqD4hTs1yT08UxaqW5lEchpaaB1E1ZahUYMnIYv5ulaEyDSLW1tZQKBR5jn5ERUXlOUpS3vQVciwa1hhymQz/XA3H5HWX8POIJujV0EHSuqgcZKUDCWFAfNjTr6Ga6c/ucJd10Fw9Mj/GNrpBxMROczTC0EqzYzN6+tXQQvPas/ou1JwyUZkBShPNrcv1DfO/r0aHD4r/nirBRaT2XI/AlPWXka0WGNjYEf8b2hgKOe8nQlTdlWkQUSqVaNasGfbv349Bg3IPa+7fvx8DBgwoy1UXi55Cjh+G+kAuA3ZcCcfkvy5j8XCgTyOGkUpNCM21FZKidHfQf4/X9KNIyntaEEbWukHEzAlICAdMbDWnTExsc783fe7I2RtbNUceinOTrmp6DYbdT4+EZKsFBjVxwsIhPgwhRASgHE7NTJ8+HaNGjULz5s3Rpk0b/PbbbwgODoafn19Zr7pY9BRyfD9Uc2Rk2+UwvLv+MtRC4BVvR6lLo+J45A88uqHp0BhzF4gNeNrBM1Fz5OLDZ45qpDzODSF6Bk8v4OQEmDlr+iA8249ixCbN6Y3ihIucjp2Ur2dDyKtNnLCAIYSInlHmQWTYsGF4/PgxvvjiC0RERKBhw4bYvXs33Nzcip65nCjkMiwc4gOZDNh6KQzvbbgCtQD6+zCMVAhp8ZpLS0ff0fTb8J2V+9qej4DA4/nPp1BqOk/mDF3t+jnQ9TPAwl1z+qSwkPHscFd6YbuuRWDqhqchpKkTFgxmCCEiXbzXzDOy1QIztlzD3xdDIZcBPwxrjAGNnYqekUpXwHFNuIi8ATy6rrn51rNmhmr6VgDAwS+AoNOATV3Aup7mehKW7pqRFAwTkvr3Wjje23AF2WqB15o647vB3gwhRNVEhbmOSGWjkMvw3WvekMuATRdC8f7GK1ALgUFNnKUurepRqzVXqgy7CIRf0lzAKud6FNc2ApfX6rY3cwZsPTVhI/uZe2d0/bz8aqZi++dqOKZt1ISQwc2c8e1rDCFElD8GkefI5TLMf9UbcpkMG86HYPqmq1CrgdeaMYy8lKRoIPScJniEXQTCLmvupZGj8cjcS1nX7qK5voZdQ80Nt+wa8A6flcizIWRIM2fMZwghokIwiORDLpfhm0GNIJPJsP5cMP7v76sQAAYzjBSPOlvTp8PSPffaGJfWAIe+1G2nZwDYe2uuEaEyyZ3e8FXNgyqdnVfDMW3DZagFMOTpkRA5QwgRFYJBpAByuQxfD2wIhRz480wwPvz7KtRqgaEtSv9Kr5VeZprm9ErQKSD4DBByVnPZ7aF/APWfDtN2bqG5gZdT06cXqGoK2Nbn/TKqkB1Xwp6ezmQIIaLiYxAphFwuw5cDGkIuk+GP00H4aMs1qIXA6y1512AAmnt67J2pCR7P3zRMaaq5EmmOWp2ASQVcrpwqvb8vhuKjv69CLYChzZ01pzcZQoioGBhEiiCTyTC3fwPIZTKsPhWIj7deh1oAI1pVozAihOYaHQ+Pam51XrenZrrKLHforLEt4NYGcG0LuLbW9O9Q8ONVHaw/F4xZ265DCGB4Sxd8PbARQwgRFRv3FMUgk8kwu199yGTAqpOBmLXtOtRC4I3WFedaKKUuLQF4eBi4+x/w4KDm+h0AULd3bhCxdAMGLgWcW2qGzRbn4l9UpfxxOhCf77gJABjTxg1z+jfI9z5SREQFYRApJplMhs9fqQ+5TIYVJwLw6fYbEEJgVBt3qUsrXWo1sG4wEHAMUD8zTFbPQHOko7avbvvGI8q3Pqowfj/+EF/tugUAeLtDTczq48UQQkQlxiBSAjKZDJ/29YJCLsNvxx7isx03kaUWeLNdTalLezHZWUDQSSDyGtD2Xc00uVxzUzh1JlCjDuDRU3OrdNc2vEAYaf1y+D4W7LsDAJjsWxv/16MeQwgRvRAGkRKSyWSY2dsTMhmw7OhDzP3HHykZ2ZjsW0fq0oonK13T1+PWDuD2biA1FoAM8BkOGFtr2vT8StP/o0ZtSUulikcIgR8P3MOig/cAANO718XUrh4SV0VElRmDyAuQyWT4uJcnDPQUWHTwHhbsu4PUjGx80KNuxf2vMPgscP534O5ezdDaHIZWgGcfIDM1d5pjk/Kvjyo8IQS+23cHS488AADM6OWJiZ0ZVono5TCIvCCZTIb3u9eFkVKBeXtu4+fD95GSkY3PXqkg58mzMgB1FqA00jx/fA+4vknzvYk94NUPqN9fM8qFo1uoCEIIfLXrFlacCAAAfNrXC291qCVxVURUFXAP9JImdKoNI6UCn+24iZUnA5CamYWvBjaS5pLWajUQcga4tgnw3w50mgG0nqh5rV4foM0UwKu/5uJicnn510eVklotMOefm/jjdBAA4MsBDapeJ20ikgyDSCkY1cYdBvoKzNhyDevPhSA1IxsLh/hAT1FOO/tH/pobxd3YAsSH5E6/fzA3iBhZAT2/Lp96qMrIylZjxpbr2HIpFDIZMP/VRhjWohpdQ4eIyhyDSCkZ0twFBvoKvL/xCrZfCUdapho/DW8CpV4ZhhF1NrCqt+bKpjmUpppTLo2GADU7lt26qcpLz8rG1PWXse/mIyjkMiwc4s07URNRqWMQKUX9fBxhqK/ApHWXsPdmJN5ZewG/vtEMBvqK0lmBEEDEVcCxsea5XAEY2wByfc1FxhoN0XzVNyyd9VG1lZKRhQlrL+L4vRgoFXIsHtEEPRvYS10WEVVBMiGEkLqIgiQkJMDc3Bzx8fEwMzOTupxiO3EvBm//cQGpmdloXcsKv49pARPVS2S+hAjgyjrg8p/AkwDg3Uu5Q2tjH2qG2uYMvSV6SfGpmXhz1TlcCo6DkVKB5aObo10dfr6IqPhKsv9mj8Uy0N7DGn+MbwkTlR7OPIzFyOVnEJucUfSMzxICCDwBbBoD/NgQOPSlJoQoTYEo/9x2VrUYQqjURCem4/XfzuBScBzMDPTw51utGEKIqEzxiEgZuhYahzErz+FJSiZq2Rhj7fhWcLIoxmmT6LvAptFA9K3caS6tgaajgQYDAaVxmdVM1VdYXCre+P0sAmKSYW2iwtrxLeHlUPl+74hIejwiUkF4O1tgs19bOJob4GF0MgYvPYX7UYn5N85Izv3e3BlIDAf0jYBmYwG/k8D4fUCTkQwhVCYeRCdhyNJTCIhJhpOFIf72a8MQQkTlgkdEykFEfCpGrTiH+1FJsDDSx6qxLdDE1VJz+iXgGHD6ZyAuGJh0JvcOtoEnAbsGgKGFpLVT1Xc1JA7jVp/H4+QM1LYxxp9vtYKDOTs8E9GLK8n+m0GknDxJzsCbq8/jSkgcTPUFNrWPhFfAGs0N5wAAMsDvBGDfUNI6qXo5cicKk9ZdQkpGNho5mWP1my1Qw0QldVlEVMmVZP/N4bvlxNJYiXVveGHnqnno9ORvOJ6O1bygZ6g55dJ6Em8yR+Vq66VQfPT3NWSpBTp4WGPpG81ebnQXEdEL4F+dcmQcfQnD434DZEC0MMea7J5w6jQJw315kzkqP0IILDv2EPP33AYADGzsiO8G+5TtxfeIiArAIFKWEsI1Q23rdNM8r90VqD8Q6tpd8WtwA6w4GwnsC0dgqgFm9PKEXIr701C1olYLfLnLH6tOBgIA3ulYCx/zs0dEEmIQKQtPgoCTP2ouQKZvBLx/A1CZajqiDl0DOYBPmwpYmt/Hwv/uYtmxhwiNS8X/hviU3lVYiZ6TnpWN6ZuuYte1CAC8gy4RVQwMIqXp8QPg+P+AqxsAka2Z5tQcSI7RBJFnyGQyTOniASdLQ3z09zXsuhaBR/FpWD66OSyNlRIUT1VZXEoGJqy9iLMBsdBXyLBwiA8GNHaSuiwiIgaRUpEQARyZpzkCkhNAavkCHT8E3NsVOuugJs6wMzPAhLUXcSHoCV5degqr32wBtxq8XgiVjsCYZIxbfR4PY5JhotLD0jeaooOHjdRlEREB4AXNSkd6AnB5rSaE1OkOvHUQGL29yBCSo21ta2yZ2BZOFoYIiEnGoCWncCn4SdnWTNXC+cBYDFpyEg9zLlQ2sQ1DCBFVKAwiLyL1CeC/I/e5TT2g2xxg3D7gjb8B5+YlXmRdO1Nsm9QWDZ3MEJucgeG/ncHeGxGlVzNVO9svh2Hk8rN4kpIJH2dzbJvcFp72lft6PERU9fCCZiWRmQqcWQKcWARkJAKTzwHWHqW6iuT0LLy7/jIO3Y4CAHzQvS6mdKkDmYyjGqh4hBBYdPAefjxwDwDQq4E9fhjWGIZKdoQmovLBe82UNrVa0wF1cTPg4BdAejxg4wWkxZf6qoxVevhtVDOMbesOAPjf/rt4d/1lpGZkl/q6qOpJy9SMjMkJIRM61sKSkU0ZQoiowmJn1aIEHAP++xSIuKp5bu4CdP0caDgYkJdNjtNTyDGnfwPUszfFZ9tv4N9rEQh6nILfRjfjPUCoQBHxqZiw9iKuhcZDIZfhq4ENMbylq9RlEREViqdmCpOeCHzfQHMERGUGdJgOtJoI6BuUWwlnHz7GxHWXEJucAWsTFX4b3QxNXS3Lbf1UOVwIjIXfn5cQk5QOSyN9/DKiKdrWsZa6LCKqpnjTu5eRngSoTHKfn/4FiA0AOn8MGEvzhz0kNgVv/3EBtyMToVTIMe/VRnitmbMktVDFs+5sEObsvInMbAFPe1MsH90cLlZGUpdFRNUYg8iLUGcDl/7Q9AF5dTng0a1s11dCyelZmL7pCvbdfAQAGNPGDZ/0rc/7g1RjGVlqzP3nJtadDQYA9G3kgAVDvGGk5BlXIpIWO6uWVMh5YHkX4N9pQGoscHGV1BXlYazSw9KRzTC1q2aUzprTQRi67DTC41IlroykEBGfihHLz2Dd2WDIZMCHPevh5xFNGEKIqNKp3kdEkqKBA3OAK39qnqvMAN9PgBZvAYqK+wf90O1HeH/jVcSnZsLSSB8/DW/Ci1RVI8fuRmPaxiuITc6AqUoPi4Y3RhdPO6nLIiLS4qmZ4ri6Adj9kaYjKgA0fgPoNhswsS3d9ZSRkNgUTFx3ETfCEiCTAe93q4spvnV4F9UqLFutuT7I4kP3IATQwNEMS0Y25e0AiKjC4amZ4lCZakKIgw8w/gAw8JdKE0IAwMXKCH/7tcXwli4QAvh+/12MW3Mej5PSpS6NykB0YjpGrzyLnw5qQsiIVq7YMrEtQwgRVXrV94iIEMDdvYBHD0BeuS/2tPlCCD7dfgPpWWrYmKrwvyE+6FiXp2qqipP3Y/D+xiuISkyHob4C37zaEIOacNQUEVVcPDVTDd2KSMDU9ZdxLyoJAPBOx1r4vx71OKqmEkvPysbCfXew/HgAAMDD1gRLRjaFh52pxJURERWOQaSaSs3Ixte7/fHnGc1wzoZOZlj0ehPUtjEpYk6qaO4+SsR7G67gVkQCAGBkK1d82rc+L9VORJUCg0g199/NSHy05RriUjJhqK/AJ329MLKVK2+cVwkIIfDH6SB8s/sW0rPUsDJW4rvXvNGtPkfFEFHlwSBCiIxPw/RNV3DqwWMAQLs6NfDta95wtuQVNyuq0CcpmLn1Oo7fiwEAdKprgwVDvGFrWn63FCAiKg0MIgQAUKsFVp8KxHf7biMtUw1jpQKz+nphREseHalI1GqBdWeDMH/PbSRnZEOlJ8fM3p4Y09adPyciqpQYREhHQEwyPtx8FReCngAA2texxrxXG/F+JBVAYEwyPtpyDecCYgEALdwt8e1r3qjFfj1EVIkxiFAe2U+Pjix4enTEQF+Od7t44O0OtTiyRgIZWWqsOBGARQfvIi1TDSOlAjN6eWJUazdelI6IKj0GESrQw+gkfLLtBk4/1PQdqW1jjK8GNkKb2jUkrqz6OHk/Bp/vuIEH0ckANP135r/qzSNURFRlMIhQoYQQ2H4lDF/vuoWYpAwAwKtNnPBxH092jCxDkfFp+GqXP/69FgEAsDZR4uPeXnitqRP7ghBRlcIgQsUSn5KJ7/bdxl/ngiEEYKRUwK9TbbzdoRavV1GK0jKzseJEAJYcvo/kjGzIZcCo1m6Y3qMezA31pS6PiKjUMYhQiVwOfoK5//jjSkgcAMDezAD/17MeXm3ixP4KLyFbLbD1Uii+338XEfFpAICmrhb4cmBDNHA0l7g6IqKywyBCJSaEwD/XIvDtntsIi0sFoLm76//1rIfOdW146qAEhBA4cjca3+65jduRiQAAJwtD/F/Puhjgw3BHRFUfgwi9sLTMbKw+FYhfDt1HYnoWAKCxiwWmdfNAJwaSQuUEkEUH7mmPLpkZ6GFKlzoY3cYdBvo83UVE1QODCL20x0np+PXoA6w9E4S0TDUAoImrBaZ29eARkucIIXDodhR+OngPV0PjAQAqPTlGt3HDZN86sDBSSlwhEVH5YhChUhOdmI5lTwNJepYmkHjYmuCtDjUxoLFTtf4vPy0zGzuuhGHliUDceaQ5BWOgL8eo1m54u2MtjkAiomqLQYRKXVRiGn47+hDrzwUjOSMbgGb46ajW7hje0gW2ZtVnpxuVkIY/zwThz7PBiE3WDH82Uiq0AcTaRCVxhURE0mIQoTKTkJaJDeeCsepkoHYkiEIuQxdPW7zewgWd6tpAT1H1rtSala3G0bvR2Hg+BIduRyFLrfm1cbIwxJi2bhjW3BXmRhyKS0QEMIhQOcjMVmPPjUisORWIi0/vYQMAdmYqDGrijFe8HdDA0axS9yURQuBmeAL+vRaBrZdCEZWYrn2tmZslxrWriZ4N7Kpk8CIiehkMIlSu7j1KxMbzIdh6OUx7qgIA3GoYoW8jB/RqaI+GjuaVYtiqWi1wPSwee25EYvf1CATHpmhfszJW4tUmThjS3AX17E0lrJKIqGKrMEHk66+/xq5du3DlyhUolUrExcWVaH4GkcolPSsbB29F4Z+r4Th0O0rbuRUAahgr0bGuDTrVtUG7OtawMa04/SiiE9Nx/F40jt2NxvF7MXj8TJgy0JfDt54t+vs4oquXHW8QSERUDBUmiMyePRsWFhYIDQ3FihUrGESqkeT0LBy6HYVd1yJw/F60toNrDvcaRmjqaommbpZo7GKBOrYm5TICJy0zG/ejknA5JA6Xg57gUvATBD5O0WljrFSgUz0b9GnkAN96tjBW6ZV5XUREVUmFCSI5Vq9ejWnTpjGIVFMZWWpcCn6Co3ejcfRONG5FJuD5T51cBrhbG6OurSnq2JrA0cIQjhYGcLQwhJ2ZAUxVesU6taNWCySmZSEyIQ3h8amIjE9D2JNU3ItKxL1HSQh8nAx1Pp/4hk5m6Ohhg451bdDU1ZJHPoiIXkJJ9t8V6l+99PR0pKfndghMSEiQsBoqLUo9OVrXqoHWtWpgRi9PxKdm4nLwE1wKeoILQU9wMzwB8amZeBidjIfRycDN/JdjotKDqYEejJQKyJ92gpXJALXQHIFJTMtC0tOrwRbGzEAPPi4WaOJqiaauFmjiYskRL0REEqlQQWTevHmYO3eu1GVQGTM31EfnerboXM8WgGZ0SlRiOu4+SsSdyEQExCQjIj4N4XGpiIhPQ3xqJgAgKb14QQMALIz0YW+mOaJib26A2jYmqGtngrp2prA1VVXq0TxERFVJiYPInDlzigwL58+fR/PmzUtczMyZMzF9+nTt84SEBLi4uJR4OVS5yGQy2JkZwM7MAB08bPK8np6VjcS0rKePTCSna/qbCGjOscggg4lKDyYGmiMmJiq9an3FVyKiyqTEQWTKlCl4/fXXC23j7u7+QsWoVCqoVBVnNAVVDCo9BVQmCl6xlIioCipxELG2toa1tXVZ1EJERETVTJn2EQkODkZsbCyCg4ORnZ2NK1euAADq1KkDExOTslw1ERERVQJlGkQ+//xzrFmzRvu8SZMmAIDDhw+jc+fOZblqIiIiqgR4iXciIiIqVSXZf/OqTURERCQZBhEiIiKSDIMIERERSYZBhIiIiCTDIEJERESSYRAhIiIiyTCIEBERkWQYRIiIiEgyDCJEREQkGQYRIiIikgyDCBEREUmGQYSIiIgkwyBCREREkmEQISIiIskwiBAREZFkGESIiIhIMgwiREREJBkGESIiIpIMgwgRERFJhkGEiIiIJMMgQkRERJJhECEiIiLJMIgQERGRZBhEiIiISDIMIkRERCQZBhEiIiKSDIMIERERSYZBhIiIiCTDIEJERESSYRAhIiIiyTCIEBERkWQYRIiIiEgyDCJEREQkGQYRIiIikgyDCBEREUmGQYSIiIgkwyBCREREkmEQISIiIskwiBAREZFkGESIiIhIMgwiREREJBkGESIiIpIMgwgRERFJhkGEiIiIJMMgQkRERJJhECEiIiLJMIgQERGRZBhEiIiISDIMIkRERCQZBhEiIiKSDIMIERERSYZBhIiIiCTDIEJERESSYRAhIiIiyTCIEBERkWQYRIiIiEgyDCJEREQkGQYRIiIikgyDCBEREUmGQYSIiIgkwyBCREREkimzIBIYGIjx48ejZs2aMDQ0RO3atTF79mxkZGSU1SqJiIioktErqwXfvn0barUay5YtQ506dXDjxg28/fbbSE5OxsKFC8tqtURERFSJyIQQorxWtmDBAixduhQPHz4sVvuEhASYm5sjPj4eZmZmZVwdERERlYaS7L/L7IhIfuLj42FlZVXg6+np6UhPT9c+T0hIKI+yiIiISCLl1ln1wYMHWLx4Mfz8/ApsM2/ePJibm2sfLi4u5VUeERERSaDEQWTOnDmQyWSFPi5cuKAzT3h4OHr16oUhQ4bgrbfeKnDZM2fORHx8vPYREhJS8ndERERElUaJ+4jExMQgJiam0Dbu7u4wMDAAoAkhvr6+aNWqFVavXg25vPjZh31EiIiIKp8y7SNibW0Na2vrYrUNCwuDr68vmjVrhlWrVpUohBAREVHVV2adVcPDw9G5c2e4urpi4cKFiI6O1r5mb29fVqslIiKiSqTMgsh///2H+/fv4/79+3B2dtZ5rRxHDBMREVEFVmbnSsaOHQshRL4PIiIiIoD3miEiIiIJMYgQERGRZBhEiIiISDIMIkRERCQZBhEiIiKSDIMIERERSYZBhIiIiCTDIEJERESSYRAhIiIiyTCIEBERkWQYRIiIiEgyDCJEREQkGQYRIiIikgyDCBEREUmGQYSIiIgkwyBCREREkmEQISIiIskwiBAREZFkGESIiIhIMgwiREREJBkGESIiIpIMgwgRERFJhkGEiIiIJMMgQkRERJJhECEiIiLJMIgQERGRZBhEiIiISDIMIkRERCQZBhEiIiKSDIMIERERSYZBhIiIiCTDIEJERESSYRAhIiIiyehJXUBhhBAAgISEBIkrISIiouLK2W/n7McLU6GDSGJiIgDAxcVF4kqIiIiopBITE2Fubl5oG5koTlyRiFqtRnh4OExNTSGTySSpISEhAS4uLggJCYGZmZkkNVRk3D4F47YpHLdP4bh9CsftUzipt48QAomJiXB0dIRcXngvkAp9REQul8PZ2VnqMgAAZmZm/LAXgtunYNw2heP2KRy3T+G4fQon5fYp6khIDnZWJSIiIskwiBAREZFkGESKoFKpMHv2bKhUKqlLqZC4fQrGbVM4bp/CcfsUjtuncJVp+1TozqpERERUtfGICBEREUmGQYSIiIgkwyBCREREkmEQISIiIskwiBAREZFkGERKoH///nB1dYWBgQEcHBwwatQohIeHS11WhRAYGIjx48ejZs2aMDQ0RO3atTF79mxkZGRIXVqF8fXXX6Nt27YwMjKChYWF1OVIbsmSJahZsyYMDAzQrFkzHD9+XOqSKoRjx46hX79+cHR0hEwmw/bt26UuqcKYN28eWrRoAVNTU9ja2mLgwIG4c+eO1GVVGEuXLoW3t7f2aqpt2rTBnj17pC6rSAwiJeDr64tNmzbhzp072LJlCx48eIDBgwdLXVaFcPv2bajVaixbtgw3b97EDz/8gF9//RWzZs2SurQKIyMjA0OGDMHEiROlLkVyGzduxLRp0/DJJ5/g8uXL6NChA3r37o3g4GCpS5NccnIyfHx88PPPP0tdSoVz9OhRTJ48GWfOnMH+/fuRlZWFHj16IDk5WerSKgRnZ2fMnz8fFy5cwIULF9ClSxcMGDAAN2/elLq0wgl6YTt27BAymUxkZGRIXUqF9N1334maNWtKXUaFs2rVKmFubi51GZJq2bKl8PPz05nm6ekpPv74Y4kqqpgAiG3btkldRoUVFRUlAIijR49KXUqFZWlpKX7//XepyygUj4i8oNjYWKxbtw5t27aFvr6+1OVUSPHx8bCyspK6DKpgMjIycPHiRfTo0UNneo8ePXDq1CmJqqLKKD4+HgD4dyYf2dnZ2LBhA5KTk9GmTRupyykUg0gJzZgxA8bGxqhRowaCg4OxY8cOqUuqkB48eIDFixfDz89P6lKogomJiUF2djbs7Ox0ptvZ2SEyMlKiqqiyEUJg+vTpaN++PRo2bCh1ORXG9evXYWJiApVKBT8/P2zbtg3169eXuqxCVfsgMmfOHMhkskIfFy5c0Lb/8MMPcfnyZfz3339QKBQYPXo0RBW+Sn5Jtw8AhIeHo1evXhgyZAjeeustiSovHy+yfUhDJpPpPBdC5JlGVJApU6bg2rVrWL9+vdSlVCj16tXDlStXcObMGUycOBFjxoyBv7+/1GUVSk/qAqQ2ZcoUvP7664W2cXd3135vbW0Na2tr1K1bF15eXnBxccGZM2cq/KGvF1XS7RMeHg5fX1+0adMGv/32WxlXJ72Sbh/S/A4pFIo8Rz+ioqLyHCUhys+7776LnTt34tixY3B2dpa6nApFqVSiTp06AIDmzZvj/PnzWLRoEZYtWyZxZQWr9kEkJ1i8iJwjIenp6aVZUoVSku0TFhYGX19fNGvWDKtWrYJcXvUPuL3M56e6UiqVaNasGfbv349BgwZpp+/fvx8DBgyQsDKq6IQQePfdd7Ft2zYcOXIENWvWlLqkCk8IUeH3UdU+iBTXuXPncO7cObRv3x6WlpZ4+PAhPv/8c9SuXbvKHg0pifDwcHTu3Bmurq5YuHAhoqOjta/Z29tLWFnFERwcjNjYWAQHByM7OxtXrlwBANSpUwcmJibSFlfOpk+fjlGjRqF58+bao2fBwcHsUwQgKSkJ9+/f1z4PCAjAlStXYGVlBVdXVwkrk97kyZPx119/YceOHTA1NdUeVTM3N4ehoaHE1Ulv1qxZ6N27N1xcXJCYmIgNGzbgyJEj2Lt3r9SlFU7KITuVybVr14Svr6+wsrISKpVKuLu7Cz8/PxEaGip1aRXCqlWrBIB8H6QxZsyYfLfP4cOHpS5NEr/88otwc3MTSqVSNG3alEMwnzp8+HC+n5MxY8ZIXZrkCvobs2rVKqlLqxDGjRun/Z2ysbERXbt2Ff/995/UZRVJJkQV7mlJREREFVrVP4lPREREFRaDCBEREUmGQYSIiIgkwyBCREREkmEQISIiIskwiBAREZFkGESIiIhIMgwiREREJBkGESIiIpIMgwgRERFJhkGEiIiIJPP/6Wa05DX8rn8AAAAASUVORK5CYII="
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# This is a demonstration: You just need to run this cell without editing.\n",
"\n",
"# Set learning rate\n",
"learning_rate = 1e-6\n",
"\n",
"# Initialize weights to 0\n",
"a = torch.tensor(0.)\n",
"b = torch.tensor(0.)\n",
"c = torch.tensor(0.)\n",
"d = torch.tensor(0.)\n",
"\n",
"print('iter', 'loss', '\\n----', '----', sep='\\t')\n",
"for t in range(1, 5001): # 5000 iterations\n",
" # Forward pass: compute predicted y\n",
" y_pred = a + b * x + c * x**2 + d * x**3\n",
"\n",
" # Compute MSE loss\n",
" loss = torch.mean(torch.square(y_pred - y_true))\n",
" if t % 1000 == 0:\n",
" print(t, loss.item(), sep='\\t')\n",
"\n",
" # Backpropagation\n",
" grad_y_pred = 2.0 * (y_pred - y_true) / y_pred.shape[0]\n",
" \n",
" # Compute gradients of a, b, c, d with respect to loss\n",
" grad_a = grad_y_pred.sum()\n",
" grad_b = (grad_y_pred * x).sum()\n",
" grad_c = (grad_y_pred * x ** 2).sum()\n",
" grad_d = (grad_y_pred * x ** 3).sum()\n",
"\n",
" # Update weights using gradient descent\n",
" a -= learning_rate * grad_a\n",
" b -= learning_rate * grad_b\n",
" c -= learning_rate * grad_c\n",
" d -= learning_rate * grad_d\n",
"\n",
"# print fitted polynomial\n",
"equation = f'{a:.5f} + {b:.5f} x + {c:.5f} x^2 + {d:.5f} x^3'\n",
"\n",
"y_pred = a + b * x + c * x**2 + d * x**3\n",
"plt.plot(x, y_true, linestyle='solid', label='sin(x)')\n",
"plt.plot(x, y_pred, linestyle='dashed', label=f'{equation}')\n",
"plt.axis('equal')\n",
"plt.title('3rd degree poly fitted to sine (MSE loss)')\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "76952906",
"metadata": {},
"source": [
"### 1.2.2 Demo - Using autograd to automatically compute gradients\n",
"\n",
"In the previous example, we explicitly computed the gradient for Mean Squared Error (MSE): \n",
"`grad_y_pred = 2.0 * (y_pred - y_true) / y_pred.shape[0]`\n",
"\n",
"In the next example, we will use PyTorch's autograd functionality to help us compute the gradient for **Mean Absolute Error (MAE)**. \n",
"In order to compute the gradients, we will use the `.backward()` method of *PyTorch* tensors.\n",
"\n",
"Once again, we fit a **degree 3 polynomial** to the sine function, using a learning rate of `1e-6` and `5000` iterations. \n",
"This time, we will use MAE instead of MSE."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "d2861c55",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-01T09:19:57.884874Z",
"start_time": "2024-04-01T09:19:57.603483Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"iter\tloss\t\n",
"----\t----\n",
"torch.Size([1000]) torch.Size([1000])\n"
]
},
{
"ename": "RuntimeError",
"evalue": "element 0 of tensors does not require grad and does not have a grad_fn",
"output_type": "error",
"traceback": [
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
"\u001B[0;31mRuntimeError\u001B[0m Traceback (most recent call last)",
"Cell \u001B[0;32mIn[5], line 23\u001B[0m\n\u001B[1;32m 20\u001B[0m \u001B[38;5;28mprint\u001B[39m(t, loss\u001B[38;5;241m.\u001B[39mitem(), sep\u001B[38;5;241m=\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;130;01m\\t\u001B[39;00m\u001B[38;5;124m'\u001B[39m)\n\u001B[1;32m 22\u001B[0m \u001B[38;5;66;03m# Automatically compute gradients\u001B[39;00m\n\u001B[0;32m---> 23\u001B[0m \u001B[43mloss\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mbackward\u001B[49m\u001B[43m(\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 25\u001B[0m \u001B[38;5;66;03m# Update weights using gradient descent\u001B[39;00m\n\u001B[1;32m 26\u001B[0m \u001B[38;5;28;01mwith\u001B[39;00m torch\u001B[38;5;241m.\u001B[39mno_grad():\n",
"File \u001B[0;32m/opt/homebrew/anaconda3/envs/cs2109s-ay2223s1/lib/python3.9/site-packages/torch/_tensor.py:363\u001B[0m, in \u001B[0;36mTensor.backward\u001B[0;34m(self, gradient, retain_graph, create_graph, inputs)\u001B[0m\n\u001B[1;32m 354\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m has_torch_function_unary(\u001B[38;5;28mself\u001B[39m):\n\u001B[1;32m 355\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m handle_torch_function(\n\u001B[1;32m 356\u001B[0m Tensor\u001B[38;5;241m.\u001B[39mbackward,\n\u001B[1;32m 357\u001B[0m (\u001B[38;5;28mself\u001B[39m,),\n\u001B[0;32m (...)\u001B[0m\n\u001B[1;32m 361\u001B[0m create_graph\u001B[38;5;241m=\u001B[39mcreate_graph,\n\u001B[1;32m 362\u001B[0m inputs\u001B[38;5;241m=\u001B[39minputs)\n\u001B[0;32m--> 363\u001B[0m \u001B[43mtorch\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mautograd\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mbackward\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mgradient\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mretain_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mcreate_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43minputs\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[43minputs\u001B[49m\u001B[43m)\u001B[49m\n",
"File \u001B[0;32m/opt/homebrew/anaconda3/envs/cs2109s-ay2223s1/lib/python3.9/site-packages/torch/autograd/__init__.py:173\u001B[0m, in \u001B[0;36mbackward\u001B[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001B[0m\n\u001B[1;32m 168\u001B[0m retain_graph \u001B[38;5;241m=\u001B[39m create_graph\n\u001B[1;32m 170\u001B[0m \u001B[38;5;66;03m# The reason we repeat same the comment below is that\u001B[39;00m\n\u001B[1;32m 171\u001B[0m \u001B[38;5;66;03m# some Python versions print out the first line of a multi-line function\u001B[39;00m\n\u001B[1;32m 172\u001B[0m \u001B[38;5;66;03m# calls in the traceback and some print out the last line\u001B[39;00m\n\u001B[0;32m--> 173\u001B[0m \u001B[43mVariable\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_execution_engine\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mrun_backward\u001B[49m\u001B[43m(\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;66;43;03m# Calls into the C++ engine to run the backward pass\u001B[39;49;00m\n\u001B[1;32m 174\u001B[0m \u001B[43m \u001B[49m\u001B[43mtensors\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mgrad_tensors_\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mretain_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mcreate_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43minputs\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 175\u001B[0m \u001B[43m \u001B[49m\u001B[43mallow_unreachable\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43;01mTrue\u001B[39;49;00m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43maccumulate_grad\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43;01mTrue\u001B[39;49;00m\u001B[43m)\u001B[49m\n",
"\u001B[0;31mRuntimeError\u001B[0m: element 0 of tensors does not require grad and does not have a grad_fn"
]
}
],
"source": [
"# This is a demonstration: You just need to run this cell without editing.\n",
"\n",
"# Set learning rate\n",
"learning_rate = 1e-6\n",
"\n",
"# Initialize weights to 0\n",
"a = torch.tensor(0., requires_grad=True)\n",
"b = torch.tensor(0., requires_grad=True)\n",
"c = torch.tensor(0., requires_grad=True)\n",
"d = torch.tensor(0., requires_grad=True)\n",
"\n",
"print('iter', 'loss', '\\n----', '----', sep='\\t')\n",
"for t in range(1, 5001):\n",
" # Forward pass: compute predicted y\n",
" y_pred = a + b * x + c * x ** 2 + d * x ** 3\n",
" if t == 1: print(y_pred.shape, y_true.shape)\n",
"\n",
" # Compute MAE loss\n",
" if t % 1000 == 0:\n",
" print(t, loss.item(), sep='\\t')\n",
"\n",
" # Automatically compute gradients\n",
" loss.backward()\n",
"\n",
" # Update weights using gradient descent\n",
" with torch.no_grad():\n",
" a -= learning_rate * a.grad\n",
" b -= learning_rate * b.grad\n",
" c -= learning_rate * c.grad\n",
" d -= learning_rate * d.grad\n",
" a.grad.zero_() # reset gradients !important\n",
" b.grad.zero_() # reset gradients !important\n",
" c.grad.zero_() # reset gradients !important\n",
" d.grad.zero_() # reset gradients !important\n",
" # What happens if you don't reset the gradients?\n",
"\n",
"# print fitted polynomial\n",
"equation = f'{a:.5f} + {b:.5f} x + {c:.5f} x^2 + {d:.5f} x^3'\n",
"\n",
"y_pred = a + b * x + c * x ** 2 + d * x ** 3\n",
"plt.plot(x, y_true, linestyle='solid', label='sin(x)')\n",
"plt.plot(x, y_pred.detach().numpy(), linestyle='dashed', label=f'{equation}')\n",
"plt.axis('equal')\n",
"plt.title('3rd degree poly fitted to sine (MAE loss)')\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "ca266605",
"metadata": {},
"source": [
"### Task 1.2 - Polyfit model\n",
"\n",
"We have demonstrated how to fit a degree-3 polynomial to a set of `x` and `y` points (following the sine curve), using two different types of loss functions (MSE and MAE). \n",
"\n",
"Now, your task is to write a function `polyfit` that takes in some arbitrary set of points. You are only allowed to use **ONE** loop for the backpropagation and weights update. You are **NOT** allowed to use a loop to raise the features to their respective powers.\n",
"1. `x`, corresponding x-values, \n",
"2. `y`, corresponding true y-values, \n",
"3. `loss_fn` to compute the loss, given the true `y` and predicted `y`, \n",
"4. `n` representing the $n$-degree polynomial, and \n",
"5. `lr` learning rate, and \n",
"6. `n_iter` for the number of times to iterate. \n",
"\n",
"Return the 1D tensor containing the coefficients of the $n$-degree polynomial , after fitting the model. \n",
"The coefficients should be arranged in ascending powers of $x$.\n",
"\n",
"For example,\n",
"```\n",
">>> y = torch.sine(x)\n",
">>> mse = lambda y_true, y_pred: torch.mean(torch.square(y_pred - y_true))\n",
">>> mae = lambda y_true, y_pred: torch.mean(torch.abs(y_pred - y_true))\n",
"\n",
">>> polyfit(x, y, mse, 3, 1e-3, 5000)\n",
"tensor([-4.2270e-09, 8.5167e-01, 1.2131e-08, -9.2587e-02], requires_grad=True))\n",
"\n",
">>> polyfit(x, y, mae, 3, 1e-3, 5000)\n",
"tensor([-9.6776e-07, 8.7905e-01, -2.4784e-06, -9.8377e-02], requires_grad=True))\n",
"```\n",
"\n",
"*Note: For this regression problem, initialize your weights to 0.0.*"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "c1f9a796",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-01T09:20:29.199401Z",
"start_time": "2024-04-01T09:20:28.547952Z"
}
},
"outputs": [
{
"data": {
"text/plain": "<Figure size 640x480 with 1 Axes>",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGxCAYAAABfrt1aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABm3UlEQVR4nO3dd3gU1dvG8e/uplfSaGn03quAUkSaiogIikhRQUEQy2sFpdiwKxaaKKggxZ+ChSaKFOm9g/RAKKGlkJ7d8/4RjUaKRAkTkvtzXXvJzp6deXbA7J0z55yxGWMMIiIiIhawW12AiIiIFF0KIiIiImIZBRERERGxjIKIiIiIWEZBRERERCyjICIiIiKWURARERERyyiIiIiIiGUURERERMQyCiJyzduyZQv33XcfZcuWxcvLCz8/P+rVq8cbb7zBmTNnrC4v3/Xp04cyZcpYXcZ/tnHjRlq0aEFgYCA2m4333nsvT+8/ePAgNpuNyZMn52wbMWIENpstV7uMjAz69+9PqVKlcDgc1KlTB4AzZ85w9913U7x4cWw2G7fffvt/+0D5aO7cuYwYMSJP73n++eeJiorCzc2NYsWKAdCyZUtatmyZ0yYlJYURI0awePHiK1aryD9xs7oAkf/i448/5uGHH6Zy5co89dRTVKtWjczMTNatW8e4ceNYuXIls2bNsrrMfPXCCy/w6KOPWl3Gf3b//feTnJzM9OnTCQoKuiLhqm/fvrRv3z7XtrFjxzJ+/Hg++OAD6tevj5+fHwAvvfQSs2bN4tNPP6V8+fIEBwf/5+Pnl7lz5/LRRx9ddhj59ttveeWVVxg6dCgdOnTA09MTgDFjxuRql5KSwsiRIwFyBRSR/KQgIteslStXMmDAANq0acPs2bNzfrgCtGnThv/7v/9j/vz5FlaYv1JSUvDx8aF8+fJWl3JFbNu2jX79+tGhQ4crts+IiAgiIiLOO463tzeDBg06b3v58uXp0aPHFTt+amoq3t7eV2x//9a2bdsAGDx4MMWLF8/ZXq1aNatKEvmTEblG3XrrrcbNzc3ExMRcVnun02lef/11U7lyZePh4WHCwsJMz549zeHDh3O1a9GihalevbpZsWKFadKkifHy8jLR0dHm008/NcYY88MPP5i6desab29vU6NGDTNv3rxc7x8+fLgBzIYNG0znzp2Nv7+/CQgIMD169DBxcXG52k6fPt20adPGlCxZ0nh5eZkqVaqYZ555xpw7dy5Xu969extfX1+zZcsW06ZNG+Pn52euu+66nNeio6NztZ85c6Zp1KiRCQgIMN7e3qZs2bLmvvvuy9Xm0KFDpkePHiYsLMx4eHiYKlWqmLfeess4nc6cNgcOHDCAefPNN83bb79typQpY3x9fc11111nVq5ceVnnfevWrea2224zxYoVM56enqZ27dpm8uTJOa9PmjTJAOc9LiU2NtZ07drV+Pn5mYCAANOtWzezcuVKA5hJkybltPvj7+IPFzrOxY7/yy+/GGOMSU9PNy+99FLOv5vQ0FDTp0+f8/4uo6OjzS233GK+/vprU6dOHePp6WmeeeYZY4wxx44dMw8++KAJDw837u7upkyZMmbEiBEmMzMzz+e6d+/eF6z3wIEDFzxX0dHR57UdPny4MSb733qLFi1yHf/vj969extjjImLizP9+vUzEREROeehadOmZuHChZf8uxL5Jwoick3KysoyPj4+pnHjxpf9ngcffNAAZtCgQWb+/Plm3LhxJiwszERGRpqTJ0/mtGvRooUJCQkxlStXNp988olZsGCBufXWWw1gRo4caWrWrGmmTZtm5s6da6677jrj6elpYmNjc97/x5dfdHS0eeqpp8yCBQvMO++8Y3x9fU3dunVNRkZGTtuXXnrJvPvuu2bOnDlm8eLFZty4caZs2bKmVatWuWrv3bt3zhfYqFGjzM8//2wWLFiQ89pfg8iKFSuMzWYzd999t5k7d65ZtGiRmTRpkunZs2dOm7i4OBMeHm7CwsLMuHHjzPz5882gQYMMYAYMGJDT7o8vpzJlypj27dub2bNnm9mzZ5uaNWuaoKAgEx8ff8lzvmvXLuPv72/Kly9vPv/8czNnzhzTvXt3A5jXX389p5Y/QsSdd95pVq5cecmQk5KSYqpWrWoCAwPNBx98YBYsWGAGDx5soqKi/jGIrFy50tx8883G29s75zjHjx83K1euNHXr1jXlypXL2Z6QkGCcTqdp37698fX1NSNHjjQLFy40EydONOHh4aZatWomJSUlZ9/R0dGmVKlSply5cubTTz81v/zyi1mzZo05duyYiYyMNNHR0Wb8+PHmp59+Mi+99JLx9PQ0ffr0yfO53rt3r7nzzjsNkFPrypUrTVpa2gXP14YNG8wDDzxgADN//nyzcuXKnPD91yCSlpZm5s+fbwDzwAMP5Ox37969xhhj2rVrZ8LCwsyECRPM4sWLzezZs82wYcPM9OnTL/lvQOSfKIjINen48eMGMHffffdltd+5c6cBzMMPP5xr++rVqw1ghgwZkrOtRYsWBjDr1q3L2Xb69GnjcDiMt7d3rtCxadMmA5j3338/Z9sfX36PP/54rmNNnTrVAGbKlCkXrNHlcpnMzEyzZMkSA5jNmzfnvPbHb8F/9Mr81d+DyFtvvWWAS4aEZ5991gBm9erVubYPGDDA2Gw2s3v3bmPMn1+ONWvWNFlZWTnt1qxZYwAzbdq0ix7DGGPuvvtu4+npeV6vVYcOHYyPj0+uGgEzcODAS+7PGGPGjh1rAPPtt9/m2t6vX79/DCLG/Nm79Hd/9IT91bRp0wxgvv7661zb165dawAzZsyYnG3R0dHG4XDknLs/PPTQQ8bPz88cOnQo1/Y//p62b99ujMnbuR44cOA/9hr91R/n4a+B+4/P/EcQMcaYkydP5uox+Ss/Pz/z2GOPXfYxRS6XZs1IkfDLL78A2TNM/qpRo0ZUrVqVn3/+Odf2UqVKUb9+/ZznwcHBFC9enDp16lC6dOmc7VWrVgXg0KFD5x3z72MNunXrhpubW04tAPv37+eee+6hZMmSOBwO3N3dadGiBQA7d+48b59dunT5x8/asGHDnOPNnDmT2NjY89osWrSIatWq0ahRo1zb+/TpgzGGRYsW5dp+yy234HA4cp7XqlULuPDn/vtxWrduTWRk5HnHSUlJYeXKlf/4ef7ul19+wd/fn9tuuy3X9nvuuSfP+/onP/zwA8WKFaNjx45kZWXlPOrUqUPJkiXPm11Sq1YtKlWqdN4+WrVqRenSpXPt44+xMEuWLMnV/t+e6/zWqFEjJk+ezMsvv8yqVavIzMy0tB4pPBRE5JoUGhqKj48PBw4cuKz2p0+fBrIDxt+VLl065/U/XGjGhIeHx3nbPTw8AEhLSzuvfcmSJXM9d3NzIyQkJOdY586d44YbbmD16tW8/PLLLF68mLVr1/LNN98A2QMd/8rHx4eAgIBLfk6A5s2bM3v2bLKysujVqxcRERHUqFGDadOm5bQ5ffr0Rc/FH6//VUhISK7nfwwM/nuNf5fX41yO06dPU6JEifO2//18XwknTpwgPj4eDw8P3N3dcz2OHz/OqVOncrW/0Gc9ceIE33///Xnvr169OsB5+/i35zq/zZgxg969ezNx4kSaNGlCcHAwvXr14vjx45bWJdc+zZqRa5LD4aB169bMmzePI0eOnDcz4u/++OF+7Nix89oePXqU0NDQK17j8ePHCQ8Pz3melZXF6dOnc2pZtGgRR48eZfHixTm9IADx8fEX3N/f18O4lE6dOtGpUyfS09NZtWoVo0aN4p577qFMmTI0adKEkJAQjh07dt77jh49CnDFzkd+HCckJIQ1a9actz0/vhBDQ0MJCQm56Owrf3//XM8v9HcUGhpKrVq1eOWVVy64j7/2sBVkoaGhvPfee7z33nvExMTw3Xff8eyzzxIXF1eoZ6dJ/lOPiFyznnvuOYwx9OvXj4yMjPNez8zM5PvvvwfgxhtvBGDKlCm52qxdu5adO3fSunXrK17f1KlTcz2fOXMmWVlZOesz/PGl9ddpxwDjx4+/YjV4enrSokULXn/9dSB70TCA1q1bs2PHDjZs2JCr/eeff47NZqNVq1ZX5PitW7fOCVx/P46Pjw/XXXddnvfZqlUrkpKS+O6773Jt//LLL/9TrRdy6623cvr0aZxOJw0aNDjvUbly5cvaxx9Tgy+0j38TRPKrl+Ry9xsVFcWgQYNo06bNef+GRPJKPSJyzWrSpAljx47l4Ycfpn79+gwYMIDq1auTmZnJxo0bmTBhAjVq1KBjx45UrlyZBx98kA8++AC73U6HDh04ePAgL7zwApGRkTz++ONXvL5vvvkGNzc32rRpw/bt23nhhReoXbs23bp1A6Bp06YEBQXRv39/hg8fjru7O1OnTmXz5s3/6bjDhg3jyJEjtG7dmoiICOLj4xk9enSu8SePP/44n3/+Obfccgsvvvgi0dHRzJkzhzFjxjBgwIDzxjn8W8OHD88ZIzFs2DCCg4OZOnUqc+bM4Y033iAwMDDP++zVqxfvvvsuvXr14pVXXqFixYrMnTuXBQsWXJGa/+ruu+9m6tSp3HzzzTz66KM0atQId3d3jhw5wi+//EKnTp3o3LnzJffx4osvsnDhQpo2bcrgwYOpXLkyaWlpHDx4kLlz5zJu3Lh/7NH7u5o1awLw+uuv06FDBxwOB7Vq1cq5VPhv+fv7Ex0dzbfffkvr1q0JDg4mNDSUoKAgWrVqxT333EOVKlXw9/dn7dq1zJ8/nzvuuOM/HVNEs2bkmrdp0ybTu3dvExUVZTw8PHKmyQ4bNizXWg9/rCNSqVIl4+7ubkJDQ82999570XVE/u6PdSL+jr/N9vhjhsL69etNx44djZ+fn/H39zfdu3c3J06cyPXeP9Yq8fHxMWFhYaZv375mw4YN583+uNhMjz9e++usmR9++MF06NDBhIeHGw8PD1O8eHFz8803m2XLluV636FDh8w999xjQkJCjLu7u6lcubJ58803L7qOyIU+94VmV/zd1q1bTceOHU1gYKDx8PAwtWvXzvXZ/rq/y5k1Y4wxR44cMV26dMk5t126dDErVqy44rNmjDEmMzPTvPXWW6Z27drGy8vL+Pn5mSpVqpiHHnrI7NmzJ6fdxf59GJM9G2Xw4MGmbNmyxt3d3QQHB5v69euboUOH5qwZk5dznZ6ebvr27WvCwsKMzWa75Doifz0P/zRrxhhjfvrpJ1O3bl3j6emZs45IWlqa6d+/v6lVq1bO2jSVK1c2w4cPN8nJyRc9rsjlsBljjCUJSKSQGjFiBCNHjuTkyZP5MvZERKQw0RgRERERsYyCiIiIiFhGl2ZERETEMuoREREREcsoiIiIiIhlFERERETEMgV6QTOXy8XRo0fx9/fP0/LWIiIiYh1jDElJSZQuXRq7/dJ9HgU6iBw9evS8u3aKiIjIteHw4cP/uHJwgQ4if9xQ6vDhw5d111ERERGxXmJiIpGRkefdGPJCCnQQ+eNyTEBAgIKIiIjINeZyhlVosKqIiIhYRkFERERELKMgIiIiIpYp0GNERERE/gtjDFlZWTidTqtLKXTc3d1xOBz/eT8KIiIiUihlZGRw7NgxUlJSrC6lULLZbERERODn5/ef9qMgIiIihY7L5eLAgQM4HA5Kly6Nh4eHFsa8gowxnDx5kiNHjlCxYsX/1DOiICIiIoVORkYGLpeLyMhIfHx8rC6nUAoLC+PgwYNkZmb+pyCiwaoiIlJo/dPy4vLvXakeJv0NiYiIiGUURERERMQyCiIiIiLXiD59+nD77bfn+X2LFi2iSpUquFyuy2q/detWIiIiSE5OzvOx8kpBRERE5BoxevRoJk+enOf3Pf300wwdOvSyx8zUrFmTRo0a8e677+b5WHmlICIiInKNCAwMpFixYnl6z4oVK9izZw9du3bN0/vuu+8+xo4dm++LwSmIiIhIkWCMISUjy5KHMSZPtf7vf/+jZs2aeHt7ExISwk033URycvJ5l2ZatmzJ4MGDefrppwkODqZkyZKMGDEi176mT59O27Zt8fLyyjkPN910E+3bt8+pKz4+nqioKIYOHZrzvnbt2nH69GmWLFny7074ZdI6IiIiUiSkZjqpNmyBJcfe8WI7fDwu7yv32LFjdO/enTfeeIPOnTuTlJTEsmXLLhpmPvvsM5544glWr17NypUr6dOnD82aNaNNmzYALF26lO7du+e0t9lsfPbZZ9SsWZP333+fRx99lP79+1OiRIlcIcbDw4PatWuzbNkybrzxxn//4f+BgoiIiEgBcuzYMbKysrjjjjuIjo4GssdsXEytWrUYPnw4ABUrVuTDDz/k559/zgkiBw8epHTp0rneEx4ezvjx4+nZsycnTpzg+++/Z+PGjbi7u5/X7uDBg1fw051PQURERIoEb3cHO15sZ9mxL1ft2rVp3bo1NWvWpF27drRt25Y777yToKCgC7avVatWruelSpUiLi4u53lqamrOZZm/6tq1K7NmzWLUqFGMHTuWSpUqnV+3t3e+36tHQURERIoEm8122ZdHrORwOFi4cCErVqzgxx9/5IMPPmDo0KGsXr36gu3/3oths9lyTdMNDQ3l7Nmz570vJSWF9evX43A42LNnzwX3febMGcqXL/8fPs0/02BVERGRAsZms9GsWTNGjhzJxo0b8fDwYNasWf9qX3Xr1mXHjh3nbf+///s/7HY78+bN4/3332fRokXntdm2bRt169b9V8e9XAoiIiIiBcjq1at59dVXWbduHTExMXzzzTecPHmSqlWr/qv9tWvXjl9//TXXtjlz5vDpp58ydepU2rRpw7PPPkvv3r1z9ZwcPHiQ2NhYbrrppv/0ef6JgoiIiEgBEhAQwNKlS7n55pupVKkSzz//PG+//TYdOnT4V/u799572bFjB7t37wbg5MmTPPDAA4wYMYJ69eoBMHz4cEqXLk3//v1z3jdt2jTatm2bM2A2v9hMXic3X0WJiYkEBgaSkJBAQECA1eWIiMg1Ii0tjQMHDlC2bNkLDtQsap5++mkSEhIYP378ZbVPT0+nYsWKTJs2jWbNml2wzaXOcV6+v9UjIiIiUsgNHTqU6Ojoy14l9dChQwwdOvSiIeRKKvjDh0VEROQ/CQwMZMiQIZfdvlKlSheczpsf1CMiIiIillEQEREREcsoiIiIiIhlFERERETEMgoiIiIiYhkFEREREbGMgoiIiIhYRkFERERELKMgIiIiIpZREBERERHL5GsQGTVqFA0bNsTf35/ixYtz++2359z9T0RExBIZyRd/ZKbloW3q5bXNo5YtW/LII4/w2GOPERQURIkSJZgwYQLJycncd999+Pv7U758eebNmwfA2bNn6dGjB2FhYXh7e1OxYkUmTZqUs7/Y2FjuuusugoKCCAkJoVOnThw8eDDPdeWXfL3XzJIlSxg4cCANGzYkKyuLoUOH0rZtW3bs2IGvr29+HlpEROTCXi198dcqtoUeX/35/M0KkJly4bbR18N9c/58/l5NSDl9frsRCXku8bPPPuPpp59mzZo1zJgxgwEDBjB79mw6d+7MkCFDePfdd+nZsycxMTG88MIL7Nixg3nz5hEaGsrevXtJTc0OSSkpKbRq1YobbriBpUuX4ubmxssvv0z79u3ZsmULHh4eea7tSsvXIDJ//vxczydNmkTx4sVZv349zZs3z89Di4iIXLNq167N888/D8Bzzz3Ha6+9RmhoKP369QNg2LBhjB07li1bthATE0PdunVp0KABAGXKlMnZz/Tp07Hb7UycOBGbzQZkfxcXK1aMxYsX07Zt26v7wS7gqt59NyEhOxUGBwdf8PX09HTS09NznicmJl6VukREpAgZcvTir9kcuZ8/tfcSbf82uuGxrf++pr+pVatWzp8dDgchISHUrFkzZ1uJEiUAiIuLY8CAAXTp0oUNGzbQtm1bbr/9dpo2bQrA+vXr2bt3L/7+/rn2n5aWxr59+65Yvf/FVQsixhieeOIJrr/+emrUqHHBNqNGjWLkyJFXqyQRESmKPPIwNCC/2v4Dd3f3XM9tNluubX/0brhcLjp06MChQ4eYM2cOP/30E61bt2bgwIG89dZbuFwu6tevz9SpU887RlhY2BWr97+4arNmBg0axJYtW5g2bdpF2zz33HMkJCTkPA4fPny1yhMREblmhYWF0adPH6ZMmcJ7773HhAkTAKhXrx579uyhePHiVKhQIdcjMDDQ4qqzXZUg8sgjj/Ddd9/xyy+/EBERcdF2np6eBAQE5HqIiIjIxQ0bNoxvv/2WvXv3sn37dn744QeqVq0KQI8ePQgNDaVTp04sW7aMAwcOsGTJEh599FGOHDliceXZ8jWIGGMYNGgQ33zzDYsWLaJs2bL5eTgREZEix8PDg+eee45atWrRvHlzHA4H06dPB8DHx4elS5cSFRXFHXfcQdWqVbn//vtJTU0tML/s24wxJr92/vDDD/Pll1/y7bffUrly5ZztgYGBeHt7/+P7ExMTCQwMJCEhocCcMBERKfjS0tI4cOAAZcuWxcvLy+pyCqVLneO8fH/na4/I2LFjSUhIoGXLlpQqVSrnMWPGjPw8rIiIiFwj8nXWTD52toiIiEghoHvNiIiIiGUURERERMQyCiIiIiJiGQURERERsYyCiIiIiFhGQUREREQsoyAiIiIillEQEREREcsoiIiIiIhlFERERETEMgoiIiJSpKRkplz0ke5Mv+y2aVlpl9U2r1q2bMkjjzzCY489RlBQECVKlGDChAkkJydz33334e/vT/ny5Zk3bx4ATqeTBx54gLJly+Lt7U3lypUZPXr0efudNGkSVatWxcvLiypVqjBmzJg815Yf8vVeMyIiIgVN4y8bX/S1G8JvYMxNf35Bt5zZktSs1Au2bVCiAZPaT8p53v7r9pxNP3teu629t+a5xs8++4ynn36aNWvWMGPGDAYMGMDs2bPp3LkzQ4YM4d1336Vnz57ExMTg7u5OREQEM2fOJDQ0lBUrVvDggw9SqlQpunXrBsDHH3/M8OHD+fDDD6lbty4bN26kX79++Pr60rt37zzXdyWpR0RERKSAqV27Ns8//zwVK1bkueeew9vbm9DQUPr160fFihUZNmwYp0+fZsuWLbi7uzNy5EgaNmxI2bJl6dGjB3369GHmzJk5+3vppZd4++23ueOOOyhbtix33HEHjz/+OOPHj7fwU2ZTj4iIiBQpq+9ZfdHXHHZHrueLuy2+aFu7Lffv8vO7zP9Pdf1VrVq1/qzJ4SAkJISaNWvmbCtRogQAcXFxAIwbN46JEydy6NAhUlNTycjIoE6dOgCcPHmSw4cP88ADD9CvX7+cfWRlZREYGHjFav63FERERKRI8XH3sbztP3F3d8/13Gaz5dpms9kAcLlczJw5k8cff5y3336bJk2a4O/vz5tvvsnq1atz2kD25ZnGjXNflnI4cgcvKyiIiIiIXMOWLVtG06ZNefjhh3O27du3L+fPJUqUIDw8nP3799OjRw8rSrwkBREREZFrWIUKFfj8889ZsGABZcuW5YsvvmDt2rWULVs2p82IESMYPHgwAQEBdOjQgfT0dNatW8fZs2d54oknLKxeg1VFRESuaf379+eOO+7grrvuonHjxpw+fTpX7whA3759mThxIpMnT6ZmzZq0aNGCyZMn5worVrEZY4zVRVxMYmIigYGBJCQkEBAQYHU5IiJyjUhLS+PAgQOULVsWLy8vq8splC51jvPy/a0eEREREbGMgoiIiIhYRkFERERELKMgIiIiIpZREBERkUKrAM/HuOZdqXOrICIiIoXOH6uQpqTk/e63cnkyMjKA/746qxY0ExGRQsfhcFCsWLGce7H4+PjkLIsu/53L5eLkyZP4+Pjg5vbfooSCiIiIFEolS5YE/rwxnFxZdrudqKio/xzwFERERKRQstlslCpViuLFi5OZmWl1OYWOh4cHdvt/H+GhICIiIoWaw+EoEHeZlQvTYFURERGxjIKIiIiIWEZBRERERCyjICIiIiKWURARERERyyiIiIiIiGUURERERMQyCiIiIiJiGQURERERsYyCiIiIiFhGQUREREQsoyAiIiIillEQEREREcsoiIiIiIhlFERERETEMgoiIiIiYhkFEREREbGMgoiIiIhYRkFERERELKMgIiIiIpZREBERERHLKIiIiIiIZRRERERExDIKIiIiImIZBRERERGxjIKIiIiIWEZBRERERCyjICIiIiKWURARERERyyiIiIiIiGUURERERMQyCiIiIiJimXwNIkuXLqVjx46ULl0am83G7Nmz8/NwIiIico3J1yCSnJxM7dq1+fDDD/PzMCIiUsAYY0jLSiMhPYFTqac4eu4o6c70nNedLifGGAsrlILCLT933qFDBzp06JCfhxARkXyWmuEkITWTpLRMEtMySUzL4lxGEnGphyntUx6HzR1vDwdrTs1h8bHvSMqMJyHjLFmurFz7mXLzFGqH1c7+884pvLf+PYK8ggjxDiHEK4QwnzCiA6KJDoimQYkGBHkFWfFx5SrL1yCSV+np6aSn/5mYExMTLaxGRKToSEjJZNfxRH47kcTeuHPExqcSG5/G0fhUEtITcPjE4PA6gt3rKA7Po9g94gFI3v8YPukBRNvisIUs5VDYngvu34Eb5rO7yHRm4nC4k+HnTpY3nEw9ycnUk+e1/7TWYzQs2xYCwtl+Zie7zuyiQckGRPlHYbPZ8vNUyFVWoILIqFGjGDlypNVliIgUaulZTrbFJrL+0BnWHTzL5iPxnEhMv2Bb9+Bl+EXPxWY7/zKKp8uPj7xfprntDACH0tw4fNyNYKeTYKeLqZk380FGV8BBBVssdTyfznlvr2ToaHdw2mHntMPBySo3E1emETGJMcSc3UOZH54EpwscniwoHcUkt+z6SnmF0Kh0U5qGX0/ziOb4efhd+RMkV5XNXKWLdDabjVmzZnH77bdftM2FekQiIyNJSEggICDgKlQpIlL4GGPYE3eOxbvjWPLbSdYePEtGluuvLbB7niA4dB/u/rtoHNSDBiUaEOkPp09+w/BDnxJt3KidkkSV8KZUafwIlYMrE5B2Dt6pmr0LnxAIKgtB0Rjf4qR6BHMiqC4HfGoRG5/G4ROnSTu6nf1nMjh7Lg13snAnC09bJsU4xzG3CILK1+eGimHcFHSc0gsfxhYfA65Mvvbz5Tt/X7Z4epL1l94Qd7s715VqzKvNXqGYd/DVPalySYmJiQQGBl7W93eBCiJ/l5cPIiIif3K5DOsOnWXOlqMs3HGCowlpuV4P9nWnSlQy7oGbiUlfwam0Yzmv9SpWg6fOJMLhVaQZJ+fsNkKdvweXardDt8+y/2wMHN+SHUC8Lv9ndGJaJlsOJ7Ax5iwbD8ezIeYs8SmZudpEh/hwc/Xi3F4mi0rmALZjm0k5uoGNp7exqlo7Fqcc5mDiQYp7BLIwNg575ZuhakdiQssQHlAGh93xr86bXBkKIiIiRZAxhi1HEpi9KZa5W4/lutzi6WbnunIhtKgURvUow6gNT7AvYV/O6x52DxqVakSL0s1o8d2zlEpLyn4hIByim2Y/oppCaCWwX9kJly6XYcexRJbtOcXS306y7tAZMp1/fjVFBHlza63S3Fk/ggqhPmBc4HBjf/x+jq4czfWrJwOQCbSJisDd3Zvu5TrRpf5gAr0Cr2itcnkKTBA5d+4ce/fuBaBu3bq88847tGrViuDgYKKiov7x/QoiIiL/LCElk9mbYpm+9jA7j/05yN/fy4221UrSvkYo4cWTqRZaCQCXy0nbmTdyJj2e5ulZtHd50vz+Zfh4+Ga/cemb4BkIFW/K7u24yoNDk9Oz+GV3HPO2HmfRrjhSM505r9WLKkbXBpHcWqsU/l7ukJUBB5fCzu/Zu2cuvYu5k+jI7g3xNnB7uVvpWedhIgMir+pnKOoKTBBZvHgxrVq1Om977969mTx58j++X0FEROTitsUm8OnyA8zZcoz038d8eLjZaV+9JLfVLk3F8Exm7/uaWXtmYTD81H4K7ltmwsYpbD93hKjMTPyNAc8AGLgaAkpb/InOl5rh5JfdcXyz4Qi/7D6J05X9leXr4eDO+hH0aVaWsqG/ByiXk/T9i5i3/iO+SNzNbx7Z8zEcNge3lb+NAVV7USq4glUfpUgpMEHkv1IQERHJzeUy/LI7jo+X7WfV/jM52yuX8OfuRpF0rhvOkZTf+Gz7Zyw8tBCnye5NCHZ480nMQSpk/H65xt0HKneAGl2gfGtw97Li4+RJXGIaszbGMmPdYfafTAayO2taVS7O/c3K0qxCSM7UXpMaz+o93zM5bjnLY5cDMDUBapWoAy2egZI1rPoYRYKCiIhIIZPpdDFrYyzjl+xj3+9fwg67jVtrlaJ30zLUjSzGjtM7eGPtG2yI25DzvsYlG3NXlbtome7C/ctuEHkd1OsF1W+HPy7FXGOMMSzfe5pPlx9g0a64nO01wgN45MaKtKlaArv9z8tJm09uZvn2aTy8eGzOtrWVWlGjxfN4hze4qrUXFQoiIiKFRNbvAeSDRXuJOZMCgL+nG90bR9GnaRlKF/POabv7zG7u/P5O3LDRITmNXmXaU+Xm97NfdLng9B4Iq2zFx8g3+0+e47MVB5m57kjOWJIqJf0Z3Loi7auXzBVIiNsJS97g2K7v6BhRkiCXiyf8qtG+3XvYimkMyZWkICIico1zugzfbY5l9E97OHg6O4CE+HrwYPNy3NM4Ch8PO/MPzudY8jH61uwLsRtg2dv8L3YxN6SkUsLphOhm0GfOVR9saoUzyRl88ut+PltxiHPp2UvLVy7hzzMdKtOqcvFcq7Fu2v0tT69+kWMmA4Bmqem8cMtkwiObWlJ7YaQgIiJyDft1zylembszZwZM8O8BpFeTaLzdHSw4tICPNn7EwcSDuNkczLNFU3Lf4j93ULEtXPcwlGtZJELIX8WnZPDp8oNMWn6ApLTsQNK4bDBDbq5K7chiOe3SstKYvPJVPt43iwwbeLt5MajOI/So2kNrkFwBCiIiItegvXFJvDp3V864B38vN/q3KE+fpmXw9XRj7fG1vLPuHbad3gZAgEcAvRxh9Ni5GD/sUKsbXP94obv88m8kpGQyZsleJi0/mLOK7C21SvFs+ypEBvvktDuYcICRy4ez7uRGAOqF1uSTND/cWr8AulzzrymIiIhcQxJSM3nnx91MWR2D02Vws9u497poBreuSLCvB7HnYnll1Sssi10GgLfDi/tq3E/Paj3xSzkLS9/IDiDB5Sz+JAVPbHwq7/z4G99sPIIx2Qu7DWhZnv4tyuPlnt3z4TIuvtnzDW+ve5sejlAG7VwK7r7QZiQ07FvkepWuBAUREZFrgDGG2ZtieWXOTk6dyx6v0KZaCZ7rUIVyYX/ezO1U6ilu+boDGc40uiQm0T+kIaH3fGVV2dekHUcTeXnODlbsOw1AVLAPwztWo3XVEjltjicfJyTpFO5zn4SYlRxxcxAQdT0Bt48rkGusFGQKIiIiBdzeuHO8MHsbK/dnfzGWC/PlpU41aFYhlExXJr8e+ZVWUa0g6TgseZ2FO2dQKT2d6Cwn1LoLOn0IDneLP8W1xRjDD1uO8cqcnRxPzL73zk1VizO8Y/Vcl2twuchcPZYe2z4gyQbvxqdRpd1bUPNOiyq/9iiIiIgUUOlZTj5atJexS/aR6TR4utl55MYK9GteDk83B6uPrWbU6lHsS9jHhJJtaLJuOmRmrxtCxXbQepgW4/qPktOzeH/RHj5ZdoAsl8HHw8HT7SrTq0mZnOm+sedieWBuL2JT4/B0uXjh9Fk6NR8JjfpZXP21QUFERKQA2nIknqe+2sLuE9k3lLuxSnFG3pb92/jx5OO8ufZNfjz0IwBBDi9eOHqYNimpEN4A2rwIZZpZWX6hszcuiSHfbGPNwewVahuWCeL1LrVyLoslpCfw3NJnWXb0VwC6lbuNZ5uNwN2unqh/oiAiIlKApGc5ef/nPYxbsh+nyxDi68FLt9egQ42SuIyLabum8f7G90nNSsVus3NX5bsYWKMvgTN6Q4P7oGZXDZjMJy6XYerqQ7w2bxfJGU483Ow8flMl+t1QFjeHHZdxMWHLBMZsGoPB0LhUY95u8TaBp/ZBeD2ryy+wFERERAqIrUcS+L+vNvHbiXMA3FqrFCNvq06InycAjy56lEWHFwFQz+lgyG1fUjm0mmX1FlVHzqYwZNY2lv52Esi+y++7d9UhOiR7GfylR5by1JKnSMlKoU1gJd7Z9FP2PWtaPAt2u5WlF0h5+f7W2RMRyQcul2Hs4n10HrOc306cI9TPg7E96vHhPfVyQghAR8+S+LkML5w6w6SYA1Q+c9jCqouuiCAfPruvIW91rY2/lxsbYuK5efQyZq47jDGG5hHN+bzD59QIqcGT3r/fwXfJ6/D1/ZCRYm3x1zj1iIiIXGHHE9J4YuamnKmiHWqU5JXONQn29WDt8bUkZSRxY2AlmPs07J5DvN1OscAouOVtqHCTxdXLkbMpPDFzM2sOZI8d6VCjJK92rkmQrwfGmOzl4jd8AT88zgmbixIlasHd0yCglMWVFxy6NCMiYpEF24/zzNdbiE/JxNvdwYjbqtGtQSTpznRGbxjNlJ1TKGb34tsjRwlOPwd2N2j2KDR/Cty9//kAclU4XYbxS/fxzo+/keUylAjw5J1udWhWITSnzcK17/PM9gmMPHWajrZicO/XUEKX1UCXZkRErrq0TCdDZm3loS/WE5+SSY3wAH4YfD13NYxi66mtdP2+K1N2TgGgdZYdj4xkiLwO+v+aPSVXIaRAcdhtPNyyArMebka5MF9OJKZz7yereWfhbzhd2b+/L888Q6bNxpCwUL6wJcGkDpB8yuLKrz3qERER+Y8OnU5mwJQN7Pj9JnUPNS/H/7WtDLYsxm0exyfbPsFlXBT3Ls7IZiO53j0EDq+Gur000PEakJKRxUs/7GDamuzxO80qhPDeXXUJ8XPnzbVv5gTMR4Pr07fjZAsrLTh0aUZE5Cr5cftx/u+rzSSlZRHs68F7d9WheaUw0rLSuHfuvew+uxuAW7zCee72GQR6BlpcsfxbszfGMmTWVlIynIT5e/JB97o0LhvMuC3jGLNpDACP1nuUvjX7gjOzSK98m5fvb7erVJOISKGS5XTx1o+/MW7JPiB7uudHPepRKjD7EouXzY06WYY4p5MXTp2hDcngKrC/98lluL1uODXCAxgwZQN74s5xz8er+L+2lRnQoj927Hy46UNGbxgNWen0XfsV1L03+6Z5ckkKIiIieRSXlMbgaRtZtT97VsV9zcrwXIeqJGae4WTKOcKSTsLsATxxfDMDbDZCKraHW98F72LWFi7/WYXi/nw7qBnPz9rGNxtjeXPBbjYdjuedbvcD8OGmDzl+eAXm6EZsRzeCMVoW/h/o0oyISB5sOhzPg5+vIy4pHV8PB6/fWYtba5Vm2ZFlPL/8eSrYvZmwawMOZzp4FYMOb0CtbloZtZAxxjBj7WGGfbudDKeLCsX9+LhXA2LTN9KsVFNsPw2DFR9kN775rSIXRnRpRkQkH8zaeIRnvt5KRpaLisX9GHtvfaJCPHh9zes5AxbPZmZx1mQSWrEd3PY++Je0uGrJDzabjbsbRVG5pD/9p6xnb9w5bvvwVz7oXheb3Q5tXiLLGLZs+Jh6c5/MflMRCyOXS8O1RUT+gdNlGDV3J4/P2ExGloubqpZg1sBm2D3j6DG3R04I6VG1B9MaDie0w5twzwyFkCKgblQQ3w+6nnpRxUhKy+K+yWsZu3gf6c4MHndL5P7SJVns7Q1zn4R1k6wut0DSpRkRkUtITMvk0Wkb+WV39j1IBrWqwBNtKjHv4FxGrhxJalYqQW6+vNT8dVpEtrC4WrFKepaTEd9tz5nie0vNEvhFfM3cgz/giZ3xR49R3y0QHlkPXoX/+0wLmomIXAEHTiXT+aPl/LL7JJ5udt7vXpcn21Umy2QyfsNoUrNSaZyaxtcnEmhRsrHV5YqFPN0cvNq5Ji/dXgM3u405W0+wc2t7mpS8gXRcDAqP4Lc7xhSJEJJXCiIiIhewYt8pbv9oOftOJlMywIuv+jfhttqlwRg81kzk7T1bGHA2gfHJboTdPg7cvawuWSxms9noeV00U/s2ppiPO1uOnGPbxtuoGlSbcyaTQRvf5FTq7yuvZqVbW2wBoiAiIvI3X68/Qu9P15CQmkmdyGJ8N6gZp1wbmLF1EkzvAQueo2J6Kg+Xao5jwHIop0sy8qfG5UKY9XAzyob6cjTeyY6NXSjuFcGx5GM88vMjpP42H96vB3E7rS61QNAYERGR3xljeP/nvbz7028A3FKrFK93qcb4rR8xeftk3AxMOXqM6k4btHs1e7EqTcuVi4hPyeChL9az+sAZ3DxPE1xxPNgy+DizGLUProHAKOj7E/iXsLrUK05jRERE8igjy8VT/9uSE0IealGOF24LZ+Cih5i8fTIA3T1KUsk3HB5YmD0VUyFELqGYjwdfPNCYLvUiyEoP4fS+e7jedzg17pwOweUhIQam3Q0ZKVaXaimtIyIiRV5iWiYPT9nAr3tPYbfBi51qUKPcWbrP6captDP4ufvxYrMXaVOyCRiXBhzKZfNws/NW11qUC/PlzQXwzSpITjrA+3fNwD65Le5HN8A3/aDb52B3WF2uJdQjIiJFWmx8KneOXcGve0/h4+Hgk94N8Qpey33z+3Aq7QwVbJ5Mv2UabaLbgKefQojkmc1mY2CrCnx4T108HHYWbD/BHd+u4tboaLZ7+8CuH+DHF6wu0zIKIiJSZO08lkjnj5bz24lzFPf3ZOZDTWhVOYzkfT+TZZy0SU5hanwW0XZvq0uVQuDWWqX5/IFG+Hu5sS99LkfTTzM4ohxn7HZY9RFsn211iZZQEBGRImnNgTN0G7+SuKR0KpXwY9bAZtQIc4dZD9Fr9ZeMPnGSt0Oa4fPg0kI5mFCscV25EL7q34TAcz1wpYcSl5XIYxXqkVWvN1S+2eryLKEgIiJFzsIdJ+j5yWqS0rJoWCaIYV18Gb68L+c+uQm2zMBmc3Dj9c9j6/aZLsXIFVelZADfDGhNWOpDGKcHGzPjeMa3OLh5WF2aJRRERKRImbnuMP2nrCc9y8VNVYvTuflhHlnSj9WntjAmMxZ8w6D3d9B0kGbFSL4JL+bN7H53UDrrPgB+PDKD15fNAGcWrBpXpBY80zoiIlJkjFuyj9fm7QLgjnolCQj/nq/3fg1Am+INePnECXzumAgBpa0sU4qQtEwnt305hGPMxTg9mZgVyXVHF0HdnnDbB9dsGM7L97em74pIoedyGUbN28nHyw4A0PuGUA7xLgv3bsaGjcH1BvNAjQewwTX7g1+uTV7uDr675yXazthLXKIn75+oRCO3xdg3fgHh9aDB/VaXmO90aUZECrVMp4sn/7c5J4Q8fJM/a5KHsP7UZnxdLj6s+yR9a/bFZrMphIglvNw9mNvtM3qXf4EVWQ15PfMuAMy8Z+HYFoury38KIiJSaKVnOXl46ga+2RCLw27j7a616RNygIykY4RnZjHlnIPmxetZXaYIfh5+PHdzVZ5uX5nxzluYYauNzZmO+aoPpCVaXV6+UhARkUIpNcNJ38/WsXDHCTzc7Ey4ty5dEiZT/NvBjDsexzSPilTouwRK1rC6VJEcvZuVokHDObwcmcZGexi2M/twfTcYCu5wzv9MQURECp2ktEx6f7qGZXtO4eMBbZr+TMaG+2HpmwBUbPAQQT2+Ae8giysVyc3d7o7D8yw4UukfVo504yBz51zST+y2urR8oyAiIoVKfEoG905czZqDZ/D3Sad6/RksPbmA4ZkxnHH3gtvHQbtXwKGx+lLweDg8eLP5m3i7eZPic4ye/m3omPYi930fz7n0LKvLyxcKIiJSaJxMSufuCavYfCSBoMCzlKgygV3xG/F19+WtgNoE954LdbpbXabIJUUFRPFkgycB2FN8D7E+XqzYd5p7J64mISXT4uquvCIbRLaf3s7Lq17G6XJaXYqIXAFH41O5a/xKdh1PIiTkAJ4R73EiNZZwv3CmdJhC8y5fQkR9q8sUuSxdK3Xl+vDryTKZlK32HYE+NuxH1jD7gyc4m5xhdXlXVJEMImlZaTy08CFm7J7B/IPzrS5HRP6jQ6eT6TpuJftPJVOy1BqcxceT7EqnniOAL2+eSoWgClaXKJInNpuNF5u+SDHPYhw6t4cuTVcy0/Mleqd+zttjPuTUucKz8mqRDCJebl70rtYbgHGbx5HlKpzX3USKgr1xSXQbv5LY+FQaBKfRxXceTuC2c8l8XO4egr2CrS5R5F8J8wnjheteoJRvKdpX78i5mtnfW4PPjebBsfM5kZhmcYVXRpFd4j05M5n2X7cnPj2eV69/lY7lO17R/YtI/ttxNJF7P1nNmeQMbgk9wfu8ju3ccRYWC6Vtx4nYyreyukSR/yw1KxVvN2/ITCVjbHM8zvzGAmcDRvkP5csHm1C6mLfVJZ4nL9/fRbJHBMDX3Zc+1fsA6hURuRZti03gnomrOJt2hvplJvFO+lAc545jD6tCu14LFUKk0PB2+z1ouHuT1fkjjN2ddo51NEqYS7fxKzl8JsXaAv+jIhtEALpX6U6QZxAxSTH8sP8Hq8sRkcu05Ug893y8isSsWIIrjOE37928EeAJFW6CB36E4HJWlyhyRRljmLFrBm1+fZxdzfoDMNx9Cs6zR7hr/EoOnkq2uMJ/r0gHER83b+6vkX1DofGbx5PpKnzTokQKm02H4+kxcTXJ9t34lxtHhv0MEV6h3FvmVug+A7wCrS5R5Iqz2WysPr6axIxEhiXvIjOiAb6k8nDAMo4mpNFt/Er2xp2zusx/pWgGEZcTNnwO42+gW5kOVA2uSs9qPaHAjpYREYANMWfpOXE1No+l+EZ9gsuWQu2w2kzt9DVlO36oRcqkUBvSeAgBHgHsOrubz2q0gY7v037QB1Qu4U9cUjp3T1jJ7uNJVpeZZ0UziBgXLB8Nx7fis3oCM26dwT1V78Hd4W51ZSJyEesPnaHXJ6vxCZiJCf8OY3PRtmQTJradqJkxUiSEeofyTKNnABi75ysOVGhOWIAX0x68jmqlAjh1LoO7J6xkW2yCxZXmTdEMIg53uGlE9p9XfoQt6bil5YjIpa09eIZen6yhqlmJI2gVAPenwZt1HsXLzcvi6kSuno7lOtIsvBkZrgxGrhyJMYZgRyr/q7+N2uEBnE3JpMfE1ddUGCmaQQSgyq0Q2RiyUmHxq7iMi3kH5tFrXi9SMq/tEcgihcnq/afp/ekaOjoXMs3+AaPjTjIsK4DHeyzCXqK61eWJXFU2m41h1w3D282b9SfW88OeWTC+BT4/Pcu0poepG1WMhNRrK4wU3SBis0Gbl7L/vHEKzhPbeX/D+2yM28gXO76wtjYRAWDlvtPc/8VcunmM4TX3ibjhpGHF2+ja62fwC7O6PBFLlPYrzUO1HsJhc3A09STU6QGAz8/P83n3CrnCyNYjBT+MFN0gAhDVOLtnxLhwX/QKg+sNBmDS9kmcSTtjcXEiRdvyvae4f9r/8I54lwXh+9jr7g4th8AdH4O7LsdI0darWi++6vgVD9V+CK5/DIpXg5TT+C99kc/vb0S9nDCyii1H4q0u95KKdhCB7LEiNgf8No92PtFUDa5KcmYyH2/52OrKRIqspb+d5IGvJuMIH0+6WyYRdm8C2r8JLZ/J7s0UKeLcHe5UDKqY/cThDre+l/3njVPwP7GWz+5vRP3oIBLTsrh34uoCHUYUREIrQrtX4IGfsJeoxmP1HwNgxu4ZxJ6LtbY2kSJo8e44hn77Im6lPsdmz6RZ6ev5rPsSite/z+rSRAqkPWf38GLsApx1e2Vv+P4x/N0Mn93fiAa/h5EeE1ez+XC8pXVejIIIwHUDILIhAE1LN6VxqcZkujL5aONHFhcmUrT8vPM4b8zrT3zxxdhshjsqdOHD1h/g4+FrdWkiBVJaVhp9f+zLV799xfQytcA3DE7thlUf4efpxuT7G9GwTBBJaVnc+8lqNhXAMKIg8ndnD/J4zQEA/LD/B2ISYywuSKRo+Gn7cab/8CCHg/YAMNgWyojGQ3Cza5EykYvxcvPi4doPA/DRjkmcbf0C1OyWM4DVz9ONSfc1olGZYJLSsug5cTUbY85aWfJ5FET+auUY+LAR1Xf/xIDaA5jQdgJRAVFWVyVS6P209TBJM/oyJnkJ16ek8kpAHfrd+xM2Nw+rSxMp8O6sdCeVgyqTlJHERxmHocvH4Fc85/XsMNKQRmWDSUrPotcna9hQgMLIVQkiY8aMoWzZsnh5eVG/fn2WLVt2NQ6bd/4lwZkOv77Lw2Vv47pS11ldkUihN3v1r/h/dQed7cuwY+eDes9wW+cvwO6wujSRa4LD7shZcfWr375i95ndf76Ymh04fD3dmNSnYIaRfA8iM2bM4LHHHmPo0KFs3LiRG264gQ4dOhATUwAveVTvDFFNsxc5Wzg8Z/PJlJO6IZ5IPpi4ehFvbevP4tA4Uuy+0ON/uDXuZ3VZItechiUb0ia6DS7j4o21b2CSTsD0HjChFWSmAdlhZPJ9DbmuXDDnfg8j6w9ZH0byPYi88847PPDAA/Tt25eqVavy3nvvERkZydixY89rm56eTmJiYq5HfkjPcvLQF+tYtOtE7hdsNujwGmCDbf+DQyuZtmsat8y6ha92f5UvtYgUVe8sn8V7O54kwc3GKm9/nPfPwVGxtdVliVyz/q/B/+Fh92DN8TUsO7UZYtfD2QOw8sOcNj4ebnzapyFNyoX8HkZWs/6Qtetm5WsQycjIYP369bRt2zbX9rZt27JixYrz2o8aNYrAwMCcR2RkZL7UNXn5QRZsP8FDX6znpx1/CyOlakP93tl/nv8MdgOpWamM2TyGhPSCv0KdyLVg5I/v8eme4djsmQTbajK5+2L8I+paXZbINS3cL5yBdQfybKNnaRLd6s/Vw5e9DQl/Lkfx1zCSnOFk8LRNZGS5LKo6n4PIqVOncDqdlChRItf2EiVKcPz4+Teae+6550hISMh5HD58OF/quv/6stxcsySZTsOAqetZ+PcwcuML4BkIxzbTJSWdCsUqkJCewLjN4/KlHpGiwuXMZMTnN/O/Y59gsxlK2Vswv/skAv1CrS5NpFC4v8b99KjaA3e7O9S8EyKvg8wUWDgsVztvDwef9mnIzTVLMr5nfTzcrJu7clWObPvbSojGmPO2AXh6ehIQEJDrkR/cHXZG312XW2qVItNpeHjqehZs/0sw8g3NXsHRzQu3rAyeavgUANN3TWd/wv58qUmk0EtL5JnJzfnaZP+C0TyzFnO6j8bb3dPiwkQKp0yTRUrbF/lzuEHuKxHeHg7G9KhPjfBAawr8Xb4GkdDQUBwOx3m9H3Fxcef1klxt7g47o++qQ8fapcl0GgZO3cD8bcf+bNDoQRi4Gho/SNPSTWkZ0ZIsk8Vba9+yrmiRa1V8DIljWtP+9AE8XIb2mc15//4puLtpZoxIflh9bDWdv+3M6KOLoH6f7I1znwaX09K6LiRfg4iHhwf169dn4cKFubYvXLiQpk2b5uehL4ubw8673WrTqU5pslyGgV9uZO7W38OIwx2CyuS0/b8G/4ebzY1lsctYemSpNQWLXINMzBrSx7YkIPE3aiZ70tk2mNfu/xCHXfeMEclPhxIPMXP3TGIa3QdegdlTec8etLqs8+T7pZknnniCiRMn8umnn7Jz504ef/xxYmJi6N+/f34f+rK4Oey8060OneuG43QZHpm2kR+2HM3d6NBKyvw4kh5VuuNmd9NqqyKXafuWqdw1vxdnsuLZ4YpmXMWPea5XP4UQkXzWuFRjmoU3I8tk8f6uL6DnbHhkHYSUt7q08+T72sl33XUXp0+f5sUXX+TYsWPUqFGDuXPnEh0dnd+HvmwOu423utbGZoNvNsTy6PRNuAzcVrs0ZKTAjB6QcpoBpYZxR8evKVesnNUlixR4S48s5fHN75Hh6c7TQeUID32fV7o1UQgRuUoer/c4K2JXsODgAnpX601Nd2+rS7qgqzJY9eGHH+bgwYOkp6ezfv16mjdvfjUOmycOu40376zNnfUjcLoMj03fyLebYsHDB24aCYDvkrcph7vFlYoUcFnpfLX7Kx75+REyXGlwriwhoaMVQkSussrBlelYviMA76x/B2NM9hiRDV9A0vkzV62ie838hcNu440utejWIAKXgcdnbGLWxiPZNw+KagKZyTD/WQC2n97OO+t+/4sVEQBM0gne/7wFL656ERcuMuPr0yZsOG93a6oQImKBQXUG4WH3YN2JdSyLXQbfDoTvBsHiUVaXlkNB5G/sdhuv3VGLuxtG4jLwxMzNfL3xKNzyDtjdYNcPxG+ZQZ95fZi0fRI/x/xsdckiBULmsc08N601HzuSszecvIFbSz3Gm3fWUwgRsUgpv1L0qJZ9J95FMYug3u8Ldm74HOJ2WVjZnxRELsBut/Fq55p0bxSFMfDk/zbzvyMB0PQRAIr9OIyelboC8Pra10nJTLGyXBHr/baAjMk3s49MHMbgd+wm2kc/xBt31lYIEbHYAzUeYEKbCQxvMhyim0CVW8G4zlvkzCoKIhdht9t45fYa3Htddhh56n+b+Z/fvRBSEc4dp1+GO+F+4RxPPs5Hmz6yulwRaxgDq8bCtLvxTT/HfceK4Yq5l+vL9+H1LrWwK4SIWC7QM5AmpZv8uZDoTSOze/j3LIAD1i9HoSByCXa7jZc61aBXk+jsnpHZu1lYaRh0+gjvZo8xtPFQAKbsnMK2U9ssrlbk6tvy3UNMXfEyGBfTna14NHUI7at1UAgRKaDi0+LZRhrUvy97w48vgMu6+8yAgsg/stlsjLytOn2algGg3y8OvsxoDjYbN0TcwC3lbsFlXAxbMYxMZ6a1xYpcRQsPLeT+hDW8FhLMw57teTazL3c2KstrdyiEiBREm+I20f6b9jy55Ekym/8fePjBsU2wY7aldSmIXAabzcbwjtW4r1kZAIbM2sqUVYcgLYFnfKsQ5BnEnrN7+H7/99YWKnIVGJeLSdsm8cTiJ0g3TjzPRTM38S56NynDq51rKoSIFFCVgyvj7eZN7LlYZh1dBjc8AQ37QXQzS+uymQI8/zQxMZHAwEASEhLy7QZ4eWGM4eU5O/nk1wP4k8KqYkPxTTvBgg4jOBtYim6Vu2G3KdtJ4ZW5bzGvLH2ar+2pAGScaUr6iVvpd0N5htxc9YI3sxSRguPLnV8yas0oinsXZ84dc/By88qX4+Tl+1vfmnlgs9l4/paqPNi8HEn4MOtcDQDaLZ/I3dHtFUKkUEta9REDF/bja3sqNgNpxzuSfuI2BraqqBAico24s9KdlPQtSVxqHF/99pXV5QAKInlms9l4rkMVHmpRjlezenDAVQISj8DcJwFIyUxh9bHVFlcpcgU5M+GHJ1i+7GVWenvhaWxkHLmHzLPNeKJNJZ5qV0UhROQa4eHwoH+t7Hu9Tdw6sUAsP6Eg8i/YbDaebV+Ffq1r8kTmwziNDbZ+xYn1k+jyXRcG/jyQ/Qn7rS5T5L9LPg1fdIZ1n9AuOZWOziqcOTiQtHO1eKZ9FQa3rmh1hSKSR7dVuI1I/0jOpJ1h+u7pVpejIPJv2Ww2Hm9TifbtO/Kh83YA/Oe+QKR3GOnOdIYsG0KmS7No5Bp2ag9LJ7XkbMxyjIc/Myq8wZe/9cGVFsHzt1RlQMuCdxdPEfln7nZ3Hqr1EG42NxLTE60uJ//vvlvYPdSiPFPchrJ5wWZqs5+79ySwrbg/209vZ+LWiQyoPcDqEkXyzBjD54d/4m0/qOMdQV2fkYze7AnAS52q07NJGWsLFJH/5JZyt9CwZENK+5W2uhT1iFwJ9zaryNEbRzPLeT1PHOtOlCt7Xf8Jmyew/fR2i6sTyQNjyHRmMnzFcN7aMgZjg3iP5oze7IbNBq93qakQIlIIuNndCkQIAQWRK6ZDy+a43fkx5+wBrNoaTQgNyTJZDFk2hLSsNKvLE/lnGSmc+aonfb/pyKy9s7Db7JSxdWfL9g447G680602dzWMsrpKESlkFESuoI61SzPu3vp4OOxU2xuEuwlgf8J+JmyZYHVpIpeWcIQ9k9twT+J6NqTE4uvmQ5mswWzdURsPh4MxPerRuW6E1VWKSCGkIHKF3VStBItq/8IYx0R6HrPj56zB7eXvtroskYs7sBQzvjnPc5JYdzfCPUMJTXyKzb+VxMfDwaT7GtKuekmrqxSRQkpBJB9ENLsbl92dx9O30HpfGI988RtnkjOsLkskN2NgxYfw+e3YUk7zmgnlhrDGuOKeZtshbwK83JjStzHNKoRaXamIFGIKIvkhvD72Ni8CMMx9Klmxm7hz3Apm7/6ZLFeWxcWJAMaQ8U0/Vi19EYwTat2NR5cf2LnjXn476iLUz5MZDzWhXlSQ1ZWKSCGnIJJfrhsAlW/Bgywmer7HKftnvLDqMUat/MDqykQ4nXaGvpkH6F+yOCubP8K+69+i68cbOXAqmfBi3vyvfxOqlrL+/k4iUvgpiOQXmw1u/wiCylKKk/R37QRg5p5JTN38s8XFSZGVlcGO0zvoPqc7G9Pj8HH3ZV9IC7qNX8XRhDTKh/nyvwFNKBPqa3WlIlJEKIjkJ+8guPtLcPdlwLndRKVVBZth1Lrnmb11h9XVSVFiDCx9i+8/vZ5e83pxLPkY0QHRDK7+Ea9+7eR0cgY1wwOZ+VATSgV6W12tiBQhWlk1v5WoBneMx+bhx+clG9Dhq7tIdTvM0OVP4sz8iC71ylpdoRR2aQlkzRrA2yeXMyUwAJxwQ/gNNAsczPMz95PlMtxQMZSx99bHz1M/EkTk6lKPyNVQtSOUb0WIrz/TO43DDV/s3ocZuvQVJi8/YHV1Upgd2wzjm/PTkV+yQwjwUK2HqOn2OEO/3keWy3B7ndJ80ruhQoiIWEJB5Cor58pkNP6ADbdia3jxx58YNXcnLpexujQpTIyBdZNgYhs4e5B2bqF0C2/FOy3e5fSRVrw+/zcAHmxejne61cHDTT8KRMQa+ulzNRkDs/rT/MAanszy4daQJ3Gll2b80v08Mn0jaZlOqyuUwmL5e/y46DmSXBlQqT22h5bwdMt3mb0imEnLDwLw/C1VGXJzVex2m7W1ikiRpiByNdls0HkceAfR+/BOXktZyLvdauLusDFnyzHunbias1r4TP6jTGcmr3GG/ysRxpBqzXDdNZV4/Oj1yRrmbDmGu8PG6Lvr0PeGclaXKiKiIHLVhZTPnknj8ICd39H59Ce8dU8E/pEzWBdzgjvGruDQ6WSrq5Rr0dGNHE8+zn0L7mPqvtkAVK7QgYNnUrljzApWHziDn6cbn/ZpSKc64dbWKiLyO5sxpsAOTkhMTCQwMJCEhAQCAgrZ4kqbZ8CsB3EBd1Spy77007il1eTsge4E+3oxsXcDrWoplycjGeY/y/KdM3k2ogzxWSn4u/vz8vUv4++qw4Ofr+NsSibhxbz5pE8DqpQsZP8viUiBk5fvb/WIWKX2XdDiWezA8IM7cbc5yPLaSni5xZxJzqD7hFXM33bM6iqloDu+FeeEFozZ/y0DSoQRn5VC1eCqzOg4g8TTlenx8WrOpmRSOyKQWQObKoSISIGjIGKlls9CvV7UDa3Jiw2fAyDR80dqVtlFepaL/lM28MHPeyjAnVZiFWNg1Tj4+EbOndnLrIAAjM1G10pd+bzD53y9OpXHZmwiw+miffWSTH+wCcX9vayuWkTkPLo0YzWXE7LSwMOXDzd+yPgt43HYHDTyeZwf12Xf9fTWWqV4887aeHs4LC5WCoTkUzD7YdizIPt5pQ5svv5hYjITaBN1M899s5VZG2MBeKh5OZ5pX0UzY0TkqtKlmWuJ3QEe2ff1GFhnIJ2KVcdpnGxI+4CB7d1ws9v4Ycsxuo1fybGEVIuLlYLAeXgNE+NW8m1AIHR4E7pPo3ZUcxqE3kS38SuZtTEWh93GqDtq8pym54pIAacgUoDYNn7BiI3zaJnuooJfJA9c15CpfRsT7OvB1tgEOn6wnA0xZ60uUywUlxLHQ4e+YXRwMV4KCyW2+i1gs7Hu4Bk6frCcLUcSCPJx54v7G9G9UZTV5YqI/CMFkYKk8s24hVXlzWOxfLJ/F8HJZ2lcLoRvBzajSkl/Tp1L5+7xq/h6/RGrK5Wr6dgW+LQDi3f9jy7fdWH18dV4u3kz9LoXKO1bmqmrD9H941WcOpdOlZL+fDfoeppWCLW6ahGRy6IxIgXNuZPw2a1wchcEhEOfH5h1ejO1QhrwxpwTLNh+AoDeTaIZeks1Lc1dmDmzYPl7pC9+jbeL+TEt0B+AqsFVeb3564T7RjPy++1MXR0DwC01S/Fm11r4eOieMSJirbx8fyuIFETn4mDyrXBqN7PCIhjmZyfCL4JP237KtFVJvP/zHgDqRBZjTI96lC6m27YXOqf2wuz+ZB5Zyz2lS7LL0wOAXtV68Wi9Rzl9zskjX25k3aGz2GzwZNvKPNyyPDabxoOIiPU0WPVa51ccen8PoZVocuYYEU7DkXNHeGDhA/RoGsinfRoQ6O3OpsPx3PL+MpbtOWl1xXKluFywegKMux6OrMXdM5CWUTcS7BXM2JvG8lTDp1i1L4Fb3v+VdYfO4u/pxie9GzCwVQWFEBG5JqlHpCBLOgGfd+JYtVu578yvxJ6LJco/ik/bfUpGuj8Dpq5nW2wiNhs8flMlBrWqoBkS17q1nxC74CkybDbKRl4PnT4iy78kCekJFPMMZvTPe/hg0R6MgeqlAxjTox7RIb5WVy0ikosuzRQmGSng4cPRc0e5f8H9xJ6LJdI/kgltJhDqVYqR329n2prDALSsHMbbXWsT4udpcdHybxhj+GbXdN5Y/SpRXqF82WU+7u7Zf5cnk9J5bMZGlu89DcA9jaMYdms1vNy1toyIFDy6NFOYePgAUNqvNJ+0HE24cXA46TC95vUiwyQz6o5avHlnLTzd7CzefZL2o5ex9DddqrlmHNsC3z3CyXPHGbRoECPWvEqKDXwCo0jISgJg+d5T3PL+MpbvPY23u4N376rNq51rKoSISKGg4fXXkPAdP/D54RgeKhnGzUERBLhnz6Lo2iCSGuGBDJ62kT1x5+j16RoebF6OJ9tW1qyagiozFZa8jln+Pj/4evJ6wkoSXOm4290ZXHcwPav1JMsFr8zZwcfLDgBQsbgfY3rUo2IJf4uLFxG5cnRp5lricsKPz5O6eixexmCrdRfc9gFOuxsOu4PUDCevzN3BlFXZ0zlrhAcw+u66lA/zs7hwyWXPTzDvKRLjD/BUWCgrfLJnPVUNrsqr179KhaAK/HYiiUenb2LnsUQAejSO4vlbqmmZfxG5JmiMSGG39hOY+xQYJymRjXggrBidK3elW+VuAPy4/ThPf72F+JRMvN0dDL2lKj0aR2lWhdXiY2D+c7DrBwBc/qXoXaYCO1KOMaDOAHpX742bzY3PVx7i1bk7Sc9yEezrwRtdanFTtRIWFy8icvkURIqCfYtgZh+mejp5LSQYgHur3suTDZ7EYXdwPCGNJ2ZuYsW+7MGNzSqE8HqXWkQE+VhZddH2eSd2H15OpNPg07g/tHiGQxlnMcZQJrAMR86m8Nw3W1m25xQALSqF8WbXWrprrohccxREioqTuzFfdmOCewYf+mYP92ke0ZzXb3gdPw8/XC7D5BUHeWPBLtIyXfh6OBhyS1XuaaTekavGmQUON85lnGPMihf58uA8epS9hadavJbTxOUyTF19iNfm7SI5w4mnm53nOlShd9My+nsSkWuSgkhRkhoPCYdZkHaUob8OJd2ZTvnAcrzb6j3KBpYF4MCpZJ76ajPrDmXfMO/6CqGMuqMmkcHqHck3J7bDjy9gQivzQ6WmvLP+HU6lZvd0tCvTjjeav4HdZufgqWSe/noLaw6cAaBhmSBe71KLchrXIyLXMAWRImrrya0M/vFBTmWdw8fhxXs3vk+T0k0AcP7eO/Lm770jXu52HrmxIv1uKKeZNVdS0gn45WXYOIXd7g5eDQ1lg6c7ANEB0Tzb6FmuD7+ejCwXn/x6gNE//0ZapgsfDwfPtK9Cz+uitSidiFzzFESKKmcWJ8c05Gn3c+z3cOerMndTvMUQcPw5S3v/yXMMnbWNlfuzx46UD/Pl5dtr0qR8iFVVFw7p52DVGPj1PchMZp6vD88WD8UFeLt582CtB+lVrRceDg+W7z3FsG+3se9kMpA9fue1O2qph0pECg0FkaIs+TRZ3w/myN75lMnKgohG0Hkccd7+FPcpDmSv4Dl7UyyvzNnJqXMZANxRN5xnb66igZH/xoFl8FVvSMkOd0Q05FSLp7l5zfNcH349TzV4ilJ+pTiekMbLc3bww5ZjAIT6efBsh6p0qReusSAiUqgoiBR1xsDm6dlTfDOSWOAfyNCwYJ5o8H90r3pvzpdeQkombyzYxZdrYjAGfDwc9G9Rnn43lNN6FXngTDrOdxOvY4tvIMObDoPqd4DNxvHk45T0LUlappNPfj3AmF/2kpzhxG6DntdF80TbygR6u1tdvojIFacgItniY2D2wzydvIN5ftk3RmtauinDmgwj3C88p9nGmLOM/H4Hmw7HA1AywIsn21XmjrrhGq/wd5mpsHEKHN2Eq9MHLIpZxJjNY9hzdg8An7b7lIYlGwLZ43K+2XCEdxb+xrGENADqRRXjpdtrUL10oGUfQUQkvymIyJ+MwWyaxrSY+byTtIN0Zzrebt4MrNGXHjXvx83u9nszw/dbjvH6vF3ExqcC2Xd3fbJdZVpWCtOlg9SzsHYirBqHSTnFLz7ejC1fj13nsm846O/uz4O1HqR71e542D1Y/NtJXp+3i13Hs+8XE17MmyfbVaJTbYU7ESn8FETkgg4kHGDkypGsP7EegKruQbx443tUKVkvp01appPJKw7y0aK9JKVnAVAnshiP3VSRFkUxkCQcgVVjYf1kyDjHUTcHj5UKZ+fv43993HzoUbUHvav3JsAjgMW/nWT0T3tyepcCvNwYdGMFejUpo5vUiUiRoSAiF+UyLmZ/dx9vn15LosPBpLPpNGj2NNTtCW6eOe1On0tn3JJ9fLHqEGmZLgDqRhVjcOuKRaeH5PAa+LQdxriwARSvTmbTwdy691POpsdnB5BqvQn0DGTRrjje/3kPm48kAODpZqdXk2gGtqpAMR8PSz+GiMjVpiAil2YMpzZP5cdVb3LP8YPZ2/xLs6ZuF6o1Hoyfb/GcpieT0hn/eyBJz8oOJBWL+9H3hrJ0qhNeuH7LT0uEM/ugdF0AElPPMnPS9fzs48Xn172Me+X2YLOx9eRWIvwj8HYE8O2mWD799SC7T2RfgvFyt9Pzumj6NS+nGUgiUmQpiMjlycrIvuTw67scTzlBx4hSeNjs9Ko7iHuq3oO/x5+3m49LSmPCkv1MWxNDcoYTyJ5+2vO6MnRvFEnxgGv0S9flhP2/wKZp2Tej8w5me8/pzNz7DfMOzCM1K3u8zOs3vM7N5W4GIC4xjSmrDjFldQxnkrOnP/t4OHICSKif50UPJyJSFCiISN5kpbN1xds8v3ca+3/v4PD38OfeKvdwl39lQsrfBL9fiklMy2T6mhgmLT+YMxPEYbdxY5Xi3N0wkhaVwnBzFPCVWo2B41th2/9gy0xIOkaqzcY8Xx9mBoey3fHn/xIVgyrSp3of2kS2Y8W+eGasPcyiXXFkubLbhBfzpnfTaO5qEEWgj6biioiAgoj8S87MNH48uIBx2z9lf8J+ANyN4eYsNx4p25kSdXtCcDkAMp0u5m07zmcrDrL+93vYAJQI8KRz3QhurVWK6qUDCuZYkiVvZi/D/gfvIPZXaU+n+OUAuNvdaVumLV0rdsUjqzxzth7nmw1HiEtKz3lL/egg7m9WlnbVSxT84CUicpUpiMh/4nQ5WRizkC9Wv8WWtBN4ulwsPHyUIJcLwhvgqnob9srtIbQS2GzsOZHEjLWH+WZjbM6lCoDoEB9uqVmK9jVKUqN04NWftnouDvYvhn2LoGZXqNCaLFcWm7fP4Kclw8gMjOT5+k9ApXbg5sn/Lf4/qodUp5LPjfz6Wxpztx4j5kxKzu6CfT24o244XRtEUrmk/8WPKyJSxBWYIPLKK68wZ84cNm3ahIeHB/Hx8Xl6v4KI9TYfWc6e3d9y59E92V/qxkXvUsUJcbq4qcVIbqjYKXssSWYa6TZ3ft4Zx/ebj7JoV1zO4FaAEF8PmlcKo0WlMJpVCCXM/wqPozAG4g/BkXXZs10OrYATWwFIttlYUeNmFpcoy9IjS4lPjwfIXu/jrsWkpXuwbM9Jlv52kmV7TnH6L2HKy91Oq8rFua12aVpXLaEbBIqIXIYCE0SGDx9OsWLFOHLkCJ988omCyLUu6QSHNn3Grfu/yNnkZnejccnG3HB0Jw0TzlChVEPsEfVJK1aRZfEhfL3HsGzvqZwBrn8oE+JDvagg6kUHUSeyGBWK+13+DJzMVDi5GzA5M1xIOg5vVz6v6ZsRFZnqnoGTP/+Z+7kHEOVdH4/UusTERnLodHqu9/h6OGhROYyba5aiVeXi+Hq6/X23IiJyCQUmiPxh8uTJPPbYYwoihYAxhl1ndvHjwQX8fHgRBxIO5Hr93oREnjkTD0AmkOIZgH/ZFqxpNJolv51kye6TlItbgMM4ScGLZLzIMG44bIbSgZ6EhYTiEVmP0sW8KV3Mi+oHPycg6xQeycewJRyBxNjs0IGB8jeScvdUDiQeYMfpHexY/BLb3R2MCmqKd2gT9vvX56uj81kU9yleFMeWUp0zJyuSlRIN5A49NcIDaF4xjOaVwqgXFaSeDxGR/yAv398F6le99PR00tP//O00MTHRwmrkQmw2G1VDqlI1pCqP1n+M/Qn7WXx4Matjl7MxbhO1y98MJc7Byd1sTj7MfSWLUTx9G1G/PUmkfySdW0VS8pcZBKUlUiUjg1Bn9uWbLCArzcaWmPJ02zsc7JnY7BnM8vyQk45zhGc5CXRlt13u7cXHgUHsyzhA/JeN/yzOM3tP7baHk5UUBOzH5igN9qdJygzOaRbg5UbtyGLUjQqiXlQx6kYGacaLiIhFClQQGTVqFCNHjrS6DMmDcoHlKBdYjvtr3E+mMxODAUf2SqKHds2A1S8Th5O4E+tYd2Jd9puKeQFejMoK4NZ0FyYrnR/dXDzj6wQy8Gdozv574g/4c93RatiTqnLUhHDY/RRO72+A7Ms9riwfXGnhONPCs/+bUpZiPu6UDPCidLHilAz0onyYH5VK+FGphD/F/T0L5mweEZEiKM9BZMSIEf8YFtauXUuDBg3yXMxzzz3HE088kfM8MTGRyMjIPO9HrOHuyN2r0KXKXbQp14GDCQc5nHQ453Ei5QTxafGUaPQMlGyIDTD758CyZ3O934YNbzdvvN18aN3pHhoXv4mktExik+LYcbYSxb0jKOEdgb97Mfw83fDzcsPfyw0/T7fCteKriEghlucxIqdOneLUqVOXbFOmTBm8vP5caVNjROSfZLmySHemk+XKvtGel5sXHnYP9VyIiFyD8nWMSGhoKKGhof+6OJELcbO74WYvUFcKRUTkKsjXn/wxMTGcOXOGmJgYnE4nmzZtAqBChQr4+fnl56FFRETkGpCvQWTYsGF89tlnOc/r1s1e8+GXX36hZcuW+XloERERuQZoiXcRERG5ovLy/a1Vm0RERMQyCiIiIiJiGQURERERsYyCiIiIiFhGQUREREQsoyAiIiIillEQEREREcsoiIiIiIhlFERERETEMgoiIiIiYhkFEREREbGMgoiIiIhYRkFERERELKMgIiIiIpZREBERERHLKIiIiIiIZRRERERExDIKIiIiImIZBRERERGxjIKIiIiIWEZBRERERCyjICIiIiKWURARERERyyiIiIiIiGUURERERMQyCiIiIiJiGQURERERsYyCiIiIiFhGQUREREQsoyAiIiIillEQEREREcsoiIiIiIhlFERERETEMgoiIiIiYhkFEREREbGMgoiIiIhYRkFERERELKMgIiIiIpZREBERERHLKIiIiIiIZRRERERExDIKIiIiImIZBRERERGxjIKIiIiIWEZBRERERCyjICIiIiKWURARERERyyiIiIiIiGUURERERMQyCiIiIiJiGQURERERsYyCiIiIiFhGQUREREQsoyAiIiIillEQEREREcsoiIiIiIhlFERERETEMgoiIiIiYhkFEREREbGMgoiIiIhYRkFERERELJNvQeTgwYM88MADlC1bFm9vb8qXL8/w4cPJyMjIr0OKiIjINcYtv3a8a9cuXC4X48ePp0KFCmzbto1+/fqRnJzMW2+9lV+HFRERkWuIzRhjrtbB3nzzTcaOHcv+/fsvq31iYiKBgYEkJCQQEBCQz9WJiIjIlZCX7+986xG5kISEBIKDgy/6enp6Ounp6TnPExMTr0ZZIiIiYpGrNlh13759fPDBB/Tv3/+ibUaNGkVgYGDOIzIy8mqVJyIiIhbIcxAZMWIENpvtko9169bles/Ro0dp3749Xbt2pW/fvhfd93PPPUdCQkLO4/Dhw3n/RCIiInLNyPMYkVOnTnHq1KlLtilTpgxeXl5Adghp1aoVjRs3ZvLkydjtl599NEZERETk2pOvY0RCQ0MJDQ29rLaxsbG0atWK+vXrM2nSpDyFEBERESn88m2w6tGjR2nZsiVRUVG89dZbnDx5Mue1kiVL5tdhRURE5BqSb0Hkxx9/ZO/evezdu5eIiIhcr13FGcMiIiJSgOXbtZI+ffpgjLngQ0RERAR0rxkRERGxkIKIiIiIWEZBRERERCyjICIiIiKWURARERERyyiIiIiIiGUURERERMQyCiIiIiJiGQURERERsYyCiIiIiFhGQUREREQsoyAiIiIillEQEREREcsoiIiIiIhlFERERETEMgoiIiIiYhkFEREREbGMgoiIiIhYRkFERERELKMgIiIiIpZREBERERHLKIiIiIiIZRRERERExDIKIiIiImIZBRERERGxjIKIiIiIWEZBRERERCyjICIiIiKWURARERERyyiIiIiIiGUURERERMQyCiIiIiJiGQURERERsYyb1QVcijEGgMTERIsrERERkcv1x/f2H9/jl1Kgg0hSUhIAkZGRFlciIiIieZWUlERgYOAl29jM5cQVi7hcLo4ePYq/vz82m82SGhITE4mMjOTw4cMEBARYUkNBpvNzcTo3l6bzc2k6P5em83NpVp8fYwxJSUmULl0au/3So0AKdI+I3W4nIiLC6jIACAgI0D/2S9D5uTidm0vT+bk0nZ9L0/m5NCvPzz/1hPxBg1VFRETEMgoiIiIiYhkFkX/g6enJ8OHD8fT0tLqUAknn5+J0bi5N5+fSdH4uTefn0q6l81OgB6uKiIhI4aYeEREREbGMgoiIiIhYRkFERERELKMgIiIiIpZREBERERHLKIjkwW233UZUVBReXl6UKlWKnj17cvToUavLKhAOHjzIAw88QNmyZfH29qZ8+fIMHz6cjIwMq0srMF555RWaNm2Kj48PxYoVs7ocy40ZM4ayZcvi5eVF/fr1WbZsmdUlFQhLly6lY8eOlC5dGpvNxuzZs60uqcAYNWoUDRs2xN/fn+LFi3P77beze/duq8sqMMaOHUutWrVyVlNt0qQJ8+bNs7qsf6QgkgetWrVi5syZ7N69m6+//pp9+/Zx5513Wl1WgbBr1y5cLhfjx49n+/btvPvuu4wbN44hQ4ZYXVqBkZGRQdeuXRkwYIDVpVhuxowZPPbYYwwdOpSNGzdyww030KFDB2JiYqwuzXLJycnUrl2bDz/80OpSCpwlS5YwcOBAVq1axcKFC8nKyqJt27YkJydbXVqBEBERwWuvvca6detYt24dN954I506dWL79u1Wl3ZpRv61b7/91thsNpORkWF1KQXSG2+8YcqWLWt1GQXOpEmTTGBgoNVlWKpRo0amf//+ubZVqVLFPPvssxZVVDABZtasWVaXUWDFxcUZwCxZssTqUgqsoKAgM3HiRKvLuCT1iPxLZ86cYerUqTRt2hR3d3eryymQEhISCA4OtroMKWAyMjJYv349bdu2zbW9bdu2rFixwqKq5FqUkJAAoJ8zF+B0Opk+fTrJyck0adLE6nIuSUEkj5555hl8fX0JCQkhJiaGb7/91uqSCqR9+/bxwQcf0L9/f6tLkQLm1KlTOJ1OSpQokWt7iRIlOH78uEVVybXGGMMTTzzB9ddfT40aNawup8DYunUrfn5+eHp60r9/f2bNmkW1atWsLuuSinwQGTFiBDab7ZKPdevW5bR/6qmn2LhxIz/++CMOh4NevXphCvEq+Xk9PwBHjx6lffv2dO3alb59+1pU+dXxb86PZLPZbLmeG2PO2yZyMYMGDWLLli1MmzbN6lIKlMqVK7Np0yZWrVrFgAED6N27Nzt27LC6rEtys7oAqw0aNIi77777km3KlCmT8+fQ0FBCQ0OpVKkSVatWJTIyklWrVhX4rq9/K6/n5+jRo7Rq1YomTZowYcKEfK7Oenk9P5L9/5DD4Tiv9yMuLu68XhKRC3nkkUf47rvvWLp0KREREVaXU6B4eHhQoUIFABo0aMDatWsZPXo048ePt7iyiyvyQeSPYPFv/NETkp6efiVLKlDycn5iY2Np1aoV9evXZ9KkSdjthb/D7b/8+ymqPDw8qF+/PgsXLqRz58452xcuXEinTp0srEwKOmMMjzzyCLNmzWLx4sWULVvW6pIKPGNMgf+OKvJB5HKtWbOGNWvWcP311xMUFMT+/fsZNmwY5cuXL7S9IXlx9OhRWrZsSVRUFG+99RYnT57Mea1kyZIWVlZwxMTEcObMGWJiYnA6nWzatAmAChUq4OfnZ21xV9kTTzxBz549adCgQU7vWUxMjMYUAefOnWPv3r05zw8cOMCmTZsIDg4mKirKwsqsN3DgQL788ku+/fZb/P39c3rVAgMD8fb2trg66w0ZMoQOHToQGRlJUlIS06dPZ/HixcyfP9/q0i7Nyik715ItW7aYVq1ameDgYOPp6WnKlClj+vfvb44cOWJ1aQXCpEmTDHDBh2Tr3bv3Bc/PL7/8YnVplvjoo49MdHS08fDwMPXq1dMUzN/98ssvF/x30rt3b6tLs9zFfsZMmjTJ6tIKhPvvvz/n/6mwsDDTunVr8+OPP1pd1j+yGVOIR1qKiIhIgVb4L+KLiIhIgaUgIiIiIpZREBERERHLKIiIiIiIZRRERERExDIKIiIiImIZBRERERGxjIKIiIiIWEZBRERERCyjICIiIiKWURARERERy/w/29NB5ixbMswAAAAASUVORK5CYII="
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def polyfit(x, y, loss_fn, n, lr, n_iter):\n",
" \"\"\"\n",
" Parameters\n",
" ----------\n",
" x : A tensor of shape (1, n)\n",
" y : A tensor of shape (1, n)\n",
" loss_fn : Function to measure loss\n",
" n : The nth-degree polynomial\n",
" lr : Learning rate\n",
" n_iter : The number of iterations of gradient descent\n",
" \n",
" Returns\n",
" -------\n",
" Near-optimal coefficients of the nth-degree polynomial as a tensor of shape (1, n+1) after `n_iter` epochs.\n",
" \"\"\"\n",
" weights = torch.zeros(n+1, requires_grad=True)\n",
" pows = torch.arange(n+1).float()\n",
" X = x.unsqueeze(1) ** pows\n",
" for _ in range(n_iter):\n",
" # Forward Pass\n",
" y_pred = torch.matmul(X, weights)\n",
" # Compute Loss\n",
" loss = loss_fn(y, y_pred)\n",
" # Compute Gradients\n",
" loss.backward()\n",
" # Update Weights\n",
" with torch.no_grad():\n",
" weights -= lr * weights.grad\n",
" weights.grad.zero_()\n",
" return weights\n",
"\n",
"x = torch.linspace(-math.pi, math.pi, 1000)\n",
"\n",
"# Original true values\n",
"y = torch.sin(x)\n",
"plt.plot(x, y, linestyle='solid', label='sin(x)')\n",
"\n",
"# MSE\n",
"mse = lambda y_true, y_pred: torch.mean(torch.square(y_pred - y_true))\n",
"a, b, c, d = polyfit(x, y, mse, 3, 1e-3, 5000)\n",
"y_pred_mse = a + b * x + c * x ** 2 + d * x ** 3\n",
"plt.plot(x, y_pred_mse.detach().numpy(), linestyle='dashed', label=f'mse')\n",
"\n",
"# MAE\n",
"mae = lambda y_true, y_pred: torch.mean(torch.abs(y_pred - y_true))\n",
"a, b, c, d = polyfit(x, y, mae, 3, 1e-3, 5000)\n",
"y_pred_mae = a + b * x + c * x ** 2 + d * x ** 3\n",
"plt.plot(x, y_pred_mae.detach().numpy(), linestyle='dashed', label=f'mae')\n",
"\n",
"plt.axis('equal')\n",
"plt.title('Comparison of different fits')\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "e60cfabe",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-01T09:20:32.269111Z",
"start_time": "2024-04-01T09:20:31.636353Z"
}
},
"outputs": [],
"source": [
"x = torch.linspace(-math.pi, math.pi, 10)\n",
"y = torch.sin(x)\n",
"\n",
"def mse(y_true, y_pred):\n",
" assert y_true.shape == y_pred.shape, f\"Your ground truth and predicted values need to have the same shape {y_true.shape} vs {y_pred.shape}\"\n",
" return torch.mean(torch.square(y_pred - y_true))\n",
"def mae(y_true, y_pred):\n",
" assert y_true.shape == y_pred.shape, f\"Your ground truth and predicted values need to have the same shape {y_true.shape} vs {y_pred.shape}\"\n",
" return torch.mean(torch.abs(y_pred - y_true))\n",
"\n",
"test1 = polyfit(x, x, mse, 1, 1e-1, 100).tolist()\n",
"test2 = polyfit(x, x**2, mse, 2, 1e-2, 2000).tolist()\n",
"test3 = polyfit(x, y, mse, 3, 1e-3, 5000).tolist()\n",
"test4 = polyfit(x, y, mae, 3, 1e-3, 5000).tolist()\n",
"\n",
"assert allclose(test1, [0.0, 1.0], atol=1e-6)\n",
"assert allclose(test2, [0.0, 0.0, 1.0], atol=1e-5)\n",
"assert allclose(test3, [0.0, 0.81909, 0.0, -0.08469], atol=1e-3)\n",
"assert allclose(test4, [0.0, 0.83506, 0.0, -0.08974], atol=1e-3)"
]
},
{
"cell_type": "markdown",
"id": "825a4e0b",
"metadata": {},
"source": [
"### Task 1.3 - Observations on different model configurations\n",
"\n",
"Run `polyfit` on these model configurations and explain your observations for <b>ALL</b> four configurations. Refer to the learning rate and degree of the polynomial when making observations regarding how well the model converges if at all.\n",
"\n",
"1. `polyfit(x, y, mse, 3, 1e-6, 5000)`\n",
"2. `polyfit(x, y, mse, 3, 1e6, 5000)`\n",
"3. `polyfit(x, y, mse, 1, 1e-3, 5000)`\n",
"4. `polyfit(x, y, mse, 6, 1e-3, 5000)`"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "4c2554da",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-01T09:20:38.151067Z",
"start_time": "2024-04-01T09:20:37.724923Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([-3.5401e-10, 5.1256e-02, -2.5358e-10, 9.4585e-02, 7.6460e-10,\n",
" -1.0178e-02, 1.0055e-11], requires_grad=True)\n"
]
},
{
"data": {
"text/plain": "<Figure size 640x480 with 1 Axes>",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABY2UlEQVR4nO3deViU5eI+8HtmgGEfNtlXN1xwRUXc0jJc0rTFJQtb1LKTmZnf09FOm/3KY6udOpqVhpmpnczUk5pY7oKKgru4AILACAjMsA7MzPv7Y3CKQARleGe5P9c115XDO8M9pHDzzLNIBEEQQERERGRFpGIHICIiImptLDhERERkdVhwiIiIyOqw4BAREZHVYcEhIiIiq8OCQ0RERFaHBYeIiIisDgsOERERWR07sQOIQa/XIy8vD25ubpBIJGLHISIiomYQBAFlZWUIDAyEVNr0GI1NFpy8vDyEhISIHYOIiIjuQE5ODoKDg5u8xiYLjpubGwDDF8jd3V3kNERERNQcarUaISEhxp/jTbHJgnPzbSl3d3cWHCIiIgvTnOklnGRMREREVocFh4iIiKwOCw4RERFZHRYcIiIisjosOERERGR1WHCIiIjI6rDgEBERkdVhwSEiIiKrw4JDREREVsekBWf//v0YP348AgMDIZFI8PPPP9/2Mfv27UN0dDQcHR3Rvn17fPHFFw2u2bRpE7p16wa5XI5u3bph8+bNJkhPRERElsqkBaeiogK9evXC559/3qzrMzMzMXbsWAwdOhSpqalYtGgR5s6di02bNhmvSUpKwpQpUxAfH4+TJ08iPj4ekydPxpEjR0z1MoiIiMjCSARBENrkE0kk2Lx5MyZOnHjLa1599VVs3boV58+fN943e/ZsnDx5EklJSQCAKVOmQK1WY8eOHcZrRo8eDU9PT6xfv75ZWdRqNRQKBVQqFc+iIiIishAt+fltVodtJiUlIS4urt59o0aNwqpVq1BbWwt7e3skJSXh5ZdfbnDNsmXLbvm8Go0GGo3G+Ge1Wt2quYmI6M6oq2tx6XoZ8lXVUKqqUVpZC61egFanh6O9DB7O9vBwdkCYtzM6tHOFl4uD2JHJQphVwVEqlfDz86t3n5+fH7RaLYqKihAQEHDLa5RK5S2fd8mSJXj77bdNkpmIiJrvuroa+y4W4sClIpy+VoqsG5UteryPqxz9wz0RE+GFIZ3aoaOvq4mSkqUzq4IDNDwC/eY7aH++v7Frmjo6feHChZg/f77xz2q1GiEhIa0Rl4iIbqO4ogZb03LxU2ouTl1TNfh4gMIRwZ5O8HN3hJeLA+xlUthJJaiu1aG0qhY3ymuQWVSB3NIqFJVrsOOMEjvOGH6pjfRzw9geAXi4bxBCvJzb+qWRGTOrguPv799gJKagoAB2dnbw9vZu8pq/jur8mVwuh1wub/3ARER0S2dyVfhyfwa2n86HVn/zl1WgZ7AH7uncDgPCvdA90B2ezXzbqUKjxbl8NY5k3EByRjGOZN5A+vUypF8vw7LfLuLeSF9MHxSOYZ18mvyll2yDWRWc2NhYbNu2rd59u3btQr9+/WBvb2+8JjExsd48nF27dmHQoEFtmpWIiBp3/GoJPk5Mx6HLN4z39QhS4JG+QRjXKxA+rnf2C6eL3A79w73QP9wLc+4FVJW12HVOiS1peTh4uQi/XSjAbxcK0CNIgfn3d8bwyHYsOjbMpAWnvLwcly9fNv45MzMTaWlp8PLyQmhoKBYuXIjc3Fx8++23AAwrpj7//HPMnz8fs2bNQlJSElatWlVvddRLL72EYcOGYenSpZgwYQK2bNmC3bt34+DBg6Z8KUREdBtXCsvx/s4L+PXsdQCATCrBuJ4BmDW0PaKCFK3++RTO9pjULwST+oXgSmE51iZdxQ8pOTidq8LTCcfQJ9QDb43vjl4hHq3+ucn8mXSZ+N69ezFixIgG9z/55JNISEjAU089haysLOzdu9f4sX379uHll1/G2bNnERgYiFdffRWzZ8+u9/gff/wR//znP5GRkYEOHTrg3XffxcMPP9zsXFwmTkTUeqprdfjs90tYuS8DWr0AqQSYFB2CF+/riGDPtp0Xc6Ncg5X7M/BtUhaqa/WQSIAp/ULwf6Mi4X2HI0dkPlry87vN9sExJyw4REStI+nKDSz86ZRxNdS9XXyxcEwXdPJzEzVXgboa/9pxAT+l5gIAvFwc8O7EKIzpESBqLro7LDi3wYJDRHR3anV6fJx4EV/suwJBAPzc5Xj7wSiMjvIXO1o9KVnF+OfPZ3BBWQYAeLBXIN6ZEAWFs73IyehOsODcBgsOEdGdyyqqwEsbUnGybsn3YwNCsGhsV7g5mmdpqNHq8e/fLmH53svQC0CwpxNWPB6NHsGtPy+ITIsF5zZYcIiI7syeCwWYuz4VZRotFE72+NfDPSzmbZ/U7BLM3ZCKnOIqONhJ8db47nhsQAhXWlmQlvz8Nulhm0REZB0EQcAX+67gmTXHUKbRol+YJ3a8NNRiyg0A9An1xP/mDMXIrn6o0eqxaPNpLPzpNGp1erGjkQmw4BARUZM0Wh3m/3AS/9pxAYIAPDYgFN/PGohADyexo7WYwtkeX8ZH4x9jukAqATYcy8EzCcegrq4VOxq1MhYcIiK6pXKNFjMSUrA5NRcyqQTvTOiO9x6KgoOd5f74kEolmH1PB3z9ZD84O8hw4FIRJq1IQm5pldjRqBVZ7t9QIiIyqRvlGkz7KhkHLxfB2UGGhKf7Iz423GrmrNzbxQ8/PBcLXzc50q+X4ZHlh3GlsFzsWNRKWHCIiKiBvNIqTPoiCaeuqeDl4oD1swZiaKd2YsdqdVFBCmx+YTA6+rpCqa7GlJVJuKBUix2LWgELDhER1ZOvqsJjXyUjo6gCQR5O+O/sWKs+7iDIwwkbnx2IbgHuKCqvwdQvk3Emt+Gp52RZWHCIiMhIqarGY18m4+qNSoR4OeGH2bHo0M5V7Fgm5+0qx/pZA9ErxAOllbV47CuWHEvHgkNERAAMxxs89lUysurKzYZnYxFkgSul7pTC2R7fzRiAfmGeKKvW4snVR3G5gHNyLBULDhERQVVVi+mrjyKzqALBnk5YP2ugTZWbm9wc7bH66f7oEaTAjYoaPPH1EeQUV4odi+4ACw4RkY2rrtVh1rcpuKAsQzs3Ob6fObDNTwE3J+6O9ljzzAB0qpt4PO3rZBSUVYsdi1qIBYeIyIbp9ALmbUjD0cxiuMntkPB0f4R62265ucnLxQHrZsYgzNsZOcVVmLkmBZU1WrFjUQuw4BAR2ShBEPDm1jPYeVYJB5kUX07vh+6BPIDyJl93R6x5egA8ne1x6poKc9enQqe3ueMbLRYLDhGRjVp9KAvfJWdDIgGWTe2N2A7eYkcyO+E+Lvj6yX5wsJNi9/kCLN52FjZ4RrVFYsEhIrJB+y4W4t1fzgEAXhvbFWMt6NDMthYd5oVlU3oDANYkXcXqQ1mi5qHmYcEhIrIxlwvKMef7E9ALwOR+wZgxJELsSGZvbI8ALBrbBQDw7i/ncPBSkciJ6HZYcIiIbEhpZQ1mrjmGsmot+od74p2JUVZztpSpzRraHo9GB0MvAHPWn+DycTPHgkNEZCN0egFzN6Qh60YlgjycsOKJaMjtZGLHshgSiQT/b2IUegYrUFpZi2fXHkdVjU7sWHQLLDhERDbi898vY//FQjjaS/HV9H7wcZWLHcniONrL8MUT0fBxdcD5fDX+8dMpTjo2Uyw4REQ24MClQiz77SIA4N2JPdAt0F3kRJYr0MMJ/5nWF3ZSCbak5SHhcJbYkagRLDhERFYuX1WFlzakQRCAxwaE4JHoYLEjWbyY9t5YNLYrAGDJ9gs4fY0Hc5obFhwiIitWq9NjzvepKK6oQbcAd7w5vrvYkazG04PDEdfNDzU6PeasP4Gy6lqxI9GfsOAQEVmxZbsv4vjVErg52mHFE33haM9Jxa1FIpHgg0d7IcjDCVdvVGLR5jOcj2NGWHCIiKzUkYwbWL73CgDgXw/3RJi3i8iJrI/C2R7/fqwPZFIJtp3Mw8ZjOWJHojosOEREVkhVVYv5P5yEIACTooPxQE/uVGwq0WGeWBAXCQB4c+tZXC4oFzkRASw4RERWRxAE/PPnM8gtrUKYtzPefJDzbkztuWHtMaSjDzRaPV75IQ1anV7sSDaPBYeIyMr8nJaLbSfzIJNK8MmU3nCV24kdyepJpRJ8MKkn3B3tcPKaCv/Zc0XsSDaPBYeIyIrkFFfijZ/PAgDm3tsJfUM9RU5kOwIUTnhnYhQA4LPfL+HUtVJxA9k4FhwiIishCAL+8dMplGm0iA7zxAsjOogdyeY82CsQD/QIgFYv4OWNaaiu5VEOYmmTgrN8+XJERETA0dER0dHROHDgwC2vfeqppyCRSBrcunf/4z3khISERq+prq5ui5dDRGSW1h/NwaHLN+BoL8WHk3rBTsbfYdvazfOq2rnJcaWwAu/vTBc7ks0y+d/+jRs3Yt68eXjttdeQmpqKoUOHYsyYMcjOzm70+k8//RT5+fnGW05ODry8vDBp0qR617m7u9e7Lj8/H46OjqZ+OUREZim3tArvbT8PAFgQF4kIHy4JF4uniwPef7QnAGD1oUwczSwWOZFtMnnB+fjjjzFjxgzMnDkTXbt2xbJlyxASEoIVK1Y0er1CoYC/v7/xlpKSgpKSEjz99NP1rpNIJPWu8/f3N/VLISIyS4IgYOFPp1Gu0aJvqAeeHhwhdiSbNyLSF1P6hQAA/rHpFN+qEoFJC05NTQ2OHz+OuLi4evfHxcXh8OHDzXqOVatWYeTIkQgLC6t3f3l5OcLCwhAcHIxx48YhNTX1ls+h0WigVqvr3YiIrMV/U65h/8VCONhJ8f6jvSCTSsSORAAWje2Kdm5yZBRV4LPfL4kdx+aYtOAUFRVBp9PBz8+v3v1+fn5QKpW3fXx+fj527NiBmTNn1ru/S5cuSEhIwNatW7F+/Xo4Ojpi8ODBuHSp8b9AS5YsgUKhMN5CQkLu/EUREZkRpaoa7/xyDgDwyv2d0dHXVeREdJPC2R7vTDDMH125LwPn8vjLdVtqkxloEkn93yYEQWhwX2MSEhLg4eGBiRMn1rt/4MCBeOKJJ9CrVy8MHToUP/zwAzp37ozPPvus0edZuHAhVCqV8ZaTw620icg6vLX1LMqqtegV4oGZQ9uLHYf+YnRUAEZ394dWL+DVTae4AWAbMmnB8fHxgUwmazBaU1BQ0GBU568EQcDq1asRHx8PBweHJq+VSqXo37//LUdw5HI53N3d692IiCzdb+evY+dZJeykEix9pAffmjJTiyd0h7ujHU7nqrD6UKbYcWyGSQuOg4MDoqOjkZiYWO/+xMREDBo0qMnH7tu3D5cvX8aMGTNu+3kEQUBaWhoCAnjWChHZhsoaLd7YYtjQb8bQCHTx5y9u5srX3RGvPdAVAPBx4kXkFFeKnMg2mPwtqvnz5+Prr7/G6tWrcf78ebz88svIzs7G7NmzARjePpo+fXqDx61atQoxMTGIiopq8LG3334bv/76KzIyMpCWloYZM2YgLS3N+JxERNbu092XkFtahSAPJ7x0Xyex49BtTO4Xgtj23qiu1ePtbWfFjmMTTH5AyZQpU3Djxg0sXrwY+fn5iIqKwvbt242rovLz8xvsiaNSqbBp0yZ8+umnjT5naWkpnn32WSiVSigUCvTp0wf79+/HgAEDTP1yiIhEdz5fja8PGt7qWDyhO5wdeNaUuZNIJHhnYneM+fQAdp8vQOK567i/W9NTNejuSARBEMQO0dbUajUUCgVUKhXn4xCRRdHrBTzyxWGkZpdidHd/fBEfLXYkaoF/7biAL/ZdQZCHE3bPvwdODjKxI1mUlvz85j7eREQWZGNKDlKzS+HiIMObD3YTOw610Nz7OiJQ4Yjc0ir8Z89lseNYNRYcIiILoaqsxQe/Gs42mh8XiQCFk8iJqKWcHezwxvi6vXH2X8GVwnKRE1kvFhwiIgvxye6LKK6oQSdfV0yPDbv9A8gsjeruh+GR7VCrE/DmlrOwwZkibYIFh4jIAqQry7A2+SoA4M3x3WHPk8ItlkQiwdsPdoeDnRQHLxdh55nb7+xPLcd/IUREZk4QBLy97Sx0egGjuvthSCcfsSPRXQrzdsFzwww7T7+34zwP4zQBFhwiIjP361klDl+5AQc7Kf75ACcWW4vZ93SAn7scOcVV3OHYBFhwiIjMWHWtDu/87zwA4Llh7RHi5SxyImotLnI7vDq6CwDgP79fRoG6WuRE1oUFh4jIjK3cl4Hc0ioEKBzx/PAOYsehVjaxdxB6hXigokaHD3elix3HqrDgEBGZKaWqGiv2GfZKWTS2K3cstkJSqQRvjDO87fjf49dwJlclciLrwYJDRGSmPtqVjupaPfqFeWJcTx4mbK2iwzwxoXcgBAF4exuXjbcWFhwiIjN0Pl+NH09cAwAseqArJBKJyInIlF4d3QWO9lIcyyrBL6fzxY5jFVhwiIjM0Hvbz0MQgAd6BKBvqKfYccjEAj2c8NwwwxyrpTsvQKPlsvG7xYJDRGRm9l8sxIFLRbCXSfD30ZFix6E28tw97eHrZlg2vi45W+w4Fo8Fh4jIjOj0At7bblgWHj8wHGHeLiInorbi7GCHeSM7AwA++/0S1NW1IieybCw4RERmZNOJa7igLIObox1evLej2HGojU3uF4wO7VxQUlmLlfuuiB3HorHgEBGZiaoaHT6q2wvlxXs7wtPFQeRE1NbsZFL8vW7zv1UHM3Gdm//dMRYcIiIzsepgBq6rNQjycML02HCx45BI4rr5ITrME9W1eizbfVHsOBaLBYeIyAwUV9Tgi30ZAIC/j46Eo71M5EQkFolEgoVjDKM4G4/l4HJBmciJLBMLDhGRGVix9zLKNVp0D3TH+J6BYschkfUL98L93fygF4ClO3mEw51gwSEiElm+qgprkq4CABaMioRUyk39CHh1dCSkEiDx3HWkZBWLHcfisOAQEYnss98vo0arx4BwLwzv3E7sOGQmOvq6YUr/EADAv3Zc4BEOLcSCQ0QkoqyiCvxwLAeAYfSGRzLQn80b2RlyOylSrpZg/6UiseNYFBYcIiIRLdt9EVq9gOGR7TAgwkvsOGRm/NwdET8wDIDh8FWO4jQfCw4RkUguKNXYcjIPALAgjkcyUONmD+8AZwcZTl1TIfHcdbHjWAwWHCIikXy066LxQM2oIIXYcchM+bjK8fTgcADAx4kXoddzFKc5WHCIiESQllOKxHPXIZUAL9/fWew4ZOaeHdoBbo52uKAswy+n88WOYxFYcIiIRPDhr4a9TR7pG4yOvq4ipyFzp3C2x6yh7QEAn+y+CK1OL3Ii88eCQ0TUxo5mFuPg5SLYyySYe18nseOQhXh6cDg8ne2RUViBn9PyxI5j9lhwiIja2Ke/Gc4XmtQvBCFeziKnIUvh5miP2fd0AGD4O1Sj5ShOU1hwiIja0NHMYhy6fAP2Mgn+NryD2HHIwkyPDYePqxw5xVX4ISVH7DhmjQWHiKgN/Xn0JtiTozfUMk4OMrwwwlCMl++5zFGcJrRJwVm+fDkiIiLg6OiI6OhoHDhw4JbX7t27FxKJpMHtwoUL9a7btGkTunXrBrlcjm7dumHz5s2mfhlERHeFozfUGh4bEApfNznyVNXYdOKa2HHMlskLzsaNGzFv3jy89tprSE1NxdChQzFmzBhkZ2c3+bj09HTk5+cbb506/TERLykpCVOmTEF8fDxOnjyJ+Ph4TJ48GUeOHDH1yyEiumMcvaHW4Ggvw3N1c3H+s+cyarmiqlESwcT7PsfExKBv375YsWKF8b6uXbti4sSJWLJkSYPr9+7dixEjRqCkpAQeHh6NPueUKVOgVquxY8cO432jR4+Gp6cn1q9ff9tMarUaCoUCKpUK7u7uLX9RREQtdCyrGJO+SIK9TII9C4az4NBdqarRYej7e1BUrsH7j/TE5LpDOa1dS35+m3QEp6amBsePH0dcXFy9++Pi4nD48OEmH9unTx8EBATgvvvuw549e+p9LCkpqcFzjho16pbPqdFooFar692IiNrSp7svAQAejeboDd09JwcZnhtm2Bfn8z2XuS9OI0xacIqKiqDT6eDn51fvfj8/PyiVykYfExAQgC+//BKbNm3CTz/9hMjISNx3333Yv3+/8RqlUtmi51yyZAkUCoXxFhJiG02XiMzDsSzDvjd2UolxgijR3Xp8YCi8XByQXVyJLdwXp4E2mWQskUjq/VkQhAb33RQZGYlZs2ahb9++iI2NxfLly/HAAw/gww8/vOPnXLhwIVQqlfGWk8OldUTUdm6O3nDuDbUmZwc74+7Gn++5DB3PqKrHpAXHx8cHMpmswchKQUFBgxGYpgwcOBCXLl0y/tnf379FzymXy+Hu7l7vRkTUFlI4ekMmFB8bBg9ne2QWVeB/pziK82cmLTgODg6Ijo5GYmJivfsTExMxaNCgZj9PamoqAgICjH+OjY1t8Jy7du1q0XMSEbWFz/dcBgA8Gh3M0Rtqda7yP0ZxPvudozh/ZmfqTzB//nzEx8ejX79+iI2NxZdffons7GzMnj0bgOHto9zcXHz77bcAgGXLliE8PBzdu3dHTU0NvvvuO2zatAmbNm0yPudLL72EYcOGYenSpZgwYQK2bNmC3bt34+DBg6Z+OUREzXYmV4W96YWQSoDnue8Nmcj02DCs3HcFlwvKsf10Psb3ChQ7klkwecGZMmUKbty4gcWLFyM/Px9RUVHYvn07wsLCAAD5+fn19sSpqanBggULkJubCycnJ3Tv3h2//PILxo4da7xm0KBB2LBhA/75z3/i9ddfR4cOHbBx40bExMSY+uUQETXb8r2G0ZvxvQIR5u0ichqyVm6O9pgxpD0+2X0Rn/1+CQ/0CIBU2vicVFti8n1wzBH3wSEiU7tcUI77P9kHQQB+nTcMkf5uYkciK6aqqsWQf/2OMo0WK+OjMaq7v9iRTMJs9sEhIrJVK/ZegSAA93fzY7khk1M42SM+1vDOyPK9V2CDYxcNsOAQEbWynOJK/JyWCwB4YURHkdOQrXhmSATkdlKczClF0pUbYscRHQsOEVEr++pABnR6AUM6+qB3iIfYcchG+LjKMaXuyIble6+InEZ8LDhERK2ooKwaG44ZNhP9G/e9oTY2a2h7yKQSHLxchFPXSsWOIyoWHCKiVrTqYCZqtHr0DfVAbHtvseOQjQnxcsaE3oZl4sv32PYoDgsOEVErKa2swXdJVwEY5t7c6vgYIlN6/h7DyOGv55S4XFAuchrxsOAQEbWSNYevoqJGhy7+bri3i6/YcchGdfJzQ1w3PwgC8MU+2x3FYcEhImoFFRotvjmcCYCjNyS+v9Wt3vs5NRe5pVUipxEHCw4RUSvYcCwHpZW1iPBxwdgeAbd/AJEJ9Q7xwKAO3tDqBXy1P0PsOKJgwSEiuku1Oj1WHzSM3txcxUIktr8NN4zibDiWjRvlGpHTtD0WHCKiu/TLqXzkllbBx9UBD/cNEjsOEQBgcEdv9AxWoLpWj28OZYkdp82x4BAR3QVBEIwTOZ8eHAFHe5nIiYgMJBIJ/lZ3iv2apCyUa7QiJ2pbLDhERHdh/6UiXFCWwdlBhidiwsSOQ1RPXDd/tPdxQVm1FhvrNqC0FSw4RER34cv9htGbqf1DoXC2FzkNUX1SqQQzh7YHAKw+mIlanV7kRG2HBYeI6A6dvqbCocs3IJNKMGNohNhxiBr1cN8g+Lg6ILe0CttP54sdp82w4BAR3aGVdaM343sGIMjDSeQ0RI1ztJfhydhwAMDKfRkQBEHcQG2EBYeI6A7kFFcafxt+dhgP1STz9sTAMDjZy3AuX43DV26IHadNsOAQEd2Brw9kQC8Awzq3Q7dAd7HjEDXJ08UBk/sFAwBW2sjGfyw4REQtVFxRg40phhUpzw1rL3IaouaZMaQ9pBJg/8VCnM9Xix3H5FhwiIhaaG3SVVTX6hEV5I5BHbzFjkPULKHezhgTZThG5KsD1j+Kw4JDRNQCVTU6rEnKAgA8N6wDD9Uki/Js3Yjj1rQ85Kus+xBOFhwiohb48XgOiitqEOzphDFR/mLHIWqRXiEeiInwglYvWP3xDSw4RETNpNML+PpPh2rayfgtlCzPzVGc749kQ11dK3Ia0+G/TiKiZko8dx1Xb1TCw9kek+pWpBBZmhGRvujo64pyjRYbjmaLHcdkWHCIiJpp1UHDxMzHY0Lh7GAnchqiOyOVSvCs8fiGLNRorfP4BhYcIqJmOJlTimNZJbCXSTC9bldYIks1oU8g2rnJoVRXY9vJPLHjmAQLDhFRM6yqm3szvmcg/NwdRU5DdHfkdjI8NSgcgGHJuDUe38CCQ0R0G3mlVfil7liGZ4bwUE2yDk/EGI5vuKAsQ5IVHt/AgkNEdBtrDmdBpxcQ294bUUEKseMQtQqFsz0ejTZMlr85QmlNWHCIiJpQodHi+7qVJjM4ekNW5unB4QCA3y4UIKOwXNwwrYwFh4ioCf9NyUFZtRYRPi64t4uv2HGIWlX7dq64r+7vtbVt/NcmBWf58uWIiIiAo6MjoqOjceDAgVte+9NPP+H+++9Hu3bt4O7ujtjYWPz666/1rklISIBEImlwq66uNvVLISIbotMLWF33Tf+ZweGQSnksA1mfmyOTPx6/htLKGpHTtB6TF5yNGzdi3rx5eO2115CamoqhQ4dizJgxyM5ufHOh/fv34/7778f27dtx/PhxjBgxAuPHj0dqamq969zd3ZGfn1/v5ujIlQ1E1Hp2n7+O7OJKKJzs8Ug0N/Yj6xTbwRtd/N1QVavD+qM5YsdpNSYvOB9//DFmzJiBmTNnomvXrli2bBlCQkKwYsWKRq9ftmwZ/v73v6N///7o1KkT3nvvPXTq1Anbtm2rd51EIoG/v3+9GxFRa1p1wDDxkhv7kTWTSCTGUZw1h7NQq7OOjf9MWnBqampw/PhxxMXF1bs/Li4Ohw8fbtZz6PV6lJWVwcvLq9795eXlCAsLQ3BwMMaNG9dghOfPNBoN1Gp1vRsRUVNOXSvF0axi2Em5sR9Zvwd7B8LH1bDx3/a6LREsnUkLTlFREXQ6Hfz8/Ord7+fnB6VS2azn+Oijj1BRUYHJkycb7+vSpQsSEhKwdetWrF+/Ho6Ojhg8eDAuXbrU6HMsWbIECoXCeAsJCbnzF0VENsG4sV+vQPgr+PY3WTe5nQzxA8MAAKsPZlrFxn9tMslYIqk/MU8QhAb3NWb9+vV46623sHHjRvj6/rF6YeDAgXjiiSfQq1cvDB06FD/88AM6d+6Mzz77rNHnWbhwIVQqlfGWk2M97zESUevLV1Xhl1OG32K5NJxsxeMDQ+FgJ8XJayocv1oidpy7ZtKC4+PjA5lM1mC0pqCgoMGozl9t3LgRM2bMwA8//ICRI0c2ea1UKkX//v1vOYIjl8vh7u5e70ZEdCtrDl+FVi8gJsKLG/uRzfBxleOh3kEArGPjP5MWHAcHB0RHRyMxMbHe/YmJiRg0aNAtH7d+/Xo89dRT+P777/HAAw/c9vMIgoC0tDQEBATcdWYism0VGi2+P3IVADCz7sRlIltx8yiSX88qkVNcKXKau2Pyt6jmz5+Pr7/+GqtXr8b58+fx8ssvIzs7G7NnzwZgePto+vTpxuvXr1+P6dOn46OPPsLAgQOhVCqhVCqhUqmM17z99tv49ddfkZGRgbS0NMyYMQNpaWnG5yQiulM/Hr8GdbUW4d7Oxg3QiGxFpL8bhnbygV4AEg5niR3nrpi84EyZMgXLli3D4sWL0bt3b+zfvx/bt29HWJhhMlN+fn69PXFWrlwJrVaLF154AQEBAcbbSy+9ZLymtLQUzz77LLp27Yq4uDjk5uZi//79GDBggKlfDhFZMZ1ewDeHDEPzzwyJ4MZ+ZJNujuJsPJaDsupakdPcOYlgDVOlW0itVkOhUEClUnE+DhEZ7T53HTO/TYG7ox2SFt4HFzn3viHbo9cLuP+TfbhSWIHXx3Uzq4n2Lfn5zbOoiIjq3BySf2xAKMsN2SypVGIcxUk4nAmd3jLHQVhwiIgAXLpehoOXiyCVAE/U7QdCZKse7hMMD2d75BRXIfFc8/atMzcsOERE+GP05v5ufgjxchY3DJHInBxkeDwmFIDlLhlnwSEim6eqrMVPJ3IBAE8NMp/5BkRimh4bDjupBMeySnAmV3X7B5gZFhwisnk/pOSgqlaHLv5uGNje6/YPILIBfu6OGNvDsL+cJS4ZZ8EhIpum0wtYk5QFAHhqUHizjpEhshVPDQ4HAGxNy0NRuUbcMC3EgkNENu2389dxraQKHs72mFC3TT0RGfQJ8UCvYAVqdHpsOJp9+weYERYcIrJp3xzKAgBM7R8KJweZuGGIzIxEIjGO4qxNvopanV7cQC3AgkNENuuCUo2kjBuQSSWIj+XScKLGPNAjEO3c5Liu1mDHGctZMs6CQ0Q2a03dxMlR3f0Q5OEkbhgiM+VgJzUuGU84ZDlLxllwiMgmlVTUYHMql4YTNce0mFDYyyQ4kV2KkzmlYsdpFhYcIrJJG1NyUF2rR7cAd/QP9xQ7DpFZ83VzxLiegQD+GPk0dyw4RGRztDo91iZdBWBYBsul4US399SgcADAtlN5KCirFjdMM7DgEJHN2X3+OnJLq+Dl4oAHewWKHYfIIvQK8UDfUA/U6gR8f8T8l4yz4BCRzbm5NPyxASFwtOfScKLmemqwYb7ad8nZqNGa95JxFhwisinn8tQ4klkMmVTCU8OJWmhMlD/83OUoKtdg++l8seM0iQWHiGxKwmHDMtcxUf4IUHBpOFFL2MukeCLG8IvBN2Y+2ZgFh4hsRnFFDX5OywMAPF23OysRtcy0mFA42ElxMqcUJ7JLxI5zSyw4RGQz1h81zBvoEaRA31AuDSe6E96ucuPk/IS6+WzmiAWHiGxCrU6P75Lrlobz1HCiu3Jzyfj20/m4rjbPJeMsOERkE3advY58VTV8XB0wrleA2HGILFpUkAL9wz2h1QtYV/eLg7lhwSEim3BzcvG0AaGQ23FpONHdunnEyboj2dBodSKnaYgFh4is3plcFY5llcBOKsHjXBpO1CpGdfdDgMIRNypqsO2k+S0ZZ8EhIquXULecdWyPAPi5O4obhshK2MmkiI+tWzJ+KBOCIIicqD4WHCKyakXlGmytWxr+FJeGE7Wqqf1DIbeT4myeGsevmteScRYcIrJq649ko0anrztHh0vDiVqTl4sDJvYOAmB+G/+x4BCR1arV6bG2boXH03XLWomodd0cGd15Rol8VZW4Yf6EBYeIrNaOM0oUlGnQzk2OsT24NJzIFLoGuGNgey/o9ALWJpnPknEWHCKyWgmHDEvDH6/bWp6ITOPmkvH1R7NRXWseS8b5L56IrJLhnJxS2MskmBYTKnYcIqs2sqsvgjycUFJZi60n88SOA4AFh4is1Jq6CY/jegbC141Lw4lM6c9LxtcczjKLJeNtUnCWL1+OiIgIODo6Ijo6GgcOHGjy+n379iE6OhqOjo5o3749vvjiiwbXbNq0Cd26dYNcLke3bt2wefNmU8UnIgtTUFaNbafqloZzcjFRm5jSL8S4ZDzFDJaMm7zgbNy4EfPmzcNrr72G1NRUDB06FGPGjEF2dnaj12dmZmLs2LEYOnQoUlNTsWjRIsydOxebNm0yXpOUlIQpU6YgPj4eJ0+eRHx8PCZPnowjR46Y+uUQkQVYfyQHtToBfUI90CvEQ+w4RDbB08UBD/UxLBlPMIMl4xLBxONIMTEx6Nu3L1asWGG8r2vXrpg4cSKWLFnS4PpXX30VW7duxfnz5433zZ49GydPnkRSUhIAYMqUKVCr1dixY4fxmtGjR8PT0xPr16+/bSa1Wg2FQgGVSgV3d/e7eXlEZGZqtHoMXvo7Css0+PdjffBgr0CxIxHZjPP5aoz59ABkUgkOvjoCAQqnVn3+lvz8NukITk1NDY4fP464uLh698fFxeHw4cONPiYpKanB9aNGjUJKSgpqa2ubvOZWz6nRaKBWq+vdTOFGuQb//u0SPvj1gkmen4hub/vpfBSWaeDnLseYKH+x4xDZlK4B7oiJMCwZX5fc+Ds1bcWkBaeoqAg6nQ5+fn717vfz84NSqWz0MUqlstHrtVotioqKmrzmVs+5ZMkSKBQK4y0kJOROX1KTMooq8HHiRXx9IBMlFTUm+RxE1LSbu6k+ERMGexnXURC1tZvz3sReMt4m//olEkm9PwuC0OC+213/1/tb8pwLFy6ESqUy3nJyclqUv7n6hXmie6A7NFo9NhwzzecgoltLzS7ByZxSOMikeIxLw4lEcX83P4zvFYj3Hu4h6i8ZJv3MPj4+kMlkDUZWCgoKGozA3OTv79/o9XZ2dvD29m7ymls9p1wuh7u7e72bKUgkEmNzXZuUBa1Ob5LPQ0SNuzmxcXyvQPi4ysUNQ2Sj7GRSfPZYH4zq7g+Z9NaDGaZm0oLj4OCA6OhoJCYm1rs/MTERgwYNavQxsbGxDa7ftWsX+vXrB3t7+yavudVztqXxvQLh7eKAPFU1Es9dFzsOkc24rq7GL6fyAXBpOBG1wVtU8+fPx9dff43Vq1fj/PnzePnll5GdnY3Zs2cDMLx9NH36dOP1s2fPxtWrVzF//nycP38eq1evxqpVq7BgwQLjNS+99BJ27dqFpUuX4sKFC1i6dCl2796NefPmmfrl3Jajvcy4a+o3h7LEDUNkQ9YdyYZWL6BfmCd6BCvEjkNEIjN5wZkyZQqWLVuGxYsXo3fv3ti/fz+2b9+OsDDDjof5+fn19sSJiIjA9u3bsXfvXvTu3RvvvPMO/v3vf+ORRx4xXjNo0CBs2LAB33zzDXr27ImEhARs3LgRMTExpn45zfJ4TBjspBIczSrGmVyV2HGIrJ5Gq8P3RwyH/N082ZiIbJvJ98ExR22xD86L61Ox7WQeJkUH44NJvUzyOYjI4KcT1zD/h5Pwd3fEgVdHcPUUkZUym31wbNnNOQBbTubhRrlG3DBEVkwQBOPbwfGxXBpORAb8TmAifUM90DNYgRouGScyqRPZJTidq4KDnRSPDeDScCIyYMExEYlEgqfr5gKsTbqKWi4ZJzKJm6M3E3sHwsvFQdwwRGQ2WHBMaGyPAPi4yqFUV2PnmcZ3WSaiO5evqsKOun9bT3JpOBH9CQuOCcntZHi8bsm4OZysSmRt1iVnQ6cXMCDCC90DuTSciP7AgmNij8eEwl4mwfGrJTh1rVTsOERWo7pWh++PGraYeJqjN0T0Fyw4Jubr7ogHegQA4CgOUWvaejIPxRU1CPJwwv3dGj+mhYhsFwtOG3h6cAQA4H8n81FYxiXjRHdLEAQk/GlpuB2XhhPRX/C7QhvoFeKBPqEeqNHp8f2R7Ns/gIiadCyrBOfy1XC0l2Jq/xCx4xCRGWLBaSM3N/777shV1Gi5ZJzobiQczgQAPNQnCB7OXBpORA2x4LSRMVEB8HWTo7BMgx1n8sWOQ2Sxckur8OvZ6wC4NJyIbo0Fp4042EkRP9BwwChPGSe6c98lX4VOLyC2vTe6+JvmLDkisnwsOG3osZhQOMikSMspRWp2idhxiCxOVY0O6+uWhvPUcCJqCgtOG/JxlWN8r0AAXDJOdCe2pOWitLIWwZ5OGNmVS8OJ6NZYcNrYzcnGv5zKx3V1tbhhiCyIIAjGXwyejA2HTCoRNxARmTUWnDbWI1iBfmGe0OoFrOOScaJmS84oxgVlGZzsZZjcj0vDiahpLDgiuLnx3/dHrkKj1Ymchsgy3Fwa/nDfICic7UVOQ0TmjgVHBHHd/RCgcERReQ3+d5JLxoluJ6e4EonnDEvDn+LScCJqBhYcEdjLpHiibsl4wuEsCIIgciIi87Y2+Sr0AjCkow86+bmJHYeILAALjkgeGxAKBzspTueqcIJLxoluqbJGiw03Tw3n0nAiaiYWHJF4uThgYm/DknFu/Ed0a5tTc6Gu1iLM2xkjIn3FjkNEFoIFR0RPDTJMNt5xRol8VZXIaYjMz59PDZ8eGw4pl4YTUTOx4IioW6A7YiK8oNML+C75qthxiMzO4Ss3cKmgHM4OMkzqFyx2HCKyICw4Irs5p+D7I9moruWScaI/u/n27aPRwXB35NJwImo+FhyRjezqhyAPJ5RU1mLryTyx4xCZjas3KvDbBZ4aTkR3hgVHZHYyKeJj65aMH+KScaKb1hy+CkEAhnVuhw7tXMWOQ0QWhgXHDEztHwJHeynO5atxNLNY7DhEoiurrsUPKTkAgGe4NJyI7gALjhnwcHbAQ30MEyh5yjgRsPFYDso1WnT0dcU9nduJHYeILBALjpm4uf38r2eVuFZSKW4YIhHp9H+cGv7M4AhIJFwaTkQtx4JjJiL93TCogzf0AvBtEpeMk+3adVaJayVV8HS2x8N9g8SOQ0QWigXHjMwYYtj4b/3RbJRrtCKnIRLHqoOGU8OnxYTC0V4mchoislQmLTglJSWIj4+HQqGAQqFAfHw8SktLb3l9bW0tXn31VfTo0QMuLi4IDAzE9OnTkZdXf/n08OHDIZFI6t2mTp1qypfSJkZE+qK9jwvKqrX4b90ESyJbcjKnFClXS2Avk2B6bLjYcYjIgpm04EybNg1paWnYuXMndu7cibS0NMTHx9/y+srKSpw4cQKvv/46Tpw4gZ9++gkXL17Egw8+2ODaWbNmIT8/33hbuXKlKV9Km5BKJXimbhTnm0NZ0Om5ZJxsy+pDhtGbcT0D4efuKHIaIrJkdqZ64vPnz2Pnzp1ITk5GTEwMAOCrr75CbGws0tPTERkZ2eAxCoUCiYmJ9e777LPPMGDAAGRnZyM0NNR4v7OzM/z9/U0VXzSP9A3Gh7vSkV1cicRz1zE6yvpeI1FjlKpq/HIqH8Afb9cSEd0pk43gJCUlQaFQGMsNAAwcOBAKhQKHDx9u9vOoVCpIJBJ4eHjUu3/dunXw8fFB9+7dsWDBApSVld3yOTQaDdRqdb2buXJykOHxGEORW103F4HIFqxJyoJWL2BAhBeighRixyEiC2eygqNUKuHr69vgfl9fXyiVymY9R3V1Nf7xj39g2rRpcHd3N97/+OOPY/369di7dy9ef/11bNq0CQ8//PAtn2fJkiXGeUAKhQIhISEtf0FtaHpsOOxlEhzNKsbJnFKx4xCZXGWNFt8fyQbA0Rsiah0tLjhvvfVWgwm+f72lpKQAQKP7VwiC0Kx9LWprazF16lTo9XosX7683sdmzZqFkSNHIioqClOnTsWPP/6I3bt348SJE40+18KFC6FSqYy3nBzznsDr5+6I8T0DAfyxooTImm06kQtVVS1CvZwxsquf2HGIyAq0eA7OnDlzbrtiKTw8HKdOncL169cbfKywsBB+fk1/A6utrcXkyZORmZmJ33//vd7oTWP69u0Le3t7XLp0CX379m3wcblcDrlc3uRzmJtnhkTgp9RcbD+dj3+M6YJADyexIxGZhF4v4Ju6Iv/04HDIpNzYj4juXosLjo+PD3x8fG57XWxsLFQqFY4ePYoBAwYAAI4cOQKVSoVBgwbd8nE3y82lS5ewZ88eeHt73/ZznT17FrW1tQgICGj+CzFzUUEKDGzvheSMYqxJysLCMV3FjkRkEnsvFiCjqAJucjtM6mfebx8TkeUw2Rycrl27YvTo0Zg1axaSk5ORnJyMWbNmYdy4cfVWUHXp0gWbN28GAGi1Wjz66KNISUnBunXroNPpoFQqoVQqUVNTAwC4cuUKFi9ejJSUFGRlZWH79u2YNGkS+vTpg8GDB5vq5YhixpD2AID1R7JRwY3/yErdfBt2Sv8QuMpNtrCTiGyMSffBWbduHXr06IG4uDjExcWhZ8+eWLt2bb1r0tPToVKpAADXrl3D1q1bce3aNfTu3RsBAQHG282VVw4ODvjtt98watQoREZGYu7cuYiLi8Pu3bshk1nXrqf3dfFFuLcz1NVa/Hj8mthxiFrdBaUahy7fgFQCPFl3HhsRUWuQCIJgc7vJqdVqKBQKqFSq287vEdu3SVl4Y8tZhHs747dXhnN+AlmVv/94Ej+kXMPYHv5Y/ni02HGIyMy15Oc3z6Iyc49GB0PhZI+sG5X47XzDSdtElqqoXIOf0wzHsHBpOBG1NhYcM+fsYIdpdRv/cck4WZPvkq+iRqtHrxAP9A31FDsOEVkZFhwL8GRsOOykEhzJLMaZXJXYcYjuWnWtDt8mXQVgGL1pzt5YREQtwYJjAfwVjhjX07AEnqM4ZA1+PH4NxRU1CPZ0wliet0ZEJsCCYyFuLhnfdjIPSlW1yGmI7pxOLxiL+owhEbCT8dsQEbU+fmexED2CFRgQ4QWtXsA3hzmKQ5Yr8dx1ZBZVQOFkj8nc2I+ITIQFx4I8O9QwivN9cjbKqmtFTkN0Z746kAEAeGJgKFy4sR8RmQgLjgW5t4svOrRzQZlGiw1HzfvAUKLGHL9ajONXS+Agk+LJ2HCx4xCRFWPBsSBSqQTPDesAwDDZuEarFzkRUct8ud8wevNQnyD4ujuKnIaIrBkLjoWZ0CcQvm5yKNXV2HYyT+w4RM2WUViOXecMm1XOGsaN/YjItFhwLIzcToanBxt+OHy5PwM2eNIGWahVBzMhCIYz1jr6uokdh4isHAuOBZoWEwoXBxnSr5dh78VCseMQ3VZRucZ4YOyzw9qLnIaIbAELjgVSONkbj29Yue+KyGmIbm9t0lVotHr0qtvugIjI1FhwLNTTgyNgJ5UgOaMYJ3NKxY5DdEtVNTp8m5QFAHh2WAcey0BEbYIFx0IFejjhwd6BAP5YmUJkjn48cQ0llbUI8XLCaB7LQERthAXHgt2cy7DjTD6u3qgQOQ1RQzq9gK/rNvabOaQ9ZFKO3hBR22DBsWBd/N0xPLId9ALw9QEe30DmZ9dZJa7eqISHsz0m9QsWOw4R2RAWHAt3c+O/H1JycKNcI3Iaoj8IgoDlew2T4KcPDIOzA49lIKK2w4Jj4Qa290LPYAU0Wj2+Tboqdhwio4OXi3A6VwUnexmeGsyN/YiobbHgWDiJ5I/jG75NykJljVbkREQGy/cYRm+mDgiBl4uDyGmIyNaw4FiB0VH+CPN2RkllLQ/hJLOQml2CpIwbsJdJMGsoN/YjorbHgmMFZH86hPPL/Rk8hJNEd3PuzcTeQQj0cBI5DRHZIhYcK/FIdJDxEM7NqdfEjkM27OL1MiSeuw6JBHjung5ixyEiG8WCYyXkdjLjvjgr9l6BTs9DOEkcX9SN3ozq5o+Ovq4ipyEiW8WCY0UeGxAKT2d7ZN2oxPbT+WLHIRuUU1yJLSfzAAB/G8HRGyISDwuOFXGR2+HpuuW4/9lzGYLAURxqW18dyIBOL2BIRx/0DPYQOw4R2TAWHCvzZGw4XBxkuKAsw+8XCsSOQzaksEyDjccMq/j+NpyjN0QkLhYcK6NwtscTsWEAgM85ikNt6JtDmdBo9egV4oHYDt5ixyEiG8eCY4VmDmkPuZ0UqdmlSM4oFjsO2QB1dS3W1u2k/bfhHSCR8FBNIhIXC44Vaucmx5T+IQCA5Xsvi5yGbMG3h7NQptGio68r7u/qJ3YcIiIWHGv17LD2sJNKcOBSEU7mlIodh6xYuUaLrw8aTrN/8d6OkEo5ekNE4jNpwSkpKUF8fDwUCgUUCgXi4+NRWlra5GOeeuopSCSSereBAwfWu0aj0eDFF1+Ej48PXFxc8OCDD+LaNW5u92fBns6Y0DsIgGFFFZGprE26itLKWrT3ccG4noFixyEiAmDigjNt2jSkpaVh586d2LlzJ9LS0hAfH3/bx40ePRr5+fnG2/bt2+t9fN68edi8eTM2bNiAgwcPory8HOPGjYNOpzPVS7FIzw/vAIkE2HXuOs7nq8WOQ1aoskaLrw5kAABeGNERMo7eEJGZsDPVE58/fx47d+5EcnIyYmJiAABfffUVYmNjkZ6ejsjIyFs+Vi6Xw9/fv9GPqVQqrFq1CmvXrsXIkSMBAN999x1CQkKwe/dujBo1qvVfjIXq6OuKsT0C8MupfHz2+yUsfzxa7EhkZdYlZ6O4ogahXs6Y0JujN0RkPkw2gpOUlASFQmEsNwAwcOBAKBQKHD58uMnH7t27F76+vujcuTNmzZqFgoI/9nM5fvw4amtrERcXZ7wvMDAQUVFRt3xejUYDtVpd72Yr5t7bCRIJsP20EheUtvO6yfSqa3VYud8wejNnREfYyTilj4jMh8m+IymVSvj6+ja439fXF0ql8paPGzNmDNatW4fff/8dH330EY4dO4Z7770XGo3G+LwODg7w9PSs9zg/P79bPu+SJUuM84AUCgVCQkLu4pVZlkh/N4yNCgAAfPYb5+JQ61l/NBtF5RoEeTjhob5BYschIqqnxQXnrbfeajAJ+K+3lJQUAGh0LwxBEJrcI2PKlCl44IEHEBUVhfHjx2PHjh24ePEifvnllyZzNfW8CxcuhEqlMt5ycnJa8Iot39z7OgEAfjmdj3RlmchpyBpU1+rwxT7DoZovjOgIe47eEJGZafEcnDlz5mDq1KlNXhMeHo5Tp07h+vXrDT5WWFgIP7/m75MREBCAsLAwXLp0CQDg7++PmpoalJSU1BvFKSgowKBBgxp9DrlcDrlc3uzPaW0i/d0wtoc/tp9W4t+/X8J/pvUVOxJZuP+m5OC6WoNAhSMeieboDRGZnxYXHB8fH/j4+Nz2utjYWKhUKhw9ehQDBgwAABw5cgQqleqWRaQxN27cQE5ODgICDG+zREdHw97eHomJiZg8eTIAID8/H2fOnMH777/f0pdjM+be1wnbTyuxvW4UJ9LfTexIZKE0Wh2W7zWM3jw/vAPkdjKRExERNWSyceWuXbti9OjRmDVrFpKTk5GcnIxZs2Zh3Lhx9VZQdenSBZs3bwYAlJeXY8GCBUhKSkJWVhb27t2L8ePHw8fHBw899BAAQKFQYMaMGXjllVfw22+/ITU1FU888QR69OhhXFVFDXXxd8eYKH8IAvDv3y+JHYcs2KbjuchXVcPPXY5J/WxnPhsRWRaTvnG+bt069OjRA3FxcYiLi0PPnj2xdu3aetekp6dDpVIBAGQyGU6fPo0JEyagc+fOePLJJ9G5c2ckJSXBze2PEYdPPvkEEydOxOTJkzF48GA4Oztj27ZtkMn4m2RTbs7F2X46Hxevcy4OtZxGq8PndQV59j0d4GjPf3NEZJ4kgg0eN61Wq6FQKKBSqeDu7i52nDY1e+1x7DyrxLieAficc3GohdYczsKbW8/C390Re/9vOAsOEbWplvz85tIHG/PnFVWXOIpDLVBVo8Pndcd+zLm3I8sNEZk1Fhwb0y3QHaO7G+bifLL7othxyIKsTc5CYZkGwZ5OmMy5N0Rk5lhwbNDL93c27m58JlcldhyyAOUaLb7YZ9i1eO59neBgx28dRGTe+F3KBkX6u2FCL8O5QR/uShc5DVmChEOZKK6oQYSPCx7uw31viMj8seDYqHkjO8NOKsHe9EIczSwWOw6ZMVVlrfHMqXkjO/HMKSKyCPxOZaPCfVwwub9hHsUHv16ADS6mo2b6+mAGyqq16OznivE9eWI4EVkGFhwbNvdew1yKY1kl2HexUOw4ZIaKK2qw+mAmAGD+/Z0hld76HDkiInPCgmPD/BWOmD4wDIBhLg5Hceivvth3BRU1OnQPdMeo7v5ixyEiajYWHBv3/PAOcHGQ4UyuGjvPKMWOQ2Ykt7QKCYezAAAL4iIhkXD0hogsBwuOjfN2lWPG0PYADKM4Oj1Hccjg410XUaPVY2B7LwyPbCd2HCKiFmHBIcwcGgEPZ3tcKazAphPXxI5DZuCCUo2fUg1/F/4xpitHb4jI4rDgENwd7fG34R0AGH5rr6rRiZyIxLZ0xwUIAvBAjwD0DvEQOw4RUYux4BAAYHpsOII8nKBUV2P1oUyx45CIkq7cwJ70QthJJVgwKlLsOEREd4QFhwAAjvYy/F/dD7MVe6+gqFwjciISgyAI+NfOCwCAxwaEIsLHReRERER3hgWHjB7sFYioIHeUa7T492+XxI5DIthxRomTOaVwdpAZT54nIrJELDhkJJVKsGhsVwDA90eykVFYLnIiaku1Oj0++NVwNtmsoe3Rzk0uciIiojvHgkP1DOrgg3u7+EKrF7C07q0Ksg0bjmYjs6gCPq4OmDWsvdhxiIjuCgsONbBwTBdIJcCvZ68jJYsHcdoCVVUtPk68CACYe18nuMrtRE5ERHR3WHCogU5+bphSdxDne9vP8wgHG/Dv3y6hpLIWnXxdMW1AqNhxiIjuGgsONerlkZ3hZC/DiexS/HI6X+w4ZEIZheVYU3ckwz/HdYOdjN8WiMjy8TsZNcrX3RHP3WOYh/HeL+e5+Z8Ve2/7eWj1Au7t4ot7OvNIBiKyDiw4dEvPDeuAIA8n5Kmq8cW+K2LHIRM4cKkQu88XwO5PK+iIiKwBCw7dkpODzPhD74t9V3CtpFLkRNSatDo93vnfOQBAfGwYOvq6ipyIiKj1sOBQk8b28EdMhBc0Wj3e235e7DjUitYfzcbF6+XwcLbHS9zUj4isDAsONUkikeCtB7tDKgG2n1bi8JUisSNRKyitrDEuC59/f2d4ODuInIiIqHWx4NBtdQ1wx+MxYQCAxdvOQavTi5yI7tYHv6ajpLIWnf24LJyIrBMLDjXL/Ps7Q+FkjwvKMnx/NFvsOHQXTuaUGv8fLp4QxWXhRGSV+J2NmsXTxQEL4joDAD78NZ2njVsonV7A61vOQBCAib0DMbC9t9iRiIhMggWHmu2xAaHoHugOdbUW7/7CCceWaP3RbJy6poKb3A6LHuCycCKyXiw41Gx2MinefagHJBJgc2ouDl/mhGNLcqNcYzwtfH5cZ/i6OYqciIjIdFhwqEV6h3ggfqBhwvE/fz4DjZY7HFuKpTsvQFVVi24B7sb/h0RE1sqkBaekpATx8fFQKBRQKBSIj49HaWlpk4+RSCSN3j744APjNcOHD2/w8alTp5rypdCfLBgVCV83OTKKKrBiL3c4tgRHM4vxQ8o1AMA7E7tzYjERWT2TfpebNm0a0tLSsHPnTuzcuRNpaWmIj49v8jH5+fn1bqtXr4ZEIsEjjzxS77pZs2bVu27lypWmfCn0J+6O9nhjfDcAwPI9V5BRWC5yImpKda0O/9h0CgAwtX8IosO8RE5ERGR6dqZ64vPnz2Pnzp1ITk5GTEwMAOCrr75CbGws0tPTERkZ2ejj/P396/15y5YtGDFiBNq3b1/vfmdn5wbXUtt5oEcA/tv5GvZdLMTrW87guxkxkEgkYseiRnz++2VkFFWgnZscC3neFBHZCJON4CQlJUGhUBjLDQAMHDgQCoUChw8fbtZzXL9+Hb/88gtmzJjR4GPr1q2Dj48PunfvjgULFqCsrOyWz6PRaKBWq+vd6O5IJBK8MyEKcjspDl2+gZ9O5IodiRpxPl9tPCj1nQndoXCyFzkREVHbMFnBUSqV8PX1bXC/r68vlEpls55jzZo1cHNzw8MPP1zv/scffxzr16/H3r178frrr2PTpk0NrvmzJUuWGOcBKRQKhISEtOzFUKNCvZ3x0kjDGUZvbzuLAnW1yInoz3R6Aa9uOgWtXsCo7n4YHRUgdiQiojbT4oLz1ltv3XIi8M1bSkoKADT6loUgCM1+K2P16tV4/PHH4ehYfznrrFmzMHLkSERFRWHq1Kn48ccfsXv3bpw4caLR51m4cCFUKpXxlpOT08JXTbfy7ND26BGkgLpai0WbT0MQBLEjUZ1vDmUa9rxxtMPiCVFixyEialMtnoMzZ86c265YCg8Px6lTp3D9+vUGHyssLISfn99tP8+BAweQnp6OjRs33vbavn37wt7eHpcuXULfvn0bfFwul0Mul9/2eajl7GRSfDipF8Z9dgC7zxdgS1oeJvYJEjuWzbt6owIf7TIcprlobFf4uXPPGyKyLS0uOD4+PvDx8bntdbGxsVCpVDh69CgGDBgAADhy5AhUKhUGDRp028evWrUK0dHR6NWr122vPXv2LGpraxEQwCF4MUT6u2HuvZ3wUeJFvLn1LAZ19OYmciLS6QW88sNJVNXqMLC9F6b041uyRGR7TDYHp2vXrhg9ejRmzZqF5ORkJCcnY9asWRg3bly9FVRdunTB5s2b6z1WrVbjv//9L2bOnNngea9cuYLFixcjJSUFWVlZ2L59OyZNmoQ+ffpg8ODBpno5dBuzh3dA90B3qKpq8drmM3yrSkRfHchAytUSuMrt8MGjvSCVcnUbEdkek+6Ds27dOvTo0QNxcXGIi4tDz549sXbt2nrXpKenQ6VS1btvw4YNEAQBjz32WIPndHBwwG+//YZRo0YhMjISc+fORVxcHHbv3g2ZTGbKl0NNsK97q8pOKkHiuev4OY2rqsRwPl+Nj+vemnpjXDeEeDmLnIiISBwSwQZ/1Var1VAoFFCpVHB3dxc7jlX5dPclfLL7Itzkdtj+0lD+gG1DGq0OEz4/hAvKMozs6ouvpvfj3kREZFVa8vOb+7VTq3phRAf0DfVAmUaLlzemQavTix3JZny6+xIuKMvg5eKAJQ/3ZLkhIpvGgkOtyk4mxbIpfeAqt0PK1RKeVdVGjmUVGzf0e3diFNq5cdUgEdk2FhxqdaHezlg8oTsAYNlvl3Aiu0TkRNatpKIGc9enQi8AD/cJwpgeXE1IRMSCQybxUJ8gPNgrEDq9gHkb0lCu0YodySoJgoD/+/EU8lXViPBxweKJ3NCPiAhgwSETkUgkeGdiFII8nJBdXIlFP3GXY1NIOJyF3eevw0EmxWePGd4aJCIiFhwyIYWTPT6d2hsyqQRbT+bhu+SrYkeyKmdyVViy/QIAYNHYLogKUoiciIjIfLDgkEn1C/fCP0Z3AQAs/t85pOWUihvISqgqa/HC9ydQo9MjrpsfnhwULnYkIiKzwoJDJjdzaARGd/dHrU7AC+tOoKSiRuxIFk2nF/DSxlRcvVGJIA8nvP8ol4QTEf0VCw6ZnEQiwfuTeiLc2xm5pVWYtzENOj3n49ypZbsvYm96IeR2UqyMj4aHs4PYkYiIzA4LDrUJd0d7rHgiGnI7KfZdLMT7v14QO5JF+vWsEp/9fhkA8K9HenDeDRHRLbDgUJvpGuCO9x/tCQBYuS8DPx6/JnIiy3K5oAyv/HASAPD04HA81CdY5EREROaLBYfa1ITeQZgzoiMAYNFPp3H8arHIiSxDUbkGTyccQ7lGi5gILywa21XsSEREZo0Fh9rc/Ps7Y1R3P9To9Hj22+O4VlIpdiSzVl2rw8w1KcgprkKolzOWP94X9jL+0yUiagq/S1Kbk0ol+GRKb3QLcMeNiho8/c0xlFZyZVVj9HU7QafllMLD2R4JT/eHtyvPmSIiuh0WHBKFs4Mdvn6yH/zc5bhUUI4Za1JQVaMTO5bZWbLjPHaeVcJBJsWX8f3Qvp2r2JGIiCwCCw6JJtDDCd8+EwN3Rzscv1qCOd+fQK1OL3Yss/HFviv46kAmAOCDST0xIMJL5ERERJaDBYdEFenvhlVP9YfcTorfLhRgIc+sAgB8l3wV/9phWEr/jzFdMKF3kMiJiIgsCwsOia5/uBf+M60vZFIJfjx+DW9uPWvTJWdz6jW8vuUMAOCFER0w+54OIiciIrI8LDhkFkZ288PSR3pCIgG+TbpqsyVn28k8LPjvKQgC8GRsGBbERYodiYjIIrHgkNl4NDoYSx+23ZKz6fg1vLQhFTq9gEejg/Hm+O48Y4qI6A6x4JBZmdw/pF7JWfjTaWhtYOLx90eyseDHk9ALwNT+IXj/kZ6QSlluiIjuFAsOmZ3J/UOw9JGekEqADcdy8Ld1J1Bda51LyAVBwIq9V7Bo82nj21LvPdSD5YaI6C6x4JBZmtwvBMsf7wsHOyl2nbuO6auOQlVVK3asVqXV6fHPn89g6U7DaqnnhrXHWw92Z7khImoFLDhktkZHBeDbZwbATW6Ho1nFeGj5IWQUlosdq1VUaLSY9W0K1h3JhkQCvDGuGxaO7co5N0RErYQFh8zawPbe2PhcLAIUjsgorMCE/xzCnvQCsWPdlSuF5Zj4n0PYk14IuZ0UKx6PxjNDIsSORURkVVhwyOx1C3TH1jlD0C/ME2XVWjyTcAyf/34JOr3lrbDacTofEz4/hEsF5fB1k2PDswMxOspf7FhERFaHBYcsQjs3Ob6fNRDTYkIhCMCHuy5i2lfJyCutEjtas1TX6vD2trN4ft0JlGu0iInwwv/mDkGfUE+xoxERWSUWHLIYDnZSvPdQD3zwaE84O8hwJLMYo5ftx9aTeWa9X87JnFI88O8D+OZQFgDg2WHtsW5mDHzdHMUNRkRkxSSCOf9kMBG1Wg2FQgGVSgV3d3ex49AdyCqqwEsb03AypxQAMCKyHRZPiEKIl7O4wf6kXKPFp7svYvWhLOj0Anzd5Fj6SE+M6OIrdjQiIovUkp/fLDgsOBarVqfH579fxvK9l1GrE+BoL8Xz93TEzKERcJHbiZZLrxew7VQe3tt+HtfVGgDA+F6BeGdCd3g4O4iWi4jI0rHg3AYLjnW5XFCO1zafxpHMYgCAj6scc+/riMn9QuBoL2uzHIIgIPHcdXyceBEXlGUAgHBvZ7z5YHeMiOSoDRHR3WrJz2+TzsF59913MWjQIDg7O8PDw6NZjxEEAW+99RYCAwPh5OSE4cOH4+zZs/Wu0Wg0ePHFF+Hj4wMXFxc8+OCDuHbtmgleAVmCjr6u2PDsQHz2WB+EeTujqFyDN7acxZCle/Dv3y6huKLGpJ+/ulaHjceyMebTA3h27XFcUJbBTW6HBXGdsXPeMJYbIiIRmHQE580334SHhweuXbuGVatWobS09LaPWbp0Kd59910kJCSgc+fO+H//7/9h//79SE9Ph5ubGwDg+eefx7Zt25CQkABvb2+88sorKC4uxvHjxyGT3f43do7gWK9anR4bjmZjxd4ryFNVAwAcZFLc28UXD/UNwvDIdpDb3f2ojl4v4FhWMf53Kh/bTuWhtNKwy7KTvQxPDw7Hs8Pa8+0oIqJWZnZvUSUkJGDevHm3LTiCICAwMBDz5s3Dq6++CsAwWuPn54elS5fiueeeg0qlQrt27bB27VpMmTIFAJCXl4eQkBBs374do0aNum0eFhzrV6vTY/vpfHx9IBOnc1XG+53sZRjY3gtDOrVD7xAPdPF3a9Z8nVqdHplFFThxtQTJGTdw+MoNFJRpjB8P8nDC9NgwTOkfwmJDRGQiLfn5Ld5MzEZkZmZCqVQiLi7OeJ9cLsc999yDw4cP47nnnsPx48dRW1tb75rAwEBERUXh8OHDjRYcjUYDjeaPH0Zqtdq0L4REZy+TYkLvIEzoHYTz+WpsTs3FlrRcXFdrsCe9EHvSCwEAEgkQ7OkEf3dH+Lo7wk1uB4lEAokEKKvWoqSiBtfV1cgsqoD2LxsLusntENfdH+N6BWBYp3aQ8QwpIiKzYVYFR6lUAgD8/Pzq3e/n54erV68ar3FwcICnp2eDa24+/q+WLFmCt99+2wSJyRJ0DXBH1wB3LBzTBReUZdh/sRBJGTdwLk+NgjINcoqrkFN8+w0DXRxk6BbojoHtvRHb3hvR4Z6t8nYXERG1vhYXnLfeeuu2ZeHYsWPo16/fHYf664GDgiDc9hDCpq5ZuHAh5s+fb/yzWq1GSEjIHecjyySRSIxl57l7OgAACss0yCyqQEFZNQrUGlTWaKHTA3pBgLuTPbxc7OHjKkeHdq4IUDjyMEwiIgvR4oIzZ84cTJ06tclrwsPD7yiMv7/hTB6lUomAgADj/QUFBcZRHX9/f9TU1KCkpKTeKE5BQQEGDRrU6PPK5XLI5fI7ykTWrZ2bHO3c+HeDiMjatLjg+Pj4wMfHxxRZEBERAX9/fyQmJqJPnz4AgJqaGuzbtw9Lly4FAERHR8Pe3h6JiYmYPHkyACA/Px9nzpzB+++/b5JcREREZFlMOgcnOzsbxcXFyM7Ohk6nQ1paGgCgY8eOcHV1BQB06dIFS5YswUMPPQSJRIJ58+bhvffeQ6dOndCpUye89957cHZ2xrRp0wAACoUCM2bMwCuvvAJvb294eXlhwYIF6NGjB0aOHGnKl0NEREQWwqQF54033sCaNWuMf745KrNnzx4MHz4cAJCeng6V6o9lvH//+99RVVWFv/3tbygpKUFMTAx27dpl3AMHAD755BPY2dlh8uTJqKqqwn333YeEhIRm7YFDRERE1o9HNXAfHCIiIotgNkc1EBEREYmBBYeIiIisDgsOERERWR0WHCIiIrI6LDhERERkdVhwiIiIyOqw4BAREZHVYcEhIiIiq8OCQ0RERFbHpEc1mKubmzer1WqRkxAREVFz3fy53ZxDGGyy4JSVlQEAQkJCRE5CRERELVVWVgaFQtHkNTZ5FpVer0deXh7c3NwgkUhEyaBWqxESEoKcnByeh9UIfn2axq/PrfFr0zR+fZrGr0/TxP76CIKAsrIyBAYGQiptepaNTY7gSKVSBAcHix0DAODu7s5/RE3g16dp/PrcGr82TePXp2n8+jRNzK/P7UZubuIkYyIiIrI6LDhERERkdVhwRCKXy/Hmm29CLpeLHcUs8evTNH59bo1fm6bx69M0fn2aZklfH5ucZExERETWjSM4REREZHVYcIiIiMjqsOAQERGR1WHBISIiIqvDgmMGHnzwQYSGhsLR0REBAQGIj49HXl6e2LHMQlZWFmbMmIGIiAg4OTmhQ4cOePPNN1FTUyN2NLPx7rvvYtCgQXB2doaHh4fYcUS3fPlyREREwNHREdHR0Thw4IDYkczC/v37MX78eAQGBkIikeDnn38WO5LZWLJkCfr37w83Nzf4+vpi4sSJSE9PFzuW2VixYgV69uxp3NwvNjYWO3bsEDvWbbHgmIERI0bghx9+QHp6OjZt2oQrV67g0UcfFTuWWbhw4QL0ej1WrlyJs2fP4pNPPsEXX3yBRYsWiR3NbNTU1GDSpEl4/vnnxY4iuo0bN2LevHl47bXXkJqaiqFDh2LMmDHIzs4WO5roKioq0KtXL3z++ediRzE7+/btwwsvvIDk5GQkJiZCq9UiLi4OFRUVYkczC8HBwfjXv/6FlJQUpKSk4N5778WECRNw9uxZsaM1icvEzdDWrVsxceJEaDQa2Nvbix3H7HzwwQdYsWIFMjIyxI5iVhISEjBv3jyUlpaKHUU0MTEx6Nu3L1asWGG8r2vXrpg4cSKWLFkiYjLzIpFIsHnzZkycOFHsKGapsLAQvr6+2LdvH4YNGyZ2HLPk5eWFDz74ADNmzBA7yi1xBMfMFBcXY926dRg0aBDLzS2oVCp4eXmJHYPMTE1NDY4fP464uLh698fFxeHw4cMipSJLpFKpAIDfZxqh0+mwYcMGVFRUIDY2Vuw4TWLBMROvvvoqXFxc4O3tjezsbGzZskXsSGbpypUr+OyzzzB79myxo5CZKSoqgk6ng5+fX737/fz8oFQqRUpFlkYQBMyfPx9DhgxBVFSU2HHMxunTp+Hq6gq5XI7Zs2dj8+bN6Natm9ixmsSCYyJvvfUWJBJJk7eUlBTj9f/3f/+H1NRU7Nq1CzKZDNOnT4c1v3vY0q8PAOTl5WH06NGYNGkSZs6cKVLytnEnXx8ykEgk9f4sCEKD+4huZc6cOTh16hTWr18vdhSzEhkZibS0NCQnJ+P555/Hk08+iXPnzokdq0l2YgewVnPmzMHUqVObvCY8PNz43z4+PvDx8UHnzp3RtWtXhISEIDk52eyHAO9US78+eXl5GDFiBGJjY/Hll1+aOJ34Wvr1IcO/IZlM1mC0pqCgoMGoDlFjXnzxRWzduhX79+9HcHCw2HHMioODAzp27AgA6NevH44dO4ZPP/0UK1euFDnZrbHgmMjNwnInbo7caDSa1oxkVlry9cnNzcWIESMQHR2Nb775BlKp9Q883s3fH1vl4OCA6OhoJCYm4qGHHjLen5iYiAkTJoiYjMydIAh48cUXsXnzZuzduxcRERFiRzJ7giCY/c8oFhyRHT16FEePHsWQIUPg6emJjIwMvPHGG+jQoYPVjt60RF5eHoYPH47Q0FB8+OGHKCwsNH7M399fxGTmIzs7G8XFxcjOzoZOp0NaWhoAoGPHjnB1dRU3XBubP38+4uPj0a9fP+NoX3Z2NudsASgvL8fly5eNf87MzERaWhq8vLwQGhoqYjLxvfDCC/j++++xZcsWuLm5GUcBFQoFnJycRE4nvkWLFmHMmDEICQlBWVkZNmzYgL1792Lnzp1iR2uaQKI6deqUMGLECMHLy0uQy+VCeHi4MHv2bOHatWtiRzML33zzjQCg0RsZPPnkk41+ffbs2SN2NFH85z//EcLCwgQHBwehb9++wr59+8SOZBb27NnT6N+TJ598UuxoorvV95hvvvlG7Ghm4ZlnnjH+m2rXrp1w3333Cbt27RI71m1xHxwiIiKyOtY/mYGIiIhsDgsOERERWR0WHCIiIrI6LDhERERkdVhwiIiIyOqw4BAREZHVYcEhIiIiq8OCQ0RERFaHBYeIiIisDgsOERERWR0WHCIiIrI6LDhERERkdf4/DgBRHCrbBL4AAAAASUVORK5CYII="
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# You may use this cell to run your observations\n",
"x = torch.linspace(-math.pi, math.pi, 1000)\n",
"# Original true values\n",
"y = torch.sin(x)\n",
"plt.plot(x, y, linestyle='solid', label='sin(x)')\n",
"print(polyfit(x, y, mse, 6, 1e-5, 5000))\n",
"\n",
"# mse = lambda y_true, y_pred: torch.mean(torch.square(y_pred - y_true))\n",
"\n",
"# a, b, c, d = polyfit(x, y, mse, 3, 1e-3, 5000)\n",
"# y_pred_mse = a + b * x + c * x ** 2 + d * x ** 3\n",
"# plt.plot(x, y_pred_mse.detach().numpy(), linestyle='dashed', label=f'3,1e-3')\n",
"\n",
"# a, b, c, d = polyfit(x, y, mse, 3, 1e-6, 5000)\n",
"# y_pred_mse = a + b * x + c * x ** 2 + d * x ** 3\n",
"# plt.plot(x, y_pred_mse.detach().numpy(), linestyle='dashed', label=f'3, 1e-6')\n",
"\n",
"# # a, b, c, d =polyfit(x, y, mse, 3, 1e6, 5000)\n",
"# # y_pred_mse = a + b * x + c * x ** 2 + d * x ** 3\n",
"# # plt.plot(x, y_pred_mse.detach().numpy(), linestyle='dashed', label=f'3, 1e6')\n",
"\n",
"# a, b = polyfit(x, y, mse, 1, 1e-3, 5000)\n",
"# y_pred_mse = a + b * x\n",
"# plt.plot(x, y_pred_mse.detach().numpy(), linestyle='dashed', label=f'1, 1e-3')\n",
"\n",
"# a,b,c,d,e,f,g = polyfit(x, y, mse, 6, 1e-5, 5000)\n",
"# y_pred_mse = a + b * x + c * x ** 2 + d * x ** 3 + e * x ** 4 + f * x ** 5 + g * x ** 6\n",
"# plt.plot(x, y_pred_mse.detach().numpy(), linestyle='dashed', label=f'6, 1e-5')\n",
"\n",
"\n",
"# plt.axis('equal')\n",
"# plt.title('Comparison of different fits')\n",
"# plt.legend()\n",
"# plt.show()\n"
]
},
{
"cell_type": "markdown",
"id": "4e2fb6c5",
"metadata": {},
"source": [
"---\n",
"# 2 Computing gradients for arbitrary graphs\n",
"\n",
"Recall the neural network for `y = |x-1|` from the lecture. We are going to implement forward propagation as mentioned during lecture. This forward pass is the act of feeding data into our input layer, which will then be passed to and processed by the hidden layers according to the different activation functions specific to each perceptron. After passing through all the hidden layers, our neural network will generate an output, $\\hat{y}$, that is hopefully meaningful to our problem at hand.\n",
"\n",
"<img src=\"imgs/img_toy_nn.jpg\" width=\"800\">\n",
"\n",
"### Task 2.1 - Forward pass\n",
"\n",
"In this task, you are required implement the function `forward_pass` that takes in 4 arguments: \n",
"1. `x`, the input values (not including bias)\n",
"2. `w0`, (2x2) weights of the hidden layer\n",
"3. `w1`, (3x1) weights of the output layer\n",
"4. `activation_fn`, the activation function of the hidden layer.\n",
"\n",
"*Note: As in the lecture, there will be no activation for the output layer (i.e. the activation function of the output layer is the identity function `lambda x: x`)*"
]
},
{
"cell_type": "code",
"execution_count": 83,
"id": "4d97ca45",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T03:01:38.802087Z",
"start_time": "2024-04-02T03:01:38.795809Z"
}
},
"outputs": [
{
"data": {
"text/plain": "tensor([[3.],\n [2.],\n [1.],\n [0.],\n [1.]], grad_fn=<MmBackward0>)"
},
"execution_count": 83,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# This is the same as unsqueeze(1)\n",
"x = torch.linspace(-10, 10, 1000).reshape(-1, 1)\n",
"y = torch.abs(x-1)\n",
"\n",
"def forward_pass(x, w0, w1, activation_fn):\n",
" n = x.shape[0]\n",
" x = torch.cat((torch.ones(n, 1), x), 1)\n",
" # Perform a forward pass\n",
" a = torch.matmul(x, w0)\n",
" h = activation_fn(a)\n",
" h = torch.cat((torch.ones(n, 1), h), 1)\n",
" y_pred = torch.matmul(h, w1)\n",
" return y_pred\n",
"\n",
"# Exact weights\n",
"w0 = torch.tensor([[-1., 1.], [1., -1.]], requires_grad=True)\n",
"w1 = torch.tensor([[0.], [1.], [1.]], requires_grad=True)\n",
"\n",
"# Performing a forward pass on exact solution for weights will give us the correct y values\n",
"x_sample = torch.linspace(-2, 2, 5).reshape(-1, 1)\n",
"forward_pass(x_sample, w0, w1, torch.relu) # tensor([[3.], [2.], [1.], [0.], [1.]])"
]
},
{
"cell_type": "code",
"execution_count": 84,
"id": "8a3184ab",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T03:01:43.470100Z",
"start_time": "2024-04-02T03:01:43.465055Z"
}
},
"outputs": [],
"source": [
"w0 = torch.tensor([[-1., 1.], [1., -1.]], requires_grad=True)\n",
"w1 = torch.tensor([[0.], [1.], [1.]], requires_grad=True)\n",
"\n",
"output0 = forward_pass(torch.linspace(0,1,50).reshape(-1, 1), w0, w1, torch.relu)\n",
"x_sample = torch.linspace(-2, 2, 5).reshape(-1, 1)\n",
"test1 = forward_pass(x_sample, w0, w1, torch.relu).tolist()\n",
"output1 = [[3.], [2.], [1.], [0.], [1.]]\n",
"\n",
"assert output0.shape == torch.Size([50, 1])\n",
"assert test1 == output1"
]
},
{
"cell_type": "markdown",
"id": "9c8033e1",
"metadata": {},
"source": [
"### Task 2.2 - Backward propagation\n",
"\n",
"In this task, you will start with random weights for `w0` and `w1`, and iteratively perform forward passes and backward propagation multiple times to converge on a solution.\n",
"\n",
"Submit your values of `w0`, `w1`, and `loss` value onto Coursemology. Your `loss` value should be less than 1."
]
},
{
"cell_type": "code",
"execution_count": 101,
"id": "d79c3395",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T03:12:29.008533Z",
"start_time": "2024-04-02T03:12:18.880737Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"iter\tloss\t\n",
"----\t----\n",
"1000\t10.045333862304688\n",
"2000\t8.946976661682129\n",
"3000\t8.083948135375977\n",
"4000\t7.403078556060791\n",
"5000\t6.8642473220825195\n",
"6000\t6.436517715454102\n",
"7000\t6.095942497253418\n",
"8000\t5.823876857757568\n",
"9000\t5.605810165405273\n",
"10000\t5.430357456207275\n",
"11000\t5.288575649261475\n",
"12000\t5.173434257507324\n",
"13000\t5.079381465911865\n",
"14000\t5.002045154571533\n",
"15000\t4.937958240509033\n",
"16000\t4.884366035461426\n",
"17000\t4.839096546173096\n",
"18000\t4.800426006317139\n",
"19000\t4.766984462738037\n",
"20000\t4.737689018249512\n",
"21000\t4.711678504943848\n",
"22000\t4.688270092010498\n",
"23000\t4.666922092437744\n",
"24000\t4.647202491760254\n",
"25000\t4.628774642944336\n",
"26000\t4.6113667488098145\n",
"27000\t4.594768047332764\n",
"28000\t4.578812599182129\n",
"29000\t4.563370704650879\n",
"30000\t4.548344612121582\n",
"31000\t4.533658981323242\n",
"32000\t4.519261837005615\n",
"33000\t4.505115032196045\n",
"34000\t4.4911980628967285\n",
"35000\t4.477519512176514\n",
"36000\t4.464054584503174\n",
"37000\t4.450812339782715\n",
"38000\t4.437822341918945\n",
"39000\t4.425092697143555\n",
"40000\t4.412638187408447\n",
"41000\t4.400496482849121\n",
"42000\t4.388679027557373\n",
"43000\t4.377202033996582\n",
"44000\t4.366110324859619\n",
"45000\t4.355399131774902\n",
"46000\t4.345085620880127\n",
"47000\t4.3351898193359375\n",
"48000\t4.325725555419922\n",
"49000\t4.316679000854492\n",
"50000\t4.308060169219971\n",
"51000\t4.299867153167725\n",
"52000\t4.292091369628906\n",
"53000\t4.284744739532471\n",
"54000\t4.277796745300293\n",
"55000\t4.271248817443848\n",
"56000\t4.265082359313965\n",
"57000\t4.259285926818848\n",
"58000\t4.25383996963501\n",
"59000\t4.248725891113281\n",
"60000\t4.2439470291137695\n",
"61000\t4.23946475982666\n",
"62000\t4.235273838043213\n",
"63000\t4.231363296508789\n",
"64000\t4.227708339691162\n",
"65000\t4.224303722381592\n",
"66000\t4.221127986907959\n",
"67000\t4.218160152435303\n",
"68000\t4.215404510498047\n",
"69000\t4.212835788726807\n",
"70000\t4.210443019866943\n",
"71000\t4.208215713500977\n",
"72000\t4.206140995025635\n",
"73000\t4.204216003417969\n",
"74000\t4.202425003051758\n",
"75000\t4.2007598876953125\n",
"76000\t4.199214935302734\n",
"77000\t4.197778701782227\n",
"78000\t4.1964430809021\n",
"79000\t4.195202827453613\n",
"80000\t4.194048881530762\n",
"81000\t4.192974090576172\n",
"82000\t4.19197940826416\n",
"83000\t4.191056728363037\n",
"84000\t4.190197467803955\n",
"85000\t4.189395904541016\n",
"86000\t4.188657760620117\n",
"87000\t4.187970161437988\n",
"88000\t4.187328338623047\n",
"89000\t4.1867356300354\n",
"90000\t4.186183452606201\n",
"91000\t4.18567419052124\n",
"92000\t4.185197830200195\n",
"93000\t4.184756278991699\n",
"94000\t4.184344291687012\n",
"95000\t4.183964252471924\n",
"96000\t4.1836090087890625\n",
"97000\t4.183282375335693\n",
"98000\t4.182975769042969\n",
"99000\t4.182692527770996\n",
"100000\t4.182427883148193\n",
"--- w0 ---\n",
"tensor([[-1.5630, -1.1366],\n",
" [-0.6188, -0.1137]], requires_grad=True)\n",
"--- w1 ---\n",
"tensor([[ 3.7083],\n",
" [ 1.5504],\n",
" [-0.6191]], requires_grad=True)\n",
"--- w1 ---\n",
"tensor(4.1824, grad_fn=<MeanBackward0>)\n"
]
},
{
"data": {
"text/plain": "<Figure size 640x480 with 1 Axes>",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGxCAYAAABfrt1aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABJFklEQVR4nO3deViU9f7G8XtYZmAURsEFSDZNTdMUd2kR09y3zknNSsXtl7lri0t6JCtR0/JkLqmIeiytjssxNZWOoCdBQ8XMJdRARYEUU0DQGZbv7w9iZGCGdWaeWe7Xdc11wcx35vkMY8y7Z2YeZEIIASIiIiIJOEg9ABEREdkvhggRERFJhiFCREREkmGIEBERkWQYIkRERCQZhggRERFJhiFCREREkmGIEBERkWQYIkRERCQZhghRKZs3b4ZMJtN7euedd3Dt2jXIZDJs3rxZe53Y2FiEhYXh/v37ldpGWFgYZDIZGjRogOzs7DKXBwQEYMCAATrnFc+wZMkSgzOfOnWqSvdVajExMZDJZPj3v/9t0u2sWrUKTz75JORyOWQyWaUfJ1M4cOAAwsLC9F4WEBCA0NBQs85DJDWGCJEBkZGRiIuL0zlNmzYN3t7eiIuLQ//+/bVrY2Nj8cEHH1T5Ce7OnTtYtmxZla6zZMkS/Pnnn1W6jj07e/Yspk2bhu7du+PIkSOIi4uDm5ubZPMcOHAAH3zwgd7Ldu/ejQULFph5IiJpOUk9AJGlatWqFTp06KD3si5duhhlG3369MFnn32GyZMnw8vLq8L1PXv2RExMDD7++GOsWLHCKDPYugsXLgAAJkyYgE6dOkk8TfmCgoKkHoHI7LhHhKiKSr80ExYWhnfffRcAEBgYqH0JJSYmpsLb+uijj5Cfn29wV31pzZs3x7hx47B69Wpcv369WvOfP38egwcPRt26deHi4oK2bdtiy5YtOmuKXzLZvn073n//ffj4+MDd3R09e/ZEYmJihdu4evUqxowZg6ZNm0KpVOKJJ57AwIED8euvv+pd/+jRI8yaNQteXl5wdXVFt27dkJCQoLMmKSkJr776Knx8fKBQKNCwYUP06NEDZ8+eNThHSEgI3njjDQBA586dIZPJtC99GHoZJCQkBCEhIdX+WRw8eBA9evSASqWCUqlEixYtEB4eDgAIDQ3F6tWrAUDnJb9r164ZnOnGjRt444030KBBAygUCrRo0QIrVqxAYWGhdk3xv8nly5fj008/RWBgIGrXro2uXbvixIkTBn8+RJaAIUJkQEFBAfLz83VO+owfPx5Tp04FAOzatUv7Mk67du0q3Ia/vz8mTZqEiIgIXL58uVJzhYWFwdHRsVq78BMTExEcHIwLFy7g888/x65du9CyZUuEhobqfYlo3rx5uH79OjZu3Ij169fjypUrGDhwIAoKCsrdTmpqKjw9PbFkyRIcPHgQq1evhpOTEzp37qz3yXvevHlISkrCxo0bsXHjRqSmpiIkJARJSUnaNf369cPp06exbNkyREVFYe3atQgKCir35bA1a9Zg/vz5AB6/1Fbdlz4q87OIiIhAv379UFhYiHXr1uH777/HtGnTcPPmTQDAggUL8MorrwCAzkt+3t7eerd5584dBAcH4/Dhw/jwww+xd+9e9OzZE++88w6mTJlSZv3q1asRFRWFlStX4quvvkJOTg769euHzMzMat1nIrMQRKQjMjJSANB7ysvLE8nJyQKAiIyM1F7nk08+EQBEcnJypbaxcOFCAUDcuXNHZGRkCJVKJf7+979rL/f39xf9+/fXuQ4AMXnyZCGEEO+//75wcHAQv/zyi87M8fHx5W731VdfFQqFQty4cUPn/L59+wqlUinu378vhBAiOjpaABD9+vXTWfftt98KACIuLq5S97NYfn6+0Gg0omnTpmLmzJna84u3065dO1FYWKg9/9q1a8LZ2VmMHz9eCCFERkaGACBWrlxZpe0KYfhn4+/vL0aPHl1mfbdu3US3bt3KzFjRzyI7O1u4u7uL5557Tue+lDZ58mRh6Fdv6ZnmzJkjAIiTJ0/qrHvrrbeETCYTiYmJQgih/TfZunVrkZ+fr133888/CwBi+/btBuchkhr3iBAZsHXrVsTHx+ucnJyM/7YqT09PzJ49Gzt37sTJkycrdZ333nsPHh4emD17dpW2deTIEfTo0QO+vr4654eGhiI3NxdxcXE65w8aNEjn+2eeeQYAKnxZKD8/H4sXL0bLli0hl8vh5OQEuVyOK1eu4NKlS2XWv/baa5DJZNrv/f39ERwcjOjoaACAh4cHmjRpgk8++QSffvopEhISdF6aMIeKfhaxsbHIysrCpEmTdO5LTRw5cgQtW7Ys896W0NBQCCFw5MgRnfP79+8PR0dHgzMSWSKGCJEBLVq0QIcOHXROpjJjxgz4+Pjgvffeq9R6d3d3zJ8/HwcPHtQ+WVfG3bt39b4M4OPjo728JE9PT53vFQoFAODhw4flbmfWrFlYsGABhgwZgu+//x4nT55EfHw82rRpo/e6+t6o6+XlpZ1HJpPhv//9L3r37o1ly5ahXbt2qF+/PqZNm6b348+mUNHP4s6dOwCARo0aGW2b5nq8iKTEECGyAK6urggLC8OxY8ewf//+Sl3nrbfeQmBgIGbPng0hRKWu4+npibS0tDLnp6amAgDq1atX+aHLsW3bNowaNQqLFy9G79690alTJ3To0AEZGRl616enp+s9r+QTq7+/PyIiIpCeno7ExETMnDkTa9as0b5RuKpcXFygVqvLnG9oxorUr18fALTvBzEGcz1eRFJiiBAZgTH+z3Ps2LFo0aIF5syZU6mXHeRyOT766CPEx8fju+++q9Q2evTogSNHjmifyIpt3boVSqXSaB9Llslk2p9Jsf379+PWrVt612/fvl0npq5fv47Y2FidT6+U1KxZM8yfPx+tW7fGmTNnqjVjQEAAzp07p3Pe5cuXK/WpIH2Cg4OhUqmwbt26csOwKv9WevTogYsXL5a5j1u3boVMJkP37t2rNSuRJWGIEBlB69atAQD//Oc/ERcXh1OnTlX5JQNHR0csXrwY58+fR0pKSqWuM2LECAQFBeGHH36o1PqFCxfC2dkZ3bt3x1dffYUffvgBb7zxBvbv34+wsDCoVKoqzWzIgAEDsHnzZqxcuRJHjhzBJ598gjFjxhh82eL27dt4+eWXsX//fnz99dfo2bMnXFxcMHfuXADAuXPn8MILL2DVqlU4ePAgjhw5gvnz5+PcuXN46aWXqjXjyJEjcfHiRUyaNAn//e9/sWnTJgwaNEi7Z6OqateujRUrVuDYsWPo2bMnduzYgejoaGzYsEHnEy7F/1aWLl2KkydP4tSpU9BoNHpvc+bMmXjiiSfQv39/bNiwAYcPH8b06dOxZs0avPXWW2jWrFm1ZiWyJDygGZERhISEYO7cudiyZQs2bNiAwsJCREdHG/w/ekOGDBmC4OBgxMbGVmq9TCbD0qVL0atXr0qtb968OWJjYzFv3jxMnjwZDx8+RIsWLRAZGWnUQ4v/85//hLOzM8LDw/HgwQO0a9cOu3bt0n6UtrTFixcjPj4eY8aMQVZWFjp16oQdO3agSZMmAIreL9KkSROsWbMGKSkpkMlkaNy4MVasWKH96HRVvfbaa0hNTcW6desQGRmJVq1aYe3atQaPeloZ48aNg4+PD5YuXYrx48dDCIGAgACMHj1aZ7vHjx/HmjVrsGjRIgghkJycjICAgDK3V79+fcTGxmLu3LmYO3cusrKy0LhxYyxbtgyzZs2q9pxElkQmKvviMhEREZGR8aUZIiIikgxDhIiIiCTDECEiIiLJMESIiIhIMgwRIiIikgxDhIiIiCRj0ccRKSwsRGpqKtzc3Iz2R6SIiIjItIQQyM7Oho+PDxwcyt/nYdEhkpqaWuavhBIREZF1SElJqfAPQVp0iLi5uQEouiPu7u4ST0NERESVkZWVBV9fX+3zeHksOkSKX45xd3dniBAREVmZyrytgm9WJSIiIskwRIiIiEgyDBEiIiKSjEW/R4SIiKyHEAL5+fkoKCiQehQyA2dnZzg6Otb4dhgiRERUYxqNBmlpacjNzZV6FDITmUyGRo0aoXbt2jW6HYYIERHVSGFhIZKTk+Ho6AgfHx/I5XIehNLGCSFw584d3Lx5E02bNq3RnhGGCBER1YhGo0FhYSF8fX2hVCqlHofMpH79+rh27Rry8vJqFCJ8syoRERlFRYfyJttirL1e/FdDREREkmGIEBERkWQYIkREZLdiYmIQEBBg8u2EhoYiLCzM5NuxRgwRIiKiGli/fj1CQkLg7u4OmUyG+/fvSz2SVWGIEBER1UBubi769OmDefPmST2KVeLHd4mIyOiEEHiYZ/4jrLo6O1b70xxbt27FpEmTkJCQgKZNmwIApk6dikOHDiEhIQG1atXSe70ZM2YAKHqZh6qOIUJEREb3MK8ALf9xyOzbvbioN5Ty6j21jRo1Cvv27cPrr7+O2NhY/Pjjj/jyyy9x/PhxgxFCNceXZoiIiP7y5ZdfIi0tDdOmTUNoaCgWLlyIjh07Sj2WTeMeESIiMjpXZ0dcXNRbku3WRN26dREREYHevXsjODgYc+bM0V62ePFiLF68WPv9xYsX4efnV6PtEUOEiIhMQCaTVfslEqkdO3YMjo6OSE1NRU5ODtzd3QEAEydOxLBhw7TrfHx8pBrRpvClGSIior/ExsZi2bJl+P777+Hu7o6pU6dqL/Pw8MCTTz6pPTk5WWdoWRr+FImIiABkZ2dj5MiRmDp1Kvr27Qs/Pz906NABAwYMwNChQw1eLz09Henp6bh69SoA4Ndff4Wbmxv8/Pzg4eFhrvGtFveIEBERAZg+fTpq1aqlfR/I008/jaVLl2LixIm4deuWweutW7cOQUFBmDBhAgDghRdeQFBQEPbu3WuWua0dQ4SIiAjApk2bcO7cOSgUCu1506ZNw927d/HEE08YvF5YWBiEEGVOoaGhZpja+jFEiIiISDIMESIiIpIM36xKRER2KyAgQHuIdlMaMmQI6tSpY/LtWCOGCBER2S1zhgjpx5dmiIiISDIMESIiIpIMQ4SIiIgkwxAhIiIiyTBEiIiISDLVDpFjx45h4MCB8PHxgUwmw549e7SX5eXlYfbs2WjdujVq1aoFHx8fjBo1CqmpqcaYmYiIiGxEtUMkJycHbdq0wRdffFHmstzcXJw5cwYLFizAmTNnsGvXLly+fBmDBg2q0bBERERUJCAgACtXrpR6jBqr9nFE+vbti759++q9TKVSISoqSue8VatWoVOnTrhx4wb8/Pyqu1kiIiKLVVBQAJlMBgcHy3jng6XNo4/ZJsvMzIRMJiv3yHJqtRpZWVk6JyIiIlMJCQnBlClTMGXKFNSpUweenp6YP38+hBAAAI1Gg/feew9PPPEEatWqhc6dOyMmJkZ7/c2bN6NOnTrYt28fWrZsCYVCgevXr0OtVuO9996Dr68vFAoFmjZtioiICO31Ll68iH79+qF27dpo2LAhRo4ciYyMjErPFRISguvXr2PmzJmQyWSQyWTlznPv3j2MGjUKdevWhVKpRN++fXHlypUy9+PQoUNo0aIFateujT59+iAtLc2UP34AZgqRR48eYc6cOXjttdfg7u5ucF14eDhUKpX25Ovra47xiIjIVDQ5hk95j6qw9mHFa6tpy5YtcHJywsmTJ/H555/js88+w8aNGwEAY8aMwfHjx7Fjxw6cO3cOQ4cORZ8+fXSexHNzcxEeHo6NGzfiwoULaNCgAUaNGoUdO3bg888/x6VLl7Bu3TrUrl0bAJCWloZu3bqhbdu2OHXqFA4ePIg//vgDw4YNq/Rcu3btQqNGjbBo0SKkpaXpBIO+eUJDQ3Hq1Cns3bsXcXFxEEKgX79+yMvL07ne8uXL8a9//QvHjh3DjRs38M4771T751ppwggAiN27d+u9TKPRiMGDB4ugoCCRmZlZ7u08evRIZGZmak8pKSkCQIXXIyIi6Tx8+FBcvHhRPHz4sOyFC90Nn7a9orv2Iy/Dazf10127NLDsmmro1q2baNGihSgsLNSeN3v2bNGiRQtx9epVIZPJxK1bt3Su06NHDzF37lwhhBCRkZECgDh79qz28sTERAFAREVF6d3mggULRK9evXTOK36+S0xMrHCuYv7+/uKzzz7TuR1981y+fFkAEMePH9eel5GRIVxdXcW3336rc72rV69q16xevVo0bNhQ730QovzHPTMzs9LP3yb9WzN5eXkYNmwYkpOTceTIkXL3hgCAQqGAQqEw5UhEREQ6unTpon1pAwC6du2KFStW4NSpUxBCoFmzZjrr1Wo1PD09td/L5XI888wz2u/Pnj0LR0dHdOvWTe/2Tp8+jejoaO0ekpJ+//137fYMzVVQUABHR0eD96f0PJcuXYKTkxM6d+6sPc/T0xPNmzfHpUuXtOcplUo0adJE+723tzdu375tcDvGYrIQKY6QK1euIDo6WudBIyIiOzGvnMM2yEo9mb57tZy1pd5JMOPX6s9UBY6Ojjh9+nSZJ/6SEeHq6qoTDK6uruXeZmFhIQYOHIilS5eWuczb27uGE5edR/z1vpLShBA665ydnXUul8lkBq9rTNUOkQcPHuDq1cf/aJKTk3H27Fl4eHjAx8cHr7zyCs6cOYN9+/ahoKAA6enpAAAPDw/I5fKaT05ERJZPXkv6tRU4ceJEme+bNm2KoKAgFBQU4Pbt23j++ecrfXutW7dGYWEhjh49ip49e5a5vF27dti5cycCAgLg5GT4adjQXMVRJJfLUVBQUOE8LVu2RH5+Pk6ePIng4GAAwN27d3H58mW0aNGi0vfLVKr9ZtVTp04hKCgIQUFBAIBZs2YhKCgI//jHP3Dz5k3s3bsXN2/eRNu2beHt7a09xcbGGm14IiKimkpJScGsWbOQmJiI7du3Y9WqVZg+fTqaNWuG119/HaNGjcKuXbuQnJyM+Ph4LF26FAcOHDB4ewEBARg9ejTGjh2LPXv2IDk5GTExMfj2228BAJMnT8aff/6JESNG4Oeff0ZSUhIOHz6MsWPH6oSFoblKbufYsWO4deuWziduSmvatCkGDx6MCRMm4KeffsIvv/yCN954A0888QQGDx5shJ9gzVR7j0hISEi5u2zMsTuHiIiopkaNGoWHDx+iU6dOcHR0xNSpU/F///d/AIDIyEh89NFHePvtt3Hr1i14enqia9eu6NevX7m3uXbtWsybNw+TJk3C3bt34efnh3nz5gEAfHx8cPz4ccyePRu9e/eGWq2Gv78/+vTpo3O8j/LmAoBFixbhzTffRJMmTaBWq8t93o2MjMT06dMxYMAAaDQavPDCCzhw4ECZl2OkIBMWXAxZWVlQqVTIzMys8I2uREQkjUePHiE5ORmBgYFwcXGRepwqCQkJQdu2bS3uCKWWOldJ5T3uVXn+ttxDrREREZHNY4gQERGRZEx6HBEiIiJLVvJw7ZbEUucyBe4RISIiIskwRIiIyCgs+LMPZALGerwZIkREVCPFHwHNzc2VeBIyJ41GAwDlHm6+MvgeESIiqhFHR0fUqVNH+3dJlEqlzqHDyfYUFhbizp07UCqV5R4dtjIYIkREVGNeXl4AYJY/kkaWwcHBAX5+fjWOToYIERHVmEwmg7e3Nxo0aIC8vDypxyEzkMvlOkeCrS6GCBERGY2jo2ON3zNA9oVvViUiIiLJMESIiIhIMgwRIiIikoxdhkhBocDC/5zH6ev3pB6FiIjIrtlliET8lIQtcdcxetPPOHODMUJERCQVuwyRN7r4o0tjDzxQ52NUBGOEiIhIKnYZIkq5EzaFdsRrje4wRoiIiCRklyECAMozG7A4YzqW1D/EGCEiIpKI3YYI1A8AAK9mb8HKev/BA3UeY4SIiMjM7DdEur0L9PoYADDkwTdY6/ktY4SIiMjM7DdEACB4CtB/BQCgb85/EOGxDblqDWOEiIjITOw7RACg43hgyFpA5oAeuT9gS91N3DNCRERkJgwRAGj7GvD3jYCDEzp3fQFdGnvyDaxERERmwBAp1urvwKQTkL8wE5tCO/I4I0RERGbAECmpXlMAfx1nZERzrPH8FgXqB4wRIiIiE2GIGKDc+yb65ezBbvflkKmzGCNEREQmwBAxpNtswEWFpzQXscdtKRzV9xkjRERERsYQMaRRB2D0PkDpiSZ5V7C39mK4qO8yRoiIiIyIIVIe72eA0ANA7Ybwz7+G/9T+GLXVfzBGiIiIjIQhUpEGTwFjfgDcG+GJ/JvY4raWxxkhIiIyEoZIZXg2Acb+ADTqCL/QDTzOCBERkZEwRCqrjh8wLgquT7TSHmdEo37IGCEiIqoBhkhVyGQAio4zsjnkEeKUb8Nfc4UxQkREVE3VDpFjx45h4MCB8PHxgUwmw549e3QuF0IgLCwMPj4+cHV1RUhICC5cuFDTeS2DEHCJ+xSehRn41mUxmmkuMkaIiIiqodohkpOTgzZt2uCLL77Qe/myZcvw6aef4osvvkB8fDy8vLzw0ksvITs7u9rDWgyZDBj+FeAXjFoiB1+7LEGrvHOMESIioiqSCSFEjW9EJsPu3bsxZMgQAEV7Q3x8fDBjxgzMnj0bAKBWq9GwYUMsXboUb775ZqVuNysrCyqVCpmZmXB3d6/pmManyQF2vAYkxUADOSZoZuK0c3tsHdcJ7fzqSj0dERGRJKry/G2S94gkJycjPT0dvXr10p6nUCjQrVs3xMbGGryeWq1GVlaWzsmiyWsBI74BmvWBHBpEyFcgOC+Oe0aIiIgqySQhkp6eDgBo2LChzvkNGzbUXqZPeHg4VCqV9uTr62uK8YzL2QUY9i+g5RA4IR8jVWf50V4iIqJKMumnZmR/fcqkmBCizHklzZ07F5mZmdpTSkqKKcczHic58PcIoO8ytJ+2HV0aezBGiIiIKsEkIeLl5QUAZfZ+3L59u8xekpIUCgXc3d11TlbD0Qno/CaUrq7YFNoRXQProE1eAmOEiIioHCYJkcDAQHh5eSEqKkp7nkajwdGjRxEcHGyKTVoUpbMjtnp9i6/k4XgtfzdjhIiIyACn6l7xwYMHuHr1qvb75ORknD17Fh4eHvDz88OMGTOwePFiNG3aFE2bNsXixYuhVCrx2muvGWVwS+fsVh8AMM95O1zzNBgVIbB1XGd+moaIiKiEaofIqVOn0L17d+33s2bNAgCMHj0amzdvxnvvvYeHDx9i0qRJuHfvHjp37ozDhw/Dzc2t5lNbOpkMeHE+4OQCHPkQM513wjVfjVERYIwQERGVYJTjiJiKxR9HpDLi1gCH5gIAtuS/hOWO47BlXBfGCBER2ayqPH9Xe48IVVLXSYCzK8S+mRjtFAXH/EKMinDgQc+IiIjAP3pnHh3GQPbylxCOCqTXf5Yf7SUiIvoLQ8Rc2gyHbPpZTHprOo8zQkRE9BeGiDm5+0Apd8Km0I4Y4JeHBQVr8H8R/2OMEBGR3eJ7RCSgdHLA51gGB6cL8Cu4jYkRhVg3rhvfM0JERHaHe0Sk4OAAh/4rIOS10dXxItaJDzE5Ipp7RoiIyO4wRKTi3xWy0XshXOqgncNVbBAfYFrEj4wRIiKyKwwRKT3RHrLQ/RDKemjlcA0RIgwzIw4zRoiIyG4wRKTm1QqyMT+gsLYXmjvcxOzCjfw0DRER2Q2GiCWo3wwOY39AQeMe2OMzix/tJSIiu8EQsRQejeE4ahdWjntJe5yRtyJiGCNERGTTGCIWpvg4I3O84rEP0/FRxHeMESIislkMEQukdAQmuEajviwTmxCGZRFfM0aIiMgmMUQskaMTHEftQYFPe9SR5WADPsRnEVsZI0REZHMYIpbKtS4cR/8HBX7Pwk32EF/iY6yOiGCMEBGRURQUCiw9+Btu3X8o6RwMEUumcIPjG/9GQWB3KGVqrMESbIxYxxghIqIaKSgUmL3zHNbG/I6RG09Ck18o2SwMEUsnV8Lx9W+Q36wvFLI8NCu4zI/2EhFRtRVHyL9P34SjgwwzX2oGuZN0OcAQsQZOCjgN/xfUg9bhhO8EHmeEiIiqpXSErBzeFgPb+Eg6E0PEWjg6Q9FuBDaN6YQujT2Qr85BZMRqxggREVWKJUYIwBCxOkq5EzaNbINvVKuwSvYJDkeEMUaIiKhclhohAEPEKildXNGybTAAYI5sM45FzGWMEBGRXpYcIQBDxDrJZHDu8xHynnsPADBDth0/R8zCmet/SjwYERFZEkuPEIAhYr1kMjj3fB+a7gsBABNlu3Bu01TGCBERAbCOCAEYIlZP3m0WNL2WAABCZftwZtMMvkxDRGTnrCVCAIaITZAHvwV1/8/xQFYbezQd+dFeIiI7Zk0RAjBEbIai42g4zPgFtQM78DgjRER2ytoiBGCI2BSlqh42hXZEl8YeaKq5hLSI15GQlCb1WEREZAbWGCEAQ8TmKOVO2PR6K0S4/hP9ZcfxcMswJCSlSj0WERGZkLVGCMAQsUnKWm5QjojEI5kLgmXnULDlbzj7e4rUYxERkQlYc4QADBGb5dI0BBi5BzmyWugguwSHrUPwy5VrUo9FRERGZO0RAjBEbJpL465wGPM9shzc8YzsKuTbBuNc4lWpxyIiIiOwhQgBGCI2z9WvPZzHHsB9h7poIbuGc1/P56dpiIisnK1ECMAQsQuujVpDMeEgjrr2xIfq4fxoLxGRFbOlCAEYInbD1fspdJz5DYIae/11nJGTOJd4WeqxiIioCmwtQgATh0h+fj7mz5+PwMBAuLq6onHjxli0aBEKCwtNuVkyQCl3KjrOSGBdTC3YCq+ve+HiLz9LPRYREVWCLUYIADiZ8saXLl2KdevWYcuWLXj66adx6tQpjBkzBiqVCtOnTzflpsmAouOMtMSdlRfQIP8enHb9DZcKd6BF0HNSj0ZERAbYaoQAJt4jEhcXh8GDB6N///4ICAjAK6+8gl69euHUqVOm3CxVQFm7DupPicLvzk3hIcuGz55h+O3UEanHIiIiPWw5QgATh8hzzz2H//73v7h8uei9CL/88gt++ukn9OvXT+96tVqNrKwsnROZhrJOA3hPPYxE55ZQyXLg+/0IXP75oNRjERFRCbYeIYCJQ2T27NkYMWIEnnrqKTg7OyMoKAgzZszAiBEj9K4PDw+HSqXSnnx9fU05nt1TunvAd/oPOC9vi1qyR/DdPxJX4v4j9VhERAT7iBDAxCHyzTffYNu2bfj6669x5swZbNmyBcuXL8eWLVv0rp87dy4yMzO1p5QUHpbc1JS166Dx9H04o+gIBfKw8WA8P9pLRCQxe4kQAJAJIYSpbtzX1xdz5szB5MmTted99NFH2LZtG3777bcKr5+VlQWVSoXMzEy4u7ubakwCkPswFys3bML61EDUVjhh67hOaOdXV+qxiIjsji1ESFWev026RyQ3NxcODrqbcHR05Md3LZDSVYkZEyeiS2MPPFDn4+2Ig0g6uk3qsYiI7IotREhVmTREBg4ciI8//hj79+/HtWvXsHv3bnz66ad4+eWXTblZqqbi44x0D1BgvfgQjaMn48bhVVKPRURkF+wxQgATvzSTnZ2NBQsWYPfu3bh9+zZ8fHwwYsQI/OMf/4BcLq/w+nxpRhq56jwc/Xwc+uYUvXH1Zqf5aNTvXYmnIiKyXbYWIVV5/jZpiNQUQ0Q6ueo8HF41CUMefAsASG33DnwGLZB4KiIi22NrEQJY0HtEyHopFc7oNXUNvnEbBQDwObMc6bvmApbbrUREVscWI6SqGCJkkFLhjIFTP8NW9wkAAK9za3Br32KJpyIisg2MkCIMESqXUu6EV6YsQYRqCq4XNsCoU4E8zggRUQ0xQh5jiFCFlHInjJj8ARb4rMfvahVGRfxcFCN8mYaIqMoYIboYIlQpSrkT1o19Xnucka8jVuLe1teBfI3UoxERWQ1GSFkMEaq04uOMvOTviA+wFnWT9yNzy3Ag75HUoxERWTxGiH4MEaoSpdwJ/xzXEys9F+KhkEOVcgRZm/4GaHKkHo2IyGIxQgxjiFCVKeVOmDnxTSyt9zEeCBe4px3Hg42DgEeZUo9GRGRxGCHlY4hQtSjlTnjvzbFYUn8JMoUStW+fQs7GAUDun1KPRkRkMRghFWOIULUp5U6Y938jsbjBJ7gr3FAr4xxuRq+XeiwiIovACKkchgjViFLuhIUTXsXiBsuxJn8Q+vzclscZISK7xwipPIYI1ZhS7oQPJ7yCY36T8EBdgFERPyMhOR3IvCX1aEREZscIqRqGCBlF8Ud7uzT2wCP1I/y5+XVo1vcAMq5KPRoRkdkwQqqOIUJGUxwj3f3l8BVpkOekIS+iN/DHRalHIyIyOUZI9TBEyKiKjjPyEj7xXoGLhf5wfpiB/E19gdSzUo9GRGQyjJDqY4iQ0RXHyAqf5Thb2ARO6vvI3zwASPlZ6tGIiIyOEVIzDBEyCaXcCavGvoh/+izDycKn4KTJRsGWwUDyMalHIyIyGkZIzTFEyGSUciesHhuC1T5LcKygNdR5BUi8w79LQ0S2gRFiHAwRMqniv9q7odHHGKZZgL/vFzzOCBFZPUaI8TBEyOSUcid8OeZZ1A7sgAfqfIyK+BmXEo4D53dKPRoRUZUxQoyLIUJmUfI4I27qP9Bwz3CIf48DzmyVejQiokpjhBgfQ4TMpjhGAgKb4EBBJ8gggL1TgZP8+zREZPkYIabBECGzUsqdEDGmM/b5voMN+f2KzvzhXeCnlZLORURUHkaI6TBEyOyUcidsGtMJ//Wdis/zhxSd+eNCIDocEELS2YiISmOEmBZDhCRRHCOxfhOxLG940ZlHl/A9I0RkURghpuck9QBkv4rfMzJ2M/DBdTkGOMXDsU4PtJV6MCIiMELMhXtESFLFMXLJ/3UMU7+PN7ZeKDrOiBBAYaHU4xGRnWKEmA9DhCRXHCMdG9fXHmckde8HwK7xQEGe1OMRkZ1hhJgXQ4QsQsnjjNTTpKD+mc+LDnj27WggXy31eERkJxgh5scQIYtRHCNegU9jQt4sqIUzkLgf2D4C0ORKPR4R2ThGiDQYImRRimPkUUAPjMl7F7lCAfz+X+DrYYA6W+rxiMhGMUKkwxAhi1McI4UBL2CUZjYeCFfg2v+Af70MPLwv9XhEZGMYIdJiiJBFKo4Rp8BgvKaZh0xRC7gZD/x+ROrRiMiGMEKkxxAhi1UcI8rAjhiuWYBFYjzOuHeXeiwishGMEMtg8hC5desW3njjDXh6ekKpVKJt27Y4ffq0qTdLNqI4RuoEtsUm9YsYFfFz0XFGcu4CmTelHo+IrBQjxHKYNETu3buHZ599Fs7Ozvjhhx9w8eJFrFixAnXq1DHlZsnGlPxo7wN1PiZHRCN300BgU1/gzySpxyMiK8MIsSwmPcT70qVL4evri8jISO15AQEBptwk2ajHh4OPR3LSVfyR8ScCZWlAZD9g1F6gfjOpRyQiK8AIsTwm3SOyd+9edOjQAUOHDkWDBg0QFBSEDRs2GFyvVquRlZWlcyIqVhwjgY2fxDD1AlwRvkB2GhDZF0g/L/V4RGThGCGWyaQhkpSUhLVr16Jp06Y4dOgQJk6ciGnTpmHrVv1/YTU8PBwqlUp78vX1NeV4ZIWKY6RJ48YYpn4fF0QgkJsBbO4P3OJ7j4hIP0aI5ZIJIYSpblwul6NDhw6IjY3Vnjdt2jTEx8cjLi6uzHq1Wg21+vHhvLOysuDr64vMzEy4u7ubakyyQrmafIzdHI+LSSnYqvgEbWWXAbkb8MZOwK+z1OMRkQVhhJhfVlYWVCpVpZ6/TbpHxNvbGy1bttQ5r0WLFrhx44be9QqFAu7u7jonIn2K94y0bOyL19Rz8LN4GhonJVC7gdSjEZEFYYRYPpOGyLPPPovExESd8y5fvgx/f39TbpbsRHGMPNPYByPV72JQznyceVBH6rGIyEIwQqyDSUNk5syZOHHiBBYvXoyrV6/i66+/xvr16zF58mRTbpbsSHGMBDX2wm9qz8fHGUn8AbiwR+rxiEgijBDrYdL3iADAvn37MHfuXFy5cgWBgYGYNWsWJkyYUKnrVuU1JrJvxe8ZOZH0J9orbuI7x/lwEPnAkLVAm1elHo+IzIgRIr2qPH+bPERqgiFCVVEcIz8nZWCFYiNelsUAkAEDPgM6jJF6PCIyA0aIZbCYN6sSmVPxyzSdGtfDLPV4fC16AxDAvhlA3BqpxyMiE2OEWCeGCNmU4hjp3Lge5qlHYZMYVHTBobnAsU+kHY6ITIYRYr0YImRzHv9tGk8sUg/HajG06IIjHwGX9kk7HBEZHSPEujFEyCaVjJFP1C9jhXgDfzYZAjTvK/VoRGREjBDrxxAhm1Xyr/auUvfDC1dH4MzNv/5+UUE+UFgo7YBEVCOMENvAECGbVjJGHqgLio4zci0D2DMR+M+koiAhIqvDCLEdDBGyeboxko/lkdshzu8CftkO7BwH5GukHpGIqoARYlsYImQXSsZIrLoxZhTORKGDM3BxD/DtSCDvkdQjElElMEJsD0OE7EbJGPmPuh3eKngXhY4K4PJBYPtwQJMj9YhEVA5GiG1iiJBdKRkjh9StMC5/DgqclEBSDLDtFeBRltQjEpEejBDbxRAhu1MyRqLVzTEqbx7y5W7ArdPA7YtSj0dEpTBCbBtDhOxSyRg5rm6MEer3cfXFtYBfF6lHI6ISGCG2jyFCdqtkjMSr/TDkcG2cuXGv6MKMK0BWmrQDEtk5Roh9YIiQXSv90d5RET/j/PmzwJaBQGRf4P4NqUckskuMEPvBECG7VzpGZn33K9TCCbiXDGzqC9z9XeoRiewKI8S+MESIoBsjl9Ue6Jv9Ph6pGgNZN4v2jNy+JPWIRHaBEWJ/GCJEfykZI0lqd/S+PwcP6z4FPPgD2NwfSPtF6hGJbBojxD4xRIhKKBkj19W10ePP95BT7xkg927R+0YYI0QmwQixXwwRolJKxkiq2gU97szCgwYdgLoBQB1/qccjsjmMEPvGECHSo2SMpKvl6P7HNPzSfTPgWkfq0YhsCiOEGCJEBpSMkTtqJ7z+1ZXHxxk5sQ74bb+0AxJZOUYIAQwRonLpO87I1djdwMHZwDcjgfM7pR6RyCoxQqgYQ4SoAqVj5O+HFPizyRBAFAA7xwMJX0k9IpFVYYRQSQwRokooGSOZaoGQq68io9mrgCgE/jMJ+HmD1CMSWQVGCJXGECGqpJIxkqUuREjiy7jdMrTowgPvALGrJJ2PyNIxQkgfhghRFei+TFOAFy/0Rfozk4ouPDwfSPlZ2gGJLBQjhAxhiBBVUekY6flLCFLbvQN0fx/w7ST1eEQWhxFC5WGIEFVD6Tew9jrdCWcCJzxeoMkBCgulG5DIQjBCqCIMEaJq0vfR3jM37gHqB8C/Xgb2TQcKC6Qek0gyjBCqDIYIUQ3oPc7IqcPAzXjgzFZg95tAQb7UYxKZHSOEKoshQlRDpWNkyOFaSOr2OeDgBPz6HfDvUCBfI/WYRGbDCKGqYIgQGUHpGBkU3QC/v7gOcJQDl74HvnkdyHso9ZhEJscIoapiiBAZSekYGRzljis9IwAnV+DKYeDrYUXvHyGyUYwQqg6GCJERlY6Rlw8qcLnXZkBeG/jjIvDgD6lHJDIJRghVF0OEyMhKx8jf9svwW69/AaP2AJ5NpB6PyOgYIVQTZguR8PBwyGQyzJgxw1ybJJJM6Rh55fs8nNE0erzgehzw4LZ0AxIZCSOEasosIRIfH4/169fjmWeeMcfmiCyCweOM3DgJbPsbENkXyLwl9ZhE1cYIIWMweYg8ePAAr7/+OjZs2IC6deuWu1atViMrK0vnRGTN9MXIhSw5oPQE7l4tipF716Qek6jKGCFkLCYPkcmTJ6N///7o2bNnhWvDw8OhUqm0J19fX1OPR2RypWNk+He3cb73dqBuIHD/OhDZD8i4KvWYRJXGCCFjMmmI7NixA2fOnEF4eHil1s+dOxeZmZnaU0pKiinHIzKb0jHy6jep+LX3DqBecyDrVtGekT8uSj0mUYUYIWRsJguRlJQUTJ8+Hdu2bYOLi0ulrqNQKODu7q5zIrIVpWNkxPbrOPfS14BXayDnNrCZe0bIsjFCyBRkQghhihves2cPXn75ZTg6OmrPKygogEwmg4ODA9Rqtc5l+mRlZUGlUiEzM5NRQjYjV5OPsZvjcSLpT9RWOOGrN5qjTcw4oHZDYNgWwNFZ6hGJymCEUFVU5fnbZCGSnZ2N69ev65w3ZswYPPXUU5g9ezZatWpV4W0wRMhWlY6RbaNaoq1/A8BJIfVoRGUwQqiqqvL87WSqIdzc3MrERq1ateDp6VmpCCGyZcUv0xTHyBtbL2LruNpo56cAhAAOzgH8nwUCny97ZWfl42DJ1wB5OYY35OQKOP/10mhBHqAp5xDzOmvzAU12OWtdAGfXoq8LCwB1OZ9wc1QAcmU11hYC6sxy1soBea2ir4UAHt03vNbBGVDUrvpaAHh4r5y1ToDCrcTa+wAM/L+dzBFwKfEL+VEmIAoNrHUAXFTVXJsFiAIDA8sA1zqPv1VnA4Xl/HVo17raCPnh9BXUdShE+Mut0aeZa9mfi2uJT0WqHwCFeYZv16UOIJMVfa3JAQrK+aOQChXg8Ne7CDS5QIG6nLXugINj1dfmPQTyHxleK3cDHP96usx7BOSX83ej5LUf79Wsytp8NZCXa3itcy3ASf7X2gr+u6/u7wiJmCxEiKh8pWNkVMTP2DquE9pl7ANOris66TNkLdD2taKvfz8CbB9ueCP9lgOdJhR9fSMO2DLQ8NqXPgSenVb0ddovwMYXDa8NmQuEzCn6+k4isLar4bXB04BeHxZ9nZkC/LON4bUdJwD9lxd9nXsXWP6k4bVtXweGrCn6WpMDLA0wvLblkKKXvYqVt7Zpb+D1bx9/v6KF4SeTgOeB0H2Pv/88CHj4p/61Pu2A/4t+/P3aZ4t+HvrUbwFMPvH4+40vARmJ+tfW8QNm/Pr4+y0DgbSz+tcq6wHv/f74+6+HA9eP61/rXAsFc29p94Rslq9CiMNZYD+KTqWFlYjGPW8Bl/bqv10AmJf2ODj3vw38st3w2neTgFqeRV8fng+cijC8dsavRT8PAIj+GIj7wvDaSSeBBk8Vff3TZ8DRpYbXTogGnmhX9PXJdcCPCw2vDd0PBDxX9PWZrcAP7xpe+9p3QLNeRV//+h3wn8mG1w7dAjw9pOjr3/YB/x5jeG11f0dIxKyHeI+JicHKlSvNuUkii6b3oGcevYF2o4v+z5xIIgLQeTmmhbdbhdchqg6TvUfEGPgeEbIXpd8zsnVcJ7RrpNK/O17m8HhXdWGh4V32NVkrRNHLKEZZK3u8C9za1gJFL1MZYy3wePe+ha8tEAJzd53Dt2fSH78npHXDop9dZW63sKD8tQ6Oj1+asbq1Ffx3VJW15vhvuSprjcgi3qxqDAwRsid6Y8Sv/KMRExkb35hKxlCV52/+9V0iC2Hwb9MQmQkjhKTAECGyIIwRkgojhKTCECGyMIwRMjdGCEmJIUJkgRgjZC6MEJIaQ4TIQjFGyNQYIWQJGCJEFowxQqbCCCFLwRAhsnCMETI2RghZEoYIkRVgjJCxMELI0jBEiKwEY4RqihFCloghQmRFGCNUXYwQslQMESIrwxihqmKEkCVjiBBZIcYIVRYjhCwdQ4TISjFGqCKMELIGDBEiK8YYIUMYIWQtGCJEVo4xQqUxQsiaMESIbABjhIoxQsjaMESIbARjhBghZI0YIkQ2hDFivxghZK0YIkQ2hjFifxghZM0YIkQ2iDFiPxghZO0YIkQ2ijFi+xghZAsYIkQ2jDFiuxghZCsYIkQ2jjFiexghZEsYIkR2gDFiOxghZGsYIkR2gjFi/RghZIsYIkR2hDFivRghZKsYIkR2hjFifRghZMsYIkR2iDFiPRghZOsYIkR2ijFi+RghZA8YIkR2jDFiuRghZC8YIkR2jjFieRghZE8YIkTEGLEgjBCyNyYNkfDwcHTs2BFubm5o0KABhgwZgsTERFNukoiqiTEiPUYI2SOThsjRo0cxefJknDhxAlFRUcjPz0evXr2Qk5Njys0SUTUxRqTDCCF7JRNCCHNt7M6dO2jQoAGOHj2KF154oczlarUaarVa+31WVhZ8fX2RmZkJd3d3c41JZPdyNfkYuzkeJ5L+RG2FE7aO64R2fnWlHstmMULI1mRlZUGlUlXq+dus7xHJzMwEAHh4eOi9PDw8HCqVSnvy9fU153hE9BfuGTEfRgjZO7PtERFCYPDgwbh37x7+97//6V3DPSJEloV7RkyLEUK2yiL3iEyZMgXnzp3D9u3bDa5RKBRwd3fXORGRdLhnxHQYIURFzBIiU6dOxd69exEdHY1GjRqZY5NEZCSMEeNjhBA9ZtIQEUJgypQp2LVrF44cOYLAwEBTbo6ITIQxYjyMECJdJg2RyZMnY9u2bfj666/h5uaG9PR0pKen4+HDh6bcLBGZAGOk5hghRGWZ9M2qMplM7/mRkZEIDQ2t8PpVebMLEZkH38BaPYwQsicW82ZVIYTeU2UihIgsE/eMVB0jhMgw/q0ZIqoyxkjlMUKIyscQIaJqYYxUjBFCVDGGCBFVG2PEMEYIUeUwRIioRhgjZTFCiCqPIUJENcYYeYwRQlQ1DBEiMgrGCCOEqDoYIkRkNPYcI4wQouphiBCRUdljjDBCiKqPIUJERmdPMcIIIaoZhggRmYQ9xAgjhKjmGCJEZDK2HCOMECLjYIgQkUnZYowwQoiMhyFCRCZnSzHCCCEyLoYIEZmFLcQII4TI+BgiRGQ21hwjjBAi02CIEJFZWWOMMEKITIchQkRmZ00xwgghMi2GCBFJwhpihBFCZHoMESKSjCXHCCOEyDwYIkQkKUuMEUYIkfkwRIhIcpYUI4wQIvNiiBCRRbCEGGGEEJkfQ4SILIaUMcIIIZIGQ4SILIoUMcIIIZIOQ4SILI45Y4QRQiQthggRWSRzxAgjhEh6DBEislimjBFGCJFlYIgQkUUzRYwwQogsB0OEiCyeMWOEEUJkWRgiRGQVjBEjjBAiy8MQISKrUZMYYYQQWSaGCBFZlerECCOEyHIxRIjI6lQlRhghRJaNIUJEVqkyMcIIIbJ8ZgmRNWvWIDAwEC4uLmjfvj3+97//mWOzRGTjyosRRgiRdTB5iHzzzTeYMWMG3n//fSQkJOD5559H3759cePGDVNvmojsgL4YOX39T0YIkZWQCSGEKTfQuXNntGvXDmvXrtWe16JFCwwZMgTh4eHlXjcrKwsqlQqZmZlwd3c35ZhEZOVyNfkYuzkeJ5L+1J7HCCGSRlWev026R0Sj0eD06dPo1auXzvm9evVCbGxsmfVqtRpZWVk6JyKiyii5ZwRghBBZCydT3nhGRgYKCgrQsGFDnfMbNmyI9PT0MuvDw8PxwQcfmHIkIrJhSrkTNo/phCO/3Ya/pxJP+6ikHomIKmCWN6vKZDKd74UQZc4DgLlz5yIzM1N7SklJMcd4RGRDXJwd0a+1NyOEyEqYdI9IvXr14OjoWGbvx+3bt8vsJQEAhUIBhUJhypGIiIjIgph0j4hcLkf79u0RFRWlc35UVBSCg4NNuWkiIiKyAibdIwIAs2bNwsiRI9GhQwd07doV69evx40bNzBx4kRTb5qIiIgsnMlDZPjw4bh79y4WLVqEtLQ0tGrVCgcOHIC/v7+pN01EREQWzuTHEakJHkeEiIjI+ljMcUSIiIiIysMQISIiIskwRIiIiEgyDBEiIiKSDEOEiIiIJMMQISIiIskwRIiIiEgyDBEiIiKSDEOEiIiIJMMQISIiIskwRIiIiEgyDBEiIiKSDEOEiIiIJMMQISIiIskwRIiIiEgyDBEiIiKSDEOEiIiIJMMQISIiIskwRIiIiEgyDBEiIiKSDEOEiIiIJMMQISIiIskwRIiIiEgyDBEiIiKSDEOEiIiIJMMQISIiIskwRIiIiEgyDBEiIiKSDEOEiIiIJMMQISIiIskwRIiIiEgyDBEiIiKSDEOEiIiIJMMQISIiIsmYLESuXbuGcePGITAwEK6urmjSpAkWLlwIjUZjqk0SERGRlXEy1Q3/9ttvKCwsxJdffoknn3wS58+fx4QJE5CTk4Ply5ebarNERERkRWRCCGGujX3yySdYu3YtkpKSKrU+KysLKpUKmZmZcHd3N/F0REREZAxVef422R4RfTIzM+Hh4WHwcrVaDbVarf0+KyvLHGMRERGRRMz2ZtXff/8dq1atwsSJEw2uCQ8Ph0ql0p58fX3NNR4RERFJoMohEhYWBplMVu7p1KlTOtdJTU1Fnz59MHToUIwfP97gbc+dOxeZmZnaU0pKStXvEREREVmNKr9HJCMjAxkZGeWuCQgIgIuLC4CiCOnevTs6d+6MzZs3w8Gh8u3D94gQERFZH5O+R6RevXqoV69epdbeunUL3bt3R/v27REZGVmlCCEiIiLbZ7I3q6ampiIkJAR+fn5Yvnw57ty5o73My8vLVJslIiIiK2KyEDl8+DCuXr2Kq1evolGjRjqXmfETw0RERGTBTPZaSWhoKIQQek9EREREAP/WDBEREUmIIUJERESSYYgQERGRZBgiREREJBmGCBEREUmGIUJERESSYYgQERGRZEx2QDNjKD7mSFZWlsSTEBERUWUVP29X5thhFh0i2dnZAABfX1+JJyEiIqKqys7OhkqlKndNlf/6rjkVFhYiNTUVbm5ukMlkRr3trKws+Pr6IiUlxSb/si/vn/Wz9fto6/cPsP37yPtn/Ux1H4UQyM7Oho+PT4V/8Nai94g4ODiU+Ts1xubu7m6z/8AA3j9bYOv30dbvH2D795H3z/qZ4j5WtCekGN+sSkRERJJhiBAREZFk7DZEFAoFFi5cCIVCIfUoJsH7Z/1s/T7a+v0DbP8+8v5ZP0u4jxb9ZlUiIiKybXa7R4SIiIikxxAhIiIiyTBEiIiISDIMESIiIpIMQ4SIiIgkY7Mh8vHHHyM4OBhKpRJ16tTRu+bGjRsYOHAgatWqhXr16mHatGnQaDTl3q5arcbUqVNRr1491KpVC4MGDcLNmzdNcA+qJiYmBjKZTO8pPj7e4PVCQ0PLrO/SpYsZJ6+8gICAMrPOmTOn3OsIIRAWFgYfHx+4uroiJCQEFy5cMNPElXft2jWMGzcOgYGBcHV1RZMmTbBw4cIK/z1a+uO3Zs0aBAYGwsXFBe3bt8f//ve/ctcfPXoU7du3h4uLCxo3box169aZadKqCw8PR8eOHeHm5oYGDRpgyJAhSExMLPc6hv47/e2338w0deWFhYWVmdPLy6vc61jT46fv94lMJsPkyZP1rreGx+7YsWMYOHAgfHx8IJPJsGfPHp3Lq/v7cOfOnWjZsiUUCgVatmyJ3bt3G3Vumw0RjUaDoUOH4q233tJ7eUFBAfr374+cnBz89NNP2LFjB3bu3Im333673NudMWMGdu/ejR07duCnn37CgwcPMGDAABQUFJjiblRacHAw0tLSdE7jx49HQEAAOnToUO51+/Tpo3O9AwcOmGnqqlu0aJHOrPPnzy93/bJly/Dpp5/iiy++QHx8PLy8vPDSSy9p/6Cipfjtt99QWFiIL7/8EhcuXMBnn32GdevWYd68eRVe11Ifv2+++QYzZszA+++/j4SEBDz//PPo27cvbty4oXd9cnIy+vXrh+effx4JCQmYN28epk2bhp07d5p58so5evQoJk+ejBMnTiAqKgr5+fno1asXcnJyKrxuYmKizmPWtGlTM0xcdU8//bTOnL/++qvBtdb2+MXHx+vct6ioKADA0KFDy72eJT92OTk5aNOmDb744gu9l1fn92FcXByGDx+OkSNH4pdffsHIkSMxbNgwnDx50niDCxsXGRkpVCpVmfMPHDggHBwcxK1bt7Tnbd++XSgUCpGZman3tu7fvy+cnZ3Fjh07tOfdunVLODg4iIMHDxp99prQaDSiQYMGYtGiReWuGz16tBg8eLB5hqohf39/8dlnn1V6fWFhofDy8hJLlizRnvfo0SOhUqnEunXrTDChcS1btkwEBgaWu8aSH79OnTqJiRMn6pz31FNPiTlz5uhd/95774mnnnpK57w333xTdOnSxWQzGtPt27cFAHH06FGDa6KjowUAce/ePfMNVk0LFy4Ubdq0qfR6a3/8pk+fLpo0aSIKCwv1Xm5Nj50QQgAQu3fv1n5f3d+Hw4YNE3369NE5r3fv3uLVV1812qw2u0ekInFxcWjVqhV8fHy05/Xu3RtqtRqnT5/We53Tp08jLy8PvXr10p7n4+ODVq1aITY21uQzV8XevXuRkZGB0NDQCtfGxMSgQYMGaNasGSZMmIDbt2+bfsBqWrp0KTw9PdG2bVt8/PHH5b50kZycjPT0dJ3HS6FQoFu3bhb3eOmTmZkJDw+PCtdZ4uOn0Whw+vRpnZ89APTq1cvgzz4uLq7M+t69e+PUqVPIy8sz2azGkpmZCQCVesyCgoLg7e2NHj16IDo62tSjVduVK1fg4+ODwMBAvPrqq0hKSjK41pofP41Gg23btmHs2LEV/qV3a3nsSqvu70NDj6sxf4fabYikp6ejYcOGOufVrVsXcrkc6enpBq8jl8tRt25dnfMbNmxo8DpSiYiIQO/eveHr61vuur59++Krr77CkSNHsGLFCsTHx+PFF1+EWq0206SVN336dOzYsQPR0dGYMmUKVq5ciUmTJhlcX/yYlH6cLfHxKu3333/HqlWrMHHixHLXWerjl5GRgYKCgir97PX9N9mwYUPk5+cjIyPDZLMagxACs2bNwnPPPYdWrVoZXOft7Y3169dj586d2LVrF5o3b44ePXrg2LFjZpy2cjp37oytW7fi0KFD2LBhA9LT0xEcHIy7d+/qXW/Nj9+ePXtw//79cv/HzZoeO32q+/vQ0ONqzN+hTka7JTMICwvDBx98UO6a+Pj4Ct8TUUxf+QohKixiY1ynsqpzn2/evIlDhw7h22+/rfD2hw8frv26VatW6NChA/z9/bF//3787W9/q/7glVSV+zdz5kztec888wzq1q2LV155RbuXxJDSj40pH6/SqvP4paamok+fPhg6dCjGjx9f7nWlfvwqUtWfvb71+s63NFOmTMG5c+fw008/lbuuefPmaN68ufb7rl27IiUlBcuXL8cLL7xg6jGrpG/fvtqvW7duja5du6JJkybYsmULZs2apfc61vr4RUREoG/fvjp7yEuzpseuPNX5fWjq36FWFSJTpkzBq6++Wu6agICASt2Wl5dXmTfb3Lt3D3l5eWXqr+R1NBoN7t27p7NX5Pbt2wgODq7UdquqOvc5MjISnp6eGDRoUJW35+3tDX9/f1y5cqXK162OmjymxZ8OuXr1qt4QKX6Hf3p6Ory9vbXn37592+BjbGxVvX+pqano3r07unbtivXr11d5e+Z+/AypV68eHB0dy/xfU3k/ey8vL73rnZycyg1NqU2dOhV79+7FsWPH0KhRoypfv0uXLti2bZsJJjOuWrVqoXXr1gb/bVnr43f9+nX8+OOP2LVrV5Wvay2PHVD934eGHldj/g61qhCpV68e6tWrZ5Tb6tq1Kz7++GOkpaVpH5TDhw9DoVCgffv2eq/Tvn17ODs7IyoqCsOGDQMApKWl4fz581i2bJlR5iqtqvdZCIHIyEiMGjUKzs7OVd7e3bt3kZKSovMP1ZRq8pgmJCQAgMFZAwMD4eXlhaioKAQFBQEoei346NGjWLp0afUGrqKq3L9bt26he/fuaN++PSIjI+HgUPVXTs39+Bkil8vRvn17REVF4eWXX9aeHxUVhcGDB+u9TteuXfH999/rnHf48GF06NChWv+WTU0IgalTp2L37t2IiYlBYGBgtW4nISFB8serMtRqNS5duoTnn39e7+XW9vgVi4yMRIMGDdC/f/8qX9daHjug+r8Pu3btiqioKJ090ocPHzbu/3wb7W2vFub69esiISFBfPDBB6J27doiISFBJCQkiOzsbCGEEPn5+aJVq1aiR48e4syZM+LHH38UjRo1ElOmTNHexs2bN0Xz5s3FyZMntedNnDhRNGrUSPz444/izJkz4sUXXxRt2rQR+fn5Zr+P+vz4448CgLh48aLey5s3by527dolhBAiOztbvP322yI2NlYkJyeL6Oho0bVrV/HEE0+IrKwsc45dodjYWPHpp5+KhIQEkZSUJL755hvh4+MjBg0apLOu5P0TQoglS5YIlUoldu3aJX799VcxYsQI4e3tbXH379atW+LJJ58UL774orh586ZIS0vTnkqypsdvx44dwtnZWURERIiLFy+KGTNmiFq1aolr164JIYSYM2eOGDlypHZ9UlKSUCqVYubMmeLixYsiIiJCODs7i3//+99S3YVyvfXWW0KlUomYmBidxys3N1e7pvR9/Oyzz8Tu3bvF5cuXxfnz58WcOXMEALFz504p7kK53n77bRETEyOSkpLEiRMnxIABA4Sbm5vNPH5CCFFQUCD8/PzE7Nmzy1xmjY9ddna29rkOgPZ35vXr14UQlft9OHLkSJ1Pth0/flw4OjqKJUuWiEuXLoklS5YIJycnceLECaPNbbMhMnr0aAGgzCk6Olq75vr166J///7C1dVVeHh4iClTpohHjx5pL09OTi5znYcPH4opU6YIDw8P4erqKgYMGCBu3LhhxntWvhEjRojg4GCDlwMQkZGRQgghcnNzRa9evUT9+vWFs7Oz8PPzE6NHj7ao+1Ps9OnTonPnzkKlUgkXFxfRvHlzsXDhQpGTk6OzruT9E6LoI2sLFy4UXl5eQqFQiBdeeEH8+uuvZp6+YpGRkXr/vZb+fwVre/xWr14t/P39hVwuF+3atdP5aOvo0aNFt27ddNbHxMSIoKAgIZfLRUBAgFi7dq2ZJ648Q49XyX9/pe/j0qVLRZMmTYSLi4uoW7eueO6558T+/fvNP3wlDB8+XHh7ewtnZ2fh4+Mj/va3v4kLFy5oL7f2x08IIQ4dOiQAiMTExDKXWeNjV/wR49Kn0aNHCyEq9/uwW7du2vXFvvvuO9G8eXPh7OwsnnrqKaPHl0yIv95NRERERGRmdvvxXSIiIpIeQ4SIiIgkwxAhIiIiyTBEiIiISDIMESIiIpIMQ4SIiIgkwxAhIiIiyTBEiIiISDIMESIiIpIMQ4SIiIgkwxAhIiIiyfw/IaIcE4j6py0AAAAASUVORK5CYII="
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# torch.manual_seed(1) # Set seed to some fixed value\n",
"\n",
"w0 = torch.randn(2, 2, requires_grad=True)\n",
"w1 = torch.randn(3, 1, requires_grad=True)\n",
"\n",
"learning_rate = 1e-4\n",
"print('iter', 'loss', '\\n----', '----', sep='\\t')\n",
"for t in range(1, 100001):\n",
" # Forward pass: compute predicted y\n",
" y_pred = forward_pass(x, w0, w1, torch.relu)\n",
"\n",
" loss = torch.mean(torch.square(y - y_pred))\n",
" loss.backward()\n",
"\n",
" if t % 1000 == 0:\n",
" print(t, loss.item(), sep='\\t')\n",
"\n",
" with torch.no_grad():\n",
" w0 -= learning_rate * w0.grad\n",
" w1 -= learning_rate * w1.grad\n",
" w0.grad.zero_() # reset gradients !important\n",
" w1.grad.zero_()\n",
"\n",
"print(\"--- w0 ---\", w0, sep='\\n')\n",
"print(\"--- w1 ---\", w1, sep='\\n')\n",
"print(\"--- w1 ---\", loss, sep='\\n')\n",
"y_pred = forward_pass(x, w0, w1, torch.relu)\n",
"plt.plot(x, y, linestyle='solid', label='|x-1|')\n",
"plt.plot(x, y_pred.detach().numpy(), linestyle='dashed', label='perceptron')\n",
"plt.axis('equal')\n",
"plt.title('Fit NN on abs function')\n",
"plt.legend()\n",
"plt.show()\n",
"\n",
"# Task 5: Submit the values of `w0`, `w1`, and `loss` values after fitting\n",
"# Note: An acceptable loss value should be less than 1.0\n",
"# You should try adjusting the random seed, learning rate, or \n",
"# number of iterations to improve your model.\n",
"\n",
"w0 = [[0.0, 0.0], [0.0, 0.0]] # to be computed\n",
"w1 = [[0.0], [0.0], [0.0]] # to be computed\n",
"loss = 0.0 # to be computed"
]
},
{
"cell_type": "code",
"execution_count": 90,
"id": "c4bfdc7d",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T03:07:05.366381Z",
"start_time": "2024-04-02T03:07:05.352349Z"
}
},
"outputs": [
{
"ename": "AssertionError",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
"\u001B[0;31mAssertionError\u001B[0m Traceback (most recent call last)",
"Cell \u001B[0;32mIn[90], line 12\u001B[0m\n\u001B[1;32m 9\u001B[0m computed_mse_loss \u001B[38;5;241m=\u001B[39m torch\u001B[38;5;241m.\u001B[39mmean(torch\u001B[38;5;241m.\u001B[39msquare(y \u001B[38;5;241m-\u001B[39m y_pred))\u001B[38;5;241m.\u001B[39mitem()\n\u001B[1;32m 11\u001B[0m \u001B[38;5;28;01massert\u001B[39;00m loss \u001B[38;5;241m<\u001B[39m \u001B[38;5;241m1\u001B[39m\n\u001B[0;32m---> 12\u001B[0m \u001B[38;5;28;01massert\u001B[39;00m isclose(computed_mse_loss, loss, atol\u001B[38;5;241m=\u001B[39m\u001B[38;5;241m1e-5\u001B[39m, rtol\u001B[38;5;241m=\u001B[39m\u001B[38;5;241m1e-2\u001B[39m)\n",
"\u001B[0;31mAssertionError\u001B[0m: "
]
}
],
"source": [
"w0 = torch.tensor(w0)\n",
"w1 = torch.tensor(w1)\n",
"\n",
"x = torch.linspace(-10, 10, 1000).reshape(-1, 1)\n",
"y = torch.abs(x-1)\n",
"\n",
"#IMPORTANT: Your forward pass above have to be correctly implemented\n",
"y_pred = forward_pass(x, w0, w1, torch.relu)\n",
"computed_mse_loss = torch.mean(torch.square(y - y_pred)).item()\n",
"\n",
"assert loss < 1\n",
"assert isclose(computed_mse_loss, loss, atol=1e-5, rtol=1e-2)"
]
},
{
"cell_type": "markdown",
"id": "413cd863",
"metadata": {},
"source": [
"### Task 2.3 - Different random seeds\n",
"\n",
"Try to fit the model on different initial random weight values by adjusting the random seed. \n",
"<br/>\n",
"What is the impact of a random seed? How should we compare different neural network models given your observation to ensure fairness?\n",
"\n",
"Submit your observations and conclusion on Coursemology.\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "c98f725f",
"metadata": {},
"source": [
"# 3 Neural Networks (using PyTorch layers)"
]
},
{
"cell_type": "markdown",
"id": "8c0f772a",
"metadata": {},
"source": [
"### 3.1.1 Demo - nn.Module\n",
"\n",
"The `nn.Module` class is an interface that houses two main methods: `__init__`, where we instantiate our layers and activation functions, and `forward`, that performs the forward pass.\n",
"\n",
"To create our own neural network, we will inherit from the nn.Module parent class and call `super().__init__()` from within our constructor to create our module. Next, we will implement the `forward` function within our class so we can call it from our module to perform the forward pass. \n",
"\n",
"In this example, we define a custom LinearLayer class that inherits from nn.Module. The __init__ method initializes the weight and bias parameters as nn.Parameter objects, which are special types of tensors that require gradients to be computed during the backward pass.\n",
"\n",
"The forward method defines the forward pass of the linear layer. It takes a tensor x as input and computes the matrix multiplication of x and self.weight using the torch.matmul function, and then adds self.bias.\n",
"\n",
"We also created our own activation function which uses `torch.sin` by inheriting from nn.Module.\n",
"\n",
"Finally, in our Model, we can combine our own LinearLayers together with our SineActivation to process our input data using the forward function. In later sections, you will see how we can train our models."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "36fd1dd3",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T05:01:23.721712Z",
"start_time": "2024-04-02T05:01:23.715216Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Original value: tensor([[1.]])\n",
"Value after being processed by Model: tensor([[1.0562]], grad_fn=<AddBackward0>)\n"
]
}
],
"source": [
"# Define a linear layer using nn.Module\n",
"class LinearLayer(nn.Module):\n",
" def __init__(self, input_dim, output_dim):\n",
" super().__init__()\n",
" self.weight = nn.Parameter(torch.randn(input_dim, output_dim))\n",
" self.bias = nn.Parameter(torch.randn(output_dim))\n",
"\n",
" def forward(self, x):\n",
" return torch.matmul(x, self.weight) + self.bias\n",
" \n",
"class SineActivation(nn.Module):\n",
" def __init__(self):\n",
" super().__init__()\n",
"\n",
" def forward(self, x):\n",
" return torch.sin(x)\n",
"\n",
"class Model(nn.Module):\n",
" def __init__(self, input_size, hidden_size, num_classes):\n",
" super(Model, self).__init__()\n",
" self.l1 = LinearLayer(input_size, hidden_size)\n",
" self.act = SineActivation()\n",
" self.l2 = LinearLayer(hidden_size, num_classes)\n",
"\n",
" def forward(self, x):\n",
" x = self.l1(x)\n",
" x = self.act(x)\n",
" x = self.l2(x)\n",
" return x\n",
" \n",
"input_size = 1\n",
"hidden_size = 1\n",
"num_classes = 1\n",
"\n",
"model = Model(input_size, hidden_size, num_classes)\n",
"\n",
"x = torch.tensor([[1.0]])\n",
"output = model(x)\n",
"print(\"Original value: \", x)\n",
"print(\"Value after being processed by Model: \", output)\n"
]
},
{
"cell_type": "markdown",
"id": "c6adbc1d",
"metadata": {},
"source": [
"_Extra: We can also define a `backward` function to perform backpropagation which will not be required in this problem set._\n",
"\n",
"In this trivial example, the Squared module takes an input x and returns x**2. The backward method calculates the gradient of the output with respect to the input, based on the gradients of the output grad_output.\n",
"\n",
"We can define the backward function for functions that are not fully differentiable that we still wish to use in our neural network."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "3e1044e1",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T04:56:54.392808Z",
"start_time": "2024-04-02T04:56:54.230366Z"
}
},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'nn' is not defined",
"output_type": "error",
"traceback": [
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
"\u001B[0;31mNameError\u001B[0m Traceback (most recent call last)",
"Cell \u001B[0;32mIn[1], line 1\u001B[0m\n\u001B[0;32m----> 1\u001B[0m \u001B[38;5;28;01mclass\u001B[39;00m \u001B[38;5;21;01mSquared\u001B[39;00m(\u001B[43mnn\u001B[49m\u001B[38;5;241m.\u001B[39mModule):\n\u001B[1;32m 2\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mforward\u001B[39m(\u001B[38;5;28mself\u001B[39m, x):\n\u001B[1;32m 3\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mx \u001B[38;5;241m=\u001B[39m x\n",
"\u001B[0;31mNameError\u001B[0m: name 'nn' is not defined"
]
}
],
"source": [
"class Squared(nn.Module):\n",
" def forward(self, x):\n",
" self.x = x\n",
" return x**2\n",
"\n",
" def backward(self, grad_output):\n",
" grad_input = 2 * self.x * grad_output\n",
" return grad_input"
]
},
{
"cell_type": "markdown",
"id": "023c28dd",
"metadata": {},
"source": [
"### 3.1.2 Demo - Activation Functions\n",
"\n",
"Pytorch also provides built-in activation functions. To help you understand more about activation functions, we have included some examples of activation functions introduced in the lecture, namely Sigmoid, Tanh, and ReLu. \n",
"\n",
"<img src=\"imgs/img_activation_fns.png\" width=\"200\"> \n",
"\n",
"Activation functions introduces non-linearity into the output of a neuron, allowing the NN to learn non-linear functions. Without non-linearity, our entire network will effectively become a linear model with only one layer, preventing us from modelling complex representations based on our inputs.\n",
"\n",
"Sigmoid, Tanh and ReLU are three examples of such activation functions introduced during lecture and the code block below shows how they map input to output values.\n",
"\n",
"The choice of activation function for the hidden layers and the output layer depends on the problem you're trying to solve.\n",
"\n",
"#### For the hidden layers, there are several commonly used activation functions:\n",
"\n",
"ReLU (Rectified Linear Unit): ReLU is a popular activation function that is widely used in deep learning models. It maps non-positive inputs to 0 and positive inputs to their original value. It is mainly used in hidden layers because it is fast to compute, has sparse activations, and helps to mitigate the vanishing gradient problem, where the gradients can become very small and cause the model to learn slowly.\n",
"\n",
"Tanh (Hyperbolic Tangent): Tanh is a activation function that maps input values to the range [-1, 1]. It is similar to Sigmoid, but instead of producing output values in the range [0, 1], it produces output values in the range [-1, 1]. Tanh is useful for solving problems where you want the activations to be centered around zero, such as in recurrent neural networks.\n",
"\n",
"Sigmoid: Sigmoid maps its input values to the range [0, 1]. It is less commonly used in hidden layers because it has a relatively slow convergence rate and can introduce saturation, where the output values become very small or very large, which can make it difficult for the gradients to flow through the model.\n",
"\n",
"#### For the output layer, the choice of activation function depends on the problem you're trying to solve. Here are some common choices:\n",
"\n",
"Sigmoid: The Sigmoid activation function maps input values to the range [0, 1]. It is commonly used for binary classification problems where the network produces a probability of one of two classes. In this case, the Sigmoid activation maps the output to a probability distribution over the two classes.\n",
"\n",
"Softmax: The Softmax activation function is a generalization of the Sigmoid activation that maps input values to a probability distribution over multiple classes. It is commonly used for multiclass classification problems. The Softmax activation function is used to convert the raw scores produced by the network into a probability distribution over the classes.\n",
"\n",
"Linear: For regression problems, the linear activation function is often used because it just maps the input values to the output values without any change.\n",
"\n",
"In summary, ReLU is a common choice for hidden layers, and the choice of activation function for the output layer depends on the problem you're trying to solve (binary classification, multiclass classification, or regression).\n",
"\n",
"---\n",
"\n",
"_Extra (Vanishing Gradient Problem):_\n",
"\n",
"_Below is an image of the derivatives of the Sigmoid, Tanh and ReLU function. We can see that the derivatives for both Sigmoid and Tanh tend to zero when the inputs are largely positive or negative, while derivative for ReLU is zero only when the inputs are non-positive. In our neural network, gradients are calculated through backpropagation using chain rule and the derivatives of each layer are multiplied down the network. The gradient is more likely to decrease exponentially as we propagate down to the initial layers if we use Sigmoid and Tanh as compared to ReLU, leading to the vanishing gradient problem._\n",
"\n",
"<img src=\"imgs/img_activation_fns_der.png\" width=\"500\"> \n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 119,
"id": "1ea9f8e6",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T03:28:32.429716Z",
"start_time": "2024-04-02T03:28:32.330845Z"
}
},
"outputs": [
{
"data": {
"text/plain": "<Figure size 600x600 with 1 Axes>",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA4AAAAJ3CAYAAADF1D9CAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAADQ6klEQVR4nOzdd3gUZdcG8HtTdtMT0gspBEIKgQABAlISUECwoTQbgmJXEOliIUiTpoDYQSkqKlJERV5QCU26gNJrIEBCIJCE9Ha+P/bbMctuKkk2sPfvuvbKZuaZmTOzM8/MmfaoRERAREREREREdzwLUwdAREREREREdYMJIBERERERkZlgAkhERERERGQmmAASERERERGZCSaAREREREREZoIJIBERERERkZlgAkhERERERGQmmAASERERERGZCSaAREREREREZoIJIBERmYxKpYJKpTJ1GDUmPj4eKpUK8fHxpg6FiIjIKCaARHewoKAgqFQqLF682NShVEtiYiLi4+NrLf6zZ8/iiy++wHPPPYeoqChYWVlBpVJhypQplRp+x44deOihh+Dh4QFbW1tERERg8uTJyMvLK3e4o0eP4oknnoCPjw9sbGzQuHFjjB49Gunp6TUwV2RO1qxZg/j4eBw4cMDUodwREhISMH36dDz88MPw8/NTTlBcuHCh2uOMj4+v8xMCubm5WL16Nd544w3cfffdcHZ2hkqlQpMmTeo0DiKqn6xMHQARUVkSExMxadIkxMbGYsiQITU+/nnz5mHevHnVGvabb77B4MGDUVxcDD8/P/j7++PQoUN455138PPPPyMhIQF2dnYGw23atAn33XcfcnNz4eHhgWbNmuHYsWOYM2cOVq9ejb/++gteXl63OmtkJtasWYMlS5YgKCgILVu2NHU4t70+ffogIyOjRsc5adIkAKjTJPD48eN45JFH6mx6RHR74RVAIjJb7u7uuP/++/Huu+/it99+Q9++fSs1XGJiIoYOHYri4mLMnDkTSUlJ+Pvvv3Hy5EmEhoZiz549GDt2rMFwN27cwMCBA5Gbm4vhw4fj4sWL2LdvH86fP4+OHTvizJkzGDp0aE3PJhFVUrNmzTBkyBB8/PHH2Lt3r6nDqTZra2u0b98ew4YNw7Jly/Dll1+aOiQiqkd4BZCIzNZbb72l9/93331XqeFmzZqF/Px89OjRA2PGjFG6BwYG4ssvv0THjh3x+eef4+2339a7mvfpp5/iypUrCA8Px/vvvw9LS0sAgJubG7799ls0btwYv/76K/7++2+0bt26BuaQiKpi+/btpg6hRjRr1gw7duxQ/k9ISDBdMERU7/AKIJEZKv2iioyMDIwYMQIBAQHQaDRo0qQJJk+ejKKionKHS0lJwdChQ+Hr6wsbGxuEh4dj9uzZFQ5nzOLFi6FSqfRu84yLi0PXrl0BAJs3b1aexVGpVAgKCqqJxVAtIoLVq1cDgNGrdXfddRfCwsJQWFiIn376Sa/fqlWrAABDhgxRkj+dgIAA3HPPPQCAH3/8sVKxnDhxAvb29rC0tMTmzZsN+qempsLT0xMqlQrffvttheO7fv06NBoNrKyscPny5TLL9e3bFyqVCvPnz1e6paSk4MMPP0TPnj0RFBQEGxsbNGjQALGxsVi2bFml5qe0il4Oo3u+NTEx0aCfiOC7775D9+7d4ebmBo1Gg+DgYAwfPhwpKSlGx7dt2zY8/PDD8Pb2hrW1NVxdXREeHo5nn30WO3furHL8AKq0jehcuHABw4cPR9OmTWFrawsXFxd07drVYJ1ITEyESqXCkiVLAABPP/203jYSHx+PkpISuLq6wtLSEtevX9cbfufOnUrZRYsWGcRR1vKtzrIFgGvXruHNN99EZGQk7O3t4ejoiPbt2+OLL75ASUmJQfkhQ4Yozy9funQJzzzzjPLMbLNmzfDRRx+VOa36Qlfv6ZT+fYwt27/++guPPPIIvLy8oFar0bBhQzz11FM4evRoHUdORHc6JoBEZiwjIwMdOnTARx99BDc3N/j6+uL06dN455138NJLL5U5XFpaGtq1a4clS5bAy8sLgYGBOHbsGMaMGYP+/fsbPaCrqubNmyMyMhIA4OTkhI4dOyqftm3b6pXVHVDVxVnu8+fPIzk5GQDQsWNHo2V03Xft2qV0Kyoqwr59+6o8XHmaNm2KWbNmoaSkBE899ZTBs0vPPvssrly5goEDB+Lxxx+vcHwNGjRAz549UVxcjB9++MFomczMTKxbtw6WlpYYMGCA0n3hwoUYPnw4tm7dCisrKzRv3hxOTk7YsmULnnrqqXLXp5pUWFiIgQMH4rHHHsPvv/+uJF6XL1/Ghx9+iNatW+PEiRN6w/z000+IjY3FmjVrUFRUhBYtWsDLywtJSUlYtGhRpa8Ml1adbWTz5s2IjIzEhx9+iAsXLiAkJAROTk5ISEhA//79MXr0aKWsjY0NOnbsCE9PTwBASEiI3jYSEBAACwsLdOzYESUlJdi2bZvetLZs2aI33dLOnz+Pc+fOISAgQO9kS3WWLQAcPnwYLVq0wLRp03Dy5EkEBQXBy8sLu3fvxvPPP4+BAwdCRIwux3PnziE6OhrLly+Hr68v3NzccOTIEbz66quYOnVqxT+ECQUEBOht66V/n44dO8LGxkbp98knn6BTp07KyaWoqChkZ2dj2bJlaN26NX799dc6j5+I7mBCRHeswMBAASBfffWVXveJEycKALG2tpYuXbrIxYsXlX5r164VS0tLASBHjx41OpyVlZU0b95czp49q/TbvHmzODs7CwBZsGCB0eEmTpxoNM6vvvpKAMjgwYP1um/atEkASGxsbLnzCUAAyKZNm8otV5HBgwcLAJk8eXKZZTZu3CgARKPRSElJidEyU6dOFQDSuXNnpdvJkyeVOC9dumR0uG+++UYAiL+/f5XivvfeewWAPPnkk0q3zz77TACIn5+fXLt2rdLjWr58uQCQDh06GO2/ePFiASD33HOPXvetW7fKn3/+KUVFRXrdDx48KOHh4QJAEhISDManWyaV7a6jW7dLr4MiIuPHjxcA0qpVK9m/f7/SPScnR15++WUBIG3atNEbJjIyUgDIxx9/rBd/SUmJbNq0SdauXVtmHDer7jZy8eJFcXV1FZVKJdOmTZO8vDyl3/bt28XPz08AyM8//6w3nG6dvXkb15k1a5YAkFGjRul17927t1hYWIi3t7cEBATo9Vu6dKkAkEGDBul1r86yzcrKksaNGwsAGT58uGRkZCj9Dh8+LM2aNTO6PHTzZW1tLf369ZPr168r/T7++GMBIDY2Nnrda4NuPUxKSrrlcZRl//79YmVlJQBk5syZUlxcLCIieXl5ynJ1dnYus96oDF1d2rhx42qPg4juHEwAie5gFSWAtra2Rg9sHnnkEQEg77//vtHhAMi+ffsMhps/f74AkKCgIL3kqLYTQD8/P/Hz85O//vqr3HIVqUwC+MMPPwgA8fLyKrOM7gA1MjJS6bZ7925l2eXm5hodbt26dQJAHBwcqhT3pUuXxM3NTQDI999/L6dOnRIHBwdRqVSyYcOGKo0rOztbHBwcjCZXIiI9e/YUALJo0aJKj/P3338XAPLcc88Z9KvJBDA1NVU0Go04OTkZXa+Li4ulbdu2AkC2bNmidNdoNNKgQYNKz095qruNjBw5UgDI66+/bnS8P//8swCQbt266XWvKAHctWuXQWJWXFwszs7O0qpVK2X40stx6NChAkAWLlyodKvustXN78MPP2w0voMHD4pKpZLg4GCj8+Xt7S1ZWVkGw7Vu3VoAyKpVq4yOt6bURQL4xBNPCAB56KGHDPqVlJQoSfLbb79d7RiYABJRabwFlMiM3XvvvWjYsKFBd90tlmfOnDE6XIcOHYy+pOSZZ56BjY0NEhMTcfz48ZoNthwXLlzAhQsX0KFDh1qflq6NP7VaXWYZjUYDQNsW183DlTesseEqw8fHB5999hkA4MUXX8TAgQORlZWFYcOGoXv37lUal52dHR566CEAhi/FuXLlCv744w9oNBqjb0y9ceMGvvjiCwwePBg9evRA586d0alTJ4wfPx4AcPDgwSrFUlXr1q1Dfn4+evbsaXS9trCwwP333w9A/7ZHf39/pKenY+PGjTUWS1W3Ed3zoc8++6zR8d17771Qq9X466+/yn2G8GatW7eGg4MD9u/fjxs3bgAADhw4gIyMDMTGxiI2NhaA/vLQ3R7apUsXpVt1l21F89WiRQsEBQXhzJkzRtvae+yxx2Bvb2/QvaI66nayYcMGAMCwYcMM+qlUKgwfPlyvHBHRreJbQInMWOPGjY121z1XlJWVZbR/eHi40e729vbw9/fHyZMnceLECYSFhdVMoPWI7rmdgoKCMsvk5+cDAGxtbQ2G0w1b+v/yhqusvn37YtCgQVi2bBn27duH8PBwzJgxo8rjAYDHH38c33zzDZYvX64kbwCwYsUKFBUV4f7774ezs7PeMPv378f999+PS5culTnea9euVSueyvr3338BaF9w0qlTJ6NldC+3uXjxotLt9ddfxyuvvIIePXogOjoa99xzDzp16oTY2Fg4OjpWK5aqbCNZWVnKC0Gef/75csebl5eHtLS0SrcVaWVlhbvuugsbNmzA9u3bce+99yoJWmxsLFq0aAFAm7QNHjwYycnJOHnyJHx8fBASEqKMp7rLVjfcO++8g2nTphkd7urVq8pwNyeX1a2jbhfp6em4cuUKACAiIsJomWbNmgGA0ecriYiqgwkgkRkzdmYd0J7NB1Dmixl0B1/GeHl54eTJk8rVhjtNgwYNAGgP3ETE6JsqdW9c1JW9+fv169fh4+NTqeGqolu3bsobNx977DGjSeb+/fuNXmno3bs3JkyYAADo0aMH3N3d8c8//+DIkSPKgeny5csBwOCFMsXFxRgwYAAuXbqE3r17Y9y4cWjWrBlcXFxgaWmJU6dOISQkBIWFhdWar8rSvQQnKSkJSUlJ5ZYtfZX15ZdfhqOjI+bMmYN9+/Zh3759mDFjBmxsbDBo0CDMmjXLIOGtSFW2kdIv76lMMwRVvUIcGxuLDRs2YPPmzUoCqFKp0LlzZ7i5uaFhw4ZKUlg6OSytustWN5zuBUiVHU6nunXUzX777TejL4155pln8Mwzz1RqHLWhdAJb1jqjS/bv1DqViOoebwEloirTnbE2JjU1FQD0rpzokqSyDtays7NrMLrapbsqkp+fX+bVLt1taaWvoAQFBcHa2lqvf2WGq6yLFy9i5MiRALTL+7333jN6xSAjIwPbt283+JQua2VlhX79+gH4L+lLSkrC9u3b4ejoqNzqp7N7926cOnUKgYGBWLVqFbp06QI3NzelqYuKEobyVGWdcXBwAAC8+eabEO0z7mV+Fi9erDfsoEGDcODAASQnJ+O7777D0KFDYWVlhS+++AJPPvlkleOuyjaiixvQXh2uKPaqNoOiu5Vz8+bNEBFs3boVkZGRcHNzU/rrbsEsKwGs7rLVDXfy5MkKh4uLi6vSfFXF5cuXja7358+fr7VpVkbp3163XtxMd2W1ulejiYhuxgSQiKqsrHapcnJylAOqpk2bKt11Z/HLOig+deqU0e7ltQNnKgEBAfD29gZQ9tUaXfeYmBilm5WVlfJMWFWGqwwRwZAhQ3D9+nU8++yzGDlyJHJycjBo0CCD58Xi4uIqlRDprvLpEsDly5dDRNCnTx+DW1R1ty9GR0crzzGWVp1n/8pbZzIyMpTbBkvTXak8dOhQlaen4+3tjYEDB2LhwoXYtWsXLCws8MsvvyhNf1RWVbYRZ2dn+Pr6AtA2mVAVldlG2rVrB1tbW+zduxe7d+/GtWvX9BI83feEhASjz/8B1V+2NfGb1IQhQ4YYXe/Lapu0rri4uMDDwwMAcOTIEaNldOtE6TqViOhWMAEkoir766+/cODAAYPuX375JfLy8hAYGIjQ0FCle3BwMABgz549BsNkZ2eX2c6aLtGo6i1vtUmlUuHhhx8GAKMNaP/11184duwYrK2t8eCDD+r1e+SRRwBoG74vLi7W63f+/Hn8/vvvAGD0BSvl+fDDD/H777+jUaNG+OCDDzB16lRERkZi9+7dZT53VZFOnTohICAAp0+fxu7du5VE8LHHHjMoq/udjDUeX1hYiLlz51Z5+uWtMwsXLjQ6zH333Qe1Wo1169bh5MmTVZ7mzSIiIpRbP8t7ttGYqm4junWjqsuqMtuIWq1GTEwMCgsLMX36dAD6V/h0yd6PP/6Io0ePwsPDw+B5tOouW918zZ8/v9K3a95pKvqNevbsCUC7Hd9MRJTuunJERLeKCSARVZmVlRWGDBmCc+fOKd22bduGd955BwAwevRovSsTXbt2hY2NDfbu3YvPP/9c6Z6eno4hQ4YgLS3N6HQaNWoEQHtmvLxb6oKCghAUFISdO3fe0nxV1pgxY6BWq7FhwwbMmjVLObA9d+6c8jzRs88+q1wp1HnxxRfh7u6Oo0ePYuTIkcozcWlpaXj88cdRVFSEXr16ITo6utKxHDt2DOPHj4eFhQWWLl0KBwcHaDQafP3111Cr1Zg8eTL27t1b5XlUqVR49NFHAQDx8fE4cOAA3N3djb5VtH379rCyssL27duxdOlSpXtGRgaeeOIJo4lhRXr16gUAeOutt/SGX79+Pd59911YWRk+wu7r64sRI0agsLAQPXv2REJCgl5/EcHu3bvx0ksvKbfbZmZm4tFHH0VCQoJe4+zFxcWYP38+rl+/Dnt7e71krTKquo2MGzcOrq6uWLJkCUaOHIn09HS98V27dg1ffvklpkyZotddlyhv2bKl3ARLl/CtXbsWgP4VvrCwMHh5eWHt2rUQEYOrf0D1li0AvPDCCwgODsamTZvwxBNPGFxJzcrKwg8//KDcvnwn0v1Gpd+OWtqoUaNgZWWFn376CXPmzFHWw4KCArz22ms4dOgQnJ2d8dJLL9VZzER0h6uxBiWIqN6pqB3AqrbLpxvulVdeEX9/f7GyspKWLVtKaGio0tbVAw88oDRkXNrkyZOVMn5+fhIdHS22trbi5eUl8fHxRqcnItKtWzcBII6OjhITEyOxsbEycOBAvTK68Va1Ifht27aJm5ub8tFoNAJA7Ozs9LqfP3/eYNglS5aIhYWFMj+tWrUSa2trASDR0dFG2y4T0baJZ2NjIwDEw8NDoqOjxc7OTmkbLjk5udLxFxQUSHR0tACQcePGGfR/7733BICEhoZKTk5O5RfM/ztw4ICybAHISy+9VGbZ0aNHK+UCAgKU39fa2lo++eQTASCBgYEGw+mGuVlqaqp4e3sLANFoNNKyZUsJCgoSADJ+/PgyG4IvLCyUJ598Uhmvt7e3tGvXTqKiosTR0VHpfvToURERuX79utLN3t5eoqKipE2bNuLu7i4ARKVSyRdffFHpZXYr28i2bduU6VpbW0vz5s0lJiZGgoODRaVSCQCDdf/UqVOiVquV5du5c2eJjY012Ob/+OMPZfrh4eEG0+7fv7/Sf/78+UbnrarLVufo0aPSqFEjASAWFhYSHh4uMTEx0rRpU7G0tBQAEhMTozdMRe0bVlSHVderr76qt+3r5qlBgwZKtwcffLBK43z33XcFgFhaWkqrVq0kNjZWYmNj9bb1jz/+WPmNvby8pG3btuLi4qKs/7/88kuV56VVq1ZKzE5OTsryLz1/M2bMqPJ4iej2xwSQ6A5WWwngxIkTJTk5WZ555hnx8fERtVotoaGhMmPGDCksLCwzno8++kgiIiJErVaLp6enDBo0SJKSksqcnohISkqKDBkyRPz8/MTKyspoIlHdBFDXOHJFH2MNoouIbN++Xe6//35xdXUVjUYjoaGhEh8fX2ZD7zqHDh2SRx99VDw9PUWtVkujRo1k5MiRcu3atSrF/9ZbbwkAadGiheTn5xv0Ly4uls6dOwsAefXVV6s0bp2IiAhlOWzdurXMciUlJTJ37lwJCwsTtVot7u7u8sADD8jOnTvl7NmzVU4ARUROnz4t/fv3lwYNGoitra20atVKWZfLSgB1fv31V+nTp494e3uLtbW1eHp6SnR0tLz66quSkJCgJGBFRUWybNkyGTRokISFhYmzs7PY2tpK06ZN5cknn5QDBw5UaXnd6jaSmpoqb775pkRFRYmDg4PY2tpKkyZNpFevXvLxxx9LSkqKwTD/+9//JDY2VpycnJQk4uZtOycnR0kUX3zxRYNxLFiwQPktKprnyi7b0jIzM+W9996TmJgYcXJyEo1GI0FBQdKtWzeZPXu2we9oqgRQN93yPrGxsVUaZ0FBgUycOFFCQ0OVk0zG1t1t27ZJnz59xMPDQ6ytrcXX11eefPJJOXz4cLXmRbeNlPep6eVHRLcHlYiZ3pRPRFUWHx+PSZMmYeLEiSZ/eQIRERERVR2fASQiIiIiIjITTACJiIiIiIjMBBNAIiIiIiIiM8EEkIiIiIiIyEzwJTBERERERERmglcAiYiIiIiIzAQTQCIiIiIiIjPBBJCIiIiIiMhMMAEkIiIiIiIyE0wAiYiIiIiIzAQTQCIiIiIiIjPBBJCIiIiIiMhMMAEkIiIiIiIyE0wAiYiIiIiIzAQTQCIiIiIiIjPBBJCIiIiIiMhMMAEkIiIiIiIyE0wAiYiIiIiIzAQTQCIiIiIiIjPBBJCIiIiIiMhMMAEkIiIiIiIyE0wAiYiIiIiIzAQTQCIiIiIiIjPBBJCIiIiIiMhMMAEkIiIiIiIyE0wAiYiIiIiIzAQTQCIiIiIiIjPBBJCIiIiIiMhMMAEkIiIiIiIyE0wAiYiIiIiIzAQTQCIiIiIiIjPBBJCIiIiIiMhMMAEkIiIiIiIyE0wAiYiIiIiIzAQTQKJ6LCEhASqVCnFxcXrdExMToVKpEBQUZJK4zMGQIUOgUqmwePFiU4dicsXFxZg2bRpCQ0OhVquNrpN0+4mPj4dKpUJ8fHyNjtfctx1zrJ9ra12qL26XdTooKAgqlQqJiYmmDqVcBw8exP333w9XV1dYWFhApVIhISHB1GHVGJVKBZVKZeowymVVWyPesmULvvzyS2zZsgUpKSmwtraGv78/evTogVdeeQWNGzeu0eklJiZi8eLFCAoKwpAhQ2p03NV14MABrFmzBi1btkSfPn1qdVrr1q3Dd999h+3bt+Py5csoLi6Gl5cXoqOj0b9/f/Tt2xfW1ta1GgMR3ZneeecdTJs2DWq1GpGRkbCzs0Pz5s0BAHPnzkV6ejpGjBgBFxcX0wZKta4+7muJbkVdHqvdijVr1uDAgQPo06cPWrZsaepwqi01NRVdu3bF9evX4efnh/DwcKhUKjg7O5s6tEpZvHgxEhMTMWTIkNv6JE+NJ4AFBQV47rnnsHTpUgCAi4sLwsPDkZeXh+PHj+Pw4cP46KOPMGPGDIwYMaLGppuYmIhJkyYhNja23uyUDhw4gEmTJmHw4MG1VqlcuXIFAwcOxKZNmwAAjo6OCA4OhrW1Nc6fP49Vq1Zh1apVCAkJwebNm+Hj41MrcVDdsra2RmhoKPz8/EwdCt3hRASffvopVCoVtm/fjjZt2uj1nzt3Ls6dO4chQ4YwATQD9XFfS3QrKnOs5uPjg9DQUJMmKWvWrMGSJUsQFBRUZgLYuHFj2NjY1OsT/t999x2uX7+Ohx56CKtWrYKFxe11M+LixYuxefNmxMXFlZkAhoaG1m1Q1VDjCeCAAQPw008/wd3dHfPmzcOAAQNgZaWdzNWrVzFlyhTMmzcPr7/+OkQEr7/+ek2HYDYyMjLQqVMnnDhxAiEhIZg1axZ69+6tt+Hv3bsXs2bNwooVK3D58mUmgHcIPz8/HDt2zNRhkBm4cuUKrl27Bk9PT4Pkj4jIHEyfPh3Tp083dRgV+uOPP0wdQoV0xy49e/a87ZK/yrodjs9qNAH87LPP8NNPP8He3h5//vmncouQjru7O+bOnYsGDRogPj4e48aNQ/fu3REZGVmTYZiNV155BSdOnEBERAS2bNkCNzc3gzJt2rTB999/j0cffRT29vYmiJKIbme5ubkAAFtbWxNHQkREtzvuU+oJqSFFRUUSGBgoAGTq1KkVlo2IiBAA8uSTT+r1mzhxogCQiRMnGh32q6++EgAyePBgpVtsbKwAMPoJDAxUyg0ePFgAyFdffSXHjx+XAQMGiIeHh9jY2EjLli1l0aJFRqdZejhjjMWsWxbGPrGxseUun8o4efKkWFhYCADZsWNHtcZRUlIiy5Ytky5duoizs7PY2NhIaGiojB07VtLS0owOo5sHEZEff/xROnfuLM7OzgJAzp49K2fPntVb7p9//rm0adNGHBwc5ObVLSkpSYYNGyYhISFiY2Mjzs7OEhcXJytWrDA67aysLJk0aZI0b95c7OzsRKPRSMOGDSU2NlamT58uBQUFStnScZSUlMj8+fMlMjJSbG1txcPDQ5588kk5d+5cmcvm6tWrMmbMGGnatKnY2NiIi4uLxMbGytdffy0lJSUG5Uuvl3l5eTJx4kRp3LixEuPrr78uWVlZZU5v1apV0qFDB7GzsxNXV1e57777ZM+ePbJp0yaj68zNy7m00r/RunXrpHPnzuLg4CBOTk5y7733yt9//11mHKdOnZJHH31U3N3dxdbWVqKiouSTTz4Rkf/W6bNnz5Y5vE5RUZH4+fkJANm7d2+Z5V555RUBIKNHj65wnH379hUAMmvWrDLLrF27VgBIq1at9GJZs2aNPP300xIRESFOTk5ia2srYWFhMmbMGLly5YrRcZW13VenPijt6NGj8vTTT0tgYKCo1WpxdXWV3r17yx9//GG0/NWrV2XUqFESGhoqGo1G7OzsJDAwUHr27CkfffRRmcvCmB07dsiYMWMkOjpaPDw8RK1WS8OGDeXJJ5+UQ4cOGZQvqw7TzX95/Tdt2qQ3rrS0NJkwYYI0a9ZM7OzsxMHBQWJiYuTzzz+X4uJig2mXXs5nzpyRwYMHi6+vr1haWpa5bEsrve0UFBRIfHy8hISEiEajEV9fX3n55ZfLrOequpxEtPXpkiVLlDrR2tpavLy8pHXr1jJmzBhJSkrSK1+d37Wqy1BEpLCwUGbMmKFMx9fXV5599llJSUmpcF29WXX2tRcvXpSnn35avL29RaPRSEREhCxYsMDo+Cv7m2dlZcnkyZOVfYGjo6O0a9dOFixYIIWFhQbjNXbcUFpZ9azO77//Ll27dhVHR0dxdnaWbt26yR9//FFmPXxz92XLlkl0dLTY2tpKgwYNpF+/fnL69OmyFnO5CgsL5ZNPPpGOHTuKs7OzaDQaCQ0NlTfffFMyMjIMyle3/tPZsGGDPPzww+Lj4yNqtVp8fHwkLi5OFixYIHl5eUq50utSenq6vPbaa+Lv7y9qtVoaN24s7777rtHfpjx1EXtlj9WM1ft1tU/SrU9lfYwdexrbTxcUFMj8+fOlbdu24ujoKHZ2dtKiRQuZMmWKZGdnG5Sv6fVYt46Ut6wr2hZra5vLzs6WWbNmSUxMjDg7O4utra00adJEnnzySUlISNCLrbx9ok7p47CbVbX+Kr1MiouLZe7cudKsWTPRaDTi6ekpzzzzjKSmppY5b2WpsQRw+/btAkCsrKwq3ChFRD744AMBIHZ2dnozXJ0E8NVXX5XIyEgBIE5OTtKxY0fl069fP6WcbgOeMGGCUnG2bt1arwIYNmyYwTSrc8DXr18/CQkJEQDi6empF9Orr76qlCu9YVfmwFpn6tSpBpVKVZSUlMjjjz+uTDs4OFhat24tarVa2YiMbSy68u+9954AEC8vL2nbtq14eHgYJIAvvviiABB/f39p06aNuLi4KONJSEhQEkdbW1tp3ry5+Pv7K+MfNWqU3nQLCwulffv2AkAsLCwkNDRU2rRpI76+vkoifP36daV86TheeuklASABAQESHR0tNjY2AkA8PDzk2LFjBvN48uRJJRa1Wi2tW7eW4OBgJbannnrKIAnUrZePP/64dOnSRVQqlTRr1kxCQ0OV+Lp37270t5gxY4Yybh8fH4mOjhYHBwfRaDQyefLkaieAn3zyiahUKvHx8ZHWrVuLvb29ABAHBwc5evSowXAHDx4UFxcX5TeJjo5Wto3hw4dXKQEUEXnjjTfK3KZERPLz88XNzU0AlHlQXdrKlSsFgLRu3brMMo899pgAkJkzZyrdkpKSlPVGtyzCwsKU9SAoKEhSUlIMxlUbCeD333+vbGOOjo7SsmVL8fb2FgCiUqlk/vz5euXT09OlcePGyroYEREhrVu3Fk9PT1GpVOLs7FzmsjBGNy43NzeJjIyUqKgove3w5qStY8eO0qZNGwEgGo1Grx6bNm2adOzYUTQajQCQNm3a6PUvfaLh0KFDygkB3Xw0btxYVCqVAJB+/foZbFO65Tx+/HhxcXFR6uuwsDCJj4+vcF51O80uXbrIfffdJwAkJCREWrZsKVZWVgJAmjRpIpcvX77l5SQiMmrUKGXbCwgIkLZt20qjRo2U33v16tVK2er8rtVZhkVFRfLAAw8ocTVt2lSioqLE0tJSAgIC5NVXX61SAljVfW18fLx4e3uLjY2NtG7dWnx9fZVYpkyZYjD+yvzmqamp0rx5c2WbbtGihYSHhyvj7d69u+Tm5uqN91YSwCVLlijL2N3dXdq2bStubm5iYWEhs2bNqvBgdPz48cr3qKgoZXvx8fGp1LFSaRkZGdKlSxdl3gMDAyUyMlJZx8LDww3W5+rWfyL/naDTbQtt2rSRwMBAZZ9Wel+gq/dGjBgh4eHhYmVlJS1btpSgoCBlHM8++2yV5rcuYq/ssZqxer+u9knJycnSsWNH8fT0VOqx0nGWvnhR1n46JydHunXrpiyT8PBwadGihbI8WrZsKVevXtUbpqbX40WLFpU5H7plXRMJYFVjPXfunF4dEhISIq1btxZXV1e9WP7++2/p2LGjODk5CQCJjIzU+x3WrVunjFM3rptVp/4qvUx0x+0hISHSrFkzZV/WrFkzvRMylVFjCaCuImzRokWlyu/bt0+Z4X379indq5MAilS80oj8twFbWVlJ165d9TLmFStWiLW1tQCQX375xehwVT3gq2inI1L9BFB3QDNixIhKD1Pahx9+qByEbtiwQemuq2gASExMjMFwuljVarV8/vnnygFHYWGhFBYWKvNjaWkp9vb28tNPPynD5uTkiIjIxYsXxdXVVVQqlUybNk1vpd2+fbtykPPzzz8r3X/88UcBIFFRUQZn0lNTU2Xu3Ll6Z7B0cVhZWYm1tbUsX75c6Xf16lW55557BIC0a9dO76CppKREOeCNjY3Vq4R/++03JYn6+OOP9WLQ/dbW1tYSEREhx48fV/rt2LFDqTB+++03veH+/vtvsbS0FJVKJQsWLFBiuXHjhgwcOFBZJ6uTANrZ2emts5mZmXL33XcLABk4cKDeMMXFxUql1KtXL7l27ZrestdoNEoslV1PT548qRw0lb46q6PbebZp06ZS48vLy1MOwksvX53s7Gyxt7cXlUol58+fV7qnp6fL4sWLDa72XL9+XTkAHjJkiMH4ajoBPHjwoGg0GrGxsTG4YrN27VpxcnISS0tLOXDggNJ99uzZAkB69OhhEP+5c+fkgw8+MBpDWZYsWWJwYqewsFAWLlwoVlZWEhwcbHAlqbx1TaTiK8NZWVlKsjN8+HC9qxSHDx+WZs2aCQCDq0K65WxpaSkPPvig3vzfvIM0RrdPsLKyEicnJ/nzzz+VfufOnZOoqCglcbpZVZdTamqqWFhYiLOzs2zbtk1vuNzcXFm+fLkcPHhQ6VbV37W6y3DevHkCQBo0aCBbt25Vup89e1YiIyOVbbqyCaBI1fa11tbW0q9fP72Tcx9//LEAEBsbG73upYcr7zfXXXVp1qyZnDp1Sum/Z88e8fLyEgAyduxYvfFWNwE8d+6c2NnZCQB56623pKioSES068L48eOV5VfWwahu3St9YJicnCwtWrQQADJu3Lgyl6Exjz76qACQu+++W2/9vHbtmjzyyCNG1+fq1n9z585V9iPLli3TW9/T0tJkzpw5esdQunrP2tpaunTpIhcvXlT6rV27ViwtLQWA0ZOPZamr2CtzrGas3q8v+6TSyqqPdSeofH199Y65T548KWFhYQJABgwYoDdMba3H5c3HrSaAVY21qKhIoqOjlWORI0eO6PXfv3+/wfGe7k4IYycCdcpKAKtTf+mWibW1tfj6+squXbuUfsePH5eGDRsKAOVurcqqsQRwxIgRAkAefvjhSpVPT09XFlDpJKEuEkCNRiPJyckG/UeOHCmA9oyxseFqIwFMSkoSPz8/8fPzM0hsytOyZUsBIPPmzav0MDolJSXKFS5jB5AXLlxQzijefFua7jcr66pO6YR2zpw5RsvolvPrr79utP/PP/8sAKRbt25Kt+nTp1dpfkvHMXz4cIP+ly9fVs62lT4w3LhxY7nryMyZM5XKp3TiqPutVSqV7Nmzp8x5vjmWJ598UgBI//79DYbJzc1VzpRVJwE09hv9888/AsDgCsP69esF0J4pTU9PNxiu9K0bVTlR0blzZwH0r37oPPjgg0YPWsvz9NNPCwCjV4CWL18uAKRz586VHp+IiL+/v8GdCCI1nwDqDtDKWod1J2WeeeYZpdsLL7xgUEfWFt26uH37dr3ut5oAzp8/v9x9w8GDB0WlUklwcLBed91y9vb2Lvf26bKUvl3n/fffNzpd3TZblduYjC2nHTt2VGn/V9XftTrLsKSkRAICAgSA0VtKS5+Era0EsKzfrnXr1gJAVq1aVaXhTpw4oVyNM3Yr+w8//CAAxN7eXjIzM5Xu1U0AdVcS7rnnHqPD6Q4EyzoYLWs/qLstsLInzEX+W18DAwP15k0nOztb/P39RaVSSWJiYqXHa6z+y8nJUe7OWLp0aaXGo6v3bG1tjR7L6Oo/Y9tiddVU7NVNAEXqxz6pNGP1cUZGhnIiw9i+ePfu3UpdWDopqY31uKL5uNUEsKqx6uoMT09PgyugZaluAljd+qv0vmzlypUGw+n2Dw8++GCl4tepsdfv3LhxAwAq/aKR0uV0w9aVRx55BN7e3gbdX375ZQDA9u3bkZ2dXSexNGzYEBcuXMCFCxfQsGHDSg9X1eVd2tGjR5GUlAQbGxs899xzBv39/PzQt29fAMCGDRuMjuOpp56qcDpllVm1ahUA4NlnnzXa/95774VarcZff/2FoqIiAIC/vz8A4Ndff0VOTk6F0y7tlVdeMejm6emJfv36AQD+97//Kd1189u/f3+j68iLL74IjUaDc+fO4fjx4wb9W7ZsafRNiW3btgUAnDlzRq+7bnovvfSSwTA2NjZ45plnypyvihhbvs2bN4eNjQ0yMjKQlpamdN+4cSMA7bZh7DXXTz/9dLVi0MW/ZMkSve5XrlzBb7/9BrVajccee6zS43v88ccBAMuXLzfop+umK3OzP//8E6+//jruu+8+dOnSBZ06dUKnTp2QkZGBnJwcnDx5stJxVFVBQQHWrVsHS0vLMl+d/+CDDwIANm/erHTTrferV69WtoVbdezYMUycOBGPPPII4uLilOWgm+7BgwdrZDo6FW3vLVq0QFBQEM6cOYMLFy4Y9O/bt+8tvcBKrVYbnXaLFi3QqVMniIjReq4qy0n3O+3atQvnz5+vMKaq/q7VWYZHjx7F+fPnYWNjY3Sda926Ndq3b1/htG/FY489ZvS3K6s+1CnrN9+4cSNEBJ06dUKrVq2MDtewYUNkZ2dj+/bttxj9f/ViWfVfZerFoUOHGnSraP6NWb16NQDtm9YdHR0N+tvZ2eGee+6BiGDr1q0G/atS/23fvh1paWnw9fXFE088UekYAe3+29ixTHXmua5jr47bYZ+0bds25OTkICAgAA899JBB/7Zt26JDhw4QEWWdv1lNrcd1oSqx/vTTTwC0xyrGXqRYk261/mrQoAEeeeQRg+7V/R1q7C2gugqpsolT6XLGKrPaFB4ebrR7cHAwNBoN8vPzcfr0abRo0aJO46qKqi7v0k6cOAEACAgIKPPAqlmzZnplb1bWMtRxd3eHu7u7QfesrCwkJiYCAJ5//vlyx5GXl4e0tDR4eXmhT58+CAoKwoYNG+Dr64t7770XnTt3RlxcnBKrMdbW1mjSpEm581B6HnXfIyIijA7j6OgIf39/nDp1CidOnEBYWJhe/8aNGxsdztPTE4B2/nXS09ORmpqqF0tZMVZHWbF4eHggKSkJWVlZSoWn29GUtc4HBgbCyckJmZmZVYqhf//+GD58OH799VdcvXpVWSe+/fZbFBYWol+/fnB1da30+Lp16wZvb28cP34c+/fvVyrR9PR0rF+/HlZWVkpir1NQUICBAwdizZo15Y772rVrVZq3qjhx4gTy8vKgVqvRu3dvo2W0Jw2BixcvKt2efvppzJo1C4sXL8Zvv/2mrPddu3ZFcHBwleOYPn063nrrLZSUlJRZpqaXw7///gvgv8bkjbl69SoA7bzffPB4K9sAoD3JVtY+Jjw8HNu2bTOo56q6nPz8/NC/f3+sWLECTZo0QdeuXREXF4fOnTujffv2SlNIOlX9XauzDHXzFBgYCDs7uzLnf+fOnWXO462qSn14c1zGVFQ/W1hYICwsDBcuXMCJEydw7733VjVkPRXVixUdI7i7uxs9oVbR/BujWwdWr16Nv/76y2iZc+fOAdCvQ6pT/x09ehQA0K5duyq/pr+6v7kxdR17ddwO+yTddhMWFgaVSmW0TLNmzbBjxw6jx3w1uR7XtqrGqltfavtkGHDr9VdNblsAUGNbh65B6tOnT1eqfOlydd2YtW5h3UylUsHDwwNA3V+VrCrdMjt79myVh9WtJGUtBwDw8vICUPZyqOiMfFn9MzIylO/bt28v81NQUADgv9cF29vbY+vWrXj66adRUlKC77//Hq+++ioiIyPRrFkz/PLLL0an5+bmVuZOwNg83uqyKWu+dTHoDvJLTwuAst6VNa3qqEosuhMJ5Z2Mqc6JGnt7ewwYMACFhYV6Z0h1VwRvvjLRv39/5Sxo6U/p2AcOHAhA/4zrypUrUVBQgB49ehiceHjvvfewZs0aeHt7Y+nSpUhMTEReXh5Eews8OnbsCAAoLCys8vxVlm69LygoKHOd1x3U5eXlKcP5+vpix44d6Nu3LzIyMrBkyRI8++yzaNy4MTp06IAdO3ZUOoYtW7ZgwoQJUKlUmD59Og4fPoysrCyUlJRARPDmm28CqPnloJv3ffv2lTnvum1Jt72XdqvN11R1W67uclq6dCkmTpwIT09PbNiwARMmTEDnzp3h6+uL2bNn6yWTVf1dq7MMdfVLWXVL6fmvLVWpgyozXE3su6qionqxojqxovmvCt06cOrUqTLXAd3V39LbUXXqP92JPhcXlyrHWd3f3Ji6jr06bod9Um0f19QnVY21LteXujy+rIwa+/XuuusuAMCRI0eUM5Hl2bJlCwDtbQtRUVFKd93ZibJmpCZuzbxy5YrR7iKi9CtdsddFTFWlW96lbxerLAcHBwBQrj4Zc/nyZQA1f3VWN21AezCsq/DK+gQFBSnlGzZsiC+//BLXrl3Dzp078d5776FNmzY4cuQI+vTpg127dhlMLy0trcyz+Lr5Lz2PdblsSi+LstbJ8uKoSbqKpbwzSNU9oLr5NtB///0X+/fvh7e3t8EZrj179hg9sClNd8vod999p2yTuh2vsdtJv/nmGwDA4sWLMWjQIAQGBkKj0Sj9k5KSqjQ/1akPdL+1n59fhev8zeMNDw/Hjz/+iPT0dGzatAnx8fEICwvDzp070aNHD+WKekV0y2HMmDEYP348IiIiYG9vr8xPVZdDZenm/eTJkxXOd1xcXI1Pv6xtCzBeB1R3OdnY2CA+Ph4XLlzA0aNH8dlnn+GBBx5AWloaxowZg/fff1+vfFV+1+osQ90wlZn/20V16+fq7sMrqhfr8kSxbt6/+OKLCteB+Ph4Zbjq1H+6ZZeenl57M1QJt0vspt4nVcSUx3xVVdfH23W5vtS336HGEsCYmBgEBASgqKgIX3zxRblli4uLlTKPPPKI3u0xugq3rJ3WqVOnjHYv67K2MbpLvjc7e/Ys8vPzYWFhoXeptS5iqqr+/fvDwsIC+/fvr/ItPE2bNgUAnD9/vswd2+HDh/XK1hRnZ2f4+vrqTaOqrKysEBMTg3HjxmHPnj149NFHUVxcjC+//NKgbGFhYZlXpXXrQel51H0/cuSI0WFu3LihVM63umxcXFyUM0HHjh0rN8bappuXf/75x2j/8+fPV/n2T5277roLYWFh2LdvHw4dOoTFixcDAJ588klYWlrqlU1MTKwwIYqJiUHjxo2RlJSEbdu2ISUlBQkJCbC1tUWfPn0Mpq87kNadNCktLS1N73apyqhOfRASEgJra2skJydX+7YejUaDuLg4TJw4EYcOHULHjh2RlZVl9NkTY8pbDkD1n/2rqJ7T3e5y6NChao3/VuludzbGWB1QE8spLCwMzz//PNauXYuPP/4YAMrcL1bmd63OMixdz5f13HR16pfa3K9VpKL6uaSkRKlLS/+m1d2HV1Qv6m7LrAvV3Y6qU//pHqvYs2dPubdB17a6iv1W1+m62idVN07denz06NEyE6vaOuarqupuq9WlW1+qchx9q79DVeuv2lJjCaClpSXGjx8PAJg6dWq5FeOUKVNw5MgRWFtbY+zYsXr9dM8/7Nmzx2C47OxsfPfdd0bHaWtrC8D4LUQ3W7lypZJll6bbUXfs2FHvUmt5MV24cEHvJSLVjamqQkJClNsOhg4dWuFB5Zo1a5TnGcLDwxEQEIC8vDwsXLjQoOylS5ewcuVKAEDPnj1rOHIoD7HOnTu3Rsanu3f70qVLRvvrftfSrly5ghUrVgAAevTooXTXze+KFSuQkpJiMNxnn32G/Px8BAYGIjQ09JZj7969OwDg008/NeiXn59vNKmtDbo4Vq1aZfSsti5pqy7dyxIWLVqknP0s62UolaE7q7p8+XJ8//33KC4uxgMPPKB3VVVHtx0a2+bnzJmD4uLiKk27OvWBnZ0devbsiZKSEsyfP79K0zPG0tJSefC7rPX+ZuUthw0bNlQ7AayontNt7/Pnz6/yLSo1oaCgAIsWLTLofujQIWzduhUqlUpZ/4GaX04V1U+llfW7VmcZhoWFwd/fH7m5uVi6dKlB/wMHDlTpFmKd2tyvVaRHjx5QqVTYtm0b9u/fb9B/1apVuHDhAuzt7ZXb6ID/ttkDBw4YvHSnpKQEX331ldHp6daLsuq/W60Xq+Lhhx8GAHz99dd6L/CqSHXqv44dO8Ld3R0XL16s9Amm2lBXsdfEOl0X+6TqxtmpUyfY2dkhKSlJeelJaXv37sWOHTsM6kJT0G2rZ86cMbqeGztmvRW6BF13d1llVPd3qG79VWuq9M7QCpSUlMj9998vgLbtr2+//VbvNbZXrlyR1157TXmd6axZswzGce3aNeX1/J999pnS/fr169KvXz+l3Z2bX9ebmpoqgLaR69Ltu5RWuh3Au+++W69ByFWrVilNH6xdu1ZvuCNHjijD/frrr0r3S5cuSZcuXcpsS2nPnj0CQBo1aqTXRl1pSUlJEhgYKIGBgVVqBkJEu6x0bUOFhITITz/9ZNDe2v79++Wxxx4TlUol+/fvV7rrXjnv5OQkv//+u9I9JSVFeXV/+/btDaap++3KUtEr40W086xrYPP11183aAsqLS1NFi1aJJMnT1a6vf/++/LBBx8YNPp67tw5pWHid955xyAOKysrUavV8sMPP+iNv0ePHgJo2325uR3Atm3bCgCJi4vTa1T3f//7nzg4OAhg2N5KdV8zvnfvXrGwsBCVSiWffPKJEktWVpY8/vjjt9QOYFmMvSa6uLhYaSfn/vvv1/tNVq9eLTY2NlVuB7C05ORksbKyUhotrWzbf2U5evSoUs/o2m1cs2aN0bK6NjMffPBBuXHjhohof+clS5aItbW1Ut/c/Ernsl5VXd36YP/+/aLRaMTS0lKmT5+utItZevi5c+fqrVsTJkyQhQsXGmwj//77r9Ko9pdfflmJJfZfW62NGzeWM2fOKN13794tfn5+ynK4Oe6Ktmnd8i2rDaIbN25IcHCwAJDHHntMLl26ZND/+++/N2gWpjKvPC9P6XYAnZ2dJSEhQemXlJQkrVq1EgDSt29fveGqs5x+//13GT16tBw+fNhg3p544gkB9JsXqurvWt1l+MEHHwgAcXV11Wu2IjExUVq0aFGtdgCrsq+talMplfnNde1oRUZG6jXfsW/fPvHx8RHAeFtfuuU6YcIEpZ7Nzc2VYcOGlVnPlm4HcOLEiXrtAL755psVtgNY3n6wonramAEDBggAadWqlcFr5IuKimTTpk3y+OOP67WtW936T9eGpL29vXz77bd6+8lr167J+++/b7QdwKo24VWeuoq9MsdqFa2bdbFP0tVNjz76qN48lVZRO4B+fn56686pU6ckIiJCAMP2gWtrPa5oWbZr104AyFNPPaUc0xYVFcn06dNrfJsrKipSfq+YmBg5duyYXv8DBw4YtAP4yiuvGK1nKpqWSPXqr+o2jVGRGk0ARbQNY+paqgcgLi4u0rp1a4mIiFAO/qytrWX27NlljmPy5MnK8H5+fhIdHS22trbi5eUl8fHxZVYi3bp1E0DbuHlMTIzExsbqrdC6le6NN94QZ2dnsbGxkejoaAkKClKm9/LLLxuNaejQoUqZRo0aScuWLcXKykrCwsKUpPbmiq+4uFhCQkIE0Lav1qFDB4mNjZXXXntNKVPdhuB1UlJSpEuXLso4HB0dJSoqSqKjo5U25ABIWFiY3kFDSUmJ3u/UpEkTad26tZIEBwQEGG0bqyYSQBGRbdu2ibu7u7I+NG/eXGJiYiQ4OFhpJ6X0b1f6xEFQUJC0a9dOwsLClMZlIyMj9dqvKx3HSy+9pHxv06aN2NraKr/JzY1+imgbRtU1rKnRaKR169bSpEkTZfqDBg0yqHyrmwCKiEybNk0Zt6+vr7Rp00YcHR1Fo9Eo20JtJ4Ai2namXFxcBNA2oNumTRtl2xg2bJgyXOkGbavigQceUGKrStt/ZdG1hamrZ/Lz842W27t3r2g0GgG0Jzyio6OVg8FBgwaV2aZPeTup6tQHItoTTboDShsbG2nZsqW0a9dOaZfz5sr/oYceEgBiYWEhTZo0kXbt2umti127djVoK6osGRkZShKhVqulefPmEhoaKgAkIiJCaauyqgng0qVLlXgiIyMlNjZWYmNj9U44HT16VBo1aqTMS3h4uMTExEjTpk2VbTgmJqbSy78ydNtcly5dlAOupk2bSqtWrZR9UXBwsEF7n9VZTqtXr1aWgYeHh7Rp00aioqKU39rZ2Vmv8eXq/K7VWYZFRUXSu3dvvf2Abl0NCAhQGp2uSgIoUvl9bW0kgKmpqdK8eXMBtA3GR0VFKQewgLbNPl2j8aUtW7bM4DdycnISBwcHmT17dpn18+LFi5V9koeHh7Rt21bc3d3FwsJCaRf25jYsa+vA+caNG9K9e3dl2ICAAImJiZHmzZsr+zUAevNf3fqvpKRE2XcC2sSmbdu2EhQUpKxvpfchtZEA1lXslTlWq8y6Wdv7pFOnTinHaIGBgdK5c2eJjY3Vi6ms/XtOTo507dpViS8iIkKioqKU5REVFWXQDp6pEsBNmzYpdbSLi4u0adNG3NzcxMrKSrl4UZMnXc6dO6fU8br9RHR0tNKe5M31wpYtW/TKdunSRWJjY+W3336rcFrVqb9umwRQJyEhQQYNGiRBQUFiY2MjDg4OEhERIa+99pqcPHmywuE/+ugjiYiIELVaLZ6enjJo0CBJSkoqtxJJSUmRIUOGiJ+fn7LylF4gpVe648ePS//+/cXDw0M0Go1ERUXJ559/XuZZlcLCQnn33XelcePGolarxc/PT1555RW5fv16uRXfiRMnpF+/fuLp6alsaKV/xFtNAHV+/vlneeKJJ6RRo0ZiZ2cnGo1GAgMDpW/fvvL9998bPUgsKSmRpUuXSufOncXJyUk0Go2EhITImDFjymwQs6KNvSorYmpqqrz55psSFRUlDg4OYmtrK02aNJFevXrJxx9/rHe17+jRoxIfHy9dunQRPz8/UavV4uXlJe3bt5cPP/zQ4GpK6ThKSkpk3rx5EhkZKTY2NuLu7i5PPPFEuY3lXrlyRUaPHi0hISGi0WjEyclJunTpIsuWLTO6jtxKAigi8uOPP0pMTIzY2tpKgwYNpHfv3rJnz54yh6uNBFBEu4N59NFHxc3NTWxsbKR58+ZKsqZL2G++alFZq1atUg6q09LSqjWO0nQHXwBk6NCh5ZbdtWuXdO/eXRwcHMTe3l5atmwp8+fPl5KSkmolgNWtD0S0V19ee+01CQsLE1tbW3FwcJDQ0FB5+OGHZcmSJXrLd8+ePTJ+/HiJiYkRb29vZVqxsbGydOnSSid/OpcuXZKnnnpK3N3dRa1WS6NGjWTkyJGSkZFRZtyV2abnzZsnLVq00DsIvXl5ZmZmynvvvScxMTFKfRMUFCTdunWT2bNnG6yPNZUAxsbGSkFBgcTHx0uTJk1Eo9GIj4+PvPTSS3p3gZRW1eV09epVmT9/vjzwwANKHezs7CwtWrSQsWPHGiSZ1f1dq7oMRUQKCgpk+vTp0rRpU1Gr1eLj4yNDhw6VlJSUCtfVslRlX2vMrSSAIto7JN59912JjIwUW1tbsbe3l7Zt28qHH35ocBdMaT/88INER0eLjY2NuLq6ysMPPyyHDx+usH7euHGjxMXFiYODgzg6OkpsbKxs2LBBDh06pBw8l1ZbB84i2mTlm2++kZ49e4q7u7tYW1uLj4+PxMTEyLhx42T37t0Gw1Sn/tP59ddf5f777xcPDw9lPe3WrZt8/PHHeglObSSAdRW7SMXHapVZN2t7nySivRMpNjZWnJyclBMTpZd5efv3goICmTdvnrRp00bs7e3F1tZWmjdvLlOmTDF65dNUCaCIyB9//CGdOnUSOzs7cXJyku7du8u2bdsqbAi+urFmZWXJ9OnTpXXr1uLg4CB2dnYSEhIigwcPli1bthiU//bbb6Vdu3Zib2+vjLf0/FQ0rarUX7WVAKr+P1CzMGTIECxZsgRfffXVLT1/RPVfYmIiGjVqhMDAwEq/JZHKlpaWBnd3d7i4uOD69evVGsenn36Kl156Cf369VOevySqLQkJCejatStiY2ORkJBg6nDoDrRy5Ur069cPDz30UIVtuhER1Sf1rxEPIqp3dC9JKOvNiJWhexGH7oUwRES3M129WCcvbCAiqkFMAIkIgPaV5p9//rneK/NFBF9//TXefvttAMCLL75YrXGvXLkSe/fuRXBwsEHbf0RE9dXKlSuxbt06vTcz5uTkYOzYsfj1119hb2+PQYMGmTBCIqKqs6q4CBGZg7S0NLzwwgt4+eWXERgYCDc3N71XMb/wwgt44IEHqjTOuLg43LhxQ3nl8ZQpU2BhwfNORHR7+PfffzFp0iTY2NigcePG0Gg0OHr0KHJzc2FpaYnPPvsM3t7epg6TiKhKmAASEQBtQ8Njx47Fhg0bkJSUhPPnz8PJyQl33303nnvuOaXdyarYvHkzLC0tERwcjFGjRiltJRER3Q4eeughXLhwAVu2bEFSUhJyc3Ph4eGBBx98EKNGjVLabSQiup2Y1UtgiIiIiIiIzBnvxSIiIiIiIjITTACJiIiIiIjMBBNAIiIiIiIiM8EEkIiIiIiIyEwwASQiIiIiIjITTACJiIiIiIjMBBNAIiIiIiIiM8EEkIiIiIiIyEwwASQiIiIiIjITTACJiIiIiIjMBBNAIiIiIiIiM8EEkIiIiIiIyEwwASQiIiIiIjITTACJiIiIiIjMBBNAIiIiIiIiM8EEkIiIiIiIyEwwASQiIiIiIjITTACJiIiIiIjMBBNAIiIiIiIiM8EEkIiIiIiIyEwwASQiIiIiIjITTACJiIiIiIjMBBNAIiIiIiIiM8EEkIiIiIiIyEwwASQiIiIiIjITTACJiIiIiIjMBBNAIiIiIiIiM8EEkIiIiIiIyExYmToAojtVSUkJLl26BEdHR6hUKlOHQ0REdNsQEdy4cQO+vr6wsOD1CqKaxASQqJZcunQJ/v7+pg6DiIjotpWUlISGDRuaOgyiOwoTQKJa4ujoCEC783JycjJxNESkKCwEvvpK+/3ppwFra9PGQ0QGMjMz4e/vr+xLiajmqERETB0E0Z0oMzMTzs7OyMjIYAJIVJ9kZwMODtrvWVmAvb1p4yEiA9yHEtUe3lRNRERERERkJpgAEhERERERmQkmgERERERERGaCL4EhMiERQVFREYqLi00dCpkBS0tLWFlZsVkSIiIiM8YEkMhECgoKkJycjJycHFOHQmbEzs4OPj4+UKvVpg6FiIiITIAJIJEJlJSU4OzZs7C0tISvry/UajWvylCtEhEUFBTgypUrOHv2LEJCQti4MhERkRliAkhkAgUFBSgpKYG/vz/s7OxMHQ6ZCVtbW1hbW+PcuXMoKCiAjY2NqUMyDY0G+OWX/74TERGZESaARCbEKzBU17jOAbCyAu67z9RREBERmQSPBIiIiIiIiMwErwASEZF5KSwEvvlG+/2JJwBra9PGQ0REVIeYABIRkXkpKACeflr7vX9/JoBERGRWeAsoERERERGRmWACSEREREREZCZ4CyhRfZKdXXY/S0ug9Gv7yytrYQHY2lZc1t6+avERERER0W2NVwDpjjd9+nS0bdsWjo6O8PT0RJ8+fXD8+PEKh9u8eTOio6NhY2OD4OBgfPrpp7UfrIND2Z++ffXLenqWXbZXL/2yQUHGyxERERGRWWECSHe8zZs345VXXsHOnTuxceNGFBUVoUePHsgu5wra2bNn0bt3b3Tu3Bn79+/HhAkTMHz4cKxcubIOIyciIiIiqlm8BZTueOvXr9f7/6uvvoKnpyf27duHLl26GB3m008/RUBAAObOnQsACA8Px969ezF79mz0vflKXE3Kyiq7n6Wl/v+pqWWXvbmx78TEaodERERUFhHBokOL8FDjh+Bh52HqcIioEpgAktnJyMgAALi6upZZZseOHejRo4det549e2LRokUoLCyEtZHXxufn5yM/P1/5PzMzs+rBVeWZvNoqS3Sn02iAH3747zsRVduPJ3/EvL/nYfnR5fj54Z9hZ21n6pCIqAK8BZTMiohg5MiR6NSpEyIjI8ssl5KSAi8vL71uXl5eKCoqwtWrV40OM336dDg7Oysff3//Go2diGqIlZW2/b/+/bXfiahajl87jhm7ZwAAnox4kskf0W2CCSCZlVdffRX//PMPli9fXmFZlUql97+IGO2u88YbbyAjI0P5JCUl3XrARERE9VBOYQ5Gbx6N/OJ8dPbrjMHNBps6JCKqJJ76JLMxbNgwrF27Flu2bEHDhg3LLevt7Y2UlBS9bqmpqbCysoKbm5vRYTQaDTS8nYyo/isqAlav1n5/+GFeBSSqIhHB5J2TkZiZCE87T0ztNBUWKl5TILpdcK9HdzwRwbBhw7B69WokJCSgUaNGFQ7ToUMH/Pzzz3rdNmzYgDZt2hh9/o+IbiP5+cCAAdrvWVlMAImqaM2pNfjlzC+wVFliZpeZaGDTwNQhEVEV8HQN3fFeeeUVfP311/j222/h6OiIlJQUpKSkIDc3Vynzxhtv4KmnnlL+f/HFF3Hu3DmMHDkSR48exZdffolFixZh9OjRppgFIiKieuHU9VOYtmsaAOCVlq8g2ivaxBERUVUxAaQ73ieffIKMjAzExcXBx8dH+Xz//fdKmeTkZJw/f175v1GjRli3bh0SEhLQsmVLTJ48GfPnz6/dJiCIiIjqsZzCHIzaPAp5xXm4y/cuDG0+1NQhEVE18L4XuuPpXt5SnsWLFxt0i42Nxd9//10LEREREd1+pu2ahjMZZ+Bh64FpnabxuT+i2xS3XCIiIiIq19rTa/HT6Z9gobLAjC4z4GZr/IVoRFT/MQEkIiIiojKdST+DKTunAABejHoRbb3bmjgiIroVTACJiIiIyKi8ojyM2jwKuUW5iPGOwfPNnzd1SER0i/gMIBERmRe1Gvjqq/++E1GZ3tv9Hk6ln4KrjSve6/IeLC0sTR0SEd0iJoBERGRerK2BIUNMHQVRvbfuzDqsPLkSKqjwXuf34G7rbuqQiKgG8BZQIiIiItKTmJGISTsmAQCea/EcOvh2MHFERFRTeAWQiIjMS1ER8L//ab/37AlYcVdIVFp+cT5Gbx6NnKIcRHtF46Wol0wdEhHVIO71iIjIvOTnA/ffr/2elcUEkOgms/bMwvHrx9FA0wAzOs+AlQW3EaI7CW8BJSIiIiIAwPrE9fj++PcAgGmdp8HL3svEERFRTeMpHaL6JDu77H6WloCNTeXKWlgAtrYVl7W3r1p8RER0x0rKTMKkv7TP/Q2NHIpOfp1MHBER1QYmgET1iYND2f169wZ+/fW//z09gZwc42VjY4GEhP/+DwoCrl41LCdSnSiJiOgOU1BcgFGbRyGrMAutPFvh1VavmjokIqolvAWUiIiIyMzN2TsHR68dhbPGGTO7zORzf0R3MG7dRPVJVlbZ/Sxvanw3NbXsshY3ndtJTKx2SEREdGf7/dzv+PbYtwCAaZ2mwdve28QREVFtYgJIVJ9U5Zm82ipLRERm48KNC3hn+zsAgCHNhqBLwy4mjoiIahsTQCIiMi9qNbBgwX/ficxUYXEhxm4ZixuFN9DCowWGtx5u6pCIqA4wASQiIvNibQ288oqpoyAyubl/z8W/V/+Fk9oJs7rMgrWFtalDIqI6wJfAEBEREZmZhKQELD2yFAAwueNk+Dr4mjYgIqozvAJIRETmpbgY2LpV+71zZ8MXLBHd4ZKzkvHmtjcBAE+GP4luAd1MHBER1SUmgEREZF7y8oCuXbXfs7L4kiQyK4UlhRizZQwyCzIR6RaJkdEjTR0SEdUx3gJKREREZCY+3P8hDl45CEdrR8yKnQVrSz73R2RumAASERERmYGtF7biq0NfAQDe7fguGjo2NHFERGQKTACJiIiI7nAp2SmYsG0CAOCxsMdwT+A9Jo6IiEyFCSARERHRHayopAjjtoxDen46wl3DMbrNaFOHREQmxASQiIiI6A728YGP8Xfq37C3tsfs2NlQW6pNHRIRmRATQCIiIqI71F8X/8LCfxcCAOLvikeAU4CJIyIiU2MzEEREZF6srYGZM//7TnSHSs1JxRvb3oBAMKDpANwbdK+pQyKieoAJIBERmRe1GhgzxtRRENWq4pJijN86HtfyriG0QSjGthtr6pCIqJ7gLaBEVKNUKhXWrFlj6jCQkJAAlUqF9PT0MsssXrwYLi4udRYTEVFd+eyfz7AnZQ/srOwwO3Y2NJYaU4dERPUEE0AiqrTU1FS88MILCAgIgEajgbe3N3r27IkdO3YoZZKTk9GrVy8TRql11113ITk5Gc7Ozrc8rl9++QVxcXFwdHSEnZ0d2rZti8WLF1d5PPHx8WjZsuUtx2MMk9kqKC4G9uzRfoqLTR0NUY3blbwLnx78FADwTod3EOQcZNqAiKheYQJIRJXWt29fHDx4EEuWLMGJEyewdu1axMXF4dq1a0oZb29vaDSmP9OsVqvh7e0NlUp1S+P58MMP8dBDD+Guu+7Crl278M8//+DRRx/Fiy++iNGj+Sr121JeHtCunfaTl2fqaIhq1NXcqxi/dTwEgr4hfXFf8H2mDomI6hkmgET1gIggp6DIJB8RqVSM6enp2LZtG2bMmIGuXbsiMDAQ7dq1wxtvvIH77vvvAOPmW0D/+usvtGzZEjY2NmjTpg3WrFkDlUqFAwcOAPjvVs3//e9/aNWqFWxtbdGtWzekpqbit99+Q3h4OJycnPDYY48hJydHGW9+fj6GDx8OT09P2NjYoFOnTtizZ4/S39gtoIsXL0ZAQADs7Ozw8MMPIy0trdx5TkpKwqhRozBixAhMmzYNERERaNKkCUaNGoVZs2Zhzpw52LVrlzLum6/A6eZV13/SpEk4ePAgVCoVVCqVchVRpVLhk08+Qa9evWBra4tGjRphxYoV5c7LgQMHoFKpkJiYiISEBDz99NPIyMhQxh0fH1/uvBHRnae4pBhvbH0DV3OvoolLE4xrN87UIRFRPcSXwBDVA7mFxYh4538mmfaRd3vCTl1xVeDg4AAHBwesWbMG7du3r9RVvhs3buCBBx5A79698e233+LcuXMYMWKE0bLx8fFYsGAB7OzsMGDAAAwYMAAajQbffvstsrKy8PDDD+PDDz/EuHHaA5qxY8di5cqVWLJkCQIDAzFz5kz07NkTp06dgqurq8H4d+3ahWeeeQbTpk3DI488gvXr12PixInlxv/jjz+isLDQ6JW+F154ARMmTMDy5csRExNT4bIYOHAgDh06hPXr1+P3338HAL3bU99++2289957mDdvHpYtW4bHHnsMkZGRCA8Pr3Dcd911F+bOnYt33nkHx48fB6D9vYjIvCz8dyF2Ju+ErZUt5sTOga2VralDIqJ6iFcAiahSrKyssHjxYixZsgQuLi7o2LEjJkyYgH/++afMYb755huoVCp88cUXiIiIQK9evTCmjLcvTpkyBR07dkSrVq0wdOhQbN68GZ988glatWqFzp07o1+/fti0aRMAIDs7G5988glmzZqFXr16ISIiAl988QVsbW2xaNEio+OfN28eevbsifHjx6Np06YYPnw4evbsWe48nzhxAs7OzvDx8THop1arERwcjBMnTpQ7Dh1bW1s4ODjAysoK3t7e8Pb2hq3tfwdn/fv3x7PPPoumTZti8uTJaNOmDT788MNKjVutVsPZ2RkqlUoZNxNAIvOyN2UvPj74MQDgzZg3EewSbOKIiKi+4hVAonrA1toSR94tPxmpzWlXVt++fXHfffdh69at2LFjB9avX4+ZM2di4cKFGDJkiEH548ePo0WLFrCxsVG6tWvXzui4W7RooXz38vKCnZ0dgoOD9brt3r0bAHD69GkUFhaiY8eOSn9ra2u0a9cOR48eNTr+o0eP4uGHH9br1qFDB6xfv77iGS+DiNzyM4alY7n5f91tskRE5bmWdw3jtoxDiZTgwcYP4qEmD5k6JCKqx5gAEtUDKpWqUrdh1gc2Njbo3r07unfvjnfeeQfPPvssJk6caDQBNJYglfXMoXWpBrlVKpXe/7puJSUleuMwNu6yErLKPutYWtOmTZGRkYFLly7B19dXr19BQQHOnDmDbt26AQAsLCwMplFYWFjlaZammxcLC+3NGqXHf6vjJqI7Q4mUYMLWCUjNTUWwczDejHnT1CERUT3HW0CJ6JZEREQgOzvbaL+wsDD8888/yM/PV7rt3bv3lqfZpEkTqNVqbNu2TelWWFiIvXv3lvnMXEREBHbu3KnX7eb/b9a3b19YWVlhzpw5Bv0+/fRTZGdn47HHHgMAeHh44MaNG3rL4uYreGq1GsVlNDtgLLawsDBl3IC2iY3qjJuI7lxfHvoS2y9th42lDWbHzoadtZ2pQyKieu72uORARCaXlpaG/v3745lnnkGLFi3g6OiIvXv3YubMmXjoIeO3Gz3++ON488038fzzz2P8+PE4f/48Zs+eDcDw6l1V2Nvb46WXXsKYMWPg6uqKgIAAzJw5Ezk5ORg6dKjRYYYPH4677roLM2fORJ8+fbBhw4YKb//UjXf06NGwsbHBoEGDYG1tjZ9++gkTJkzAqFGjlBfAxMTEwM7ODhMmTMCwYcOwe/dug7YCg4KCcPbsWRw4cAANGzaEo6Oj8jKdFStWoE2bNujUqRO++eYb7N69W3mesUmTJvD390d8fDymTJmCkydPGiSlQUFByMrKwh9//IGoqCjY2dnBzo4HgkZZWwO6FwDddKWZ6Hby9+W/sWD/AgDAGzFvIKRBiIkjIqLbghBRrcjIyBAAkpGRYdAvNzdXjhw5Irm5uSaIrHry8vJk/Pjx0rp1a3F2dhY7OzsJDQ2Vt956S3JycpRyAGT16tXK/9u3b5cWLVqIWq2W6Oho+fbbbwWAHDt2TERENm3aJADk+vXryjBfffWVODs7601/4sSJEhUVpfyfm5srw4YNE3d3d9FoNNKxY0fZvXu30t/YeBctWiQNGzYUW1tbeeCBB2T27NkG0zHmp59+ks6dO4u9vb3Y2NhIdHS0fPnllwblVq9eLU2aNBEbGxu5//775fPPP5fS1WxeXp707dtXXFxcBIB89dVXyjL76KOPpHv37qLRaCQwMFCWL1+uN+5t27ZJ8+bNxcbGRjp37iwrVqwQAHL27FmlzIsvvihubm4CQCZOnGh0Xm7HdY+IDF3PvS53/3C3RC6OlLGbx0pJSYmpQ6pR5e1DiejWqESq8WAMEVUoMzMTzs7OyMjIgJOTk16/vLw8nD17Fo0aNdJ7QYo5+Oabb5Q260q/BdOcqVQqrF69Gn369Kn1aZnzukd0pyiREgz7cxi2XNiCIKcgfHf/d7C3tjd1WDWqvH0oEd0a3gJKRLVq6dKlCA4Ohp+fHw4ePIhx48ZhwIABTP7IdEpKAN3bYsPDAQs+Dk+3l6WHl2LLhS1QW6gxK3bWHZf8EVHtYgJIRLUqJSUF77zzDlJSUuDj44P+/ftj6tSppg6LzFluLhAZqf2elQXY8+CZbh8HrxzEvL/nAQDGtRuHMNcwE0dERLcbJoBEVKvGjh2LsWPHmjqMeo134hNRZWTkZ2DM5jEokiL0DOqJ/k37mzokIroN8b4XIiIionpORPDW9reQnJ0Mf0d/xHeIv6W3KROR+WICSERERFTPfX30ayQkJcDawhqzY2fDQe1g6pCI6DbFBJCIiIioHjt09RDe3/c+AGB0m9GIcIswcUREdDtjAkhERERUT2UWZGL05tEoKinCPQH34LGwx0wdEhHd5pgAEhEREdVDIoKJ2yfiYtZF+Dn4YVLHSXzuj4huGRNAIqo3VCoV1qxZY+ow6E5nbQ2MHq39WFubOhqiMi0/thy/n/8dVhZWmB07G05qNohORLeOCSARVYpKpSr3M2TIEJPFdvjwYQwYMAAeHh7QaDQICQnB22+/jZycnCqNJyEhASqVCunp6TUeY2JiIlQqFQ4cOFDj46YqUquBWbO0H7Xa1NEQGXUk7Qhm750NABgZPRKR7pEmjoiI7hRsB5CIKiU5OVn5/v333+Odd97B8ePHlW62tramCAs7d+7EPffcg3vuuQe//vorvLy8sHv3bowaNQp//vknNm3aBDUP8onoNpJVkIUxm8egsKQQcf5xeDL8SVOHRER3EF4BJKJK8fb2Vj7Ozs5QqVTK/9bW1njxxRfRsGFD2NnZoXnz5li+fLne8HFxcRg+fDjGjh0LV1dXeHt7Iz4+3mA6V69excMPPww7OzuEhIRg7dq1ZcYkIhg6dCjCw8OxatUqtGvXDoGBgejfvz9+/vln7NixAx988AEA41fg0tPToVKpkJCQgMTERHTt2hUA0KBBA72rmnFxcXj11Vfx6quvwsXFBW5ubnjrrbf0GnA3dvuqi4sLFi9eDABo1KgRAKBVq1ZQqVSIi4urxFKnWlFSAiQmaj8lJaaOhkiPiGDSjkk4f+M8fOx9MKXjFD73R0Q1igkgUX0gAhRkm+ZTKomprry8PERHR+OXX37BoUOH8Pzzz2PQoEHYtWuXXrklS5bA3t4eu3btwsyZM/Huu+9i48aNemUmTZqEAQMG4J9//kHv3r3xxBNP4Nq1a0ane+DAARw5cgQjR46EhYV+dRYVFYV77rnHIBEti7+/P1auXAkAOH78OJKTkzFv3jy92K2srLBr1y7Mnz8fH3zwARYuXFipcQPA7t27AQC///47kpOTsWrVqkoPSzUsNxdo1Ej7yc01dTREelacWIH1iethpbLCzC4z4axxNnVIRHSH4S2gRPVBYQ4wzdc0055wCVDb39Io/Pz8MHr0aOX/YcOGYf369VixYgViYmKU7i1atMDEiRMBACEhIViwYAH++OMPdO/eXSkzZMgQPPaY9jXn06ZNw4cffojdu3fj3nvvNZjuiRMnAADh4eFG4woPD8e2bdsqNQ+WlpZwdXUFAHh6esLFxUWvv7+/Pz744AOoVCqEhobi33//xQcffIDnnnuuUuP38PAAALi5ucHb27tSwxCReTl+7Thm7J4BABjeejhaerY0bUBEdEfiFUAiumXFxcWYOnUqWrRoATc3Nzg4OGDDhg04f/68XrkWLVro/e/j44PU1NQyy9jb28PR0dGgTGWJSI3dOtW+fXu9cXXo0AEnT55EcXFxjYyfiMxbdmE2Rm8ejYKSAnT264zBzQabOiQiukPxCiBRfWBtp70SZ6pp36I5c+bggw8+wNy5c9G8eXPY29tjxIgRKCgo0J/UTa/cV6lUKLnpGazKlNFp2rQpAODIkSNo2bKlQf9jx44hJCQEAJRbREs/t1dYWFiJuasclUqlN+6aHj8R3blEBJN3TkZiZiI87TwxtdNUWKh4jp6IagcTQKL6QKW65dswTWnr1q146KGH8OST2jfVlZSU4OTJk2XemllTWrZsibCwMHzwwQd49NFH9Z4DPHjwIH7//XdMnz4dwH+3YCYnJ6NVq1YAYNAkg+5tocau6u3cudPg/5CQEFhaWirjL/2m1JMnT+o1Q1HeuInIvK0+tRq/nvkVlipLzOoyCw1sGpg6JCK6g/H0EhHdsiZNmmDjxo3466+/cPToUbzwwgtISUmp9emqVCosXLgQR44cQd++fbF7926cP38eK1aswAMPPIAOHTpgxIgRALTNVLRv3x7vvfcejhw5gi1btuCtt97SG19gYCBUKhV++eUXXLlyBVlZWUq/pKQkjBw5EsePH8fy5cvx4Ycf4rXXXlP6d+vWDQsWLMDff/+NvXv34sUXX9S7munp6QlbW1usX78ely9fRkZGRu0uHCK6LZy8fhLTd2lPVL3a6lW09mpt4oiI6E7HBJCIbtnbb7+N1q1bo2fPnoiLi4O3tzf69OlTJ9Pu2LEjdu7cCUtLS/Tu3RtNmjTBG2+8gcGDB2Pjxo3QaDRK2S+//BKFhYVo06YNXnvtNUyZMkVvXH5+fpg0aRLGjx8PLy8vvPrqq0q/p556Crm5uWjXrh1eeeUVDBs2DM8//7zSf86cOfD390eXLl3w+OOPY/To0bCz++/2WisrK8yfPx+fffYZfH198dBDD9XiUiGi20FOYQ5Gbx6NvOI8dPTtiGcinzF1SERkBlRy80MrRFQjMjMz4ezsjIyMDDg5Oen1y8vLw9mzZ9GoUSPY2NiYKEKqrLi4OLRs2RJz5841dSi3jOsegPx8YORI7ff33wdKnSQgqktvbnsTa0+vhYetB1Y8sAJutm6mDqneKG8fSkS3hlcAySxs2bIFDzzwAHx9fY022H2zhIQEqFQqg8+xY8fqJmAiqj0aDfDRR9oPkz8ykZ9O/YS1p9fCQmWBGV1mMPkjojrDl8CQWcjOzkZUVBSefvpp9O3bt9LDHT9+XO/Mo+5FIkRERNV1Jv0Mpu6aCgB4KeoltPVua+KIiMicMAEks9CrVy/06tWrysMZaxCczE9CQoKpQ6CaJAJcvar97u6ufQsvUR3JLcrFqM2jkFuUi/Y+7fFc8+dMHRIRmRneAkpUjlatWsHHxwd33303Nm3aZOpwiKgm5OQAnp7aT6mmOojqwozdM3Aq/RTcbNwwvfN0WFpYmjokIjIzvAJIZISPjw8+//xzREdHIz8/H8uWLcPdd9+NhIQEdOnSxegw+fn5yM/PV/7PzMysq3CJiOg28OuZX7Hy5EqooMJ7Xd6Du627qUMiIjPEBJDIiNDQUISGhir/d+jQAUlJSZg9e3aZCeD06dMxadKkugqRiIhuI4kZiXh3x7sAgBeiXkB7n/YmjoiIzBVvASWqpPbt2+PkyZNl9n/jjTeQkZGhfJKSkuowOiIiqq/yi/MxevNo5BTloK13W7zY4kVTh0REZoxXAIkqaf/+/fDx8Smzv0aj0Wt0nIiICABm7p6J49ePw9XGFe91fo/P/RGRSTEBJLOQlZWFU6dOKf+fPXsWBw4cgKurKwICAvDGG2/g4sWLWLp0KQBg7ty5CAoKQrNmzVBQUICvv/4aK1euxMqVK001C0REdBtan7geP5z4ASqoML3TdHjaeZo6JCIyc0wAySzs3bsXXbt2Vf4fOXIkAGDw4MFYvHgxkpOTcf78eaV/QUEBRo8ejYsXL8LW1hbNmjXDr7/+it69e9d57EREdHtKykxC/F/xAIBnmz+Lu/zuMm1ARETgM4BkJuLi4iAiBp/FixcDABYvXqzX1tvYsWNx6tQp5Obm4tq1a9i6dSuTPwBDhgyBSqWCSqWClZUVAgIC8NJLL+H69etVGo9KpcKaNWsMuicmJkKlUuHAgQMG/fr06YMhQ4aUO97c3FxMnDgRoaGh0Gg0cHd3R79+/XD48OEqxQcAQUFBmDt3bpWHq4y4uDiMGDGiVsZNlWBlBQwerP1Y8Two1Y6C4gKM2jwK2YXZaO3ZGi+3fNnUIRERAWACSERVdO+99yI5ORmJiYlYuHAhfv75Z7z8sukPbPLz83HPPffgyy+/xOTJk3HixAmsW7cOxcXFiImJwc6dO00dItUXGg2weLH2w+d2qZbM2TsHR68dhYvGBTO6zICVBU82EFH9wASQiKpEo9HA29sbDRs2RI8ePTBw4EBs2LBBr8xXX32F8PBw2NjYICwsDB9//HGtxzV37lzs2LEDv/zyCwYMGIDAwEC0a9cOK1euRHh4OIYOHQoRAWD8ClzpK4xxcXE4d+4cXn/9deWKJ6C9Uuzi4oI1a9agadOmsLGxQffu3fXe+DpkyBD06dNHb9wjRoxAXFyc0n/z5s2YN2+eMu7ExMTaWCREZCK/n/sd3x77FgAwrdM0eNt7mzgiIqL/8HQUUT0gIsgtyjXJtG2tbJUEp6rOnDmD9evXw9raWun2xRdfYOLEiViwYAFatWqF/fv347nnnoO9vT0GDx5cU2Eb+Pbbb9G9e3dERUXpdbewsMDrr7+OJ554AgcPHkTLli0rHNeqVasQFRWF559/Hs8995xev5ycHEydOhVLliyBWq3Gyy+/jEcffRTbt2+vVJzz5s3DiRMnEBkZiXff1bYJ5uHhUbmZpJohAuTkaL/b2QHVXP+JjLlw4wLe2f4OAODpyKfRuWFnE0dERKSPCSBRPZBblIuYb2NMMu1dj++CnbVdpcv/8ssvcHBwQHFxMfLy8gAA77//vtJ/8uTJmDNnDh555BEAQKNGjXDkyBF89tlntZoAnjhxQu9FP6WFh4crZSqTALq6usLS0hKOjo7w9tY/c19YWIgFCxYgJkb7ey1ZsgTh4eHYvXs32rVrV+G4nZ2doVarYWdnZzBuqiM5OYCDg/Z7VhZgb2/aeOiOUVhciDGbx+BG4Q1EeURhWKthpg6JiMgAE0AiqpKuXbvik08+QU5ODhYuXIgTJ05g2DDtQc6VK1eQlJSEoUOH6l05KyoqgrOzs6lCVm79rO6VztKsrKzQpk0b5f+wsDC4uLjg6NGjlUoAiejO9cHfH+BQ2iE4qZ0wq8ssWFtYVzwQEVEdYwJIVA/YWtli1+O7TDbtqrC3t0eTJk0AAPPnz0fXrl0xadIkTJ48GSUlJQC0t4HqrpDpWFpW3PCxLknMyMgw6Jeeno7AwMAyh23atCmOHDlitN+xY8cAACEhIQC0t4XqkkKdwsLCCuPTMZZI6rrd6riJ6Pa06fwmLDuyDAAwtdNU+Dj4mDgiIiLj+BIYonpApVLBztrOJJ9bvSo2ceJEzJ49G5cuXYKXlxf8/Pxw5swZNGnSRO/TqFGjCsfVoEEDeHh4YM+ePXrdc3NzcfjwYYSGhpY57KOPPorff/8dBw8e1OteUlKCDz74ABEREcrzgR4eHkhOTlbKFBcX49ChQ3rDqdVqFBcXG0ynqKgIe/fuVf4/fvw40tPTERYWZnTcAAyatShr3ER0e7qUdQlvbX8LADAoYhDi/ONMGxARUTmYABLRLYmLi0OzZs0wbdo0AEB8fDymT5+uvOzk33//xVdffaX3nCAAnD17FgcOHND7ZGVlYfTo0Zg2bRqWLVuG06dPY+/evXjqqadgZWWFJ598ssw4Xn/9dbRr1w4PPPAAVqxYgfPnz2PPnj3o27cvjh49ikWLFinJbrdu3fDrr7/i119/xbFjx/Dyyy8jPT1db3xBQUHYsmULLl68iKtXryrdra2tMWzYMOzatQt///03nn76abRv3165/bNbt27Yu3cvli5dipMnT2LixIkGyWVQUBB27dqFxMREXL16VblySkS3n8KSQozdMhaZBZlo7t4cr7d+3dQhERGViwkgEd2ykSNH4osvvkBSUhKeffZZLFy4EIsXL0bz5s0RGxuLxYsXG1wBHDlyJFq1aqX32bt3L0aPHo0pU6Zg9uzZiIqKQp8+fSAi2Lp1K5ycnMqMwcbGBn/++ScGDx6MCRMmoEmTJrj33nthaWmJnTt3on379krZZ555BoMHD8ZTTz2F2NhYNGrUyOAFMu+++y4SExPRuHFjvbd02tnZYdy4cXj88cfRoUMH2Nra4rvvvlP69+zZE2+//TbGjh2Ltm3b4saNG3jqqaf0xj169GhYWloiIiICHh4eOH/+fLWWOxGZ3od/f4iDVw7C0doRM7vMhLUln/sjovpNJTc/rEJENSIzMxPOzs7IyMgwSFzy8vJw9uxZNGrUCDY2NiaKkKpq8eLFGDFihMHVwtsJ1z0A2dl8CyjViC0XtuCVP14BAHwQ9wHuCbzHxBHdOcrbhxLRreFLYIiIyLxYWgL9+v33nagaUrJT8Oa2NwEAj4c9zuSPiG4bTACJiMi82NgAK1aYOgq6jRWVFGHclnFIz09HhFsERrUZZeqQiIgqjc8AEhFV0pAhQ27r2z+JqGZ8fOBj/J36N+yt7TG7y2yoLdWmDomIqNKYABIRERFV0l8X/8LCfxcCAOLvioe/k7+JIyIiqhomgEREZF6yswGVSvvJzjZ1NHQbSc1JxRvb3oBAMKDpANwbdK+pQyIiqjImgEQmxJfwUl3jOkdUPcUlxRi/dTyu5V1DaINQjG031tQhERFVCxNAIhOwtta2E5WTk2PiSMjc6NY53TpIRJXz6T+fYk/KHthZ2WF27GxoLDWmDomIqFr4FlAiE7C0tISLiwtSU1MBaBsXV6lUJo6K7mQigpycHKSmpsLFxQWWbP6AqNJ2Ju/EZwc/AwC80+EdBDkHmTYgIqJbwASQyES8vb0BQEkCieqCi4uLsu4RUcWu5l7F+C3jIRD0DemL+4LvM3VIRES3hAkgkYmoVCr4+PjA09MThYWFpg6HzIC1tTWv/BFVge65v7S8NDRxaYJx7caZOiQiolvGBJDIxCwtLXlQTkRUD33x7xfYlbwLtla2mBM7B7ZWtqYOiYjoljEBJCIi82JpCfTu/d93IiP2pOzBJwc/AQC81f4tBLsEmzgiIqKawQSQiIjMi40N8Ouvpo6C6rG03DSM2zIOJVKChxo/hAcbP2jqkIiIagybgSAiIiL6fyVSggnbJuBK7hU0dm6MCTETTB0SEVGNYgJIRERE9P++PPQl/rr0F2wsbTA7djbsrO1MHRIRUY1iAkhEROYlOxuwt9d+srNNHQ3VI39f/hsL9i8AAEyImYAmDZqYOCIioprHZwCJiMj85OSYOgKqZ67nXceYLWNQLMW4L/g+9GnSx9QhERHVCl4BJCIiIrNWIiV4c9ubSM1JRZBTEN5u/zZUKpWpwyIiqhVMAImIiMisLTm8BFsvboXaQo3ZsbNhb21v6pCIiGoNE0AiIiIyWwdSD2De3/MAAOPajUOoa6iJIyIiql1MAImIiMgsZeRnYOyWsSiWYtwbdC/6N+1v6pCIiGodE0AiIiIyOyKCt7a/heTsZAQ4BmBih4l87o+IzALfAkpERObFwgKIjf3vO5mlr49+jYSkBFhbWGNW7Cw4qB1MHRIRUZ1gAkhERObF1hZISDB1FGRCh64ewvv73gcAjGk7BhFuESaOiIio7vDUJxEREZmNzIJMjN48GkUlRege2B2Phj5q6pCIiOoUE0AiIiIyCyKCidsn4mLWRfg5+GHSXZP43B8RmR0mgEREZF6yswEPD+0nO9vU0VAdWn5sOX4//zusLKwwJ3YOHNWOpg6JiKjO8RlAIiIyP1evmjoCqmNH0o5g9t7ZAIBR0aPQzL2ZiSMiIjINXgEkIiKiO1pWQRZGbx6NwpJCdPPvhifCnzB1SEREJsMEkIiIiO5YIoL4HfFIupEEX3tfvNvxXT73R0RmjQkgERER3bFWnFiB/yX+D1YqK8yKnQVnjbOpQyIiMikmgERERHRHOnbtGGbsngEAGBE9Ai08Wpg4IiIi02MCSERERHec7MJsjNk8BgUlBYhtGIunIp4ydUhERPUC3wJKRETmxcICaNPmv+90xxERvLvjXSRmJsLLzgtTOk7hc39ERP+PCSAREZkXW1tgzx5TR0G1aPWp1Vh3dh0sVZaYHTsbLjYupg6JiKje4KlPIiIiumOcvH4S03dNBwAMazUMLT1bmjYgIqJ6hgkgERER3RFyCnMwevNo5BXnoaNfRzwd+bSpQyIiqneYABIRkXnJyQGCgrSfnBxTR0M1aOquqTiTcQaetp6Y1mkaLFQ8zCEiuhmfASQiIvMiApw79993uiP8dOonrD29FhYqC8zoMgOuNq6mDomIqF7iqTEiIiK6rZ1OP42pu6YCAF6OehltvNuYOCIiovqLCSARERHdtnKLcjF682jkFuWivU97PNv8WVOHRERUrzEBJCIiotvWe7vfw6n0U3C3dcf0ztNhaWFp6pCIiOo1JoBERER0W/rlzC9YdXIVVFDhvc7vwd3W3dQhERHVe0wAiYiI6LaTmJGIyTsmAwBejHoRMT4xJo6IiOj2wLeAEhGReVGpgIiI/77TbSe/OB+jN49GTlEO2nm3wwstXjB1SEREtw0mgEREZF7s7IDDh00dBd2Cmbtn4vj143C1ccV7nd/jc39ERFXAW0CJiIjotrE+cT1+OPEDVFBheufp8LDzMHVIRES3FSaAREREdFs4n3ke8X/FAwCebf4s7vK9y7QBERHdhpgAEhGRecnJAZo1035yckwdDVVSQXEBRm8ejezCbLT2bI2XW75s6pCIiG5LfAaQiIjMiwhw5Mh/3+m2MHvvbBy9dhQuGhfM6DIDVhY8hCEiqg5eASSzsGXLFjzwwAPw9fWFSqXCmjVrKhxm8+bNiI6Oho2NDYKDg/Hpp5/WfqBERGRg47mNWH5sOQBgaqep8Lb3NnFERES3LyaAZBays7MRFRWFBQsWVKr82bNn0bt3b3Tu3Bn79+/HhAkTMHz4cKxcubKWIyUiotKSbiRh4vaJAICnI59Gl4ZdTBwREdHtjfdPkFno1asXevXqVenyn376KQICAjB37lwAQHh4OPbu3YvZs2ejb9++tRQlERGVVlhciLGbx+JG4Q209GiJYa2GmTokIqLbHq8AEhmxY8cO9OjRQ69bz549sXfvXhQWFhodJj8/H5mZmXofIiKqvvf3vY9DaYfgpHbCzC4zYW1hbeqQiIhue0wAiYxISUmBl5eXXjcvLy8UFRXh6tWrRoeZPn06nJ2dlY+/v39dhEpEdEf68/yf+Pro1wC0z/35OPiYOCIiojsDE0CiMqhUKr3/5f/fFnhzd5033ngDGRkZyicpKanWYySialCpgMBA7aeM7ZlM61LWJby9/W0AwFMRTyHOP860ARER3UH4DCCREd7e3khJSdHrlpqaCisrK7i5uRkdRqPRQKPR1EV4RHQr7OyAxERTR0FlKCwpxJgtY5BZkInm7s0xovUIU4dERHRH4RVAIiM6dOiAjRs36nXbsGED2rRpA2trPoNCRFRb5v89H/9c+QeO1o7a5/4sWecSEdUkXgEks5CVlYVTp04p/589exYHDhyAq6srAgIC8MYbb+DixYtYunQpAODFF1/EggULMHLkSDz33HPYsWMHFi1ahOXLl5tqFoiI7nhbLmzB4sOLAQCTO05GQ8eGpg2IDBUXAlmXgRsp2k9WCnDjMuDgCbR7ztTREVElMAEks7B371507dpV+X/kyJEAgMGDB2Px4sVITk7G+fPnlf6NGjXCunXr8Prrr+Ojjz6Cr68v5s+fzyYgiO4EublAl/9vS27LFsDW1rTxEAAgJTsFE7ZNAAA8HvY47g6828QRmaGSYuBGMpBxQf+TeQnIvKjtl5UKQAyH9WvDBJDoNqES3ZstiKhGZWZmwtnZGRkZGXBycjJ1OESkk50NODhov2dlAfb2po2HUFhSiKH/G4r9qfsR4RaBZb2WQW2pNnVYdx4RIOcacP0scD0RuPb/f9PPARlJ2mSvpKji8VhYAQ7egKPXf389woCYF2osVO5DiWoPrwASERGRSX20/yPsT90PB2sHzO4ym8nfrcrPAtJOAVdPAmkngbTTwLXTQNoZID+j/GEtrAEnX8DZH3D2A5z8tH8dfbXdnXwBO3fAgq+RILpdMQEkIiIik9l2cRsWHVoEAIi/Kx7+TmxDtdLyMoErx4HUI8CVY0DqUW3Sl3mh/OGc/IAGQf99XAIBlwDtx9EbsLCsg+CJyFSYABIREZFJXM6+jAlbtc/9DQwdiJ5BPU0cUT0lor1tM/kf4PJh4PIh7Sf9fNnD2HsA7k0Bt8aAWxPAtbH2e4MgwJrPvRKZMyaAREREVOeKSoowbus4XM+/jjDXMIxpO8bUIdUPumTv4t/Apf1A8kFt4lfWrZuOPtrn7zwjAM8wwD0UcA8B7FzrNm4ium0wASQiIqI698nBT7Dv8j7YWdlhduxsaCw1pg7JNPIygAt7gaTdwMV92k/uNcNylmptkufdHPCKBLwjtf8z0SOiKmICSERE5sfd3dQRmLUdl3bgi3++AABM7DARgU6BJo6oDmVcBM79BZz/Czi/S/v83s3NKliqtYmebyvApyXg21J7lc/S2gQBE9GdhgkgERGZF3t74MoVU0dhtq7kXMH4reMhEPQN6Yvewb1NHVLtyrwEnNkMJG4Fzm3XNrtwswZBQMN2gH87wK+19gqflZleESWiWscEkIiIiOpEcUkx3tj6Bq7lXUNIgxCMbzfe1CHVvPwbwNktwOlNwNnNwNUT+v1VFoBPFBDYEQhor038HL1MEysRmSUmgERERFQnPv/3c+xK2QVbK1vMjp0NGysbU4d060S0t3Ge3Aic+h04vxMoKSxVQKW9lbNRFyCos/Yqnw0bNici02ECSERE5iU3F+jVS/v9t98AW74Svy7sSdmDTw9+CgB4q/1bCHYONnFEt6CoADi3DTj+G3B8PZBxU3MMDRoBTe4GguOAoE6AbQOThFkbiopLcDE9F2evZiPxajYS03KQmJaNIDd7xD/YzNThEVElMAEkIiLzUlICbN7833eqdWm5aRi3ZRxKpAQPNX4IDzZ+0NQhVV1BDnBqI3BkLXDif0DBjf/6Wdlqr/A1uUeb+Lk1Nl2cNUBEcCUrH2euZOPMlWycvZqFs1ezceZqNpKu5aCwWAyGCfPOM0GkRFQdTACJiIio1pRICSZsm4AruVfQ2LkxJsRMMHVIlVeQA5z4DTi8Rnt7Z2HOf/0cvICmPYHQ3kCjWEBtZ7Iwq6ugqATn0rJx+koWTl/JxunULJy+koUzV7JxI7+ozOE0VhYIcrNHoJsdGrnbI9DNHk08HeowciK6FUwAiYiIqNYs+ncR/rr0F2wsbTArdhbsrOt5olRcqH2By78rgGO/AoXZ//VzCQQiHgTCHwL8ogELC9PFWQV5hcU4fSULJy9n4WTqDZy8nIVTV7JwPi0HRSWGV/MAwEIFNGxgh2APezRyt0ewhwOC3e0R5G4PHycbWFio6nguiKimMAEkIiKiWrHv8j4sOLAAADAhZgJCGoSYOKIyiAAp/wAHvtUmfjlp//VzCQQi+wIRD2nf3qmqv4lPflExzlzJxonLN/7/k4WTl2/g/LUclJHnwV5tiSaeDmjs4YDGng5o7KFN9gLd7KCxsqzbGSCiOsEEkIiIiGrc9bzrGLtlLEqkBPcH348+TfqYOiRD2WnAP99pE7/Lh/7rbu8JRD4CRPYDGrapd0mfiODC9VwcS7mBY8mZOHb5Bk6k3MCZq9koLiPTc7a1RlMvBzTxdESIpwOaeDogxMsB3k42UNWz+SOi2sUEkIiIiGqU7rm/1JxUBDkF4e32b9efJENE2yD73q+Ao2uB4gJtd0s1EHYf0PIJILgrYFk/DpFyC4pxLCUTR5Nv4GhyJo4mZ+J4yo0yn9FztLFCqJcjQrwcEerlgKb//93dQV1/fgMiMqn6UbsRERHVJbt6/hzabW7x4cXYdnEbNJYazI6dXT+e+8vL1F7p27MQSDv5X3efKKD1U0CzRwA7V9PFB+BqVj4OX8rEkUuZOHwpA0eSM5F4Ndvo7ZvWlio09nBAmLcjQr2d/v+vI3yceUWPiMrHBJCIiMyLvT2QnV1xOaqWA6kHMP/v+QCAce3GIdQ11LQBXT0J7P5cm/wVZGm7qR2A5v2A6CHaRtrrmIggOSMPhy5m4NClTBy+mIFDlzJwOTPfaHl3BzXCfZwQ4eOEcB8nhPk4ItjdAWqr2+MlNERUvzABJCIiohqRnpeOMVvGoFiK0SuoF/qF9DNNICJA4lZg+3xt2306HmFAu+eBFgMAjWMdhSK4nJmPfy6k49+LGdrPhQykZRcYlFWpgEZu9ojwddJ+fLR/PR1t6iRWIjIPTACJiIjolokI3t7+NlKyUxDgGIB3OrxT97ciFhcBR3/SJn7JB/6/owoI7QXEvKBtr6+WY7qeXYCDF9Lxz4UM/HMhHQcvZODKDcMre5YWKoR4OiDSzxmRvk5o5ueMCB8n2Gt4aEZEtYu1DBERmZe8PKBvX+33lSsBG15dqQnLjixDwoUEWFtYY3bsbDio67Bh8KJ87S2e2z4A0s9pu1nZAq2eBDq8DLgG18pk8wqLcfhSJg4kpeNgUjoOXkjHubQcg3K6ZK9FQ2c093NGpJ8zwn2cYGPNZhaIqO4xASQiIvNSXAysW/ffd7pl/175Fx/8/QEAYGzbsQh3C6+bCRfmAn8vA7bPBTIvarvZuWlv82z7HGDvVmOTEhGcv5aD/efT8ff56ziQlI6jyZkoLDZ8Q0uwuz1aNHRGi4YuiPJ3RoSPM2zVTPaIqH5gAkhERETVllmQiTFbxqCopAjdA7tjYOjA2p9oUT6wbzGwdQ6QdVnbzdEH6Pga0HowoL71t47mFRbjnwsZ2HfuOvadu44DSddxNcvwuT13BzVa+rugVUADRDV0QfOGznC2tb7l6RMR1RYmgERERFQtIoJ3tr+Di1kX0dChISbdNal2n/srLgIOLgc2zwAykrTdnBoCnUYArQYB1tW/nTc1Mw97z13H3sTr2Hf+Og5fzEDRTe0vWFuq0MzXGa0DGqBVgAta+rugYQNbNrtARLcVJoBERERULd8e+xZ/nP8DVhZWmB07G47qWnqzpghwZA3w5xQg7ZS2m6MP0GU00OopwEpdxdEJTl/Jwu6z17E38Rr2nLuGpGu5BuU8HTVoE9Tg/xO+Bmjmy+f2iOj2xwSQiIiIquxw2mHM2TsHADC6zWg0c29WOxM6vwvY8BZwYbf2f1tXoPNIoO2zgLVtpUZRVFyCI8mZ2H32GnafvYa9567j2k3NMFiogDBvJ7QJaoDoQO3Hz4VX94jozsMEkIiIiKrkRsENjE4YjcKSQtwdcDceD3u85idy7Qzwezxw5Cft/9Z2wF3DgQ6vADZO5Q5aUFSCfy6kY9fZa9h19hr2JV5DdoH+C39srC3Q0t8F7YJc0SbIFa0CXOBow2f3iOjOxwSQiIiIKk1EMGnHJFzIugA/B7+af+4vP0v7cpcdC4DiAkBloW3OIW4C4ORjdJCCohIcvJCOnafTsPNsGvadu468whK9Mo42VmgX5Iq2jVzRrpErIn2dobayqLm4iYhuE0wAiYjIvNjba58po2pZcWIF/pf4P1iprDCzy0w4a5xrZsQiwKGVwIa3gRuXtN0adwN6TAG89G8vLSwuwT8XMrDzTBp2nE7D3nPXDBI+V3s12gW5IibYFTGN3BDq7QhLC97OSUTEBJCIiIgq5di1Y5ixewYAYET0CLTwaFEzI049Bvw6Eji3Xfu/SyBw73QgtDegUqGkRHAkORM7Tqfhr9NXsfus4S2dbvZqtA92Q0ywK9oHu6GJhwMsmPARERlgAkhEREQVyi7MxujNo1FQUoDYhrF4KuKpWx9pYS6wZTawfR5QUghY2QKdR0HuehXnM0uwbfd5/HVKm/RdzynUG9TFzhrtG7mhQ2PtJ8TTgS9sISKqBCaARERkXvLygEGDtN+XLQNsqt92nLkQEby7412cyzwHb3tvTOk45daTrdN/Ar+MBK6fBQAUNLkXW5qMwR+X1Nj6/g5cuK7fLIO92hLtGrmiYxN3dGjshnBvJ17hIyKqBiaARERkXoqLgR9/1H5fvNikodwuVp1chXVn18FSZYlZXWbBxcal+iPLvQ6snwAc/BYAcEPtgfnq57HwcATk0GWlmLWlCq38G6BjE3d0bOKGKH8XWFvypS1ERLeKCSARERGV6cT1E5i+ezoAYFirYWjp2bJa4xERJO9ZDZffx8Ku4ApKRIUlxT0wJ68/smAHAAj1ckSnEHd0auKOdo1cYa/hYQoRUU1jzUpERERG5RTmYPTm0cgvzkcnv054OvLpKg1/I68Q20+lYfeR02h3bAbuLdkMADhd4oMxhS/gvH1zdA9xR+f/T/o8nXg7LhFRbWMCSEREREZN3TUVZzPOwtPWE1M7TYWFqvxbMEW0b+tMOH4Fm09cwd/nrqMDDmK29afwUqWjWFT4zbEfklu/jilh/gj3ceSLW4iI6hgTQCIiIjKw5tQarD29FhYqC8zoMgOuNq5Gy2XmFWL7yavYdDwVm09cweXMfACABgV402o5nrb6HwAg27ERLB/5BPc36lBn80BERIaYABIREZGe0+mnMW3XNADAKy1fQRvvNko/EcGp1Cz8eSwVm46nYm/idRSViNLf1toSA/zTMSJzJhpkndZ2bPsc7Lu/C6jt6nQ+iIjIEBNAMqn09HS4uLiYOgwiIvp/uUW5GL15NHKLctHBpwOebf4s8gqLseNMGjYdS8Wfx1INmmgIdrdHXKgnuoa6o/3VlbD+4x2guACw9wT6fAyEdDfR3BAR0c2YAFKdmTFjBoKCgjBw4EAAwIABA7By5Up4e3tj3bp1iIqKMnGERGQW7OyArKz/vpOe6bum41T6Kbhq3NDW4RU8v/RvbD91FbmFxUoZtZUFOgS7oWuoB+JCPRHkbq9t3uGnV4Fjv2gLhfYGHvwQsHc30ZwQEZExTACpznz22Wf4+uuvAQAbN27Exo0b8dtvv+GHH37AmDFjsGHDBhNHSERmQaUC7O1NHUW9IyL4dN8KrD61GhAVLpx4BFMPXFD6ezvZoFu4J7qFeuKuJm6wU5c6hEjaDfw4FMg4D1iqgR5TgHbPa5c1ERHVK0wAqc4kJyfD398fAPDLL79gwIAB6NGjB4KCghATE2Pi6IiIzE9eYTF2nE7DxqOX8fvJf5HjMQcqCyD/ajeU5DZGqwAX3B3miW5hXsbf2CkC7PwE2Pg2UFIENGgE9P8K8G1lmhkiIqIKMQGkOtOgQQMkJSXB398f69evx5QpUwBozzoXFxdXMDQRUQ3JzwdeeEH7/bPPAI3GtPHUsWvZBfjj6GX8fvQytpz4/1s7VYWwC/oKlhYFcEY4Xo0dgbvDfODhWM6yyc8C1g4DDq/S/t/sEeCBeYCNU93MCBERVQsTQKozjzzyCB5//HGEhIQgLS0NvXr1AgAcOHAATZo0MXF0RGQ2ioqAJUu03z/6yCwSwMSr2dh45DI2HrmMveeuodRLO+HtZAP3RhtwriAFrjau+PGBj+Bh51H+CK+eAr5/ErhyFLCwAnpO4y2fRES3CSaAVGc++OADBAUFISkpCTNnzoSDgwMA7a2hL7/8somjIyK6c4gI/rmQgQ1HUrDxyGWcuJyl1z/CxwndI7zQPcIL5/O3Y9zWP6GCCtM7T684+Tu2Dlj1PFBwA3DwBgYsAQLa1+LcEBFRTVKJiFRcjIiqKjMzE87OzsjIyICTE2+JIqo3srOB/z8BhaysO+aFMIXFJdh15ho2HEnBhsOXkZKZp/SzslAhJtgV3cO9cE+EFxo20L799FzmOQz4eQByinLwXPPnMLz18LInIAJsnQ38qb19H4EdgX5fAY5etTlbZKa4DyWqPbwCSLVq7dq16NWrF6ytrbF27dpyyz744IN1FBUR0Z0ht6AYm09cwYbDKfjjWCoycguVfvZqS8SFeqJ7hBe6hnrC2c5ab9j84nyM3jwaOUU5aO3ZGi+3LOdOjIIcYO2rwKGV2v/bPa+97dPSuuxhiIioXmICSLWqT58+SElJgaenJ/r06VNmOZVKxRfBEBFVQkZOIf44dhn/O5yCzSeuIK+wROnnZq9G9wgv9GzmjQ6N3WBjbVnmeGbtmYVj146hgaYBZnaZCSuLMg4JMi4C3z0OJB/QPu933xwgekjNzhQREdUZJoBUq0pKSox+JyKiyku9kYeNRy5j/aEU7DidhqJSb3Fp2MAWPZt5495Ib7QOaABLi4pfxLIhcQO+P/49AGBa52nwsi/jNs5LB4BvBwJZKYCdGzBgGRDUsSZmiYiITIQJINULOTk5sLOzM3UYRET1xoXrOfjf4ctYfygZe89dR+kn9kO9HNGzmRd6NPNGM18nw/b5ypGUmYSJf03E/7V35/FR1Pcfx1+b+06AnCCQgNyHyiWoCHgEEClaRSiI4NWipRatt78W1CqCVK0iItWiAlVUxKKAgkpAyyFokPtQCOFISAIhN7l2fn8MJESSQCC7s5t9Px+PfeS7M9/Z/UyGsHnnOzNfgLs638VVza6qvuOuL+Dju6C0AKI7wu8+gEYtL2SXRETEBSgAitP079+fefPmcdFFF1VZvn79esaMGcPu3bstqkxEPEpQEGRkVLZdyL6sApZtTeOLrelsPphTZd0lzSMY2CmGQZ1iaRUVcl6vX1JewsOrHya/NJ9Loy5lwmUTqu/4/b9g2aNg2KH1NTD8Xc3vJyLSQCgAitOEhYXRtWtXZs6cyciRI7Hb7TzzzDNMmTKFP/3pT1aXJyKewmaDqLNMdeBEe47ksXRLOsu2prEzPa9iuZcNesQ3ZnDnWAZ2iqVpROAFv9dLP7zE9qPbCfcP58V+L+Lr9aubuNjtsOKvsHaG+bzbHTDkJd3sRUSkAVEAFKdZvHgxs2bN4p577mHx4sWkpKSQmprKkiVLuO6666wuT0TEKQzDYEdaHsu2prF0Sxq/ZBZUrPPxstGndRMGd44jsVMMkSH1N0n91/u/Zv6O+QD8/cq/ExscW7VDWTEsGg/bPjGfXzsJrnpQk7uLiDQwCoDiVOPHj2f//v1MnToVHx8fkpKSuOKKK6wuS0Q8SXExPPSQ2X7pJfCvv5BVE8Mw2Hool6Vb01i2JY2Uo4UV6/y8vejbJpJBnWO5vmMMEUF+9f7+h/IP8dc1fwVgbMex9G/ev2qH4jz4YDTsWwVevnDzLOhya73XISIi1lMAFKfJzs7mnnvu4euvv+bNN99k1apVJCYmMm3aNO6/v5b5p0RE6lNZGcycabanTXNYADQMg58O5rB0iznSdzC7qGKdv48X/dtFcUOXOK5pH01ogONOsSwtL+XRVY+SV5JH18iu/Lnbn6t2yM+A+bdC2k/gFwIj5prX/YmISIOkAChO07lzZxISEkhOTiYhIYF7772XBQsWcP/997NkyRKWLFlidYkiIhfEMAySDxxn6eY0lm1N59DxytAX4OvFNe2juaFLHAPaRRPs75yP4H/++E82Z20m1C+Uaf2m4Xv69XzH9sHcmyF7HwRFwuiPoFk3p9QlIiLWUAAUpxk/fjxPPfUUXl5eFctGjBjBlVdeyZ133mlhZSIi589uPxn6tpindx7OOVGxLsjPm2s7xHBD51j6tYsiyM+5H7urDqzi3e3vAvDslc/SLKRZ5crMXfDeMMhLg4gWMOZTaNLaqfWJiIjz2Qzj9JmFRKS+5ObmEh4eTk5ODmFhun26iMsoKICQk9Mo5OdDcHCdX+JU6FuyOY1lW9NIOy30Bft5c13HGAZ3jqN/uygCfL3rq/I6SS9I59bPbiWnOIfRHUbzeK/HK1em/WSO/BUehaj2ZvgLi7OkTpHq6DNUxHE0AihOV1hYSGpqKiUlJVWWd+3a1aKKRETOrrbQF+Lvw3UdzNM7r25rXeg7pdReyiOrHiGnOIeOTTryUPeHKlemrof5w6E4B+Iuhds/geAmltUqIiLOpQAoTpOZmcmdd97JsmXLql1fXl7u5IpERGrnTqHvdDOSZ7ApcxMhviFM7zcdP++TdxbduwreHwmlhdCiD4xaAAHh1hYrIiJOpQAoTjNx4kSys7NZt24dAwYMYNGiRRw5coS///3v/OMf/3D4+8+cOZMXX3yRtLQ0OnXqxCuvvELfvn2r7ZuUlMSAAQPOWL5jxw7at2/v6FJFxEKnbuSyZPOZ1/S5cug75btD3/Hvrf8G4OkrnqZ5aHNzxc9fwwejoOyEeZfPEfPBL8jCSkVExAoKgOI033zzDf/973/p2bMnXl5etGzZkuuvv56wsDCmTJnCkCFDHPbeCxYsYOLEicycOZMrr7ySN998k8GDB7N9+3ZatGhR43a7du2qcu1BVFSUw2oUEScJDIR9+yrbmKFv08kbuSzdUvXunaeu6RviwqHvlCMFR3jy2ycBGNFuBInxieaKPV+Z4a+8GNoOhtveBR/Hz38oIiKuRwFQnKagoIDo6GgAGjduTGZmJm3btqVLly78+OOPDn3vl156ibvvvpt77rkHgFdeeYUvv/ySN954gylTptS4XXR0NBEREQ6tTUSczMsL4uMxDIPNB3NYsiWNJZvTqg19N3SJo5+Lh75TyuxlPPbtY2QXZ9O+cXse6fmIuWL3clgwGspLoP2NcOsc8Kn/yeZFRMQ9KACK07Rr145du3YRHx/PpZdeyptvvkl8fDyzZs0iLs5xd58rKSnhhx9+4PHHH6+yPDExkTVr1tS67WWXXcaJEyfo2LEj//d//1ftaaGnFBcXU1xcXPE8Nzf3wgoXkXpnGAZbD+Xy+ZbDLNlcdXL2U1M2DOli7d07z9cbP73BD0d+IMgniOn9puPv7Q+7voAPx5jhr8NQM/x5O27SeRERcX0KgOI0EydOJC0tDYBJkyYxcOBA5s+fj5+fH++8847D3jcrK4vy8nJiYmKqLI+JiSE9Pb3abeLi4pg9ezbdu3enuLiYuXPncu2115KUlMTVV19d7TZTpkzh6aefrvf6ReTCGIbBtsO5fL45jaVb0kg9VlixLsjPm2vaR3Nj1zj6t4t2u9B3ytrDa/nX5n8BMKnPJFqGtYQ9K2DB7WAvhY43wS1vKfyJiIjmARTrFBYWsnPnTlq0aEFkZKTD3ufw4cM0a9aMNWvW0KdPn4rlzz33HHPnzmXnzp3n9DpDhw7FZrOxePHiatdXNwLYvHlzzWEkYoFToW/JFjP07T9aGfoCfb24ZnMSN+78jv7rlxEY4d4/n5mFmdz62a0cO3GMW9rcwuQrJsMvK+E/I8xr/jreBLe8Dd76m6+4D80DKOI4+jQQywQFBdGtWzeHv09kZCTe3t5njPZlZGScMSpYm969ezNv3rwa1/v7++Pvr5sqiFjFMAy2p+Wy9OQ1fSmnhb4AXy+ubR/DkK5x9L8omKDGN5gr3HTE75RyezmPf/s4x04co02jNuZk7yn/g/d/Z4a/9jeeHPnTx72IiJj0iSANnp+fH927d2fFihXcfPPNFctXrFjBsGHDzvl1kpOTHXqtoojU3emhb+mWdPZlFVSs8/fx4pr25pQN13aIJsjv5EdeQUENr+Z+Zm+ezffp3xPoE8j0ftMJSNsM/7kNyoqgTSLc+m+d9ikiIlUoAIpHeOihhxgzZgw9evSgT58+zJ49m9TUVMaPHw/AE088waFDh3jvvfcA8y6h8fHxdOrUiZKSEubNm8fChQtZuHChlbshIpihb0daHku2HK429PVvF8WQrk25tn00wf4N92Pu+7TveeOnNwD4a++/0qogF+bdAiX50Ko/3DZXUz2IiMgZGu4no8hpRowYwdGjR3nmmWdIS0ujc+fOLF26lJYtWwKQlpZGampqRf+SkhIefvhhDh06RGBgIJ06dWLJkiXccMMNVu2CiEc720hf/3ZRJ0f6YghpwKHvlKyiLB779jEMDG66+CaGhrWFOYOhOBdaXgkj/wO+AVaXKSIiLkg3gRFxEF3ALnJhTt3IZenJG7mcfk3fBYW+ggIICTHb+fkQHFzPlTuW3bAzfsV41qatpXV4a/5z5VSC3rsJ8g5D08vgjsUQoP9zxL3pM1TEcRr+n0nFJXXp0oWlS5fSvHlzq0sRERdiGAZbDuWwdEs6y7ZWvXunJ470VeetLW+xNm0tAd4BTO/5BEHvjzTDX1R7GL1Q4U9ERGrlmZ+eYrmUlBRKS0utLkNEXIBhGPx0MIdlW9JYujWNA8cqJ2f39/FiQLtobugaxzXto+sn9AUGwtatlW03sjF9I69veh2AJ7tN5OL/PgjH9kJECxizCIKbWFyhiIi4OgVAERFxOrvdIPnAcZZuSeOLrekcOl4Z+gJ8K+/eOaCdA27k4uUFnTrV72s6wbETx3hs9WPYDTtD4wdz09p3IWMbhMTAHf+FsKZWlygiIm5AAVAs0bdvXwLd7C/vInJhyu0GG1OOsWyreXrnkdziinVBft4MaB/NkC5x9G8XVTllgwDmdX9PffcUGUUZJIQl8H+HUrAdWA/+4ebIX+NWVpcoIiJuQp+wYomlS5daXYKIOEFpuZ11e4+ybGs6y7elk5VfUrEuxN+H6zpEM7hLHP3aRhHgrEnZS0rg+efN9pNPgp+fc973Aryz7R2+O/Qd/t7+TKcJQbs/AW9/+N37EON+o5kiImIdBUAREalXxWXlfLcni2Vb0/lqxxGOF1Ze7xsW4ENip1gGd47lqjaR+Ps4KfSdrrQUnn7abD/yiMsHwE0Zm3j1x1cBeDysK21/XAg2L3OS9/grLa5ORETcjQKgiIhcsILiMlbtzmTZ1nRW7swgv7isYl2TYD8SO8UwqHMcfVo1wc/Hy8JK3cvxE8d5ZPUjlBvlDA5ryy0/LjRXDHkJOtxobXEiIuKWFABFROS85BSW8tWOI3yxLZ3VuzMpLrNXrIsJ82dQp1gGdY6jV0JjvL1sFlbqngzD4K//+yvpBem0DIhk0paV2AD6PwE97rS6PBERcVMKgCIics6O5J5g+fYjLN+WztpfjlJmNyrWtWgcxODOsQzsHMulF0XgpdB3Qd7b/h5JB5Pw8/Jhespugu3l0O0O6PeY1aWJiIgbUwAUpxk3bhx33XUXV199tdWliEgd7M3M58ttR/hyWzqbDhyvsq59bCiJnWIZ2CmGjnFh2GwKffVhS+YWXvnhFQAezSmkfVE+XHydeeqnvsciInIBFADFafLy8khMTKR58+bceeedjB07lmbNmlldloj8it1usOVQDsu3p/PltiP8nJFfZf1lLSIY2CmWgZ1iSYgMtqjKhiunOIeHVz1MmVFGYqkXt2WlQ2wXGP4OePtaXZ6IiLg5BUBxmoULF3L06FHmzZvHO++8w6RJk7juuuu4++67GTZsGL6++sVGxColZeZ0Dcu3p/PV9gzSc09UrPPxstGndRMSO8WS2DGGmLAACytt2AzD4G//+xuHCw5zkeHN5EMp2MIuglEfgX+o1eWJiEgDYDMMwzh7N5H6l5yczL///W/eeustQkJCuP3227n//vtp06aN1aXVi9zcXMLDw8nJySEsLMzqckTOkFNUStKuDL7akUHSzgzyTrtzZ7CfN/3bRZPYKYb+7aIJD2xAf6ApL4cffzTb3bqBtwVTUdRg/o75vPD9C/hiY+6hw3SyBcJdX0JMR6tLE3EqfYaKOI5GAMUSaWlpLF++nOXLl+Pt7c0NN9zAtm3b6NixI9OmTePBBx+0ukSRBunAsUK+2nGEr3YcYf3eY1Vu4hIV6s91HWJI7BhDn9ZNnDcxu7N5e0PPnlZXcYZtWduYvnE6AH85epROpXa4/T2FPxERqVcKgOI0paWlLF68mDlz5rB8+XK6du3Kgw8+yOjRowkNNU9t+uCDD7jvvvsUAEXqid1usOngcb7ecYSvtmew60helfVtokO4rqMZ+i7RnTstk1eSZ173Zy/juoJCRuXmw42vQOsBVpcmIiINjAKgOE1cXBx2u53f/e53fP/991x66aVn9Bk4cCARERFOr02kIckvLuO7PZl8vSODlbsyyMovqVjn7WWjR8tGXN8xhus6xBDviTdxKSmBf/7TbP/5z+DnZ2k5hmEwac0kDuYfpFlZOU9nHcXWZ4Lm+hMREYfQNYDiNHPnzmX48OEEBHjGDSR0/YI4U+rRQr7ZeYSvd2awfu8xSsorJ2UP9fehX7sorusQQ/92UUQEWRt4LFdQACEhZjs/H4KtDcELdi7g7+v/jo9h8N7hI3RJuA5GzAOvBnoKrsg50GeoiONoBFCcZsyYMVaXINJglJbb2ZiSzcpdGXyzM+OMqRpaNgni2vYxXNshmp7xjfHz8bKoUqnNzmM7mbZhGgATjx2nS+N28Nt/KfyJiIjDKACKiLiJjLwTJO3KJGlXBt/uzqpy185Tp3Ze0z6aazvE0DoqWJOyu7iC0gIeTvoLJfYS+hcUckd5IPzuA/APsbo0ERFpwBQARURcVFm5neQDx0nalUHSrky2Hc6tsr5JsB/92kZxTYdo+raJalhTNTRwhmHw9Nqn2Z+XSmxZGX/Pzsd2x+cQfpHVpYmISAOnACgi4kLSc06wencmq3Zn8u2eTHJPlFVZ36VZOAPaR3NN+2i6NgvXXTvd1MI9C1m2bxnehsGLGVmE3/gaNHe9qSlERKThUQAUEbHQidJyNqZks3pPJqt3Z7Izveo0DRFBvlzdJor+7aLo2yaKqFB/iyqV+rLr2C5eWP88AA9kH+fSHvfBJSMtrkpERDyFAqCIiBMZhsGejHy+3ZPF6t2ZrN93lBOllXfstNngkosi6Nc2in7torjkogi8NcrXYBSWFvLwygcptpdyVWER46L6wLWTrC5LREQ8iAKgiIiDZeYVs+aXLL7dk8W3ezI5kltcZX1MmD9920Rxddso+l4cSaNgD5+mwdECAmDlysq2kxiGwd/XPkNK/gGiy8p43h6B161v646fIiLiVAqAIiL1rLCkjO/3HeO7PVl893PWGad1+vt40SuhMX3bRHJ12yjaxYTqjp3O5O0N/fs7/W0//flTPtu3BG/DYNrxEzQa+xkEaH4zERFxLgVAEZELVFJm56eDx/nfz1ms+fkoyQeyKS03qvTpGBdG3zaR9G0TRY/4RgT4atTHk/yc/TPPr30GgD9m59B96NsQebHFVYmIiCdSABQRqaNyu8H2w7ms+SWLNb8cZUPKMQpLyqv0aRYRyJUXN+GqNlFc2boJTUJ08xaXUVoKs2eb7d//HnwdO31GYWkhD391PyeMMq4oLOLu7hOhbaJD31NERKQmCoAiImdhtxtsT8tl3d6jrNt7jO/3HT1jeoZGQb5c0TqSKy5uwlUXR9KicZBO63RVJSUwYYLZHjfO4QFwynd/5ZfCNCLLynm+yeV49X3Ioe8nIiJSGwVAEZFfOTXCt36fGfg2pBwjp6i0Sp9Qfx8ub9WEPq2bcEXrJrSLCdWcfHKGz3Z/wqepy/EyDKaVhtDkptnmrV5FREQsogAoIh6vpMzOlkM5fL/PHN3bmJJNXnHVEb5gP296JjSmz8nQ16lpuKZnkFrtzdnLsyev+xufX0zPUYvBL9jiqkRExNMpAIqIxykoLuPH1Gw27DvGhpRskg9kV5mLD8wRvh7xjejdqgmXt2pCp6Zh+Hp7WVSxuJsTZSd4+Iu7KaKcy4tO8PvEGdC4ldVliYiIKACKSMN3JPcEG1Oy2ZByjB/2Z7M9LZdye9W7dDYK8qVXQmN6JTShV3xjOjYN0wifnLcXkh5mz4ksGpeX80KbMXi3HWh1SSIiIoACoIg0MGXldnYdyePH/dn8sD+bjfuzOZhddEa/ZhGB9EpoTI/4RvSKb0zrqBBdwyf1YumOBSw8tAqbYfCC/8VEDvg/q0sSERGpoAAoIm7taH4xmw4c58fUbJJTj/PTgeMU/GpKBi8btI8No2d8I7rHN6ZHy0Y0jQi0qGJpyFKO7+Xp758D4N4Sb/qMnAteOnVYRERchwKgiLiN4rJydqTlsSk1m+QDx9l04Dj7jxae0S/U34fLWjaiW4sIurdsxKXNIwgNcOyt/sWN+PvD559XtutJcXkxDy+9g0IMuheXct9vPoaA8Hp7fRERkfqgACgiLsluN9h3tICfDpijepsO5rDjcC4l5fYz+l4cHcJlzSPo1rIRl7WIoE10qK7fk5r5+MCQIfX+si8u/xO7SnNoVF7O1O6P4hPbud7fQ0RE5EIpAIqI5QzD4GB2EZsP5rD50HE2H8hh66GcM6ZiAPNmLZc2j+DS5mbYu+SiCMKDNLon1lq+ZS4LMtYC8Hzj3sR0v8viikRERKqnACgiTmUYBgeOFbH1cA5bDplBb8uhHI4Xlp7R19/Hi05Nw7i0eSMuaR7OZc0b0bxxIDZNpC0XorQU5s8326NHg++F/QHhwLE9TPphGtjgrvJgrrpxVj0UKSIi4hgKgCLiMGXldvZmFbD9cC7bDuew9ZD5NffEmSN7vt422sWG0vWiCC65KJwuzSJoGxOCj+bek/pWUgJ33mm2hw+/oABYUl7Cw0vvIN8Gl5aUM2H4h+DjV0+FioiI1D8FQBGpF/nFZexMy2VHWi7b03LZnpbHzrRcisvOvGbPz9uLdrGhdGoaRpeLwunaLIK2sSH4+3hbULnI+Xtp2e/ZXp5PeHk5L141Bd+IFlaXJCIiUisFQBGpk3K7QeqxQnal57IjLY8dabnsTM8j9diZd+MECPbzpkNcGJ2ahtGpaTidmoXRJjoUPx+N7Il7+3rT28w/+gMAzzVNJLbDMIsrEhEROTsFQBGplmEYZOQVs/tIHrvSTz6O5LH7SB4nSs8c1QOIDQugQ1woHZuG0SEujI5xYcQ3CdYE69LgHDq6k78mvwJeMNbWmH6JL1ldkoiIyDlRABTxcIZhkJlXzJ6MfHYfyWNPRj57juSx+0g+OUVn3pgFzJuztI0JpX1sKO3jwuhw8mvjYF37JA1faVkJjyy5gzwv6FoGfx7xkSZ7FxERt6EAKOIhyu0Gh7KL+CUzn58z8tmTkcfPGWa7upuyAHjZID4ymHYxoRWBr11sKC2bBGuePfFY/1xyJ1uMIkLtdqb1ewnfkGirSxIRETlnCoAiDUzuiVL2ZhawNzPf/Jp16msBJdXckAVOBr0mwVwcHUKbmBDaRJuBr1VUMAG+ujGLyCmrfpzNu8c3A/Bsi6E0u3igxRWJiIjUjQKgiBsqLClj/9FCUrIK2He0wPx68pGVX1Ljdn4+XrSKDKZ1VAgXR4dUBL74Jgp64kH8/eHDDyvb5yg9cwdP/fQqeNm43SeGa6+Z4qACRUREHEcBUMRNHM0v5r75P5KSVUBGXnGtfaND/WkVFUyrqJCKwNc6KoRmjQJ16qaIj485/18dlJYV88jSO8jxstGp3MZDwz8Em36WRETE/SgAiriJsEBfftifTbndAKBRkC8tmwSTEGk+4iODaRUZTMsmQYQGnP/E1iJyphmfj2MTJwix23lxwGv4BjW2uiQREZHzogAo4iZ8vb2YObobMWEBJDQJJjxIIU/kvJSVwaJFZvvmm80RwVp8+8Ms/p2zFYBn4m+mecI1jq5QRETEYRQARdzIwE6xVpcg4v6Ki+G228x2fn6tAfBI5jae2jwDvGyM9I3j+v7POqlIERERx9DERSIiItUoKyvm0aXjyPay0b7cxsM3f6Dr/kRExO1pBFBERKQaMz8fx4+cIMhuZ/qAGfgH6ro/ERFxfxoBFBER+ZU1P77JW8e3ADA5/mZaJgywuCIREZH6oQAoIiJymsyM7Tzx02sYNhvD/eIYPODvVpckIiJSbxQARURETiovK+HxZWM55mWjbbkXjw77wOqSRERE6pUCoIiIyElvfj6O7zlBoN1g+oCXCdB8fyIi0sDoJjAiIuJZ/PxgzpzK9knrf3yLWcc3g83G3+KHkaD5/kREpAFSABQREc/i6wvjxlVZlJW1k8c3vYLhbeO3vrHcOOA5a2oTERFxMJ0CKiIiHq28rJQnltxBlreNi8ttPH6TrvsTEZGGSyOAIiLiWcrK4MsvzfbAgby15C7WUUSg3eAf/V8iMKiJtfWJiIg4kEYAxWPMnDmThIQEAgIC6N69O99++22t/VetWkX37t0JCAigVatWzJo1y0mViohDFRfDjTfCjTfyxfqXmZmdDMBTLW6kVavrLS5ORETEsRQAxSMsWLCAiRMn8tRTT5GcnEzfvn0ZPHgwqamp1fbft28fN9xwA3379iU5OZknn3ySBx54gIULFzq5chFxhOwQbx6Z0JxHfn4Pu83Gb3yjGXbtC1aXJSIi4nA2wzAMq4sQcbTLL7+cbt268cYbb1Qs69ChAzfddBNTpkw5o/9jjz3G4sWL2bFjR8Wy8ePH89NPP7F27dpzes/c3FzCw8PJOXyYsLCwMzt4e0NAQOXzgoKaX8zLCwIDz69vYSHU9GNus0FQ0Pn1LSoCu73mOoKDz6/viRNQXl4/fYOCzLrBHPUpK6ufvoGB5vcZoKQESkvrp29AgPnvoq59S0vN/jXx9wcfn7r3LSszvxc18fMzb6hS177l5eaxq4mvb+XdOevS1243/62dpe/Xu5fwzBcPcyzcB2/D4K5ib+67dSm+ARGVfX18zO8FmD8ThYU1v25d+tbl517/R1TfV/9H1L2vG/4fkZuRQXjTpuTk5FT/GSoi503XAEqDV1JSwg8//MDjjz9eZXliYiJr1qypdpu1a9eSmJhYZdnAgQN5++23KS0txffUh9RpiouLKT7twy03N9dsNG1afWE33ABLllQ+j46u+RfHfv0o/vpLrvvoOvP5sWM1/7Lk4wMREZXPa+vr7Q2NGlU+z86u+ZclLy9ofNqcaMeP1/wL0K/75uTU/IuKzQZNTrvmKje39l8+IiMr23l5tf9C0aRJ5S9sZ+vbuHHlL2H5+bWHjkaNKn+xKiioPXSc3rewsPZwEBFR+YtVUVHtv8SHh1f+snTihFlzTcLCKkPS2fqGhlaGmeJi8/tWk5CQyoBSUmIeu3PpW1pq/puoSXBwZUApKzP/rdUkKKgydJSXm/+GaxIYiBEcRE5xDoT70LqkhOfSjtLp9WwYf1HVvvffD6+/brazssyfz5qMHQvvvGO2CwvNfa3JrbfCRx9VPq+tbx3/jyApqfJ5fLxZd3V69IANGyqfd+wI+/dX37djR9i2rfJ5z56wfXv1fVu2hJSUyudXXw0bN1bfNzISMjMrnw8eDKtWVd83KKjqz8Itt8DSpdX3haoBdcwY+Pjjmvvm51cGxj/8Ad59t+a+GRkQFWW2H3oIZs6sue++feYxAHjqKZg+vea+W7dCp05m+/nn4emna+77/ffmMQD45z/h0Udr7rtyJfTvb7Znz4YJE2ru+/nnMGSI2Z4/H+68s+a+H34Iw4eb7UWL4Lbbau47Z07lHXe//NI89bomM2bAH/9otr/9FgYMqLmviFwQBUBp8LKysigvLycmJqbK8piYGNLT06vdJj09vdr+ZWVlZGVlERcXd8Y2U6ZM4enaPrjrwfHi42Yj2Itaz+A+1a+ufYNs1Prfwul9Azn3vgFAwDn29Qf8z7GvH+BXS9+S00LG2fqWnhZefAHfWvqW5cGp7OsDhJ5jX++z9C3Ph1P52+ssfe0FcCrP2s7S1yiE4sJz60sRFJ8WaGvtewKKTwvK9da3uGpYr7VvCRSf9geDWvuWQnEOXti4M/s49x/PwW9xEWTUMvIkIiLSwOgUUGnwDh8+TLNmzVizZg19+vSpWP7cc88xd+5cdu7cecY2bdu25c477+SJJ56oWPa///2Pq666irS0NGJjY8/YproRwObNm9fbKaD2AH9SclLM50W1jCJ5eYH/aa97oqj2U7YCAs+vb/GJ2k/ZCgw6v74lxbWfslWXvgGBlSOAJSVQXsspW3XqGwC2k6G6tBTKznIa1rn29fcHL++69y0rrf1UMD8/8Pape9/ystpHY319wce37n3t5bWPxvr4Vo5u1qWvYa995NbHF8oLiPjPKBrnpsHmUnhjX/WjcDoF1KRTQM+vr04BNekUUBGXpBFAafAiIyPx9vY+Y7QvIyPjjFG+U2JjY6vt7+PjQ5Mm1d8i3t/fH/9TvwSeLji46i8kNTlLHy+gVUQr80nE2V9ORH7FMOD9kZCbBlnlsKTIDH9n+/m02c7tZ7iufcE1+p4e2uqz7+khsz77nh6K67Ovv39lkK/Pvn5+ladgW9XX97Q/lNRnXx+fyjBYn329vev2b1hE6kR3AZUGz8/Pj+7du7NixYoqy1esWMEVV1xR7TZ9+vQ5o//y5cvp0aNHtdf/iYgbWDsDdn8B3v7Q5s/w0oxz/wVaRESkgdAIoHiEhx56iDFjxtCjRw/69OnD7NmzSU1NZfz48QA88cQTHDp0iPfeew8w7/g5Y8YMHnroIe69917Wrl3L22+/zfvvv2/lbojI+TqwAb6abLYHTYGed1tajoiIiFUUAMUjjBgxgqNHj/LMM8+QlpZG586dWbp0KS1btgQgLS2typyACQkJLF26lAcffJDXX3+dpk2b8uqrr3LLLbdYtQsicr4Kj8HHd4K9DDrdDD3usroiERERy+gmMCIOUjEPoC5gF7GOYcAHo2DXUmiUAH9YDb7B5m3mAfr2rbxRhoi4DH2GijiORgBFRKThWveGGf68/eC2dyEgzLxD5qk5xk6fB05ERMQD6CYwIiLSMB38AVb8zWwPfB7iLrG2HhERERegACgiIg1PUTZ8NA7spdBxGPS8x+qKREREXIICoIiINCyGAZ/+EXJSoVE8/Oa1ygm8RUREPJwCoIiINCzr3oBdS8zr/oa/CwHhVlckIiLiMhQARUSk4Ti4EVb81WwPfB6aXmppOSIiIq5GAVBERBqGomz46OR8fx1v0nV/IiIi1dA0ECIi4v4MAxbdd/K6vwT4zas1X/fn6wvTplW2RUREPIgCoIiIuL81r8HuZeDtD8Pfqf26Pz8/eOQRp5UmIiLiSnQKqIiIuLfUdfDVZLM9+AVd9yciIlILjQCKiIj7Ksgyr/szyqHLcOh+59m3KS+HH3802926gbe3Y2sUERFxIQqAIiLinux2+OT3kHcYmrSBG185t/n+TpyAXr3Mdn4+BAc7tEwRERFXolNARUTEPX37D/jla/AJhNveBf8QqysSERFxeQqAIiLifvYmQdLzZnvIPyCmk6XliIiIuAsFQBERcS+5h2HhPWDY4bIxcNloqysSERFxGwqAIiLiPspL4eO7oCATYrrADS9aXZGIiIhbUQAUERH38fXTkLoW/MPM6/58A62uSERExK0oAIqIiHvY8bk54TvATTOhSWtr6xEREXFDmgZCRERc39Ff4NP7zHafCdBh6Pm/lq8vTJpU2RYREfEgCoAiIuLaSgpgwRgozoXmveG6yRf2en5+MPkCX0NERMRN6RRQERFxXYYBnz8IGdsgONq87s9bo3YiIiLnSyOAIiLiuja8BZsXgM0bhr8DobEX/pp2O+zYYbY7dAAv/S1UREQ8hwKgiIi4pgMb4IsnzPZ1kyH+yvp53aIi6NzZbOfnQ3Bw/byuiIiIG9CfPUVExPXkZ8JHY8FeCh1+A1f8yeqKREREGgQFQBERcS3lZfDxnZB7CJpcDMNeB5vN6qpEREQaBAVAERFxLV9NgpRvwS8ERsyHgDCrKxIREWkwFABFRMR1bPkY1s4w2zfNhOj21tYjIiLSwCgAioiIaziyDRafvNbvyonQcZil5YiIiDRECoAiImK9ouPwwWgoLYRW/eHav1ldkYiISIOkaSBERMRa9nJYeDdk74PwFnDrHPDydtz7+frCww9XtkVERDyIAqCIiFjrm2fh56/AJxBGzoOgxo59Pz8/ePFFx76HiIiIi9IpoCIiYp2tC+G7l832sBkQd4m19YiIiDRwGgEUERFrpG+BT/9otq94ALrc6pz3tdshNdVst2gBXvpbqIiIeA4FQBERcb6Co/D+KCgrgtbXwnWTnffeRUWQkGC28/MhONh57y0iImIx/dlTREScq7wUPhoLOanQuBXc+rZjb/oiIiIiFRQARUTEeQwDlj4CKd+CXwiM/A8ENrK6KhEREY+hACgiIs6z4S34YQ5gg1vehugOVlckIiLiURQARUTEOfYmwbLHzPZ1k6HdICurERER8UgKgCIi4nhHf4EPx4JRDl1HwpV/troiERERj6QAKCIijlV0HN4fCSeOw0U9Yeg/wWazuioRERGPpGkgRETEccpL4cM7IGs3hDWDEfPAN8Damnx84P77K9siIiIeRJ98IiLiGIYBS/4C+1aBbzCMWgChsVZXBf7+8PrrVlchIiJiCZ0CKiIijrF2Bvz4Lti84NZ/Q2wXqysSERHxeBoBFBGR+rdzCSz/q9lOfM617vhpGJCVZbYjI3U9ooiIeBQFQBERqV+Hk2HhPYABPe6C3vdZXVFVhYUQHW228/MhONjaekRERJxIp4CKiEj9yd4P/xkBpYXQ+hoYPE0jbCIiIi5EAVBEROpHUTbMHw75RyCmMwx/F7x9ra5KRERETqMAKCIiF66sGD64HbJ2QWhTGPUhBIRZXZWIiIj8igKgiIhcGLsd/vtH2P8d+IXC6I8gvJnVVYmIiEg1FABFROTCfP00bPkIvHxgxHsQ29nqikRERKQGCoAiInL+1r0B/3vFbA991bzxi4iIiLgsTQMhIiLnZ8vH8MXjZvvav8Flo62t51z5+MDYsZVtERERD6JPPhERqbtfvoFF4812rz/AVQ9ZW09d+PvDO+9YXYWIiIgldAqoiIjUzeFNsGAM2Euh080w6AXN9SciIuImNAIoIiLnLmsPzLsFSvIh4Wq4+U3wcrO/JRoGFBaa7aAghVcREfEobvapLSIiljl+AN67CQqzIO4SGDEffPytrqruCgshJMR8nAqCIiIiHkIBUEREzi4/A94bBrkHIbIt3P6JJnoXERFxQwqAIiJSu6LjMPe3cOwXCG8BYz6F4EirqxIREZHzoAAoIiI1K86H/9wGR7ZAcDTc8SmEN7O6KhERETlPCoAiIlK9kkJ4fyQcWA8B4TBmETRpbXVVIiIicgEUAKXBy87OZsyYMYSHhxMeHs6YMWM4fvx4rduMGzcOm81W5dG7d2/nFCziCkpPwILRkPIt+IXC7YsgtrPVVYmIiMgF0jQQ0uCNGjWKgwcP8sUXXwDw+9//njFjxvDZZ5/Vut2gQYOYM2dOxXM/Pz+H1iniMspK4MM7zMnefYPh9o/hou5WVyUiIiL1QAFQGrQdO3bwxRdfsG7dOi6//HIA/vWvf9GnTx927dpFu3btatzW39+f2NhYZ5Uq4hrKS+HjO2HPl+ATCKMWQIsGNvrt7Q233lrZFhER8SA6BVQatLVr1xIeHl4R/gB69+5NeHg4a9asqXXbpKQkoqOjadu2Lffeey8ZGRm19i8uLiY3N7fKQ8StlJfCx3fBzs/B2x9+9x9I6Gt1VfUvIAA++sh8BARYXY2IiIhTKQBKg5aenk50dPQZy6Ojo0lPT69xu8GDBzN//ny++eYb/vGPf7BhwwauueYaiouLa9xmypQpFdcZhoeH07x583rZBxGnKCuBj8bBjsXg7Qcj5kHra6yuSkREROqZAqC4pcmTJ59xk5ZfPzZu3AiAzWY7Y3vDMKpdfsqIESMYMmQInTt3ZujQoSxbtozdu3ezZMmSGrd54oknyMnJqXgcOHDgwndUxBnKiuGjsZUjfyPfh7aJVlclIiIiDqBrAMUtTZgwgZEjR9baJz4+ns2bN3PkyJEz1mVmZhITE3PO7xcXF0fLli3Zs2dPjX38/f3x9/c/59cUcQllxbBgzMlr/gJg5H/g4mutrsqxCgogJMRs5+dDcLC19YiIiDiRAqC4pcjISCIjI8/ar0+fPuTk5PD999/Tq1cvANavX09OTg5XXHHFOb/f0aNHOXDgAHFxcedds4jLKSk0p3r45Rvzhi+/ex9aD7C6KhEREXEgnQIqDVqHDh0YNGgQ9957L+vWrWPdunXce++93HjjjVXuANq+fXsWLVoEQH5+Pg8//DBr164lJSWFpKQkhg4dSmRkJDfffLNVuyJSv07kwLzfnpzqIci826fCn4iISIOnACgN3vz58+nSpQuJiYkkJibStWtX5s6dW6XPrl27yMnJAcDb25stW7YwbNgw2rZty9ixY2nbti1r164lNDTUil0QqV+Fx+Dd30DqWvAPhzGfQqt+VlclIiIiTmAzDMOwugiRhig3N5fw8HBycnIICwuzuhwRU146vHcTZO6AoCYwZhHEXWJ1Vc6lawBFXJ4+Q0UcR9cAioh4iqO/mKd9ZqdAaBzc8V+IanfWzURERKThUAAUEfEEhzfB/FuhIBMaxZunfTZOsLgoERERcTYFQBGRhm5vEnwwGkryIbYLjF4Ioec+DUqD4+0NN9xQ2RYREfEgCoAiIg3Z1k9g0R+gvAQSroYR8yHAw6+nCQiAJUusrkJERMQSuguoiEhDZBjwv1fh47vM8NfxJhj9scKfiIiIh9MIoIhIQ1NeBssehY1vm897/R4GvQBeOt1RRETE0ykAiog0JMV55qjfnuWADQY+D73vA5vN6spcR0EBREeb7YwMTQMhIiIeRQFQRKShyDkE74+A9C3gEwi3vAUdbrS6KtdUWGh1BSIiIpZQABQRaQgObIAPRkFBBgRHwagF0Ky71VWJiIiIi1EAFBFxd5veh88eMG/2EtMZRv4HGrW0uioRERFxQQqAIiLuyl4OX02CNa+Zz9vfCDe/Cf4h1tYlIiIiLksBUETEHRUeg4X3wC9fm8+vfhT6PwFemt1HREREaqYAKCLibg4nw4I7ICfVvNnLTTOh82+trkpERETcgAKgiIg7+XEuLPkLlBdD41YwYh7EdLK6Kvfi5QX9+lW2RUREPIgCoIiIOygphC8egx/fM5+3HQw3z4LACEvLckuBgZCUZHUVIiIillAAFBFxdRk74eM7IWM7YINrnoKr/qLRKxEREakzBUAREVdlGLBpPix9BEoLITgabvkXtOpvdWUiIiLiphQARURc0YlcWPowbF5gPm/VH377LwiJtrSsBqGgAOLjzXZKCgQHW1mNiIiIUykAioi4mv1rYdHv4Xgq2LxhwJNw1UM65bM+ZWVZXYGIiIglFABFRFxFWQmsegG+exkMO0S0MEf9WvS2ujIRERFpIBQARURcQeYu+OT3kLbJfH7JKBg8FQLCLC1LREREGhYFQBERK5WXwZpXIWkKlJdAYCO48RXodJPVlYmIiEgDpAAoImKVI9vhv/fD4WTzeZuBMPQVCGtqaVkiIiLScCkAiog4W1kxfPcKrH4R7KUQEA6DpsIlI8Fms7o6ERERacAUAEVEnCnlf/D5RMjabT5vOxhufBnC4iwty6N4eUGPHpVtERERD6IAKCLiDIXHYMVfIXme+Tw4GgZNgc63aNTP2QIDYcMGq6sQERGxhAKgiIgj2cvhx/fg62eg6Ji5rPudcN0k84YvIiIiIk6kACgi4igHvoelj1RO7RDVwbzJi+b1ExEREYsoAIqI1Lfcw+aI30/vm8/9w2DAk9DzHvD2tbY2gcJC6NjRbG/fDkFB1tYjIiLiRAqAIiL1pTgP/vdPWDMDyorMZZfdDtdOhpAoS0uT0xgG7N9f2RYREfEgCoAiIheqvAyS58LK56Egw1zWvDcMfB4u6m5tbSIiIiKnUQAUETlfdjts+wSSpsDRn81ljVvBdU9Dh6G6u6eIiIi4HAVAEZG6MgzY/QV883c4stVcFtgY+j0GPe4CHz9r6xMRERGpgQKgiMi5OhX8Vr8Ih34wl/mHwRV/gt73gX+otfWJiIiInIUCoIjI2djtsPMzM/ilbzGX+QTC5X+AK/8MQY2trU9ERETkHCkAiojUpKwEtnwEa16FzJ3mMt9g6HUP9PmT7uzprmy2ymkgdJ2miIh4GAVAEZFfO5EDG+fA+lmQl2Yu8w8zR/x6368RP3cXFATbtlldhYiIiCUUAEVETjn6C3w/G5LnQ0meuSw0Di4fDz3uhIBwa+sTERERuUAKgCLi2ex22PsNrH8T9iyvXB7V3ry5S5fh4ONvXX0iIiIi9UgBUEQ8U34mbJoHP7wL2fsql7cZCJf/HlpdA15e1tUnjlNYCD17mu0NG8xTQkVERDyEAqCIeA57OexbDT++Czs+B3upudw/DC4dDb3uhSatra1RHM8wYPv2yraIiIgHUQAUkYYv62f46T/w0weQe6hyebMe0H0cdP4t+AVbVp6IiIiIsygAikjDlJ8B2z41p3E4+H3l8oBw87q+7uMgtotV1YmIiIhYQgFQRBqOomzYuRS2fgx7k8Cwm8ttXnDxdXDpKGg7GHwDLC1TRERExCoKgCLi3gqOws7PYcdiM/TZyyrXNe0GXW6FTr+FsDjLShQRERFxFQqAIuJ+jv4Cu5bB7i9g/xowyivXRXUwr+nrfItu6CIiIiLyKwqAIuL6yoohdR38/BXs/hKydlVdH9sVOv4GOgyDqLbW1Cjuw2aDli0r2yIiIh5EAVBEXI9hmKN8+5Lg569h7yooLahc7+UDLa+EdoOh7SBonGBZqeKGgoIgJcXqKkRERCyhACgiriHnEKR8B/tWmYEv92DV9cHR5o1cLr7W/BoYYUmZIiIiIu5MAVBEnM8w4Nhe87TO/f8zg9/x/VX7ePtB88uhVX9ocz3EdAEvL0vKFREREWkoFABFxPFKCiHtJ3M+vgPfw4H1UJBZtY/NC+IugYSrIaEftOgDfkHW1CsNW1ERXH212V69GgIDra1HRETEiRQARaR+lZVA5g44vAkO/wgHf4CM7VXv1AnmCF/cpRB/pXk9X/PLISDMiorF09jtsHFjZVtERMSDKACKyPkrOm6Gu/StcGSLOcp3ZDvYS8/sGxILF/Uwg17zy83RPk3ILiIiIuJUCoAicnYlhZC1GzJ3QsaOykdOavX9AyLMgNesGzTrbj7Cmjq1ZBERERE5kwKgiJjsdsg7bE6/cOwXyNpjhr6s3XD8AGBUv114c4jpBDGdzdAXdwlEtND8aiIiIiIuSAFQxJOUFkHOQcjeD9n7IDsFju0z28f2QtmJmrcNbAzRHSCqvfk1uoMZ/AIbOa18EREREbkwCoAiDYXdDoVZkHvInFMv56A5l17OQXME73gqFGTU/hpePhDREpq0hsi2ENnm5Ne2ENREo3oiIiIibk4BUMRdlBVDyreQdwTy0yHv9Eea+bW6m6/8ml+IeYpmowRoFA+NT31tZYY/b/23IB4gMtLqCkRERCyh3/RE3EXZCZh3y1k62SAkBsKbQVgz8/q88JNfI1qYj8BGGskTzxYcDJmZZ+8nIiLSACkAirgL/zBo2s0McKGxZtALjYPQGAhtCmFx5jJvX6srFREREREXpQAoDd5zzz3HkiVL2LRpE35+fhw/fvys2xiGwdNPP83s2bPJzs7m8ssv5/XXX6dTp06OL7gmNhv8fqV17y8iIiIibs/L6gJEHK2kpIThw4dz3333nfM206ZN46WXXmLGjBls2LCB2NhYrr/+evLy8hxYqYg4RVER9O9vPoqKrK5GRETEqWyGYdQwuZdIw/LOO+8wceLEs44AGoZB06ZNmThxIo899hgAxcXFxMTEMHXqVP7whz+c0/vl5uYSHh5OTk4OYWFhF1q+iNSXggIICTHb+fnmNYEi4lL0GSriOBoBFPmVffv2kZ6eTmJiYsUyf39/+vXrx5o1a2rcrri4mNzc3CoPERERERFXogAo8ivp6ekAxMTEVFkeExNTsa46U6ZMITw8vOLRvHlzh9YpIiIiIlJXCoDiliZPnozNZqv1sXHjxgt6D9uvpkowDOOMZad74oknyMnJqXgcOHDggt5fRERERKS+6S6g4pYmTJjAyJEja+0THx9/Xq8dGxsLmCOBcXFxFcszMjLOGBU8nb+/P/7+/uf1niIiIiIizqAAKG4pMjKSyMhIh7x2QkICsbGxrFixgssuuwww7yS6atUqpk6d6pD3FBERERFxBp0CKg1eamoqmzZtIjU1lfLycjZt2sSmTZvIz8+v6NO+fXsWLVoEmKd+Tpw4keeff55FixaxdetWxo0bR1BQEKNGjbJqN0SkPgUFmQ8REREPoxFAafD+9re/8e6771Y8PzWqt3LlSvr37w/Arl27yMnJqejz6KOPUlRUxP33318xEfzy5csJDQ11au0i4gDBweZUECIiIh5I8wCKOIjmMBIRETk/+gwVcRydAioiIiIiIuIhFABFRMSznDgBQ4aYjxMnrK5GRETEqXQNoIiIeJbycli6tLItIiLiQTQCKCIiIiIi4iEUAEVERERERDyEAqCIiIiIiIiHUAAUERERERHxEAqAIiIiIiIiHkJ3ARVxEMMwAHMyWxFxIQUFle3cXN0JVMQFnfrsPPVZKiL1RwFQxEHy8vIAaN68ucWViEiNmja1ugIRqUVeXh7h4eFWlyHSoNgM/WlFxCHsdjuHDx8mNDQUm81WL6+Zm5tL8+bNOXDgAGFhYfXymlbTPrmHhrZPDW1/QPvkLrRP58YwDPLy8mjatCleXrpiSaQ+aQRQxEG8vLy46KKLHPLaYWFhDeYXh1O0T+6hoe1TQ9sf0D65C+3T2WnkT8Qx9CcVERERERERD6EAKCIiIiIi4iEUAEXciL+/P5MmTcLf39/qUuqN9sk9NLR9amj7A9ond6F9EhGr6SYwIiIiIiIiHkIjgCIiIiIiIh5CAVBERERERMRDKACKiIiIiIh4CAVAERERERERD6EAKOLCUlJSuPvuu0lISCAwMJDWrVszadIkSkpKat3OMAwmT55M06ZNCQwMpH///mzbts1JVdfuueee44orriAoKIiIiIhz2mbcuHHYbLYqj969ezu20Do4n31y5WMEkJ2dzZgxYwgPDyc8PJwxY8Zw/PjxWrdxteM0c+ZMEhISCAgIoHv37nz77be19l+1ahXdu3cnICCAVq1aMWvWLCdVeu7qsk9JSUlnHA+bzcbOnTudWHHtVq9ezdChQ2natCk2m41PP/30rNu48nGq6/64wzGaMmUKPXv2JDQ0lOjoaG666SZ27dp11u1c+TiJeDoFQBEXtnPnTux2O2+++Sbbtm3j5ZdfZtasWTz55JO1bjdt2jReeuklZsyYwYYNG4iNjeX6668nLy/PSZXXrKSkhOHDh3PffffVabtBgwaRlpZW8Vi6dKmDKqy789knVz5GAKNGjWLTpk188cUXfPHFF2zatIkxY8acdTtXOU4LFixg4sSJPPXUUyQnJ9O3b18GDx5Mampqtf337dvHDTfcQN++fUlOTubJJ5/kgQceYOHChU6uvGZ13adTdu3aVeWYtGnTxkkVn11BQQGXXHIJM2bMOKf+rn6c6ro/p7jyMVq1ahV//OMfWbduHStWrKCsrIzExEQKCgpq3MbVj5OIxzNExK1MmzbNSEhIqHG93W43YmNjjRdeeKFi2YkTJ4zw8HBj1qxZzijxnMyZM8cIDw8/p75jx441hg0b5tB66sO57pOrH6Pt27cbgLFu3bqKZWvXrjUAY+fOnTVu50rHqVevXsb48eOrLGvfvr3x+OOPV9v/0UcfNdq3b19l2R/+8Aejd+/eDquxruq6TytXrjQAIzs72wnVXTjAWLRoUa193OE4nXIu++Nux8gwDCMjI8MAjFWrVtXYx52Ok4gn0gigiJvJycmhcePGNa7ft28f6enpJCYmVizz9/enX79+rFmzxhklOkRSUhLR0dG0bduWe++9l4yMDKtLOm+ufozWrl1LeHg4l19+ecWy3r17Ex4eftb6XOE4lZSU8MMPP1T5/gIkJibWWP/atWvP6D9w4EA2btxIaWmpw2o9V+ezT6dcdtllxMXFce2117Jy5UpHlulwrn6czpc7HaOcnByAWj+HGupxEmkoFABF3Mgvv/zCa6+9xvjx42vsk56eDkBMTEyV5TExMRXr3M3gwYOZP38+33zzDf/4xz/YsGED11xzDcXFxVaXdl5c/Rilp6cTHR19xvLo6Oha63OV45SVlUV5eXmdvr/p6enV9i8rKyMrK8thtZ6r89mnuLg4Zs+ezcKFC/nkk09o164d1157LatXr3ZGyQ7h6seprtztGBmGwUMPPcRVV11F586da+zX0I6TSEOjAChigcmTJ1d74f/pj40bN1bZ5vDhwwwaNIjhw4dzzz33nPU9bDZbleeGYZyxrL6cz/7UxYgRIxgyZAidO3dm6NChLFu2jN27d7NkyZJ63IuqHL1P4NxjBHXbp+rqOFt9Vhyn2tT1+1td/+qWW6ku+9SuXTvuvfdeunXrRp8+fZg5cyZDhgxh+vTpzijVYdzhOJ0rdztGEyZMYPPmzbz//vtn7duQjpNIQ+NjdQEinmjChAmMHDmy1j7x8fEV7cOHDzNgwAD69OnD7Nmza90uNjYWMP8CGxcXV7E8IyPjjL/I1pe67s+FiouLo2XLluzZs6feXvPXHLlPVhwjOPd92rx5M0eOHDljXWZmZp3qc8Zxqk5kZCTe3t5njIzV9v2NjY2ttr+Pjw9NmjRxWK3n6nz2qTq9e/dm3rx59V2e07j6caoPrnqM/vSnP7F48WJWr17NRRddVGtfTzhOIu5MAVDEApGRkURGRp5T30OHDjFgwAC6d+/OnDlz8PKqfeA+ISGB2NhYVqxYwWWXXQaY1w+tWrWKqVOnXnDt1anL/tSHo0ePcuDAgSrhqb45cp+sOEZw7vvUp08fcnJy+P777+nVqxcA69evJycnhyuuuOKc388Zx6k6fn5+dO/enRUrVnDzzTdXLF+xYgXDhg2rdps+ffrw2WefVVm2fPlyevToga+vr0PrPRfns0/VSU5OdvrxqE+ufpzqg6sdI8Mw+NOf/sSiRYtISkoiISHhrNt4wnEScWtW3X1GRM7u0KFDxsUXX2xcc801xsGDB420tLSKx+natWtnfPLJJxXPX3jhBSM8PNz45JNPjC1bthi/+93vjLi4OCM3N9fZu3CG/fv3G8nJycbTTz9thISEGMnJyUZycrKRl5dX0ef0/cnLyzP+8pe/GGvWrDH27dtnrFy50ujTp4/RrFkzl9gfw6j7PhmGax8jwzCMQYMGGV27djXWrl1rrF271ujSpYtx4403Vunjysfpgw8+MHx9fY23337b2L59uzFx4kQjODjYSElJMQzDMB5//HFjzJgxFf337t1rBAUFGQ8++KCxfft24+233zZ8fX2Njz/+2Om116Su+/Tyyy8bixYtMnbv3m1s3brVePzxxw3AWLhwoVW7cIa8vLyKnxfAeOmll4zk5GRj//79hmG433Gq6/64wzG67777jPDwcCMpKanKZ1BhYWFFH3c7TiKeTgFQxIXNmTPHAKp9nA4w5syZU/HcbrcbkyZNMmJjYw1/f3/j6quvNrZs2eLk6qs3duzYavdn5cqVFX1O35/CwkIjMTHRiIqKMnx9fY0WLVoYY8eONVJTU63ZgWrUdZ8Mw7WPkWEYxtGjR43Ro0cboaGhRmhoqDF69OgzblXv6sfp9ddfN1q2bGn4+fkZ3bp1q3Lb+rFjxxr9+vWr0j8pKcm47LLLDD8/PyM+Pt544403nFzx2dVln6ZOnWq0bt3aCAgIMBo1amRcddVVxpIlSyyoumanpkH49WPs2LGGYbjfcarr/rjDMarpM+j0/8/c7TiJeDqbYZy8KldEREREREQaNN0FVERERERExEMoAIqIiIiIiHgIBUAREREREREPoQAoIiIiIiLiIRQARUREREREPIQCoIiIiIiIiIdQABQREREREfEQCoAiIiLnwWaz8emnn1pdhoiISJ34WF2AiIiIO0pLS6NRo0ZWlyEiIlInNsMwDKuLEBEREREREcfTKaAiIuKSMjMziY2N5fnnn69Ytn79evz8/Fi+fPkFvfa8efPo0aMHoaGhxMbGMmrUKDIyMirWP/PMMzRt2pSjR49WLPvNb37D1Vdfjd1uB6qeAlpSUsKECROIi4sjICCA+Ph4pkyZckE1ioiIOIICoIiIuKSoqCj+/e9/M3nyZDZu3Eh+fj633347999/P4mJiRf02iUlJTz77LP89NNPfPrpp+zbt49x48ZVrH/qqaeIj4/nnnvuAWDWrFmsXr2auXPn4uV15kfnq6++yuLFi/nwww/ZtWsX8+bNIz4+/oJqFBERcQSdAioiIi7tj3/8I1999RU9e/bkp59+YsOGDQQEBNTre2zYsIFevXqRl5dHSEgIAHv37uXSSy/l/vvv57XXXmP27NmMHj26YhubzcaiRYu46aabeOCBB9i2bRtfffUVNputXmsTERGpTxoBFBERlzZ9+nTKysr48MMPmT9/fq3hb/DgwYSEhBASEkKnTp1q7JecnMywYcNo2bIloaGh9O/fH4DU1NSKPq1atWL69OlMnTqVoUOHVgl/vzZu3Dg2bdpEu3bteOCBBy74FFURERFHUQAUERGXtnfvXg4fPozdbmf//v219n3rrbfYtGkTmzZtYunSpdX2KSgoIDExkZCQEObNm8eGDRtYtGgRYJ4aerrVq1fj7e1NSkoKZWVlNb5vt27d2LdvH88++yxFRUXcdttt3HrrrXXcUxEREcfTNBAiIuKySkpKGD16NCNGjKB9+/bcfffdbNmyhZiYmGr7N2vW7KyvuXPnTrKysnjhhRdo3rw5ABs3bjyj34IFC/jkk09ISkpixIgRPPvsszz99NM1vm5YWBgjRoxgxIgR3HrrrQwaNIhjx47RuHHjc9xbERERx1MAFBERl/XUU0+Rk5PDq6++SkhICMuWLePuu+/m888/P+/XbNGiBX5+frz22muMHz+erVu38uyzz1bpc/DgQe677z6mTp3KVVddxTvvvMOQIUMYPHgwvXv3PuM1X375ZeLi4rj00kvx8vLio48+IjY2loiIiPOuU0RExBF0CqiIiLikpKQkXnnlFebOnUtYWBheXl7MnTuX7777jjfeeOO8XzcqKop33nmHjz76iI4dO/LCCy8wffr0ivWGYTBu3Dh69erFhAkTALj++uuZMGECt99+O/n5+We8ZkhICFOnTqVHjx707NmTlJQUli5dWu0dQ0VERKyku4CKiIiIiIh4CP1pUkRERERExEMoAIqIiIiIiHgIBUAREREREREPoQAoIiIiIiLiIRQARUREREREPIQCoIiIiIiIiIdQABQREREREfEQCoAiIiIiIiIeQgFQRERERETEQygAioiIiIiIeAgFQBEREREREQ+hACgiIiIiIuIh/h9i2TldQXsGMQAAAABJRU5ErkJggg=="
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"x_sample = torch.linspace(-2, 2, 100)\n",
"sigmoid_output = nn.Sigmoid()(x_sample).detach().numpy()\n",
"tanh_output = nn.Tanh()(x_sample).detach().numpy()\n",
"relu_output = nn.ReLU()(x_sample).detach().numpy()\n",
"\n",
"f = plt.figure()\n",
"f.set_figwidth(6)\n",
"f.set_figheight(6)\n",
"plt.xlabel('x - axis')\n",
"plt.ylabel('y - axis')\n",
"plt.title(\"Input: 100 x-values between -1 to 1 \\n\\n Output: Corresponding y-values after passed through each activation function\\n\", fontsize=16)\n",
"plt.axvline(x=0, color='r', linestyle='dashed')\n",
"plt.axhline(y=0, color='r', linestyle='dashed')\n",
"plt.plot(x_sample, sigmoid_output)\n",
"plt.plot(x_sample, tanh_output)\n",
"plt.plot(x_sample, relu_output)\n",
"plt.legend([\"\",\"\",\"Sigmoid Output\", \"Tanh Output\", \"ReLU Output\"])\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "971aac32",
"metadata": {},
"source": [
"### Task 3.1 - Forward pass\n",
"\n",
"In part 2, you manually created the Linear layers and explicitly specified weights and biases for the forward pass to connect every input neuron to every output neuron which will be extremely tedious for larger networks. \n",
"\n",
"In this task, you will be using `nn.Linear(in_dimensions, out_dimensions)` provided by pytorch which abstracts all these details away. `nn.Linear` represents a fully connected layer with bias automatically included. We can also choose to remove the bias column by simply calling `nn.Linear(in_dimensions, out_dimensions, bias=False)` instead.\n",
"\n",
"We inherit from PyTorch's `nn.Module` class to build the model from the previous task `y = |x-1|` from the lecture. \n",
"\n",
"<img src=\"imgs/img_toy_nn.jpg\" width=\"400\"> \n",
"\n",
"Pytorch is widely used in machine learning due to the ease of being able to combine many different types of layers and activation functions to create neural networks. This task should allow you to appreciate how easily we can build neural networks using PyTorch. \n",
"\n",
"The model has been built for you in `__init__`. You need to implement the `forward` method, making use of the layers `self.l1`, `self.l2`, and the activation function `self.relu`. You need to combine the linear layers AND the activation function in the forward pass function!\n",
"\n",
"_Extra: PyTorch has many other layers implemented for various model architectures. \n",
"You can read more in the glossary as well as in the docs: https://pytorch.org/docs/stable/nn.html \n",
"For now, we will only be using fully connected `nn.Linear` layers._"
]
},
{
"cell_type": "code",
"execution_count": 127,
"id": "81145ccc",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T04:13:25.016858Z",
"start_time": "2024-04-02T04:13:25.013710Z"
}
},
"outputs": [],
"source": [
"class MyFirstNeuralNet(nn.Module):\n",
" def __init__(self): # set the arguments you'd need\n",
" super().__init__()\n",
" self.l1 = nn.Linear(1, 2) # bias included by default\n",
" self.l2 = nn.Linear(2, 1) # bias included by default\n",
" self.relu = nn.ReLU()\n",
" \n",
" # Task 3.1: Forward pass\n",
" def forward(self, x):\n",
" '''\n",
" Forward pass to process input through two linear layers and ReLU activation function.\n",
"\n",
" Parameters\n",
" ----------\n",
" x : A tensor of of shape (n, 1) where n is the number of training instances\n",
"\n",
" Returns\n",
" -------\n",
" Tensor of shape (n, 1)\n",
" '''\n",
" x = self.l1(x)\n",
" x = self.relu(x)\n",
" x = self.l2(x)\n",
" return x\n",
" \n"
]
},
{
"cell_type": "code",
"execution_count": 128,
"id": "1040ee57",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T04:13:26.388248Z",
"start_time": "2024-04-02T04:13:26.383984Z"
}
},
"outputs": [],
"source": [
"x_sample = torch.linspace(-2, 2, 5).reshape(-1, 1)\n",
"\n",
"model = MyFirstNeuralNet()\n",
"\n",
"state_dict = OrderedDict([\n",
" ('l1.weight', torch.tensor([[1.],[-1.]])),\n",
" ('l1.bias', torch.tensor([-1., 1.])),\n",
" ('l2.weight', torch.tensor([[1., 1.]])),\n",
" ('l2.bias', torch.tensor([0.]))\n",
"])\n",
"\n",
"model.load_state_dict(state_dict)\n",
"\n",
"student1 = model.forward(x_sample).detach().numpy()\n",
"output1 = [[3.], [2.], [1.], [0.], [1.]]\n",
"\n",
"assert allclose(student1, output1, atol=1e-5)"
]
},
{
"cell_type": "markdown",
"id": "b897a701",
"metadata": {},
"source": [
"### 3.1.3 Demo - Optimisers in PyTorch\n",
"\n",
"Optimizers in PyTorch are used to update the parameters of a model during training. They do this by computing the gradients of the model's parameters with respect to the loss function, and then using these gradients to update the parameters in a way that minimizes the loss. \n",
"\n",
"In the following code example, we will simply demo a few basic functionalities of optimisers. Only in 3.1.4 Demo will you see an actual optimizer at work to train a Neural Net.\n",
"\n",
"We first create a tensor x with requires_grad set to True. Next, we define our loss function to be the simple equation y = x ** 2 + 2 * x. Next, we define an optimiser (in this case, Stochastic Gradient Descent, SGD) and pass it our tensor x as a parameter to optimise. After updating the gradient stored in x using `backward()`, we will call the `step()` function to let the optimiser update x. We will then set the gradient of our tensor x back to zero using `zero_grad()`.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "7c2a8f0e",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T04:57:09.043600Z",
"start_time": "2024-04-02T04:57:08.881544Z"
}
},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'torch' is not defined",
"output_type": "error",
"traceback": [
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
"\u001B[0;31mNameError\u001B[0m Traceback (most recent call last)",
"Cell \u001B[0;32mIn[1], line 1\u001B[0m\n\u001B[0;32m----> 1\u001B[0m x \u001B[38;5;241m=\u001B[39m \u001B[43mtorch\u001B[49m\u001B[38;5;241m.\u001B[39mtensor([\u001B[38;5;241m1.0\u001B[39m], requires_grad\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mTrue\u001B[39;00m)\n\u001B[1;32m 3\u001B[0m \u001B[38;5;66;03m#Loss function\u001B[39;00m\n\u001B[1;32m 4\u001B[0m y \u001B[38;5;241m=\u001B[39m x \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39m \u001B[38;5;241m2\u001B[39m \u001B[38;5;241m+\u001B[39m \u001B[38;5;241m2\u001B[39m \u001B[38;5;241m*\u001B[39m x\n",
"\u001B[0;31mNameError\u001B[0m: name 'torch' is not defined"
]
}
],
"source": [
"x = torch.tensor([1.0], requires_grad=True)\n",
"\n",
"#Loss function\n",
"y = x ** 2 + 2 * x\n",
"\n",
"# Define an optimizer, pass it our tensor x to update\n",
"optimiser = torch.optim.SGD([x], lr=0.1)\n",
"\n",
"# Perform backpropagation\n",
"y.backward()\n",
"\n",
"print(\"Value of x before it is updated by optimiser: \", x)\n",
"print(\"Gradient stored in x after backpropagation: \", x.grad)\n",
"\n",
"# Call the step function on the optimizer to update weight\n",
"optimiser.step()\n",
"\n",
"#Weight update, x = x - lr * x.grad = 1.0 - 0.1 * 4.0 = 0.60\n",
"print(\"Value of x after it is updated by optimiser: \", x)\n",
"\n",
"# Set gradient of weight to zero\n",
"optimiser.zero_grad()\n",
"print(\"Gradient stored in x after zero_grad is called: \", x.grad)\n"
]
},
{
"cell_type": "markdown",
"id": "65c27fd8",
"metadata": {},
"source": [
"### 3.1.4 Demo - Training Your First Neural Net\n",
"\n",
"Now, let's make use of an optimiser to train our neural network in Task 3.1!\n",
"\n",
"Take note, if you make changes to your model (e.g. fix any bugs in your forward pass), then you will have to re-run your previous cell to update the model definition.\n",
"\n",
"In the example below, we are applying what we have learnt in the above section about optimisers to train our neural network.\n",
"\n",
"We will using `torch.optim.SGD(model.parameters(), lr=1e-3, momentum=0)` as the optimiser. This SGD optimiser will implement stochastic gradient descent for us. As mentioned previously, `optimiser.zero_grad()` will set all the gradients to zero to prevent accumulation of all the previous old gradients we have calculated using backpropagation. `optimiser.step()` causes our optimiser to update the model weights based on the gradients of our parameters.\n",
"\n",
"We can see clearly from our example below that we are calling `optimiser.zero_grad()` at the start of the loop so we can clear the gradient from the previous iteration of backpropagation. Then after we compute the loss in the current iteration using our loss function and model predictions, y_pred, we will call `loss.backward()` to let pytorch carry out the backpropagation for us. After backpropagation, gradients for each of our parameters will be computed for us to update our model weights using `optimiser.step()`."
]
},
{
"cell_type": "code",
"execution_count": 137,
"id": "587d9e4d",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T04:20:54.253141Z",
"start_time": "2024-04-02T04:20:53.150715Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch\tLoss\t\n",
"-----\t----\n",
" 1000\t0.3594755232334137\n",
" 2000\t0.13189829885959625\n",
" 3000\t0.05977959558367729\n",
" 4000\t0.03069353848695755\n",
" 5000\t0.016396140679717064\n",
" 6000\t0.00869276374578476\n",
" 7000\t0.004500675015151501\n",
" 8000\t0.0022789021022617817\n",
" 9000\t0.0011381806107237935\n",
"10000\t0.0005686104414053261\n"
]
},
{
"data": {
"text/plain": "<Figure size 640x480 with 1 Axes>",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGxCAYAAABfrt1aAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABYqklEQVR4nO3dd1QU198G8GeWDtKRskqzoUaj2DUWLLG3FHsjdsWexJaiMSrRaOIbxYqixthii4qJYhQ1gsbeo1FQFCR2QNAFdu/7h3F/QQFpy+wuz+ecOSc7c2fmO3vJ7uOUu5IQQoCIiIhIBgq5CyAiIqKSi0GEiIiIZMMgQkRERLJhECEiIiLZMIgQERGRbBhEiIiISDYMIkRERCQbBhEiIiKSDYMIERERyYZBhIzO6tWrIUlSttMnn3yCmzdvQpIkrF69WrtOVFQUpk+fjidPnuRpH9OnT4ckSXB1dUVKSspry318fNCxY8cs817W8M033+RY88mTJ/N1rIbqZR9ERkYW6XbXrl2Lnj17ws/PDwqFAj4+Pjm29fHxwfTp0/O03YULF6JChQowNzeHJEl5/jvRhT179uRYt4+PDwIDA4u1HqLCMpW7ACJdCQsLQ+XKlbPMUyqVcHNzQ3R0NMqXL6+dHxUVha+++gqBgYFwcHDI8z7u37+PuXPn4uuvv87zOt988w2GDh0KJyenPK9DefPjjz8iMTER9erVg0ajQUZGRqG3efbsWYwZMwaDBw/GgAEDYGpqCltb2yKotmD27NmDkJCQbMPI9u3bYWdnV/xFERUCgwgZrWrVqqFOnTrZLmvQoEGR7KNt27b4/vvvERQUBHd39ze2b9WqFSIjIzFr1izMnz+/SGqg/9m7dy8Uihcnejt27IiLFy8WepuXLl0CAAwZMgT16tUr9PZ0yd/fX+4SiPKNl2aoxHn10sz06dPx6aefAgB8fX21l1Dyctlg5syZyMzMzPMpfj8/PwwaNAghISG4detWgeq/ePEiunTpAkdHR1haWqJmzZpYs2ZNljaRkZGQJAkbNmzAZ599BqVSCTs7O7Rq1QpXr17NdftHjhzRrvuqtWvXQpIknDhxokC15+abb76BQqHArl27sswPDAyEtbU1Lly48MZtvAwhRSUgIAB9+/YFANSvXx+SJGkvfeR0GSQgIAABAQHa1/nti99++w0tW7aEvb09rK2tUaVKFQQHBwN48V6EhIQAQJZLjjdv3syxpri4OPTt2xeurq6wsLBAlSpVMH/+fGg0Gm2bl/9PzJs3D9999x18fX1RqlQpNGzYEMeOHSvgu0eUNwwiZLTUajUyMzOzTNkZPHgwRo8eDQDYtm0boqOjER0djVq1ar1xH97e3hg5ciRWrlyJa9eu5amu6dOnw8TEBF988UXeD+ZfV69eRaNGjXDp0iX88MMP2LZtG6pWrYrAwEDMnTv3tfZTp07FrVu3EBoaiuXLl+Pvv/9Gp06doFarc9xHkyZN4O/vr/3C+69Fixahbt26qFu3LgBACPHae5zT9CaTJk1Cu3btMGDAAG1ICwsLw5o1a7Bw4UJUr149r29TkVm8eDE+//xzbS3R0dEF6jcgb32xcuVKtG/fHhqNBkuXLsWuXbswZswY3LlzBwDwxRdf4MMPPwQA7d9pdHQ0PDw8st3n/fv30ahRI+zbtw9ff/01du7ciVatWuGTTz7BqFGjXmsfEhKCiIgILFiwAD/99BNSU1PRvn17JCUlFeiYifJEEBmZsLAwASDbKSMjQ8TGxgoAIiwsTLvOt99+KwCI2NjYPO1j2rRpAoC4f/++ePDggbC3txcffPCBdrm3t7fo0KFDlnUAiKCgICGEEJ999plQKBTi3LlzWWo+ceJErvvt2bOnsLCwEHFxcVnmt2vXTlhbW4snT54IIYQ4ePCgACDat2+fpd3mzZsFABEdHZ3rfl7Wc+bMGe28P//8UwAQa9asea1dXqb/etkHBw8ezDL/wYMHomzZsqJevXri9OnTwtraWvTt2zfXWnPSoUMH4e3tneNyb29vMW3atDduJ6e+8fb2FgMGDHitfbNmzUSzZs20r/PaFykpKcLOzk40btxYaDSaHOsJCgp67f3MqabJkycLAOL48eNZ2o0YMUJIkiSuXr0qhPhff1SvXl1kZmZq273s8w0bNuRYD1Fh8YwIGa21a9fixIkTWSZT06K/LcrZ2RmTJk3C1q1bcfz48TytM3HiRDg5OWHSpEn52teBAwfQsmVLeHp6ZpkfGBiItLQ0REdHZ5nfuXPnLK/ffvttAHjjZaFevXrB1dU1y1mRhQsXonTp0ujRo4d2XqdOnV57j3Oa8sLZ2RmbNm3C6dOn0ahRI3h5eWHp0qVZ2rx6puu/lxj02Zv6IioqCsnJyRg5ciQkSSqSfR44cABVq1Z97d6WwMBACCFw4MCBLPM7dOgAExOTHGsk0gXerEpGq0qVKjnerFrUxo0bh0WLFmHixIk4dOjQG9vb2dnh888/x7hx43Dw4ME87+fhw4fZnoZXKpXa5f/l7Oyc5bWFhQUA4NmzZ7nux8LCAsOGDcP8+fPx7bffIiMjA5s3b8aECRO02wAAJycn2Nvb57n+vKhfvz7eeustnDt3DiNGjICNjU2W5eXLl8/yxTht2rQ836Mjpzf1xf379wEAZcuWLbJ9Pnz4MNtHmIv674WoMHhGhKgIWFlZYfr06Th8+DDCw8PztM6IESPg6+uLSZMmQQiRp3WcnZ1x9+7d1+YnJCQAAFxcXPJedB7qy8jIwKpVq7BixQpkZmZi+PDhWdqsWbMGZmZmeZryatq0abhw4QJq166NL7/8EjExMVmW79q1K8uZlqFDhxbJ8eaXpaUlVCrVa/MfPHhQoO2VLl0aALT3gxSF4vx7ISoonhEhQtH8y2/gwIH4/vvvMXny5DxdLjA3N8fMmTPRp0+fPH8htGzZEtu3b0dCQoL2X7XAi8tQ1tbWRfZYMgB4eHigW7duWLx4MdLT09GpUyd4eXllafPy0kxRiYiIQHBwsPZsUc2aNdGjRw8cPXoU5ubmACDLTavZ8fHxwfnz57PMu3btGq5evVqgL/hGjRrB3t4eS5cuRc+ePXO8PPPfv1UrK6tct9myZUsEBwfj9OnTWW6+fvn0U/PmzfNdJ1FRYxAhwv++3P7v//4PAwYMgJmZGfz8/PI1cJWJiQlmz56N9957D8D/rq/nplevXpg3bx5+/fXXPO1j2rRp2L17N5o3b44vv/wSTk5O+OmnnxAeHo65c+cW+WWSsWPHon79+gBePDXyKmdn59dO5xfU3bt30bdvXzRr1gzTpk2DQqHApk2b0LRpU0ycOBELFix44zYuX76My5cvAwASExORlpaGLVu2AACqVq2KqlWrFkmtANCvXz/07dsXI0eOxAcffIBbt25h7ty52jMb+VWqVCnMnz8fgwcPRqtWrTBkyBC4ubnh+vXrOHfuHBYtWgTgf3+rc+bMQbt27WBiYoK3335bG9T+a/z48Vi7di06dOiAGTNmwNvbG+Hh4Vi8eDFGjBiBSpUqFfwNICoivDRDhBdjP0yZMgW7du1C48aNUbduXZw6dSrf2+natSsaNWqU5/aSJGHOnDl5bu/n54eoqCj4+fkhKCgIXbt2xcWLFxEWFqYdC6Uo1atXDz4+PqhSpQpatmxZ5Nt/Sa1Wo1evXpAkCevXr9eOB9KgQQPMnj0b//d//4cdO3a8cTubN29Gt27d0K1bN5w6dQr379/Xvt68eXOR1ty7d2/MnTsXe/fuRceOHbFkyRIsWbKkUF/ugwYNwp49e6BWqzF48GB07NgRCxYsyHImqnfv3hg8eDAWL16Mhg0bom7dutpLLa8qXbo0oqKi0KJFC0yZMgUdO3bE3r17MXfuXCxcuLDAdRIVJUnk9eI0EZU458+fR40aNRASEoKRI0cW2XZv3rwJX19fHDx4MMvgX8Xp5eBfhnCjK5Ex46UZInrNjRs3cOvWLUydOhUeHh78ITUi0hlemiGi13z99dd499138fTpU/z888+wtraWuyQiMlI8I0JEr1m9erX2t3iIiHSJ94gQERGRbHhphoiIiGTDIEJERESy0et7RDQaDRISEmBra1tkPwJFREREuiWEQEpKCpRKpXZcoJzodRBJSEh47VdGiYiIyDDcvn37jT/kqNdB5OXw2rdv34adnZ3M1RAREVFeJCcnw9PTM08/k6HXQeTl5Rg7OzsGESIiIgOTl9sqeLMqERERyYZBhIiIiGTDIEJERESy0et7RIiIyHAIIZCZmQm1Wi13KVQMzMzMYGJiUujtMIgQEVGhpaen4+7du0hLS5O7FComkiShbNmyKFWqVKG2wyBCRESFotFoEBsbCxMTEyiVSpibm3MQSiMnhMD9+/dx584dVKxYsVBnRhhEiIioUNLT06HRaODp6Qlra2u5y6FiUrp0ady8eRMZGRmFCiK8WZWIiIrEm4byJuNSVGe9+FdDREREsmEQISIiItkwiBARUYkVGRkJHx8fne8nMDAQ06dP1/l+DBGDCBERUSEsX74cAQEBsLOzgyRJePLkidwlGRQGESIiokJIS0tD27ZtMXXqVLlLMUh8fJeIiIqcEALPMop/hFUrM5MCP82xdu1ajBw5EmfOnEHFihUBAKNHj8bevXtx5swZ2NjYZLveuHHjALy4zEP5xyBCRERF7lmGGlW/3Fvs+708ow2szQv21da/f3/s3r0bffr0QVRUFPbv349ly5bh6NGjOYYQKjxemiEiIvrXsmXLcPfuXYwZMwaBgYGYNm0a6tatK3dZRo1nRIiIqMhZmZng8ow2suy3MBwdHbFy5Uq0adMGjRo1wuTJk7XLZs+ejdmzZ2tfX758GV5eXoXaHzGIEBGRDkiSVOBLJHI7fPgwTExMkJCQgNTUVNjZ2QEAhg8fju7du2vbKZVKuUo0Krw0Q0RE9K+oqCjMnTsXu3btgp2dHUaPHq1d5uTkhAoVKmgnU1PDDFr6hu8iERERgJSUFPTr1w+jR49Gu3bt4OXlhTp16qBjx47o1q1bjuslJiYiMTER169fBwBcuHABtra28PLygpOTU3GVb7B4RoSIiAjA2LFjYWNjo70P5K233sKcOXMwfPhwxMfH57je0qVL4e/vjyFDhgAAmjZtCn9/f+zcubNY6jZ0DCJEREQAVq1ahfPnz8PCwkI7b8yYMXj48CHKlCmT43rTp0+HEOK1KTAwsBiqNnwMIkRERCQbBhEiIiKSDW9WJSKiEsvHx0c7RLsude3aFQ4ODjrfjyFiECEiohKrOIMIZY+XZoiIiEg2DCJEREQkGwYRIiIikg2DCBEREcmGQYSIiIhkU+AgcvjwYXTq1AlKpRKSJGHHjh3aZRkZGZg0aRKqV68OGxsbKJVK9O/fHwkJCUVRMxERERmJAgeR1NRU1KhRA4sWLXptWVpaGk6fPo0vvvgCp0+fxrZt23Dt2jV07ty5UMUSERHRCz4+PliwYIHcZRRagccRadeuHdq1a5ftMnt7e0RERGSZt3DhQtSrVw9xcXHw8vIq6G6JiIj0llqthiRJUCj0484HfasnO8VWWVJSEiRJynVkOZVKheTk5CwTERGRrgQEBGDUqFEYNWoUHBwc4OzsjM8//xxCCABAeno6Jk6ciDJlysDGxgb169dHZGSkdv3Vq1fDwcEBu3fvRtWqVWFhYYFbt25BpVJh4sSJ8PT0hIWFBSpWrIiVK1dq17t8+TLat2+PUqVKwc3NDf369cODBw/yXFdAQABu3bqF8ePHQ5IkSJKUaz2PHz9G//794ejoCGtra7Rr1w5///33a8exd+9eVKlSBaVKlULbtm1x9+5dXb79AIopiDx//hyTJ09G7969YWdnl2O74OBg2NvbaydPT8/iKI+IiHQlPTXnKeN5Pto+e3PbAlqzZg1MTU1x/Phx/PDDD/j+++8RGhoKAPjoo49w9OhRbNy4EefPn0e3bt3Qtm3bLF/iaWlpCA4ORmhoKC5dugRXV1f0798fGzduxA8//IArV65g6dKlKFWqFADg7t27aNasGWrWrImTJ0/it99+wz///IPu3bvnua5t27ahbNmymDFjBu7evZslMGRXT2BgIE6ePImdO3ciOjoaQgi0b98eGRkZWdabN28efvzxRxw+fBhxcXH45JNPCvy+5pkoAgDE9u3bs12Wnp4uunTpIvz9/UVSUlKu23n+/LlISkrSTrdv3xYA3rgeERHJ59mzZ+Ly5cvi2bNnry+cZpfztO7DrG1nuufcdlX7rG3n+L7epgCaNWsmqlSpIjQajXbepEmTRJUqVcT169eFJEkiPj4+yzotW7YUU6ZMEUIIERYWJgCIs2fPapdfvXpVABARERHZ7vOLL74QrVu3zjLv5ffd1atX31jXS97e3uL777/Psp3s6rl27ZoAII4ePaqd9+DBA2FlZSU2b96cZb3r169r24SEhAg3N7dsj0GI3Ps9KSkpz9/fOv2tmYyMDHTv3h2xsbE4cOBArmdDAMDCwgIWFha6LImIiCiLBg0aaC9tAEDDhg0xf/58nDx5EkIIVKpUKUt7lUoFZ2dn7Wtzc3O8/fbb2tdnz56FiYkJmjVrlu3+Tp06hYMHD2rPkPzXjRs3tPvLqS61Wg0TE5Mcj+fVeq5cuQJTU1PUr19fO8/Z2Rl+fn64cuWKdp61tTXKly+vfe3h4YF79+7luJ+iorMg8jKE/P333zh48GCWTiMiohJiai7DNkivfJl+ej2Xtq/cSTDuQsFrygcTExOcOnXqtS/+/4YIKyurLIHBysoq121qNBp06tQJc+bMeW2Zh4dHISt+vR7x730lrxJCZGlnZmaWZbkkSTmuW5QKHESePn2K69f/90cTGxuLs2fPwsnJCUqlEh9++CFOnz6N3bt3Q61WIzExEQDg5OQEc3PzwldORET6z9xG/rZvcOzYsddeV6xYEf7+/lCr1bh37x6aNGmS5+1Vr14dGo0Ghw4dQqtWrV5bXqtWLWzduhU+Pj4wNc35azinul6GInNzc6jV6jfWU7VqVWRmZuL48eNo1KgRAODhw4e4du0aqlSpkufj0pUC36x68uRJ+Pv7w9/fHwAwYcIE+Pv748svv8SdO3ewc+dO3LlzBzVr1oSHh4d2ioqKKrLiiYiICuv27duYMGECrl69ig0bNmDhwoUYO3YsKlWqhD59+qB///7Ytm0bYmNjceLECcyZMwd79uzJcXs+Pj4YMGAABg4ciB07diA2NhaRkZHYvHkzACAoKAiPHj1Cr1698OeffyImJgb79u3DwIEDswSLnOr6734OHz6M+Pj4LE/cvKpixYro0qULhgwZgj/++APnzp1D3759UaZMGXTp0qUI3sHCKfAZkYCAgFxP2RTH6RwiIqLC6t+/P549e4Z69erBxMQEo0ePxtChQwEAYWFhmDlzJj7++GPEx8fD2dkZDRs2RPv27XPd5pIlSzB16lSMHDkSDx8+hJeXF6ZOnQoAUCqVOHr0KCZNmoQ2bdpApVLB29sbbdu2zTLeR251AcCMGTMwbNgwlC9fHiqVKtfv3bCwMIwdOxYdO3ZEeno6mjZtij179rx2OUYOktDjxJCcnAx7e3skJSW98UZXIiKSx/PnzxEbGwtfX19YWlrKXU6+BAQEoGbNmno3Qqm+1vVfufV7fr6/9XeoNSIiIjJ6DCJEREQkG52OI0JERKTP/jtcuz7R17p0gWdEiIiISDYMIkREVCT0+NkH0oGi6m8GESIiKpSXj4CmpaXJXAkVp/T0dADIdbj5vOA9IkREVCgmJiZwcHDQ/i6JtbV1lqHDyfhoNBrcv38f1tbWuY4OmxcMIkREVGju7u4AUCw/kkb6QaFQwMvLq9Chk0GEiIgKTZIkeHh4wNXVFRkZGXKXQ8XA3Nw8y0iwBcUgQkRERcbExKTQ9wxQycKbVYmIiEg2DCJEREQkGwYRIiIikk2JDSIb/4xDYtzfcpdBRERUopXIILLxzzhE71gC51UN8OjYOrnLISIiKrFK5FMzTSuVhoV1DMwyM2H/22g8ggSnBn3kLouIiKjEKZFnRJQOVqgftAo7Td6FCTSw/20UHh37Se6yiIiISpwSGUQAQOlogzqj1mBXljCyXu6yiIiISpQSG0SAF2Gk9qg12G3S6t8wEoRHxzfIXRYREVGJUaKDCPAijNQatRbhJi1hAg0iIvYg/skzucsiIiIqEUp8EAFehBH/UT9iuvknmPS0B3otP8YwQkREVAwYRP6ldLTB0JGfwMvJBnGP0tB/2RHcv3hA7rKIiIiMGoPIfygdrLBxaAOUdzTD1Kez4bjlAzw8sVnusoiIiIwWg8grlA5W+HFII2SY28MUGtiHD8PDEz/LXRYREZFRYhDJhtKpFN4O+gm/mQT8L4yc3Cp3WUREREaHQSQHL8PIXpNmMIUa9ruHMIwQEREVMQaRXCidSqF60HpEmDT9Xxg5tV3usoiIiIwGg8gbKJ1K4a2g9YgwaYLnwgxf7LvLR3uJiIiKCINIHiidbPFW0AaMsPoWe5K8Oc4IERFREWEQySOlky3mDO8GLydrxD1KwxdL1+Ph6V/kLouIiMigMYjkw8txRho6JGH+sy9ht3MgwwgREVEhMIjkk9LBCvOHdsIZ07dhhkzY7hyIB2d2yl0WERGRQWIQKQClky0qj9yEgyaNYI5M2P3yER6c2SV3WURERAanwEHk8OHD6NSpE5RKJSRJwo4dO7IsF0Jg+vTpUCqVsLKyQkBAAC5dulTYevWG0tkOfiM3IdKk4b9hJBAPzuyWuywiIiKDUuAgkpqaiho1amDRokXZLp87dy6+++47LFq0CCdOnIC7uzveffddpKSkFLhYfaN0tkOlkZu1YcT2l0Dcv3xI7rKIiIgMRoGDSLt27TBz5ky8//77ry0TQmDBggX47LPP8P7776NatWpYs2YN0tLSsH79+kIVrG9ehpFDJg1wRlMBfXal8dFeIiKiPNLJPSKxsbFITExE69attfMsLCzQrFkzREVF5bieSqVCcnJylskQKJ3tUHHkz/jSZhquPdZwnBEiIqI80kkQSUxMBAC4ubllme/m5qZdlp3g4GDY29trJ09PT12UpxNKZzusHhagHWckPORjPDj3q9xlERER6TWdPjUjSVKW10KI1+b915QpU5CUlKSdbt++rcvyitzLcUYC7U5haMZPsN3eHw/O75W7LCIiIr2lkyDi7u4OAK+d/bh3795rZ0n+y8LCAnZ2dlkmQ6N0sMLQYWPxh0k9WCAdttv6MowQERHlQCdBxNfXF+7u7oiIiNDOS09Px6FDh9CoUSNd7FKvKJ3tUW7kFhw1qfufMLJP7rKIiIj0ToGDyNOnT3H27FmcPXsWwIsbVM+ePYu4uDhIkoRx48Zh9uzZ2L59Oy5evIjAwEBYW1ujd+/eRVW7XlM628N3xBZEKerAAukota0P7l+IePOKREREJYgkhBAFWTEyMhLNmzd/bf6AAQOwevVqCCHw1VdfYdmyZXj8+DHq16+PkJAQVKtWLc/7SE5Ohr29PZKSkgzyMg0AJDx4gpuLP0AjzUk8gwWeDPkTHmV85C6LiIhIZ/Lz/V3gIFIcjCGIAP8LI3tU1XHYvgs2DG2AMg5WcpdFRESkE/n5/uZvzRQDpYsDfEbvxGH7Loh7lPZinJHHaXKXRUREJDsGkWKidLTBxqEN4OVkjeRH/+Dhwha4f/F3ucsiIiKSFYNIMXo5zshnpXbhbc0VlNrSC/cvHpC7LCIiItkwiBQzpYMV3hm+CH8qasIKKths6YX7lw7KXRYREZEsGERkoHRxRNmR23FCUQPWeA6bn3vi/qVIucsiIiIqdgwiMlG6OKHMyB3/CSM9cP/yIbnLIiIiKlYMIjJ6NYyk/jwS8Y+eyl0WERFRsWEQkZnSxQllR2zHbybN0f/5x+gVegLxT57JXRYREVGxYBDRAx6lnfH26A2Ao692nJGEew/kLouIiEjnGET0xMtHe72crFHhyR+wXuyPe1f+kLssIiIinWIQ0SNKBytsHFIfwyx/hwOSYb2pG+5dOSp3WURERDrDIKJnlI7W8BqxBWcUb6EU0mC96UOGESIiMloMInrIo7QL3EfsxNn/hpG/GEaIiMj4MIjoKY/SLnAbvhPnFFVRCmmw2vgh7v0VJXdZRERERYpBRI95uLrAdfgunFNUgS3S8MeWhXy0l4iIisyucwlIS8+UtQYGET33MowsM+uHT572Rq/lxxhGiIio0FYeuooxG04hMOwE0jM1stXBIGIAPFxLo1PQtyjrVApxj9LQZ9lRJN68LHdZRERkoFYd+gte+4djhulqNPB1gpmJJFstDCIG4uU4Iz6OFhj79DtYr34X964dl7ssIiIyMGGRV+C9fzjeNTmNXuZHML6mBpLEIEJ5oHSwwoaPaqKC2UPY4Sks17+Pe9f+lLssIiIyEGGRV+Dz+zC0NDmDDIUFTPpsguRaRdaaGEQMjIerC1yG7cJFhR/s8BQW69/HvWsn5C6LiIj0XFjkZZT7fSiam5xDhsICpn03QyrfXO6yGEQMkYebK5yH7cZFhR/skQKL9e8xjBARUY5Cj8Rg5d4/UVkRhwyFJUz7bYVULkDusgAwiBisF2FkFy4pKmnDyD9/n5S7LCIi0jOhR2IwM/wK7ojS2FNrBUz7b4Pk20TusrQYRAyYh5sbnIbtxmVFRViK5/h2yyE+2ktERFphkZewZ88vAIAxLSsisPO7kHzekbmqrBhEDJyHmxsch+7GJ5bTsSWpMscZISIiAMDqyEuo/PtgbDCfhfm1HmJ8q4qyPh2TEwYRI+Dh7o4pIwbBy8kacY/S8OnSbfjn+mm5yyIiIpmsPngRVQ4MREOTy4CpOd5v4KeXIQRgEDEaL8cZaeCQhO+efQaLdV0YRoiISqA1By+g6sGBqK/4CyoTG5gH7oDkVV/usnLEIGJElA5W+D6wOZ6YOMEByTBf15VhhIioBFlz8DzeOjgQ9RRX8dykFMw/2gnJs57cZeWKQcTIeLh7wH5oOK5K5eCIpBdh5MYZucsiIiIdW3PwAqof/Ah1FNfw3MQWFgN3QSpbR+6y3ohBxAh5uHvAblg4rv0bRsx+7IJ/bpyVuywiItKR0CMxmLE3FreFK56b2sFi0C5IZWrJXVaeMIgYKQ93JWz/DSNOSILpj13wT8wFucsiIqIi9nKcEDVMENNkPixGREJS+stdVp4xiBix/4URXyRoHDDw51g+2ktEZETWHjiLJ7/NhgIajGlZEePerQLJubzcZeWLqdwFkG55uCuBoeEYt/Y0Lj9WoNfyY9gwtAHKOFjJXRoRERXCjwfOwD8yENXNbqK5txlqteqot4/o5oZnREoAD48yCB3WSjvOyLrFX+OfmPNyl0VERAX044HTqBU5ANUVN5Fm5ohaXUYZZAgBGERKjJfjjAyyO4FJ6SEwXduZ94wQERmgH38/hTqRA/CW4hZSzZxgNeRXSG5vyV1WgTGIlCBKBysM+WgwbkjecMZjmPzYCf/EXpS7LCIiyqN1v59EnUMDUEURh1QzJ1gP+RWSaxW5yyoUnQaRzMxMfP755/D19YWVlRXKlSuHGTNmQKPR6HK3lAt3j7KwGRKOG5I3XMRjKNZ2QmLsJbnLIiKiN1h5+G/UODQIVRS38dTMBdZDf4PkWlnusgpNp0Fkzpw5WLp0KRYtWoQrV65g7ty5+Pbbb7Fw4UJd7pbewF3pCZsh4YiRvFBaPILJ2o4MI0REeiz0SAy+3nMNCzI/QJKFEjbDfoNU2k/usoqEToNIdHQ0unTpgg4dOsDHxwcffvghWrdujZMnT2bbXqVSITk5OctEuuGu9IT1f8KIYm0nJCTelbssIiJ6xctxQgDgreY9YffJGUguFWWuqujoNIg0btwYv//+O65duwYAOHfuHP744w+0b98+2/bBwcGwt7fXTp6enrosr8RzV3rBekg4YiVPrE1vgZ5r/+I4I0REemT9/mOotK8/PKV/MKZlRYxvVRGSmaXcZRUpSQghdLVxIQSmTp2KOXPmwMTEBGq1GrNmzcKUKVOyba9SqaBSqbSvk5OT4enpiaSkJNjZ2emqzBLv7v0H6BF2AXGP0uDlZM1xRoiI9MD6iGNocGQAyikSccfOH2XGHYCkMIxnTJKTk2Fvb5+n72+dHtGmTZuwbt06rF+/HqdPn8aaNWswb948rFmzJtv2FhYWsLOzyzKR7nmUdsHGoQ3g5WSNB48e4fLCbki89ZfcZRERlVjrI6LR8Eh/lFMkIsnCA2UCwwwmhOSXTs+IeHp6YvLkyQgKCtLOmzlzJtatW4e//nrzF11+EhUVXsKTZzizsA86qH/HP5ILRGA43L0N/45sIiJDsiHiKBodCYS34h6SLJSwG/4rJEcfucvKF705I5KWlgbFKwnOxMSEj+/qKaWDFWoP/A63pDJwEw+A1R15ZoSIqBht3PcH3jkyAN6Ke3hiWQZ2I/YaXAjJL50GkU6dOmHWrFkIDw/HzZs3sX37dnz33Xd47733dLlbKgT3Mj6wGLwHt6QycBf3IVZ3ROKtq3KXRURk9EKPxMDpyJfwUtzHE8uysB++F5KDl9xl6ZxOL82kpKTgiy++wPbt23Hv3j0olUr06tULX375JczNzd+4Pi/NyCcx/ibSQ9vBSyTgrlQaUmA43L2N45l1IiJ98/IRXUckY3OZTajQfxEk+7Jyl1Vg+fn+1mkQKSwGEXkl3olF+sr28BIJuKKoALvRR1DG0VrusoiIjMrqgxcxfe8tAPjfI7oG+gN2L+nNPSJk2NzL+sJ80B5cUFTG+GeD0WvFcY4zQkRUhDbvjcS7kZ3R2+R3owkh+cUgQrlyL+sL59EHkeZYGXGP0tBr+THEP06VuywiIoO3ee9BNIn6CGWkh5hgdwDjA7xKXAgBGEQoD5SO1tpxRtwfn8KzHxoh8fZ1ucsiIjJYP/92AM2iAuEhPcIDK184j9xrdCOm5hWDCOWJ0sEKG4fUwyzLH1FB3IR6VXuGESKiAvj5t9/RLPojuElP8MC6PJyD9kGydZO7LNkwiFCeKR1tYDtwK+IlN5QR/yBzVQfcvX1D7rKIiAzGlt/2IyD6I7hKT3DfusKLMyGlXOUuS1YMIpQv7p4VYPpROBIkN5QViVCvao+7d2LkLouISO+FHonBtSNbUFpKwj2binAJ2gupVGm5y5Idgwjlm5tXRZj8N4ysZBghIsrNy3FClqs74vdyE1F65F5INi5yl6UXGESoQF6Ekd24K7mirLiLqNWf8dFeIqJs/Pzb75gffgYAMKZlJbToNxWSjbPMVekPBhEqMDevSlB8FI7tpu0w+WnPF4/2MowQEWlt2/MrWkb3x0qzeZgQ4Fkixwl5EwYRKhQ3r0qoPyoMHk52L8YZWRaNhMS7cpdFRCS77eHhaH58MJykp/C1lzC6mSdDSDYYRKjQlA5WL8YZcbTCwJQlyFzWAnfjb8pdFhGRbLaH70bzP4fAUXqKBNvqcA/aA8nKUe6y9BKDCBUJpYMVNveriLZmZ+AlEpAe2p5hhIhKpB3hu9DizyFwkFKRYFsdHkHhkKwc5C5LbzGIUJFxV3pCCtyNfyQXeIt4pId2wN34W3KXRURUbH7ZvRMt/hwKeykN8bY14BG0B5Klvdxl6TUGESpSbt6VgcBw3JOc4S3uQBXaHnfj4+Qui4hI50KPxGD50dvQQMIdO38og3ZDsuQvx78JgwgVOTfvyhADXoQRH3EHqtB2DCNEZNRejhNySfjgl1orUSZoF0NIHjGIkE64+VSBGLAb9yRneGri8cOaDXy0l4iM0q5d27Bnzy8AgDEtK6J/57aQLGxlrspwMIiQzrj5VIUYsBvTLCZiQ3I1jjNCREZn184taHFyONaYz8GMBoLjhBQAgwjplJtPVYwcOR5eTtaIe5SG0UvDcTfhttxlEREV2u6dm9Hi1EjYSCo8cqiOfu2bM4QUAIMI6dzLcUZqO6RhftpUPFvRnmGEiAxa+M7NaHFqFGwkFW46NIBX0C+QzG3kLssgMYhQsVA6WGFxjyqwNUlHORGHtBUdGEaIyCDt+WUDWpwKgrWkQqxDQ3gH7YBkbi13WQaLQYSKjZtvNaj77cIDyRHlxa0XYeTuHbnLIiLKsx3hu9Di9GhYSemIcXwHPkHbIZlZyV2WQWMQoWLlVq461P124SFehJHU5QwjRGQYQo/EYOIRNaI1VXHDsTF8R25jCCkCDCJU7NzKVYe6/048hAMqiJt4urwD7t6Nl7ssIqIcvRwnJB1mON84BOVGboVkZil3WUaBQYRk4VrubW0YUahVGLmWj/YSkX7au30NMvd+CUBgTMuKGNO6GkNIEWIQIdm4lqsBdf+d+MRmFs48tuQ4I0Skd/ZtC0Pzs+Mx3HQXllT/m+OE6ACDCMnKtVwNhAxrrx1nZOHiH5CQmCB3WURE2LdtFQLOfQxzSY1rzi3RtkcQQ4gOMIiQ7F6OMxJodwqzVMFIWdYBCYl35S6LiEqwiG0rEXDuE5hLalx1boWKIzZCMjWXuyyjxCBCekHpYIWRPTohWbKDn4hB8nKGESKSx/6tKxBw7lOYS2r85dIalUYyhOgSgwjpDdfy/sjouwOPYY/KmhtIWt4RCYmJcpdFRCXI+ohoNDk/GWaSGldc2sBvxHpIJmZyl2XUGERIr7hWqIWMvjvwBHaoormOpOUdGEaIqFiEHonB1N8fYUrGYFxy7YDKDCHFgkGE9I5rhVpI77MDT2D7bxjpiIT7D+Qui4iM2MpDVzEz/AoAoGzzQag64idIJqYyV1UyMIiQXnKtWBvpfX7BE9jiaHoF9Fx1no/2EpFOHPx5EZr8/h5K4zHGtKzIR3SLGYMI6S3XirXxfNAhrLUdirjHzzjOCBEVucjNP6Dpxc9RSRGPHyqeZQiRAYMI6TV3z/LYOKwhvJyskfgoCQcXBSHhn3tyl0VERuDQ5gVoeulLmEgC593eQ4OP5jCEyEDnQSQ+Ph59+/aFs7MzrK2tUbNmTZw6dUrXuyUj8nKckYU2q9A3cyseLevIMEJEhXJ403docmk6FJLAWfcPUH3YSkgKE7nLKpF0GkQeP36Md955B2ZmZvj1119x+fJlzJ8/Hw4ODrrcLRkhpYMV/Ht8jmSUQjXNVYYRIiqwIxvnoemVr/4NId1QY2goQ4iMJCGE0NXGJ0+ejKNHj+LIkSN5aq9SqaBSqbSvk5OT4enpiaSkJNjZ2emqTDIg968dh8X692GHp7igqAzn4buhdC0td1lEZCDCIq+g2YGuKKdIxGmPHvAfshSSgncpFLXk5GTY29vn6ftbp+/+zp07UadOHXTr1g2urq7w9/fHihUrcmwfHBwMe3t77eTp6anL8sgAla5UH6peW5ECG1TX/IWHSzsi4d59ucsiIgMQeiQGX/0Wg97pn+EPr5EMIXpCp2dELC1f/EzyhAkT0K1bN/z5558YN24cli1bhv79+7/WnmdEKK/uX42G5YYPYItUHFPUgueYPSjjYCV3WUSkpzbvjcTEg6kAwEd0i0F+zojoNIiYm5ujTp06iIqK0s4bM2YMTpw4gejo6Deun58DoZLn/tVoaDb2xcjnQbjv6I8NQxswjBDRa6LWz0K9q/MwNmMUyjfvxxBSDPTm0oyHhweqVq2aZV6VKlUQFxeny91SCVHaryHUQadw39EfcY/SOM4IEb0m+qev0ejaXJhKGvQvl8wQood0GkTeeecdXL16Ncu8a9euwdvbW5e7pRJE6eKAjUMbwMvJGraPL+PGwq4cDp6IAADHfvoKDf+eBwA4XjYQ9QZ+zxCih3QaRMaPH49jx45h9uzZuH79OtavX4/ly5cjKChIl7ulEkbpYIWNg2pjheX/oan6GP5Z0plhhKiEO75uGhr8/R0A4FjZgS9CCG9M1Us67ZW6deti+/bt2LBhA6pVq4avv/4aCxYsQJ8+fXS5WyqBlM52MOsehqewhr/mEhKXdGEYISqh/vzxC9S/vgAAEO05BPUHzmcI0WM6vVm1sHizKuXX/St/wHrTh7DBM5xWVIf7iF+gLO0sd1lEVExCD9+A2b5JGGAagSjPoWg4cC4vx8hAb25WJSpupas0RlqPLUiFFWppLvx7ZuSh3GURUTEIPRKDmXv+wrTMQGyruoAhxEAwiJDRKV2lMdK6/6wNI5HLPubTNERG7sDPizEn/AIAYEzLSnivWyBDiIFgECGjVLpqE6R134yjJnXx9dNOfLSXyFgJgdNrPkWLS1OwyOwHjGlRno/oGhgGETJapas2he/oXSjt5PS/cUYepchdFhEVFSFwZs0nqBW7HABgWb4Rxr/rxxBiYBhEyKgpHay044y8n7wWCYs6IP7BI7nLIqLCEgJnV0+A/81QAECkz3g0HTCDIcQAMYiQ0VM6WOHnnmUx1HQP6mrOIX7xewwjRIZMCJxbPQ41b60CABz0nYBmA6YxhBgoBhEqEdy8KiKt20akwRL1NGdxZ/H7iH/wWO6yiKgAzqz5BDVurQYAHCj3KQL6f8kQYsAYRKjEcHmrOdI+3IA0WKK+5gzuLH6PYYTIwIQeiUHwVQ+kCgv8Xm4imvf7jCHEwDGIUIniUq0F0j5cj2ewQH3NGdzmmREigxF6JAYzw6/gT1EFP9bbgRb9pjKEGAEGESpxXKq1ROoHL8JIA81pLFy+nI/2Eumzf29M/XnPXgDAmJYVMax9Q4YQI8EgQiWSS/VWSP1gPeaZjcDG5GocZ4RIXwmBy6FDUfPmSqwzn41PmnlwnBAjwyBCJZZL9VboHTQNXk7WiHuUhiHL9iP+wRO5yyKilzQaXAkdjKrxm6EREk5XHIugtv4MIUaGQYRKtJfjjFRzzMS3qZ/h5uIPGEaI9MG/IaRK/BZohIR9lb5E6z4fM4QYIQYRKvGUDlZY3dEeFRR38Y7mJG4u+ZBhhEhOGg3+Ch2IKglboRYS9laajja9xzOEGCkGESIALm8FIOX9dVDBHO+oTyB2STfEP0ySuyyiEun4+q9QOWH7ixDiNwNte49lCDFiDCJE/3J5uw2evvcjVDBHY/WfiFn8IcMIUTELPRKDgRer45imCn7zm4l2vUYzhBg5BhGi/3Cu0RZP31sLFczQRP0nbizmmRGiYqHRaMcJSYUVohqvRvteQQwhJQCDCNErnGu0w9OuL8JIhcy/MXbFXj7aS6RL6kxcX9Ybj38LBvBinBD+im7JwSBClA3nmu2R8v5PmGA1Cyef2HCcESJdUWfixvLeqPDPrxhnuhVfNjLnOCElDIMIUQ5c3m6D74a/px1nZMaStYh/mCx3WUTGQ52BmGW9UP6fvUgXJthTZQ4+6tSKIaSEYRAhysXLcUZ62F/CD8+n4u/F3RlGiIqCOgMxy3qi3L19L0JI1bno3GMwQ0gJxCBC9AZKBytMbFsZkgQEqKNxbXEPhhGiwshMR+zS7ih3bz9UwhThVb9Fl+6DGEJKKAYRojxw9u+MlM6rkAFTNFdH4eriHoh/lCJ3WUQGae+u9fC9fwAqYYo9Veeha/eBDCElGIMIUR451+qC5H/DSAt1FK6GMIwQ5VfokRgMO14a0zIGIPyt+ejaPZAhpIRjECHKh6xh5Cj+CunJMEKUF5kqrD1wDjPDrwAA7ANG4b1uAxhCiEGEKL+ca3VBcqeVyIApHqcr0GfFcT7aS5SbjOe4veQ91IwcADs8fTFOCB/RpX8xiBAVgHPtrnjSazcWlRqLm49VHGeEKCf/hhDPh0dRUYrHlLoKhhDKgkGEqIBK+zXE+mHvwMvJGncePcWGkK8Q/+ip3GUR6Y+MZ7izpAs8H0UhTVhgV/Uf0PP9DxlCKAsGEaJCeDnOyCKbVfgkYwkuhfRmGCECgPQ0xC/ujLKPjiH13xDS7YOeDCH0GgYRokJSOlihQbs+yIQJWqsP4SLDCJV06alIWNIZZR7/iafCErurL0T3D3owhFC2GESIioBTnQ+R3GEZMqFAG/UhXAjpwzBCJdbGyNNQPLqOFGGF3W8vQvcPujOEUI4YRIiKiFPdbtow0lYdifMhfRlGqMQJPRKDyQeS0Sv9c+yuEYIevCeE3oBBhKgIOdXtjuT2S5EJBdqpD+Lk4oF8moZKBlUKdu3aqh0npFOLpuj53vsMIfRGDCJERcypXg8kt1+CNFhiW1pNPtpLxu95Mv5Z3AGtTw5DE8V5jhNC+VJsQSQ4OBiSJGHcuHHFtUsi2TjV64nkoScR69AQcY/SGEbIeD1Pwj+L28Mt6Ryewwxt61RhCKF8KZYgcuLECSxfvhxvv/12ceyOSC+4Kz2xcWgDeDlZQ/H4BiIXjUT841S5yyIqOs+e4N7i9nBLvoAnwga7ai5F7/e6MIRQvug8iDx9+hR9+vTBihUr4OjoqOvdEekVpYMVNg6sgY2W36BP5jacXjSAYYSMw7MnuLekPVyTL+KxKIVdNZehT9fODCGUbzoPIkFBQejQoQNatWr1xrYqlQrJyclZJiJDp3RxgkWbr6CGAp3UETjFMEKG7nky7i9uB9fkS3gkSmGX/zL07dqRIYQKRKdBZOPGjTh9+jSCg4Pz1D44OBj29vbaydPTU5flERUbxwZ9kNx2IdRQoLM6AqdCAhlGyGCt/PMfHH3igIfCFrv9l6Nflw4MIVRgOgsit2/fxtixY7Fu3TpYWlrmaZ0pU6YgKSlJO92+fVtX5REVO8cGfZHc5ocXYSRzH06GfMQwQgYn9EgMvt5zDR9njMD2Oj+iX5f2DCFUKJIQQuhiwzt27MB7770HExMT7Ty1Wg1JkqBQKKBSqbIsy05ycjLs7e2RlJQEOzs7XZRJVOweR/8I+72joYDAErP+6Bw0F2UcrOQuiyh3qQ9xesscfHilCTRQ8BFdylV+vr9NdVVEy5YtceHChSzzPvroI1SuXBmTJk16YwghMlaODfvhMQQSI37A0pTG2LD8GDYMbcAwQvor9QEeLWmLWk//xmTTO3jabDpDCBUZnQURW1tbVKtWLcs8GxsbODs7vzafqKRxbNgfz/zeg33oSe04IwwjpJee3sejJW3hlHod/wgHmNUNZAihIsWRVYlkonSy1Y4z0jxpO6IXDUL84zS5yyL6n6f38HhJazilXkeicER47VAEdn6XIYSKlM7uESkKvEeESoJ7MefhsrYpFBDYYtoBDYNCUcbRWu6yqKRLScSTJW3hkBaLu8IJv9ZegY86tWQIoTzJz/c3z4gQycy13NtIenc+NJDwYWY4okOG8MwIyUujxsNlHeGQFot44Yw9tVcyhJDOMIgQ6QHHdwYhudV8AMCHmbsRxTBCMgo9egufPuqCWI0bfqsTioGdmjOEkM4wiBDpCYfGg/Dk3zDSLXM3joYMZRihYhd6JAYzw6/ggKYWfnlnOwZ2ZAgh3WIQIdIjDo0H40nLeQCA7pm7MH/pMv5qLxWPpDtIWNASa/dEAgDGtKyIsa2rMoSQzuns8V0iKhiHJkPwBAI/Hz6LbUmVcJKP9pKuPbmN5GVtoXx2B3NM0xDddA0f0aViwzMiRHrIoclQdAiaDy8na8Q9SsOAZYd5mYZ04/EtpCxtDbtnd3BL44oL9ecwhFCxYhAh0lNKBytsHNoAfo4SZqd+gUMhIxhGqGg9vomUZa1h+zwBsRo37K23CkM6NGEIoWLFIEKkx5QOVtjQMg31FFfRO3MHDoWMZBihovEoFk+XtoHt80Tc0Hggot4qDOnQmCGEih2DCJGec6rzIZ40DwYA9M7cjsiQIIYRKrS49WNQSvUihOyvtxJDOrzDEEKyYBAhMgAOzUbiSfPZAIA+mdtwIGQUwwgVWOiRGHS50xu71Q2wv/4qDO3QiCGEZMMgQmQgHJoF4UnALABAv8yt+D1kNMMI5c/zZO04IY9hh2tNF2Jo+4YMISQrBhEiA+IQMApJzWYCADpm/IZRy3/lOCOUN/evIfU7f8T+thDAi3FC+HQM6QOOI0JkYOybj8YTEzNMiLLAmceW6MVxRuhN7l9F2op2sEl/iD4mv6N008EYyxBCeoJnRIgMkEPT4Zg9vKd2nJFxS3fyzAhl794VPFveFtbpD3FZ443IBqEcMZX0CoMIkYF6Oc5IJ/tYrH42GvsWjWUYoaz+uYxnK9rBKuMRLv0bQka0q8sQQnqFQYTIgCkdrDCzgYCNpMJHmZuwN2Qcwwi9kHgRz0LbwSrjMS5qfHCowUqGENJLDCJEBs6++WgkNZkGABiYsRG/hYxnGCH8uX8zrDKe4LzGF4carsSIdnUYQkgvMYgQGQH7lhOQ1PgLAMCgjA34lWGkRAs9EoPuF+thasYgHGm4EiPb1mYIIb3FIEJkJOxbfYLkxp8DAAZnbMCvIRMYRkqae1ewOvISZoZfASDBJWA4RratxRBCeo1BhMiI2LX6FMnvfAYA8Hp+FX2WHWUYKSniT0O1/F34/T4YllBxnBAyGAwiREbG7t2JeNQhFN+Umoybj1XotfwYw4ixu3MKqrBOsMhMgZmUiVHNfBhCyGAwiBAZIae63bBuWJN/xxlJxfwlSxlGjNWdk0gP6wyLzKf4U+OH6IbLEdTWnyGEDAaDCJGRUjpYYeOQ+phf6id8p5qOXSETGUaMze0/kR7WGebqpziuqYzjjZZjVNuaDCFkUBhEiIyY0tEa79atDgAYnrEWOxlGjEfccaSv7gJzdSqi1VVfhJA2NRhCyOAwiBAZObs2U5HccCIAYETGWvwSMplhxAhsu/gIaZkSjqrfwol3lmF0m7cZQsggMYgQlQB2bT5DcoNPAQAjM1ZjB8OIQQs9EoMJhzX4MH0aTr2zFKPbVGcIIYPFIEJUQti1/RzJDT4BAARlrMb2kCkMI4Ym9gh27dr27zghQPsWzRlCyOAxiBCVIHZtv0BK/QlQQ4GrqTZ8tNeQxEQi88cPEXByJCpLcRwnhIwGgwhRCWPb9ks87Ps7zjm0QtyjNIYRQ3DjIDLXdYep5jlOaPzQLqAxQwgZDQYRopJGkuBaoRY2Dm0ALydrpD+6jbWLZzGM6KvrvyPzpx4w1ajwu9ofFxqHYEzragwhZDQYRIhKKKWDFTYNqIotVrMxJX0htoR8zjCib/7ej8z1PWGqUSFCXQsXGy/CmNZvMYSQUWEQISrBPFxdYV+nGwBgbEYofmYY0R93TkK9vidMNenYp66NSwwhZKQYRIhKMkmCbfsZSKkzGgAwLiMUP4d8wTCiB1bdKIVDmW/hN3VdXG68EGNbV2UIIaPEIEJU0kkSbDt8jZQ6owAA4zJWYHPIlwwjMgo9EoMZv97AiIxx+KvxDwwhZNR0GkSCg4NRt25d2NrawtXVFV27dsXVq1d1uUsiKghJgm2HmUipHQQAGJ+xHKsWf8MwUtyu7Mb5VaMxM/wyAGBYy7cwtnUVhhAyajoNIocOHUJQUBCOHTuGiIgIZGZmonXr1khNTdXlbomoICQJth1n4WntkbgllcHO5Ip8tLc4Xd4JzeYBeDtuLboojnKcECoxJCGEKK6d3b9/H66urjh06BCaNm36xvbJycmwt7dHUlIS7OzsiqFCIoIQuHvvHnqsuYy4R2nwcrLGhqENUMbBSu7KjNelHdBsGQiFUGOHuhFim8zHuHd5JoQMV36+v4v1HpGkpCQAgJOTU7bLVSoVkpOTs0xEVMwkCR5ubtpxRmo92Yf1i7/imRFdubhVG0K2qRsjtsl3DCFUohRbEBFCYMKECWjcuDGqVauWbZvg4GDY29trJ09Pz+Iqj4heoXSwwtYulvjOfAk+TV+CnxbPYBgpahe2QLNlMBRCjS3qprjVZB7GvVuZIYRKlGILIqNGjcL58+exYcOGHNtMmTIFSUlJ2un27dvFVR4RZaN0pQZI8x8CAJiYvhjrFn/NMFJUntyGetswKKDB5sxmuN1kLkMIlUimxbGT0aNHY+fOnTh8+DDKli2bYzsLCwtYWFgUR0lElBeShFKd5+KpECh1dgUmpYdgzmKg78gveM9IIYVeyMBF1VDUU/yFxKbBGP+uH0MIlUg6PSMihMCoUaOwbds2HDhwAL6+vrrcHRHpgiShVJdv8bTmYADAp6rF+HHxTJ4ZKajMdIQeicHM8CvYoWmMxGZzGEKoRNNpEAkKCsK6deuwfv162NraIjExEYmJiXj2jB9gRAZFklCqyzw8rTkICklgoioEM5asZRjJrzPr8Pi7elgWHgUAfESXCDp+fDen/7nCwsIQGBj4xvX5+C6RnhECT3dMQPilB5j0tAe8nGz4aG9enV4LsXMMJAh8m9EdJgGfMoSQ0crP93exjiOSXwwiRHpICCQ8eYaeK46/GGfE0QobhjVkGMnNyTBg9zgAwOrM1njUdCbGv1uJIYSMlt6OI0JERkCSoHS0xsahDVDe0QxfPP0aqxcH8zJNTk6s1IaQVZltGUKIXsEgQkQFonSwwtYGN/CuyWlMUf2AMIaR1/25AgifAABYkdkeT5rOYAghegWDCBEVmEOTYUit3h8KSWCq6gf+UN5/ZTxD0sEfAADLMjsgpel0hhCibDCIEFHBKRSwee//kFqt379h5P+wavEchhEAocfuos3jTzErozdSm05jCCHKAYMIERWOQgGb939AarW+MJEEpqoWYGVJDiMP/taOE5IIZ1gFjGcIIcoFgwgRFZ5CAZv3FyK1Wh+YSAJjVcswZNn+khdGjv4ATUh9nP11FQCOE0KUFwwiRFQ0FArYvL8IT2sOxhTLL3H5sQK9lh8rOWHkj++BiC+gEGqUlxIYQojyiEGEiIqOQoFSXefj8xGB8HKyRtyjNAxcdtD4w8iR+cD+6QCA7zM+gAiYzBBClEcMIkRU5JQOVtg4tAFaOiRiXdowLFs833jDyKFvgd9nAADmZXRjCCHKJwYRItIJpYMV/q/yFZSWkvGlar5xhpHIOcDBmQCAuRk9oAiYyBBClE8MIkSkM6U6z0Fale4wlTT4UjUfSxd/ZzxhRAhcun4DAPBNRk+YBnzCEEJUAAwiRKQ7ChNYd1uKtCofwlTSYJpqntGEkdA/YtHhemf0TZ8C84CPGUKICohBhIh0S2EC627LtWHkS9V8LFn8vWGGESGAc5uw6tBVzAy/AkBCrebvM4QQFQKDCBHp3sswUvkDmElqtH62B72WRRtWGBHixZMx24ei7P4RkKDhI7pERYBBhIiKh8IE1t1XIPmdzzCz1GeIe/zMcMYZEQKI+AI4ugAAcFRTDaNb+jGEEBUBBhEiKj4KE9i9OxGrhwVoxxmZtHSLfocRIYC9nwFRCwEAX2QEwj5gFEMIURFhECGiYqd0sMLGIfUxrdQvWPVsLBYt/kE/w4gQwG9TgGMhAIDPMgbCMSCIIYSoCDGIEJEslPaW6FFOBXNJja9Uc7BQH8PI/mnA8SUAgCkZg+AcMIIhhKiIMYgQkTwUClj3WIlnlbrAXFJjhmoOfliyUK/CyK5nbyNFWGFSxhCUDhjOEEKkAwwiRCQfE1NY9ViFZxU7vwgjz+fg/5Ys0oswEnokBqOjrNBM9T3cAoYyhBDpCIMIEcnLxBRWPcPwrGInWEiZ+Pr5N/KFEY0G2Pc5tu359d9xQoC+LWszhBDpEIMIEcnvZRip0AEWUiZMn94t/kd7NRpg1xggaiGaHB8GGzzjOCFExYBBhIj0g4kZrHqtwcOuG/CHfSfEPUorvjCiUQM7RwNnfoRaSJiZ0QeDWr7NEEJUDBhEiEh/mJjBuWZ7bBzaAF5O1kh+9A++WRKq2zCiUQO/jALOroNaSBifEQTv5h8xhBAVEwYRItI7SgcrbOpfGT9bBWPe86/w3ZIlugkjGjWwYyRwbj0yhQJjM0bBp/kAhhCiYsQgQkR6ycPFGWV9K8NCysDs57N1E0aOLgDOb0SmUGBMxiiUa96fIYSomDGIEJF+MjWHVe8f8axcW1hIGZj1fDbmL1lapGFkdWYbRKmrYlTGGFRo3o8hhEgGDCJEpL+0YaQNLKUMzH4+C/OWLCtcGNFoALwYJ2T63lvonfEZKjXvwxBCJBMGESLSb6bmsOq9Ds/LtYallIHg5zMxd8mKgoURdQawJRAn107RjhMypmUlhhAiGTGIEJH+MzWHZe91eO77LpIVdjiTXCr/j/ZmpgNbPgIu/4LqN5bDW0rkOCFEeoBBhIgMg6kFLPv8BM3AfYCjb/7GGXkZQq7sgkqYYVjGeHRp0YQhhEgPMIgQkeEwtYC7ZwXtOCMVnvzx5ss0mSrg5wHAX7uhEmYYmjEBbzfvzhBCpCdM5S6AiCi/lA5W2NZRAfvNC5D5XIFJS4BJI4agjINV1oaZKmBTP+DvvXguzDAk42P4N/+AIYRIj/CMCBEZJJeK9aH2bgIrKR3fPJ+Z/QisNw5oQ8igjE8YQoj0ULEEkcWLF8PX1xeWlpaoXbs2jhw5Uhy7JSJjZmYJy74b8dy7OawlFeY8/xo7Qibj8eGlQOJFAEDoPT98kRGIgRmfonbz9xlCiPSQzoPIpk2bMG7cOHz22Wc4c+YMmjRpgnbt2iEuLk7XuyYiY/dKGAnKWA3HA5Pw5MoBhB6JwczwK/hR3Rp1mr/HEEKkpyQhhNDlDurXr49atWphyZIl2nlVqlRB165dERwcnOu6ycnJsLe3R1JSEuzs7HRZJhEZsoznSNk3G6dPH0dauho/q5vhgKYWAPARXSIZ5Of7W6c3q6anp+PUqVOYPHlylvmtW7dGVFTUa+1VKhVUKpX2dXJysi7LIyJjYWYJ2w4zUPGdZ+i5/BjiHqUBYAghMgQ6DSIPHjyAWq2Gm5tblvlubm5ITEx8rX1wcDC++uorXZZEREZM6WCFn4c3xLpjt1C+dCl0qalkCCHSc8Vys+qrHwRCiGw/HKZMmYKkpCTtdPv27eIoj4iMiJudJT5u7Yeu/mUYQogMgE7PiLi4uMDExOS1sx/37t177SwJAFhYWMDCwkKXJREREZEe0ekZEXNzc9SuXRsRERFZ5kdERKBRo0a63DUREREZAJ2PrDphwgT069cPderUQcOGDbF8+XLExcVh+PDhut41ERER6TmdB5EePXrg4cOHmDFjBu7evYtq1aphz5498Pb21vWuiYiISM/pfByRwuA4IkRERIYnP9/f/K0ZIiIikg2DCBEREcmGQYSIiIhkwyBCREREsmEQISIiItkwiBAREZFsGESIiIhINgwiREREJBsGESIiIpINgwgRERHJhkGEiIiIZMMgQkRERLJhECEiIiLZMIgQERGRbBhEiIiISDYMIkRERCQbBhEiIiKSDYMIERERyYZBhIiIiGTDIEJERESyYRAhIiIi2TCIEBERkWwYRIiIiEg2DCJEREQkGwYRIiIikg2DCBEREcmGQYSIiIhkwyBCREREsmEQISIiItkwiBAREZFsGESIiIhINgwiREREJBsGESIiIpINgwgRERHJRmdB5ObNmxg0aBB8fX1hZWWF8uXLY9q0aUhPT9fVLomIiMjAmOpqw3/99Rc0Gg2WLVuGChUq4OLFixgyZAhSU1Mxb948Xe2WiIiIDIgkhBDFtbNvv/0WS5YsQUxMTJ7aJycnw97eHklJSbCzs9NxdURERFQU8vP9rbMzItlJSkqCk5NTjstVKhVUKpX2dXJycnGURURERDIptptVb9y4gYULF2L48OE5tgkODoa9vb128vT0LK7yiIiISAb5DiLTp0+HJEm5TidPnsyyTkJCAtq2bYtu3bph8ODBOW57ypQpSEpK0k63b9/O/xERERGRwcj3PSIPHjzAgwcPcm3j4+MDS0tLAC9CSPPmzVG/fn2sXr0aCkXesw/vESEiIjI8Or1HxMXFBS4uLnlqGx8fj+bNm6N27doICwvLVwghIiIi46ezm1UTEhIQEBAALy8vzJs3D/fv39cuc3d319VuiYiIyIDoLIjs27cP169fx/Xr11G2bNksy4rxiWEiIiLSYzq7VhIYGAghRLYTEREREcDfmiEiIiIZMYgQERGRbBhEiIiISDYMIkRERCQbBhEiIiKSDYMIERERyYZBhIiIiGSjswHNisLLMUeSk5NlroSIiIjy6uX3dl7GDtPrIJKSkgIA8PT0lLkSIiIiyq+UlBTY29vn2ibfv75bnDQaDRISEmBrawtJkop028nJyfD09MTt27eN8pd9eXyGz9iP0diPDzD+Y+TxGT5dHaMQAikpKVAqlW/8wVu9PiOiUChe+52aomZnZ2e0f2AAj88YGPsxGvvxAcZ/jDw+w6eLY3zTmZCXeLMqERERyYZBhIiIiGRTYoOIhYUFpk2bBgsLC7lL0Qken+Ez9mM09uMDjP8YeXyGTx+OUa9vViUiIiLjVmLPiBAREZH8GESIiIhINgwiREREJBsGESIiIpINgwgRERHJxmiDyKxZs9CoUSNYW1vDwcEh2zZxcXHo1KkTbGxs4OLigjFjxiA9PT3X7apUKowePRouLi6wsbFB586dcefOHR0cQf5ERkZCkqRspxMnTuS4XmBg4GvtGzRoUIyV552Pj89rtU6ePDnXdYQQmD59OpRKJaysrBAQEIBLly4VU8V5d/PmTQwaNAi+vr6wsrJC+fLlMW3atDf+Pep7/y1evBi+vr6wtLRE7dq1ceTIkVzbHzp0CLVr14alpSXKlSuHpUuXFlOl+RccHIy6devC1tYWrq6u6Nq1K65evZrrOjn9f/rXX38VU9V5N3369NfqdHd3z3UdQ+q/7D5PJElCUFBQtu0Noe8OHz6MTp06QalUQpIk7NixI8vygn4ebt26FVWrVoWFhQWqVq2K7du3F2ndRhtE0tPT0a1bN4wYMSLb5Wq1Gh06dEBqair++OMPbNy4EVu3bsXHH3+c63bHjRuH7du3Y+PGjfjjjz/w9OlTdOzYEWq1WheHkWeNGjXC3bt3s0yDBw+Gj48P6tSpk+u6bdu2zbLenj17iqnq/JsxY0aWWj///PNc28+dOxffffcdFi1ahBMnTsDd3R3vvvuu9gcV9cVff/0FjUaDZcuW4dKlS/j++++xdOlSTJ069Y3r6mv/bdq0CePGjcNnn32GM2fOoEmTJmjXrh3i4uKybR8bG4v27dujSZMmOHPmDKZOnYoxY8Zg69atxVx53hw6dAhBQUE4duwYIiIikJmZidatWyM1NfWN6169ejVLn1WsWLEYKs6/t956K0udFy5cyLGtofXfiRMnshxbREQEAKBbt265rqfPfZeamooaNWpg0aJF2S4vyOdhdHQ0evTogX79+uHcuXPo168funfvjuPHjxdd4cLIhYWFCXt7+9fm79mzRygUChEfH6+dt2HDBmFhYSGSkpKy3daTJ0+EmZmZ2Lhxo3ZefHy8UCgU4rfffivy2gsjPT1duLq6ihkzZuTabsCAAaJLly7FU1QheXt7i++//z7P7TUajXB3dxfffPONdt7z58+Fvb29WLp0qQ4qLFpz584Vvr6+ubbR5/6rV6+eGD58eJZ5lStXFpMnT862/cSJE0XlypWzzBs2bJho0KCBzmosSvfu3RMAxKFDh3Jsc/DgQQFAPH78uPgKK6Bp06aJGjVq5Lm9offf2LFjRfny5YVGo8l2uSH1nRBCABDbt2/Xvi7o52H37t1F27Zts8xr06aN6NmzZ5HVarRnRN4kOjoa1apVg1Kp1M5r06YNVCoVTp06le06p06dQkZGBlq3bq2dp1QqUa1aNURFRem85vzYuXMnHjx4gMDAwDe2jYyMhKurKypVqoQhQ4bg3r17ui+wgObMmQNnZ2fUrFkTs2bNyvXSRWxsLBITE7P0l4WFBZo1a6Z3/ZWdpKQkODk5vbGdPvZfeno6Tp06leW9B4DWrVvn+N5HR0e/1r5NmzY4efIkMjIydFZrUUlKSgKAPPWZv78/PDw80LJlSxw8eFDXpRXY33//DaVSCV9fX/Ts2RMxMTE5tjXk/ktPT8e6deswcODAN/7Su6H03asK+nmYU78W5WdoiQ0iiYmJcHNzyzLP0dER5ubmSExMzHEdc3NzODo6Zpnv5uaW4zpyWblyJdq0aQNPT89c27Vr1w4//fQTDhw4gPnz5+PEiRNo0aIFVCpVMVWad2PHjsXGjRtx8OBBjBo1CgsWLMDIkSNzbP+yT17tZ33sr1fduHEDCxcuxPDhw3Ntp6/99+DBA6jV6ny999n9P+nm5obMzEw8ePBAZ7UWBSEEJkyYgMaNG6NatWo5tvPw8MDy5cuxdetWbNu2DX5+fmjZsiUOHz5cjNXmTf369bF27Vrs3bsXK1asQGJiIho1aoSHDx9m296Q+2/Hjh148uRJrv9wM6S+y05BPw9z6tei/Aw1LbItFYPp06fjq6++yrXNiRMn3nhPxEvZJV8hxBsTcVGsk1cFOeY7d+5g79692Lx58xu336NHD+1/V6tWDXXq1IG3tzfCw8Px/vvvF7zwPMrP8Y0fP1477+2334ajoyM+/PBD7VmSnLzaN7rsr1cVpP8SEhLQtm1bdOvWDYMHD851Xbn7703y+95n1z67+fpm1KhROH/+PP74449c2/n5+cHPz0/7umHDhrh9+zbmzZuHpk2b6rrMfGnXrp32v6tXr46GDRuifPnyWLNmDSZMmJDtOobafytXrkS7du2ynCF/lSH1XW4K8nmo689Qgwoio0aNQs+ePXNt4+Pjk6dtubu7v3azzePHj5GRkfFa+vvvOunp6Xj8+HGWsyL37t1Do0aN8rTf/CrIMYeFhcHZ2RmdO3fO9/48PDzg7e2Nv//+O9/rFkRh+vTl0yHXr1/PNoi8vMM/MTERHh4e2vn37t3LsY+LWn6PLyEhAc2bN0fDhg2xfPnyfO+vuPsvJy4uLjAxMXntX025vffu7u7Ztjc1Nc01aMpt9OjR2LlzJw4fPoyyZcvme/0GDRpg3bp1OqisaNnY2KB69eo5/m0Zav/dunUL+/fvx7Zt2/K9rqH0HVDwz8Oc+rUoP0MNKoi4uLjAxcWlSLbVsGFDzJo1C3fv3tV2yr59+2BhYYHatWtnu07t2rVhZmaGiIgIdO/eHQBw9+5dXLx4EXPnzi2Sul6V32MWQiAsLAz9+/eHmZlZvvf38OFD3L59O8sfqi4Vpk/PnDkDADnW6uvrC3d3d0RERMDf3x/Ai2vBhw4dwpw5cwpWcD7l5/ji4+PRvHlz1K5dG2FhYVAo8n/ltLj7Lyfm5uaoXbs2IiIi8N5772nnR0REoEuXLtmu07BhQ+zatSvLvH379qFOnToF+lvWNSEERo8eje3btyMyMhK+vr4F2s6ZM2dk76+8UKlUuHLlCpo0aZLtckPrv5fCwsLg6uqKDh065HtdQ+k7oOCfhw0bNkRERESWM9L79u0r2n98F9ltr3rm1q1b4syZM+Krr74SpUqVEmfOnBFnzpwRKSkpQgghMjMzRbVq1UTLli3F6dOnxf79+0XZsmXFqFGjtNu4c+eO8PPzE8ePH9fOGz58uChbtqzYv3+/OH36tGjRooWoUaOGyMzMLPZjzM7+/fsFAHH58uVsl/v5+Ylt27YJIYRISUkRH3/8sYiKihKxsbHi4MGDomHDhqJMmTIiOTm5OMt+o6ioKPHdd9+JM2fOiJiYGLFp0yahVCpF586ds7T77/EJIcQ333wj7O3txbZt28SFCxdEr169hIeHh94dX3x8vKhQoYJo0aKFuHPnjrh79652+i9D6r+NGzcKMzMzsXLlSnH58mUxbtw4YWNjI27evCmEEGLy5MmiX79+2vYxMTHC2tpajB8/Xly+fFmsXLlSmJmZiS1btsh1CLkaMWKEsLe3F5GRkVn6Ky0tTdvm1WP8/vvvxfbt28W1a9fExYsXxeTJkwUAsXXrVjkOIVcff/yxiIyMFDExMeLYsWOiY8eOwtbW1mj6Twgh1Gq18PLyEpMmTXptmSH2XUpKiva7DoD2M/PWrVtCiLx9Hvbr1y/Lk21Hjx4VJiYm4ptvvhFXrlwR33zzjTA1NRXHjh0rsrqNNogMGDBAAHhtOnjwoLbNrVu3RIcOHYSVlZVwcnISo0aNEs+fP9cuj42NfW2dZ8+eiVGjRgknJydhZWUlOnbsKOLi4orxyHLXq1cv0ahRoxyXAxBhYWFCCCHS0tJE69atRenSpYWZmZnw8vISAwYM0KvjeenUqVOifv36wt7eXlhaWgo/Pz8xbdo0kZqamqXdf49PiBePrE2bNk24u7sLCwsL0bRpU3HhwoVirv7NwsLCsv17ffXfCobWfyEhIcLb21uYm5uLWrVqZXm0dcCAAaJZs2ZZ2kdGRgp/f39hbm4ufHx8xJIlS4q54rzLqb/++/f36jHOmTNHlC9fXlhaWgpHR0fRuHFjER4eXvzF50GPHj2Eh4eHMDMzE0qlUrz//vvi0qVL2uWG3n9CCLF3714BQFy9evW1ZYbYdy8fMX51GjBggBAib5+HzZo107Z/6eeffxZ+fn7CzMxMVK5cucjDlyTEv3cTERERERWzEvv4LhEREcmPQYSIiIhkwyBCREREsmEQISIiItkwiBAREZFsGESIiIhINgwiREREJBsGESIiIpINgwgRERHJhkGEiIiIZMMgQkRERLL5f9GLnc7x/MBwAAAAAElFTkSuQmCC"
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"torch.manual_seed(6) # Set seed to some fixed value\n",
"\n",
"epochs = 10000\n",
"\n",
"model = MyFirstNeuralNet()\n",
"# the optimizer controls the learning rate\n",
"optimiser = torch.optim.SGD(model.parameters(), lr=1e-3, momentum=0)\n",
"loss_fn = nn.MSELoss()\n",
"\n",
"x = torch.linspace(-10, 10, 1000).reshape(-1, 1)\n",
"y = torch.abs(x-1)\n",
"\n",
"print('Epoch', 'Loss', '\\n-----', '----', sep='\\t')\n",
"for i in range(1, epochs+1):\n",
" # reset gradients to e\n",
" optimiser.zero_grad()\n",
" # get predictions\n",
" y_pred = model(x)\n",
" # compute loss\n",
" loss = loss_fn(y_pred, y)\n",
" # backpropagate\n",
" loss.backward()\n",
" # update the model weights\n",
" optimiser.step()\n",
"\n",
" if i % 1000 == 0:\n",
" print (f\"{i:5d}\", loss.item(), sep='\\t')\n",
"\n",
"y_pred = model(x)\n",
"plt.plot(x, y, linestyle='solid', label='|x-1|')\n",
"plt.plot(x, y_pred.detach().numpy(), linestyle='dashed', label='perceptron')\n",
"plt.axis('equal')\n",
"plt.title('Fit NN on y=|x-1| function')\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "6f845c8d",
"metadata": {},
"source": [
"### 3.2 Concept - Save and load models\n",
"\n",
"Your model weights are stored within the model itself. \n",
"You may save/load the model weights:\n",
"```\n",
"torch.save(model.state_dict(), \"path/to/model_state_dict\")\n",
"\n",
"model = MyFirstNeuralNet()\n",
"model.load_state_dict(torch.load(\"path/to/model_state_dict\"))\n",
"```\n",
"\n",
"Alternatively, you can save/load the entire model using\n",
"```\n",
"torch.save(model, \"path/to/model\")\n",
"\n",
"model = torch.load(\"path/to/model\")\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "ac548df6",
"metadata": {},
"source": [
"### Task 3.2 - Model weights\n",
"\n",
"For this task, you will print out the trained model's `.state_dict()` and submit this to Coursemology.\n",
"\n",
"*Note: An acceptable loss value should be less than 1.0. If your loss is greater than 1, try re-running with a different random initialization, or adjust your model configuration.*"
]
},
{
"cell_type": "code",
"execution_count": 142,
"id": "481bb0cc",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T04:23:56.764275Z",
"start_time": "2024-04-02T04:23:56.758484Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--- Submit the OrderedDict below ---\n",
"OrderedDict([('l1.weight', tensor([[ 1.],\n",
" [-1.]])), ('l1.bias', tensor([-1., 1.])), ('l2.weight', tensor([[1., 1.]])), ('l2.bias', tensor([0.]))])\n"
]
}
],
"source": [
"# To submit this output\n",
"print(\"--- Submit the OrderedDict below ---\")\n",
"print(model.state_dict())"
]
},
{
"cell_type": "code",
"execution_count": 143,
"id": "d6e437cc",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T04:23:59.123744Z",
"start_time": "2024-04-02T04:23:59.119881Z"
}
},
"outputs": [],
"source": [
"def get_loss(model):\n",
" model.load_state_dict(state_dict)\n",
" x = torch.linspace(-10, 10, 1000).reshape(-1, 1)\n",
" y = torch.abs(x-1)\n",
" loss_fn = nn.MSELoss()\n",
" y_pred = model.forward(x)\n",
" return loss_fn(y_pred, y).item()\n",
"\n",
"assert model.load_state_dict(state_dict)\n",
"assert get_loss(model) < 1"
]
},
{
"cell_type": "markdown",
"id": "9151f959",
"metadata": {},
"source": [
"### 3.3 Concept - Using NN to recognize handwritten digits\n",
"\n",
"In the final part of this problem set, we will be building a neural network to classify images to their respective digits. \n",
"\n",
"You will build and train a model on the classic **MNIST Handwritten Digits** dataset. Each grayscale image is a $28 \\times 28$ matrix/tensor that looks like so:\n",
"\n",
"<img src=\"https://upload.wikimedia.org/wikipedia/commons/2/27/MnistExamples.png\" width=\"500\" />\n",
"\n",
"MNIST is a classification problem and the task is to take in an input image and classify them into one of ten buckets: the digits from $0$ to $9$. "
]
},
{
"cell_type": "markdown",
"id": "eda667dc",
"metadata": {},
"source": [
"### 3.3 Demo - Loading an external dataset\n",
"\n",
"The cell below imports the MNIST dataset, which is already pre-split into train and test sets. \n",
"\n",
"The download takes approximately 63MB of space."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "2ce62735",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T04:57:52.145541Z",
"start_time": "2024-04-02T04:57:52.090524Z"
}
},
"outputs": [],
"source": [
"# DO NOT REMOVE THIS CELL – THIS DOWNLOADS THE MNIST DATASET\n",
"# RUN THIS CELL BEFORE YOU RUN THE REST OF THE CELLS BELOW\n",
"from torchvision import datasets\n",
"\n",
"# This downloads the MNIST datasets ~63MB\n",
"mnist_train = datasets.MNIST(\"./\", train=True, download=True)\n",
"mnist_test = datasets.MNIST(\"./\", train=False, download=True)\n",
"\n",
"x_train = mnist_train.data.reshape(-1, 784) / 255\n",
"y_train = mnist_train.targets\n",
" \n",
"x_test = mnist_test.data.reshape(-1, 784) / 255\n",
"y_test = mnist_test.targets"
]
},
{
"cell_type": "markdown",
"id": "e092f6c4",
"metadata": {},
"source": [
"### Task 3.3 - Define the model architechure and implement the forward pass\n",
"Create a 3-layer network in the `__init__` method of the model `DigitNet`. \n",
"These layers are all `Linear` layers and should correspond to the following the architecture:\n",
"\n",
"<img src=\"imgs/img_linear_nn.png\" width=\"600\">\n",
"\n",
"In our data, a given image $x$ has been flattened from a 28x28 image to a 784-length array.\n",
"\n",
"After initializing the layers, stitch them together in the `forward` method. Your network should look like so:\n",
"\n",
"$$x \\rightarrow \\text{Linear(512)} \\rightarrow \\text{ReLU} \\rightarrow \\text{Linear(128)} \\rightarrow \\text{ReLU} \\rightarrow \\text{Linear(10)} \\rightarrow \\text{Softmax} \\rightarrow \\hat{y}$$\n",
"\n",
"**Softmax Layer**: The final softmax activation is commonly used for classification tasks, as it will normalizes the results into a vector of values that follows a probability distribution whose total sums up to 1. The output values are between the range [0,1] which is nice because we are able to avoid binary classification and accommodate as many classes or dimensions in our neural network model.\n",
"\n",
"*Note: When using `torch.softmax(...)` on the final layer, ensure you are applying it on the correct dimension (as you did in NumPy via the `axis` argument in popular methods)*"
]
},
{
"cell_type": "code",
"execution_count": 56,
"id": "596d04f8",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T05:18:11.250227Z",
"start_time": "2024-04-02T05:18:11.246543Z"
}
},
"outputs": [],
"source": [
"class DigitNet(nn.Module):\n",
" def __init__(self, input_dimensions, num_classes): # set the arguments you'd need\n",
" super().__init__()\n",
" \"\"\"\n",
" YOUR CODE HERE\n",
" - DO NOT hardcode the input_dimensions, use the parameter in the function\n",
" - Your network should work for any input and output size \n",
" - Create the 3 layers (and a ReLU layer) using the torch.nn layers API\n",
" \"\"\"\n",
" self.l1 = nn.Linear(input_dimensions, 512)\n",
" self.l2 = nn.Linear(512, 128)\n",
" self.l3 = nn.Linear(128, num_classes)\n",
" self.relu = nn.ReLU()\n",
"\n",
" \n",
" def forward(self, x):\n",
" \"\"\"\n",
" Performs the forward pass for the network.\n",
" \n",
" Parameters\n",
" ----------\n",
" x : Input tensor (batch size is the entire dataset)\n",
"\n",
" Returns\n",
" -------\n",
" The output of the entire 3-layer model.\n",
" \"\"\"\n",
" x = self.l1(x)\n",
" x = self.relu(x)\n",
" x = self.l2(x)\n",
" x = self.relu(x)\n",
" x = self.l3(x)\n",
" return torch.softmax(x, dim=1)\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 64,
"id": "95c1a075",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T05:20:17.458629Z",
"start_time": "2024-04-02T05:20:17.454070Z"
}
},
"outputs": [],
"source": [
"model = DigitNet(784, 10)\n",
"assert [layer.detach().numpy().shape for name, layer in model.named_parameters()] \\\n",
" == [(512, 784), (512,), (128, 512), (128,), (10, 128), (10,)]"
]
},
{
"cell_type": "markdown",
"id": "d356b9ad",
"metadata": {},
"source": [
"### Task 3.4 - Training Loop\n",
"\n",
"As demonstrated in Section 3.2, implement the function `train_model` that performs the following for every epoch/iteration:\n",
"\n",
"1. set the optimizer's gradients to zero\n",
"2. forward pass\n",
"3. calculate the loss\n",
"4. backpropagate using the loss\n",
"5. take an optimzer step to update weights\n",
"\n",
"This time, use the Adam optimiser to train the network.\n",
"<br/>\n",
"<br/>\n",
"Use Cross-Entropy Loss, since we are performing a classification.\n",
"<br/>\n",
"_(PyTorch Softmax normalize logits while CrossEntropyLoss accepts unnormalized logits and CrossEntropyLoss already applies LogSoftmax, however, we will use Softmax here as we want to showcase how Softmax can convert the raw scores produced by the network into a probability distribution over the classes)._\n",
"<br/>\n",
"<br/>\n",
"Train for 20 epochs. \n",
"\n",
"*Note: refer to the command glossary to find out how to instantiate optimisers, losses, and more*"
]
},
{
"cell_type": "code",
"execution_count": 57,
"id": "60ab3632",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T05:18:29.674622Z",
"start_time": "2024-04-02T05:18:15.183827Z"
}
},
"outputs": [],
"source": [
"def train_model(x_train, y_train, epochs=20):\n",
" \"\"\"\n",
" Trains the model for 20 epochs/iterations\n",
" \n",
" Parameters\n",
" ----------\n",
" x_train : A tensor of training features of shape (60000, 784)\n",
" y_train : A tensor of training labels of shape (60000, 1)\n",
" epochs : Number of epochs, default of 20\n",
" \n",
" Returns\n",
" -------\n",
" The final model \n",
" \"\"\"\n",
" model = DigitNet(784, 10)\n",
" optimiser = torch.optim.Adam(model.parameters())\n",
" loss_fn = nn.CrossEntropyLoss()\n",
"\n",
" for i in range(epochs):\n",
" optimiser.zero_grad()\n",
" y_pred = model(x_train)\n",
" \n",
" loss = loss_fn(y_pred, y_train)\n",
" loss.backward()\n",
" optimiser.step()\n",
" return model\n",
" \n",
"digit_model = train_model(x_train, y_train)"
]
},
{
"cell_type": "code",
"outputs": [],
"source": [],
"metadata": {
"collapsed": false
},
"id": "7cf3b5fd0f653e73"
},
{
"cell_type": "code",
"execution_count": 58,
"id": "a99b7049",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T05:18:29.879303Z",
"start_time": "2024-04-02T05:18:29.675746Z"
}
},
"outputs": [],
"source": [
"x_train_new = torch.rand(5, 784, requires_grad=True)\n",
"y_train_new = ones = torch.ones(5, dtype=torch.uint8)\n",
"\n",
"assert type(train_model(x_train_new, y_train_new)) == DigitNet"
]
},
{
"cell_type": "markdown",
"id": "01fdee35",
"metadata": {},
"source": [
"### 3.5 Demo - Explore your model\n",
"\n",
"Now that we have trained the model, let us run some predictions on the model."
]
},
{
"cell_type": "code",
"execution_count": 63,
"id": "6f83aa93",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T05:18:55.537241Z",
"start_time": "2024-04-02T05:18:55.510301Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"true label: 1\n",
"pred label: 1\n"
]
},
{
"data": {
"text/plain": "<Figure size 640x480 with 1 Axes>",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGFCAYAAAASI+9IAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAHgElEQVR4nO3cr4uVWwOG4b0PalBENE3yRxsF/wOjSZsMmExiGsRiM1oEwazJJhpUTEbBpFhExKAYBLtpgqC8Xznc9Zu1cbtn5lxX3g/vSnO7gms+TdM0A4DZbPbPqg8AwM4hCgBEFACIKAAQUQAgogBARAGAiAIA2bfdH87n82WeA4Al287/VXZTACCiAEBEAYCIAgARBQAiCgBEFACIKAAQUQAgogBARAGAiAIAEQUAIgoARBQAiCgAEFEAIKIAQEQBgIgCABEFACIKAEQUAIgoABBRACCiAEBEAYCIAgARBQAiCgBEFACIKAAQUQAgogBARAGAiAIAEQUAIgoARBQAiCgAEFEAIKIAQEQBgIgCABEFACIKAEQUAIgoABBRACCiAEBEAYCIAgARBQAiCgBEFACIKAAQUQAgogBARAGAiAIAEQUAIgoARBQAiCgAEFEAIPtWfQDYzebz+fDmypUrC33r4cOHw5v79+8Pb27cuDG8+fXr1/CGnclNAYCIAgARBQAiCgBEFACIKAAQUQAgogBARAGAiAIAEQUAIgoAZD5N07StHy7w8BfsdWtra8Ob79+/L+Ekf87ly5eHN0+fPl3CSfjTtvPn3k0BgIgCABEFACIKAEQUAIgoABBRACCiAEBEAYCIAgARBQAiCgBk36oPALvZ5ubmqo/wx12/fn1440G8vcNNAYCIAgARBQAiCgBEFACIKAAQUQAgogBARAGAiAIAEQUAIgoARBQAiFdS4V9ra2vDm0uXLi3hJLA6bgoARBQAiCgAEFEAIKIAQEQBgIgCABEFACIKAEQUAIgoABBRACAexIN/nT9/fnizvr4+vJmmaXjzNz158mTVR2CF3BQAiCgAEFEAIKIAQEQBgIgCABEFACIKAEQUAIgoABBRACCiAEA8iMeetLGxMby5du3aEk6yWl++fBnePH78eAknYbdwUwAgogBARAGAiAIAEQUAIgoARBQAiCgAEFEAIKIAQEQBgIgCAPEgHnvSrVu3hjdnz55dwklWa2tra3jz48ePJZyE3cJNAYCIAgARBQAiCgBEFACIKAAQUQAgogBARAGAiAIAEQUAIgoARBQAiFdS2fE2NjaGN2fOnFnCSXafBw8erPoI7DJuCgBEFACIKAAQUQAgogBARAGAiAIAEQUAIgoARBQAiCgAEFEAIB7EY8c7fPjw8Objx4/Dm0Ue0ZvP58ObT58+DW9ms9ns+PHjw5t3794t9C3+u9wUAIgoABBRACCiAEBEAYCIAgARBQAiCgBEFACIKAAQUQAgogBAPIjHjvfhw4fhzYkTJ4Y3+/fvH94s8iDeIg/bzWaz2cWLF4c379+/X+hb/He5KQAQUQAgogBARAGAiAIAEQUAIgoARBQAiCgAEFEAIKIAQEQBgHgQjx1vkYfqDhw4MLxZ5HG7f/4Z/3fVz58/hzez2Wz29u3bhXYwwk0BgIgCABEFACIKAEQUAIgoABBRACCiAEBEAYCIAgARBQAiCgDEg3j8NefOnVto9+LFi+HNwYMHhzfTNA1vfv/+Pbx59uzZ8Ab+FjcFACIKAEQUAIgoABBRACCiAEBEAYCIAgARBQAiCgBEFACIKAAQUQAg82mbT0PO5/Nln4Vd5NChQ8ObN2/eLPSt06dPL7T7G7a2toY3R44cWcJJ4P/bzp97NwUAIgoARBQAiCgAEFEAIKIAQEQBgIgCABEFACIKAEQUAIgoAJB9qz4Au9O9e/eGNzv5YbtFPX/+fNVHgD/KTQGAiAIAEQUAIgoARBQAiCgAEFEAIKIAQEQBgIgCABEFACIKAMSDeCxkfX191UfYEW7fvr3qI8Af5aYAQEQBgIgCABEFACIKAEQUAIgoABBRACCiAEBEAYCIAgARBQDiQTz2pG/fvg1vXr16Nbz5+vXr8AZ2MjcFACIKAEQUAIgoABBRACCiAEBEAYCIAgARBQAiCgBEFACIKAAQD+KxJ929e3d4c/To0SWcBHYXNwUAIgoARBQAiCgAEFEAIKIAQEQBgIgCABEFACIKAEQUAIgoABBRACBeSWUhL1++HN6cPn16oW8dO3ZseHPz5s3hzaNHj4Y3sNe4KQAQUQAgogBARAGAiAIAEQUAIgoARBQAiCgAEFEAIKIAQEQBgHgQj4XcuXNneHPq1KmFvnX16tXhzcmTJ4c38/l8eAN7jZsCABEFACIKAEQUAIgoABBRACCiAEBEAYCIAgARBQAiCgBEFACIB/H4azY3Nxfaff78eXhz4cKF4c3r16+HN7DXuCkAEFEAIKIAQEQBgIgCABEFACIKAEQUAIgoABBRACCiAEBEAYDMp2matvXD+XzZZwFgibbz595NAYCIAgARBQAiCgBEFACIKAAQUQAgogBARAGAiAIAEQUAIgoARBQAiCgAEFEAIKIAQEQBgIgCABEFACIKAEQUAIgoABBRACCiAEBEAYCIAgARBQAiCgBEFACIKAAQUQAgogBARAGAiAIAEQUAIgoARBQAiCgAEFEAIPu2+8NpmpZ5DgB2ADcFACIKAEQUAIgoABBRACCiAEBEAYCIAgARBQDyP8EelCLH7ieYAAAAAElFTkSuQmCC"
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# This is a demonstration: You can use this cell for exploring your trained model\n",
"\n",
"idx = 190 # try on some index\n",
"\n",
"scores = digit_model(x_test[idx:idx+1])\n",
"_, predictions = torch.max(scores, 1)\n",
"print(\"true label:\", y_test[idx].item())\n",
"print(\"pred label:\", predictions[0].item())\n",
"\n",
"plt.imshow(x_test[idx].numpy().reshape(28, 28), cmap='gray')\n",
"plt.axis(\"off\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "fcc94586",
"metadata": {},
"source": [
"### Task 3.5 - Evaluate the model\n",
"\n",
"Now that we have trained the model, we should evaluate it using our test set. \n",
"We will be using the accuracy (whether or not the model predicted the correct label) to measure the model performance. \n",
"\n",
"Since our model takes in a (n x 784) tensor and returns a (n x 10) tensor of probability scores for each of the 10 classes, we need to convert the probability scores into the actual predictions by taking the index of the maximum probability. "
]
},
{
"cell_type": "code",
"execution_count": 84,
"id": "a5684246",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T05:30:31.386136Z",
"start_time": "2024-04-02T05:30:31.326612Z"
}
},
"outputs": [
{
"data": {
"text/plain": "tensor(0.6764)"
},
"execution_count": 84,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def get_accuracy(scores, labels):\n",
" \"\"\"\n",
" Helper function that returns accuracy of model\n",
" \n",
" Parameters\n",
" ----------\n",
" scores : The raw softmax scores of the network\n",
" labels : The ground truth labels\n",
" \n",
" Returns\n",
" -------\n",
" Accuracy of the model. Return a number in range [0, 1].\n",
" 0 means 0% accuracy while 1 means 100% accuracy\n",
" \"\"\"\n",
" idxes = torch.argmax(scores, dim=1)\n",
" ints = (idxes == labels).float()\n",
" return torch.mean(ints)\n",
"\n",
"scores = digit_model(x_test) # n x 10 tensor\n",
"get_accuracy(scores, y_test)"
]
},
{
"cell_type": "code",
"execution_count": 87,
"id": "beafdef0",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-02T05:30:33.432831Z",
"start_time": "2024-04-02T05:30:33.428819Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"passed\n"
]
}
],
"source": [
"scores = torch.tensor([[0.4118, 0.6938, 0.9693, 0.6178, 0.3304, 0.5479, 0.4440, 0.7041, 0.5573, 0.6959],\n",
" [0.9849, 0.2924, 0.4823, 0.6150, 0.4967, 0.4521, 0.0575, 0.0687, 0.0501, 0.0108],\n",
" [0.0343, 0.1212, 0.0490, 0.0310, 0.7192, 0.8067, 0.8379, 0.7694, 0.6694, 0.7203],\n",
" [0.2235, 0.9502, 0.4655, 0.9314, 0.6533, 0.8914, 0.8988, 0.3955, 0.3546, 0.5752],\n",
" [0,0,0,0,0,0,0,0,0,1]])\n",
"y_true = torch.tensor([5, 3, 6, 4, 9])\n",
"acc_true = 0.4\n",
"assert isclose(get_accuracy(scores, y_true),acc_true) , \"Mismatch detected\"\n",
"print(\"passed\")"
]
},
{
"cell_type": "markdown",
"id": "9ce50c78",
"metadata": {},
"source": [
"# Submission\n",
"\n",
"Once you are done, please remember to submit your work to Coursemology, by copying the right snippets of code into the corresponding box that says \"Your answer\", and click \"Save\". After you save, you can make changes to your submission.\n",
"\n",
"Once you are satisfied with what you have uploaded, click \"Finalize submission\". **Note that once your submission is finalized, it is considered to be submitted for grading and cannot be changed.** If you need to undo this action, you will have to reach out to your assigned tutor for help. Please do not finalize your submission until you are sure that you want to submit your solutions for grading. \n",
"\n",
"### HAVE FUN AND ENJOY CODING!"
]
}
],
"metadata": {
"kernelspec": {
"name": "cs2109s",
"language": "python",
"display_name": "CS2109S"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.9"
},
"vscode": {
"interpreter": {
"hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Reference in New Issue View Git Blame Copy Permalink