2050 lines
223 KiB
Plaintext
2050 lines
223 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5c60acfe",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Problem Set 7: Building ConvNets with PyTorch\n",
|
||
"\n",
|
||
"**Release Date:** 02 April 2024\n",
|
||
"\n",
|
||
"**Due Date:** 2359h, 13 April 2024\n",
|
||
"\n",
|
||
"In PS6, you looked at the basics of building learning pipelines with `PyTorch`. Computer Vision, the field that aims to make computers understand and learn visual objects like images, is a hot topic in Machine Learning. Knowledge of building these Computer Vision pipelines is pertinent given the fast-paced nature of the field.\n",
|
||
"\n",
|
||
"<img src=\"imgs/logo.png\" width=\"600\">\n",
|
||
"\n",
|
||
"In *Problem Set 7*, we will take you through data `PyTorch` API for Computer Vision. You will be building a __Convolutional Neural Network__ (CNN/ConvNet) and training it on two datasets, *MNIST* and *CIFAR-10*. You'll also learn how to build __data augmentation pipelines__ to enhance your dataset. Finally, you'll look through the eyes of your ConvNet to see __why__ it's making certain predictions."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "adfd1c67",
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-27T17:03:32.690848Z",
|
||
"start_time": "2024-04-27T17:03:30.945851Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"/tmp/ipykernel_95268/3863716717.py:17: UserWarning: 'has_mps' is deprecated, please use 'torch.backends.mps.is_built()'\n",
|
||
" device = torch.device(\"mps\") if torch.has_mps else torch.device(\"cpu\")\n",
|
||
"/tmp/ipykernel_95268/3863716717.py:18: UserWarning: 'has_mps' is deprecated, please use 'torch.backends.mps.is_built()'\n",
|
||
" torch.has_mps\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": "False"
|
||
},
|
||
"execution_count": 1,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# do not remove this cell\n",
|
||
"# run this cell before moving on\n",
|
||
"\n",
|
||
"# DL libraries\n",
|
||
"import torch\n",
|
||
"import torch.nn as nn \n",
|
||
"import torch.nn.functional as F\n",
|
||
"from torchvision import datasets, transforms\n",
|
||
"\n",
|
||
"# Computational libraries\n",
|
||
"import math\n",
|
||
"import numpy as np\n",
|
||
"\n",
|
||
"# Visualization libraries\n",
|
||
"import matplotlib.pyplot as plt\n",
|
||
"import seaborn as sns\n",
|
||
"device = torch.device(\"mps\") if torch.has_mps else torch.device(\"cpu\")\n",
|
||
"torch.has_mps"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "06e51671",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Chapter 1: Vision Layers\n",
|
||
"\n",
|
||
"Here, you'll be building the two fundamental layers that are the cornerstone of Computer Vision: the convolutional layer and the pooling layer (specifically, __max__ pooling). \n",
|
||
"\n",
|
||
"## Task 1.1: Convolution Under The Hood\n",
|
||
"\n",
|
||
"Your task is to write the `conv2d` function that performs the convolution operation on a 2-dim image, `img : torch.Tensor`, using a certain kernel, `kernel : torch.Tensor`. Assume there is no padding and the stride is 1. \n",
|
||
"\n",
|
||
"> Don't work on the channels – assume they remain the same. Your code should only work on the spatial dimensions: Height and Width.\n",
|
||
"\n",
|
||
"We've given you two images `x1` and `x2` and their convolutional outputs `c1` and `c2` respectively. Run them to verify whether your function is working as expected.\n",
|
||
"\n",
|
||
"$$\n",
|
||
"c1 = \\texttt{conv2d}\\Bigg(\n",
|
||
"\\begin{bmatrix}\n",
|
||
" 4 & 9 & 3 & 0 & 3 \\\\\n",
|
||
" 9 & 7 & 3 & 7 & 3 \\\\\n",
|
||
" 1 & 6 & 6 & 9 & 8 \\\\\n",
|
||
" 6 & 6 & 8 & 4 & 3 \\\\\n",
|
||
" 6 & 9 & 1 & 4 & 4 \\\\\n",
|
||
"\\end{bmatrix},~\n",
|
||
"\\begin{bmatrix}\n",
|
||
" 1 & 1 \\\\\n",
|
||
" 1 & 1\n",
|
||
"\\end{bmatrix}\\Bigg) = \n",
|
||
" \\begin{bmatrix} \n",
|
||
" 4+9+9+7 & 9+3+7+3 & 3+0+3+7 & 0+3+7+3 \\\\\n",
|
||
" 9+7+1+6 & 7+3+6+6 & 3+7+6+9 & 7+3+9+8 \\\\\n",
|
||
" 1+6+6+6 & 6+6+6+8 & 6+9+8+4 & 9+8+4+3 \\\\\n",
|
||
" 6+6+6+9 & 6+8+9+1 & 8+4+1+4 & 4+3+4+4 \\\\\n",
|
||
" \\end{bmatrix} =\n",
|
||
"\\begin{bmatrix} \n",
|
||
" 29 & 22 & 13 & 13 \\\\\n",
|
||
" 23 & 22 & 25 & 27 \\\\\n",
|
||
" 19 & 26 & 27 & 24 \\\\\n",
|
||
" 27 & 24 & 17 & 15 \\\\\n",
|
||
"\\end{bmatrix}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"$$\n",
|
||
"c2 = \\texttt{conv2d}\\Bigg(\n",
|
||
"\\begin{bmatrix}\n",
|
||
" 1 & 9 & 9 & 9 & 0 & 1 \\\\\n",
|
||
" 2 & 3 & 0 & 5 & 5 & 2 \\\\\n",
|
||
" 9 & 1 & 8 & 8 & 3 & 6 \\\\\n",
|
||
" 9 & 1 & 7 & 3 & 5 & 2 \\\\\n",
|
||
" 1 & 0 & 9 & 3 & 1 & 1 \\\\\n",
|
||
" 0 & 3 & 6 & 6 & 7 & 9 \\\\\n",
|
||
"\\end{bmatrix},~\n",
|
||
"\\begin{bmatrix}\n",
|
||
" 6 & 3 & 4 & 5 \\\\\n",
|
||
" 0 & 8 & 2 & 8 \\\\\n",
|
||
" 2 & 7 & 5 & 0 \\\\\n",
|
||
" 0 & 8 & 1 & 9 \\\\\n",
|
||
"\\end{bmatrix}\\Bigg) = \\begin{bmatrix} \n",
|
||
" 285 & 369 & 286 \\\\\n",
|
||
" 230 & 317 & 257 \\\\ \n",
|
||
" 306 & 374 & 344 \\\\\n",
|
||
"\\end{bmatrix}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"__Note:__ You are not allowed to use the `torch.nn.functional.conv2d` function."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "a63a577557da6e87",
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"outputs": [],
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "7873ce10",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-27T17:03:40.731264Z",
|
||
"start_time": "2024-04-27T17:03:40.726572Z"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"torch.manual_seed(0)\n",
|
||
"\n",
|
||
"def conv2d(img, kernel):\n",
|
||
" \"\"\"\n",
|
||
" PARAMS\n",
|
||
" img: the 2-dim image with a specific height and width\n",
|
||
" kernel: a 2-dim kernel (smaller than image dimensions) that convolves the given image\n",
|
||
" \n",
|
||
" RETURNS\n",
|
||
" the convolved 2-dim image\n",
|
||
" \"\"\"\n",
|
||
" iH, iW = img.shape\n",
|
||
" kH, kW = kernel.shape\n",
|
||
" result_shape = (iH-kH+1, iW-kW+1)\n",
|
||
" \n",
|
||
" result = torch.zeros(result_shape)\n",
|
||
" for i in range(result_shape[0]):\n",
|
||
" for j in range(result_shape[1]):\n",
|
||
" result[i, j] = torch.sum(img[i:i+kH, j:j+kW] * kernel)\n",
|
||
" return result\n",
|
||
" \n",
|
||
" \n",
|
||
" \n",
|
||
" # YOUR CODE HERE\n",
|
||
" pass"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"id": "38048231",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-03T08:51:32.066584Z",
|
||
"start_time": "2024-04-03T08:51:32.056798Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"tensor([[29., 22., 13., 13.],\n",
|
||
" [23., 22., 25., 27.],\n",
|
||
" [19., 26., 27., 24.],\n",
|
||
" [27., 24., 17., 15.]]) True\n",
|
||
"tensor([[285., 369., 286.],\n",
|
||
" [230., 317., 257.],\n",
|
||
" [306., 374., 344.]]) True\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"x1 = torch.tensor([\n",
|
||
" [4, 9, 3, 0, 3],\n",
|
||
" [9, 7, 3, 7, 3],\n",
|
||
" [1, 6, 6, 9, 8],\n",
|
||
" [6, 6, 8, 4, 3],\n",
|
||
" [6, 9, 1, 4, 4]\n",
|
||
"])\n",
|
||
"k1 = torch.ones((2, 2))\n",
|
||
"o1 = torch.tensor([\n",
|
||
" [29., 22., 13., 13.],\n",
|
||
" [23., 22., 25., 27.],\n",
|
||
" [19., 26., 27., 24.],\n",
|
||
" [27., 24., 17., 15.]\n",
|
||
"])\n",
|
||
"\n",
|
||
"x2 = torch.tensor([\n",
|
||
" [1, 9, 9, 9, 0, 1],\n",
|
||
" [2, 3, 0, 5, 5, 2],\n",
|
||
" [9, 1, 8, 8, 3, 6],\n",
|
||
" [9, 1, 7, 3, 5, 2],\n",
|
||
" [1, 0, 9, 3, 1, 1],\n",
|
||
" [0, 3, 6, 6, 7, 9]\n",
|
||
"])\n",
|
||
"k2 = torch.tensor([\n",
|
||
" [6, 3, 4, 5],\n",
|
||
" [0, 8, 2, 8],\n",
|
||
" [2, 7, 5, 0],\n",
|
||
" [0, 8, 1, 9]\n",
|
||
"])\n",
|
||
"o2 = torch.tensor([\n",
|
||
" [285., 369., 286.],\n",
|
||
" [230., 317., 257.],\n",
|
||
" [306., 374., 344.]\n",
|
||
"])\n",
|
||
"\n",
|
||
"# TEST YOUR conv2d FUNCTION HERE\n",
|
||
"c1 = conv2d(x1, k1)\n",
|
||
"print(c1, torch.all(torch.eq(c1, o1)).item())\n",
|
||
"c2 = conv2d(x2, k2)\n",
|
||
"print(c2, torch.all(torch.eq(c2, o2)).item())"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2d6a9790",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Task 1.2: Max Pooling Under The Hood\n",
|
||
"\n",
|
||
"Your task is to write the `maxpool2d` function that takes in an image, `img : torch.Tensor`, and a square kernel size `size : int`. Assume stride is 1 and there's no padding.\n",
|
||
"\n",
|
||
"We've given you two images `x1` and `x2` to test your `maxpool2d` function with `size=2` and `size=3` respectively. \n",
|
||
"$$\n",
|
||
"m1 = \\texttt{maxpool2d}\\Bigg(\n",
|
||
"\\begin{bmatrix}\n",
|
||
" 4 & 9 & 3 & 0 & 3 \\\\\n",
|
||
" 9 & 7 & 3 & 7 & 3 \\\\\n",
|
||
" 1 & 6 & 6 & 9 & 8 \\\\\n",
|
||
" 6 & 6 & 8 & 4 & 3 \\\\\n",
|
||
" 6 & 9 & 1 & 4 & 4 \\\\\n",
|
||
"\\end{bmatrix},~2\\Bigg) =\n",
|
||
"\\begin{bmatrix} \n",
|
||
" max(4,9,9,7) & max(9,3,7,3) & max(3,0,3,7) & max(0,3,7,3) \\\\\n",
|
||
" max(9,7,1,6) & max(7,3,6,6) & max(3,7,6,9) & max(7,3,9,8) \\\\\n",
|
||
" max(1,6,6,6) & max(6,6,6,8) & max(6,9,8,4) & max(9,8,4,3) \\\\\n",
|
||
" max(6,6,6,9) & max(6,8,9,1) & max(8,4,1,4) & max(4,3,4,4) \\\\\n",
|
||
"\\end{bmatrix} =\n",
|
||
"\\begin{bmatrix} \n",
|
||
" 9 & 9 & 7 & 7 \\\\\n",
|
||
" 9 & 7 & 9 & 9 \\\\\n",
|
||
" 6 & 8 & 9 & 9 \\\\\n",
|
||
" 9 & 9 & 8 & 4 \\\\\n",
|
||
"\\end{bmatrix}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"$$\n",
|
||
"m2 = \\texttt{maxpool2d}\\Bigg(\n",
|
||
"\\begin{bmatrix}\n",
|
||
" 1 & 9 & 9 & 9 & 0 & 1 \\\\\n",
|
||
" 2 & 3 & 0 & 5 & 5 & 2 \\\\\n",
|
||
" 9 & 1 & 8 & 8 & 3 & 6 \\\\\n",
|
||
" 9 & 1 & 7 & 3 & 5 & 2 \\\\\n",
|
||
" 1 & 0 & 9 & 3 & 1 & 1 \\\\\n",
|
||
" 0 & 3 & 6 & 6 & 7 & 9 \\\\\n",
|
||
"\\end{bmatrix},~3\\Bigg) = \\begin{bmatrix} \n",
|
||
" 9 & 9 & 9 & 9 \\\\\n",
|
||
" 9 & 8 & 8 & 8 \\\\ \n",
|
||
" 9 & 9 & 9 & 8 \\\\\n",
|
||
" 9 & 9 & 9 & 9 \\\\\n",
|
||
"\\end{bmatrix}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"__Note:__ You are not allowed to use the `torch.nn.functional.max_pool2d` function."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"id": "cb5469f7",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-03T08:53:52.828251Z",
|
||
"start_time": "2024-04-03T08:53:52.823369Z"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"torch.manual_seed(0)\n",
|
||
"\n",
|
||
"def maxpool2d(img, size):\n",
|
||
" \"\"\"\n",
|
||
" PARAMS\n",
|
||
" img: the 2-dim image with a specific height and width\n",
|
||
" size: an integer corresponding to the window size for Max Pooling\n",
|
||
" \n",
|
||
" RETURNS\n",
|
||
" the 2-dim output after Max Pooling\n",
|
||
" \"\"\"\n",
|
||
" iH, iW = img.shape\n",
|
||
" result_shape = (iH-size+1, iW-size+1)\n",
|
||
" \n",
|
||
" result = torch.zeros(result_shape)\n",
|
||
" for i in range(result_shape[0]):\n",
|
||
" for j in range(result_shape[1]):\n",
|
||
" result[i, j] = torch.max(img[i:i+size, j:j+size])\n",
|
||
" return result"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 14,
|
||
"id": "cb2954e1",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-03T08:53:54.157039Z",
|
||
"start_time": "2024-04-03T08:53:54.148566Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"tensor([[9., 9., 7., 7.],\n",
|
||
" [9., 7., 9., 9.],\n",
|
||
" [6., 8., 9., 9.],\n",
|
||
" [9., 9., 8., 4.]]) True\n",
|
||
"tensor([[9., 9., 9., 9.],\n",
|
||
" [9., 8., 8., 8.],\n",
|
||
" [9., 9., 9., 8.],\n",
|
||
" [9., 9., 9., 9.]]) True\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"x1 = torch.tensor([\n",
|
||
" [4, 9, 3, 0, 3],\n",
|
||
" [9, 7, 3, 7, 3],\n",
|
||
" [1, 6, 6, 9, 8],\n",
|
||
" [6, 6, 8, 4, 3],\n",
|
||
" [6, 9, 1, 4, 4]\n",
|
||
"])\n",
|
||
"k1 = 2\n",
|
||
"o1 = torch.tensor([\n",
|
||
" [9., 9., 7., 7.],\n",
|
||
" [9., 7., 9., 9.],\n",
|
||
" [6., 8., 9., 9.],\n",
|
||
" [9., 9., 8., 4.]\n",
|
||
"])\n",
|
||
"\n",
|
||
"x2 = torch.tensor([\n",
|
||
" [1, 9, 9, 9, 0, 1],\n",
|
||
" [2, 3, 0, 5, 5, 2],\n",
|
||
" [9, 1, 8, 8, 3, 6],\n",
|
||
" [9, 1, 7, 3, 5, 2],\n",
|
||
" [1, 0, 9, 3, 1, 1],\n",
|
||
" [0, 3, 6, 6, 7, 9]\n",
|
||
"])\n",
|
||
"k2 = 3\n",
|
||
"o2 = torch.tensor([\n",
|
||
" [9., 9., 9., 9.],\n",
|
||
" [9., 8., 8., 8.],\n",
|
||
" [9., 9., 9., 8.],\n",
|
||
" [9., 9., 9., 9.]\n",
|
||
"])\n",
|
||
"\n",
|
||
"# TEST YOUR maxpool2d FUNCTION HERE\n",
|
||
"m1 = maxpool2d(x1, k1)\n",
|
||
"print(m1, torch.all(torch.eq(m1, o1)).item())\n",
|
||
"m2 = maxpool2d(x2, k2)\n",
|
||
"print(m2, torch.all(torch.eq(m2, o2)).item())"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "1933b2f8",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Chapter 2: MNIST Classification with CNNs\n",
|
||
"\n",
|
||
"As done in PS5, we will be working on the MNIST handwritten digits classification problem. This time, however, your images are no longer flattened to form input vectors $\\in \\mathbb{R}^{784}$. You'll be working the images as they are in the form of $1 \\times 28 \\times 28$ tensors, where $28$ is the image height and width, and $1$ is the number of colour channels (grayscale image in this case)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "c78ee5e1",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-27T17:04:36.032494Z",
|
||
"start_time": "2024-04-27T17:04:35.981931Z"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# do not remove this cell\n",
|
||
"# run this before moving on\n",
|
||
"\n",
|
||
"T = transforms.Compose([\n",
|
||
" transforms.ToTensor(),\n",
|
||
" transforms.Normalize([0.5], [0.5])\n",
|
||
"])\n",
|
||
"\n",
|
||
"\"\"\"\n",
|
||
"Note: You can update the path to point to the directory containing `MNIST` \n",
|
||
"directory to avoid downloading the MNIST data again.\n",
|
||
"\"\"\"\n",
|
||
"device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
|
||
"mnist_train = datasets.MNIST(\"./\", train=True, download=True, transform=T)\n",
|
||
"mnist_test = datasets.MNIST(\"./\", train=False, download=True, transform=T)\n",
|
||
"\n",
|
||
"\n",
|
||
"\"\"\"\n",
|
||
"if you feel your computer can't handle too much data, you can reduce the batch\n",
|
||
"size to 64 or 32 accordingly, but it will make training slower. \n",
|
||
"\n",
|
||
"We recommend sticking to 128 but do choose an appropriate batch size that your\n",
|
||
"computer can manage. The training phase tends to require quite a bit of memory.\n",
|
||
"\"\"\"\n",
|
||
"train_loader = torch.utils.data.DataLoader(mnist_train, shuffle=True, batch_size=256)\n",
|
||
"test_loader = torch.utils.data.DataLoader(mnist_test, batch_size=10000)\n",
|
||
"\n",
|
||
"def get_accuracy(scores, labels):\n",
|
||
" ''' accuracy metric '''\n",
|
||
" _, predicted = torch.max(scores.data, 1)\n",
|
||
" correct = (predicted == labels).sum().item() \n",
|
||
" return correct / scores.size(0)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "6e4c7f78",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Concept 1: DataLoaders\n",
|
||
"\n",
|
||
"PyTorch __DataLoaders__ accept datasets and can iterate through the datasets as we deem fit.\n",
|
||
"\n",
|
||
"`train_loader = torch.utils.data.DataLoader(mnist_train, shuffle=True, batch_size=256)` means that this dataloader takes in the MNIST training data, and outputs training features and labels in batches of 256. It also reshuffles all the data in the dataset for the next epoch once it has outputted all the data in the dataset.\n",
|
||
"\n",
|
||
"Run the following code to get a better idea of how dataloaders work."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "4e0fdf18",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-27T17:04:39.560907Z",
|
||
"start_time": "2024-04-27T17:04:39.424591Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Feature batch shape: torch.Size([256, 1, 28, 28])\n",
|
||
"Labels batch shape: torch.Size([256])\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": "<Figure size 640x480 with 1 Axes>",
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGFCAYAAAASI+9IAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAJbklEQVR4nO3cMWieVR/G4ZM2CDHYGCrUCCo2GAex0EEq6FLo1kExDkZLzKAg1CVrwaK4BjsKSqBQiJkK3TNkqLabmkUMVixCFEEFNRmC4XUQbj6ww/t/vuRtTK9rzs15oE1/nsEz1Ov1eg0AWmuH7vYHALB/iAIAIQoAhCgAEKIAQIgCACEKAIQoABDD/f7g0NDQXn4HAHusn/9X2U0BgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBi+2x/A7jp8+HB5c/Xq1fJmamqqvHn22WfLm9Za+/PPPzvtgDo3BQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQDCK6kHzPBw/Y/0yJEj5c2TTz5Z3oyMjJQ3rXklFQbJTQGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgPIh3wJw6daq8OXHixB58CfBf5KYAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEB7EO2Dee++98mZsbKy8WVtbK282NzfLG2Cw3BQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAwoN4+9Tp06c77Z5//vld/pI7u3TpUnmztbW1B18C7CY3BQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYDwIN4+NTk52Wl3+PDhXf6SO1tfXx/IOcBguSkAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEF5JpW1sbAxkA+x/bgoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIA4UG8fero0aMDO+uHH34YyIZ/vPDCC512s7Ozu/wld/bTTz+VN5988kl54+/Q/uSmAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABAexNunXnvttYGd9d133w3srIPm/Pnz5c3Fixc7nTXIRxKr3nzzzfJmcXGx01nvvvtupx39cVMAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQAiKFer9fr6weHhvb6W/gfX331Vafd008/Xd6cO3euvFleXi5vDqLffvutvHnggQf24Evu7PPPPy9vHnroofJmamqqvNnc3CxvWmttbGys047W+vnn3k0BgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBi+2x9wL+jyguQjjzzS6awuL09+8cUXnc7az0ZHR8ubpaWl8ubBBx8sb7755pvyprXW3nnnnfJmZWWlvJmYmChvurzG+thjj5U3rbU2Pz9f3ly6dKnTWfciNwUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGA8CDeADz66KPlzfj4eKezfv755/Km6wNt+9nc3Fx5c/bs2fJma2urvHn//ffLm9a6PW7XxY8//jiQTZffi9Zae+ONN8obD+L1z00BgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIDyIx4H00ksvDeSchYWF8ubTTz/dgy/ZPTMzM+XNM888swdfcmdLS0sDO+te5KYAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEB7EG4DV1dXy5uuvv+501uOPP17ePPfcc+XNzZs3y5uuHn744fLm+PHje/Al//brr78O5JyuDh2q/3ffK6+8Ut6MjIyUNysrK+VNa619+OGHnXb0x00BgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIDyINwA7OzvlzZdfftnprKeeeqq8uXbtWnkzPT1d3ly/fr28aa210dHR8mZ8fLzTWVXHjh0byDldLSwslDcvvvhiebO+vl7eXLhwobxprbW//vqr047+uCkAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEEO9Xq/X1w8ODe31t7ALfv/99/Lm/vvvL29u3LhR3pw/f768aa21tbW18uatt94qbz766KPyZnt7u7z54IMPypvWuv05zc3NlTcTExPlze3bt8ubkydPljetdfs7zj/6+efeTQGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgPIh3wLz66qvlzeXLl8ub4eHh8uaXX34pb1pr7YknnihvdnZ2ypsrV66UN9PT0+VNn79yd83GxkZ5c+bMmfJmfX29vOH/40E8AEpEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAIj6q2bsa8vLy+VNl8cOP/744/Lm6NGj5U1rrd2+fbu8mZ2dLW9u3bpV3ux333//fXkzPz9f3njc7uBwUwAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFACIoV6v1+vrBzs8msbBNTMzU94sLi52Ouu+++7rtBuELr8Xff7K/cvq6mp58/bbb5c33377bXnDf0M/f/fcFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQDCg3gMzOTkZKfdxYsXy5vXX3+901lVf/zxR3nz8ssvdzrrs88+K2+2t7c7ncXB5EE8AEpEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACC8kgpwj/BKKgAlogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQAz3+4O9Xm8vvwOAfcBNAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgPgbLvku86SCoJIAAAAASUVORK5CYII="
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Label: 6\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# no need to code\n",
|
||
"# run this before moving on\n",
|
||
"\n",
|
||
"train_features, train_labels = next(iter(train_loader))\n",
|
||
"print(f\"Feature batch shape: {train_features.size()}\")\n",
|
||
"print(f\"Labels batch shape: {train_labels.size()}\")\n",
|
||
"img = train_features[0].squeeze()\n",
|
||
"label = train_labels[0]\n",
|
||
"plt.imshow(img, cmap=\"gray\")\n",
|
||
"plt.axis(\"off\")\n",
|
||
"plt.show()\n",
|
||
"print(f\"Label: {label}\")"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "aa2edfb3",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Task 2.1: Building a Vanilla ConvNet\n",
|
||
"\n",
|
||
"Your task here is to build a ConvNet using PyTorch layers. You can refer to the attached command glossary to read more about the layers. Use the following architecture:\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\text{Conv(32, (3,3))} \\rightarrow \\text{MP(2,2)} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\text{Conv(64, (3,3))} \\rightarrow \\text{MP(2,2)} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\text{Flat} \\\\ \\rightarrow \\text{L(1600, 256)} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\text{L(256, 128)} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\text{L(128, 10)} \\rightarrow \\text{Softmax}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"where \n",
|
||
"- [`Conv`](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html) is a Convolution layer with the specified output channels and kernel size, with no padding and a stride of 1 by default.\n",
|
||
"\n",
|
||
"- [`MP`](https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html) is the Max Pooling layer with the specified kernel size, with no padding, the stride set to the same shape as the kernel by default.\n",
|
||
"\n",
|
||
"- [`LReLU`](https://pytorch.org/docs/stable/generated/torch.nn.LeakyReLU.html) is Leaky ReLU with the specified negative slope.\n",
|
||
"\n",
|
||
"- `Flat` is the flattening operation, which should flatten/reshape the tensor from a multi-dimensional tensor (batch_size, num_channels, width, height) into a \"flat\" tensor (batch_size, num_channels x width x height). The 2-dimensional result represents that each sample has only 1 dimension of \"flattened\" data. This has already been implemented for you\n",
|
||
"\n",
|
||
"- [`L`](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html) is a fully-connected layer with the specified input and output features.\n",
|
||
"\n",
|
||
"You are highly encouraged to initialise all your layers in the `__init__` method.\n",
|
||
"\n",
|
||
"__Note:__ The only constructor argument here is `classes`. For all your networks hereon, do not add any parameters to the `__init__` method other than the ones mentioned. Remember not to hardcode for the number of classes and use the `classes` argument instead.\n",
|
||
"\n",
|
||
"__Note:__ There is no need to include a Softmax layer in your neural network, as technically, [CrossEntropyLoss](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html) which we are going to use as our loss function later, already applies Softmax implicitly."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "6dd33b95",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-27T17:04:45.792747Z",
|
||
"start_time": "2024-04-27T17:04:45.419030Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"input torch.Size([20, 1, 28, 28])\n",
|
||
"conv1 torch.Size([20, 32, 26, 26])\n",
|
||
"maxpo torch.Size([20, 32, 13, 13])\n",
|
||
"lrelu torch.Size([20, 32, 13, 13])\n",
|
||
"conv2 torch.Size([20, 64, 11, 11])\n",
|
||
"torch.Size([20, 10])\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"class RawCNN(nn.Module):\n",
|
||
" def __init__(self, classes):\n",
|
||
" super().__init__()\n",
|
||
" \"\"\"\n",
|
||
" classes: integer that corresponds to the number of classes for MNIST\n",
|
||
" \"\"\"\n",
|
||
" self.conv1 = nn.Conv2d(1, 32, 3, stride=1, padding=0)\n",
|
||
" self.conv2 = nn.Conv2d(32, 64, 3, stride=1, padding=0)\n",
|
||
" self.mp = nn.MaxPool2d(2)\n",
|
||
" self.lRelu = nn.LeakyReLU(0.1)\n",
|
||
" self.l1 = nn.Linear(1600, 256)\n",
|
||
" self.l2 = nn.Linear(256, 128)\n",
|
||
" self.l3 = nn.Linear(128, classes)\n",
|
||
" # YOUR CODE HERE\n",
|
||
" \n",
|
||
" def forward(self, x):\n",
|
||
" # YOUR CODE HERE \n",
|
||
" print('input', x.shape)\n",
|
||
" x = self.conv1(x)\n",
|
||
" print('conv1',(x.shape))\n",
|
||
" x = self.mp(x)\n",
|
||
" print('maxpo',(x.shape))\n",
|
||
" x = self.lRelu(x)\n",
|
||
" print('lrelu',(x.shape))\n",
|
||
" x = self.conv2(x)\n",
|
||
" print('conv2',(x.shape))\n",
|
||
" x = self.mp(x)\n",
|
||
" x = self.lRelu(x)\n",
|
||
" x = x.view(-1, 64*5*5) # Flattening – do not remove this line\n",
|
||
"\n",
|
||
" x = self.l1(x)\n",
|
||
" x = self.lRelu(x)\n",
|
||
" x = self.l2(x)\n",
|
||
" x = self.lRelu(x)\n",
|
||
" x = self.l3(x)\n",
|
||
" return x\n",
|
||
"\n",
|
||
"# Test your network's forward pass\n",
|
||
"num_samples, num_channels, width, height = 20, 1, 28, 28\n",
|
||
"x = torch.rand(num_samples, num_channels, width, height).to(device)\n",
|
||
"net = RawCNN(10).to(device)\n",
|
||
"y = net(x)\n",
|
||
"print(y.shape) # torch.Size([20, 10])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f2f0bc12",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Concept 2: Dropout\n",
|
||
"\n",
|
||
"__Dropout__ (*Srivastava et al., 2014*) is a regularisation technique that randomly shuts off neurons in a given layer. This means the output of the neuron is __zero__. As users, we need to specify a probability value `p` that is the probability of a neuron being shut off or not; there's a $p$ chance of a neuron being shut off.\n",
|
||
"\n",
|
||
"Suppose a layer has $n$ neurons/units. Mathematically, \n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\text{Prob}(i = 1) = p \\\\\n",
|
||
"\\text{Prob}(i = 0) = 1 - p\n",
|
||
"$$ \n",
|
||
"\n",
|
||
"where $i \\in \\{1, \\dots, n\\}$ and $1$ represents neuron $i$ being shut off and $0$ represents neuron $0$ left untouched.\n",
|
||
"\n",
|
||
"Essentially, Dropout does this:\n",
|
||
"\n",
|
||
"<img src=\"https://production-media.paperswithcode.com/methods/Screen_Shot_2020-05-23_at_6.19.24_PM.png\" width=600>\n",
|
||
"\n",
|
||
"### Why Dropout works\n",
|
||
"By randomly dropping/zero-ing out neurons in a layer, it has a regularising effect on the model. It prevents overfitting because the loss of certain features means the model doesn't accidentally compute very complex functions to model the relationship between $x$ and $y$.\n",
|
||
"\n",
|
||
"#### Dropout in PyTorch\n",
|
||
"To use Dropout in a network, we can create a `Dropout` layer in our `__init__` method of the model class:\n",
|
||
"\n",
|
||
"```python\n",
|
||
"class Model(nn.Module):\n",
|
||
" def __init__(self, ..., drop_prob):\n",
|
||
" super().__init__()\n",
|
||
" self.l1 = ...\n",
|
||
" ...\n",
|
||
" self.dropout = nn.Dropout(p=drop_prob)\n",
|
||
" ...\n",
|
||
" self.ln = ...\n",
|
||
" \n",
|
||
" def forward(self, x):\n",
|
||
" x = self.l1(x)\n",
|
||
" ...\n",
|
||
" x = self.dropout(x)\n",
|
||
" ...\n",
|
||
" out = ...\n",
|
||
" \n",
|
||
" return out\n",
|
||
"```\n",
|
||
"\n",
|
||
"__Note:__ Other that `nn.Dropout`, there is a `nn.Dropout2d` in PyTorch. Instead of randomly zero-ing out neurons, `Dropout2d` randomly zero-es out the entire channels of the input. \n",
|
||
"\n",
|
||
"`nn.Dropout` is best used with non-spatial data or data that has been flattened, which is typical for fully connected layers. `nn.Dropout2d` is designed for spatial data, making it ideal for use right after convolutional and pooling layers in CNNs.\n",
|
||
"\n",
|
||
"For the sake of this problem set, You should choose one of them to be but __NOT both__ in your neural network.\n",
|
||
"\n",
|
||
"---"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "d263bdc8",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Task 2.2: Building a ConvNet with Dropout\n",
|
||
"\n",
|
||
"Here, you must build the exact same network but with Dropout inside the architecture. You can refer to the attached command glossary to read more about the layers. Use the following architecture:\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\text{Conv(32, (3,3))} \\rightarrow \\text{MP(2,2)} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\textbf{DO(prob)} \\rightarrow \\\\\n",
|
||
"\\text{Conv(64, (3,3))} \\rightarrow \\text{MP(2,2)} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\textbf{DO(prob)} \\rightarrow \\\\\n",
|
||
"\\text{Flat} \\rightarrow \\text{L(1600, 256)} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\textbf{DO(prob)} \\rightarrow \\\\\n",
|
||
"\\text{L(256, 128)} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\text{L(128, 10)} \\rightarrow \\text{Softmax}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"where \n",
|
||
"- [`Conv`](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html) is a Convolution layer with the specified output channels and kernel size, with no padding and a stride of 1 by default.\n",
|
||
"\n",
|
||
"- [`MP`](https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html) is the Max Pooling layer with the specified kernel size, with no padding, the stride set to the same shape as the kernel by default.\n",
|
||
"\n",
|
||
"- [`LReLU`](https://pytorch.org/docs/stable/generated/torch.nn.LeakyReLU.html) is Leaky ReLU with the specified negative slope.\n",
|
||
"\n",
|
||
"- `Flat` is the flattening operation, which should flatten/reshape the tensor from a multi-dimensional tensor (batch_size, num_channels, width, height) into a \"flat\" tensor (batch_size, num_channels x width x height). The 2-dimensional result represents that each sample has only 1 dimension of \"flattened\" data. This has already been implemented for you\n",
|
||
"\n",
|
||
"- [`L`](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html) is a fully-connected layer with the specified input and output features.\n",
|
||
" \n",
|
||
"- [`DO`](https://pytorch.org/docs/stable/generated/torch.nn.Dropout.html) is Dropout with a dropping probability. Choose between `nn.Dropout` and `nn.Dropout2d` but __not both__ for your network.\n",
|
||
"\n",
|
||
"You are highly encouraged to initialise all your layers in the `__init__` method.\n",
|
||
"\n",
|
||
"__Reminder:__ Do not hardcode for the number of classes and the dropout probability. Use the `classes` and `drop_prob` constructor arguments instead.\n",
|
||
"\n",
|
||
"__Note:__ There is no need to include a Softmax layer in your neural network, as technically, [CrossEntropyLoss](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html) which we are going to use as our loss function later, already applies Softmax implicitly."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 89,
|
||
"id": "e9a58c35",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T03:15:50.305024Z",
|
||
"start_time": "2024-04-07T03:15:50.294454Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"torch.Size([20, 10])\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"class DropoutCNN(nn.Module):\n",
|
||
" def __init__(self, classes, drop_prob=0.5):\n",
|
||
" super().__init__()\n",
|
||
" \"\"\"\n",
|
||
" classes: integer that corresponds to the number of classes for MNIST\n",
|
||
" drop_prob: probability of dropping a node in the neural network\n",
|
||
" \"\"\"\n",
|
||
" self.conv1 = nn.Conv2d(1, 32, (3, 3), stride=1, padding=0)\n",
|
||
" self.conv2 = nn.Conv2d(32, 64, (3, 3), stride=1, padding=0)\n",
|
||
" self.mp = nn.MaxPool2d((2, 2))\n",
|
||
" self.lRelu = nn.LeakyReLU(0.1)\n",
|
||
" self.l1 = nn.Linear(1600, 256)\n",
|
||
" self.l2 = nn.Linear(256, 128)\n",
|
||
" self.l3 = nn.Linear(128, classes)\n",
|
||
" self.dropout = nn.Dropout(p=drop_prob)\n",
|
||
"\n",
|
||
" # YOUR CODE HERE\n",
|
||
" \n",
|
||
" def forward(self, x):\n",
|
||
" x = self.conv1(x)\n",
|
||
" x = self.mp(x)\n",
|
||
" x = self.lRelu(x)\n",
|
||
" x = self.dropout(x)\n",
|
||
" \n",
|
||
" x = self.conv2(x)\n",
|
||
" x = self.mp(x)\n",
|
||
" x = self.lRelu(x)\n",
|
||
" x = self.dropout(x)\n",
|
||
" \n",
|
||
" x = x.view(-1, 64*5*5) # Flattening – do not remove this line\n",
|
||
" x = self.l1(x)\n",
|
||
" x = self.lRelu(x)\n",
|
||
" x = self.dropout(x)\n",
|
||
" \n",
|
||
" x = self.l2(x)\n",
|
||
" x = self.lRelu(x)\n",
|
||
" x = self.l3(x)\n",
|
||
" return x\n",
|
||
"\n",
|
||
"# Test your network's forward pass\n",
|
||
"num_samples, num_channels, width, height = 20, 1, 28, 28\n",
|
||
"x = torch.rand(num_samples, num_channels, width, height).to(device)\n",
|
||
"net = DropoutCNN(10).to(device)\n",
|
||
"y = net(x)\n",
|
||
"print(y.shape) # torch.Size([20, 10])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c779f0ec",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Task 2.3: Training your Vanilla and Dropout CNNs\n",
|
||
"\n",
|
||
"Here, write down the training loop in the function `train_model` to train the CNNs you have just created. It will take in the respective NN (vanilla or dropout), as well as training and testing __data loaders__ (more on this later) that return batches of images and their respective labels to train on. \n",
|
||
"\n",
|
||
"Use the `torch.optim.Adam(...)` optimizer and Cross Entropy Loss.\n",
|
||
"\n",
|
||
"> Return the model and epoch losses.\n",
|
||
"\n",
|
||
"Remember to extract the loss value from the `loss` tensor by using `loss.item()`.\n",
|
||
"\n",
|
||
"__Tip:__ Don't be worried if your model takes a while to train. Your mileage may also vary depending on your CPU. But if you would like to speed things up, you can consider making use of your device's GPU to parallelize the matrix computations."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "1573889a",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-11T04:04:14.704810Z",
|
||
"start_time": "2024-04-11T04:04:10.568283Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"input torch.Size([256, 1, 28, 28])\n",
|
||
"conv1 torch.Size([256, 32, 26, 26])\n",
|
||
"maxpo torch.Size([256, 32, 13, 13])\n",
|
||
"lrelu torch.Size([256, 32, 13, 13])\n",
|
||
"conv2 torch.Size([256, 64, 11, 11])\n",
|
||
"input torch.Size([256, 1, 28, 28])\n",
|
||
"conv1 torch.Size([256, 32, 26, 26])\n",
|
||
"maxpo torch.Size([256, 32, 13, 13])\n",
|
||
"lrelu torch.Size([256, 32, 13, 13])\n",
|
||
"conv2 torch.Size([256, 64, 11, 11])\n",
|
||
"input torch.Size([256, 1, 28, 28])\n",
|
||
"conv1 torch.Size([256, 32, 26, 26])\n",
|
||
"maxpo torch.Size([256, 32, 13, 13])\n",
|
||
"lrelu torch.Size([256, 32, 13, 13])\n",
|
||
"conv2 torch.Size([256, 64, 11, 11])\n"
|
||
]
|
||
},
|
||
{
|
||
"ename": "KeyboardInterrupt",
|
||
"evalue": "",
|
||
"output_type": "error",
|
||
"traceback": [
|
||
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
|
||
"\u001B[0;31mKeyboardInterrupt\u001B[0m Traceback (most recent call last)",
|
||
"File \u001B[0;32m<timed exec>:33\u001B[0m\n",
|
||
"File \u001B[0;32m<timed exec>:21\u001B[0m, in \u001B[0;36mtrain_model\u001B[0;34m(loader, model, device)\u001B[0m\n",
|
||
"File \u001B[0;32m/opt/homebrew/anaconda3/envs/cs2109s-ay2223s1/lib/python3.9/site-packages/torch/_tensor.py:396\u001B[0m, in \u001B[0;36mTensor.backward\u001B[0;34m(self, gradient, retain_graph, create_graph, inputs)\u001B[0m\n\u001B[1;32m 387\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m has_torch_function_unary(\u001B[38;5;28mself\u001B[39m):\n\u001B[1;32m 388\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m handle_torch_function(\n\u001B[1;32m 389\u001B[0m Tensor\u001B[38;5;241m.\u001B[39mbackward,\n\u001B[1;32m 390\u001B[0m (\u001B[38;5;28mself\u001B[39m,),\n\u001B[0;32m (...)\u001B[0m\n\u001B[1;32m 394\u001B[0m create_graph\u001B[38;5;241m=\u001B[39mcreate_graph,\n\u001B[1;32m 395\u001B[0m inputs\u001B[38;5;241m=\u001B[39minputs)\n\u001B[0;32m--> 396\u001B[0m \u001B[43mtorch\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mautograd\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mbackward\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mgradient\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mretain_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mcreate_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43minputs\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[43minputs\u001B[49m\u001B[43m)\u001B[49m\n",
|
||
"File \u001B[0;32m/opt/homebrew/anaconda3/envs/cs2109s-ay2223s1/lib/python3.9/site-packages/torch/autograd/__init__.py:173\u001B[0m, in \u001B[0;36mbackward\u001B[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001B[0m\n\u001B[1;32m 168\u001B[0m retain_graph \u001B[38;5;241m=\u001B[39m create_graph\n\u001B[1;32m 170\u001B[0m \u001B[38;5;66;03m# The reason we repeat same the comment below is that\u001B[39;00m\n\u001B[1;32m 171\u001B[0m \u001B[38;5;66;03m# some Python versions print out the first line of a multi-line function\u001B[39;00m\n\u001B[1;32m 172\u001B[0m \u001B[38;5;66;03m# calls in the traceback and some print out the last line\u001B[39;00m\n\u001B[0;32m--> 173\u001B[0m \u001B[43mVariable\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_execution_engine\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mrun_backward\u001B[49m\u001B[43m(\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;66;43;03m# Calls into the C++ engine to run the backward pass\u001B[39;49;00m\n\u001B[1;32m 174\u001B[0m \u001B[43m \u001B[49m\u001B[43mtensors\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mgrad_tensors_\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mretain_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mcreate_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43minputs\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 175\u001B[0m \u001B[43m \u001B[49m\u001B[43mallow_unreachable\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43;01mTrue\u001B[39;49;00m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43maccumulate_grad\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43;01mTrue\u001B[39;49;00m\u001B[43m)\u001B[49m\n",
|
||
"\u001B[0;31mKeyboardInterrupt\u001B[0m: "
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"%%time \n",
|
||
"# do not remove the above line\n",
|
||
"device = \"cuda\" if torch.has_cuda else \"cpu\"\n",
|
||
"def train_model(loader, model, device=device):\n",
|
||
" model = model.to(device)\n",
|
||
" optimiser = torch.optim.Adam(model.parameters())\n",
|
||
" loss_fn = nn.CrossEntropyLoss().to(device)\n",
|
||
" # loss_fn = nn.CrossEntropyLoss()\n",
|
||
" epoch_losses = []\n",
|
||
" for i in range(10):\n",
|
||
" epoch_loss = 0\n",
|
||
" \n",
|
||
" for idx, data in enumerate(loader):\n",
|
||
" x, y = data\n",
|
||
" x = x.to(device)\n",
|
||
" y = y.to(device)\n",
|
||
" \n",
|
||
" optimiser.zero_grad()\n",
|
||
" y_pred = model(x)\n",
|
||
"\n",
|
||
" loss = loss_fn(y_pred, y)\n",
|
||
" loss.backward()\n",
|
||
" optimiser.step()\n",
|
||
" # COMPUTE STATS\n",
|
||
" epoch_loss += loss.item()\n",
|
||
"\n",
|
||
" epoch_loss = epoch_loss / len(loader)\n",
|
||
" epoch_losses.append(epoch_loss)\n",
|
||
" print (\"Epoch: {}, Loss: {}\".format(i, epoch_loss))\n",
|
||
" return model, epoch_losses\n",
|
||
" \n",
|
||
"\n",
|
||
" # YOUR CODE HERE\n",
|
||
"vanilla_model, losses = train_model(train_loader, RawCNN(10))\n",
|
||
"# do_model, losses = train_model(train_loader, DropoutCNN(10))\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 17,
|
||
"id": "15a66fad",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T02:51:31.721190Z",
|
||
"start_time": "2024-04-07T02:51:29.073203Z"
|
||
},
|
||
"scrolled": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"vanilla acc: 0.9872\n",
|
||
"drop-out (0.5) acc: 0.9912\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# do not remove – nothing to code here\n",
|
||
"# run this cell before moving on\n",
|
||
"\n",
|
||
"with torch.no_grad():\n",
|
||
" vanilla_model.eval()\n",
|
||
" for i, data in enumerate(test_loader):\n",
|
||
" x, y = data\n",
|
||
" x = x.to(device)\n",
|
||
" y = y.to(device)\n",
|
||
" pred_vanilla = vanilla_model(x)\n",
|
||
" acc = get_accuracy(pred_vanilla, y)\n",
|
||
" print(f\"vanilla acc: {acc}\")\n",
|
||
" \n",
|
||
" do_model.eval()\n",
|
||
" for i, data in enumerate(test_loader):\n",
|
||
" x, y = data\n",
|
||
" x = x.to(device)\n",
|
||
" y = y.to(device)\n",
|
||
" pred_do = do_model(x)\n",
|
||
" acc = get_accuracy(pred_do, y)\n",
|
||
" print(f\"drop-out (0.5) acc: {acc}\")\n",
|
||
" \n",
|
||
"\"\"\"\n",
|
||
"The network with Dropout might under- or outperform the network without\n",
|
||
"Dropout. However, in terms of generalisation, we are assured that the Dropout\n",
|
||
"network will not overfit – that's the guarantee of Dropout.\n",
|
||
"\n",
|
||
"A very nifty trick indeed!\n",
|
||
"\"\"\";"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5e39d607",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Task 2.4: Observing Effects of Dropout\n",
|
||
"\n",
|
||
"Here, train your `DropoutCNN` with your `train_model(loader, model)` from Task 2.3, but with `p=0.1` and `p=0.95` respectively. \n",
|
||
"\n",
|
||
"Explain why extreme values of Dropout don't work as well on neural networks. Look back at first principles – what does Dropout do in the first place? How does the `p` value affect how it does it? "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 19,
|
||
"id": "eaa1389b",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T02:55:00.432275Z",
|
||
"start_time": "2024-04-07T02:52:00.630718Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Epoch: 0, Loss: 0.279120754735901\n",
|
||
"Epoch: 1, Loss: 0.06485465260499969\n",
|
||
"Epoch: 2, Loss: 0.04437660489905071\n",
|
||
"Epoch: 3, Loss: 0.03242283386990745\n",
|
||
"Epoch: 4, Loss: 0.027098283476810505\n",
|
||
"Epoch: 5, Loss: 0.022629741275128214\n",
|
||
"Epoch: 6, Loss: 0.019325615923871543\n",
|
||
"Epoch: 7, Loss: 0.01702770205521758\n",
|
||
"Epoch: 8, Loss: 0.01503490183522251\n",
|
||
"Epoch: 9, Loss: 0.013751926652571939\n",
|
||
"Epoch: 0, Loss: 2.383768850691775\n",
|
||
"Epoch: 1, Loss: 2.270876559805363\n",
|
||
"Epoch: 2, Loss: 2.070882184454735\n",
|
||
"Epoch: 3, Loss: 1.7385065088880822\n",
|
||
"Epoch: 4, Loss: 1.5343923441907192\n",
|
||
"Epoch: 5, Loss: 1.4513092396107126\n",
|
||
"Epoch: 6, Loss: 1.395279371484797\n",
|
||
"Epoch: 7, Loss: 1.348512672870717\n",
|
||
"Epoch: 8, Loss: 1.3113447128458227\n",
|
||
"Epoch: 9, Loss: 1.2827049635826273\n",
|
||
"CPU times: user 2min 59s, sys: 158 ms, total: 2min 59s\n",
|
||
"Wall time: 2min 59s\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"%%time \n",
|
||
"# do not remove – nothing to code here\n",
|
||
"# run this before moving on\n",
|
||
"\n",
|
||
"do10_model, do10_losses = train_model(train_loader, DropoutCNN(10, 0.10))\n",
|
||
"do95_model, do95_losses = train_model(train_loader, DropoutCNN(10, 0.95))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 21,
|
||
"id": "e8874ce7",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T02:57:55.059022Z",
|
||
"start_time": "2024-04-07T02:57:52.392158Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"drop-out (0.10) acc: 0.9924\n",
|
||
"drop-out (0.95) acc: 0.6142\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# do not remove – nothing to code here\n",
|
||
"# run this cell before moving on\n",
|
||
"\n",
|
||
"with torch.no_grad():\n",
|
||
" do10_model.eval()\n",
|
||
" for i, data in enumerate(test_loader):\n",
|
||
" x, y = data\n",
|
||
" x = x.to(device)\n",
|
||
" y = y.to(device)\n",
|
||
" pred_do = do10_model(x)\n",
|
||
" acc = get_accuracy(pred_do, y)\n",
|
||
" print(f\"drop-out (0.10) acc: {acc}\")\n",
|
||
"\n",
|
||
" do95_model.eval()\n",
|
||
" for i, data in enumerate(test_loader):\n",
|
||
" x, y = data\n",
|
||
" x = x.to(device)\n",
|
||
" y = y.to(device)\n",
|
||
" pred_do = do95_model(x)\n",
|
||
" acc = get_accuracy(pred_do, y)\n",
|
||
" print(f\"drop-out (0.95) acc: {acc}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c11012be",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Concept 3: Confusion Matrix Analysis\n",
|
||
"\n",
|
||
"A __Confusion Matrix__ (CM) is a $k \\times k$ matrix that represents the number of correctly classified and misclassified samples from the dataset. For a binary classification problem, the CM is simply $2 \\times 2$. \n",
|
||
"\n",
|
||
"> For every non-diagonal row $i$ and non-diagonal column $j$, the entry $CM(i, j)$ represents the number of times the model classified a sample with label $j$ as $i$ (for example, calling a `cat` a `dog` or vice versa). \n",
|
||
"\n",
|
||
"#### TP, FP, TN, FN\n",
|
||
"Let's start small: to understand a $2 \\times 2$ CM and its 4 quadrants, you need to first understand the following concepts:\n",
|
||
"\n",
|
||
"- __True Positive__: when the prediction is positive and label is positive\n",
|
||
"- __True Negative__: when the prediction is negative and label is negative\n",
|
||
"- __False Positive__: when the prediction is positive and label is negative (also known as Type I error)\n",
|
||
"- __False Negative__: when the prediction is negative but label is positive (also known as Type II error)\n",
|
||
"\n",
|
||
"<img src=\"imgs/confusion_matrix.png\" width=1000>\n",
|
||
"\n",
|
||
"---\n",
|
||
"\n",
|
||
"#### Types of Errors\n",
|
||
"__Type 1 Error__: When you support and make the __False Positive__ conclusion. Eg: The ART says you have COVID but you actually don't have it.\n",
|
||
"\n",
|
||
"__Type 2 Error__: When you support and make the __False Negative__ conclusion. Eg: The ART says you don't have COVID but you actually have it. \n",
|
||
"\n",
|
||
"Obviously, we want the model to score high on the True Positives and True Negatives (i.e., the diagonals). Here's a comic to better understand the above terms:\n",
|
||
"\n",
|
||
"<img src=\"imgs/doc_confusion_matrix.png\" width=1000>\n",
|
||
"\n",
|
||
"---\n",
|
||
"\n",
|
||
"This concept of FP, TP, FN, TN and the confusion matrix can be scaled to a classification problem with $k > 2$ classes as well. The diagonals represent the number of samples the model correctly classified where each column (or row) corresponds to class label (or prediction). "
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "1cce640a",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Task 2.5: What did the model misclassify?\n",
|
||
"\n",
|
||
"Your task is to run the cell below, and check out the CM for the vanilla model and dropout model you have previously trained in Task 2.3. On Coursemology, post a screenshot of the two CMs, identify which class (i.e., which digit) each model misclassified the most (the class with the most **False Positives + False Negatives**), and explain your reasoning on how you came to this conclusion for both models."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 93,
|
||
"id": "dd4b56d6",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T03:37:27.228083Z",
|
||
"start_time": "2024-04-07T03:37:23.893030Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"vanilla acc: 0.9919\n",
|
||
"drop-out (0.5) acc: 0.9924\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"<Axes: title={'center': 'Confusion Matrix for do_model'}>"
|
||
]
|
||
},
|
||
"execution_count": 93,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
},
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAuwAAAJbCAYAAAC/78ExAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABjUElEQVR4nO3de3zO9f/H8ecsc2azYUOOY5nDzKLvWGGlnCqHRH0TSxKhCBFSxJziK2eSSqw5xDeHTkr9KnTwJXIohBxWNkwOmzHX7w83q8uhdmmf631d1+dx73bdbl3vXdfn83zt89m897re1+fyczgcDgEAAADwSPlMBwAAAABwfUzYAQAAAA/GhB0AAADwYEzYAQAAAA/GhB0AAADwYEzYAQAAAA/GhB0AAADwYEzYAQAAAA/GhB0AAADwYEzYgVzYtWuXhgwZovj4eNWuXVvR0dFq27at5s6dq/T0dEv3vWPHDj3yyCOKiYlRRESE3njjjTzfR0REhKZOnZrn2/077777riIiIhQREaGvv/76qq87HA41a9ZMERER6ty58w3tY+HChXr33Xddes7XX3993Uw3as2aNWrVqpXq1KmjiIgI7dy5M8+27UkGDx6s+Ph4p7H4+HgNHjw45/6hQ4cUERHh8nHxFlOnTlVERMQNPfda3z8AuMl0AMDTLV68WC+99JIqV66sbt26KTw8XBcuXNAPP/ygd955R1u2bNH06dMt2//zzz+vjIwMTZo0SSVKlFC5cuXyfB/JyckKDQ3N8+3mVpEiRbR06VLddtttTuPffPONfvnlFxUpUuSGt52UlKSgoCC1a9cu18+pWbOmkpOTFR4efsP7/bPjx49r0KBBiouL04gRIxQQEKBKlSrlybY9Ta9evfToo4+ajgEAPoUJO/AXNm/erBdffFENGzbUjBkzFBAQkPO1Ro0aKSEhQV988YWlGXbv3q0OHTqocePGlu2jbt26lm07N1q2bKmVK1dqxIgRKlq0aM740qVLFR0drdOnT7slx/nz5+Xn56eiRYvm6fdk3759On/+vO677z41aNAgT7aZkZGhQoUK5cm28lKFChVMRwAAn8OSGOAvzJ49W35+fho1apTTZP2ygIAA3XnnnTn3L168qLlz56p58+aqVauWYmNjNWjQIP36669Oz+vcubNat26trVu36uGHH1ZUVJTuvPNOzZkzRxcvXpT0x3KRCxcuKCkpKWfpiHT9l9wvP+fQoUM5Yxs2bFDnzp112223qU6dOmrSpIn69OmjjIyMnMdca0nMTz/9pJ49e6p+/fqqXbu27r//fi1fvtzpMZeXjqxatUqTJ09WXFyc6tWrp65du+rnn3/O7bdZrVq1kiStWrUqZ+zUqVP66KOP1L59+2s+Z9q0aerQoYMaNGigevXqqW3btlqyZIkcDkfOY+Lj47V792598803Od+/y8sNLmdfsWKFxo4dq9tvv121a9fWgQMHrloSc/z4cTVu3FidOnXS+fPnc7a/Z88e1a1bVwMHDrxubYMHD9bDDz8sSerXr99Vy3s++eQTdezYUVFRUYqOjlZCQoI2b97stI3Lx3v79u3q27ev6tevr2bNml1zf7t27VJERISWLFly1dc+//xzRURE6JNPPpEkHThwQEOGDNHdd9+tqKgo3X777XryySf1448/Oj3PleN8o0s6cpslN+Lj49WjRw+tW7dObdq0UZ06ddSiRQutW7dO0qWfkxYtWqhu3bp64IEHtG3btqu2kZvjIkmfffaZ7r//ftWqVUvx8fGaN2/eNTM5HA4tXLhQ999/v+rUqaP69eurb9++OnjwoMv1AbAfJuzAdWRnZ2vjxo2qWbOmwsLCcvWcF198URMnTlSjRo00c+ZMPf300/riiy/UqVMnHT9+3OmxqampGjhwoO677z7NnDlTd9xxh1555RW99957kqQmTZooOTlZknTPPfcoOTk5535uHTp0SD169FD+/Pk1evRovfbaa3r22WdVqFAhp4nnlX7++Wd16tRJe/bs0dChQzVt2jSFh4dr8ODBmjt37lWPnzRpkg4fPqzRo0dr1KhROnDggHr27Kns7Oxc5SxatKjuueceLVu2LGds1apVypcvn1q0aHHN5xw+fFgdO3bUf/7zH02bNk3NmjXTyy+/7LQ8adq0abr55psVGRmZ8/2bNm3aVdmPHDmil156SbNmzVJwcPBV+ypZsqQmTZqkbdu2aeLEiZIudbiffvpphYWF6aWXXrpubb169dILL7wgSerfv7+Sk5M1YsQISdLKlSvVq1cvFS1aVK+88opGjx6tkydPqnPnzvruu++u2lafPn1UoUIFTZkyRS+++OI193fLLbcoMjLymuvDly9fruDg4JxXa44eParAwEA9++yzeu211zRixAj5+/vrwQcfvOYfXP/0OP8VV7P8nV27dmnSpEnq3r27pk6dqmLFiqlPnz569dVXtWTJEvXv318TJ07UqVOn9OSTTyozMzPnubk9Lhs2bFCvXr1UpEgRTZ48WYMGDdL7779/ze/9Cy+8oMTERDVs2FDTp0/XiBEjtHv3bnXq1ElpaWk39k0DYBssiQGu48SJE8rIyFD58uVz9fi9e/cqOTlZDz/8sIYPH54zHhkZqQ4dOujNN99Uv379csbT09M1d+5c1alTR5LUsGFDffPNN1q5cqXatGmjkiVLqmTJkpKkkJCQG1qisX37dp07d06DBg3SLbfckjN+7733/uXzpk2bpvPnz+utt97K+WOlcePG+v333zV9+nR16tRJxYoVy3l8eHh4zkRWkvLly6dnnnlG27Zty3Xu9u3b69FHH9Xu3btVrVo1LVu2TM2bN3daIvNniYmJOf9/8eJFNWjQQA6HQ2+99Zaeeuop+fn5KTIyUgULFvzLJS4VKlTQq6+++rf5YmJi9Mwzz2jixImqX7++1q5dq0OHDmnx4sUqXLjwdZ9XoUKFnLXwFStWzMlx8eJFjR8/XtWrV9fcuXOVL9+l/knjxo3VrFkzTZw4Ue+8847Tttq0aaO+ffv+bdZ27drp5Zdf1r59+1S5cmVJ0smTJ/XJJ5/okUce0U03XfrVX79+fdWvXz/nednZ2WrcuLFat26t5ORkDRkyxGm7eXGcr8fVLH8nPT1dixcvVpkyZSRJZcqU0f3336/Fixfr448/dlpO9NRTT2n9+vWKj4936bhMnjxZwcHBmj9/vgoUKCBJiouLc3rVTZK2bNmixYsXa/DgwUpISMgZv/XWW3XPPfdo/vz5f/kqDQDQYQfyyOXlE23btnUar1OnjqpWraoNGzY4jZcqVSpnsn5ZRESEjhw5kmeZatSoofz582v48OFavnx5rl9+37hxo2JjY696ZaFt27bKyMi4amnAlUsgLi/XcaWWBg0aqEKFClq2bJl+/PFHbdu27brLYaRL3c2uXbsqJiZGNWrUUM2aNfXqq68qPT1dx44dy/V+77777lw/9vHHH1eTJk3Uv39/LV++XMOGDbvhq4Hs27dPR48e1f33358zKZQuvQH37rvv1vfff++0bMmVrPfee68CAgKcljCtWrVKWVlZTm++vXDhgmbNmqWWLVuqVq1aioyMVK1atbR//37t3bv3qu3mxXG+Hlez/J0aNWrkTNYlqUqVKpKk2267zWmyXrVqVUl/1JDb43L27Flt27ZNd999d85kXbr0alHTpk2dsqxbt05+fn667777dOHChZxbSEiIbrnlFn3zzTcu1wfAXuiwA9cRFBSkQoUKOa0H/yuXL+9YunTpq75WunTpqyY1gYGBVz0uICBA586dcznr9VSoUEFvvPGGXnvtNY0cOVJnz57VzTffrM6dO6tLly7XfV56erpKlSp11fjl2q68lOWVtVxe7//nZQZ/x8/PT+3atdOCBQt07tw5VapUSbfeeus1H7t161Z169ZNDRo00KhRoxQaGqr8+fNr7dq1mjVrlkv7vVadf5Wxbdu2+uyzz1SqVCndf//9uX7ulU6cOHHd/ZcuXVoXL17U77//7jS5vNa5dS2BgYGKj4/XihUr9PTTT8vf31/Lly9XnTp1VK1atZzHjR07VgsXLlT37t1Vv359lShRQn5+fho2bNg1z8O8OM7X42qWv1OiRIlrZr1yPH/+/JKUs4/cHheHw6GLFy8qJCTkqsddOXbs2DE5HA41bNjwmllvvvnm3JQEwMaYsAPX4e/vr3/961/64osv9Ouvv/7tZQ8vT2aOHj161WOPHj2qoKCgPMt2uaOXlZXl9GbYy5ONP7v11lt16623Kjs7Wz/88IMWLFigMWPGKCQkJOfNnlcKDAxUamrqVeNHjx6VpDyt5c/atWunV199Ve+8847T8qErrV69WjfddJNmz57t1N1cu3aty/v08/PL9WOPHj2qkSNHqkaNGtqzZ4/Gjx+vYcOGubxP6Y/v4fW+z/ny5VPx4sVvaNvSpe/lBx98oK+++kply5bVtm3brlr3/t5776lNmzbq37+/0/iJEyf+0b5vhKdkye1xcTgc8vPzu+b68yvHgoKC5Ofnp4ULF173zesA8FdYEgP8hR49esjhcGjYsGHKysq66uvnz5/Xp59+Kkn617/+JUk5bxq9bOvWrdq7d2/O1/PC5Wux79q1y2n88lUwrsXf319RUVE5b3jcvn37dR8bGxurjRs36rfffnMa/+9//6tChQpZdhnIMmXKqFu3bmratKnatGlz3cf5+fnJ39/faclCZmbmVd976dJkKC86wNnZ2Xr22Wfl5+enuXPnqn///lqwYIE++uijG9pe5cqVVaZMGa1atcrpyjZnz57VRx99pLp16/6jyzbGxcWpTJkyevfdd7Vs2TIVKFBArVu3dnqMn59fTof5ss8+++yq4+4OnpIlt8elcOHCqlOnjj766COnVwBOnz591c9hkyZN5HA49Ntvv6l27dpX3W50WRUA+6DDDvyF6Ohovfjii3rppZfUvn17derUSdWqVdOFCxe0Y8cOLV68WNWqVVN8fLyqVKmijh076u2331a+fPl0xx136PDhw5oyZYrCwsLUtWvXPMvVuHFjBQYGaujQoU5LHlJSUpwel5SUpI0bN6pJkyYKCwvTuXPncq7Ecr2X56VLb8Jbt26dHn30UT311FMqUaKEVq5cqc8++0wDBw50esNpXhswYMDfPqZx48aaP3++nn32WXXs2FHp6emaN2/eNTuV1atX1+rVq7VmzRqVL19eBQoUuKEJ0quvvqrvvvtOr7/+ukqVKqXHHntM33zzjYYOHaoaNWq4vKwhX758GjhwoAYMGKAePXqoY8eOysrK0rx58/T777/r2WefdTnjn/n7+6tNmzaaP3++ihYtqmbNml113Jo0aaLly5erSpUqOZeNnDdvnpEP0fKULK4cl6efflqPP/64EhIS9Nhjjyk7O1tz585VoUKFnJaNxcTEqGPHjnr++ef1ww8/qH79+ipUqJBSU1O1adMmVa9ePefSnwBwLUzYgb/x4IMPqk6dOjlrwVNTU5U/f35VqlRJrVu31iOPPJLz2BdffFE333yzli5dqkWLFqlo0aK6/fbb9eyzz+bpMpKiRYtq7ty5GjNmTM4EukOHDrr99tudlmjUqFFDX331laZOnarU1FQVLlxY1atX18yZMxUXF3fd7VepUkXvvPOOJk2apJEjRyozM1NVq1ZVYmKiS58YapXY2FiNGTNGc+fO1ZNPPqkyZcrowQcfVMmSJTV06FCnx/bp00epqakaNmyYzpw5o3LlyuW8KpJbX331lebMmaNevXopNjY2Z3zs2LFq27at+vXrp0WLFrm8tOHee+9VoUKFNGfOHPXr1y/nVZC33npL9erVc2lb19KuXTvNnj1bx48fv+YbeIcOHaqbbrpJc+bM0dmzZxUZGampU6dqypQp/3jfrvKkLLk9Lo0aNdL06dP1n//8R88884xKlSqlhx56SOfOnbvq8qEjR45UVFSUkpOTlZSUpIsXL6p06dKqV6/eVW8+B4Ar+Tn+/JofAAAAAI/CGnYAAADAg7EkBgDgFbKzs/VXLwpffjMyAPgalsQAALxCfHy8Dh8+fN2vN2jQQAsWLHBjIgBwDybsAACv8OOPP17z8qqXFSlSJOcTTQHAE1y4cEFTp07VypUrlZaWplKlSqlt27bq1auX06WJ/w5LYgAAXoHrlQPwNnPnztU777yjcePGKTw8XD/88IOGDBmiYsWK/eUnjl+JCTsAAABggS1btujOO+9UkyZNJEnly5fX6tWr9cMPP7i0Ha4SAwAAAORSVlaWTp8+7XS73nK9mJgYbdy4Ufv27ZN06RPKN23apMaNG7u0T4/psGdeMJ0AAIAbt3l/uukIcKPoSoGmIxhR0GNmjs4KRfd2277GPxZx1Yej9e7dW3369Lnqsd27d9epU6fUokUL+fv7Kzs7W/369VPr1q1d2qeHftsBAAAAz9OjRw8lJCQ4jV3vk67XrFmj9957T6+88orCw8O1c+dOJSYmqnTp0mrbtm2u98mEHQAAAN7Nz32rvAMCAq47Qb/S+PHj9cQTT6hVq1aSLr15/siRI5o9e7ZLE3bWsAMAAAAWyMzMlJ+fn9OYv7//X34I3LXQYQcAAIB3u2JS7CmaNm2qWbNmqWzZsjlLYubPn6/27du7tB0m7AAAAIAFhg0bpilTpuill17SsWPHVLp0aXXs2FFPPfWUS9vxmE865SoxAABvxlVi7IWrxHiWQjFPu21fGZumuG1fl3notx0AAADIJTe+6dQE364OAAAA8HJ02AEAAODdPPRNp3mFDjsAAADgweiwAwAAwLuxhh0AAACAKXTYAQAA4N1Yww4AAADAFDrsAAAA8G6sYQcAAABgCh12AAAAeDfWsAMAAAAwhQ47AAAAvBtr2AEAAACYYusJe3LSQrW4O171o2urU4d2+t+m70xHcgvqpm47oG7q9mWrFr+hl57pqicfaKo+DzfXlFEDlXLogOlYlrNr3ZfZ7Tx3iZ+f+24G2HbC/sH7azR+bKK6P9FTyUtXqF69GPXq0V0pR46YjmYp6qZu6vZd1G2fundt26z4Vg9o+CvzNPDlV3UxO1sTh/XVucwM09EsZde6JXue5/iDn8PhcJgOIUmZF9y7v3936qAakZEa9sJLOWNt7m2hpvF36el+z7o3jBtRN3VTN3X7Gk+pe/P+dLft60q/nzyhvg8315BxsxRRK9pYDnczWXd0pUC37s9TzvOCHvrux0Jxw922r4wvR7ltX5fZssN+PitLO3dsV2zDOKfx2IaN9P2WzYZSWY+6qVuibl9F3faq+0oZZ05LkooULW44iXvZpW7Oc7j8d9Kvv/6qpKQk/e9//1NaWpr8/PwUHBysevXqqVOnTgoLC7MiZ546kX5C2dnZCg4OdhoPDg5RWlqqoVTWo27qlqjbV1G3ver+M4fDoaS5U1S9ZpTKV6pqOo7b2KluznO4NGH/7rvv1L17d4WFhalRo0YKCQmRw+HQsWPHtHbtWi1YsEBz585VTEyMVXnzlN8VbxxwOBxXjfki6r6Eun0bdV9C3b5vwcwJOrh/j4ZOmG06ilvZsW47n+d/y8e/Dy5N2BMTE9WhQwc9//zz1/z6mDFjNGbMGC1btixPwlklKDBI/v7+SktLcxo/fvyYgoNDDKWyHnVTt0Tdvoq67VX3ZQtmTtSWr7/QkHGzVTKkjOk4bmO3uu1+nsPFNey7d+9Wp06drvv1jh07avfu3f84lNXyBwSoRmRNbVz/ldP4xvXrFVXXd9+sQ93ULVG3r6Jue9XtcDi0YOYEbdrwmQaNma5SoWVNR3ILu9Zt1/PcJX753HczwKUOe6lSpbR582ZVqVLlml/fsmWLSpUqlSfBrNa5S4KGDh6kyFq1FBUVrWVLkpWSkqIOHa//B4kvoG7qpm7fRd32qXvBjAna8PmHenr4BBUsVETpx49JkgoXKaKAAgUNp7OOXeuW7Hme4w8uTdgfe+wxjRgxQtu3b1fDhg0VEhIiPz8/paamav369VqyZMl1l8t4muYtWupk+gnNmTlDqalHFV6tuqbPmqOyZcuZjmYp6qZu6vZd1G2fuj9dc2np6djBPZ3Guz0zXLc3a20iklvYtW7Jnue5Swx1vt3F5euwr1mzRm+88Ya2b9+u7OxsSZK/v79q1qyprl27qmXLljcUxN3XYQcAIC+ZvA473M/d12H3FB57HfbGI922r4zPX3Dbvi5z+dvesmVLtWzZUufPn9eJEyckSUFBQcqfP3+ehwMAAAD+Vj6uEnNN+fPnV+nSpfMyCwAAAIAreOgLGwAAAEAu+fgadt+uDgAAAPBydNgBAADg3Xz8k07psAMAAAAejA47AAAAvBtr2AEAAACYQocdAAAA3o017AAAAABMocMOAAAA78YadgAAAACm0GEHAACAd2MNOwAAAABTmLADAAAAHowlMQAAAPBuvOkUAAAAgCl02AEAAODdeNMpAAAAAFPosAMAAMC7sYYdAAAAgCl02AEAAODdfHwNOxN2AADyQHSlQNMRAPgoJuwAAADwbqxhBwAAAGAKHXYAAAB4NzrsAAAAAEyhww4AAADv5uNXiaHDDgAAAHgwOuwAAADwbqxhBwAAAGAKHXYAAAB4Nw9dwx4fH6/Dhw9fNf7www9rxIgRud4OE3YAAADAAkuXLlV2dnbO/d27dyshIUHNmzd3aTtM2AEAAAALlCxZ0un+nDlzVKFCBTVo0MCl7TBhBwAAgHfzgjedZmVl6b333lNCQoL8XFzCw4QdAAAAyKWsrCxlZWU5jQUEBCggIOAvn7d27VqdOnVKbdu2dXmfnv/nCAAAAPBX/Pzcdps9e7ZiYmKcbrNnz/7biMuWLdMdd9yhMmXKuFweHXYAAAAgl3r06KGEhASnsb/rrh8+fFjr16/X1KlTb2ifTNgBAADg1VxdE/5P5Gb5y5XeffddBQcHq0mTJje0T5bEAAAAABa5ePGi3n33XbVp00Y33XRjvXI67AAAAPBq7uywu2r9+vU6cuSI2rdvf8PbYMIOAAAAWCQuLk4//vjjP9oGE3YAAAB4N89tsOcJW69hT05aqBZ3x6t+dG116tBO/9v0nelIbkHd1G0H1E3ddkDd1A17sO2E/YP312j82ER1f6KnkpeuUL16MerVo7tSjhwxHc1S1E3d1O27qJu6qdt32bXu3PLz83PbzUh9DofDYWTPV8i84N79/btTB9WIjNSwF17KGWtzbws1jb9LT/d71r1h3Ii6qZu6qdvXUDd1U7f76i7ooYupiz74htv2dXpxV7ft6zJbdtjPZ2Vp547tim0Y5zQe27CRvt+y2VAq61E3dUvU7auom7ol6vZVdq3bFb7eYbflhP1E+gllZ2crODjYaTw4OERpaamGUlmPuqlbom5fRd3ULVG3r7Jr3fhDnk/YU1JSNGTIkLzerCWu/CvJ4XB49HU88wp1X0Ldvo26L6Fu30bdl1A36LC76OTJk1qxYkVebzZPBQUGyd/fX2lpaU7jx48fU3BwiKFU1qNu6pao21dRN3VL1O2r7Fo3/uDyhP2TTz75y9vGjRutyJmn8gcEqEZkTW1c/5XT+Mb16xVVN9pQKutRN3VL1O2rqJu6Jer2VXat2xW+3mF3+b2+Tz31lPz8/PRXF5fxhpdnOndJ0NDBgxRZq5aioqK1bEmyUlJS1KFjJ9PRLEXd1E3dvou6qZu6fZdd68YlLk/YS5UqpREjRuiuu+665td37typdu3a/eNgVmveoqVOpp/QnJkzlJp6VOHVqmv6rDkqW7ac6WiWom7qpm7fRd3UTd2+y65155rn94r/EZevw/7kk0+qRo0aevrpp6/59V27dqlNmzbatWuXS0HcfR12AAAAuMZTr8Ne4uEFbtvXyUWd3bavy1z+tj/++OM6e/bsdb9eoUIFvfXWW/8oFAAAAIBLXJ6w33rrrX/59cKFC6tBgwY3HAgAAABwhTe8f/KfsOUHJwEAAADewkNXIgEAAAC5Q4cdAAAAgDF02AEAAODV6LADAAAAMIYOOwAAALwaHXYAAAAAxtBhBwAAgHfz7QY7HXYAAADAk9FhBwAAgFdjDTsAAAAAY+iwAwAAwKvRYQcAAABgDB12AAAAeDU67AAAAACMocMOAAAA7+bbDXY67AAAAIAnY8IOAAAAeDCWxAAAAMCr8aZTAAAAAMZ4TId98/500xGMiK4UaDoCYLmU9EzTEYwICyxoOoIRdj3e3RZtNh3BiHkPR5uOYIRdf749FR12AAAAAMZ4TIcdAAAAuBF02AEAAAAYQ4cdAAAAXo0OOwAAAABj6LADAADAu/l2g50OOwAAAODJ6LADAADAq7GGHQAAAIAxdNgBAADg1eiwAwAAADCGDjsAAAC8Gh12AAAAAMbQYQcAAIB38+0GOx12AAAAwJMxYQcAAAA8GEtiAAAA4NV40ykAAAAAY+iwAwAAwKvRYQcAAABgjC077KsWv6FN6z9TyqEDyh9QQOE1auvBhN4KK1/RdDS3SE5aqDfmz1NaaqqqhlfToMHPq17MraZjWY667VP3ti2btHTRG9q9a6eOH0vVC4mT1fCOeNOx3ILjbY/j3eW28upy281OY8fPZOmBeZsMJXIfOx7vy+z4851bdNh90K5tmxXf6gENf2WeBr78qi5mZ2visL46l5lhOprlPnh/jcaPTVT3J3oqeekK1asXo149uivlyBHT0SxF3faqOzMjQ5XDI9Sr/2DTUdyK422v473v2Fm1f+27nFu3Rd+bjuQWdj3edv35xiW2nLAPGDVFtzdrrXIVq6hClerq1m+4jqX+qv17dpmOZrkFb85X2/bt1e6BDqpStaoGDRmq0LBQLU5OMh3NUtRtr7rrx8ap6xO9FdfkLtNR3Irjba/jnX3RoRNnz+fcTmZcMB3JLex6vO36851bfn5+bru56rffftOAAQN02223KSoqSvfff79++OEHl7Zhywn7lTLOnJYkFSla3HASa53PytLOHdsV2zDOaTy2YSN9v2WzoVTWo2571W1XHG/7KRdYUIsfi9HCLtEa1ryawooXMB0JFuHn23udPHlSDz30kPLnz6+5c+dq9erVGjx4sIoXd23Oacs17H/mcDiUNHeKqteMUvlKVU3HsdSJ9BPKzs5WcHCw03hwcIjS0lINpbIeddurbrvieNvLzl9Pa+xHe3QoPVNBhfPrkfrlNLVDLT228Hv9nmmPTrud8POdCx66hH3u3LkKDQ1VYmJizlj58uVd3o7LHfbMzEx999132rNnz1VfO3funFasWOFyCJMWzJygg/v36MlBo0xHcZsrX85xOBw+/2YNibovs0vddsXxtodvDqTri73Hte/YWf3v4Ek9/96lJZ131yhlOBmsxM+3Z8jKytLp06edbllZWdd87KeffqpatWqpb9++io2NVZs2bbR48WKX9+nShH3fvn1q2bKlHnnkEd17773q3Lmzjh49mvP1U6dOaciQIS6HMGXBzIna8vUXGpw4QyVDypiOY7mgwCD5+/srLS3Nafz48WMKDg4xlMp61G2vuu2K421vmRcu6udjZ1W+REHTUWABfr7/njvXsM+ePVsxMTFOt9mzZ18z18GDB5WUlKRKlSpp3rx56tSpk15++WWXG9wuTdgnTpyo6tWra/369frggw9UpEgRPfTQQzriZe9QdjgcWjBzgjZt+EyDxkxXqdCypiO5Rf6AANWIrKmN679yGt+4fr2i6kYbSmU96rZX3XbF8ba3/P5+qliykI6dPW86CizAz7dn6dGjhzZt2uR069GjxzUf63A4VLNmTfXv31+RkZHq1KmTHnzwQSUlufZmYZfWsG/evFnz589XyZIlVbJkSc2aNUsvvfSS/v3vf+utt95SoUKFXNq5KQtmTNCGzz/U08MnqGChIko/fkySVLhIEQUU8O3uROcuCRo6eJAia9VSVFS0li1JVkpKijp07GQ6mqWo2151Z5w9qyOHfsm5/+uRw9r70y4VK15CpUPDDCazFsf7Ejsc7yfjKmr9vhM6euqcAgvlV+cG5VQ4wF8f7Tz690/2cnY83pJ9f75zy51LgwICAhQQEJCrx5YqVUpVqzq/R7JKlSr68MMPXdqnSxP2zMxM3XST81NGjBihfPny6ZFHHtErr7zi0s5N+XTNMknS2ME9nca7PTNctzdrbSKS2zRv0VIn009ozswZSk09qvBq1TV91hyVLVvOdDRLUbe96v5p13Y91+fxnPtzpk6UJN3V4j4NGOa771fheF9ih+MdUjRAw+6pphKFbtLJjAva8esp9V78g347de11tL7Ejsdbsu/Pt7erV6+e9u3b5zS2f/9+lSvn2nHzczgcjtw++IEHHtAjjzyiNm3aXPW1kSNHauXKlTp9+rR27tzpUghJ2rAn3eXn+ILoSoGmIwCWS0nPNB3BiLBA337F7nrsery7LbLn5fXmPWzPJRl2/fku6KHXFwwf8L7b9rVnYotcP3br1q166KGH1KdPH7Vo0UJbt27V8OHDNXLkSN1333253o5La9ibNWum1atXX/NrL7zwglq1aiUX5v8AAACAz6pTp46mTZum1atXq3Xr1poxY4aef/55lybrkosddivRYQd8l107rnbtwNn1eNNhtxe7/nx7aoe92sAP3Lav3ROau21fl/FJpwAAAIAH89C/kwAAAIDc8fXPj6LDDgAAAHgwJuwAAACAB2NJDAAAALyaOz84yQQ67AAAAIAHo8MOAAAAr+bjDXY67AAAAIAno8MOAAAAr5Yvn2+32OmwAwAAAB6MDjsAAAC8GmvYAQAAABhDhx0AAABejeuwAwAAADCGDjsAAAC8mo832OmwAwAAAJ6MDjsAAAC8GmvYAQAAABhDhx0AAABejQ47AAAAAGM8psMeXSnQdAQAFgkLLGg6ghEp6ZmmIxhh1+M9umUN0xEA2/LxBjsddgAAAMCTMWEHAAAAPJjHLIkBAAAAbgRvOgUAAABgDB12AAAAeDUfb7DTYQcAAAA8GR12AAAAeDXWsAMAAAAwhg47AAAAvJqPN9jpsAMAAACejA47AAAAvBpr2AEAAAAYQ4cdAAAAXs3HG+x02AEAAABPRocdAAAAXo017AAAAACMocMOAAAAr+bjDXY67AAAAIAno8MOAAAAr8YadgAAAADG2HrCnpy0UC3ujlf96Nrq1KGd/rfpO9OR3IK6qdsO7Fj3ti2bNGJQHz18311q3ihK6//vU9OR3MZux3vV4jf00jNd9eQDTdXn4eaaMmqgUg4dMB3LLTjP7XOeu8LPz303E2w7Yf/g/TUaPzZR3Z/oqeSlK1SvXox69eiulCNHTEezFHVTN3X7rsyMDFUOj1Cv/oNNR3ErOx7vXds2K77VAxr+yjwNfPlVXczO1sRhfXUuM8N0NMtxntvnPMcfbDthX/DmfLVt317tHuigKlWratCQoQoNC9Xi5CTT0SxF3dRN3b6rfmycuj7RW3FN7jIdxa3seLwHjJqi25u1VrmKVVShSnV16zdcx1J/1f49u0xHsxznuX3Oc/zBlhP281lZ2rlju2IbxjmNxzZspO+3bDaUynrUTd0SdcO3cLwvyThzWpJUpGhxw0lgBc7zv+fn5+e2mwkuXyVm79692rJli+rWrauqVatq7969euutt5SVlaX77rtPsbGxVuTMUyfSTyg7O1vBwcFO48HBIUpLSzWUynrUTd0SdcO3cLwlh8OhpLlTVL1mlMpXqmo6DizAeQ6XJuz/93//p169eqlIkSLKyMjQtGnT9Nxzz+mWW26Rw+HQ448/rtdee80rJu3S1ZcAcjgcPn9ZIIm6L6Nu32bXuu3Kzsd7wcwJOrh/j4ZOmG06Cixm5/P87/j6t8GlJTEzZsxQt27d9PXXXysxMVEDBgzQgw8+qPnz5+uNN97Q448/rrlz51qVNc8EBQbJ399faWlpTuPHjx9TcHCIoVTWo27qlqgbvsXux3vBzIna8vUXGpw4QyVDypiOA4vY/TyHixP23bt3q127dpKkFi1a6MyZM7r77rtzvt6qVSv9+OOPeZvQAvkDAlQjsqY2rv/KaXzj+vWKqhttKJX1qJu6JeqGb7Hr8XY4HFowc4I2bfhMg8ZMV6nQsqYjwUJ2Pc9dwRr268iXL58CAgJUvPgfb3ApUqSITp06lSfBrNa5S4KGDh6kyFq1FBUVrWVLkpWSkqIOHTuZjmYp6qZu6vZdGWfP6sihX3Lu/3rksPb+tEvFipdQ6dAwg8msZcfjvWDGBG34/EM9PXyCChYqovTjxyRJhYsUUUCBgobTWYvz3D7nOf7g0oS9XLlyOnDggCpWrChJSk5OVljYHz8cKSkpKlWqVN4mtEjzFi11Mv2E5sycodTUowqvVl3TZ81R2bLlTEezFHVTN3X7rp92bddzfR7PuT9n6kRJ0l0t7tOAYaNMxbKcHY/3p2uWSZLGDu7pNN7tmeG6vVlrE5HchvPcPue5K3x9Lb+fw+Fw5PbBSUlJCgsLU5MmTa759cmTJystLU2jR492OUjmBZefAgAeLSU903QEI8ICfbvDez2b96ebjmBEqE2Pt13P84I3vDbDWndM+urvH5RH/q9/I7ft6zKXvu0PPfTQX369X79+/ygMAAAA4Cofb7Db84OTAAAAAG/hoS9sAAAAALnj62vY6bADAAAAHowOOwAAALyapzbYp06dqmnTpjmNhYSE6KuvXHuTLBN2AAAAwCLVqlXT/Pnzc+77+/u7vA0m7AAAAPBqnryG3d/f/x9/ThETdgAAACCXsrKylJWV5TQWEBCggICAaz7+wIEDiouLU0BAgKKiotS/f3/dfPPNLu3TpQ9OshIfnATA1/DBSfbCByfZi13Pc0/94KQ7p25w277a6Lur1qX37t1bffr0ueqxn3/+uTIzM1WpUiUdO3ZMM2fO1M8//6xVq1YpKCgo1/tkwg4AFmHCbi9M2O3Fruc5E3bp/R4xLnXY/+zs2bNq1qyZHn/8cSUkJOR6nx76bQcAAAA8T24n59dSuHBhVa9eXfv373fpeUzYAQAA4NXyefCbTv8sKytLe/fuVUxMjEvPY8IOAAAAWGDcuHFq2rSpwsLCdPz4cc2cOVOnT59W27ZtXdoOE3YAAAB4NU9tsP/666/q37+/0tPTFRQUpLp162rx4sUqV66cS9thwg4AAABYYPLkyXmyHSbsAAAA8Gqe/MFJeSGf6QAAAAAAro8OOwAAALxaPt9usNNhBwAAADwZHXYAAAB4NdawAwAAADCGDjsAAAC8mo832JmwA4BVwgILmo5gREp6pukIRkRXCjQdwQi7Hu+g+r1NRzAiY/M00xFsiQk7AAAAvJqffLvFzhp2AAAAwIPRYQcAAIBX4zrsAAAAAIyhww4AAACvxnXYAQAAABhDhx0AAABezccb7HTYAQAAAE/GhB0AAADwYCyJAQAAgFfL5+NrYuiwAwAAAB6MDjsAAAC8mo832OmwAwAAAJ6MDjsAAAC8Gh+cBAAAAMAYOuwAAADwaj7eYKfDDgAAAHgyOuwAAADwalyHHQAAAIAxdNgBAADg1Xy7v27zDnty0kK1uDte9aNrq1OHdvrfpu9MR3IL6qZuO6Bu+9S9bcsmjRjURw/fd5eaN4rS+v/71HQkt+F42+N4+/vn04herbVz1Ys6vmGSdqx8UUOeaO7zlzLEH2w7Yf/g/TUaPzZR3Z/oqeSlK1SvXox69eiulCNHTEezFHVTN3X7LrvWnZmRocrhEerVf7DpKG7F8bbP8X62azM9/kCc+o1dorrtXtbQKSvU79G71KtTY9PRPIafn5/bbibkyYTd4XDkxWbcasGb89W2fXu1e6CDqlStqkFDhio0LFSLk5NMR7MUdVM3dfsuu9ZdPzZOXZ/orbgmd5mO4lYcb/sc79vqVNaqz7fqgy+365eU41q+dos+2bhL9SIrmI4GN8mTCXvt2rW1d+/evNiUW5zPytLOHdsV2zDOaTy2YSN9v2WzoVTWo27qlqjbV9m1brvieNvLhi171bRBhMIrlJYk1a5eTrF1q+jDr7YbTuY58vm572aCS286TUxMvOZ4dna25syZo8DAQEnSkCFD/nEwK51IP6Hs7GwFBwc7jQcHhygtLdVQKutRN3VL1O2r7Fq3XXG87WXi/I9VvGghfb98mLKzHfL399OI6au0+INNpqPBTVyasL/55pu65ZZbVKxYMadxh8OhvXv3qlChQl71BogrszocDq/Kf6Oo+xLq9m3UfYld6rYrjrc9dLgnRg+1rK+uz7+pHXtTVCeinCYMeEApqSe1cOXXpuN5BF8/712asPfr10+LFy/Wc889p9jY2JzxmjVrauzYsQoPD8/zgFYICgySv7+/0tLSnMaPHz+m4OAQQ6msR93ULVG3r7Jr3XbF8baXMc+00cT5H2vJh5c66tv3HFGFsJIamNCMCbtNuLSGvUePHpo8ebJefPFFjRs3TufPn7cql6XyBwSoRmRNbVz/ldP4xvXrFVU32lAq61E3dUvU7avsWrddcbztpVDBAF10XHQay77oUL58tr3Yn+24/MFJderU0bvvvquRI0eqffv2mjBhgle+DNG5S4KGDh6kyFq1FBUVrWVLkpWSkqIOHTuZjmYp6qZu6vZddq074+xZHTn0S879X48c1t6fdqlY8RIqHRpmMJm1ON6X2OF4r/m/bXqu2z06mHJCO/amqO4t5dX3kaZ6a8VG09E8hhdORV1yQ590WqRIEY0bN06rV6/WY489puzs7LzOZbnmLVrqZPoJzZk5Q6mpRxVerbqmz5qjsmXLmY5mKeqmbur2XXat+6dd2/Vcn8dz7s+ZOlGSdFeL+zRg2ChTsSzH8b7EDse7/7glGtGrtaY831GlgooqJfWk5i39SmPmvG86GtzEz/EPL6L+66+/6ocfflDDhg1VuHDhG95O5oV/kgIA4ClS0jNNRzAiLLCg6QhG2PV4RzYbYDqCERmbp5mOcE2PLtrqtn299XAdt+3rshvqsP9ZaGioQkND8yILAAAAgCv84wk7AAAAYJKpDzRyF95eDAAAAHgwOuwAAADwat54xUJX0GEHAAAAPBgddgAAAHg13+6v02EHAAAAPBoddgAAAHi1fKxhBwAAAGAKHXYAAAB4NR9vsNNhBwAAADwZHXYAAAB4Na7DDgAAAMAYOuwAAADwaj7eYKfDDgAAAHgyOuwAAADwalyHHQAAAIAxTNgBAAAAN5g9e7YiIiI0evRol57HkhgAAAB4NW9YEbN161YlJycrIiLC5efSYQcAAAAsdObMGQ0cOFAvv/yySpQo4fLzmbADAADAq/n5+bntdiNGjhypxo0bq2HDhjf0fJbEAAAAALmUlZWlrKwsp7GAgAAFBARc8/GrV6/Wjh07tHTp0hvep8dM2FPSM01HMCIssKDpCEbY9XhHdn/bdAQjTix53HQEAMhTOz6eaDoC/sSdS0Zmz56tadOmOY317t1bffr0ueqxKSkpGj16tF5//XUVKFDghvfpMRN2AAAAwNP16NFDCQkJTmPX665v375dx44dU7t27XLGsrOz9e2332rhwoXatm2b/P39/3afTNgBAADg1W50bfmN+KvlL1f617/+pZUrVzqNDRkyRFWqVFH37t1zNVmXmLADAAAAlihatKiqV6/uNFa4cGEFBgZeNf5XmLADAADAq+Xzguuw/xNM2AEAAAA3WbBggcvPYcIOAAAAr+brHXY+OAkAAADwYHTYAQAA4NXceZUYE+iwAwAAAB6MDjsAAAC8GmvYAQAAABhDhx0AAABezceXsNNhBwAAADwZE3YAAADAg7EkBgAAAF4tn4+viaHDDgAAAHgwOuwAAADwar7egfb1+gAAAACvRocdAAAAXs3Hl7DTYQcAAAA8mW0n7Nu2bNKIQX308H13qXmjKK3/v09NR3Kb5KSFanF3vOpH11anDu30v03fmY5kOTse712zOypj+eNX3SY/0dB0NLew43ku2bNuO/58X8bxts/xtmvduZXPz89tNyP1GdmrB8jMyFDl8Aj16j/YdBS3+uD9NRo/NlHdn+ip5KUrVK9ejHr16K6UI0dMR7OUHY933MD/qlLCwpxbyxFrJEnvfrXPcDLr2fU8t2vddvz5ljjedjvedq0bl9h2DXv92DjVj40zHcPtFrw5X23bt1e7BzpIkgYNGar167/U4uQkPd3vWcPprGPH4532e6bT/QHtorQ35aS+2J5iKJH72PU8t2vddvz5ljjedmPXunOLNezwGeezsrRzx3bFNnT+gY9t2Ejfb9lsKBXcIf9N+dSpcbje/OQn01EsZ9fz3K512xXHG7CXf9RhP3nypJYvX64DBw6oVKlSatu2rcLCwvIqG/LYifQTys7OVnBwsNN4cHCI0tJSDaWCO9zXoKICiwTo7U93m45iObue53at26443oCzfHTY/xAXF6cTJ05Ikg4ePKhWrVrptdde04EDB5ScnKx7771Xe/futSQo8o7fFa8bORyOq8bgW7rcFaEP/3dIKSfOmo7iNnY9z+1at11xvAF7cGnCnpaWposXL0qSJk+erMqVK+vjjz/W66+/ro8//li33nqrpkyZYklQ/HNBgUHy9/dXWlqa0/jx48cUHBxiKBWsVqFUUcXXKas31u4yHcUt7Hqe27Vuu+J4A864Ssx1fP/99+rVq5cKFSokSQoICFDPnj31/fff51k45K38AQGqEVlTG9d/5TS+cf16RdWNNpQKVuscX11HT2bq/e8Omo7iFnY9z+1at11xvAF7cXkN++WX2rKyshQS4vxXfHBwsI4fP543ySyWcfasjhz6Jef+r0cOa+9Pu1SseAmVDvXddfiduyRo6OBBiqxVS1FR0Vq2JFkpKSnq0LGT6WiWsuvx9vOTHo2vpoWf7Vb2RYfpOG5j1/PcrnXb9eeb432JXY63XevOLV9fCebyhL1Lly666aabdPr0ae3fv1/VqlXL+VpKSoqCgoLyNKBVftq1Xc/1eTzn/pypEyVJd7W4TwOGjTIVy3LNW7TUyfQTmjNzhlJTjyq8WnVNnzVHZcuWMx3NUnY93vF1yqlC6WJ685MfTUdxK7ue53at264/3xzvS+xyvO1aNy7xczgcuW67TZs2zel+VFSUbr/99pz748aN02+//aZJkya5HGRfWubfP8gHhQUWNB3BiJR0ex7vyO5vm45gxIklj//9g+Az7Przze9z2EHlEM88z0d/ssdt+xp6Z7jb9nWZSx323r17/+XXn3vuuX8UBgAAAIAz237SKQAAAHyDn3x7ETufdAoAAAB4MCbsAAAAgAdjSQwAAAC8Wj7fXhFDhx0AAADwZHTYAQAA4NXosAMAAAAwhg47AAAAvJqfn2+32OmwAwAAAB6MDjsAAAC8GmvYAQAAABhDhx0AAABezceXsNNhBwAAADwZHXYAAAB4tXw+3mKnww4AAAB4MDrsAAAA8GpcJQYAAACAMXTYAQAA4NV8fAk7HXYAAADAk9FhBwAAgFfLJ99usdNhBwAAADyYx3TY1+07ajqCETWCipuOYER0pUDTEYw4seRx0xGMSEnPNB0BbhQWWNB0BLgRxxuwnsdM2AEAAIAbwZtOAQAAABhDhx0AAABejQ9OAgAAAGAMHXYAAAB4tXw+voidDjsAAADgweiwAwAAwKv5eIOdDjsAAADgyeiwAwAAwKt56hr2RYsWKSkpSYcPH5YkVatWTb169VLjxo1d2g4TdgAAAMACoaGhGjBggCpUqCBJWrFihZ566iktX75c1apVy/V2mLADAADAq3log13x8fFO9/v166ekpCRt2bKFCTsAAABghaysLGVlZTmNBQQEKCAg4C+fl52drQ8++EBnz55VdHS0S/tkwg4AAACv5s6rqMyePVvTpk1zGuvdu7f69Olzzcf/+OOP6tSpk86dO6fChQtr+vTpCg8Pd2mffg6Hw3HDifPQ69/+YjqCETWCipuOYER0pUDTEeBGKemZpiPAjcICC5qOAMAiBT201fuGG+eRD0eFutRhz8rKUkpKin7//Xd99NFHWrJkid5++22XJu0e+m0HAAAAcsfPjYvYc7P85crHV6xYUZJUu3Ztbdu2TW+99ZZGjhyZ621wHXYAAADATRwOx1Ud+r9Dhx0AAABezUMvEqNJkybpjjvuUGhoqM6cOaM1a9bom2++0WuvvebSdpiwAwAAABZIS0vToEGDdPToURUrVkwRERF67bXX1KhRI5e2w4QdAAAAXs1TP+l0zJgxebId1rADAAAAHsyWE/bNa1fq9SFPaPLj92vy4/drwYt9tff7b0zHstyqxW/opWe66skHmqrPw801ZdRApRw6YDqW2yQnLVSLu+NVP7q2OnVop/9t+s50JLewY93btmzSiEF99PB9d6l5oyit/79PTUdyC7vWLdnzPJeom7rtUTdsOmEvVjJEjTt2U5dR09Vl1HRVjKyrdyeNUOqh/aajWWrXts2Kb/WAhr8yTwNfflUXs7M1cVhfncvMMB3Nch+8v0bjxyaq+xM9lbx0herVi1GvHt2VcuSI6WiWsmvdmRkZqhweoV79B5uO4lZ2rduu5zl1U7cd6s4tPzfeTLDlhD28Xqyq1r1NJcPKq2RYed3x4GMKKFhIR/bsNB3NUgNGTdHtzVqrXMUqqlClurr1G65jqb9q/55dpqNZbsGb89W2fXu1e6CDqlStqkFDhio0LFSLk5NMR7OUXeuuHxunrk/0VlyTu0xHcSu71m3X85y6qdsOdeMSW07Y/+zixWzt2LBO589lqly1SNNx3CrjzGlJUpGivv1pq+ezsrRzx3bFNoxzGo9t2Ejfb9lsKJX17Fo37MWu5zl1U7fk+3W7ws/PfTcTXLpKzI4dO1SsWDHdfPPNkqT//ve/SkpKUkpKisqWLatHHnlErVq1siRoXks9uE8LXuyrC+ezFFCwkNo+M0Ih5SqajuU2DodDSXOnqHrNKJWvVNV0HEudSD+h7OxsBQcHO40HB4coLS3VUCrr2bVu2Itdz3Pqpm7J9+vGH1zqsA8dOlSHDx+WJC1ZskQvvPCCatWqpSeffFK1a9fWsGHDtHTpUkuC5rWSYeWVMHqWOr/4qqLvvFerZ09Q2mH7vAFzwcwJOrh/j54cNMp0FLe58mOLHQ6HWz/K2BS71g17set5Tt2XUDf8/PzcdjPBpQ77vn37VKFCBUnSokWL9Pzzz6tjx445X69du7ZmzZqlBx54IG9TWsD/pvwKCi0nSQqrEqGUn3/Udx8sV/Nuz5gN5gYLZk7Ulq+/0JBxs1UypIzpOJYLCgySv7+/0tLSnMaPHz+m4OAQQ6msZ9e6YS92Pc+pm7ol368bf3Cpw16gQAEdP35ckvTbb78pKirK6etRUVE6dOhQ3qVzJ4dD2ReyTKewlMPh0IKZE7Rpw2caNGa6SoWWNR3JLfIHBKhGZE1tXP+V0/jG9esVVTfaUCrr2bVu2Itdz3Pqpm7J9+t2RT433kxwqcN+xx13KCkpSaNHj1b9+vX1wQcf6JZbbsn5+vvvv5/TgfdknyfPU5WoBioeXEpZmRnauWGdftm5VR0G5c2nUXmqBTMmaMPnH+rp4RNUsFARpR8/JkkqXKSIAgoUNJzOWp27JGjo4EGKrFVLUVHRWrYkWSkpKerQsZPpaJaya90ZZ8/qyKFfcu7/euSw9v60S8WKl1Dp0DCDyaxl17rtep5TN3XboW5c4tKEfcCAAXrooYf0yCOPqFatWpo/f76++eYbVa1aVfv27dOWLVs0ffp0q7LmmTO/p2vVrHE6k35cBQoXUambK6vDoDGqXDvGdDRLfbpmmSRp7OCeTuPdnhmu25u1NhHJbZq3aKmT6Sc0Z+YMpaYeVXi16po+a47Kli1nOpql7Fr3T7u267k+j+fcnzN1oiTprhb3acAw333fhl3rtut5Tt3UbYe6c8vX1/L7ORwOhytP+P333zVnzhytW7dOBw8e1MWLF1W6dGlFR0era9euql279g0Fef3bX/7+QT6oRpBvX1LxeqIrBZqOADdKSc80HQFuFBbo26/YAXZW0KVWr/ss3uK+D5B6sK77lxS7/G0vXry4BgwYoAEDBliRBwAAAHCJb/fX+eAkAAAAwKN56AsbAAAAQO74+hp2OuwAAACAB6PDDgAAAK/m6x1oX68PAAAA8Gp02AEAAODVWMMOAAAAwBgm7AAAAIAHY0kMAAAAvJpvL4ihww4AAAB4NDrsAAAA8Go+/p5TOuwAAACAJ6PDDgAAAK+Wz8dXsdNhBwAAADwYHXYAAAB4NdawAwAAADCGDjsAAAC8mh9r2AEAAACYQocdAAAAXo017AAAAACMocMOAAAAr+br12H3mAn7w9EVTEcwYtHmX0xHgBtFVwo0HcGIsMCCpiMAsEhKeqbpCEas23fUdAQjHqtvz/maaR4zYQcAAABuBGvYAQAAABhDhx0AAABejQ47AAAAAGPosAMAAMCr8UmnAAAAAIxhwg4AAAB4MJbEAAAAwKvl8+0VMXTYAQAAAE9Ghx0AAABejTedAgAAADCGDjsAAAC8Gh+cBAAAAMAYOuwAAADwaqxhBwAAAGAMHXYAAAB4Na7DDgAAAMAYOuwAAADwaqxhBwAAAGAMHXYAAAB4NV+/DrutJ+zJSQv1xvx5SktNVdXwaho0+HnVi7nVdCzLbF67Ups/WamTqb9JkkLKV1TDto+oalQDw8mstWrxG9q0/jOlHDqg/AEFFF6jth5M6K2w8hVNR3MLu53nl1E3dVO3b9q2ZZOWLnpDu3ft1PFjqXohcbIa3hFvOpal7PrvN/5g2yUxH7y/RuPHJqr7Ez2VvHSF6tWLUa8e3ZVy5IjpaJYpVjJEjTt2U5dR09Vl1HRVjKyrdyeNUOqh/aajWWrXts2Kb/WAhr8yTwNfflUXs7M1cVhfncvMMB3NcnY8zyXqpm7q9mWZGRmqHB6hXv0Hm47iNnb999sVfm68uWL27Nlq3769oqOjFRsbq169eunnn392uT7bTtgXvDlfbdu3V7sHOqhK1aoaNGSoQsNCtTg5yXQ0y4TXi1XVurepZFh5lQwrrzsefEwBBQvpyJ6dpqNZasCoKbq9WWuVq1hFFapUV7d+w3Us9Vft37PLdDTL2fE8l6ibuqnbl9WPjVPXJ3orrsldpqO4jV3//fYF33zzjf79739r8eLFmj9/vrKzs9WtWzedPXvWpe3YcsJ+PitLO3dsV2zDOKfx2IaN9P2WzYZSudfFi9nasWGdzp/LVLlqkabjuFXGmdOSpCJFixtOYi27nufUTd0SdcN32fnf77+Sz8/PbTdXzJs3T+3atVO1atV0yy23KDExUUeOHNH27dtd2o5La9hHjRqlFi1a6NZbvXt93In0E8rOzlZwcLDTeHBwiNLSUg2lco/Ug/u04MW+unA+SwEFC6ntMyMUUs4ea7klyeFwKGnuFFWvGaXylaqajmMpu57n1E3dEnXD99j9329fcerUKUlSiRIlXHqeSxP2hQsXatGiRapQoYLat2+vtm3bqlSpUi7t0JP4XfFXksPhuGrM15QMK6+E0bOUefa0fvr2S62ePUEPD3vFNj/0C2ZO0MH9ezR0wmzTUdzGjue5RN2XUbdvs2vddmT3f789SVZWlrKyspzGAgICFBAQ8JfPczgcSkxMVExMjKpXr+7SPl1eEvP666/rjjvu0Ouvv66mTZuqZ8+eWrdunS5evOjqpowJCgySv7+/0tLSnMaPHz+m4OAQQ6ncw/+m/AoKLaewKhFq3LGbSleoou8+WG46llssmDlRW77+QoMTZ6hkSBnTcSxn1/Ocuqlbom74Hjv/+50b7nzT6ezZsxUTE+N0mz377xuBI0eO1E8//aRJkya5XJ/LE/bq1atr6NCh+uKLLzR+/HhlZWXpqaeeUuPGjTV58mQdOHDA5RDulj8gQDUia2rj+q+cxjeuX6+outGGUhnicCj7QtbfP86LORwOLZg5QZs2fKZBY6arVGhZ05Hcwq7nOXVTt0TdsAEb/PvtqXr06KFNmzY53Xr06PGXzxk1apQ+/fRTvfnmmwoNDXV5nzd8Hfb8+fOrZcuWatmypY4cOaJly5Zp+fLlmjNnjnbu9Px3LXfukqChgwcpslYtRUVFa9mSZKWkpKhDx06mo1nm8+R5qhLVQMWDSykrM0M7N6zTLzu3qsOgMaajWWrBjAna8PmHenr4BBUsVETpx49JkgoXKaKAAgUNp7OWHc9zibqpm7p9WcbZszpy6Jec+78eOay9P+1SseIlVDo0zGAy69j132+XuHElWG6Wv1zmcDg0atQoffzxx1qwYIFuvvnmG9pnnnxwUtmyZdWnTx/17t1b69evz4tNWq55i5Y6mX5Cc2bOUGrqUYVXq67ps+aobNlypqNZ5szv6Vo1a5zOpB9XgcJFVOrmyuowaIwq144xHc1Sn65ZJkkaO7in03i3Z4br9matTURyGzue5xJ1Uzd1+7Kfdm3Xc30ez7k/Z+pESdJdLe7TgGGjTMWylF3//fYFL730klatWqUZM2aoSJEiSk299KbwYsWKqWDB3DcN/RwOhyO3D46Pj9eyZcsUFBTkeuK/kXkhzzfpFRZt/uXvH+SDagT59iUVrye6UqDpCACQp1LSM01HMGLdvqOmIxjxWP0KpiNc09d7T7ptX7dVzf0VXiIiIq45npiYqHbt2uV6Oy512D/99FNXHg4AAADY1o8//pgn28mTJTEAAACAKb5+NVNbftIpAAAA4C3osAMAAMCr+XiDnQ47AAAA4MnosAMAAMC7+XiLnQ47AAAA4MHosAMAAMCr+fl4i50OOwAAAODB6LADAADAq3EddgAAAADG0GEHAACAV/PxBjsddgAAAMCT0WEHAACAd/PxFjsddgAAAMCDMWEHAAAAPBhLYgAAAODV+OAkAAAAAMbQYQcAAIBX44OTAAAAABhDhx0AAABezccb7J4zYV+0+RfTEYx4OLqC6Qhwo837001HMCK6UqDpCIDlUtIzTUcwIiywoOkIRtQIKm46AmzEYybsAAAAwA3x8RY7a9gBAAAAD0aHHQAAAF6N67ADAAAAMIYOOwAAALwa12EHAAAAYAwddgAAAHg1H2+w02EHAAAAPBkddgAAAHg3H2+x02EHAAAAPBgddgAAAHg1rsMOAAAAwBgm7AAAAIAHY0kMAAAAvBofnAQAAADAGDrsAAAA8Go+3mCnww4AAAB4MjrsAAAA8G4+3mKnww4AAAB4MDrsAAAA8Gq+/sFJtpywb167Ups/WamTqb9JkkLKV1TDto+oalQDw8ncIzlpod6YP09pqamqGl5NgwY/r3oxt5qOZTm71b1q8RvatP4zpRw6oPwBBRReo7YeTOitsPIVTUdzC7sd78uo2z51b9uySUsXvaHdu3bq+LFUvZA4WQ3viDcdyy3sdrzt/vscNl0SU6xkiBp37KYuo6ary6jpqhhZV+9OGqHUQ/tNR7PcB++v0fixier+RE8lL12hevVi1KtHd6UcOWI6mqXsWPeubZsV3+oBDX9lnga+/KouZmdr4rC+OpeZYTqa5ex4vCXqtlvdmRkZqhweoV79B5uO4lZ2PN52/n2eW35+7rsZqc/hcDjM7NrZ69/+YnT/U3q0U5OHuiuqSQu37vfh6Apu3d+/O3VQjchIDXvhpZyxNve2UNP4u/R0v2fdmsWdPKXuzfvT3bavK/1+8oT6PtxcQ8bNUkStaLfuO7pSoFv35ynH292o22zdKemZbtvXlZo3ijLWYQ8LLOjW/XnK8bbr7/PY8EC37i+3fvz1rNv2FRFa2G37usyWHfY/u3gxWzs2rNP5c5kqVy3SdBxLnc/K0s4d2xXbMM5pPLZhI32/ZbOhVNaza91XyjhzWpJUpGhxw0msZdfjTd32qtuuON6X2OX3uSv83HgzwZZr2CUp9eA+LXixry6cz1JAwUJq+8wIhZTz7bVgJ9JPKDs7W8HBwU7jwcEhSktLNZTKenat+88cDoeS5k5R9ZpRKl+pquk4lrLr8aZue9VtVxxve/0+xx9cnrAvWLBAP/zwgxo3bqyWLVtqxYoVmjNnji5evKi7775bffv21U03ef7fASXDyith9Cxlnj2tn779UqtnT9DDw17x+Um7JPldsQDL4XBcNeaL7Fq3JC2YOUEH9+/R0AmzTUdxG7seb+q+xC5125Wdj7cdf5/nio8ffpdm1tOnT9e8efMUFxenMWPG6NChQ5o3b566du2qfPny6Y033tBNN92kvn37WpU3z/jflF9BoeUkSWFVIpTy84/67oPlat7tGbPBLBQUGCR/f3+lpaU5jR8/fkzBwSGGUlnPrnVftmDmRG35+gsNGTdbJUPKmI5jObseb+q2V912Zffjbbff5/iDS2vYly9frrFjx+rVV1/Va6+9pv/85z8aOnSoevbsqR49emjkyJFatWqVVVmt5XAo+0KW6RSWyh8QoBqRNbVx/VdO4xvXr1dUXfe+acWd7Fq3w+HQgpkTtGnDZxo0ZrpKhZY1Hckt7Hq8qdtedduVXY+3XX+fu8LPjf+Z4FKHPTU1VbVq1ZIk3XLLLcqXL59q1KiR8/XIyEgdPXo0bxNa4PPkeaoS1UDFg0spKzNDOzes0y87t6rDoDGmo1muc5cEDR08SJG1aikqKlrLliQrJSVFHTp2Mh3NUnase8GMCdrw+Yd6evgEFSxUROnHj0mSChcpooAC7r2qg7vZ8XhL1G23ujPOntWRQ39cYe3XI4e196ddKla8hEqHhhlMZi07Hm87/z7HJS5N2ENCQrRnzx6VLVtW+/fvV3Z2tvbs2aNq1apJkvbs2aOSJUtaEjQvnfk9XatmjdOZ9OMqULiISt1cWR0GjVHl2jGmo1mueYuWOpl+QnNmzlBq6lGFV6uu6bPmqGzZcqajWcqOdX+6Zpkkaezgnk7j3Z4ZrtubtTYRyW3seLwl6rZb3T/t2q7n+jyec3/O1ImSpLta3KcBw0aZimU5Ox5vO/8+zy1ffwuDS9dhnzx5shYvXqw777xTGzZsUKtWrbRy5Uo98cQT8vPz06xZs3TPPfdoyJAhLgcxfR12U9x9HXaYZfK6vSa5+zrsgAkmr8Nukruvw+4p7Pr73FOvw77nqPs+RCq8dCG37esylzrsffv2VcGCBfX999+rY8eOeuKJJxQREaEJEyYoIyND8fHxevrpp63KCgAAAFzFxxvsrk3Y/f391bOn88sxrVq1UqtWrfI0FAAAAIBLbP9JpwAAAIAn8/xPOAIAAAD+io+viaHDDgAAAFjk22+/1ZNPPqm4uDhFRERo7dq1Lm+DCTsAAAC8mid/cNLZs2cVERGhF1544YbrY0kMAAAAYJHGjRurcePG/2gbTNgBAADg1Xz9g5OYsAMAAAC5lJWVpaysLKexgIAABQQEWLZP1rADAADAq/m58TZ79mzFxMQ43WbPnm1pfXTYAQAAgFzq0aOHEhISnMas7K5LTNgBAADg7dy4ht3q5S/XwoQdAAAAsMiZM2f0yy+/5Nw/dOiQdu7cqRIlSqhs2bK52gYTdgAAAHi1G7k+urv88MMPevTRR3PuJyYmSpLatm2rsWPH5mobTNgBAAAAi9x222368ccf/9E2mLADAADAq/n6ddi5rCMAAADgweiwAwAAwKv5eIOdDjsAAADgyeiwAwAAwKuxhh0AAACAMUzYAQAAAA/m53A4HKZDSFLmBdMJAFhl8/500xGMiO8wzHQEI058O810BCMWbf7l7x/kgzYdOmM6ghGv3FvDdAQjCnroYupDJ7Lctq/yQQFu29dldNgBAAAAD+ahfycBAAAAucObTgEAAAAYQ4cdAAAAXs3HG+x02AEAAABPRocdAAAAXo017AAAAACMocMOAAAAr+bn46vY6bADAAAAHowOOwAAALybbzfY6bADAAAAnowOOwAAALyajzfY6bADAAAAnowOOwAAALwa12EHAAAAYAwddgAAAHg1rsMOAAAAwBg67AAAAPBuvt1gp8MOAAAAeDIm7AAAAIAHs/WSmOSkhXpj/jylpaaqang1DRr8vOrF3Go6luWom7p9ue5Vi9/QpvWfKeXQAeUPKKDwGrX1YEJvhZWvaDqapfz982lYj5bq1PJWlQkurl/TfteClRs1du6HcjgcpuNZzm7n+ea1K7X5k5U6mfqbJCmkfEU1bPuIqkY1MJzMve6uHqz7a5bWp3uOa9m230zHsZzdznNX+PiKGPt22D94f43Gj01U9yd6KnnpCtWrF6NePbor5cgR09EsRd3U7et179q2WfGtHtDwV+Zp4Muv6mJ2tiYO66tzmRmmo1nq2a7N9PgDceo3donqtntZQ6esUL9H71KvTo1NR7OcHc/zYiVD1LhjN3UZNV1dRk1Xxci6enfSCKUe2m86mttUCCyoRpUCdehkpukobmHH8xx/sO2EfcGb89W2fXu1e6CDqlStqkFDhio0LFSLk5NMR7MUdVO3r9c9YNQU3d6stcpVrKIKVaqrW7/hOpb6q/bv2WU6mqVuq1NZqz7fqg++3K5fUo5r+dot+mTjLtWLrGA6muXseJ6H14tV1bq3qWRYeZUMK687HnxMAQUL6cienaajuUUBfz91rV9Wizan6GxWtuk4bmHH89wVfn7uu5lgywn7+aws7dyxXbEN45zGYxs20vdbNhtKZT3qpm7J9+u+UsaZ05KkIkWLG05irQ1b9qppgwiFVygtSapdvZxi61bRh19tN5zMWpzn0sWL2dqxYZ3On8tUuWqRpuO4xYN1Q7X919P6MfWs6ShuwXkOl9ewHz16VElJSdq0aZNSU1Pl7++v8uXL684771S7du3k7+9vRc48dSL9hLKzsxUcHOw0HhwcorS0VEOprEfd1C35ft1/5nA4lDR3iqrXjFL5SlVNx7HUxPkfq3jRQvp++TBlZzvk7++nEdNXafEHm0xHs5Sdz/PUg/u04MW+unA+SwEFC6ntMyMUUs6336shSTHliuvmEgU1/rP9pqO4jZ3P89zig5P+ZNu2bWrZsqXWrVunrKws7d+/X5GRkSpUqJDGjx+vf//73zp9+rRVWfOc3xWvazgcjqvGfBF1X0Ldvm/BzAk6uH+Pnhw0ynQUy3W4J0YPtayvrs+/qdiHx+nxFxbomc536t/33mY6mlvY8TwvGVZeCaNnqfOLryr6znu1evYEpR0+YDqWpQIL3aQH6pTRm98d0YWLvv9m6ivZ8TzHJS512MeMGaOuXbuqd+/ekqT//ve/WrhwoRYvXqyTJ0+qS5cu+s9//qNhw4ZZEjavBAUGyd/fX2lpaU7jx48fU3BwiKFU1qNu6pZ8v+7LFsycqC1ff6Eh42arZEgZ03EsN+aZNpo4/2Mt+fBSR337niOqEFZSAxOaaeHKrw2ns46dz3P/m/IrKLScJCmsSoRSfv5R332wXM27PWM2mIUqBBZU8YI36bmmlXPG/PP5KTyksBpXCdLT/90lX5zG2/k8zy1f/7vFpQ77jh07dP/99+fcv/fee7Vjxw6lpaWpRIkSGjhwoD788MM8D5nX8gcEqEZkTW1c/5XT+Mb16xVVN9pQKutRN3VLvl+3w+HQgpkTtGnDZxo0ZrpKhZY1HcktChUM0EXHRaex7IsO5cvn229Vsut5fk0Oh7IvZJlOYakfU8/q5bU/K/HTfTm3Aycy9N3B35X46T6fnKxLnOdwscMeHByso0eP6uabb5YkpaWl6cKFCypatKgkqWLFijp58mTep7RA5y4JGjp4kCJr1VJUVLSWLUlWSkqKOnTsZDqapaibun297gUzJmjD5x/q6eETVLBQEaUfPyZJKlykiAIKFDSczjpr/m+bnut2jw6mnNCOvSmqe0t59X2kqd5asdF0NMvZ8Tz/PHmeqkQ1UPHgUsrKzNDODev0y86t6jBojOloljp34aJSTp27aux0VvZV477Gjuc5/uDShP3OO+/Uiy++qIEDByogIEAzZsxQ/fr1VbDgpX8E9+3bpzJlvOOl5+YtWupk+gnNmTlDqalHFV6tuqbPmqOyZcuZjmYp6qZuX6/70zXLJEljB/d0Gu/2zHDd3qy1iUhu0X/cEo3o1VpTnu+oUkFFlZJ6UvOWfqUxc943Hc1ydjzPz/yerlWzxulM+nEVKFxEpW6urA6Dxqhy7RjT0WARO57n+IOfw4WPwDtz5oyGDh2qjz/+WNnZ2apbt64mTJiQ03H/8ssvderUKbVo0cLlIJkXXH4KAC+xeX+66QhGxHfw7PfzWOXEt9NMRzBi0eZfTEcwYtOhM6YjGPHKvTVMRzCioMvXF3SP9Az3XY8/sJD7r4jo0re9SJEi+s9//qNz587pwoULKlKkiNPX4+LirvNMAAAAADfihv5OKlCggAoUKJDXWQAAAACXcR12AAAAAMZ46EokAAAAIHe4DjsAAAAAY+iwAwAAwKv5eIOdDjsAAADgyZiwAwAAAB6MJTEAAADwbj6+JoYOOwAAAODB6LADAADAq/HBSQAAAACMocMOAAAAr8YHJwEAAAAwhg47AAAAvJqPN9jpsAMAAACejA47AAAAvJuPt9jpsAMAAAAejAk7AAAAvJqfG/+7EQsXLlR8fLxq166tdu3a6bvvvnPp+UzYAQAAAIusWbNGiYmJ6tmzp1asWKGYmBh1795dR44cyfU2mLADAADAq/n5ue/mqvnz56t9+/bq0KGDqlatqqFDhyo0NFRJSUm53gYTdgAAACCXsrKydPr0aadbVlbWdR+7fft2xcXFOY03atRImzdvzvU+PeYqMQU9JgmAvBYbHmg6ghEZm6eZjgA3eqx+BdMRjHisvukEgHvnkVOnzta0ac6/33v37q0+ffpc9dgTJ04oOztbwcHBTuMhISFKTU3N9T6ZJgMAAAC51KNHDyUkJDiNBQQE/OVz/K5YS+NwOK4a+ytM2AEAAIBcCggI+NsJ+mVBQUHy9/dXWlqa0/ixY8cUEhKS632yhh0AAACwQEBAgGrWrKmvvvrKaXz9+vWKjo7O9XbosAMAAAAWSUhI0KBBg1SrVi1FR0crOTlZKSkp6tSpU663wYQdAAAAsEjLli114sQJzZgxQ0ePHlX16tU1Z84clStXLtfb8HM4HA4LMwIAAAD4B1jDDgAAAHgwJuwAAACAB2PCDgAAAHgwJuwAAACAB7P1hH3hwoWKj49X7dq11a5dO3333XemI1nu22+/1ZNPPqm4uDhFRERo7dq1piNZbvbs2Wrfvr2io6MVGxurXr166eeffzYdy3KLFi3Svffeq3r16qlevXrq2LGjPv/8c9Ox3G727NmKiIjQ6NGjTUex1NSpUxUREeF0a9SokelYbvHbb79pwIABuu222xQVFaX7779fP/zwg+lYloqPj7/qeEdEROill14yHc1SFy5c0OTJkxUfH686derozjvv1LRp03Tx4kXT0Sx3+vRpjR49Wk2bNlWdOnXUqVMnbd261XQsuIltL+u4Zs0aJSYmasSIEapXr57eeecdde/eXatXr1bZsmVNx7PM2bNnFRERoXbt2qlPnz6m47jFN998o3//+9+qXbu2srOzNXnyZHXr1k2rV69W4cKFTcezTGhoqAYMGKAKFSpIklasWKGnnnpKy5cvV7Vq1Qync4+tW7cqOTlZERERpqO4RbVq1TR//vyc+/7+/gbTuMfJkyf10EMP6bbbbtPcuXNVsmRJHTx4UMWLFzcdzVJLly5VdnZ2zv3du3crISFBzZs3N5jKenPnztU777yjcePGKTw8XD/88IOGDBmiYsWKqUuXLqbjWWrYsGHavXu3xo8fr9KlS+u9995TQkKC1qxZozJlypiOB4vZdsI+f/58tW/fXh06dJAkDR06VF9++aWSkpL07LPPGk5nncaNG6tx48amY7jVvHnznO4nJiYqNjZW27dvV/369Q2lsl58fLzT/X79+ikpKUlbtmyxxYT9zJkzGjhwoF5++WXNnDnTdBy38Pf3V6lSpUzHcKu5c+cqNDRUiYmJOWPly5c3mMg9SpYs6XR/zpw5qlChgho0aGAokXts2bJFd955p5o0aSLp0rFevXq1z7+ikpmZqY8++kgzZszI+XerT58+Wrt2rRYtWqR+/foZTgir2XJJTFZWlrZv3664uDin8UaNGmnz5s2GUsFdTp06JUkqUaKE4STuk52drdWrV+vs2bMufRSyNxs5cqQaN26shg0bmo7iNgcOHFBcXJzi4+PVr18/HTx40HQky3366aeqVauW+vbtq9jYWLVp00aLFy82HcutsrKy9N5776l9+/by8/MzHcdSMTEx2rhxo/bt2ydJ2rVrlzZt2uTzjagLFy4oOztbBQoUcBovWLCg/ve//xlKBXeyZYf9xIkTys7OVnBwsNN4SEiIUlNTDaWCOzgcDiUmJiomJkbVq1c3HcdyP/74ozp16qRz586pcOHCmj59usLDw03Hstzq1au1Y8cOLV261HQUt6lTp47GjRunSpUq6dixY5o5c6Y6deqkVatWKSgoyHQ8yxw8eFBJSUlKSEjQk08+qa1bt+rll19WQECA2rRpYzqeW6xdu1anTp1S27ZtTUexXPfu3XXq1Cm1aNFC/v7+ys7OVr9+/dS6dWvT0SxVtGhRRUdHa8aMGapSpYpCQkK0atUqff/996pYsaLpeHADW07YL7uyE+FwOHy+O2F3I0eO1E8//aRFixaZjuIWlStX1ooVK/T777/ro48+0nPPPae3337bpyftKSkpGj16tF5//fWrulG+7MoOY926ddWsWTOtWLFCCQkJhlJZz+FwqFatWurfv78kKTIyUnv27FFSUpJtJuzLli3THXfcYYt1zGvWrNF7772nV155ReHh4dq5c6cSExNVunRpn/+DZfz48Xr++ed1xx13yN/fX5GRkWrdurV27NhhOhrcwJYT9qCgIPn7+ystLc1p/NixYwoJCTGUClYbNWqUPv30U7399tsKDQ01HcctAgICcrovtWvX1rZt2/TWW29p5MiRhpNZZ/v27Tp27JjatWuXM5adna1vv/1WCxcu1LZt22zxZszChQurevXq2r9/v+kolipVqpSqVq3qNFalShV9+OGHhhK51+HDh7V+/XpNnTrVdBS3GD9+vJ544gm1atVKkhQREaEjR45o9uzZPj9hr1Chgt5++22dPXtWp0+fVunSpfXMM8/Y4j0bsOmEPSAgQDVr1tRXX32lZs2a5YyvX79ed955p8FksILD4dCoUaP08ccfa8GCBbr55ptNRzLG4XAoKyvLdAxL/etf/9LKlSudxoYMGaIqVaqoe/futpisS5fWNe/du1cxMTGmo1iqXr16OeuZL9u/f7/KlStnKJF7vfvuuwoODs55E6avy8zMvOqVcH9/fzkcDkOJ3K9w4cIqXLiwTp48qS+//FIDBw40HQluYMsJuyQlJCRo0KBBqlWrlqKjo5WcnKyUlBR16tTJdDRLnTlzRr/88kvO/UOHDmnnzp0qUaKEz17O8qWXXtKqVas0Y8YMFSlSJOd9CsWKFVPBggUNp7POpEmTdMcddyg0NFRnzpzRmjVr9M033+i1114zHc1SRYsWver9CYULF1ZgYKBPv29h3Lhxatq0qcLCwnT8+HHNnDlTp0+f9vmuY5cuXfTQQw9p1qxZatGihbZu3arFixf79KtIl128eFHvvvuu2rRpo5tussc/502bNtWsWbNUtmzZnCUxl6/65uu++OILORwOVa5cWb/88ovGjx+vypUrO72aCN/l57DTn6VXWLhwoebNm6ejR4+qevXqGjJkiE9f5k+Svv76az366KNXjbdt21Zjx441kMh617sGd2Jiok//onv++ee1ceNGHT16VMWKFVNERIS6d+9umw/T+bPOnTvrlltu0dChQ01HsUy/fv307bffKj09XUFBQapbt66efvppn36/wmXr1q3TpEmTtH//fpUvX14JCQl68MEHTcey3Jdffqlu3brpgw8+UOXKlU3HcYvTp09rypQpWrt2rY4dO6bSpUurVatWeuqppxQQEGA6nqXWrFmjSZMm6ddff1VgYKDuvvtu9evXT8WKFTMdDW5g6wk7AAAA4OlseR12AAAAwFswYQcAAAA8GBN2AAAAwIMxYQcAAAA8GBN2AAAAwIMxYQcAAAA8GBN2AAAAwIMxYQcAAAA8GBN2AAAAwIMxYQcAAAA8GBN2AAAAwIMxYQcAAAA82P8D8RtvnZAoN6UAAAAASUVORK5CYII=",
|
||
"text/plain": [
|
||
"<Figure size 1000x700 with 2 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAuwAAAJbCAYAAAC/78ExAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABiRklEQVR4nO3deVxUZf//8TegiKi4oIRr7uQurqEWpmmpWS6ZWJmSmeVSt7mUYaWW4VLa4gZWmmaEaVkqdXd72923JCvLJde0NDcMUNxFFOf3hz+pUSvGOHPNzHk9e8zj0VzMnPP+cM7gxYdrzvg5HA6HAAAAAHgkf9MBAAAAAPw5JuwAAACAB2PCDgAAAHgwJuwAAACAB2PCDgAAAHgwJuwAAACAB2PCDgAAAHgwJuwAAACAB2PCDgAAAHgwJuyAm2zfvl1jxoxRu3bt1KBBA0VGRqp79+6aO3eujh49aum+t27dqvvvv19NmzZVRESE5s+fX+D7iIiI0Ouvv17g2/07H3zwgSIiIhQREaFvvvnmiq87HA516NBBERER6tu37zXtY9GiRfrggw9ces4333zzp5muVUpKirp06aKGDRsqIiJC27ZtK7Bt59el7/f+/fvdvm93uNbzeP/+/YqIiHD5PAGA/ChkOgBgB4sXL9b48eNVrVo1DRgwQDVr1tT58+e1efNmvffee9qwYYNmzpxp2f6ffvppnTlzRtOmTVPJkiVVsWLFAt9HcnKywsPDC3y7+VWsWDEtWbJELVu2dBr/9ttvtXfvXhUrVuyat52UlKTSpUurR48e+X5OvXr1lJycrJo1a17zfv/oyJEjGj16tNq0aaPnnntOgYGBqlq1aoFsGwDg2ZiwAxZbv369xo0bp1atWmnWrFkKDAzM+1rr1q0VGxurL7/80tIMO3fuVK9evRQdHW3ZPho3bmzZtvOjc+fOWr58uZ577jkVL148b3zJkiWKjIzUyZMn3ZLj3Llz8vPzU/HixQv0e7J7926dO3dOd955p1q0aFEg2zxz5oyKFi1aINsCAFiHJTGAxRISEuTn56fnn3/eabJ+SWBgoNq3b593/8KFC5o7d65uv/121a9fX1FRURo9erQOHTrk9Ly+ffvqjjvu0KZNm3TvvfeqUaNGat++vRITE3XhwgVJvy9fOH/+vJKSkvKWjkjS66+/nvf/f3S1JQ9ff/21+vbtq5YtW6phw4Zq27athg0bpjNnzuQ95mpLCX766Sc9+uijat68uRo0aKC77rpLH374odNjLi0dWbFihaZPn642bdqoSZMm6t+/v3755Zf8fpvVpUsXSdKKFSvyxk6cOKHPPvtMPXv2vOpzZsyYoV69eqlFixZq0qSJunfvrvfff18OhyPvMe3atdPOnTv17bff5n3/2rVr55R92bJlmjRpkm666SY1aNBAv/766xVLYo4cOaLo6GjFxMTo3LlzedvftWuXGjdurFGjRv1pbU899ZTuvfdeSdLw4cOvWN7z3//+V71791ajRo0UGRmp2NhYrV+/3mkbl473li1b9Nhjj6l58+bq0KHDX35PN2zYoJiYGDVo0EBt2rTRyy+/rPPnz1/xuPyes3/n0rn39ddfa+zYsWrZsqWaNGmi0aNH6/Tp08rIyNDjjz+uZs2aqU2bNpo8ebLT91KSjh49qnHjxummm25S/fr11b59e02fPl05OTlOjzt58mTePiIjIzVgwADt3r37qrn27NmjESNGKCoqSvXr11enTp20aNEil2oDgH+CDjtgodzcXK1du1b16tVT+fLl8/WccePGKTk5Wffff7/atm2rAwcO6NVXX9W3336rDz74QGXKlMl7bEZGhkaNGqXY2FgNHTpU//nPf/Tyyy8rLCxM3bp1U9u2bZWcnKzevXvrtttu04MPPuhyDfv379egQYPUrFkzTZw4USEhIfrtt9/05Zdf6ty5c3/aof3ll18UExOj0NBQxcXFqXTp0vr444/11FNPKTMzUwMHDnR6/LRp09SkSRNNnDhRJ0+e1EsvvaRHH31UKSkpCggI+NucxYsX12233aalS5cqJiZG0sXJu7+/vzp16qS33377iuccOHBAvXv3VoUKFSRdnKC+8MIL+u233zR06FBJFyf1jz32mEqUKKHnnntOkq74xWvatGlq3Lixxo8fL39/f4WGhiozM9PpMWXKlNG0adP0wAMP6KWXXtKYMWN05swZPf744ypfvrzGjx//p7UNHjxYDRo00IQJE/TEE0+oZcuWeX9FWL58uUaOHJk3oc7JydEbb7yhvn37av78+WrWrJnTtoYNG6bOnTsrJiZGp0+f/tN97tq1S/3791fFihU1adIkBQUF6d1333X6hegSV87Z/Bg7dqw6duyoadOmaevWrZo+fbpyc3O1e/dudejQQb1791Zqaqrmzp2rsLAwxcbGSpLOnj2rBx54QPv27dOwYcMUERGhdevWKTExUdu2bVNiYqKki+9rGDx4sNavX68hQ4aoQYMG+uGHH644Jy99H2JiYlS+fHk9+eSTKleunL766iu98MILysrKyjtPAMBKTNgBC2VlZenMmTOqVKlSvh7/888/Kzk5Wffee6+eeeaZvPG6deuqV69eevvttzV8+PC88aNHj2ru3Llq2LChJKlVq1b69ttvtXz5cnXr1k1lypTJmyyVLVv2mpZobNmyRWfPntXo0aN1ww035I137dr1L583Y8YMnTt3TgsWLMj7ZSU6OlrHjx/XzJkzFRMToxIlSuQ9vmbNmnrppZfy7vv7++tf//qXfvzxx3zn7tmzpx544AHt3LlTtWrV0tKlS3X77bc7LZH5o/j4+Lz/v3Dhglq0aCGHw6EFCxZoyJAh8vPzU926dRUUFPSXS1yqVKmi11577W/zNW3aVP/617/00ksvqXnz5lq1apX279+vxYsXKzg4+E+fV6VKlby18Ndff31ejgsXLmjKlCmqXbu25s6dK3//i380jY6OVocOHfTSSy/pvffec9pWt27d9Nhjj/1t1pkzZ8rhcOjtt99W2bJlJUlt27bVHXfc4fQ4V8/Z/Ljlllv05JNPSrq4bGzDhg1asWKFxowZo/79+0u6eK5/9dVXWr58ed6E/cMPP9SOHTv0yiuvqFOnTnnPDw4O1ksvvaQ1a9aodevW+vLLL/XNN98oLi5ODzzwQN7jChcurOnTpztliY+PV7FixZSUlJR3HrVu3Vo5OTlKTExU3759VbJkSZfqAwBXsSQG8CCXlk90797dabxhw4aqUaOGvv76a6fxcuXK5U3WL4mIiNDBgwcLLFOdOnVUuHBhPfPMM/rwww+1b9++fD1v7dq1ioqKuuIvC927d9eZM2euWLJxaZnJJZeW67hSS4sWLVSlShUtXbpUO3bs0I8//viny2Gki0t9+vfvr6ZNm6pOnTqqV6+eXnvtNR09elSHDx/O9347duyY78c+9NBDatu2rZ544gl9+OGHGjt27FWXJuXH7t27lZ6errvuuitvsi5dfANux44dtXHjRqdlS65k/eabbxQVFZU3WZekgIAAde7c+YrHSfk/Z/Ojbdu2Tvdr1KghSVe8B6NGjRo6cOBA3v21a9cqODhYt99+u9PjLr1Z+FKWS5kv/6Xz8l9Gzp49q7Vr16pDhw4KCgrS+fPn824333yzzp49qw0bNrhcHwC4ig47YKHSpUuraNGi+b4E3qXLO4aFhV3xtbCwsCsmr6VKlbricYGBgTp79qzLWf9MlSpVNH/+fL3xxhuaMGGCTp8+rcqVK6tv377q16/fnz7v6NGjKleu3BXjl2q7/FKWl9dyadlJdnZ2vrP6+fmpR48eWrhwoc6ePauqVatesSTkkk2bNmnAgAFq0aKFnn/+eYWHh6tw4cJatWqV5syZ49J+r1bnX2Xs3r27/ve//6lcuXK666678v3cy2VlZf3p/sPCwnThwgUdP37cadnS1c6tqzl69KjTZP2Sy8dcPWfz4/KOdeHChf90/I9r0y9l9vPzc3pcaGioChUqlJf16NGjKlSokEqXLu30uMu/j0ePHtX58+e1cOFCLVy48KpZLx0DALASE3bAQgEBAbrxxhv15Zdf6tChQ3972cNLk9b09PQrHpuenn7FBOOfKFKkiCQpJyfHaU321SYgzZo1U7NmzZSbm6vNmzdr4cKFevHFF1W2bNm8N3terlSpUsrIyLhiPD09XZIKtJY/6tGjh1577TW99957f7kUY+XKlSpUqJASEhLyvheStGrVKpf3efkE8a+kp6drwoQJqlOnjnbt2qUpU6Zo7NixLu9T+v17+GffZ39/f4WEhFzTtkuVKnXFOnxJV4y585z9O6VKldLGjRvlcDicjsnhw4d1/vz5vCylSpXS+fPnlZWV5ZTv8u9jSEiIAgICdNddd+W96fdy+V3uBgD/BEtiAIsNGjRIDodDY8eOveJKFdLFywCuXr1aknTjjTdKkj7++GOnx2zatEk///xz3tcLwqVrsW/fvt1p/PPPP//T5wQEBKhRo0Z5b77csmXLnz42KipKa9eu1W+//eY0/tFHH6lo0aKWXQbyuuuu04ABA3TLLbeoW7duf/o4Pz8/BQQEOC0lyc7OvuJ7L13s9rvScf8zubm5GjFihPz8/DR37lw98cQTWrhwoT777LNr2l61atV03XXXacWKFU5Xtjl9+rQ+++wzNW7c+Jov29iyZUt9/fXXThP03NxcpaSkOD3Onefs34mKitLp06ev+KVr2bJleV+XlHet/uXLlzs97vI31BYtWlQtW7bU1q1bFRERoQYNGlxxc+cvJADsiw47YLHIyEiNGzdO48ePV8+ePRUTE6NatWrp/Pnz2rp1qxYvXqxatWqpXbt2ql69unr37q133nlH/v7+uvnmm/OuuFG+fPm8N9wVhOjoaJUqVUpxcXF6/PHHFRAQoA8//FBpaWlOj0tKStLatWvVtm1blS9fXmfPntXSpUslXXzj358ZMmSIPv/8cz3wwAMaMmSISpYsqeXLl+t///ufRo0a5fSG04I2cuTIv31MdHS05s2bpxEjRqh37946evSo3nzzzateerN27dpauXKlUlJSVKlSJRUpUuSa1p2/9tprWrdund566y2VK1dODz74oL799lvFxcWpTp06qly5skvb8/f316hRozRy5EgNGjRIvXv3Vk5Ojt58800dP35cI0aMcDnjJY8++qhWr16tfv36aciQIQoKCtKiRYuuWBPvznP273Tr1k2LFi3Sk08+qQMHDqh27dr6/vvvlZCQoOjo6LzztU2bNmrevLmmTp2qM2fOqH79+vrhhx/00UcfXbHNuLg43XvvvbrvvvvUp08fVaxYUadOndLevXu1evVqLViwwG31AbAvJuyAG9xzzz1q2LBh3lrwjIwMFS5cWFWrVtUdd9yh+++/P++x48aNU+XKlbVkyRK9++67Kl68uG666SaNGDGiQLt5xYsX19y5c/Xiiy/mTaB79eqlm266yWmJRp06dbRmzRq9/vrrysjIUHBwsGrXrq3Zs2erTZs2f7r96tWr67333tO0adM0YcIEZWdnq0aNGoqPj3fpE0OtEhUVpRdffFFz587VI488ouuuu0733HOPypQpo7i4OKfHDhs2TBkZGRo7dqxOnTqlihUr5v1VJL/WrFmjxMREDR48OK/TK0mTJk1S9+7dNXz4cL377rtX/YXhr3Tt2lVFixZVYmKihg8fnvdXkAULFqhJkyYubeuPateurXnz5mny5Ml68sknVbJkSd1555267bbbnK4GI7nvnP07RYoU0YIFCzR9+nS98cYbysrK0nXXXacHH3zQ6fKL/v7+mj17tuLj4/XGG2/o3LlzatKkiRITE/OuLnNJzZo19cEHH2jWrFl65ZVXdOTIEZUoUULXX3+9pR9EBgB/5Of4499RAQAAAHgU1rADAAAAHowlMQAASzkcDuXm5v7lYwICAly62g4A2AkTdgCApT788EONGTPmLx+zYMGCvKu3AACcsYYdAGCprKysv/3wsGrVqql48eJuSgQA7nPy5Em9+uqrWrVqlQ4fPqy6devq6aefvuKTyv8KE3YAAADAIv/617+0c+dOjRs3TmFhYfr44481f/58paSk6LrrrsvXNpiwAwAAABbIzs5WkyZNNGvWLLVt2zZv/K677lLbtm3/8hO5/4g17AAAAEA+5eTkXPHJ5YGBgVf9HI3z588rNzdXRYoUcRoPCgrSDz/8kO99esyEPfu86QQAAFy79ONnTUeAG4WFFPn7B/mgII+ZOTorGjn07x9UQKY8GKEZM2Y4jQ0dOlTDhg274rHFixdXZGSkZs2aperVq6ts2bJasWKFNm7cqOuvvz7f+/SYJTFM2AEA3owJu70wYfcs7pywH/tmWr477JK0d+9ePf300/ruu+8UEBCgunXrqmrVqtq6datSUlLytU8P/bYDAAAA+eTnvs8C/avJ+dVUqVJF77zzjk6fPq2TJ08qLCxM//rXv1SpUqV8b4NPOgUAAAAsFhwcrLCwMB07dkxfffWV2rdvn+/n0mEHAACAd/PgT0r+8ssv5XA4VK1aNe3du1dTpkxRtWrV1KNHj3xvgwk7AAAAYJETJ05o2rRpOnTokEqVKqWOHTtq+PDhKly4cL63wYQdAAAAsEjnzp3VuXPnf7QNJuwAAADwbm5806kJvl0dAAAA4OXosAMAAMC7efCbTgsCHXYAAADAg9FhBwAAgHdjDTsAAAAAU+iwAwAAwLuxhh0AAACAKXTYAQAA4N1Yww4AAADAFDrsAAAA8G6sYQcAAABgCh12AAAAeDfWsAMAAAAwxdYT9uSkRerUsZ2aRzZQTK8e+uH7daYjuQV1U7cdUDd1+7pN69fpmZFD1btre3WIaqg1X6w2Hckt7Fq3ZM/zPN/8/Nx3M8C2E/ZPP0nRlEnxGvjwo0peskxNmjTV4EEDlXbwoOlolqJu6qZu30Xd9qo7O/uMqteK0NARY0xHcSu71m3X8xwX2XbCvvDteeres6d63N1L1WvU0OgxcQovH67FyUmmo1mKuqmbun0Xddur7hZRNyl20DDd1PZW01Hcyq512/U8zzc/f/fdDLDlhP1cTo62bd2iqFZtnMajWrXWxg3rDaWyHnVTt0Tdvoq67VU37IXzHC5fJebQoUNKSkrSDz/8oMzMTPn5+Sk0NFRNmjRRTEyMypcvb0XOApV1NEu5ubkKDQ11Gg8NLavMzAxDqaxH3dQtUbevom571Q174TyHSxP2devWaeDAgSpfvrxat26tsmXLyuFw6PDhw1q1apUWLlyouXPnqmnTplblLVB+l71xwOFwXDHmi6j7Iur2bdR9EXUDvoPz/C/4+PfBpQl7fHy8evXqpaeffvqqX3/xxRf14osvaunSpQUSziqlS5VWQECAMjMzncaPHDms0NCyhlJZj7qpW6JuX0Xd9qob9sJ5DpfWsO/cuVMxMTF/+vXevXtr586d/ziU1QoHBqpO3Xpam7rGaXxtaqoaNY40lMp61E3dEnX7Kuq2V92wF87zfPDxN5261GEvV66c1q9fr+rVq1/16xs2bFC5cuUKJJjV+vaLVdxTo1W3fn01ahSppe8nKy0tTb16//kvJL6Auqmbun0Xddur7jOnT+vA/r159w8dPKBdP21XSEhJhYV7/vvJrpVd67breY6LXJqwP/jgg3ruuee0ZcsWtWrVSmXLlpWfn58yMjKUmpqq999//0+Xy3ia2zt11rGjWUqcPUsZGemqWau2Zs5JVIUKFU1HsxR1Uzd1+y7qtlfdP23fopFDBuTdn/PaVElSh853avQzL5iKZTm71m3X8zzfDHW+3cXP4XA4XHlCSkqK5s+fry1btig3N1eSFBAQoHr16ql///7q3LnzNQXJPn9NTwMAwCOkHz9rOgLcKCykiOkIRgS5fH1B9ygaPcFt+zrzxbNu29clLn/bO3furM6dO+vcuXPKysqSJJUuXVqFCxcu8HAAAADA3/LnKjFXVbhwYYWFhRVkFgAAAACX8dA/bAAAAAD55ONr2H27OgAAAMDL0WEHAACAd/PxTzqlww4AAAB4MDrsAAAA8G6sYQcAAABgCh12AAAAeDfWsAMAAAAwhQ47AAAAvBtr2AEAAACYQocdAAAA3o017AAAAABMYcIOAAAAeDCWxAAAAMC78aZTAAAAAKbQYQcAAIB3402nAAAAAEyhww4AAADvxhp2AAAAAKbQYQcAAIB38/E17EzYYUT68bOmIxgRFlLEdAQAKFD8XAOsx4QdAAAA3o017AAAAABcdf78eU2fPl3t2rVTw4YN1b59e82YMUMXLlxwaTt02AEAAODdPLTDPnfuXL333nuaPHmyatasqc2bN2vMmDEqUaKE+vXrl+/tMGEHAAAALLBhwwa1b99ebdu2lSRVqlRJK1eu1ObNm13ajmf+OgIAAADkl5+f2245OTk6efKk0y0nJ+eqsZo2baq1a9dq9+7dkqTt27fr+++/V3R0tEvl0WEHAAAA8ikhIUEzZsxwGhs6dKiGDRt2xWMHDhyoEydOqFOnTgoICFBubq6GDx+uO+64w6V9MmEHAACAd3PjGvZBgwYpNjbWaSwwMPCqj01JSdHHH3+sl19+WTVr1tS2bdsUHx+vsLAwde/ePd/7ZMIOAAAA5FNgYOCfTtAvN2XKFD388MPq0qWLJCkiIkIHDx5UQkICE3YAAADYiId+0ml2drb8LssWEBAgh8Ph0naYsAMAAAAWuOWWWzRnzhxVqFAhb0nMvHnz1LNnT5e2w4QdAAAAsMDYsWP16quvavz48Tp8+LDCwsLUu3dvDRkyxKXt+Dlc7clbJPu86QRwp/TjZ01HMCIspIjpCAAsws812EGQh7Z6i3Z/w237OvPhQ27b1yVchx0AAADwYB76exIAAACQTx76ptOCQocdAAAA8GB02AEAAODVLr90oq+hww4AAAB4MDrsAAAA8Gp02AEAAAAYQ4cdAAAA3s23G+z27rAnJy1Sp47t1DyygWJ69dAP368zHckt7Fj3pvXr9MzIoerdtb06RDXUmi9Wm47kNnY83hJ1U7fv183PNXsdb8m+dcPGE/ZPP0nRlEnxGvjwo0peskxNmjTV4EEDlXbwoOlolrJr3dnZZ1S9VoSGjhhjOopb2fV4Uzd126Fufq7Z63jbte788vPzc9vNBNtO2Be+PU/de/ZUj7t7qXqNGho9Jk7h5cO1ODnJdDRL2bXuFlE3KXbQMN3U9lbTUdzKrsebuqnbDnXzc81ex9uudeMiW07Yz+XkaNvWLYpq1cZpPKpVa23csN5QKuvZtW67suvxpm7qlny/bruy6/G2a92uoMPug7KOZik3N1ehoaFO46GhZZWZmWEolfXsWrdd2fV4Uzd1S75ft13Z9XjbtW78rsAn7GlpaRozxjvW013+W5LD4fD563hK9q3brux6vKn7IuqGL7Lr8bZr3flBh91Fx44d07Jlywp6swWqdKnSCggIUGZmptP4kSOHFRpa1lAq69m1bruy6/GmbuqWfL9uu7Lr8bZr3fidyxP2//73v395W7t2rRU5C1ThwEDVqVtPa1PXOI2vTU1Vo8aRhlJZz65125Vdjzd1U7fk+3XblV2Pt13rdoWvd9hd/uCkIUOGyM/PTw6H408f4w1/nunbL1ZxT41W3fr11ahRpJa+n6y0tDT16h1jOpql7Fr3mdOndWD/3rz7hw4e0K6ftiskpKTCwssbTGYtux5v6qZuO9TNzzV7HW+71o2LXJ6wlytXTs8995xuvfXql5Hatm2bevTo8Y+DWe32Tp117GiWEmfPUkZGumrWqq2ZcxJVoUJF09EsZde6f9q+RSOHDMi7P+e1qZKkDp3v1OhnXjAVy3J2Pd7UTd12qJufa/Y63natO988v1f8j/g5/qpVfhWPPPKI6tSpo8cff/yqX9++fbu6deum7du3uxQk+7xLD4eXSz9+1nQEI8JCipiOAMAi/FyDHQS53Op1j5L3LnTbvo6929dt+7rE5W/7Qw89pNOnT//p16tUqaIFCxb8o1AAAAAALnJ5wt6sWbO//HpwcLBatGhxzYEAAAAAV3jD+yf/CVt+cBIAAADgLTx0JRIAAACQP3TYAQAAABhDhx0AAABejQ47AAAAAGPosAMAAMCr0WEHAAAAYAwddgAAAHg3326w02EHAAAAPBkddgAAAHg11rADAAAAMIYOOwAAALwaHXYAAAAAxtBhBwAAgFejww4AAADAGDrsAAAA8G6+3WCnww4AAAB4MibsAAAAgAdjSQwAAAC8Gm86BQAAAGCMx3TY04+fNR3BiLCQIqYjGGHXuu2K1zfsIKL9CNMRjNjx35dNRzCC17dnocMOAAAAwBiP6bADAAAA14IOOwAAAABj6LADAADAq9FhBwAAAGAMHXYAAAB4N99usNNhBwAAADwZHXYAAAB4NU9dw96uXTsdOHDgivF7771Xzz33XL63w4QdAAAAsMCSJUuUm5ubd3/nzp2KjY3V7bff7tJ2mLADAADAq3lqh71MmTJO9xMTE1WlShW1aNHCpe0wYQcAAADyKScnRzk5OU5jgYGBCgwM/Nvnffzxx4qNjXX5Fwwm7AAAAPBq7uywJyQkaMaMGU5jQ4cO1bBhw/7yeatWrdKJEyfUvXt3l/fJhB0AAADIp0GDBik2NtZp7O+665K0dOlS3Xzzzbruuutc3icTdgAAAHg3Ny5hz8/yl8sdOHBAqampev31169pn1yHHQAAALDQBx98oNDQULVt2/aans+EHQAAALDIhQsX9MEHH6hbt24qVOjaFrewJAYAAABezVMv6yhJqampOnjwoHr27HnN22DCDgAAAFikTZs22rFjxz/aBhN2AAAAeDVP7rAXBNawAwAAAB7MthP2TevX6ZmRQ9W7a3t1iGqoNV+sNh3JbZKTFqlTx3ZqHtlAMb166Ifv15mO5BbUbZ+6eX3b63hL9qy7eHARTR3ZUztSJujI19P0+fwn1LRuFdOxLMfr217neX75+fm57WaCbSfs2dlnVL1WhIaOGGM6ilt9+kmKpkyK18CHH1XykmVq0qSpBg8aqLSDB01HsxR126tuXt/2Ot52rXv2s/eq3Y036MGxb6vZPS9q1dfbtXLOMFUoV9J0NEvx+rbXeY6LbDthbxF1k2IHDdNNbW81HcWtFr49T9179lSPu3upeo0aGj0mTuHlw7U4Ocl0NEtRt73q5vVtr+Ntx7qDihRWt/aNFffKMq354Wf9si9TExNStOfgYQ3sdZPpeJbi9W2f89wVdNjhM87l5Gjb1i2KatXGaTyqVWtt3LDeUCrrUbe96rYrux5vu9ZdKMBfhQoFKDvnnNN49tlzahVZw1AqWMWu5zl+x4TdRrKOZik3N1ehoaFO46GhZZWZmWEolfWo215125Vdj7dd6z55+qzWbvxFYwZ2UvlyJeXv76eYzs3VvP71Ci8bYjoeCphdz3OX+LnxZoDLE/bs7GytW7dOu3btuuJrZ8+e1bJlywoiFyx0+Z9zHA6Hz18OSaLuS+xSt13Z9Xjbse4Hxy6Qn5/0y2cTdeybVzSkT7SSP1mn3AsXTEeDRex4nuMil67Dvnv3bg0YMEAHDx6Un5+fmjVrppdffllhYWGSpBMnTmjMmDHq1q2bFVnxD5UuVVoBAQHKzMx0Gj9y5LBCQ8saSmU96rZX3XZl1+Nt17olaff+THV86FUFBwUqpHiQDmUe18JJsdpz4LDpaChgdj7P88vXf3FxqcP+0ksvqXbt2kpNTdWnn36qYsWKqU+fPjrIO5S9QuHAQNWpW09rU9c4ja9NTVWjxpGGUlmPuu1Vt13Z9Xjbte4/Op2do0OZx1WqRFHd2qqOVvzvR9ORUMA4z+FSh339+vWaN2+eypQpozJlymjOnDkaP3687rvvPi1YsEBFixa1KmeBO3P6tA7s35t3/9DBA9r103aFhJRUWHh5g8ms1bdfrOKeGq269eurUaNILX0/WWlpaerVO8Z0NEtRt73q5vVtr+Nt17pvjaojPz/ppz3pqlG5nF4c3k0796Rrwcdfm45mKV7f9jrP88vXO+wuTdizs7NVqJDzU5577jn5+/vr/vvv18svv1yg4az00/YtGjlkQN79Oa9NlSR16HynRj/zgqlYlru9U2cdO5qlxNmzlJGRrpq1amvmnERVqFDRdDRLUbe96ub1ba/jbde6SxYP0oRhd6ridaV05NhpffTfDXpu5nKdP+/ba9h5fdvrPMdFfg6Hw5HfB9999926//77r7pGfcKECVq+fLlOnjypbdu2uRxk75GzLj/HF4SFFDEdAbBc+nFe3/B9pZsPNR3BiB3/9Z5mXUGy6+s7yKVWr/vUHPmJ2/a166VObtvXJS6tYe/QoYNWrlx51a89++yz6tKli1yY/wMAAAD4Gy512K1Ehx3wXXTYYQd02O3Frq9vT+2w1xr1qdv2tXPq7W7b1yV8cBIAAADgwTz09yQAAAAgf3z8IjF02AEAAABPxoQdAAAA8GAsiQEAAIBX8/UPTqLDDgAAAHgwOuwAAADwaj7eYKfDDgAAAHgyOuwAAADwav7+vt1ip8MOAAAAeDA67AAAAPBqrGEHAAAAYAwddgAAAHg1rsMOAAAAwBg67AAAAPBqPt5gp8MOAAAAeDI67AAAAPBqrGEHAAAAYAwddgAAAHg1OuwAAAAAjPGYDntYSBHTEeBG6cfPmo5ghF3Pc7vWvWJLmukIRtxRr7zpCEZkfTfDdATAtny8wU6HHQAAAPBkTNgBAAAAD+YxS2IAAACAa8GbTgEAAAAYQ4cdAAAAXs3HG+x02AEAAABPRocdAAAAXo017AAAAACMocMOAAAAr+bjDXY67AAAAIAno8MOAAAAr8YadgAAAADGMGEHAACAV/Pzc9/NVb/99ptGjhypli1bqlGjRrrrrru0efNml7bBkhgAAADAAseOHVOfPn3UsmVLzZ07V2XKlNG+ffsUEhLi0naYsAMAAMCreeoa9rlz5yo8PFzx8fF5Y5UqVXJ5OyyJAQAAACywevVq1a9fX4899piioqLUrVs3LV682OXt0GEHAACAV3Nngz0nJ0c5OTlOY4GBgQoMDLzisfv27VNSUpJiY2P1yCOPaNOmTXrhhRcUGBiobt265XufTNgBAACAfEpISNCMGTOcxoYOHaphw4Zd8ViHw6H69evriSeekCTVrVtXu3btUlJSEhN2AAAA2Ic717APGjRIsbGxTmNX665LUrly5VSjRg2nserVq+vf//63S/tkwg4AAADk058tf7maJk2aaPfu3U5je/bsUcWKFV3ap63fdJqctEidOrZT88gGiunVQz98v850JLewY92b1q/TMyOHqnfX9uoQ1VBrvlhtOpLb2PF4S/ar+5vPPtJrIx/UhH6dNaFfZ82JG6wd678xHctt7Ha8L6Fu6sZFnnod9n79+mnjxo2aM2eOfv31Vy1fvlyLFy/Wvffe69J2bDth//STFE2ZFK+BDz+q5CXL1KRJUw0eNFBpBw+ajmYpu9adnX1G1WtFaOiIMaajuJVdj7cd6w4pU0633fuwBscnaHB8gqrXb6JFU+L0277df/9kL2fH4y1RN3Xbo25v17BhQ82YMUMrV67UHXfcoVmzZunpp5/WnXfe6dJ2/BwOh8OijC7JPu/e/d0X00t16tbV2GfH541169pJt7S7VY8PH+HeMG7kKXWnHz/rtn1drkNUQ42b9IpaR7dz+77DQoq4dX+ecrzdzVPqXrElzW37upoXYrvq9r6PqFm7Lm7d7x31yrt1f55yvN2NuqnbRN1BHrqYutWU/3PbvlJH3+y2fV1iyw77uZwcbdu6RVGt2jiNR7VqrY0b1htKZT271m1Xdj3edq37jy5cyNWmNf9VztlsValdz3QcS9n1eFM3dUu+X7cr/Pz83HYzweXfk37++Wdt2LBBjRs3Vo0aNfTzzz9rwYIFysnJ0Z133qmoqCgrchaorKNZys3NVWhoqNN4aGhZZWZmGEplPbvWbVd2Pd52rVuSDu39RQlxg3X+XI4Cg4rqvpHPK6xSVdOxLGXX403d1C35ft34nUsT9v/7v//T4MGDVaxYMZ05c0YzZszQk08+qRtuuEEOh0MPPfSQ3njjDa+YtEtXXgLI4XB47EfbFiS71m1Xdj3edqy7bIXKGjr1DZ05dVJbvvk/LZkZr4HjX/X5Sbtkz+MtUfcl1A1f/za4tCRm1qxZGjBggL755hvFx8dr5MiRuueeezRv3jzNnz9fDz30kObOnWtV1gJTulRpBQQEKDMz02n8yJHDCg0tayiV9exat13Z9XjbtW5JKlSosELDK6lSjRt0270Pq3zVGkpNWWo6lqXserypm7ol368bv3Npwr5z50716NFDktSpUyedOnVKHTt2zPt6ly5dtGPHjoJNaIHCgYGqU7ee1qaucRpfm5qqRo0jDaWynl3rtiu7Hm+71n01Dod0/lzO3z/Qi9n1eFM3dUu+X7crWMP+J/z9/RUYGKiQkJC8sWLFiunEiRMFEsxqffvFKu6p0apbv74aNYrU0veTlZaWpl69Y0xHs5Rd6z5z+rQO7N+bd//QwQPa9dN2hYSUVFi4e69o4U52Pd52rPuzd+eqdmRLlQwtp7PZZ7RpzWrt3rJB/eOmmI5mOTseb4m6qdsedeMilybsFStW1K+//qrrr79ekpScnKzy5X+f7KSlpalcuXIFm9Ait3fqrGNHs5Q4e5YyMtJVs1ZtzZyTqAoVXPvkKW9j17p/2r5FI4cMyLs/57WpkqQOne/U6GdeMBXLcnY93nas++SxLL0/Y6JOZB1RUHAxhV9fXf3jpqhmw2amo1nOjsdbom7qtkfd+eXra/ldug57UlKSypcvr7Zt217169OnT1dmZqYmTpzochB3X4cdZpm8DrtJ7r4OO8wyfR12U9x9HXYA7uOp12G/edqav39QAfm/J1q7bV+XuPRt79Onz19+ffjw4f8oDAAAAOAqH2+w2/ODkwAAAABv4aF/2AAAAADyx9fXsNNhBwAAADwYHXYAAAB4NR9vsNNhBwAAADwZHXYAAAB4NdawAwAAADCGDjsAAAC8mo832OmwAwAAAJ6MCTsAAADgwVgSAwAAAK/m7+NrYuiwAwAAAB6MDjsAAAC8mo832OmwAwAAAJ6MDjsAAAC8Gh+cBAAAAMAYOuwAAADwav6+3WCnww4AAAB4MjrsAAAA8GqsYQcAAABgDB12AAAAeDUfb7AzYYcZYSFFTEcALHdHvfKmIxixZf9x0xGMqFcpxHQEuFHp5kNNRzDizPoZpiPYEhN2AAAAeDU/+XaLnTXsAAAAgAejww4AAACvxnXYAQAAABhDhx0AAABejeuwAwAAADCGDjsAAAC8mo832OmwAwAAAJ6MCTsAAADgwVgSAwAAAK/m7+NrYuiwAwAAAB6MDjsAAAC8mo832OmwAwAAAJ6MDjsAAAC8Gh+cBAAAAMAYOuwAAADwaj7eYKfDDgAAAHgyOuwAAADwar5+HXYm7AAAAIAFXn/9dc2YMcNprGzZslqzZo1L22HCDgAAAK/myf31WrVqad68eXn3AwICXN6GrdewJyctUqeO7dQ8soFievXQD9+vMx3JLaibuu2Auu1R90fvzdPYYQ/owW7ReuSejnp53Egd3LfHdCy3sdvxvsSOdRcPLqKpI3tqR8oEHfl6mj6f/4Sa1q1iOhbyISAgQOXKlcu7lSlTxuVt2HbC/uknKZoyKV4DH35UyUuWqUmTpho8aKDSDh40Hc1S1E3d1O277Fj3tk0/qEPXXprwylsaEz9DF3JzNenpYcrOPmM6muXseLwl+9Y9+9l71e7GG/Tg2LfV7J4Xterr7Vo5Z5gqlCtpOppH8PPzc9vNVb/++qvatGmjdu3aafjw4dq3b5/r9TkcDofLz7qMw+H4xxeszz7/T1O45r6YXqpTt67GPjs+b6xb1066pd2tenz4CPeGcSPqpm7qpm6rbdl/3G37utzxo1l6pHdHPfNSguo0aOLWfderFOLW/XnK8XY3T6m7dPOhbttXUJHCyvjqJfUanqhPv9qSN772vaf0yf9t1vhZK9yW5cz6GX//IAP6LNjgtn29HVNXOTk5TmOBgYEKDAy84rFffPGFsrOzVbVqVR0+fFizZ8/WL7/8ohUrVqh06dL53meBdNgbNGign3/+uSA25RbncnK0besWRbVq4zQe1aq1Nm5YbyiV9aibuiXq9lV2rftyp0+dlCQVL+HeybO72fV427XuQgH+KlQoQNk555zGs8+eU6vIGoZSeRZ/P/fdEhIS1LRpU6dbQkLCVXNFR0frtttuU0REhFq1apX3uGXLlrlUn0tvOo2Pj7/qeG5urhITE1WqVClJ0pgxY1wK4W5ZR7OUm5ur0NBQp/HQ0LLKzMwwlMp61E3dEnX7KrvW/UcOh0PvJE5XRL3Gqly1puk4lrLr8bZr3SdPn9Xajb9ozMBO2rH7N/12+Ljuub2Zmte/Xrv2+m7dnmrQoEGKjY11Grtad/1qgoODVbt2be3Zs8elfbo0YX/77bd1ww03qESJEk7jDodDP//8s4oWLfqPl8a40+VZC2Jpjzeg7ouo27dR90V2qVuS5s+cor27d+m5l+eajuI2dj3edqz7wbELlDDuPv3y2USdP5+rDdv3KfmTdWpcp7LpaB7Bncf/z5a/5EdOTo5+/vlnNW3a1KXnuTRhHz58uBYvXqwnn3xSUVFReeP16tXTpEmTVLOmd3Q0SpcqrYCAAGVmZjqNHzlyWKGhZQ2lsh51U7dE3b7KrnVfMn/mVH3/9f/p2ZcTFVruOtNxLGfX423XuiVp9/5MdXzoVQUHBSqkeJAOZR7Xwkmx2nPgsOlo+AuTJ0/WLbfcovLly+vIkSOaPXu2Tp48qe7du7u0HZfWsA8aNEjTp0/XuHHjNHnyZJ07d+7vn+SBCgcGqk7delqb6nzR+rWpqWrUONJQKutRN3VL1O2r7Fq3w+HQvBlT9N2azxU3ZbbCwiuajuQWdj3edq37j05n5+hQ5nGVKlFUt7aqoxX/+9F0JPyFQ4cO6YknnlCnTp00dOhQFS5cWIsXL1bFiq79rHL5g5MaNmyoDz74QBMmTFDPnj01depUr/wzVN9+sYp7arTq1q+vRo0itfT9ZKWlpalX7xjT0SxF3dRN3b7LjnXPmzFZqZ//WyPGvaSiRYN19MjFzmtwseIKLBJkOJ217Hi8JfvWfWtUHfn5ST/tSVeNyuX04vBu2rknXQs+/tp0NI/gqVPR6dOnF8h2rumTTosVK6bJkydr5cqVevDBB5Wbm1sgYdzp9k6ddexolhJnz1JGRrpq1qqtmXMSVaGCb3dnqJu6qdt32bHuVSuWSpKeH/WI0/igEc8qumNXE5Hcxo7HW7Jv3SWLB2nCsDtV8bpSOnLstD767wY9N3O5zp+/YDoa3OAfX4f90KFD2rx5s1q1aqXg4OBr3o67r8MOALCGyeuwm+Tu67DDLHdeh92TeOp12B94d5Pb9rXg3oZu29cl19Rh/6Pw8HCFh4cXRBYAAAAAl/nHE3YAAADAJH8PXcNeUArkk04BAAAAWIMOOwAAALyaN16x0BV02AEAAAAPRocdAAAAXs23++t02AEAAACPRocdAAAAXs2fNewAAAAATKHDDgAAAK/m4w12OuwAAACAJ6PDDgAAAK/GddgBAAAAGEOHHQAAAF7NxxvsdNgBAAAAT0aHHQAAAF6N67ADAAAAMIYJOwAAAODBWBIDAAAAr+bjK2LosAMAAACejA47AAAAvBofnAQAAADAGI/psG/Zf9x0BCPqVQoxHQFuNPV/u0xHMGJU25qmI8CNyoUUMR0BsNyO/75sOgL+wNc70L5eHwAAAODVPKbDDgAAAFwL1rADAAAAMIYOOwAAALyav2832OmwAwAAAJ6MDjsAAAC8Gh12AAAAAMbQYQcAAIBX4yoxAAAAAIyhww4AAACvxhp2AAAAAMbQYQcAAIBX8/El7HTYAQAAAE/GhB0AAADwYCyJAQAAgFfz9/E1MXTYAQAAAA9Ghx0AAABezdc70L5eHwAAAODV6LADAADAq/n4EnY67AAAAIAns+WE/aP35mnssAf0YLdoPXJPR708bqQO7ttjOpbbJCctUqeO7dQ8soFievXQD9+vMx3JLexa9yWb/71Y7wzponVLEk1HcQu7Hm871r1p/To9M3Koendtrw5RDbXmi9WmI7mNHY+3ZM+67Xye54e/n5/bbkbqM7JXw7Zt+kEduvbShFfe0pj4GbqQm6tJTw9TdvYZ09Es9+knKZoyKV4DH35UyUuWqUmTpho8aKDSDh40Hc1Sdq37ksxff9LONZ+qVMVqpqO4hV2Pt13rzs4+o+q1IjR0xBjTUdzKrsfbrnXb9TzHRbacsD/14uuK7thVlarW0PU1amvQiGeVmX5Iu3duMx3NcgvfnqfuPXuqx929VL1GDY0eE6fw8uFanJxkOpql7Fq3JJ3LPqM186fqxnuHKTC4uOk4bmHX423XultE3aTYQcN0U9tbTUdxK7seb7vWbdfzPL/8/Nx3M8GWE/bLnT51UpJUvESI4STWOpeTo21btyiqVRun8ahWrbVxw3pDqaxn17ov+W7xbFWs11zlb4g0HcUt7Hq87Vq3Xdn1eNu1buAfTdiPHTum+fPna/z48Zo1a5bS0tIKKpfbOBwOvZM4XRH1Gqty1Zqm41gq62iWcnNzFRoa6jQeGlpWmZkZhlJZz651S9KedV/oyL5diryrv+kobmPX423Xuu3KrsfbrnXj7/n7ue/2TyQkJCgiIkITJ050rT5XHtymTRtlZWVJkvbt26cuXbrojTfe0K+//qrk5GR17dpVP//8s0sBTJs/c4r27t6loWNeMB3Fbfwu+3uOw+G4YswX2a3uU1kZWrckUa37jVRA4UDTcdzObsf7ErvWbVd2Pd52rRvebdOmTUpOTlZERITLz3XpOuyZmZm6cOGCJGn69OmqVq2aEhMTVbRoUeXk5Oixxx7Tq6++qtdee83lICbMnzlV33/9f3r25USFlrvOdBzLlS5VWgEBAcrMzHQaP3LksEJDyxpKZT271n1k7y5lnziqlMmP5405LlxQ+q7N2vHFcvV5dZn8/QMMJrSGXY+3Xeu2K7seb7vWjb9n6uot+XXq1CmNGjVKL7zwgmbPnu3y8695SczGjRs1ePBgFS1aVJIUGBioRx99VBs3brzWTbqNw+HQvBlT9N2azxU3ZbbCwiuajuQWhQMDVaduPa1NXeM0vjY1VY0a++76ZrvWHR7RSHfEzVSXMa/n3cpUqaVqzdqqy5jXfXKyLtn3eNu1bruy6/G2a93wfhMmTFB0dLRatWp1Tc93+ZNOL/3JKScnR2XLOv82GxoaqiNHjlxTEHeaN2OyUj//t0aMe0lFiwbr6JGLv6kHFyuuwCJBhtNZq2+/WMU9NVp169dXo0aRWvp+stLS0tSrd4zpaJayY92Fg4JVqkJVp7FCRYJUpHjIFeO+xo7HW7Jv3WdOn9aB/Xvz7h86eEC7ftqukJCSCgsvbzCZtex6vO1at13P8/xyZ4M9JydHOTk5TmOBgYEKDLz68tOVK1dq69atWrJkyTXv0+UJe79+/VSoUCGdPHlSe/bsUa1atfK+lpaWptKlS19zGHdZtWKpJOn5UY84jQ8a8ayiO3Y1Ecltbu/UWceOZilx9ixlZKSrZq3amjknURUq+PZfGexat13Z9Xjbte6ftm/RyCED8u7PeW2qJKlD5zs1+hnffX+SXY+3Xeu263nuiRISEjRjxgynsaFDh2rYsGFXPDYtLU0TJ07UW2+9pSJFilzzPv0cDocjvw++PFyjRo1000035d2fPHmyfvvtN02bNs3lIN/vOe7yc3xBvUq+fSlJOJv6v12mIxgxqq1vX4EJztKPnzUdwYiwkGv/xxjex67neZUynnmeT/yv+/59HXVTlXx32FetWqUhQ4YoIOD3Zai5ubny8/OTv7+/fvzxR6ev/RmXOuxDhw79y68/+eSTrmwOAAAA8Cp/tfzlcjfeeKOWL1/uNDZmzBhVr15dAwcOzNdkXbqGJTEAAACAJ/GTZ14lpnjx4qpdu7bTWHBwsEqVKnXF+F/hk04BAAAAD0aHHQAAAHCThQsXuvwcJuwAAADwav6euSKmwLAkBgAAAPBgdNgBAADg1eiwAwAAADCGDjsAAAC8mp+fb7fY6bADAAAAHowOOwAAALwaa9gBAAAAGEOHHQAAAF7Nx5ew02EHAAAAPBkddgAAAHg1fx9vsdNhBwAAADwYHXYAAAB4Na4SAwAAAMAYOuwAAADwaj6+hJ0OOwAAAODJ6LADAADAq/nLt1vsdNgBAAAAD+YxHfaUXemmI8CN6lUKMR3BiFFta5qOYET68bOmI8CNwkKKmI4AWI7zHO7kMRN2AAAA4FrwplMAAAAAxtBhBwAAgFfjg5MAAAAAGEOHHQAAAF7N38cXsdNhBwAAADwYHXYAAAB4NR9vsNNhBwAAADwZHXYAAAB4NdawAwAAADCGDjsAAAC8mo832OmwAwAAAJ6MDjsAAAC8mq93oH29PgAAAMCr0WEHAACAV/Pz8UXsdNgBAAAAD0aHHQAAAF7Nt/vrdNgBAAAAj0aHHQAAAF6NTzoFAAAAYAwTdkmb/71Y7wzponVLEk1HsdRH783T2GEP6MFu0Xrkno56edxIHdy3x3Qst0lOWqROHdupeWQDxfTqoR++X2c6klvYse5N69fpmZFD1btre3WIaqg1X6w2Hckt7Fq3ZM/zXKJu6rZH3WDCrsxff9LONZ+qVMVqpqNYbtumH9Shay9NeOUtjYmfoQu5uZr09DBlZ58xHc1yn36SoimT4jXw4UeVvGSZmjRpqsGDBirt4EHT0Sxl17qzs8+oeq0IDR0xxnQUt7Jr3XY9z6mbuu1Qd375ufFmgq0n7Oeyz2jN/Km68d5hCgwubjqO5Z568XVFd+yqSlVr6PoatTVoxLPKTD+k3Tu3mY5muYVvz1P3nj3V4+5eql6jhkaPiVN4+XAtTk4yHc1Sdq27RdRNih00TDe1vdV0FLeya912Pc+pm7rtUDcusvWE/bvFs1WxXnOVvyHSdBQjTp86KUkqXiLEcBJrncvJ0batWxTVqo3TeFSr1tq4Yb2hVNaza92wF7ue59RN3ZLv1+0KPz/33UxwacK+detW7du3L+/+Rx99pJiYGEVHR6tPnz5auXJlgQe0yp51X+jIvl2KvKu/6ShGOBwOvZM4XRH1Gqty1Zqm41gq62iWcnNzFRoa6jQeGlpWmZkZhlJZz651w17sep5TN3VLvl83fufShD0uLk4HDhyQJL3//vt69tlnVb9+fT3yyCNq0KCBxo4dqyVLllgStCCdysrQuiWJat1vpAIKB5qOY8T8mVO0d/cuDR3zgukobnP5xxY7HA6f/yhjyb51w17sep5T90XUDT8/P7fdTHDpOuy7d+9WlSpVJEnvvvuunn76afXu3Tvv6w0aNNCcOXN09913F2zKAnZk7y5lnziqlMmP5405LlxQ+q7N2vHFcvV5dZn8/QMMJrTW/JlT9f3X/6dnX05UaLnrTMexXOlSpRUQEKDMzEyn8SNHDis0tKyhVNaza92wF7ue59RN3ZLv143fudRhL1KkiI4cOSJJ+u2339SoUSOnrzdq1Ej79+8vuHQWCY9opDviZqrLmNfzbmWq1FK1Zm3VZczrPjtZdzgcmjdjir5b87nipsxWWHhF05HconBgoOrUrae1qWucxtempqpRY999/4Jd64a92PU8p27qlny/blf4u/Fmgksd9ptvvllJSUmaOHGimjdvrk8//VQ33HBD3tc/+eSTvA68JyscFKxSFao6jRUqEqQixUOuGPcl82ZMVurn/9aIcS+paNFgHT1y8Tf14GLFFVgkyHA6a/XtF6u4p0arbv36atQoUkvfT1ZaWpp69Y4xHc1Sdq37zOnTOrB/b979QwcPaNdP2xUSUlJh4eUNJrOWXeu263lO3dRth7pxkUsT9pEjR6pPnz66//77Vb9+fc2bN0/ffvutatSood27d2vDhg2aOXOmVVnxD61asVSS9PyoR5zGB414VtEdu5qI5Da3d+qsY0ezlDh7ljIy0lWzVm3NnJOoChV8+68Mdq37p+1bNHLIgLz7c16bKknq0PlOjX7Gd9+3Yde67XqeUzd126Hu/PL1tfx+DofD4coTjh8/rsTERH3++efat2+fLly4oLCwMEVGRqp///5q0KDBNQV5ftWua3qet+tcM8x0BCPqVfLtS0nCWfrxs6YjwI3CQoqYjgDAIkEutXrdZ/EG932A1D2NK+T7se+++66SkpLyLtpSq1YtDR48WNHR0S7t0+UJu1WYsNsLE3Z7YcJuL0zYAd/lqRP29904Ye/lwoR99erVCggIyFsyvmzZMr355pv68MMPVatWrXxvx0O/7QAAAIB3a9eundP94cOHKykpSRs2bGDCDgAAAPvwhjXsubm5+vTTT3X69GlFRrp2dR8m7AAAAEA+5eTkKCcnx2ksMDBQgYFX/zDOHTt2KCYmRmfPnlVwcLBmzpypmjVd+5R5U5eTBAAAAAqEO6/DnpCQoKZNmzrdEhIS/jRbtWrVtGzZMiUnJ6tPnz568skntWuXa+/d5E2nhvGmU9gBbzq1F950CvguT33T6Qcb09y2rzvqhLrUYb9c//79VaVKFU2YMCHf+/TQbzsAAACQP+5cw+7K5PxqHA7HFRP+v8OEHQAAALDAtGnTdPPNNys8PFynTp1SSkqKvv32W73xxhsubYcJOwAAAGCBzMxMjR49Wunp6SpRooQiIiL0xhtvqHXr1i5thwk7AAAAvJqnXtTxxRdfLJDtcJUYAAAAwIPRYQcAAIBX84LPTfpH6LADAAAAHowOOwAAALyav8euYi8YdNgBAAAAD0aHHQAAAF6NNewAAAAAjKHDDgAAAK/mxxp2AAAAAKbQYQcAAIBXYw07AAAAAGPosAMAAMCr+fp12P0cDofDdAhJyj5vOoEZW/YfNx0BblSvUojpCABQoNKPnzUdwYgMm9bdtKpn/jv26ZYMt+3r9nrl3LavS+iwAwAAwKuxhh0AAACAMXTYAQAA4NXosAMAAAAwhg47AAAAvBqfdAoAAADAGCbsAAAAgAdjSQwAAAC8mr9vr4ihww4AAAB4MjrsAAAA8Gq86RQAAACAMXTYAQAA4NX44CQAAAAAxtBhBwAAgFdjDTsAAAAAY+iwAwAAwKtxHXYAAAAAxtBhBwAAgFdjDTsAAAAAY+iwAwAAwKv5+nXYbT1hT05apPnz3lRmRoZq1Kyl0U89rSZNm5mOZZmP3pun79Z8roP7flVgYBHVqttQfQYMVYXKVU1Hs5Rd677Ebuf5JdRN3dTtmzatX6f3F83XTzu26UhmhsZNekWto9uZjmUpu/87Bhsvifn0kxRNmRSvgQ8/quQly9SkSVMNHjRQaQcPmo5mmW2bflCHrr004ZW3NCZ+hi7k5mrS08OUnX3GdDRL2bVuyZ7nuUTd1E3dviw7+4yq14rQ0BFjTEdxGzv/O5Zffm68meDncDgchvbtJPu8e/d3X0wv1albV2OfHZ831q1rJ93S7lY9PnyE23Js2X/cbfu63PGjWXqkd0c981KC6jRoYiyHu5msu16lELfuz1POc3ejbuqmbvfVnX78rNv2dbkOUQ2NddgzDNZt8t+xplXd++9Yfq3ZmeW2fbWuVdpt+7rElh32czk52rZ1i6JatXEaj2rVWhs3rDeUyv1OnzopSSpewjNffFaxS912Pc+pm7ol6oZvs8u/Y67w9/Nz281Ifa48+Pnnn9e6deusyuI2WUezlJubq9DQUKfx0NCyyszMMJTKvRwOh95JnK6Ieo1VuWpN03Hcxk512/U8p27qlqgbvstO/47hdy5N2BctWqS+ffvqtttuU2JiojIyvPuHgt9lvyU5HI4rxnzV/JlTtHf3Lg0d84LpKG5lx7rtep5T90XU7dvsWred2fHfMVzDkpi33npLN998s9566y3dcsstevTRR/X555/rwoULVuSzROlSpRUQEKDMzEyn8SNHDis0tKyhVO4zf+ZUff/1/2nslNkKLXed6ThuY7e67XqeUzd1S9QN32S3f8dc4etvOnV5wl67dm3FxcXpyy+/1JQpU5STk6MhQ4YoOjpa06dP16+//mpFzgJVODBQderW09rUNU7ja1NT1ahxpKFU1nM4HJo3Y4q+W/O54qbMVlh4RdOR3MKuddv1PKdu6paoG77Frv+O4XfXfB32woULq3PnzurcubMOHjyopUuX6sMPP1RiYqK2bdtWkBkt0bdfrOKeGq269eurUaNILX0/WWlpaerVO8Z0NMvMmzFZqZ//WyPGvaSiRYN19MjFzkxwseIKLBJkOJ117Fq3ZM/zXKJu6qZuX3bm9Gkd2L837/6hgwe066ftCgkpqbDw8gaTWcfO/47lm4+vBHPpso433HCD1qxZc8WbXC5xOBxKTU1V69atXQ7i7ss6Sv//AyfeelMZGemqWau2Rj05Rk2bNXdrBnde1vHe265e26ARzyq6Y1e35XA3T6rb3Zd1lDzjPDeBuqmbut3D3Zd13PjDdxo5ZMAV4x0636nRz7hvXbc7L+voSf+OeeplHdf+fNRt+7qxRim37esSlybs7dq109KlS1W6dMFff9LEhN0TmLwOO9zPxIQdAKxk8jrsJpm8DrtJnjph/+bnY27bV8saJd22r0tcWhKzevVqq3IAAAAAuIprXsMOAAAAeAJfv5qpLT/pFAAAAPAWdNgBAADg1Xy8wU6HHQAAAPBkdNgBAADg3Ty0xZ6QkKDPPvtMv/zyi4KCghQZGamRI0eqevXqLm2HDjsAAABggW+//Vb33XefFi9erHnz5ik3N1cDBgzQ6dOnXdoOHXYAAAB4NT8PbbG/+eabTvfj4+MVFRWlLVu2qHnz/H/IGR12AAAAwA1OnDghSSpZ0rUPX6LDDgAAAK/mzuuw5+TkKCcnx2ksMDBQgYGBf/k8h8Oh+Ph4NW3aVLVr13Zpn0zYAQAAgHxKSEjQjBkznMaGDh2qYcOG/eXzJkyYoJ9++knvvvuuy/v0czgcDpefZYHs86YTmLFl/3HTEeBG9SqFmI4AAAUq/fhZ0xGMyLBp3U2reua/Yz/scd98qn6FIJc77M8//7xWrVqld955R5UrV3Z5n3TYAQAAgHzKz/KXSxwOh55//nn95z//0cKFC69psi4xYQcAAIC388yLxGj8+PFasWKFZs2apWLFiikjI0OSVKJECQUFBeV7O0zYAQAAAAskJSVJkvr27es0Hh8frx49euR7O0zYAQAAAAvs2LGjQLbDhB0AAABezVM/OKmg8MFJAAAAgAejww4AAACv5s4PTjKBDjsAAADgweiwAwAAwKv5eIPdcybsK7akmY5gRIvKZUxHgBvZ9RMBw0KKmI5ghF1/rt1Rr7zpCEbw+raXt3/YZzqCEZ76Sae+zmMm7AAAAMA18fEWO2vYAQAAAA9Ghx0AAABejeuwAwAAADCGDjsAAAC8GtdhBwAAAGAMHXYAAAB4NR9vsNNhBwAAADwZHXYAAAB4Nx9vsdNhBwAAADwYHXYAAAB4Na7DDgAAAMAYJuwAAACAB2NJDAAAALwaH5wEAAAAwBg67AAAAPBqPt5gp8MOAAAAeDI67AAAAPBuPt5ip8MOAAAAeDA67AAAAPBqvv7BSbacsH/z2Uf65rOPdDTjkCQprFJV3XJ3P0VEtjSczHqb1q/T+4vm66cd23QkM0PjJr2i1tHtTMeyHHXbq25JSk5apPnz3lRmRoZq1Kyl0U89rSZNm5mOZRk7/1yT7He8JV7fdjvef7T534u14eO3dcMtd6nZ3Q+bjgM3sOWSmJAy5XTbvQ9rcHyCBscnqHr9Jlo0JU6/7dttOprlsrPPqHqtCA0dMcZ0FLeibnvV/eknKZoyKV4DH35UyUuWqUmTpho8aKDSDh40Hc0ydv65ZsfjLfH6ttvxviTz15+0c82nKlWxmukoHsXPz303E2zZYa/TrJXT/Y59HtK3n32kfTu36rrKvv0CaBF1k1pE3WQ6httRt70sfHueuvfsqR5395IkjR4Tp9TUr7Q4OUmPDx9hOJ017PxzzY7HW+L1bbfjLUnnss9ozfypuvHeYfrx02TTceBGtuyw/9GFC7natOa/yjmbrSq165mOA+AfOpeTo21btyiqVRun8ahWrbVxw3pDqdzLTj/XON72Yvfj/d3i2apYr7nK3xBpOorH8XPjzQRbdtgl6dDeX5QQN1jnz+UoMKio7hv5vMIqVTUdC8A/lHU0S7m5uQoNDXUaDw0tq8zMDEOp3MOOP9fsfLztyM7He8+6L3Rk3y51Gv2K6SgwwOUO+8KFC/Xkk08qJSVFkrRs2TJ17txZt99+u6ZNm6bz588XeEgrlK1QWUOnvqFBE2epRce7tGRmvNL37zEdC0AB8btsoaHD4bhizNfY+eeaHY+3ndnteJ/KytC6JYlq3W+kAgoHmo7jmXy8xe5Sh33mzJl688031aZNG7344ovav3+/3nzzTfXv31/+/v6aP3++ChUqpMcee8yqvAWmUKHCCg2vJEmqVOMGHfh5u1JTlqrbw769/g3wdaVLlVZAQIAyMzOdxo8cOazQ0LKGUrmHHX+u2fl425Fdj/eRvbuUfeKoUiY/njfmuHBB6bs2a8cXy9Xn1WXy9w8wmBBWc2nC/uGHH2rSpEnq2LGjtm/frh49emjSpEm68847JUnVq1fX1KlTvWLCfjmHQzp/Lsd0DAD/UOHAQNWpW09rU9eo/a0d8sbXpqaqbbv2BpO5nx1+rnG87cWuxzs8opHuiJvpNJa68BWVvK6S6nW8m8m6uA67k4yMDNWvX1+SdMMNN8jf31916tTJ+3rdunWVnp5esAkt8Nm7c1U7sqVKhpbT2ewz2rRmtXZv2aD+cVNMR7PcmdOndWD/3rz7hw4e0K6ftiskpKTCwssbTGYt6r7ILnX37ReruKdGq279+mrUKFJL309WWlqaevWOMR3NMnb+uWbH4y3x+rbT8S4cFKxSFao6jRUqEqQixUOuGIdvcmnCXrZsWe3atUsVKlTQnj17lJubq127dqlWrVqSpF27dqlMmTKWBC1IJ49l6f0ZE3Ui64iCgosp/Prq6h83RTUb+v6HLvy0fYtGDhmQd3/Oa1MlSR0636nRz7xgKpblqPsiu9R9e6fOOnY0S4mzZykjI101a9XWzDmJqlChoulolrHzzzU7Hm+J17fdjjf+mg+/hUGS5OdwOBz5ffD06dO1ePFitW/fXl9//bW6dOmi5cuX6+GHH5afn5/mzJmj2267TWPGuP4hDks2prn8HF/QorLn/4ID/FNhIUVMRzBixRZ7/ly7o57vdnf/Svrxs6YjGGHX1/fU/+0yHcGIZ26taTrCVe1KP+O2fdUMK+q2fV3iUof9scceU1BQkDZu3KjevXvr4YcfVkREhKZOnaozZ86oXbt2evzxx/9+QwAAAEAB8fEGu2sT9oCAAD366KNOY126dFGXLl0KNBQAAACAi2z/SacAAACAJ7PtJ50CAADAR/j4mhg67AAAAIAHo8MOAAAAr+brH5xEhx0AAADwYHTYAQAA4NV8/YOT6LADAAAAHowOOwAAALyajzfY6bADAAAAnowOOwAAALybj7fY6bADAAAAFvnuu+/0yCOPqE2bNoqIiNCqVatc3gYTdgAAAHg1Pzf+56rTp08rIiJCzz777DXXx5IYAAAAwCLR0dGKjo7+R9tgwg4AAACv5s7rsOfk5CgnJ8dpLDAwUIGBgZbtkyUxAAAAQD4lJCSoadOmTreEhARL90mHHQAAAF7NnReJGTRokGJjY53GrOyuS0zYAQAAgHyzevnL1TBhBwAAgFdz5xp2E5iwAwAAABY5deqU9u7dm3d///792rZtm0qWLKkKFSrkaxtM2AEAAACLbN68WQ888EDe/fj4eElS9+7dNWnSpHxtw8/hcDgsSeei7POmEwCwSvrxs6YjGPHtviOmIxhxR73ypiMYsWJLmukIRrSoXMZ0BCPCQoqYjmBEkIe2evdn5fz9gwpIpdLuXb8ucVlHAAAAwKN56O9JAAAAQP74+ptO6bADAAAAHowOOwAAALyajzfY6bADAAAAnowOOwAAALwaa9gBAAAAGEOHHQAAAF7Nz8dXsdNhBwAAADwYHXYAAAB4N99usNNhBwAAADwZHXYAAAB4NR9vsNNhBwAAADwZHXYAAAB4Na7DDgAAAMAYOuwAAADwalyHHQAAAIAxdNgBAADg3Xy7wU6HHQAAAPBkTNgBAAAAD2brCXty0iJ16thOzSMbKKZXD/3w/TrTkdyCuqnb121av07PjByq3l3bq0NUQ635YrXpSJb75rOP9NrIBzWhX2dN6NdZc+IGa8f6b0zHchu7ned2Pt52fH1fYrfz3BV+bryZYNsJ+6efpGjKpHgNfPhRJS9ZpiZNmmrwoIFKO3jQdDRLUTd126Hu7Owzql4rQkNHjDEdxW1CypTTbfc+rMHxCRocn6Dq9Zto0ZQ4/bZvt+lolrPjeW7n423H17dkz/Mcv7PthH3h2/PUvWdP9bi7l6rXqKHRY+IUXj5ci5OTTEezFHVTtx3qbhF1k2IHDdNNbW81HcVt6jRrpYgmN6pshcoqW6GyOvZ5SIFBRbVv51bT0Sxnx/Pczsfbjq9vyZ7nuSv8/Nx3M8GWE/ZzOTnatnWLolq1cRqPatVaGzesN5TKetRN3ZLv1w3pwoVcbVrzX+WczVaV2vVMx7EU57m9jrddcZ7D5cs6pqenKykpSd9//70yMjIUEBCgSpUqqX379urRo4cCAgKsyFmgso5mKTc3V6GhoU7joaFllZmZYSiV9aibuiXfr9vODu39RQlxg3X+XI4Cg4rqvpHPK6xSVdOxLGXn89yOx9uu7Hye5xcfnPQHP/74ozp37qzPP/9cOTk52rNnj+rWrauiRYtqypQpuu+++3Ty5EmrshY4v8v+ruFwOK4Y80XUfRF1w9eUrVBZQ6e+oUETZ6lFx7u0ZGa80vfvMR3LLex4ntv5eNuVHc9zXOTShP3FF19U//79tWzZMr333nuaNGmS9uzZo+nTp2vVqlXKzs7WK6+8YlHUglO6VGkFBAQoMzPTafzIkcMKDS1rKJX1qJu6Jd+v284KFSqs0PBKqlTjBt1278MqX7WGUlOWmo5lKTuf53Y83nZl5/M8v1jD/gdbt27VXXfdlXe/a9eu2rp1qzIzM1WyZEmNGjVK//73vws8ZEErHBioOnXraW3qGqfxtampatQ40lAq61E3dUu+Xzd+53BI58/lmI5hKc7z39nheNsV5zlcWsMeGhqq9PR0Va5cWZKUmZmp8+fPq3jx4pKk66+/XseOHSv4lBbo2y9WcU+NVt369dWoUaSWvp+stLQ09eodYzqapaibuu1Q95nTp3Vg/968+4cOHtCun7YrJKSkwsLLG0xmnc/enavakS1VMrSczmaf0aY1q7V7ywb1j5tiOprl7Hie2/l42/H1LdnzPMfvXJqwt2/fXuPGjdOoUaMUGBioWbNmqXnz5goKCpIk7d69W9ddd50lQQva7Z0669jRLCXOnqWMjHTVrFVbM+ckqkKFiqajWYq6qdsOdf+0fYtGDhmQd3/Oa1MlSR0636nRz7xgKpalTh7L0vszJupE1hEFBRdT+PXV1T9uimo2bGY6muXseJ7b+Xjb8fUt2fM8x+/8HA6HI78PPnXqlOLi4vSf//xHubm5aty4saZOnZrXcf/qq6904sQJderUyeUg2eddfgoAL5F+/KzpCEZ8u++I6QhG3FHPd7ucf2XFljTTEYxoUbmM6QhGhIUUMR3BiCCXry/oHkfP5LptX6WKuv+KiC5924sVK6ZXXnlFZ8+e1fnz51WsWDGnr7dp0+ZPngkAAADgWlzT70lFihRRkSL2/M0SAAAAnoXrsAMAAAAwxkNXIgEAAAD54+ufH0WHHQAAAPBgdNgBAADg1Xy8wU6HHQAAAPBkTNgBAAAAD8aSGAAAAHg3H18TQ4cdAAAA8GB02AEAAODV+OAkAAAAAMbQYQcAAIBX44OTAAAAABhDhx0AAABezccb7HTYAQAAAE9Ghx0AAADezcdb7HTYAQAAAA/GhB0AAABezc+N/12LRYsWqV27dmrQoIF69OihdevWufR8JuwAAACARVJSUhQfH69HH31Uy5YtU9OmTTVw4EAdPHgw39tgwg4AAACv5ufnvpur5s2bp549e6pXr16qUaOG4uLiFB4erqSkpHxvgwk7AAAAkE85OTk6efKk0y0nJ+dPH7tlyxa1adPGabx169Zav359vvfpMVeJCfKYJAAKWpUyRUxHMKJKmfKmI8CN7m7E8QZMcec88vXXEzRjxgynsaFDh2rYsGFXPDYrK0u5ubkKDQ11Gi9btqwyMjLyvU+myQAAAEA+DRo0SLGxsU5jgYGBf/kcv8vW0jgcjivG/goTdgAAACCfAgMD/3aCfknp0qUVEBCgzMxMp/HDhw+rbNmy+d4na9gBAAAACwQGBqpevXpas2aN03hqaqoiIyPzvR067AAAAIBFYmNjNXr0aNWvX1+RkZFKTk5WWlqaYmJi8r0NJuwAAACARTp37qysrCzNmjVL6enpql27thITE1WxYsV8b8PP4XA4LMwIAAAA4B9gDTsAAADgwZiwAwAAAB6MCTsAAADgwZiwAwAAAB7M1hP2RYsWqV27dmrQoIF69OihdevWmY5kue+++06PPPKI2rRpo4iICK1atcp0JMslJCSoZ8+eioyMVFRUlAYPHqxffvnFdCzLvfvuu+ratauaNGmiJk2aqHfv3vriiy9Mx3K7hIQERUREaOLEiaajWOr1119XRESE061169amY7nFb7/9ppEjR6ply5Zq1KiR7rrrLm3evNl0LEu1a9fuiuMdERGh8ePHm45mqfPnz2v69Olq166dGjZsqPbt22vGjBm6cOGC6WiWO3nypCZOnKhbbrlFDRs2VExMjDZt2mQ6FtzEtpd1TElJUXx8vJ577jk1adJE7733ngYOHKiVK1eqQoUKpuNZ5vTp04qIiFCPHj00bNgw03Hc4ttvv9V9992nBg0aKDc3V9OnT9eAAQO0cuVKBQcHm45nmfDwcI0cOVJVqlSRJC1btkxDhgzRhx9+qFq1ahlO5x6bNm1ScnKyIiIiTEdxi1q1amnevHl59wMCAgymcY9jx46pT58+atmypebOnasyZcpo3759CgkJMR3NUkuWLFFubm7e/Z07dyo2Nla33367wVTWmzt3rt577z1NnjxZNWvW1ObNmzVmzBiVKFFC/fr1Mx3PUmPHjtXOnTs1ZcoUhYWF6eOPP1ZsbKxSUlJ03XXXmY4Hi9l2wj5v3jz17NlTvXr1kiTFxcXpq6++UlJSkkaMGGE4nXWio6MVHR1tOoZbvfnmm0734+PjFRUVpS1btqh58+aGUlmvXbt2TveHDx+upKQkbdiwwRYT9lOnTmnUqFF64YUXNHv2bNNx3CIgIEDlypUzHcOt5s6dq/DwcMXHx+eNVapUyWAi9yhTpozT/cTERFWpUkUtWrQwlMg9NmzYoPbt26tt27aSLh7rlStX+vxfVLKzs/XZZ59p1qxZef9uDRs2TKtWrdK7776r4cOHG04Iq9lySUxOTo62bNmiNm3aOI23bt1a69evN5QK7nLixAlJUsmSJQ0ncZ/c3FytXLlSp0+fdumjkL3ZhAkTFB0drVatWpmO4ja//vqr2rRpo3bt2mn48OHat2+f6UiWW716terXr6/HHntMUVFR6tatmxYvXmw6llvl5OTo448/Vs+ePeXn52c6jqWaNm2qtWvXavfu3ZKk7du36/vvv/f5RtT58+eVm5urIkWKOI0HBQXphx9+MJQK7mTLDntWVpZyc3MVGhrqNF62bFllZGQYSgV3cDgcio+PV9OmTVW7dm3TcSy3Y8cOxcTE6OzZswoODtbMmTNVs2ZN07Est3LlSm3dulVLliwxHcVtGjZsqMmTJ6tq1ao6fPiwZs+erZiYGK1YsUKlS5c2Hc8y+/btU1JSkmJjY/XII49o06ZNeuGFFxQYGKhu3bqZjucWq1at0okTJ9S9e3fTUSw3cOBAnThxQp06dVJAQIByc3M1fPhw3XHHHaajWap48eKKjIzUrFmzVL16dZUtW1YrVqzQxo0bdf3115uOBzew5YT9kss7EQ6Hw+e7E3Y3YcIE/fTTT3r33XdNR3GLatWqadmyZTp+/Lg+++wzPfnkk3rnnXd8etKelpamiRMn6q233rqiG+XLLu8wNm7cWB06dNCyZcsUGxtrKJX1HA6H6tevryeeeEKSVLduXe3atUtJSUm2mbAvXbpUN998sy3WMaekpOjjjz/Wyy+/rJo1a2rbtm2Kj49XWFiYz//CMmXKFD399NO6+eabFRAQoLp16+qOO+7Q1q1bTUeDG9hywl66dGkFBAQoMzPTafzw4cMqW7asoVSw2vPPP6/Vq1frnXfeUXh4uOk4bhEYGJjXfWnQoIF+/PFHLViwQBMmTDCczDpbtmzR4cOH1aNHj7yx3Nxcfffdd1q0aJF+/PFHW7wZMzg4WLVr19aePXtMR7FUuXLlVKNGDaex6tWr69///rehRO514MABpaam6vXXXzcdxS2mTJmihx9+WF26dJEkRURE6ODBg0pISPD5CXuVKlX0zjvv6PTp0zp58qTCwsL0r3/9yxbv2YBNJ+yBgYGqV6+e1qxZow4dOuSNp6amqn379gaTwQoOh0PPP/+8/vOf/2jhwoWqXLmy6UjGOBwO5eTkmI5hqRtvvFHLly93GhszZoyqV6+ugQMH2mKyLl1c1/zzzz+radOmpqNYqkmTJnnrmS/Zs2ePKlasaCiRe33wwQcKDQ3NexOmr8vOzr7iL+EBAQFyOByGErlfcHCwgoODdezYMX311VcaNWqU6UhwA1tO2CUpNjZWo0ePVv369RUZGank5GSlpaUpJibGdDRLnTp1Snv37s27v3//fm3btk0lS5b02ctZjh8/XitWrNCsWbNUrFixvPcplChRQkFBQYbTWWfatGm6+eabFR4erlOnTiklJUXffvut3njjDdPRLFW8ePEr3p8QHBysUqVK+fT7FiZPnqxbbrlF5cuX15EjRzR79mydPHnS57uO/fr1U58+fTRnzhx16tRJmzZt0uLFi336r0iXXLhwQR988IG6deumQoXs8c/5Lbfcojlz5qhChQp5S2IuXfXN13355ZdyOByqVq2a9u7dqylTpqhatWpOf02E7/Jz2OnX0sssWrRIb775ptLT01W7dm2NGTPGpy/zJ0nffPONHnjggSvGu3fvrkmTJhlIZL0/uwZ3fHy8T/+ge/rpp7V27Vqlp6erRIkSioiI0MCBA23zYTp/1LdvX91www2Ki4szHcUyw4cP13fffaejR4+qdOnSaty4sR5//HGffr/CJZ9//rmmTZumPXv2qFKlSoqNjdU999xjOpblvvrqKw0YMECffvqpqlWrZjqOW5w8eVKvvvqqVq1apcOHDyssLExdunTRkCFDFBgYaDqepVJSUjRt2jQdOnRIpUqVUseOHTV8+HCVKFHCdDS4ga0n7AAAAICns+V12AEAAABvwYQdAAAA8GBM2AEAAAAPxoQdAAAA8GBM2AEAAAAPxoQdAAAA8GBM2AEAAAAPxoQdAAAA8GBM2AEAAAAPxoQdAAAA8GBM2AEAAAAPxoQdAAAA8GD/D745k9d0A0psAAAAAElFTkSuQmCC",
|
||
"text/plain": [
|
||
"<Figure size 1000x700 with 2 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"from sklearn.metrics import confusion_matrix\n",
|
||
"\n",
|
||
"with torch.no_grad():\n",
|
||
" vanilla_model.eval()\n",
|
||
" for i, data in enumerate(test_loader):\n",
|
||
" x, y = data\n",
|
||
" x = x.to(device)\n",
|
||
" y = y.to(device)\n",
|
||
" pred_vanilla = vanilla_model(x)\n",
|
||
" acc = get_accuracy(pred_vanilla, y)\n",
|
||
" print(f\"vanilla acc: {acc}\")\n",
|
||
" \n",
|
||
" do_model.eval()\n",
|
||
" for i, data in enumerate(test_loader):\n",
|
||
" x, y = data\n",
|
||
" x = x.to(device)\n",
|
||
" y = y.to(device)\n",
|
||
" pred_do = do_model(x)\n",
|
||
" acc = get_accuracy(pred_do, y)\n",
|
||
" print(f\"drop-out (0.5) acc: {acc}\")\n",
|
||
"\n",
|
||
"cm = confusion_matrix(mnist_test.targets, pred_vanilla.argmax(dim=1).cpu())\n",
|
||
"plt.figure(figsize=(10,7))\n",
|
||
"plt.title('Confusion Matrix for vanilla_model')\n",
|
||
"np.fill_diagonal(cm, 0) # you can zero-out the diagonal to highlight the errors better\n",
|
||
"sns.heatmap(cm, annot=True, cmap='Blues', fmt='g')\n",
|
||
"# print(cm) # if seaborn does not work, you can always print out the array\n",
|
||
" \n",
|
||
"cm = confusion_matrix(mnist_test.targets, pred_do.argmax(dim=1).cpu())\n",
|
||
"plt.figure(figsize=(10,7))\n",
|
||
"plt.title('Confusion Matrix for do_model')\n",
|
||
"np.fill_diagonal(cm, 0) # you can zero-out the diagonal to highlight the errors better\n",
|
||
"sns.heatmap(cm, annot=True, cmap='Blues', fmt='g')\n",
|
||
"# print(cm) # if seaborn does not work, you can always print out the array"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3e0bf87e",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Concept 4: Classification Metrics (Precision, Recall, F1-score, ROC Curve, and AUC-ROC)\n",
|
||
"\n",
|
||
"### Introduction\n",
|
||
"\n",
|
||
"In classification tasks, we often want to evaluate the performance of our predictive model beyond just the accuracy. In some cases, we would like to minimize the number of false positives, while in other cases, we would like to minimize the number of false negatives. \n",
|
||
"\n",
|
||
"For instance, in a cancer diagnosis task, we would like to minimize the number of false negatives (i.e., patients who have cancer but are diagnosed as healthy), but don't care as much for the number of false positives (i.e., patients who don't have cancer but are diagnosed as ill) because delaying cancer treatment is dangerous while suggesting a person to go through more advanced checkup poses no additional harm. On the other hand, in a spam detection task, we would like to minimize the number of false positives (i.e., emails that are not spam but are classified as spam), because users may miss out on an important email. \n",
|
||
"\n",
|
||
"In this section, we will delve deeper into the metrics that help us to understand how well our model is doing in different aspects: Precision, Recall, F1-score, ROC Curve, and AUC-ROC.\n",
|
||
"\n",
|
||
"### Definitions\n",
|
||
"\n",
|
||
"#### 1. __Precision__\n",
|
||
"\n",
|
||
"Precision helps us to understand the correctness of our model in predicting the positive class.\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\text{Precision} = \\frac{\\text{True Positives}}{\\text{True Positives} + \\text{False Positives}}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"#### 2. __Recall__\n",
|
||
"\n",
|
||
"Recall, also known as sensitivity or true positive rate, indicates how well the model identifies positive instances.\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\text{Recall} = \\frac{\\text{True Positives}}{\\text{True Positives} + \\text{False Negatives}}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"#### 3. __F1-score__\n",
|
||
"\n",
|
||
"The F1-score is the harmonic mean of precision and recall and provides a balance between the two metrics.\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\text{F1-Score} = 2 \\cdot \\frac{\\text{Precision} \\cdot \\text{Recall}}{\\text{Precision} + \\text{Recall}}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"### Graphical Metrics\n",
|
||
"\n",
|
||
"#### 4. __ROC Curve__\n",
|
||
"\n",
|
||
"The ROC (Receiver Operating Characteristic) curve is a graphical representation of the true positive rate against the false positive rate, helping to visualize the performance of the binary classification model.\n",
|
||
"\n",
|
||
"#### 5. __AUC-ROC__\n",
|
||
"\n",
|
||
"AUC (Area Under the ROC Curve) represents the model's ability to discriminate between positive and negative classes; a higher AUC value indicates a better model performance.\n",
|
||
"\n",
|
||
"\n",
|
||
"<img src=\"imgs/Roc_curve.png\" width=500>\n",
|
||
"\n",
|
||
"### Further Reading\n",
|
||
"You can read more about the metrics and how to use them in the following links:\n",
|
||
"1. [Scikit-learn Classification Metrics](https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics)\n",
|
||
"2. [Understanding Confusion Matrix](https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e3dfa092",
|
||
"metadata": {},
|
||
"source": [
|
||
"### Evaluate your model\n",
|
||
"\n",
|
||
"Run the cell below to evaluate the model you built previously with the metrics."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 95,
|
||
"id": "d3169d65",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T03:38:00.781825Z",
|
||
"start_time": "2024-04-07T03:38:00.765999Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"0.9923320262116722\n",
|
||
"0.9922976098863504\n",
|
||
"0.9923886815131227\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"from sklearn.metrics import f1_score, precision_score, recall_score\n",
|
||
"\n",
|
||
"print(f1_score(pred_do.argmax(dim=1).cpu(), mnist_test.targets, average='macro'))\n",
|
||
"print(precision_score(pred_do.argmax(dim=1).cpu(), mnist_test.targets, average='macro'))\n",
|
||
"print(recall_score(pred_do.argmax(dim=1).cpu(), mnist_test.targets, average='macro'))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c800da3f",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Chapter 3: Training on CIFAR-10\n",
|
||
"\n",
|
||
"## Concept 5: CIFAR-10\n",
|
||
"Using what you've learned with MNIST, apply the techniques to CIFAR-10, a dataset of 60K training and 10K testing images comprising of real-life objects corresponding to the following 10 classes:\n",
|
||
"\n",
|
||
"- airplane\t\t\t\t\t\t\t\t\t\t\n",
|
||
"- automobile\t\t\t\t\t\t\t\t\t\t\n",
|
||
"- bird\t\t\t\t\t\t\t\t\t\t\n",
|
||
"- cat\t\t\t\t\t\t\t\t\t\t\n",
|
||
"- deer\t\t\t\t\t\t\t\t\t\t\n",
|
||
"- dog\t\t\t\t\t\t\t\t\t\t\n",
|
||
"- frog\t\t\t\t\t\t\t\t\t\t\n",
|
||
"- horse\t\t\t\t\t\t\t\t\t\t\n",
|
||
"- ship\t\t\t\t\t\t\t\t\t\t\n",
|
||
"- truck\n",
|
||
"\n",
|
||
"Each image is $3 \\times 32 \\times 32$, meaning we operate on 3 color channels RGB. Some of these images look like so:\n",
|
||
"\n",
|
||
"<img src=\"imgs/cifar.jpg\" width=600>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7e003539",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Concept 6: Data Augmentation\n",
|
||
"\n",
|
||
"In reality, however, finding a well-representative, balanced dataset is difficult. To address this issue, we use __Data Augmentation__. It refers to the process of transforming data in a training dataset in one or more ways to create more samples to expand the training dataset. \n",
|
||
"\n",
|
||
"Here, we will pick images from the original dataset `x_train`, perform some transformations $F$ on them, and append them to `x_train`. So, for example, if I have a training dataset of 200 car images, I can perform augmentations on the 200 images to get 300 more images, thereby making my new training dataset 500 images large.\n",
|
||
"\n",
|
||
"Of course, the impact of data augmentation on model training depends on the types of augmentation used. Here are some common ones Computer Vision practitioners use:\n",
|
||
"\n",
|
||
"- Normalisation\n",
|
||
"- Horizontal and Vertical Flipping\n",
|
||
"- Rotation\n",
|
||
"- Blurring\n",
|
||
"- Adding noise\n",
|
||
"- Skewing\n",
|
||
"- Cropping (zooming in or out)\n",
|
||
"- Brightness and Contrast\n",
|
||
"- Shuffling pixels\n",
|
||
"\n",
|
||
"This results in a wide variety of new samples being created that can be used for training."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "239c0069",
|
||
"metadata": {},
|
||
"source": [
|
||
"## The `transforms` module\n",
|
||
"\n",
|
||
"Here, we are going to use the `transforms` module from PyTorch to transform the images in our dataset. It contains all kinds of image transformations from `rotate` to `resize`. Check out the full list of augmentations on the PyTorch documentation: https://pytorch.org/vision/stable/transforms.html.\n",
|
||
"\n",
|
||
"Explore the following example to see how the transformations work."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 24,
|
||
"id": "9fc794d7",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:10:43.402021Z",
|
||
"start_time": "2024-04-07T04:10:42.575847Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Files already downloaded and verified\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAxoAAAGICAYAAADGcZYzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABEK0lEQVR4nO3deXxdVb3//8+Zz8mcJmlLJ6C0J3SEDhAtQ22ZfIjgBXEAWxC4wOULooAySgWvSi+XQaGW4apclTKIZRYEFMpoUbAoiFyGltIxQ5vpJGfe6/cHv0ZCg31v3b3I9fV8PPpHk3dW1t5r7bX255wkO+SccwYAAAAAAQp/0B0AAAAA8H8PhQYAAACAwFFoAAAAAAgchQYAAACAwFFoAAAAAAgchQYAAACAwFFoAAAAAAgchQYAAACAwFFoAP+H8PxNAP8XsJZ9+DBmGAqFxofAwoULrbm5edC/2bNn2/HHH2+//e1v/6Y2r7zySmtpabG9997b7rnnnmA7vAPr16/f7niG+nfXXXf9r/br/fz4xz+2/fff36ZPn25Lly79oLvzvt544w079thjB32subnZrrvuug+oRwBgdt5551lzc7PddNNN230uk8nY6aefbnvttZfts88+9tZbb9n1119vP/zhD3dKX5S95x9lzXzwwQdt3rx5Nm3aNFu0aNEH3Z33tXnzZjvttNNsw4YNAx+bP3++XXDBBR9gr/CPIvpBdwCayZMn2ze+8Q0zMyuXy9bZ2Wm33XabnXzyyXbXXXfZxIkT5bZee+01+6//+i/77Gc/a5/61Kds/PjxO6vbQxo+fLjdcccdA/9vb2+3M888004//XT72Mc+NvDxcePG/a/2ayiZTMYWL15sc+fOtZNPPtnGjBnzQXfpfT300EO2atWqQR+74447bOTIkR9QjwD8s8tkMvbII49YOp22n/3sZ3bKKadYKBQa+Pw999xjjz32mC1atMgmTpxoY8aMse9+97t25pln7pT+vHvvMTP73Oc+Z8ccc4x95jOfGfjYP8qaedlll9luu+1mixcvthEjRnzQ3Xlfzz77rK1YscIuueSSgY8tWbLEqqqqPsBe4R8FhcaHRFVVle29996DPjZnzhz76Ec/anfddZedf/75cltdXV1mZnb44Yfb7NmzA+ylJh6PDzqW9evXm9k7hcV7j/GD1t3dbZ7n2SGHHGL77LPPB90d3/7RzieAfy6/+MUvrFwu29e//nU7/vjj7emnn7YDDjhg4PPb9qPjjjtuUAGyswy1Jo4cOfIfcq3s6uqy/fbbz1paWj7orvg2efLkD7oL+AfBj059iKVSKUskEtstzr/61a/s6KOPtmnTptl+++1n3/rWt6y/v9/MzK677jpbuHChmZmdcMIJNn/+fDN7512SZcuW2RFHHGHTp0+3j33sY3bllVdaPp8faPeCCy6wE044wb7xjW/Y7Nmz7aijjrJSqWSe59lNN91khxxyiE2dOtUOO+ww++lPfxrIMS5cuNC++tWv2llnnWUzZ860U0891czeKU7OO+8823///W3KlCn20Y9+1M477zzr7Owc+Nr58+fbtddea//xH/9hc+bMsenTp9vJJ59sa9asGchs3brVvvrVr9p+++1n06ZNs0996lMDP0p21113DZyfiy66yJqbmwe+7sEHH7Sjjz7aZsyYYfvtt58tWrTIuru7Bz5/3XXX2SGHHGJLliyxlpYWO/jgg62zs9Pmz59vS5Ysscsvv9xaWlpsxowZdu6551pfX5/ddNNNduCBB9qsWbPsS1/60qBjyeVydtVVV9mhhx5qU6dOtZkzZ9qJJ55of/7znwe+35IlS8xs8I9LvffHANra2uzCCy+0uXPn2vTp0+2YY46xX//614POeXNzsy1btswuvvhi23fffW3GjBl21llnWUdHx98+kAD+KS1fvtxaWlqspaXFdt99d7v99tsHPrdw4cKB9WnPPfe0Cy64YGCdXbJkyaA197XXXrPTTjvNZs6caTNnzrQzzjjD1q1bN/D55557zpqbm+3222+3efPm2Zw5c+zpp5/+m/v9fvvd1q1b7bLLLrN58+bZ1KlTbd9997Uzzjhj4AWzbcd18cUX20033WQf+9jHbNq0afb5z3/e/vCHPwxk8vm8XXbZZXbggQfa1KlT7eMf/7j96Ec/GnQsZmbf//73rbm5eaD9Z555xo477jibNWuWtbS02LnnnmubNm0aaPeuu+6yyZMn25133mn777+/HXjggfb666/bwoULbdGiRXb99dfbAQccYHvttZedcsop1tHRYcuXL7dDDjnEZsyYYV/84hcHHUu5XLabbrrJPvnJT9r06dNt7733ts9//vP2m9/8ZuD7XXjhhWZmdtBBBw38uNR7f3Sqt7fXLr/8cjv44INt2rRp9slPftJ+/vOfDzrnyp6NDyGHf3gLFixwX/jCF1yxWHTFYtEVCgXX1tbmrrzySrfXXnu5N954YyB73333uXQ67c4991z3xBNPuFtvvdXts88+7oQTTnCe57lNmza5W265xaXTaXfLLbe4P/3pT8455y666CI3efJkd/XVV7unn37a3XTTTW6vvfZyJ510kvM8zznn3Pnnn+8mT57sTjjhBPfss8+6Rx991Dnn3CWXXOKmTJnirr32WvfUU0+5q6++2u25555uyZIl0vGtW7fOpdNpt3z58iGPffLkye6cc85xzz77rHvqqadcf3+/mzdvnjv66KPdI4884n7zm9+4JUuWuEmTJrmvf/3rA187b948N2vWLHfqqae6FStWuHvvvdftu+++7rOf/exA5qSTTnKf+tSn3KOPPuqeffZZd8EFF7h0Ou1WrlzptmzZ4h555BGXTqfdNddc41atWuWcc+773/++S6fT7tJLL3VPPvmkW7Zsmdt3333dEUcc4bLZrHPOuWuvvdZNnjzZHXnkke7pp592999//0CfZs6c6c4880z3zDPPuBtuuMGl02l32GGHueOPP96tWLHC3XzzzW7SpEnu0ksvHejnl770JfeRj3zE3Xnnne65555zd9xxh5szZ4477LDDBsb1oosucul02q1atcpt2rTJOedcOp121157rXPOufb2dnfAAQe4+fPnu7vvvtutWLHCnXXWWa65udnde++9A98rnU67WbNmuQsuuMA99dRT7tZbb3XTpk1zZ599tjSeAOCcc2+88YZLp9PugQcecM45d+ONN7pJkya5zZs3O+ece/311wetW2vXrnWrVq1y6XTaXXTRRQNr7urVq92MGTPcpz/9affwww+7Bx980B1xxBFuv/32cx0dHc4551auXOnS6bTbd9993UMPPeTuuece19vbu8M+vnuNfLeh9jvP89wxxxzjDjnkEHf//fe7lStXuptvvtntvffe7sQTTxz42gULFrhZs2a5z372s+7RRx91jzzyiDvooIPcgQce6EqlknPunX1z3rx57oEHHnArV650V1xxxcA+2Nvbu915yOfz7p577nHpdNp95StfcStWrHB33323mzdvnjvggAMGzsPy5ctdOp128+fPd48//rhbvny58zzPLViwwM2cOdMtWLDAPfHEE+62225zkydPdocddtjAHnjnnXe6vffe251yyikDx7J48WI3ffp095Of/MQ999xz7t5773WHHnqo22effVxfX5/bsmWLu+aaa1w6nXaPPPKIW7t2rXPunb3u/PPPd845l81m3Sc/+Un3kY98xC1btsw9+eSTbtGiRS6dTrvrr79+4HspezY+fCg0PgQWLFjg0un0kP9uuOGGgZznee7AAw90J5988qCvf/bZZ106nXaPP/64c+4vC/LKlSudc+8s9ul02i1dunTQ121b1FasWOGce2fhTafT7q233hrIrF692jU3N7sbb7xx0Ndec801btq0aW7r1q07PL4dFRpTp051fX19Ax975ZVX3LHHHjuwoG1z2mmnuUMPPXTg//PmzXPz5s0bWNidc+66665z6XR6oF9Tp04ddNzlctktXrzY/e53vxuyb11dXW7q1Knu4osvHvS9f/e737l0Ou2WLVvmnHun0Ein0+6ZZ54ZlNu2KRSLxYGPHXbYYW7GjBmup6dn0LEceeSRzjnn8vm8O+mkk9wvfvGLQW396Ec/cul02rW2tg76nu/27k30iiuucFOmTHFvv/32oMwJJ5zg9ttvP1culwe+5thjjx2UueCCC9zee+/tAEC1ePFiN3v2bJfL5ZxzzrW2trpJkya56667biCzo3XLOefOOecc99GPfnRQ4dDZ2elmzZrlFi9e7Jz7y7529dVX++rjXys03rvfbd682S1cuHBgf9jm3//9392UKVMG/r9gwQK31157Derv3Xff7dLptHvppZecc++s++/dR5YsWeIee+yxIftWLpfdfvvt5774xS8O+pq1a9e6KVOmuCuuuMI595dC42c/+9mg3IIFC9y0adNcV1fXwMdOOukkl06nB+0J3/zmN92sWbMG/n/OOee4m2++eVBbDz/8sEun0+73v//9oO+5bt26gcy7C41ly5a5dDrtnn/++UHtXHTRRW7atGmus7Nz4Gt2tGfjw4ff0fiQmDJlil122WVm9s6fkOvp6bEnn3zSrrnmGuvv77ezzz7bVq9ePfDXH0ql0sDX7rPPPlZVVWXPPPPMoF+23mbbX6464ogjBn388MMPtwsvvNCee+45mzt3rpmZJZPJQb+kvXLlSnPO2fz58wd9z/nz59v1119vL7zwgh188MF/17GPGTPGKioqBv4/adIku/XWW83zPFu3bp299dZb9vrrr9vq1asH9cHMbNq0aRaJRAb+v+2X/LLZrNXX11tLS4tdd9119uqrr9rcuXPtwAMP/Ku/7/Liiy9aoVDY7lzNnj3bRo8ebc8995wdd9xxAx9Pp9PbtTF9+nSLRv9y6TU1NVlVVZVVV1cPfKyurs5ee+01M3vnd1q2/QWWtrY2W7t2ra1evdoef/xxMzMrFovv2993++1vf2szZsywsWPHDvr4kUceaRdeeKGtXr3aJkyYYGbb/xzzyJEjLZvNSt8HAEqlkt1333128MEHWz6ft3w+b8lk0lpaWuzOO++0008/fdDa/NesXLnSWlpaLJlMDqzxVVVVNnv2bHv22WcHZd/941Z/r/fudyNGjLCf/OQnZma2ceNGW7t2rb355pv2+9//frt1eMKECYN+GXrbL3NvW0dbWlrs9ttvt9bWVps3b57NnTvXzjjjjPfty5o1a6y9vd3OOeecQR8fN26czZgxw5577rlBHx9q79ljjz2strZ24P9NTU02bNiwQXtCXV2d9fb2Dvz/qquuMrN3fsx47dq1tmbNGnvsscfMzN/eM3r0aJs1a9agjx955JH285//3P7whz8M3GPsaM/Ghw+FxodEZWWlTZs2bdDH9t9/f+vv77cf/OAHdvzxxw/8Ut1ll102UJS8W1tb25Btb/vdgqampkEfj0ajVl9fP2jRaWhoGPQ7Ie/+xfKhtLa2/vUDEzQ2Nm73sZtvvtluvPFG6+zstMbGRpsyZYqlUqlBfTV75/dY3i0cfufXkjzPMzOza665xm644QZ76KGH7Je//KWFw2GbM2eOXXrppdvdkJv95VwN1afGxsbtvv9QuaH+Esd7+/leTz31lH3nO9+x1atXW2VlpTU3N1tlZaWZ6X+7vLu7e8i/mrWtjz09Pe/bn3A4zN9IByBbsWKFdXR02F133TXknyp//PHH5Rehurq67MEHH7QHH3xwu88NGzZs0P8bGhr+tg4P4b37nZnZfffdZ1dffbVt2rTJ6urqbM8997RkMrnd1+5o77n44ott5MiRdt999w3s1zNmzLBFixYN+YvU2/ba99t7Xnnlle36/l5/y97z0ksv2WWXXWYvvfSSJZNJmzBhgo0ePdrM/O0979dvsx3vPWZ/OW/48KHQ+JCbNGmS3XnnnbZ+/Xqrqakxs3f+Zvm+++67Xfbdr2QM9fH29vZBN6LFYtE6Ozv/6qsI277nj3/844Eb33cbNWqUfjCi+++/3xYvXmznnnuuHXPMMQMbzZe//GV76aWXfLVVXV1tX/va1+xrX/uarV692n7961/b0qVL7bLLLrMf/OAH2+W3nauOjg7bY489Bn2uvb19yOLk7/X222/bGWecYQcddJDdeOONA6+wLVu2zJ566im5ndra2iF/obu9vd3MjFeLAATm5z//uY0ePdouv/zy7T531lln2e233y4XGtXV1TZnzhw78cQTt/vcu98d3tmef/55O//8823BggV28sknD7zafsUVV9gLL7zgq614PG6nn366nX766bZx40Z7/PHHbenSpXbuuefaQw89tF2+rq7OzOx91/CdsX5nMhn713/9V2tubrYHHnjA9thjDwuHw/bEE0/Yww8/LLdTW1tra9eu3e7j7D3/HPirUx9yq1atskgkYmPHjrXx48dbQ0ODrV+/3qZNmzbwb+TIkXbVVVdt94rHNtuKkvvvv3/Qx7f9WcL3vt35btv+5GtnZ+eg79nV1WXf/e53B16FCdILL7xg1dXVduqppw4UGX19ffbCCy/4etVjw4YNNnfuXPvlL39pZmbjx4+3U045xebMmWObN28e8mv22msvi8fj252r559/3jZu3GgzZ878G4/q/b388suWz+fttNNOG/Q2/rYiY9urStte+Xk/++yzj61atWrQX2oxe+cVuqamJtt1110D7jmAf0YdHR321FNP2eGHHz7wF6fe/e8Tn/iEPfPMM9utRdu8dy3bd9997Y033rBJkyYN7DFTp061//7v/7ZHH330f+OQzOyd/dbzPDvrrLMGioxyuTzw41vq/pPL5eywww4b+CtTo0aNsi984Qt2+OGHv+/es/vuu1tTU9N2e8+6devsxRdf3Cl7z+rVq62rq8uOP/54mzhx4sC4PPnkk2b2l+NV9p4NGzZsV4zdd999FovFbPr06YH3Hf84eEfjQyKTydiLL7448P9isWi//vWv7f7777fPfe5zAzfcZ599ti1atMgikYjNmzfPenp6bOnSpdba2mpTpkwZsu0JEybYUUcdZUuWLLFcLmctLS325z//eeBPs777b56/VzqdtiOPPNIuueQS27Bhg02dOtXWrFlj11xzjY0ZM8Z22223IE+Dmb3zOw633XabLV682ObNm2dtbW32wx/+0Do6Ot73XZuhjB492kaOHGnf+ta3LJPJ2Lhx4+zll1+2J554wk477bQhv6aurs5OPfVUW7JkicViMTvooINs/fr19r3vfc8mTJhgRx99dFCHOWDKlCkWjUbtP//zP+2kk06yQqFgd911l61YscLMbOBPF297d+mBBx6wvfbaa7t3V0488US777777MQTT7QzzzzT6uvr7Z577rGVK1fad77znR1uFgCguPvuu61UKr3vj9QeddRRduutt9rPfvYzi8fj232+pqbGVq1aZb/73e9s9uzZ9v/+3/+zz3/+83baaafZsccea4lEwu644w771a9+Zddee+3OPpwB226Iv/nNb9qnP/1p6+npsVtuucVeffVVM3tnLVYeUpdMJm3KlCkD+0hzc7OtWbPG7r77bjvssMOG/JpwOGznnHOOXXjhhXb22Wfbv/zLv1hnZ6ctWbLEamtrh3y35++1++67W1VVld1www0WjUYtGo3aww8/PPBnabf9vsm2vefRRx+1Aw88cLt3+48++mi79dZb7cwzz7SzzjrLxo4da4899pgtX77czjzzzIGvx/9NFBofEq+88op97nOfG/h/IpGwcePG2dlnn20nn3zywMc/85nPWGVlpf3gBz+wO+64wyoqKmzmzJl25ZVX/tUf6/n2t79tu+66qy1fvtx++MMf2vDhw23hwoV2xhln7PAG9PLLL7cbb7zRbr/9dtu8ebM1NDTYJz7xCfvKV74i/7KfH0cddZStX7/eli9fbrfeequNGDHC5s6da8cdd5xdcskl9sYbbwz8UvOOLFmyxK6++mr73ve+Z52dnbbLLrvYmWeeOfC8jqF86UtfssbGRrvlllvszjvvtLq6Ovv4xz9uX/nKV3b4865/i1133dWuuuoqW7JkiZ1++ulWW1tre++9t/30pz+1hQsX2vPPP2/Nzc126KGH2r333msXXHCBHXPMMXbppZcOaqepqcluu+02u+qqq+zb3/62FYtF23PPPW3p0qV20EEHBd5vAP+c7r77bps4caLtueeeQ35++vTpNn78eFu+fLkdc8wx233+3/7t32zp0qV2yimn2IMPPmh77rmnLVu2zK655ho777zzzDln6XTavv/97/+vrl0tLS22aNEiu/nmm+2Xv/ylNTY2WktLiy1ZssTOOOMMe+GFFwZ+qXlHvvnNb9p3v/td+9GPfmTt7e3W0NBgxxxzjH35y19+3685+uijrbKy0m688UY744wzrKqqyg444AA755xztvsdyyBUV1fb0qVL7YorrrAvf/nLVllZaZMmTbJbbrnFTjnlFHv++edt/vz51tLSYnPmzLGrrrrKfvOb39hNN900qJ1UKmU//elP7aqrrrJrr73WMpmMjR8/3r797W8POf74vyXk+A1PAAAAAAHjZyUAAAAABI5CAwAAAEDgKDQAAAAABI5CAwAAAEDgKDQAAAAABI5CAwAAAEDgKDQAAAAABE5+YF/jLiPlRlMuL2eb6vQHnJVLRTlr5ufxICG9VR/NHnnCV+VsxfDRcvaZp34tZ71sj5zd8uYqORt2JTk7seUTcjZS6JOzUxq3f6Ls+yn6rKl/+9KLcjYe9/HcSx/zx89c8/M4HOe8ndSun6wcNb23Zs7Xdf/BH5uf7IOP6dfnP5MqH08Vjpm+blWlYnLW88pydqfxMZemtRwsZ+PVtXJ29Zv/I2e9Yk7O9rWvl7Mh08di+G5T5Gy4VJCzu1TpD6ot+9yb3tqon4toZOe8luzr6Wt+1k5fG6Sf6E5av/WoTztnb/LVAx/N/um1vz4neUcDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQODkRxpHYgm50YiPJwo2NQ2Xs2Uf7Zby+lNHcz6yFtKfIh6v0p+o2tObkbMFH/2N+niaeiKmP+HaOb1GLef65ezsibvI2bkfbZGzr72tP03VzOyP//OKHtanhFnog3/CtOf56fDO4eMyMl9PSfUxGM7HeQiF/QyGjz7oreJ9hCL6uuXnlbWqqmo56+fp9Z6PNblY0p9k7kckkZSzuXxezpZ89Ddc1p/g7e8J1/r15xX1sRjXpO/nE8fvKmfbtnbJWTOzDa2b9PAHv9T764OPC8n52EtDPtZkP3uTt5NWcOenv2EfHfbRXX979F/HOxoAAAAAAkehAQAAACBwFBoAAAAAAkehAQAAACBwFBoAAAAAAkehAQAAACBwFBoAAAAAAkehAQAAACBwFBoAAAAAAkehAQAAACBwUTUYj8flRgt9/XI2FI7I2fRuu8nZXRpq5OxLr74qZzMFOWqhWEzOdna0ytlioSRnI+WinI3H9bEolvXn09dZr5ydNXWunK0dN0nORjd1ylkzs3BIr8G9kNMb9vSsczspKyd9hndSH5yftI9xC4d9jPHOGWLzPE8PY0jRqL5ulfL6Ah7yMT9GDBsmZ2sqk3J2Y6u+L+T1bcFCEf2c9ffo63e5VJazYadno1F9LMo+LsAKy8nZcaMmyNlU/Ug5G+7Oylkzs1BI33v9rfX+VmW92X+AvclPf/0064s+bjtrjP0MsZ9x2xHe0QAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGLqsF4LCE32lMqy9liuShnU5WVcnb/A+bJ2TEjG+Xscy+/KWfznhy13r6MnC1l9Wx/sVfvhBeRo9FYXM6OH1ErZxt3GStnE8mknI3rh2ZmZqFwSM76adoL6e2GnZOz+hVn5qMLvsKe3l1ffHU3rL920tevrz1bu/TrqFAqydlEjNd6/l6RiLyNWdHTF+Wyp19VsYS+P+4xIS1n62qq5OxbmzrkbMnHtZrL5+WsV9SzhbKeNc/HuhnRV+SG6pScraytl7PRWEzP+lwCQj7W5JDpA+1r+fYR9tOur73Jx87g+dhLdxY/41Yo6mtPX39OzpZ9rH/RiK/B+KvY5QAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOCicjCRkBstlvTHnIdCPmqdcEyOxmub5Oyc+R+Xs9HkM3L2d2v/JGf721rlbKTYJ2ejPs5ZbeNYOVuV0Me4oqJGznqm9zdUKsrZUrksZ99pPKRnnfPRrI92w/q1UcwX5GzExzUXj0XkbCGnj0ckrLcb8nN+TT+/3Rn9OtrQtlXOlsolOTtmpH5tYGiRqL5mlD19LpmPuWQ+rqlIskrOjm+erLcbe1POrt26Sc4WMr1yNlzW16FwSF8DUlX1cjYR1cc4Hk/KWWd6f83HflP29L30HT7mpe2c+e5nGyuX9eML++hDNKJfc6WiPh5hH/uur73JxznL+tjPu3r75azn9LGoq9GvjR3hHQ0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABC4qBqMJ308jjwUkaOe02udnu5OObth/To5u8u+s+Xs7uN3l7Mvv/ILOTsyXpazI8bvKmcLRTlqw4dV6tk6fT7UDquXs319PXK2vC4rZ7s7t8hZM7POPv3EtbdukrN11VVyNhGPydn+XEnONtbq41yTlJcIC4VScjZX0Ptrzul90Fs15/S03gOzcFhf/0J+OowhRaL6HPXz2przMZtyWR9rUZe+j9XW6mt9Q2OjnN246WU5WxPx9GzjMDlb0rc8q66I+8jq62aqokLOFvI5OetK+v6R6++Ts2Zm/QX9xGV6u+VsKpGQs9GovsYVivr8qUrp45yM+XidPKTPiVJJ76+/ncFHqzunWQv52HCC3Jp4RwMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAAQuqgZTSf3x9IViQc5m83k5m8n0ytmOLZ16H3oyctbCcTk6Yfx4OduXL8nZzoyezRaKcnZYbUrPNjTIWRfS+5DP62Nc7tHnTimzVc6amfX36W2vW9chZztT3XI2GY/I2fph9XI2HKqSs4WiPnaxSEzOFs2TsyUry9lwSH/tJBQKyVlzehT/u2IxeRuzcllfO4slPZvP5+Rspi+r9yGnr0MW0teLpsZGOVso6ddqf16/VotlPVuZ1PfdispKOetCeh9KJX2MvZw+d8r5PjlrZlYo+LhX6NTvbfpj+ryMRvV1tqKiQs6GTL/PLPmYPxEf10bZx2KvXxk7cW/6EOAdDQAAAACBo9AAAAAAEDgKDQAAAACBo9AAAAAAEDgKDQAAAACBo9AAAAAAEDgKDQAAAACBo9AAAAAAEDgKDQAAAACBo9AAAAAAELioGqxI6Y+RLzn98em5gv4Y+XKpKGdff+1lOTtmWErOVlUm5WwkqtdxqbDeh+7+Xjkbi0TkbFVSH+NsNidny1F9PnR1det96OmRs61bO+WsmVlthT7Oqbh+jstlfb6Xyvp5C5mTs/0+xs5HsxaL++hvRM+aj/XE/ER9ZfVwOKxnE4mY3gkMKR6Ly9myjwlSLHly1vP067q9baOcravU50ciLm/nvuZoLKb3IeRjPw+H9P0x4aMPxaJ+n+B8vNTqZ88r5LJytre/X++EmaVi+jjHIvoBep6+2PvJ+tmbCj7Gzo+Ij/0m5OPaMG/n7E3+7Jy9KRrV72t2+H0DawkAAAAA/n8UGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACJz/LvqZptNxodeMYOVsKxeRsX3+vnH3ttVfkbE1SPg02Y+/pcjaTK8nZeFR/NHxVZUrOVlbXydnGumo529a6Wc7WNw2Xs5mebjmbzebkbMMuo+SsmdnMUETO9nVvlbPtWzvlrOecnM34OBfJigo5Gyvr8zJcLsvZYtmTs6Z3wSe94ZCPl2RCPtp13k47uH8aSR9rXLJKz3o+1oBCIS9n29r0tTMZ0yfemDH6Hp0v6ddfJKzP0URC38+rE/o6VFWRkLO9PT1yNlVdI2fzuaycLRT0vb+ypk7OmpmNHaPPiXyuX85m+vSsM31vyhf1cxGL+5iXPtbOUFhvt+zpx/aPwN/epPNx+7FDvKMBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACF1WDldXVcqOxyjo56+L6c849K8rZsI/nsm/Y3CFnR7VtlbO19cPlbDysn4eJo0bL2araBjkb89GHWEVczvb19cjZXD4nZ/O5gpytrKmQs2ZmE8aPk7M9nZPl7G9/+zs563llOVt0+tjl8vp5i0YTcjaZ0udEzEpytlTSs6afBjMf58wP56MToVBkp/Thn0kioc/RSFxfB1zEz96kX6thC8nZrp4+OVvb2y9nUxVVcjaid9eG19bJ2USq0kcf9LGIxOTbGssX9P2mVNLvP/ysWfGkvm6amTU1DpOzuX59/ry1dq2cdT7WzrKPbLHk4zoK6+Mci+nrbCTiyVmvrGd97U07iZ8uhHzcQ+8I72gAAAAACByFBgAAAIDAUWgAAAAACByFBgAAAIDAUWgAAAAACByFBgAAAIDAUWgAAAAACByFBgAAAIDAUWgAAAAACByFBgAAAIDAyc9wr6qulRsNV1TJWc/rlrMhT3+AeigakrPJZELO9nTr/c17eTk7ZuwYOVtRlZKzxZzeh0gqLme9cEnOVtRE9Gx8pJz906oX5Wwuq4+bmdluu4+Ts3s2T5Cz7W2b5ezGjRvlbFy/NKxYzMnZfF5/LcLH5WmxaEzOlstlveGdxcexmb70WCjMaz1/r0RSXw9DcX2tdy6rt+tnfoT1CRKLylu05bJ6f0tOX7/r6urkbDzh47ou6n0Ix/Q9xIX09SKe1MciHqmRs5vWr5ezftZjM7OGhno5O2JEk5zNZHrkbJeP+6Coj2ujXC7K2VJJHzs/l2ckrM81z/N8tLyT7Ky9KeQjvAPscgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHBRNVhRWSE3moj4eHR5LidHC7mCnI0nknLWFX30oViWs+PH7iJnt3a3ydmI3gVrqB8mZ7d09svZda+/JmcnTJokZwteUc6O2qVJzkZMnztmZpXJuJxNjhouZ2ftNUXOZvv18ejo7JKzqURMzkYictRCIR/XvY9o2dMnfCyqj1s47OPgfHBOz5aK+nzH0GJxfcyjPuadlfWxKfnYF6JReds156cPZU/ONtTVyNn+XK+cDfuY+5UV+j1Fpl9fvzvb9b20acRIOVty+hjX1lTL2bCV5KyZWTyqr1t1tXo/xo7W71cKBX08+vqzcjbm49jCvl4m93Ph6zwfi30krPch7Gcv9cHP3lQu+7jR3AHe0QAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGLqsFkPC436uMp51b29GyprLdcFZMPzVIxvRPtrRvk7J7FPeVszKuUsx2r/yxn95g5Sc62v/26nK3wYnLWlfSxyPZ1ydlxY0bJ2bpqff6amWUzvXK2UNLnz7jRI+TspPTucvaPr/yPnPXTXz/Xcn8uL2crqyrkrOf0XsTi+rz0k91ZSqXyB92FD71YNLJT2vUx7czz9HAkovc3FtHbzfR2ydkRnr4ORZy+dmY6NsvZxrEj9XY72+Rs3NPPr/ORLRaycnZYfa2cTSX8zd9iXl9nS56+1g+rq5GzfcMb5eyGza1ytuzjHs/P3lQoleRsIuHjXtfHIhHxsU75WSN2Fj9r2o7wjgYAAACAwFFoAAAAAAgchQYAAACAwFFoAAAAAAgchQYAAACAwFFoAAAAAAgchQYAAACAwFFoAAAAAAgchQYAAACAwFFoAAAAAAhcVA2mkim5UReOydmi02sdFwrJ2YqKCjm7x7gxcnZze4ec/fNLf5SzE9KT5Wyut0vOFjo2ydnMllY5Wzda729EHzarqdLnWV19rZxNRT29E2bm8v1ytlR2cjYa1rOjRgyTs29vrJGz3Rn92LK5op7N5+RsVVWlnI2EInI2HPaRjehZfdT8KXnlndTyP49YNC5nnY/5UXY+Fi4fe1M8rve3aVidnO3J9MnZ1g0b9T4MHylni7msnC1nuuVsvq9XzqZq9f6GfQxxMqHf16QqknI25mNPMDOzUkGOej7mcDik96O2Wr+/6uzWz0U2rx9bsaivncWSvo8lEvr1GQ7p968hH2tEKOzjvlhO+uN5/u6Z/hre0QAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGLqsGqioTcaH1Cfyi6K6fkbCgVk7O1dcPk7G5jxsjZquoqOfvKm5vlbKZ9rZytqa7Q2+3TH3vfOGZPOZuqq5az+dxWOTtmt/FytqGhVs72d7bKWTOzUFSfa2WvIGddKS9n62r0cfYz3/NlOWqey8rZktNft4hG5KXHIhG93VwuJ2eLBX3czPlY08xH1tOzGFoirs+lCj1qLh7Xw7GIHE2l9Ot6WF2dnE0k9D16U0ePnM1n9PU7mdTXzXxB35sq60bI2XgqKWeLxT45W9fQKGcrK/X7mmK/PhZmZsWIPte8gr7YO68kZyuS+rWRTFXK2ZInR825opz1TJ9rkZB+fsMhvd1SUT+/5ZKPTdrH3mR+9iZf7f51vKMBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACR6EBAAAAIHAUGgAAAAACF1WD1amE3OiIKv3R5aWY3q7nyd21fFl/ln0onpKzk6bvIWcbxvTI2Z4Nq+Xs9H2OlLOdmzrl7ISJzXI2379FzrZuXC9n62qr5GwqLkfNJWN62Mz6oxE527p5k5yNFHvlbHVNtZytH9YoZ3uyZTkbS1bK2c7ejJwthfXXOKIJ/fosFvJytlwqyVk/nNPXHs/TsxhaMqbvC9UJfW/yInq7ztPnc8nT+xCK6ovcyNFNcrayPidnc10dcnbUrtPlbH93v5xtGj5CzpYKfXK2t1vfHyuS+r1KXN8+zGJ+wmaFvD7Xenq65WzY09fOhI9zUVGp7yG5or4eRmL6tdGf87EvhEJyNhzV7yvKZX2/8Tx9j/bDmb72OKdnd4R3NAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOCiajCeSumNRvVHuBdz+qPWQ05/PH3rlm4929UnZ4fvoh/b8BG7yNmQi8jZVNPuctY1TJSzMR9jnKiqkbPZkn7OIslqOZsv5+RsLq/PMzOzvr6snC2XinK2Z+tWORtNxOVsQ70+Hn0FfTyiEf21iAof5yyT7Zezoah+3Yd9jIXzsZ5ks3k5WyyW5Gx9tX7NYWiRWEzPhvW5X/YxP8ycnOz1cZ309uvzrlpfAqzaT9jp5yxe1aC3W9kkRyMxfS2MJpJytuhjiEOxhJwtefoaUPTTCTPL532scZ6+7/X36fdB4ah+v1KZ0sej4ONeIezjWo4l9XOWLxbkbCiiX/chH2PhfKwnfvabclmfaxUJfV3dEd7RAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABC4qByMJ+RGO3tycnb9W+vlrFcqyNm66go5+/b6zXJ2cnoPOVs/YpicLRSdnH3r9VflbOPYCXK2r5SXs4WCPsbDR42Ws6lKfdz6u1rlbGdPVs6ambVt7ZGzIdPHrqenW876OW+1Nfp5q82F5GxvX5+cdV5Z70NllZztKZfkbMH0Yyt7+rgVinofigUf2ZInZzG0SFTexqw/V5SznVu65Kzz9DFPJeJydmuXvg6NHN4kZyuqK+VsSb+sbUubviZX1ev9LfhYA0o+stW1dXI2HtfHrZDVx83PnDQzy/Tre68fuZzerp/zlkrp5y1X0tfvXF6/XzGnr7OphH6vm8vq7ZZ87E0+umvlsh4ul/WL2c/+uCO8owEAAAAgcBQaAAAAAAJHoQEAAAAgcBQaAAAAAAJHoQEAAAAgcBQaAAAAAAJHoQEAAAAgcBQaAAAAAAJHoQEAAAAgcBQaAAAAAAIXVYPJZEputBhvkLM95TY5W+7LytlwOCdn16zfLGe39vTL2USdnh07dhc529PTJWdb33pRzhadPB2svq5Wzsbrx8jZ7g59LF5/eZWcfXvdejlrZjZql0Y5G6utkbPdVfp5i8WTcjZa1l8zSCVjcrY/6+RsIdcnZ8OpCjkbiejH5jw5ak4/NDNPD3s+soVSyUcnMJRoTJ/P5UilnM25XjnrFQpyNhQqytktXT1yti+n9yFaoWfr6/X1LZfT9+ierfqaXHb6GlCR0u9VIhV1cjab0ceibdM6OdvZ2SVnzcxqa6vkbEVK30NyCT0bierXXNgLydlYNCJnC/plZKWSPt9jIR/HFtaPzXzsN362Jj8bmZ89r+z52Ex3gHc0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4KJqsLIiJTda6eOx99GKKjkbj+qPe09Ey3K2K9MvZ19//U05Wyjoj72vnDJJzrqi3t+mhjo52zhqnJzN9vfJ2XVvvy1n21vb5ez6N9+Qs5neLjlrZtZYVyFnS4WinN1l7K5ytqauQc52d3TL2ZB5crbfxzhHQvr12d+nt5svluSsc3LUvLJ+Hkw/NIvFfLx+46O/GFoiFpez8XhMzkZiCTkbDesTJBrW5102r68t7e0dcrZc1q+p+C4j5awr63tedaWPe4raYXK2WMjL2c6tnXI209srZ7va9X0sn8/KWTOzqpQ+38tl/T6opl4/x8lUpZzNZvwcn74gFnyMs59X1At5fQ6XPP1a9rM3OR/t+tmbIhEf4QD3Jt7RAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgYuqwWRFhdxoqiLlowv6I9FjCb3dUFh/jHwmm5OzHVs75Wx9dVzOrl+t13w9XV1yNu9F9HYL8nSwjr6ynG3dtFHOxrJtctZMH+OmYTU+2jXr6e7We1HI6/1obJCzLhLTs54+Hj09PXJ2/QZ9PPxcG3W11XLW87FGdHX3ydnuTL+cjUT16ygZ06/7mI92MbRoXL9OYjE962dvikT1dkOhkpzNF4pyNtOnz+eKhD7vujr0vSmX1ftQ8vR2a0p6tq/g5KyfdT5S7JWzZvp6XFWR9NGuWS6XlbOlkj7Xqqoq9U6EfaxbzpOjfo6tqysjZzP9+rxMJRNy1vlYI7I5/T4hm9ev+1BYvzaiER9rWiS49yF4RwMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAAQuqgZTiaTcaLWPR9nrD0Q381MXubDe355sWc6u29whZ3dtSsnZbr1Z6+kryNnWrd1ytjert9tX0keuIubp2VBWziarKuRsKp6Qs2Zm7V29crahsUHO1g0bJmeLZf28FQolOdu2pUfORmP6eRs7ericHV6vrxF+ruUVK1+Ss+1bOvU+OCdno9GInPU8fe3B0OLRmJxNJuI7qRf6euhCen+zRX0N6OzJyNlhVT76oDdruYI+n3v79LU+V9TXt4Knj0U8ol/X8VBRzjof8ywWlW/DzMwsk83J2Uof92IVFfp+Wvb0eVnysY9l+vRjC0f081ZfWy1nqyv0sfNzLb/+1gY5m+nrl7PmY2+KhPW9yfkY4x3hHQ0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4+RnulcmE3OiBs3aVs7HC23K2a2uXnLWS/qj1uqo6vd1wSY62dfbK2Uy+rGcLelYeYDOLhPRH2Vd4nXK2sXa4nA0V5KiFnSdnezL6WJiZNTQ1ydmRIxvlbD5XlLNtHVvl7OtrW+Vsv0vK2XHj6uRsdbhPzu7SUClnC6EKObvqz9V6u0X9OopE/LwmE5KT4ZCexdDiMX2VmzBumJyNlPU1LtvXL2fN0+dSKpHS2w3p62Emm5Oz+ZLebr6sZ/Ud2izs4zKJO30sKlNVcjZUiutZp++lubw+FmZmlVV6n2tq9GzJx3rYm9HPcftWfe8tWEzODqvX94VEOC9nayv1e92Sj/6ub9X33bKP6yjk5+LwsTeFAtybeEcDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAELqoG6+sq5Ubn7rOrnG1uyMrZxlCbnP3NSxk52xkdq2c7NsvZJ1atkrO5fv08DGtokrNzZjTL2Vg8ImeLIb0PsUhMzrqok7NdnZ1ytmwhOWtmNmGcPif6fIzdMy+vk7MrX3hZzq596y052zx9hpwdVpOSs+1trXJ27DB9jegvl+VsMqWvU4m4Pi8t5MnRcEh//aZc1uc7hlaRisvZibsOk7MjKotytjLUK2fXbMjL2f5IvZ7N9MjZ19etl7PFQkHOVlZWy9ndxw6Xs5GIj2vKqvR2Q/qe5+dl2f7+fjnrfO5N9fX6nCgU9Dm8epO+n655e5Oc3bpli5wdMXqMnK1I6ut3ple/Pusr9HY9p6/f0Zi+TkWjPual6X0IhfS55nnB7U28owEAAAAgcBQaAAAAAAJHoQEAAAAgcBQaAAAAAAJHoQEAAAAgcBQaAAAAAAJHoQEAAAAgcBQaAAAAAAJHoQEAAAAgcBQaAAAAAAIXVYMNVQm50Y6Cng2H6uWsRUp6H/IFOdvW0ydnu7bq2Y1b9Mfe9/d0ydm+vCdne/ZMy9ndU9VythTR+1DKZeRs55Z2OWuJSjm6+66j9XbNbPXbm+Ts48+9LGfXbNXn8NsbO+Xs+o2tcnbUhH45G4836e2OGSFnvai89Fg2U5SzjfVVcnb0iAY527plq5y1kJOjZf0ywvuoTMTkbKakz7tQqELvREgfyEyprGdz+j7W369nu/tycraQy+rZkj73dxk5XM42xvV7Cs9HH7xSXs729+n7mEXjcrRhWJ3erpl1dPbI2dfe2ihnt/Tpc7izW99DOrv1+6DaJn2tj0b0+5XKOj3rwvrr78W8fi1XVehzuLZav7fp7dPvSc18XBt6dId4RwMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAAQuqgYjITlqfaVaOftqmxy1V1dn5Oy6te1ytrvzdTmb7dP70NPdLWdz/f1ytm1Lp5wd2dQoZydOmChnE4mUnC3ksnK2rUM/tmSFp7dbU5SzZmb3P/ZbOfvsqlflbMOo3fROlPNyNOzjJYM1q9+Ss7Obd5Wz43cbJ2fD0YScTdTH5eymrfpcy2f1a86V9bkWkpNmzuntYmjhkD75C56+brX26mvR5g79Wu3c2itnc/36Blko6H3IZfXrpFgoyNnePv2aqqmqkrPDG5vkbDQWk7Olor4v9Gb0Y4vFnd5usixnzcxe+p+35Oya9a1ytrK2Qe+EV5KjYR8L4paOLXJ23IhhcraxQc+Gwvq9brQiIme7+/W5Virq15zz9LlmenfNnI92d4B3NAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQODkZ62//NrrcqPrOrbK2VffXC1ns7msnK1taJSz8WRSzrZt2ihnw7G4nE1VhuRsTX29nF27Qe/vqpf+LGfTE3aXs7vU6+d3xKhRcrZcKsnZtzdukrNmZn94+TU5m+3LyNlitlfPFvNy1pyToxGnn7exo0bI2YqqmJwNxVNytjJWJWcLuT/I2XxOP79lz5Oz+kj4axdD29jWJmc7M/1ydnNHh5wtFotyNlWpz+doTL+meru75WwoIm/9Fkvoe1OyokLObvXR33UbN8vZ4U363l9boZ+HmtpaOev5uK47u3vkrJnZhk36fC8U9DUuWczJ2XJZ30P8rIhh089bfW21nI0nInLWIvo1l4gk5Gy5uEHOlor6+XU+9n4/vADb5R0NAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQOAoNAAAAAIGj0AAAAAAQuKga/M0Lf5Qb7e/PyNl8WX/MudOfTm+lXF7OFkp6w5FUhZwdNXa0nB3ZOEzOhnyciD+u0sft/gcflLMHz91Pzu7ysX3lbH3jSDlrXlmOZjd36e2aWcSV9G4UeuVssXO9nI2X9T6EPT2bDOvzZ/RofTzGjh0hZ7u69XNW0g/NImEfr52E9KgvTl/TfC1qGNKatzfK2UJB3xd8bAu+htwr6hO65GN/DMficra2vk7O1lTpe56fE7Fx/QY5+/Kf/iRnmyfuIWdrJ+4qZyuqauSsn+u62JPV2zWzsI+2XSknZ8v9XXI24mPv9XO/Eg3p86euTh+P+vpqOduf1dcIz8caEQr52HB21t7ki49FbQd4RwMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAASOQgMAAABA4Cg0AAAAAAQuqgYbGkfKjbr2TXJ205rVcrawtU3OducLcjZUVSdnezOdcjaa07ORbK+c7erS202EPTnb1rNVzq787fNydtTwBjk7cfw4OVtbVSlnN2x6Vc6amXX16OfYKxflbD7TLWeTsYicDTknZyOmz4lCPi9nh4/Wx65sG+Vsa+sWOWvhkByNRGN6u6GSHC37GAte6fn7VVbVyFnn4/rr2dIhZ0t9+vqdLelzKZSokLO5fL+cjRT1bLiYk7P9/Xq70ZB+nfTm9HbfemutnK2t1veQ4Q31cjaZSMjZru5WOWtmlvVxLpynr/WlfFbORiM+Vi5f66GP/vq4jqprh8lZz/Q1orenT8762ZvCYX3vt5B+zjwfY6H3dsfY5wAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOAoNAAAAAAEjkIDAAAAQOCiarC2tlZutK11o5x9/Q8vyNlkZoucLY8YJ2drGkfq7W7VHzmf29IuZ7s3b5az0ag8bDZqlH5sYdMfT79xc6uc/eVjT8vZzW3NcrahRp+Tq/7wRzlrZpbN5/VwSB+PotPPcUzvgSXi+msGyZSPlsueHM33ZeVsf6ZfznZs6ZSzhWJZzu4yWr82qn30t+SV5GwoxGs9f69UMilne3u65Gzb+rflbCyv7wteTb2cTVbVyFnXr69Z+b6MnM329MjZcESfz3W1+rGFfOxNXb29cvaV/3lTzvaMGi5nK5MpObt+wwY5a2ZWKOnri/lYX8o+9qaI3gOLRkNyNhbz0bKn97dYKMrZQr4gZzN9+r5Q9rGX1tTp10bCR389p/chZPq47Qi7HAAAAIDAUWgAAAAACByFBgAAAIDAUWgAAAAACByFBgAAAIDAUWgAAAAACByFBgAAAIDAUWgAAAAACByFBgAAAIDAUWgAAAAACFxUDZZ8PPbe+ahfyiX90fDRqP54+sZdx8vZfleWs142I2eLPh4NXy75eDR8SD+/zjk5W1dbLWd7+rJy9pXXXpOz7e2tcrYylZSzmZ4uOWtmloiE5GyyoUnODmtokLO9bWvlbJ2Plwy6e7bK2Q0b1svZXCYtZ7P9vXLWfFyflRUpOTt+913lbLmQl7OFor6mhWJ6fzG0sqevnX5eW/M8fd6FI3q7lcMa5WzRx/rtCn72G30/9zy9D6GQvm76ODRLJfW1PpfXr7/NbW1yNpPpkbOJWEzO5nP6XmpmFvWx1sdS+p5eUVkpZ/O9+h5SkZCjls31y9nuri45W8oPl7PFQk7OmulrTzyuz4nGhmF6D3xcy2Ufa1oorPd3R3hHAwAAAEDgKDQAAAAABI5CAwAAAEDgKDQAAAAABI5CAwAAAEDgKDQAAAAABI5CAwAAAEDgKDQAAAAABI5CAwAAAEDgKDQAAAAABC6qBvNZ/bHs+kPOzZIV1XK2JpSUs1X19XK2/c0/y1lXKshZz8lRS1VWyNlIVH80fDgsD7ElEnE5O2JEo5zty+XlbHd3l5zN9kXkbESPmplZIhqSs7V1+lyrHzFGzhYzHXK2nO2Vs7mCPh6r16yWs5lMi5xNplJydli9J2dDIX3c4j7meySZkLOFsn7hJ1OVchZDKxWLclafSWaxuL7fJE1fkxMV+lqfad8sZ51X0rNy0iwW16+TsI+FNhTSX+eMxvR9rLqmSs7mS11yNpvNytmijzU2rC9ZZmYW9fHycCqlz7WKmjo56+Uzeraon4tiWZ/DHVv0/TGf31XOxmL6fK9I6VdSyPSBjkZ9XEc+ro2yj5vSWEzf83aEdzQAAAAABI5CAwAAAEDgKDQAAAAABI5CAwAAAEDgKDQAAAAABI5CAwAAAEDgKDQAAAAABI5CAwAAAEDgKDQAAAAABI5CAwAAAEDgQs45/ZnkAAAAACDgHQ0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgaPQAAAAABA4Cg0AAAAAgfv/AHoIsKIlMyaBAAAAAElFTkSuQmCC",
|
||
"text/plain": [
|
||
"<Figure size 1000x700 with 2 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"cifar_train = datasets.CIFAR10(\"./\", train=True, download=True, transform=transforms.ToTensor())\n",
|
||
"cifar_train_loader = torch.utils.data.DataLoader(cifar_train, batch_size=128, shuffle=True)\n",
|
||
"\n",
|
||
"train_features, train_labels = next(iter(cifar_train_loader))\n",
|
||
"img = train_features[0]\n",
|
||
"\n",
|
||
"fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10,7))\n",
|
||
"transform = transforms.Compose([transforms.RandomHorizontalFlip(),\n",
|
||
" transforms.RandomVerticalFlip(),\n",
|
||
" transforms.ColorJitter(brightness=0.5),\n",
|
||
" # transforms.RandomResizedCrop(32),\n",
|
||
" # YOUR CODE HERE\n",
|
||
" ]) # add in your own transformations to test\n",
|
||
"tensor_img = transform(img)\n",
|
||
"ax1.imshow(img.permute(1,2,0))\n",
|
||
"ax1.axis(\"off\")\n",
|
||
"ax1.set_title(\"Before Transformation\")\n",
|
||
"ax2.imshow(tensor_img.permute(1, 2, 0))\n",
|
||
"ax2.axis(\"off\")\n",
|
||
"ax2.set_title(\"After Transformation\")\n",
|
||
"plt.show()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "bd138177d2e4877",
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"outputs": [],
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "fc1e7ce8",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Task 3.1: Picking Data Augmentations\n",
|
||
"\n",
|
||
"Your task is to pick your favourite data augmentations and apply them to the images from the dataset (in the later cell). \n",
|
||
"\n",
|
||
"We've already started you off with the necessary one `ToTensor()` that converts the original JPEG-format image to the PyTorch `Tensor` format. Refer to the command glossary to add your custom data augmentations from the list we've provided. \n",
|
||
"\n",
|
||
"**Choose at least 2 additional augmentations.** Tell us which augmentations you chose to use _and_ why. Then tell us which augmentations you avoided _and_ why. \n",
|
||
"\n",
|
||
"__Note:__ Feel free to use any augmentations you wish from the full list of augmentations shown on the [PyTorch documentation](https://pytorch.org/vision/stable/transforms.html)! There's no need to be restricted to the list that we've provided.\n",
|
||
"\n",
|
||
"The point is to improve your model performance as much as possible! Use trial and error to get the best performing network in Task 3.2!\n",
|
||
"\n",
|
||
"Be creative :D\n",
|
||
"\n",
|
||
"__Note:__ Do ensure your augmentations retain the 3-dimensional shape of the CIFAR-10 images. The final images should still have the shape `(3, 32, 32)`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 118,
|
||
"id": "1d132349",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:52:35.817710Z",
|
||
"start_time": "2024-04-07T04:52:35.813776Z"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# pick your data augmentations here\n",
|
||
"def get_augmentations():\n",
|
||
" T = transforms.Compose([\n",
|
||
" transforms.ToTensor(),\n",
|
||
" transforms.RandomHorizontalFlip(),\n",
|
||
" transforms.RandomVerticalFlip(),\n",
|
||
" transforms.ColorJitter(brightness=0.5),\n",
|
||
" ])\n",
|
||
" return T"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b91f7700",
|
||
"metadata": {},
|
||
"source": [
|
||
"Create your data loaders that return batches of data:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 119,
|
||
"id": "8f15171f",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:52:39.178114Z",
|
||
"start_time": "2024-04-07T04:52:37.715878Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Files already downloaded and verified\n",
|
||
"Files already downloaded and verified\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# do not remove this cell\n",
|
||
"# run this before moving on\n",
|
||
"\n",
|
||
"T = get_augmentations()\n",
|
||
"\n",
|
||
"cifar_train = datasets.CIFAR10(\"./\", train=True, download=True, transform=T)\n",
|
||
"cifar_test = datasets.CIFAR10(\"./\", train=False, download=True, transform=T)\n",
|
||
"\n",
|
||
"\"\"\"\n",
|
||
"if you feel your computer can't handle too much data, you can reduce the batch\n",
|
||
"size to 64 or 32 accordingly, but it will make training slower. \n",
|
||
"\n",
|
||
"We recommend sticking to 128 but dochoose an appropriate batch size that your\n",
|
||
"computer can manage. The training phase tends to require quite a bit of memory.\n",
|
||
"\n",
|
||
"CIFAR-10 images have dimensions 3x32x32, while MNIST is 1x28x28\n",
|
||
"\"\"\"\n",
|
||
"cifar_train_loader = torch.utils.data.DataLoader(cifar_train, batch_size=128, shuffle=True)\n",
|
||
"cifar_test_loader = torch.utils.data.DataLoader(cifar_test, batch_size=10000)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "2f0c6794",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Concept 7: Sequential Model Building with PyTorch\n",
|
||
"\n",
|
||
"All this while, you've been adding layers one by one as attributes inside the `__init__` method. This is so that you can quickly debug which layer(s) is causing issues later down the road. However, for the most part, there should be no major issues when creating parts of your network or your entire network. \n",
|
||
"\n",
|
||
"This is why PyTorch lets you combine layers together using the `nn.Sequential` API. It allows you to stack layers inside and chain layers together. It allows you to build isolated modules that can exist on their own (either within a `nn.Module` class or otherwise) and be used as independent \"mini models\" on data tensors. Refer to https://pytorch.org/docs/stable/generated/torch.nn.Sequential.html for more information about combining PyTorch modules to create your own.\n",
|
||
"\n",
|
||
"__Note:__ You should not add an array of layers inside `nn.Sequential` i.e., it's `nn.Sequential(xyz, abc, mno)`, **not** `nn.Sequential([xyz, abc, mno])`.\n",
|
||
"\n",
|
||
"#### DEMO 1: 3-layer Multilayer Perceptron for MNIST"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 40,
|
||
"id": "fd7ede5a",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:12:19.583175Z",
|
||
"start_time": "2024-04-07T04:12:19.572009Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"torch.Size([15, 10])\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"densenet = nn.Sequential(\n",
|
||
" nn.Linear(784, 512),\n",
|
||
" nn.ReLU(),\n",
|
||
" nn.Linear(512, 128),\n",
|
||
" nn.ReLU(),\n",
|
||
" nn.Linear(128, 10),\n",
|
||
" nn.Softmax(1) # softmax dimension\n",
|
||
" )\n",
|
||
"\n",
|
||
"x = torch.rand(15, 784) # a batch of 15 MNIST images\n",
|
||
"y = densenet(x) # here we simply run the sequential densenet on the `x` tensor\n",
|
||
"print(y.shape) # a batch of 15 predictions"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "035610c7",
|
||
"metadata": {},
|
||
"source": [
|
||
"#### DEMO 2: 2-layer ConvNet for MNIST"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 41,
|
||
"id": "a68d4ba9",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:12:21.336248Z",
|
||
"start_time": "2024-04-07T04:12:21.181299Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"torch.Size([15, 10])\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"convnet = nn.Sequential(\n",
|
||
" nn.Conv2d(1, 32, (3,3)),\n",
|
||
" nn.ReLU(),\n",
|
||
" nn.Conv2d(32, 64, (3,3)),\n",
|
||
" nn.ReLU(),\n",
|
||
" nn.Flatten(),\n",
|
||
" nn.Linear(36864, 1024),\n",
|
||
" nn.ReLU(),\n",
|
||
" nn.Linear(1024, 512),\n",
|
||
" nn.ReLU(),\n",
|
||
" nn.Linear(512, 128),\n",
|
||
" nn.ReLU(),\n",
|
||
" nn.Linear(128, 10),\n",
|
||
" nn.Softmax(1) # softmax dimension\n",
|
||
" )\n",
|
||
"\n",
|
||
"x = torch.rand(15, 1, 28, 28) # a batch of 15 MNIST images\n",
|
||
"y = convnet(x) # here we simply run the sequential convnet on the `x` tensor\n",
|
||
"print (y.shape) # a batch of 15 predictions"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "3a60c57c",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Task 3.2: Build a ConvNet for CIFAR-10\n",
|
||
"\n",
|
||
"Your task is to build a decently-sized ConvNet (i.e., $\\geq 4$ layers). Design your ConvNet with the following architecture\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\text{Conv(32, (3,3))} \\rightarrow \\text{MP((2,2))} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\text{Conv(64, (3,3))} \\rightarrow \\text{MP((2,2))} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\text{GAP} \\\\ \\rightarrow \\text{L(64, 256)} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\text{L(256, 128)} \\rightarrow \\text{LReLU(0.1)} \\rightarrow \\text{L(128, 10)}\n",
|
||
"$$\n",
|
||
"\n",
|
||
"where \n",
|
||
"- [`Conv`](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html) is a Convolution layer with the specified output channels and kernel size\n",
|
||
"\n",
|
||
"- [`MP`](https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html) is the Max Pooling layer with the specified kernel size\n",
|
||
"\n",
|
||
"- [`LReLU`](https://pytorch.org/docs/stable/generated/torch.nn.LeakyReLU.html) is Leaky ReLU with the specified negative slope\n",
|
||
"\n",
|
||
"- `GAP` is the Global Average Pooling operation (already implemented for you)\n",
|
||
"\n",
|
||
"- [`L`](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html) is a fully-connected layer with the specified input and output features\n",
|
||
"\n",
|
||
"You are highly encouraged to initialise all your layers in the `__init__` method.\n",
|
||
"\n",
|
||
"---\n",
|
||
"\n",
|
||
"You must use the [`nn.Sequential`](https://pytorch.org/docs/stable/generated/torch.nn.Sequential.html) API to build two parts:\n",
|
||
"1. The `self.conv` attribute must contain all the Convolutional, Pooling, and Activation layers\n",
|
||
"2. The `self.fc` attribute must contain all the fully-connected layers after the flattening\n",
|
||
"\n",
|
||
"The `self.conv` and `self.fc` attributes are already given to you. All you need to do is chain the arbitrary `nn.XYZ` layers together based on the architecture stated above.\n",
|
||
"\n",
|
||
"__Note:__ The flattening is already done for you via Global Average Pooling (GAP) in the `forward` method. Do not add the Softmax activation in the `self.fc` Sequential module.\n",
|
||
"\n",
|
||
"__Reminder:__ Do not hardcode for the number of classes. Use the `classes` argument instead.\n",
|
||
"\n",
|
||
"__Note:__ There is no need to include a Softmax layer in your neural network, as technically, [CrossEntropyLoss](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html) which we are going to use as our loss function later, already applies Softmax implicitly."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 120,
|
||
"id": "3edf5056",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:52:44.926878Z",
|
||
"start_time": "2024-04-07T04:52:44.922955Z"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"class CIFARCNN(nn.Module):\n",
|
||
" def __init__(self, classes):\n",
|
||
" super().__init__()\n",
|
||
" \"\"\"\n",
|
||
" classes: integer that corresponds to the number of classes for CIFAR-10\n",
|
||
" \"\"\"\n",
|
||
" self.conv = nn.Sequential(\n",
|
||
" nn.Conv2d(3, 32, (3, 3)),\n",
|
||
" nn.MaxPool2d((2, 2)),\n",
|
||
" nn.LeakyReLU(0.1),\n",
|
||
" nn.Conv2d(32, 64, (3, 3)),\n",
|
||
" nn.MaxPool2d((2, 2)),\n",
|
||
" nn.LeakyReLU(0.1),\n",
|
||
" )\n",
|
||
"\n",
|
||
" self.fc = nn.Sequential(\n",
|
||
" nn.Linear(64, 256),\n",
|
||
" nn.LeakyReLU(0.1),\n",
|
||
" nn.Linear(256, 128),\n",
|
||
" nn.LeakyReLU(0.1),\n",
|
||
" nn.Linear(128, classes)\n",
|
||
" )\n",
|
||
" \n",
|
||
" def forward(self, x):\n",
|
||
" # YOUR CODE HERE\n",
|
||
" x = self.conv(x)\n",
|
||
" x = x.view(x.shape[0], 64, 6*6).mean(2) # GAP – do not remove this line\n",
|
||
" x = self.fc(x)\n",
|
||
" return x"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "22de4211",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Train your ConvNet on CIFAR-10"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 121,
|
||
"id": "c9ea9d9c",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:54:08.922882Z",
|
||
"start_time": "2024-04-07T04:52:50.295512Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Epoch: 0, Loss: 1.9733156856063687\n",
|
||
"Epoch: 1, Loss: 1.7190505208261788\n",
|
||
"Epoch: 2, Loss: 1.6375392404053828\n",
|
||
"Epoch: 3, Loss: 1.5758821574013557\n",
|
||
"Epoch: 4, Loss: 1.518964537269319\n",
|
||
"Epoch: 5, Loss: 1.4709320745199843\n",
|
||
"Epoch: 6, Loss: 1.4300793857525682\n",
|
||
"Epoch: 7, Loss: 1.3909132538549125\n",
|
||
"Epoch: 8, Loss: 1.3572429686860965\n",
|
||
"Epoch: 9, Loss: 1.3212151067031315\n",
|
||
"CPU times: user 1min 4s, sys: 15.5 s, total: 1min 19s\n",
|
||
"Wall time: 1min 18s\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"%%time\n",
|
||
"# do not remove – nothing to code here\n",
|
||
"# run this cell before moving on\n",
|
||
"\n",
|
||
"cifar10_model, losses = train_model(cifar_train_loader, CIFARCNN(10), device=\"cpu\")\n",
|
||
"cifar10_model_gpu, losses = train_model(cifar_train_loader, CIFARCNN(10), device=\"mps\")\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f1376a81",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Test the CIFAR-10 ConvNet model using the testing data loader"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 131,
|
||
"id": "20bdce79",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:56:22.594761Z",
|
||
"start_time": "2024-04-07T04:56:20.868553Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"cifar accuracy: 0.5123\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# do not remove – nothing to code here\n",
|
||
"# run this cell before moving on\n",
|
||
"\n",
|
||
"with torch.no_grad():\n",
|
||
" cifar10_model.eval()\n",
|
||
" for i, data in enumerate(cifar_test_loader):\n",
|
||
" x, y = data\n",
|
||
" # x = x.to(\"mps\")\n",
|
||
" # y = y.to(\"mps\")\n",
|
||
" pred = cifar10_model(x)\n",
|
||
" acc = get_accuracy(pred, y)\n",
|
||
" print(f\"cifar accuracy: {acc}\")\n",
|
||
" \n",
|
||
"# don't worry if the CIFAR-10 accuracy is low, it's a tough dataset to crack.\n",
|
||
"# as long as you get something shy of 50%, you should be alright!"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "89a05019",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Concept 8: Class Activation Map (CAM) Analysis\n",
|
||
"\n",
|
||
"A __Class Activation Map__ (CAM) is an analysis technique that lets one see through the eyes of the model. It ultimately answers the following question: \"why is the model predicting this label for a given image\". \n",
|
||
"\n",
|
||
"CAM creates a superimposable heatmap that's placed on top of the test image. This heatmap is coloured more strongly for areas the model is focusing on more than others and coloured less strongly for areas the model chooses to ignore.\n",
|
||
"\n",
|
||
"For example, when a picture of a dog (left) is passed through a trained ConvNet, CAM generates a heatmap (center) that's embossed on top of the image (right). The darker (redder) the region of the heatmap, the more the model focuses on that part of the image. \n",
|
||
"\n",
|
||
"<img src=\"imgs/cam.png\" width=1000>\n",
|
||
"\n",
|
||
"You can think of the model looking out for the \"most interesting\" parts of an image when classifying it. In fact, __Saliency__ is the measure of \"interestingness\" in an image. ConvNets look out for the *most salient* features/regions in an image and give out their predictions based on that. "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e321bf5d",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Task 4.1: Building CAM for CIFAR-10\n",
|
||
"\n",
|
||
"Earlier we asked you to divide your CIFAR-10 ConvNet into two parts `self.conv` and `self.fc`. You'll now understand why we did so. \n",
|
||
"\n",
|
||
"Your task is to write the `get_CAM` method that takes in,\n",
|
||
"\n",
|
||
"- `feature_map`: the output of the final activation layer of the CNN. You can get this by running the image through the `self.conv` module i.e., `output = self.conv(x)`\n",
|
||
"\n",
|
||
"- `weight`: the weights of the immediate first Linear layer after the flattening operation\n",
|
||
"\n",
|
||
"- `class_idx`: the label index that your ConvNet outputs/predicts\n",
|
||
"\n",
|
||
"### The CAM Algorithm\n",
|
||
"\n",
|
||
"The first few steps of CAM are already written for you. We'll let you handle the minor implementation details of the rest of the algorithm.\n",
|
||
"\n",
|
||
"1. remove the first dimension of `cam` using `torch.squeeze(...)`\n",
|
||
"2. reshape `cam` to $h \\times w$\n",
|
||
"3. get the difference of `cam` and the minimum elements of `cam`\n",
|
||
"4. divide `cam` by the maximum elements of `cam`\n",
|
||
"5. clip the values of `cam` so they are within the $[0, 1]$ range\n",
|
||
"\n",
|
||
"Refer to the command glossary to find the respective methods."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 133,
|
||
"id": "4e0cde54",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:56:28.220258Z",
|
||
"start_time": "2024-04-07T04:56:28.215689Z"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def get_CAM(feature_map, weight, class_idx):\n",
|
||
" \"\"\"\n",
|
||
" PARAMS\n",
|
||
" feature_map: the output of the final pre-GAP layer in the ConvNet\n",
|
||
" weight: the parameters of the first linear layer post-GAP\n",
|
||
" class_idx: the final prediction label of the ConvNet\n",
|
||
" \n",
|
||
" RETURNS\n",
|
||
" a CAM heatmap of the areas the ConvNet is focusing on more\n",
|
||
" \"\"\"\n",
|
||
" \n",
|
||
" # do not remove these lines\n",
|
||
" size_upsample = (32, 32)\n",
|
||
" bz, nc, h, w = feature_map.shape\n",
|
||
"\n",
|
||
" before_dot = feature_map.reshape((nc, h*w))\n",
|
||
" cam = weight[class_idx].unsqueeze(0) @ before_dot\n",
|
||
" \n",
|
||
" \"\"\"\n",
|
||
" YOUR CODE HERE - perform the steps listed above\n",
|
||
" \"\"\"\n",
|
||
"\n",
|
||
" cam = torch.squeeze(cam)# YOUR CODE HERE ## remove the first dimension of cam using torch.squeeze(...)\n",
|
||
" cam = torch.reshape(cam, (h, w)) # YOUR CODE HERE ## reshape cam to h x w\n",
|
||
" cam = cam - torch.min(cam)# YOUR CODE HERE ## get the difference of cam and the minimum elements of cam\n",
|
||
" cam = cam / torch.max(cam)# YOUR CODE HERE ## divide cam by the maximum elements of cam\n",
|
||
" cam = torch.clip(cam, 0, 1) # YOUR CODE HERE ## clip the values of cam so they are within the [0, 1] range\n",
|
||
" \n",
|
||
" # here, `cam` is the final processed heatmap\n",
|
||
" # we upsample/resize the heatmap to the original image's dimensions\n",
|
||
" # do not remove these lines\n",
|
||
" img = transforms.Resize(size_upsample)(cam.unsqueeze(0))\n",
|
||
" \n",
|
||
" return img.detach().numpy(), cam"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "88a9cf44",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Task 4.2: Visualising CAM Heatmaps\n",
|
||
"\n",
|
||
"Once you run the `plot_cam` method in the cell below, you'll be presented two images: the original test image `x`, the raw heatmap and the image with the heatmap on it. \n",
|
||
"\n",
|
||
"Take a screenshot of all three plots and post it on Coursemology. Then, explain what you think the ConvNet was looking at that convinced the model to predict that class. Talk about this in terms of the \"saliency\" of the image. \n",
|
||
"\n",
|
||
"__Note:__ This is an open-ended question that tests your understanding of saliency and what features ConvNets rely on when predicting a class label. Also, the resolution of images from CIFAR-10 isn't fantastic, try your best to identify discerning features of the image!"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 134,
|
||
"id": "8f397744",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:56:31.521488Z",
|
||
"start_time": "2024-04-07T04:56:31.517416Z"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# do not remove this cell\n",
|
||
"# run this cell before moving on\n",
|
||
"\n",
|
||
"cifar10_classes = [\n",
|
||
" \"airplane\",\n",
|
||
" \"automobile\",\n",
|
||
" \"bird\",\n",
|
||
" \"cat\",\n",
|
||
" \"deer\",\n",
|
||
" \"dog\",\n",
|
||
" \"frog\",\n",
|
||
" \"horse\",\n",
|
||
" \"ship\",\n",
|
||
" \"truck\",\n",
|
||
"]\n",
|
||
"\n",
|
||
"def plot_cam(img, cam):\n",
|
||
" ''' Visualization function '''\n",
|
||
" img = img.permute(1, 2, 0)\n",
|
||
" fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(10,7))\n",
|
||
" ax1.imshow(img)\n",
|
||
" ax1.set_title(f\"Input image\\nLabel: {cifar10_classes[y]}\")\n",
|
||
"\n",
|
||
" ax2.imshow(cam.reshape(32, 32), cmap=\"jet\")\n",
|
||
" ax2.set_title(\"Raw CAM.\")\n",
|
||
"\n",
|
||
" ax3.imshow(img)\n",
|
||
" ax3.imshow(cam.reshape(32, 32), cmap=\"jet\", alpha=0.2)\n",
|
||
" ax3.set_title(f\"Overlayed CAM.\\nPrediction: {cifar10_classes[idx[0]]}\")\n",
|
||
" plt.show()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 139,
|
||
"id": "463cced4",
|
||
"metadata": {
|
||
"ExecuteTime": {
|
||
"end_time": "2024-04-07T04:57:02.907611Z",
|
||
"start_time": "2024-04-07T04:57:02.758721Z"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"true class: horse\n",
|
||
"predicated class: horse\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAzQAAAEzCAYAAADn6EbmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAACZsklEQVR4nO2dd5gUVdbG36rqPDMwM2QHJTNKFFRAQECUFREUURd2Ma+KgvqJioJrVsSwKCKgGFgTiopZEcO6LiqogIqIoiSJEocJPR0rfH90mO6ZPqeHmSE0nN/z9FNVfSrculV17j11b71XsSzLgiAIgiAIgiAIQgaiHuwECIIgCIIgCIIg1BQJaARBEARBEARByFgkoBEEQRAEQRAEIWORgEYQBEEQBEEQhIxFAhpBEARBEARBEDIWCWgEQRAEQRAEQchYJKARBEEQBEEQBCFjkYBGEARBEARBEISMRQIaYZ+RsVgFQRAEQRCEQwUJaDKIJ554AoWFhQft+Nu3b8eYMWOwdetWcp1vv/0WhYWF+Pbbbw9gygRBEARBqA7Lly/Hddddhz59+qBz58447bTTcPvtt2PdunUHNB0XXXQRLrroogN6TI6JEydi4MCB1Vr3448/xj/+8Q/07t0bxx9/PIYOHYqZM2fC6/WS24wePRqFhYVYsGBBSvtFF12EwsJCjBo1itzH+PHjUVhYiIkTJ1YrnUcSEtAI1Wbx4sX44osv2HU6duyI1157DR07djwwiRIEQRAEoVo8/fTTGD16NHw+HyZNmoTnnnsOV199NX755Rece+65+PDDDw92Eg9pTNPETTfdhBtvvBEFBQW499578eSTT2Lo0KF4/vnnceGFF6KkpKTKdhs3bsSyZcvQvn17vPrqq+T+VVXFjz/+iD///LOKze/3p62DHclIQCPUKdnZ2Tj++OORnZ19sJMiCIIgCEKU//73v5g6dSquueYaPPfccxg6dCh69OiBCy64AK+99hoGDBiAiRMnYs2aNQc7qYcszz77LD744ANMmzYN9957L04//XScfPLJuPrqq/HMM8/gt99+wxNPPFFluzfffBNNmzbF2LFj8d1335GtYR06dIDT6cTChQur2D7//HM4nU40adKkzs/rcEACmgzmrbfeQocOHbBixQqMHDkSnTt3xoABA/DMM8/E19myZQsKCwvx4Ycf4uqrr0bXrl3Rv39/PPHEEzBNM77ewIEDqzRhvvXWWygsLMSWLVvw1ltvYdKkSQCA0047jWzurNzl7IknnsDgwYPx2WefYejQoejcuTPOOecc/PDDD/jxxx9xwQUXoEuXLhg6dCiWLFmStK/PPvsMf//739GtWzd06tQJgwcPxssvv5y0zrp163DllVeie/fu6N27Nx577DFMmjQpqRnbNE08/fTTGDRoEDp16oQzzjgDL730Ug1yXBAEQRAykxkzZqBVq1a4/vrrq9jsdjvuueceaJoWr0NcfvnlGD58eJV1b7jhBpx11lnx5WXLluHCCy9E165d0aNHD9x6660oKiqK22N1lTfeeAN9+/ZFv379UgZNRUVFuOeee3DqqaeiU6dO6NGjB8aNG4ctW7YAAObOnYvCwkJs2LAhabsPP/wQxx57bHy9bdu24cYbb0SPHj3QtWtXXHLJJfjll1+StikpKcGkSZPQs2dPnHTSSXjkkUeS6kSpCIfDmDNnDvr164dBgwZVsR9//PG44YYb0K5du6T/DcPAO++8gwEDBmDgwIHIycnBa6+9lvIYHo8H/fv3x0cffVTFtmDBAgwePBg2m41N55GKBDQZjmmauOGGGzBkyBA8/fTTOOGEE/Cvf/0LX375ZdJ6d999N7Kzs/HEE09g+PDhmDVrFh5++OFqH2fAgAG45pprAESc4tixY6u97fbt2zFlyhRcffXVmDZtGkpKSnD99dfjxhtvxF//+lc8+uijME0T48ePRyAQAAB88cUXGDduHDp27IhZs2bhiSeeQEFBAe677z58//33ACLO78ILL8Sff/6JKVOm4Pbbb8fChQvxwQcfVDn36dOn4+yzz8ZTTz2FwYMH44EHHsDMmTOrfQ6CIAiCkKkUFRXh559/xqmnngpFUVKuk5eXh969e+M///kPAOCcc87Br7/+ivXr18fXKS8vx3//+1+cc845AIClS5fi0ksvhcvlwrRp03Dbbbfhu+++w8UXXxwvz4FIpf6pp57C/fffjxtuuAFt27ZNOrZlWRgzZgy+/vpr3HTTTXjuuecwduxYLF68GHfeeScAYNiwYXA6nXj33XeTtn377bfRo0cPNG/eHEVFRRg1ahRWrVqFO+64A1OnToVpmhg9enS8VcQ0TVxxxRX44osvcPPNN+Ohhx7CDz/8QH7bEmPVqlXYu3cvTj31VHKdMWPGYOTIkUn/ffXVV9ixYwfOPfdcOJ1ODBkyBO+8805S/iQyZMgQrFixAtu2bYv/5/V6sWjRIgwdOpRN45GMhHkZjmVZGDt2LC644AIAwAknnIBPP/0UX3zxBU455ZT4eh06dMC//vUvAEC/fv3g8/nw8ssvY+zYsahXr17a4+Tn5+OYY44BABx33HFo3rx5tdPo9/tx1113oV+/fgAirSpTp07F5MmTcf755wOIOLvrr78eGzZswHHHHYe1a9di+PDh+Oc//xnfT7du3dCzZ08sXboU3bt3x0svvYTy8nK888478SbYrl274owzzohvs2HDBrz++uu48cYbcdVVVwEA+vbtC0VRMHv2bPz9739HXl5etc9FEARBEDKNmJhPurK7RYsW+M9//oOSkhIMGjQIHo8HCxYswLXXXgsA+PTTTxEMBjFs2DAAwNSpU9GqVSvMnj0bmqYBiJTDZ511Ft58802MHj06vu+rr74aAwYMSHncnTt3wu1249Zbb8WJJ54IAOjZsye2bNmCefPmAQDq1auHQYMG4b333sP//d//QVEU7Ny5E4sXL8YDDzwAAHjhhRdQXFyMV199FQUFBQAidZ4hQ4bg8ccfx/Tp07Fo0SL89NNPmD17djw9vXr1SisIsH379mrlYWXefPNNtG7dGscffzwA4Pzzz8drr72Gjz76COeee26V9QcMGACPx4OFCxfi8ssvBxDJ9/z8fJxwwgn7dOwjCWmhOQzo1q1bfN7hcCA/Px8+ny9pnbPPPjtp+YwzzkA4HMaPP/54IJKI7t27x+cbNmwIAPGHGwByc3MBAKWlpQCAK664Ag899BB8Ph9Wr16Njz76CE8//TSASLMvAHzzzTfo1q1bUn/SgoKCpPz45ptvYFkWBg4cCF3X47+BAwciGAxi+fLl++V8BUEQBOFQITbcgt1uZ9eLBSWWZcHj8WDQoEFJLRcffvghevTogWbNmsHv92PFihXo378/LMuKl69HH3002rRpg6+//jpp3+3btyeP26RJE7z44os48cQTsW3bNixZsgQvv/wyvv/++3iZD0SCga1bt2LZsmUAgHfffRculyv+InPJkiU47rjj0KRJk3h6VFVFv379sHjxYgCRLnJ2uz3+khWo6OrFoaqRKnO6rmmJ7N27F59//jnOPPNMlJaWorS0FC1btkSrVq3igVplXC4XBg4cmNTt7MMPP8SQIUPI1jVBWmgOC1wuV9KyqqpVxopp3Lhx0nJ+fj6AigBif5NKJKByuhMpKirCXXfdhc8++wyKoqBFixbxNxOxcysqKkqpptaoUSPs2rULAFBcXAwASf19E9mxY8c+nYcgCIIgZBqx1gpu2AUA2Lx5MzweT/wl4/Dhw/Huu+9i9erVaNy4MRYvXox7770XQKT+YJomnnnmmaRvd2M4nc6k5QYNGrDHfu+99/Doo4/izz//RG5uLo499tgq9YRevXqhefPmeOedd3DSSSfhnXfewZlnngm32w0gUuZv3LiRVFr1+/0oKSlBbm5uPECJ0ahRIzZ91cnDoqIiZGVlxc/93XffRTgcxsyZM1N2c1+9ejWOPfbYKv+feeaZ8e+HsrKysGTJEtxwww1s+o50JKA5QohV7GPs2bMHQLKDMQwjaZ3KrTwHkptvvhnr1q3Dv//9b3Tv3h0OhwN+vx9vvPFGfJ2mTZvGzyORxP9i3eleeOEFZGVlVVn3qKOO2g+pFwRBEIRDhwYNGuD444/HJ598ghtuuCHlm36v14uvv/4ap512Wvy/Xr16oUmTJvjoo4/QpEkT2Gy2eGtIVlYWFEXBpZdemvKlYSzIqA7Lli3DrbfeigsvvBD/+Mc/0LRpUwDAww8/nNSTQlEUnHvuuXjxxRcxevRorF27Nh5gAUBOTg569OiBW265JeVxHA4H8vLysHfvXhiGEW+RAqrWkypz3HHHoWHDhli0aFFSV7pE7r77bnzzzTdYtGgRXC4X3nrrLXTt2hU33XRT0nqBQADXXHMNXn31Vdxzzz1V9tOvXz/k5OTg448/Rk5ODpo3b45OnTqx6TvSkS5nRwiff/550vLHH38Mt9uNrl27Aoi0oMT6h8aIfXwfo/LbjP3J8uXLccYZZ6BXr15wOBwAgEWLFgGoaO496aST8MMPP8RbYwBg165dSd3oTjrpJACRZt/OnTvHf8XFxZg2bVpaByYIgiAIhwPXXnst1q9fj2nTplWxGYaBu+66C4FAAFdccUX8f1VVMXToUPznP//BwoULcdppp8V7XGRnZ6NDhw5Yv359Uvnarl07zJgxY58G2P7hhx9gmiauv/76eDBjGEa8m1hiN6/zzjsPZWVlmDJlClq2bJn0XUmPHj2wYcMGtGrVKilN7733Ht544w1omoaTTz4Zuq7js88+i28XCoWqdJGrjKqquPTSS/HFF1/EhRMSWbp0KT7//HOcccYZcLlcWLlyJX777TeMGDECPXv2TPr1798fffv2xfvvv4/y8vIq+3I4HDjttNPwySef4KOPPiJ7mQgVSAvNEcLChQvRsGFD9O/fH9999x3mzp2L8ePHw+PxAABOPfVUzJ49G0899RSOP/54fPHFF1VklGOtHZ9++in69euHNm3a7Lf0dunSBe+//z46duyIpk2b4ocffsDs2bOhKAr8fj8A4OKLL8bcuXPxj3/8A+PGjQMAzJw5E6FQKP72qX379jj77LNxxx13YOvWrejUqRM2bNiAxx57DM2bN0fLli332zkIgiAIwqHCKaecgokTJ+Lhhx+OD6TZuHFjbNmyBa+++ip+/fVXTJ48uUoXqOHDh+O5556Dpml48sknk2wxwZ2bbroJZ599NgzDwJw5c7BixYq4Mmp16NKlCwDg3nvvxXnnnYfS0lK8/PLLWL16NYBIj5FYINWsWTP07t0bX331FcaPH5+0n0svvRTvvvsuLr30Ulx++eXIy8vDggUL8Prrr8eHnjj55JPRt29f3H777dizZw8KCgrw4osvoqioKG23uEsvvRRLly7F9ddfjwsuuAADBgyAqqpYtmwZXnrpJbRr1w633norgIgYgN1uTxIqqpyv//vf//D+++9j1KhRVexDhgzBmDFjoKoqbr/9djJNmzZtQlFRUdJ3yUciEtAcIfzf//0fvvvuO7z22mto1qwZ7rzzTvztb3+L28eMGYOioiLMmTMH4XAYAwYMwOTJk5McUs+ePdG7d29MnToVS5YsiX+kvz948MEHcd999+G+++4DALRs2RL33HMP3nvvvfjHgPXq1cOLL76IyZMn45ZbbkFWVhb+/ve/w+PxxAM1AJgyZQpmz56NefPmYfv27WjQoAGGDBmCG264Iam5WRAEQRAOZy677DJ069YNL7zwAh566CEUFRWhUaNG6NOnDyZPnlxFThmIvBg87rjjsGPHDvTp0yfJ1rdvXzz33HOYMWMGrr/+etjtdnTs2BH//ve/96mC3bNnT9x5553497//HX8B27NnT8yYMQPjxo3D8uXLkz7aP/XUU7F48eIq4+Q0adIE8+bNw9SpU3H33XcjGAyiZcuWSaqqQGT4iX/961+YPn06gsEghgwZgr/+9a8pW14SsdvtmDVrFl577TW8++67+OijjxAKhdC8eXOMGTMGF110EbKyshAMBvHhhx+iT58+pJLq6aefjnr16mHevHkpA5revXujXr16aNasGfsCedasWXj77bfx22+/sWk/3FGsyl+PC4cVW7ZswWmnnYYpU6ZgxIgRBzs5dcqKFStQXFyc5OR0XceAAQNw1llnxd/GCIIgCIJw+HDllVdC0zQ89dRTBzspwiGCtNAIGcu2bdswfvx4jBs3Dj169IDf78e8efNQVlaGv/71rwc7eYIgCIIg1CEzZ87Ehg0bsGjRIrz88ssHOznCIYQENELGcuaZZ6K4uBivvPIKnnvuOdjtdnTt2hUvv/zyfv2+RxAEQRCEA8/nn3+OjRs3YsKECXHRH0EApMuZIAiCIAiCIAgZjMg2C4IgCIIgCIc08v5d4JCA5hDnoosuwkUXXVTr/bz11lsoLCzEli1bar2vgQMHYuLEift9G0EQ9i8XXXQRCgsLk37HHnssTjjhBFxwwQX48MMPD2r6lixZgmuvvRannHIKunbtijPOOAMPPvggdu/eTW5zyy23oLCwkFRhnDhxIgoLC9GvXz+ygvSvf/0LhYWFdeJ7BeFQINWz3qlTJwwYMAD33HMPSkpK9tuxK9c/nnjiCRQWFlZ7++3bt2PMmDHYunVr/L+DWaeoy/qUUHfINzSCIAhHMB06dMBdd90VXzYMA9u3b8fzzz+PG2+8ETk5OejXr98BT9fUqVPxzDPPYPDgwfjnP/+J3Nxc/P7773jmmWfwySef4KWXXkJBQUHSNl6vF5988gnat2+P119/HVdeeWXKEdFVVcWOHTuwfPlynHjiiVXsH3300X47L0E4WFR+1sPhMFatWoVHH30Uv/76K1599dWUz0tdc8EFF+CUU06p9vqLFy/GF198gTvuuCP+34wZM+Lj0ggCIAGNIAjCEU12dnbK8SL69++Pk08+GW+++eYBD2gWLFiAp59+GpMmTcKll14a/79Xr14YMGAAhg8fjvvuu6+KZOuHH34IwzBw++234+KLL8ZXX32VsuLUrFkzWJaFhQsXVglofvzxR2zfvh3t27ffL+cmCAeLVM/6SSedhPLyckyfPh0rVqw4IIMzNm3aFE2bNq3VPjp06FBHqREOF6TL2WHCG2+8gREjRuD4449Hly5dcM4552DBggVV1vv+++8xfPhwdO7cGcOGDauyTjAYxMMPP4z+/fujU6dOKdepzEUXXYSBAwemTWM4HMbDDz+MPn364Pjjj8fll1+OjRs3Jq3z9ddf4+9//ztOOOEE9OzZEzfddBP+/PPPuP2tt95Chw4d8MYbb6Bv377o168f1qxZg82bN+Oaa65Bz5490bVrV4wcORL/+9//kvb9+++/Y8yYMejevTu6d++OcePGYfPmzWnTLQhHIg6HA3a7Pem/QCCAqVOn4i9/+Qs6deqE7t2747LLLsOvv/4KAHjggQfQo0cPmKYZ3+aOO+5AYWEh1q9fH//vlVdeQZcuXeD3+1Mee/bs2Wjbti0uueSSKrZjjjkGt9xyC0444YSk4wCRkbl79uyJnj17olWrVpg3bx55foMHD8bHH39cZR8LFixA7969kZubS24rCIcTnTp1AhAZCgGIlOk333wzrr/+enTv3h1XXXUVgOrVD0zTxKxZszBgwAB07doVY8eOrdKdLVWXsw8//BAjRoxA165dMWDAADzyyCMIhUJ466234mPKnXbaafFuZpW7nJWVlWHKlCk4/fTT0blzZwwdOhTz589POsbAgQMxffp0PPTQQ+jduze6dOmCf/zjH9iwYUN8nW+//RaFhYV466230ubbihUrMGrUKHTu3BkDBgzAc889l2SvbpoeeOABXHLJJejevTvuvPNOAMBLL72EwYMHo3PnzjjllFNw9913w+v1JuXz008/jUGDBqFTp04444wz8NJLL6VN8+GMBDSHAXPnzsWdd96J0047DbNnz8YjjzwCu92OCRMmxB1UjDvuuAODBw/GzJkz0bZtW4wfPx5fffUVgMgHd+PGjcO8efNw2WWX4cknn0S3bt0wfvx4vPPOO+Tx77rrLsyYMSNtOhcsWIA1a9bgwQcfxJ133omVK1di/Pjxcfu7776Lyy+/HE2aNMGjjz6KSZMm4YcffsDIkSOxZ8+e+HqGYeCpp57C/fffjxtuuAGtW7fGmDFj4PP58PDDD2PWrFnIzc3F2LFj4wHThg0bMGrUKOzZswcPPvggJk+ejM2bN+Nvf/tb0r4F4UjDsizouh7/BYNBbNy4EbfffjvKy8txzjnnxNe95ZZbMH/+fFx11VWYM2cOJk6ciN9//x3jx4+HZVk49dRTUVJSgp9//jm+zTfffAMAWLp0afy/RYsW4eSTT4bb7a6Snl27dmH16tUYMGAA2f1l1KhRuPLKK6GqFUXYunXrsGLFCpx77rkAgBEjRuC///0vduzYkXIfQ4YMwc6dO7F8+fL4f6ZpYuHChTjrrLOqk3WCcFgQq9AfffTR8f8++ugj2O12zJw5ExdffHG16wePPPIIZs6cifPOOw8zZsxAXl4epk6dyh5/3rx5uPHGG3HcccdhxowZGDNmDF555RXcfffdGDBgAK655hoAkW5mY8eOrbJ9IBDA3//+d7z33nu4/PLLMWvWLJxwwgn45z//WaUV98UXX8T69esxZcoU3H///fj555+TAqOOHTvitddew4ABA9Lm2913342hQ4di9uzZ6NKlCx5++GH897//3ec0zZ07F4WFhXjiiSdwzjnn4MMPP8RDDz2E0aNH47nnnsO4cePw7rvv4v7770869vTp03H22WfjqaeewuDBg/HAAw9g5syZadN9uCJdzg4DNm/ejMsvvxzjxo2L/9e8eXOMGDEC33//PY466qj4/+PGjYu/benXrx/++OMPzJgxA3379sXixYvx5Zdf4rHHHsOQIUMAAKeccgr8fj/+9a9/YejQobDZqt4ybdu2rVY6mzRpglmzZsXf+m7cuBFPPfUUvF4vPB4PHnnkEfTu3RuPPfZYfJvu3btjyJAhmDNnDiZMmBD//+qrr447nF27dmHdunW4+uqr0b9/fwBAly5dMGPGDASDQQARR+hyufD888/H+92efPLJOP300/Hss8/i1ltvrdY5CMLhxtKlS9GxY8ek/xRFQfv27fH444/HW19DoRDKy8txxx13xP1Djx49UF5ejgcffBC7du3CiSeeiKysLCxZsgRdunTB1q1bsWnTJnTs2BHfffcdRo4ciVAohG+//ZZ85rZv3w4g4sP2hfnz56NevXo4/fTTAQDDhw/HtGnT8MYbb+Daa6+tsn7nzp1xzDHHYOHChfHxLJYtW4bi4mKcfvrpePPNN/fp+IJwqBN7eRGjpKQE3333HZ588kkcf/zx8ZYaIPKd2X333QePxwMg0nsiXf3A5/PhpZdewsUXX4zrrrsuvs6OHTvw5ZdfpkyTaZp44oknMGjQIEyePDn+fzAYxNtvv43s7Gwcc8wxAIDjjjsupV9466238Pvvv+OVV17BCSecED+uruuYNWsWRo0aFW9xrVevHmbNmgVN0wAAmzZtwhNPPIG9e/ciLy+P7IKbihtvvBF/+9vfAADHH388Pv/8c3zzzTc49dRT9ylNjRs3xsSJE+MvaO68804UFBRg9OjRUFUVPXr0gMfjwd69ewFEAtDXX38dN954Y7w+17dvXyiKgtmzZ+Pvf/878vLyqnUOhxPSQnMYMHHiREyYMAFlZWVYuXIl3n//fcydOxdApJtXImeeeWbS8umnn44ff/wR5eXlWLJkCRRFQf/+/ZPe2A4cOBC7du3CmjVrapXOLl26JHVhib0NKi0txYYNG7Br1y4MGzYsaZtjjjkG3bp1w7fffpv0f2L/9oYNG6Jt27a44447MHHiRCxYsACWZWHSpEnx9b755hv07NkTLpcrfl7Z2dk48cQTsXjx4lqdlyBkMh07dsT8+fMxf/58zJw5E+3bt0fLli3x2GOPYfDgwfH1HA4HnnvuuXjLxtKlS/Haa6/F30iGw2HY7Xb06dMn/kwtWbIELVq0wJlnnonvvvsOQCSA8vl8OPXUU1OmJ1aoV+4KxqHrOt577z2cfvrpCAaDKC0thcvlQs+ePfHGG2/AMIyU25155plJ3c4+/PBDDBgwQD42Fg5LYi8vYr/evXvjxhtvRMeOHfHoo48mtYg2b948HswAqFb94Mcff0Q4HMZpp52WdNzK9Y5ENmzYgN27d8dfRMS49NJL8e6778LhcKQ9r++++w4FBQXxwCHG2WefjWAwiBUrVsT/69y5czyYARD/lofq/sqR+P2dx+NBw4YNUVpaus9patOmTVJrc69evfDHH39gxIgRmDVrFn755RcMGzYs3gX3m2++gWVZGDhwYJVrEQwGk1qdjySkheYwYNOmTbjzzjvxzTffwGazoXXr1vH+qZVlSRs1apS03KBBA1iWBa/Xi+LiYliWhe7du6c8zs6dO3HcccfVOJ2JzhFIrrgUFxcDiAQnlWnYsCF++eWXKumOoSgK5syZgyeffBKffvop3n77bdjtdpx++um4++67kZubi+LiYixYsCDl90D5+fk1PidByHSysrLQuXNnAJHCvlu3bjjnnHNw+eWX4+233056Pr788ks88MADWL9+PbKyslBYWIisrCwAFb6mf//+uOeeexAIBLBkyZL4Ny3/+te/8Mcff2DRokXo2LEjmjRpkjI9zZo1g6IoSRKtlSktLYWmafFjf/HFF9i9ezfeeuutlH3f//vf/1apMAGRbmezZ8/GsmXLcMIJJ+CTTz7B3XffXb2ME4QMo2PHjrjnnnsARMpNp9OJZs2apQzgK5fF1akfxCrzlcvUyvWOyvsFksv0faWkpISsOwCIpwtAlW6uNXmBwu0r5gf3JU2V1xsyZAhM08Qrr7yCGTNm4PHHH0dBQQFuuukmnHXWWfE8o7rGUt1sD3ckoMlwTNPEVVddBbvdjtdffx0dOnSAzWbD2rVr8d5771VZv6SkBC6XK768e/duaJqG+vXrIycnBx6PBy+++GLKY7Vo0WK/nUes6TXV+BK7du1K23zapEkT3H333bjrrruwevVqLFy4EM888wzq16+Pe+65Bzk5Oejduzcuu+yyKtum6kYnCEcqDRo0wJ133onrrrsOkydPjvd/37RpE8aNGxf/Vi/WDWTu3LlJ3Un69++PcDiM5cuXx7uWdezYEdnZ2Vi6dCkWLVoU77KSivz8fHTs2BFffvklJkyYkPI7mieffBIvvfQSPv30UzRr1gzz589HQUEBpkyZUmXd66+/HvPmzUsZ0Bx77LFo1aoVFi5ciHA4jGAwWK2+84KQiSS+vNhXqlM/+OmnnwAAe/bsQevWreO2WAU8FfXq1QMAFBUVJf1fXFyMVatWVav7V/369asIDAGRugOAg9L9qrZpGjp0KIYOHYqysjJ89dVXeOaZZzBhwgSceOKJ8Tx74YUX4i91Ekn8zOBIQrqcZTh79+7Fhg0bcP7556NLly7xyvmiRYsAVH3rkFjxiH0A27VrV7hcLvTo0QM+nw+WZaFz587x35o1azBz5sykvrd1TatWrdCoUSO8//77Sf9v3rwZP/74I/lWCAB++OEH9O7dGz/99BMURcFxxx2H8ePHo3379vH++D169MDatWtx3HHHxc+rU6dOeP755/Hpp5/ut/MShEzkL3/5C0455RR88MEH8e6eP//8M4LBIMaMGRMPZoAKnxJ7M9moUSN06NABr776Knbt2oUePXpA0zScdNJJePvtt7F+/Xqyu1mMf/zjH/j9999TqvasX78eb7zxBnr06IFmzZph9+7d+PLLL3HWWWfFW4MSf0OGDMHXX39NKhoOGTIEn376KT788EMMGjQITqezRnkmCIcz1akfdOvWDS6XCwsXLkzaNtYtNRWtW7dGXl4e/vOf/yT9//777+PKK69EMBhM6o6VipNOOglbt26t0tXqvffeg91uR5cuXfbxbGtPbdJ0ww03xL/7y8nJwZlnnomxY8fCMAzs3Lkz/s3f3r17k65FcXExpk2bxgaQhzPyajoDiA1yV5m2bduib9++KCgowNy5c9G0aVPUq1cPX331FV544QUAVfuFTps2DYZhoFmzZnj11VexYcMG/Pvf/wYQebN60kknYezYsRg7dizatGmDn376CU888QT69u1Lds1au3YtQqFQrXThVVXFjTfeiEmTJmH8+PEYPnw49u7dixkzZqB+/fopW1ZidOjQAS6XC7fccguuu+46NGzYEIsXL8avv/6Kiy++GAAwduxYjBo1CmPGjMHf/vY3OJ1OvPbaa/jss88wffr0GqdbEA5XbrvtNpx99tm4//778fbbb6Njx46w2Wx45JFHcPnll8clVb/44gsAgM/ni287YMAAzJw5E61atYp3LevZsycefPBBNG7cuIoIQWWGDBmCxYsXY/LkyVixYgUGDx6MrKwsrFy5EnPmzEG9evXirTFvv/02dF0nu1+ce+65eOWVV/D666/jpptuSnmsmTNn4r333sOsWbPING3fvh3bt29Hhw4dqtWvXxAOJ6pbPxg7diymTZsGt9uNXr164X//+x8b0Giahuuuuw733nsv7r77bgwaNAh//PEHpk2bhr/97W/Iz8+Pt0h8+umn6NevH9q0aZO0jxEjRuCVV17Btddei+uvvx5HH300Pv/8c7z55pu49tpr49tXB6/Xi7Vr1+KYY46pVXf02qSpV69euOuuu/DQQw+hX79+KC0txYwZM9CyZUsce+yxsNvtOPvss3HHHXdg69at6NSpEzZs2IDHHnsMzZs3R8uWLWuc7kxGApoMYNOmTSm7Upx77rno27cvZs2ahcmTJ2PixIlwOBxo27YtnnzySTzwwANYtmwZLrroovg2kydPxsMPP4yNGzeiffv2eOaZZ9CjRw8AkaDi6aefxuOPP47Zs2djz549aNKkCS699NIkBbXK3HPPPdi6dSs+//zzWp3niBEjkJWVhdmzZ2PcuHHIzs7GKaecghtvvJHtg+t0OjFnzhxMnToVkydPRmlpKVq2bIl7770XI0aMABDpWjJ37lw89thjuOWWW2BZFtq3b4+ZM2dW+YBREITIm9OLLroIc+bMwcsvv4xLL70UU6dOxYwZM3DNNdegfv36OP744/HSSy/hoosuwrJly+Lf7sUCmphvASIBTcxWndHI77//fvTs2ROvv/467rrrLni9XhQUFOC8887DFVdcEa9svP3222jXrh2OPfbYlPvp0qULWrdujTfffDOuvJRI27Zt0b59e+zatQu9e/cm0/PGG29gxowZ+M9//rPPCmyCkOlUt34wZswYeDwevPDCC3jhhRfQrVs33Hrrrey3aaNHj4bH48Fzzz2H+fPno0mTJrj88svjCl49e/ZE7969MXXqVCxZsgRPP/100vZutxsvvfQSpk6diunTp8Pr9aJ169aYPHkyzj///H06z1WrVuHiiy/GlClT4vWHmlCbNI0aNQrhcBjz5s3DK6+8ApfLhZNPPhkTJkyICytNmTIFs2fPxrx587B9+3Y0aNAAQ4YMwQ033JAkenAkoViVvxoXBEEQBEEQBEHIEOQbGkEQBEEQBEEQMhYJaARBEARBEARByFgkoBEEQRAEQRAEIWORgEYQBEEQBEEQhIxFAhpBEARBEARBEDIWCWgEQRAEQRAEQchYJKARBEEQBEEQBCFjqdOAZs+ePRg7dixOPPFE9OzZE5MnT4au63V5CEEQDnPEjwiCUBvEhwjCkYetLnd2ww03oEmTJvjyyy+xe/duXHPNNXj++edxxRVXVHsfdzx8E4KhAJwOF+67ZWp82W53kdu43PSoqDYHvZ2m8vFcbraHtOXl0LZmuQ1IW7ZiJ22maZC2MrNiXtNs6HvKefjqyzdhGDq2+/zkdt5gkLQBQJgZVzWs0Plq2hy0jblW+Rqd5+1z6TzNVcKkLWRW7FNVbeh84hCsXLYAphkpwIp99DkGQI9YzmyGUn8ZafPr5aTNYEZIt2l0nqrpxr9VA6SpqdMV3b8dZ/YfjY/+Nxe6EcnPcOne1GmxOXD20HEpbfubuvAjuKggMn1pa2SeuV7Vooix7eY39e2ibX/Stwq28bslacTYGic+0jk5yN+0FUXHFABlZcjPZzbMS3PQxjVMUFPG1qyGNm6fXJ020VXacoC/bAU+KQD0sqr2yoQYG/1oAvUYWxvaFG5B2zbVLyBtW0DbAGA7m7ERbHDhPDyDN3EldPbkKtY90NSFD3nynssQCvrhcLpxzV3/ji/bmLLP4aKrVDam7FdNvi6SZaPLVLfbSdrqZdcnbR4uPRZdFwknFEWq3YETLx6PZS8+BjMcQnGYfhB8FveQAGGVLuMMB10XsVxMnYqx1XPTeX5MHp3fOYwT0fWEuojmQNte12DtN0/CNCLnXuanzzFo0XWDgEmaUB720dvZ6Lqh6aSPZ9rdpC1s0XkK8NexgaPi2bGpDgw94Wp8sPwp6GYIujd1GW3THBjebwJ7zPi61VqrGmzcuBHfffcdFi1aBLfbjaOPPhpjx47FI488sk9OJBgKIBAMVFk2mfqcotGnYTIVyHQBTThM71fXaZth0JVvLj2xCnjKfaa4oQ1Dh2GEEdbp43E2AAixAQ39FJlMIGAygVAYtM0w6PM3VcZmVL2OpqnDjO7PMOhz1Jnz0Jl7jsvXEGPjAhqLcWjpAxr6mLotOc91IxwPaHSdL2QONHXlR5ICGH9Z7QMauswAvGm2ZQ5tMgENXaXgYco+WAm3QvxuKyuDVVYG0PU03gbw+UOXqTUPEji3xmUcZ0vlYvSyioCGC4a49HA2bp/chWQ3o29Ig71QQJi9WMnoCOzT+geKuvIhoaAfoaC/yrJp0DeRotD1Akuht0sX0OjMvaDTRSoMJsAwFaZMYQKaVMWpGQ7BCIegh+kHWjf5skbXmHKaqVNYGlNPsTH71Ok8T1WniB+PeWhT1kWMUDygMZm6iMGU/8wtB11n8pxxsIaNqcOpTB0uTV2Es+spzkM3Q9CNEHSDf/leHeosoFmzZg1yc3PRpEmT+H9t2rTBtm3bUFpainr1uFdRFTijLSqVp1wLjZOJ3m12+u1FuoDGbqdLce4tjaYxbz4Yhwcu+EowadEALja12+gHzJ4qEkqEufkUtoWGeSvC2OxMC43GBKYq43xVJLfQJE4j+6W3tTEBDeMLYWfO0QHaxrfQcG/M0gU0tMeL7bfyFAAs4j7m7u/9SV35EbhzUs/XFLrxEMhOsy1T5+NcEFNPYeG8mlKphSY2VQD+PLLSHJTLH/plH0C7Zz6I4l4SchnHPUaJ7seWkzwF+GCIc7PcdlwJXMMO4SpzITX2QgF29mJFsMGVNK3OugeSuvIhDqc75ZTzjXbn/mmhsTEtNDYHU8dh6jBqDVtotEotNIlT7na2MRV2ALCYN/uKnQlo7DWri9hsdJ6rTG8JhXkwVSu5habyvlSmLqIx+aMxWWez6PqfzUY7JpXZqcmcP9K00IC5jom9UGyqI2kKLfV9zPVcqbJutddMQ3l5OdzuZGcYW/b5fNV2IvfdMpVdFiroe8p5BzsJhyydTxxysJNwyHJm/9EHOwkkdeVH8NLW1PMHAa4KyfQqYm11Sf6mg5s/hzR/OTTzhqtSHKh76mB0JasOdeVDrrnr3+yyUMGJF48/2Ek4ZGnb65qDnYRDmqEnXF1n+6qzgMbj8cDvT34VGVvOykr3eq8C+YYmNfINjXxDw3KYfENTV35EvqFJjXxDw9jkG5rD4huauvIh8g1NauQbGvmGhuOw+IamXbt2KC4uxu7du9GwYUMAwLp169C0aVPk5FS/y4d8Q0PsU76hoW3yDc1h8w1NXfkR+YYmNfINDWOTb2gOi29o6sqHyDc0xD7lG5p92k6+oUlIT6Z8Q9OyZUuccMIJeOCBB3Dvvfdi7969mDVrFs4///x92k92dg7sDgec0X6hsWWXi36z4sli+rS66Wiyvofv39s8l3Z+jbLofspO5mYwAvRDHQrRD6YrYZ+xRg6XGnkICnLp13zpbpISJj0BhWmFcdARvMK8TWrsot8m5brofHOa9JtAw1dRY4j1YdUsFUp0vh7z1szFOHUP02/VybRClYZo72OqdJ46bXQLFfetDwDYFbqikudI/uaqQU52PHgMEYUs9x3Y/qSu/IggCEcmdeVD3J4s2DQV9ui3M7FlB1P2uZj6hs1Gl0PZab4TaOSm6z9ZXFnM1UWCTKTNRVAJwYUafUFnVxRoioJG2XQ6DZUvU7zMC92QnXlpzdTjFBedr/nZtC3HRQc0DoMJEhOkwWPf9mpQ49/dZDEtTQ7mAz8XE0M4nPR5lGtMYMq0XlkOuk5ppnm7ZTEvwnPsifXYyPHrezwwTDt0YjttH76hqdNxaKZPnw5d13Haaafhr3/9K0455RSMHTu2Lg8hCMJhjvgRQRBqg/gQQTjyqNNxaBo2bIjp06fX5S4FQTjCED8iCEJtEB8iCEceddpCIwiCIAiCIAiCcCCRgEYQBEEQBEEQhIxFAhpBEARBEARBEDIWCWgEQRAEQRAEQchY6lQUoC5wu7Oh2exw2J1Jy1lZtISy20PLz2ncwEqM3DMA5Dro7Mli5H5Vi5YfDDAa5gqjtY5ESefYoDxhCzBMhMvpgS40RrYPAOrbaBllJyez6GAGgWTC5GxuIAhGRjIrh5ZmdCRIKMdkKuvXy4YVHag0bNEJKgvQctBKmBlPyMbIYVrMY8XIITvtTJ6G+UEpcuz0dXRH9f3V6IXxqCrM6LsMlyf1dupBkm0WBEE4FHA53dAUBfbo4NyxZbebltd3MkMEqCoj22zn/W2OxgxpwNRFdKYuYjJlscW9606UdI5J7eomoJvsODSag6+LZDMSyyFG8hqMpDMzLBDcXF0kROeby0OnM3FwbCVaf8rKcsMyIwnRmfFtfMwg6AqTHou5N0wm3ywm3ywHfT/q3MBHAOxM3ciZMOimpkYkv92qAgMKTOL6a8xQF5WRFhpBEARBEARBEDIWCWgEQRAEQRAEQchYJKARBEEQBEEQBCFjkYBGEARBEARBEISMRQIaQRAEQRAEQRAyFgloBEEQBEEQBEHIWA452Wan0wlFVeCIStjGlx20LK2Nk4lzZZG23GxafhEAsmy0VJ5Do2PBUDhE2nwBH2kzLTqtChLkAKOXTYENCgDNpOWFVU5DGYDHTcthOxkZafqIgMtGyyHmaLTEZA4jBQ1GflJNkC1UorKAqmbBispZWjq9rc5IM5eX0tfKbzDylPStysp6agotzexyKPROARTk1GeskftRicqG5mV5YJmRPPEHU+cNJzGaEeRVmnfXcn+cUmWarCKUsQEA7UppW5M9tC1M39LIZ24FJdHFZEfXbwYgBwDtfgDu9gIA5hzZbfMZW2PGdhRjO5qxVfe2jj1uHQDQLqvWhJnzX1e/BWn7Ay1J2wbG9gdasenZzGZeBDfsGAngE/wFfrYkqFg3E3G4HVA0E/ZouRRfZuSFNSdd3joctBPKdtPDEgCAx6LLVDfzXjoYoK9POEwPWaCpdHqUBIlpNSr9a7fbYMKEwZS1SpoyxcPIYTsctAPWmd06nHSZmuWiH+wsZigQMPUbNeH6x+siWRas2FAbBp0/BjOkh1+nr1VAoa9xyEnfq5aTdsw6mLqxjb9Xs7OyGWtCWmOa2q7GgKUjHCpPuYXJDR9SCWmhEQRBEARBEAQhY5GARhAEQRAEQRCEjEUCGkEQBEEQBEEQMhYJaARBEARBEARByFgkoBEEQRAEQRAEIWORgEYQBEEQBEEQhIzlkNNm1TQVmqlCi8oix5Y5bDZa1s3OSL5paU+flvWz2Wid0jCt6geVkbwr89KyfVZC7KlZEU3RgG7A0HUECOldAFDT5V2ITqzdSeePndltHqNTm21jNmQklH0+Tia5QmIwJpWo63pcmlg3aMnjYIg+ZsBPy29bzL2hMOdoGbSMpKLQ55jNyC8CgBpipKmtyDnG8sYMhuN5YxF5bmX6q478SvN01lYPTqmSU6msnJbKpFaqBADUY2xgfEy1JaZjj01zAD7wpQEny5zOzsk2N2BsNZRmLmpHXyw/I0fqS7CpyEYbAOtaFsCEFwAQAv0M+pj9BpntdqIJaVuLNqSNk19ex2y3Fm1JGwBs3pNetjlHAZ7LB94pOgdlaSStY+tmIopdhQoVSrSwiy1zz4nGlJk2OyPZz8gSR6AzWrMxMtLMbu0q7US8ftqWWBdRo/sIqgZM1UCQkZdW0xQqNp0uG21u+kSY00dODl3/83jo9OgafR5lOm1TbAl1ESWyf5+qworOBxT6mMUqff4lTL6GLNrfBQ1aKtw06CE7DIV26C4HX+CV+GlfGKuLAIBNjVzTIn82dNNASE+dN7Z9GEIi06stgiAIgiAIgiAcwUhAIwiCIAiCIAhCxiIBjSAIgiAIgiAIGYsENIIgCIIgCIIgZCwS0AiCIAiCIAiCkLFIQCMIgiAIgiAIQsZSpwHNggUL0KFDB3Tr1i3+mzBhQl0eQhCEwxzxI4Ig1AbxIYJw5FGn49CsXLkS55xzDqZMmVLjfThtFlTLgt1mJS17nPRYIm47rVHutOjxORRm3BMAMDVai9vQmfFtFEZvntGitzsCpC2cMJaKoirxqaKp0E1ao9zw+kkbABjMODRZ2bSGeW5uFmlz6LQwPHfDBYP09QgzY8kYVoVNjYbo4QBgmpH/gzo9CIll0nmuhEpJ256du0hbvYKmpM3upjXaPTb6Xs3X+Hs14KPtSnRMJ1WL6NybYRNmdDycoD913sTWPRjUhR9BXqV5ekih6sGNpULfQhG4MWO4bbmxc+hbhbclpiX2eB8NgHcT/D4B/sGux9gaMzZmHJrS1rT/5cZo2cMMfFOGinEZbHCjDYBV6AA9mjnFyCW35Wx7GdufzElyY8b8hvakbfuq1qQNP9MmAMDqNHYAYQeASUB4dj2E0zxXsXUPNHXhQxxOC6pqwWa3kpZdLroscjnpuoidGdtEUfgHzHLQZarJjNOhqXR6NGaQGptGX9jEcd1iY64pLhWKTYXBjLMWDvKDgZnMWCsuZty/HKaewo1DCBt9HQMhOm/KDfpaWFbCWFbRY5eFPTDNcHRbZnwbkx5rqFSny/edRfSAZbbGdMGluPJImzNI31O5fr4dJFBM3zuKWrGtoilAB8D4U4FhKAgGU+eNYVOALuwh49RpC83KlSvRqVOnutylIAhHGOJHBEGoDeJDBOHIo85aaEzTxKpVq+B2u/Hss8/CMAz0798fN998M+rX515vCoIgRBA/IghCbRAfIghHJnUW0BQVFaFDhw4444wzMH36dOzduxe33norJkyYgKeffrra+7HbHOw0FTYb3axo05jmWMYGACrTlKtwNtBNhyrT7Mylx4RSZb3YlDt/ulEVabfVGJvKpJXNG5VOEbedyuVpQrO6Ep1XVC3e/MillctzLm/sdroJnM1TNi2kKX5eFJw51uUslr+KakvIm9Rppf7f39SVH4EjJ/V8TeF64NG3ZgSuyxkH9/DWtMtZ4nm4cpKnHLXpcka7bn47tv8A45tAd4fVQHf5tMFdZT7xPzvori0OuEibE7SvcDHn4QH9UGczN0cOl2/pSnzuWsX270ieVmfdA0ld+RBb1MdT01RoTNcorouXypQZAKBodEYqTLcqti7C3AusTUno4h2tl8WmGpM3SpoOQZqdPkeVqf+pbN4wNoVOj6Iy+WbR19FKuBax+mNiPVKz6GNqzDFtzDnabXR3NBtTjnP1LTvXHZE5BwAw2HpMxba26DHiU1vq9FD/p9y/ZVnpiuMa89NPP+Gvf/0rli1bhuzs7P11GEEQDmPEjwiCUBvEhwjC4U+dtdCsXr0aH3zwAW666SYo0Qg+FApBVVU4mA/aKvPCG9MR1kOw2xy45ILr48seN+2EXB76zZmTeSOe56bfJgBAvpP5EI15Y2BwH3eF6A/jvOXVEwXQNBt6DxyBxZ+/BcPQUVpKf7xuMB+TAYCDeaPiYT62q1+fftNZj/3wnX6zGAzQXyYHmDxV1Yp0KqqGNscPxLofP4cVFUvwG/RHal6/l7SV7i4mbbt27SFtOUfRXzp7GjQkbfUZUYBmLr6FJsh81J3YQtO6+1lY//2HsMzIsbz+opTbqJodnXtfyB5zf1BXfgSPFkSmN26NzIfKapcwroWmNqIA3Hey+0MUoHILzcytwLgCIJAmf2rTQtOOsRXWzFZaSL95XK/RH8UXIZ+0eVFRxtjgxhC8jAW4MC4KUMJ83F/CqEYUM7btoAVE1oM+jzWMYMCOX2lRBPxCmwAAv6exI9LqsvUmoGAqUJZGFCC27oGkrnzI+3Nvgx4OwmZ3YtjoB+LLLhdT33DSLXXcW+96dr6Fph7TepHFtNCYBl1ulofoi1fur54ogGpz4Pjz/g8/vvk4TD2E8nL6A3UzjQiTnbk2zlzaltOQvh5Z9ZnrkUW3NHjDdL75mCYIS6toHVZVGzq2vwCrfn8DZrS89XGiAAE6z0tKfKRt115GFKBRM9Km1G9C2pxh+hwbpWmhCRbTtsotNGcOOAUfffEldMOAjxBhstlsOG/QEPaY8XWrtVY1yM3Nxdy5c1G/fn1cdtll2LlzJx555BGce+65++REwnoI4QTplNhy2E5fbJvOdONhmlwNrm0MgGkyN65JX1SLqXybBl0zMFhb1UDAMHQYhg6dCVqMNDI0KtPsajD5yp1HrLKc2sblG70d55gTa4yxvVumATMa0NQ0z7l8DYfp2iZ7Pbi0MEo33D0FAIzQXbypvyJv9HiemAZfyBxo6sqPJAUwobLaBzQ1DUrSbXswVc7iaSgD/PsxoOFcUHWDryowlTjQBbzBdA3TU3Tx0uGPBzRhputYiLEFmWMGQD9/PubG8TLlWhmXb+mu4z6oAZaF0gc0B4O68iF6OAg9HKiyrGt0GWbT6Bd2XFclU+H9O1eH5DrZWEy5aer0xWNtKeoiph6CqYdgMOViuoBGZfKAqRrA5AIMrr7BdR3j6nAm/YBZStVzNE09rnJmcIq0Jp0/OvNSNsxcK4sp3xUmU1WmzNLTdOrSubpIim11w4j89HTOKT11FtA0bdoUs2fPxqOPPoonn3wSTqcTZ5111j5rv3vUMMJaGPbotxax5fp2OhPrcVKJTP87Z1qNN3qFMHPRDJN2aqEg8zBwFR/q9C3+rVBQ5+V39RB9E4WYFhO/j97Ow9xVOlPwK0w/4kCAdpSBhMpq7BuVvQFvRfAQpt9uFG1aT9p+/onWON285U/SNmjY2aQtqyEtlahxgVCa6xhmHHAoKocd+y7G6/PGA5kwEfCq2n7ricpSV34kSQ64McDUG2tP7f1w3R6Trs8n22KPYgtEgqfqbrev6eG+w6ZVlGExss3bNNr4B1rS2zEyyTsSbhonnDgbwE/ogmA0stwJ+o3mDkZ/mttuA5NWVn55GW3Cj4yN2w6olmwzshGRYn4aAN3AnbzuAaSufIjLqUNXddjsetJytpv2jdluJtixMfWUdHURO12n0Jlv7UymLhJmgh2Li/sS6ymxst4BQAUcHnrDUHmaughTE9YNuvwPMtsxCtswDLrXT0ijW31KmRdN5XpFnmqqhc4AdgSteK+dMFOO79xO9/pYvWYTadu8g+6h0/2k5qQtz2C+WWbqd4bB36w6nRyEE4IWW3QIE99WP/Qw/WJeYer+lanTcWh69OiBefPm1eUuBUE4whA/IghCbRAfIghHHnU6Do0gCIIgCIIgCMKBRAIaQRAEQRAEQRAyFgloBEEQBEEQBEHIWCSgEQRBEARBEAQhY5GARhAEQRAEQRCEjKVOVc7qAo9qQVdN2FQradnDaHTby+kxFBzuLNKmKYymHwCDGQQz7KBjQU2ls9XBDPRZFqA1ML3+CgllW1Te2FtcAl0Pw+6gpZBVJi0ALdsL8OOpBIO0VGIoRMsv222MLrxCp1VR6WsVKCuJz2tRaeKArxRGVJp43dIl5LbrliwmbbuKaf3BvYzcdcmmdaStbQt6oCuLuR39QV660M+MrBmKplWL3jeBYHl8wFVVSy0jqTDjXGQEeZXmayutzD1G/Pi8/LY13S83Rg2t/gkkjqMae0ybIyJrXd3tUsGNp8PJNtNqx9iRT2+4GUeTNk62mZNJ/gMVA1K6oxfmJ3SBP3rzcMfcbNC2otUFpA20MjwvsczZvmFsgQ2MEQC+T2MHELQBOAfY8y5QlubBiq2bgbhsJgyY0Gxm0rJLpc/ZxgwRYNPo4RU0G+9EDJUZo4Spi6gqJxVN28pD9Hn4gxXORzMj6faHvDDCQdiYQT5VRrYaAHRmnBqdKY9CFi2F7GfqjWDGqPEptPx0UKW3K/ZVOEJbdLyikmBknBUAWL+KHrl2/U+/krbdu+h6WmkZM16VZwtpO7qQHuTbYmT6g2kGXguG6MIinDDeiemI3Csh358Ih8JkXcRS+EFnE5EWGkEQBEEQBEEQMhYJaARBEARBEARByFgkoBEEQRAEQRAEIWORgEYQBEEQBEEQhIxFAhpBEARBEARBEDIWCWgEQRAEQRAEQchYDjnZZsUwoOgGFMVIWg56aR05VaNlFN2ubNKmcTq5ADRGnk9VaanAYJBOqx6kJe9CjGxz0Fcho2jYI5KCQb8XejgEPUzvU7Xxknd2O30LqIxUMkdZOX3+CidpbaNlLR2M/OJviyukl+1OF045/RKs+e5bhIMRCcVfPv+UTk9xMWkzDDqtpkXfG6uW0dKnHQpbk7asBrSMYhC85GXYoKUbg6FI3mmmEl3W47LNQOrtNFvNrv0hQ36leTp7qgeX/em8KGevzX7rejtOgZeTZQYARuaTtTH7dTDyoBpzQTmbHx7StgcN4vOe6IUpQj580f39EWxJbluyrClpYyWWf2RsnKTzasYW4PS31zI2ANiZxg5U6H3vQkTvuzrrZh6KYsZ/icvhEC2Rr6r0vefMcjPbpamL2Gm7aqPrIuywDJxMMiPbHApUPLRatNwJBXwwwgEYFn29FYt3TDambqByowjQpw9fgD4PVaXrhiYnsW3S6Vy34pf4vMPuBHoAG1auRihaR/vluxXktsYO2hkaZXRBYfrousj672hH0TqrOWlz5dB5E0pTGOgmfc+FE++56HMVDgcj92k49bNjmtUvvKWFRhAEQRAEQRCEjEUCGkEQBEEQBEEQMhYJaARBEARBEARByFgkoBEEQRAEQRAEIWORgEYQBEEQBEEQhIxFAhpBEARBEARBEDKWQ0622QgCRjiu6BZf1m107OXKzSVtikZLJfKaqYDdxkjl6bQcYHnpXtKm04p2gEXLlNq1Cm1CW3TerplQTBOeLEYa08ZfYoedljHVNAezJa3xqhuM5KNOy086FFqb8del35C2Hz7+JD7v8mQDtwE/ffY5Ar6oDLa3hNyWk7X8Yxd9HvDUI007tu0gbWtXrSRtHU84ibSFbFl0WgD4AvS9Ew5FZA81I/IMBYM6DD1y/TTiHreUDH/X0ajSPCPtWS04SWM669Nvy0kac9vRtzSvvLstYd4ZnW5B5Byqu10quPNowNjyGVN9Wh60TTtaftgH2ufvZhKzAS3j8+5o0eiGH1b0Qvi8tK/EH7SJlW3mpJl3Mzbu3mAzvDu3IZCQBzQx33Aq0j9YmetHTCvyixVLsWVDo8/JUZ+Wu1XsTtIGha+L2FSmLhKinxN/aRlpMzj13RAt6WzTK8ppLZo5Nt2ColtwOZj6hpmmvqXSUsns8BM6Letr+eiTVJ2043Yq9Ha//fI7aft9SUU9xeXyAP8A1n73LQJR+egAfTkQLqefpT83cReLrhvsMYpI2+Z1tA9tXdiBtOkacx8DCDD3jh6uuI6WEpkPhd0Ih21QtdT3h2JWX/o9c72NIAiCIAiCIAhHPBLQCIIgCIIgCIKQsUhAIwiCIAiCIAhCxiIBjSAIgiAIgiAIGYsENIIgCIIgCIIgZCwS0AiCIAiCIAiCkLHUWLa5qKgII0eOxP3334+ePXsCAFasWIH7778fa9euRV5eHq655hpccMEF+7RfVbNDNS2omj1p2cnI5KpOWraOU0l2mrRMMAD4fLRsr2ajdTPtjHShTaVli40wLduX46qQUNZskfmG9RwwdMDhoWXtyg1a0hAAbBqdBw4HLbOoqox0JSMV7bDofNv2+6+kbdmiz0lbWXGFhq0ezUNvSSn85RGNRLtJ50HQTktTtzypK2nzeunz2F20hbR9v+I30tasVTvSVr8pIxkLwMZIbCv2yD2nRaUvHXY3DCVyjSzieqhpJM3rgv3lQwBUlW3mH/X0cDLJ6eAUN72MrZSx7WFsnMRyoi2mkroZkTTuL9lmRpqZtdWnTS2ydpE2/1HrSNsfaEXanAmlhSMqR+xACEZUIzkcYGTs6Ueel21ezdg4uJKbVr8FbJykM8BLPkeJKRNnt0n/XNEqxnXG/vIjisMORTWgRP1mbNnupusbqoeWDA8zeWVPk5GBACM/DLp8szF+XFOYuohF10U8jor6hmqP3Ii5DhtMxQ47U/b7A7zEt8aY7Yzks8q8l7fZ6e3sOi3bvGUrrae+5vvlpM3vTXDoRuSEAuVe+P0RBxkM0Q9EyKDrcc1ataePWURf/+JiegiJ1b/9QdoaHnU0actu2Ii0AYBmMtLlCfdcrJ5sd9QHoNN1EbX6YUqNWmiWL1+OkSNHYtOmTfH/SkpKcNVVV2H48OFYunQpJk+ejClTpuCnn36qySEEQTiMER8iCEJtET8iCEKMfQ5o3n77bdx8880YP3580v+ffPIJcnNzMXr0aNhsNpx88skYNmwY5s6dW2eJFQQh8xEfIghCbRE/IghCIvvc5axv374YNmwYbDZbkiNZs2YN2rdPbhZr27Yt5s+fv28JinYBqjK10c39mkY31SnE6KMA320qYmdsGt1cqzJtp5ZFb6cx56gmjBQc6zoUm6rM+WtpRnzXVHpbVaNvD1Wlm8i581e582dGUnYyzfzurJz4vCsrO2kKADamy5nKNEm7PPQxDZPucmYE6SZX7jy48+euMQBoNjpfgeQuZ1rCqMuWlfr+0LiRmWvJ/vYhAAAlJ/V8TeEeo3Svhbjee5wH5i4B0/sJ3EDOid2RXDnJU65XI33bRuBuP7oXDn8eXN4wx1OZfk52pj+WO+GAsfnE/3K4c+TOg8u7mt6aXN7U1FZNcrKTp9VZd3+wv/2IZnOy01SoXNdfq2bdpgBAtegyTGG6nHH1FIspM1Q7/ZyoCXUKNVpmxadc3hh8lzOVeYhUrm5kq1ldRFHpfNNs9Pk7nbSjdLsr0uJye5KmAGCp9La2MNNVn0mPEmTqmyaTVqZ7pN1BFz52O+9E7IxzNo2ELmfR/cSmFnEZ0x0vkX12b40ape4/V15eDrc7OYNcLhf7HUoqho68kV0WKjjxzKsPdhLqlHYn07ZT/n7bPu1rxqLfa5maw5e+Z156UI+/v30IAKDh1tTzNYXvNpzZPFEH+XMIQfc2521jUvz3MoZULBzFbDy+hrYMZutBdrH724/0O+shdlmooOOoew52EuqUFozt5DP3bV9PPUN//ysAF48ZVmf7qoP3NRHcbjfKysqS/gsEAsjKSvdqL5kPXnsUejgEm92BoSNvjC97PPQXovXr55E2roXGkaaFRlPpt/CajY7u9SD9sZml09Grz1tM2iq30Jx45tVY9tFTMPQw+5Gij2mdAACbSm9rd9ARPNdC47DRbwwczJum7Rvoj3k//+AD0rY54eM2V1Y2Ziz6Hdf2a49AeeQDPa6FJsS10BQUkrZyRhTAu5euIBYWNidtZwwdTNrqNW5K2gCgPMS0GOkVLTR9z7wUX330PAw9DACwiOuh2ew4Zcg/2GPWNXXlQwAAuwsi04ZbI/NWGb9+OrgP9LkP4tNty9m4JBcxtj+raXPlRIKZ6wqAQBmwu4b7BACurtiZsXVjbF1qts/fmx1D2v6LAaTtE/wlPu+GDS9jCC7EAvijogCfbvsLtSnwGvMlPtfTqaZBwUFuodn6O1DQHijjRC0S1j2Q1JUfWfThrTD0IDSbE/3Oeii+7GLK2+xsusmNa6Gxp2mh0WrYQmP6w6TNCtF1EX8p7XxUPbmFpuOoe7Bq3l0ww0HYbXSdIa0oANN6aq/HtBhl03URWzZ9TJuTzrdtu2jRkf9++QNp+31TRWHgcnvw1DOf4+orByLgjzhIX5C+d8KlXAsNHWIF9tLnWF5GK720atGYtPU6hX67nN2gIWkDAF+YuR8rtdBcPGYYXpz9PsJhWhTAbrfhkqvPZY8Zo84Cmvbt2+Prr79O+m/t2rVo145WbkqFHg5BD4eqLOs6rVdmGPRDqzDqISbR3Sa+LaPIpSi0zWTSYxmMsghzjpZa1RkaehiGHoJm0M2xRhqVM0Wj06oxqhtk+yAAU6EfMJPJUyNMB4JBP11rjKmZJRIo91ZT5YzpD+Rjjumjz8NXTpfy3Hlw58/dUwBg6OkDmorlcEJAQ293oKkrHwIgOYCxymof0HBlMV9Og6lvAFz2c5eck2+kb6PUimuBssiPC0rSBW2c3c/YuPPg8oYRhTIZ6bgwIznnT3FAP/T4/2WcEBV3Hlze1PS2PIgBTYwyL1BWy8dqf1BXfsTQgzD0QJVlQ6cz0TToLldcQGOmCWgUJqDhVM5Mg6lTMC9XzTAjzahXPQ8zHIz8mDqVyai4AoDC5IGpM/mjM3URppubxdQLEq97ZYJB2lH6U5TvAb8v/r8/SOd5yEffV4aN8Vvl9Hn4vExafbRjDofowicc5usM4VD1AprE/YVDdECzL9SZexs0aBAeeeQRPP/88xg9ejSWL1+O999/H7Nmzdqn/aiKBVUxoSpW0jInd8x9J2IwmWRyOooADIW2Mw0/cDNSwOFyWm9V30m3UJTsLY7P2xxuYBiw+7fl0EN+lIXpxOS14J14s2YtSZuLaaGCwtzwzEP028rvSNuq7xaTtl1b6DcNRb6KGpxHjeT9Xn8Qvuj/OQp9m7tzaZnSc/42grS5HPSblq8WfU3aNm7aQNrCKtOnlekLDQAuOx18GtFgOPZdjMvhhBFtnTSJb5o0Rn5zf1FXPgQASvMj51ovPl+7b4LqcdEFF0Cks3MtNLTiJt9iwkksJ+4z1r16OyLBDLNdOE0LTSnzpr4BI7/MSTOz354wtkLXRtK2Mp/OVEfChXJEK4kOBOOyzfAyrTB/0CZemvkXxka35kKnhzEA9wKV2WXabWPE7puTwQfBieseQOrKj1h2C5ZiwYp+oxFbVhxM5Zp5QWYxLxctpoUdAMKM+0nxrjOOi/F7ejmtRW9sp3XIS4oqolgt+j1J8dpfYQR98PnpxGQ3pKWAASA/j+7T6eYq0UHapjOBycatq0jbuvW0Il7ZTvo6lgUqynA9Wu8oCwThD0R8SzhIP7dOjXaG/fufSm9np+sGP35P+5c/t9O+UGe+57V5+LqI08MEZmbFsxP7NsaRa4cSVurkG5o6G1gzLy8Pc+bMwcKFC9GzZ0/cfvvtuP3229GrV6+6OoQgCIcx4kMEQagt4kcE4cikVq9hf/steZDAzp07Y968ebVKkCAIRw7iQwRBqC3iRwRBqLMWGkEQBEEQBEEQhAONBDSCIAiCIAiCIGQsEtAIgiAIgiAIgpCxSEAjCIIgCIIgCELGcuC1WdOgWWFYVhhaVM88tmwEaF1Qv8LIyGn0KVoGo3cIwOai7cXF9Mh2f676kbSFtq+hD1hOD+ZUXlahGe5wZwMANi3/CiG/F39sZzTK7fSgowBQ0LwlaWvdhtH4ZMbh+WP9ZtJWtv0P0qaXFZM2GzMol8NWcZ3s0Xm7TYv/7w3SMostmheQtqZN8klbVlY2aRs0hJZY3LrlWNKWk08PR1/CaMYDkXEAKCwzJtsckbQO+b3xMY8UhcgbZmyjTGCPFrl29aLzFmp3Plp9WjY8qyjNQDScIiutnArQh+Slmatri0kfbwFQDviY7f5IMw4NN85nT1qpHPaayjYzCsqMSjsa96Ez1ZkwmIwzOtCNE2GYMclubhDJPxgbljO27xlbH8bWgTZx0sudGBsA0O6pgtij1Bf8+DuJ62YgZvSnVFoOhpiTLqcfFDszDITlTzOwJvPuuWw3rf2+bjU9qmn4z030AUv3kqZAcUVZY/NEBhLd/tMK6L4ybNlO502pSg86CgD5jWnZ5kJmEEjFwwyQWURLE/sYh6eb9ABLLjs9vIJD8yTMK/GpEZ0vYcZoaZpPn2PDfFp+2+Wm76uefbqStp07aYfmyafrjV6FH6fAtDFjHyWOxRQdGiJs9yGs6OQAsaqt+kMuSAuNIAiCIAiCIAgZiwQ0giAIgiAIgiBkLBLQCIIgCIIgCIKQsUhAIwiCIAiCIAhCxiIBjSAIgiAIgiAIGYsENIIgCIIgCIIgZCwS0AiCIAiCIAiCkLEccuPQ2FQFUJXINGEZVpjcJhykNcOdbnqgA7eLHksEACyN1vdesfQn0rZl+bekrb5BjzWTpdLjyejhitgz7ImMe1FeVIqQrwzwMePphLfSNgCbdm4kbf4tdP5odnrsn127aX1zN6MpnpdF3471nPTxHK6K7ZxZkf0fk29H0BmZ37CLHsOlcSMPaQvt+YO0+XbTg4voCn092janB4kwTB9pM5kxDADA4aDfTVhReffYkEwOO2BEszOsU7r4tF5+JrAbEU3/VtF5gx3UJD1OJ62tn5XFjcIC3svWdIyaPTWzWYm2UGSMDWsvgDJgBzPWDDckTrrkbGHOoxU3Zk49xsZdTsbWpA89LoUbFc+fC/bo1AfExqHhrlUxY8Naxsb5Z2bsKa7oasvYejG26tiBitegpyMyMEt11s1ALMUOSzFgKfakZZ1xxRYzRo3mcpI2p5u2AQBC9EHXrKLHttvxw8+kLdtPjzXj1umxRoyEMXPsgUhBEijyIlzuRdhHX3Bdp+s+ALCl6E/SZu2ky2l3Fl032Bugy1QnV9/Ip8twWzZdNpYmlMMuV2S+oJ6KQPT/ojL6Oubn0INrhUvowbwCxXR9S1fosfSObkaP+2M66Lqolc3XRexZTF0kwV/Yo9lvzwegA7pBOBNbOidTQQa7G0EQBEEQBEEQjnQkoBEEQRAEQRAEIWORgEYQBEEQBEEQhIxFAhpBEARBEARBEDIWCWgEQRAEQRAEQchYJKARBEEQBEEQBCFjOeRkm02bDSYMmDZb0rJqY5JqoyX2TDst+aa6GbljAJu20RKf3y1bSdo8JbTkod9HS+wd1ZA+x7BRIU1oROf9hoKgocAfpmUEm+XSMskAkJ1FSwVqdnq/FqPcl9XUQRstOs89DjqtDkYmWi2qkGZ0uCKymQ1dIYTMyHyoKS35mJNN73frOvoahxVaw1Wz03kayM4lbVmNaBlFNYveDgB0hX43YUZPUVMi1zOgGDCi84aa+kJqxP+Zwhq0AwCcFJ0PcxK41SAIWlZVO/o3dtumOqNbTKtjAoyMMmgXw3p1JdEWlTdWmkeO1Yq+beHh5JXByza3aswY6VseaMLYGjA2RrZZY+TI8xK0l52I+LBclCCIiB9xHUvLcwdOpOVR8c0w2obutKltO9o2nNklY2vXZwWzIdAT37F2ICZpfSlGdn4eAdDDKSSum4kEbB7o0GCzOZOWNaYuommMzUaXNdkupswEsGcLLZz+8zpaFtwVpGWkg2W0pHGjLLqc1hPOw9Ciss2agrBNARS6vtU8m69uOtx0HmS56f1ajOK1i6v/MEMduJi0eph9NiypcOhOZ2QfDZxBBK3I/00KaD/haUjfHzu30JLXukIXIloWXU8Jeegy0Z3TiLY15OvNWQq9XzPhMtq0yPXOcu+CboRgmql9c2y96iAtNIIgCIIgCIIgZCwS0AiCIAiCIAiCkLFIQCMIgiAIgiAIQsYiAY0gCIIgCIIgCBmLBDSCIAiCIAiCIGQsEtAIgiAIgiAIgpCx1Fi2uaioCCNHjsT999+Pnj17AgDuuusuvPnmm7DbK2TtJk6ciJEjR1Z7vz+tXYtQ0A+H040zE5Y1Rpq5QT4thRfSaX1TxeRP/9ffaa3STdtpCc/GflpGr6mdPg+3h5anC5cnSNqpCVMVcLho3UKXi5fYM006f8wgLSOo02qACDOKvzY7fY679tBpycmmz8OuaVXm7ZoGKzqf0ziH3NZw0Vq1xUFGtpqRfISfljItD+8lbUEvff67eXVUaIxss2FELpbT6Ub/IcDiFd8jGIxIKzrsqe8dh8OFgZzabB2wv3wIAKxGYdJ8THq3poRA37camIcBgKPVL6Qtv5zRbS5ldsrJNjMypkk2d3TaAoAfrNxxE8YGAE0YZWq0YmycpDOjhIz6jI07f4bcBNlmR3Qn9VGCUDSzj66/mdx2DSfbfDwtGw8vI818IW3ChbSTHdjmQ9J2FhYwOwVOwZesHQBUZAG4FFfjKZistnjFuvuT/eVHft60C8FwAE67C8MSljWNLovq169H2lSjjLS5Td7Bb99E33vby2hHkR+g/V4D5jyc9WhpYr0sQXvXlTA1AXsOI73s4t+fqyadVp2uisBg3K/OHFJz0udYXEZfD42RtHZqFc+lIzrv0CwgOp/XiD6mmUfnXVmIlq2G6qZtjGyz30nfN85SWibaKuIKJqA+4xOMBGlmh90N9B2Hbb9/iVDYD7st9fnb7W6gH3vIODUKaJYvX46JEydi06ZNSf+vXLkS9913H84999ya7FYQhCME8SGCINQW8SOCIMTY5y5nb7/9Nm6++WaMHz8+6f9QKITff/8dnTp1qrPECYJw+CE+RBCE2iJ+RBCERPa5haZv374YNmwYbDZbkiNZvXo1dF3H9OnTsXz5cuTk5OC8887DFVdcAVWtftzkcLhSTjWN3oed6DYDAJZKt1Wm63LmctFNeZ6sbNLmZppyXTa66dDhoc/DaSU01Xmyk6YupjuMw8N3OVOV6o/CmrQd08yrWHRXLZudzlOzhudhGQl5485OmgKA001fK7uTTo/GnIfFZZtJ33M2ZiRpk8kbJ9PkDgAq0+XMNCMXq/IzBYBs5k1cp67Z3z4EqBjpvfJ8TbGDzg+N66sFAKC7PEJlRrJmTGy3Ku7SJd5i7pzkKdMzKu0p0o8Kv1+mtwR7jlzeMG5dBe0LHAkHjM0n/pfFvP/L4dKTLu/oBNEw/sDDZICDvTliXcR41OgFVdkLi2qvU1P2e13E7ko5VZny3WGjb1pFobsx2U2+nHY46Hx0MeWbM8SUqQZ9E9nd9A1tN8yE9bKTpg6T2c5Z8y5nnC/gLqnioOtbmpveqcl0AdSYLv4Os+JaOJxZSVMAcDnp6+h00PtVXVyXM9rE5ZvGfP7gsNP3hqXw3bcdoNOa2OXMHq3vxKY2sstZ9esiimVZzBcPPIWFhXjxxRfRs2dPfP3115g9ezauvfZadOvWDb/++ivGjRuHSy65BFdccUVNDyEIwmGM+BBBEGqL+BFBEGosClCZPn36oE+fPvHlLl264JJLLsGCBQv2yYk89q9xCIUCcDhcGH/zzPgy10KTn5dH2kIGHU2ma6H5fe2fpO2Hr34gbY0CftLWxEZ/UNW6JR1Ol5Unt9Dc+s46PDS8DUI+L8p89FuYZvXTtdBwXxfT8KIANWuhCfjotGQzogCVW2jGzV+Lmee3RcjvBQAUZeeS2x7dsStpO9AtNEE7/famKI0oQHVbaG666UlMnXoNQqHIx+hcC831//c4f9A6pq58CABMxlQAwD9xEyZjaq1FAVpjPWkrxG/stoVYTdryf2FEAVYyO13H2HYytsRvPd05wLNbgSsKAH9Zso3bLhXcd6ItGNvRNdyuJWNrTpvW/aWAtL2OC+LzDjhxE/6JqZgcFwVItFfZ7yud6YPOok3sN/V/ZWwj6feQ/Vt/TNrOwCfMToGTsZi1A5FWl274HD9gIEz4qrXugaSu/MjUF8cjFA7AYXfhposfiy9zLTT169GtsYrhJW3pRAF2bNlK2tYu+ZW05ZXQfi+fSU/zhnRLS7k3uYXm0jc34PnzWiHs98LnpbdruJ9aaAxmtzrXQsP0iAlyLTRN6O2KzAq1EoczC9c9/C2euKUnQsHIg77RfQy5bW5uF9Km7qphC00ubdIaMi00WfS1sPR0ogC0T6jcQjPmon9j9kuXIRz2sy00//jb0+wxY9RZQPPZZ59h9+7dGDVqVPy/UCgEF6MilYpQKIBQ0F9lmVM5C4fpinC4FipnASYw8ZXTzsDvp7cL2Gmlk5CPvomCvqqV65DPi6CvDIFyunYdYpoVgQMf0Jh2esMgE9A41OoFNDFCfi9CvkheB5kgIhykr5XJBTRcuyYT0Fg2Ji0WvV2wDlTOYoRCgbjKmWUyjvIAU1c+BEBSABNEqNYBTRh04GGkUXoC6GceJhPQcNece2yZXSLV7e4vi/y4umm6U6TdIb9f+vHjz5HLG8Y3mUxCQykOGIrePQBQznSlYESR+Lzj8o27ZRn/42MyIMTeHEirWpa8rm+f1j9Q1FldJBxAMByosqwx3cNCOl0Wqwad97Y0AU0oRD9EAT99E3Flakintwtn0YFJ2Ff1OQj7vQj7yhAqZ7bjIg+kCWiYLq1sQKPTz6zJdJ0KM9fDZJTjQinujVCwHKFAJK8DKn0dgyH6WqmBGgY0jA/VwkzdkKlTWzrvQ0KMU08MaOKHCvsRCvthMfWt6lJnAY1lWZgyZQpatGiBXr164ccff8SLL76ISZMm7dN+Fi/7CT5fOTyerKRlu42+avXr029FistpPVEb04cUADZvpLct20tfVDtTSWx3FPOdjEJf0J0JUomu6P53eU0Eyk0UBejSbcdupjIFIIeRUnQzec6YoDMFqmHS6SlP4ShjNGQ6qbs8FbexM9q5vMynIFgemd+j0teqfPMW0gaVaYZhWgy5QMjG9JPduncjadtewtV8ACWFo4jhiEqXetyRZ+qr5Svg80cqIhrxDYfHnYWb2SPWPXXlQwBgNY5NmvenkVZOhwHmbVaaYMnDOHhPK/rtqmsPs1OuTOG+PUnsbh9bryUiBR/3+US6Tys42Wb6pSRwFGPjJJ1pdVw2rZzEdrJssyv6X0k8CDgajGxzL7qlF6fTJrYEvpSuVJ3TYj5pOxfvkLbzg/R2AJD1XjVecNhygHOBE97+BdD58iW27oGkrvzIt79ugS/gg8flSVq2Mb4/O5tuZQ8HdtPbMcEOAJT+STeRmkEmUDLo63lMfbpMdTjp7TbvrbA5ogHDXt1ESDdRatJ1kT07+da8LI2ujznddJ5rXOuNTpeLBhckMvnmttNpCSR862JZkfmg30QwGpD4FPp52WtnmtaDjKNg7sdUL3tjNChlXu5soYcsMctoGwBoe+lj2hNe6Lpc2cDlwJpvViIQ8EJVUp+jy5UNXMweMk6dBTSDBg3CpEmTcPfdd2PHjh1o2LAhrrvuOpxzzjl1dQhBEA5jxIcIglBbxI8IwpFJrQKa335L7js+atSopGZeQRAEDvEhgiDUFvEjgiDs8zg0giAIgiAIgiAIhwoS0AiCIAiCIAiCkLFIQCMIgiAIgiAIQsYiAY0gCIIgCIIgCBlLnamc1RWlxbtQXu6FHspOWva4aInBYDkjacgoM7sJydoY5aX0AEKhML1jTrqwNEzbikO0NOzaBKlET1TNc30x4PMCZYxmeoCTUwXgVhkJSgctv+fUGGliZuCxLCedb4ad1l8MgJZQLiutkK10ReUWd5dGJK0BoITRTfe66QHLAsxgMw6NTo9u0tKwAUYXP8goC6uM/CQARlQYsMKRa2xF5YUt/15YPi97TC2UndqQIaxB26R5bgyR2uJOM7hgDjMOTU4WbWt3DCMpzo3Rwg23kShpHLuFmyMy5kltZJu54UiaMDZOmrkBY6vP2FjZZvo5SpRttsMdPUwJwlHZbU62uUUhPXjqxsHHkjZu4Lu/tFhA2s7Hm6RtdBFtU2bQxwMAvJHGDkTy91wADyH9+ESxdTOQPWU+lAd8yAonL7scdLWpjBm7zsE8tJrKj0NTUk7bVWb8snKVvt+9Kl2+lTHjmm1OGBPFFS14tgaBQADwMWVfKM1QYE6LGUKCGQrBwUgsa8wgNW4PnW+mg64baowufmnCdXJG62WlvjCC/sh8iBnM1O/cRNrKDPqYdo1Oa5aX1v73ljAy4sG9pE3fzpelKr0prIT7yvJE7j9rYxksXxlCxK2jetIMwpe4brXXFARBEARBEARBOMSQgEYQBEEQBEEQhIxFAhpBEARBEARBEDIWCWgEQRAEQRAEQchYJKARBEEQBEEQBCFjkYBGEARBEARBEISM5ZCTbW7TRIHfr8DtVpKWHTZaKs7OaNa6HLQ0n83G6ZsCW/+k5VjtjFBuToiWmduwk5Zu/DNASwG7cnPi886siKSus342TBtwQidaYnd3CS8pu2YNbd+6i5Zg1H30reMGLQeZm0Vvp9jp+FrT6OvvSpACVKMxeggqQtH5Jg3zyW1zmtLpMVVak9Sh0ulRTPo6GiYjeUhnG8DIjwOAYXFSipGNna5Iuk5s6UAweq9ZhK6500XLRGYC6zZFpXKPicyXcXlbHVrQJk6WGQDyEuSAK5PL2Jocs4O01WNkXFmJ5XoJ87FbvwUAHbwUcj3GBvDyvdx+OWlm+rHl98kojtuqKdtsi0qc10cJ9KgsNyfb3AG/kDbfaR7SdhS2kbZheJ+0nV9eM2lm38O0DQAeTifDDMCZA0wCMOUXIMjf+vF1M5HGOW747BY8LnfSss1Gl1OMCfWY57KextdF9uymLwxXF3HZ6Pt9Wxmto7xbp6WAnQ0rTsTpicw7G2TBcps4roWb3K64hB4+AQA2/0Hbd5XS52GE6fN3MkMv5DDDS4CRrTaYoRcspcJmRYcKsGDG55vm0vmDhrTJ24j29x6VzrdGJn3f5Ou0hLSyg77+Jj/aCbY1qF5h63RH8j8vT0PQZaPrIu7qhynSQiMIgiAIgiAIQsYiAY0gCIIgCIIgCBmLBDSCIAiCIAiCIGQsEtAIgiAIgiAIgpCxSEAjCIIgCIIgCELGIgGNIAiCIAiCIAgZyyEn29zQCQRNwOlMXrZptBScnTkLO2iZZIuROwQAVael8iydPmj9LFqCUTFp25+ltIRy344V0p9Od2S+TQsPgn4TbZv6ye2OacTL77ZuSuuxbi+i5QCLy2mb4ac1hvUQLVv46297SJsWpGUbj2tdcR1dWZEY3V1PhaJF5o8qYOJ2O30emkafh8rIQZoqnVaF0WZWLGa7NLLNIYuW2IYV2dhhj+gt5tiDcBqx5yJ13jictKRnJlBwzKboXBsUHLMJXnCy1ulpCPredDA+BgCCoJ/BPYxu8QatJWk7uhMtIZy/k5FHTZSOjV365gBMAJxybLqSgs4e4Ki6t5UfRT/TO5yNSds6tCVtm3F0fN4ezYwtKEAYkfz0IifldungZL052W5uO1cJc0BGelln3ISQTKOsEAK2EFxOW9IyVy7YmCqFB3T57kyjy68YdNkQNuiH0+6mNcw1RpqfG+7h+LYVjsIR3UfBMU6EAmEc3ZT2hU3z6eEMAKCgIa1rvcdHl0dlBm3jRkkIMw7v9030Q1ReQud3s4KKfSquyLzicUFRIw9eg0b0tgEbfR52lRnSgyn7sxS6DptjMnURpg5jpikLspltY3URAHDYIunOsumw28KgxqZwOKrvtKSFRhAEQRAEQRCEjEUCGkEQBEEQBEEQMhYJaARBEARBEARByFgkoBEEQRAEQRAEIWORgEYQBEEQBEEQhIxFAhpBEARBEARBEDKWfZJtXr16NR566CGsWrUKdrsdffr0wcSJE5Gfn48VK1bg/vvvx9q1a5GXl4drrrkGF1xwwb4nyAQMMzJNXHba6NjLwSoeMqdo0ZJ2ANCiIS3rtzFE6zPuLqYlDxWN0XW00xqDzfIqpFgdrsg5Nc0NIOQKwPR5ye1UnZembmCnpRvzGbljQ2X0EG30Pv/cRctI/rC6iLQV+2gpQHf9imvs9ERuBlc9BYotOu+i5U8Nk7l5DEbyWqHzNSbRmApGKRGmybxfSCPbbDLvJgwjcq1iydJ1IKxb0WMS+cpd31pyIPxIB6yKzrVBB6yCH7XTq+XkdfMYGwAYjDx8MXJJ2x9oRdr88JC2Zo23kbajnLuSlu0Awi2i81xpUJvs46SZ29GmjY0bkTYub/5AS9K2jUnMTlTIPTsRkUndgFYIIpR2Wx9zPTQm85yM5LcGRv6Uu1a0+i3q1We2A9BkJ28HIvcMADQCwJegFevWNQfCh2RbAdgsP1yWmrTsYHytjfHTNuZ62hjJfgBomstILIfpQmVnGSMFrNHnobhoW4OGFfes3Rl5TvIbhBAOBmEa9BASaph/f57rpKWJ6+fRGWu46bIq5KKfy217G5K2ok10WssCtK1ZdsJ1ckXPJ9sRrxMpTvqJ4YpczWCuFVMXUVX63mBGnoDBtXWkqYuoTCXHTNTRNqyKqWHBJLYzbdWvi1S7hSYQCOCKK65At27d8NVXX+GDDz5AcXExbrvtNpSUlOCqq67C8OHDsXTpUkyePBlTpkzBTz/9VO2ECIJw+CN+RBCE2iA+RBCEVFQ7oNm2bRuOPfZYjBs3Dg6HA3l5eRg5ciSWLl2KTz75BLm5uRg9ejRsNhtOPvlkDBs2DHPnzt2faRcEIcMQPyIIQm0QHyIIQiqq3eWsdevWePbZZ5P++/jjj9GxY0esWbMG7du3T7K1bdsW8+fP3+cEOV3ZqacOulnNzvaqYvpKKHw85/bQTWdZ2XTWOZguJmC6nOka3TzsSGg6dUTzJDa1u+g2QDNNlzPFRjfzWkw3P40bDVaj9+l00f0hsnPokbizwky3DU/F8ZzRkZGdCSMkx7ropcLgWjOtmnU5s2rc5Ywxpu1yxhwz2rRb+b6JHDP1Nonr1CUHyo+4E1ybe9961hL7ozvPxLonUThA30d2ZrRqG9uNiX6OFHDXLpAwn5M85dxhuizk+hZxLoi5r7nz4M7fDjdp465F4nWMzSf+x90DHuYks5iM9TAZy50HFNpXsrdjmsfaTvcYqlgn6qftjL+uvG5dcsDqIk5PyqmdKRdtzHNiY+4frjsaALhddFclj5suN2jvAtg1+piKytS3nBXnYXdkJ01tCp0BFtelGoBiMXURJ+0oFAd9HpaD8QVO2od4PExXPZ0+D6erIsed0f07E47jYNIDk74/VIsuCxwKXfY7VPqBt3N1BhdTF2FOAQAcTBexxC5njmgdLTYl6yLu6tdFFMvietKlxrIsTJs2Da+88gpefvllvPjiiwiHw3j44Yfj67zxxht4+umn8emnn+7r7gVBOAIQPyIIQm0QHyIIQox9fnXp9XoxadIkrFq1Ci+//DIKCwvhdrtRVpb84XUgEEBWFvN1IsF9V3VGMOCF05WNO55eWbF8EFpoVqyjY73N25kWGi/zmquGLTTDT01uobl22grMuKErQgEv6rnKye32VwuNWcMWmu27c0nb2x9tIW3ZTAtNv17JLTR3z1uNu0cdi6A/IpZwVGtpoQEi981N//oZU2/uhFDAGz1m6m0crmxMePRn/qC1YH/7kQuxAADwMobgQiyoA1GAEtLWCPyX1PmgxS4aM9s2wu4abdcEf5K2ZiWJ+8yBvf5WhEsKAJTBvpXcDKAfzQh7GVsLxtaGNm1qRH+0u4n58H8TjiFtf6IpaduFChECJxy4B9fhLjwRFwXYjmbktjtBCxhwx2yJTaRtFOaRtpG73iVteI424RnGBuDZXbwdiLS6XLJ1K14oKEC4jBZcSVx3f7C/fcg/Jw9GMOiD0+nB5H8ujC/XvIWGLt/TtdCs30K30OzZzbTQ+Om6Ad9CQ9dhBnVLbqEZdesqzHuoI8IhL7IVejvLX4sWGg8jCuCizyPsopsT/ixuQNo+/pp2aFwLTefOyS0090z5CndN6otgMHIdcgvo9ISYFppwDVto8lS6DMkNFZM2k7mnsIc2AcDOcPVbaG6YsxrTLj8WIb+XbaG58fnV/EGj7FNAs2nTJlx55ZU46qijMH/+fOTn5wMA2rdvj6+//jpp3bVr16JdO0bChiAY8MYroknLBlOBZM+i5gGN30df1HIvfdCwt2YqZ2EmoAkFql7tUMCLUMCLMGiVs/QBDV1p5wIag6m0cypnwQD90HqZwlFhApqgr6ojDPq9CPoi+wsFuICGiRQ4FbwMCmiMSlFbKOBFMFAWPSa/3/3BgfAjiQGMH3qtAxono+cUZCoqABBilKzCSV3AktFB+xEDdEXFYvwBkOoZK4v8uHshXfZxcldcXY255S2mwwx3/mHQlSruWqS6jkGE4v/7mZP0MSdZzmSsj8lY7jxgMYEEdztytwaAMB+fVFq3LG1As784IHWRoA+BYHmVZZNRnDKYe91gLoyRJqDxB5h7z88oWTEKqDob0NC+JxysWoaHQ16Eg2XQuYAmWIuARmMCGublapixhYK0f/Ex+cYFNMFA1eMFg+UIBiL3UShE+4IgG9CQJlhMQBNS6fMIh5h6Y4A5YJpuqaEwnedmigpHyO9FyF9WJ3WRaosClJSU4JJLLkH37t3x3HPPxR0IAAwaNAi7d+/G888/j3A4jG+++Qbvv/8+zjvvvNqnUBCEwwbxI4Ig1AbxIYIgpKLaLTRvvfUWtm3bho8++ggLFy5Msv3www+YM2cOJk+ejOnTpyM/Px+33347evXqtc8JsmDAgg4r+sYitmxQ42UA0HU6mgwxIZvN5F9XtW1Oi/bvLaPfGGzeQ79NCTGvJFseTUfododZZd7uMGGZJsp1+sNLg3ljAgBmkGkGDzEfqWl0OK156PPfXkQ35ZYzzRCak75VbVkVb1ps7kjrkM3jhKFE0mEZ9BsMhTkm2xtNp89RsZhXpCrXWkTfrGk/dLPo1Cqx9xaGEp8q0beMKtFCpTJvIWvDgfIjxyLWRH02jsXqtK0o6XAw23PjjKRjN+huVWWgn2tu/Jog8+G7Ub+iZVFBNloB2FK/CSxkoZWxndxOoRtEInBZwIxDw4018ys6kLbfUEjafmG24/ItMb9jQhJr0C7euseNNeNnvpTluhO5mVY47r7ieiUo3JfgacahKajGODSxQx+F9A13tZfjqMqB8iFOeAGUwxn1vrFlm0WXGTamLqIzY4KYJt1SCwBHN6a7zJWV0756ewnTk4AZF6ZlE/p4VkIXL8sR2YflVGEpGvxh+hkxbXTrKABYQbpM1QymRwxTVoWY7TaX0nfvLsb3uuz08XR3Rb7ZomIAhqs+9KhYQojp426x6incoDFM2abTNoupU3Nj4qX76t7GdDlLbENxRK+bw1ABQyX360gjJpF07OqueNlll+Gyyy4j7Z07d8a8eXSfX0EQBPEjgiDUBvEhgiCkYv+8hhUEQRAEQRAEQTgASEAjCIIgCIIgCELGIgGNIAiCIAiCIAgZiwQ0giAIgiAIgiBkLBLQCIIgCIIgCIKQsewPVcVaUR4MIRAMQVdDScs2Rh9SU2gZRV2jNeacIX4wq4BCywzuZGSbc5rQ0oU6M7Bkwzx6n96EgSUd0UGmvGEFobACnZFX1tMMHgqTlkp0MNLMNka7z/TR8oO7yugL2SAvm7QZJn0tjITbODZvwBafD7Mj15ImdmBNk5E85EYSDHNSiQr9OFrMPR5ZgZHRViP3gBnNelMHYkrWhpH6Oqq2tELRhzQd8UvSPDtIYTXwMbK8fkbOF+Dll2tq24EmpM1g5D8TpYA1ZKEVgO1oBgPlyMmnZewbMTYAYMYHRflRtA/6A61IGyfNvAwnkrblOIG0cZLWiXmTFX3Xtx6t4gNjphvNnd4vvZ2zhnLi3HjJdlpxF8hnbABzV1UQO3Rj8GOmJq6biVghb+SnWEnLnIqszrjpMDM4pMVK3QJ+hS6LdjHlrZbP6HSH6PS469HlbYleoeHu0CKDYZYaDoR0B2w6M3xAuqEAmOEVNJUZkNKgn+nyAO2bt5YzviCH9r0Bi06nDxWDg5rRMsMHNwIx6W9GRpkdV5s5pmnS+9SZJzTADD3BDmTPDUYOAMw9oCbUR2NjnqpG5EfVxZR9cLvSQiMIgiAIgiAIQsYiAY0gCIIgCIIgCBmLBDSCIAiCIAiCIGQsEtAIgiAIgiAIgpCxSEAjCIIgCIIgCELGIgGNIAiCIAiCIAgZyyEn2xwMKwiGFCg2JWnZUGmpOLuNlphTGNk6xXSQNgAIMzp6ikHvt3kBLQdYsp2WQ8yy0ekpD1bI9ulRvTtfyEAwqEP30/J7VhqFPaedjmkN0PJ7FqN37A/QMoLl5XS+NalPX8e9Pno7VfUlzGvRqT/+v85IGuuMXCYX7ZtMxhqMLWzQxzMsRtKZSQsAqCqX2sj9YaiRe8gX0hEMRv7Tw6nvcR2MpGMG0AGrkuZNlDNrp2cnGpO2bTiK3ZaTX96DBjXarwO0H+FsblQ8K7aorOgWFECHH3koJrdr2ICXbVaY7N3hpPPuD7Qkbb+gA2njpJl/XdGdTgxT4qkNK04iRwHQGNiwszXKoo9ITm4ZuW2us5i2MfmaeD0qw8lEG8x5sLLNjIovABTwZgBAzLs1A5BO3D1N8XNIo+kBaGE/NM2WvMzURWwaXYapTImimLzAtc4Mk8AUKchvTMvNl++my2lVo4eCKAlXpNUZLW9LwxqCYQ2OAO17XMzQCwDgYIaYCFp0/pSH6bQWhegHZY+f3i5Lo+WeS4O0Rn04YRgELTofNqz4/wFO8tigy1zOF5gWfR3DzH0T0ul9Wkz9lqmmRNDp62gmSOOb9si8qeswwzoMPXVaDa36dRFpoREEQRAEQRAEIWORgEYQBEEQBEEQhIxFAhpBEARBEARBEDIWCWgEQRAEQRAEQchYJKARBEEQBEEQBCFjkYBGEARBEARBEISM5ZCTbTZ1Lf5LXOYklGGjdQttdkbujpEJBABVp20t8ultWzSkZZt3BOm0ZntoaUIzxbwZm7czUpGMbB/Ay26ajMQgJwdtmXTeOJgsb9KMPo/wVnpDl7viGjvdZvQ/E0o0pwzu3lE4OUxOn5BOj8XZGAlpKHSmKmkEUE1GRjsmnhpbxzTN+LxF3B9sOjOA5tiaNG+BltytLT7QEp8AUIxc0sbJ9mqMVCcn48nZnAglrKfF/9MQggO0/CcnywwAnCo2dx6czQDtD3TGBk5hOps2abaKtGhKxX9a9BHhpJmbYAdpa4ZtpO1obCZtjbGTtLmKSBN7LdLh5tWDAQCKVrEuo6abtG4mYjMMGIYBW1TCNrasEFL3AKDaGIlZZngJwMWnRaf9cUE9eruGubQ08d4Q/exluTj/b6WYtwBY8eE2UqGG+PoWJwfMFUcWU4abTF3EptLXo34evZ2+h7Y5XRXX3+k0o/+ZsJTYCdCVShszVILNoqWi7YxMv8b5dEImGeDzW2HqKWk3TqzHxNazTMAyoRB1EWUf6iLSQiMIgiAIgiAIQsYiAY0gCIIgCIIgCBmLBDSCIAiCIAiCIGQsEtAIgiAIgiAIgpCxSEAjCIIgCIIgCELGIgGNIAiCIAiCIAgZyz7JNq9evRoPPfQQVq1aBbvdjj59+mDixInIz8/HXXfdhTfffBN2e4VM4MSJEzFy5Mh9S5GhVvwSlk2Dlm5TNFpGzm2jJZRNTvoTQD1XLmnrysg26watP9i+bRPSZli0bF8gXGFTo/KP4bCGUNgGm42WZlTSxKy6QcsIKoysNadAGQgwsoZO+nj59elr9ecuWppQ1yuuv6bbov/ZoEfng0FantCmMjKTjKSsqjHSnSpt05h7FZw0cxqJ8XCYkb+NyY7qkbw3dB16dJ7UXeXVvmvFgfAj7mBUDtkZm6flkatDrnMvaWsMN7stJz/MyRZ74CdtDkaqsz1+I22FCTY1qmHcDmthwou2RVvI7bCGNgEAo0yMFq12kbY/j1rL7LIZadsB2o/uPJG25Thp+e6jEk4iCxqAvuia/xPKo9eIk1jmpJmPwp+krTN+qpENv9ImbGJsnNwzgFJOqT6KYgD50XU5qd3EdeuaA+FDVL3il7gMZhgAjZG0dTlpHx5MU07nObNIW736DtIWMOg6RaOjabl5h0U/J5qeUBfRIvOqHoamh+Hiyg2dl6Y2gvTNZHcx+eqiq7EKU/bnOOg6Zf2G9PFKw3SeJvplR7R67UAIZvR/LUDLKDsVer9ORtLZo9GSzlkqbWMUnQGdG5cjTZ2SGe4iaXgJPbJeSDcQ0g1yv0x1quq61V0xEAjgiiuuQLdu3fDVV1/hgw8+QHFxMW677TYAwMqVK3Hffffhhx9+iP/2OZgRBOGwRvyIIAi1QXyIIAipqHZAs23bNhx77LEYN24cHA4H8vLyMHLkSCxduhShUAi///47OnXqtD/TKghChiN+RBCE2iA+RBCEVFS7y1nr1q3x7LPPJv338ccfo2PHjli9ejV0Xcf06dOxfPly5OTk4LzzzsMVV1wBVd23z3Rc7uyUU4vpcuZy0s1jTjdtM9PEc3Y7Pay0Zqf3qzFptWt09xSuyxmMivZBpys7aaoxXc7UNOdoMV3O7My1szF3josZndftoY/ndNNN4J4sOk+d7oo2ycp5AwCGcWC7nHEjF3P3hlGLLmeqjWnmjR7TGX2WYlMAZJezpHXqkAPlR4AcYr5mKMwQ8xro7iAAYGO6pDmYEcKdoLuS0Bb+eGrCeajRdKvx9DP5lG7Ed9oFsT0pubzj8sbNFF05zAGzGX+YlXCSnui8J+E/N3OSLvZa0V1pbaB9Hn89mAynD4c0tyqUajwqSk5O0rQ669YlB8qHOKJlSOUpTKZ8Z+oFdif3kHAXDbBs9IUzbPS9Zxn0g2vXmHuWGZ1dNSt8iMOZnTxl+iAqaboqgSlvNaaOZzrpZ8hp0vnmdtHbuZ20D/W46TqMy1nhl5zRdDkT0qeZ9D3Adzmjt3NodL7amS6QcNHnYYWZ7Yyad383E7prOqJ1jNiU6nLm8FS/LqJYlrXPveUty8K0adPwyiuv4OWXX8bu3bsxe/ZsXHvttejWrRt+/fVXjBs3DpdccgmuuOKKfd29IAhHAOJHBEGoDeJDBEGIsc8BjdfrxaRJk7Bq1So8+eSTKCwsTLnes88+iwULFuCtt97apwT93/kdEPB74XJn4/H5v8SXa9pCk1WrFppc0sa10Oj7oYUmaFR8lOx0ZeO+p3/CHVd1QTDgPeRaaErK6e0276aP1/Io+o3J6vWlpK1b6+QWmklP/YwpV3dCMOAFcGi10HD3Rm1aaNi3IgktNPfNWYc7Lm+DoN8bTSzdQnPfv39nj1kb9rcfKQ8eDQDIcm6OztMfuVaHnc5GpG078/F6xN60Rtv+yWznYL7oLMRq0tYJv8TnVWShEIvxG3rDRDmOLdpAbqesJE0RtjO2frRpabPOpO1r9CZtixnbkiBty3Z6SVuzhI/3PdDwAU7GUCyBL+oHCrCV3LYp8+F/UyZzOmIVaeuBb0lb/n+Yj33/R5uwiLEB+ON73g5EWl1abN2KjQUFsMr45yq27v5gf/uQhyYUIhTwwuHKxq2P/BZfrnELjZsup0NpW2jySBvXQhOocQsN/Zyo5s6K9ZzZuPGfP+LRyccjFPQim2uhYcSCooklTVo9Ol+DTEvLn3767f76Enq7+g1oKYtN2+h7vlNBQk8apwcP3LYAtz0wBMGoSI1m0n7bxbbQ0Nu5NVrwxqOUkzYUMy00ZUxdhKnfAUA5oyxSuYXmhjc3YNp5rRDye9kWmhveXMceM8Y+qZxt2rQJV155JY466ijMnz8f+fmRi/7ZZ59h9+7dGDVqVHzdUCgEl4tXtUhFwO9FwFdWZZkLaGDQmW9jKonpVM5Mnc4eTa+ZypmpMRVaTuXMqOpgggEvAn4vq3JWm4DGZAIag7lzAn56O7+PPl7QT+eNr5x2sKm2Cwa8CPoj9xEX0BhHQEBjVDpm0O9FIJo3pMrZfuRA+JHkAKYMtQ1oLKYblwGm0ACgM2plIdAV0yBTiFmMyhl3PBNVnyMT5dH/mTxKp37F9JblFPO4vOPyxg/aj5SxEn3081ee4iR9MOL/+5mTDDDXKsReK059j7seTEBDHw5pblUw4lYp1i1LG9DsLw6EDwkFvAgGyqouMwGNxdRFLJUup4PsAwRYBh18GIySWZAJaEyNqdBa9I2imVV9SCjojfz2U0BjY15ahxT6egSD9Hb+AO0nHEHa3/v89DMbCFZ9+IJBHwLBSH7amIBGYQIa7qHWmIDGrtD1JgSYul+AqYsw9TsACPmqF9DE1/d7EfKVpVVPqw7VDmhKSkpwySWXoFevXpg8eXJSf1TLsjBlyhS0aNECvXr1wo8//ogXX3wRkyZN2ucEKbDiv8Rly2Iqnkw+cCq520t4J1I/h47uszT6xly7sYS0tWxB7zPgpxO7Y0+Fg3F7Iie8ZZsPfl85nC7G2dH3bNTOtdAwb3eYJpo9pfR5lIdpJ7JLo51ouZ+upBUXV6TTFYqcT0mJjkA0eDIt5rsdO33zOGz0eTgczHcy9ppJjOsB+n40uWAHgMpcKyXWjzZhGvuvcrATPx5TaNeGA+VHPOXR9Dtj87U7n1xnMWkz0rhRGxMNuJkKbS7oY3qY7TjZ5jbl6xOWcoAsoFX5BgBlUDgp4FrINuNo2tTmKPrN2zYcRdp2MrLNe5wNSBuXp4myzJFvYvqiM1bGgxVOtrkl/qjWfitTyFyr/GVM0MJdD/pwwB7GBjChcAWxJzaA9E/V/hjo7kD5EM204r/EZS5oYd4twqHQfmC3l895l4f2MXaVLt92b6frIgVNmVafFBXzGGV7K/bpdEXugJIdpQgGyuBn3nSmiIOSj+lnApr6zIu3XCZfmQA+rNN3b4ApTxGkg3ijrOJaGOHI/g1vGYxAJCEK84QpNqYupnGtN/Q+bUF6n0wcDIN58Wwy9VQAsJiXr1bCNz2xeUtRYCkKTOLFs8HIpFem2gHNW2+9hW3btuGjjz7CwoULk2w//PADJk2ahLvvvhs7duxAw4YNcd111+Gcc86pdkIEQTj8ET8iCEJtEB8iCEIqqh3QXHbZZbjssstI+6hRo5KaeQVBECojfkQQhNogPkQQhFTsjxZhQRAEQRAEQRCEA4IENIIgCIIgCIIgZCwS0AiCIAiCIAiCkLFIQCMIgiAIgiAIQsayT+PQHAhsmhr/JS5z0m2WQUso68x4MWWMDjkAFJXQgzk2a0xn3c69tIygL7SDtB1zFC1TqiErYd4Tn2owEQ7S5x/Q00jeMdrfnKi17qWt2/fQxwwz8tsNmYHHNAc9joDLXaEZ73RlR6c5QPRYNhstcasykpcKM9aFjZF0VhgdcV1nxrZhxsRJN/4tJb8MVGi/W1EpZss047LMCiGxqPDKjIc8SuyS50fn92n44KrkaIyEbv3d7LYOZgyBHGaskYaMxi63XfuS9aTNnjjmpgbgRMD1KyLjzHBSwOlkm2m3xso2NzqG1nItbEVLGu8BLc1cjFzS1oDJ05aoGFjUjoi/6YwVCEfHw2nFSDNzss1tSjaSNjs9BirwI2PjJLY30aZwEbMdqjdaU0zQ1ov0wxMd+FGu6g4NCmxQoEUl82PLBuOLrTAztgszJkqIsQFAmXcvacvPo4dt8JXSZd+fYVrut2V+PdIW8leUxbboc2ILuGD4w9AZmeCQN40TDjHlJnMjGTpd//EG6Ds6xFxHxUkPdJplp+/6XFfFOTidkfn6ThWuaNuBhxn7x6XQ18Nl0DY7M+6UwownY/mY4SV05n5kxvUDgDBXV0+wadE6iGGa0E2THGuPqTJWQVpoBEEQBEEQBEHIWCSgEQRBEARBEAQhY5GARhAEQRAEQRCEjEUCGkEQBEEQBEEQMhYJaARBEARBEARByFgkoBEEQRAEQRAEIWM55GSbTdOK/xKXFY2WgjMVWtPPVOhTbNkki7QBwN5iWpq4ecNc0lbfTevMFZXSMoJNcmkZvTZNG8XnHa5Iuo8vbIhQwA1v2S5yu2CaS6xpdEzLySHqjExwTg59PF+Qvo65ebQ08+7N9PFKyipkFF1GZL7UayDgi8y3PpqWYNRNWkbRZDRJORVlnZHK1sPMfUyfIpQ07x4UJj1KNLGJ09i8aaa+Vy3i/4whWGmeydvqYKdvE+SXM5LOAPJctD3opJ/dkIuWMc8poX2Twkn6JtrsAE4EsAoRjXZOmvl3xgYAOxkbrUbP2trn0/LTe+vnkrYy0A6Ik20uRIVMdEwmvytWwkA5AKAN1pHbNlpDy0+zEsucbDN9+vy1YmSbd5Qw2wGgxWEriJW2fhzess2mYcV/icucpL3JDINgmXRuHJ3jJm0AUMr4mGb16G3zGPnh8tJy0tbARjvMo5vmxeft0WESCpvkIhywwVdEy0uH7GnqIkyZY7hpf2d46MLPHabPY49J79OZQ2+n76DrRWZ5xT5j9QfTVw4zEPEPjZsww0T4mf1yRQzz0OrM0CQGI6NtcftkhkIB+LpR4vATsXnLsmBZFii15zQq0UlIC40gCIIgCIIgCBmLBDSCIAiCIAiCIGQsEtAIgiAIgiAIgpCxSEAjCIIgCIIgCELGIgGNIAiCIAiCIAgZiwQ0giAIgiAIgiBkLBLQCIIgCIIgCIKQsRxy49CEwwZCYQNa2Ehe1miNckWjT8MforXGXS5edb9JI1rD3WGjBf1zs+hjul10Ws3wbtLmtyoU/01E9MoDgb0I+r2wTFq/HMz4GQCgW7TeusVs7LDRmvqtmtLjZ6iMhr+iMGNrmHS+aagQabepjug0BJsaGYwkHKbjdpMRTTcMejtuPBlON11V6fNQmcFkLEqkPYrCjLMS18NPmMbmA0Eiz1X6WmQE5ZXmazkOTW1ghsICPfIS4AJzDejhVPgxSjYkzMfc20ZExurZUHX1OJsZG8CPQ8Pt9xjaZGfGqGl1/B+kbQ8akrYm2EHa2ieMQ4PoWDZtsAZAZOyw/J+ZwSBW0ib8zNi4a7WtZrZS5lrQZx/dNo0dqKg0lCFt8XLoVTD2AT1sIhwyoWpm0rLGFGIKUy4GA3RuOR1B0gYADbLpMtVh0GMg5TJXyONixu8rKyZtgYQx6IxQZD5QWopwwAuLOUfQw95E9sWUqRYzKJzdTpfTzXPoO7Cphy4UfAqd2ByLyVNUjDPoiCbLo3phU6P/M8NVmT6mTlHOjNHDjCfDVQ0Vg77+FlOJYYbZixyTKWv1BJsanQ+bkV8olDpfTVs6L5Owz2qvKQiCIAiCIAiCcIghAY0gCIIgCIIgCBmLBDSCIAiCIAiCIGQsEtAIgiAIgiAIgpCxSEAjCIIgCIIgCELGIgGNIAiCIAiCIAgZyz6rKi5ZsgSPPvoo1q1bB7fbjcGDB2PChAlwuVxYsWIF7r//fqxduxZ5eXm45pprcMEFF+zT/kPhyE8LJy8rtGpfZAUCl5PTCeZ2CkChNe/K/LSEp8mkRw/R6QkxUnlKgmy1KxSR2ysq8SPg90FVGfnFMK9XazEaeyojhx1mFGVVxUfa3E76eBp9GmicR8tW6kZF3jjdkTysl2PBaYvMhxg5SEWhhXODAfr6h8P0vWMweaqpdFo0W83fL5iMsqFl2apMragMtkHIQRuMTHZt2d8+RBCEw5/97UfCeuSn6snLOlOlQJiTZq5FXYSRivYx5ZSp0+kx/PQ+w4yEsJKQAQ4jUi6XlgcR8geghpghEkK83q8VYuoiTrpyoDNq6oqdNqpOOq1MlQp5ObTNnlCeOpyR+WyPhZAWmQ+zMsoO0hYO0tdRDzF1kSBT32Dqm5ZJ3xvphpCwuFvZSshYU6uYmhqo3ZpW9esi+1SDKioqwpgxY/C3v/0Ny5Ytw9tvv43vvvsOTz/9NEpKSnDVVVdh+PDhWLp0KSZPnowpU6bgp59+2pdDCIJwGCM+RBCE2iJ+RBCEyuzTa9j8/HwsXrwY2dnZsCwLxcXFCAaDyM/PxyeffILc3FyMHj0aAHDyySdj2LBhmDt3Lrp06bJfEi8IQmYhPkQQhNoifkQQhMrsc7+S7OxsAED//v2xY8cOnHjiiRgxYgSmTZuG9u3bJ63btm1bzJ8/f5/27/Jkp5wq3BDzzDDgTqbLmZNpxgQAF2N3OpnRWTW6P5am0elRmaa8xPN3urOTpmyXMzuXb3zzocqkldurotDn72S6nDGDLMNQ6C5nmkHnTWS/zHkoTCMlcx6ajWnmtZguZ0zGqWyXM76Zl+tyFnvIU+WNqaXerythnbpmf/sQAICak3r+YLA/vlTkXBf9qADOhHlHTvLUzWyXlSY93O3C7ZdLK/dogr6mNnhIm8aeSOI+sytNAahMYrmS1MnYuLyhT4PPb+Z2V+leLQCqVyGw5eQkTauz7v5gf/sRpys75VRRalgXYcpim4N3Eg4nfWUcNvqY3CjrhsWUi1Y1u5xF8yQ2VTWmy5nFn6PFnIfqprdVmGdIcdFlONflzNLonXJnYTcrHjC7MztpCgBcdUzhrkeIvo6qydRFiPIdADSma1htupyB2W9iLzdntG4fm1rEKcbs1UGxLObOZQgEAigpKcHNN98Mp9OJJk2aIBwO4+GHH46v88Ybb+Dpp5/Gp59+WpNDCIJwGCM+RBCE2iJ+RBAEoAYtNDFcLhdcLhcmTJiACy64ABdddBHKysqS1gkEAsjKSvdqL5kxQwsR8Hnh8mRj9ge/xZczqoWmhqIA4X1ooXngpdW47aJjEfR7+RYa9gvGA99Cw4kC1LSFRq/UQnPHM6tw35UdEfR7o/utWQtNMMBcR04U4BBroYml1OnOxgMvr8FtF7aL542P+GjQ5c7GY2+sZY9ZW/aXDwEA/FwQmXbaGpk3y/j19yf7Q1+hiLGtY2wbE+YdOcDErcCDBUCojN+OswHALsbWi7H1qdl227rkk7Yf0I20NcZO0tYGaxKWspGP9ShCawCRZyX/F+br41W0Cb8yNi5f/2Rs22lTKbPduiCzzzSHjGHLycFftm7FJwUF0Mv45yq27v5kf/mRyWPaIhjwwunKxj9nr40vH4wWGmdNW2iMGooC+KvfQnPtM2sw48p2CAW8vCiAL00LDScKkMO00DCNgEo200JTjyn7a9xCU/HVv92Zjavv+BVP3XccwsGID7FzogAB+nropYwoQDlTF2GEGGraQhNM00LDaBRUaaGZ9N56TDm7NYI+LwJMC809C9azx4yxT0Xt999/j9tuuw3vvfceHI5I01ooFILdbkfbtm3x9ddfJ62/du1atGvXbl8OgYDPC7+vrMpyTQMay2Cazgw+oAGn9MQoWe1vlbMYQb8XAX9ZGpWzNE6EVTljnAhXMWfU4VTmeNzlqG5AEyPo9yLoL4vulzsPLqDZHypnpCmNylnNAxq90qZBvxeB6DMWYFRQ9gcHwocASA5gzLKDG9DwQoM1g1OSYRQIkapCGyoDgmWAn9muPE16vIyN2y+XVibfLKavmg5aZdFgTyTVPeKt+N9kAhrm+UuZ5zG4vKFPg89v5lY30wQ03GlUWbesLG1As784EH4kGKgoQxKXaxrQgKmLmCZfTiuc0pOdqYtwKmdMBbq6AU2MUMCLkL9s/wU0TNmoMFmj2GrW5SzE1PbZUjrFAxYOehEKRu+jGgY0YT8T0PhrqnJGp4ULaEJpAppgNQOa+Po+L4K+MgS5sqCa7FNAU1hYiEAggKlTp+Kmm27Crl278NBDD+H888/HGWecgalTp+L555/H6NGjsXz5crz//vuYNWvWPiVINzTougY9WruNLUNhJObY/GU6MJt8QGMy2sQBH9OnkUmPjfOFTGASTnB4WtS5hS0bwqYNCtM3VVPSyDYzieWy1WIea0OhO3+bjKyljekLqjM1HzPBZEaDKX8ghEA0ILGrtDSzqjIOmJFR1Ji+qVxLk8m03nDyi7rBy3qazKOsx3QUo8FtIKwiEJ2nYi8zzbNRUw6EDxEE4fDmQPgRw1Djv8RlDr6qx3zAlCagsZjAJMhVvpkEcRVartquJ6RVjZYTuqlBNzUozHmoaeoiXGL5ugjz0poZloFpvEKQCSBV0C82DL2iMmJGe6oEQmGEoi+5rQD9IoYNBpmX66pK25iRN2AyL151ZriPoM7XRYLMB55JvVeiz1LQUBHUVaYuUv2PUPcpoMnKysKzzz6LBx54AH369EFOTg6GDRuGcePGweFwYM6cOZg8eTKmT5+O/Px83H777ejVi+tvIAjCkYT4EEEQaov4EUEQKrPPvbvbtm2LOXPmpLR17twZ8+bNq3WiBEE4fBEfIghCbRE/IghCIvtDUFQQBEEQBEEQBOGAIAGNIAiCIAiCIAgZiwQ0giAIgiAIgiBkLPtjhIRa4Y6OClp5yquc0aoLDjutcua080pOdmaUXVXjVM4YBSxG8tFkFLf0BHnh2CjusanCqHxoaXRXLEZ1S2UkjblY2GLOQ1PofNM0ZowaTuUs4S52RvPEmTDSvU2l7wFOIcQ06PQozP3Iinoy94bJKJnVhcpZ5fsGACjxnMR1MhI1J/X8wWB/vDbiXBczoH2S6KMjJ3nKjVqfbggP7nbh9sullck3BfQ1tYFWWdTYE0ncZ3alKQCVSSxXkjJCm2ze0KfB5zdzu6uM0BZQvQqBLScnaVqddTORymWJsxo+0WL01O02pi7CSvYDGlPfUFiVM062l6mLMAOmJQ6D4IjmSWyqMOMSqGoalTNGWUtxMw7PzaicuRg1Vu651Jg6A+O07FqFzeHMTpoCgIPZVlWY4TcYRTKumsaMdgGTGb9IYa5VuroIVzglqpw5o3X72JTabcxeHRTL4kWPBUEQBEEQBEEQDlWky5kgCIIgCIIgCBmLBDSCIAiCIAiCIGQsEtAIgiAIgiAIgpCxSEAjCIIgCIIgCELGIgGNIAiCIAiCIAgZiwQ0giAIgiAIgiBkLBLQCIIgCIIgCIKQsUhAIwiCIAiCIAhCxiIBjSAIgiAIgiAIGcshGdDs2bMHY8eOxYknnoiePXti8uTJ0HX9YCfroFJUVIRBgwbh22+/jf+3YsUKXHDBBejWrRsGDhyIN9544yCm8MCzevVqXHbZZejRowf69OmDW265BUVFRQAkb5YsWYILLrgA3bt3R58+fXDfffchEAgAODLyRnxIVcSHVEV8CI/4EfEjlRE/UhXxIzQH1IdYhyAXXnihddNNN1k+n8/atGmTddZZZ1nPPPPMwU7WQWPZsmXW6aefbrVv39765ptvLMuyrOLiYqtHjx7Wyy+/bIXDYWvx4sVWt27drBUrVhzk1B4Y/H6/1adPH+vxxx+3gsGgVVRUZF155ZXWmDFjjvi82bNnj9W5c2frzTfftAzDsHbs2GENHTrUevzxx4+YvBEfkoz4kKqID+ERPyJ+pDLiR6oifoTmQPuQQ66FZuPGjfjuu+8wYcIEuN1uHH300Rg7dizmzp17sJN2UHj77bdx8803Y/z48Un/f/LJJ8jNzcXo0aNhs9lw8sknY9iwYUdMPm3btg3HHnssxo0bB4fDgby8PIwcORJLly494vMmPz8fixcvxogRI6AoCoqLixEMBpGfn39E5I34kGTEh6RGfAiP+BHxI4mIH0mN+BGaA+1DDrmAZs2aNcjNzUWTJk3i/7Vp0wbbtm1DaWnpQUzZwaFv37749NNPMWTIkKT/16xZg/bt2yf917ZtW6xevfpAJu+g0bp1azz77LPQNC3+38cff4yOHTse8XkDANnZ2QCA/v37Y9iwYWjUqBFGjBhxROSN+JBkxIekRnxIesSPiB+JIX4kNeJHeA6kDznkApry8nK43e6k/2LLPp/vYCTpoNKoUSPYbLYq/6fKJ5fLdUTmkWVZeOyxx/Df//4X//znPyVvEvjkk0+waNEiqKqK66+//ojIG/EhyYgPSY/4EB7xIxHEj4gf4RA/QnMgfMghF9B4PB74/f6k/2LLWVlZByNJhyRutzv+YVWMQCBwxOWR1+vF9ddfj/fffx8vv/wyCgsLJW8ScLlcaNKkCSZMmIAvv/zyiMgb8SHV40i4F6qD+JD0iB+JIH6kKkfCvVAdxI/wHAgfcsgFNO3atUNxcTF2794d/2/dunVo2rQpcnJyDmLKDi3at2+PNWvWJP23du1atGvX7iCl6MCzadMmnHfeefB6vZg/fz4KCwsBSN58//33GDx4MEKhUPy/UCgEu92Otm3bHvZ5Iz6kehzpzwkgPoRD/Ij4keogz4r4EYoD7UMOuYCmZcuWOOGEE/DAAw/A6/Vi8+bNmDVrFs4///yDnbRDikGDBmH37t14/vnnEQ6H8c033+D999/Heeedd7CTdkAoKSnBJZdcgu7du+O5555Dfn5+3Hak501hYSECgQCmTp2KUCiErVu34qGHHsL555+PM84447DPG/Eh1eNIf07Eh/CIHxE/Uh2O9GdF/AjNgfYhimVZVh2mv07YvXs37r33Xnz77bdQVRXDhw/HzTffnPTR1ZFIYWEhXnzxRfTs2RMAsHLlSkyePBm///478vPzMXbsWIwYMeIgp/LA8O9//xsPPvgg3G43FEVJsv3www9HdN4AkTcdDzzwAFauXImcnBwMGzYsrsJyJOSN+JDUiA+pQHxIesSPiB9JhfiRCsSP8BxIH3JIBjSCIAiCIAiCIAjV4ZDrciYIgiAIgiAIglBdJKARBEEQBEEQBCFjkYBGEARBEARBEISMRQIaQRAEQRAEQRAyFgloBEEQBEEQBEHIWCSgEQRBEARBEAQhY5GARhAEQRAEQRCEjEUCGkEQBEEQBEEQMhYJaARBEARBEARByFgkoBEEQRAEQRAEIWORgEYQBEEQBEEQhIxFAhpBEARBEARBEDKW/welJ3wLUrlhzQAAAABJRU5ErkJggg==",
|
||
"text/plain": [
|
||
"<Figure size 1000x700 with 3 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"# do not remove this cell\n",
|
||
"# run this cell before moving on\n",
|
||
"\n",
|
||
"rand_idx = torch.randint(0, 10000, size=[1]) # pick a random index from the test set\n",
|
||
"\n",
|
||
"x = cifar_test[rand_idx][0] # test image\n",
|
||
"y = cifar_test[rand_idx][1] # associated test label\n",
|
||
"\n",
|
||
"cifar10_model.eval()\n",
|
||
"scores = cifar10_model(x.unsqueeze(0)) # get the raw scores\n",
|
||
"probs = scores.data.squeeze()\n",
|
||
"probs, idx = probs.sort(0, True)\n",
|
||
"\n",
|
||
"print('true class: ', cifar10_classes[y])\n",
|
||
"print('predicated class: ', cifar10_classes[idx[0]])\n",
|
||
"\n",
|
||
"assert y == idx[0], \"We want to visualize what the model is focusing on for a correct prediction, run again for another random sample!\"\n",
|
||
"\n",
|
||
"# if the printed prediction and label are different, it means the model misclassified it. \n",
|
||
"# Rerun this cell until you get the same class printed for both. It will help for the visualisation later.\n",
|
||
"\n",
|
||
"# Get the first Linear layer's weights and final Feature Map\n",
|
||
"params = list(cifar10_model.fc.parameters()) # access the model layers\n",
|
||
"weight = params[0].data # grab the first layer's weights\n",
|
||
"\n",
|
||
"feature_maps = cifar10_model.conv(x.unsqueeze(0))\n",
|
||
"\n",
|
||
"# Creating the heatmap\n",
|
||
"heatmap, _ = get_CAM(feature_maps, weight, idx[0])\n",
|
||
" \n",
|
||
"plot_cam(x, heatmap)\n",
|
||
"# Red \"hot\" areas represent where the model is focusing on more\n",
|
||
"# if the shading isn't that great, rerun the cell to get another random sample"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a4c0a351",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Submission\n",
|
||
"\n",
|
||
"Once you are done, please submit your work to Coursemology, by copying the right snippets of code into the corresponding box that says __Your answer__ and click __Save__. After you save, you can make changes to your\n",
|
||
"submission.\n",
|
||
"\n",
|
||
"Once you are satisfied with what you have uploaded, click __Finalize submission__. \n",
|
||
"\n",
|
||
"\n",
|
||
"__Note:__ Once your submission is finalized, it is considered to be submitted for grading and cannot be changed. If you need to undo this action, you will have to email your assigned tutor for help. Please do not finalize your submission until you are sure that you want to submit your solutions for grading. "
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.11.8"
|
||
},
|
||
"vscode": {
|
||
"interpreter": {
|
||
"hash": "5c7b89af1651d0b8571dde13640ecdccf7d5a6204171d6ab33e7c296e100e08a"
|
||
}
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|