{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Intro to neural net training with autograd\n", "\n", "In this notebook, we'll practice\n", "\n", "* using the **autograd** Python package to compute gradients\n", "* using gradient descent to train a basic linear regression (a NN with 0 hidden layers)\n", "* using gradient descent to train a basic neural network for regression (NN with 1+ hidden layers)\n", "\n", "\n", "### Requirements:\n", "\n", "Follow the Python environment setup instructions here:\n", "https://www.cs.tufts.edu/comp/150BDL/2018f/setup_python_env.html\n", "\n", "All the specific Python packages you'll need are in the 'bdl_basic_env' conda environment:\n", "https://www.cs.tufts.edu/comp/150BDL/2018f/conda_envs/bdl_basic_env.yml" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pickle\n", "import copy\n", "import time" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "## Import plotting tools\n", "\n", "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "## Import numpy\n", "import numpy as np\n", "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "## Import autograd\n", "import autograd.numpy as ag_np\n", "import autograd" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# PART 1: Using autograd's 'grad' function on univariate functions\n", "\n", "Suppose we have a mathematical function of interest $f(x)$. For now, we'll work with functions that have a scalar input and scalar output. \n", "\n", "Then we can of course ask: what is the derivative (aka *gradient*) of this function:\n", "\n", "$$\n", "g(x) \\triangleq \\frac{\\partial}{\\partial x} f(x)\n", "$$\n", "\n", "Instead of computing this gradient by hand via calculus/algebra, we can use autograd to do it for us.\n", "\n", "First, we need to implement the math function $f(x)$ as a **Python function** `f`.\n", "\n", "\n", "The Python function `f` needs to satisfy the following requirements:\n", "* INPUT 'x': scalar float\n", "* OUTPUT 'f(x)': scalar float\n", "* All internal operations are composed of calls to functions from `ag_np`, the `autograd` version of numpy\n", "\n", "**Important:**\n", "* You might be used to importing numpy as `import numpy as np`, and then using this shorthand for `np.cos(0.0)` or `np.square(5.0)` etc.\n", "* For autograd to work, you need to instead use **autograd's** provided numpy wrapper interface: `from autograd.numpy as ag_np`\n", "* The `ag_np` module has the same API as `numpy`, so you can call `ag_np.cos(0.0)`, `ag_np.square(5.0)`, etc.\n", "\n", "Now, if `f` meeds the above requirements, we can create a Python function `g` to compute derivatives of $f(x)$ by calling `autograd.grad`:\n", "\n", "```\n", "g = autograd.grad(f)\n", "```\n", "\n", "The symbol `g` is now a **Python function** that takes the same input as `f`, but produces the derivative at a given input.\n", "\n" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "def f(x):\n", " return ag_np.square(x)\n", "\n", "g = autograd.grad(f)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.0" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# 'g' is just a function. You can call it as usual, by providing a possible scalar float input\n", "\n", "g(0.0)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[-2.0, 2.0]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[g(-1.0), g(1.0)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Plot to demonstrate the gradient function side-by-side with original function" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Text(0.5,1,u'gradient of f(x)')" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmYU+X99/H3dxZAFMEF1GopVvmprb1+IhSNIg6ytS4Fn+plrX3AFUfU2lprq7WVKlp9auvyw4WhqOC+b4ggSyPIHFFafy64a7UiLqhYV2b9Pn+ckzEMGWaGJJPt87quXJOTc5JzT86db+7cq7k7IiJS/MpynQAREekaCvgiIiVCAV9EpEQo4IuIlAgFfBGREqGALyJSIhTwM8DMdjOzp83sMzP7efTYGDO7v4PPf9LMvpvdVG70/LuY2RQz+06u0iCFLco/N0f3+5vZ52ZWngfp2uCz2Wr/dma2JNr/l+ix75jZig6+/r1m9oNMpztbFPAz42wg7u693P2q6LGLgUs6+PzLgAuykrJ2mNn2wKPACOBRM+vfav8hZva4mX1iZu+Z2Qwz65WLtEphcPd/u/sW7t6U7muZWdzMTkzjJVJ9NpNNAj4EtnT3X0WPXUj4meyIS4CL0khfl1LAz4xvASsTG2b2faC3uz/Rwec/CIwwsx2ykbi2mNmWwCPAre5+IHA5MM/Mtkk6rDcwFfgGsAewE/DnrkyndC0zq8h1GjJovc9mG/tf8GgEavQZHAF06Ne5uz8JbGlmQ9JNaJdwd93SuAGLgSZgHfA58F/AH4C/JR2zH2Ep4pvR9n8DnwC7Jx2zAJiY4bTNBf6StH0HcH10vzvwd+CcVs+ZDNQCm7fxmv8HeC7X77tunc4LewNPA58Bd0V5YWq0rwpYBfwGeA+4CdgKmAOsAdZG93dKer2dgcei11sATANujvYNAByoiLZ7AzOBd4F3CAsQ5dG+Y4HHCUvUa4F/AT+M9l3U6rM1rY3/7UeEQf0TIA7sET2+wWez1fNuBBqA+mj/KGACsDDpmF2Aj4G9o+1vRJ/lqqRjZgDn5/oadygf5DoBxXCLMtmJSdt3Ab9udcxFUQbcDHgWOK3V/quAv7bx+sOizNzWbVgbz9se+AA4CDgGeAPoleb/egVwe67fc906dc26AW8BZwCVhF/a9a0CfiNwKWFBYDNgG+DHQE+gV5Sn7096zQD4a3T88CjwtxXw7wemA5sD/YAngZOjfcdGQfckoBw4BVgNWLR/vc9Wiv/tv4AvgNHR/3Y28BrQrYPPvzHxPkTbfwaubnXMScCL0XsxH7is1f4zgXtzfZ07ciumn275pA/hByDZFOAJwsz+DnB1q/2fASmrdNz98eg1O8Xd3zOzamAW4Yd4vLu3TleHmdloYCKwz6a+huTEvkAFcJWHEepeM3uy1THNhKXUumj7K+CexE4zu4jwFyFRO8/3gVHR8UvM7KFUJzaz7YAfAn3c/SvgCzO7nLDufHp02FvuPiM6fhZwDbAd4a+N9hwFPOzuC6LnX0b4xbYfYbDvrD7AR8kPuPsMMzsMWE74RfajVs/5jE34fOaC6vCzYy1hqaiFuzcQlib2JKxmaT1rXS/C0nqmzSEsOb0cfXFsEjPbF7gVOMLdX8lU4qRLfAN4p1Wee7vVMWvcfV1iw8x6mtl0M3vLzD4FlgB9op433wDWuvsXSc9/q41zf4uw5P1u1PD/CWGg75d0TEtgd/cvo7tbdOJ/azm3uzdH/9uOHXx+axt8diMzCD+7/5P0pZiQrc9uxingZ8ezhD81W5jZjsD5wA3AX8yse6vn7AE8k+rFzOyAqJtbW7cDNpKWiwh/ju5gZkdvyj9jZoMIG5aPd/dFm/IaklPvAjuamSU99s1Wx7QugPwK2A3Yx923JKy2AbDo9bYys82Tju9Pam8DdcC27t4num3p7h3thtzedL6rCb9UwsSF/+M3CX9Fb4pUn90tCKsyZwJTzGzrVs9p87ObbxTws2MucGBiI8qENxJmmBMIPzAXJu3vDgwmbPzagLsv9bCbW1u3pameZ2bDgeMIG6ImAP8TffF0mJntCcwDTnf3lD/bJe8FhI2Xp5lZhZmNA4a285xehNU6n0QB7vzEDnd/C1gB/NHMupnZMOCwVC/i7u8Sdvv9i5ltaWZl0biPA1Mdn8L7wLc3sv9O4BAzG2lmlYRfVHWEHQ82xQJgbzPrkfTYlcA/3P1E4GHgulbPOZCwt1veU8DPAnf/J/AfM0vUdf+csE7y99HP6uOA45JK5j8i7Cu8OlNpiLpcziZsHH4nqs6ZCdzQqqTXnl8BfYGZSb8oNtbNTfKMu9cTNtSeQFj18DPCqr7WVRPJriBs9/mQsO1pXqv9PyVsy/mY8Mtg9kZeawJhw/ELhFUmd9NGe1UKVwJHmNlaM9ugH727v0z4//xPlNbDgMOi/7nT3P19ws4V4wCiL8cfANXRIWcSfiEcE+3/PvCFh90z816iJVwyzMzGAJPdfXwHjl0OnODuz2c/ZSItee46d78h12nJN9GI81nA0BRtba2PvQeY6e5zuyRxaVLAFykBURXKy4Sl4GMIqyW+HVW5SIlQt0yR0rAbYX33FsDrhL2tFOxLjEr4IiIlQo22IiIlIq+qdLbddlsfMGBArpMhReof//jHh+7eNxfnVt6WbOpo3s6rgD9gwABWrOjQNNQinWZmbY0GzTrlbcmmjuZtVemIiJQIBXwRkRKhgC8iUiIU8EVESoQCvohIiehwwDez683sAzN7Pumxrc1sgZm9Gv3dKnrczOwqM3vNzJ41s72zkXgREem4zpTwbyScNS7Zb4FF7j4QWBRtQ7jCzcDoNgm4Np1EBkHAn/70J4IgSOdlpMgpn4hsXIf74bv7EjMb0OrhcYTrYUI4u1yccCHkccDsaKa5J8ysj5ntsClzdwRBQFVVFQ0NDfTo0YNFixYRi8U6+zJS5IIg4KCDDqKuro7u3buzePFi5RORVtKtw98uEcSjv4lly3Zk/SXUVtHGkmNmNsnMVpjZijVr1mywPx6P09DQgLtTX19PPB5PM8lSjOLxOPX19bg7DQ0NeZFP2svbIl0tW422qRbYSDlLm7vXuPsQdx/St++GI4OrqqqoqAh/iFRWVlJVVZXJdEqRqKqqoqwszM7dunXLi3zSXt4W6WrpBvz3zWwHgOjvB9Hjq1h/zcydCNee7LRYLMb1118PwK9//Wv9TJeUYrEYAwcOZMCAAar2E2lDugH/QWBidH8i8EDS4xOi3jr7Av9JZ+7to48+ms0335y1a9eml1opWnV1dbz++usceeSRCvYibehwo62Z3UbYQLutma0iXMfyEuBOMzsB+DdwZHT4XOBg4DXgS8I1XDdZeXk5Q4YM4cknC2LZSMmBZ599lvr6eoYObW9tbpHS1ZleOke3sWtkimMdOHVTE5XK0KFDufzyy7nwwgsZNWqUSnGynttvvx0ICwciklrBjLTt06cPjY2NTJkyhZEjR6qvtbQIgoArr7wSgGOOOUZ5Q6QNBRPwP/30UwCam5vVPVPWE4/HaWpqAlDeENmIggn448aNA8DM8qbbneSHvfcOZ+5Q3hDZuIIJ+LFYjOHDh7PVVlup252sJ2wyguOPP155Q2QjCibgAxxyyCF8/PHH7LrrrrlOiuSRJ554AjPj8ssvV7AX2YiCCvj77LMPAGeddZYa5qTFvHnz6NevH88//3z7B4uUsIIK+ImGuZtuukk9dQSA2tpali9fzgcffKA8IdKOggr4y5cvB9BEatLinnvuAZQnRDqioAJ+VVVVy8Aa9cYQgJ49ewLhgCvlCZGNK6iAH4vFOPfccwGYMWOGGuiEDz/8kJ49e3LBBReoh45IOwoq4AP87Gc/A+Crr77KcUokHwRBwP7778+5556rYC/SjoIL+AMHDmTrrbdW45zw2Wef8dxzzynQi3RQwQV8M2O33XZjzpw5CvolbtasWTQ3N9O7d+9cJ0WkIBRcwA+CgKeeekrd8EpcEASceeaZAJx33nnKByIdUHABPx6P09zcDISLXqgbXmlKrHUMmjBNpKMKLuBXVVXRvXt3AMrKytQNr0QdeOCBgCZME+mMggv4sViMRYsWscMOO7DXXnupwa5EbbPNNgAcfvjh6o4p0kEFF/AhDPqHH344L730Ust0C1JaamtrAZg6daqCvUgHFWTAB9hvv/34/PPP+eUvf6kGuxJ033330aNHDz7++ONcJ0WkYBRswO/RowcA06ZNU2+dEhMEAXPmzGHdunWMHj1a116kgwo24L/88suAJs0qRXPnzm1Z9ETXXqTjCjbgjxgxQhOplajEQKuysjJde5FOKNiAH4vFOPXUUwG4/fbb1XBXQj7++GPKy8v5wx/+oB46Ip1QsAEfvp5Ira6uLscpka60bNkyBg8ezPnnn69gL9IJGQn4ZvZLM1tpZs+b2W1m1sPMdjaz5Wb2qpndYWbdMnGuZHvttRfdu3fn8ssvV8NdiViyZAm1tbXsvPPOuU6KSMFJO+Cb2Y7Az4Eh7r4nUA78BLgUuNzdBwJrgRPSPVdrK1asoKGhgSAI1FOnBARBwJgxY2hsbOS+++7T9RbppExV6VQAm5lZBdATeBc4CLg72j8LGJ+hc7WIx+PqrVFC4vE49fX1QLi+sa63SOekHfDd/R3gMuDfhIH+P8A/gE/cvTE6bBWwY6rnm9kkM1thZivWrFnTqXNXVVXRrVtYU1RRUaHeGkWuqqoKMwMKo2dWOnlbJBsyUaWzFTAO2Bn4BrA58MMUh3qq57t7jbsPcfchffv27dS5Y7EYc+bMwcz46U9/qga8IrfPPvuwxRZbMHjw4ILonZNO3hbJhkxU6YwC/uXua9y9AbgX2A/oE1XxAOwErM7AuTY8+ahR7LXXXrz11lvZeHnJIy+//DKffvopkydPzvtgL5KPMhHw/w3sa2Y9Lfy9PRJ4Afg7cER0zETggQycK6UDDjiAZcuWMXXqVDXkFbFZs2YBsMUWW+Q4JSKFKRN1+MsJG2f/CTwXvWYN8BvgTDN7DdgGmJnuudrSr18/6urqOP/889Vbp0gFQcBll10GwLHHHqtrLLIJMtJLx93Pd/fd3X1Pd/+/7l7n7m+4+1B339Xdj3T3rI2O+uKLLwBobm5Wb50iFY/HW6bC1jUW2TQFPdI24bDDDiuo3hvSebvvvjug+XNE0lEUAT8Wi3HIIYfQo0cPFixYoAa9IvTll18CMHny5ILooSOSj4oi4AMceeSRrFu3ji233DLXSZEsWLJkCb179+aKK65QsBfZREUT8IcPHw7A7373OzXoFaH58+ez/fbb8+STT+Y6KSIFq2gC/urVqzEzHnroIfXUKTIPP/wwb731Fq+88oqurUgaiibgP/bYY5pXp0jdcsstgFY3E0lX0QT8qqoqKisrAaisrFQvjiKSmDCtvLxcPXRE0lA0AT8Wi3HzzTcDcPrpp6thr4i89NJLDB06lAsvvFA9dETSUDQBH8KeOttuuy3z589XPW+ReOSRR1i5ciV7770355xzjoK9SBqKKuA/8cQTrF27lmeffVaNe0UgCALGjw+XUbjhhht0PUXSVFQBPx6P09zcDKjhthjE43EaGhoAaGxs1PUUSVNRBfyqqiq6d+8OhEPw1bhX2JKvnxprRdJXVAE/FouxaNEievXqxfDhw1XfW+B23XVX3J0xY8aosVYkA4oq4APst99+/OhHP+L5559v6ZcvhSlRhfPHP/5RwV4kA4ou4AOMGDGC999/nzPPPFMNfQXstttuo1u3bi31+CKSnqIM+L179wbgyiuvVG+dAhUEAffffz/19fWMHTtW11AkA4oy4L/yyiuAhuIXsgcffFBTZYhkWFEG/BEjRlBeXg6od0eh6tatG6DpFEQyqSgDfiwW47zzzgPgmmuuUYNfAXr77bfp1asXF1xwgXroiGRIUQZ8gJNPPhmAO+64Q/W/Baa2tpZ7772XwYMHc+655yrYi2RI0Qb8N998EzNj3rx5argtIEEQMHLkSP7zn/+wbNkyXTeRDCragJ/cyKdGv8IRj8epq6sDoLm5WddNJIOKNuAnz49fUVGhRr8CUVVVRVlZmC3VWCuSWUUb8GOxGHPmzMHMOProo1UPXCCGDh1Kz549GTx4sBprRTIsIwHfzPqY2d1m9pKZvWhmMTPb2swWmNmr0d+tMnGuzhg9ejTf/e53WbhwoeqCC8SsWbP47LPPOOywwxTsRTIsUyX8K4F57r478N/Ai8BvgUXuPhBYFG13qSAIeOmll1i1apUabgtAEARUV1cDcMkll+h6iWRY2gHfzLYEhgMzAdy93t0/AcYBs6LDZgHj0z1XZyXPj19XV6cGwDyXPP99Q0ODrpdIhmWihP9tYA1wg5k9bWZ/M7PNge3c/V2A6G+/VE82s0lmtsLMVqxZsyYDyfla8vz4ZqYGwDy3zz77AOG1KoYG22zmbZFNkYmAXwHsDVzr7oOAL+hE9Y2717j7EHcf0rdv3wwk52uJ+fF33313tttuO9UJ57lEd8xjjz22KBpss5m3RTZFJgL+KmCVuy+Ptu8m/AJ438x2AIj+fpCBc3VaLBbj1FNPZfXq1Zx11lmqF85jN954I+Xl5UyYMKHgg71IPko74Lv7e8DbZrZb9NBI4AXgQWBi9NhE4IF0z7WpEqWrv/71r2q8zVNBEHDXXXfR1NTEwQcfrGskkgWZ6qVzOnCLmT0L7AVcDFwCjDazV4HR0XZOvP7664CmS85nDzzwgKZDFsmyiky8iLv/LzAkxa6RmXj9dCWmS25qaiqKxsBilBhdq+mQRbKnaEfaJovFYkydOhWAyy67TPXDeejVV19l22231XTIIllUEgEfYPLkyZSXl3PnnXeqfjjPLF26lDlz5rDvvvtqOmSRLCqZgL9y5Urcnccee0wNt3kkCAJGjx7NunXrePTRR3VdRLKoZAJ+PB5Xo2Aeisfj1NfXA9DU1KTrIpJFJRPwq6qqWtZJLSsrU6Ngnki+DmqsFcmukgn4sViMxYsX06tXL4YNG6Z64jwxYMAA3J0xY8aosVYky0om4APst99+HHXUUSxfvpypU6eqvjgPTJs2DYBjjjlGwV4ky0oq4APssssufPnll5x//vlqvM2xIAi49NJLAaiurta1EMmykgv469atA8L1UtV4m1uLFi2iqakJUEO6SFcouYA/duxYrZmaJ3r37g2Ejei6FiLZV3IBPxaLcfrppwPhcnqqN86df/3rX1RWVvL73/9eDbYiXaDkAj7QEvBnz56teuMcuuuuuxgwYABjx45VsBfpAiUZ8D/44APMjDlz5qjhNkfuuOMOVq1axWuvvaZrINJFSjLgJzcOqrEwN2bNCpc71pTVIl2nJAN+8qjb8vJyNRbmwOrVqzEzTYcs0oUyMh9+oYnFYixcuJCxY8dq1G0OfPjhhzz33HMce+yxDBw4kKqqKl0DkS5QkgEfYNiwYVRVVbF48WKWLl3KAQcckOsklYyrrrqK5uZmDjjgAI477rhcJ0ekZJRklQ6EozwXLlzIunXrGD16tBoNu0gQBFx88cUAnHrqqXrfRbpQyQb8eDyuUZ45sGDBAr3vIjlSsgE/ueEW4MADD8xhakpH9+7dAY2uFcmFkg34sViMRYsW8eMf/xh359Zbb1X1QhdYsmQJ3bp148QTT9ToWpEuVrIBH8Kgf+KJJwJwzTXXaABQli1dupS5c+dSX1/PTTfdlOvkiJSckg74AE8//TSgAUBd4cYbb2y5r/dapOuVfMCvqqqisrISgMrKStUpZ9HatWsBNNhKJEcyFvDNrNzMnjazOdH2zma23MxeNbM7zKxbe6+RC7FYjLvvvhuAPffcM8epKV7Lli1jwYIFDBo0iAsvvFD19yI5kMkS/hnAi0nblwKXu/tAYC1wQgbPlVF9+/alrKyMFStWqB4/C4IgYOTIkXz++eesXLlSI2tFciQjAd/MdgIOAf4WbRtwEHB3dMgsYHwmzpUN8XgcdwdUt5wN8Xic+vp6AJqamvT+iuRIpkr4VwBnA83R9jbAJ+7eGG2vAnZM9UQzm2RmK8xsxZo1azKUnM6pqqpar3+46pYza/jw4S1fqKVUd58PeVskWdoB38wOBT5w938kP5ziUE/1fHevcfch7j6kb9++6SZnk8RiMRYvXkz//v3p168f8Xhc1ToZ9NxzzwEwcuTIkqq7z4e8LZIsEyX8/YEfmdmbwO2EVTlXAH3MLDE5207A6gycK2tisRiHH34477zzDuedd57q8jMkCIKWFcZqa2tznBqR0pZ2wHf3c9x9J3cfAPwEWOzuxwB/B46IDpsIPJDuubKtR48eADQ3N6suP0MWL15MY2NYs6f3VCS3stkP/zfAmWb2GmGd/swsnisjxo0bR1lZ+JaUUl1zNvXu3RvQ3Dki+SCjAd/d4+5+aHT/DXcf6u67uvuR7l6XyXNlQywW45e//CUAhx56aI5TUxwWLFhARUUFxx9/fEnV34vko5IfadvaQQcdBMDdd9+tevw0LVmyhAcffJDGxkZuueWWXCdHpOQp4LfyzDPPAJpbJxNmzvy6Fk/vpUjuKeC3kjxPfkVFheqc0/DOO+8AmjtHJF8o4LcSi8WYO3cu5eXl7LLLLrlOTsFauHAhjz32GAceeKDmzhHJEwr4KfTs2ROAF154QfX4myAIAg455BAaGxt54oknNHeOSJ5QwE8heW6duro61T13UvLcOY2NjXr/RPKEAn4KyXPrJLal4/bYYw8AzEx19yJ5RAE/hcR6t8OGDQPg4YcfVrVOJ8ybNw+Ao48+WnX3InlEAb8NsViMk08+mebmZi6++GLV5XdQbW0tNTU1ANx33305To2IJFPA34i3334bUJ/8zpg9e7bWFhDJUwr4G5G83q365HeM+t6L5C8F/I2IxWI89NBDlJWVseuuu+Y6OXlv4cKFzJ8/n/33319970XykAJ+O7bcckvMjJUrV6oefyOCIODggw+moaGBFStWqO+9SB5SwG+H+uR3TDwep6GhAVDfe5F8pYDfjuQ++e7Ov//9b5XyU/jyyy8B9b0XyWcK+O1I9MnfZ599cHdqampUtdNKEARccsklQNhYe8UVV6g6RyQPKeB3QCwWY9SoUYCWP0xl/vz5LcsYujsfffRRjlMkIqko4HfQIYccQnl5OaDlD1tLVOdoGUOR/KaA30GxWIxrrrkGgO9///s5Tk3+cHfuuecettlmG0466SR1xRTJYwr4nbDnnntiZixZskT1+JFrr72WN954g48//pjZs2fnOjkishEK+J3w2GOPtdxXF81Q4lePpp8QyX8K+J1QVVVFjx49gDDAbbPNNjlOUW7de++9rFy5kvLyck2lIFIAFPA7IRaLccUVV1BWVoa7c8YZZ5RstU4QBBx11FFA2BVT9fci+U8Bv5M++ugjzAwo7WqdefPmtXTFbGpqon///gr2Inku7YBvZt80s7+b2YtmttLMzoge39rMFpjZq9HfrdJPbu5VVVXRrVs3zAx355VXXinJUv4LL7wAqCumSCHJRAm/EfiVu+8B7AucambfAX4LLHL3gcCiaLvgJUbeHnHEEQDMmjWr5HrsLF26lHvuuQcIA75G1opkVxDAn/4U/k1H2gHf3d91939G9z8DXgR2BMYBs6LDZgHj0z1XvojFYgwaNAgozd4pV111VcuEchpZK5IdiSB/2mkwbBicey6MHJle0K/IXPLAzAYAg4DlwHbu/i6EXwpm1q+N50wCJgH0798/k8nJqsSkanV1dSXVY2fZsmUsXLgQM1N1TjsKNW9L7tXUwKmnQtRM1qKuDuJx2NQf1BlrtDWzLYB7gF+4+6cdfZ6717j7EHcf0rdv30wlJ+tisRhXXXUVZkZzczO/+MUvir5aJwgCDjroID755BP1zOmAQs3bkhtBAKecAqNGQXX1hsEeoLwc0ilfZaSEb2aVhMH+Fne/N3r4fTPbISrd7wB8kIlz5ZNEjx13b+mxU8zB7+9//zv19fVAWJWjnjkimVFTA5MnQ1NT6v1mYbCfNm3TS/eQgYBvYR/FmcCL7v7XpF0PAhOBS6K/D6R7rnyTqNb56quvaG5u5s033yQIgqINgqtXrwY0571IJgQBzJoFK1fCsmUQNYutp6ICzjwT+vQJS/ZphxZ3T+sGDAMceBb43+h2MLANYe+cV6O/W7f3WoMHD/ZCU1tb62PHjnXAzcw322wzr62tzXWyMm7ZsmVuZg54RUWFT58+PddJ6jRghaeZ3zf1Voh5W7Jn+nT3sjL3MMxveKusdK+udu9oKOlo3k67hO/ujwPWxu6R6b5+vovFYgwfPpz58+fj7qxbt47Zs2cXXSn/ggsuUM8ckTQEAdx4I/zzn7BiRepjysvhsMPg7LMzUJpPIaO9dErViBEjqKyspKGhAXfnhhtuYMKECUUT9B9//HHmz5/fsl1RUaHqHJEOCgK44Qa4/vq26+grK+GEE2DChOwE+gRNrZABsViM448/vmW72BbxvvLKK1vumxnHHXdc0XyZiWRLEISl9WHDYMaM1MG+vBzGj4fHHoNrr81usAcF/IyZOHHieoudF0u//Hg8ziOPPIKZUV5eTo8ePZgwYUKukyWSt4IAjj8+DPRz5kBz84bHVFaGXS+XLoX77st+oE9QlU6GJPrlV1dX09zczBlnnMH3vve9gi4JB0HAmDFjaGhooLKykhNOOKGoqqpEMiUIwgFR77wTltRTBflE/fz222e/6qYtCvgZlNwvf926dUyZMoUpU6YUbIC87rrraGhoAMLF29XvXmRDbY2KTVZeDtdcA5MmdV26UlHAz6DkfvkACxYsYOnSpQU5GjUIAm6++eaWbTXUinwtUaJ//XWYOTP1MflQom9NAT+DEjNpnnfeeSxevHi9idUKLeDfdNNNNEe/S9VQK/K19kr0iVGxV1+d+xJ9awr4GRaLxZg6dSpVVVXU19cXZAPu0qVLufPOOwFali5UQ62Uso6Mii0rg7POyuCo2GzoyOisrroV02jE6dOnt4xM7dGjR8GMvq2trfXKysqWEbXV1dUFk/b2oJG2sgnaGxVr5l5RER6XKx3N2yrhZ0mhNuBOmzatpaHWXROkSWnqyKjYjM9z0wUThBpzAAAKzklEQVQU8LOkEBtwly1bxm233dayrYZaKTX5NCo2GzTwKksSDbijRo0CWG+enXx19tlnt8yXo4ZaKSX5OCo2GxTwsygWi3HBBRfQrVs3gJZ5dvJxoZSbb76Z2tralm011EopyOdRsdmgKp0sS8yzc9111wFQV1eXd7NpLl68mOrq6pZtle6lmBXKqNhsUMDvAhMmTOD6669vWS1qxowZLY/nOqjW1NRQXV3dUpVTVlZG9+7dVbqXolRIo2KzQVU6XSBRyg8XB4OmpiamT5/OyJEjc1q9EwQBp5xyynr19qNGjcrrhmWRzgoC+NOf4MQT4eST214rdvz4r6tuijHYg0r4XWbChAnMmjWLdevWtfSJzeViKUEQcNppp7WMpoWwV04hdB0V6ahCHhWbDQr4XSTRa2f27NnMmDGDpqamnC2WEgQBI0aMoK6uDqBl6uNp06Yp2EvBK5pRsdnQkdFZXXUrldGI1dXVTrgOsAM+ZsyYLh3NetRRR7Wcu6ysrMvPnytopG3RK4RRsdnQ0bytOvwcmDBhAptttlnL9qOPPsrw4cOpqanJ+rn/+Mc/cscdd7RsV1ZWqhpHCloQhHXzQ4eGf1P1uqmoCNeJvegiWLKkNKpvUlGVTg4kqnemTJnCo48+CoTLIp5yyikATMpCbgyCgD//+c/cd999LY+p+6UUsmIfFZsVHfkZ0FW3UvvZW1tb6xUVFetV75SXl2d8wrLp06d7eXn5eucBvLKysiSqchJQlU5RqK11P/TQjVfdlJe7jx8fHlsKOpq3VaWTQ7FYjKuvvpqysq8vQ1NTE9ddd11GqniCIOCkk06iurqaplZFoIqKCjXSSkEptVGx2aAqnRxLVN+cdtppNDY2En5Zh1U8kydP5umnn96kXjw1NTVMnjx5g0BfXl7OSSedlBeDvkTaU8qjYrOiIz8D0rkBPwBeBl4DfruxY0v5Z29tba1XV1enrHopLy/38ePHt1v9Ultb6xdffLFPnDhxg9cgmt9+erF1T+gEVKVTUKZPD3vUtFVtk6i6KeEs3aKjeTurJXwzKweuBkYDq4CnzOxBd38hm+ctRLFYjFgsxqBBgzYo7Tc1NXH//ffz0EMPMWbMGL71rW8xaNAgnn76ad577z369etHY2MjN95443oDqRJUqpdCkSjRv/EG/O1vqY9RiX7TZbtKZyjwmru/AWBmtwPjAAX8NkyaNInvfe976w3QSmhqauKRRx7p1OuVlZVxzTXXZKXnj0gmaVRs9mU74O8IvJ20vQrYJ/kAM5sETALo379/lpNTGJJL+6nq4TsiMXr26quvVrDPEeXt9gUBzJ4Nzz+vUbFdIdsB31I8tt4ldfcaoAZgyJAhKS536Uou7b/33ns8/PDDLcsPppII8meeeSZ9+vShqqpKVTg5pLy9cTU1cMopqRtiQSX6bMh2wF8FfDNpeydgdZbPWVQSpX0Iu1kmVsxKrsPffvvtGTRoEB999JGCvOS1xFqxTz8NTz2V+phCXCu2UGQ74D8FDDSznYF3gJ8AP83yOYtWcvAXKSQaFZsfshrw3b3RzE4D5gPlwPXuvjKb5xSR/BEEcPHFMHdu21U3iV43Z5+tQJ9tWR945e5zgbnZPo+I5I8gCBcDnzWr7UCvEn3X00hbEckIjYrNfwr4IpK2Ul8rtlAo4IvIJtGo2MKjgC8inaZRsYVJAV9EOiQxKnblSnj8cY2KLUQK+CLSLo2KLQ4K+CKSkkbFFh8FfBFZj0bFFi8FfBEBNCq2FCjgi5Q4jYotHQr4IiVIo2JLkwK+SInRqNjSpYAvUgI0KlZAAV+k6GlUrCQo4IsUIY2KlVQU8EWKjEbFSlsU8EWKgEbFSkco4IsUuGuvhdNOUx96aZ8CvkgBCgKYOROefBKeey71MRoVK60p4IsUEI2KlXQo4IsUgCCACy+EefNS97gBleilfQr4InksCMLeNLfdphK9pE8BXyTPJI+KnTkzdYleo2JlUyjgi+QRzXMj2ZRWwDezPwOHAfXA68Bx7v5JtO8c4ASgCfi5u89PM60iRSlRon/+ebj11tTHqEQvmZBuCX8BcI67N5rZpcA5wG/M7DvAT4DvAt8AFprZf7l7G+vniJSmmhqYPLntlaU0KlYyKa2A7+6PJm0+ARwR3R8H3O7udcC/zOw1YCgQpHM+kWLy8MNw8smp92meG8mGTNbhHw/cEd3fkfALIGFV9NgGzGwSMAmgf//+GUyOSG61l7efeSbVc1Sil+xpN+Cb2UJg+xS7fufuD0TH/A5oBG5JPC3F8Sl7D7t7DVADMGTIkDZ6GIsUnvby9ogRsNlmUFcXlug1z41kW7sB391HbWy/mU0EDgVGurd0IFsFfDPpsJ2A1ZuaSJFiFIvBokVhg62CvHSFdHvp/AD4DXCgu3+ZtOtB4FYz+ytho+1A4Ml0ziVSjGIxBXrpOunW4U8DugMLzAzgCXevdveVZnYn8AJhVc+p6qEjIpJb6fbS2XUj+y4CLkrn9UVEJHPKcp0AERHpGgr4IiIlQgFfRKREKOCLiJQI87ZWU8gBM1sDvNXG7m2BD7swORuTL2nJl3RA/qRlY+n4lrv37crEJBRI3s6XdED+pCVf0gEZyNt5FfA3xsxWuPuQXKcD8ict+ZIOyJ+05Es6OiNf0pwv6YD8SUu+pAMykxZV6YiIlAgFfBGRElFIAb8m1wlIki9pyZd0QP6kJV/S0Rn5kuZ8SQfkT1ryJR2QgbQUTB2+iIikp5BK+CIikgYFfBGREpH3Ad/MjjSzlWbWbGZDWu07x8xeM7OXzWxsF6Zpipm9Y2b/G90O7qpzJ6XhB9H//ZqZ/barz98qLW+a2XPRe7GiC897vZl9YGbPJz22tZktMLNXo79bdVV6OiMf83V07pzmbeXrlnNnJ2+7e17fgD2A3YA4MCTp8e8AzxBOz7wz8DpQ3kVpmgKclcP3pDz6f78NdIveh+/kMD1vAtvm4LzDgb2B55Me+3/Ab6P7vwUuzdX70k7a8y5fR+fPWd5Wvl7v3FnJ23lfwnf3F9395RS7WhZKd/d/AYmF0kvBUOA1d3/D3euB2wnfj5Li7kuAj1s9PA6YFd2fBYzv0kR1kPJ1SsrXkWzl7bwP+BuxI/B20nabC6VnyWlm9mz006urqw1y/b+35sCjZvaPaOHuXNrO3d8FiP72y3F6Oisfrm2u8nY+/O/J8ilfQwbydrorXmVERxZKT/W0FI9lrI/pxtIEXAtcGJ3vQuAvwPGZOncHZPV/3wT7u/tqM+tHuPrZS1EJpaTlY76GvM7bytdZlhcB39tZKL0NWV0ovaNpMrMZwJxMnbeD8mqReHdfHf39wMzuI/xpnqsPxvtmtoO7v2tmOwAf5CgdeZmvIa/ztvL1xqWdtwu5SudB4Cdm1t3MdqYLF0qP3uyEw4Hn2zo2S54CBprZzmbWDfgJ4fvR5cxsczPrlbgPjKHr349kDwITo/sTgbZK0vkqZ/kacp63la83Lv28nasW8E60Vh9O+M1fB7wPzE/a9zvCVv2XgR92YZpuAp4Dno0uwg45eF8OBl6J/v/f5fD6fJuwN8UzwMquTAtwG/Au0BDlkROAbYBFwKvR361z9d60k/a8y9fRuXOat5WvW86flbytqRVEREpEIVfpiIhIJyjgi4iUCAV8EZESoYAvIlIiFPBFREqEAr6ISIlQwBcRKRH/H6S6W5/R9M1pAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "x_grid_G = np.linspace(-10, 10, 100)\n", "\n", "fig_h, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, squeeze=False)\n", "subplot_grid[0,0].plot(x_grid_G, [f(x_g) for x_g in x_grid_G], 'k.-')\n", "subplot_grid[0,0].set_title('f(x) = x^2')\n", "\n", "subplot_grid[0,1].plot(x_grid_G, [g(x_g) for x_g in x_grid_G], 'b.-')\n", "subplot_grid[0,1].set_title('gradient of f(x)')\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise 1a:\n", "\n", "Consider the decaying periodic function below. Can you compute its derivative using autograd and plot the result?\n", "\n", "$$\n", "f(x) = e^{-x/10} * cos(x)\n", "$$" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEICAYAAABbOlNNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFglJREFUeJzt3H2QZXV95/H3RyaSqDwzII+CEY3o1iY6Ykg0YhSEpHCIYgXFcmIgk6zLZrd0N4GyIs9buBExKkkAQYmUUYNPE0VxxLDGlQiDGBENMhJZBhAGQSMawNHv/nHOsPd3uT3d0/dO953h/ao61fec3++c873dv9Ofex66U1VIkrTR4xa7AEnSdDEYJEkNg0GS1DAYJEkNg0GS1DAYJEkNg2GBJHlGkhuS/DDJH/fLjkjy8Tmuf22SZ23ZKje5/19MclqSgxerBm3d+vFzWf96/yQPJNluCup61LE51L5nki/07ef2yw5OsmaO2/9okiMnXfeWZDAsnD8Brq6qHarqnf2y/wmcM8f13wacsUUqm0WSJwOfBV4MfDbJ/kPtv53ki0m+n+S7SS5KssNi1KqtQ1X936p6UlX9dNxtJbk6yYljbGLUsTloJXAvsGNVvalfdibdMTkX5wBnj1HfgjMYFs5TgJs2ziR5HrBTVf3THNdfBbw4yV5boriZJNkR+DTwgap6EXAe8Jkkuw102wk4C9gbeCawL/DnC1mnFlaSJYtdwwQ1x+YM7d+o/q+B+2PwxcCczvar6lpgxyTLxi10wVSV0xaegM8DPwUeBB4Ang68BXjPQJ9fo/tUsl8//x+B7wO/NNBnNbBiwrVdAZw7MP8h4JL+9fbAPwCnDK3zBuBLwBNn2OYrgBsX+/vutNlj4TnADcAPgb/rx8JZfdthwDrgT4HvAu8HdgE+CawH7u9f7zuwvQOB/91vbzXwbuCyvu0AoIAl/fxOwMXAXcAddB80tuvbfg/4It0n9PuBfwWO6tvOHjq23j3De3s53S//7wNXA8/slz/q2Bxa733AT4CH+/aXAq8DPjfQ5xeB+4Dn9PN798fyYQN9LgJOXeyf8ZzHwmIX8FiZ+sF44sD83wH/Y6jP2f1A/QXga8BJQ+3vBN4+w/Zf0A/6maYXzLDek4F7gN8EjgduBXYY872+A/jgYn/PnTbrZ/Z44DbgvwI/RxfuDw8FwwbgrXQfGH4B2A14JfAEYId+TH98YJvXAG/v+/9GHxAzBcPHgQuAJwJ7ANcCf9i3/V7/y/kPgO2A/wTcCaRvb46tEe/t6cCPgMP79/YnwFrg8XNc/30bvw/9/J8D5w/1+QPgm/334krgbUPtbwQ+utg/57lO29Lp4NZmZ7oDZdBpwD/RHRR3AOcPtf8QGHkpqaq+2G9zs1TVd5P8EXAp3cF+TFUN1zVnSQ4HVgDPn+82tCh+FVgCvLO632QfTXLtUJ+f0X3qfaif/3fgIxsbk5xNd4ZJfx/qecBL+/5fSPL3o3acZE/gKGDnqvp34EdJzqO7tn9B3+22qrqo738p8JfAnnRnL7P5XeBTVbW6X/9tdAH4a3ShsLl2Br43uKCqLkpyNPBlusB7+dA6P2Qex+di8R7D4rmf7lPWI6rqJ3SfTp5Nd3ln+D8c7kD36X/SPkn3SezmPmDmJcmvAh8Ajq2qb02qOC2IvYE7hsbc7UN91lfVgxtnkjwhyQVJbkvyb8AXgJ37J432Bu6vqh8NrH/bDPt+Ct0n+bv6Bxi+TxcIewz0eSQAqurH/csnbcZ7e2TfVfWz/r3tM8f1hz3q2O1dRHfsvmsgPDfaUsfuFmEwLJ6v0Z3iPiLJPsCpwHuBc5NsP7TOM4F/HrWxJC/sH/+baXrhJmo5m+40eK8kr57Pm0nyK3Q3yH+/qq6azza0qO4C9kmSgWX7DfUZ/qDyJuAZwPOrake6y0UA6be3S5InDvTfn9FuBx4Cdq+qnftpx6qa6+PZs/2L6DvpwqcrrnuP+9Gdlc/HqGP3SXSXUC8GTkuy69A6Mx6708hgWDxXAC/aONMP1vfRDawT6A6sMwfatweeS3cT71Gq6h+re/xvpukfR62X5DeA19PdUHsd8K4+oOYsybOBzwD/papGXi7Q1LuG7ibsSUmWJFkOHDLLOjvQXU76fv+L8NSNDVV1G7AGOD3J45O8ADh61Eaq6i66x6HPTbJjksf1fzfzolH9R7gbeOom2j8M/HaSlyT5ObpAe4juAYr5WA08J8nPDyz7C+D6qjoR+BTw10PrvIju6b6tgsGwSKrqK8APkmy8Fv/HdNdM/6w/nX898PqBT/ovp3vW+s5J1dA/ivo3dDe57+gvI10MvHfok+Ns3gQsBS4eOEPZ1ON/mjJV9TDdDecT6C55vJbuEuPwJZFB76C7L3Uv3b2xzwy1v4buXtN9dKHxN5vY1uvoboB/g+5SzeXMcD9thL8Ajk1yf5JH/R1CVd1M937e1dd6NHB0/543W1XdTfeQyHKAPkSPBP6o7/JGuuA4vm9/HvCj6h5b3SpsvKuvRZDkCOANVXXMHPp+GTihqr6+5SuTHhlzf11V713sWqZN/x8ALgUOGXEvcLjvR4CLq+qKBSluAgwGSQD0l25upvtUfTzd5ZCn9pd69Bji46qSNnoG3fX4JwHfpnu6zFB4DPKMQZLU8OazJKmxVV5K2n333euAAw5Y7DK0jbr++uvvraqlC71fx7W2pOuvv/7fgGuqatZ/Ab5VBsMBBxzAmjVz+lfo0mZLMtNf6G5RjmttSUlumUsogJeSJElDDAZJUsNgkCQ1DAZJUsNgkCQ1DAZJUsNgkCQ1DAZJUsNgkCQ1DAZJUsNgkCQ1DAZJUsNgkCQ1DAZJUsNgkCQ1DAZJUsNgkCQ1DAZJUsNgkCQ1DAZJUsNgkCQ1DAZJUsNgkCQ1DAZJUsNgkCQ1JhIMSY5McnOStUlOHtG+fZIP9e1fTnLAUPv+SR5I8t8nUY8kaf7GDoYk2wHnA0cBBwOvTnLwULcTgPur6mnAecBbh9rPAz49bi2SpPFN4ozhEGBtVd1aVQ8DHwSWD/VZDlzav74ceEmSACQ5BrgVuGkCtUiSxjSJYNgHuH1gfl2/bGSfqtoA/ADYLckTgT8FTp9tJ0lWJlmTZM369esnULa0+BzXmkaTCIaMWFZz7HM6cF5VPTDbTqrqwqpaVlXLli5dOo8ypenjuNY0WjKBbawD9huY3xe4c4Y+65IsAXYC7gOeDxyb5H8BOwM/S/JgVb17AnVJkuZhEsFwHXBQkgOBO4DjgNcM9VkFrACuAY4FPl9VBbxwY4ckpwEPGAqStLjGDoaq2pDkJOBKYDvgkqq6KckZwJqqWgVcDLw/yVq6M4Xjxt2vJGnLmMQZA1V1BXDF0LK3DLx+EHjVLNs4bRK1SJLG418+S5IaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqWEwSJIaBoMkqTGRYEhyZJKbk6xNcvKI9u2TfKhv/3KSA/rlhye5PsmN/dffnEQ9kqT5GzsYkmwHnA8cBRwMvDrJwUPdTgDur6qnAecBb+2X3wscXVX/AVgBvH/ceiRJ45nEGcMhwNqqurWqHgY+CCwf6rMcuLR/fTnwkiSpqhuq6s5++U3AzyfZfgI1SZLmaRLBsA9w+8D8un7ZyD5VtQH4AbDbUJ9XAjdU1UMTqEmSNE+TCIaMWFab0yfJs+guL/3hjDtJViZZk2TN+vXr51WoNG0c15pGkwiGdcB+A/P7AnfO1CfJEmAn4L5+fl/gY8DrqurbM+2kqi6sqmVVtWzp0qUTKFtafI5rTaNJBMN1wEFJDkzyeOA4YNVQn1V0N5cBjgU+X1WVZGfgU8ApVfV/JlCLJGlMYwdDf8/gJOBK4JvAh6vqpiRnJHl53+1iYLcka4E3AhsfaT0JeBrwZ0m+2k97jFuTJGn+lkxiI1V1BXDF0LK3DLx+EHjViPXOAs6aRA2SpMnwL58lSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSY2JBEOSI5PcnGRtkpNHtG+f5EN9+5eTHDDQdkq//OYkL5tEPZKk+Vsy7gaSbAecDxwOrAOuS7Kqqr4x0O0E4P6qelqS44C3Ar+b5GDgOOBZwN7A55I8vap+Op9aXvva13L55ZezYcMGqmpjfY+0L/Qy9zed+0vC7rvvzumnn87KlSuZdtdcA294A9x4I1TBwNuif1sk7etR7ZNetq3vbxpq2Nx1tt8envc8OOccOPRQ5q+qxpqAQ4ErB+ZPAU4Z6nMlcGj/eglwL5DhvoP9NjU997nPrWHHH398AU5OmzVdcMEFjxpLwJpxj4v5TKPG9Ze+VAVOTps3LVnSjZ35jutJXEraB7h9YH5dv2xkn6raAPwA2G2O6wKQZGWSNUnWrF+//lHtn/70p+dbvx7DPvKRjyzq/mcb11dfvfA1aeu3YcN4Y2cSwZARy2qOfeaybrew6sKqWlZVy5YuXfqo9qOOOmq2OqVHeeUrX7mo+59tXB92GDzOR0S0mZYs6cbOvNefQA3rgP0G5vcF7pyhz7okS4CdgPvmuO6cXHbZZQDeY3B/sy7bmu4xHHoofPGL3mPwHsPc1pnUPYZJBMN1wEFJDgTuoLuZ/JqhPquAFcA1wLHA56uqkqwCPpDk7XQ3nw8Crp1vIZdddtkjASFtKw49FG64YbGr0GPJ2MFQVRuSnER343g74JKquinJGXQ3O1YBFwPvT7KW7kzhuH7dm5J8GPgGsAH4zzXPJ5IkSZMxiTMGquoK4IqhZW8ZeP0g8KoZ1j0bOHsSdUiSxudtLUlSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDUMBklSw2CQJDXGCoYkuyZZneSW/usuM/Rb0fe5JcmKftkTknwqyb8kuSnJOePUIkmajHHPGE4Grqqqg4Cr+vlGkl2BU4HnA4cApw4EyNuq6peAXwF+PclRY9YjSRrTuMGwHLi0f30pcMyIPi8DVlfVfVV1P7AaOLKqflxV/wBQVQ8DXwH2HbMeSdKYxg2GPavqLoD+6x4j+uwD3D4wv65f9ogkOwNH0511SJIW0ZLZOiT5HPDkEU1vnuM+MmJZDWx/CfC3wDur6tZN1LESWAmw//77z3HX0nRzXGsazRoMVfXSmdqS3J1kr6q6K8lewD0juq0DDhuY3xe4emD+QuCWqnrHLHVc2Pdl2bJltam+0tbCca1pNO6lpFXAiv71CuATI/pcCRyRZJf+pvMR/TKSnAXsBPy3MeuQJE3IuMFwDnB4kluAw/t5kixL8h6AqroPOBO4rp/OqKr7kuxLdznqYOArSb6a5MQx65EkjWnWS0mbUlXfA14yYvka4MSB+UuAS4b6rGP0/QdJ0iLyL58lSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSQ2DQZLUMBgkSY2xgiHJrklWJ7ml/7rLDP1W9H1uSbJiRPuqJF8fpxZJ0mSMe8ZwMnBVVR0EXNXPN5LsCpwKPB84BDh1MECSvAJ4YMw6JEkTMm4wLAcu7V9fChwzos/LgNVVdV9V3Q+sBo4ESPIk4I3AWWPWIUmakHGDYc+qugug/7rHiD77ALcPzK/rlwGcCZwL/Hi2HSVZmWRNkjXr168fr2ppSjiuNY1mDYYkn0vy9RHT8jnuIyOWVZJfBp5WVR+by0aq6sKqWlZVy5YuXTrHXUvTzXGtabRktg5V9dKZ2pLcnWSvqroryV7APSO6rQMOG5jfF7gaOBR4bpLv9HXskeTqqjoMSdKiGfdS0ipg41NGK4BPjOhzJXBEkl36m85HAFdW1V9V1d5VdQDwAuBbhoIkLb5xg+Ec4PAktwCH9/MkWZbkPQBVdR/dvYTr+umMfpkkaQrNeilpU6rqe8BLRixfA5w4MH8JcMkmtvMd4Nnj1CJJmgz/8lmS1DAYJEkNg0GS1DAYJEkNg0GS1DAYJEkNg0GS1DAYJEkNg0GS1DAYJEkNg0GS1DAYJEkNg0GS1DAYJEkNg0GS1DAYJEkNg0GS1DAYJEkNg0GS1DAYJEkNg0GS1DAYJEkNg0GS1DAYJEmNVNVi17DZkqwHbpuheXfg3gUsZybTUgdMTy3TUgdsupanVNXShSwGtppxDdNTy7TUAVtHLQcB11TVkbNtYKsMhk1JsqaqllnH/zcttUxLHTBdtczFNNU7LbVMSx2w7dXipSRJUsNgkCQ1tsVguHCxC+hNSx0wPbVMSx0wXbXMxTTVOy21TEsdsI3Vss3dY5AkjWdbPGOQJI3BYJAkNbaJYEjyqiQ3JflZkmVDbackWZvk5iQvW+C6TktyR5Kv9tNvLfD+j+zf99okJy/kvkfU8p0kN/bfhzULvO9LktyT5OsDy3ZNsjrJLf3XXRayprlybM+4/8f82N6S43qbCAbg68ArgC8MLkxyMHAc8CzgSOAvk2y3wLWdV1W/3E9XLNRO+/d5PnAUcDDw6v77sZhe3H8fFvp57/fR/fwHnQxcVVUHAVf189PIsT3Esf2I97GFxvU2EQxV9c2qunlE03Lgg1X1UFX9K7AWOGRhq1s0hwBrq+rWqnoY+CDd9+Mxp6q+ANw3tHg5cGn/+lLgmAUtao4c2yM5ttmy43qbCIZN2Ae4fWB+Xb9sIZ2U5Gv9ad9CXq6Yhvc+qIDPJrk+ycpFrGOjPavqLoD+6x6LXM/mmoafr2O7M01jeyLjeslES9qCknwOePKIpjdX1SdmWm3Esok+n7upuoC/As7s93kmcC7w+5Pc/6ZKG7FsMZ9N/vWqujPJHsDqJP/Sf+J5zHNsb35pI5Y5tidoqwmGqnrpPFZbB+w3ML8vcOdkKurMta4kFwGfnOS+Z7HF3/vmqKo7+6/3JPkY3eWAxTx47k6yV1XdlWQv4J7FKsSxvdkc2zObyLje1i8lrQKOS7J9kgPp/rvgtQu18/4Hs9Hv0N1IXCjXAQclOTDJ4+luVK5awP0/IskTk+yw8TVwBAv7vRhlFbCif70CmOmT+bRybDu2R5nMuK6qrX6iG5jrgIeAu4ErB9reDHwbuBk4aoHrej9wI/C1/ge21wLv/7eAb/Xv/82L+PN5KvDP/XTTQtcC/C1wF/CTfpycAOxG99TGLf3XXRfr+zNL7Y7t0ft/zI/tLTmu/ZcYkqTGtn4pSZK0mQwGSVLDYJAkNQwGSVLDYJAkNQwGSVLDYJAkNf4fa8CDeTtsR+UAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "def f(x):\n", " return 0.0 # TODO compute the function above\n", " \n", "g = f # TODO define g as gradient of f\n", "\n", "# TODO plot the result\n", "x_grid_G = np.linspace(-10, 10, 500)\n", "fig_h, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, squeeze=False)\n", "subplot_grid[0,0].plot(x_grid_G, [f(x_g) for x_g in x_grid_G], 'k.-');\n", "subplot_grid[0,0].set_title('f(x) = x^2');\n", "\n", "subplot_grid[0,1].plot(x_grid_G, [g(x_g) for x_g in x_grid_G], 'b.-');\n", "subplot_grid[0,1].set_title('gradient of f(x)');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# PART 2: Using autograd's 'grad' function on functions with multivariate input\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, imagine the input $x$ could be a vector of size D. \n", "\n", "Our mathematical function $f(x)$ will map each input vector to a scalar.\n", "\n", "We want the gradient function\n", "\n", "\\begin{align}\n", "g(x) &\\triangleq \\nabla_x f(x)\n", "\\\\\n", "&= [\n", " \\frac{\\partial}{\\partial x_1} f(x)\n", " \\quad \\frac{\\partial}{\\partial x_2} f(x)\n", " \\quad \\ldots \\quad \\frac{\\partial}{\\partial x_D} f(x) ]\n", "\\end{align}\n", "\n", "Instead of computing this gradient by hand via calculus/algebra, we can use autograd to do it for us.\n", "\n", "First, we implement math function $f(x)$ as a **Python function** `f`.\n", "\n", "The Python function `f` needs to satisfy the following requirements:\n", "* INPUT 'x': numpy array of float\n", "* OUTPUT 'f(x)': scalar float\n", "* All internal operations are composed of calls to functions from `ag_np`, the `autograd` version of numpy\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "def f(x_D):\n", " return np.sum(np.square(x_D))\n", "\n", "g = autograd.grad(f)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[0. 0. 0. 0.]\n", "0.0\n", "[0. 0. 0. 0.]\n" ] } ], "source": [ "x_D = np.zeros(4)\n", "print(x_D)\n", "print(f(x_D))\n", "print(g(x_D))" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[1. 2. 3. 4.]\n", "30.0\n", "[2. 4. 6. 8.]\n" ] } ], "source": [ "x_D = np.asarray([1., 2., 3., 4.])\n", "print(x_D)\n", "print(f(x_D))\n", "print(g(x_D))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 3: Using autograd gradients within gradient descent to solve multivariate optimization problems " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Helper function: basic gradient descent\n", "\n", "Here's a very simple function that will perform many gradient descent steps to optimize a given function.\n", "\n", "Make sure you understand its basic properties (the gradient descent algorithm is one of the prereqs of this course)." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "def run_many_iters_of_gradient_descent(f, g, init_x_D=None, n_iters=100, step_size=0.001):\n", "\n", " # Copy the initial parameter vector\n", " x_D = copy.deepcopy(init_x_D)\n", "\n", " # Create data structs to track the per-iteration history of different quantities\n", " history = dict(\n", " iter=[],\n", " f=[],\n", " x_D=[],\n", " g_D=[])\n", "\n", " for iter_id in range(n_iters):\n", " if iter_id > 0:\n", " x_D = x_D - step_size * g(x_D)\n", "\n", " history['iter'].append(iter_id)\n", " history['f'].append(f(x_D))\n", " history['x_D'].append(x_D)\n", " history['g_D'].append(g(x_D))\n", " return x_D, history" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Worked Example 3a: Minimize f(x) = sum(square(x))\n", "\n", "It's easy to figure out that the vector with smallest L2 norm (smallest sum of squares) is the all-zero vector.\n", "\n", "Here's a quick example of showing that using gradient functions provided by autograd can help us solve the optimization problem:\n", "\n", "$$\n", "\\min_x \\sum_{d=1}^D x_d^2\n", "$$" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "def f(x_D):\n", " return np.sum(np.square(x_D))\n", "\n", "g = autograd.grad(f)\n", "\n", "# Initialize at x_D = [-3, 4, -5, 6]\n", "init_x_D = np.asarray([-3.0, 4.0, -5.0, 6.0])" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "opt_x_D, history = run_many_iters_of_gradient_descent(f, g, init_x_D, n_iters=1000, step_size=0.01)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3sAAADTCAYAAAA4agr9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAGQRJREFUeJzt3Xu0ZGV55/Hvb7oVIyo3G0WaScPQxrQ63k5AJE5QBNGozYq4RDPaYzAkrmC8xNFGJxKvA2qCunTUXkpk1BEd1Ngi2iKXP5I4yOGi0FykBSMtIAebYIxR7PDMH7Uby3Oq29PnnKp9zq7vZ61aVfvdb+391Hv2Wc96au/9VqoKSZIkSVK3/Ie2A5AkSZIkLTyLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeqg5W0HsLse+tCH1qpVq9oOQ5I0ZJdffvmdVbWi7TiWCvOjJI2P2ebIJVfsrVq1isnJybbDkCQNWZJ/ajuGpcT8KEnjY7Y50ss4JUmSJKmDLPYkSZIkqYNaL/aS7J3k3CTXJ7kuyRFtxyRJkiRJS91iuGfvfcBXq+qEJPcHHth2QJIkSZK01LVa7CV5CPBfgP8GUFX3APcMc5+3v/Od/Py664e5C0kae3v89qN4+Bvf2HYYkiSNtbYv4zwEmAL+NsmVST6aZM/pnZKcnGQyyeTU1NToo5QkSZKkJabtyziXA08EXllVlyZ5H7Ae+Mv+TlW1AdgAMDExUfPZod80S5IkSRoHbZ/Z2wpsrapLm+Vz6RV/kiRJkqR5aLXYq6rbgVuS/FbTdDRwbYshSZIkSVIntH0ZJ8ArgU81M3HeBLys5XgkSZIkaclrvdirqquAibbjkCRJkqQuafuePUmSJEnSEFjsSZIkSVIHWexJkiRJUgdZ7EmSJElSB1nsSZIkSVIHWexJkiRJUgdZ7EmSNCRJjktyQ5ItSdYPWL9Hks806y9Nsmra+v+Y5CdJXjeqmCVJ3WGxJ0nSECRZBnwQeBawBnhRkjXTup0E3FVVhwJnAmdMW38m8JVhxypJ6iaLPUmShuMwYEtV3VRV9wDnAGun9VkLnN28Phc4OkkAkhwP3ARsHlG8kqSOsdiTJGk4DgRu6Vve2rQN7FNV24G7gf2S7Am8AXjLrnaQ5OQkk0kmp6amFixwSVI3WOxJkjQcGdBWs+zzFuDMqvrJrnZQVRuqaqKqJlasWDHHMCVJXbW87QAkSeqorcBBfcsrgVt30mdrkuXAXsA24HDghCTvAvYG7k3ys6r6wPDDliR1hcWeJEnDcRmwOsnBwA+AE4EXT+uzEVgHfAM4Abioqgp46o4OSf4K+ImFniRpd1nsSZI0BFW1PckpwCZgGXBWVW1O8lZgsqo2Ah8DPpFkC70zeie2F7EkqWss9iRJGpKqOh84f1rbm/te/wx4wa/Zxl8NJThJUuc5QYskSZIkdZDFniRJkiR1kMWeJEmSJHXQoij2kixLcmWS89qORZIkSZK6YFEUe8CrgOvaDkKSJEmSuqL12TiTrAR+H3gH8Nqh7/Ar6+H2q4e+G0kaaw9/LDzr9LajkCRprC2GM3vvBV4P3LuzDklOTjKZZHJqamp0kUmSJEnSEtXqmb0kzwHuqKrLkxy1s35VtQHYADAxMVHz2qnfNEuSJEkaA22f2TsSeF6S7wHnAE9P8sl2Q5IkSZKkpa/VYq+qTq2qlVW1CjgRuKiq/mubMUmSJElSF7R9Zk+SJEmSNAStz8a5Q1VdAlzSchiSJEmS1Ame2ZMkSZKkDrLYkyRJkqQOstiTJEmSpA6y2JMkSZKkDrLYkyRJkqQOstiTJEmSpA6y2JMkSZKkDrLYkyRJkqQOstiTJEmSpA6y2JMkaUiSHJfkhiRbkqwfsH6PJJ9p1l+aZFXTfkySy5Nc3Tw/fdSxS5KWPos9SZKGIMky4IPAs4A1wIuSrJnW7STgrqo6FDgTOKNpvxN4blU9FlgHfGI0UUuSusRiT5Kk4TgM2FJVN1XVPcA5wNppfdYCZzevzwWOTpKqurKqbm3aNwMPSLLHSKKWJHWGxZ4kScNxIHBL3/LWpm1gn6raDtwN7Detz/OBK6vq59N3kOTkJJNJJqemphYscElSN1jsSZI0HBnQVrvTJ8mj6V3a+SeDdlBVG6pqoqomVqxYMedAJUndZLEnSdJwbAUO6lteCdy6sz5JlgN7Adua5ZXAF4CXVtV3hx6tJKlzLPYkSRqOy4DVSQ5Ocn/gRGDjtD4b6U3AAnACcFFVVZK9gS8Dp1bVP4wsYklSp1jsSZI0BM09eKcAm4DrgM9W1eYkb03yvKbbx4D9kmwBXgvs+HmGU4BDgb9MclXz2H/EH0GStMQtbzsASZK6qqrOB86f1vbmvtc/A14w4H1vB94+9AAlSZ3W6pm9JAcluTjJdUk2J3lVm/FIkiRJUle0fWZvO/AXVXVFkgcDlye5oKqubTkuSZIkSVrSWj2zV1W3VdUVzet/oXdPw/TfIJIkSZIk7aZFM0FLklXAE4BLB6zzR2MlSZIkaTcsimIvyYOAzwGvrqofT1/vj8ZKkiRJ0u5pvdhLcj96hd6nqurzbccjSZIkSV3Q9mycofcbQ9dV1d+0GYskSZIkdUnbZ/aOBF4CPL3vR2Of3XJMkiRJkrTktfrTC1X190DajEGSJEmSuqjtM3uSJEmSpCGw2JMkSZKkDrLYkyRJkqQOstiTJEmSpA6y2JMkSZKkDrLYkyRJkqQOstiTJEmSpA6y2JMkSZKkDrLYkyRJkqQOWr6rlUn+YFfrq+rzCxuOJEmjl2R/4EjgEcC/AdcAk1V1b6uBSZI0D7ss9oDnNs/7A08BLmqWnwZcAljsSZKWrCRPA9YD+wJXAncADwCOB/5TknOBv66qH7cXpSRJc7PLYq+qXgaQ5DxgTVXd1iwfAHxw+OFJkjRUzwb+uKq+P31FkuXAc4BjgM/NZeNJjgPeBywDPlpVp09bvwfwv4EnAT8CXlhV32vWnQqcBPw78OdVtWkuMUiSxtevO7O3w6odhV7jh8AjhxCPJEkjU1X/fRfrtgN/N9dtJ1lG74vRY4CtwGVJNlbVtX3dTgLuqqpDk5wInAG8MMka4ETg0fQuLf16kkdW1b/PNR5J0viZbbF3SZJNwKeBopeALh5aVJIkjVCSTwCnVNXdzfIq4GNVdfQ8NnsYsKWqbmq2eQ6wFugv9tYCf9W8Phf4QJI07edU1c+Bm5Nsabb3jXnE82u95UubufZWr1iVpGFa84iHcNpzHz2Sfc1qNs6qOgX4MPA44PHAhqp65TADkyRphP4euDTJs5P8MfA14L3z3OaBwC19y1ubtoF9mjOJdwP7zfK9JDk5yWSSyampqXmGK0nqmtme2aOqvgB8YdC6JN+oqiMWLCpJkkaoqj6SZDO9q1buBJ5QVbfPc7MZtKtZ9pnNe6mqDcAGgImJiRnrd9eovmmWJI3GQv3O3gMWaDuSJI1ckpcAZwEvBT4OnJ/kcfPc7FbgoL7llcCtO+vTTAizF7Btlu+VJGmXFqrYm/e3iZIktej5wO9W1aer6lTgT+kVffNxGbA6ycFJ7k/vfveN0/psBNY1r08ALqqqatpPTLJHkoOB1cA35xmPJGnMLFSxN2dJjktyQ5ItSda3HY8kafxU1fFVdUff8jeBw+e5ze3AKcAm4Drgs1W1Oclbkzyv6fYxYL9mApbX0vvNP6pqM/BZepO5fBX4M2filCTtrlnds5dkzbSpoklyVFVdsmNxLjuf5bTUkiQNRZL/Afyvqto2fV1V3ZPk6cADq+q8uWy/qs4Hzp/W9ua+1z8DXrCT974DeMdc9itJEsx+gpbPNtNSv4ve/XnvAiaAHZOyvGSO+5/NtNSSJA3L1cCXkvwMuAKYopfnVtObffrrwDvbC0+SpLmb7WWch9O7Ufwf6d2DcCtw5I6VVXXNHPfv1NKSpDadUFVH0rvUcjOwDPgx8EngsKp6TVWZeCRJS9Jsz+z9Avg34DfofeN5c1XduwD7b2VqaUmSGk9K8pvAHwJPm7buN+jlPkmSlqTZFnuXAV8Efofej71+JMkJVXXCPPfv1NKSpDZ9mN4EKIcAk33toffl4yFtBCVJ0kKYbbF3UlXtSIK3A2ub3ySar/umpQZ+QG9a6hcvwHYlSfq1qur9wPuTfKiqXtF2PJIkLaRZFXt9hV5/2yfmu/Oq2p5kx7TUy4CzmummJUkaGQs9SVIXzfbM3tAMmpZakiRJkjQ/rf+ouiRJkiRp4VnsSZIkSVIHWexJkiRJUgdZ7EmSJElSB1nsSZIkSVIHWexJkiRJUgdZ7EmSJElSB1nsSZIkSVIHWexJkiRJUgdZ7EmSJElSB1nsSZIkSVIHWexJkiRJUgdZ7EmSJElSB1nsSZIkSVIHWexJkrTAkuyb5IIkNzbP++yk37qmz41J1jVtD0zy5STXJ9mc5PTRRi9J6gqLPUmSFt564MKqWg1c2Cz/iiT7AqcBhwOHAaf1FYXvqapHAU8AjkzyrNGELUnqEos9SZIW3lrg7Ob12cDxA/o8E7igqrZV1V3ABcBxVfXTqroYoKruAa4AVo4gZklSx1jsSZK08B5WVbcBNM/7D+hzIHBL3/LWpu0+SfYGnkvv7OAMSU5OMplkcmpqakEClyR1x/K2dpzk3fQS2D3Ad4GXVdU/txWPJEm7I8nXgYcPWPWm2W5iQFv1bX858Gng/VV106ANVNUGYAPAxMREDeojSRpfbZ7ZuwB4TFX9Z+A7wKktxiJJ0m6pqmdU1WMGPL4I/DDJAQDN8x0DNrEVOKhveSVwa9/yBuDGqnrvsD6DJKnbWiv2quprVbW9Wfx/eD+CJKk7NgLrmtfrgC8O6LMJODbJPs3ELMc2bSR5O7AX8OoRxCpJ6qjFcs/eHwFf2dlK70mQJC0xpwPHJLkROKZZJslEko8CVNU24G3AZc3jrVW1LclKepeCrgGuSHJVkpe38SEkSUvbUO/Z29X9DM1lLiR5E7Ad+NTOtuM9CZKkpaSqfgQcPaB9Enh53/JZwFnT+mxl8P18kiTtlqEWe1X1jF2tb35A9jnA0VVlESdJkiRJC6TN2TiPA94A/F5V/bStOCRJkiSpi9q8Z+8DwIOBC5r7ET7cYiySJEmS1CmtndmrqkPb2rckSZIkdd1imY1TkiRJkrSALPYkSZIkqYMs9iRJkiSpgyz2JEmSJKmDLPYkSZIkqYMs9iRJkiSpg1r76YW2vOVLm7n21h+3HYYkddqaRzyE05776LbDkCRprHlmT5IkSZI6aOzO7PlNsyRJkqRx4Jk9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSFliSfZNckOTG5nmfnfRb1/S5Mcm6Aes3Jrlm+BFLkrqo9WIvyeuSVJKHth2LJEkLZD1wYVWtBi5sln9Fkn2B04DDgcOA0/qLwiR/APxkNOFKkrqo1WIvyUHAMcD324xDkqQFthY4u3l9NnD8gD7PBC6oqm1VdRdwAXAcQJIHAa8F3j6CWCVJHdX2mb0zgdcD1XIckiQtpIdV1W0AzfP+A/ocCNzSt7y1aQN4G/DXwE93tZMkJyeZTDI5NTU1/6glSZ3SWrGX5HnAD6rqW7PoazKTJC0qSb6e5JoBj7Wz3cSAtkryeODQqvrCr9tAVW2oqomqmlixYsVuxS9J6r7lw9x4kq8DDx+w6k3AG4FjZ7OdqtoAbACYmJjwLKAkqXVV9YydrUvywyQHVNVtSQ4A7hjQbStwVN/ySuAS4AjgSUm+Ry9P75/kkqo6CkmSdsNQi72dJcIkjwUOBr6VBHoJ7ookh1XV7cOMSZKkEdgIrANOb56/OKDPJuCdfZOyHAucWlXbgA8BJFkFnGehJ0mai6EWeztTVVfTd/9C8+3lRFXd2UY8kiQtsNOBzyY5id4kZC8ASDIB/GlVvbyqtiV5G3BZ8563NoWeJEkLopViT5KkLquqHwFHD2ifBF7et3wWcNYutvM94DFDCFGSNAYWRbFXVavajkGSJEmSuqTtn16QJEmSJA2BxZ4kSZIkdZDFniRJkiR1kMWeJEmSJHWQxZ4kSZIkdZDFniRJkiR10KL46YVROuObZ3D9tuvbDkOSOu1R+z6KNxz2hrbDkCRprHlmT5IkSZI6aOzO7PlNsyRJkqRx4Jk9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSJEnqoFRV2zHsliRTwD/NczMPBe5cgHC6xDEZzHGZyTGZyTGZaSHG5DerasVCBDMOFig/gsfzII7JTI7JTI7JYI7LTCPLkUuu2FsISSaraqLtOBYTx2Qwx2Umx2Qmx2Qmx2Tp8m83k2Myk2Myk2MymOMy0yjHxMs4JUmSJKmDLPYkSZIkqYPGtdjb0HYAi5BjMpjjMpNjMpNjMpNjsnT5t5vJMZnJMZnJMRnMcZlpZGMylvfsSZIkSVLXjeuZPUmSJEnqNIs9SZIkSeqgsSv2khyX5IYkW5KsbzueUUlyUJKLk1yXZHOSVzXt+ya5IMmNzfM+TXuSvL8Zp28neWK7n2B4kixLcmWS85rlg5Nc2ozJZ5Lcv2nfo1ne0qxf1Wbcw5Jk7yTnJrm+OV6OGPfjJMlrmv+ba5J8OskDxvE4SXJWkjuSXNPXttvHRpJ1Tf8bk6xr47NoJvOj+XE68+NM5siZzJGLOz+OVbGXZBnwQeBZwBrgRUnWtBvVyGwH/qKqfht4MvBnzWdfD1xYVauBC5tl6I3R6uZxMvCh0Yc8Mq8CrutbPgM4sxmTu4CTmvaTgLuq6lDgzKZfF70P+GpVPQp4HL2xGdvjJMmBwJ8DE1X1GGAZcCLjeZx8HDhuWttuHRtJ9gVOAw4HDgNO25EA1R7zo/lxJ8yPM5kj+5gj7/NxFmt+rKqxeQBHAJv6lk8FTm07rpbG4ovAMcANwAFN2wHADc3rjwAv6ut/X78uPYCVzT/g04HzgAB3AsunHzPAJuCI5vXypl/a/gwLPB4PAW6e/rnG+TgBDgRuAfZt/u7nAc8c1+MEWAVcM9djA3gR8JG+9l/p56O1v6v58Zef3fxY5sedjIk5cuaYmCN/ORaLMj+O1Zk9fnlA7rC1aRsrzSnzJwCXAg+rqtsAmuf9m27jMlbvBV4P3Nss7wf8c1Vtb5b7P/d9Y9Ksv7vp3yWHAFPA3zaX7nw0yZ6M8XFSVT8A3gN8H7iN3t/9csb7OOm3u8dG54+ZJcq/C+bHacyPM5kjpzFH7tKiyI/jVuxlQNtY/fZEkgcBnwNeXVU/3lXXAW2dGqskzwHuqKrL+5sHdK1ZrOuK5cATgQ9V1ROAf+WXlx0M0vkxaS6hWAscDDwC2JPeJRjTjdNxMhs7GwfHZ3Ea+7+L+fGXzI87ZY6cxhw5JyPNj+NW7G0FDupbXgnc2lIsI5fkfvQS2aeq6vNN8w+THNCsPwC4o2kfh7E6Enheku8B59C7VOW9wN5Jljd9+j/3fWPSrN8L2DbKgEdgK7C1qi5tls+ll9jG+Th5BnBzVU1V1S+AzwNPYbyPk367e2yMwzGzFI3138X8OIP5cTBz5EzmyJ1bFPlx3Iq9y4DVzQxB96d3A+nGlmMaiSQBPgZcV1V/07dqI7Bjtp919O5V2NH+0mbGoCcDd+84Fd0VVXVqVa2sqlX0joWLquoPgYuBE5pu08dkx1id0PTv1LdRVXU7cEuS32qajgauZYyPE3qXpjw5yQOb/6MdYzK2x8k0u3tsbAKOTbJP843wsU2b2mV+ND/ex/w4mDlyIHPkzi2O/Nj2zYyjfgDPBr4DfBd4U9vxjPBz/y69U8HfBq5qHs+md530hcCNzfO+Tf/Qm5ntu8DV9GZZav1zDHF8jgLOa14fAnwT2AL8X2CPpv0BzfKWZv0hbcc9pLF4PDDZHCt/B+wz7scJ8BbgeuAa4BPAHuN4nACfpndPxi/ofQN50lyODeCPmvHZArys7c/l476/i/nR/DhofMyPvzoe5siZYzL2OXIx58c0G5YkSZIkdci4XcYpSZIkSWPBYk+SJEmSOshiT5IkSZI6yGJPkiRJkjrIYk+SJEmSOshiT2pRkn9snlcleXHb8UiStFiYI6X5s9iTWlRVT2lergJ2K5ElWbbgAUmStEiYI6X5s9iTWpTkJ83L04GnJrkqyWuSLEvy7iSXJfl2kj9p+h+V5OIk/we4OsmeSb6c5FtJrknywtY+jCRJC8gcKc3f8rYDkATAeuB1VfUcgCQnA3dX1e8k2QP4hyRfa/oeBjymqm5O8nzg1qr6/eZ9e7URvCRJQ2SOlObIM3vS4nQs8NIkVwGXAvsBq5t136yqm5vXVwPPSHJGkqdW1d0txCpJ0iiZI6VZstiTFqcAr6yqxzePg6tqx7eW/7qjU1V9B3gSvYT2P5O8uYVYJUkaJXOkNEsWe9Li8C/Ag/uWNwGvSHI/gCSPTLLn9DcleQTw06r6JPAe4ImjCFaSpBEyR0pz5D170uLwbWB7km8BHwfeR2/2sSuSBJgCjh/wvscC705yL/AL4BUjiVaSpNExR0pzlKpqOwZJkiRJ0gLzMk5JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6qD/DwxARpf6nCQPAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Make plots of how x parameter values evolve over iterations, and function values evolve over iterations\n", "# Expected result: f goes to zero. all x values goto zero.\n", "\n", "fig_h, subplot_grid = plt.subplots(\n", " nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)\n", "subplot_grid[0,0].plot(history['iter'], history['x_D'])\n", "subplot_grid[0,0].set_xlabel('iters')\n", "subplot_grid[0,0].set_ylabel('x_d')\n", "\n", "subplot_grid[0,1].plot(history['iter'], history['f'])\n", "subplot_grid[0,1].set_xlabel('iters')\n", "subplot_grid[0,1].set_ylabel('f(x)');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Try it Example 3b: Minimize the 'trid' function\n", "\n", "Given a 2-dimensional vector $x = [x_1, x_2]$, the trid function is:\n", "\n", "$$\n", "f(x) = (x_1-1)^2 + (x_2-1)^2 - x_1 x_2\n", "$$\n", "\n", "Background and Picture: \n", "\n", "Can you use autograd + gradient descent to find the optimal value $x^*$ that minimizes $f(x)$?\n", "\n", "You can initialize your gradient descent at [+1.0, -1.0]" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "def f(x_D):\n", " return 0.0 # TODO\n", "\n", "g = f # TODO" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "# TODO call run_many_iters_of_gradient_descent() with appropriate args" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3sAAADTCAYAAAA4agr9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAGQRJREFUeJzt3Xu0ZGV55/Hvb7oVIyo3G0WaScPQxrQ63k5AJE5QBNGozYq4RDPaYzAkrmC8xNFGJxKvA2qCunTUXkpk1BEd1Ngi2iKXP5I4yOGi0FykBSMtIAebYIxR7PDMH7Uby3Oq29PnnKp9zq7vZ61aVfvdb+391Hv2Wc96au/9VqoKSZIkSVK3/Ie2A5AkSZIkLTyLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeqg5W0HsLse+tCH1qpVq9oOQ5I0ZJdffvmdVbWi7TiWCvOjJI2P2ebIJVfsrVq1isnJybbDkCQNWZJ/ajuGpcT8KEnjY7Y50ss4JUmSJKmDLPYkSZIkqYNaL/aS7J3k3CTXJ7kuyRFtxyRJkiRJS91iuGfvfcBXq+qEJPcHHth2QJIkSZK01LVa7CV5CPBfgP8GUFX3APcMc5+3v/Od/Py664e5C0kae3v89qN4+Bvf2HYYkiSNtbYv4zwEmAL+NsmVST6aZM/pnZKcnGQyyeTU1NToo5QkSZKkJabtyziXA08EXllVlyZ5H7Ae+Mv+TlW1AdgAMDExUfPZod80S5IkSRoHbZ/Z2wpsrapLm+Vz6RV/kiRJkqR5aLXYq6rbgVuS/FbTdDRwbYshSZIkSVIntH0ZJ8ArgU81M3HeBLys5XgkSZIkaclrvdirqquAibbjkCRJkqQuafuePUmSJEnSEFjsSZIkSVIHWexJkiRJUgdZ7EmSJElSB1nsSZIkSVIHWexJkiRJUgdZ7EmSNCRJjktyQ5ItSdYPWL9Hks806y9Nsmra+v+Y5CdJXjeqmCVJ3WGxJ0nSECRZBnwQeBawBnhRkjXTup0E3FVVhwJnAmdMW38m8JVhxypJ6iaLPUmShuMwYEtV3VRV9wDnAGun9VkLnN28Phc4OkkAkhwP3ARsHlG8kqSOsdiTJGk4DgRu6Vve2rQN7FNV24G7gf2S7Am8AXjLrnaQ5OQkk0kmp6amFixwSVI3WOxJkjQcGdBWs+zzFuDMqvrJrnZQVRuqaqKqJlasWDHHMCVJXbW87QAkSeqorcBBfcsrgVt30mdrkuXAXsA24HDghCTvAvYG7k3ys6r6wPDDliR1hcWeJEnDcRmwOsnBwA+AE4EXT+uzEVgHfAM4Abioqgp46o4OSf4K+ImFniRpd1nsSZI0BFW1PckpwCZgGXBWVW1O8lZgsqo2Ah8DPpFkC70zeie2F7EkqWss9iRJGpKqOh84f1rbm/te/wx4wa/Zxl8NJThJUuc5QYskSZIkdZDFniRJkiR1kMWeJEmSJHXQoij2kixLcmWS89qORZIkSZK6YFEUe8CrgOvaDkKSJEmSuqL12TiTrAR+H3gH8Nqh7/Ar6+H2q4e+G0kaaw9/LDzr9LajkCRprC2GM3vvBV4P3LuzDklOTjKZZHJqamp0kUmSJEnSEtXqmb0kzwHuqKrLkxy1s35VtQHYADAxMVHz2qnfNEuSJEkaA22f2TsSeF6S7wHnAE9P8sl2Q5IkSZKkpa/VYq+qTq2qlVW1CjgRuKiq/mubMUmSJElSF7R9Zk+SJEmSNAStz8a5Q1VdAlzSchiSJEmS1Ame2ZMkSZKkDrLYkyRJkqQOstiTJEmSpA6y2JMkSZKkDrLYkyRJkqQOstiTJEmSpA6y2JMkSZKkDrLYkyRJkqQOstiTJEmSpA6y2JMkaUiSHJfkhiRbkqwfsH6PJJ9p1l+aZFXTfkySy5Nc3Tw/fdSxS5KWPos9SZKGIMky4IPAs4A1wIuSrJnW7STgrqo6FDgTOKNpvxN4blU9FlgHfGI0UUuSusRiT5Kk4TgM2FJVN1XVPcA5wNppfdYCZzevzwWOTpKqurKqbm3aNwMPSLLHSKKWJHWGxZ4kScNxIHBL3/LWpm1gn6raDtwN7Detz/OBK6vq59N3kOTkJJNJJqemphYscElSN1jsSZI0HBnQVrvTJ8mj6V3a+SeDdlBVG6pqoqomVqxYMedAJUndZLEnSdJwbAUO6lteCdy6sz5JlgN7Adua5ZXAF4CXVtV3hx6tJKlzLPYkSRqOy4DVSQ5Ocn/gRGDjtD4b6U3AAnACcFFVVZK9gS8Dp1bVP4wsYklSp1jsSZI0BM09eKcAm4DrgM9W1eYkb03yvKbbx4D9kmwBXgvs+HmGU4BDgb9MclXz2H/EH0GStMQtbzsASZK6qqrOB86f1vbmvtc/A14w4H1vB94+9AAlSZ3W6pm9JAcluTjJdUk2J3lVm/FIkiRJUle0fWZvO/AXVXVFkgcDlye5oKqubTkuSZIkSVrSWj2zV1W3VdUVzet/oXdPw/TfIJIkSZIk7aZFM0FLklXAE4BLB6zzR2MlSZIkaTcsimIvyYOAzwGvrqofT1/vj8ZKkiRJ0u5pvdhLcj96hd6nqurzbccjSZIkSV3Q9mycofcbQ9dV1d+0GYskSZIkdUnbZ/aOBF4CPL3vR2Of3XJMkiRJkrTktfrTC1X190DajEGSJEmSuqjtM3uSJEmSpCGw2JMkSZKkDrLYkyRJkqQOstiTJEmSpA6y2JMkSZKkDrLYkyRJkqQOstiTJEmSpA6y2JMkSZKkDrLYkyRJkqQOWr6rlUn+YFfrq+rzCxuOJEmjl2R/4EjgEcC/AdcAk1V1b6uBSZI0D7ss9oDnNs/7A08BLmqWnwZcAljsSZKWrCRPA9YD+wJXAncADwCOB/5TknOBv66qH7cXpSRJc7PLYq+qXgaQ5DxgTVXd1iwfAHxw+OFJkjRUzwb+uKq+P31FkuXAc4BjgM/NZeNJjgPeBywDPlpVp09bvwfwv4EnAT8CXlhV32vWnQqcBPw78OdVtWkuMUiSxtevO7O3w6odhV7jh8AjhxCPJEkjU1X/fRfrtgN/N9dtJ1lG74vRY4CtwGVJNlbVtX3dTgLuqqpDk5wInAG8MMka4ETg0fQuLf16kkdW1b/PNR5J0viZbbF3SZJNwKeBopeALh5aVJIkjVCSTwCnVNXdzfIq4GNVdfQ8NnsYsKWqbmq2eQ6wFugv9tYCf9W8Phf4QJI07edU1c+Bm5Nsabb3jXnE82u95UubufZWr1iVpGFa84iHcNpzHz2Sfc1qNs6qOgX4MPA44PHAhqp65TADkyRphP4euDTJs5P8MfA14L3z3OaBwC19y1ubtoF9mjOJdwP7zfK9JDk5yWSSyampqXmGK0nqmtme2aOqvgB8YdC6JN+oqiMWLCpJkkaoqj6SZDO9q1buBJ5QVbfPc7MZtKtZ9pnNe6mqDcAGgImJiRnrd9eovmmWJI3GQv3O3gMWaDuSJI1ckpcAZwEvBT4OnJ/kcfPc7FbgoL7llcCtO+vTTAizF7Btlu+VJGmXFqrYm/e3iZIktej5wO9W1aer6lTgT+kVffNxGbA6ycFJ7k/vfveN0/psBNY1r08ALqqqatpPTLJHkoOB1cA35xmPJGnMLFSxN2dJjktyQ5ItSda3HY8kafxU1fFVdUff8jeBw+e5ze3AKcAm4Drgs1W1Oclbkzyv6fYxYL9mApbX0vvNP6pqM/BZepO5fBX4M2filCTtrlnds5dkzbSpoklyVFVdsmNxLjuf5bTUkiQNRZL/Afyvqto2fV1V3ZPk6cADq+q8uWy/qs4Hzp/W9ua+1z8DXrCT974DeMdc9itJEsx+gpbPNtNSv4ve/XnvAiaAHZOyvGSO+5/NtNSSJA3L1cCXkvwMuAKYopfnVtObffrrwDvbC0+SpLmb7WWch9O7Ufwf6d2DcCtw5I6VVXXNHPfv1NKSpDadUFVH0rvUcjOwDPgx8EngsKp6TVWZeCRJS9Jsz+z9Avg34DfofeN5c1XduwD7b2VqaUmSGk9K8pvAHwJPm7buN+jlPkmSlqTZFnuXAV8Efofej71+JMkJVXXCPPfv1NKSpDZ9mN4EKIcAk33toffl4yFtBCVJ0kKYbbF3UlXtSIK3A2ub3ySar/umpQZ+QG9a6hcvwHYlSfq1qur9wPuTfKiqXtF2PJIkLaRZFXt9hV5/2yfmu/Oq2p5kx7TUy4CzmummJUkaGQs9SVIXzfbM3tAMmpZakiRJkjQ/rf+ouiRJkiRp4VnsSZIkSVIHWexJkiRJUgdZ7EmSJElSB1nsSZIkSVIHWexJkiRJUgdZ7EmSJElSB1nsSZIkSVIHWexJkiRJUgdZ7EmSJElSB1nsSZIkSVIHWexJkiRJUgdZ7EmSJElSB1nsSZIkSVIHWexJkrTAkuyb5IIkNzbP++yk37qmz41J1jVtD0zy5STXJ9mc5PTRRi9J6gqLPUmSFt564MKqWg1c2Cz/iiT7AqcBhwOHAaf1FYXvqapHAU8AjkzyrNGELUnqEos9SZIW3lrg7Ob12cDxA/o8E7igqrZV1V3ABcBxVfXTqroYoKruAa4AVo4gZklSx1jsSZK08B5WVbcBNM/7D+hzIHBL3/LWpu0+SfYGnkvv7OAMSU5OMplkcmpqakEClyR1x/K2dpzk3fQS2D3Ad4GXVdU/txWPJEm7I8nXgYcPWPWm2W5iQFv1bX858Gng/VV106ANVNUGYAPAxMREDeojSRpfbZ7ZuwB4TFX9Z+A7wKktxiJJ0m6pqmdU1WMGPL4I/DDJAQDN8x0DNrEVOKhveSVwa9/yBuDGqnrvsD6DJKnbWiv2quprVbW9Wfx/eD+CJKk7NgLrmtfrgC8O6LMJODbJPs3ELMc2bSR5O7AX8OoRxCpJ6qjFcs/eHwFf2dlK70mQJC0xpwPHJLkROKZZJslEko8CVNU24G3AZc3jrVW1LclKepeCrgGuSHJVkpe38SEkSUvbUO/Z29X9DM1lLiR5E7Ad+NTOtuM9CZKkpaSqfgQcPaB9Enh53/JZwFnT+mxl8P18kiTtlqEWe1X1jF2tb35A9jnA0VVlESdJkiRJC6TN2TiPA94A/F5V/bStOCRJkiSpi9q8Z+8DwIOBC5r7ET7cYiySJEmS1CmtndmrqkPb2rckSZIkdd1imY1TkiRJkrSALPYkSZIkqYMs9iRJkiSpgyz2JEmSJKmDLPYkSZIkqYMs9iRJkiSpg1r76YW2vOVLm7n21h+3HYYkddqaRzyE05776LbDkCRprHlmT5IkSZI6aOzO7PlNsyRJkqRx4Jk9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSFliSfZNckOTG5nmfnfRb1/S5Mcm6Aes3Jrlm+BFLkrqo9WIvyeuSVJKHth2LJEkLZD1wYVWtBi5sln9Fkn2B04DDgcOA0/qLwiR/APxkNOFKkrqo1WIvyUHAMcD324xDkqQFthY4u3l9NnD8gD7PBC6oqm1VdRdwAXAcQJIHAa8F3j6CWCVJHdX2mb0zgdcD1XIckiQtpIdV1W0AzfP+A/ocCNzSt7y1aQN4G/DXwE93tZMkJyeZTDI5NTU1/6glSZ3SWrGX5HnAD6rqW7PoazKTJC0qSb6e5JoBj7Wz3cSAtkryeODQqvrCr9tAVW2oqomqmlixYsVuxS9J6r7lw9x4kq8DDx+w6k3AG4FjZ7OdqtoAbACYmJjwLKAkqXVV9YydrUvywyQHVNVtSQ4A7hjQbStwVN/ySuAS4AjgSUm+Ry9P75/kkqo6CkmSdsNQi72dJcIkjwUOBr6VBHoJ7ookh1XV7cOMSZKkEdgIrANOb56/OKDPJuCdfZOyHAucWlXbgA8BJFkFnGehJ0mai6EWeztTVVfTd/9C8+3lRFXd2UY8kiQtsNOBzyY5id4kZC8ASDIB/GlVvbyqtiV5G3BZ8563NoWeJEkLopViT5KkLquqHwFHD2ifBF7et3wWcNYutvM94DFDCFGSNAYWRbFXVavajkGSJEmSuqTtn16QJEmSJA2BxZ4kSZIkdZDFniRJkiR1kMWeJEmSJHWQxZ4kSZIkdZDFniRJkiR10KL46YVROuObZ3D9tuvbDkOSOu1R+z6KNxz2hrbDkCRprHlmT5IkSZI6aOzO7PlNsyRJkqRx4Jk9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSJEnqoFRV2zHsliRTwD/NczMPBe5cgHC6xDEZzHGZyTGZyTGZaSHG5DerasVCBDMOFig/gsfzII7JTI7JTI7JYI7LTCPLkUuu2FsISSaraqLtOBYTx2Qwx2Umx2Qmx2Qmx2Tp8m83k2Myk2Myk2MymOMy0yjHxMs4JUmSJKmDLPYkSZIkqYPGtdjb0HYAi5BjMpjjMpNjMpNjMpNjsnT5t5vJMZnJMZnJMRnMcZlpZGMylvfsSZIkSVLXjeuZPUmSJEnqNIs9SZIkSeqgsSv2khyX5IYkW5KsbzueUUlyUJKLk1yXZHOSVzXt+ya5IMmNzfM+TXuSvL8Zp28neWK7n2B4kixLcmWS85rlg5Nc2ozJZ5Lcv2nfo1ne0qxf1Wbcw5Jk7yTnJrm+OV6OGPfjJMlrmv+ba5J8OskDxvE4SXJWkjuSXNPXttvHRpJ1Tf8bk6xr47NoJvOj+XE68+NM5siZzJGLOz+OVbGXZBnwQeBZwBrgRUnWtBvVyGwH/qKqfht4MvBnzWdfD1xYVauBC5tl6I3R6uZxMvCh0Yc8Mq8CrutbPgM4sxmTu4CTmvaTgLuq6lDgzKZfF70P+GpVPQp4HL2xGdvjJMmBwJ8DE1X1GGAZcCLjeZx8HDhuWttuHRtJ9gVOAw4HDgNO25EA1R7zo/lxJ8yPM5kj+5gj7/NxFmt+rKqxeQBHAJv6lk8FTm07rpbG4ovAMcANwAFN2wHADc3rjwAv6ut/X78uPYCVzT/g04HzgAB3AsunHzPAJuCI5vXypl/a/gwLPB4PAW6e/rnG+TgBDgRuAfZt/u7nAc8c1+MEWAVcM9djA3gR8JG+9l/p56O1v6v58Zef3fxY5sedjIk5cuaYmCN/ORaLMj+O1Zk9fnlA7rC1aRsrzSnzJwCXAg+rqtsAmuf9m27jMlbvBV4P3Nss7wf8c1Vtb5b7P/d9Y9Ksv7vp3yWHAFPA3zaX7nw0yZ6M8XFSVT8A3gN8H7iN3t/9csb7OOm3u8dG54+ZJcq/C+bHacyPM5kjpzFH7tKiyI/jVuxlQNtY/fZEkgcBnwNeXVU/3lXXAW2dGqskzwHuqKrL+5sHdK1ZrOuK5cATgQ9V1ROAf+WXlx0M0vkxaS6hWAscDDwC2JPeJRjTjdNxMhs7GwfHZ3Ea+7+L+fGXzI87ZY6cxhw5JyPNj+NW7G0FDupbXgnc2lIsI5fkfvQS2aeq6vNN8w+THNCsPwC4o2kfh7E6Enheku8B59C7VOW9wN5Jljd9+j/3fWPSrN8L2DbKgEdgK7C1qi5tls+ll9jG+Th5BnBzVU1V1S+AzwNPYbyPk367e2yMwzGzFI3138X8OIP5cTBz5EzmyJ1bFPlx3Iq9y4DVzQxB96d3A+nGlmMaiSQBPgZcV1V/07dqI7Bjtp919O5V2NH+0mbGoCcDd+84Fd0VVXVqVa2sqlX0joWLquoPgYuBE5pu08dkx1id0PTv1LdRVXU7cEuS32qajgauZYyPE3qXpjw5yQOb/6MdYzK2x8k0u3tsbAKOTbJP843wsU2b2mV+ND/ex/w4mDlyIHPkzi2O/Nj2zYyjfgDPBr4DfBd4U9vxjPBz/y69U8HfBq5qHs+md530hcCNzfO+Tf/Qm5ntu8DV9GZZav1zDHF8jgLOa14fAnwT2AL8X2CPpv0BzfKWZv0hbcc9pLF4PDDZHCt/B+wz7scJ8BbgeuAa4BPAHuN4nACfpndPxi/ofQN50lyODeCPmvHZArys7c/l476/i/nR/DhofMyPvzoe5siZYzL2OXIx58c0G5YkSZIkdci4XcYpSZIkSWPBYk+SJEmSOshiT5IkSZI6yGJPkiRJkjrIYk+SJEmSOshiT2pRkn9snlcleXHb8UiStFiYI6X5s9iTWlRVT2lergJ2K5ElWbbgAUmStEiYI6X5s9iTWpTkJ83L04GnJrkqyWuSLEvy7iSXJfl2kj9p+h+V5OIk/we4OsmeSb6c5FtJrknywtY+jCRJC8gcKc3f8rYDkATAeuB1VfUcgCQnA3dX1e8k2QP4hyRfa/oeBjymqm5O8nzg1qr6/eZ9e7URvCRJQ2SOlObIM3vS4nQs8NIkVwGXAvsBq5t136yqm5vXVwPPSHJGkqdW1d0txCpJ0iiZI6VZstiTFqcAr6yqxzePg6tqx7eW/7qjU1V9B3gSvYT2P5O8uYVYJUkaJXOkNEsWe9Li8C/Ag/uWNwGvSHI/gCSPTLLn9DcleQTw06r6JPAe4ImjCFaSpBEyR0pz5D170uLwbWB7km8BHwfeR2/2sSuSBJgCjh/wvscC705yL/AL4BUjiVaSpNExR0pzlKpqOwZJkiRJ0gLzMk5JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6iCLPUmSJEnqIIs9SZIkSeogiz1JkiRJ6qD/DwxARpf6nCQPAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# TRID example\n", "# Make plots of how x parameter values evolve over iterations, and function values evolve over iterations\n", "# Expected result: ????\n", "\n", "fig_h, subplot_grid = plt.subplots(\n", " nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)\n", "subplot_grid[0,0].plot(history['iter'], history['x_D'])\n", "subplot_grid[0,0].set_xlabel('iters')\n", "subplot_grid[0,0].set_ylabel('x_d')\n", "\n", "subplot_grid[0,1].plot(history['iter'], history['f'])\n", "subplot_grid[0,1].set_xlabel('iters')\n", "subplot_grid[0,1].set_ylabel('f(x)');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 4: Solving linear regression with gradient descent + autograd" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We observe $N$ examples $(x_i, y_i)$ consisting of D-dimensional 'input' vectors $x_i$ and scalar outputs $y_i$.\n", "\n", "Consider the multivariate linear regression model:\n", "\n", "\\begin{align}\n", "y_i &\\sim \\mathcal{N}(w^T x_i, \\sigma^2), \\forall i \\in 1, 2, \\ldots N\n", "\\end{align}\n", "where we assume $\\sigma = 0.1$.\n", "\n", "One non-Bayesian way to train weights would be to just compute the maximum likelihood solution:\n", "\n", "\\begin{align}\n", "\\min_w - \\log p(y | w, x)\n", "\\end{align}\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Toy Data for linear regression task\n", "\n", "We'll generate data that comes from an idealized linear regression model.\n", "\n", "Each example has D=2 dimensions for x.\n", "\n", "The first dimension is weighted by +4.2.\n", "The second dimension is weighted by -4.2\n" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "N = 300\n", "D = 2\n", "sigma = 0.1\n", "\n", "true_w_D = np.asarray([4.2, -4.2])\n", "true_bias = 0.1\n", "\n", "train_prng = np.random.RandomState(0)\n", "x_ND = train_prng.uniform(low=-5, high=5, size=(N,D))\n", "y_N = np.dot(x_ND, true_w_D) + true_bias + sigma * train_prng.randn(N)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Toy Data Visualization: Pairplots for all possible (x_d, y) combinations\n", "\n", "You can clearly see the slopes of the lines:\n", "* x1 vs y plot: slope is around +4\n", "* x2 vs y plot: slope is around -4" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhIAAAIVCAYAAACEOBYHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsvXl4FNed9/ut6l3dEhJCYrHk2EOwbIWIQGPWuQkOMSFjJVxbLG9AwoaxEcYeZ/zGBmdsTfJe4meMgWHCdbAEXgEzAbPEfp03XkLC+A4YL4KEENmCEGxLbFpQa+m9u+r+0TqlWk51V7ckupHO53l4bEm9VHed5Xd+y/fHiaIIBoPBYDAYjFTg030BDAaDwWAwrl+YIcFgMBgMBiNlmCHBYDAYDAYjZZghwWAwGAwGI2WYIcFgMBgMBiNlmCHBYDAYDAYjZZghwWAwGAwGI2WYIcFgMBgMBiNlmCHBYDAYDAYjZZghwWAwGAwGI2WGhCExf/58EQD7x/7R/qUdNj7Zvzj/0g4bn+xfnH+GGBKGRFtbW7ovgcHQhY1PRibDxiejvwwJQ4LBYDAYDEZ6YIYEg8FgMBiMlGGGBIPBYDAYjJRhhgSDwWAwGIyUYYYEg8FgMBiMlDGn+wKGCjc98RvDj/38mbsG8UoYDCWCIKLdG0IoEoXVbEK+0wqe54bM+zEYQ4VU50665xwzJBiMDIW2OABIasEQBBGNV7rxwM5P0NzhR1GeAzuWT0XJ6OxBWWiu9fsxGEOFVOdOoueRdUQQBERFQBTFATc2WGiDwchAyOJw97ajmL3hD7h721E0XunG5+1eze8EQV83pt0bkhYYAGju8OOBnZ+g3RsalOvWe7+LnX60dAfiXqsaQRDR2h3EhQ4fWruDST031dclf2vrDuBChw9ftHtx0eNHJCIMyHszri8iEQEXPX580e7FhQ4fPL4gIhEhqXGZaByT97jY6U9prsab42QdefLQKfy11YvFdR8YXjuSgRkSDEYGorc4fNHuS2qhCUWi0uMJzR1+hCLRQbluvfe70OHHPduOaRYvvUVWz5Dq78IX73XJ3154/6+42BnAku3H8a2NR7C47gN8dqUb7d4A9Trl15/sJsNIP4IgoqU7gC+vxoyFq97YfWz3BvDZlW4srvsA39p4BEu2H8eXV/242OXHk4dOGRqX6vH25KFTaO7wSeMjHI5K73Ghw5/SXI03x8k6UuEuxroDpwbtQMFCG0MQlq9x/aEOYwiCQF0csqwmze/iLTRWswlFeQ7FaxXlOWA1K18nmRhrvMfqvZ/HH5YWr0NrZqMg2xbXJatnSJHnpkq81wWAB3Z+gpfvux0rXvlY8ZjVu+uxfsFErHjlY+k6JxS4cLa1R3q9eaWFeGTuLVi9u56Fda4TaGPwl0snw8TzaO0OouaN04pxsOa1E3hlxTRUuIvxbkNLwnEpH2+Ti3Nx76ybsfSFD6X3qqt0Y+vhM2ju8MPjD1PnDsdxkqFCm3fx5jgxMnIdFunvk4tzsXrOeOQ6LAhFohAEsd/jk3kkGIw0Qzslt3lDmFdaqHhcUZ4DvlBU8zu1USAn32nFjuVTUZTnkB6/Y/lUKd9C/v5PHjqF0xe78EW7F80dPqo7P5GngPZ+GyrKUHvkHACl4aO3qV/qjD2mwKVcmAtcNoQi0X6d9uOd3sjfTDwX14gj19nSE1Rcf4W7WDIi5I8brDASo//QxuBVbxird9cjy2qijgOeA3IdFsXv5Ma83EsVikQx6+/yUVflxr8vnqTxClTvrkeFuxgAUHvkHDZUlGnmzs/ePI0LHh+ae0Ntpy924clDp6R5F2+OEyODGCmTi3Px2HdLsP6tBizZfhxLth8fEE8f80gwGGmGtphV76rHnvuno+FSt+J0azPz0umDZhSo4XkOJaOzcWjNbEWyVbs3JJ1o2r0hbHmvEffOulla6IryHKircuO2MTmK00oiTwF5v33VM+EPRfHlVR82vdOIk00eAErDR29Tb+7w48ev/wkbF5bh2bdjz51cnIu180uwZPvxhKf9VDwm5JqK8hyICqKuV0V+nZGo0mskP/XJHzdYYSRG/6GNQWJA6HkIBBHw+MPSyT7faVV4DeQejnmlhXj42xOw5rUT2LxoEnV8kPl7ssmDTe80Yv2Cibi5wInGy93Y9E4jAKDDF8aa105IY3/zoknoCUTQ7PHBYTFjQoELh9bM1ox5YmRsea8RGyrKEIoI1BBHfz19zJBgMNKM3oZq4jnN4gCAumDEgywoemGEUCRKjaFW76rXLDBGci54nsOYHDs+b/fCbuHR2hMEoPWGJAqDPL7/lBROeGTuBDy+X7sAHlwzCxw46fvIc1g04Yan7iqFqdcFnOewYOfKafii3Ycsqwm+UBRfyc+SrmnH8qnY/8mX2LZsimLhJkaN/DrNJl5x/XobTzyPESO90MagLxRFUZ5D8hDIjevaSjdyHGYU5TqwafEkfNnuw9O/+RStPUHUVbkxymnVeKnIONIbHwXZNun3rT1BjMq2Ydvv/4p99c0AgLoqt/QaACRDe/2CiVj07AdxjWpi2D99dxkEQUAgQg+Z9tfYZYYEg5Fm4p2SaaeEVE4O8TwJxCAxssAYzbngeQ435TuRm2XB3lUzEBUBu4XHKKdNWuzIaUlu3GyoKJNOYc0dfny10IWj6+5ARBBRU16K2iPnJO9Gc4cfvmAUlS/2xZz33D89bkyaeHVI7Jv8jlxzyehs3P/Nr4KDGLtuQYTFxMMfjmoMokKXTXH9B+qbUFvp1uRIxPMYMdILbQyOdFqk+0g8BDeNcuJKVwA7j32Ou6fcIBm18jFbvaseu1ZOk8be6jnjMaHQJc0VmmGyY/lUjMuxY1/1TESiAswmHllWHsf+1i5do56nSx1q0/Mq8Dwn/b61Ozgoxi4zJBiMNENbzOqq3BCEWAXAQNR7x/MkjB3hgD8UMbTA0K5Vb7PkeQ4jnTbASb8medjFH47iXEuPJgwiiiKW7PhQs2ifbPKgKM+B821ehXHU0h2Ufl49ZzzVjbt+wcS4oRnaYiwIItUTRK5f7hFJ1mPESB/kHh5cMwuBsAATBzisJuTY+u4jx3H42Zun8W5DC+qq3BrP2LoDp1BTXorqXfWIiiLmlRZKYcKa8lJpXslDF+MLXXBY+sbHuFyHdE2CICrmGPGQJAq16XkV5KE+h9VkeP4mAzMkGIw0o85jCAsi2ntC+GNzJw7UN+HRO0v6nfkfz5PA8xzGjYgZL9W74p+maZtnfzZLsnELgghvMKI49ddVufHz3zRQF+31bzWgrtKNp359WvF67d6Q9DkTneTkv0vk2tUzMGi/70+smXHt4XkOhdl2ze/JfbzQ4cO7DS0A9L0DuQ4LivIcuNwZwBPfuw3LX/oIzR1+jReitSeIMSPsKMp16M4Z9Ryjbf60UBvNqyAIIj5v9ypCeV8tdOLgmlkIR4QBM3Yz1pDgOM4E4BMAF0RRLE/39QxVkikVBVi56GBB8hg+vdyl2Mw3VJRhy3uNePrusn5tUIk8CWYzj9vG5BgyEPQ21f5AM1AEQZAWcEJzhx+3jYk9zsRDMjwIB+qbJINILyad77JhcnEuNQGUwVBjNZswr7QQFe5iFMryGQikmop4y/71+6XS34kXoqa8FLeNyYbDajac1ySfY7kOqzQ3LGYePYGIbu6RHI8/hCtdAUUob+PCMpSMyUZhXtZAfD0AMtiQAPAjAJ8CyEn3hTAY14J2b0gyIgDlCdwfjpU9pnqCMOJJGAwDIdlrlL+/XjzXYTVLXgySkV7hLka+04rCbBvG5tgl747ay7KhogzPvv0p1s4vwbNvN6K1J8jyGBhxyXNYJH2QApcNGxeWKXIkaivdGOWy4uX//htONnng8SkN2JNNHqx/q6FflRHquTHKSQ+1AcpQhghoQjGP7z+Fvatm6IYcUyEjDQmO44oA3AXgaQD/M82Xw2BcE/TyGPKdVpxr6VGIIaUS6uivoaAuq8xzWNDhDw9aPkAiLwrPc5hQ4MKPvnOLJiRDvp+C7Fgi20WPH+3ekJRf0XCpG3tXzWB5DIyEdPjDUgJtc4cfz77diE2LJmF0jh2ft3lR8+vTUtXGyr//O1jMvOE8BPmc4jgOJg7geT7hmIyXyyOvzvrd//wWdU2JDrDgakYaEgD+A8BaANnpvhAG41qhl8cw0mnF07/5FMDA1X0ni3qBiqfiCCTXWEwPI16UDn9Y48VRJ0+KooiFtR8oXps8nuUzMBKhNvBPNnnQ6Q/jsdf/pPg9KZce6bQpQhHycatOfLzSFdRULb167HzKeVHq6qzLnX7qmmK3DKwWZcYZEhzHlQNoEUWxnuO4OXEetwrAKgC48cYbr9HVMQaToZSvkcr4pFZvVLoVJY9AekSO1AuUnorjwTWz0N4ToupVpNoOOd5mb0TXwmjJ6nCCrZ/GoY2feOXSrd1BafyOHeFQhBzkxvjL992ukeAmocxUDwvq+bD53TPYvGgSftxr9JD5OMo5sAZ0xhkSAGYD+AHHcf8AwA4gh+O43aIoVsofJIridgDbAWDq1KmsMw4jo0hlfNJO4CYeippyID2boHqB0steD4QFbHkvllyW67DA4w8bThZNpY2yxcxTjQSLue/ElUzJ6nCBrZ/GoY2fAp2ky6gg4u5tR6njlyjIkrmR76IbI2RupXJYUBs9J5s8ePG//4Z91TMHpX04IeMMCVEUfwLgJwDQ65F4TG1EMBhDFVK9QU7lFjOPnSunSeVkZHEy8ehX8mWyqBcovYoIMweN1PaGipiqXiJSadRl5jlN8tuWxZMAUfn9DGTJKmN4QXJx9lXPRDgqwMxzeO8vlzTiUnWV2nJlZZM6QTE3Xr7vdl19CKOHBbUHL9duxp77p6OlO4h2b0gqHx+TYx/U8Z5xhgSDMZzRO5W/+fBs+EN9pV8/eI5+6hks1KcyPRVHs4nXiECtO3AK+6pnJnyPVFqe+0NRPPt23ykvHBVgMfO45/ljmu+H5UMwUkEQRIXsOjGO3zh5QeF5G+m0UMuVyfgVRCjmxtbDZzVGMMmRMOIxM5K3VFflxoQC16AbzRltSIiieATAkTRfBoNxzYh3Kr8hLwut3UHJO6H++2BulEZVHC92+qnGgCgm9p6nkstgNZvQ2hNE9a56ALG+BE8c/PM1/34YQxfanJSrWQKxcbqveqbu+BUEEUFVn4uTTR48+3ZjrBQTkKo2nr67zJDHzEjeEq1fzmDA2ogzGBlEvFO5IIjwhyNJn9oJ8vbGqbThJiVnN+RloSDbBrOZV/zM8xx4jkNRnkPxvKI8Bzgu8YnISMvzRM8x2jOEwTBKvLJsQNt7hTZ+270hnG/zauZGa08QVrMJN+RlYVyuA6NHOKS5lGi+Gs1buhZjP6M9EgzGUEcd44yXPNh4pRuXOwMpVSCkksiYCiYOmtjxhooymAy8RSry2+rncL2GzLWs0IjXtpyRfvp7f/Q8ZeNyHTi67o64vVfI70ORKLYePkvNq9DTl0g0X9VrhV7ekjzxeLBgHgkGI02QxeLubUcxe8MfcPe2o+gJRKinGjPP4YGdn0iLUaJTu/o00+YNUkMm7d7QgH4mnufx6rHzqCkvxd5VM1BTXopXj50HzxtbatReDyMLvvw5Y3LsSXs1jKB3OqTdw8Yr3Ul7exiDA+3+fHq5C1c6/Ya9cnqesjE5ds041Ru/JARH5LL3rpqB9QsmYmwuPQlSL8R5uSsgXbOJBzYu7FsLDtQ34fllUxTXuXFhGXoCkUEfj8wjwWCkCdpisfylj/Dmw7M1p5pLvbkHzR1+aTEijYLGjogtHKR+nSRkyis9dv/j9AF1e+qd8vKdVjx6Z0naSi0HuqkYEP90qLfg76ueOeiZ8oz4CIKIy10Bzf2p3lUvNX4z4pUbiDElT1au3lUvJULm2CzUx+uFU/yhKL68GmvAFY4KikRjjz8MQRSxfsFEZFlN8PjDkgz8YOdJMEOCwUgT8RaLG1QNdeQu+5NNHmkxItUQ6o1u48IyFLhskvFB4rMD4fJP5HZNZdHtr/t5MMML8RJg9e7hRY8fnf7woFfTMOiQMeoN0nOKSD5BokRc9biSC0zpPYY29kgJqbw08xe/O6OrYKkXTvnyqk+Syt+2bAoKsq1SwicA7F89Eyte+VjzOQY7T4KFNhiMNEEWCzm0zV0QREAU8erKaXj5vtsxuThXkXtA2+ge338Kq+eMl15j6+GzqKt0D4jLX29jJWGSZMMT/Q0PDHZ4IV4CrN49JN/RQIeOGMaQf/+0++PxhwHE98oZGVfJjL0OfxhLX/gQC2s/QPWuerzb0KI7RmjhlI0Ly7D18Fnpute8dgI15V/D/tUzUVflxrzSQkkoS/15B1vAjhkSDEaaMFKlQBaqxduP47F9fwIA/PuSSdi5chreb7wCnud1N7pcR5/btLUniIJsG/aumoH3196Bg2tmpXxaTkXvQY0854Dmfk5mE05k2PSXeAZfnsOCuiqlgbahogy1R86xapE0QsZo7ZFzmpwicn/Iz3qbrJFxpfeYNm9Q83pG5w3xcIzMsmDvqhnYv3omdq2chmffjjWcm1yci7oqNzYvmoSIIODp33yK9W814OFvT0BulnlQcoQSwUIbDEaaMBIGIAtVgcuGx75bosj4rq1093bgBNUN6gtFpf+vrXTjqV//Ge82tGj09pMNC/S3d4U6NLJ/9cx+GSYDYdjEQ09iO89hwdnWHvzid2fw8n23o9MfVnQYHe79PNIJqWg42eSRcorynVaMGWHH//O//yLdn3ibrJFxpfcYXzAKwSkq5pHevHFYTbr5TURk6vN2H1p7gphcnKtZBzZUlGHTO41Y89oJ6YBwrVVcmUeCwUgjicIAZKFaPWe8RjFy9e56dPjDup6NScUjcHTdHdhXPRNbD5+RVPfkJ6tUwgKp6D3IUZ/i9NzPRjfhZEJERnU05I9t94YwocCFQ2tm4+i6O3BozWyUjM5Ghz+MB3Z+gncbWrB2/ykEIwLWv9VgaJNiDC5EOp0YE+vfakA4KsBp4/GzH0zE+4/Pwb7qmZLqI21sGBlXeo853+bVeMRo82bnymm40hWU5t89247hSlcABa6Ykf9uQwu2Hj6Dr43LQV2lG4/MnUBVjl09ZzyaO/wIR4SUKp/6C/NIMBgZDFmoEonNjM6JhS2iImC38Bjl7F1AnLGeE3rSvan0t+hvFrv6FEfcz/JTVjKbsJGmXMnoaBh9rPxzyE++t43JhsNqZnoSaUQtnU4qGLYs+QYqX/xQcV8nFLg0Etjk94nGVb7TirpKN6plstTEQ/Dc0smKa6LNGxEilm87pslvkqtmtnaHEI4KGOWywmU364Yx0+kBGzaGxFBqUZ1O2Pd4bSGbpJ4QFRGqUi928jbB8UIRqYYFyKknFWgdCl89dj7lDoV6hg3QVxLLcZxhg8mocUX7HOvfamBy3BmAWjod6PMU0Ep1affbSJiA5zmMzbUrSi43vdMoKVaqUc+bCx2+uPlNk4tzsXZ+CZZsP47mDv1GX75QNK0eMBbaYDAyGLJJTioeQa26IEJV8RLC4oUijIYFBhLa9ZAOhXru2ERhCbU7F4AiZHPRQ+8BQjOYjBpX/Q3xMAYP2r2pq3RLVQ+E5g4/wlGBer99waj0WsToJuFAObmOWO7Fj1//E6p31aO1J2h4HOjNP5Lf9MjcCVJTL6Cv0Zf6c00qHpHWUuNh45FgMK5XeJ7DSKcNuQ6rrlCVHPWmFy8UYSQsMBifJ5nQSCry3np5GEYSRI0mkw6G+BVjYKDdGxMfq16SU5TngMVEl6U/3+ZFjsOMK13BuGOvP+NAb/6NzrHh6Lo7EBVFxXWpG31lyphjHgkGI4OId/KmJVEZ9SjoJWDJF0F5IuFALky0z5RMQpheqMHjD+l+V3p5GEa8B+rT7LzSQuy5fzpCkWhKzc4Y1w51kmyewyJ5EyKCiJ0rp2nGQKHLpvD2zSstxM6V05CbZYE/FMWW9xrjevz6I4amN/9GOmNzw2Exa+Z3a0/QUBO8awnzSDAYGUIqJ2/aiaauyg1BENDaHTS0qMnjtgOtEDkQzcJooYYClw2XPAFFkpv8dfuTh0EW9zcfno1wREBrTwhLX/hQ8z6AVlF0MBqhMYyhHmukdHK1bIzUVbnx1j/NhjeoHN8kz2GUywpBhEJefkNFGVq7QzjZ5AGg9PglO7715pfe/MtzWDTzu7bSjZ+9eVpRyp3uMcc8EgxGmiGnqGaPD5c7+0q/jAgrqU80e+6fjl/87gym/9vvB1Uh0mgpZTKCPXrQvC6PzJ0gGRHy1yXfVSp5GPLP5vGHcMkTQMOlbmkjUr/PYAthMZJDfT8q3MWae1e9qx5d/gjGytp1A315Dm09ITy05wS1vJIg9/jpNcOjecsSzS/a38+29ihKj0kpd2t3SBKlutwZgMef3jHHDAkGI43IF49vPnsENW+cxmPfLcHk4lwAyVVQWM0mLH3hQ6pehBGMbozJGBxxBXsMGjg0o+DmUc64uSGphmzIZ/tTUyeqd9cjy2rSfR+9z0a8QUb0Khj9Q27QhiJRyQgHoFsy3dLbDVcOGS/jC+njioTA5CExQRDhC+p4yzoDmvnh8cefX3rzr8MflsKAoiiitTuEx75bgvVvNWDJ9uOoeeM0LnkCaR1nGWlIcBxXzHHcHziO+5TjuL9wHPejdF8TgzEY0BaPdQdO4dmFZVJPDaMVFP1VeDT6fPkpbHJxLmrKS+ENRhQtjgnJCPboQTMKsmyJc0NSEeYh94MYEB5/WPd9aJ9tXmkh2rwh1lb8GqA2aJdsP4618/uMcL171+4NIRAWNK/H8xw1J6Eoz4FxuQ6NQdruDUnN8AiTi3OxefEk+ENR1JSXYnJxrmQQBELx51ei+ScIIjiOw78vmaQRpareXZ9WT1hGGhIAIgB+LIribQBmAHiI47jSNF8TgzHg6C0enf4w1s4vwSsrbpdOuIk2o/4oPJJFijyf6PnvXz0THMcp3jsQjkpGBDkZLaz9AIvrPkDjlW5EIn0nchMPai+KrYfPJiVhrTYKRjlt/S69pH0P5H6QTShekibxlMwrLZS+q3/9/tfwi9+dYeGOa4BcPp64+QNhAf/yD7cBAA7UN2lKpp9fNgUjs6ywmeLnHKnvNy0kFopEsfXwWWxeNAlFeQ5J82H5Sx9hYe0HWP9WAx77bgkWu4tQU16KsCBKTfdo8yve/I1EBHx6uQuL6z5AS1ewXweGwSAjky1FUbwE4FLv/3dzHPcpgBsANKT1whiMAUav1DAcFRAIC+A5Dn9s7sSJz9uxbMZNMPUmEtISBfuj8Ggz83jmt59iQ0UZXj12HvfOulmjNElOYqZeg4Mm2/3Azk+w5/7piuTEV1bcjk2LJoEDEgr2GKW/pZd638PoHJvCgFh34BQ2vdOI9Qsm4uZRTmTZTH2qoQAmFLjwo+/cgupdSmVDveQ8xsBBQhm0HjQf/uTb4HkeZpOI3f84HYIo4ot2H/71jb+gtSeIuio3CrLt1HbfRseV1WxCQbYVNguP9Qsm4qb8LFT1JmkCfd7FnSunKZI3f7l0MgJhAT9+/U+KsadW0pxXWoin7ipFMBLFle7Y+KkpL4UgioZLma8VmeqRkOA47iYAkwF8mN4rYTAGHtoJ6Lmlk8FzHGreOI1vb/4vHKhvwl2TbsDSFz6M6y7XywsAkLDT5hftMRntTe804vHv3ko1EMip2mE1YePCMuQ7rboxaPlz73v5Y4xLUbAnHsRLMXZE7Lu71Ok3nJOgF4+OCCJ2LJ+K1p6gZEBsWfINfO2GHNw4MguFqs2nwx+WjAjyOvGS8xgDh9VsovaeWL27HjzPoyDbhhF2G0w8h+UvfYQVr3yMk00eKemSjGe1ZwqAoZBYvtOKp+4qxcN7TmLFKx8rxj2hucOPq96Q4vquesOSEUF+98DOT3DVH5Lm74c/+TZ+9J1bsPSFD/H3vWEbjy+MA/VN4DkOzy2dnFFCaBnpkSBwHOcCcADAP4ui2KX62yoAqwDgxhtvTMPVpQ6TmR76GB2fZPPfVz0TFz1+tHtD6AlE8MTBPyuyz9e8dkKz8NCkmNUSvEY7bWZZYxvdySaPYuGTP4acqnMdVozOsUMQ6V1HA+Eo6qrcUo+D2iPnYOK5QRFuSrW8VC+kFI4I0mIuCAKiIiCKIjjQX0vvdWjJeZnE9bx+EvKdVmrSbYHLhlAkigsdPljNJnAcdMdzf8qTeZ6DzcxL/TxGOq3U+aAOa+kl8PqCUcAZM2Jau4NUA7WmvBQ/fv1PeOaer2P9gokYX+iCw5J+UaqMNSQ4jrMgZkS8JoriQfXfRVHcDmA7AEydOpVlMjEyimTGJ89zGJNjR6c/jH/e+0dsXjRJsdAkatgVD/XJOxCO6mr1E0h+gJ7rlOc53JTvhMcf0jQsqqtyIxIVJEOoKM+BjQvL4LCaMNKpNHBIH4xUDAt5vX0yTcfI88hnon1GoviZaIMheSX7V89EuzeE2iPnpM6fJDkvU5QH1QyF9ZPnOWTZTJhXWogKdzFyHRaEowKy7WapN0VRngN77p9OvdcWM59U0zqaxkObN4T1bzVIoYhty6ZIRj+Rr/7F4TOK1/GF6HPwfJsXTpsZBdk2XQOVrAV2iwl5ThOKch0ZMbYy0pDgYrJdLwL4VBTFf0/39TAYg406NitfaBJt7PGQL0iTi3PhspuxcWGZpN9PEtBcdhNevu92ZFlN4DkOtZVuhZCP+lStJ9vN8yIWPKftZnhwzSzpuf0VqZI/X210kfekGVny5xW4bJrvQf4ZE20wtM9A8kuIXkUmLPBDnZEOq0Z0auPCMhS4bGjuiMnH//w3DaircivyWDYuLENPIAKrmTc0fmj3u67KrUisJWXXv1o1AxwgGRuP3lmChkvd0vNGOi3YvGiSIkdC3TFUL3eKrAVjRtgxhpLjkS4y0pAAMBtAFYA/cxz3x97f/Ysoiv8njdfEYAwqJCwh9MbpyaJ1oL4Jz1e68SBlY0+kRClfkFbPGY+H95xEgcsmuWN9oahUCvefH30hqeXtXDkNB9fMQjgiJFSBJNccC8vohwwIqbQulyN/fjJGlvx5zR1+PPt2o657OFEpnl7Z7r7qmcyIuIZ0+MMa0Sl1G+4K7wiMAAAgAElEQVR3G1rwvxZMVHTofPbtWMLv3lUzqONHLUFNu9/Vu+pRU14qGRDkvX76/a/hhrwsADEDxGbmsWnRJIwdYcffWr342ZuxmoGa8lLkO60Y4bBg7f5TUh+Q1u4gVdGSGKp1VW6MybbDbM6cFMeMNCREUfxvQCcoyWAMYciGPDLLopBzznNYqG2yE53s5ZUcxC1KFkHC3lUz8OBrJ6RFsbnDj+UvfYRDa2ZLC2KiaybXUVNemnBjH0i9C3l1hZ73RO99TzZ5sOKVj3F03R0aAyZR4y69zyCKIjMiriHxQgCEojwHBEHEilc+1jyf56AZPxsqyqCuDk2UCyN/L4tsg2/3hrD8pY9QU14KM88proHMwb2rZqC1J4iNC8vw8J6TUjLyhAIXDq6ZhWBYAM/FjPaf3/11RdVQppA5Jg2DMcyRC+xM/7ffY3HdB+gKRJDvtMJs5jWZ5EaUKOUhk6I8B7VO3eMPaxbfZDZ2+XXQdBd2rpwGEaKUFW8x84b0LvSQ19ufbPJI1RXvr42vYJlMy/RELcLT0X6doSVRG25y3xxW+uNEcHj12HnUlJdi76oZqCkvxavHzoPnlVuj3vuQBEvy88aFZTDLxh4xQHIdFqkDrfo1bshzYP2CiXj27UapquSBnZ+gKxhGe08IP9xxHLM3/AGLaj9Ae09m6pEwQ4LByBCS7d1g9GQvL5NUC/RsqChD7ZFzkkFBSLQp6kkTk429prwU7z8+B28+PBvBiIB7th2TSld7ApF+iUmpN/nWniDGjLCjKNeRsFzPyPsKgoi2niBGOMzYu2oG3l87BwcfnEX19GRSCd5whHYf6irdKB2bjf+WlUDnOuj3q9Blw6N39slNr3+rAY/eWaK5j3kOC2rV4laVbnxyvh0v33c7fv/jb+Hl+27HoRMX4JclLhMDxOOPlW6qjey6SjcsvZ4KojsCxOaxP0RPJM5EcbOMDG1kAsmWaDIY/SVZl7/D2pcgScos4wk9kS6HmxZNQkG2DV+2+/DGyQt4ZO4E3DQqC1e6gphcnJtQ54GWeLZxYZl0ojrZ5MH6txpwaM1sRAVoFsPlL32ENx+enXI5qJ5oEIC4lSBGxIYEQUTj5W48sKtPFOiJ790GnxBFW08Qo1wxQ6W/gliMgYHnOUwocGFf9UyEowLCURHb/+scjv2tHc9XunFrYZ9hSTq6+kNRREURdovJ8H3s8Iex9fAZKbfI4w/j/z18Bk987zaF2BSpUAJiY0lETBDrclcAa+74Krb94a9SbsRIpxX5Lgu8QYFa+RMVxX6FAK8lzJBgMNIMyYsw85xhw0AQRFzpCqLmjdOKRWx0jj3uqTjXEUvu2vDbT7Fi9s2omvkVPKgqVxuba0euQ39TpHlOHt9/CusXTMSKVz5WnM4vdfqpi6E/FDWcf0FLJk2kl6FXCaJ+HvWz7errI3LvrJsVG4X8NRO9FmPwEQQRZ1t7NEmJZ1t68ODueuxdNUMxzq50BaljJN9plcZZuzekMSZCkSjebWhRJFYCwKpvjqdWKFGrPCrdeKq8FO09IXh8YYzOseGiJ0hNqHzie7fBzPNUAyMTw2cstMFgpBGy4Dx56BQ+b/eh5o3Tkot17fwS7Fw5jWoY6G3mLrs57qmYnMCevrsMN47MkowI8hrVu+sRFZCSmNONI7Owd9UMrF8wEbbehLP+5BIk02V0oFp6yz+bngR4JrqWhyt61TOr58Q2+IhsrMRraZ9onOmNY/VYIBVK1CqP3fUw8zwKs22YeMMI2Cwm6rX/rx9MRDAsYHHdB4qeHfNKCzM2fMYMCQYjjZAFp8JdrJHNjWcYxFNmTIT8JJ2K61RvUf1raw+WbD+OFa98jOUvfSSd7PRyCWhNs4C+/Itmjw+XOwNS/kW8jby/lSC0z9YfITDGtSFe1UZRnoOa+Kh+bCAsYMt7jYqEyy3vNSok12nj+JdLp+BAfZPi9YiRHK+qhyRMhyMCfQ4LouQVI79bd+AU1s6/DaNzMq9iA2ChDQYjrcizupMxDBKVJxoh1degNQcjgjryaw9FonHzGdSu350rp2FElhmXPUGFWiZ5bZLRTtvI+/N9yMMnDqsJO6qm4oFdn/RLCIxxbdC7775QFM9XulHosiV8rM3EaZrU/XLpFPhCESzZflwKf5SMzsbeVTMk/ZLXjn+Be2fdrBCbIkYyqdCIN3b0rsekI+nd3hOEw8IDzgH9CgcE5pFgMNKIPKs7mRDAQFQNpPoa6uZg+6pn4tVj5xVZ5/JrV7cAp5WuFrhsuNIVwKmmLsmIALRNsFIt19RDHT75wXNHYbPwOPjgLJTdkKOpcslU1/JwhXbfayvduHVsNka5LDCZ+LiP3bF8KgQRmhDWQ3tOwGIyocBlk7xgfG/nXdJ8bl99M7X0GABMPBKOHb3rsVvp5dG+UDRjjVjmkWAw0ghZTLa812hYWAlIrY02rVeAzcxLin++UFTKbUiEPDwiCKJGBjjRhqt2/a6eMx6P7z+lK3lNXNVyTQq5d6PdG0KO3Yx91TNh4gCe5w1VUdBi2USMa2xuFkbniHGrQxxWEyKCmFABlNE/5GOX4zjFPVY3vav59WkpMVEuad7uDSHbHivp5TlABIdClw1XugPUMdfWE8TqOeNRvate8oKpvXHy0mOe5zQy7OsXTMRNo5ywm3kUuGyaxF/1HM5zWPBlhw9bFk/Co/v6JLSNJFKnE2ZIMBhphOc5jM6x4YfTvoIcuxkv33c7eoIRKas7UeJkvMZUiUIJe+6fLlUkEOSLbzKfoWR0Ng6umYVAWICJg1QCp4farUtCO3rhhKI8B958eDaudAWxfNsxRTgkGBFS7tuRKLdC/R1HIgIaW7pRvaseBS4b1s4v0fTrMPreDGMk6mtSMjoboihiYe0HiueR+5jo+flxunYSA9ZhNUnG4+gcm658vFqGnVQxbVo0CeGoAFOvV0Ov+qi1O4jlL32EApcNz9zzdYwZYYeJ5+C0mqXS40yEhTYYjDTjD0Wx4pWP8f3njuLOLe/j7m3HsOKVjxXCNkbRq3Tw+LUn75bu4IAmE7b3hLC0V4XvB8/pV1gAWrcu6Yh4uOEKti2bonFVj862UzUpvmj39ataI5mqEkEQcbHTLzV/Il4UVtUxuOhVZlS4i6XvO959TPT8qCBg40KlUNTmRZNwoL4JvlAUO1dOw5WuoCL81d4TwtgRWgE0PcO0INuGpS98mLD6iDz/ZJMHlS9+hO/8+/u4Y9N/IRwVMtaIAJhHgsFIOwOROEnQK3EjSWJywlEBL993O3KzLHDZzAiEo2jrCSX0JiTzvnreDbVb12E1YcfyqegJRBAMC9i1chqioojLnQFsPXwGT99dRl2ks6ymfhlDtMRRvbBMuzekML5YVce1IVFL7VAkirEjHLr3UU/LJNdhQYHLhkBEgMum9AbyHIcfzb0FY3PtiAii5AUjz6WN7UhvYvTvf/wtRAURO97/G/bVN6Moz4Ev232G5sZArgXXEmZIMBhpJpnNLBF6i25UhGKBmlycCzPPoeaN0yhw2fDI3Am4MT8LjlAUHl84riBVovedXJyL1XPGI9dhkVzL8TqHEnJsFnx2pVvREnpDRRlau/vCNOpFlngy1Asvx3G676u+BqO5JqFIVDK+sqwmjHRaMa+0UCFSdD0s+tcbVrMJ80oLUeEullQlD9Q3SWGwWPt6er4B8Q7Rxkg4KmDt/BJUvfiRYrxtPXwWv/jhZIzLsaPDH4Y/FEVNeakkCgVoDcZwOIqLXQG0dgfR7g3hQH0THv72BORlmVE+6Qb86xt/UXwmPYNzINeCawkzJK4Drme57uv52q8VAym3rHeisVt4xQL1yNwJeHTfn1DgsuGx75Yokjw3LixDbpYFI53G8yTI+9Jez2jeAK0l9LoDp7Bp0STpO1Evsl8tdKKuyi2FG8hm8LM3T0vxcyPGRLycEJJzwvOA02bG4/v7lEC3LZsCAFL79eth0b/eyHNY8E9zb8GDMgPz+WVu1H/ehroqt/R9qxOASV7ErL/Lx7ZlU7BGpuC6edEkjM21Y+mODzXjbf2CiXDZTFTFTFKGLDcYBUHEmZYeTcnyc78/i5ryr6G5wye1CCfoGZzXq/Q6J4r0GOb1xNSpU8VPPvkk7mPYhjb0+PyZu4w8LO0z0Mj4HCjiSUUDkJIwo6KIbz57BHVVbqx/q0FjeKilhY2+7+XOgCTbLX89IwmcFzp8mL3hD5rf/+GxOSjOdcBs5jWaD1e6gtjyXiMq3MVS/4LaI+ckl3KyiaN6n4u0SKd9V/J27yks+sNqfKZCS3cA98hCC0Dse3/t/ukYl2OHxaLdkFu7YzkNxLB99dh5aYwUZNvAcUBLV1CToAkARx6bgyybifqeZAzIjWPyXrTH3jomG2aeg8cfVhi711FSrqELZB4JBiNDkG+SFjMPM8/BH4pqyhzjnVQSnWjIptraHURRnkM3zh/tVZdMpqR0dI4NTlvqOQt63pTP27xw2cxSYpv8MxCDiYQXyAK+r74ZzR1+CIIQt5FXIuS5H4XZNupnI2qFjIFHEET4gvRwXWt3EFlWMwpkhgQZj75QBDXlpcixm6WEWPkYWb9gIkJRgTresmwmXdXJW8fEqpNGOfvGoC8UoT4232mFmecwdoQDY0c4pDlJylc9/hBEiAiEBKmJ2Chn5lZmxINVbTAYGYC62uKebcfQeLkbD+85KWV5f97uVVRjfN7uRUt3QCMxTROAUstR5zks2LF8qpRjIKcoz4Evr/riZpfLr/nJQ6dw+mIX/tbqBc9xmFdaqHk9I3kD+U4rtc351sNnqYZIvCQ8AJhXWog2b8hQrw495O/hspmp3xXHXX8L//VCuzeE821e6vdODFiCfA59a+MRrH+rIWaAupQeqeYOP7KsJtQeOadp671j+VSMctoko3ZycS7qqtzYu2oGXr7vdrR0BdHeE1K812eXu6nXV5Btw+hsu2QYiBARjsa6y25+9wyarvpw5nJMVv6bzx6Jzfkkx2emMOCGBMdxdw7Q68znOK6R47i/chz3xEC8JoORqeg14SLNhx7Y+Qm+kGV+EyXIe7YdS7hJ0kpCz7b2YEKBC5OKR2g2782LJmHzu2dQ4LLhcmcAzR6loSK/5i3vNeLeWTdj/VsNWFj7Af7H9uP4p7m3SMZEMnkDpM35+gUTpZ4Hm95p1O2AqlfyR5LwnrqrVHInk+802fJM+XsEwlFsXjRJ812ZmB0xaAiCALuFx/OqkuDnl8X6XMjHBW0OPfjaCTwyd4LiNYlK5MkmDza9E+uxsX/1TOyrnimFG/KdVuxcOQ1r55dg/VsNWLL9OGreOA1BFPHrE0240h2ANxjzehxuuKIxSGor3bCaOYVI1T3bjmHOpiOoeeM0ls34CkIRcciUDw9GaONFADf25wU4jjMB+CWAOwE0A/iY47g3RVFsGIDrYzAyjkSna3KKIuhpGNByAhKVZuY6rDi0Zjb8oQg+vdyNZ377GQAkTJoMRaKocBdr5IUf3F2PfdUz8dPvJ583kOuwYswIOzVrnabMqU6+rKtyY5TTKrmR+1ueKU/w7AlGkGU1KZRA7RbecIUIwziCIMLjD6GtJ4TH95+SVCJvzM9Ca3cQgbAgiUkR9O73TaOcUgijKM+BHVVTYbPEZKhPNnmknIcxOX3eA57n4LKbsfwl5dh+8b//hofumIAl248rEivfOHkBNeWlmFDowtmWHtT8+jRae4I4tGY2AK3+yUN7TmDXymlDpnw4JUOC47g39f4EID/1y5GYBuCvoij+rff9fgVgAQBmSDCGJHr5AR5/WPp/n0ygKhkNA6Pqja3dkJIJ66rc1BbackOFGAkDmTeQTJOvHcunYkKBSzcfhOSB9KcmX3494UgUi3s3EPnrbVo0Ce3e0PWSPJfx0BJ3mzv6VCJfu386su1mTYmy3hwy85xSBt7C48a8rISVEbQ8iQp3MR7ac0JT6UGSMGvKY14wApljtDkiqEqyyfVej+XDqYY2/i8AdQA2U/71DMB13QBA3p+1ufd3DMaQhNbAZ+PCMtQeOSdtml/Jz9IoQcrRW4SMqjfKr8GIoZLvtKIw22b4OoxipMkXMWw6/GHNY2mfh1xXKuWZ5HpE0DcEDrhuXdKZCLnXemJjZp7DSEpSIu1+x6qS/oIVr3ysaHEfb9wQaPNGz3DOd1ql+Uog80Bv/pl4TqOoeb2WD6ca2jgOwCeK4n+p/8BxXCPl8clCM+sVAVqO41YBWAUAN97Yr0gKgzHgJDs+1SdxUrXx3NLJilO5WgnSiHCNUZEb9TUkOi3xPIdxIxwaHYfBWAxTCVMMdE1+PK/R9eaSzuT1k9zrZNu40+63IAgKwTDAePiANm8Keg1n9TWNy3UgEI5KehHqeaAJwVW6kZtlRrbDhb2rZiAqAnYLf91WbfRLR4LjuFJ13gLHcXNEUTzSr4viuJkAfiaK4nd7f/4JAIii+G+0xzMdieHJcNeRoDXn0luE4nVPpD0nnh6F+vHJXEeq6NXq91cnIhn0mj+RhNAUr2XIjs9UUWtApCJupn6tVMcNLS9HLVRF8nIsZl63E+y1mCODhKGL7K8hcRrALgDPArD3/neqKIozU37R2OuaAZwBMBfABQAfA1gqiuJfaI9nhsTwZKgaEoO16CRjHAzmdRi5TiPdS5PdVAbi8wiCiDZvEL5gFOfbvNh6+Cxae4L9ERi67sbnYKNuxf3I3Am4eZQTWbY+nQWj95K8lly0rDDbhnEjYgJnqV6fJOwmiPj5bxoUyqbxxsF1aFBcE0PCCWADADeAbACvAdggiqKQ8ov2vfY/APgPACYAL4mi+LTeY5khMTwZioZEspt9MiQ6nSW7yA3kokheSxAEtHlDVBVAILEgV7zXH8jvdQA/+3U1Pq8V8b7fZO+lvPW73ItQUphNNSb0DFn179p7NUqMejsGc24PIoYurL86EmEAfgAOxDwS5wfCiAAAURT/jyiKt4iiOD6eEcFgDCX0kgoHIpEvXp6BXvvxRGJUqYg9qcWxIhFBeq0/Nnfqaj/QkjCNYuR7VV+X/LOo/wYg5WthxIcWTmj3hqTv3uNPbo50yOSpgZgGS0tXEBc6/WjpDmjuM21cq8XgYmOdrn6pl38xmHM73fTXkPgYMUPidgB/D+CHHMft7/dVMRjDFNpmX+CyIRSJUje4ZIhXvZHsIpfqokhdqFtirmeimzEYtfWJkjX1NpBwOIqWrgAaLnX1SyGTYQxBiCk/yr/rz3rVU8nPlzwBqlql3hhRd6Z97LslqHnjNL61UasmqTeuv6C0AScddeWok0HlBmgoEk3quq8n+mtI/KMoiv8qimJYFMXLoiguAPDGQFwYgzEcUW/2k4tzsXZ+CZZsP97vTSxeOWSyVRGpVFEIgojLXQHNQl29qx4V7mIAkDL15QxEbX2iEljaBrLlvUY0tvTgLxe7NF1Jh8pJMtNo6wnigV3K+7B6dz2Wz7xJ+rl6dz1VrVJvjMjv/eo546n6KORe6o1ruRgc+Z0oinHLi9XG6ZLtx7F2fgkmF+cauu7riX4ZEqIoagJroiju6s9rMhjDGfVm/8jcCQMmoysvjzu67g4cWjNbis8a1ZogJPt4sqhe9PipC/VXC2JlcDl2s0aGuq7SDRMfi3XrhR4SkUhTgraBVLiLsXp3va6ewVA4SWYa/jB9Ix8zwq74+eZetUpAeS+JB+BKpx8XPX5c6PDBYoIkA5/I46U3ruVicOR3VrNJdz4B+rL3xAi6nnUj1LDunwxGBqGuhY+K4oBuYvLumXKMak2k+niyqNaUl1Lr8L+86pOUC3+5dDJ+tWoGIlER59u8eKpXbri20o2th88YzpBXf+54mhI0jQgiPpSsngEjdUw8R/2uTbLGaEV5sQ6deuqnpP/LugMxae2180vw8tHzqCkvxbhcB/X1SeM1vXFtM/NKme3esa43nwB978b4QheOrrvjeqnaMAQzJBiMDEO+OLV0BwxvYokqCeL9PVnxpmQfTxZV0nFRrg2wcWEZnn07pmPX3OHHQ3tOYl/1TFS+qJSjXr27HjXlpXi3oUXyzCSj3RBv0Y8nPkS75qFyksw0rCYeGxeWSV44Mj7aemIeOPLd04SbSFv5mvJS6V7VlJcq2oj/74dna+7lhooyqfFaPIl2tVjcpU6/lAza4Q9r5oFFZnwQivIccFhM10z75FrBDAkGI0MRBBE9gYhmYaVtYnqlZRMKXOjwh+OWVcqNiWQWuGQeT0788o6L+U4rxo6w4+E9J3GyySM9trnDj0iUnhFPmpiRnwcqvEDbQHLtZtRWurF6dz02vdOI9Qsm4qZRTjitJphNfRvJUDlVpgu5gWsx8bghz6HojVGQbUOe05LwFE+MVXn4Qh3KuNgZwIH6JtSUlyLXYYHHH8arx87j6bvLpMfojWtSIi2fZ/NKC/FPc2/Bg7vrNfPO6NwdCjBDgsHIUNq9ISx/6SMUuGzSwucLRTE6R3sa08s233P/dCx94UOpqZD679dKGVJ+4pd3XHRYTZKsMKEozwGziX6aI03MyM8DGV6gbSC3js7GvuqZiEQFmE08CpxW/LXNS1U2jKcUytBCOnxe8gRQLd+Iq6bia+NyEAhHYTbxKHTZYnoPzvivR4xVeShKHZaqPXIOa+eXpLy5q+dZhbtYMiKAvnm1r3qm4bk7FGCGBIORoZATFqlsIBxdd4dmUdWLx7Z0Bwe1rNIo8VzGtJh0ocum+T3JkQCuXaKa2cxjTI5dOjG3ekNSqSrQV3VCDLXrQGAoI1B3+JRvuJe7Ahg9woYbcxJYDiqIsbrlvUYpfFF75JzCK9DaE8ToHDsOrplFlbJOhHqe6c0r4lEzMneHAsyQYDAyFL0mUfE6fKofO8JhweTi3GueMKiXj0HzfujlWqh/n+ew4Om7y/DT71875U29/hqt3SEpHNPc4ce4EXbUlJfCG4zgclcAY3LszJigQO6JPxzB5c4ARrms1J4adZVuTZvwRPeTjJmn7y6DIAjYVz0ToijCYTVJhgPJb/CHUhsT6nmmN6/0PGpDNUG3XxLZmQKTyB6eDEWJbDlGJHXjSUtvXjQJPMdhpMuKDm8IZhOHh/ec1H2tgSITpIAH6hr0ZMVrykulk+a80kI8dMcEPLTnRCrvdd2Oz2Sh3ZNfLp2CUETAo/v+qPmO5WG3gbifA/EakYiAC51+tHYH0e4N4cTn7Sj/RhE1R0Ld3Os69VYZuljmkWAwMpRElRG0xK/X7p8Ojy+M7kAYNguvMBy2LJ6E16tnQBAxKEmC8hMjabgkuas7AxidY8NI57XJVtfLGaHlhMQ76eqFjEhIpSjPgSe+dxuWv/RR2vJPrhdo9+ShPSfwnw/MoH7H/nBMup3nuaTupx4efwiXOwPYvGgSPP4wao+cwwM7P8HBNbNQmG1P+HxBEDXGQW2lG5+cb5OSh+UNwQayhX2mwwwJBiODiVcZoV5c321oQcOlbtSUlwIAnjj4Z8XC++i+P+HgmlkYa2DRlGMkRCA3ajYvmmTYXT1YGFXeTHRK1QsZjct14P9bewcEUYTHF2aCVQbQvSdRgfodn2vpgTcYQcno7JSUVOUIgohLnlg+hjxEtemdRviCUUQcArWEUw7NmCElycQ7JfekJFsFdT3TX4lsBoORJvQW1zE5dt0ksHAkuZ56RptzyRdZjz+MR+ZO0EgRV++uv2ay0kaVNxP1DKEpYtZVueELRfDDHcfxz7/6I5w286DIeg819O6J3cyjrsqt+I43VJRh6+Gz0r1IVklVTbs3JFWGALH7vO5ATGWytTuIxpbEY1xvvg1WSfL1BDMkGIwMR68rpd7imptlgYjEDYWMYLQ5l3yRrT1yDjfmZ6X1lJ5IEpt23bTrVMuKv149A1lWEzy+MGrKSzGh0IXW7gCeXzYl4XsNd/IcFkmqGoCUI7Hz2HmMzLJg76oZ2LtqBmrKS7HpnUacbPJI98Lo/dRD7z7fmJ+FYCSq23FWjt58C0cF1FW5sXfVDLx83+1wWIefAclCGwxGBhPP9Z7vtKKuyq1IsNxQUYanf9OAn//fX0ddpVtRn79z5TSIEHGhw2c4ZmvUpSwPAZxs8uCSx5/WrHWjyptGKmOIi/qqN4jGy90KDYJty6bgud+fRWt3COsXTMTNo5zIspmoyovDGZJf8OYfm7Fz5TRc9YbQ7g3hl384i3tn3YyrvjDysqz48et/ot6LZJVU1ejd50seP+wW/V4qCrEsM4+dK6dJ+TBFeQ48v2wKBFGUwohkfl6rEF6mwDwSDEYGE88jwPMcRjmtqCkvVZzk3m1oQTgq4LaxOdJJ+s2HZyMYEXDPtmNJdRHVO4VxHIcLHT60dAdw1RtEKBLFnvunY15pIQBg5wefo1Z1+hzIU7qel0YOMQBuyMuSYtZqkjnp+kNRTQO1Na+dQIW7GCebPFjxyseofPFDcOCG1SZiBDKOp9yUj+UvfYSFtR+gelc93m1owboDp2A18TBxiHsveJ5DvtMKq9mEUCTaW61krOqQdp9rK90oyLZhzAg7dYw7rCZFWO+ebccQjAg4sHqmNN8igoCHehOageHbGZZ5JBiMDCax651XKFYCylMcSfYifQiSzXqXK1IWuGx4ZO4E3DQqC19e9WHDbz9Da09Q6pXR2hNEXZUb6xdMBM/zyHNYBiVrnVat8tRdpTD1Jkcm8z7JnHT1GqixGHliaPLVhOYOP3yhKHKzrLr3QhBEtHmD8AWjON/mxdbDZ9HaEzRcUsnzHCYUuLDn/uno8IUxwmFBa3cQXlFEhy+Ml1fcjhUvf6zwKkQEkTpn9lXPlDwne1fRK06G2xjIOEOC47iNAL4PIATgHIAVoih64j+LwRiaJHK9G+3CmWrWO1mAX6+egbaeEB58rU8rgWS9P77/lJS5Xr2rXmGcDEbWutxLM7k4F/fOuhlLX/gw5Xp9o9n1dgv9Xqhlux1WE1q7g8Oi7E8PdaWPw6qVr8khJSUAACAASURBVCYU5TmQl2XRFS3TEwXb9E5jUiWgHf4wlr7wITYuLIMvFFGEqLYsnoT9q2ciKojSPbvUSW95L4qiNOdYZ9gYmRjaeA/ARFEUywCcAfCTNF8Pg5E29FzvJh640OFDuzeECQUuKYRxaM1s6iban6z3Dn8Yn13ukYwIoC/rffWc8YpT+bU4jcmNotVzxmuqQwbLtTzKaaO6xw/UN0k//+cD03GpM5CwAmAoQ6v0udIZxM6V03CgvgkbKso0VTDjRjh0jS1aeE8+9oyONzJuxuTYNSGqR/f9Cf5wzOghYTC9OWMx8xidY8PeVTNQVjRCU3EyHBNtM84jIYriu7IfjwNYmK5rYTDSjdr1bjHz6AlE8IPnjuqewEn+gPxEbNRzQSMUiSLLSk9Iy3VYFKfya3Eak3tpBrKHiFEJZj3ZbofVhJ5AhFoBMJzEqdq8lDDark9w8MFZGvlqjuNg4mLGaqKunnLkY8/oeCPjRi9EFRVExbihzpmqqegJRBQJlztXTku5d8dQIeMMCRUrAexN90UwGOlEnetAU1Ek6nzxqjxSzXq3mmPtnGkuXBGQciQG4jRmRPxKvsAPlGvZqHwyzfUuvzekSZqc4RYzD4TpG38gIuDGkVkAkpOr1gvv8RyHPfdPjzVT6w4mHM/5Tit2rpwmPV/9em09IYwvcCmeYzPzipbmFjOHZ377qWL+LX/pIxxaMxs35GWl8G0NDdIS2uA47nccx52m/Fsge8yTACIAXtN5jVUcx33Ccdwnra2t1+rSGQxDDNb4lJ/OJhfnoq7Kjc2LJiEUFhCJCAmrPBJVMdDId1rxlfwsbFk8SeHC3bZsCsbl2nHb2Gw8t3SybljFKGRzefLQKZy+2IUv2r1o7vAhohLRknsGvpGCa5lW8WFULyMepJLgehCnGsz108Rx1O/AJBsWet/35a6AJgxEC+/teWA6RufY0NIdxOmLXXjy0KmEISRBEGMNu8JR/HKpUvdjy+JJKMy2KcZNuzeE5S99hBWvfIwl249j6+GzaLrqx9r5t6Kuyo3JxbnStQ8nQ5FGWjwSoih+J97fOY67F0A5gLmiTlcxURS3A9gOxJrODPhFMhj9YLDGJzmdFbhseOJ7t0rZ4yTWPMppHfATMc9zuCnfiREOs0ID4Lnfn8WK2TfDNSZ7QE5jbd4gtrzXiHtn3ayU1q5y47YxObqegYJse0JPS7zmZjuWT0WO3dzv781qNkk5AOrrz7SY+WCunw6rSdG6uyjPgY0LyxRCTXrhiosePzp8IZQUZsNsjp1z1SElh9WES50BjX7Klvca8fO7v07tmyEIIhpbuqXnzCstxM6V09AdiGCUywqH1aTRfpBf42J3EZbN+IqiMRtJ+GztCWacoXitybjQBsdx8wGsA/AtURR96b4eBiOTyHdaUVfpRncwohDvae7wo3pXPV6vnjkoWeQ8zyEYFhRhFQBouNSNvatmAM6+x6bavjsQjqLCXayV1lZVgtCuLV7+gdyNXlNeqiiXlZf0zSstRIW7GLkOCzz+MA7UNyX1veU7rXj0zhJsea9R08RpOMXMcx1WjM6xK0ICo3PsyHXEjClBiOVG7F89E+3eEGqPnMPJJg+K8hxo94awfm8D9tw/HUV5WdL3pg7vqfNQ1h2IVQ4FwoL0HvIxKEJUPEfel2aUy0ptJic32lfPGa8JKa47cArrF0zUeDKGIxlnSAB4DoANwHscxwHAcVEUV6f3khiMzGGk0wqX3Yya8lJpEQZ6e2kIgiZBLBVFSzWCICIYFehJarLzbCQi4GKnHy29bZYP1Dfh0TtLUDI6GwDiGhgmLiY4NNAeFbkbXS8502ri8MjcW7BapgRaW+lGnkwjIhHk5Pz03WXDuvSTeLCy7RaqHgStlPPVY+dx76ybsemdRjR3xMaPw2qmGog0b0aBy4ZbRrsgiiJaugMIhqP4a4tXMmRuGeOi3vcxOXaFsUhTsvyi3Yer3pCimy3pHnrjyCzkOS3D7h6ryThDQhTFr6b7GhiMTCQSERTuWbl7FQAemTsBogjku6x48+HZ8IdibuArXUEs33YsZZ0FILYZf97mo3o77JaYC1rtPpa7nJ+pKMOVrqBucp0giOC5mJGUyKMiCCI8/hD8oSiiogi7Jb4ktXzj0UvODEVFyYgA+jo7JlttMZw6PsZD73vQK+V8+b7bsXb/KYVnojjPgZbuAALhKEwcB4fVhBybBVFBVNzDycW5WDu/BFUvfiSFLR7+9gRFp8/aSjfmlRbi3YYW6VqK8hzId1klb4Ke0NnfFcTcbf/yD7fi0X19ocSNC8uQZeUlT8twJhN1JBgMhgpBEHGx00916f543i1YO78ENW+cxj//6o/4y4UudPkjsPTGmPuTREgSE32hCOwWHpsXKRMu6yrdGNXrFm6X5R7Ir6/CXQx/KBr3Otq9Ifz0zb8gFI1iW5wGWIIg4vN2Lxovd2PJ9uP45rNHcM+2Y3ET7eR6ALVHzml0DHYsnwpRpyRwuCfRDTR6uRGd/rBkRGyoKMOJz9vR5g3hnm3H8M1nj2DJ9uNovNyNz6968drxzxXj8JG5ExS6EBXuYqxRaZ6s3l2Pn3zvNsV937xoEix8n5y53MhZ7C7CE9+7DS3dQXx2uRstXUHJiCCv+fj+UzDx/LD3RgAZ6JFgMBha2r0h3dLC4pFZWPbChyhw2fDYd0uUiX6VbhS4bIrnNXf44Q9HE4Y6aG7o55ZOxjP3fB0WEw9fKIob8uySKxgA9b3ynVbd2n3yvFAkincbWtDaHcLa+SXYtXIaoqKILIsJY2Q5Bu3eEL5o90mnTfI68bQa5OWiJ5s8ePXYeey5f7pCUptUWySTW5JqLshQJ973olfKOWaEXcqZePXYeTx1V6mkVgr0bdzrF0zElJvy8cxvP8Mz93wdY0bYYTHxitfLdVioYQhwUORt2C08zGZO0lwhY3RycS6WzfiKpjkXbWwHVBVFwxVmSDAY1wHy0kL1ImziOTR3+FFTXqpNVNxdj/ULJmLFKx9Lz5lXWohQREB7TxC+UBRfyc/CTflOzSZIc0M/vOdkb8Lin7Fz5TRc6lSGK4imBMnbKMpzoDDbpisvTTZqssGcbPLghzs+lP5+aM1sTSa9njiWnvfASD+NZAW7ktFBGE7Qwm/y70Xvex6TbYfNbMLYEXZMubFM13ORZTUhmzfjZJMHlS9+BAB4+b7bFWNLEEWsnV+iqRpp6w4p5kFRngO/WjUDZ6/0IMtqQr7LJiXckuoM8r4PvnZCM4/UJa3DGWZIMBjXAfLSwlePnUeFuxj5TisKsm2wm3kU5Tl0EwlvHuWUFloSP77v5Y8Ui2xulkWTua63mN82JrYpixCl3AvyN3JqXPHKxxr543gbtdGNPJ44VjzvQaLchWTbVFPVG2XCYMMRvfCb3FsU73uW35/W7iD1HvtCUYwv6BvPRXkOfCU/SzF2OI7DY69rwxDP3PN1xfXGjE9BkUuxbdkUiCIU+Rer54xHrsOCcbkOKc+CVtI6nGGGBINxHUBKC399ogkP3TFBUc/+H0u+gZfum4oLHQHq4ptlM0kLNwAs2X5cs8iqSziB2KZNK4kk2fQXOnxUQ2N8oRNH192h2YzjbdRGN3IijqXWKRiI/gbJJErqqjeGh6+rO174Te4tMvI90wzLjQvLMDrHjnEjHJpxAkD6XUSgh9HG5cYMW3m56RftPsVcWPPaCexdNUMq+1SHCrctm4J/+vYEtPWEFCWtwx1mSDAYGYDRPg/3zv47LK77QLH4/fPeP2LTokmwW3g8v2yKokPnjuVTFRUNX171UhfZKCVPMc9h0ZREPl/pBodYAqal1xOiNlwgAmNztdoJRrwCiTYYUlqYm2XB3lUzEBUBu4WPW7UxGBD1Rk2YaRi7uuOF35LVMSHj/eCaWQiEBZg4KESj9HJhGq9043In3aA+29KD9W81SF69R+begppfn1a8RnOHH4IoYkNFGUIRQRMqJIZGvssGExcznlhuDDMkGIy0k0yfB73qAg7A/9jxISYX50pJZkV5DoxViSHp5SqQEk45Hf6wpiTywd31kqjTzpXTUFfl1pR7/vw3DXj67rJBK4PkeS4WhnEmfuxAoTb0XHaTxmjbsnjSsHV1Exnz0rHZeGXF7Wi66peSGotHOlLyFvE8l1SYiOT0FLhsGnVRUiZNKon2Vc+EzcKhtSeoeA3ipdj87hlsXFRGnWvhqIjKF4+z3BgZzJBgMNKMXt8BWhWCXta7LxRzHZ9s8qB6Vz01URHoa4WtNlpGUZT94nVdbO6INSt6ffUMRXb8pndiiZY//X5mlk2mUmlBM/RqK92wmjlFFUCe04ocm3EBq6FCJCLgsyvdWL27HgUuG/7lH25V5B3sqJoqPXYwK13IeG3u8GPTOzF10QmFLpxt6ZHGJRAbw6IoIs9h18yFuio3fvG7MzjZ5MG5Vi91rp1v8xqaq8MJZkgwGGlGb8OmVSHQYsebe8Ma8gQ0vZyBZJIK9YwWQRRRV+VGrsMCQQAO1DdphH4ysfdAqpUWNENvNaUahhhvw21DaekJSp6rmvJSjd7CA7tiGy0JPQxWpQstp6e9J6SQRAf6xqdeW/hH7yxBw6Vu1B45p8nFqat04ylKOGS4640wQ4LBSDN6GzZtM1YvfhzHwcQBFjOPg2tmIRwREp70jCYV0oyW55ZORjAsSIszOZ0DkLLZByLxkRCJCGjpCSIcFWAx8Sh02aRmTsmSjOdHTrxSRPXvhuOGEpZJp+tVDpH8CSPff6r3nJrTs2wKNW+IjE/aXJDPL4uJx75VMxDpzcUx8/RwSCYaztcSZkgwGGkmWQ0DORYTP2jJXmqjJSqIuNgZ0JTWrd5dj33VM/HT74sD6q6Wu8zlRktJoQueQCRp93gynh9CvAZTJJxEGK4bisXU5w0TRBEv33c7sqwmSQiKdMdM9P0Lgoi2niC8oQg+b/Nh6+GzaO0JorbSjVtHZyc0Jqg5Pa+dwDP3fB3rF0zEzaOcyLLFl1NXw3GcQhBNEMSU5+pQhhkSDEaaSSbccK2FkOQnNkEQAQ7UzSAiiANesSB3mZP32Xr4DH409xZU76YLHsXDqOcnXstxecZ/VBAMhZOGOoUuG2or3dh6+Ax4jlPkR5CSzUTqoXrNvDa90ygZquNyHdT3J/fLF4pQx+aNI7PgsJpTzoeRj69k9UaGC6zXBoORAZAN+4a8LEm4h4aee9ho74z+XqPDYpb6FRCK8hw419KD2Rv+gLu3HdX0vSD9Oi50+NDaHdTtiaEmTOk2WuEulowIILnPTzw/en08yLU2XunG3duO4o/NndTeIT/7wUSUFLowdkRMGfH9x+fg4IOzhmXmviCI6PCHMSbHhp9+/2ua1vaP7z8Fiym2zcT7/vWaea2eMz5mqEb72oPLx1IkIkj367PL3dSxSXRPjNwbI/PL6FwdTjCPBIORgehlt6finu/ve8rREwp69u1G6Vrkce9UPSiCIMLMa0MKidqMx/sMRk6TRlqOi6KIv7Z5tZUvruGVZKm+t/tXz6R+X1e6ArBbTCjItlG1IQD9sFNhtg1FeQ6YTTx1LO25f7r0M2nIJi/73LF8Kkw8EvaVIcivQ65qGYpEYx1qmdFAhRkSDEaaUW9+eQ4Lzrb2UDffZBIzk70Go1oW8s0YAB7ec1IqrQOUG3sqCY56bu5Xj51HQe/GYtQ9rv4M8RJNBUFEKBLF5kWT4PGHEY4K1PfiOC6lpM2hhvre6oUuAuGowtBt7wlpcwxc9PbxIxwWvLLidhS6bNSxJFfSPNnkkco+bxuTDbvVhJ5ABD947qhhI5bML7mqZYHLhkfmTkA4KiadYzFcYKENBiONyF3pUmigpRtb3mukuleNuOdTIZmQidy1azWb4maxp+JB0XNz//T7X0Nulhl1lW7D7nGjYQ9yH5ZsP44l249j/VsNcNnNmrbpO5ZPhUknT2S4VWyo723tkXOaFvAbF5Yh226WPA969wgiNPd1Q0UZnvntp8ixW2A289SxRIwXwskmD9a/1QCH1QwOnNTBU/5e8cYDmV+PzJ0gGRGPfbcENW+cxpxNiVvWD1eYR4LBSCO0hbV6V0w9Uq7NQDaqwUr2SjVkkqjiJBUPit61RAQRj79+Cq3dIWoWfn/CPnqdTrcs/saAtBwfiqjvLfFKEZEujz+MZ99uRGtPEAfXzAKgf29be4LIdph1xM0E6vsBMQ0TtboqGX+XOv1Jjwcyv5y2WIdZWkfd4eh9SkTGGhIcxz0GYCOAAlEU29J9PQzGYKC3sKo9DPKNKpnmUkbR2/AdVhNau4MJe4DoGTaplLbqXcvfWr24d9bN2PROo9RdVK7e2Z+wj959GD3CjqLeviHyENSe+6fj579pGBTtjOuFPIcFe+6fjpbuINq9IRyob0KO3YIfPHdU89hwRN8YmFdaiEhUxLkWr654FEAfS4/eWYIJBS4cWjMbgiAgKgKiGLtPDmtq40GeVBxPF4PRR0YaEhzHFQO4E8CX6b4WBmMw0dv8CmW5ANdio6It0jtXTsOVrqChvAk9wyYVDwrtWkgpYGtPEDXlpdIJVL6g90ePQ34fSJJdvtMKs0w/QJ5/Ma+0EE/eVYqn7iqFuVc0abjEzQVBRJs3CF8wivNtXmw9fBYF2VY8eVcpOAAv33c7th4+K3koEhkDT3zvNix/6SNqjwy1eJTeWNJTzdy5cpoU3khmPJDr1GsANty8T4ngRPH/Z+/dw6Oq8nzv79p1T1WSCrkBJgrNYDSNoUMh1zndKC2tR5RBULsxQbCV0IzHHl8F7GkzOm/0HAFtTvsqneAFBLQFQUcHxlvbTfcMF5VIw9DRSNOoCQIJIYGkUqnbXu8fxd7Zu2rtql2VSlJJrc/z8DykUrX22pW1v/u3f+t3Sb29HkLITgA1AN4GMDmWR2Ly5Mn00KFDUccc88ie5E2QkxJ89dTNet426OoebX1qBQiOz3eg3eMf0Fz18KBPCorbNuyPEFFlRkZ/9U0QRSq7pqXCRtKNafuyabhz40FmSepE5yT9HdZ/2Ii7Z4yNuJkVZlnkoL3yYmdEe+k+1PJI6fUZDmu9PnP7RFhMAu5/7XBEJk9rlzfiu1H+jQghaO30Yt7zIS+GMlOiKCey6ZxyHsq/s0GA/PeRKMqx4c0VM0BAElqjokjR4fHhdEdPQnVLhgm6TjLlPBKEkFsBnKKUHiEkLf5QnDQm2lNW+A0y2hZDsuaiPOap9m5Nt25/F8YSLsUkKOsSAKGbQ4fHr/l0mei2j/R3ePzWCRFt2u/bcgjbl02TX1s+a1za7puzYkkeeuMIauZNUL22cudRbF82TXM7TFnkLKDIjonVdE76TPjaq6twId9hUa2V5nYP/AERl+VkRD0nLeNT6jLrtJl5AaoYDErWBiHkd4SQY4x/8wD8EsC/6BhjGSHkECHkUGtra/9PmsOJg3jWZ6wCN8zMjiRHjrOKRknufiWSW3cgCmOxMlTqKl34XlE23loxM+lPhdHatAcp5HkMh33zRPUznr4jAGIWbBIEgtHZob+r3kwkZoDytno8MHu86n1FOTaYYpTV1nNt8QJUsRkUjwSl9Ies1wkh1wAYC0DyRhQB+IwQMoVSeiZsjI0ANgIh11z/zpjDiY9krs9zbi/zpv3mihkoyLTGPZ7euhXj8x2aMQdaEfEef2ThnkS3G/TGVyRzi0UrZsVqEuTvQvKIDOV980TXZ6w29srXtEqPh/+djEYBV4/MYv6dlZ+xmQ0IiBQeH9uYGZtnV8UVrVtYhoseP4wCgdPGXhPJvrbSlZTa2qCU/jeAAulnQshX0BEjweEMZ3r8bOHs8YtxjxWrOqA0tiSmWjdyrRvKiZYudHr8GOW0wmkLPVH2ZQsk1laFKFJ81ebG123dyDAb0O0L4orcDIzJtSdkTGgFbI64dC6v3TcVRkKweem1WLLp0wELhk0Vcu1mbLlniur7Lh7R60nQ+j4CARHfXvCoMjwevKFE1cMi/O+sXKv5DgtW3ViClTuPonpuqYYhJzBTT2vmTcDIbCtzzSXz2kpnUsqQ4HA4kRgIYQpnIk2yYlUHlGhu96DbGwTsYN7IY2VWSOKd6zD3axXIDo8PZy/2RDSKcmaYMMKeeKyE0njS8ti8c/9MeHzpt2/uDYiq71vyXml5jkSRorGlM6L52foPG/Hk/DLNdaBcq9VzS7FyZyguhVUKu7bChfZuH5Zu/jRinAyzQXPNJfPaSmdS2pCglI4Z7DlwOIOF3IWSUmy5ZwqeevdzuW7BuoVlcrXAeIhWHTBcTE+ec8NuMTKFXrrhSkGIygJCQK94v3bf1H6NJ/D4gvINRhpbCvSDPbExw5+OWzvZ7u+3VsyMGcg33Eik5HmbooOq9JnVu0KehWjrQLlWlXEp4aWwjQYBj79zDAtcxcx13OHxa645m9mA5xeV47zbL3tYRthNCV1b6UxKGxIcTrrC2oLYcNck/K/rx+Nclw+FWb1bB/GgWR2wwqVKcZO8C88tKtccK1ZmRXO7R/OJL1nxBMEowZHJIlrFzP5MgU0Fws8vkeqh0YquRVsHyrUaHpcilcKWvCAfNLSgtdMX4amQ1rHWmsuymGAQBJWHpbbChSyLKZ6vKe3hvTY4nBSE9eS34tXPMMJuxoTLsvscA6CMkH/whhJclmNFzbwJ2L5sGqrnlspbFFJlS60W4Kzx1iwoQ+3eE3JlzP7oDSJhNfVmlpQXO1FX6cLO5dNhvBSolwy0sldMRqHfs2kGE1ZGQ1Ckmpk8rM+3dnoRpBSbllyL8mKn6jMFmZao60C5tmr3nsC6hWXyseeUFuDVe6fKtSjmlBbInoqnbrsGv3/oB6iZN0Fex6w1J4oULV1eLA9rS798Wz3aPf6Ev7d0hHskOJwUROspDmDHLChJpJU2AIzMtsZd2VIa7+37Z8DjFdETEHHmggf5mWb879uugdNm7lMefqwn/jy7BS8snqxZSKqkMBMAoo4R6xiseJC6SheoSId1F1CWMfvEnoaI3hZ1FS4YBMgGVJvbB1EUcU6xpRFepKquwoUsW/TbjyAQjM93YEfVdPiDIkRK8f/95HuhYmkUuOvFj+Wxf1PhAgB80NCCR978b2y5Zwq+OzoLv/5JuapduYRkJLm9gSGfypsKcEOCw0lBiMaWQKwibYm20hZFCosi6r3bF4TJIDC7J76+bBrMRkHVTrnloi/iRjs+3yH/PpEbq95ziVZI6s0VM5htq6Ux4jmGZAwFRYon9jTgp3//nWF9E2IZsx80tOCxW76LN382A92+UInsR//tGFq7vNhyzxR4AyLu23II1XNLVX0zpNiV1++bhuMtXfJnomXwiCJVBblKZckNAsGPNx5Ujf2zbfXYvHQK/vl/liLDYsAIm5kZICsdSzKStDNAeIxEPPCtDQ4nBTEQYM2Csogtg1jR5IkWimpz+7D45U+wdPOnuHPjQSzd/CnOXuxh3ihPtXtU7ZS1Opj21T0cPm6+w4IzF3rQ3KHeZolWSKrHL0b9PvR+X5LxZTYasOjFj/FBQ4u8b69kON2EtLZ0vjzbhYBIUfHSx1i6+VMcbupAc7sHX7d1y99leNGu8mJnKLgyKMIXDKVWxlqbyr9NebETd88Yi7te/Bin2tk1TNq6vKh46WMQELR7/FH/rpKRJGWA9NfWW7rADQkOJwURBAGv7D+J6rmlctzCK/tPQhCiX7KJttKOls2hRBlIKQlzX9p3652T1N+i+u1j+P7avRHxCFo3PQNB1LnFO3fl+4f7TSjXbkZdhSvCmH32o+MIBMWI7y3DbJBfUxpZ0t+uZncDrn/mj6jZ3YCHf1SC8mKn7u9aWZZcy4BTZmfE+rtK60WZAbJz+XTsqJqeTn00kgY3JDicFCTXbsaDN4TE986NB1GzuwEP3lAS8yYVrax1vJ/bVd8UUbpYCqQEeoU50WPGQjmuVn8L6QmTFfT5wuLJcitprbnFO3fl+6WbUM28CfjTquv6pWz3YCIIBKOc7CBco0GI+N66fUH5NaWRxfrbrd51FMtnjdP9XSs9HCwDThngazYaYv5dletFygCxW4wYmWUdNn+/gYTHSHA4KUgi7beBxFtpsz734A0lcrCbxxfEN+e7VbUiJGHuS/tuvXOK1d8iWhBptLnFO/fw97d2eTEy24oiJ7tL5VDHaTNHBOG+sHgyChyWiO/titwM+bXDTR14Zf9JvHbvVFCwvULSd6nnu1amfyoNuMtHZIACWPnGkYjsjGh/10SvLw6blGwjHi+8jXh6MhzaiPcHfWmlzfqcVIb67MUeufgTK2ixP+opKMe9UxFgB6hbmvfl+4h37kk+15Rfn9HWRfjrQGSGTJvbh/kbIlt876iaHtMDIBdl08gCKcyywplhYlYYHe41PgaIodlGnMPh9I2+tNLWqmI5JtcOZ4YJ25dNQ5ACVpM6ayPRY+qdkyjShL0eseYW79z761xTlWjrgvV6+GtaXh892wjKY+RnWvHmihno8YtySqfcjItRxTTd/k6DCTckOBxOTASBhHpXJFh2OhnH567ooUmy/naCQHhHzhSFGxIcDmdIwJ8why78bze84VkbHA6Hw+FwEoYbEhwOh8PhcBKGGxIcDofD4XAShhsSHA6Hw+FwEoYbEhwOh8PhcBJmWBSkIoS0Avg6xtvyAJwbgOmkAvxcezlHKb1xoCbDgq9PFelynoC+cx0q6zPZpOo6SMV5DeacdK3PYWFI6IEQcohSOnmw5zEQ8HMdegyX84hFupwnkF7nGi+p+t2k4rxScU7h8K0NDofD4XA4CcMNCQ6Hw+FwOAmTTobExsGewADCz3XoMVzOIxbpcp5Aep1rvKTqd5OK80rFOalImxgJDofD4XA4ySedPBIcDofD4XCSDDckOBwOh8PhJAw3JDgcDofD4SQMNyQ4HA6Hw+EkDDckOBwOh8PhJAw3JDgcDofD4SQMNyQ4HA6Hw+EkDDckVek7dgAAIABJREFUOBwOh8PhJAw3JDgcDofD4SQMNyQ4HA6Hw+EkDDckOBwOh8PhJAw3JDgcDofD4SQMNyQ4HA6Hw+EkDDckOBwOh8PhJMywMCRuvPFGCoD/4/9Y/wYdvj75vyj/Bh2+Pvm/KP90MSwMiXPnzg32FDgcTfj65KQyfH1y+sqwMCQ4HA6Hw+EMDtyQ4HA4HA6HkzDckOBwOBwOh5Mw3JDgcDgcDoeTMNyQ4HA4HA6HkzDGwZ5AuiGKFG1uH3yBIMxGA3LtZggCGexp6Waoz5/DSZQxj+yJ6/1fPXVzP80k9RkoneB6lBpwQ2IACQRENLZ0omprPZrbPSjKseGFxZNRUpg5JBa/KFI0nu3EfVsODcr8uWhwOKnPQOlEso7DdaXv8K2NAUIUKb694JGNCABobvfgvi2H0Ob2DfLs9NHm9skXLaBv/qJI0drpxan2brR2eiGKumucRIzTeLYT8zfsw8w1f8D8DfvQeLYz4fE4HE7/cM7tZerEObc3qceJpUd6tIfrSnLghsQA0eb2oaXTKy96ieZ2D3yBoOq1ZN18k40vENQ1f4lkXqSJGDEcDmfg6fGzdaLHLyZV26LpkV7t4bqSHPjWxgDhCwTR5vahKMemWvxFOTaYjQb558HePtBCFCkIIdi5fDra3D7U7j2Bw00dEfNXorxIy4udWD5rHNzeAM5c7MHILGtc5xOvEcPhcAYHAyFMnbMYSFK1zWw0oCjHhnyHBctnjYPTZkK3Lwib2aBbe7iuJIe0MyQGaz/MbDRgV30T1iwow+pdR+ULqa7SBYMAnGrvhtloAAVlWshvrZiJ/ExLn+eRyPmzjJs1C8rwyv6T+PnsK2EQQu9RjiOKFB5/QL6QH/5Rieq84xUQk1GIaYRxOJzkkohe2MwGrFtYhpU7e6/3dQvLIFIkrG2seeTazdhyzxScvdijOlZdpQs5GSZd2iMZI+G6YjJyZ308pJUhkcyn/fCFnWMz4aLXD48viCClsJoMyLNb5HFz7WY8eEMJ1n/YiOq5pci1m1GQaQEhwK3P7ZPns+2nU/vNQo73/KVz9PgDOHOhB/kOC5rbPWhu92D1rqPYvHQKVr5xBK1dXtU40nHOXOhBUY4Ny2eNky9k6XziMY5EkaKrJxAhTi8snoxcu7nP3wuHw4kkHr1Q6qHVJCA/04KaeROQYTag2xdEYZYV3qDI1DaPL4DWTmgaKdHm4bAa8dS7J1E9txROmwkdHj9+/bsvsfqmq3VpT67djBcWT1aNvW5hGbp6AsizUx50qZO0MiS09sOUNzSW5St9Vmk0HG/tUi2+zUuvRbvbhwd3HGFedIJAUFKYiSfnl8njGIReI0Kaz8lzbqaFHBRpxFN/OLGeHvScv3Islhfi6fcbcbipA83tHrR1eXG4qQMAVONIx8l3WLBmQRksRqFPxlGb24fFL3+CfIdFFoyQOFn4hc7h9BNaevHO/TMRFEPbAiajALOB4FRHD6q21iPfYcGqG0uwad9JLHAVI1MwYly+HVk2Iw5/c4GpbZ+f6UTN7oaIhxFJywghWP9hI1O3qEhx94yxKo/DmgVlMBsI1i0sg8kQXXsEgaAwq9fo6fD4sfa9RrR2eZPmBU4H0sqQiLUfpmX5WowCFr/8ifzab++bFnGBNZ33oPrtY1Fv0oJAVAvzVHt3xHye/eg46ipcqNpWr7owntjTgCfnl2kubD1PD/HsB7JEZPWuo6ieW4qqrfUoyrGhw+NnjiMdp7ndg6ffb8TahWV92pZQjle1tV5+fd/q6wC7riFSHl6jgJNqsPQi32HB6Y4elT5tuGsSnvv9cTS3e1A9t1T2Gn7Q0AIgdK1vXzYNz350PGJrV3o4Ueplrt3MfIhp7fTJDy6S3hBCIjwOq3cdxW/vm4a17+nTHo8viKWbP2WeP0cfabMRJAULFuXYVK8rF5WWBf51W7fqNW8g0kWXYTbE/dQt7c8pae3yItNmRPXcUry1YgY2LbkWWVYjFriKIYqi5lh6oo+lOAOt81eiZXQ4bSbZ/Ve79wRzHOV5HW7qwKqdR7Hu0gUtvTeebQnW98TjIzic/oV13T0we7xsRAAhTVjx6mdY4CoGADhtJqZuBGlI255+P7S1+/uHfoDquaV4+v1GAEBdpQvP3D4RvkCQmT66etdRLJ81Th6zKMcGQghESpnH8wdFtHZ5sWrnUaxZEF17uL70nbTwSEhP6+s/bIywiJWLSuvmmWFWLygDQYSV2+0Lxv3UnWMz4bV7p6Kl04s2tw+76ptw94yx+KatW/7///rtYVUQUX4mO9vBFwiqXP/+oAiDQOAPBvFthwciDaU9vbxkMu7Z3Gvp11a4kGMzRYynFYRUlGPDmytmoKsngNYur/y68nsM33ds7fKiMMuKN1fMgD8gxh3kytrH5PERHE7/wrruLs/NYGqkdC12ePxM3bCaBHms2r0nsO72ici1m/HQnCvhsBpx/2u9OqeMEysvduKhOVfi8twMCITgo4d+gNZOL6wmAY+/cwyP3lzKPN63HR45purp9xtRM28CxubZkWFRx65pnSfXl/hIC0NC+bTe2umTgx1HO22qVCCtzIDwTOdzXb6IwL8RdhN+c9ck/OzVz3QtRlGkEXEWzy+ahFcPfo3jLV145o6J8nYKANmtr7VvZzMbsOrGEtWcnl9UjqbzHjz0Rm/cxvo7JmLdwjIIhKDD48ezH33J3DLRurhGZdsgCAR5doq3VsxkxmNI8SBav4+XZI/H4XBiw4ofON3hYWrkCLs59GCy9wQzKDrPbkGe3YJdy6ejpdOLJZt6t4rXLSxTBXJLcWL5DgseuekqvPRff4uIg/i/d34PrZ0+PLGnAXWVLlW14LpKF/LsZpiMgq6HF64vfSdlDQlCiAHAIQCnKKVz+zKW0tNwuKlD3mf/+BfXq4ITpQCd8LSlK0bY5IunKMeGXIcZJgPB5qVTIBBApIDFSJBpNWL7smkIUsBqEiIsXyWsrYh/fO0zVM8txY76ZnT2BDS3SlhBlQGRyvOW3nve7Y+I23hwxxE5zkHisVsit19iXVzh8R6sz0cLYOUXKYeT+oTHD5QXO/HconK0u/1yRkaewwxfMCgbHA6LEa8vm4agSGEyCChw9OpgkEJ+2AJCmrRy51GVJr3736fx6r1T4Q9SNJ3vxtKZYyO07Z+2/1n+TM28CbqNAC0tCtczqXAW1yx9pKwhAeDnAD4HkNXXgVhu+jmlBTjn9qkt2QoX3vrslCqVaO17jXhuUblqobKyNl5YPBmXOTPgzNC32KLFIMwpLUCuw8ws/mQzG5hBlSMyIvcmteI2nIqtjGjbL7GMBT0GQjJSblO1SBeHM9xhaac/IMoPKEU5Njxz+0TsOtCM2aWFuGqEA+e7/fjxxoPMa5VqxDQUZFpQV+nC6GwrKIC7XvxY/vxv7pokeyzCP1OUY4MgCFGD0CWNMhkFdPUEVIHzLB3hehM/KRlsSQgpAnAzgBeTMZ7kplcG3Dx6c2lE34uqbfW46ZpRqNpajzs3HkTV1nq0dnlhNhqQn2nBZTkZyM+0oN3j73NZVa0An5HZVjww+0r8eONBLKw9gJrdDXj4RyWYU1qAFyono8cvMo8dpIgYT4rbCD9Gty8o//+FxZORYzPFXbZ2IEvQ8jK2HM7gEK6dD8weL6e4A6Fr8aE3jmB2aSFqdjeAgkTtJ6Sle9k2E2p2N+DbCz1YEeax+Nmrn+GB2eOZn9lyzxRN/QrXqNs27MfZi6F6OKy5SXC9iZ9U9Uj8XwCrAGQmYzCWm17LIzA2z67axlDGOUjWbbcvgOq5pbKnQPpsPOlCrBiEugoXbCaDbI1L467edRQ7qqaj2xfAtx0e5rwppRHjjbCb8MztE1UxEnUVLuQ5zPjTqutgNQkYYTMzvSusfG6l10GrMU94DEcyStDyMrZ9g6eWchIlXDuDGh4FSc8MBCqPrqSR0rWqpXtPvfu57C1ljX95boZKl5+5fSKeevdz/J8F1zD1qzDLAo8vGKFRK3cexVO3XYOKlz6RXwvXEa438ZNyhgQhZC6AFkppPSFkVpT3LQOwDAAuv/zymOOGu+lbO73MoKEMi4G53xarQFO86UJaMQinL7ANhUBQxJJNn6J6LjtK2Ww0oKTQJo9HAHx9vhsOixGbllyLLm8ALZ1e5DnMKMzufSJo7WQbBG/+bAbyHBami298vgPdXn0Xm1b2RzzflVZNfas5JR1qAOJfnxzOQBLP+lRqp5ZujnbaUOCwoLElVFxKqZGv7D8pX+8s3RNFUa45oZn1YRTw2/umwR8U8W2HB0+9+wUON3XgsVvYHlopXoOlUaOcNpQXOzV1W6lZUp8O6WEyEBBh5OWzI0jFb2QmgFsJIV8BeB3A9YSQbeFvopRupJROppROzs/Pj/sgrO0OKbpYuY0h7YlFy21ONF1IukCVx9Jy/QkCQXO7B7V7T2jmRSvHMxkNWLnzKG55bh9uWP8nzN+wHzW7GyAI6j+5lvXt8QdxrottZLR0eeXI6vB5moyCys2YYzMxv+d4viuppv6qG0tQs7sBd248iOq3j+F0hxeBgHZtjcGkr+uTw+lPEl2fWro5MsuKdo8/Yltj9a6jePTmUtX1Hql7vfVtWPq2ZkEZDALBT144iOuf+SMqXvpENgK0PCRSlglLo75p646q29I5ziktwMM/CmnOwtoDuHPjQXxxtjNlNWcwSTmPBKX0FwB+AQCXPBIPU0orkn0cLY8AgIhoXQCaT+BXj8yUq7ElIxBHK+3SbAhdbIebOuSiLrl2M0ZlW+WUTD3jhF80Wh6Dv7W6MSaPnTPuD4rMKnV1FS5mMNP4fEefUqsEgcBmMkREbi/fVo8dVdMx2mmLMQKnP4ln64Rvmwxtoumm1KRPSXO7JyJ9PhylVh1u6sAr+09iyz1TcMHjR0unF6/sP4kn/uEapp5ZTWz9krZVtCpp/uqOiaiZNwEWhndBOsfHbvku7rwUNCqdC9ccNilnSAwkrJSfcFd+XaULI+wmBETKXLA2s5EZMcxq6tXu8ce8mWpdqKJIUVvhwvJt9Tjc1IGa3Q2orXChUKNAld7c6Fy7WR43/GJ7YPZ45jn7g1RVpU7aasjPtOC23+yPGTeRCD6Nhj+BIH864HAGEi3dlJr0hevF6Qs9yFDoZHgfDQMBCrMscs2HgEjx5J4GfNDQIhfNG5FhRp7DwjRgWE231r7XKD901cybgOIRNpxodePp90N9NAIixdLNn6Iox8bUJ0EgCIjaVTNj9T1KN1LakKCU7gWwd6COx4rWrdpaj+q5pdhV34QNd02SI4qlJ3CWi55lkNRWuPDsR1/KF0e0dCJW2qUgEFxVmIkdVdMRuFS10igQtHv8UY2SWDdwQSAoyLRg60+noOWiFx0evxz3wer7UVfhwsY/9lr6Ut+NugqXZmpXMoKUjAJhipSBX8wczqAS3qRP6QFYt7AMVpMgl/fXijV7Zf9JPHhDCUoKMyGKFI/fOgGP3izCaBCQbzerHsLCPbAlhZl4fdk0nLp0k7eZDXLV3dYuL3IdZqx97wtZe9csKEOPP6RJ0fTJZGAXKPQHQ4YQb+jVS0obEslET82DaLUdpGCgTUuuxXm3D92+IEY52d4AlkGyfFvIIPmgoSXhJ3WjUcDILGvSc5xHZJjh8Qfl7A6J1i4vRjmtqqcAgwDs/1sbjrd0qbwRo5xWBMXI0uHxBlZqYTEKEYbchrsmMV2THA6n/1FmsTW39zbpk3ShINOC/2fHEbR2ebGjajqA6M0AlZoobR3oqekgCARmgyDrV3mxU7X1u2V/qBPpT//+O+jw+PHKpZ+B6PpU4LDgNxUu/CysQdnGP57Az384nvmZdCUtVFhvzQOtQEepy+UHDS047/bhoTeOYGS2FU4bO2AwmkGi/FlpCUuV1GLVckhGjnP4sQSBYHSWFXUVroggKqfNrAqMctpC+5mtXV5Uba1XfRdagVjJqFlPQbDnyClsWnItfv/QD7BpybXYc+QUKLhHgsMZaJSa+sWZTvmalyoHP/TGEXzZ0oXDTR1yejoQXRub2z0QRVGlTR0efXqX57DghcrJchxZze4G2M1GFGZa8Q+TiuUg7ZrdDVg6cyxq956IqU9Go4DiHCs2L50ia862A19j/9/aeEOvMNLCI6F18w33CLACFKV4AaC3aVWs4EqtAEZl222lJRxPJbW+5jhHO9bVo7JixlTEir3or5r1uXYz/mFSMZZu/jRq8CiHw+l/lJoaLagRYHcGZmmjVrVhVlVLlt5ZTIKc9tntC8JiEiL0ymQUYBQInltUrkufsqxmnL7gjRm0nu6khSGh9+YbUXxFpHhiT4OcaqRsWhUNlkEixUgAkU/qeg0doO91GWIdS0+p2VjBov2xd8gb63A4qUN4/yJpS+PqkaEaguG6qdUZWBkj8ejNpVgUVoyvaluol4ay3wdL79rcPlWTQ+l9kq7l2s29AZ6MOAstuO7oIy0MiXhuvuHNpp6cX4bHbolvAbEWX47NpDlWPF6Gvra8TcSjkSq15/vLSOFwOPERrqnSdoLkrdXSunBtlLI2npxfllC1YYloutZX/eK6E5u0MCQSvfn2ZQFJn5We5M929jAjjoH4DZ2+WMiJeDTi8Zgkk0RTaDkcTv8STVPj6QwM9F7nADtYW6vasJJoupYM/Qpv/mUUCDw+rkMSaWFIJNs9pdfNr9cSjtfQ6YuBk4hRNRi15/uaQsvhcPqPRDWV9XAg9crId1iwbmGZXHhOWW04ke1kSde02g70Ja5MqlXR2uXlOoQ0MSSA5Lmn4nGT6bWEE7ko9Roz4cRzrFhPCv0ZudxfKbQcDic5xKupWgX/fv27L+X00bXvhQpIjStwwGaKrU1KDdPSNS1vhUln6jhLi1buDKWsVm2t5zqENDIkkkU8brJ4nuTjuSgHYs9PeQytJwWtYlyJGDjhnwXAjNaOlkLLGX7wzqXDh2gF/6Q6PYebOrB086fYt/q6qIHfWvrH+gzLW7FuYRm6egLIs9OoXY6BkI7nOywRXU0lLeI6xA2JuNFrHIgiRVCjrHZfn+QHImZBeQy9Twp9MXCiuQ+lVu3RUmg5HE5qo6Wd4Q8kRTk2BEWqWYY6Xv0TBILCLIucGtrh8cvbElJwaDTdspkNWHVjiepBat3CMoiXamNwHUqTglSA/oJPsdAqWsVKR3piT0NEJ7u6SnZZ7XgYiJiF8GNITwoGAlVXVCV9KZal5T58YHaogpwUI7Grvkn+medzczipRTSd1dLO/EyLSiPXLCjDE3saNHUjEf3z+IJYuvlT3LnxIKq21suFsnyBYEzdCog0omHgyp1H5QdFrkNp4pFIZvqi3mBFXyCIDxpa0NrpU7nE8pIQ+6An86IvWwx6jxE+b63uf3oMHJY45DssGF/owB9XzoLpUs39RNJxORxO/xNLZ7W00242qDRS6vXz2C2RXt42tw9BSrFpybV49qPjKm+l2WjQ1L1oehbLMPEH1A0Dy4udWD5rHIpHZGBH1XQUOGIHgw530sKQSOZWgN5gRWnhSiVjgd4CKVokK8sjGYZTPNkdsbr/6XH7hV/o5cVOrLqxBD++1MaXZ2lwOKlNLJ3V0s42tw81uxtiPhjFypzIsZk0dS+anrW5fUzdkrZXlNpUXuzEwz8qUVXx5LqUJlsbyd4KkIIVpf4TrAWUSN8JvVsDygty3+rr8NaKmaqFnIx+HLGOwZr3sx8dj9jK0ev2C/++Hpg9PsKdGO85cDicgUOPzrK0U49Wam19PreoXNamdo9fU/ei6Vmu3Yy6SnWfIeX2inJ+y2eNk42I8GOkM2nhkehrWelESCSlM1lZHskynPRmkkjHa25Xd/8ryrExC3BpuR9V5cn7sSU5h8NJPonqrB6t1NI0ALJGxdI9LT0TBII8u1lze0UQCMbnO7Cjajr8QZF5DI8/qBkcmg6khUeiL10p+xKkqcdzoURvIGcskjVOIsdTdv8zGw2amR2sTqzK78tqYp+D3txvDoczsOjVWZamxtJKPZrWF90TBEHuECoFYyrjLo63duGOugOqTqfKY5xo6WJ2lE4X0kKVlRbvx7+4HjuqpiPLakSb2xf1D6+3/XiySFYbbtY4dZUuuUVv+Pz7mtESz7z1brsYBYJ1C9XbJOsWlsGYphY/h5PqhG8fvHP/TOQ6QpUlJV1JVFP1aIwe3dPSumjjszqdhm+DPPvR8bTe4kjJrQ1CSDGALQBGAhABbKSU/rovY0p7YfEEIQ50j4lklfLW6mLKKi2djMBMrXkDQGunV/Wa3m0Xjy+Ite81qtyNa99rxHOLygF7XF8Hh8MZIJQ9hli6UphlSUhT9WhjLN3bcs8UeAOiptZpaZgyG03Z6XR8gQPHW7rkbRAAabv1mqoeiQCAhyilVwOYBuAfCSGlfR1UyzBQWsxK+qteQzQPQLzbIVpI45iNBix68WO5cly+w4IzF3rQ3BE6docnuodAr7cifN4AmE8eJqOgy/1oNhrQ2uVF1dZ62d3Y2uVN+8IvHM5QQEtru31BVM8tRXmxU36vHk3Vm84eTfdECri9Afn40pzOub1o7fTi9IXQXEdl21QadqLFrdIsqdPp123d8jYIkN6FqVLSI0EpPQ3g9KX/dxJCPgdwGYCGvoyrZRg0t3vw0BtHUFfpQklBJoyX9uH7I0hzoFtyK8+ZlbpUV+FilqLua/tdLSF55/6ZEWlYdZUuEFB82+EBpVRu5tOXdukcDmfw0NLaU+0e1OwOFeqTnuRjaWoiOsTSvSWbPpE/rzx+tzeIipc+jhhb0rB8hwVrFpSpUz4rJ8NuNWDTkmuRYTag2xfEFbkZaatPqeqRkCGEjAFQDuDjvo6lFYzT4fGjuT1U9/3bCx5d+2aJkozUTL2IIgUhBDuXT0ddpQsPzbkyInWpalu9XD1SQrqwY801mrdCS0g8vqBqH/W1e6fincPNaDwbCmaSvBfHW7vwd3l27Kiajj+tnIUdVdMxPt+RtlHRHM5QIpbWrt51FMtnjUtqWrzW8Vm6pzz+yXNu1e/Wf9iIMxd70O0LeS8AyNsZ25dNw/Zl0+CwGnD2Qg+q3z6GOzceRPXbx9DjD6K1s6dPlZOHKiltSBBCHAB2AfgnSunFsN8tI4QcIoQcam1t1TUeyzBYs6AMtXtPAAgtopZOr7xA46mloJeBasktWfF31B3AwtoDqNndgFFOG/PYY/PsEcZSjs0UtVJlrKCpaBHU4e7HSWNymbnZpy/24I66A/j+ur24o+4Ajrd2DZkLNJH1yeEMFP29PvVo7fiCUEplLE1NRDOl488pLdDUvVy7GXUVLjz70XH59fJiJ+6eMRZ31B3AD9btRc3uBjz8oxIAkLPRCCH4a4sbD+44otKsZVvr8efmC/0elJ+KpKwhQQgxIWREvEopfTP895TSjZTSyZTSyfn5+brGVBoGf1p1HWrmTVAFyhTl2FTdJ6XPxBOzIIoU592hp/RvzrvR0tmjq958svfWWFb8N23dzGNnWAwqY2l8vgPHW7si9gaVc431lKDHmyMJhNNmYl7oLZ3eIVv4JZH1yeEMFP29PvVo7fGWLlCqXXtBFClaOnsQECn+8PAP8Nv7psqxFUU5NhBCosZslRRm4vFbJ2jq3minDSOdFrR2eeXXWQWnwr0nBgJkmA1MzZK0bChpVTJIyRgJQggB8BKAzymlv0rm2Mqo4s4ev7yIJIv5lf0nMenyMuZnYwX8iCLFV21unL3YE9FyW7K6c2wm1Fa4sHxbvfz72goXchTtsZMBy4p/9qPjqKtwoUpx7BcWT0aeXW0gtXZ6tfcGLxkDpy94YhZ/iRVlLRlVHR4/MxYl/ELkBal6ibe9dqqMnUqky3kOFvFqrVJfTUYBXT0BLH65N65h3cIyPHZrKTb84a+4e8ZYPP7OMTx4Q4mmR0MQCCilcsXd8NiwkVlWdHh8WLewTNbrXLuZqWtXj8yUO4W2uX3o9gWZmiV1J043rUpJQwLATACVAP6bEPLnS6/9M6X0P+IdSOvmLwgEJQWZeO3eqfJ2xiv7T+LBG0qi9pOIFvDT5vbh67ZuVL99TDO9qd3jx7MffalKa3z2oy/x5PyypKaUsgJFW7u8GOm0YEfVdASCIowGgdlwRk+lSj2BqLEqY0pei/UfNkZe6JUu/Pp3X6ren85R0RzOUEUQCK7Md+D1ZdPgC4gIihQ7D32j0lqtXhpSIHhze6gkds28CVj5o6uwaudRHG7qQMPpzqipo1L2l1LHun1B5DnMl+oIiao08+xLOheuazazUT5Grt2MK3IzVAZIUY4Nz9w+EU+9+4X8mXTSqpQ0JCil/wWgz1F1sW7+RqOAopwM2MxGjMq2YtLlZbKhEW6AUNCY+c++QFDT5SVZp1JXUCktSSK80x3rXGKlPynfYzMbmFkPF7rVVj4r+llpJEiVKotyQg3HpPfF09RLC8lr8eT8MoiiiB1V01VZGw/eENqbXOAqRq7djIJMS9I9NxwOp38RRYq/nnNHZGpJwdOiSHHmYg+zl0b13FK56WFzuwcZZgPOu33yFkn4k3+4TiqzvyQdW3/HRCzf9hlau7yoq3QhP9MsH6O82BlhIITrmiAQjMm1w5lhwvZl0xCkFAQENbv/gsNNHZhTWoBHby6FLxBEa6c3LboUp6QhkSz0FJRiPTWzDJBtP50aM+DHbDRourwk6zSRlFI93hDWe7bcMwVvrpgBf0CE2WiAQQBufW5f1O8D0GckhG9dEEJgIJCb3MRTzErraWJ8vgM//+GVqNpaH9Xw4XA4qQtLh6u21stbBY1nO+H2sgO7nYoHh6IcG7p9QfiCouo1STu1NNBiFFAzb4KcpimV2Zfm8dq9U9FwuhPN7R60dnlRmGVV6WaOzcR8iBtht8jF8USR4sn5ZaiZJ+Kc24dFL0amkw5nzRrWhkSiGRKshX/ynDumAaDl8lLehBN5ktdjELHes/jlT/DWipm4LCdWixu1AAAgAElEQVQDAHCqvVvX96G3wmYi1ULjod3jl40IrfPmcDipTTQdlnSrem4pU1+7fUH5/+sWhrZ/177Xu32g1E6WBoZvNUufkzwdze0eGASiqXV6a1hID0Stnd601KxhbUhoPf1L0b7RipnkOyyqOIZ3//s06ipd6qfjylguL8BqElTBjP3VFVTPe6J9H6fauyNiSPQs/P4sIz5QqbIcDqf/iK7DoW6aHzWcxfOLJuEfX/tMFRCZn2nBn1bNgoEQ2MwGZFlMeHJ+GR67JVI7WXoRLbtCmoc0juR1UHpV49W3dNWsYW1IsJ7+1ywoixntazMbsOrGEpVX4dc//h4cVmOYiyzSIAl3ebHQe5OW0LMdouc9rO+jtsKFx985xuzDoYf+LCMuFdNqc/tQu/eErip4HA4ntYimwz//4ZWo+h9j8P2SQjz/h+OonluKXLsZI+xm1O49gf1/a0NthQtXFfZWHI4WWBmugdGyK5Q1c7S8DvHqmzSHfIcFy2eNk4M7bebhrVnD2pCQnv53VE3Htx0etLl9ci5ztGjfgEixcudRlVfioicAUQSWbv5Ufl9Rjg07qqZjtNMWMUYyybWbseWeKfi6rVuzHGuicQ2SEQEk5k0YqDLiUrrYz394pdzNLx2CmDicoU4sHX592TT8eOPBUIxCpw/LZ40DANz3/e/geEsXlm+r16WzSg3Md1jwwOzxuLLQEeFJrqt0Ic9uVqVzankd4tU3SavDSwDUVbrgD4gQBGFY6tawNiSA3lzihbUHVK9Hsyr9ARH5DktEX4rf3DUJ5cVOVcRwQBH4w0Jvs5lYeAOivNcnGQnh56k3rkEyEk61d0dkj8TrTUhG9kY4rAt79a6jeH3ZNPy///6XhL0nHA5ncIimw0GRorndw+wFJPXECNdZLV0tKczEO/fPxOmOHrlezpzSArx271QYLqWth+tiNK/DqGxbXPomCAQOqxGLXw5rRbC1HtVzS1Gzu2FY6tawNySA+J+azUYDHpg9PqLC2c9e/UyVjlSUY4PRoF0clPVkXVfhwpUFDphM+p/YWTfW9R824vFbJ8jpktLFofQkSL0wRFFEkAIipRAuZVcIggC7pbfpTIfHj9q9J+LusJms1udKtC7sMxd6+uQ94XA4g4eWDpsMoY7AWlUla+ZNUOlsLF0NipCNCACyZjx+6wQ5BiLHZkK7xw9RDBko/7lqFgACX1DE6Q4Pthz4Si7nH6+++QOiZlzGcNWttDAk4n1qzrWbMTbPzlwM0mek+IICh/ZiYKY9bavHq/dOxWXZNnnPLxbhN1ZlPfhY6aDrP2zE3TPGRlj5f2o8i1u+V6TycqxbWIbCLGvc3oR4Yz6UsJ4stASHV7rkcIYuWjpc4LDghcWTNVNAx+TZVTobS1f16GVthQv//udmfL+kEK/sPxmhkcpqw/HomxTblW5VL9PCkIjXqhQEggwL+2Y2KtuKP62cJVeFZBkD0s2x28e+MFo7vTAZBN2xFeE3VpblrpUOWj23lGnlb1pyLZZu/lT1+sqdR/HmihlJc7npKSnOCnIan++I2Osck5eBsxe9qq0lHnjJ4QwdoulwSWEmzlzsYWqu3WxQ6ayWx1LSVT16uXxbvayBLI1cvq0eb66YgYJMa8zzUupcUKR49eBXqkq9c0oL8MhNV+OCx4+6Shd21TcNO91KC0MCiP+pOc9uYVrPUoloLZQ3R63c6Da3DwVxzCXckteqB89KB9VqiGUQCPN1fyB6zIde9JYU1wpyYu11Sl6Tte81orXL2+dYDA6HM7Bo6bAgEIzMsjI1N8+h3q4NijSqroaPo6WXkgZqaWS3NwjRrl0mQJoPKzD87cOnZP0HhaqacH/0VhpsUrb752CTaAtx5c2xdu8J1Fa4QosJvc1qdtU3RY2tYJHrMOO1+6Zi3+rrcJnTJo8pUZRjkyu2AZENscLfK12M4a8ny1KO1R0UiB7kJAgkYq9T8po8t6g8KS3dORxO6qBHc9vcPjyxpyGqroaPo6WXkgZqaeTJc+6YHTy1AsNnlxbKmSLhGrZ8Wz3aL21zDBe4IRGFeFuIA+qb4+GmDvzh87N49d6p2Ll8OqrnluKV/SfxwOwro8ZWKJEs3ts27Mf31+7FnRsPwu0LYP0dE1UX0rqFZTAq5id5MXbVN2HNgrKIi27noW+w4a5JqtfrKlxJe8KPp0CWEqUxoxyjvNiJukoXnrl9onx+3IjgcIYXsTRX6lUUS1elcUZl2+DxB7FuoVoDf3PXJOw89I1sgIRr5DO3T8SzHx2PGcugpXNSwatwb0h5sRPVc0vR7QtcCoRnt0EfaqTN1sZAEb4/98zvjqPL68fiGWNRkGnB47dO0IytYMGyeJds+hTrFpapKm+ufa8Rzy0qlwthaTXEIoSAEMAXyMe2A1+rOuKNdOozlhL5HgB9BbKU2xXK4i7haWHDMYWKw+FER9KEZ353HE3tHtz3/e9ghN2Mx275LgozrRG62ub2YfHLn6hqAnX7ggiIIiaNycV38jLw+K0TQAjFb++bBn9QxLcdHjz17hdo7fLGrIKspXOSl6Mg0yL/npXeOlx0jHskkox0c1Rat/8wqRijsm24PNeO0U792RqAtsVrMgio2lqPOzceRNXWembapmSVF2bbMNppw2U5GRjttKEw04qR2Vbs/1sbqrbW46E3jsBqEnChO5A0C5n1PUQrkMVyZUpjsFJxw7dJOBzO8EepKzvqm7F086fwBkSM0siCk/RT6mJ858aDWLr5U/gCFDW7G2A0GC5pog1d3lBn5IqXPkFrl1euvtl4tlNTF1k6V1fpwveKsvHWipkYfakOhVZ663DRMe6RSDLJrqugZfEqLd14i0AJAkFhlkUu9y15NFq7vKrMj0SLaUmfG5FhUrUGj1UgizXPksJM2C3RW7NzOJz0IF591dLPbl9QpZl6qiAr+3GEZ5y8uWIGevwiDCTUYsFp652TNF+tLL7hoGPckOgH+lJXIZxw97/U614QCHZUTZeLS8VrrHh8QVW5bwlpUevteheO9ueiZ7toIQgENpMx6WW4OYPDmEf2DPYUOEOcePSVtX26eem1sJuNEClVNeiKVn1TFEVNPQSAti6fplb2dgbFsNWxhLc2CCFZhJBxjNfL+jYleZwbCSGNhJC/EkIeScaYQxGlBX7ol7Px89lXYtGLH+Pv1/wBd9QdwPluf0Iej1iBjnqyLlgk+rlo6Nkm4XA4nHDCt0/fuX8m/AGK2+sOYOaaP2D+hn2qrQstXQxSaOqaXs0bzjqWkCFBCLkDwBcAdhFC/kIIuVbx6819nRQhxADgeQA3ASgF8BNCSGlfxx0opNLUp9q7kxKZKwgEuXYzun3BiFSiRG/SsRZ1tKyLaOfXH91AE03F5XA46UE0TVJmggREivu2Rt70z7m9ALR1kVKqqWu+QBDP3D4RdZUulBc7Vb9TMpx1LNGtjX8G4KKUniaETAGwlRDyz5TSNwEk41uZAuCvlNK/AQAh5HUA8wA0JGHsfiXRLYFYtLl9aOn0Ju0mHWuvUbMuvlGIen790Q1Umu9wqk3P6R/41kn6EY/m9vjZDzo9/lAhPi1dbHP7mLoWFCkWvfixqhjV0+83avYsGq46lujWhoFSehoAKKWfALgOwC8JIQ8ASEbY/2UAmhQ/N196Leno8R7E42HoD9c+ALnZTDKLSIXnbAOQz9MggGmZGwUin5+UE+32BnDmYg9EkQ5r9x2HwxlYwrU3EBAjtDgezTVc6oOhpCjHBoPC3mDVstDKznhiTwPyHRa5xo0vIOKf/+fVaad5iXokOgkh4yilJwDgkmdiFoB/A/DdJMyL9eiuunsTQpYBWAYAl19+eUIH0WPJxuth6A/XPhDyEEiFU5R5yHWVySkixTrPLfdMwZsrZsAfEGXL/PQFT8yc6GR3Aw2fZzLasvc3yVifHE5/MRTWJ0uTaitcePajL/FBQ4usOVlWo27NtZkNWLewDCt39urWuoVlsJljP4xZjIKc6dbtC8JiFOC0mZkNv8bnO3Tr0lDRtGgQSuN3IBBCJgLoBmCilDYoXjcB+DGldGufJkXIdACPU0p/dOnnXwAApfT/sN4/efJkeujQobiP09rpxfwN+yLcVcoUSD3viXdMLaItKGU3zwWuYuTazSjItGB0HF1Eo6F33tL7queWomZ3Q0LnmSiBgIjGlk659KzObaNBvyL1rE/ukh9+fPXUzXreNiTW52CgpUnVc0tRtbVe/nlH1XS5s6fyfeFaJIoU59zeUEGqIIXHF8C5Lh+uyM3AmFx71Ju31lw2L52CJZs+SVgH+2srPInomkRCdyBK6RFK6XEAOwghq0kIG4BfAViRyJhhfApgPCFkLCHEDODHAN5Jwrgq9HgP4vUwJOralxbU/A375Gjiz89cRCCg3rt7cn4ZJozOwhW5dhTlZCTFiAD0n6d0fnoahyUTUaT49oJHNiKk4w2Xgi4cDkdNrPLT0s8Gwt6GVWqustXArHV7sWTTJyCEoKw4K6YREW0uJgO7+aFeHTzn9jK3ZaTgz6FCX+tITAWwBsB+AJkAXgUws6+TopQGCCH3A3gfgAHAy5TSv/R13HD0BAbGGzwYHqxDCIGBQJWvzIK1z1e1tR6v3TsVRTkZqnzkeNDrNosWXNna6VWdS67DDKfNlHBQpShSdHh88PiCCFIKkyBAiFEPI9nBphwOJ7WJVn5a+bMgCBHbqTk2k0r3KGiEvi7fFtLXsJAJGaV2kkuxFRF6ZxCwacm1cmG/2r0nNAMtRZGi3eOFxyciKFKYDQIMBkQN/hwq9NWQ8APwALABsAI4SSlNyjdAKf0PAP+RjLG0iNXrQe97whEEghybCd9eCKClswdtbh921TfhwRtK4o6taOn0wmY2JrRdEI/bTOs8u3oCqha4axaU4ZX9J/HITVfH/b1Ic/qqzY2zF3tU+5TP3D4RL/3X3zS/I2Wwab7DguWzxsl18/Xsb3I4nKEFS5OkGAlA7XlQPmRJGvN1W7cczzA23x6XvoZr55zSAtRWuLB8m3pb1eMLovrtY6p4i/xMS0SbcC3dq61woep/jEHdf34FINTU64HZ40FpKMh0qMRLJBQjIX+YkCMA3gZQAyAXQB0AP6V0YXKmp4++7PHpeWKPNxhGFCk+P3NRtZcv3YCfnF/GNApaOntw24b9zP3ACaOzcFlORtznprWvt33ZNOZ5hJ+nQQBufY69R1mzuwHv3D8TQRFxBQm1dnpx7NQF+eJjjcvaX2zt9OKXbx3Fiuv+Dh5fUHUxxthTHPSrkMdIpCc8RqLvhGtSjs2Edo8/quacd3vReKZTpRFb7pkiPxBJRNNXlnbOKS3AY7d8F0EKWE0CjAJh6uPTt0/EuHxHRGyZlu799r5p+MkLB5HvsGDVjSXxaNtA0H8xEgp+Sin9F0qpn1J6hlI6DyHDYkig10CIt514m9sXsZe/etdRLHAVM93wokjR1ROIaOsttbhNNL1Ty8vR3O6JqOimPM9R2aE5dPawa8M7bSY0t3vg8QUTarOeYWb3zpDGZX1HuXYzHryhBD1+Ub7QpM/xOAkOZ3gSrklnO3sAAKOybZqao3zQAEIa8dS7n6OuwsXU16BII1L6Wdr5QUMLmts9WPTCQbR1hbZmWTpGgAgNi6Z7IqWomTcBv7pj4pDVtj5tbVBKI8zYvmZs9Bcsy/ab9m6V+0tP9G6scXPtZs0beK7dzDQKpFa3M76Tiy33TMH5S2VXX9l/Eg/eUBJ3eqc0pyCl2LTkWjz70XEcbuoA0LvHKC3S8GY0NrMBZy+GAoCq55Zq7lEmWr/CbAx91/GOK8WeNHd08zgJDidNkDMtvEGcudgDSilMBgEeX4CZsSaKFAExsgrlBw0t+Nd538Wr905Fa6dX1te7Z4zFE3saIjzFUnxG+DaqSCnyHRacudCDLKtDsyFYuIZF0z2TQcCEy7KHdFOvtGgjzsqIaGzpRGePH9VvH8OdGw+i+u1jOHuxBx0etvXHKkrFHPdsJ0xGgVn0pCDTgly7mTGWiOZ2D3bUN+OhHUfQ5vbBaTPhsVu+G7dbSzmn76/di+q3j2HVjSUoL3bKVnjt3hMA1M1opHM40nRB3hes3XsCaxaUyecyp7QAW+6ZgpFZ1ktBSqZoU2GSazfjitwMrFtYpno6eOb2idhV3xQ1zkLZwCv8ux0OjW84HE4vqkyLp/fi4TeOAACe3PM5Fr34MRpbOmUdlvS0ub0b7RqF+wyCALvZgGybCeMLHFj5o6vw9uFT+KChhZmdtuWeKVh1YwlqdjfI9wiTQcBjt5ai+u1j+OZ8N9bfMVGlY88vKsffFTjgCwRVxQu1dK+2woUChwX5mRZkmIeutqVF90+tjIiaeRNUr23adxKP3fJdnPJ1q7Y6tIIWC7MsqnElS9VhMeC1e6fiiT0NcuGUukoXRl9yz4WPVVfpwpzSAnzQ0ILDTR2o2lov5yLHuzfGSidaufMoXl82DcfPdsmtcQF2Mxql++1wUweefr8R1XNLUXZZFs67/arAy0T27wSBYEyuHc4ME7YvmyZnbZgMBI/fOgE0rCNfOIkEv3I4nKEHS7dX7jyK6rmlqN17Ai0XvciydoNSqLT2uUXlqKt0oeWiV+VtHmEzR9ShWbOgDB0eX8TNWhAIHFYjFr+s3mr4p+1/lu8bAiH43//xBZ667RqMzLbCahRwvtuPn7xwkKmRY3LtyM4w4vVl0yCKIc9Knt0sx3yYjIIqlmMoaVtaGBJaWw0Zimj/8mIn7p4xFndujFwEWiVYty+bJr/GqvRYV+lCzbwJqrTG1s7IG72U5tlwulP3AtKK79CqJU8pMDLbitauUH6ydAypGU15sRPLZ41DrsOs2g453NSBmt0N2FE1ndkwbEfVdIzMssZtTIywWwB777nozS6J1SOEw+EMD7R0e3S2NUJr1ywoQ2unD4ebOrDhD3/F/dePV2VTvLB4Mi56/czYtdfunRpRc6LNHYqBkIwW6eFLed/o8PiRn2mG2xdEW5cPI+xmrHj1s6gamWu3xtS9d+6fCY9vaGlbWhgSWvnI3b5ed9byWePkhQn0LgLphsVa0EHa21+e9fmqrfURGQhaYxkEovvmGO3Ga9DIdxYINJvRzCktiCjzum5hGda+F2o+E6373bcdHlzw+DU9E3oCWrUMNa3qcMO18Q2Hw+lFS7etJgN+FnbDXr3rqFzxcoGrmHlDVz74SUjaG60lgtSI63BTh+q+8VHDWdx//Xj5WDuXT4+pkQBUtSm0dC+RLL3BJC1iJLSqTV6RmyG/Fq1SY3iP+vJiJzYtuRYAxWv3TsWc0gI544D1eSXSWOXFTtRVurB92TRsWnItbGaD7gyIaE1qpFryynOVaslrNaN59ObSCCNo5c6jeG5RudzqNvw7kMaW5sKKLNaKIdETIa387pLdlp3D4aQ+LN1et7AMXV7tbLLyYieuLHQw23pLD34AZP3duXw6CCGyprC0dfWuo1g+a5wcyzXCHirGN7u0UGWwaDVVlMY81+VV6eG3HR5N3RtqmpcWHgktdzgAVQVKrUqNyn15Vq5vXaULeXazZmXIls4e9PiDMBACq1nAb++bKt+slR4Fp02fGyvajXdUtg2FWVZVc5nCLCucNu0ARoPALvMKQH7yz7WbUVfpithffPr9Rs3IYr2ehmjVQ4dALXoOh9MPhOu2yRiq3eDRyH4QKcWqG0tQ+dInERrV2uWF1Sjgt/dNgz8owmQQcOZCD57c87nsdS0pzNTU1vEFDlTPLcVL//U3PHLT1XhzxYyI9E8pOD18y0XSyB5/UKWHWq3JTUZhyGleWngktNzryif0kVlWzXrtygX93KLyiFzfqq31ECmj3nvlZFz0+HHbhv34/tq9uHPjQXx5pgsUACFEtprzHZa48oW1vANmo0EO6plwWTaKcmyYcFl2zJTWaONJCAJBnj0UP7Fz+XRUzy1VuftYkcV6+3fk2Eyoq3Qxv/v+asvO4XBSH6VG59ktCIqAUSARelFX6cIVIzIitHn1rqN4YPZ41FW60NUTwE9eOIjrn/kjanb/BXmZZvzqzol46rZrsP7DRrS5fZpaaDUZMGF0Fp6cX4YxuXYUZFojsiwON3Xglf0nsX3ZNKZGCmEPbOFZcUU5NtRVuAAKrP+wcUhp3rD3SOh9oo0VxCct6FCKUeTN8VSHByPsZrnttskoIBgUcXvdQdWC2LTvJO6/fjwefuNIhNWqN184VuZCvDEEejMhBEHAuve/wN0zxsqdP6WLmBUYqqdPiShSHG/twq9/9yWq55aqupoKAum3tuwcDmfowCpZ/eq9U2EUiKzVpy+wtwqKR4TiGhZf6tIpBdaHey5EUUR+ppWphayAcpZuPnhDCewWA/xBUaWR6xaWwWwQVHooGR5vVE2HNyDi5Dk3Hv23Y2jt8qqCR6XzSGXNG/aGhFZ3tTdXzEBBplX1Xj03YK2bY5vbh3/a/mc5UKa104tTHT0RC5sVCLR611E8fftEEEJwqr07ZrBlsjMX9I4nVZdc/2Ej86Yfjh4DRelx+KChRf4+pe2PeJumcTic4Ue4Z/KDhhY0nO6UMy6ESwYFSytOtLoxTtFrgxUYv3rXUWxfNg2nL3hQmGWRHwhjVTzW2jJnbS+PyGAbHgYDQUXdx6p5K4NHpfNIZc0b9oaEVjqkVne1WFkGrJuj5FHId1jgCwRxqr1bPnb4wtYK6izMsuKOugO698SSnbmgZzxlK3M9BoweAyWWx4HXjeBwOHqaGkpbpKw4rl9dKhwlBWWyxjp9oQcLaw/EFZOgpZtjcu3ItJoidI+lh1qeFEnjhoLmDXtDQpkOKddKsJthICGjIbxpVaxtEGkx7Kiajm87QgE0AgH+5ZZSZNtM+Nd//4tcGOX5ReV45vaJeEixjZGfaWFazV+dc8cMSkwF4jVgYr0/lseB143gcDjhOiF1yXRmhG7WgYAYsUU6wm6Wq/gSQrD1p1Pw1bluiJRqepWB5Oivlu6xXtfSwNFOG/atvm5IaN6wNySkdMhN+05G1EoINxL0ZhkIAsHILCt6/MGItrDKva1/fO0w1t/xPXlh52dakGU1YP0dE/Hgjl7j4jd3TcK/vP0X1bwT3ROLt1PpYKPH48DrRnA46Y0ya0wrc+7Xv/sSHzS0qLZI19/xPfiCIipe+rhXbytceOnuyfjpK5FeZYlo+ptsjdXSwHgL/Q0mw96QcNrMKMyyYtWNV2PJpk+iGgnxBPZplVBV7m01t3tQmG3FaKdVtX+W6wio9s/yHGa54qREInti/Zkq2V8GCvc4cDicWAgCQUlBJl67dyqClMqBkkBv5lz13FLZiJBeH+W04scb1QHvP9tWj6duuwbVc0sxvsCBoEix7v0v5MBGQFt/E9HYWNo5HDRw2Kd/SumQZiO7VoLSSNCTBqnEHxCZYzovNbMqyrHBZjKoCkCx0jPzHdqpp/HQX6mSegtLJUq8bdo5HE76YTQKKMrJ0Kx7E66XUm0J1ntNBgE1uxtgNRmQZTXi5z+8Upf+xquxerVzqGtgynkkCCHrANwCwAfgBICllNKO6J+KjrJrZLTo/3gD+7T2tqR22FqfZbnqk2GR9leqpNbFs6NqOiilQ9KC5nA4Q49oWl6giD+T9Ndq0m6PoNw+yM+06tJfLY31+IMRMXeAtnZuXzZtWOlmKnokPgQwgVJaBuBLAL9IxqBaRY8MAuQypEDvDX3f6uvw1oqZGJ/vQJvbxyxVyirhWlfpwveKsuXS0noXSTwWqVb5VC2PSvBSq91E0bp4vu3w9IuHQslQKxXL4XD6Fy3dtVsM2L5sGv606jq8uWIGSgozMcJmjixeVeHCxOLsiCD6aPor6ZA0hpKiHBtOtHTFVf6/ud0Tl26mug6mnEeCUvqB4seDABb2dUxW0aPCLCv8QRGP7DqKBa5i5NrN8PgCGJ1tk70FsfbDBmNvK9qctMpYP7GnAU/OL4srYFG5r6dVPjyZUc56z7Wu0oWSgkwYjaloA3M4nP5G0t03V8xAtzeIMxd74A+KmPf8fuQ7LHhg9niMzbPjHPWiyxtgFruLRz+UOpTvsGDdwrKIAHupDLfe8v8dHr9u3RwKOphyhkQY9wDY3tdBWEWPNi25Fr/95OuITI66SheuHpkFQSC6sjgGOqMg1pzy7GZUzy2F02ZCh8ePtw+fwuLpY9DtC+BUexA2syFmTw9WFbkNd02SC2lFi3JOZlAm61yllutFORnDwiXI4XDiRxAICAgqXvoY1XNL8fAbDch3WCLai69bGMqiUxZ2iveBR6lDze0erH2vETXzJqB4RKjYlaSDT912TYTO5thMqK1wYfm2yNoWQEjTRFFEa6dXUzOHgg4OiiFBCPkdgJGMX/2SUvr2pff8EkAAwKsaYywDsAwALr/88qjHY7mXMswGLHAVR239nYrlmWPNSRAEuTRrebETj9x0laqOxbqFZSjMsqr6b4Tf/A0CIqrIASHj67zbhxF2MzPK2WY2JDVrRE8RmlQlnvXJ4Qw0w2F9SvogFZiqnsvuYqysEJmIfofr0OGmDizd/Cl2Lp+Oqq31mjpblGODURAQFCk2LbkWFpOAEy1uuf8GAMwpLcA5t0/lRQ7XzKGgg4PiF6GU/pBSOoHxTzIi7gYwF8BdlFLmZhCldCOldDKldHJ+fn7U47FiB7p9waitw7U+N9ilSmPNSbl/uHzWOHlxA70XVkunF2cu9uBUezfOu70RUcWnO3qQ71Avzg8aWnDe7cOdGw9i1c6jWDpzbES8SUCkSc0aida6PJXrzgPxrU8OZ6AZDutT0gcpuF2rYqWURQckpt9aOiQFd7J0dtO+k2jv9uP2ugOY9/w+LN38KU539CDHbpJT/YtybPjlzb2lAqTPhmvmUNDB1NhgUUAIuRHAagC3Ukq7kzEmKzjnitwMeSEo0bopS78b7FKlseakjNu4amQm88LKz7SgvduHmWv+gCNNF1IZh64AACAASURBVCLdZtvqsfqmq1BX6cL2ZdNQV+nCnNICdPtCi7a1y4vCLCveXDFDDkotKczUTIdNdLFLMR/Kc12zoAy76ptSuu48h8PpfyQt3FXfhDULytB9qb24EilDQ/p/Ivqtpbmjs22aOsvqqfTQG0eQaTVh89Ip+M9V12FH1TTNVFalZg4FHUzFGInnAFgAfEgIAYCDlNLlfRlQKyhSFGlEcKLWTTnWnr8oUpxze9HjD8JAiK5YhGSeS3iBk/xMC1o7wQz0+aatW16UGWYDcyEXZllVHUp/c9ckiJRi5/LpGO20MauuJbvBlrIITUunF21uH17ZfxIP3lCS0nXnORxO/6Ps/SOKIgghqKtwoUoRj1BX4UJ+pgUf/+J6CIIQoZWBgIiWLi/8QREmg4AChyUigDGa5mrprJa3u93tk/t5rL9jIopyMmJq5lDQwZQzJCilf9cf42oFRdpMBlWVSQtjEcXag2JF1bJiEZJ9LlJsw+kLHs0GY+EXlhTo8y+3lAKA7BYMX8hfnXMj32GRAzd7/CJyHWa0dnqhsdvULw22pCI0NrMRo7KtmHR52bDJveZwOH0jXJ/zHBY5m0PZlpsVqxUIiPjibKcqELK2woWrCiOzIbTuA6JIYRAQEVA5wm5m6mqPP4i6ShdGZ1thMxvhF0W8du9UPLGnQe7RxNLMVNdBonVTGEpMnjyZHjp0KO7PtXZ6MX/Dvog/drxRvVrj1MybEKpc2Q/BMHpLtZ53e3Gk6QIyzAZ0ePyo3XsCrV1eOQCpvNgZUbf+N3dNwtYDX2Ne+WVYvesos7a9VhBltKyNQeoDMuhXmp71OeaRPQM0G85A8dVTN+t525BYn0MJvbr+bYdH7risfN+OqpDXNRZKDV63sAw9flHW2Y8azmL+pMtUmvnM7RNhMQnY8Ie/MrMF8+xmldckRfom6TpgynkkBpJkZWVojZNhNvS58YvW+/Q2GHPazBiZbVUZHBvumoTnfn8cgDrewR8ItVb/5nw3brpmlLzQq+eWyhdEtGMB0S33RDI6UuRi4nA4QwS9uu4PsmO6AkFR/pmlP0AoJdPjD+DMhVBg+tr3GvHwj0pUmRuLpl2Op2+fGCpwRYBznT7c/9phZnaJMltQOm40vUw1XUxrQyJZe/pa43T7gn1q/BLtfVoXCysnWSre4vEF8bdWN7Yd+BoLXMX46d9/B92+IJwZJoywhxZwa6cXm/adxOqbrpbH14qGjsdI0mv4JPI9cTgcjoReXTcZBOb7jIbQtoaW/liMAha//EnEdvHbh0/hlXumwCgQGAQCtzeAsxe9WPPuFwCAp++YqEpXVaLUU1GkOHOxR1Mvc+3mlNPFlMvaGEiSlZXBGmf9HRNx1UgHfIFgRElTvY1for2PlRIk5SSHN4gBgIJMK4wCwdLNn2JHfTOqttbjzo0HsXTzp/D41BHCD95QgtZOrzy+FEehRMvg0mpSk4j3R+v8z7m9KV0ulsPhDB56db3AYUFthToborbCBYuJ4FR7t+bN/Ou2btVrq3cdxUNzrsS88stw98uf4Afr9uLHGw/iXJcPtXtP4HBTBw43deB0h0eVrqqkKMcGk1GQ9fPbDo+mXmrpopTSPxiamNYeiWSVuFaWbO3xizASoNsfxO11B5kWo96barT3jcq2RQQ2PnpzKRa9+LHmU78eS106lw6PTw7UrN17IqIsbLzd8XZUTY/b+6N1/t3eICpe+jhlrHEOh5M66NV1o1HAVYWZ2FE1HYFLWRsefxDzntuP5nYPdi6frrllHf7aKKcNd7/8SYSBoSyGteXAV6ircOHXH32JNQvKIipwdvUEYBQI7ttyCNVzSzX1MlrvIykjZKA1Ma0NCSB5Ja4FgaAg0wogtD2wZOPBhG7oym0BipCXQaosqXwf62KJZaDozaoQBIIRdgucNjN2VE3Htx0e+IMinrrtGpgMArp9QRRmsRuLac3BQBB3RofW93TynDuuLRIOh5Ne6NV1o1GQAytbO71YrAi+bHP7mPqT67Bg+7JpquB1s0Fg6p6kb0U5Njx4QwnG5zvw5PwyBEQRm5dOQWePHy2dXqx9L9SrY/uyaWhu96B274kIY0PSS6159Xfvo2ikvSHRHyR6Q8+xmSL2vjbcNQkA5NSgukqXqs6FcqFI2xFaT/3xemAEgYBSioW1ByJ+t2/1dYA98jOsm/+c0gIEKTAiwxRX63HW91RX4cKj/3ZM87vlcDicRAjXbdbNvLbChbXvfS7r8bqFZbCZDThzoYepvaOdNuxbfV1E7YlT7d344a/+GDGHIA197nBTB55+vxHVc0sxMsuKPIcZxktB9jk2U4QuavU+GijSxpAYyChXrSdpQkJ7b2ajAePzHRE3dNa2wIpXP8OmJdfip3//HXR4/JdShNjzjuVxSOQ7iDcgNXwOc0oL8MDsK+U0q163my3msVmGj0GAXGJWz3w4HE56Eq/ehWvd4aYOvLL/JLYvm4agSGEQCP713/8ie4ib20MtB5667Ro888GXETV7Xlg8WS7cFz4Xm5mtq1aTIOvn4aYO7KpvwgOzr8SdG9Xb5Mr7ByEEj79zLKL30UBqYloYEgMd/c+6oddWuPD4O8dURUfCj6/lyZB6XEi50P8/e+ceH0WV7ftfVb/TnaRDSAKYKIgYjRhOaN6c66CMDB5RrvJwBggCIonRw1yOAp4zMnpu5FwQOFw9PgioIK8R5DE6OCrqyHiPgkhAGCeKDAOa8EoT8uh0Ov2qun90dqUeu7o7ISGv/f18+Gg61VW7qnb2XnvttX5Lj2geh9Y+g5aKTKnbwHGcIle7pW43tddFEMQ2F71iMBjdi9aMd7SxbuH4myXjYVfhaMU2MxAZz/okW5GWaEZfp7VFY+/meSMU2R8bZg9Db7sFve2WFo2fgiBi0d3ZKLvg6bAxsUcYEq1JPWwtxPJMshqxs2A0DFzEXUWMiGjXNxnp6UjBsBB359DbG2ztM9AzTgDolr6Vt+FcdQPVOGqt262tAmQZDEb3pTXjnXxsEQQBIUHE5foApriy4PYEdGMTyq/4sHD8zUiymDSKmNHasqdoDHVclXsu4gnM7wxjYo8wJNq7HDgxHgRBoJaE7ZVgolqy6usbeU6THfHKjDykOSzY/uhIWE2td1VdzTOgeQXitfZtZgM2zhmuUda8GrdbWwXIMhiM7klrxzue5yKxapUexTj+yoyh+PxkpSZmYuWUXLx77Bzuub0vkmxG2ExGzSSu15ZgSMB1KQnSZ7Rxdfv8kZLxkpflROG4gUi1m8FxEU8HbfHWEfQIQ6Kti0nJkb/8FQ/ejqf3/EVjeb4TZ+qjLxDGCx+elOpbCKKIxqCA6evpaaQtoS2fQbzWviCIuFTnx7J3v1WkOWUkWdlWBIPBaDdaO94JgojztT5Nae/Htx/F5nkjsOPwj3h7wSicq/ahxhfEu8fOSaUE9MboeNtCG1eff78MJfkuvPjJDxpZ7c6U9t4jBKnasxy4/OX3SbbSLU9BiHp9QRDh9vgRFkUsHD8I6w6cxkPrD6GuMaSpc08TroqHtnwGsVQ1iSjKZa9f84exeNcJOKzGTtH5GQxG96S1412VN4BKj183Vm3mqP4wG3m88d9/BwA8Nm6gRu5aPUbH2xbauLq/rBIZiRY8e99tMa/TkfQIj0R77iHJX76B46iWp4GLLwgyzWHBwvGDsHr6EFyo8SHR2jJp6mi0tCR6tGhnvRRP9bbO1kdG6rr0GAwGo71o7ZhPlCP1dBr6JlvRy2bGwvE3o3BrKf7rV3ltEsMgCCI4yvwxIScdlzx++ALtuz1/tfQIjwTQvId0XUpCpIhKG62I5VLVF+sasWpqrsLyXDU1FyYDjwu1PviCIZiMvKITEY9GmsOCp36RjWXvfovxa/6Mp/f8Bc4EU9zS1HKIh0Mtl6r3DOTHV3oacbbKq5G4lkuu0izsZ+7N0bgDz1z2tqr9DAaDcbW0Zsw3Gw3YXVqOV2YMVYxvK6fkYndpOQDgXK0P9f4QxtyYimRb68ZogiCIqPQ04qcrDfjpSgNenpFHHVeJcdPa67Q3PcIj0Z6k2ExSLfoXPjyJ5+7PQfHkwUgwG9AQCCMt0YILtT48vv1Ys6hSvgt9k61w2sySR4NWEW550/6YOngzmnuupWlPtONXTc1FmsOCimofNf4hXlXNlz49Rc2tZvERDAajM0JqDf3+aDk2zxuBK95ItsZbX57BE3cNklJBiVjgjsM/agIw5aKBgP6YPCjNgVPuesXna6YNkRaf/Zw2iKKIiuroSpedAWZIXCXVviD+8E0FNs4ZDkPTRM1xHJwJESWyc9U+LNr5jcJAKNhSiuLJg9En2YpUhxmZKTZqRbj9ZZUonjy4Re65lqY90Y5fvEupEU9zocWjqumu90fNre5MZXAZDAaDLJIeHnsjqhsCSEu0ICPJiux/ysHFuka4Pc0y1EXbjmLZpBxJgdJpM1FFA6PVH1J//uQ7x6WxlyhiqpUuU+1m9HPa0CepuSRDR4+jndaQ4DjuKQCrAKSJoni5o9sD0GMHBEHAHdkZmLvpa4VVmZHIoTEkoLfDLK3uCRXVkcIvj24+gveeGIsNs4fpSqzyPN+itJ6Wpj3pHe+0mRTtiOVC0xOuctq0HZuVB2cwGJ0ZURQhCCJqGoJ4fPtRRRoozwHnaxux7sBppNrNOFZeIy26aKKBuumfYUF37CVjrnxcPVZeg+J9ZZJiJoBOM452yhgJjuOyANwN4KeObgtBrzy2IEKzJfHkO8dRdsGDn606gPw3D2PJxGzkZTmlc2WmRErJVlT74AuEkZ2RiCFZyShRlbRtjeuKVl48miGgd3xDU2nxlghhke2Or/71LuwsGI0kqxFV3gBCoUg2x6XaSIW6C7W+uMqo09CL/2AwGIyrhYzzD60/hPO1jZIRATSngZ6vbUTxvjIsmZiNfk5rzDFbb4w1GXjdsZecRz6ufrH0TuwtGotBaQ5UeQOoqGnAxdpGpDksUvto4+i1GDM7q0diLYAlAN69FheLx82u557a/ig9M4GUmiVbBcWTB2Pupq8lj8WKD76XJnh5tc2rzSyhFrrKdyFF5mGIdfyG2cOQkWRRFJsBYrvQeJ5Dqt2ssZLXzXLhD99U4I7sDCzdfQJrpg1pVQQy82QwGIy2Rj7+cxyHtR+flDwDeh4DMq6/OnMoiicPxsB0B2wm+rioV3wwwcxTP+/rjMTPAcoxt29yxOhQj4GkYNex8hrNOHqtxsxOZ0hwHHc/gHOiKB7nuPafHEIhQaNi1pI6GHopnzW+oOK4G9Ps+PPicbhQ24gVH3wPd70fm+eNgAhRKuSVajdftToZz3MYlObA9vkjUenxo8obwIuf/IBFd2dTO0/U1KSm6p4t6Yw0g6twayk2zhkubf/U+ILUNCd5UbOWGHOshDiDwWgNtLFt5ZRcuD0B6jglH9srqn2wm41wpBphNfFS6qh67CJj7J6iMWjwh3HmshfP/P5baQ7YUzQGwZAQV32OjCSLZgxcurs5po14OogBIjeMyPHtMWZ2iCHBcdwnAPpQfvUbAP8GYEIc51gAYAEAXH/99a1qh56KGe1B66mT2cwGjVW5amouXvjwpOI4k4GHgQOu75WAl2fkwWY24FKdH7Nf/bJFlmI83pNqXxAzXv9K0dayCx7dzhNLXrUlE7iewWU0cNLn6ghk/Qqh8RlznSWXWk5b9E8Go71g/TMCTTSPTMy0TAl5ue7MlEi1zrrGkO44rvZ2/McfyxTlEma/eRh7i8YqpLIB/TF3x4JR1DFwYJodE3LSUXTnTXDX+xULY2IYkeqg7TFmdoghIYriz2mfcxx3O4ABAIg3IhPAUY7jRoiieFF1jvUA1gPAsGHD4tr0UU/CBh66KmbqBx0tmJBsSfiCYZyv8cFq4qVS12RrgZSTJbg92g6sJzNN2kyMj1iegbaecFtyPt0S6uAwIScd+8sqpQhk4hI08vFVCDUbDZiQk44priwpQnp3aXmnyaWW05r+yWBcK1j/jIytekJPJIjyrS/PYPv8keA4IBgWseKD73CsvEahEaQ3jtNqdsQ7qeuNuWER1PG1/IoPj985CBwHzcJYbhjp1eq4WjrV1oYoin8BkE5+5jjuLIBhbZG1QVOQ7N87ASl2szTBEWjBibHUydISLRAEEV5/CGs/bk7TSU+0oF+yTfPC4pmc1e6tjXOGS3UryPEt8Z60dsLVq0pKO1+q3azRvlg5JRf/8ccyPH3PrVKpW3e9H32Srch02nCh1heXoZJiM0mKcvL4C734DwaDwaBBxtaLtY3UxUnfZKsmRuxslRe/GnEDHvnHG9EQCCMjyaqbeREIhXG+NqQ7qcuzPGjjqN4YbuRAHV9Xf3QS7no/3n6U7rHITLHhqV9kt1utjk5lSLQnagVJ+QN9deZQAJCERuSCIi3ROyDGxvIHcmMeH89kr3ZvJZgNcU+48hiJ3aXlWHR3dqvESwRBRH1jSFOVVG8C53kOve1mRV41CQR65t4cRWl1UYw8W5s5PsOn2heUjAhy74VbS1mMBIPBaBFkbB1zYyqeuGsQirYdVcwHdosBSVYzqrwBXKj1wWw04PqUBCRaTYqxXU9Om+M4VHoadb0dQCQu7Jl7cxAIheH2+BVzBc0DvnJKLp77w1/x+J03SVsc8vEVAAwGUKstO22muLbwW0unNiREUezfVueKpiBZtO0oNs4ZjgV3DFR4EFoT8RpvOVe9rRL5ZK/2WugF/8jdVIIgatTSSvJdGJTmaJXlWeUNYPabh5HmsEjGQUMgDF8gjGpfUCfmgkfxvjJKPImRmtWxYfYwbJ43ArPfPCzFTND+wLpSjASDwei8kLFkfE6GZEQAzfPBnsfGxDX2643jBg66RkbfZCs+XzwOIoDn3y+TFrDy85NF6c6C0Thf40OVNyAZDGUXPNg0d4SioCMQMUyueIOaasskxXTNtCGScUHL8LgaOqWORHtAPAB6KT0WI48bUu3ITEmA0Rh5LHoBL21RcY2WH6zupOr843UHTmtqeayckovn3vtWqodBa3PBllJUy7JIoqHOOSZ/cER05aH1hzB309cQRFFT7ZPkJ0erdqf3TB1WI/YWjcWR34zHb+7NQaXHj2/P1+E3e09I99ZSjQwGg8GgEWs+aAwJcY39+uM4j92l5Vg5RTlevzbLhX//w19xx6oDmPn6V3h4zADkZTmp5+d5DqIoYuq6gyjYUqqIq+A5YJ1Kd4hW72jxrhMw8hx+uFQfuW8Dj2fvz0FelrNNx85O7ZFoS8jkpqcgaTMb485AaDMrLob3Qm3tuuv9yEiy6lqpJH5D7j0gFqhem+VbNyYjj/rGkOQZyEyxYfv8kdTnlZZogccfwpyNX1Mtdr14El2Vt5CAvsk2fHexTrP/t/bjk1j+QG5cXhwGg8GIRaz5wMAh7rGfNo6Tmh3yeLk+yVb876ZaHeR8S3efwMY5w3HFG0k3FYRIZWRBEHHZ64eoE1wpiMDmL8+iePJgDOhtR4LFgGBIG6+R5rDgcn1A46X4t3+6FQ6rsc3Gzh5jSJDJLSPJEnchqXjiGNqzZoTehHyh1oep6w4qjiWd3GY2YMnEbEU8w6qpubCZtW0WBEFT+ltdsOt5SuGwlVNyseKD7zB37ADd4l56RlK0Z1olawu5JxKcFAiF27UcPIPB6DnEmg+ixW3FM+bT4uUCobAiqB+IjHG1viAeWn9I2obulWDGD+56FGwpRZrDoolPe2XGULzw4Xd4eMwAKchyb9FY6ti6cPwgPKbaulm86wTeXjCKmgTQWnqMIQGgxQqSsYIWr4VqGG1CjjYZhwRR6nRAc8fZUzRG0+Zlk3IUsQzkWHlUMSkcJveCvHvsHKa4smAy8Hhhai6W7DoR955bNK+CXvZGnyRrTLEqBoPBaAnR5gMA0jhFsvwG9LZDFEWcrfIqvLZ6Y348hQ0zU2zSdgbZhn57wShpQVVR7cMLH0ZS5bN62XDa7QXPAW5PAIGQgFXTcnHa7YUgCEhLtGrG1v69E6hjqiCIUhBpW4ynPcqQIMQTEBlP0GJHKS22ZjIOhgRNm6NJwBIyUyKFw0KhMKauO4i8LKcm60WefqS35ya34jOSLFQ1Nz0DyZlgiilWxWAwGK1Bbz7IzkjEe0+MxYWaRoXHQu21jXfM18vEIAJXAPEsK7cojpXXYO6mr7FjwSgU7yvDigdv14zBJfkupCVaJQXNxqAAAxe5N9qYetrtlUo2tMV42mOCLVtKPEGLHZVFEC1QM1ZAorzNJAtEfSytYBc5b+G4gZqsl6W7T2Dh+EG6W0Tqgmf3v/wFquoD6Jtsk7ZBAHqQZsksF5a/X6Yx1toi4JXBYDD04HkOYQGSEQE0e20Lxw2Ujot3zFeP2zsLRuOtL89IQZRAZMwLC6LuuPzKjKEw8JxmDC7YUiqNiVX1AczYcAhjV36GZ9/9VhOUuWpqLl769JT03bYYT3ukRyIe4jES2lr4qSXoWdGxAhLlbV534DRemzlU2kPLTIkUFBNEEZ8vHiela5JiXBtmD4PXH6I+l4HpDmQ66Xtu8XpuaDEQgiBQ9xVZyieDwWhv9OYBtdc23jFfPm4LgohFd2dLIn3Es7DryE+acfm1mUNR7w/BaODAc5zu3KQea8nYubNgNEQxklX3xPZjCuOlLcZTZkjoEI+R0BmzCGIFJKrr2zcGBRRPHiwJmJCCYnqT/MU6nawXk0HXNdYSz028+4os5ZPBYLQ3evMAzWvbUmhjdYrNhP85NAv1jSHFuPzbd/8Kd70fOwtGw2kz6Y6JtLF2f1klnr1PxHUpCXB7/FL5BvV3rwZmSOgQj5HQWbMIosWAqNusV7+D9ofB8xz6JGkDemL9IV2N56YzGmsMBqNnoDf+ZCRZFBLarR3zaWN1dkYianwBTWzGhtnD0CfJCgC6Y6KeCBYZa9trPOWIu6MrM2zYMPHIkSNtft72TO3sTLT0Pltz/NVkt1zle+jwFxZP/+z/9PvXqDWMa8XZFffGc1iX6J89mY6aB6JdV+938Yy1LbyfuG6UeSSiEK/cdVenpffZmuOvxnPTU94Dg8HofHTU+BPLs0wvTxB7rG2P+2GGBOOawIwBBoPBaH86Yqxl6Z8MBoPBYDBaDTMkGAwGg8FgtBpmSDAYDAaDwWg1zJBgMBgMBoPRapghwWAwGAwGo9V0Cx0JjuPcAH6McVhvAJevQXM6A+xem7ksiuLEa9UYGqx/Kugp9wnEd69dpX+2NZ21H3TGdnVkm+Lqn93CkIgHjuOOiKI4rKPbcS1g99r16C73EYuecp9Az7rXltJZn01nbFdnbJMatrXBYDAYDAaj1TBDgsFgMBgMRqvpSYbE+o5uwDWE3WvXo7vcRyx6yn0CPeteW0pnfTadsV2dsU0KekyMBIPBYDAYjLanJ3kkGAwGg8FgtDHMkGAwGAwGg9FqmCHBYDAYDAaj1TBDgsFgMBgMRqthhgSDwWAwGIxWwwwJBoPBYDAYrYYZEgwGg8FgMFoNMyQYDAaDwWC0GmZIMBgMBoPBaDXMkGAwGAwGg9FqmCHBYDAYDAaj1TBDgsFgMBgMRqthhgSDwWAwGIxWwwwJBoPBYDAYrYYZEgwGg8FgMFpNtzAkJk6cKAJg/9g/2r8Oh/VP9i/Kvw6H9U/2L8q/uOgWhsTly5c7ugkMhi6sfzI6M6x/Mq6WbmFIMBgMBoPB6BiYIcFgMBgMBqPVMEOCwWAwGAxGq+lwQ4LjOAPHccc4jtvX9PMAjuO+4jjuFMdxOziOM3d0GxkMBoPBYNDpcEMCwK8BfCf7eSWAtaIoDgJQDeCRDmkVo9MgCCLcHj/OVTfA7fFDEOIOJu4y9IR7ZDAY3RNjR16c47hMAPcCWA7gXziO4wDcBWBG0yFvAXgOwGsd0kBGhyMIIk5e8uDRzUdQUe1DZooNG2YPQ3ZGInie6+jmtQk94R4ZjI6i/9Pvx33s2RX3tmNLui8d7ZH4vwCWABCafk4FUCOKYqjp5woA13VEwxidgypvQJpgAaCi2odHNx/BhVpft1m5693jZa+feSkYDEanp8MMCY7jJgGoFEWxVP4x5VDq6Mlx3AKO445wHHfE7Xa3SxsZHU8gFJYmWEJFtQ8V1T488OoXOHnJ0ykn2Jb0T717bPCH8cCrX2Dsys869b0yuh5s/GS0JR3pkRgL4H6O484CeBuRLY3/C8DJcRzZcskEcJ72ZVEU14uiOEwUxWFpaWnXor2Ma4AgiKj0NOKnK16cq26AycAjM8WmOCYzxYYaX1BauVd5Ax3UWn1a0j/NRoN0j3lZTpTku7CrcDQMPIdVU3ORl+VkXgpGm8LGT0Zb0mGGhCiK/yqKYqYoiv0B/BLAn0RRnAngMwBTmw57GMC7HdRExjWGxAo8+OqXuOOFA3ho/SH8WOXFprnDpYk2M8WGlVNyse7AaQCRlXsgFO7IZl81qXYzNswehgk56XjqF9ko3leGqesO4lcbDgEAnr0/RzImmJeCwWB0Njo02FKHpQDe5jjueQDHALzRwe1hXCNosQKLdh7H6mlDsLNgNEJhAafdXqz+6CSOldcAiBgWkRjdrgvPc8jOSMRz9w/G9JKDivtfvOsEiicPRuG4gSjeV4Yzl72aWIq9RWORlmjpyFtgMLoFLQnMBFhwJqFTGBKiKB4AcKDp//8OYERHtofRMejFCnAARFGExcjDbOThrvcDaPZOGLq2HQEgYkyIoki9/wSzAYm8ESWzXHjm999qft/VPTIMBqNr0ykMCUb3RxBEVHkDCITCMBsNSLWbNamNJFZAPplmptjQEIh8BwDe+vIMlk3KgdNmQo0viLe+PIPlD+Re03tpS+TPheM43fvP7pMIq6nZiJL/njwbBoPB6AiYIcFod9Q6CRNy0vHMvTkw8JxkVACAgQdK8l0o2FIq6SmsmpqLjCSrdMyiu7M1egvkd10N2nNZN8uFwq3N9//qzKFISTDBauLh0XkLCgAAIABJREFUC4Sxff5IPP9+GfaXVXb5+2cwGN0DZkgw2h157ENelhMPjxmAGa9/pTAGLEYes988jDSHBaunDUHfZCuMPAeb2QCnrdl7kZ2RiL1FY6N6NroK6piQ/WWVGJCagG3zR8Lt8aPKG8DLfzqFheNvRvG+ZuOhJN+F4smDwfN8l75/BoPRPWCGBKPdkcc+FI4biKW7T2gCBosnD5b0IX65/hAyU2zYWzQWvezKIEKe57pNYCEtJmRo/1TMbDKyCGUXPFg2KQf7yypRUe1DwZZSFmDJYDA6DR2tbMnoAch1Epw2k25AofozvSDC7lKXQv5cCKl2M/X5OG0mxc8tCbDsLs+LwWB0TpghwWh3iE4CEZKiCUw1BMKaz2hBhCSuoDtoKcifCxC55/REi64Al/zneAMsu9PzYjAYnRNmSDDaHaKTsKdoDHIzk1Eyy6WYPEvyXbghNUHxmV4QoV5dis6obhkL8lz2Fo3FF0vvxN6iseiXbNMYF+tmubC7tFz6uSTfBVEU4/IwdKfnxWAwOicsRoLRLsjTGk1GHmYDB3edHwVbSzHmxlRsnjcCV7wBVHkDePGTH/D0PbdiT9EYBENC1CBKPa2JrqqlQIv5UAeUOq1GPHf/YPzmXgFhQUQgHMaDr30ZV6XQ7va8GAxG54MZEowWE0sTglYW+9WZQ/Hyn06hotqH8TkZmP3mYU1A4d6isbguJSHqtWxmutZEV9ZSoD1PYlzQnuWqqblIc1ik4NRo6pZybY68LCcKxw1Eqt0MjuMgCCLL+GAwGFcN29pgtAjanvt3F+pwxdvsYqe504u2HcUUVxYA/YBL9SqZdq1LdX5snjcirm2QrgD1eV6sw6WmMumXvX7Ns1y86wQKxw2UzhHNw6BXx2N6yUEWK8FgMNoEZkgwWgTNSCjYWorj5bXSxCQIApZNysGOBaNQku+SCk6RyV4v4FLtVdDb33dYjYq4Aj23fleA+jy3lOKbilo88OoXaPA3b02QyqBrpg3BzekO5GU5AUT3yMjreNDSblmsBIPBuFrY1gYjbgRBhC8Y0k3ffHTzEbz3xFhc9gZQvK9McsWvnJKLt748g152cyR48MBprJqai8W7TkRVqNTb3w+GBM0WSFdF7x4HptmR5rDgzGUvMlNsSHNY8NQvsiVjQP5cF92dHdUjE62OB/FkxCNhzmAwGDSYIcGIC+KCv1jbSI1RqPEFUVHtgy8QliSugchktXT3Cfzu0VG4VNeItdP/AWmJFiTaDDGDK81GAybkpGOKK0uqrbG7tBxhQew2+/t69UXKr/jw1C+y8e6xcyiZ5UKlx6/xKCzdfQI7C0ajT5IVAOD2+HUNAb3rmI0G6d2u/fgkpriykGo3wxcIoV+yDUYjc1oyGIzoMEOCERdkrz7NYcHKKbmalfHqj04iM8WGsM7K93yND0++cxwls1y4LtkKkyl2cGSKzYR/Hn8zHlPVnth26Czm33FTt1B2JDEM8mBK8jzd9X4UTx6M3g4zHFYj9bmKYiTGQR2QuWH2MGQkWeBrKniWYjNprkO8QFXeANZ+fBIPjxmgeK8l+S7c2iepWxhsDAaj/WCGBCMmoZCAQFDAmmlDUOML4t1j56QKnOlJFvzLjuNw10eCII08H9VjUbC1FNvnj0RmSkLMCaraF5SMCKA5aHPZpJxuk75IYhh2LBiFimofanxBrP7oJABg2aQcDEiz41RlPYwG+nM1Gw3UgEwiOz5309eS0TAozUGtUxIIhTHFlaXxeDApbgaDEQ/MkGAo9sdtZgNCgihtOaTYTDhZ6VFU5JSvmHcWjMbLM/JgMxtwqc6PFR98p+uxACITVKXHD5vZGHOC0osfSLWbu3S6Jw2O42DgOSRZjfjtfbfCyPN4bNtR6Rm+PCMPa6cPwaKdxxUehRSbCRU1vqiy4/GkiOpJc3cXg43BYLQfzJDo4ch1CtIcFiyZmK0Igtw+fyQKtpQizWGRvBANgTD+7Z9uRYLZgD5JVvA8B7eneVXs9gSwbFIObkpz4KcrDVj90UkcK69BXpYTC8cPgjPBhEAoHDPOQW9fPz3R0mXTPdXQnn9NQwjL3v1W4R149bO/4el7bsW2+SMVVVGrvAEpIJPmBSJUVPsgCAI1joLERHQ3fQ4Gg3FtYJFUPZwaXwAXaxuxZtoQvDA1Fxu/OKOYwCo9filjoHhfGR5afwjL3v0WNrMB16VYJUNA7j04Vl6Dgi2leOqd40hLtMBd70delhNLJmZj2bvf4uf/+TkeWn8opo4BrRZFySwX+iZZu82+vTz9c8nEbDQGBdyQmqCY0Enp9dlvHsbPVh3AQ+sP4VKdH0Dkub/06SmsnJKreE6vzRyKdQdOS+eYkJOOy94AteYGz3PolxyJiYhHn4MVAWMwGHKYR6IHQrYyBEGAuz4grX7JNoTbE8Cx8hoAkYlu4fhBmv3zwq2l2FM0RjonzXvgrvcjPdGC7fNHIiyKyH/jsOIc0dztQCR+YFCaA9vnj0Slxx+R0/70Byy6O7tLa0fIIQZYXpYTDosRi3dFYkDkz1Kv9PreorEwGw1w1/ux+qOTCo9Rb4cZ7vqIsZGZYsMz9+Zghqw8ufr5G408bu2TRI2hkENT2owm0c1gMLo/zCPRw5ArKX5TUYvCrdpUTblq4u7ScvTvnUDdP28MCtKqlGQFyFe0K6fk4pnf/wUhQYSB51q1B1/tC2LG619h6rqDKNhSiv1llVj78UlcrGvsFitiYoAVjhsoxUSsO3Ba4WGIFr9AvDbuej8KtpTiyXeOIz3JApOBx86C0fhi6Z3YUzQGQtN3aOcgkLof16UkIC3RQjUM4ikCxjwWDEbPgnkkehjyiUBPqpq4szNTbFh0dzbsZiN1//x0Zb0mK2BnwWicr/GhyhuQYiPKLniws2B0q/bg1QGXxM0/veRgt1gRp9rNKMl3wRdQbg0RD8MtfRJhipKxIa8gGgiFERZEPP9+GfaXVTZvTzjMOOOmx1G0NAYiVhEw5rFgMHoezCPRw5BPBHpS1f2cNny+5E7sWDAKGUkW9ErQxiqsmpqLlz49BaB5VVrtC0IURcl7QLZHKqp9MHDQnIPswUdbwZIVO0HPzd9VpZ55nkNvuxnJNpPiPo+V16B4XxlsTQGt0Z4dybjhOA4zXv8K+8sqATQ/m8YgPY6iZJarRUGrgiCC4zjsKhwtSZ+TcxGDhJUtZzB6Hswj0cOQxzIQF7o8VXPD7GFoDIal6pw0DQIAeGL7MclQAJpXpXqZFjzPa8pjk0ks2gpWLtiU5rBgULqj26Up8jyPVR99r3kXJfku9LZbNF4HvWe3q3A09dkYOI4aR9HXGX/QKs3TQJPo1vNYNARCqPSI0v0wGIzuQ4cZEhzHZQHYDKAPAAHAelEUX+Q4rheAHQD6AzgLYLooitUd1c6uSLS6CfKJ+Vh5Dd768gy2zx8JA8/BbDTAwAP3v/xF1KBIt8cvBfIRyKo0xWZSBEfuLi2XJhqyBy+HpI3K00sv1jZGPCGySfS9J8biStOqtrulKabazVh0dzZ+f7QcG+cMb3oXPDIczZNutGdHnkWVN0B9NjazQXrnBVtKJWPNaYtM/vHU2aB5Gt768gyeve826fdE34PWhu8velC8r4xtczCumv5Pv9/RTWCo6EiPRAjAk6IoHuU4LhFAKcdxHwOYA+BTURRXcBz3NICnASztwHZ2KWLtUdNWtyk2E6p9QWlVn+awKCaCNIcFgVAY56obosotp9hMOOWuV3xeku/CoDSH7sQRCIWpBalKZrngtJmlNocFoPyKD787/CN15d6VdSVIdsr9eZmYu+nruGMLSJVVUofk07JLVA9TksWEkEPE9kdHwsA1a1DwPBd3TINerMpD6w9pPFd6kt/xZOowGIyuR4cZEqIoXgBwoen/PRzHfQfgOgCTAYxrOuwtAAfADIm40dujlg/e8tUtbSJZNTUXL3zYLCK1ZGI2dcJQu9r1SmJHmzjMRgM1vbRgq/J7gVAYCWYD9pdVSoJXZALt3Q0qVVb7gppiZ49uPoIdC0ZRvQSCIFKrrH5+8hJ2FoyGKIqS0ac27uTeiHj6C6BN742Wkkokv/0hAacq66WgW3JcV96GYjAYWjpFsCXHcf0B5AH4CkBGk5FBjI30jmtZ1yNWVL0a2kSyeNcJLBw/CACwcPwgLN51AmkOC0ryXVgzbQgu1jaizh/UpAq29NpAxK0/oLc95vfMRgMaAmFkptgkwauH1h9C8b4y8Hyn6MZXhd6zq6j2KcSjCFXeALXK6sxR/dEnySq9l2pfUPN+5emzxCOkvq76nanFwaKlpPJN22Q/VjWgeF+ZIpamq29DMRgMLR0+AnMc5wCwG8D/EkWxrgXfW8Bx3BGO44643e72a2AXg+M4aiYGx+lvLdAmhIHpDnz1r3dhUIYDr80ciufuv02hbHmhplGjD6DOsCDXjjZx8DyHBEvs76XazbghNQGrpiozD/TUFzualvZPvWdHip09uvkILnv9UnaLLxiiB1Y2bQUR1O93uisTj985CNNLDmLsys/w0PpDWDIxW8rAINdVvzP5ltgXS+9EP6ct6jvrau+rp8HGT0ZbwpEyxB1ycY4zAdgH4CNRFP+z6bOTAMaJoniB47i+AA6Iopgd7TzDhg0Tjxw50v4N7kToBchdqvXhb26vpmhW/9QE9E22abYA3B4/Hnj1C8Vkk5liw56iMaiqj3grlk3KkVzo8mPkLvRoGRjyctaknfL2mwy8JKikt08fceX74Q+GwXORydLAc/FkAXT4nkc8/VNdc2Ph+EG4PjUBF2p8WLP/Bxwrr8GBp8Zh1hsRdcqNc4Yr6nEAkXei3pKQv9+8LCfWTB8iZeTIvyevFBrJFjGD53lq4CUQqQirLuZG3pkgiKis9yMYFmDkOViMPBqCAqwmXvO+4gn07OZ0+M12tfGzKwdbnl1xb0c3oaXE1T87MmuDA/AGgO+IEdHEewAeBrCi6b/vdkDzOjXRA+R4vPXlGUUMwVtfnsGvRtwAs9GgiVWQZ3HIzwURMYWrztf4MHXdQcX15YGcJiOP+saQlAVCjruptx0/uOsVk9DLM/KwetoQ9EmyIsFiUEw4PUHkSJ6dcr6mUVIcladZnrnsld7DS5+ewqqpuYoCa7TVvvz9Fo4biCveAN0DlWbH/1syDoIILFcJWqmfsyCIOOWux4uf/IBlk3KQajcjPdGCfsk2CIKI7y95FO1fN8uFWzISYTQqHaA94b0yGD2BjtzaGAsgH8BdHMd90/TvnxAxIO7mOO4UgLubfmbIiCb6k2o3Y8nEW2A2RF6t2cCj6M6b8NKnp6Tqj3LhJ7XLek/RGDgsBnj8za5zPeEqIjIkv75cZpkDp1j9kv3583WNmv39J7YfQ60viFlvfAUOSvd8TxE5ItkpNNny39ybIwmAARHBqr1Hz2HHglH4fPE47CwYLWXHyAW+qrwBKTD2lj6JUoqonMwUG0QAF2v9mEkRtFI/Z/I+9pdVomBLKaauO4gZr3+Fal8Q7nq/pv2FW0s16cLy83T398pgdHc6Mmvjv6HvNhl/LdvS1YgV1OgPCYpCXGumDcGI/k5clgXoqVd/co2Iv1V6ATTrNaw7cBprpg3Bk+8cV5xzxQffU68frZ1TXFlwe/zU9hPPh/w8giDqxgN0x+h/vXfLN4lKEfKynHhg6HXUbBpalkZ2RiKASO0UdYroKzOGYvn7ZVg26Tbp2nlZThSOGwinTVvyPVr/Cwki9XfBsBD3vfoCIbg9UKQl99BtDwajS9DhwZaMlhFLppgWzf/kO8eRP2YANb1QvfoLhMJwJpg0kspWE4/iyYOxY8EoFE8eDKtJ2XVoAXq0AEKSJqoXWCg/D3F9n670Uo8PC2K3KwhFe2YTctJh4DlsfWQkNs4Zjrwsp5RNo36flfX+qN6qX4+/Wdr62lU4GpvnjcC2Qz9if1klAiEBmSk25GU5FWXj1SXfowXVGnh6sC/NANA7z3cXI0Xlvr/kwW/2ntCUPWcwGJ0LZkh0IcjEOr3kIKauO4jifWV46hfZmJCTLu2P663ywjorRbLaJK5wjuOQ2lSCmkgq/+f0IXh8+zHM3fQ1Hlp/CHM3fY3Htx+TUkRj7c/Lo/bTEy3Sqlj++SszhmJ3abniPMT1TasTsXJKLp5/v6zbucHVz2xCTjoWjr8Z00sOYtzqA1j27rd4/n8Oxi196VLhwbAQ1XuTlmjBskm3SQbdugOnMT4nA7sKRwMc8PKMPKquh9zopL1X8t5sZl6TqbFqai5sZu1QQzvPyim5WHfgtLQlMsWVRW0Dg8HoPLBaG50cdVEm9Wpz6e4T2FkwGn2SrFL+Pk2iWK+CJM9pMy1KZrnw8ow8PLH9GAq2lOrWcBiYbscXS+/UdTvrqWguujsbaz8+KQXqpSVaYDcbsPyBXMV5iFFUUe1T1IlIT7TgX3Yex7HyGjx7X/fa3lA/M47jpEqnQLNYl141Vb33bDLyiqyQJROzsbu0HA+PGaDY5lgzbQgGpkfX9dCr/cHzHFJsFmQkhVA8eTASzBHtj4wkK1JsWkEy+Xl8gRC+u+jRiFc5bSZqGxgMRueBeSQ6McQD8cCrX2Dsys9wvsZHHeBFsXn/Wm+1mN4kKEVc1xvnDMdb80bAFxS0apRbS9EYjMgv71gwSlOZkpyX5ziYjPopggAUwZdpiRYYjZHiXcsfyMXgfkm4IdWOrJQE9E60SsJW5N7lmhhEhOrJd47jh8p6HCuv6bbiRuSZ9U22IRCiexhEUdR9z7TPjXyzEXqsvAYvfHgSSybeqvE8PPnOcfiDQkxdD0EQEQwLCDX9l2w58DyH/ql2DL4uGZkpNgy+Lhn9U+0x+4fNbKSKVwXDAkryXdixYBQ2zhkOm7n7vW8Go6vDPBKdGHVUu15RJvkAH2212NtuxqqpuXBYjHhs21FUVOtXjOyTZMVTTcGVE3LSsW6WS5HSt2pqLv55+zG46/0tTtlTS3SrdQSAiJdk7ccnNYGBa6cPwX/88fseIW5U5Q3gzGWv7jvPzrBR3zPt/V+oVRqhx8pr4GkM6m6PqN+3/FmHQkLUFE9agbFY0NKQS2a5AIh4es9fmj/LdyEYEqJqXDAYjGsLMyQ6Mep4B72y3/LJVE/gRxBE+EMCGoMCFu86GtM4SbAYNFsSex4bA39IgD8k4GJt9NoM8aCnI5DqMEufkboaqXYznAlmACLWzRraIyaSQCgsxYeoC5rpVVMF6JVCbWYDNs4ZjgSzATW+INYdOI1Eq4n67pNtJmQkWqlGCgBUylI8SXZHYzCMS55GquhZPJDCZfLKsQ2BMBbt/EbpLdtSKgmkMc0JBqNzwAyJTow63oGU/VYrSsYSbrqptx2V9X6ERRGDMhwYc2MqdpZWANA3TmgKhG6PHwUqoSSyp61O2YxHrVBPR2D7oyMV91ywpRQAsGPBKDz5zvEeUz3SbDQogl6dNhMaAmH0dVpbNHkKgohLdX5FSnDEM8XjlRlD8fj2o4pUUIuRh9HI6z5jEtBJsjvUfSeeyZ3WR6p9Qcx4/Svp3e9YMCpqmjCrJMpgdA6YIdGJobl7F92dLQVWqqFNzF+ddiPBbIC7aZW3u7QcT9wVybbYWVoR0ziRn7uAIpREVofqlM141Ar1MkwMTbER6pUyqTvRUwLu5O+/YEup9CxJ5c54ofWLjV+cwcLxN+OVz05hxYO3o2+yDWYjj+qGAGp8QfSKIj1OAjqjVQCNNrnr9ZEkq1Hxzkk6MK0fkOsFQmG4PX6mNcFgdCDMkOhAYq3cibt3Z8FoBMMCTAYe6Q79AV49MedlOTFsQG/MbFrlES/Cy386hSUTb8XO0oqYxoneuYHIQJ5qN6Mk3wUD33w/ZIIgbm+vP4SLdY2aa+hlmNjMBo0BRbwf3TXAkoZcNtsXCCMsirCa9O9drz/pCYMVbi1FmsMCjuPw8MbDCm9FL7sZvex0YyDdYcG6WS40BvWFqeSTu1pYysBDY9is/fgknr3vNuwqHC2lpa47cFojA076AdCsJUJqiTCJbQajY2CGRAehtyqTF7hKsZl0VQrVha/MRgNMRmXqX+G4gVS55WWTcmAycDjw1DiprgUAafA3GXkYeU7RjrAgUif9XnYzntx5XAq67JVgitvtrVfng6y4V08bgrREC36qasDqj07CXe9HSb6rWwdY0rhU54/ZBwRB0FUupRlspAz4skk5Gq/C4l0nsGPBKMBOb4/RyCM73YHKej+1T4QEEQ81Te5EB0MemFkyy6XYXsvLcuLhMQMUKp2kvkhGkhV7isYgGBIQFkQ8/36ZlLFTku/C8++XKdrOtjsYjGtPh1b/bCu6WvU6QL/qprwK4/b5IxV7xuSYvUVjkWo3awyRzfNGwB8SsPbjk5jiysKgdAfuWvNnzbV3FY5G32SrFBhHM2pWTc3FCx+eRFqiGb+97zbU+oIIhUXFfjo5hqTsZaZEKoJOLzmoWzFUPcjLJ8KwCGl7RRAEFG49iicn3Iw+yVYYOA4X6xpxY287MpJtmnuKQocvTa+mf+r1E3UfiPa89frK7DcPY820IXho/SHNdT9fPA42s5G6VUD6y9qPT2p0KF6ZMRR/PHEOQ/unwmkzoZfdjFUffS/V7yDt2jxvBJ5s0gIpyXfpVpeVe7HUhrMgCBj5f/6kkPOu8QXhut4JEVxX2e7o8IZ1tfGTVf+8pnTu6p89Hb2tgoSmPPmKah8qdWpSBEJh6r737DcPY98/j8Wvf36zFN1OWzGmJVqQkdg8QNPOtXjXCax48HZwHIdfNq0UJ+SkY/O8Eaj1BeG0mSRRKHnbiL6B1x9/fQwDD1TWBRSBnCX5Ljx3fw4e335MYbiYjD1L+iRaXQv5e9Or0OoLRp53dkYidhaMxvkan7R1sHJKLhoCYWof+elKA57e8xfqVoH8uvKsmmSbCYCIO7IzNGXs3Z6AQmjqijeAheMHYe6mryXviLrtcn0UQJuN4vb4MSEnHY/8442KOjCvzXLhvz79IWoFUwaD0Xb0rFG5HZDLS5OKmvGgV2eABJIBzamZ6mPMRoPuBOMPCqis82PNtCGwmw14eUaeQpyoZJYL/ZKsipLOeufqk2xVuL33l1Vi9puHUenx42xVg6aiY7O+QSL6OW1xiRqdvOTB8fJaTSBnwZZSBEKiJIq1bFIONn5xBqFuWmtBrx9Fq2shf296FVpPV9bju4t1cHsaIYoilr//HQq2lGJnaQVWf3QSVhOPdbNcij6yZtoQrNn/A9IcFlysbURFjbJN8uuSrJqp6w7iijcAgNNslSzdfQKF4wYq2lXlDWBgugNfLL0zrr5CI9VuxrJJt0lGBLneYzJpbb17YDAYbQczJK4CtfJkrMJC8snCwEOjQLhqai4+LbskKfn1SjDjNdUgvyE/ohuhV9zJXR/Asne/xUPrD+HpPX+BPyhg1dRcfL54HPYWjcWtfZNgMikncxL/ICczxQYDx+kGWKbYTVgzbYhGQZG4kfskWXXrMRDIyjbBbNBcJ81hiRQKayocVbyvDA+PGYCwILbYaOvsROtHekqlTVXipc+Jh0FtEFhNPHyBMOoaQ9j85RmsmT4Ee4vGoCTfBQBYvOsE+jkjmhGfLx6HZZNypKquT/0iG8ve/RZ3vHBA0aZoRnC9jieKvHfiodhdWg6byYDrUhLi6is0eJ6DCHoNGafNJMXpqO8hFBJaZfwzGAw6LEbiKoi2fy13wYZCAtz1fjSGwjh7uQEvfXoK7no/Ns8bAYuRR0V1RE2wd6IZ1d6gIkp945xhqPIGwQFoCISR1cuGG3s7AGhrZOjFVBRPHozB1yVTA9DcHj9+s/eEZq971dRcAFBUmCTne3vBKPzvP/wVbk/EPT2gt10K2lRrT0TLSjlX3YCxKz+j7pFvnDNc0j2QX3vT3BH4+X/+uSUu6w73Z8fqn7H6kfw5mow86htDmP3mYYy5MRWP33WTlNp79GwVHhpxA2p9QfSym1HdEMATsq2hV2cOxct/OiW5/FdNzUVGklWSsJa3Qy9uQS/mgpx7iiuL+r23F4zCxdpGKQV50d3ZincXr/aImnPVDVKQpvx6yyblAAC1LfK/k06w9dHp+2dng8VIXFNYjER7Q9sSSHNYIAgCzlU3ICyKsBgN8DQGMWfj15oUttlvHsbOgtGSa3brIyMkOWAgsrKauykSSEdEmTJTbHjvibEIC0CS1YidBaNh4ACe53W3KAb0tuuu7gKhMPaXVcLtCWDV1Fz0SbJCEAGbiYfVzGPb/JEKDYpFd2ejT6IVyx/IjTnox5JKJitbmijWgN70wlGexmYNge4SoR8tDgJofo6CIOJyvR8cB7w1bwR4Dlj+fplkGLw6cyjWHTiNL/9ehY1zhktGBDlf0bajWDYpB/vLKqU4mD1FY6R3l2IzScqSzgR6zEUgFNbIcAPAi59EjIh+yVaNyNXmeSNgNvBIT7Kgb7IVw/vnIsli0hgOrXmPNrNBkyJKjJpH/vFG6j1UNwQlga8aXxBrPz6J5Q/kdvl+xGB0FMyQuArUaXV5WU48d38O/n7ZqxjYVk3NRZrDgopqnyIFs2BLKQwcpBRIk4HXddMS0hwWXKhpVAQmkhVVNLlrWrqoeotEEIH8Nw/rp+3luzAozaFRPSRbNvGuJuXt2D5/JJ5/vwyrPzqJ1dOGoG+yFUaeA68jSuX1hxTPpjuIU+npaajjSc5WeXGprlGjq0ACGYu2HcXqaUMwe0x/NAToxom6mmYwJEjnl6cab5wzPGqb5Eai2+PHl3+vUqRzFk8ejIHpdjgsRlyq82P2m18qDItoKa009DwWTpsZGUlWRbXRtEQznrt/MARRP2X5lc+aPTMrp+RCEIRWvTsGg8FiJKISK5BSvX+9cPwgXJFtTQDNGRDyYDMyoGem2MDzvLS6y0yhB53JAzAXjh+kCUx8dPMRVHkD1P30knwXetnMuvvwKTYTNswehoXjBymC5Ii5XljRAAAgAElEQVRgkToAslrWFvKMWhonIj9+xutf4dc/vxkbZruQaDVi5utfYezKz/Dse9/i1ZlDFffyyow89O9tx5+e/Bk+XnQHCv5H/24hTkU8AbsKR6Mk34UJOenUeJIfqxo0fUseyFhR7UO/ZCtuyUhEWqIlZl+SGwbqzJ2XPj2FVVOVMRe0uAVBEGHgIVWWBQB3vR9WE49QWERIEDUZQT9WNVCl0au8AerzidbH1NVGh2Qlo6YhhOklB/HP249p7mHllFwU7/srHh4zAHlZTukZhsXma7k9flyq9eF8jY/FUTAYccA8EjrEI/WsVh4MCSIEQZS8DwT1SjAzJaLeuH3+SCmNjwzQaoEmkspGvqfn8icu50FpDvzu0VEIhATwHHC5PoCfqhvgsBqpg/feorHIzkiE3aIMeNRLJ1R7APTqZZC9dPUqknZ8wZZS7CwYLYkpAZB0BzbOGY4r3gAEUURjUJBSUTNTIhUnndau3YXVngC550e+Og+EwlJQqlo3oV+yFUBTgCzPgec5WE08Sma5FJ6rjXOHo6o+gB0LRqEhEMYNqQlSv1Nvr5BS4zsWjAIAqqdJrifx9D23YtPcETBwwMW6RrzwYURATF43hUALro3mXYrWx0jpedK3fIGwdGxFtQ8vfHgSxZMH4/peCfibu16qDVN2wSN5BSuqI6mm8vtRp5SyFFJGW9CS+I6uFE/RtUfhdkRv8NpTNAbpiVbFsWo3LU2oqSEQlv7/zTnDEAiJ1IAvdQnoFJsJyx/IxbP3RX4WQXfXkpVlnT+IiuoGzZ6x1cRj2aQcrDtwWpHPTwwQm8moOK9enQO1ByDa/n489RTI8aGmQlBy9pdV4pF/vBEPrT+kCf6rqPahcGsp3ikYjb7OFglUdSr0DKs9RWPAyUSVbE1u+wk56VQRqIL/0R8PjbgBYVFERXUDnn+/DG5PAMWTB2NAbzsSbQZU1gak0vDkfRBo2yvuej/MRoNu7ECVNyCJUs1+s1lie820IVgyMRs8x1HrpuhpV+h5l2LFkMiN/jXThmgMormbvsaOBaOkOCPyfWLck2uTd7Hiwds1KaV6f/sMBoNtbeiiN3g1+MMKN+dlr18zESzedQL/+dAQyU394i//AVYTj12Fo7F53gjU+UKabQPi2iV7z9elJCAt0SLFI5Cfe9stUVPlfIGwxv1dtO0o/nKuDsX7yvDUL7KRl+WUvksGb/W2yO7Sco2+AM21rZcKyHEc1RDjmiYW9fHGpkJQ6s+JAdYnyUp9H8Fw197bjtbP5K78S3V+3JRux9P33KrRaXh8+1HMGNUfs988jF//7hucdnux9J5b8eSEm/HSp6cw642v4A+KeHSL/naCXppptBTMQCiMKa4sTXuefOc4GoMCHlp/CP/+h79q+tENqQktulY0LQ1AaYzp6WmQfiT/jBxLrk3eRZ9kel9rDITZVgeDQYF5JHTQC4A7c9kLu8UordL0ChdV1vlRvK8Mr84cin7JVpx2e1HlDWD5+9/h6XtuaZFrV06sQk6RFal+6WV5xU6iR3CuugFmowGD0hwab4j8Z5prW4SIrY+MxJnLXimtdcPsYTBwoLZDHlwqXxmnOyzUzzOSLPjqX+9CY0igvo+u7mqO1s9oK2KA3t/cHj/SHBZNfROSIRSkeHzkfU6diRFP0KzNbMDNGQ6smTYENb6g5O2qqG5WaCVbVOrqsgCwp2gMGoMCDFzkXHro1WRJsZng9vjREAhJ3jZaBtCG2cNgkdWhIdtHve1maQuO5znpXehVnz3t9kry9Wyrg8FohhkSOqTazZo9ZjIov/jLf4DbEznGGKPkddG2o3h7wSi88d9/lwbVeLcNoqEX9W410ScmeenlW/skYk/RGNQ3hnD/y9ErJ+q5tWkxJCWzXOjrtMJpM+tmkMiDS9UTlt7nbo8fy9//VjNBvDpzKGzmru1Uo02SJbNceOb33yqOq6iOZFhYVVtQQLNSJK2s99LdJ7B62hAYeXo/lfe5WOm6cgRB1PRB8vfhrvcrgjr3l1Xi2ftEXJeSoPh+VX2A2ofVkzOtb9AK2pHrr/7oZFPWiAM2U7PhEstIIu/icn1Ak1JKtivJc+0uqccMRlvQtUfhdoTnOfR1RtLKiEQzGSS/u+iRIsctJp4aGb7uwGkAzbLVC8ffjAk56QDi3zbQQy9+o8oboG59yNsTCfQ0ggMn7Wurz9HaNhRsLUVYgBT8pue+Vm/fyINXaZ8TrYvVH51USGan2s1IsXXtgVw+SX6x9E7sLRqLvk6rrvx4is2k6TvrZrmwu7RcN0A2I8mKf//DXzXKly3pc2po73/p7hNYOH4QVk1t7m/ytsf6frT+p+4b1b4g9fqF4wbCXe9Hn2QrMp02qR/p9S31NbIzEjEw3Y60RIv0t7/lkRGKmCdyve6QesxgtAWd1iPBcdxEAC8CMAB4XRTFFde6DU6bGX2SrdRVDxn4diwYhRc+jExwN6U7pJLX8kDLC7U+PL3nL9hZMBrP3ifGtW0QjWjBZ+rVm7r0Mpk83J5GSYAqLIq4WNuINft/iHtw1GuDLxDC+RoBoigi1WGWtmCupgojcTmTug5A5LnKxZS6MmpPgCCIVFc+yUx46dMfFIJKf/imAo/fOQhXdLxAZy97JdExYoD1c9oUlTVbit77H5huRygsSoaQnsGiJ+YWCIVxqdanqARL6zd617+1T6K0XQFAoW+SYjOh2heM+jfH8xx62S1w2sxItkXiJjiO0zXsWqvIyWB0J2IaEhzHPQFgmyiK1degPeSaBgCvALgbQAWArzmOe08UxbL2uJ7eYCCflH2BEL676FEYCRXVkQHPXe9HwZZS5GU58fQ9tygG0TXThmDFB9+jojqSYiZ377ZW1CmWgJF8YhIEUZH1QQZYT5OwU74s2v61mUNhtxiiPpNYbQgJIma8fjCmu7olkFW4XBxr3ayIPkZ3RM+VX+UNoCEQwhRXliL7BgDuz8uE1cTjtZlD8di2ZlVJ+TaJ3BD7Yumdcb+TaEJm6vdvMxmR6jTHNJJpYm5LJmbj3//wV01WColn4HleOpdJFvOguL7ZKKmAyrfeaAJr0fqm+m9IL0YjVoo4g9ETiGdrow8iE/lOjuMmchx3Lf5CRgD4myiKfxdFMQDgbQCT2+rkcqGpyrpG/P1yva6gEhlQbGYjiveVKQbvzBQbrCZecuEfK6/BG//9d7y9YBR2FY6WCiARb4BeDERLRZ1aEmEvd+mSFW1FTQMATpPd8di2o6jzhRAKCVJ7nth+DN+eq8VPVxpQ6WmU2kRrwyszhmLFB9+1ertEj2pfEC99+gNWPHg7PvmXO7Bl3ohIISp/MPaXuyhke4hU+Txf68Nv9p7Az1YdoGbfJFqMWLzrBH777l9RPHkwDjw1DnuKxkTdJomHWEJmLdm6kkMTc1u86wQ1C6RgSym+qahVFN2qbwxpthRLZrlg4JsNH/nWB01gLd6+Sdt+ys5IpG6vrP34JC7WNbLsDkaPIqZHQhTFZziOWwZgAoC5AF7mOG4ngDdEUTwd/dut5joA5bKfKwCMbIsT04IE1RLWtEAqvcjx3vZISuZ7T4xFYyCMoCCCB+BMMOF/7fhG45qmEUtwR01LI+wFQcRlrx8N/rCUXfGbe2+lZ5t4/DAaeDy6+Qg1C0C+4lJ7a3iuOUpffs6r3UsOhMJwewLgOE5Rs6RklgtOW/d0JdP6qVwOW5190y/ZRu0PtNX05nkjEAoL+LHKC5OBR7rDoigrLyda32xplocczRZcU7ZRP6eN2i/TEy3StXcWjMbsNw8jzWHBigdvR1+nDT9VNeCZ338rZQ31UtUKiVdgLVp71X+L6u2VvCwnHh4zANNL2tYj193pykW4GBHiipEQRVHkOO4igIsAQgBSAOziOO5jURSXtEO7aH91CtOe47gFABYAwPXXXx/XSQVBxMW6Rqrug7wwFm2AiTZ50yLY104fgncKRkEQtaqAaldxLMEdGvFE2AuCiBpfQFObY+WUXDQG6aJAVd4A0poG7WWTcjSrQ7WiYFqiBW5PpMriskk5V52NQsNsNGgkvElwZ2eNnG9N/5SjF8woV2O8RRYPoNcf1P3WauZxocavEJBaN8uFWzISqcZErHicq3n26nodE3LSkWihZ6U4LEbp2iSVtaLaB28gjIdlQcMAJGNDfp62yJRSo96eoWXNdNbsjqvtnwyGnJhbGxzHLeQ4rhTACwC+AHC7KIqPAXABmNJO7aoAkCX7ORPAefkBoiiuF0VxmCiKw9LS0mKekKzwztf4qAOjWsKaNsDouWxpg/6incfx/cV6SRlQbkScrfLitLsevmAY9f4gDDxdpOlqBjlyv8fLazW1OZbuPgEDz2mi+NdMG4LdpeUwNYlDxbuKI96a3aXlbZoZID9/NGnwzkhL+6cavQlcrsZobOqPsVa78n7rD4oaF3/h1lJUqrY/CHIxqLwsJ0ryXdhVOBoAcKnW12bu+1S7Gc/cm4P/+GOZpg+tm+UCWUdkptik/gnoexpEUWyVwFpL2yy/Rqrd3GX66NX2TwZDTjweid4AHhRF8Uf5h6IoChzHTWqfZuFrAIM4jhsA4ByAXwKYcTUnJJO93qpZLmFdku9q0QCjN+gnmA2aQaTGF0BVvV8hVbx2+hBsnDscc2Vue70CSTW+gEKIqredPpGQ+1VLBpO2ZSRZUVUfwM4Fo+ALCrhQ68Mb//13LLo7WxKHuljbGNcqjqx6lz8QqaKoFh+6Wrcuz3NIsMSukNmd4KLok5DtuGgiTnroCVOFdBRCU+1mlOS78OInP2iCIFdOycVbX57BoruzkZ2RCABRA3SJJ04QBGpWhoHnFNklJCslEpXFSdsyRp6TRNCCYbpQmdloQHaGTROw2tqtGBpqb4/eO+uufZTBIMQTI/HbKL/7rm2bI5031JQt8hEi6Z9viqL416s5J5nsacp3a6cPQUgQsatwNNITLeiX3DLFRL0I9oamtEf5VgYALNqp1PFftPM4Vk8bgk1zR8DTGESi1QSbicdlr18yFPTKSOvtwZL71XPpnqqsj+yv5w9DX6cFVhOPZyblwGoySANkRpJFI8oVK6hTfa9A7AyQeCD6GLSUyO6IgYOmn74yYyh62U0onjwYGUkR4a+WQlbz6v5gNNCdkzzPobfdjMW/uAVzN32t8WytePB2KQ2apBrvL6vEhJx0PHNvDgxNipFEQIrU5qDF3eil+S6blINb+0TUXNUlyTfNHY6SfBcKtpQizWHBwvGDMKC3HWKTB0O9pdASwS15n9VLHY0nu6O79lEGg9BpdSREUfwjgD+21fnkgxQRNiL59FYTf1VaB7RAzFVTcyODvdWIiuoGVHr8qPIGdGtGcABEUUCyzYQr3gBOewPYXVourfZIGell736r2YPdWTBaowlA7pdmOMm1MNZ+chK//vnN0r67fGAn+fQtCepUBwhunjcC/pBw1SlyrZFw7srwPI+3vjyjWJm/8tkpPHvfbRh8XXKr7z3dYaGm0qY79CdYnudR26TUKqei2ofrUmxSoPKT7xzHyim5cNrMmJx3HZ5/vwxTXFlItZvhT7Zi7ccnqVkZRP7byHNUNdm3vjyD5Q/kIixAs4U4Z+PXeO+JsXjvibGaWKDWBjq2NnW0p/VRBoPQaQ2JtkY+2R8rr5Gi3aUJ2N76c5MB5N0nxsAXEBAWRJgNPFITTPihsl4xuG2eN4K6IuQ5Dv6QiMKthxWD6NqPT2L5A7mKMtJyKqp9OF/jQ60vqBjY5PdLJIP797ajpsEPrz+Mp++5BTW+IJKsRkX5bnmAmLwMuM1sQEgQcaHWpztA0mJF9Iyf1gSgXW1wX1ci1W7GoruzNQZY3xZ6y9QYjTxuyUjEzoLRCIUFGGNkbZB6Kn2SrNR+K4rAv/3TrbjSEJC8FBvnDMeqj76nboXoVX9t8Icx642vMObGVGyeNwJXvAFUeQPS1kmq3YwLtfT4pmAoslXiC4YVFW5p/UzuaSB9OhgSFH063tRRWh/uSX2UwSD0GEOivVYL8oFJ7trNTLFh2/yRmkDHFR98h1dmDMXj25tFg1ZNzUVGsgUzNnxFjdIXBAEcx+mWX062mbDqo++x/IFcaRCj3a/JAFxsUtkk135t5lBpRUkgAWJkVZbmsGDJxOyYWyq0WBFi/ORlOVE4bqC0uhaErl21s71pz9Wt0cijXxyl18l22o9VDXAmmLBt/kgsl/XvlVNyseKD77Bs0m34jz9Gdjkrqn0w8BzV80CMDD0hM1L86/3j53FPbj+kJVrw3P2Dke6wKIpqyb87IScdl70BhUeNeNyOlddottji6dPqfhwr6JipWzJ6Oj3GkADafrUQK9ff7fFrBqD9ZZX49c9vxlvzRkAURRg4DpfrA4BIr5bZJ8mKy94AXvzkBxTdeZOmmNDKKbnS6o+DqJEElhMIiVQRquLJgzF309fScZkpyjLgyyblaL736OYj2F04GkFBhNAU+Gkzawf6hkAYE3LSqWqFaYmtl2juCXT06rbGF8ClukbJo0TiNP75rkE4X9soTdZL77lVIQkfFkTdDIaGQFiz1fbaLBde+PA7KbbiibsGYc7Gw5oJPsVmwvb5I6Vtwt2l5Vg26Tb8asMhqgFevK9MEego9zTo9em9RWNhMxuwcc5w9HaYYTUZYDXx+ORffgZPYxCVHj/WHTgNd71fin9i6pZamDZEz6LHGBKtXTVE+95lr1/jypfn+utVwHRYjIqV3YbZw3SrdqY6zPjl+shA6fYEsGRiNn736ChcqmtElTcgDeZlFzzYNn8kZr7+le6+7pZHRlAH9wG97dK1SXvkZcBpK7I0hwWVHr9CjnlD/jBsnjdCEgtaOH4Qcvom4rf33SbdA7lmwZbOqwHR1Wmrvh4IhTWT7ePbjyo0VzJTbHB7miXhS/JdSEkwwawjYZ2WaEGfJIuU2cNxHJ5771tJyGyKKwtFTX2KXJPEUKirhW6aOxxhQaT2abK1Jw90lHsaonkZ6hpD+N3hH/HwmAH4rz+d0hjBJP4p1W6mjgF6cUsMRnelRxgS0VYNgH7KWqzvNfi1ynaF4wZiULoDJfkuHD1bpdnGKJnlQr8kK7X+haacdL4LJp6TrnGsvAa/2vAVdiwYhYfWH1LcY8TQaPaA0PZ1z15uoA7uCRaDxoUuN4JomR8Lxw+SjAhy/ke3RAZ8Evj24qeRlEGLkY/qGma0HfGskGmGBgDN97bNH6k7SQOQJnO72Yg/Lx6nUMl0hgQpm0LeDvXkeq66QaGGqjfBNwbCmgm7/IpPaoe6TzsTzFDP4fKtEb1sJuKJI2JsNFG2xbtOYE/RGADaMYAcQ4tbYjC6Kz2ijLiezG+NLxC1xkW0UsdV3gDOXPYqxHqe+kU2iveV4a41f0bxvjLc9w+ZcFgibtL3F/4jts8fieyMRJhMBo2wFU3P/9Y+SeD5ZvEdAomVkJOZYlPUDaANyC99egqvzRyqEeXpbbdo2iMX21l34LSmrsH1qQn6gW8CULC1VNonJ0aJur0sv77tiVWeW692Ro1P+70LTToicjJTbEixm7GrcDR+9+hIBEICppUcxM9WHcD0koM45a5HlbcR5+t86G03452C0Yr6FHrFuwhkgldfk0hoy0kwG/DSp6c0AlavzRyKxe8cx+w3Dyv+JmL1abknjvz96Bk2wZCgGQPk7SXv4WrrzDAYXYEeYUjol7zWrnLkf/x63xMEAYFQGM4EEzbPG4EJOelUedzCraX4m9uLuZu+RlgQwXGIujqhKWcSQSD5gJdiN2Ht9CGKz0ryXdhd2lyehDYgu+v9qPeHsGxSDj5fPE53cCdtyc6I5O6/PCMP/XsnYGfBaPy/JXdi09wRcHv8usYBeW5kECYpqOpBm+XXtz2x5Nb1DA1fQPu9lR98r+l7r84cCkGIpCkHQhFp+LSm1FFyrhPldbjjhQOYsu4gzlZ54Q8JutsranVImgLlyim5kjianIZAGO56v5TOvWPBKBRPHox6fwjHyms0Xi+5sf7yjDxk90nEnqIxKkMnYrjX+IKYkJOOXnZz1H5OM2RWTsnFugOnmdeN0WPoEVsbeoJRYVFEmsOiyNVfd+C09Mcfb5T4ulkupNjpKxcymRY1BTVaTcYWxQUQQSB5G//9vUg19R0LRkntTLGZsOjubJRd8KCi2oejZ6uwbf5IuGWBaQ+PGYAXPjwJd70/7vgEdQ2RDbOHwWExwGbiNYGfxDggHghizNC0O9j+cfsQq8S8nqHB8xw2zhmOBLNB+jtw1/vRN9kaKcwWDON0ZT22HvwRk/Ouo+qSkMk7oUlxk2wDFE8ejESrKe4CdCk2E3YsGCVtQaz+6CQArUDXDakJ0nZgwZZSqS0vfHhSum+b2aAIQE61m6P2e2LY/P5oOZ64axBWffS95rryfk4MmY1zhqPWF1TELTGvG+NqaGnA6tkV97ZTS2LTIwwJvcqddotBkwL2yow88ByHc9UNMBl5KXiQ/P6Ze3Mw43Vlmmbh1lLdtLYaX1A6jiaZHQ8cz6F4X5nm3CYjj/REq/QZGZAFQcBlb0AKvCQrya0Hf4S73o/N80ZAhIhz1Q3S4CoIIq40/P/2zj0+qvLO/+9nJjPJ5AIJIQgILZQfopSiEASU/lpcWqtdW6qgbhUUvACtrO5uve2v9be21K2KrrvWKtFW8Ua9oK5Uq2Kp1n0hKCCKiCK1qKBgAiSQy2Suz/4xcw7nzJwzmYSQSWa+79eLF8yZk5xnwvM8+T7f2ydMOJbogxHweUGlNwC6/KGNPP2jUxlUUUxMa35/+VQisTg+r4chSePA+Hnf8fJ2cxN27N0hdDtuc93w/rgZxwdaw7bqDCOhsF9xoqOjV8GomjLOqR1mtneH9ARj65w33u9o3jtVp/iLvPzE8hyAB1/fyfL5k9nfEjJb2o+uKT+sIJosvzZ+iT90yWRHQzjVC5eaM/J/BpZx0akjOd+S5OxkBFcFfGZzr2tXbuHaM8aY61S8bkIhURCGhFtN/r6WkC0rvaa8mLZwjHNTZIBXLZ5mdr50O9G1hKKuHSTB3jK7sxR5VNrpf+ns8RSl/DI+rMYZSmsy9eNH3+LxBVO53n98os3w3YfbDP/+8imEonEamkO2ZzxyqXOyXXs0zgWWkjvj81lVQY+m9obgTkf9J5wMDSfj+JqVW3j2ilPZ0dBiu9et8qcy4DPnpeERAPdW8R3NBadxzp82kmuefMdWamr1rMXj2pbErNHmPDfGmdpIyqmL5VUzjiMYObzOrS271153mjnmxmCEO9d8aHoL41pz27knMrQyQMAnc10oHArCkID0U088rmlLiQsvmj7KtbZ8SP/DyYxOngejvvzmc77GsAGlxOOam1943zwd3XHeiVSXF3fphBIMx7j1xe228MatL27nrgsmOHbkdDN2AMc2w4nUUoXP67F1BjQSyVI/q1fh2sTKINc9EAqZTD97J0PDPYconjZX3Cp/hlYmqjxC0UTegnHd8GxUBXyd6reQOk6AxSs2m0aEMZ5Mc+6zxraM+SKQnjMyq3Y4Cx/ZZIr71ZQXm43U2sIxm1BaOBozRcaszdZGVpfK3BcKioIxJFJPQxqd9osyU225tSOeU1MoIy4aiWnm/PYNcwO69OtfoS0c48vVZQws71jy2Ql/kZeGlpB5KoLMVQ+Z4uSpvzQmDK/kUDBq6zdhfJ4X3t3jmGfx8z+8xy9mfpWWUJRbX5R4cF8j9ReukTibOl/ao+lKoXeu2ZGmh3H3hRNZ8tx7ZkOphy+djNej8CpFwO+lMuB3TfLMlKtjHWdDc8g0UKxjzDTnOsoXgXSj25ogfNcFEwiGY2l5QJWBhKch4Pfy2IKp1FQU8+n+Nm5+4QMaWkKJsu2iI9PvEYS+REFUbTiVvLWF0jOu3coqrV0eN+9q4tYXE9oVr107nRWXTeHB13eav0xHDiwz71v48CbOv3c985dvIBKLd3kzSc1s7yj+mun+1HK7RdNHpfWbuO6pLfzk9OM4e+KxXPjbN5i9bB1LntvGFaeN5tnNn7F6Wz0/evQt2iNxrv7OGE4fO0jiwX2Y6jI/dY6VEkHHyp8hlSVmmfITC0/hrj/vMHtBrN5Wz9zfvUmJz8uxVaUMSKrXdlRNks0YO7MGsv0at/LTzbuaaGmPOnooE1LoiYqVq598hxm3/4Ubnt3K1d8ZQ015MQsf3sQ7uw46lpQLQj5SEB4Jp9PQzn2tttKxyoAPj1KOTXSsXR4hETOdv3wDa687jSGVgbS4rNMpSAO7G9vwez0MKPW7iiQ50VndhUz3p8ae3VoZD+kf4OJkm2LjmtHV8IlNu80kup88+Y508evjGCf/JTPHmVUbt720nZoKv6OMvHEih/SGUnC4z4KVbLwDANFoPC3p1/DkdVZ7JJuvSV0PT23aZX5mn9e9kZrTnmJNOrVWrnRVpE4Q+goFYUgYpyGrcFQkFmf5/JOZ98AGM9v8oUsmU15cxIrLp6S5Zd02QafcCydJ8X9csZmGlhBLZ4/nUEUxIwaUddqYSH1OallbqqRxNuV2SinHz1bsc95EK5P6HUZ2/u7GIFprMSL6OIlkWI9ZKTGsKsDV3xnDkMpEO2ujMmeQJTwXjydaXK9cdAr7W8Nmbo2TgdBRNQlAJBLj48Y29qUk/d43dxJjBld0Ke/G+jVuyZ5O5adP/+hUwrF41iFCsCedplauSD8JIZ8pCEMi4Pfy5MJTKPF5bNoQdXNrzYqMgN+bqGa4//W001dVwOfoqXBzqxYXeVgycxzDBwTYdSBo5hEAZl19qb8oKwVGJ45UKCh1c3Xa4EsdBLisvSGMPArJjcgPPB4PD76+05bQ+9r2L6gs9dnmfd2cWoZUJspCUys6bpk13ib7bWD8Ah9Q6nOt4InHNV+0hNh9IJguO//wkZ/oO1ozqd97UL8S17WR2kLewKhQcapckTUi5DNK674fu5s0aZLeuHGj43vGBrL3YLttg4LEAgbCchoAAB6QSURBVLeKB52XLPu0vm+IBd3x8nZm1Q6nuszPoIpihvYPOHoUGppDnH33WnY3Bh01MSDRSGpw/xKKktLInU3Gsj7DOtbUzTb1BFYV8HEoFCEYjhFLKnYOLEvcn432Qt3cWioDPj78ooU71+ygoSXUF5QOcz6wTPOzt+D0i3bFZVNsZaGQmGdLZo5jVE2Z43uPL5jKMRUl5trI1uhtaA7R3B6hoTnkuGbWXncax1aV2sbbGWGybNeM089lX2uI9kgcr4ISvwevUoQicRpawrYk5bo5tQyuLOZgW9TWe6aDNZKX81PUP3ueo9SQKqv5mfceif2tCSPg2jOOd3RFft4UZPaydaxcdIpzz4TI4RI4IxacaQOyujzdhIHawjEiMc03l76attFks0Fmk7jmtIEvm1NLLB7nihWb0zY5p8/iFF+GRLz7rgsmSEZ6HtGZstAvV5cSjjmrbu452M6h9qg5n7Ot1jAaShkJz5lyKbI1TqxrCTouWXYjVXX07gsnctefd9DQHGbJzHGMHFhGaXHCKE80qtKdyuUQhL5O3ldtxONxLj51JLsOpGegD6s63BvCTVgqNdESMm9A1izwZa9+xH+ef5ItazwhFFRCayjC4wumcsNZY7nj5e1mJngmETGnZ1jHat1sjQ28pryYurm13H7uiTQ0hyjyeExdAuuznXDS/nC6JuQHqf+3bvNsR30Luw60ua6nyx/ayN5D7cTjOutqDX+Rl5UbP+XYqpJ0Ma259jBiqnFSU17M3oPt7G5qo6E5RDyu09bS+feu59ozxjBheKVtvB2FHJyetb8lzLVnHM+i6aN44d097NzXSnskZq5hWSNCoZH3HomYhuue2kJNeXHGzpOGsFRqT/2AS66A2wZkTSrbvKuJe1/7iMcWTDU3GJ9H0RgM27wCt8xKdIDM9vRWFfCx4rIp1Fv6O6THpeMsnT2e8uIiW17IPXMS4l6rt9Xbni0IqTglSFrXjFs/FcPTdzAYobrcn9X6qS7z84OJw3lq4y4umDqC318+lbi2V20YWI0TQ3U3TQuj3J+2loz8pPnLNzjmOTl5A1Ofdf2Zx9sSUu+5cCK/Tpa/djZXSRDyhbzPkfissY1pt7wCYKvaOLYqwC/+8J6tdO30sYP4t+99FSBjrkCmzeJwTDVmq/ww7v28KeiYi2HkahhjtWKNDzu5devm1jJm0GFXcjgaIxrX7DnYbtNFMJ5llKhZn93VxM8+QM539L6QI+GGMZ+D4Rgf7G02KzMgsZ5+fcEE9h5sT8x3j8KjFG3hGCU+D9es3MKqxdNoaovwyf42Sv3eZHO2UkZUl6Wtn2zzHg60hnhn10FK/V4GlPlZ+tIHtnU8rCrAisun8I1bX0372teuPQ2vwjHZ02mdV5f7OSfZZvuRSydz/dPvpq2nm8/5GnN+96b5upOJoXk5PyVHoufJZY5E3oc2rO5Zo0nUT558h1hc849/N9rmQp0/bSTBSIwh/QM2V74RO7bLDTsbEdu/aOZnz7zL+3ua2XOwnZb2qC00obVzbNnIZM82ZGE9aS18eBOHQhGbK/fC375BTUWx47OMEk7rswXBCY9HMaiihIDfy5LnttlaVDe0hPAXeRhQ5sfrUVyzcgvn37ueG57dikcpasqLiUTjhKJxbnh2q/leKOrsAcsmJGA0gvr9m5/QFIxwMBjh+jNP4LzaYeY9uxuDeJNlzVaGVSU0MJy+v5s3sMijzKZWQyoDjuvJaoRLqadQiOS9IVFd5qdubnrXvpue30Z5SRFLZo4z8wVufXE78x7YkJYzkG3M00jsvPjUkSx5bhuzl63jgt++wfb6w3kOmYyFbDrxuesixNI2wk/3O8exrTXuUpomZMPAsmLHuTmwrJjykqK0DpA/efIdrpwxmph2VpB1y8vpCKc1dtH9b3Lh1C+b+Q/DqhLy4Z3phJlpXRkHiZIij8va9dieLetJKDTyPkfC41EMLPPb6uMNXYyf/v1Y5i/fkPY1bieKaDROfUvI1pzHWgIajsaYVTvcjNfCYY+B4e7M1Jino0580eRJ7vkrv47f66ElFKW+OcRTm3YRd/B03LlmB/dcONGWI7FsTi13rvkQyK7NsCBA5i6REQdNjt2NQUYOLHP1wGV7ak9dc3GtHdeY0XXVkKqvDPipDPizrp5w67xZ4vew91A7JKXtHrxkMp/ubzPLn2+ZNR6tNVd/Z4xjDw1BKARyYkgopZYC3wPCwEfAfK11U/K9fwUuBWLAlVrrl470eR6PhyXPbUvbJHxeT8ZEMGvM1uf1cKg9wrwHNth+KR9/TIVpTBibVaaN021DBmydKof0D9g2vWg0zgdfNHPnmg+5+NSRtuSyZXNqKXbYCBtaQrSEorYStQEBv62lt5SmCdni1lnS7ZdwabEXhXPn1I5O7fG4Zl9LiNZwlI/3Hf7F/ehlU1zX2AmDE+vKOqezyVWIxzUazSOXTmHnvlbzWQ9dMpk9TSHHNXfPhRNpCUV5YO1OLjplBNc//a60ihcKllyFNl4GxmmtxwMfAv8KoJQaC/wD8FXgDOBupdQR+wndQgaDyp3dtdVl/rTysXPueZ2G5hA15YmNaXdjkEWPbKLeokhoNKvqKM8hNVQCdFj2Wd8SYtEjmxxPY4se2UQ0FrcJkA2rCvCbCyYyorqUccf250sDShmUbBQkpWlCd+K2vgaWFWcVrkvFWHvn3PM6p91mF8S66flt1LissYC/qNNz2nzW3a8z/bZXueHZrfzyB+NYtXgaAZ/Xdc0ZonWXfv0r3L76QzPXSNaTUIjkxCOhtV5tebkemJ3890zgMa11CNiplPorMBlYdyTPy+SWdbte39yeFtt9YO1Obp09ngOtYZqCEZa9+hHRWNz2nKH9A2nttOvm1OL1YJaAppJN2WckFjcTJZ1OY1FNWovj37yyg5vOHi9iQcJRpaOQXGfFttwEsR6YdzLXrtxCebE3Td6+o5CCW0WIY/LyI4lQZLiDNTd8QIBrntziqi8iCIVCb8iRuAR4PPnvY0kYFga7k9fSUEotABYAfOlLX+rwIZlErFKvx+OatpA9+WrC8EouPnUk85cfDm0snT2eEp998ygq8nDC4H48/eNTaQvF2LmvlZ/999aM7aRTE72MMtW2cJSG5sSJzwjDuHXLLPF5+Odvj8koiiT0HJ2dn32dTIJanRXbckt8PBiMcOP3x7L3UMhuqM+tZXRNuatxkqkTZqaGWcaai7iId+06EDSNiL621gptfgpHl6MW2lBK/UkptdXhz0zLPT8FosCjxiWHb+VYm6i1vldrPUlrPammpqZbx76vNcTOfa029+mi6aPS3JvXrNzi+PUej0KhmPO7N7hzzQ4WTR/F7eeeyN6D7TQF07PVrZUcRoOdJc9t45tLX+Xsu9fy/p5DFBcpls8/mac27UoLYRhu5GzLVIWjz9Gcn/mOW2XT/tYwB1ojphEBh5OZGy2VSKlk6oSpXMpE/UVeBpUXs3z+yVSUFKV127x3bi1jh1Tw2rWn8fSPT+1za03mp9CdHDWPhNb6W5neV0pdDJwFzNCHGxnsBoZbbhsGfH50RuhOeyTGnWt22DpduiV4RWLONfHhaIya8uK0rnt1c2ptDarA3kHQyWBZ+Mgmlswcx+B+JfzyB18jFo/z+IKpQHpjHQljCH2dTB01rz/TWTMnUxVIpk6Yp48dxLI5tTbxLWsVVb8SH/Me2EBNebEZNmwLx2gNxfjunX+2GfKCUKjkqmrjDOA64Jta6zbLW6uAFUqp/wCGAqOBN3t6fF6laGgJcdtL283No7I0u1a/Bv4iL1fOGG1uWEa4IhiJsfdQuy272xpjbgtHHTfKUr/XlFMeUlnq9EhByAuM9fDEwlP4vCnI/tawWbKdjahXKtaqklRD3eiIaVUB9qqEF6O6zG/mJhmeDwPDkDfymR5fMFWqoISCJVdVG3cBFcDLSqm3lVLLALTW7wFPANuAF4ErtNY93iYu4PeydPZ4GlpCZifM1lAkrbFVprhodZmfkQPLTCPCCFfMXraO8+rW2aoyUstM3ZpISdc8oVDweBSD+5VQVlxkdtQcVhWgqszH7eee2KkqEGvliFPi5Opt9XgVHGqPcl7dOqb86s9m5ZTPpQmVtambYWi4iewJQr6T91obXSEe13y8v5VP9rdRWeqjosRHwOehxO8hFodINN7h6SMe1+w91M7nTUH6B3yOegCrFk8jmkzs3HuoHa01/QI+0LDQ4mo13LoNLaHO9vEX8lTLoFCwGtmxuOaXz2+joTnMlTNGp8l3p95vXaPW6+ffu95R6+bGVVuZVTvcrHp6atMubp41ni8Ohcwwy+ljB3H9mSdwMBihvjnEslc/oqElZOrXiNZGAtHa6HlyqbXRG6o2eoRsBYEgcRoaUV1GZamPPU3tzHvgTVv8dHRNOY3BCHsOBh2/l1OW+C2zxtPQHDa1CmrKi9nT1G4zGJbOHs/PntnK0nPH8+Alk/F6FNGY5t6/JDarujm1fSozXBC6kxK/l5tnjScYdl7DmaozjMqRaDTuqJzrVaQ1nbpl1ngi0bgZdozG4zQ0h7jo/jdtazbg97Ji/afUza2lMuAjHI25lnoLQj5SEB6JjjYYNxqaQ5x991rb6eX0sYO46lvH2crPUr+X09elqm4+MO9kbnh2a9o9v/7hSXiUhytWWKS/L5xIeyTOqEFlDJCkrs6S891cPBJdo7Pr1m3dGR6CTMq59S0hV1VeQ5Trs8Y2R2/Gr394Em3heJqUeZaVHHk5P8Uj0fOI+udRxq3hU1MwTH1zO58eaOWzxjYOtIZs8U2nGvNZtcPTys9SRYjcatMNb8KwqoCZP5F6z8DyEtOIMK796NG3OKZfMZUB8UYIhYPbuk0V/IpG43zeFKQtHOWGs8aaAlrG1xh5RW7KuY3BSEZVXoOYyz2Vpf60SqsjESYThL5G3oc24nFNOBrj9nNPNLtRbt7VlAgtHGy3eRaWzh7PMf1KGFFdhsejHDUEOtLSAHftgaGVAdZedxr+Ii8a7XhPyEUAyZuUNBeEQiGbRm3xuOaDL5pt5ZtGTlFqx8lMzafc1qy1GqTE53yPQnW6JFUQ8om89kgYrszz713P+feuZ8lz27j6O2OYMLySK2eMTvMsXLNyC5/sbzNPEk46AdloabjpCwzuV2JqXAwsK6ZuTrq8+d6DwQ6/vyAUAh01atv+RTMH2sKmEQGH22kvmj4qraLDrdGVkW/RkSaIk5T6LbPGs7uxTdasUNDkdY6EW8x0+fzJ+LyKD/Y2mx4Kg8cXTGVYVYBjqxK9GlKTNKsCPnY0tHQYt80mufNAa4h3dh2k1O81vSU1FX6umnGcLQmzE/FWIZ2c/9AkR6JrWHMaDInw1LX82IKpfP2WV4DDHovKgI9jqwKU+r225m8d5Vxks2at9wD8/A/v0dAcTms8JzkSkiPR00jVxlHCzZXZ1BZm9rJ1jm7QtmRGuIGTTkA2IkRu+gK2nhFFHr5cXWrLAv/3c77G6JryTokcCUI+kk2jtng8ESJ06iJ730WTbHlFHYmLZaMJYr0nHtemvs1tL21nycxxiZJUv5cir3Kt6hKEfCOvDQm3uKcRujDcoMZpx8iR6KjEsrMiRAbRaJzt9c1pFR+rFk9LK2mTXhGCcHitNTTjIlbnZdmcWhqaQ44Jj6k9HbpzbTkZJpUlRXzY0JKxqqsvIh4GIRN5nSPhFPe8ZdZ4lr36kXnP7sYgxw+u4PEFUxkzuMJMtITEiaOhOcRnjW3UN7dzoDXx74bmENFo3HyvoTlR7WG937hmEI9rPj8YdKz4iMUxcyf68mYjCEcLtxyGgeXFHH9MBV+pca6C6kzCo3X9ft4U5IuDwYxr2ynsuedQe4dVXYKQb+S1RyL1xACJmKY1J2JYVYBSf5GjlHhqPPU3F0yguT1KRYmPtnCUm57fxupt9YlNbe4kfEWKeQ9scDyJ7G8NU98ckuxuQegCmcISHo+i1F/kWnWRbe6DUxO517Z/wZxTRhKLa3bua+XONTtoaAlx30WTKC7y2MKSRqMrWeNCoZHXhgQkNqDqMj/bv2jmjpe3c/GpI9m2pzlN6S8VJ+nhtnCM659+17Fb5eUPb2TJzHGurtVwNMb+1nCnBYcEQUiQKSzhpBh630WTqAr4smpq5dRj4sHXd3LFaaP54X3r00pLL38ofb0b3TJljQu5oLPhp+5Mzszr0IaBsUms3lZvKnquXHQKTyw8xbHaoqE5lNbcZtH0UVyz0h6DNcrMjNelfvtmYT2J+Iu8PLVpF7fMGm9zz9bNlbbXgnCkWD0Wa687jWd+PI0xx1TQGIxk1dQqVWq8bm4t/3rmCWnN4Yw177TejZbbssaFQiPvPRJg3yQ272oy21Svve60rDQybntpu6Nq4O7GIJUBH4BZ8WHFehKpLvPzz98ewx0vJwyZ6jI/gyqKGdo/IHkRgtANOHksMjWhsmIkZlurP24/90TXNe+03p/atIurZhzHf635UNa4UFAUhCGRTdc6cHZvGlUdbeGY4/doCkYYVhXgjvNOpKrMb96TehIxTkw3nT1eyjoFoYfIdu0boZG9B9vN6g9jbad+bVs4xj0XTiSutW29z582ksH9i2WNCwVHQYQ2sulaB+6nlxMGV/DVof2om2vvRPmbCyYytH8JN5w1ln//4wcUF3nMsMmKy6YwZpA9bGKcmKRCQxB6hmzXvmHojxp0uPpj2asfpYcp5tQydkgFADeu2sYNZ43l8QVTueGssdz64nbaI3FZ40LBURAeCY9HMbqmnCcWnkI0FqfI62FQefoidzu9AJxzz+vUlBezZOY4vlJTxt8aWrlx1eEKkGFVATxKMW5oPzmJCEIvoaMmVGBvElfk8Zh7wOZdTWajqVGDyinxeSjyKILhGPtawjS0hMwwKUhSpVC4FIRHIh7X7Gho4by6dXxj6aucV7eOHQ0ttj4PAFUBH8tS9C/umVPLo+s/NjeW+cs3sGL9xwysKKahJWTet2xOLcdUlMhJRBB6GZk8gUZe1Nl3r2XaLa9w46qttj2goSXE4P4lDO1Xwv6WMN+/ay2LV2xmQJmPpbMlqVIQoEA8Em5yxKsWTyMWxzypeD3wh7d388C8k/F6FLG4ZuXGT5k4ohr+52Pz+9X9z8dc/o1RaR6OoqKCsMsEIW9I3RtWb6sH4ImFp6C1Nj0YTcEwew+2myrCr21v4KyThvLoZVPwKMW+lnYGihdSKFAKwpBwyn2oKS9mT1O7TRyrbm4tZ44fyvzlG2xVG5WlPtvXDqsKoFEMrSzpyY8hCEI347Q3rN5Wz799T9uE+/Y0tXPDs1vNfeHuCyfyqz++bzakWzp7PD45SAgFSkHMfCf54CtnjDaNCEh4KRY+vInG1kha1Ub/ZLkXuCdrCYLQ98gkLW6wvzWctlf8+NG3mFU73Hx9zcotRON9X0lZELpCQXgknLrejRzo3JvfqamUAlHjFIQ8xK0jpvWg4FbNZfSQMV5HovEeG7cg9CYKwpCwVm1EYnF83kT2tVuNuBXjdCJqnIKQf2RT1eFWzdUUjNheS8WGUKgURGjDWrXxT4+9zfa9zURi8bS+EPddNIkvV5dKGEMQCoiO+rs49aK4+8KJPLVpl/m6bk4tXg9plWCCUAjk1COhlLoaWArUaK33KaUU8F/Ad4E2YJ7W+q0jfY6RmW1tf7u7McjpYwex4rIpeD3KPImAhDEEQTiM1WsRjMT4qL6FR9Z9wkWnjOC6M0/g0/1t/Oy/t5qqoKn6PYKQ7+TMkFBKDQe+DXxquXwmMDr5ZwpwT/LvI8KIcd5w1ljTiIBEdva2Pc2mQqeBhDEEQbBieC0+a2xj/vINAMwYewwXJ2XEDayKv4JQKOQytHEHcC1g9QXOBB7SCdYDlUqpIUf6ICPG6Sa8lSrgIwiC4IS1ykP2E0FIkBNDQin1feAzrfU7KW8dC+yyvN6dvHZEGDFOQ3jLiiRJCYKQLdZ8CUPUy4rsJ0IhctQMCaXUn5RSWx3+zAR+Cvx/py9zuOaYvaSUWqCU2qiU2tjQ0JBxLEaM88Th/ambk55gKcmUQnfTmfkp9B2s+RInDevvmLDdF/YTmZ9Cd6K07tksY6XU14A1JJIpAYYBnwOTgZ8Dr2qtf5+8dzswXWu9J9P3nDRpkt64cWNWz7cK9EgyZUGQ8//czsxPoW/RDftJn5ifI65/vodGI/QUH9/899ncltX87PFkS631u8Ag47VS6mNgUrJqYxWwWCn1GIkky4MdGRGdxUiaEgRBOFJkPxGE3teQ6o8kSj//SsJjMT+3wxEEQRAEIRM5NyS01iMs/9bAFbkbjSAIgiAInaEgOlsKgiAIgnB0EENCEARBEIQuI4aEIAiCIAhdpsfLP48GSqkG4JMObhsI7OuB4fQG5LMeZp/W+oyeGowTMj9tFMrnhOw+a1+Zn91Nb50HvXFcuRxTVvMzLwyJbFBKbdRaT8r1OHoC+ax9j3z5HB1RKJ8TCuuzdpbe+rPpjePqjWNKRUIbgiAIgiB0GTEkBEEQBEHoMoVkSNyb6wH0IPJZ+x758jk6olA+JxTWZ+0svfVn0xvH1RvHZKNgciQEQRAEQeh+CskjIQiCIAhCN1NQhoRS6kal1GdKqbeTf76b6zF1J0qpM5RS25VSf1VKXZ/r8RxNlFIfK6XeTf4/9nlpzXyfmyDzUwCl1FKl1AdKqS1KqWeUUpXJ6yOUUkHL/F/Ww+PK+dxUSg1XSr2ilHpfKfWeUuqq5PVevzcUVGhDKXUj0KK1vi3XY+lulFJe4EPg28BuYAPwQ631tpwO7ChhVY3N9Vi6g3yemyDzU0iglDod+LPWOqqUugVAa32dUmoE8JzWelwOxtQr5qZSaggwRGv9llKqAtgE/AA4j16+NxSURyLPmQz8VWv9N611GHgMmJnjMQmCgcxPAa31aq11NPlyPTAsl+NJ0ivmptZ6j9b6reS/m4H3gWN7ehxdoRANicVJt9r9SqmqXA+mGzkW2GV5vZs+Mgm7iAZWK6U2KaUW5How3US+zk2Q+SmkcwnwguX1SKXUZqXUX5RS/7cHx9Hr5mbSQzMBeCN5qVfvDXlnSCil/qSU2urwZyZwDzAKOAnYA9ye08F2L8rhWj7HraZprScCZwJXKKW+kesBdUQBz02Q+dnr52d30cE8N+75KRAFHk1e2gN8SWs9AfgXYIVSql9PDdnhWs7mplKqHHgK+Cet9SH6wN5QlOsBdDda629lc59S6j7guaM8nJ5kNzDc8noY8HmOxnLU0Vp/nvy7Xin1DAn35Gu5HVVmCnhugszPXj8/u4uO5rlS6mLgLGCGTibpaa1DQCj5701KqY+A44CeSFTtNXNTKeUjYUQ8qrV+GkBr/YXl/V65N+SdRyITyWQWg7OBrbkay1FgAzBaKTVSKeUH/gFYleMxHRWUUmXJZCSUUmXA6fTx/8s8n5sg8zPf/j+7hFLqDOA64Pta6zbL9Zpk0iNKqa8Ao4G/9dCwesXcVEop4HfA+1rr/7Bc7/V7Q955JDrgVqXUSSTcVh8DC3M7nO4jmQW9GHgJ8AL3a63fy/GwjhbHAM8k1h1FwAqt9Yu5HdIRk7dzE2R+5sH87C7uAoqBl5M/n/Va60XAN4BfKKWiQAxYpLU+0BMD6kVzcxowF3hXKfV28tr/A37Y2/eGgir/FARBEASheymo0IYgCIIgCN2LGBKCIAiCIHQZMSQEQRAEQegyYkgIgiAIgtBlxJAQBEEQBKHLiCGRhyilXlRKNSmlel3jEqGwUUqdpJRal1Q33KKUOj/XYxIE4ciQ8s88RCk1AygFFmqtz8r1eATBQCl1HKC11juUUkNJKByeoLVuyvHQBEHoIuKR6MMopU5OnupKkt303lNKjdNarwGacz0+obBxmp+AX2u9A8w20vVATU4HKhQ8SqklSqmrLK9vUkpdmcsx9SXEI9HHUUr9EigBAsBurfWvktenA1eLR0LIJW7zM/neZOBB4Kta63iOhigIhtrm01rriUopD7ADmKy13p/TgfURCq1Fdj7yCxK94tsBsaCF3obj/EzqBzwMXCxGhJBrtNYfK6X2K6UmkGhxvlmMiOwRQ6LvMwAoB3wkTn6tuR2OINhIm59JeejngZ9prdfncnCCYOG3wDxgMHB/bofSt5DQRh9HKbUKeAwYCQzRWi9OXp+OhDaEHJM6P4F/AV4A/qC1/s9cjk0QrCSVP98lYfSO1lrHcjykPoN4JPowSqmLgKjWekVSgvd1pdTfAT8HjgfKlVK7gUu11i/lcqxC4eE0P0lINH8DqFZKzUveOk9r/bbLtxGEHkFrHVZKvQI0iRHROcQjIQiCIBQ8ySTLt4BzjcoiITuk/FMQBEEoaJRSY4G/AmvEiOg84pEQBEEQBKHLiEdCEARBEIQuI4aEIAiCIAhdRgwJQRAEQRC6jBgSgiAIgiB0GTEkBEEQBEHoMmJICIIgCILQZf4XDh3UGPk2hAEAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.pairplot(\n", " data=pd.DataFrame(np.hstack([x_ND, y_N[:,np.newaxis]]), columns=['x1', 'x2', 'y']));" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "# Define the optimization problem as an AUTOGRAD-able function wrt the weights w_D\n", "def calc_neg_likelihood_linreg(w_D):\n", " return 0.5 / ag_np.square(sigma) * ag_np.sum(ag_np.square(ag_np.dot(x_ND, w_D) - y_N))" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "4649826.583041586" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Test the function at an easy initial point\n", "init_w_D = np.zeros(2)\n", "calc_neg_likelihood_linreg(init_w_D)" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([-1053476.33939486, 1159525.45599207])" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Test the gradient at that easy point \n", "calc_grad_wrt_w = autograd.grad(calc_neg_likelihood_linreg)\n", "calc_grad_wrt_w(init_w_D)" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [], "source": [ "# Because the gradient's magnitude is very large, use very small step size\n", "opt_w_D, history = run_many_iters_of_gradient_descent(\n", " calc_neg_likelihood_linreg, autograd.grad(calc_neg_likelihood_linreg), init_w_D,\n", " n_iters=300, step_size=0.000001,\n", " )" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3sAAADTCAYAAAA4agr9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xuc3WV94PHPd86ZzOQySSAZIdcmlLQVUAEjsqu1ihdAbWNbXbHdil1301pt6XZbhXbXW2u1625RrNXShYrWFmnVwnZxERFqu8olXAQiRSIXSQgkISEJIZe5fPeP85tkEmeSzGTm/Ob8zuf9ep3XOef5/c55vs/rF+bhe57n9zyRmUiSJEmSqqWj7AAkSZIkSRPPZE+SJEmSKshkT5IkSZIqyGRPkiRJkirIZE+SJEmSKshkT5IkSZIqyGRPkiRJkirIZE+SJEmSKshkT5IkSZIqqF52AGM1f/78XLZsWdlhSJIm2Z133rklM3vLjqNV2D9KUvs42j6y5ZK9ZcuWsWbNmrLDkCRNsoh4rOwYWon9oyS1j6PtI53GKUmSJEkVZLInSZIkSRVksidJkiRJFWSyJ0mSJEkV1HILtEw1e/sH2LRjL1ue3cvmnXvZ9tw+9vQNsqdvoPHcP0D/wCCZMJgwmElmkjReDyZkQmYymFl2c8alRcOWNImWzZ/Ju191ctlhaAx27xvg/dfezxtftJCf+QkXQZWkKjDZG6Pd+wa4+cFN3PTAJtY+sZ11m56lf3D0bKezFtQ7OugI6IggAiJixPdB430ratGwJU2SHXv6yg5BYzSQyd/duZ4VJ8wy2ZOkipgSyV5E1IA1wIbMfGPZ8Yxk4/bdfOqb6/jqXRvY3TfA8TOn8aLFczjnp57Hj82bQW9PF8/r6WbujE6md9boLh61DtMgSdLUVy/6q8P9gClJai1TItkDLgIeAGaXHchIvnznej5w3Vr29g/wC2csZtUZC3np8nkmcpKkyhhK9gYGTPYkqSpKT/YiYjHwBuAjwO+UHM5BMpNLv/EQl930EC9dfjwff/OLWDpvRtlhSZI04YZ+wOxzZE+SKqP0ZA/4BPBeoGe0EyJiNbAaYOnSpU0KCz7/nce47KaHeMuLF/PRX3gB9ZqLl0qSqikiqHcEA4ODZYciSZogpWYvEfFGYFNm3nm48zLz8sxcmZkre3ubc9P43T/cxof/8Xu85vkn8LFffKGJniSp8mod4T17klQhZWcwLwN+LiIeBa4GzomIvy43JOgbGOSSr9xH76wuLn3ri7w3T5LUFuodQb/37ElSZZSa7GXmJZm5ODOXARcA38zMf19mTADXrHmcf31yJx/8uVPp6e4sOxxJkpqiXutgwJE9SaqMskf2ppz+gUE++08/4PQlczn31BPKDkeSpKapdwT93rMnSZUxZZK9zLxlKuyxd/39T/L41t2865U/TrTqDueSJI1DzWmcklQpUybZmyr+bs3jLDl+Oq99vqN6kqT20lnrcIEWSaoQk71hNu3cw/9bt4VVL1pEh4uySJLaTGNkz2mcklQVJnvD/J97NzKYsOr0hWWHIklS09XdekGSKsVkb5ivr32KnzyhhxUnjLq/uyRJlVWvhatxSlKFmOwVdu8b4M7HtvGKn5hfdiiSJJWi1tFBnwu0SFJlmOwV1jy2lX0Dg7zsZJM9SVJ7qncEA269IEmVYbJX+Jd1W+isBWctP77sUCRJKkW95j17klQlJnuFW3/wNGcsOY4Z0+plhyJJUinq7rMnSZVisgfs6x/kgY07OePH5pYdiiSpJBFRi4i7I+Ifi/fLI+K2iHgoIr4UEdOK8q7i/bri+LJh33FJUf5gRJw7rPy8omxdRFw8rHzMdUymekeHC7RIUoWY7AHff2on+wYGecGiOWWHIkkqz0XAA8Pe/wlwaWauALYB7yzK3wlsy8yTgUuL84iIU4ALgFOB84A/LxLIGvBp4HzgFOBtxbljrmOyNaZxes+eJFWFyR5w/4btACZ7ktSmImIx8AbgfxXvAzgH+PvilKuANxWvVxXvKY6/ujh/FXB1Zu7NzEeAdcBZxWNdZj6cmfuAq4FV46xjUtXcZ0+SKsVkD7hvw3Z6uussPX5G2aFIksrxCeC9wNCw1jzgmczsL96vBxYVrxcBjwMUx7cX5+8vP+Qzo5WPp46DRMTqiFgTEWs2b9489lYfot7R4T17klQhJnvA/U/s4LSFc2jCj6aSpCkmIt4IbMrMO4cXj3BqHuHYRJUfqf4DBZmXZ+bKzFzZ29s7wkfGprH1gsmeJFVF2yd7A4PJv27cwakLZ5cdiiSpHC8Dfi4iHqUxxfIcGiN9cyNiaInmxcATxev1wBKA4vgcYOvw8kM+M1r5lnHUMalqtaDPe/YkqTLaPtl74pnd7O0fZMUJs8oORZJUgsy8JDMXZ+YyGgusfDMzfxm4GXhzcdqFwLXF6+uK9xTHv5mZWZRfUKykuRxYAdwO3AGsKFbenFbUcV3xmbHWMak6HdmTpEpp+03lHtmyC4Dl8032JEkHeR9wdUT8EXA3cEVRfgXwhYhYR2O07QKAzFwbEdcA3wP6gXdn5gBARLwHuAGoAVdm5trx1DHZat6zJ0mVYrK3P9mbWXIkkqSyZeYtwC3F64dprKR56Dl7gLeM8vmPAB8Zofx64PoRysdcx2Sqd7j1giRVSdtP43xkyy5mddWZP2ta2aFIklSqes1pnJJUJW2f7P1g87Msnz/TlTglSW2v3hH0OY1Tkiqj7ZO9R7bscgqnJEk07tlzZE+SqqOtk709fQNseGa3yZ4kSUBnzXv2JKlK2jrZW79tN5mwbP6MskORJKl0tY5wNU5JqpC2Xo1z4/bdACycM73kSCRJEyEiVgI/DSwEdgP3A9/IzEnfkLwKGqtxJpnpveySVAFtPbK3cfseABaY7ElSS4uId0TEXcAlwHTgQWAT8HLgxoi4KiKWlhljK6jXGv9b4G17klQN7T2y90wj2TthTlfJkUiSjtFM4GWZuXukgxFxOrAC+GFTo2oxtY7GaF7fwCC1jlrJ0UiSjlVbJ3tP7tjN/FnT6KrboUlSK8vMT492LCKmZeY9zYynVXXWGsmeK3JKUjWUOo0zIpZExM0R8UBErI2Ii5pZ/8btezhxTnczq5QkTaKIuCUilg17fxZwR2kBtZhaR+N/C/pN9iSpEsoe2esH/ktm3hURPcCdEXFjZn6vGZVvfGYPS+e5EqckVchHgf8bEZcBi4DzgV8tN6TWUS+mcfYPuP2CJFVBqcleZm4ENhavd0bEAzQ65+Yke9t389KTjm9GVZKkJsjMGyLi14EbgS3AGZn5ZMlhtYy60zglqVKmzGqcxbSbM4DbRji2OiLWRMSazZs3T0h9u/b2s2NPv9M4JalCIuK/AZ8CXgF8ELglIt5QalAtZP/InsmeJFXClEj2ImIW8GXgtzNzx6HHM/PyzFyZmSt7e3snpM6hbRfcY0+SKmU+cFZmficz/wI4F/jtkmNqGfvv2XNjdUmqhLLv2SMiOmkkel/MzK80q94ni2TvhNmO7ElSVWTmRYe8fwx4bUnhtJyh1Tj7B71nT5KqoOzVOAO4AnggM/+0mXVveXYvAM+b7R57kiTBgX32nMYpSdVQ9jTOlwG/ApwTEfcUj9c3o+KhZG/ezGnNqE6SpCnvwGqcJnuSVAVlr8b5L0CUUffWXfuodQSzuzvLqF6SpCmnXtyz52qcklQNZY/slWbrrn0cN2MaHR2l5JqSpCaIiN+IiLdGROn3qLeCWnHPXp/37ElSJbRtsvf0rn3Mn+UUTkmquABeDjRtAbBWNjSN05E9SaqGtv2l8+ln93K89+tJUqVl5qfLjqGV1N16QZIqpW1H9rbu2meyJ0kVExE/iIgvRsSvR8QpZcfTaupuvSBJldK2yV5jGqfbLkhSxZwC/AUwD/gfEfFwRHy15JhaRt2tFySpUtoy2dvXP8jOPf2O7ElS9QwAfcXzIPAUsOlwH4iI7oi4PSK+GxFrI+JDRfnyiLgtIh6KiC9FxLSivKt4v644vmzYd11SlD8YEecOKz+vKFsXERcPKx9zHZNp/2qcTuOUpEpoy2Rv6659ACZ7klQ9O4BPAI8AF2bmv8nMXzvCZ/YC52Tmi4DTgfMi4mzgT4BLM3MFsA14Z3H+O4FtmXkycGlxHsW00QuAU4HzgD+PiFpE1IBPA+fTGHl827AppmOqY7Id2FTdaZySVAVtmew9vauxobqrcUpS5bwN+BbwG8DVEfGhiHj14T6QDc8WbzuLRwLnAH9flF8FvKl4vap4T3H81RERRfnVmbk3Mx8B1gFnFY91mflwZu4DrgZWFZ8Zax2TqrPmNE5JqpK2TPYOjOx5z54kVUlmXpuZvwf8GnA98A7gH4/0uWIE7h4aUz5vBH4APJOZ/cUp64FFxetFwONFff3Adhr3CO4vP+Qzo5XPG0cdh8a9OiLWRMSazZs3H6mZR1Rz6wVJqpQ2T/Yc2ZOkKomIL0fED4BPAjOBtwPHHelzmTmQmacDi2mMxD1/pNOGqhnl2ESVH66OgwsyL8/MlZm5sre3d4SPjM3QPXt93rMnSZXQlvvsPf1sI9mbZ7InSVXzMeCuzBwYz4cz85mIuAU4G5gbEfViZG0x8ERx2npgCbA+IurAHGDrsPIhwz8zUvmWcdQxqYa2Xhjwnj1JqoS2HNnbvrsPgNnTO0uORJI0kTLzjrEmehHRGxFzi9fTgdcADwA3A28uTrsQuLZ4fV3xnuL4NzMzi/ILipU0lwMrgNuBO4AVxcqb02gs4nJd8Zmx1jGp3HpBkqqlLUf2duzpo6ervv/eBElSW1sAXFWsmtkBXJOZ/xgR36OxyMsfAXcDVxTnXwF8ISLW0RhtuwAgM9dGxDXA94B+4N1DiWdEvAe4AagBV2bm2uK73jeWOibb/tU4ncYpSZXQnsne7n5H9SRJAGTmvcAZI5Q/TOP+vUPL9wBvGeW7PgJ8ZITy62ksGHPMdUymeq0x4ceRPUmqhracxrlzTx893W2Z50qSNKr90zgHvGdPkqqgLTOeHXv6HNmTpAqJiEcYYbXK4acUxz+RmZc1J6rWU/OePUmqlPZM9nb3s3Du9LLDkCRNkMxcXnYMVdBZTON0nz1Jqoa2nMbZGNlryzxXkqRRDa1b5jROSaqG9kz2dvcxu9tpnJIkDRcR1DvCaZySVBFtl+wNDiY79/Yz2wVaJEn6EfVaOI1Tkiqi7ZK9Z/f1k+mG6pJURRFxWtkxtLp6Rwd97rMnSZXQdsnejt19AE7jlKRq+mxE3B4RvxERc8sOphU1Rva8Z0+SqqANk71+ABdokaQKysyXA78MLAHWRMTfRMRrSw6rpXjPniRVxxEznoj4FIfZuygzf2tCI5pkO/Y4sidJVZaZD0XEfwXWAJcBZ0REAL+fmV8pN7qpr7PWwb5+R/YkqQqOZmRvDXAn0A2cCTxUPE4HBiYvtMmxfxqn9+xJUuVExAsj4lLgAeAc4Gcz8/nF60tLDa5FdNU72OfWC5JUCUcc2cvMqwAi4h3AqzKzr3j/WeDrkxrdJNixp5jG6cieJFXRnwF/SWMUb/dQYWY+UYz26Qi66jX29pnsSVIVjOWevYVAz7D3s4qyYxIR50XEgxGxLiIuPtbvO5IDI3vesydJVZOZr8jMLwxP9IYd+0IZMbWars4O9vS33MQdSdIIxpLsfQy4OyI+FxGfA+4C/vhYKo+IGvBp4HzgFOBtEXHKsXznkQzdszery2RPkqoiIv53RPxsRPzItI2IOCkiPhwR/6GM2FpNV73DkT1Jqoijzngy868i4mvAS4uiizPzyaHjEXFqZq4dY/1nAesy8+HiO64GVgHfG+P3HLUTn7yZ93TdS732hsmqQpLUfP8J+B3gExGxFdhM417z5cA64M8y89oS42sZXfUaz+3rLzsMSdIEGNPwVpHcjdZZfoHGAi5jsQh4fNj79RxIJveLiNXAaoClS5eOsYqDLdv6bc6Nb9IYUJQkVUHRP70XeG9ELAMWALuB72fmcyWG1nK66h1se86RPUmqgoncZy8m6DM/ss1DZl6emSszc2Vvb+84qjng7OcvY25t7zF9hyRpStsD9NLYa292ybG0nK7ODva69YIkVcJEJnvj2YF1PY3OeMhi4ImJCWcUXT3EwF7oN+GTpKqJiP8I3A78AvBm4Fbv1RubrnqNvS7QIkmVUPYqJXcAKyJiObABuAD4pUmtsXtO43nvTqh3TWpVkqSm+z3gjMx8GiAi5gHfBq4sNaoW4gItklQdRz2yFxFfiIj/FBE/Ncop+8ZaeWb2A+8BbqCxAe4141jkZWy6it0j9myf1GokSaVYD+wc9n4nB98briPo7qw5jVOSKmIsI3t/Bbwc+FREnATcA3wrMz8JkJlnjyeAzLweuH48nx2XoWRv787DnydJakUbgNsi4loatxesAm6PiN8ByMw/LTO4VtBV73AapyRVxFi2XvhmRPwT8BLgVcCvA6cCn5yk2CZHV3Gv/t4d5cYhSZoMPygeQ4ZWkO4pIZaW1Ej2BslMIsaz9pokaao46mQvIm4CZgLfAf4ZeElmbpqswCaNI3uSVFmZ+aGyY2h1XZ01MqFvIJlWN9mTpFY2ltU476VxX95pwAuB0yJi+qRENZm6i5G9PY7sSZIgIpZExM0R8UBErI2Ii4ry4yPixoh4qHg+riiPiLgsItZFxL0Rceaw77qwOP+hiLhwWPmLI+K+4jOXRTFkNp46JltXvfG/Bk7llKTWd9TJXmb+58x8BfDzwNM07uF7ZrICmzT7p3E6sidJAqAf+C+Z+XzgbODdEXEKcDFwU2auAG4q3gOcD6woHquBz0AjcQM+ALwUOAv4wFDyVpyzetjnzivKx1RHMxxI9lykRZJa3VhW43xPRHyJxsIsb6KxjPX5kxXYpNk/jdPVOCVJkJkbM/Ou4vVOGqtDL6KxuMtVxWlX0ej7KMo/nw23AnMjYgFwLnBjZm7NzG3AjcB5xbHZmfmdzEzg84d811jqmHRd9RpgsidJVTCW1TinA38K3FlsmdCa6l1Q63JkT5IqKCIuG6F4O7AmM68d4dihn18GnAHcBpyQmRuhkRBGxPOK0xZx8HYO64uyw5WvH6GccdSx8ZB4V9MY+WPp0qVHat5R6eosRvb6nMYpSa1uLNM4P56Zt7V0ojeke7b37ElSNXUDpwMPFY8XAscD74yITxzugxExC/gy8NuZebhOYqRVS3Ic5YcN52g+k5mXZ+bKzFzZ29t7hK88Ok7jlKTqGMvIXnV09TiyJ0nVdDJwztAPkxHxGeDrwGuB+0b7UER00kj0vpiZXymKn4qIBcWI2wJgaAXq9cCSYR9fDDxRlL/ykPJbivLFI5w/njomndM4Jak6xrIaZ3V0zXafPUmqpkU0tgkaMhNYmJkDwN6RPlCsjHkF8MAhm65fBwytqHkhB/bsuw54e7Fi5tnA9mIq5g3A6yLiuGJhltcBNxTHdkbE2UVdbz/ku8ZSx6QbGtnb4zROSWp5juxJkqrkvwP3RMQtNKZCvgL444iYCXxjlM+8DPgV4L6IuKco+33gY8A1EfFO4IfAW4pj1wOvB9YBzwG/CpCZWyPiD4E7ivM+nJlbi9fvAj5H4/73rxUPxlpHM+y/Z8+RPUlqee2Z7HXPga2PlB2FJGmCZeYVEXE9ja0PAvj9zBya/vh7o3zmXxj5HjmAV49wfgLvHuW7rqSxWvWh5Wto7FN7aPnTY61jsu2fxunIniS1vPZM9hzZk6Qqewnw08XrAZp0r1tVuECLJFVHG9+z5z57klQ1EfEx4CLge8XjtyLio+VG1VpcoEWSqqO9R/YyIUabuSNJakGvB07PzEGAiLgKuBu4pNSoWsiBe/acxilJra49R/a6Z0MOwr5dZUciSZp4c4e9nlNaFC1q/zTOPkf2JKnVtefI3vTjGs+7t0HXrHJjkSRNpI8Cd0fEzRxYjdNRvTFwGqckVUd7Jnsz5jWen3sa5i45/LmSpJaRmX9bbLvwEhrJ3vsy88lyo2ot0+pO45SkqjDZkyS1vIg485Ci9cXzwohYmJl3NTumVlXrCDpr4cieJFVAmyd7Ww9/niSpVfzPwxxL4JxmBVIF3fWa9+xJUgW0ebLnyJ4kVUFmvqrsGKqkq7PDaZySVAFtuhrnHIgOkz1JkkbQVa85jVOSKqA9k72OWmNFTpM9SZJ+RFe9w2RPkiqgPZM9aEzlNNmTJOlHTKt3sLfPaZyS1OpM9iRJlRARFxXPv1l2LK2uq9NpnJJUBW2e7LkapyRVyLMR8XvArrIDaXVd9Q72OLInSS2vjZO94x3Zk6SKiIgPAD8J/CHwkxHx/pJDamkzp9XYta+/7DAkSceojZO9YhpnZtmRSJKOUWZ+CNgHvBboy8wPlxxSS+vp7mTnHpM9SWp1pSV7EfHxiPjXiLg3Ir4aEXObGsCMeTDYB3t3NrVaSdKk+VZm/jNwS9mBtLqe7rrJniRVQJkjezcCp2XmC4HvA5c0tXY3VpekSsnMrxfP3yg7llbXGNnrI539IkktrbRkLzO/nplDPxveCixuagBDyd6uLU2tVpKkqa6nu07fQLoipyS1uKlyz95/AL422sGIWB0RayJizebNmyemxp4TG887N07M90mSVBGzu+sATuWUpBY3qcleRHwjIu4f4bFq2Dl/APQDXxztezLz8sxcmZkre3t7Jya42YsazzuemJjvkyRNSRExq+wYWs2s/cleX8mRSJKORX0yvzwzX3O44xFxIfBG4NXZ7BsDZsyD2jTYsaGp1UqSmu57wNKyg2glPV2dgCN7ktTqJjXZO5yIOA94H/AzmflcCQFAzwKncUpSBUTE74x2CDjiyF5EXEnjx8dNmXlaUXY88CVgGfAo8O8yc1tEBPBJ4PXAc8A7MvOu4jMXAv+1+No/ysyrivIXA58DpgPXAxdlZo6njmbocRqnJFVCmffs/RnQA9wYEfdExGebHsHsRU7jlKRq+GPgOBr9yvDHLI6ur/sccN4hZRcDN2XmCuCm4j3A+cCK4rEa+AzsTw4/ALwUOAv4QEQcV3zmM8W5Q587bzx1NEtP99DIntM4JamVlTayl5knl1X3frMXwIam/VAqSZo8dwH/kJl3HnogIv7jkT6cmd+KiGWHFK8CXlm8vorG/n3vK8o/X9x+cGtEzI2IBcW5N2bm1qLeG4HzIuIWYHZmfqco/zzwJhoLk42pjsxsynQUR/YkqRqmymqc5Zi9sDGy5z5CktTqfhV4bJRjK8f5nScMJVfF8/OK8kXA48POW1+UHa58/Qjl46mjKWYXI3s7HNmTpJbW3slez0IY2Au7t5UdiSTpGGTmg5l50MapEXFiceypCa4uRgphHOXjqePgkyZjayKGr8bpyJ4ktbL2TvZmL2w8uyKnJFXR9cf4+aeK6ZkUz5uK8vXAkmHnLQaeOEL54hHKx1PHQSZlayKg1hHMnFbj2b0me5LUyto82Rvaa88VOSWpgkYaHRuL64ALi9cXAtcOK397NJwNbC+mYN4AvC4ijisWZnkdcENxbGdEnF2ssvn2Q75rLHU0TU93pwu0SFKLK22BlilhTpHsPTPabR6SpBb2l0d7YkT8LY2FUuZHxHoaq2p+DLgmIt4J/BB4S3H69TS2RFhHY1uEXwXIzK0R8YfAHcV5Hx5arAV4Fwe2Xvha8WCsdTTTrO660zglqcW1d7LXswA6Z8DWh8uORJI0wTLzz8dw7ttGOfTqEc5N4N2jfM+VwJUjlK8BThuh/Omx1tEsPSZ7ktTy2nsaZwTM+3F4el3ZkUiSNKU4jVOSWl97J3sA806GLQ+VHYUkSVOKI3uS1PpM9uad3Lhnr39f2ZFIkjRlzO7udJ89SWpxJnvzVkAOwrZHy45EkqQpo3fWNJ7etY++gcGyQ5EkjZPJ3ryTG8/etydJ0n4nzplOJmzeubfsUCRJ42SyN++kxvPT3rcnSdKQE+d0AfDkjj0lRyJJGi+TvenHNbZgeGpt2ZFIkjRlnDC7G4CntpvsSVKrMtkDWHgGbLir7CgkSZoyFsyZDsBGkz1Jalkme9BI9p5+CPbsKDsSSZKmhONmdDKt3sFTTuOUpJZlsgew8MzG88Z7yo1DkqQpIiI4YXaX9+xJUgsz2YPGyB44lVOSpGEWzJ7uNE5JamEmewAz58HcpbBhTdmRSJI0ZZwwp9tpnJLUwkz2hix/BTzyLRjoLzsSSZKmhAVzunly+x4ys+xQJEnjYLI35OTXwJ7tju5JklRYMKebvf2DbHl2X9mhSJLGwWRvyEmvgqjBQzeWHYkkSVPCT504G4C1T2wvORJJ0niY7A2ZPhcWvwS+f0PZkUiSNCWcumgo2XNrIklqRSZ7w532i/DUfbDxu2VHIklS6WZ3d7Js3gzuW+/IniS1IpO94V74Fqh1wV2fLzsSSZKmhFMXzeF+p3FKUksy2Rtu+nFwyiq49xrYva3saCRJKt0LFs1h/bbdbNvlIi2S1GpM9g71st+CvTvhXy4tOxJJkkr3osVzAbj14adLjkSSNFalJ3sR8bsRkRExv+xYADjxBfDCt8Ktn4Ut68qORpKkUr1k2XH09nTx1bs3lB2KJGmMSk32ImIJ8Frgh2XG8SNe8wGYNgP+7h3Qt7vsaCRJKk291sGqFy3k5gc3OZVTklpM2SN7lwLvBbLkOA42eyH8/OXw1P3wN2+FfbvKjkiSpNL8/JmL6BtIPvftR8sORZI0BqUlexHxc8CGzDziPgcRsToi1kTEms2bNzchOuAnXgdv+nN49J/h8lfCY99pTr2SJBUi4ryIeDAi1kXExWXFcerCOaw6fSGfvnmd2zBIUguZ1GQvIr4REfeP8FgF/AHw/qP5nsy8PDNXZubK3t7eyQz5YKf/EvzKPzRG9v7qPLjidfDtT8GWh2BwsHlxSJLaTkTUgE8D5wOnAG+LiFPKiueDP3sq82ZN45f+8lb+4e4N9A/YD0rSVBeZzZ9BGREvAG4CniuKFgNPAGdl5pOH++zKlStzzZo1kxzhIfbtgjVXwne/1Nh0HaDeDfNWQM8JMP14mHE8dM2GWmfj0dEJtWlQq0PUDnxXxLAvjsOXH9W5kjQFzZgPK15zTF8REXdm5soJiqhEljeeAAAJKklEQVTlRMS/AT6YmecW7y8ByMyPjnR+M/rH9due49f/+k7u37CDeTOncfqSuSycO53eni6md9ao14J6rYPOjqDWEcQx9lfH2tvZXUqail560jwWzZ1+TN9xtH1k/ZhqGafMvA943tD7iHgUWJmZW8qI54imzYR/+5uNx7ZH4ZFvweYHYcv3Ydfmxkjf7m2wd0fZkUrS1LDk7GNO9sQi4PFh79cDLx1+QkSsBlYDLF26dNIDWnzcDK5998v5xgNPccPaJ1m7YQdrHtvG9t19k163JFXFX/zKi4852TtapSR7Le24ZY3HSDJhcAAG+2BgHwz0N55zaKpLHnzugTcjlB/FuZI0VdW7y46gCkYalzqoE8jMy4HLoTGy14ygah3BuaeeyLmnnri/bG//APv6B+kfSPoGG88Dg8cWzrFOPEr7S0lTVG9PV9PqmhLJXmYuKzuGCRHRmLZZq0Nnc7J1SVJlrQeWDHs/dMvDlNNVr9FVrx35RElSU5W99YIkSRrZHcCKiFgeEdOAC4DrSo5JktRCpsTIniRJOlhm9kfEe4AbgBpwZWauLTksSVILMdmTJGmKyszrgevLjkOS1JqcxilJkiRJFWSyJ0mSJEkVVMqm6sciIjYDjx3j18wHpuaefhPLdlZLO7SzHdoItvNo/Vhm9k5UMFU3Qf0j+O+zStqhjWA7q8Z2Hp2j6iNbLtmbCBGx5mh2nG91trNa2qGd7dBGsJ2a2trlurVDO9uhjWA7q8Z2TiyncUqSJElSBZnsSZIkSVIFtWuyd3nZATSJ7ayWdmhnO7QRbKemtna5bu3QznZoI9jOqrGdE6gt79mTJEmSpKpr15E9SZIkSao0kz1JkiRJqqC2S/Yi4ryIeDAi1kXExWXHM1Ei4tGIuC8i7omINUXZ8RFxY0Q8VDwfV3ac4xERV0bEpoi4f1jZiG2LhsuK63tvRJxZXuRHb5Q2fjAiNhTX9J6IeP2wY5cUbXwwIs4tJ+qxi4glEXFzRDwQEWsj4qKivGrXc7R2VuqaRkR3RNweEd8t2vmhonx5RNxWXM8vRcS0oryreL+uOL6szPh1sKr2j1DdPrId+kdojz7S/rFy13Pq9I+Z2TYPoAb8ADgJmAZ8Fzil7LgmqG2PAvMPKfvvwMXF64uBPyk7znG27RXAmcD9R2ob8Hrga0AAZwO3lR3/MbTxg8DvjnDuKcW/3S5gefFvulZ2G46ynQuAM4vXPcD3i/ZU7XqO1s5KXdPiuswqXncCtxXX6RrggqL8s8C7ite/AXy2eH0B8KWy2+Bj/7WsbP9YtK+SfWQ79I+HaWfV/p7aP1brek6Z/rHdRvbOAtZl5sOZuQ+4GlhVckyTaRVwVfH6KuBNJcYybpn5LWDrIcWjtW0V8PlsuBWYGxELmhPp+I3SxtGsAq7OzL2Z+Qiwjsa/7SkvMzdm5l3F653AA8Aiqnc9R2vnaFrymhbX5dnibWfxSOAc4O+L8kOv59B1/nvg1RERTQpXh9du/SNUoI9sh/4R2qOPtH8cVatezynTP7ZbsrcIeHzY+/Uc/h9YK0ng6xFxZ0SsLspOyMyN0PiPC3headFNvNHaVrVr/J5iesaVw6YYVaKNxRSFM2j82lXZ63lIO6Fi1zQiahFxD7AJuJHGr67PZGZ/ccrwtuxvZ3F8OzCvuRFrFC37b/AotVMfWdm/pyOo1N/TIfaP1bieU6V/bLdkb6QMuSp7T7wsM88EzgfeHRGvKDugklTpGn8G+HHgdGAj8D+L8pZvY0TMAr4M/HZm7jjcqSOUtUxbR2hn5a5pZg5k5unAYhq/tj5/pNOK55ZtZxuo+rWxj6zeNa7c31Owf6RC13Oq9I/tluytB5YMe78YeKKkWCZUZj5RPG8CvkrjH9VTQ0P6xfOm8iKccKO1rTLXODOfKv5QDAJ/yYFpCy3dxojopPEH/ouZ+ZWiuHLXc6R2VvWaAmTmM8AtNO5JmBsR9eLQ8Lbsb2dxfA5HPzVLk6vl/w0eTpv1kZX7ezqSKv49tX+s1vUcUnb/2G7J3h3AimIlnGk0boC8ruSYjllEzIyInqHXwOuA+2m07cLitAuBa8uJcFKM1rbrgLcXq1SdDWwfmv7Qag6Ze//zNK4pNNp4QbFy03JgBXB7s+Mbj2L++RXAA5n5p8MOVep6jtbOql3TiOiNiLnF6+nAa2jcf3Ez8ObitEOv59B1fjPwzcxsiV9o20Al+0doyz6yUn9PR1PBv6f2jwdU4XpOnf7xaFdyqcqDxupF36cxb/YPyo5ngtp0Eo2Vir4LrB1qF425vjcBDxXPx5cd6zjb97c0hvT7aPzy8c7R2kZjGPzTxfW9D1hZdvzH0MYvFG24t/gjsGDY+X9QtPFB4Pyy4x9DO19OY1rCvcA9xeP1Fbyeo7WzUtcUeCFwd9Ge+4H3F+Un0eiM1wF/B3QV5d3F+3XF8ZPKboOPg65n5frHol2V7SPboX88TDur9vfU/rFa13PK9I9RVCBJkiRJqpB2m8YpSZIkSW3BZE+SJEmSKshkT5IkSZIqyGRPkiRJkirIZE+SJEmSKshkTypRRHy7eF4WEb9UdjySJE0V9pHSsTPZk0qUmf+2eLkMGFNHFhG1CQ9IkqQpwj5SOnYme1KJIuLZ4uXHgJ+OiHsi4j9HRC0iPh4Rd0TEvRHxa8X5r4yImyPib4D7ImJmRPyfiPhuRNwfEW8trTGSJE0g+0jp2NXLDkASABcDv5uZbwSIiNXA9sx8SUR0Af8vIr5enHsWcFpmPhIRvwg8kZlvKD43p4zgJUmaRPaR0jg5sidNTa8D3h4R9wC3AfOAFcWx2zPzkeL1fcBrIuJPIuKnM3N7CbFKktRM9pHSUTLZk6amAH4zM08vHsszc+hXy11DJ2Xm94EX0+jQPhoR7y8hVkmSmsk+UjpKJnvS1LAT6Bn2/gbgXRHRCRARPxERMw/9UEQsBJ7LzL8G/gdwZjOClSSpiewjpXHynj1pargX6I+I7wKfAz5JY/WxuyIigM3Am0b43AuAj0fEINAHvKsp0UqS1Dz2kdI4RWaWHYMkSZIkaYI5jVOSJEmSKshkT5IkSZIqyGRPkiRJkirIZE+SJEmSKshkT5IkSZIqyGRPkiRJkirIZE+SJEmSKuj/A80v6PWT4y5VAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# LinReg worked example\n", "# Make plots of how w_D parameter values evolve over iterations, and function values evolve over iterations\n", "# Expected result: x\n", "\n", "fig_h, subplot_grid = plt.subplots(\n", " nrows=1, ncols=2, sharex=True, sharey=False, figsize=(15,3), squeeze=False)\n", "subplot_grid[0,0].plot(history['iter'], history['x_D'])\n", "subplot_grid[0,0].set_xlabel('iters')\n", "subplot_grid[0,0].set_ylabel('w_d')\n", "\n", "subplot_grid[0,1].plot(history['iter'], history['f'])\n", "subplot_grid[0,1].set_xlabel('iters')\n", "subplot_grid[0,1].set_ylabel('-1 * log p(y | w, x)');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Try it Example 4b: Solve the linear regression problem using a weights-and-bias representation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The above example only uses weights on the dimensions of $x_i$, and thus can only learn linear models that pass through the origin.\n", "\n", "Can you instead optimize a model that includes a **bias** term $b>0$?\n", "\n", "\\begin{align}\n", "y_i &\\sim \\mathcal{N}(w^T x_i + b, \\sigma^2), \\forall i \\in 1, 2, \\ldots N\n", "\\end{align}\n", "where we assume $\\sigma = 0.1$.\n", "\n", "One non-Bayesian way to train weights would be to just compute the maximum likelihood solution:\n", "\n", "\\begin{align}\n", "\\min_{w,b} - \\log p(y | w, b, x)\n", "\\end{align}\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "An easy way to do this is to imagine that each observation vector $x_i$ is expanded into a $\\tilde{x}_i$ that contains a column of all ones. Then, we can write the corresponding expanded weights as $\\tilde{w} = [w_1 w_2 b]$.\n", "\n", "\n", "\\begin{align}\n", "\\min_{\\tilde{w}} - \\log p(y | \\tilde{w},\\tilde{x})\n", "\\end{align}\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Now, each expanded xtilde vector has size E = D+1 = 3\n", "\n", "xtilde_NE = np.hstack([x_ND, np.ones((N,1))])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# TODO: Define f to minimize that takes a COMBINED weights-and-bias vector wtilde_E of size E=3" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# TODO: Compute gradient of f" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# TODO run gradient descent and plot the results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 5 setup: Autograd for functions of data structures of arrays" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Useful Fact: autograd can take derivatives with respect to DATA STRUCTURES of parameters\n", "\n", "This can help us when it is natural to define models in terms of several parts (e.g. NN layers).\n", "\n", "We don't need to turn our many model parameters into one giant weights-and-biases vector. We can express our thoughts more naturally." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Demo 1: gradient of a LIST of parameters" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [], "source": [ "def f(w_list_of_arr):\n", " return ag_np.sum(ag_np.square(w_list_of_arr[0])) + ag_np.sum(ag_np.square(w_list_of_arr[1]))\n", "\n", "g = autograd.grad(f)" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Type of the gradient is: \n", "\n", "Result of the gradient is: \n" ] }, { "data": { "text/plain": [ "[array([0., 0., 0.]), array([0., 2., 4., 6., 8.])]" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "w_list_of_arr = [np.zeros(3), np.arange(5, dtype=np.float64)]\n", "\n", "print(\"Type of the gradient is: \")\n", "print(type(g(w_list_of_arr)))\n", "\n", "print(\"Result of the gradient is: \")\n", "g(w_list_of_arr)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Demo 2: gradient of DICT of parameters\n" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "def f(dict_of_arr):\n", " return ag_np.sum(ag_np.square(dict_of_arr['weights'])) + ag_np.sum(ag_np.square(dict_of_arr['bias']))\n", "g = autograd.grad(f)" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Type of the gradient is: \n", "\n", "Result of the gradient is: \n" ] }, { "data": { "text/plain": [ "{'bias': array(8.4), 'weights': array([0., 2., 4., 6., 8.])}" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dict_of_arr = dict(weights=np.arange(5, dtype=np.float64), bias=4.2)\n", "\n", "print(\"Type of the gradient is: \")\n", "print(type(g(dict_of_arr)))\n", "\n", "print(\"Result of the gradient is: \")\n", "g(dict_of_arr)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 5: Neural Networks and Autograd" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Let's use a convenient data structure for NN model parameters\n", "\n", "Use a list of dicts of arrays.\n", "\n", "Each entry in the list is a dict that represents the parameters of one \"layer\".\n", "\n", "Each layer-specific dict has two named attributes: a vector of weights 'w' and a vector of biases 'b'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Here's a function to create NN params as a 'list-of-dicts' that match a provided set of dimensions" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [], "source": [ "def make_nn_params_as_list_of_dicts(\n", " n_hiddens_per_layer_list=[5],\n", " n_dims_input=1,\n", " n_dims_output=1,\n", " weight_fill_func=np.zeros,\n", " bias_fill_func=np.zeros):\n", " nn_param_list = []\n", " n_hiddens_per_layer_list = [n_dims_input] + n_hiddens_per_layer_list + [n_dims_output]\n", "\n", " # Given full network size list is [a, b, c, d, e]\n", " # For loop should loop over (a,b) , (b,c) , (c,d) , (d,e)\n", " for n_in, n_out in zip(n_hiddens_per_layer_list[:-1], n_hiddens_per_layer_list[1:]):\n", " nn_param_list.append(\n", " dict(\n", " w=weight_fill_func((n_in, n_out)),\n", " b=bias_fill_func((n_out,)),\n", " ))\n", " return nn_param_list" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Here's a function to pretty-print any given set of NN parameters to stdout, so we can inspect" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [], "source": [ "def pretty_print_nn_param_list(nn_param_list_of_dict):\n", " \"\"\" Create pretty display of the parameters at each layer\n", " \"\"\"\n", " for ll, layer_dict in enumerate(nn_param_list_of_dict):\n", " print(\"Layer %d\" % ll)\n", " print(\" w | size %9s | %s\" % (layer_dict['w'].shape, layer_dict['w'].flatten()))\n", " print(\" b | size %9s | %s\" % (layer_dict['b'].shape, layer_dict['b'].flatten()))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example: NN with 0 hidden layers (equivalent to linear regression)\n", "\n", "For univariate regression: 1D -> 1D\n", "\n", "Will fill all parameters with zeros by default" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Layer 0\n", " w | size (1, 1) | [0.]\n", " b | size (1,) | [0.]\n" ] } ], "source": [ "nn_params = make_nn_params_as_list_of_dicts(n_hiddens_per_layer_list=[], n_dims_input=1, n_dims_output=1)\n", "pretty_print_nn_param_list(nn_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example: NN with 0 hidden layers (equivalent to linear regression)\n", "\n", "For multivariate regression when |x_i| = 2: 2D -> 1D\n", "\n", "Will fill all parameters with zeros by default" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Layer 0\n", " w | size (2, 1) | [0. 0.]\n", " b | size (1,) | [0.]\n" ] } ], "source": [ "nn_params = make_nn_params_as_list_of_dicts(n_hiddens_per_layer_list=[], n_dims_input=2, n_dims_output=1)\n", "pretty_print_nn_param_list(nn_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example: NN with 1 hidden layer of 3 hidden units" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Layer 0\n", " w | size (2, 3) | [0. 0. 0. 0. 0. 0.]\n", " b | size (3,) | [0. 0. 0.]\n", "Layer 1\n", " w | size (3, 1) | [0. 0. 0.]\n", " b | size (1,) | [0.]\n" ] } ], "source": [ "nn_params = make_nn_params_as_list_of_dicts(n_hiddens_per_layer_list=[3], n_dims_input=2, n_dims_output=1)\n", "pretty_print_nn_param_list(nn_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example: NN with 1 hidden layer of 3 hidden units\n", "\n", "Use 'ones' as the fill function for weights" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Layer 0\n", " w | size (2, 3) | [1. 1. 1. 1. 1. 1.]\n", " b | size (3,) | [0. 0. 0.]\n", "Layer 1\n", " w | size (3, 1) | [1. 1. 1.]\n", " b | size (1,) | [0.]\n" ] } ], "source": [ "nn_params = make_nn_params_as_list_of_dicts(\n", " n_hiddens_per_layer_list=[3], n_dims_input=2, n_dims_output=1,\n", " weight_fill_func=np.ones)\n", "pretty_print_nn_param_list(nn_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example: NN with 1 hidden layer of 3 hidden units\n", "\n", "Use random draws from standard normal as the fill function for weights" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Layer 0\n", " w | size (2, 3) | [-1.89696697 0.10497134 -0.11371797 1.09290398 0.28420754 1.60185133]\n", " b | size (3,) | [0. 0. 0.]\n", "Layer 1\n", " w | size (3, 1) | [-1.35481542 1.56242066 0.28087651]\n", " b | size (1,) | [0.]\n" ] } ], "source": [ "nn_params = make_nn_params_as_list_of_dicts(\n", " n_hiddens_per_layer_list=[3], n_dims_input=2, n_dims_output=1,\n", " weight_fill_func=lambda size_tuple: np.random.randn(*size_tuple))\n", "pretty_print_nn_param_list(nn_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example: NN with 7 hidden layers of diff sizes\n", "\n", "Just shows how generic this framework is!" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Layer 0\n", " w | size (2, 3) | [0. 0. 0. 0. 0. 0.]\n", " b | size (3,) | [0. 0. 0.]\n", "Layer 1\n", " w | size (3, 4) | [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n", " b | size (4,) | [0. 0. 0. 0.]\n", "Layer 2\n", " w | size (4, 5) | [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n", " b | size (5,) | [0. 0. 0. 0. 0.]\n", "Layer 3\n", " w | size (5, 6) | [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", " 0. 0. 0. 0. 0. 0.]\n", " b | size (6,) | [0. 0. 0. 0. 0. 0.]\n", "Layer 4\n", " w | size (6, 5) | [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", " 0. 0. 0. 0. 0. 0.]\n", " b | size (5,) | [0. 0. 0. 0. 0.]\n", "Layer 5\n", " w | size (5, 4) | [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n", " b | size (4,) | [0. 0. 0. 0.]\n", "Layer 6\n", " w | size (4, 3) | [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n", " b | size (3,) | [0. 0. 0.]\n", "Layer 7\n", " w | size (3, 1) | [0. 0. 0.]\n", " b | size (1,) | [0.]\n" ] } ], "source": [ "nn_params = make_nn_params_as_list_of_dicts(\n", " n_hiddens_per_layer_list=[3, 4, 5, 6, 5, 4, 3], n_dims_input=2, n_dims_output=1)\n", "pretty_print_nn_param_list(nn_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup: Function that performs **prediction**" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [], "source": [ "def predict_y_given_x_with_NN(x=None, nn_param_list=None, activation_func=ag_np.tanh):\n", " \"\"\" Predict y value given x value via feed-forward neural net\n", " \n", " Args\n", " ----\n", " x : array_like, n_examples x n_input_dims\n", " \n", " Returns\n", " -------\n", " y : array_like, n_examples\n", " \"\"\"\n", " for layer_id, layer_dict in enumerate(nn_param_list):\n", " if layer_id == 0:\n", " if x.ndim > 1:\n", " in_arr = x\n", " else:\n", " if x.size == nn_param_list[0]['w'].shape[0]:\n", " in_arr = x[ag_np.newaxis,:]\n", " else:\n", " in_arr = x[:,ag_np.newaxis] \n", " else:\n", " in_arr = activation_func(out_arr)\n", " out_arr = ag_np.dot(in_arr, layer_dict['w']) + layer_dict['b']\n", " return ag_np.squeeze(out_arr)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example: Make predictions with 0-layer NN whose parameters are filled with the 'true' params for our toy dataset" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [], "source": [ "true_nn_params = make_nn_params_as_list_of_dicts(n_hiddens_per_layer_list=[], n_dims_input=2, n_dims_output=1)\n", "true_nn_params[0]['w'][:] = true_w_D[:,np.newaxis]\n", "true_nn_params[0]['b'][:] = true_bias" ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Text(0,0.5,u'predicted y|x')" ] }, "execution_count": 59, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xuc1PV97/HXZ/YCmmgwiPGKaGptMImgezCoWddgMbFR8RD7iGIhSl1uGnJyYBfqeSSmacpFT8RUBH4RqVu11oZ6DSkocQQ9P0MgwcSILSYapYoaWpqkVfb2OX/Mb9bZ3Vlm2N2Z3+zM+/l47GPn9/vN7nwixjffu7k7IiIiB5OIuwARESl9CgsREclJYSEiIjkpLEREJCeFhYiI5KSwEBGRnBQWIiKSk8JCRERyUliIiEhO1XEXMFiOPvpoHzNmTNxliIgMKTt27PiNu4/K9b6yCYsxY8awffv2uMsQERlSzOzX+bxP3VAiIpKTwkJERHJSWIiISE4KCxERyakkwsLMqszsp2b2eHR9ipn9yMx2m9k/mFlt3DWKiFSykggLYD6wK+N6GXCbu58G/AcwM5aqREQEKIGwMLMTgT8B7oquDfgM8L3oLfcAU+KpTkSkdIVhyJIlSwjDsOCfVQrrLFYATcAR0fVIYL+7t0fXe4AT4ihMRKRUhWHIpEmTaG1tpba2ls2bNzNx4sSCfV6sLQsz+zzwtrvvyLyd5a1ZDwo3s0Yz225m2995552C1CgiUoqSySStra10dHTQ2tpKMpks6OfF3bI4D7jMzC4BhgNHkmppjDCz6qh1cSLwRrYfdvcACADq6uqyBoqISDkJw5BkMsnIkSOpra3talk0NDQU9HNjDQt3XwwsBjCzBmCBu08zs38EvgA8AMwAHomtSBGREtGz62nFihXs27ePhoaGgnZBQfwti740Aw+Y2V8BPwXWxlyPiEjsenY97du3j8WLFxfls0smLNw9CSSj178CJsRZj4hInNLdTZmthoaGhqJ2PWUqmbAQEZFUSCxfvpzHHnsMd2fYsGFdM50mTpzI5s2be4VIMSgsRERKRBiGNDQ00Nra2nXvwIEDJJPJrmBIh0axxb4oT0REUpLJJG1tbd3uJRKJonY39UVhISJSIhoaGqipqem6rqqqYuXKlbG0JHpSN5SISImYOHEiyWSSlpYWAKZPn14SQQEKCxGRkhLXmEQu6oYSESmSIAi4+OKLCYIg7lIOmVoWIiJF0NzczPLlywHYtGkTAI2NjXGWdEjUshARKbAwDLn11lu73Vu/fn1M1fSPwkJEpMCSySTu3fc6nTp1akzV9I/CQkSkwBoaGhg+fDiJRIJEIkFTU9OQ6oICjVmIiAyabPs5AbFu0zFYFBYiIoMg18l1pTolNl/qhhIRGQTFPrmu2BQWIiL9EIYhS5YsIQxD4P3tw6uqqoq+fXgxqBtKROQQpbucDhw4QCKRYOXKlTQ2Ng75cYmDiTUszGw4sAUYFtXyPXf/upmdQupI1Q8DPwH+zN1b+/5NIiKFlx7Afu211zhw4ACdnZ10dnZyww038IlPfGLIj0scTNwtiwPAZ9z992ZWAzxjZj8Avgrc5u4PmNlqYCawKs5CRaSypc+aaGtro6qqCjPretbR0dHtzIlyFOuYhaf8Prqsib4c+Azwvej+PcCUGMoTEemyaNEiWltbcXfa29s577zzqKmpIZFIMGzYsLIbo+gp7pYFZlYF7AD+AFgJ/BLY7+7t0Vv2ACfEVJ6IVLB0t9P+/fvZsmVLt2djx45l6dKlZTtG0VPsYeHuHcA4MxsBPAR8LNvbsv2smTUCjQCjR48uWI0iUnmCIGDevHl0dnb2emZmXWdNlHtIpJXM1Fl33w8kgU8BI8wsHWQnAm/08TOBu9e5e92oUaOKU6iIlL0wDLnhhhtob2/vGsTOtHDhwooJibS4Z0ONAtrcfb+ZHQZcBCwDngK+QGpG1AzgkfiqFJFKkTnbqaOjo+t+dXU1X/3qV9m5cydTp04dcvs6DYa4u6GOA+6Jxi0SwIPu/riZvQg8YGZ/BfwUWBtnkSJS/jK7naqrq6mpqema+XTHHXdUZEBkijUs3P1nwPgs938FTCh+RSJSaYIgYO3atWzfvr2ru6m9vZ3GxkZGjx5dEYPX+Yi7ZSEiUnSZs5zSp9dlSiQSXQPYkqKwEJGK0tzczK233oq7d1tYl1ZTU8Mdd9yhoOihZGZDiYgUWvoc7M7OTty91yynKVOm8PTTT1f8+EQ2almISNkLw5CWlhbWrFnT7X4ikWDBggUVPcspXwoLESlrQRB0rZnoeQ72ggULWLZsWUyVDS0KCxEpW0EQMHv27F4hkW5RKCjyp7AQkbIUhiFz587tFhSJRILGxkbNdOoHhYWIlJUgCFi/fj2HH354rwHsyy67jFWrdNpBfygsRKQsBEHAihUr2LVrV9e9qqqqrm07ampqaGpqiqu8IU9hISJDWhiGLF++nIcffrjXs7PPPpuzzjoLQF1PA6SwEJEhKT0d9u6776a1NfupyzNnztR02EGisBCRIScMQyZNmsR7773Xa6YTpA4mmj9/voJiECksRGTISSaTXUecQuowourqasaPH6/WRIEoLERkSEh3OwGMHz+e2tpaWltbqa6u5tprr9WYRIEpLESkpKVD4q677qK9vR2AYcOG8Z3vfId9+/ZpC/EiUViISMlKH0iUDom01tZW9u3bx+LFi2OqrPLEuuusmZ1kZk+Z2S4z+4WZzY/uf9jMnjCz3dH3o+KsU0SKKwxDLrjgAmbNmtUrKABqa2tpaGgofmEVLO6WRTvwv939J2Z2BLDDzJ4AvgRsdvelZrYIWAQ0x1iniBRJEATMmTOn1+rrqqoqLr30Uo499liNT8Qg7mNV3wTejF7/zsx2AScAlwMN0dvuAZIoLETKWnps4rvf/W6voDAz7rzzTs1yilHcLYsuZjaG1HncPwI+EgUJ7v6mmR0TY2kiUmAH20YcYOHChQqKmJVEWJjZB4H1wFfc/bfZjjrs4+cagUaA0aNHF65AERl06XOwR44c2WsQ28wwM04//XS+8pWvKChKQOxhYWY1pILiPnf/p+j2W2Z2XNSqOA54O9vPunsABAB1dXW9/zoiIiUpvQK7tbWVRCLRtdkfpMYmrr/+eo1LlJhYw8JSTYi1wC53/3bGo0eBGcDS6PsjMZQnIgWSXoHd0dGBu1NVVUVnZyeJRIKVK1eqJVGC4m5ZnAf8GfBzM9sZ3fsLUiHxoJnNBF4DroypPhEZJEEQcPvtt+PuXHrppV0rsGtra1mxYoUW2JW4uGdDPQP0NUAxqZi1iEjhnHPOOWzbtq3reteuXTQ1NTFixAgFxBARd8tCRMpYGIbMmDGD3bt393q2c+dONm7cGENV0h+xruAWkfLV3NzMueeemzUoAKZOnVrkimQgFBYiMqiCIOCMM85g+fLlWZ8fccQRrFmzRoPYQ4y6oURk0ARBwKxZs/p8PnnyZHU9DVFqWYjIoGhububGG2/M+iyRSLBmzRoFxRCmloWIDNg111zDfffd1+v+ySefzPjx42lqatKMpyFOYSEiAxKGIffff3+v+1OmTOGhhx6KoSIpBIWFiBySMAxZvnw5b7zxBjNnzmTfvn293lNdXU1TU1MM1UmhKCxEJG9hGFJfX9+16d+2bdtoampi+PDhvPfeewCceeaZ3Hnnnep2KjMKCxHJS3Nzc7dzsNN27tzJ5s2bSSaTWo1dxhQWInJQYRgyd+5cdu7cmfX51KlTmThxokKizCksRCSrIAhYu3YtO3bs6LaFOMDhhx/Oxz/+cWbOnKnFdRUir7Aws7Hu/mKPew3unixIVSISq+bm5j5XYAPccMMNLFu2rIgVSdzyXZT3oJk1W8phZvY3wJJCFiYixReGIVdccUWfQWFmTJs2TUFRgfLthjoHWAb8P+AI4D5SZ1GISBkIw5CWlhbWrl1LW1tbr+f19fWMHTtWp9dVsHzDog14FzgMGA684u6dBatKRIrmmmuu4f7778e998nEZsbChQvVkpC8u6F+TCos/gdwPnCVmX1voB9uZneb2dtm9kLGvQ+b2RNmtjv6ftRAP0dEsmtubua+++7LGhRVVVWsXr1aQSFA/mEx092/5u5t7r7X3S9ncM7F/lvgsz3uLQI2u/tpwOboWkQGUXob8dtuu63XsylTpjB79my2bt2qmU7SJa9uKHffnuXe3w30w919i5mN6XH7cqAhen0PkASaB/pZIpJysJlO06ZN49577y1yRTIUHDQszOwpoHf7tLe/dfeWwSmJj7j7mwDu/qaZHTNIv1ekYqX3c3ruuefYu3dvr+eHHXYYN954o7qcpE+5WhZfyvP37B9gHf1iZo1AI8Do0aPjKEGk5IVhyKc//eleC+syrVixQl1OclAHDQt3/zWAmX3c3V842HsH0VtmdlzUqjgOePsg9QVAAFBXV5dPC0ik4ixatKjPoBgzZgyLFy9WUEhO+Q5wrzazbWY218xGFLQieBSYEb2eweAMpItUnCAIGDt2LFu2bMn6vKmpiVdeeUVBIXnJd4D7fDM7DbgO2G5m24B17v7EQD7czP6e1GD20Wa2B/g6sJTUivGZwGvAlQP5DJFKkt7076WXXuraMjybpqYmjU/IIcl7I0F3321m/wfYDnwHGG9mBvyFu/9Tfz7c3a/q49Gk/vw+kUrW19Gmmerr61m6dKlWYcshy3cjwU8C1wJ/AjwBXOruPzGz44EQ6FdYiMjgSC+u68vYsWOZP3++upyk3/JtWdwBfJdUK+Ld9E13fyNqbYhITMIwZPXq1VmfHXvssXzjG99QSMiA5TtmUX+QZwNenCcihya98d/evXvZsGEDra2tvd4zefJkNm7cGEN1Uo50+JHIEJPrrAkzY/Xq1WpNyKBSWIgMIQcbxK6treW6667TNuJSEAoLkSEgCAJuv/12XnzxxazPJ0yYwIoVKxQSUjC59oZ6jIPsDeXulw16RSLSJb2n08MPP9znexKJhIJCCi7XCu5bgf8LvELqPIvvRl+/B4q1/YdIRWpububcc8/tMyiqq6upr6/nmWeeUVBIweXaG+ppADP7Zo8ZUY+ZWfY9BERkQMIwZMaMGezevbvP91RXV7NlyxaFhBRNvntDjTKzU9MXZnYKMKowJYlUpjAMueKKKzj33HP7DIrq6mpmz56toJCiy3eA+38BSTP7VXQ9BphVkIpEKlAYhlxwwQW0tbX1+Z4pU6bQ1NSkkJBY5Lso75+jjQT/KLr1krsfKFxZIpUjCAK++c1v9hkUI0eO5K//+q+1bkJile/eUIcDXwVOdvfrzew0Mzvd3R8vbHki5Su9Q+zOnTuzPj/mmGP40pe+pN1hpSTkO2axDmgF0u3fPcBfFaQikQrQ3NzMeeedlzUoqqqqWLNmDW+99ZaCQkpGvmHxUXdfDrQBRJsJWsGqEilj6e063HsvYZoyZQpbt25Vl5OUnHwHuFvN7DCiBXpm9lFAYxYihygMQ2699dasz6ZNm8a9995b5IpE8pNvWNwM/DNwkpndB5xH6nwLEckhvUNsWs8Whc7BlqEg39lQm8xsB/ApUt1P8939N4UszMw+C9wOVAF3ufvSQn6eyGALw5BFixaxdevWroCorq6mtra2a+bTggULNC4hQ0K+s6E2u/sk4PtZ7g06M6sCVgJ/TGow/cdm9qi7Z99FTaTE9LWNeEdHB3/+53/O6NGjaWho0JoJGTJybSQ4HDgcONrMjuL9Qe0jgeMLWNcE4GV3/1VUxwPA5YDCQkpaujWxZUv23XBqamq0hbgMSblmQ80CdpBajLcj4+sRUn/zL5QTgNczrvdE97oxs0Yz225m2995550CliOSW3rjv76Cor6+nmQyqaCQISnXRoK3A7eb2Y3u/jdFqgmyT8vtNc/Q3QMgAKirq+tzK3WRQgqCgCVLlvDqq69mfT5ixAiWLVumAWwZ0vKdDdVpZiPcfT9A1CV1lbvfWaC69gAnZVyfCLxRoM8S6bdcR5xWVVWxYcMGtSZkyMt3Ud716aAAcPf/AK4vTEkA/Bg4zcxOMbNa4IvAowX8PJFDFgQBt9xyS5/P6+vr2bp1q4JCykK+LYuEmZlH8/+i2Uq1hSrK3dvN7AZgI6mps3e7+y8K9XkihyoIAubMmZN1FbbWTUg5yjcsNgIPmtlqUmMHs0kt0isYd98AbCjkZ4gcqiAIWLt2Ldu3b6ezs7Pbs/r6epYuXaqWhJSlfMOimdTMqDmkBp83AXcVqiiRUhMEAbfffjsvvth79nYikWDVqlVqSUhZy3cFdyewKvoSqRjNzc2sW7eOvqZm19TUcMcddygopOzlWpT3oLv/qZn9nOxTVz9ZsMpEYnbxxRezadOmPp/r5DqpJLlaFvOj758vdCEipSAMQ5YvX87OnTv7XDdx+OGHc9ttt6k1IRUl16K8N6Pvvy5OOSLxCYKAWbNyHy2voJBKlKsb6ndk6X5Kc/cjB70ikRiEYcjs2bMP+p4JEyYwc+ZMBYVUpFwtiyMAzOwvgb3A35GaDTUNOKLg1YkUWHoq7Ntvv511zcTJJ5/M+PHjNTYhFS/fqbMXu/s5GderzOxHQN/7HIiUsDAMmT59Oi+//HKf75k8eTIbN24sYlUipSvf7T46zGyamVWZWcLMpgEdhSxMpFCCIOD888/vMyjMjGnTpikoRDLkGxZXA38KvBV9XRndExlSmpubmT17dq/V15AKidmzZ/Pss8/qLGyRHvJdlPcqqcOHRIakMAyZMWMGu3fvzvrczFi9erUGr0X6kFfLwsz+0Mw2m9kL0fUnzez/FLY0kcGRPpQoW1B86EMfor6+nmeffVZBIXIQ+XZDfRdYDLQBuPvPSG0bLlKygiBg7NixfZ430dTUxP79+3n66ac100kkh3xnQx3u7tvMuh1g116AekQGLL0K++GHH+7zPZMnT2bZsmVFrEpkaMs3LH5jZh8lWqBnZl8A3ixYVSL9FIYhkyZN4t133836fPjw4Xz5y19WUIgconzDYh6ps67/yMz+DXiF1MI8kZIQhiEtLS08/fTTvPfee1nfU11dzQ9/+EN1OYn0Q86wMLMEUOfuF5nZB4CEu/9uoB9sZlcCNwMfAya4+/aMZ4uBmaTWcnzZ3TXhXfoUhiHnn39+r+mwVVVVnH322TQ0NDBixAgaGhoUFCL9lDMs3L0zOuL0QXf/r0H87BeA/wmsybxpZmNJDZ6fARwPPGlmf+juWgQo3YRhSDKZZN26dVnXTVx//fWsWqUjWEQGQ77dUE+Y2QLgH4CuwHD3f+/vB7v7LkjNb+/hcuABdz8AvGJmLwMTgLC/nyXlJwgCZs+enXU/J4Bhw4Yxffr0IlclUr7yDYvrou/zMu45cOrglgPACcBzGdd7onu9mFkj0AgwevToApQipSjXVuJnnnkmq1atUpeTyCDKdwX3Kf355Wb2JHBslkc3ufsjff1YthL6qCsgNfBOXV1dn1upS3kIgoAVK1bw0ksvZX1uZlx99dXaqkOkAPIKCzMbDswFzif1H+6twGp3zz7tJOLuF/Wjpj3ASRnXJwJv9OP3SJnIZ91EfX09S5cuVWtCpEDy7YZqAX4H/E10fRWpsy2uLEBNjwL3m9m3SQ1wnwZsK8DnyBAQBAFz586loyP7/IZRo0Zx7bXXat2ESIHlGxanu/uZGddPmdnzA/lgM7uCVPiMAr5vZjvd/WJ3/4WZPQi8SGqV+DzNhKpMYRj2GRSJRIJVq1ZpPyeRIsk3LH5qZp9y9+cAzOwc4NmBfLC7PwQ81MezbwHfGsjvl6EpvbgurWdQHH300Zx//vk6uU6kyPINi3OA6Wb2WnQ9GthlZj8H3N0/WZDqpKL07HKqqamhpqaGtrY2ILUC+9FHH1VIiMQg37D4bEGrkIoXhmGvdRPt7e3dpshOnz5dQSESk3ynzv660IVI5QrDkKuuuqrXArva2loFhEiJyLdlITKo0mMTe/fuZcOGDbS2tnZ7/oEPfIAnnnhCQSFSIhQWUnRBEDBnzpys+zmlzZs3T0EhUkIUFlI0YRgyZ84cnn++96xrMyORSHDcccdx9dVXa92ESIlRWEhRHKw1kUgkaGxs1PiESAlTWEhBNTc3s27dOt55552sz7W4TmRoUFhIwVxzzTXcd999fT4fN24cd955p1oTIkOAwkIKIgiCrEFhZowcOZLrrrtO4xIiQ4jCQgZdGIbMmzev130zY/Xq1epyEhmCEnEXIOXhmmuu4UMf+hDjx4+npaWl10D2uHHjePbZZxUUIkOUWhYyYJljEzt37uT555+npqaG9vZ2EokEK1euVEiIDHEKC+m3MAxJJpM89thj3e67O9dddx2jR4+moaFBA9giZUBhIf0ShiGTJk3qtU0HpKbDas2ESHnRmIUckiAIOOOMM7j88st57733urYTHzduHEceeSTjxo3jmWeeUVCIlJnYWhZmdgtwKdAK/BK41t33R88WAzOBDuDL7r4xrjolJQxDFi1axJYtW7rdTyQS1NbWar2ESJmLs2XxBPDx6OCkfwUWA5jZWOCLwBmkztG408yqYqtSCMOQ+vr6XkEBcOqpp7J582YFhUiZiy0s3H2Tu7dHl88BJ0avLwcecPcD7v4K8DIwIY4aJaWlpYX29vaszxYuXKigEKkApTJmcR3wg+j1CcDrGc/2RPd6MbNGM9tuZtv72ntICuPkk09mzZo1mhIrUiEKGhZm9qSZvZDl6/KM99wEtAPpvSEsy6/yLPdw98Dd69y9btSoUYP/P6AChWHIFVdcwTnnnEMQBEDqONNhw4YBUFVVxZo1a3j11VcVFCIVpKAD3O5+0cGem9kM4PPAJH//TM09wEkZbzsReKMwFUqmntuIb9u2DYDGxkaeeuopksmk1k2IVCjree5x0T7Y7LPAt4EL3P2djPtnAPeTGqc4HtgMnObuHQf7fXV1db59+/YCVlze+jpvYvLkyWzcqMloIuXKzHa4e12u98W5KO8OYBjwhJkBPOfus939F2b2IPAiqe6pebmCQgamubmZW265hWx/cZg6dWoMFYlIqYktLNz9Dw7y7FvAt4pYTsUKgoDly5d3u2dmfOxjH2P+/PkalxARQNt9VLz169d3u9Y24iKSTalMnZUiCMOQOXPmMGfOHMIwBHp3My1cuFBBISK9qGVRIcIw5MILL+TAgQMA3H333SSTya5gWL9+PVOnTlVQiEhWCosyl95G/LXXXuu2Q2xbWxvJZJKJEyfS2NiokBCRg1JYlKkgCFiyZAmvvvoqZkZNTQ3V1dW0tbUBUFNTQ0NDQ7xFisiQobAoQ83Nzd1mOLk7bW1tzJo1q+uezpsQkUOhsCgz2abCQmqWkwJCRPpLs6HKSBiGzJ07N+uzBQsWKChEpN/UshjiwjCkpaWl67rndh0jRoxg2bJlGsAWkQFRWAxR6ZC46667us6ayDaIvWHDBrUoRGTAFBZDUBiGNDQ0dJsKC9De3q5BbBEpCIXFENTS0tIrKABqa2sVECJSEAqLMjB27Fjq6+sVFCJSMAqLIWj69OmsW7eO1tZWamtrueuuuxQSIlJQCosSl96uI/OEuokTJ+rkOhEpKoVFiUrPdrr77rvp6OigtraWzZs3dwsMhYSIFEtsi/LM7Jtm9jMz22lmm8zs+Oi+mdl3zOzl6PlZcdUYlyAIuOCCC1izZg2tra10dHTQ2tpKMpmMuzQRqVBxruC+xd0/6e7jgMeBr0X3PwecFn01Aqtiqi8WYRgyb9482trauo45NTNqa2u18Z+IxCa2sHD332ZcfgBIHwB9OdDiKc8BI8zsuKIXWERhGHLBBRdw0kknsWjRom6rsKuqqpg1a1a3LigRkWKLdczCzL4FTAf+E7gwun0C8HrG2/ZE994sbnXF0XOH2D179lBdnfpjSSQSrFy5Ult1iEjsChoWZvYkcGyWRze5+yPufhNwk5ktBm4Avg5Ylvd7lnuYWSOpripGjx49OEUXSRiGLF++nIcffrjXs4985CPMmzdPM51EpGQUNCzc/aI833o/8H1SYbEHOCnj2YnAG338/gAIAOrq6rIGSikKw5BJkybx7rvvZn0+bdo0Fi9eXOSqRET6FudsqNMyLi8DXopePwpMj2ZFfQr4T3cvqy6oZDKZdbuOo446iqamJpYtWxZDVSIifYtzzGKpmZ0OdAK/BmZH9zcAlwAvA/8NXBtPeYMrCALWrl3L8ccfz+c+9zlqa2tpbW0lkUgwfvx4Zs6cqbEJESlZsYWFu0/t474D84pcTkEFQdBtN9jHH3+clStXsm/fPo1LiMiQoBXcRbB+/fpu1+3t7ezbt0/jEiIyZOhY1SKYOrV7I6q6uloL7ERkSFHLYpBl2/gvPRaRHrNoampS15OIDCmW3lJiqKurq/Pt27fH8tnpTf/27t3LD37wA9rb23tt/CciUorMbIe71+V6n1oWAxSGIRdeeCEHDhzodj+98Z/CQkTKgcYsBijbmglt/Cci5UZh0Q9BEHDxxRcTBAENDQ3U1tZ2PauurtbGfyJSdtQNdYgy10xs2rSJNWvW8NRTT9HS0gKgc7BFpCwpLA5RzzUT69evp7GxUQEhImVN3VCHqOeaiZ7XIiLlSC2LQ5ReM7F+/XqmTp2q/ZxEpCJonYWISAXLd52FuqFERCQnhUUkDEOWLFlCGIZxlyIiUnI0ZsH7J9e1trZqmw4RkSwqvmURhiE333wzBw4coKOjo2ubDhEReV/sYWFmC8zMzezo6NrM7Dtm9rKZ/czMzirUZ6dbFE8++SSdnZ0kEglt0yEikkWsYWFmJwF/DLyWcftzwGnRVyOwqlCfn97XKR0UF110kbqgRESyiLtlcRvQBGTO370caPGU54ARZnZcIT48va9TVVUVw4YN4+abb1ZQiIhkEdsAt5ldBvybuz9vZpmPTgBez7jeE917c7BrmDhxIps3b+51WJGIiHRX0LAwsyeBY7M8ugn4C2Byth/Lci/rykEzayTVVcXo0aP7VePEiRMVEiIiORQ0LNz9omz3zewTwClAulVxIvATM5tAqiVxUsbbTwTe6OP3B0AAqRXcg1e5iIhkimXMwt1/7u7HuPsYdx9DKiDOcve9wKPA9Gh3Y/v4AAAGcElEQVRW1KeA/3T3Qe+CEhGR/JXiorwNwCXAy8B/A9fGW46IiJREWESti/RrB+bFV42IiPQU99RZEREZAhQWIiKSU9mcZ2Fm7wC/zuOtRwO/KXA5/aXaDl2p1gWqrb9KtbZSrQsGVtvJ7j4q15vKJizyZWbb8znoIw6q7dCVal2g2vqrVGsr1bqgOLWpG0pERHJSWIiISE6VGBZB3AUchGo7dKVaF6i2/irV2kq1LihCbRU3ZiEiIoeuElsWIiJyiCouLOI8me8gNX0z+uydZrbJzI4vhdrM7BYzeyn67IfMbETGs8VRXf9iZhcXs67o8680s1+YWaeZ1fV4FmttUQ2fjT7/ZTNbFEcNGbXcbWZvm9kLGfc+bGZPmNnu6PtRMdR1kpk9ZWa7oj/L+SVU23Az22Zmz0e1fSO6f4qZ/Siq7R/MrLbYtUV1VJnZT83s8aLV5e4V80VqN9uNpNZjHB3duwT4Aamt0T8F/CiGuo7MeP1lYHUp1EZqC/nq6PUyYFn0eizwPDCM1O7BvwSqilzbx4DTgSRQl3G/FGqrij73VKA2qmdssf+9yqinHjgLeCHj3nJgUfR6UfrPtsh1HUdqA1GAI4B/jf78SqE2Az4Yva4BfhT9f/BB4IvR/dXAnJj+TL8K3A88Hl0XvK5Ka1nEejJfX9z9txmXH8ioL9ba3H2Tu7dHl8+R2i4+XdcD7n7A3V8htenjhGLVFdW2y93/Jcuj2GuLPu9ld/+Vu7cCD0R1xcLdtwD/3uP25cA90et7gClFLQpw9zfd/SfR698Bu0gddFYKtbm7/z66rIm+HPgM8L04azOzE4E/Ae6Krq0YdVVMWGSezNfjUV8n8xWVmX3LzF4HpgFfK6XaIteRauVAadXVUynUVgo15PIRj7b+j74fE2cxZjYGGE/qb/AlUVvU1bMTeBt4glRrcX/GX6Di+nNdQeovvZ3R9chi1FUSu84OlkKfzDcQB6vN3R9x95uAm8xsMXAD8PVi1Jarrug9NwHtwH3pHyt0XfnWlu3Hstwr9pS/UqhhyDCzDwLrga+4+297HLMcG3fvAMZFY3UPker67PW2YtZkZp8H3nb3HWbWkL6d5a2DXldZhYUX+GS+QtSWxf3A90mFRcFry1WXmc0APg9M8qhDtBh15VNbH4pS2xCoIZe3zOw4d38z6tp8O44izKyGVFDc5+7/VEq1pbn7fjNLkhqzGGFm1dHf4uP4cz0PuMzMLgGGA0eSamkUvK6K6IbyEj+Zz8xOy7i8DHgpeh1rbWb2WaAZuMzd/zvj0aPAF81smJmdApwGbCtWXTmUQm0/Bk6LZqjUAl+M6ioljwIzotczgL5aagUT9bWvBXa5+7dLrLZR6dl/ZnYYcBGpMZWngC/EVZu7L3b3E6P/jn0R+KG7TytKXXGM5Mf9BbzK+7OhDFhJqj/y52TMrCliPeuBF4CfAY8BJ5RCbaQGh18HdkZfqzOe3RTV9S/A52L4Z3YFqdA/ALwFbCyV2qIaLiE1u+eXpLrNil5DRi1/D7wJtEX/zGaS6ufeDOyOvn84hrrOJ9Vd8rOMf8cuKZHaPgn8NKrtBeBr0f1TSf3l42XgH4FhMf65NvD+bKiC16UV3CIiklNFdEOJiMjAKCxERCQnhYWIiOSksBARkZwUFiIikpPCQiQPZjbCzObGXYdIXBQWIvkZAWQNCzOrKnItIkWnsBDJz1Lgo5Y6c+QWM2uIzmK4H/i5mY3pcV7EAjO7OXr9UTP7ZzPbYWZbzeyPMn+xmSWicwhGZVy/bNGZKyKlQGEhkp9FwC/dfZy7L4zuTSC1Ontsjp8NgBvd/WxgAXBn5kN37wTuJbXjMKS2lnje3X8zaNWLDFBZbSQoUmTbPHVmRp+iHVXPBf4xYzfVYVneejep/XxWkNoOft0g1ikyYAoLkf77r4zX7XRvqQ+PvidInTUw7mC/yN1fN7O3zOwzwDm838oQKQnqhhLJz+9IHf3Zl7eAY8xspJkNI7WtO546BfEVM7sSus5VP7OP33EXqe6oBz11loJIyVBYiOTB3fcBz5rZC2Z2S5bnbcBfkjrp7XHe32YeUq2EmWb2PPAL+j5i9VHgg6gLSkqQdp0VKRFmVgfc5u6fjrsWkZ40ZiFSAsxsETAHjVVIiVLLQkREctKYhYiI5KSwEBGRnBQWIiKSk8JCRERyUliIiEhOCgsREcnp/wOy2NHLiuBm8AAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "yhat_N = predict_y_given_x_with_NN(x_ND, true_nn_params)\n", "assert yhat_N.size == N\n", "\n", "plt.plot(yhat_N, y_N, 'k.')\n", "plt.xlabel('true y')\n", "plt.ylabel('predicted y|x')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example: Make predictions with 0-layer NN whose parameters are filled with all zeros" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Text(0,0.5,u'predicted y|x')" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHI1JREFUeJzt3X2UHXWd5/H3p9Okg/KQhAQhJNkEJz5EQHR6kd5RN8PDAK5DWKJzcDkbdsxO3AE9MrNMgMEzq6McJO4OiqBOKzNDHBnIGI2MqANm6RVmWrB5DBCZxICahEhIeJCHpNPd3/3jVuNN6K57uberq3+Xz+uce259q+re+v7oDt+u+lX9fooIzMzM8rSVnYCZmU18LhZmZlaTi4WZmdXkYmFmZjW5WJiZWU0uFmZmVpOLhZmZ1eRiYWZmNblYmJlZTe1lJzBWZsyYEfPmzSs7DTOzpNxzzz1PRcTMWvu1TLGYN28efX19ZadhZpYUST+vZz9fhjIzs5pcLMzMrCYXCzMzq8nFwszMapoQxULSJEn3SfpuFs+XdJekjZJukjS57BzNzF7LJkSxAD4ObKiKrwSuiogFwNPAslKyMjMzYAIUC0mzgf8EfC2LBZwEfDPb5XrgrHKyM2tOb28vV1xxBb29vWWnYtaUifCcxeeBFcDBWXwY8ExEDGTxFuCoMhIza0Zvby8nn3wy/f39TJ48mXXr1tHV1VV2WmYNKfXMQtL7gScj4p7q1SPsOuJE4ZKWS+qT1Ldjx45CcjRrVE9PD/39/QwODtLf309PT0/ZKZk1rOzLUL8DnCnpceBGKpefPg9MlTR81jMb2DbShyOiOyI6I6Jz5syaT6ubjatFixbR3t6OJNrb21m0aFHZKZk1rNRiERGXRsTsiJgHnAP834g4F7gd+EC223nAd0pK0awpEbHPu1mqyj6zGM3FwJ9K2kSlD+O6kvMxe9V6enoYHBwkIhgcHPRlKEvaROjgBiAieoCebHkzcEKZ+Zg1a/gy1NDQkC9DWfIm6pmFWUsYGBggIhgYGKi9s9kE5mJhVpCVK1cyODgIwODgICtXriw5I7PGuViYFWTbtm25sVlKXCzMCrJs2bLc2CwlE6aD26zVLF++HIA1a9awZMmSl2OzFKlV7v/u7OwMT6tqZvbqSLonIjpr7efLUGYF6u7u5rTTTqO7u7vsVMya4stQZgXp7u7mIx/5CAC33norgC9FWbJ8ZmFWkOuuuy43NkuJi4VZQWbNmpUbm6XExcKsICtWrOCAAw4A4IADDmDFihUlZ2TWOPdZmBWkq6uLa6655uVbZz3xkaXMxcKsIL29vVx44YX09/dzxx13cOyxx7pgWLJ8GcqsIJ4pz1qJi4VZQTxTnrUSFwuzAlVPfmSWslKLhaQpku6W9ICkhyV9Kls/X9JdkjZKuknS5DLzNGvEqlWrXp7HYmBggFWrVpWckVnjyj6z2AOcFBFvB44HTpd0InAlcFVELACeBjxcpyVn+/btubFZSkotFlHxfBYekL0COAn4Zrb+euCsEtIzM7NM2WcWSJok6X7gSeA24GfAMxExPA/lFuCosvIza9SuXbtyY7OUlF4sImIwIo4HZgMnAG8dabeRPitpuaQ+SX07duwoMk2zV+2pp57Kjc1SUnqxGBYRzwA9wInAVEnDDwzOBkacjzIiuiOiMyI6Z86cOT6JmtVpxowZubFZSsq+G2qmpKnZ8oHAKcAG4HbgA9lu5wHfKSdDs8YtXLgwNzZLSdnDfRwJXC9pEpXCtToivivpEeBGSZ8B7gM8trMl5x3veEdubJaSUotFRDwIvOJfUERsptJ/YZas++67Lzc2S8mE6bMwazWPPPJIbmyWEhcLs4L4bihrJS4WZgWZPHlybmyWEhcLs4L09/fnxmYpcbEwK4ifs7BW4mJhVpDp06fnxmYpcbEwK4jHhrJW4mJhVpDdu3fnxmYpcbEwK8iCBQtyY7OUuFiYFcRPcFsrcbEwK8iLL76YG5ulxMXCrCBz587Njc1S4mJhVhAPUW6txMXCrCBLly6lvb0ysHN7eztLly4tOSOzxrlYmBVk/fr1DAxUppIfGBhg/fr1JWdk1jgXC7OCXHfddbmxWUrKnlZ1jqTbJW2Q9LCkj2frp0u6TdLG7H1amXmaNWLKlCm5sVlKyj6zGAD+Z0S8FTgRuEDSQuASYF1ELADWZbFZUp577rnc2CwlpRaLiHgiIu7Nln8NbACOAhYD12e7XQ+cVU6GZo3bunVrbmyWkrLPLF4maR6V+bjvAt4QEU9ApaAAh5eXmVljZs2alRubpWRCFAtJBwFrgAsjou5zdUnLJfVJ6tuxY0dxCZo1YP78+bmxWUpKLxaSDqBSKL4REd/KVv9K0pHZ9iOBJ0f6bER0R0RnRHTOnDlzfBI2M3sNKvtuKAHXARsi4q+qNt0MnJctnwd8Z7xzM2vWEUcckRubpaTsM4vfAf4rcJKk+7PX+4DPAqdK2gicmsVmSVm6dCltbZV/Ym1tbX6C25LWXubBI+JOQKNsPnk8czEba9deey1DQ0MADA0Nce2119LV1VVyVmaNUUSUncOY6OzsjL6+vrLTMHtZR0cH/f39L8eTJ09mz549JWZk9kqS7omIzlr7lX0ZyqxlDQ4O5sZmKXGxMCtIR0dHbmyWEhcLs4IMd26PFpulxL+9ZgWZMWNGbmyWEhcLs4IcfvjhubFZSlwszAqyYcOG3NgsJS4WZgU5+uijc2OzlLhYmBXkmGOOyY3NUuJiYVaQu+66Kzc2S4mLhVlBpk+fnhubpcTFwqwgjz32WG5slpK6ikU2L/b+6xaNeTZmLWR4EMHRYrOU1HtmsVrSxao4UNIXgSuKTMwsdQcddFBubJaSeovFu4A5wL8CPwG2UZmLwsxGMW3atNzYLCX1Fou9wEvAgcAU4LGI8Dm1WY7q4clHis1SUm+x+AmVYvHvgXcDH5L0zWYPLulvJD0p6aGqddMl3SZpY/buP8csSXv37s2NzVJSb7FYFhF/ERF7I2J7RCxmbObF/jvg9P3WXQKsi4gFwLosNkvOSy+9lBubpaSuYhERr5iCLiK+3uzBI+JHwK79Vi8Grs+WrwfOavY4ZmU4+OCDc2OzlOTOwS3pdqCeeVf/LiJWjU1KvCEingCIiCckeahOS9LWrVtzY7OU5BYL4L/V+T3PNJlHQyQtB5YDzJ07t4wUzEbl5yysleQWi4j4OYCkYyLiobx9x9CvJB2ZnVUcCTyZk1830A3Q2dlZzxmQ2bhxn4W1kno7uL8i6W5J50uaWmhGcDNwXrZ8HmPTkW427jwHt7WSeju43w2cS+XBvD5JN0g6tdmDS/oHoBd4s6QtkpYBnwVOlbQRODWLzZJzyCGH5MZmKanVZ/GyiNgo6RNAH3A18A5JAv48Ir7VyMEj4kOjbDq5ke8zm0iee+653NgsJfUOJHicpKuADcBJwO9HxFuz5asKzM8sWe6zsFZS75nFNcBXqZxFvPwbHxHbsrMNM9tPW1sbg4OD+8Rmqaq3z+K9EfH16kJRta3ph/PMWtH8+fNzY7OU+E8ds4LMmjUrNzZLiYuFWUH6+vpyY7OUuFiYFcQd3NZKao0N9U/kjA0VEWeOeUZmLaK9vX2fYcnb2+u+U91swqn12/u/s/ezgSOAv8/iDwGPF5STWUvo6OjYp1j4CW5LWa2xof4fgKRPR8R7qzb9k6QfFZqZWeI8kKC1knr7LGZKOno4kDQfmFlMSmatYc+ePbmxWUrqvYj6J0CPpM1ZPA/4SCEZmZnZhFNXsYiIH0haALwlW/XTiPCfSWY5Dj30UHbt2rVPbJaqeseGeh3wZ8BHI+IBYK6k9xeamVni9p+QyxN0Wcrq7bP4W6Af6MriLcBnCsnIrEVs3749NzZLSb3F4o0RsRLYC5CNEaXCsjJrAS+88EJubJaSeotFv6QDyR7Qk/RGwH0WZjle//rX58ZmKam3WHwS+AEwR9I3gHXAxUUlZdYKTjzxxNzYLCX13g11q6R7gBOpXH76eEQ8VWRikk4HvgBMAr4WEZ5e1ZLypje9KTc2S0m9d0Oti4idEXFLRHw3Ip6StK6opCRNAq4FzgAWAh+StLCo45kV4aabbsqNzVJSayDBKcDrgBmSpvGbTu1DgCIH5z8B2BQRm7M8bgQWA48UeEyzMbVz587c2CwltS5DfQS4kEphuIffFIvnqPzlX5SjgF9WxVuAd+2/k6TlwHLwPew28ew/jaqnVbWU1RpI8AvAFyR9LCK+OE45wci35b5iqPSI6Aa6ATo7O0cdSt2sDLt3786NzVJS7586Q5KmDgeSpkk6v6CcoHImMacqng1sK/B4ZmNucHAwNzZLSb3F4o8i4pnhICKeBv6omJQA+AmwQNJ8SZOBc4CbCzye2ZibPHlybmyWknqLRZukly8NZXcrFfabHxEDwEeBfwY2AKsj4uGijmdWhDe/+c25sVlK6h2i/J+B1ZK+QqXv4H9QeUivMBHxPeB7RR7DrEjz5s3j/vvv3yc2S1W9xeJiKndG/TGVzudbga8VlZRZK6gennyk2Cwl9T7BPQR8OXuZWR1+8Ytf5MZmKan1UN7qiPgDSesZ+dbV4wrLzCxxO3bsyI3NUlLrzOLj2bsnOjJ7lV566aXc2CwltR7KeyJ7//n4pGPWOqpuIBwxNktJrctQv2aEy0/DIuKQMc/IrEW0tbXt8yCeh/uwlNU6szgYQNJfAtuBr1O5G+pc4ODCszNLmM8srJXU+6fOaRHxpYj4dUQ8FxFfBpYUmZhZ6g499NDc2Cwl9RaLQUnnSpokqU3SuYAHujHL8eyzz+bGZimpt1j8F+APgF9lrw9m68xsFB5I0FpJvQ/lPU5l8iEzq9P06dP3ebZi+vTpJWZj1px6p1V9k6R1kh7K4uMkfaLY1MzS5jMLayX1Xob6KnApsBcgIh6kMmy4mY3i+eefz43NUlJvsXhdRNy937qBsU7GrJW0t7fnxmYpqbdYPCXpjWQP6En6APBEYVmZtQBPfmStpN5icQHw18BbJG0FLqQyp4WZjeK4447Ljc1SUrNYSGoDOiPiFGAm8JaIeHez40VJ+qCkhyUNSercb9ulkjZJelTSac0cx6wsc+bMyY3NUlKzWGRzWXw0W34hIn49Rsd+CDgb+FH1SkkLqXSevw04HfhSNo2rWVJuueWW3NgsJfVehrpN0kWS5kiaPvxq5sARsSEiHh1h02LgxojYExGPAZuAE5o5llkZ+vv7c2OzlNR7e8aHs/cLqtYFcPTYpgPAUcCPq+It2bpXkLQcWA4wd+7cAlIxMzOo/wnu+Y18uaQfAkeMsOmyiPjOaB8bKYVR8uoGugE6OztHHUrdrAxDQ0O5sVlK6ioWkqYA5wPvpvI/7juAr0TE7rzPZZ3ir9YWoLoncDawrYHvMSvVQQcdxO7du/eJzVJVb5/FKiodzl8ErgEWUpnbogg3A+dI6pA0H1gA7P9AoNmE5/ksrJXU22fx5oh4e1V8u6QHmjmwpP9MpfjMBG6RdH9EnBYRD0taDTxC5SnxCyLCg+pYcg499NB9BhL0fBaWsnqLxX2SToyIHwNIehfwL80cOCK+DXx7lG2XA5c38/1mZdt/lFmPOmspq/cy1LuAf5X0uKTHgV7gP0paL+nBwrIzS9iCBQtyY7OU1HtmcXqhWZi1oI0bN+bGZimp99bZpob2MHstevrpp3Njs5TUexnKzF4lz8FtrcTFwqwg+9/95LuhLGUuFmYF2bt3b25slhIXC7OC7Ny5Mzc2S4mLhVlBfGZhrcTFwqwgU6ZMyY3NUuJiYVYQz2dhrcTFwqwgs2bNyo3NUuJiYVaQU045JTc2S4mLhVlB7r333tzYLCUuFmYFWb9+fW5slhIXC7OCHHvssbmxWUpcLMwKctZZZ+XGZikprVhI+pykn0p6UNK3JU2t2nappE2SHpV0Wlk5mjWjp6cnNzZLSZlnFrcBx0TEccC/AZcCSFoInENlzu/TgS9JmlRalmYNuvPOO3Njs5SUViwi4taIGMjCHwOzs+XFwI0RsSciHgM2ASeUkaNZM/wEt7WSidJn8WHg+9nyUcAvq7Ztyda9gqTlkvok9e3YsaPgFM1enfe+9725sVlKCi0Wkn4o6aERXour9rkMGAC+MbxqhK+Kkb4/IrojojMiOmfOnDn2DTBrwooVK2hrq/wTa2trY8WKFSVnZNa4eufgbkhE5D6yKuk84P3AyRExXBC2AHOqdpsNbCsmQ7PirF27lqGhIQCGhoZYu3YtXV1dJWdl1pgy74Y6HbgYODMiXqzadDNwjqQOSfOBBcDdZeRo1owbbrghNzZLSZl9FtcABwO3Sbpf0lcAIuJhYDXwCPAD4IKIGCwvTbPGzJgxIzc2S0mhl6HyRMRv5Wy7HLh8HNMxG3OHHHJIbmyWkolyN5RZy9m9e3dubJYSFwuzgixbtiw3NkuJi4WZmdXkYmFWkDVr1uTGZilxsTAryPHHH58bm6XExcKsILfeemtubJYSFwuzgmzcuDE3NkuJi4VZQaZNm5Ybm6XExcKsILNmzcqNzVLiYmFWkAULFuTGZilxsTAriPssrJW4WJgVxDPlWStxsTAryMKFC3Njs5S4WJgVZOnSpXR0dCCJjo4Oli5dWnZKZg0rbYhys1bX1dXF1VdfzZo1a1iyZIlnybOkuViYFaS3t5ePfexj7N27l56eHo499lgXDEtWmdOqflrSg9ksebdKmpWtl6SrJW3Ktr+zrBzNmrFq1Sr6+/uJCPr7+1m1alXZKZk1rMw+i89FxHERcTzwXeAvsvVnUJl3ewGwHPhySfmZNWX79u25sVlKSisWEfFcVfh6ILLlxcCqqPgxMFXSkeOeoFmTjjjiiNzYLCWl9llIuhxYCjwL/G62+ijgl1W7bcnWPTG+2Zk1Z/PmzbmxWUoKPbOQ9ENJD43wWgwQEZdFxBzgG8BHhz82wlfFCOuQtFxSn6S+HTt2FNMIswbdeeedubFZSgo9s4iIU+rc9QbgFuB/UTmTmFO1bTawbZTv7wa6ATo7O0csKGZlOfDAA3nxxRf3ic1SVebdUNWjqp0J/DRbvhlYmt0VdSLwbET4EpQl5z3veU9ubJaSMu+G+mx2SepB4PeAj2frvwdsBjYBXwXOLyk/s6asWLECqXJVVRIrVqwoOSOzxpXWwR0RS0ZZH8AF45yO2Zhbu3YtlV9niAjWrl3rh/IsWR4byqwg3/rWt3Jjs5S4WJgV5Oyzz86NzVLisaHMCnLllVeydetWvv/973PGGWdw5ZVXlp2SWcN8ZmFWkN7eXlavXs2uXbtYvXo1vb29Zadk1jAXC7OCrFy5kr179wKwd+9eVq5cWXJGZo1zsTAryLZt23Jjs5S4WJgVZNmyZbmxWUpcLMzMrCYXC7OCrFmzJjc2S4mLhVlBlixZkhubpcTPWZgVZPny5UDljGLJkiUvx2Yp0vDYNanr7OyMvr6+stMwM0uKpHsiorPWfr4MZWZmNblYmBWot7eXK664wk9vW/LcZ2FWkN7eXk4++WT6+/uZPHky69at8xDlliyfWZgVpKenhz179jA4OMiePXvo6ekpOyWzhpVeLCRdJCkkzchiSbpa0iZJD0p6Z9k5mjXisMMOY2hoCIChoSEOO+ywkjMya1ypxULSHOBU4BdVq88AFmSv5cCXS0jNrGk7d+6kra3yT6ytrY2dO3eWnJFZ48o+s7gKWAFU37+7GFgVFT8Gpko6spTszJqwaNEiOjo6mDRpEh0dHSxatKjslMwaVloHt6Qzga0R8cDwpPaZo4BfVsVbsnVPjGN6Zk3r6upi3bp19PT0sGjRInduW9IKLRaSfggcMcKmy4A/B35vpI+NsG7EJwclLadyqYq5c+c2mKVZcbq6ulwkrCUUWiwi4pSR1ks6FpgPDJ9VzAbulXQClTOJOVW7zwZGnAggIrqBbqg8wT12mZuZWbVS+iwiYn1EHB4R8yJiHpUC8c6I2A7cDCzN7oo6EXg2InwJysysRBPxobzvAe8DNgEvAn9YbjpmZjYhikV2djG8HMAF5WVjZmb7K/vWWTMzS4CLhZmZ1dQy81lI2gH8vOw8XqUZwFNlJzHO3ObXBrc5Hf8uImbW2qllikWKJPXVM+lIK3GbXxvc5tbjy1BmZlaTi4WZmdXkYlGu7rITKIHb/NrgNrcY91mYmVlNPrMwM7OaXCwKJmm6pNskbczep42y33nZPhslnTfC9pslPVR8xs1rps2SXifpFkk/lfSwpM+Ob/avjqTTJT2azex4yQjbOyTdlG2/S9K8qm2XZusflXTaeObdjEbbLOlUSfdIWp+9nzTeuTeimZ9xtn2upOclXTReORciIvwq8AWsBC7Jli8Brhxhn+nA5ux9WrY8rWr72cANwENlt6foNgOvA34322cycAdwRtltGqWdk4CfAUdnuT4ALNxvn/OBr2TL5wA3ZcsLs/07qIzA/DNgUtltKrjN7wBmZcvHUJnPpvQ2FdXequ1rgH8ELiq7Pc28fGZRvMXA9dny9cBZI+xzGnBbROyKiKeB24DTASQdBPwp8JlxyHWsNNzmiHgxIm4HiIh+4F4qw9RPRCcAmyJic5brjVTaXq36v8U3gZNVGZd/MXBjROyJiMeoDJx5wjjl3YyG2xwR90XE8HQDDwNTJHWMS9aNa+ZnjKSzqPwh9PA45VsYF4vivSGyIdaz98NH2Ge02QEBPg38Hyoj8Kai2TYDIGkq8PvAuoLybFbNNlTvExEDwLPAYXV+diJqps3VlgD3RcSegvIcKw23V9LrgYuBT41DnoWbEKPOpq7GjIB1fcUI60LS8cBvRcSf7H8dtGxFtbnq+9uBfwCujojNrz7DcVHPrI6j7VP3jJATTDNtrmyU3gZcycgzZU40zbT3U8BVEfH8flNHJ8nFYgzEKDMCAkj6laQjI+IJSUcCT46w2xZgUVU8G+gBuoDflvQ4lZ/V4ZJ6ImIRJSuwzcO6gY0R8fkxSLco9czqOLzPlqwAHgrsqvOzE1EzbUbSbODbwNKI+Fnx6Tatmfa+C/iApJXAVGBI0u6IuKb4tAtQdqdJq7+Az7FvZ+/KEfaZDjxGpYN3WrY8fb995pFOB3dTbabSP7MGaCu7LTXa2U7levR8ftP5+bb99rmAfTs/V2fLb2PfDu7NpNHB3Uybp2b7Lym7HePR3v32+SSJd3CXnkCrv6hcq10HbMzeh/+H2Al8rWq/D1Pp5NwE/OEI35NSsWi4zVT+cgtgA3B/9vrvZbcpp63vA/6Nyh0zl2Xr/hI4M1ueQuVOmE3A3cDRVZ+9LPvco0zQO77Gss3AJ4AXqn6u9wOHl92eIn/GVd+RfLHwE9xmZlaT74YyM7OaXCzMzKwmFwszM6vJxcLMzGpysTAzs5pcLMzqIGmqpPPLzsOsLC4WZvWZSmV00VeQNGmcczEbdy4WZvX5LPBGSfdL+pykRZJul3QDsF7SvOr5RiRdJOmT2fIbJf0gm8PhDklvqf5iSW3ZnB4zq+JNkmaMY/vMcrlYmNXnEuBnEXF8RPxZtu4EKk/0Lqzx2W7gYxHx28BFwJeqN0bEEPD3wLnZqlOAByLiqTHL3qxJHkjQrHF3R2UuilFl85H8B+Afq0YeHWkOh78BvgN8nsowKH87hnmaNc3FwqxxL1QtD7DvmfqU7L0NeCYijs/7ooj4ZTZa70lURis9N29/s/Hmy1Bm9fk1cHDO9l9RGUL+sGz2t/cDRMRzwGOSPgigireP8h1fo3I5anVEDI5d6mbNc7Ewq0NE7AT+RdJDkj43wva9VEYivQv4LvDTqs3nAsskPUBles39p+UcdjNwEL4EZROQR501myAkdVKZWe09Zeditj/3WZhNAJIuAf4Y91XYBOUzCzMzq8l9FmZmVpOLhZmZ1eRiYWZmNblYmJlZTS4WZmZWk4uFmZnV9P8BhGqYmlP6qMwAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "zero_nn_params = make_nn_params_as_list_of_dicts(n_hiddens_per_layer_list=[], n_dims_input=2, n_dims_output=1)\n", "yhat_N = predict_y_given_x_with_NN(x_ND, zero_nn_params)\n", "assert yhat_N.size == N\n", "\n", "plt.plot(yhat_N, y_N, 'k.')\n", "plt.xlabel('true y')\n", "plt.ylabel('predicted y|x')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup: Gradient descent implementation that can use list-of-dict parameters (not just arrays)" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [], "source": [ "def run_many_iters_of_gradient_descent_with_list_of_dict(f, g, init_x_list_of_dict=None, n_iters=100, step_size=0.001):\n", "\n", " # Copy the initial parameter vector\n", " x_list_of_dict = copy.deepcopy(init_x_list_of_dict)\n", "\n", " # Create data structs to track the per-iteration history of different quantities\n", " history = dict(\n", " iter=[],\n", " f=[],\n", " x=[],\n", " g=[])\n", " start_time = time.time()\n", " for iter_id in range(n_iters):\n", " if iter_id > 0:\n", " # Gradient is a list of layer-specific dicts\n", " grad_list_of_dict = g(x_list_of_dict)\n", " for layer_id, x_layer_dict in enumerate(x_list_of_dict):\n", " for key in x_layer_dict.keys():\n", " x_layer_dict[key] = x_layer_dict[key] - step_size * grad_list_of_dict[layer_id][key]\n", " \n", " fval = f(x_list_of_dict)\n", " history['iter'].append(iter_id)\n", " history['f'].append(fval)\n", " history['x'].append(copy.deepcopy(x_list_of_dict))\n", " history['g'].append(g(x_list_of_dict))\n", "\n", " if iter_id < 3 or (iter_id+1) % 50 == 0:\n", " print(\"completed iter %5d/%d after %7.1f sec | loss %.6e\" % (\n", " iter_id+1, n_iters, time.time()-start_time, fval))\n", " return x_list_of_dict, history" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Worked Exercise 5a: Train 0-layer NN via gradient descent on LINEAR toy data" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [], "source": [ "def nn_regression_loss_function(nn_params):\n", " yhat_N = predict_y_given_x_with_NN(x_ND, nn_params)\n", " return 0.5 / ag_np.square(sigma) * ag_np.sum(np.square(y_N - yhat_N))" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "completed iter 1/100 after 0.1 sec | loss 1.632803e+02\n", "completed iter 2/100 after 0.4 sec | loss 1.626805e+02\n", "completed iter 3/100 after 0.5 sec | loss 1.622965e+02\n", "completed iter 50/100 after 9.2 sec | loss 1.598157e+02\n", "completed iter 100/100 after 20.8 sec | loss 1.596883e+02\n" ] } ], "source": [ "fromtrue_opt_nn_params, fromtrue_history = run_many_iters_of_gradient_descent_with_list_of_dict(\n", " nn_regression_loss_function,\n", " autograd.grad(nn_regression_loss_function),\n", " true_nn_params,\n", " n_iters=100,\n", " step_size=0.000001)" ] }, { "cell_type": "code", "execution_count": 74, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Layer 0\n", " w | size (2, 1) | [ 4.20137427 -4.2023327 ]\n", " b | size (1,) | [0.08737985]\n" ] } ], "source": [ "pretty_print_nn_param_list(fromtrue_opt_nn_params)" ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 70, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X98VPWd7/HXJ0wSftgWhAjWH4tW0Ba11Bu5TAvtJGClsF7wUa/9LbvVotVatRdccJfSXmERr6tyt72uaKnKYwFZRerDrfjYDUSwjUBYxeJSl263rYASEBV/NAlkPvePOUOHmEkmkwlnMuf9fDzymJnvnHPmex5H5833x/mOuTsiIiJlYVdARESKgwJBREQABYKIiAQUCCIiAigQREQkoEAQERFAgSAiIgEFgoiIAAoEEREJxMKuQHcMGzbMR44cGXY1RET6lO3btx9096qututTgTBy5EgaGxvDroaISJ9iZr/PZTt1GYmICKBAEBGRgAJBREQABYKIiAQUCCIiAigQREQkEIlAaGhoYPHixTQ0NIRdFRGRotWn7kPIR0NDA4lEgiNHjtC/f3/q6uqIx+NhV0tEpOiUfAuhvr6eI0eO4O60trZSX18fdpVERIpSyQdCIpEgFks1hCoqKkgkEuFWSESkSJV8IMTjcebMmQPAihUr1F0kIpJFyQcCwIQJEwD46Ec/GnJNRESKVyQCYcSIEQC8/vrrIddERKR4KRBERASISCBUVVVhZuzfvz/sqoiIFK1IBEIsFqOqqkotBBGRTkQiEACGDx+uQBAR6URkAmHEiBEKBBGRTigQREQEiGAguHvYVRERKUo5BYKZLTezJjPb2a78RjN7xcxeNrM7g7JxZvZi8LfDzC7PcsyzzGyLme02s0fNrKLnp5PdiBEjaGlp4e233+7NjxER6bNybSE8BEzJLDCzGmA6cKG7jwHuCt7aCVS7+9hgn/vNrKNVVZcA97j7KOBN4OruVz936XsRNPVURKRjOQWCu28CDrUr/jZwh7u3BNs0BY/vu/vRYJv+wAf6aMzMgFrgsaDoYWBGt2vfDcOHDwd0c5qISDY9GUMYDUwMun2eNbOL02+Y2X83s5eBXwHXZQRE2lDgrYzyPcBpPahLl3S3sohI53oSCDFgCDAemAOsCf7lj7tvCbqRLgbmmVn/dvtaB8frcLTXzGaZWaOZNR44cCDvyioQREQ615NA2AOs9ZStQBIYlrmBu+8C3gPOb7fvQWBwxtjC6cC+jj7E3Ze5e7W7V1dVVeVd2SFDhlBeXq5AEBHJoieBsI7UOABmNhqoAA4Gs4diQfmfAecCv8vc0VNzPzcCVwRFM4Gf9aAuXSorK9PdyiIinch12ukqoAE418z2mNnVwHLg7GAq6mpgZvBFPwHYYWYvAk8A17v7weA4Pzez9I8S/BXwPTP7DakxhZ8U8sQ6opvTRESy62g66Ae4+1eyvPX1DrZdAazIcpypGc9/C4zL5fMLZfjw4ezb12HPlIhI5EXmTmVQC0FEpDORC4Smpiba2trCroqISNGJXCC0tbXxxhtvhF0VEZGiE7lAAN2LICLSEQWCiIgAEQsErWckIpJdpAJBLQQRkewiFQgnnXQSAwcO1BLYIiIdiFQgmJnuRRARySJSgQC6OU1EJJvIBUJ5eTk7d+6koaEh7KqIiBSVSAVCQ0MDv/jFL2hqamLSpEkKBRGRDJEKhPr6+mPLVrS2tlJfXx9uhUREikikAiGRSFBeXg5ALBYjkUiEWyERkSISqUCIx+Pcf//9AMyfP594PB5yjUREikekAgHgi1/8IpD6BTUREfmTyH0rfuhDH2LEiBHs3r077KqIiBSVyAUCwKhRo/iP//iPsKshIlJUIhkIo0ePVgtBRKSdLgPBzJabWZOZ7WxXfqOZvWJmL5vZnUHZJWa23cx+FTzWZjnmD8xsr5m9GPxN7Wi73jJq1Ciampo4fPjwifxYEZGilksL4SFgSmaBmdUA04EL3X0McFfw1kHgMne/AJgJrOjkuPe4+9jg7+fdrnkPjBo1CkCtBBGRDF0GgrtvAg61K/42cIe7twTbNAWPL7j7vmCbl4H+ZlZZwPoWhAJBROSD8h1DGA1MNLMtZvasmV3cwTZfBF5Ih0YHvmNmLwVdUkOyfZCZzTKzRjNrPHDgQJ7VPd4555wDoIFlEZEM+QZCDBgCjAfmAGvMzNJvmtkYYAlwbZb97wM+BowFXgP+LtsHufsyd6929+qqqqo8q3u8AQMGcMYZZ6iFICKSId9A2AOs9ZStQBIYBmBmpwNPAFe5+392tLO773f3NndPAg8A4/KsR95GjRqlQBARyZBvIKwDagHMbDRQARw0s8HAPwPz3P0X2XY2s1MzXl4O7My2bW9RIIiIHC+XaaergAbgXDPbY2ZXA8uBs4OpqKuBme7uwHeAc4D5GVNKTwmO86CZVQeHvTOYmvoSUAPcUvhT69zo0aM5dOgQb7zxxon+aBGRohTragN3/0qWt77ewbYLgYVZjnNNxvNv5FrB3pI502jo0KEh10ZEJHyRvFMZNPVURKS9yAbC2WefTVlZmQJBRCQQ2UCoqKhg5MiRCgQRkUBkAwGgqqqKTZs26beVRUSIcCA0NDSwfft29u3bx6RJkxQKIhJ5kQ2E+vp62traAGhtbaW+vj7cComIhCyygZBIJKioqACgX79+JBKJcCskIhKyyAZCPB5n/fr1lJWV8aUvfYl4PB52lUREQhXZQIBUK+Giiy5i7969YVdFRCR0kQ4EgHHjxrFt2zaSyWTYVRERCZUCYdw43nnnHV555ZWwqyIiEqrIB8LFF6d+22fr1q0h10REJFyRD4Rzzz2XD33oQwoEEYm8yAdCv379qK6uViCISORFPhAgNY6wY8cOmpubw66KiEhoFAikAuHIkSPs2LEj7KqIiIRGgUAqEEADyyISbQoE4LTTTuPUU09VIIhIpOXym8rLzawp+P3kzPIbzewVM3vZzO4Myi4xs+3B7yVvN7PaLMc82cz+xcx2B49DCnM6+TEzzjnnHNavX69VT0UksnJpITwETMksMLMaYDpwobuPAe4K3joIXObuFwAzgRVZjjkXqHP3UUBd8Do0DQ0NPP/88xw8eFBLYYtIZHUZCO6+CTjUrvjbwB3u3hJs0xQ8vuDu+4JtXgb6m1llB4edDjwcPH8YmJFH3QsmcynslpYWLYUtIpGU7xjCaGCimW0xs2fN7OIOtvki8EI6NNoZ7u6vAQSPp+RZj4JIJBJUVqZyy8y0FLaIRFK+gRADhgDjgTnAGjOz9JtmNgZYAlzb0wqa2SwzazSzxgMHDvT0cB2Kx+PU1dUxZswYhg4dyvjx43vlc0REilm+gbAHWOspW4EkMAzAzE4HngCucvf/zLL/fjM7Ndj+VKAp2we5+zJ3r3b36qqqqjyr27V4PM5NN91EU1MTu3bt6rXPEREpVvkGwjqgFsDMRgMVwEEzGwz8MzDP3X/Ryf5Pkhp0Jnj8WZ71KKgpU1Jj5+vXrw+5JiIiJ14u005XAQ3AuWa2x8yuBpYDZwdTUVcDM93dge8A5wDzzezF4O+U4DgPmll1cNg7gEvMbDdwSfA6dGeccQZjxozh6aefDrsqIiInnKW+x/uG6upqb2xs7NXPmD17Nn//93/PoUOHGDRoUK9+lojIiWBm2929uqvtdKdyO1OmTKG1tVVTT0UkchQI7UyYMIGBAwdqHEFEIkeB0E7//v0ZO3YsK1eu1B3LIhIpCoR2Ghoa2LZtG4cOHaK2tlahICKRoUBop76+nmQyCWgZCxGJFgVCO4lEgoqKimOvP/e5z4VYGxGRE0eB0E56GYvp06fj7pp6KiKRoUDoQDwe58EHHyQWi7Fy5cqwqyMickIoELIYNmwYl156KatWrTo2piAiUsoUCJ346le/yquvvspzzz0XdlVERHqdAqET06dPZ+DAgeo2EpFIiIVdgWI2aNAgZsyYwcqVKzn99NOZNGkS8Xg87GqJiPQKtRC68KlPfYp33nmHBQsW6PeWRaSkKRC60NKS+gXQZDKpRe9EpKQpELpQW1tLLJbqWSsvL9fvLYtIyVIgdCEej7N27VrKysqYMWOGxhBEpGQpEHJw2WWXccUVV/DMM8/w/vvvh10dEZFeoUDI0Q033MCbb77Jo48+GnZVRER6haad5mjixImMGTOGO++8k9dff51EIqHuIxEpKV22EMxsuZk1mdnOduU3mtkrZvaymd0ZlA01s41m9q6Z/aiTY/7AzPaa2YvB39Sen0rvMjOmTJnCr3/9a/7mb/5GU1BFpOTk0mX0EDAls8DMaoDpwIXuPga4K3irGZgPzM7huPe4+9jg7+e5Vzk8J510EqApqCJSmroMBHffBBxqV/xt4A53bwm2aQoe33P350gFQ8m59NJLNQVVREpWvoPKo4GJZrbFzJ41s4vzOMZ3zOyloEtqSLaNzGyWmTWaWeOBAwfyrG5hxONx1q1bRywWY/LkyRpDEJGSkm8gxIAhwHhgDrDGzKwb+98HfAwYC7wG/F22Dd19mbtXu3t1VVVVntUtnGnTpjFr1iyeeeYZ9u7dG3Z1REQKJt9A2AOs9ZStQBIYluvO7r7f3dvcPQk8AIzLsx6hmD17NslkknvuuSfsqoiIFEy+007XAbVAvZmNBiqAg7nubGanuvtrwcvLgZ2dbV9szjrrLL785S/z4x//mAEDBjB16lR1H4lIn5fLtNNVQANwrpntMbOrgeXA2cFU1NXATHf3YPvfAXcDfxFs/4mg/EEzqw4Oe6eZ/crMXgJqgFsKfWK9bcqUKTQ3N7No0SJNQRWRktBlC8Hdv5Llra9n2X5klvJrMp5/I5fKFbNXX30VAHc/NgVVrQQR6cu0dEWeEokE/fv3B1I3rWkKqoj0dQqEPMXjcTZs2MDYsWMpKytj5MiRYVdJRKRHFAg9EI/Heeyxx0gmk9x4440sXrxYYwki0mdpcbse+tjHPsa0adN4/PHHeeKJJ6isrKSurk7jCSLS56iFUADnnXceoDWORKRvUyAUwPTp04+tcRSLxTTALCJ9kgKhAOLxOE8//TSDBg3ivPPOY/z48WFXSUSk2xQIBTJ58mTuvvtuduzYwdq1a8OujohItykQCuib3/wm559/Pt/97ne5/fbbNeNIRPoUBUIBxWIxrrnmGvbt28eCBQu0pIWI9CkKhAJ7//33geOXtBAR6QsUCAWWuaRF+rWISF+gQCiw9JIWkydPpq2tjcOHD4ddJRGRnFiwanWfUF1d7Y2NjWFXIyctLS1ccMEFvP/++1x77bX6yU0RCY2ZbXf36q62Uwuhl1RWVnLDDTewd+9eDTCLSJ+gQOhFGmAWkb5EgdCLMgeY3Z2JEyeGXCMRkewUCL0oPcB85ZVXkkwm6SvjHyISTbn8pvJyM2sKfj85s/xGM3vFzF42szuDsqFmttHM3jWzH3VyzJPN7F/MbHfwOKTnp1Kc4vE4q1evZurUqcydO5c5c+ZoLEFEilIuLYSHgCmZBWZWA0wHLnT3McBdwVvNwHxgdhfHnAvUufsooC54XbLMjG9961u0tLRw1113aYBZRIpSl4Hg7puAQ+2Kvw3c4e4twTZNweN77v4cqWDozHTg4eD5w8CM7lS6L9q1axdmBkBzc7MGmEWk6OQ7hjAamGhmW8zsWTO7uJv7D3f31wCCx1PyrEef0X6A+cwzzwy5RiIix8s3EGLAEGA8MAdYY+l//haYmc0ys0Yzazxw4EBvfMQJEY/Hqaur47bbbmPw4MH88Ic/ZOHCheo6EpGikW8g7AHWespWIAkM68b++83sVIDgsSnbhu6+zN2r3b26qqoqz+oWh3g8zqJFi5g7dy67d+/m+9//vsYTRKRo5BsI64BaADMbDVQAB7ux/5PAzOD5TOBnedajT0omk5gZ7k5LS4vGE0SkKOQy7XQV0ACca2Z7zOxqYDlwdjAVdTUw04NFkczsd8DdwF8E238iKH/QzNJradwBXGJmu4FLgteRkTmekEwmufDCC0OukYiIFrcLTUNDA48++ij33Xcfn/zkJ5kxYwY1NTVaAE9ECi7Xxe1iJ6Iy8kHxeJx4PE7//v1ZsmQJjY2N9O/fn7q6OoWCiIRCS1eE7MMf/jCAxhNEJHQKhJDV1NQwYMAAIDWecN5554VcIxGJKgVCyNL3J9xyyy0MHDiQ2267jdtvv11TUUXkhNOgchFZsmQJc+fOxcw0niAiBaNfTOuDdH+CiIRJgVBE2t+fUFlZGXKNRCRKFAhFJD2esGDBAkaNGsX8+fO5+eabNZ4gIieExhCK1Lp167j88ssB6N+/Pxs2bNB4gojkRWMIfdyuXbsoK0tdnubmZp555pmQayQipU53KhepRCJBZWUlLS0tJJNJ/umf/olYLMakSZPUUhCRXqEuoyLW0NBAfX09u3btYsWKFQAMGDBA01FFpFvUZVQC4vE48+bN4+Mf//hxP7+5cePGkGsmIqVIgdAHpKejpu9R+O1vfxt2lUSkBCkQ+oD0dNSFCxcydepUfvKTnzBt2jRNRxWRgtIYQh/z3HPPkUgkaGtro7y8nGeffVbjCSLSKY0hlKjNmzcfe37kyBH+9m//NsTaiEgp0bTTPiaRSFBRUUFraysATz31FDfffDPDhw8nkUiotSAieVMg9DHp8YT6+nomTJjA97//fZYuXaoVUkWkx7rsMjKz5WbWZGY725XfaGavmNnLZnZnRvk8M/tN8N6lWY75kJn9l5m9GPyN7fmpREd6OurEiROZNGkSkPrFNU1JFZGeyGUM4SFgSmaBmdUA04EL3X0McFdQ/gngy8CYYJ//Z2b9shx3jruPDf5ezLP+kTdp0iQGDBhwbErq+vXrWbRokWYgiUi3ddll5O6bzGxku+JvA3e4e0uwTVNQPh1YHZT/l5n9BhgH6Nupl6S7kDZu3MjmzZtZv349mzdv1h3NItJt+c4yGg1MNLMtZvasmV0clJ8GvJqx3Z6grCOLzOwlM7vHzLIu/G9ms8ys0cwaDxw4kGd1S1s8Hue2227js5/97LE7mv/4xz9y//33s3jxYrUWRCQn+Q4qx4AhwHjgYmCNmZ0NWAfbdnSjwzzgdaACWAb8FfC/O/ogd18WbEN1dXXfuWkiBOk7mtML4j3yyCOYGZWVlWotiEiX8m0h7AHWespWIAkMC8rPyNjudGBf+53d/bVg3xbgp6S6laSHMu9onjFjBu5OMpnUz3GKSE7yDYR1QC2AmY0m9S/9g8CTwJfNrNLMzgJGAVvb72xmpwaPBswAdrbfRvKTnoF06623HvdznDt37lT3kYh0qssuIzNbBSSAYWa2B1gALAeWB1NRW4GZnloD42UzWwP8O3AUuMHd24Lj/By4xt33Af9oZlWkupheBK4r+JlFXDweZ8OGDaxfv54nnniClStX6l4FEemU1jKKgEWLFjF//nzS1/prX/saY8aM0Z3NIhGR61pGCoQIaGhoYNKkSccGmwHKyso02CwSEbkGgpauiIDM5S5eeuklVq9eTTKZpLm5mZ/+9KfU19ertSAiaiFETbq10NzcjLtjZpqaKlLitPy1dCjdWli0aBGf//znj5uaqnWQRKJNXUYRFI/HicfjJBIJNm3aRHNzM8lkktWrV/Puu+9y2WWXqaUgEkHqMoq4hoYGNm7cyI4dO1izZg0AFRUVbNy4kU9/+tMh105ECkFdRpKT9DpIY8eOpaws9Z9Da2srV155JbfeeqtuZBOJELUQBPjTYHP6l9ja2toAKC8v59577+Xtt9/WTCSRPkr3IUi3NTQ0UF9fzx/+8AceeOCBY6EAum9BpC9TIEjesrUWAL761a9y/vnnq7Ug0ocoEKRH0q2FoUOHcvPNNx+7bwHQmkgifYwCQQomHQ67du1ixYoVx8rj8Thf+MIXmDx5soJBpIgpEKTgMruSksnksRZDeXk5S5cu5a233lJXkkgRUiBIr8gceF62bNmxxfJAXUkixUqBIL2qs4HnCRMm8PnPf15dSSJFQoEgva79wHNLSwvuflxX0pIlS2hublZXkkiIFAhyQqXD4fe//z0PPPDAB7qSKisr2bBhg0JBJAQKBAlFZleSmXH06NFj751zzjnMmjWL1tZWamtrFQ4iJ0hBA8HMlgN/DjS5+/kZ5TcC3yH1+8n/7O63BuXzgKuBNuC77v5MB8c8C1gNnAz8G/ANd2/trB4KhL6hfVdSOhwGDRrE22+/DUAsFmPhwoUkk0l1J4n0skIHwmeBd4FH0oFgZjXAXwPT3L3FzE5x9yYz+wSwChgHfBT4V2C0u7e1O+YaYK27rzazfwB2uPt9ndVDgdD3pMMhkUhQV1fHggULjutOgtTqqnfffTeHDx9WOIj0goJ3GZnZSOCpjEBYAyxz939tt908AHdfHLx+BviBuzdkbGPAAWCEux81s3iwzaWd1UGB0Ld11p0EqbGGiooKli5dyqFDhxQOIgVyIn5TeTQw0cwWAc3AbHffBpwGPJ+x3Z6gLNNQ4C13P9rJNlJiMn/bObM7CVLTVt2dlpYWrrvuOoWDSAh6EggxYAgwHrgYWGNmZwPWwbbtmyG5bJPa0GwWMAvgzDPPzLuyUhzSv9YGcMEFF3QrHO655x7dDS3Si3oSCHtIjQE4sNXMksCwoPyMjO1OB/a12/cgMNjMYkEroaNtAHD3ZcAySHUZ9aC+UmS6Gw7XX389kBpzuOOOO3R/g0iB9SQQ1gG1QL2ZjQYqSH3RPwmsNLO7SQ0qjwK2Zu7o7m5mG4ErSM00mgn8rAd1kT4u13CA1C+6fe973wNSs5XmzJnDSSedRE1NjcJBpAdynWW0CkiQagHsBxYAK4DlwFigldQYwoZg+78GvklqOurN7v50UP5z4Bp33xd0L6Wnnb4AfN3dWzqrhwaVo6ejKaxw/FIZaf369ePKK69kxIgRXHHFFZjZsRlOCgqJMt2YJiUn2/0N6W6l9srKynB3jT9I5CkQpKRlC4dkMvmB+xwyxWIxrr32WgYPHsy0adMA1IqQkqdAkMjoKBz69et33L0OHXUxwfGtiMcff5yTTz75WECAwkJKgwJBIinzzmigW11MmdKBkkwmqays5N577+WNN95QUEifpEAQydBVKyIWi3HRRRfR0NDQ6XEUFNIXKRBEsuioFZF+nl5ao1+/fgDHupw6G5eAVNeTmeHulJeXs3jx4mP3SbT/DIWGnGgKBJE8dNXl1N2ggD+NU5SVlVFWVkZbW9uxmU9vvvmmQkN6nQJBpIByCYrMQexkMtnlOEWamR1rXaRbGslkkvLychYuXEhLSwu1tbXHPjcdFJl1UnBIZxQIIidAd4Iin9ZFpnRo9OvXj4kTJ/Lcc8/R1tZGeXk5119/PUePHqWmpoaPfOQjbNmyhZqammN16qwVkvlcwVKaFAgiIepsnCLX0Ei3FDr6fzQWi31g+fCOpEOko1ZIW1vbsc9OB8v8+fP54x//yOTJk6moqODZZ5/NOUwyn6sFU1wUCCJFLt/QqKio4N577+1wKm36S7/Q2gdLMpk8rnsrM2TKy8u56qqreOSRRzh69Cjl5eXMnj2b9957jwkTJlBRUcG2bduora2lvLycTZs25RU6hXoehfBSIIiUgGyhkfkl1llrI5fn7k5bW1unLZITLTOAgA7DKF2efp5Z3j6ksoVXLBbjuuuu4/777+fIkSOUl5dzyy23cPjwYcaNG0e/fv3Yvn0748ePp6ysjG3btvGZz3yGsrIynn/+eSZOnEhZWRm//OUv+dznPkdZWRmbN2+mpqaGsrKyogksBYJIhHTV2sjleU+DJf28vLycuXPnsnjxYo4cOXJc0PRWC6ZUmRmnnHIKTU1NmBmVlZXU1dV1OxQUCCLSbYUIlkK3YAr1HDg2bpJ+HovFuOmmm1i6dGmn4ZXL896QDoT9+/cDqRsjb7/9dubNm9fd4ygQRKQ4FCpoeuN5MYZXtvGiiooKtRDSFAgi0tuKLbw0hpCFAkFEpPtyDYSyE1EZEREpfgoEEREBFAgiIhLoMhDMbLmZNZnZzoyyH5jZXjN7MfibGpRXmNlPzexXZrbDzBJZjtnh/iIiEp5YDts8BPwIeKRd+T3ufle7sm8BuPsFZnYK8LSZXezuHa3i1dH+IiISki5bCO6+CTiU4/E+AdQF+zUBbwFdjmyLiEj4cmkhZPMdM7sKaAT+l7u/CewAppvZauAM4L8Fj1tz3P8DzGwWMCt4+a6ZvZJnfYcBB/Pcty+L4nlH8ZwhmucdxXOG7p/3n+WyUU73IZjZSOApdz8/eD08qIwDtwOnuvs3zSwG/B+gBvg9UA7c7+4/a3e8DvfP6bTyZGaNuczDLTVRPO8onjNE87yjeM7Qe+edVwvB3fenn5vZA8BTQflR4JaM934J7M51fxERCU9e007N7NSMl5cDO4PygWY2KHh+CXDU3f891/1FRCQ8XbYQzGwVkACGmdkeYAGQMLOxpLp8fgdcG2x+CvCMmSWBvcA3Mo7zIPAP7t4I3Jll/9607AR8RjGK4nlH8ZwhmucdxXOGXjrvPrWWkYiI9B7dqSwiIkBEAsHMppjZK2b2GzObG3Z9eoOZnWFmG81sl5m9bGY3BeUnm9m/mNnu4HFI2HUtNDPrZ2YvmNlTweuzzGxLcM6PmllF2HUsNDMbbGaPmdmvg2seL/VrbWa3BP9t7zSzVWbWvxSvdZbVITq8tpbyf4PvtpfM7KKefHbJB4KZ9QN+DHyB1I1zXzGzT4Rbq15xlNT9HB8HxgM3BOc5F6hz91GkbhosxUC8CdiV8XoJqTvhRwFvAleHUqvetRRY7+7nAZ8kdf4le63N7DTgu0B1MP29H/BlSvNaPwRMaVeW7dp+ARgV/M0C7uvJB5d8IADjgN+4+2/dvRVYDUwPuU4F5+6vufu/Bc/fIfUFcRqpc3042OxhYEY4NewdZnY6MA14MHhtQC3wWLBJKZ7zh4HPAj8BcPdWd3+LEr/WpCbBDAjudxoIvEYJXussq0Nku7bTgUc85XlgcLtZnN0ShUA4DXg14/WeoKxkBTcSfgrYAgx399cgFRqkZoKVknuBW4H0ellDgbeCe2KgNK/32cAB4KdBV9mDwXTvkr3W7r4XuAv4A6kgeBuUoWdbAAAB7UlEQVTYTulf67Rs17ag329RCATroKxkp1aZ2UnA48DN7n447Pr0JjP7c6DJ3bdnFnewaald7xhwEXCfu38KeI8S6h7qSNBnPh04C/goMIhUd0l7pXatu1LQ/96jEAh7SK2nlHY6sC+kuvQqMysnFQb/6O5rg+L96SZk8NgUVv16wWeA/2FmvyPVFVhLqsUwOOhWgNK83nuAPe6+JXj9GKmAKOVrPRn4L3c/4O5HgLXApyn9a52W7doW9PstCoGwDRgVzEaoIDUQ9WTIdSq4oO/8J8Aud787460ngZnB85nAz9rv21e5+zx3P93dR5K6rhvc/WvARuCKYLOSOmcAd38deNXMzg2KJgH/Tglfa1JdReOD1RCMP51zSV/rDNmu7ZPAVcFso/HA2+mupXxE4sY0S/0Az72kZiYsd/dFIVep4MxsArAZ+BV/6k+/jdQ4whrgTFL/U/1Pd891OfM+w1I/xjTb3f/czM4m1WI4GXgB+Lq7t4RZv0IL7vR/EKgAfgv8Jal/4JXstTazHwJfIjWj7gXgGlL95SV1rTNXhwD2k1odYh0dXNsgHH9EalbS+8BfBqtB5PfZUQgEERHpWhS6jEREJAcKBBERARQIIiISUCCIiAigQBARkYACQUREAAWCiIgEFAgiIgLA/wfnrZDXTf4chQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.plot(fromtrue_history['iter'], fromtrue_history['f'], 'k.-')" ] }, { "cell_type": "code", "execution_count": 71, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "completed iter 1/100 after 0.1 sec | loss 4.649827e+06\n", "completed iter 2/100 after 0.3 sec | loss 2.520065e+06\n", "completed iter 3/100 after 0.6 sec | loss 1.367394e+06\n", "completed iter 50/100 after 9.4 sec | loss 1.634750e+02\n", "completed iter 100/100 after 20.2 sec | loss 1.598624e+02\n" ] } ], "source": [ "fromzero_opt_nn_params, fromzero_history = run_many_iters_of_gradient_descent_with_list_of_dict(\n", " nn_regression_loss_function,\n", " autograd.grad(nn_regression_loss_function),\n", " zero_nn_params,\n", " n_iters=100,\n", " step_size=0.000001)" ] }, { "cell_type": "code", "execution_count": 72, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Layer 0\n", " w | size (2, 1) | [ 4.20137256 -4.2023501 ]\n", " b | size (1,) | [0.08325989]\n" ] } ], "source": [ "pretty_print_nn_param_list(fromzero_opt_nn_params)" ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 75, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAD8CAYAAACyyUlaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAGJJJREFUeJzt3X2QXXWd5/H3t5M0tzNT0gECsknHMDXRlRHHh5jQ62o1ZEaDY02kSty4o1AWa6qmmFK3xppBS4vV4UGqtpBBGUsKWHFKQEqtNWUxi6lgF7tFGwgyK2DWJQszIQISHlUeuunku3/c06HTuc+3b04/vF9VXX3O7/zOPd9TB/qT3++ce29kJpIkdaOv7AIkSfOfYSJJ6pphIknqmmEiSeqaYSJJ6pphIknqmmEiSeqaYSJJ6pphIknq2tKyCzhWTjrppFy7dm3ZZUjSvHLfffc9nZkrm/VbNGGydu1adu/eXXYZkjSvRMS/ttLPaS5JUtcME0lS1wwTSVLXDBNJUtcME0lS1wwTSVLXDJMmxsbGuOKKKxgbGyu7FEmasxbN+0w6MTY2xllnncXExASVSoWdO3cyPDxcdlmSNOc4MmlgdHSUiYkJMpOJiQlGR0fLLkmS5iTDpIGRkRGWLq0O3pYtW8bIyEi5BUnSHGWYNDA8PMxnP/tZAG6++WanuCSpDsOkiTPOOAOA008/veRKJGnuMkyaqFQqALz88sslVyJJc5dh0sTAwAAAr7zySsmVSNLcZZg0MRUmjkwkqT7DpAmnuSSpOcOkCae5JKk5w6QJp7kkqTnDpAnDRJKaM0ya8J6JJDVnmDThPRNJas4wacKRiSQ1Z5g0sWTJEpYtW2aYSFIDhkkLBgYGnOaSpAYMkxYMDAw4MpGkBgyTFlQqFcNEkhowTFrgNJckNWaYtMBpLklqzDBpgdNcktSYYdICp7kkqTHDpAVOc0lSY4ZJC5zmkqTGWg6TiFgSEfdHxI+K9dMiYldEPBwR342I/qL9uGJ9b7F97bTX+FzR/suIeP+09s1F296IuHhae9vH6AVHJpLUWDsjk08De6atXwl8NTPXAc8BFxbtFwLPZeYfAl8t+hERpwNbgT8CNgP/UATUEuBa4BzgdOCjRd+2j9Er3jORpMZaCpOIWA38GXB9sR7A2cD3ii43AR8qlrcU6xTbNxX9twC3ZuZ4Zj4K7AU2FD97M/ORzJwAbgW2dHiMnnCaS5Iaa3VkcjXwN8ChYv1E4PnMnCzW9wOriuVVwGMAxfYXiv6H22fsU6+9k2P0hNNcktRY0zCJiA8CT2XmfdOba3TNJttmq73Z8Q+LiG0RsTsidh84cKDGLq0ZGBhgfHyczKMOIUmitZHJu4E/j4h/oToFdTbVkcpgRCwt+qwGHi+W9wNDAMX244Fnp7fP2Kde+9MdHOMImXldZq7PzPUrV65s4VRr8wuyJKmxpmGSmZ/LzNWZuZbqDfQ7M/MvgJ8AHy66XQD8sFjeXqxTbL8zq/+k3w5sLZ7EOg1YB9wD3AusK57c6i+Osb3Yp91j9IRfkCVJjS1t3qWuvwVujYhLgfuBG4r2G4B/jIi9VEcLWwEy86GIuA34BTAJXJSZBwEi4q+AO4AlwI2Z+VAnx+gVRyaS1FhbYZKZo8BosfwI1SexZvZ5BTivzv6XAZfVaL8duL1Ge9vH6IWpMHFkIkm1+Q74FjjNJUmNGSYtcGQiSY0ZJi3wnokkNWaYtMBpLklqzDBpgdNcktSYYdICp7kkqTHDpAVOc0lSY4ZJC5zmkqTGDJMWOM0lSY0ZJi1wmkuSGjNMWtDf309EGCaSVIdh0oKI8AuyJKkBw6RFfg+8JNVnmLTI74GXpPoMkxY5zSVJ9RkmLXKaS5LqM0xa5DSXJNVnmLTIaS5Jqs8waZHTXJJUn2HSIqe5JKk+w6RFTnNJUn2GSYsME0mqzzBpUaVS8Z6JJNVhmLTIkYkk1WeYtGgqTDKz7FIkac4xTFpUqVQ4dOgQk5OTZZciSXOOYdIiv7pXkuozTFpkmEhSfYZJi6a+utcnuiTpaIZJixyZSFJ9hkmLDBNJqs8waZFhIkn1GSYt8p6JJNVnmLTIkYkk1dc0TCKiEhH3RMT/joiHIuJLRftpEbErIh6OiO9GRH/RflyxvrfYvnbaa32uaP9lRLx/Wvvmom1vRFw8rb3tY/SKYSJJ9bUyMhkHzs7MPwbeBmyOiDOBK4GvZuY64DngwqL/hcBzmfmHwFeLfkTE6cBW4I+AzcA/RMSSiFgCXAucA5wOfLToS7vH6CWnuSSpvqZhklW/K1aXFT8JnA18r2i/CfhQsbylWKfYvikiomi/NTPHM/NRYC+wofjZm5mPZOYEcCuwpdin3WP0jCMTSaqvpXsmxQjin4GngB3A/wOez8ypD6raD6wqllcBjwEU218ATpzePmOfeu0ndnCMmXVvi4jdEbH7wIEDrZxqXYaJJNXXUphk5sHMfBuwmupI4s21uhW/a40QchbbGx3jyIbM6zJzfWauX7lyZY1dWuc0lyTV19bTXJn5PDAKnAkMRsTSYtNq4PFieT8wBFBsPx54dnr7jH3qtT/dwTF6xpGJJNXXytNcKyNisFgeAP4E2AP8BPhw0e0C4IfF8vZinWL7nVn9EpDtwNbiSazTgHXAPcC9wLriya1+qjfptxf7tHuMnunr66O/v98wkaQaljbvwqnATcVTV33AbZn5o4j4BXBrRFwK3A/cUPS/AfjHiNhLdbSwFSAzH4qI24BfAJPARZl5ECAi/gq4A1gC3JiZDxWv9bftHKPXKpWKYSJJNTQNk8z8OfD2Gu2PUL1/MrP9FeC8Oq91GXBZjfbbgdtn4xi9NDAw4D0TSarBd8C3we+Bl6TaDJM2OM0lSbUZJm1wmkuSajNM2uA0lyTVZpi0wWkuSarNMGmD01ySVJth0ganuSSpNsOkDYaJJNVmmLShUqk4zSVJNRgmbXBkIkm1GSZtMEwkqTbDpA2VSoWJiQkOHjxYdimSNKcYJm2Y+k6T8fHxkiuRpLnFMGnDk08+CcBdd91VciWSNLcYJi0aGxvj2muvBeDcc89lbGys5Iokae4wTFo0Ojp6+F7Jq6++yujoaLkFSdIcYpi0aGRkhGXLlgGwdOlSRkZGyi1IkuYQw6RFw8PDfO1rXwPgsssuY3h4uOSKJGnuMEza8N73vheAU089teRKJGluMUzaMDg4CMDzzz9fciWSNLcYJm04/vjjAcNEkmYyTNpQqVSoVCqGiSTNYJi0aXBw0DCRpBkMkzYZJpJ0NMOkTYaJJB3NMGmTYSJJRzNM2mSYSNLRDJM2GSaSdDTDpE1TYZKZZZciSXOGYdKmwcFBXn31Vb++V5KmMUza5EeqSNLRDJM2GSaSdDTDpE0rVqwADBNJmq5pmETEUET8JCL2RMRDEfHpov2EiNgREQ8Xv1cU7RER10TE3oj4eUS8Y9prXVD0fzgiLpjW/s6IeKDY55qIiE6P0WuOTCTpaK2MTCaBv87MNwNnAhdFxOnAxcDOzFwH7CzWAc4B1hU/24BvQDUYgEuAjcAG4JKpcCj6bJu23+aiva1jHAtTYfLcc88dq0NK0pzXNEwy84nM/Fmx/FtgD7AK2ALcVHS7CfhQsbwF+HZW/RQYjIhTgfcDOzLz2cx8DtgBbC62vS4zx7L6vO23Z7xWO8foOUcmknS0tu6ZRMRa4O3ALuCUzHwCqoEDnFx0WwU8Nm23/UVbo/b9Ndrp4Bg953eaSNLRWg6TiPh94PvAZzLzN4261mjLDtobltPKPhGxLSJ2R8TuAwcONHnJ1hx33HEMDAwYJpI0TUthEhHLqAbJdzLzB0Xzr6emlorfTxXt+4GhabuvBh5v0r66RnsnxzhCZl6Xmeszc/3KlStbOdWW+JEqknSkVp7mCuAGYE9mXjVt03Zg6omsC4AfTms/v3ji6kzghWKK6g7gfRGxorjx/j7gjmLbbyPizOJY5894rXaOcUwYJpJ0pKUt9Hk38HHggYj456Lt88BXgNsi4kJgH3Bese124APAXuAl4BMAmflsRPwdcG/R78uZ+Wyx/JfAt4AB4J+KH9o9xrFimEjSkZqGSWb+L2rfowDYVKN/AhfVea0bgRtrtO8G3lKj/Zl2j3EsDA4OMlv3YCRpIfAd8B1wZCJJRzJMOmCYSNKRDJMO+J0mknQkw6QDg4ODTE5O8tJLL5VdiiTNCYZJB/zkYEk6kmHSAT+fS5KOZJh0wDCRpCMZJh0wTCTpSIZJBwwTSTqSYdIBw0SSjmSYdMDvNJGkIxkmHejv72f58uWGiSQVDJMODQ4O+j3wklQwTDrk53NJ0msMkw4ZJpL0GsOkQ4aJJL3GMOmQYSJJrzFMOmSYSNJrDJMOrVixwu80kaSCYdKhwcFBDh48yIsvvlh2KZJUOsOkQ8888wwAO3fuLLkSSSqfYdKBsbExrrrqKgC2bt3K2NhYyRVJUrkMkw6Mjo4yOTkJwMTEBKOjo+UWJEklM0w6MDIyQn9/PwBLlixhZGSk3IIkqWSGSQeGh4f58Y9/TETwsY99jOHh4bJLkqRSGSYdes973sOqVas4dOhQ2aVIUukMky6sWbOGxx57rOwyJKl0hkkXhoaG2LdvX9llSFLpDJMuTI1MfBe8pMXOMOnCmjVrGB8f58CBA2WXIkmlMky6MDQ0BOBUl6RFzzDpwpo1awDDRJIMky5MhYlPdEla7AyTLpxwwgkMDAw4MpG06DUNk4i4MSKeiogHp7WdEBE7IuLh4veKoj0i4pqI2BsRP4+Id0zb54Ki/8MRccG09ndGxAPFPtdERHR6jGMtIlizZo1hImnRa2Vk8i1g84y2i4GdmbkO2FmsA5wDrCt+tgHfgGowAJcAG4ENwCVT4VD02TZtv82dHKMsQ0NDTnNJWvSahklm3gU8O6N5C3BTsXwT8KFp7d/Oqp8CgxFxKvB+YEdmPpuZzwE7gM3Fttdl5lhW36zx7Rmv1c4xSuHIRJI6v2dySmY+AVD8PrloXwVM/2f6/qKtUfv+Gu2dHOMoEbEtInZHxO5evRdkzZo1PPnkk0xMTPTk9SVpPpjtG/BRoy07aO/kGEc3Zl6Xmeszc/3KlSubvGxnhoaGyEx+9atf9eT1JWk+6DRMfj01tVT8fqpo3w8MTeu3Gni8SfvqGu2dHKMUvtdEkjoPk+3A1BNZFwA/nNZ+fvHE1ZnAC8UU1R3A+yJiRXHj/X3AHcW230bEmcVTXOfPeK12jlEK3wUvSbC0WYeIuAUYAU6KiP1Un8r6CnBbRFwI7APOK7rfDnwA2Au8BHwCIDOfjYi/A+4t+n05M6du6v8l1SfGBoB/Kn5o9xhlmQoTn+iStJg1DZPM/GidTZtq9E3gojqvcyNwY4323cBbarQ/0+4xyrB8+XJOOukkRyaSFjXfAT8L/F4TSYudYTIL/MZFSYudYTILfOOipMXOMJkFQ0ND/OY3v+GSSy5hbGys7HIk6ZgzTGbBK6+8AsCll17Kpk2bDBRJi45hMgueeKL6NpdDhw4xMTHB6OhouQVJ0jFmmMyCj3zkI0D1I+n7+/sZGRkptyBJOsaavs9EzY2MjPCmN72JiYkJvvOd7zA8PFx2SZJ0TDkymSWbNm3iwIEDbNiwoexSJOmYM0xmycaNG/nd737Hnj17yi5Fko45w2SWTI1I7rnnnpIrkaRjzzCZJW984xs5/vjj2bVrV9mlSNIxZ5jMkr6+PjZs2GCYSFqUDJNZtGHDBh588EFefPHFskuRpGPKMJlFGzdu5ODBg/zsZz8ruxRJOqYMk1k0dRPeqS5Ji41hMotOOeUU3vCGN/hEl6RFxzCZZRs3bnRkImnRMUxm2caNG9m3bx+f//zn/fRgSYuGYTLLli9fDsCVV17px9FLWjQMk1n29NNPA34cvaTFxTCZZZs2bWLp0uqHMS9btsyPo5e0KBgms2x4eJibb74ZgPPPP9+Po5e0KBgmPXDeeedx1llnsWPHDg4dOlR2OZLUc4ZJj3zyk5/k0Ucf5c477yy7FEnqOcOkR84991xOOOEErr/++rJLkaSeM0x6pFKp8PGPf5zvf//7fOELX/ARYUkLmmHSQ+9617uYnJzk8ssv9z0nkhY0w6SH9u3bR0SQmYyPj/ueE0kLlmHSQyMjI1QqFaD6Jsa3vvWtJVckSb1hmPTQ8PAwO3fu5FOf+hTLli3j0ksv5fLLL3e6S9KCs7TsAha64eFhhoeHWbp0KVdddRW7du2iUqmwc+dO39AoacFwZHKMnHjiiYfvn7z88st8/etf54orrnCUImlBmLcjk4jYDPw9sAS4PjO/UnJJDZ111llUKhXGx8c5dOgQN998MxFBpVLh6quv5plnnmFkZMTRiqR5KTKz7BraFhFLgP8L/CmwH7gX+Ghm/qLePuvXr8/du3cfowprGxsbY3R0lAceeIBbbrnlcHtfX3WAeNxxxx0RLACjo6OGjKTSRMR9mbm+ab95GibDwH/JzPcX658DyMwr6u0zF8JkytjYGJs2bWJ8fJzMZPo1iAigGjARwaFDh1i2bBlf/OIXeeWVVzj77LPp7+/nrrvuOipwWlkeHh4+HGqd7N/r5ble33yqda7XN59qXSj1dfKP0lbD5PAfs/n0A3yY6tTW1PrHga832ued73xnziV33313Xn755fnNb34zBwYGsq+vL/v6+hKY9Z+IOPz75JNPPrw+fdtcWI6IfP3rX196HQuh1rle33yqdSHU19fXlwMDA3n33Xe3/bcK2J0t/F2er/dMokZbHtUpYhuwDWDNmjW9rqktU095AZxxxhmMjo5y4okn8pnPfIaJiQmWLFkCwOTk5OERSnY4ipy+X19f3xHrc2l5+n+Yc6Wm+VrrXK9vPtW6EOrLzMNf1terKfP5Gib7gaFp66uBx2d2yszrgOugOs11bEprX61gmT48nRkyEcHk5GTby/39/XzpS1+aldfqxXJ/fz9f/vKX52x986nWuV7ffKp1odTX399/+O9KL8zXMLkXWBcRpwG/ArYC/7HckmbH9GCZWofaIdPp3OpsvVYvlud6ffOp1rle33yqdaHU18sHeeblDXiAiPgAcDXVR4NvzMzLGvWfSzfgJWm+iBZvwM/XkQmZeTtwe9l1SJJ8B7wkaRYYJpKkrhkmkqSuGSaSpK4ZJpKkrs3bR4PbFREHgH/tcPeTgKdnsZz5YjGe92I8Z1ic570YzxnaP+83ZObKZp0WTZh0IyJ2t/Kc9UKzGM97MZ4zLM7zXoznDL07b6e5JEldM0wkSV0zTFpzXdkFlGQxnvdiPGdYnOe9GM8ZenTe3jORJHXNkYkkqWuGSRMRsTkifhkReyPi4rLr6YWIGIqIn0TEnoh4KCI+XbSfEBE7IuLh4veKsmudbRGxJCLuj4gfFeunRcSu4py/GxH9Zdc42yJiMCK+FxH/p7jmw4vkWv/n4r/vByPiloioLLTrHRE3RsRTEfHgtLaa1zaqrin+tv08It7RzbENkwYiYglwLXAOcDrw0Yg4vdyqemIS+OvMfDNwJnBRcZ4XAzszcx2ws1hfaD4N7Jm2fiXw1eKcnwMuLKWq3vp74H9k5r8F/pjq+S/oax0Rq4BPAesz8y1Uv7piKwvven8L2Dyjrd61PQdYV/xsA77RzYENk8Y2AHsz85HMnABuBbaUXNOsy8wnMvNnxfJvqf5xWUX1XG8qut0EfKicCnsjIlYDfwZcX6wHcDbwvaLLQjzn1wHvBW4AyMyJzHyeBX6tC0uBgYhYCiwHnmCBXe/MvAt4dkZzvWu7Bfh2Vv0UGIyIUzs9tmHS2CrgsWnr+4u2BSsi1gJvB3YBp2TmE1ANHODk8irriauBvwEOFesnAs9n5mSxvhCv9x8AB4D/VkzvXR8Rv8cCv9aZ+SvgvwL7qIbIC8B9LPzrDfWv7az+fTNMGosabQv28beI+H3g+8BnMvM3ZdfTSxHxQeCpzLxvenONrgvtei8F3gF8IzPfDrzIApvSqqW4T7AFOA34N8DvUZ3mmWmhXe9GZvW/d8Oksf3A0LT11cDjJdXSUxGxjGqQfCczf1A0/3pq2Fv8fqqs+nrg3cCfR8S/UJ2+PJvqSGWwmAaBhXm99wP7M3NXsf49quGykK81wJ8Aj2bmgcx8FfgB8O9Y+Ncb6l/bWf37Zpg0di+wrnjio5/qDbvtJdc064p7BTcAezLzqmmbtgMXFMsXAD881rX1SmZ+LjNXZ+Zaqtf1zsz8C+AnwIeLbgvqnAEy80ngsYh4U9G0CfgFC/haF/YBZ0bE8uK/96nzXtDXu1Dv2m4Hzi+e6joTeGFqOqwTvmmxiYj4ANV/sS4BbszMy0ouadZFxL8H/ifwAK/dP/g81fsmtwFrqP7PeF5mzry5N+9FxAjw2cz8YET8AdWRygnA/cDHMnO8zPpmW0S8jepDB/3AI8AnqP7DckFf64j4EvAfqD69eD/wn6jeI1gw1zsibgFGqH4y8K+BS4D/To1rW4Tq16k+/fUS8InM3N3xsQ0TSVK3nOaSJHXNMJEkdc0wkSR1zTCRJHXNMJEkdc0wkSR1zTCRJHXNMJEkde3/A4oxCktFxKnBAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.plot(fromzero_history['iter'], fromzero_history['f'], 'k.-')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Create more complex non-linear toy dataset\n", "\n", "True method *regression from QUADRATIC features*:\n", "\n", "$$\n", "y \\sim \\text{Normal}( w_1 x_1 + w_2 x_2 + w_3 x_1^2 + w_4 x_2^2 + b, \\sigma^2)\n", "$$" ] }, { "cell_type": "code", "execution_count": 76, "metadata": {}, "outputs": [], "source": [ "N = 300\n", "D = 2\n", "sigma = 0.1\n", "\n", "wsq_D = np.asarray([-2.0, 2.0])\n", "w_D = np.asarray([4.2, -4.2])\n", "\n", "train_prng = np.random.RandomState(0)\n", "x_ND = train_prng.uniform(low=-5, high=5, size=(N,D))\n", "y_N = (\n", " np.dot(np.square(x_ND), wsq_D)\n", " + np.dot(x_ND, w_D)\n", " + sigma * train_prng.randn(N))" ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhIAAAIVCAYAAACEOBYHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsvX18VNW97//Ze54zkydCAmiCIiKag6EwCAR6e7BUqpXK1QRtMSBQSSJSezxV8JyWU+9N/R0Rudx6FBNsRQE5B+ShevFYQSu354IIRCq10Ugp0ISnhJAJycxk9szs9ftjsnf2nr33zJ6nzCSs9+vlq2Uys/ea2evhu77r+/18GUIIKBQKhUKhUOKBTXcDKBQKhUKhDF6oIUGhUCgUCiVuqCFBoVAoFAolbqghQaFQKBQKJW6oIUGhUCgUCiVuqCFBoVAoFAolbqghQaFQKBQKJW6oIUGhUCgUCiVuqCFBoVAoFAolbqghQaFQKBQKJW6GhCFx9913EwD0P/qf2n9ph/ZP+l+E/9IO7Z/0vwj/6WJIGBKXL19OdxMoFE1o/6RkMrR/UhJlSBgSFAqFQqFQ0gM1JCgUCoVCocQNNSQoFAqFQqHEDTUkKBQKhUKhxA01JCgUCoVCocSNMd0NGCrc+Mx7ut975vl7U9gSCkUOzxN0uDlwgSDMRgMK7GawLDNk7ke59khVH6N9Nz6oIUGhZChqkxqAmCY6nidovtSNZZuPobXTi+J8G15bNAXjR2SnZIIc6PtRrj1S1ceSdV0tY2QoGynUkKBQMhCtSc1iZLHo9SO6J7oONydeAwBaO71YtvkY9iyficJsS9LbrXW//6ieDrORxXC7RffkmY5dp/A3nufBE8DP8zAwDGxmA/JsQ2fiH8xcdvtU+9iOmnIYGIBlWdHodnk59HJB+HkCIxv5OSZjrGiN23GFDpxs7xmyBjY1JCiUDERrUqubNyGmiY4LBMX3C7R2esEFgilpt9b9znV68dO3P1dMnpF2bwO96wSA5kvdWL+/GT/65k346dufi+9Z/+BEFOfbwBNE3GXm20zo9PqH5K4zE+B5Ao8viEKHBavnliLPZoLL60f9gVPo9QdxtdePDR//Bc/ccxsAoKPHhyd36HuOXCCIdfMngicEQZ7AZGDh8vrB87zu9mmN2x015QNq0A801JAYgtB4jcFH+ILE87zqgpxlNihei2QUmI0GFOfbZNcqzrfBbJRfJ5bdf6T3at3P5fUrJs9Ii3qqPCmRrgsAyzYfw+q5paIRIbznyR2fo27eBCx546jmLnNOaRGemH0Larc2DsldZybQ4ebQ3u3DyrvH4+mdJ8TfeW1lGTp6OPT4AqhwluBshwcAsPqdLyI+x4YqJ67Pt+JCV8jLUeiwYOXd4/HM7j+J125Y6ERhtlXXM9QypP1B9fEcaewOpqMQmrVBoaQZYUG9f8NBzFzzMe7fcBCX3RzmlBbJ3lecb4OHCypeCzcKpBTYzXht0RQU59vE97+2aIro+pXe/2d7TuCL81dxtsON1k4PAgHlTkytrc2XusHzRPN+ayrKUH/gFAD55Km1qF/oCr2n0CE3GAodFnCBIM51etDe7RPvGQuRPDTC3/JspohGnNDOth65i73CWSIaEdL3dbi5mNtJUYcLBOELBEUjAgj9zk/vPAGGAbLMBuTZTMgyG5BlNkR8joUOC9q6fXB5ArjY1YtChwW1s8Yqrl2zpVH3MxQMaSnF+TaYDKzq61pjN9o4yzSoR4JCSTNqC2rNlkZse3Qami50K2IkhB2/mlEQDssyGD8iG3uWzwTP8wgSgJDQTkfY4XS4Oazf34xHZozBql0nZDux20bmyHZB0TwFwv121JTDywXxtysevPhBM463uADIJ0+tRb217xhkbWUZXvhd6LOTSvKw8u7xeGjj4ai7/Xg8JkKbBO+JlldF2s5A2C5TywBJ1THStYjZaIDVpG4gDLObYTIw8AV4AAwu9/g0n+Okkjw89d3xsv6+pqIMOVYjWju9mFSSh9pZY8WjE73HG4IhHe5lK3JYVF/XGrsDHduUKNSQoFDSjNaCamAZ7Fk+U5G1Ef5aNHcnyzIosJs1jxG4QBAVzhJxUhXuX7OlUTFx6Ym5YFkGI3OsONPhhtXEor3HB0DpDYl2DPL0zhOiG/qJ2eMUO8Vlm49h9/IZYMDIYhTCjxt+fm8pDCwj/n3z0qk42+FBltkADxfEDQVZYpteWzQF6/c3Y938ibIYCcGokbbTaGBl7dcyQCJ5jCixUWA3w8sFVH/nsx0eFDjMuNDViwK7GTcU2FBf5ZQdNQnPsXbWWEV/X7XrBDYtvgNzSotUjWo9xxtSwz18jGq9rsZAxzYlCjUkKJQ0E2mXrLb7iGdHEmmHI0xqeiYuvTEXLMvgxgI78rJM2F49HUECWE3yrA213duaijK8+EGzeP+bixw4uOpOBHiC1XNLUX/glOjdaO30wuMLouo3n4qf3/boNPF6k0ry8MiMMVjw609lxpPFyIpn58JrQpvHj8jGc/eXged5vF1TDj/Pw8gw8PiDCoMofJe5q7FFsXBF8xhRYoNlGVyXqzQQ1lSU4Z3j57Cw/AbU7W0SX39lwSQ8/8DtsJoMGJFjhS8Qeo5a3qMeXwDP3HObmBklvK5mVEdqo9r7tF5XQ+84yxSoIUGhpBm1BbVhoRM8z6O925eUIKtIO5xRuTbNXV74xKXlulVbLFmWwTC7BbCrt0m6S/P6gzjV1qM4BiGE4KHXPlUYGsdbXCjOt+H0Zbdswm/r9on/Vtt1Rst80ZrseZ7o2mXm20wxe4wosWE0sijKtqBu3gRkmQ1wef148YNmPDF7HB576zPZs31823GsnluKqt8cQXG+De+umIndy2eA8/Oq/b0o2wKekLR7A/JtJjQsdKJmy+AwSqkhQaGkmfA4Bj9P0NHD4Y+tXdjV2IIn7xqfcOR/pB2OsMvTM3HF6qLV892FDA63LyDb9TcsdOKX7zUp3M+r55aibm8TGqqc+Plvv5Bdr8PNid8zWtCk9LVoi0Qsu8xMPMMeagzLMqMw2yLzSowuyFJ93nk2k/j/vVwQ1+dnIRDgVfv7qFybrA8JDKQ3gOcJTrb34Fcffo21lWUYmWsFyzCwmTLTGwFkuCHBMIwBwDEA5wghc9PdnqFILKmiAE0XTRVCHMOXF6/KJrc1FWVYv78Zz91fltACFc2TYDSyuG1kji4DIRYXrV7UDBSe57GvqU32vtZOL24bGXqfgYVoeAjsamwRFwitmIWiHAsaFjrFY5JMdhlT1DEaWdw6Ihvbq6cjwBO0XPHAwDCqz9sfDBkNBXYzGCakGRGpv8fidUsFwjFkocMCngALf6NfgC5dZLQhAeAnAL4EkJPuhlAoqabDzYlGBCDfgXv9obTHeD0AejwJqTAQYm2j9P7t3epR9zazUfRiCMGRFc4SFNjNKMq2YFSOVfTuhO8611aW4R+3f472Hh/WVJThzUOn8eRd4zPWZUzRxmhkMSrXhtZOD57Z/SfMuKkAryyYjMe3fSY+71cfdiIvy4i6vU3Y19SmWIy1vEzJ9LpFIzzLSDiGXD23VPVoLhMzNzLWkGAYphjAvQCeA/CPaW4OhZJytOIYCuxmnGrrkYkhxbMrSdRQGGgVx2g7Q5ZlMK7QgZ985xaFi7p/obDKYjCEdFIAWLXrBHbUlGNkjj6xIUrmEX4sd7KtBy/On4gROVacuezGv7zzhWg0tneHtCAudvXCbjHAyLIySe10GNVqomzbHp2G4nzboEonzlhDAsD/BrASQHa6G0KhDARacQzD7GY8996XANK3Kwmf8CKpOAKxFRbTQs/OsNPrV3hx1IInz3V6sOSNo7Lrt3Z6QQihRsQgJ/yYgmEYPNjwiWwcrdp1Am89Og1XvX4xIFM4OhS8Uuk4MlDLpvrle01oWOhE21V1j1wmHsNlpCHBMMxcAG2EkEaGYWZpvKcaQDUAjB49egBbR0kVQyleI57+qZq9UeWUpTwC6dmVhE94WiqOu5fPQEcPp6pXoWeSVhOTimQw6c23H2zpdKlmqM2fUg/CuU6Pap/gCVFkdQhHh+k6MlDrv/ua2lA3bwJG5VrRUOVEzSBIJ85IQwLATAD3MQzzPQBWADkMw2wlhFQJbyCEbASwEQCmTJmSmbqhlGuWePqn2g7cwAKH/tohe186FsDwCU/L7drr57F+f7OsoJLeYNF4CnWZjKyqgWAyytX/0x1Al2kM5flTy2gM8uppnUJfTseRgVZbWZbFMLsFeTbzoEgnzshaG4SQfyKEFBNCbgTwAwC/lxoRFMpQRYgaF4KuAjzB5qVTFbUyDCwSqjkRK+E1BISMCCnF+TYYGeCRGWNQt7cJD208jLq9TXhkxhhdEsNaolmR6hwYWQZrK8tkv8/6BycCRP77SI20g6vuxJ7lMzMy+p2SOAV2MxoWOhX1Xlwe9T7rD/JpMc55nsDAAg1VTsX4lsYBFWZbcH1+SH21w80N6LjXS6Z6JCiUaxKtXfm7K2bCywVhMrLo6Q3gvpcPDmhKWPiOXkvF0WhgVaWHd9SUR71HPLLAXi6IF37X7wHxB3mYjCweePWQRvBlZkW7U5IPyzIYbjdjy9KpaOv2YZjdjLUffIUlM8dgbWWZomqokWUH3DslHeeFjpC41pjhdmRZDDL1V7X3Z2IqaMYbEoSQAwAOpLkZFMqAEEnK+vr8LLR3+xTyvQNxvqtXxfF8l1fVGCAk+u4pnjgGs9GA9h4farY0AgAaFjrFEtDCvTM1ZY6SOliWxZmObqx+5wsUOix49r6/AwA8++6fZcduL/yuGS/9cBKuz7MN6IIsHeetnV4xI2vP8pmq7cj0Il4Zb0hQKNcSkXblPE/g9QcSSgmLVBkzGnpUHFkNUSCGiX6PeOIYwj+jt2YIZWgTKtqVJXogWAbIzTLLjE4g1DcNLBO3ERHveIo2zsOvmelFvKghQaGkkfBJI1LwYPOlblzs6o07+2Ag3KMGBlhTUaYoz2zQcfl4hIDCP8NoGDKpPP9OxDijpIbwonEEQJfXH3ffVEMYT1JBNC8XwHW5NhiNkcMPtbxvwjhXGNMOc0z9eqD7ZEYGW1Io1wLCRHT/hoOYueZj3L/hIHp6A3ht0RRF8JWRZbBs8zG89NFJrKko0wzOkl67vdsnC8yKJ5gxVliWxZuHTmP13FJsr56O1XNL8eah02BZfVONNLhM0IGI5TMjc6yqv1+i599qv6fwevgzbL7UnVGBcNcqQtG46/OzcF2uDdkWY0J9M7wPXHb7sH5/sxhcXFn/CRb8+lM0t3XL+odavxE8aVrjPHyMGljo7tfp6JPUI0GhpAm1hX3R60fw7oqZil35hb7Yg9ZOL178oD+4sDjfhlG5ocmlvdsHLtAfkCnEUgiTTo7VmDT3qNaOp8BuxpN3jU9bmmUq5I0jeXK0jDOqmJlZCAqYaiqoQt+MtItX6wNbfzQNFc4SRXBxzZZGbK+erjkOxxU60On1I8dqxI6acpm65gWNGKNubxDjCh26+nU64imoIUGhpAmtc0+hQqEUqcv+eIsLNVsaUZxvE7Mhwie5tZVlKHRYRONDWNyS4faPdkQSz0KeqCs2la7cSBOz1jM87/Kiy+vPmKh6Sr8C5u7lM9DrD4JlGJhYBi4vhxyLCSfbezT7tFofOH3ZrRmT09rpxU/f/lwxDtfvb44o6a515HH6sht2i1GXIZCOeAp6tEGhpIlwbQZAfWHneQIQgjeXTsWmxXdgUkme7HxXbZJ7eucJ1M4aK16jtdMLA6PfPRqJaEcksR5PJOqKTbUrN9LErPUMhd8omcdGlOTQ0cNhwWuf4ptrPkZF/SdovtiNM1fcWL+/WbNPq/WBlz46icJsi+rzd3n9quOwwlmiKuku3KfAblboSqypKMNLH50Ez/OqxyTh6J1Xkgk1JCiUNKF1Tipd2IVF8sGNh/HUjs8BAP/roYnYvHQq/tB8CSzLai50eTaT+O/ifBuCBBiWZcKOmvKERJmSseORnh1fvNqbUOxGqmM/Ik3M+TaTqvhR/YFTGRVVTwmhZXS3XPGiwlkie6/0+an1gfYeH3JtRs3nL1xDOg6jZRWxLINReVbUzZuA7dXTsbayDIQQPHvf36E3wONne05ENZb1zCvJhh5tUChpQs8xgDDxFToseOq742UR5/VVzr4KnFB1h3q4oPj/66ucePbdL8JKKdtkZ8B6jwYSrVsRfjSys7Y8IcMk1a5crbTUfFvIHf6rD7/GpsV3oMvrR4ebw4sfhCqMXsu1PDIVrb6SZTYgm5Uvh9LnF94H5pQW4ef3lsLtC2JkrgX/vmw6uACPv13xiM9fuIZ0HBb1eTAijZ08mxkjc61iIKegiyIYKe3dHI63uDTjHga6DDpADQkKJa1EU1sUJr7Vc0sVQV21WxuxZ/lMzYVuRI4FB1fdCYZhRCNC+Kx0Eoo1LTTRuhXhu8ION5eQYaLXsInFWAp/r1qgm/R7tHdzeOq741G3tyktQaYUfWj1FQ8XxM1FdmxafAeyzAZ4uCBuKMiSPb8RORZsr54OlmVwxc1hwa8/lcUk7fnsHOZNuh7tPT7xuuvmTwTLMPjDyjthM4U8WNHGjmAIPHvfBFkV09bO/iJjwvGIlrE80Cqu1JCgUDIYYeLTKpIlTCTCJBckgNXE9svs2kM1JwQjQu2zsUZ5J7rjCd8V1h84pcjvj2UR1mPYxGIs6X2v9Hscb3GJ2TS3jcyGzWykehIZiFpfWVtZhny7CZeu+rD6nS9kzxxQ9odNi+8Q3wf0H4+snlsqy6gqyrbgH3d8juMtLhxcdac4ltQUYtUMXEK0i4wBmVW99poxJIZSiep0Qn/HgUWY+LSEqLQEbIbb+w2AaDv2eI4GEtnxhLfneIsLbx46jR015SCExKW4qWXYSD0Leo0lvYaV2veo29uUMbLFFCVCX3m7phweLgiWAS5e7UVPbxD/sP2Pqs8cgKw/ZJkNmgu8NKNq9dxS1SMu6diJZLRqjVuhYF4mebxosCWFksEIE9/EklzVKoFaAjbSQMNowVcDHeWt1p4n7xqPkTlWzUwPLWEfAbVMEWk2h5B+J0XLWNJrWKUjqI2SOCzLYESOFVyQx7++/yWCPMHwbAtWzy3FpJI88X3CMw/vD1qVb6WxEGsrQwGX0fpEpEBhtf7VsNCJbxTnZlz12mvGI0GhDFYEhb48m1lTqEpK+KIX7Sgi0ZiHeL5PLEcj8Up7SydpYfLXE4ehN+YiHUFtlOTAsgzGFToUmg5rKspUg2Wl/aH+wClFFVFpTJLJyMLIMnh5waSofSKS0TqY+hc1JCiUDCJSQKDacUIsi56Wuz3VE5bWd9Lr/o901CAEPaq1WzpJxxKHoRWhzwWCaO/2KZ6JtA3CTjITJ3uKnE6vHzVbGlHosIhxDR4uiJV3j8fTO0+gocop9g9pf2jv8YlicP4gD5OBRZHDEqqvYe/v73qINn7jOUJMR+0XakhQKBlCPDtvNW9Cw0KnKF6jdxIRJixhErrQ5U25vHS8wZlAyJjgeT7itaWTtBAMWTdvAsYWOWAzaX83wbB6d8VM+AM82nvkEfrSewxEITRKauACQc206vUPfgMj8/qP2KSGtt1iwDlXr8yT0bDQifFFoWceS1CvgQUaqpyo2aou2x0r6eqPNEaCQkkzwvl/q8uDi129KHSEdiB6hJWk3oSDq+7Etken4Vcffo1p//r7lCpERotZENDyJly82qu7XVoxHEGCiPEh4WfM7T0+jMy1ojjPFlFxk+cJXF4OF1y9aLrQjdqt2kqEA1EIjZIazEYDnpg9TjWtusDRX3+jvduHC12hvxc5LOjyBhTqlDVbGnG+ywuXV19/EMbafS8fxM9/+wXq5k3AgadmYffyGQkt+pH6o94xGw/UI0GhpBG1HYT0nFaPsJLgTWjv9omBhUDsxXr0ZivEsutJRi0KrRgOrfQ4qUpgrEc2wne72NWL1e98gXXzJ0a8h9r3K3RYwAWCONfpyehz7WudArsZY4bbVZ9vl9cPi5FFc1c3lm3p73dvPTotZPSrfKat2wcDy2j2F+mRA8P0B0m3dnqx5I2jKM63Yc/ymQn1lXi9d4mSkR4JhmFKGIb5mGGYLxmG+TPDMD9Jd5solFSgtniv2nUCL1SWiTU19GZPJKrwqPfzl90+sc2TSvKwem4p3L6AqpchGbUowr0uQsS6nmyTWOt+CO0SUvy0IvSFe4S3YVJJHlbePR4PbTxMy4pnOCzLIMui3odcHj+4II/1HzbLyo67PH5RQC38Mx1uDkEC1b/ZzAaZt++8S38WUSxI++Okkjw0LHRiZ205AjxJqecsIw0JAAEAPyWE3AZgOoDHGYYpTXObKJSko7V4d3n9WHn3eLyx5A4x3iHaYhRLETA1F6fJyKpOQgzDyO7d6w+KRoSg5lhZ/wkebPgEzZe6EQj0FxcysMpCYfHUolAzCBJNv1T7HYTnIRgQQpCm1j0K7GZs7iumtr16OtbOn4hNB0/To45BwnC7RdGH1laWYbjDDLORxSMzxmBXYwtcXj8K7GYMs5vx2ZkORZ/Y8PBk7GpsgZFlsHnpVBTn2/Cgsxgf/uPf461Hp8HLBfHbz1rEfqFljCSSci2NuZhTWiQbmxe6elNiuAhk5NEGIeQCgAt9/7+bYZgvAVwPoCmtDaNQkoxW1HaHm8OuxhY8c89tONPhAcswfaWPoekuT0ThcVyhAz29AaytLMOmg6fxyIwxigwHwQ1q6CtpXjtrrOJ8ednmY9j26DRZcOLmpVPxdk05zrm8Sa1FkUi2idbvMCLHIjMgVu06IQZpjhluR5bF0K8a2ocvwMsUEaX1EITfhRbvykzUBKpcXg5BQsDzwJuHlGPh1Son9v6xFavnlorGhT8YxPI7b8az736BJ+8aj/eemIm/XenF4k1HxM9teHgyOj0B7GhsTUjNVS0rAwDOdLhxtsODvCwTVs/9O9Tt/bPCcNGT/hwPGWlISGEY5kYAkwB8mt6WUCjJR23xX1NRhneOn8MjM8Zg0etHUOiwYOXd4/HD1w5HPN/UWlgBoL3bpzibBfoX/x015eK9Xqgsw5I3jmrGStjMBqytLIPJwGqeFUs/u+j1I9i9fAbsFqOoHpgsrYp40y+14kF2L58hPo9oBoTWdaT1EIDMkjKmKBEEqpovdWP9/mY8/d1b0eX1gycEFc4ShbH82NZGvLFkKjp6fOhwc3juvS/R3uPD8w/cjn1NbWi60I3t1dPxWFiQ7vK3PsO2ZdNR4SyGy+vHH5ovxazmqmUAj8q14NLVXk2DNlEZ+mhktCHBMIwDwC4A/0AIuRr2t2oA1QAwevToNLQufqjM9NBHb/8UFv8dNeU4L9mxS3f7q+eWiuI3QOQgyvC8c72VNv1BXgz8uuLmIrpB82xmjMixgifqVUd7/UE0LHQiz2aCy+tH/YFT8Af4lGhVxJvupnWkJG0nz/MIEoAQAgaxCQoJE3Smql0O5vkzFUgFqgQjetPiOzTLfnf0+PD8+1+hdtZYPHPPrXB5/cjtq4HR2ulFgFcPBA4EeTy08bCYZirqT+hEywDeXj1dMUdIDdpEZeijkakxEmAYxoSQEfEWIWR3+N8JIRsJIVMIIVMKCwsHvoEUSgRi6Z8sy2BkjhV2ixF1e5twvMUlm8CiFeyKRPjE0+sPqp7N+oNEfD1agCHLMrixwI5hdpNCtrthoRPZ1tD3eGjjYdTtbcLKu8fDZjbI4hwEL0K8qWjStLxYgsiEzwntVfuOgpfjisePBxs+0Qya5HkChmGws7YcDQudorxycb4N1+XZZIGhmZa1QedPJV0+Dm1XfVg3fyIaFjrx/p8uYJjdrNpPCCDGIAj9nCcQA6QNLKP5OaA/zbTT64+pjVqGq5bhIjVoo8nQJ0JGeiQYhmEA/AbAl4SQ/5Xu9lAoqSb8WILpi0OQBv7Fc74pnXgmleTBYTUq5H1frXLCFwji35dNw6WrPviDPNY/OBFP7vhc0w2qJdvNsgTzXj4kW9if3nkCu5fPED+bqGiO9PPR0jO1PlfosKjKHAvfMVoqrFba7puHTosTdqYZDxRtAgEeF1w+xdHAe5+fR32VU9QSKc634bWFU5CXZcT8hsOy/vH4ts/wxpKpsBhD8UyKcfbwZPiDQUwqydOd2h2OVkyVycCqvi4YtKlOQ85IQwLATAALAfyJYZg/9r32z4SQ/0xjmyiUlBJeFVA4q9fS9i+wm6PK4UonntpZY7Fi23GFJHBPrx9P7zyBtZVleOF3zWjv8eGNJXdg92Mz4A/yESehcEXMnl7tIwOBWMuWhxNvDQ3p51o7vXjhd9pKl9FSYbViI3bUlFMjYhDS1uNTCI+t2nUCdfMmoLvXLxsvI3It8HLq/cPl4dDrD+KZ3X9SjLMAz+OxN/8oHjdE2wyojW2tgOoih0X19YHqixlpSBBC/h+gcShJoQxhpJPHiBwLdi+fAX+Ah81sEP+/NIgy2s5eOvEIRyStnV4xEBAAtldPFz0HwiS3eNNR7Fk+E9fnZ+lqs9CO1XNLoy7sydS7iCWILPy+x1tcWPLGURxcdWfMNUy0vgMhhBoRgxAhRkhKa6cXowuy8NSOz8UMHADiDl8r22psoT3iOMuzmaLGzkTy2mnFGkULtE6lVyJjYyQolGuNcInq+14+iI4eDqNybRhmt6AoW36+qUeeWTrBFOfb1MV3+s5phUlO+P96F3ZpO9R0FzYvnQoCIsZDSPUqpO3Qm9kg1cuQ1tD4w8rIMQmxlEvPtNLrlNQiHA1IKc63KXazxfk2MIy6homgjyKkR4d/TvCeFefbosbORBrbWiJr4a8D0C15nyjUkKBQMoRY6zbo3dkLE8yoXJsiOFKY/IR/C0aFHrerIOYkFD8C+hf21XNL8YenZ+HdFTPhC/B4YMMhcTLr6Q0kJCQVbw0NvQJWPE9wuceHXJsR26un4w8rZ2H3YzNUPT3xfgdKZlHksKBeZWw8//6XWHn3eDzoLMamxXfgzaVTAULA8wRF2Wb8R/V0/N+nZ2FH9XScvNiF4y0uXLzai7WVZYpr7WpsQUOVE6NyI/dTIHGvHTCwdWAy8mgjE4g1RZNCSZRYJw+b2YBNi+9Altkgplm29/g0DQDfVwLeAAAgAElEQVSWZTAqz4q6eROQl2VCrs2E7UfOonbWWFFYp/7AqbjcrkJ8xfEWF463uFC3twl7ls9EkFcW1lr0+hG8u2Jm3Kmg8bpx9QhY8TxB88X++gpzSovwzD23wcMHcbnHh+GO0AKQ6tLrlIHFaGRx64hsbK+ejss9HLLMBvT6g6hwlmDTwdP42b2leLhPZG1OaRGemH0LareGSpA/MXscRhdk4dulowAAez47hx/PvhlbfjQVPAFYBjCyDJZ+8yaMytMXsyB4vAodFtTOGivGWdjM+j1eyTBG9EINCQolzQhxEQAwp7QIFc4SUYNhV2OLqmHA8wSXrsqjzNdWlmFEjjXirjjPZsbIXCuWbT6GGTcVoKr8Bix/6zPxGg1VTjzzvVuRZ9NeFNV2Ok/vDAWmCcWHBEPkQpd6TQEvF9Qdf6EWTBpNLyOSaFekgM4ONycaEZNK8kRRMLVrRrsWZXAReqZAjy+Ax7f1j4k1FWXo8QXEflzhLBGNiPAS5A0LnfjmLUX44Wv9yq6vLJiM/zxxDv99cgnybPq9bpuXTsWlq72KIOtIY1OKljFiNbNJj5ugRxsUShqRxkX86sOTWPHtcbLc9Cdm34L8vrgFKVqLucNqjDgpSHfSP/nOONGIEK5Rs7URQR5xuV1HD8vC9urpqJs3AZY+kZ1EYgliKWueLDeu9LtpSYDTuhlDD6GvuX284pmv2nUCZkP/UikELav1j5otjWi54pW99vi2z/DIzJti0hNhWQYOq1FViE5v/xOMkZV39+tdrH7nC1xw+fCzPSeSGjdBDQkKJY1IF8DZpSMUC7uWaE0kZcZoSHfS8bg+tYyDv7T34KGNh7HkjaNY9PoRUa46UiyBWuEs4bVWlwcXu3rF+ItIE2my3LjS75aIEBhlcNHh5rB+fzOMLCMKUgkCY62dXni4/mcuBE1q9Y+ssOOHeLN5/AH1TJJYCt2pGSO1WxtR4SwR/50M45gebVAoaUS6AMaycEVLT9RDvNfQqg/y4gfNinZHiiUIP46YU1qE/3Hf3+GK248aiQCQcO1IIj6J/B7S4xOb2YDXFk7Bsi3HEhICowwueJ7Hj755E6p+86mi37X3+JBtNYp9YVdjC16tcuJyt0+1f0iNDuG1ePpMMsa4ljGSJ/FyJsM4ph4JCiWNSHfA0aSppSQjayDea0iNg4Or7sSOmnK8eei0LNc+XFJbLV1N6o0R4hG+utgjGhFAv2u5dtbYlPweaim3FhOL3Y/NQNn1OYosF5qZMTThCfDTtz9X9LsnZo8TM5uef+B2/P6nf48fTr0BWw6dwahcZabHa4um4IaCrKT0mWSMcS3voUvi5UyGcUw9EhRKGpHu7iMpWIYTb9ZA+O7bYmRRN28CsswGeLigGNsQjXAVzifvGo+mC91R2y1FLR5BS/JaEPGRalJIszU63BxyrEbsqCmHgQFYlo27Cuii149gz/KZGJWXhRE5JGJ2iM1sQIAnMqEwmrkx+PDz6jv3mwrt2Hb4DHY0tuKe20eJgbcAsKOxFXNKi2SFsPKsRpzt9IhjigDIsxlxzuUByzCwmQ26gyWTkRmk5j3c8PBkvPz7kwCSZxxTQ4JCSSOhEsYWceLhCcHzD9wOq8mA6/JsESVutbIGtDIdwo8SNi2+Q8z6EBDEcmLJRhAmvN3LZ6DXz8PAQFeamtR1KxzraB0nFOfb8O6Kmbh01YdFGw6Jk+LmpVPhC/Bx1+2IFlsR/hsHAjya27pRs6VRLO8ebvhlYpEuSmSMkto2AsX5Nvy13Y37vlGMxd+8SVX9cl9TG37xfYLr87PA8wStnR4s3nRU9LI99d3xYk0OaWbVjQV2cUxGkrhPNDNIrbrw1k/OosJZgupvjY06x+iFGhIUSprxckEseeOo4vWDq+6MeYBHSoMM331nmQ1JDSbs6OFiWtCluyXBgPio6RI2PDxZlpJaX+XEiGwrOr1+hffgbIdHZgwJwWN6jaFYzqF5nuB8Vygyv7UztvLulMzGaGAV3kBpjETdvAkYW2iP2Fcuu31o6/ZFzPoR0qSzrSYU2M0JFa/TC8syIISgsv4T8bUdja0A4ptj1KCGBIWSZpIRVCUQqSBW+O7b5fVjTmkRFpXfiJG5VhgYBpd7uJhEb/TcV2tRlbpueZ5Hw0InPL4gfH4eW5ZORZAQXOzqxUsffY3n7i9T9R4kagxpFUFSc/V2uDnZQkGzOoYOvYEg9nx2Dv++bDrOu0KeMSHAFwj1s1++14SGhU7RkAzvK73+IPxBHm/XlGO4I+RZeP6B27Fu39fidYSsDi4QTLh4XSwkc45RgxoSFEqaiWUxi0YkV334ZPJR0yX8+Nvj8G+/P4kKZ0mouqDDDLcvoPscV+2+k0ryRAEcLhAEz2unvkldtwV2C7661C0r2bymogzt3Zxq+wHAwwVVJ0ggFMcQ7Uw5lnNoLhBaKAQ10WF2M+aUFmFfU5vs3jSrY/BhMbC4f/L1+Etbj+pxnz/IY19TG+rmTdDsK1mmUP8MEoAQgvMuLzZ/cgbP3HMrnn//KxxvcYlZHWajYUCVJ5M5x6hBDYlBwGCW6x7MbR8okim3HGnnET6Z3HP7KPzb70/ikRljZOp8Gx6eDLuFwzC7/l2RVEUvXO1Pr7u20+tXLeX84vyJqu0vzrfh5iK7Ype4trIMK7YdR3uPT9e9o51DC+fYLAvYLUY8vfMz2W8FhM7KaVbH4IUHsOngaSyZOQavVjnx2FZ5f7KZDZhTWgSWZRWKqiH9Ex6X3ZyiHy6/82Zs+PgvqJ01FnV7m2Tqsx1ubsDSi1Mt6c4QkvxKYAPNlClTyLFjxyK+hy5oQ48zz9+r521pj3rT0z+TRTSpaGlwV4An+OpiN+r2Nikms+3V03VJWIff92JXb9wBnOc6PZi55mPF6x8/NQsleTYYjawi6+TSVR/W728WPSpCvRDhDDie4FG17yWUSFf7raRR+3FMztdU/8xEeJ7gQpcXZzo8WLXrBNZWlqHXzytq2Gx7dBqK87PE56unb9TNmwAuyOPWkdkwsvKsDb2y7mlGV0OoR4JCyRCki6TJyMLIMvByQUWaY6QdRbSdh3T33dbdiwK7WdW9GuzbaUVbGKVtHpFjgd0Sf8yCljflzGU3HBajqEEhtL+92ydOwsLxQnG+DavnloqGRKHDAi4QlKWLxjJJS8+xi7Itqt+NEBKT0UXJLDrcHLz+fmlslmHw0kcnxeO52lljUX/gFBiGUXxu2eZjKHRYcHORQz1tOcuEIE9gMxtQlG2V/T0ZKdzhn1H7m9DWVBaXo4YEhZIBRKqoKbjoLUZWVkBq89KpcFiNCg0DNVe92gQz3G5Br0aMwd+uePDM7j9F3CEJbZZ6BEbmWuOOGyiwm9FQ5VRVtXx5wSTF+7XOmAXVvkkleVh593g8tPFw3Ds+6T0cFqPqbxW+wFAGF1wgiO5ev/hceUIUab0bHp6Mq14/enwBsf9wgaB4lEcIVPtGttUEs4GBkWVUjdlIx2paRoGWF0Ptb4mmR+sl6cqWDMPclaTr3M0wTDPDMH9hGOaZZFyTQslUtIpw1c4aK0Zzn+3wiH8vdFhw6WovHthwKGrxHa3iVwBwXW6oYqFUPW/d/IlYt+9rFDosuNjVi1ZXfx2M8Dav39+MR2aMQd3eJlTWf4IfbDyMH8++BXNKi8Tr6Y0bkJY53149Havnlorpd2qGiJZqnyBR/MTscQkVPQq/R68/iHXzJyp+KwO1IwY1ZqMBLk+/qmyQJ4p+s/ytz5BlNmD9/mZ0uDnwPAHDMFg7fyK4AA+TAXhlwWRZ31hTUQariQXDAPe9HL3wnBStMevyahenU5tDznZ4klLMLhqp8Ej8BsDoRC7AMIwBwCsA7gLQCuAowzDvEkKaktA+CiXjiLa7FtLGBGpnjdWtYRAtzey2kTnYs3wmvFwAX17sxvPvfwUAUYMmuUAQFc4SRa78Y1sbsaOmHL/4fuxxA9Iy5+HR5eE7tHybSTUSfUSOBQdX3YkgIQlHxUsDPHt8AWSZDTIl0NBCwUTMTKFkNvk2E24d6cCrD0/GY299BpOBVe03HW4OFc4S8DyvqBHz42/fglc+PonVc0vFWJ3tR85i8Tdvwvz6wzGneGqN2e3V0yP26WSnR+slLkOCYZh3tf4EoCD+5ohMBfAXQshf++73HwDmAaCGBGVIohUfIGjihxcDikXDQK96Y3s3xICxhoVO1RLa0glQMBKSGTegdW4MqLt0xxU61M+Y7aEYikSj4qXt8QeCeHDjYcX1Xpw/ER1uLtOC5Cg64HmCk+09YqxD3bwJuC7PptpvhGq2QQLZIl/hLMFjbzUqYnU2L50Kty+AQoc8tkbPQq41ZoMaRyhCn9abHp3szJB4jzb+G4AGAOtU/utJQruuB9Ai+Xdr32sUypBErUDP2spQsSC1YkDCBCFFa4LQOgIIf6+0DXoMlQK7GUXZFt3t0ItakS+tHVqn169aECz8+wjtiic9U2gPgXrZdQZIibuYknqk/ep4iwtL3jiKur1/VhRrW1NRhl2NLSjKtoCEebq0xkqX14/Fm47iidnjZH/TMz60xqzVxGr2abX+fkNBVlLGQDTiPdo4DMBDCPm/4X9gGKZZ5f2xombWyw6VGIapBlANAKNHJ3SSQqEknVj7Z/hOXMjaeHnBJNmuXPi7zWzQLTCjV4wmvA3RdjIsy4gxFlpqf8kiHvGeZOfOR/IapUpIKFXQ+TOEWr/a19SG/+/+27Ht0Wlo6/ahw83hzUOn8ZPv3ILrcm3oDKsHo1UfRlBBHTO8X1pb7/jQGrPD7RYMt1s0+7SWNy9V+hECCelIMAxTGh63wDDMLELIgYQaxTDlAJ4lhHy379//BACEkH9Vez/Vkbg2udZ1JKIV/NF6L8MwUStkxpLjHks74qW924f7NxyMS58iWaj9JtJ6DHG2Zcj2z8FApH4liEZFK343p7QIT8y+RaHIKvSL3Y/NAMMwMY+PgRhXOtB1w0QNiS8AbAHwAgBr3/9OIYSUx33R0HWNAL4GMBvAOQBHASwghPxZ7f3UkLg2GaqGRKomkFgFcNI1kcWa9qa3Tcn4PjxPcNntg8cXxOnLbrz00UndCpoaDLr+OZQIBHic7/KKnoddjS148q7xGJFjkWm4RDOe820mXPFyin4RXu0zlaRovA6IINU0AGsAHAKQDeAtADMTvCYIIQGGYVYA+ACAAcDrWkYEhTKUSKXaXbTsDbWJSI90dDImLuFaalLDwvdP5JgiWb8ryzIoyraCtxPYLUbZ0RMNtBxcSAMthT7RsNAJh8WAZ3adELVRvFwA1+WGlFW1EPrFFdYHAHjmnlvh8vpFHZhUe87SrZKZqCHhB+AFYEPII3GaEMIn3CoAhJD/BPCfybgWhTJYSGVFwEhxBvF4K+KduNR2c8KEHi41HP794/0N9PyusSoG0lLhg5vL7pC8+uq5pcizmeDy+vGrD7/Gim+PU9Sfaahy9lX0ZJFvM+FvnR6c7fCIacA3FGThxgI7vFwQS944qrhXquNnBrKSqBqJGhJHAbwD4A6E0j4bGIapJIRUJtwyCuUaRG2xT1TmWSBSQa9YJ6J4Jy41A6RhoRO/+vBrUTcjFXnv0YI1tQyjm4fb0en1o63bJzsDz8CaCJQYCfK8wmBYU1GGwmwLlr/1maxv12xtxOq5pdjV2IL/OW8CLl3trykjZFjlZZlSXq5by9gdyEqiaiSqbPkjQsi/EEL8hJCLhJB5CBkWFAolDsLTvqQyz7Eo46kRKR0y1oko3onrstunMEBqtjSiwlkCoD8CXkoyJuJoKbBqhtH6/c1obuvBn89fVVQlpemegx+eh0IrZdWuEwjy6kJmRdkWPDJjDLz+oEIM7umdJ+DlgklLOVZvr7raJc8T3SneqSIhQ4IQoojQIYRsSeSaFMq1TPhElAyZZwFpOuTBVXdiz/KZ4q461okonomL5wk8PnUD5OZCB7ZXT0eO1aiQoW6ocsLAhgLj2rt9ONepLtkdiWgTvJphVOEsQe3WxgFTB6QMHH5/UFP5lOeJat92WIwRDY0giTzGEkXLCygIZQ2EXoQWtGgXhZJBhGsfJEPmOfz6akcPerUm4n0/EJoIT192q7p+/3bFgyVvHEVxvg2vLJiE/6iejkCQ4PRlN37+2y/Q3uNDfZUTL330NfY1tcV8vBBNU0LNJS2odmrpBAzUbo+SXPz+IL5q64HNpH4MYTKwCm2UNRVl6PWHjM2LXb2qn7OaQvvySIW4EiGSFzDZmimxkvSiXRQKJTGkyo5Wk/6dP99X+ltrxx7p77HupOLZeXGBIF766CTWVJTJdk6vPjwZL310EkBoYnx823GwDIOq33yKJW8cxfEWF1o7vajd2n8EEo9nRk0xU0BtR1fYp9pZf+CUos0DudujJJe2Hh8e29oILxdQPNc1FWVgGYj1Zw6uuhPbHp2GNw+dxvk+A2Ldvq8VXjNBLCraGASij1MtonkBI/XvVEM9EhRKhsLzBD29AaytLJOVNFZbxLSCBccVOtDp9UdMq9RT0liNWN9vNhrQ3uPDix/0R8p7uCB6fAEcb3GJ72vt9CIQ5FV3X0IRM+HfyTpeUNvR5VmNqK9yonZrI178oBl18ybgxuF22M0GGA0MLnR5aernICTQdzRxvqsXuxpbZFkbbx46jefuL5P1bZ4neO7+MvA8L/aHXY2teGPJVJgMDEwGFkWO0HsVgcRVTozKsyLPpi5mFYtnLd9mEu8vfLa+yol8yZhIF9SQoFAylA43h0WvH0GhwyJbeEfkKHcbWuen2x6dhgW//jRqWuVAID0OqdnSKMvYkFKcb4PRwGrKUUv/nczjBTXD6NYR2dhRU45AkIfRwKLQbsZfLrsVWSfD7eaISqGUzMHIMqKnSa3CrZp0vNAvCuwW7H5sBtq6fVi86YjMYCjMtmD9/mZFtkfdvAkYmWvF+BHZCaVpdnr9eOmjr2WGz0sffY3n7i9LeyoyNSQolAxFOBMVMhsEDq66E7Crv1dKa6dX1PtPVVplLKjt+vNtJjx513g0XeiWTeZFDosiBkOIkQAG7njBaGQxMscqpty1uznlYrGlUTTUaFpo5lPksODVKiceU/E0DXcojfTwlEujgVFk8QgGwyMzxqC9mxM9bK2dXmSZDaKxkEiaJhcIYl9Tm1hhVOAX309/0C81JCiUDCWWnHSt9+baTJhUkjfgAYNa+e5qu36tIDE1o+O5+8vwi+/rDyZLVH1Tq75G+GKRZzOh0GHBxa5e2C0G2ExG6p3IUEwmA24Zbsf26ukI8ARGlsHwLDMsFuVyqKp7UuVUlAYvdFhwY0EWOtwcXqgsw8qdJ3C8xYXifBuG2c1YN3+iWIwv3nGYao2KREio1kamQGttXJsM1VobAnrOUyNJS6+bPxEsw2CYw4xONwejgcGKbcdTLqqUbrneZLZDq6jT6rmlopdIECTiCRRuch33GrT9c7ASCPD46lK3Itbg1hHZChlsredfN2+CqGApaL1I45heWTAZRgMDh8WIbYfPoOG/zqA434bNS6fCF+AVR2Pji5T3DidN42pAam1QKJQUES2lS60K4VuPToPL40d3rx8WEyszHNY/OBFv10wHT5CSIEHp7n/Z5mOy2I6LXb0YkWPBMPvAneXqPY+O5LXQckULRyqChyLIEzyz+09pjUGh6OOKh0N7tw/r5k+Ey+tH/YFTqN3aiO3V0wHIx4bW85eWBlfTenl822ficdeaijIcOePC8RYXFr1+BHt/PFNWovxXH36NJ+8aH9UgSHeKZySoIUGhZDCRMiPCF8p9TW1outCN1XNLAUCxsD2543PsXj4Do7KtMbcj2hGB1KhZN38iCh0WRSBbQ5VTjF4fCPScR0fb5Wm5k6/Ls+G/Vt4JnhA8914TfvTNm9Ieg0KJDs8TtHX7ZPLWQsnvC129qKz/ROYl0Hr+FiOL3Y/NgIcLilkgUqRxSat2nRA9WK2dXrh9QSz49aeyzzRd6NZldKZKoyJRqI4EhTJI0VooR+ZYNYMr/YHYa+pFkuYVkBo1Lq8fT8wep5AfrtnaOKCy0nrUNyOpBQIaipgLp8DDBfDD1w7jH/7jj/jh1BtwfZ4trRLFFH10uDlFoOSqXSfwxOxx4jMXAmjPd3mRbzMpnv+aijL84t0v0OHmUJxnQ5ZZvZ8JGUatnSF57YaFTuysLdc0PAaz0UkNCQolw9ESsNFaKPOyTCB9/z/8b/EsbNEWW0Bu1NQfOIXRBVlpnyz1yAZH81qEC2+9s2IGcmxGuDx+0fPz0kcncaGrF/VVTipaleFoPe/RBVmoP3BK9lpbtw9dPg4OixFbfzQNHz/19/i3H34DLAP86Js34WJXL676/BiZY1U1NoTrCUHPdXubUFn/Cf7a7tYcm/GKVaUberRBoWQwkVzvBXazqpTvc+814Zf//XY0VDlRIwko27x0KghIzFVE9RwRSF3Ax1tcuODypj3CXM+Zsp5IeMGdzPMEX168qghoFWJRCh0W1M2bgDHD7ciyGDDcPrDqgpToaD3v9m6fTBStON8Gf5DHBZe86uuGhyfjwFcXMPnGAhTYzejuDSDHYpLL2vMEv3yvSczaaKhy4vn3vxTv+dJHJ1VF5vJtpowIUo4HmrVBGbQM9awNQDtqXDhPvdTlxR9bu0SBmvoDp3C8xYWDq+7EqFybGNdgMxtw6aovrklKqw3bq6fDZGRhZBl4uf4JdF9TG+aUFuGJ2bekrPR2ommd0uvonbz1RPALr8UQZDmo+2emotU/AgEezW3dMmPw1Ycnw2Y2YPGmozKD3MAyeOrtzxXPe/PSqVj0ukSMaqETt43MUWRSCffmeR7T/vX3svZNKsnDywsmAegP7uxwcxHHepqgWRsUymAnuuudlSlWAv07amlgVnu3sny33qwC4Yhg/f5mVDhLUGA3Y5jdjF99eBKH/tqBtZVleOF3zWjv8aFhoRN18yaAZVnk20wpiTBXy1b5+b2lMPQFR8Zyn1gi4bWeRZbZoHhtMJ93D3aE/iHtr14ugFE5VvzlshvvHm/F1h9NQ5AnYBngcg+H4dlmWSbFm4dO45+/V6r6vK+4OYUgmXQchQdEtnf7FF6Q9h4fzEaD7H2JiFWlm4wzJBiGWQvg+wA4AKcALCGEuCJ/ikIZmkRzveutwpnoJOWwGLDi2+Ow/K3PZLu2k209eHpnf1R6+KSaip2UNGZjUkkeHpkxRoyCj8fzoTcSXutZeDj5bzintAgMw8R8hERJDh196qOPzBijyBr61UdfY8nMMbjQ5ZUdLaytLMPYQjtGF2RhVK4Vk0eXgYCoPu8ON4dJJXmonTVW9ATyvHYQs94xmsmCU9HIxGDL/QAmEELKAHwN4J/S3B4KJW1oBQzm20xo7/bhQpcXI3Is2L18RsQqnHoyGLTocHP4S5tbNCKA/mj32lljxVQ34fVU76CkRlHtrLGK7JBYq4LqRe1ZNCx04oaCLPG1mv92I56YfQsebPhEM8OFklq4QBAVzhLVrKEKZwlG5lgVug9P7zwBNxcEA0asnjncblE+7yonivNs+J/z/g51e5vw0MbDqNvbhMtuTvMZsyyDcYUO7Kgpxx+enoUdNeUYV+hQjFE9wcGZSsZ5JAgh+yT/PAygMl1toVDSjZZU9Mn2Hs1zfSHyW+qq17srUoMLBJFlNmjmyktT3QZiByXduSWzhki0uAutYxAA2LN8JnieR2+Ax8MSjQAqTDXwCM9FS0iM7/v/4X8L8kTWb8Kft8nIoscbwNkrHlGHQvhszZZG7Kgpx8gcq2qtjkjjVet+g8mblYkeCSlLAbyf7kZQKOlEcL0LO6VOr18zHVNL8wGALI1Ry3OhhtlogIcLqno0CIC1laFUt2TsoPSkv0l3bkINkfB2xWrM6NHKAJTPQlo/hGVZtPcVSZMyWM65BzPSfkNAcH2eVdEv5pQWYVSuFea+yrJSivNtuNzDKfqN9HkzYLBo0xFNo/q8y6vaZy671eOT1Lxmav1rMJAWjwTDMB8CGKnyp58RQt7pe8/PAAQAvKVxjWoA1QAwevToFLWUQomPVPZPwbWvdk4bTRY6nl1xgd2MGwqysP7BiXhyx+eyVLi8LBNyrEa8vGBSwjsotSA5DxfAdTlWmEzydEzBKOJ5XpECG82YUfM8JFLeWYALBEMiRYPgnHsozZ88T3Cmw42zHR5kmUNG7w3DsrBt2TScanNjuMMMhzW01H19qQfv/+mCIv1yw8OT4bAYI/YbYdxpFcDrcHP4h+1/FPsMzxNcdvvg5YJYPbdUzKgChp5xmRZDghDynUh/ZxjmEQBzAcwmGvmphJCNADYCofSlpDeSQkmAVPZPhmEwp7QIP/rmTfjp2/0Le8NCJ4ZruHQTmbRYlsGNBXbk2ozYvHQqrrg5dLg5vPz7k1gycwwcI7NxfX5Wol9LM0hOraCSNECyMNuqyx3M8wQuL4cLrl6ZvsZri6Ygx2pM+HczGw3Y1diCNRVl8iC/hc6MO+ceSvNnp9eHS1d7ZbLXayvLcONwO/79yFk8MmMMHgsLEt7z2TnUzZuAkmE2nGp3ozDbghHZymMJKcKRWv2BU4pnLMhsC31GLc1UeI+gL5FpxmUiZNzRBsMwdwNYBeA+Qogn3e2hUDINAwP88/dKRSMC6D+n5UnyFC2lsCyDXj+PRa8fQWX9J6jZ0oh9TW14eucJeMOyFuJV59MKkqvd2oi2Hl/EtkVzBwvejs9bukQjQrj+ss3HROOsYaET26uno2GhE3NKi2L63QrsZjx513i8eeg0Vs8txc7acmx7dBrGF2W+oNBgxsvxqsGTQZ6g+lvKYNxVu05gdukILHnjKDp6ONTtbYKRZaM+I+FIrb3Hhxc/aEbdvAn4/U//Hs8/cLvMQLCZDTjf5RWNCOl9a2eNFYM2M824TISMC7YE8DIAC4D9DMMAwGFCSG16m0ShZAY8TxAkABioukv9PK8Iqtd/ThUAACAASURBVIxX0VJxb6JeI0BqJwQCPM53ecV8/F2NLWJlQwARgxlNRlYzSC4QjL1GiBTh6GLd/Imq1zcbGIWAVn2VE/l92Sh6EI5cnru/bNAFyw02hOMpIe1y3fyJ4AlBkCcwGVi4vH64fQEMd1hUn3dRtkVM3Y10FBZ+DDau0CEPvuwNiMXxBO9WoK8wmNrxY+mobNTNm4BReZG9H4ONjDMkCCE3p7sNFEomEs1dOqe0CAaGQY7ViB015TAwocX50lUfFm04lLDCpIFlVc+GDdKy5irtW7+/Gc9XlEVU1uR5gp7eAAr7JvjwexgN/c5T4YjCywURJARWU3Q56mjn21yQKIo51W5tjDnbIlOrMw4lhJgILxcEAVC7tRGFDgtW3j1etqjXVzlhM0P1eWdbTWiocmJUnlWzIm0k1VMgZJxajKw41lg2ZAhf6PKiw81hTmmR4phuw8OTcUNBFvJsQ8cbAWTg0QaFQlHC8ySiu3ROaRFWfHsc/sf/+TO+OH8V511e+AI8WAa6I8Yj3bu92weA4JUFkxXFiQx9c3CHm1NtX4WzBF4uGLEdHW4Oi14/gnePn1MUv6qvcqLIYRHbcqbDjeaL3Xho42F864UDeGDDoahaDeHn2+G5+kTD2zKUAuKGCi4vh+5eP2xmg2j81c4aqzjeqN3aCAPD4NWHlX3WbjHgtlE5GBbBAFULwF2/vxmXrvbib1c8+OJcF1ZsO44HGz7BFY9f9D4JsTLP3HOb4lhl+VufwWE1DilvBJCBHgkKhaKkw82J7lIprZ1ejCtyYOXdt+GF332pquZXGObeFRbIcK0JPbuyOaVF2Lx0Krq8frR1+/CH5ksYM/wmnOsMhTOp3avAbkYwykIteAzWfXgSLZ1ebFp8BwwsA4uRxYhsqxho2eHmcLZDmccfLcNCqqMhnG9Li2vFk22RrHoflNjwckFccfvhD/b3qetyrVg9t1RRc+ZCVy+ee+9L1M2bgNEFWbjg8uLNQ6fx3P1lYu2Nth4f/EEeJgOLIodF7GvharCCiur8hk8UHkFp/yuwm/GT79yC7t6Aap/3BxI7pstEqEeCQhkESFMLpRTn22AxsjAbGU01vydmj5N9Zk5pEQiAsx1ufHH+Kn6254Tmjj58V7avqQ2LXj+Ctm4fdjW24PvfKBZVHB/aeBgr7x6PSSV5svYVZVtgNUVW1pQqb+5obMVd6/+ARa8fgdlokGVrRBLHiuQ9kKaMvrxgEiZcn4vRw7JQ1BepH6uqoF7dCUryCRKCLLNBHA+TSvLAE8iUJp/67njMKS1Ch5vD8RYXlrxxFI+8fgT+IMGTd41Hgd0MjgvgXFdI/+Gri9149t0v8NWlbgT6FvpwNVg1FVWpuqu09PxwuxlZ5vjVZAcb1JCgUAYB0tRCIbtgZ2053np0GixGFkZWO1BxzHC7OKHNKS3Cj2ffgh9sPIzK+k9Qt7cJj8wYg/X7m1WPO7RqdNw2MhvP3jdBEVfw9M4TouEipD5el2tTlRuWLtR6F/JI4ljRJuhI2R1SQ0OPYJeWyNBlt3Z2CSU5WE2hPiCMhydmj8Pj25Ty7c/ccxvqD5wSP9fa6cXYIjvGj8juU5t04+FffyobBy999LWYIRTeJ7XG13W5VmxafAeChIhZSizLYu0HXymO0TIxFTgZ0KMNCmUQIKQW/vazFjx+Z//EKRbPutiFO28bqeqez7IYxEhzhmHwYJ9rFuifdFfPLVXd0dvMBmxafAeyzAbRZdze44PNbNQ0MsYW2nFw1Z0Kd38k+V+98sCCOFa4oFAyahLEEijZ61f/7r3+oee2zjSG2y24oSALS2aOwaaDp7Hy7ltVn0U4xfk22Eyh+ISLV3sVacDCOBAyhIQ+uXv5DHh8QQR4ZREvwbsn1bB4bdEUjCt04Mm7xmP9/masnluKArsZRdkWXJdrG5LHX9SQoFAyAL11Hh6ZeZOqIbBp8R3YdvgMXlkwWWZkhESq+nff5zo9qpNugd2s2NHzPMGlqz6F0E9htgUGNpQRoma4nGp3Y2SuFeNH2BTfIdJCrWchF8Sx8rJM2F49HUECWE1s1KyNZGNgGPUMlqG3RmQcLMtgdH4WbCYW//S922DUyCY62+HByrvHiyXupcamP8hrjgNphhDLMmDAoOo3n6LQYVEIUf3s3lLN2ip6U4GHQqwNNSQolDQTKc0sfCHWyi4wsAwa/usMjpxxyYLOhodNSlqliov6gsSkqEWtP73zBJ5/4HYs3nQUm5dOVWhWCMFn7T2+lBWqYlkGw+wWwJ70S2sSPtk7rAa8+vBkmWLi+gcnwmYeeuffmUin14/5DYdFrQYtpcn2Hh+2V08HyzBgGaDV5YHVZBDrbYSPg8Jsi5ghJCB43lo7vXjxg2ZxfBVlW3DFzWnG6+gxjPWO/UyHGhIUSpqJpc6DliEQ7HO7Hm9xoWZLo/j6nuUzZZ9XqwIqxDGET1xaRxcmA4vWTm8oXXPFTGyvni5qNAiaFsLnM414dn9qk319lRNmI4O6eRPE+g75djNyLPoFrCixIX12QH+G0PEWF178oBlblk5FW7dP2Q+DBFe9Pjy+7bj4/DYvmYr6KqdMgKyhyolR2RZZcC8gH3PC+CrOt6Fu3gRwQT6h2irJqPGSCVBDgkJJM1oLttpCrGYIrKkow85jf1NMjGpxA7GUKtYyWnhC0LDQiTybCV4uCJvZIJPrFt6XadHp8e7+1Cb72q2NqJs3AUveOCq+TzDcBtMCMFiQPrtChwVPzB6HFx+ciAsuL9bt+xrHW1w4E5YWDISeyZnLbgCQPb9Fm47gnRUzsKOmHIEgD2NY6qcUtTH32qIpsBhZPP/+lwpvSCzxOrGM/UyGGhIUSprRWrDVFuJwQ4BhGBgY4JYRNyPfZtJlIOgNKlSbQF9eMAk+P4+6vU2yiXPz0qlY9PqRpAY/AvGpWGoR7+5Pa7LPCjvGGIwLQCYR6VkLz67QYcFT3x2vOMZ489Bp5NtNiuMm4YjjmXtuld2rtdOLXo7XVWxOy/gGgOfuLwPP89hRUw5CSMwxDrGM/UyGGhIUSprR2vHoWYhNBlY2cSVzNxw+gQZ5gvNdvXgqrFjYss3HsHv5DF1GTCwIKpaXrvbKMjQ2L50Kh9UIf4CP6V7x7P54noBhGOysLUeHmxOFjoQ6DVIG4wKQKfA8wTmXB/4gEavLSuu0CM9u9dxSmZZDocMCLsDjn79XitZOD/KzzOJxk3DE0d7jS/hZCVojwtFKh5tDgd2c8HhLZOxnEtSQoFDSTCzHDQMdnCX1XvA8ARhlap2Q9mhgkNSoczUVy0KHBZeu9mLR6ydi/v6x7P60So4Lu98nZt+CIN9/Pj5YF4BMweXl0OnxY7nEm7Bu/kT09AbQ6vLAyLKYU1qEPJtJfH6TSvIU3omGhU7cUJAl847VVzkTflapGnexjP1MhiFk8CuxTZkyhRw7dizie2585r0Bag1loDjz/L163pb2Eamnf+qlvduH+zccVCyGA3U2r3V/IV5AbYKNN71NSFV9aONh8bWGhU7xWEV6fz3fX+9iILzvYlev6pn7jppyFDks6PJx8HI8+L6Kk4UaZ+xRGFL9M17OdXrw0MbDEftVfZUTPb6A6BHT6gs7qqcDDCMeNeTbTLjq8/cdmcSXLpzucZdGdP1IVNmSQslAhEJZ5zo9oloekNrgLK17SlFToFxbWYaXPjoptkVajCsRKWmb2YAChwU7a8vRsNCJSSV5sh2pgPT7R/oOetUrhfN4LSluQghYlkHbVQ4/2HgY31p7APMbPsHJ9h4qka0T6XM67/Jq1mIR4lCEANcxBVliES6tvtDS6cWDDZ/gam8gpAthZDHMHlI0lcqix0Kkcadn3Ax16NEGhZJmwnfs+TYTTrb3qO6cUxWcFYuWhdQVCwArth0XU+0A+cIeb4CjIIYVnp1CoF4W2mw06PoOkQJNhefg4QJYPbcU/gipfUMlbS8dqD2ntx6dpvpbu7x+8d+tnaGKtv/yzp+xem4pijRKzru8/qQ/D61xZzMbIva5oSA2pQfqkaBQ0ojqjr2tG+v3NysWKSHAK5biUnrRWhjV6m9Ia1aYjQa098jrS0gNm3g9KGrtWbXrBMYUZKFhoVP1+8fyHcKRPoe/X3sAdXub4LAasW7+RNV7DZW0vXSgVqfkufeaFCXq11aWyWplFOfbYDKwaO/xoWZLI/5xx+eKWhavLJgsfiaZz0Nr3AV4otnnrqXCbtQjQaGkEbXFr2ZLI1bPLcW+pjbxfcKkmKrgrHgXxmhR5/F6ULTac/GqD8+++2dFGXCWZRJa3NWew4ptx7G2skz1XkMlbW+gCQR4eDnlc9rX1IaffOcWUTXSH+SRZe43UgXDwmxixP52vMWFNw+dlpW1ZxmI3rFkPg+tcXehy6vZ564lr1XGGhIMwzwFYC2AQkLI5XS3h0JJBVqLX7iHQTopxlJcSi+RFsZI7tlohk286W1a7cm2hqYsIQBvz/KZ4r0SWdy1nsP1eTbYzEbxOwnn4VwgiG2PTsMv32vCvqY2mrWhA54naG7rRttVn/oxgcmAur1NouBUjs2EN5dOhccXwPmuXrzwu2b86gffwIgcC96uKcc5lxcdbg4/3fG5mJJbN2+CeL1kPw+1cRepz11LXquMNCQYhikBcBeAv6W7LRRKKolU+2IgUwu1Fvx8mymhuIN4PSgFdjMaFjpRs0Weevn8+1+idtZY8XXppJxITr70OUwqyUPtrLFiASepESH9LeaUFuFn95bi5/eWisqIQ/H8O1l0uDnUbGlULX61pqIM24+cxe7HytHezSlSboWqs+dcoVTjMcPtGJFjxcY/9Ot6vLZoCkbkWFQrz6aKSH2uw81dM16rjEz/ZBhmJ4A6AO8AmBLNI0HTP69NhkL6p1aA4LhCBzq9/gEN0lLzPHS4Oc20N6lATyraeKnLiz+2dokFyAQxqO3V0/HQxsOq6XfxBrcJz2H9/mY8MmOMQvJ4/Ihs2W+hpmEQp65ARvfPZHKu04OZaz4GADzoLMayb90EA8vAbGRhNrIwsgx6egNYIKmmCfSngebbTfD5eVGOXdCNCBWmYwdkjKih1eeGSEEuXQ3NOI8EwzD3AThHCPmcYQbNj02hxEWkHXv4Aim41FNlWKh5FrTcszzPp3ySZFlWVSfA5fVrehviPfYRnsOz901QlGkXzrWlv0XtrLEyhcWhfP6dLASvT6HDgnmTrseSN47K+s6IHAvaun2q/a1kmA0Xu3rxzO4/KeKJkvGbJ5JdodXnhorYlB7SYkgwDPMhgJEqf/oZgH8GMEfHNaoBVAPA6NGjk9o+CiVRYumf0Ra/gdjZaE2kmtVGCVIeSKZVqXS43Sx6RJI5KUcq0y78LsJvEU3PItNJx/wpPM+LXb2qRtj26uno9QdV+1vLFa+mrkciv7mWgmmyxlcq4pkykbSkfxJCvkMImRD+H4C/AhgD4HOGYc4AKAbwGcMwCqODELKREDKFEDKlsLBwYL8AhRKFZPZPtXS5ZZuP4bLbF+WT6oQL6AQCvGaamlbam9aC6/UHFelt8Qr2qAlI3TYyByNybSjM7o9HSKYgkGAsSBHOtaW/heAVUXvfYCCV8yfPE1xxh57Hha6Q4NS5Tg863BzGFTowtsiu7uUiQLbViLWV8pTOhoVO3DLCgQKHBXNKi2SfS+Q3Fwz0z1u6RCNCaIvetGFKiIw62iCE/AmA2FP6jImoMRIUylCm169+vNDr52O+lpp3Y9uj01QNld3LZ6Ao26rqntUKJDvV1oNurx+j8qzIs4WOHRLxpujx1pzpcONshwdZZgM8XBA3FGThxgJ7XLvJSEGnHW4OOVYjdtSUw2xgFMGg13rWBs8TXHaHCmTxPMH2I2fxrfEjlCW2HWZF35lTWgSWAfxBgiBPsLayDCzDwMMF4fEFce9L/w/F+Ta8WuUEgKRkygjpmevmTxzU3qVMIKMMCQqFosTAMKqLtiEOr6tabrvWubTHFwRvJ6qLudqCK5Rsbu/xoW7eBIzMtaLAYU7pEYjLy+HS1f6aGILeQF6WCcPs8cdKSA0nTaXRomvj/FsPagaqtLQ80P/s310xU9Z35pQWYcW3x4m1NoS+9MLvvhKDa4XPP7a1ETtqyvGL78desjscIeZF8C5dC9kVqSKjDQlCyI3pbgOFki6EuAWeEGxeOhXPv/+luBNbW1kGmzn2iU4teFLLu3D6sht2izFiINn26uniZPziB82iGFCW2YBlm49h27JpKd3tebmgWGJcuPbTO0+EFh97fNcMN5zau9WPlmhgZT9qBuoV9//P3rvHR1Hf+/+vmd2d3c1ukg25cEtQSmM0paEQ7vbRo+V46THKsSC23ARUQqnlHL9esEfTeg71fEXwcOpDMUGPILcWBC0eraKlpX5/AooBoTQaEUETbrmQy2az9/n8/tjMZGZ3Znb2kmST/TwfDx4PSGY/M7PM5z3vz/vzfr/ePuXtL19QdNbc/iB8AR5LNssdjtV7T6KqohRr3qqLkMkmhGB0TkbC18wZDbi5tAA2zoBXl03FN63deO7AaTR3edM+uhQrKe1IUCjpitIKb+OCSfjFD4vR0uXD8KzerYNYUEqe3FvbgJqF5RG1++v31+P5+RNVxxKSMYVyPAFpvwO1aEqyVntqzZ6CSaxqj9awKR16KUQjFgeVMxpEZ+18Wzdau5QjYrk2DuvmluGZd+sjPp8McqwmrJp1DVZInvvqheUY6TAjx0o1QWKBOhIUSgqitMJbueMYdi+fjhHZ1rhfWEpbEg/eVIKR2WasmT0eGZxBjC40d3lh5QyaZadaWxxCU6N4RaL0YDGpCEn11PEn42WgVrliMrJDQScgbqROFMMwuLm0QCbrfuxcK7bfOw0tXV60unzYW9uAf/nHa2T/95zRoNqIbZTDCo8/KJPJTuTZCXf6CIjoRAC9HUalaqkUfVBHgkJJQVRXwUGCEdnaTkQ8ktYAMCLbErHHHd6BU0nVsmR4JvY9MBNuLw9PgMelDjfyMzn854+/C4eVg8PKxZ1LEG3Fn2cz46XFkzWFpABojhHtHGplqESlYdNQ3fKQfk9WzoDLHV7cv+0T2WoeCCVC3lxagIrvFWLh/3wk+/1ohxmXOj0ghIj5J1cNC23VCVtUwtZdBsfCYmKxa/l0BAlgMbFir5N4rj3c6dt+b99uu6UT1JGgUFIQtVWwVt4CoE9zQil5kucJzEZWjEp0+4IwGVjFZLnfL58Ozig36k2dvogXbXG+Xfx9PC9WvfeiJST1+sqZaO3yqY4RyzkEZyjIE/zm7Trc+/1vpc2LKPx7+sPKmXjgd8fF+8+3m9Hs9OLx20pRVfEdMAzwk57kSaB3tS9NwBS+60yLEc+8Wy827Gp3+/HGsfPImnl1hLZDXhwJtIByhO9si4smWSYJ2kacQklBcm0cahbK22WvnVOG5w6c1nxRxdtKu9Xlw+JXPsbSLUdx96YjWLrlKC53ehRflOfb3PjxxkOi1oRaB9M2SZJcPCiNu+H9elzq9Mg0I7SEpDx+XvP70Pt9Cc4XZzRg/ssf4b26pkGvJaEHQaOjsb0blzo8yLeHXuSODE78zgS58Kp9p/AP6w7ipy8dgTfAK/5/SBMwhe+aZRixNfjdm46gclstfvTdkUnVdlCK8D134HTEHKNJlvFBIxIUSgrCsgxGOiyKeQtaL6p4Ow7GkiwnJFIqSUfHcs5ohI87sciBe2aOFSMP0uiBWgTHwEDz2mK9dunx1QfPRDSfGkovIqVojZD/YmB7k2iV5MK/ae1W/P8IdwRCCbmI2Doam6csWhXvM6X0fDR3eTHSYaElvEmARiQolBTFYeUwItuCh147gcpttbrK0rSUGbVQ+tze2gbULIqMilQfPAMgUjo61nNGI3xctf4WrS6fqgKnldO+tlivXXr88YZ2rN9fjzWzx+ODR0PKm0Mp0VIpWrN670msuGEceEJEBUolufDnDpzGiwsmyf4/Xpg/CXtrG2THFeZYwbJshIJphjm5z5Ta8+GwcsjPNGN0ToZMLZUSGzQiQaGkKPE0/Ym3lbZaNUdxvh27K2fA7QvimyvdMq2IcOnoZFdmhI+ba+NUV6laSaRa1xbrtYcf39zlxYhsCwod1iH3ElKL1jisJuw8cg5zp4zBmtnjZS3vBZq7vPD4eeyunAFCQmqVO46cwz0zx6LuojPiu1ZqUpfMZyqdGmgNBCnZRjxWaBvx9GQotBHvCxJppa3WDvlcqwuXOz2yzPrwpMW+0FMILzGUJlQCvS3No8loJ1K1Eet4MZKyz2ez06vYQn7X8ungjAY4LEY0u3xgQHDF5ZclRq6bW4bhWRZRqlwUV+N5BAnEqg2t745qdKQEg7ONOIVCSYxEWmmrqVhenWuDI8OkWorXV10OpePGu0qNdm2xXnu6dHRUi9aMzO6NvoxyhLYKhmcRvL5yJjx+HgYGsHIGOKxcQs9HunzPQwHqSFAolKiwLBPqXRGn7HSyroGGp/uPWL5vlmVQkGkZgKukpALUkaBQKIMGukrtX+j3TdEDrdqgUCgUCoUSN9SRoFAoFAqFEjfUkaBQKBQKhRI31JGgUCgUCoUSN9SRoFAoFAqFEjdDQpCKYZhmAF9HOSwPQEs/XE4qQO+1lxZCyK39dTFK0OdTRrrcJ6DvXgfL85ksBsP/P73GXnQ9n0PCkdADwzCfEEImD/R19Af0XgcfQ+U+opEu9wmk173qZTB8J/QaY4dubVAoFAqFQokb6khQKBQKhUKJm3RyJDYN9AX0I/ReBx9D5T6ikS73CaTXveplMHwn9BpjJG1yJCgUCoVCoSSfdIpIUCgUCoVCSTLUkaBQKBQKhRI31JGgUCgUCoUSN9SRoFAoFAqFEjfUkaBQKBQKhRI31JGgUCgUCoUSN9SRoFAoFAqFEjfUkaBQKBQKhRI31JGgUCgUCoUSN9SRoFAoFAqFEjfUkaBQKBQKhRI31JGgUCgUCoUSN9SRoFAoFAqFEjfUkaBQKBQKhRI3Q8KRuPXWWwkA+of+Ufoz4NDnk/7R+DPg0OeT/tH4o4sh4Ui0tLQM9CVQKKrQ55OSytDnk5IoQ8KRoFAoFAqFMjBQR4JCoVAoFErcUEeCQqFQKBRK3FBHgkKhUCgUStxQR4JCoVAoFErcGAf6AtINnidodfngCwTBGQ3ItXFgWWagL0s3g/36KZR4ufqxt2M6/tzTt/XRlVAGmnjs4FC2ndSR6EcCAR71TU5UbqtFY5sbhTlWvLR4MkqGZw6KB4rnCeovO3H/1k8G5PqH8kSkUCipgZad4XmCdrcPF9s9qNyu344PtO3sa+jWRj/B8wQXOtyiEwEAjW1u3L/1E7S6fAN8dfpodfnEiQDou36eJ2h2enG+rRvNTi94XrfGScQ49ZeduHPjh7h+7V9w58YPUX/ZGfd4FAqFEo6WnRF+d6KhQ3QiALkdVLN38djOwQR1JPqJVpcPTU6v+CAJNLa54QsEZT9L1ss32fgCQV3XL5DMl/9Qn4gUCmXg0bIzwu8yOEOEHcy3m+ELBPHNlW6cOt+BB3Yel9m7WG3nYINubfQTvkAQrS4fCnOssgeqMMcKzmgQ/52qITCeJ2AYBntWzECry4fqg2dwvKE94vqlSCflxCIHVtwwDi5vAJc6PRiRZYnpfob6RKRQKANPNDvT2OZGu9svs+MTixx49NYS3L3piGiz184pw77j53GpwwOb2QAjy+Lm0gK8V9ckjqtlOwcbaedIDNQ+O2c0YG9tA9bOKcPqvSfFB65mUTkMLHC+rRuc0QACougRv7HyeuRnmhO+jniThMKdm7VzyvDqobP4l3+8BjzPo9npjdhLdPsDohPx8C0lsvuO1TnijIaoThiFQhmcDHT+k3B+AJp2pjDHiuqDZ2R2fNWsYjyy56TMZq/eexJbl03F4lc+Fm1e9cJyjM3NQM3/OyfawFwb12/32JeklSORzNV++IOfYzWh0+uH2xdEkBBYTAbk2cziuLk2Dg/eVIIN79ejqqIUuTYOBZlmMAxwx/Mfitez/d5pfbbyjvX+hXt0+wO41OFBvt2Mxja3OFF+v3w6/uN//4736ppkYwFA/WUnLnV4UJhjxYobxomTTrifWJ2jHKsJ1QvLsUKS4FS9sBw5VlPC3wuFQhk4+iMKq+SoAEC72wd/gEdzlw8rttci327GurllomMgfeHzPBFt0Pr99Vgzezy+lW+DP0gUbTYhwLN3TUC724/qg2ewYnstdtw3Dcu+/y2wLDukksXTypFQ2/+SvtDUHrhwp+F0c5fswd+ydAraXD48uPuE4mRgWQYlwzPx1J1l4jgGtteJEK7nbItL0SMO9iT7aD140bx6PfcvHUspCrF+fz2ON7Sjsc2NSx0eMVQnHQsA7t/6CfLtZqydUwazkU3YOWpz+/HcgS9QVVEKh9WE9p5/P3VnWVIiNRQKZWCI1S5pVVQo/U6tWs5uDuU6ePw8qvadEhdJz7wbchLGFdhhNfWO0+ryyWyQP8iDJwRnmrsUbfY3V7qxdMtRme1sdnpxVa5tyNmstHIkou1/qXnGZiMrC1H97v7pEQ9+wxW3+DAKPwufDCzLyB6g823dEdfz3IHTqFlYListWjunDL95u07zpanHq48lz0Bpcq/eexJVFaWo3FaLwhxrRKJj+F5iY5sb6/fX45m5ZQlvS/gCQbxX1yTbYwSAX98+NHIkqEYBJV3Ra5e0bBwAxd8V59tVq+W2LJ2KR/acxLN3TZCd/3hDO5ZuOYoPV98os7fhNqhmUTnOtXQrbllvXDAJv973d/F8q/eexJrZ49Hq8mFktiX5X+IAkzZVG0KyYGGOVfZz6QtNzTP+urVb9jNvgI948JUyeaOtuoV9fynNXV5kWo2oqijFGytnYvOSKciyGDGnvAg8z6uOpaeqwWRkNe9fitrkdlhNYm7H3toGxbGk93W8oR2P7jmJdT3OhHBcrPuDSt8VucqzJwAAIABJREFUzZGgUAYXShVpeue2noqK8N81dXlVq+VYRp48qXZ+nie44vIiwBPxuIlFDlxTYEeencOqWdfg1UNnUVVRij0rZmDHfdOw/fDXON7QLjvfmNwM7K1tGJI2Ky0iEoInu+H9+gjPUfpCU3t5ZnDy/3gDE5mQ0+0LxrzqzrGasPO+aWhyetHq8mFvbQPumTkW37R2i3//xe+OyxIz8zOVqx18gSDy7WZZ6P9A3WXwPI/zbd0IEgIDw+CVJZOxbEuv166WZ6CW3FiYY8UbK69HjtWEB28qQd1Fp+J3+dLiyeLEbu7yYniWBa+vnAl/gI8rmSrXxsnGHGrJShTKUEctolCcb486t6XJ21LCo6Dhv/MHedVqOZ4oJ09Kzx8I8LjQ4YYvSNDS5cXz8ydi41++xD0zx2JRT5T65tICPFFRCpBQ1JllgB+XF+J0U5foTBTmWNHs9OLBm0qGpM1KC0dC6q02O31isuMoh1VWhiis2MMfuHDVg5YuX0RCzjCbCS8umISf7Tim60XH8yQiz+KF+ZOw48jXON3UhWfnTRC3U4DQpKjcVquaoGjlDHj01hLZNb2yZDK+udIty9v47U++h3Vzy8AyjGaegdqLe2S2Vfy+SoZn4o2V1yvuV2r9Lh6EHJNkjkmhUPoPrVwIrbktOCBC8rZWRUX470wGVrVazmxkRDsuJE+OzbMhwxxKlAcQkVvx7F0T8PhtpVjw8kfiuZqdPjjdfnR6AjL7u2HeBPznHz9Hc5cXNQvLMdJhgcM6NG1WSjsSDMMYAHwC4DwhpCLecaSRhuMN7ajcVgsA+OiXP5Ql53AGJsJBWDe3DFcNs4oPaWGOFbl2DiYDgy1Lp4JlAJ4AZiODTIsRu5ZPR5AAFhMrq9oIR2lS/XznMVRVlGJ3bSOcHnXvWympKMCTiBKk822eiLyNf/n9p2Keg4BSnoGeF3d4zkf457USWBOtkqFOBIUyuIhXC0awlULytlpEWVj45NvNWDWrGGPzbDCyDB770XV4+p3PUFVRisIcK7J7IrCckcV1IzNVbXaz0xuRW/HQayfwu/uny6K/w2wc2rv9Efb3wd0n8Pvl08EAsgXYUCSlHQkA/wLgMwBZiQyiFKa/ubQALS6fzNusWViON46dl20PPPNuPZ6fP1H2QlWq2nhp8WSMdmTAkaHvYdHKQbi5tAC5dk5R/MnKGRTDg8MyTLrzNhySrQyt7RctRwHQ93JPRmlXqop0USgU/ahtl6rZNGF+C7ZSSN4W7HNhjjUiQvrmA9cr9sF4ek4Z/AEeLS4ffiIRjtKyI2o2GiB48o5SXHH5xZ8V5ljE8njpsf4gD7PRMOTtVMomWzIMUwjgNgAvJzqWEKaXJvs9cVtphLdZub0WP/ruSFRuq8Xdm46gclstmru84IwG5GeaMTonA/mZZrS5/QnLNaslGI3ItmDVrGvwk01HMLf6MNa8VYeHbynBzaUFeGnxZHj9vOK5gz37fVKEvI3wc3T7guLfX1o8GTlWU8yS3Hrlr5MhbU3lsSmUwY+SHX5p8WQEeGURPmF+hydvV26rxUOvnQAX9oJmWQZBHop9MII8wLJsTL2O1Gy00cCi2xdE1b5TuHvTEVTtO4WWLh/+7Z+uizjWZGBRYB9apZ5KpHJE4r8BPAogM9GBlML0at7m2DybbBtDGjoTVuDdvgCqKkrFSIHw2Vh0EZRyEGoWlsNqMsj234TSod2VM+DxB3G+3a143YSQiPFG51iwYd4EWY5EzaJy5Nk4fPDojbCYWAyzcorRFcFLV4s6tLi8umq/kyFtTeWxEyeW8lJaWkrpC9S2Sy92KNs0YX5rJVqH2yeej6yoi5aQqWZHcm0cahaVy6LWa+eUwevnI7YxHtlzEr+7f7rs3VG9sBwFNg5GY8qu15NGSjoSDMNUAGgihNQyDHODyjHLASwHgDFjxkQdMzxM3+z0KobZMswGxbyAaAJNsZYixjqpAjzB4lc+Fvf5lBKOSoZbxfEA4Ld/Oo07J43GtmVTESQElzo8yLNxGJ7d62U3O5Udgtd/NhN5drNqlnW3V9/LPRnS1sIY+XYzVtwwDg6rCd2+ICxc6k7QWJ9PCqU/GajnU2m7NJqNULOVQKR2RM2icllPi4lFDqyaVYwAT2BkmZj6XbAsgzwbJ9vqXr+/HuvuKlO0fTwh2HbvVBgYBiYDi3y7OS2cCCBFHQkA1wO4g2GYfwJgAZDFMMx2QshC4QBCyCYAmwBg8uTJMbeTVPNy1RIklVbggkDTmrfq4ipFjGVSGXpqnpVKlWoWlosOjzBes9OLQ1+1Yndto2wcQXlSQG217/YH0dKl7GTsrpyhqsBpMrJodnpl+SSJlm3m2jhsXTYVlzs9skTY6oXlyDanpsef6PNJofQlqfR86intVkveDrdPldtqsfO+aai76ES+3RxRybZxwSQAkMn6a9kilmWx5q06mZ1r6VIuJ2240o2F//MxPnj0Rox0WJWGG7KkpCNBCPklgF8CQE9E4mGpE5EMtLxc6YtQ+JnaCvy6EaExklVFoDaprFzIwTje0C4mHOXaODgyOAyzmSLOrVd3Qc1x+arZhavzMhTv2R/k8dyB04oOTZcnIFMBFSIYiZRtsiwDq8kQEU5csb0WuytnYFSaTdpUg6pyUhIh3BYzDAMDE8qNCrcV0shwuCIlELILBpYRxxI6cgq/W7njGLYsnYonKr4jk79WQ8mO2jhDhPrws3dNwNPvfI7CHCssptRb2PQ1KelI9BfhEQGl7YuaReUYZjOJqmaRGcdGTdnq8B4dbW6/5gtVy8ERHujjDe1Y81Zdjx4E4LBGetR6dRdybVxEMyxhy2bVrGLFe/YHCZq7vLIM6m5fEPmZZvz4xUNR8ybiwRdU3vsMBNXVPikUyuCAZRnk2jhF+5tn48QmV9LE6/B23kDvVkWujcM3VyJbEAjRVquJhS8QVHRWwq9reJYZa2aPx5jcDBACtPUkZ66/awLyM834prUbT7/TqxchaFCkEynvOhFCDiaiIRELStUBldtqcbKxE8+8+xk2LpgkyzgWthSUUKpq+PyyE4+/cVKzygHodXCEKhFp06/XV87E/3v0RuxaPh1X52XAblH3BZXGUTqmINOMbfdOxa7l01FVUSrmfQh9P8LvedNfQ9srzV1eMYO6INMMQpS74CUjKdLIKsubG4Z4WRWFki6o2d9PGzsk9rJ3QSFs84ZXgQgOh7D9KqUwx4osixH//r9/j2qHBfwBHr4gDwbAI6+dwDA7h//842cI8gRr3/kMviCPx2+7Djvum4YR2eraQUOZtIpIRNM90NJ2EBJ0Ni+ZgisuH7p9QYx0KMtVA8qTYsX2WlRVlOK9uqa4Vuuh5B/lBMhENBWGZXBw+4N46LUTsvtv7vJipMMii2oYWODQV6043dQli0aMdFgQ5JXV5ZKhLW82sti4YBJWSpRDNy6YBHMK5kdQKOlIoqJxWvZXmp8l2Bhhm1epU6cvEMRzB05H2Iy1c8rw1Nt1mFNepMsO8zxBi8uHNW/VoaqiFM1dXlxsd4sRWSH52+kJYJTDgmzL0JO/1kPaWGE9ugdqdcPt7pDwyHt1Tbji8uGh105gRLZFcUtBQGtSSP8tXa0rNbQJJxmaCuHnYVkGo7IsEdGHlxZPhsPKyaIaDmtoz1AajRC+C7U68WRoyxMwePvEeWxeMgV/fugfsHnJFLx94jwI0s/7p1BSDb26MlpEs7/SMnfhuOYuL0ZkW1DosMqirpzRgOYuLxgAVRWlsmjre3VNEXaY53lF29sqES2sPngGz8+fCLPRgBcXTIqIyObb0qdKI5y0iUjo6XmvlFgj5AsA8qZV0bxttSRGYVII/5Z2mNMTaUhUU0HrPNeNzIqaUxEt96Kv+mHk2jj886QiLN1yNO7qDwqF0jfosa/R0GN/w8vctXK/Xlo8GZc6PBFVF+F2WEnlWLCJ4fbW6+fx0GsnkG83R/TmSMctDYG0cST0vIDDX5JBnuA3b9eJOhHhTau0UJoU1QvL8dyBLwBErtb1TsREdRminUdv4qiagxBNVjteaNMuCiV1SYZonB77G17mHm2s4VnmiAqLcDv8xG2lmB8mAijYRM5owM2lBZhTXoRx+TY0XHEj327G8YZ2LN1yVFxYprsdShtHQu8LOLxe+ak7y/Dr22N/cSm9+HKsJtXx9E7ERNtpxzPhU6XXRV85KRQKJTGSITwHJM/+CmMNs4W2Y7XssJZNHJ4ZalmgVNV2vKE9acnkg5202dCJZ/9eT9WDFkJJk/Cwtrn9yLVxiuOp7Q8qOTqCg/Lh6hvFFrx6r03veaQMVK+L8FyOQEB5H5NCoQwsfZEfFYv9VcovCwR4XGh3o6GtG/4gj+GZFuRnhvIYpONq2cQ2t190IoBeIcIVN4yTHZfupE1EIpmhcb1h/lhW8rFEGhJZmccT0RiIXhdK350QkpSq0tEOoBTKwNMfW49qdlfJVvzu/mnocAdkkYTqheW4dnhmREKklk1Ua1kgdB+leVoh0saRAJITGo/FOYglASnWiRhvqVUs5xHOAfRdWacafVE+S6FQ+o6+3HrUsrtKtsIbIBGRBDUl3HCbaDKyMLIMLna4wTKMou3Tm3SfLqSVI5EMYnEOYl3J652IieYs6DmP9Bz5djPWzS2TadareeKJ1JJLPwsA+Xaz7PuLVj5LGXpQ+W0KoG13w+3sxCIHjCwj/mxikUPUe+AJAc8TxWq0XBuHFpcX3d4gzra48M7fLmLB9DGKtk9v0n26QB2JGNHrHPA8QVBFVjvRlXwySq1iOUdjmxvPvKss/CIlEQdH6bPr5pbhmXfrxVbtWuWzFApl6KJld6WJnhOLHHj4lhKxpUG+3YyHbymR9QRSsklK9ueF+ZPwwl9Oo9npk4nvDc9K71JPJdIm2RLQJ/gUDb3Jiq0uH37zdl2EhGvNInVZbb30R85C+DmEcicDA9XEp0SSMpU++8iek1g1qxgAxD3OvbUN4r/p/iSFMnAkw57qRcvuShM9V9wwDqv3nsSmv57BxgWTsGpWsehEAOo2Scn+/HznMcwpL8LxhnZUbqvF3ZuOYOmWo3D7aBQ0nLSJSCSrhFFvsqIvEMR7dU0yb7bd7e9pQKOdfRxta0BPqVWicrWxlnPxPIHbH4jbwVFyjvLtZhQPt+Ovj9wAk4FFvo1LqByMQqEkh/4uCdeyu9IcB38wKNpbX4DHNcPtijZJULIU7KO0hwfQux1SXGBHzaJyVB88I+pZ0ChoJGnjSCRrO0BvsqLwIha8WQBigo4aeidnNGcmGZM8luoO4XyXOjxxb+WEOy4Tixx49NYS/KSnDTCt0qBQUof+2F6VEs3uCjkOn13qFJUshe2Jm0sLxF5JgLKSZfXCcvE4YXtEuh2ydk4ZXj10Fg/eVEKjoAqkzdZGMrcD9NQ3x1NXrXdrIJqWRDJ0H2LRqxDO99yB06rd+KIR/n2tmlUsJjjFew8UCqVvGIiS8Gh2V9oXQ7ien+88hsdvK5XZpCduK404bsX2Wjz2o+tk2yPh2hFP3jGeLmRUSJuIRLKU1/QST111LJNTq/IiWZNcbxWJcL7GNjfW768XQ4uFOVbF7Ga1bReZPG4ftiSnUCiJ0d/2VA9qds/IMnhj5fXgeR5BAviDvOJxHW4/qipKUVygvB1CSGS1ByVE2kQk4lVeSyShKFZlzHhUJ/tynHjOJ2zlPPTaCXBGg2plh1KXQOn3ZTUZFe8hyBOqaEmhDDDJUrJMVsKmtEpOijQh80q3H/NqDuPzS07F45qcoW6ep5u6+tV+DgXSxpEQVrxvPhAK1e+8fxpy7dGdiERb48ZCsian0jg1i8rFBKPw6090Msdy3Xq3XXJtHGoWyduar51Tht+8XUe3NyiUAYZlGRTn27G7cgb++sgN2F05A8X59phW7Mm0r9Gq5KR2p/rgmYjjpBVhx861Ysd907BnxQzULArlTtAKMW1ScmuDYZgiAFsBjADAA9hECPltMsa+3OnVnYSYaglF8Y4jdNFTkpZORmKm2nUDkGVG59o43dsuLMsgz8bJKl6ERjm/vp1ub1AoAwnPE5xu7krIbiTTvkarkpPaneMN7eIW7LUjMmUVYWtm82ju8mFBTzdQwcn4dp6NbmtokJKOBIAAgIcIIccYhskEUMswzPuEkLpEBlV7cHctn6740u6rhCKt0sxkycwK4zQ7vbhz44fifeTbzbjU4YHNbIDVZISBRdTJrKeUNPy61RyUXDune2+VZVkxAzvasRQKpf9Qs6W7K2eAEJL0nLBoqFXJ7a6cgfNt3WDCpK6PN7RjzVt1qKooxZq36kQn6HKnB88d+ELmjDx34As8ecf4CGltSi8pubVBCLlICDnW83cngM8AjE50XLUHt7HNjTs3fojPLnUiEODF3/VFrkF/b5dI71koa6radwo/eOYg7tz4IS62e5Bvlzsu0skc7/WqGRojyyhuuzAguNDulm2v9EVHQQqFkjhqtvRCu1u0E59d7MQVl/pWaTLta7ituLm0ADvum4ZmpxenLnTi1Q+/wosLI7dKqw+eEW1Tu9uHIOFxz8yxWPNWHe7edARr3qrDPTPHggHNy9IiVSMSIgzDXA1gIoCPEh1LLdO43e1HY5sbldtqsfO+aSjMyRDrkmPtlBmN/twu4XkChmGwZ8UMtLp8sHGGiLKmyu21WDN7PJZuOSp+TjqZta5X2HtUilSoGRq3Lxix7bLjyDn8oGR4hIxtcb4dw7PM2LV8OoIEsJhY5NmoPC2FMtCo2VIhf0lqW0ZkWxLueKyFEDEdlmHC7soZMDBAS9j2xNo5ZXjr00b8fvl08DzBZ5ec4lapID7V5QmAZRnF0s9dy6cn+I0NbVLakWAYxg5gL4B/JYR0hv1uOYDlADBmzBhd4yk9uGvnlGH9/noAoYemyemFlTOKVRbJbo3bX/XXSlsLry6bqnjusXk20ShIJ7OWWiXP85q5FVrlYeHbLlUVpYoytjvvm4b5EmPw0uLJyLMNjk6f8TyfFEp/kejzGc2WAqF5nMEZktbxWAklO7fzvmmoDOv8uXrvSVRVlOJShwejHFZxyzRcfGrfz69XtHcUbVJyawMAGIYxIeRE7CCEvB7+e0LIJkLIZELI5Pz8fF1jSh/cDx69EWtmjxe9UqDXo5a+1GMt4QSAQIDHhXY3vm514UK7W9d2icmY3P8KpUjCN63diufOMBsihKcAoP6yE2eaXMplmEQ5t0JYkejZlhCcKofVpDh5m5zeQStIFc/zSaH0F4k+n+GCdbsrZ+DVQ2dFWwrIo73ROh7HYl+BkANxxeXFhQ43TAYWT//4u5hY5IiwGwKNbW4xgmpgENGbQzj+UqeHln7GQUpGJBiGYQD8D4DPCCH/lcyxhQeX5wmcHj+au7wAevfMXj10FpPGlCl+Vk/SYSDA4/PLTqzYLpdfvXZ4JoxGVtGTXze3DF2eAPJsyRM8UYp8PHfgNGoWloveunSVH37eZqdXbCG+dk5ZxLYDiSIYpWe1IThV7W6/ZphUafx0J9b22qkydqqRTveabKQJ1jxP8OBNJai76IyIUOh9EevtD8TzBOdaXbjc6ZG19372rgl4+p3P4fEHFe3JMBuHTR+cwaQxOaJt6vbJI65CaWi4vaN5WdqkpCMB4HoAiwD8jWGYT3t+9m+EkD/GOpDaw8myDEoKMrHzvmlocnrR6vJpaqnrLZNs6vKKTgTQK7+6u3IGRjlCKo/Ds8xYM3s8MjgD2t1+PPNuPZq7vEnNk1DaWmju8mKEw4zdlTMQCPIwGlgU2JVXAdHUKltdvqjVF9EqUASnasP79RGTt2ZROX77py9kx9OVAYXSv+h9uQsLh9dXzkS3N4izLS6s3x+ya3qF//SWobe6fPi6tRtV+07J7OxDr51AVUUpzEYD1s0tkzkZL8yfhF0ffy3a997tVURUc7x66Cw2L5mCDrcfI7Mtiuq8FDkp6UgQQv4/AAn/z0V7OI1GFoU5GbByRozMtmDSmDLxIQufQAREV5KkmvxqINi7veH2BWXJjQJaq209E1p6jJUzKCYydXQHsPiVj6NOVqkjIpRUFeaEmo4lKxFVMD5P3VkGnudlpWM5VpO4wsm3m7FqVjHG5tlAQEQVTAqF0nfEqjHDsgwKMi3gbQQ2sxHPz5+oaKuUbFksSei+QBAZnEF1+yLPbsaDuz5FVUUpCjLNsJuN8PiDWDxzrOgUCNfA8zxqFpXLGnjdM3Ms1u3/HA/8sBg2c6Q6LyWSlHQkkoWeh1Np1aw0gbbfO01XkqTJwCqu1I2G3hyIeFp0R5vQSsdsXTYVr6+cCX+AB2c0wMACdzz/oa7JGs1RCN+6YBgGBib0nceSMKUVtRCUSC+2eyK2Y2jzHAqlb4m3wkxrTqvZsiyLUXcSOmc0oNunvH0x2mGFwcCgucsr6kkIvxMWQdJrmPmtXPxi1rex7d6p4AnAMoCRZbB4xtV4/s+n8Zs7vxvTd5aupGyyZTKIt0JCaQKdbVFOOgx/+RfYzagOq1euXliOAolWQ6z6CHpkpZWOWfzKx2DAiIlMbl9sTcGidf8UIhOdngDm1RzGtP/756TqYrAsgyCPiAzswZR0SaEMVvqiwkzNlgmCUVLUFle5Ng5Fw0K5ZVIbumHeBJh7SsS17KtwDfl2MxZMvwo/fekj3Lj+r7jnlY9xod2DNW/VweULKWX6JYnyFHWGdERCbeXPMIxmeNwXCCLfbpapm73zt4sRIbCaReXIsZpknzUaWVw7PDMiD8EoqcqItexJz4TWc4zW93G+rTsulc2+1sUYiHbFFApF3V4AoWTseErheZ6X2dXqg2dwvKFdrKSQRiqU7CsQskuZZiOanV5sWzYVQUJwqcOD//zj53h+/kQMs2nbV8GmVFWU4uc7jymWiQr5YDQnSx9D2pFQq3V+8s1TePCmEtXwuJUz4NFbS2TJOhvmTcAwmwlbl03FFZcPrS4ffvunLxTHMRrZqHKqsUhh69kK0XOM0vdRvbAcT755SrEPhx76WkY8SAg2L5mC5w6clpXp0glOofQtahVmD+w8LiZRxmIreJ6gxeUTNRyklXImIwu72aDLvgIh+XzBPgMh1d5Vs4oRJER0ctTsq2Ar1crOc20cnJ4AqhcqOzKUSIa0IyGs/HdXzsCFdjdaXT5RN6LuolN11RzgifiQCqpnDMMgEASefuczvFfXJB6rNU6yyLVx2LpsKr5u7UYGF9ofvCo3Q7YVoif5USmvQXAigPiiCbHme+hBaR913dwysbqlZmE5DCxo0iWF0odI7YXbH8SZpi48826v7k6stqLV5RMjuoJdNRtZ/Or274CA4Msml6wSA1C2rzxPYGAhlrLn280RCz8tJ0ewpzyBou3KzzTDH+Tx3IEv8NSdZX1q24cKQ9qRAEKTgRCCudWHZT/XWjX7A7z4sEtVzwQPutnpEydTtNW33vKpaHgDvDjJhIkSfp96tkukkZDzbd0yp0jP/YTTXzLij+w5id/fPx2nm7rwxB9OxbUiolAosSHYi/Nt3RGVZrHaCiF6qWRXaxaWI8/ORY1uShcZ+fZQGX1xgR3/8dbfZVsmG96v13QCvAFesez8hfmT8MKfv8Tu2kYAoJ2GdTLkHQkg9lWzcHy46pl0D03aYS6WaouaheW4psAOk0n/il3pxbrh/Xo8ecf4iE574Z57qAEWjyABeELA9lRXsCwLm9mAzUumiHoW1QfPoLnLG1M0oT9lxC91emTGrC9bulMolF6SEXnUsquV22uxZenUqOeQ2sLGNjeWbjmKt1d9H8t/MA7/uutT0c7+993fA88rJ0pKx3BYOWxeMgVGA4MMkwFBAvz8h9/GL2YV40DdRbqFqpMhXbUhEGuVhHB8rk3ZQxY+F0+1ReX2Wlzo9Mhks6MR/mKdWOTAPTPHYl7NYdWOnIIT8/gbJ/Flswvzag7j+2v/gnk1h/Flswsvf/AlGts8qNp3Suxy9+itJdi6bGrM0YR4ZW6F62x2emVdP9VkxKnSJYUyMCSjE280u2oyMBGVGGqy+lJsnFF0IoSx/nXXp+CBCNsiHWNikQOzJ47G0i1H8X92ncBXLSE7+Q/rDuKnLx1B+dg8ZJupI6GHtIhIxLpqFo4XdNfDPeSR2RZ88OiNqt0ohe2McPlVIPSQNzu9MBmiJ2QKhK8GlDz68NW54MQoNcRavfckNi+ZgqVbjkZsH7y+cmbStgqibeuo1ZQX59vF7RJBjOrqvAxc7vRiYpGDJl1SKP1MMiKP0eyqlTOgZESmZrdfwRYKeREjsixgmMjGWqFFBo+7Xj4ckTdh5UKR2KtzM7CoR5yvqqJUlrzZ2ObGz7bXYtfy6RjNpcVrMiHSIiIBxL5qZlkGI7Isil74yGwrxgzLQEGmRdGJqL/sxJ0bP8Tnl5yqK2up0mU0wlcDah69UjmoWmaygWUUf56sumnp96AWNVErHW1z+0Uxqt/883hU7TuFG9f/FQ+/dgKP3lqCiUUOqoFPofQziUQepWOo2dU8mxnDbKHx1eyrkCj55B2lAIBFr3ysamdZhomwLe1uHy53elG175SsuZeanQwkQQ8nHaCulgbxeOHSl2P1wTOoXlgua+AllDs9ecf4mK6jON+O3ZUz4A/yMLKMokcv7SAarSFWkCdJr7aQokdfQqt0VE2M6pE9J7Fr+fSk5GJQKJT+J5HoBssysFuMsl4bSo221s0tA0/kTkBjmxtuX1CMdA6zcaINVLOTRmpfdEEdiSjEovcAyF+Oxxva8ZfPLmPHfdPQLGkMtmrWNTKly2jwPMHp5i7xxXxzaQE2LpiElTuOySaO9KHXaoi1dk4Z9nzyTcQYNQvLk7bCT0QgS3BmpGMI5WKOnrpu6kRQKIOXWO2qFH+AR56dk1Vp7Dt+HlUVpSgusOPr1m7kZZqx88g52ecKc6wIEoJ8uxkP31KCdfs/F21j9cEzEY2+XlxYDiuXNkH7hKCBTc0PAAAgAElEQVSORJIJfzk++6fT6PL6sXjmWBRkmvHkHeMjlC6jEb66F0o2Ny+Zgisun9hB9Pn5EwFb6DNqDbEYhgHDAL5APrYf/lqcjN2+IEY44gtX6vkeAH0CWdLtCul+aHi5GC39pFDSEytnAHFBUdjqV7d/ByUjMpFv4/DPk4rw9qnLMpthMRmwalaxaEuanT5UVZQityc6sWv5dAT4UHXbgbqLGD5h9EDf7qCAOhJJRunl+M+TihJqRau0un+vrgn3fv9buHvTEQDK2xJqXj/PE7R3+3Hoq1bsrm0UIxod3QHkWJMj8hSPQFZ4iFMY41KHJ2pyKYVCSQ8CPBEjqUBvAvnO+6ZhlMTOKtkWABibZ5NFjYVS/gMP/QPukXRGXje3DF2eAPJsVPguGtSRSDJ9oaugtrrv9gXFv8eSeMiyDIZnhcRcBA0JQTVS+nKOV0xL+NywDJOsNXg0gSyl6ywZngmbWbllMC39pFDSD0EwUIqQQB7eVFDJtmSYle2pyxvA7+6fjkCQR4AneOmDr3Doq1a6YNEBdST6gET2/5SQru6FcsixeTaYjSw++uUPwbJszM6K2xeMUKoDIL6c9bQuV0L9c/FFZFiWgdVk7NPEUEr/cvVjbw/0JVAGMYmKYwndQaX2tHi4DR3dAfz0pSOy7ZLTTV10waKDuDNJGIbJYhhmnMLPyxK7JHGcWxmGqWcY5kuGYR5LxpiDFWFl/tYvesshb1h/EHfVHMaVbn9cEQ810SdhMuppXa5EvJ/TIhliOBQKZWiQqD0QIrIb5n0P/9FjT/9+wRlRIbZ670msmlVMFyw6iCsiwTDMPAD/DaCJYRgTgCWEEGF5uwXApEQuimEYA4AXANwEoBHAUYZh3iSE1CUybn+RrP4a4XS6AxEPe7y5AtFyGLSqLrTury+6gfbFdhGFQhmcSO2BIP9PSMgm6bULbl8Q3kAQD+7+VFNvZ2yejS5YdBDv1sa/ASgnhFxkGGYqgG0Mw/wbIeR1AMmw7lMBfEkI+QoAGIb5PYDZAFLekYh3SyAarS6fTEBFIN6XdLSXs1r40GRkNe+vL7qBCtdL9ykp0aDbJukByzLItXFx21rOaMCI7F47paYjkcEZ6IJFB/FubRgIIRcBgBDyMYAbATzOMMwqAMmQAhsNoEHy78aenyUdpV4P8Rwj0BehfSC00m91+TS3I2IlXKkO6NWmN7BQDB8aWUa8v4lFDlRVlMLlDeBSpwc8T+g2BIVCiclmxvuZRGxtro2DxciKdkoQtZLarRcXTBKvi6JNvBEJJ8Mw4wghZwCgJzJxA4A/APhOEq5LyQWU/W8yDLMcwHIAGDNmTFwn0RM9iDXC0BehfSDkQe+tbYgQl6pZlBwRKaX73LpsKl5fORP+AC9GLC52uEUnQk3boS+3Ifpq2yjZJOP5pFD6ir58PuOJysbzmURsLcsysJpZUYTqeEM7Xj10FtvunQoGDM62uPCrfX9Hc5eXatboIN6IxM8AsAzDlAo/IIQ4AdwK4L4kXFcjgCLJvwsBXJAeQAjZRAiZTAiZnJ+fH9dJ9Hi0sXq90ZIYtdDyyHNtHB68qQSvHjqLqopS7FkxAzvvm4aSguQ84Er3ufiVj8GAkWnra7UCFr6XZGjyKxEI8PjsUqdm/45UIRnPJ4XSV/Tl8xlPpCDWz/A8EWX+peixtYKd9fh4FOZkYMO872HX8un46dSrYGQZLPyfj7B0y1Ecb2hPWkR5qBOXI0EIOUEIOQ1gN8Mwq5kQVgD/BWBlEq7rKIBihmHGMgzDAfgJgDeTMK4MPR5trF5vvKF9pSZXn13qFNuNS5Uqx4/KwlW5NhTmZMSkkKmF3vuM1gq4r0qleJ7gQocbldsik03pJKdQUod4IgWxfqbV5cNv3q6L2I6oXliOnB4ZfSXC7exPXzqCDM6AbKsRviCPli4f1ayJg0R1JKYBWAvgEIBMADsAXJ/oRRFCAgzDPABgPwADgFcIIX9PdNxw9CQGxpo8GJ7EyDAMDAyiZhQreeSV22qx875pKMzJANsjthJrwqHerQCt5Mpmp1d2L7l2Dg6rKe6kSp4naHf74PYFESQEJpYFy0BTDyPZyaYUCqVviCfhOtbP+AJBvFfXJEpcCz03LCYWTV3eCBE8wQ76g3yknd1ei63LpqL64BmsuGGcsh000J4bWiT67fgBuAFYAVgAnCWEJKUPNSHkj4SQawgh4wghTyVjzHD0RA/iiTCwLIMcqwlBnuBCuxufNnbg8TdOaobh1Tzypp5mX/Ggp5V3tPvs8gTEz8+rOYwvm1144o2/we0Pxh15OdfqQv0lJ+7edAQ/eCakh3GutVvzO5Imm04scqBmUTl2LZ+OzUumwMrROm8KJVWIx2bG+hkrZ8DmJVPw2I+uBQA8/c7nOHauFUaWxYV2N05d6BTtSSDAo/6yEy9/8CW8Knb2isuHVbOKUX3wDH77k+/JrmPd3DJ0ePzw++mCRQ2GkPj3lxmGOQFgH4A1AHIB1ADwE0LmJufy9DF58mTyySefxPVZPSv2WBP8eJ7gs0udYhhe2lTmqTvLFKMKTU4PfrzxUIQnXFVRivGjsjA6JyPme2t2enHnxg8jxlRrwx1+nwYWuOP5yM9XVZRizVt1ePOB6xHkEVPiY7PTi1PnO8QWwErjKuliNDu9ePyNk1h547fh9gVlXfqiJEMNeIaUnueTli0OPc49fZuewwbF8xkr8SRF6/2MUmLmC/Mnwsiyos6O1OY+ecd4zKs5jM1LpqCxza1qe64bmQUDAxhZBnUXnWL7gOqDZ9Dc5cWu5dPjssODHF3PZ6IRiXsJIb8ihPgJIZcIIbMRciwGBXof3FiTB1tdvoi9/NV7T2JOeZFiGJ7nCbo8AWxcMEnmCa+dU4a9tQ1xl3eqRTka29yK0QnhPkdmh67B6Qkofl4Qb3H7gjEnVfoCQWRwyr0zhHGVviMh2dTj50UnQvgczZOgUFILQeeBMxrEaGK0pGi9dlZpG/iKy6+oTDmnvAj+IC/24njuwGnFvIq9tQ2wmFiMzsmAJ8Bj6ZajuHvTEVRuqxWTLgM8EXPWKHISypEghES4sYSQbYmM2VeEOw05VhNON3dhw/v1mFNehFwbB7cvgFHZ1pgTGMPHVnuBCxMrnFaXD4tf+Rgzv5WLrcum4orLh1aXD68eOosHbyqJubxTuJ4gIdi8ZAqeO3AaxxvaAYQmTrvbL1PFzLVx4vVbOQMud3px/9ZPUFVRqrhfKIi3xOPgcEYDun3BmMcVck8a27tpngSFkgJoLcSEqEEy7Gs4SvZVbXGSa+NgMoT0IoI8QXOXF+v314t5Fd2+ILKsRiy9fiyMPdduZBkU5liRbzdjxQ3jxOM4I4v6JieuG5FFS0HDSIsMEsVcgSYn/nCsAffMHIs1b9VhbvVhzH/5I9Q3Ke/Rq5VmKo2tVpZUkGkWnQLpeL5AEPl2M3bXNuKh3SfQ6vLBYTXh17d/J+b6Zen1/OCZg6jadwqP3lqCiUUOMcpRffAMgNBE43ledv0nGjpEbz9cpOXm0gJsXTYVI7Is2HnfNM3saDVybRyuys3AurnyVcGzd03A3toGzX1RaQOv8O+W6uFTKP1HtPyrVpcPG96v121f1c6hZHOVSuyFxYkUweYW2ENNuvZ88g02LpiE5i4vKrfV4qHXTiAv04w2lw/PvFsPd0835QK7GZuXTsGjt5Zgb20D2t1+ZFqM8Pp5vHm8ES0ub6Jf35AjoRyJVCHaHp9arsDmJVOwdMtR8ecTixxYNasY4wpssJqMsoxfNbGUVpdPHHtikQMrbhiHEVkWODJMeOrtOrxX1yQKR5UUZMJoZBXHWze3DM+8Wy+LHMTTQ0PtXn+/fDpOX+6KiE7srpyBeTWHxeN3LZ+OuzcdET8r3FPZ6CxZ+DAR6W+lqg2TgYEvSDRbjgufjVG4ZsCXDjRHIj0ZyjkSanZGsFnn27px6kIn1rxVp3iMNAqqlq+lNs8BROSgvXxPOcxGA1q7QtHcvbUN+Jd/vEZmc1tcXrAAPAEeQZ6AZRgE+CD+z66TaO7yyuztpQ43frXvFO6ZOVYmuvfC/EkoyOIwMjttciV0PZ9p0UZcbavBwDIyJ0JNqVFNLEUo8VRTeqxZVI41s8dHlDUqjffInpNYM3s8lm45qivLWS2sqHavrV0+FGSZ0dwV8qaFcxBCZE5Qrp2TbYccb2jHmrfqsLtyhmLDsN2VMzAiyxKTM8GyDIbZzICt9170Oge0gReFMvBE030Q5qXSMUIUVGu+a9nc/Ewz8mycuD3BE4KO7gAeeq1WZnsFJwII2Q0jy6D+klOWqL1ubhn+7Z+ug91ilNlbBsAjt1yLDrcfVRWlqD54Bscb2vHzncewa/l08DyhNkdCWmxtqKtN9mqtayk1ak0aLaXHym21YFk2InFIbbxxBXZ8uPpGvLHyel1yskphRbV7vdTpQZ6Nwxsrr5edgzMacHNpAR6+pQRr3qrDP/7XBxHbIVKHI/yaL7S7Ncta9ejnx6pq11fKmRQKRR/RFHxzbRwKMs2KxwQJos73aI4Ky7JY81Yd7t50BJ2eAB567USE7W1z+2Wfl1Z7Ccc9suckRjosYqSj2enF5Q43Wrp8WLrlKOZWH8aat+rw8C0he9jY5sbFDk/KKuoOFGnhSKjVKA+3m1GzqByFOVbVNrJSZ0FgYpEDm5dMQZAQEBBsXTY1JqVH4eUtaCHULCrHzaUFsJoMul6OWi/eXBsn3pNwr0L1h+DUSM+Ra+PwxG2lEU7QI3tO4vn5E2UOh5JREK5F6aWvV8cimtGIpwEQhULpO6LpPrAsg1HZ1ghbpLUokdpKLUeF50N2d/u907B5yRSMyrYojuf2B2U2I6hyXp4nMlv1aWOHYgWIIFalZfPSlbTY2tAKh183Ikv8uZqymjBp7t/6CfLtZjx6a0mEjsGYYRmqypBNTg88/iAMDAMrZ4DDYsSqWddghSTfIJq0qxStFy/LMigpyMTO+6aJYlZa1R8sy8i2eKTjARD3DAUHJVwbY/3+elWHKVp4UkBL1a6v2rJTKJT40bPFaDSyMvsqHCMIyynNdwGpzZXO+xyrKcIe1CwMLcTeq2uSjXemqQtLtxzFzaUFeOK2Uhh6qjHCz3um2QWnN4Df/ukLWRm6FKECJJrNS1fSwpHQKlMSwuQ8TxQfXOFY6aS5e9ORiJfja5UzIj+/aDI63X4s2XxUtic3OseKLm8Az941QRQ8WbG9VndyZTQ5WaORRWFOBqycESOzLZg0pkwzj0CPPC3LMsizhfInOtx+tLp8WL8/lByqVjURa/8Ope9erzNCoVD6F6lkv5qNVZL115rv0rGVHBXFVgLba7Hjvmmou+iMSF6fV16IBdOvwvyXP0K+3Sx2+8y3m7FqVjGuys3AhXY3fvunLzCnvAjv1TWJZejh9jDbasKjPZ1CaaWYnCHvSCitaMMTcYDoHrYwIc63KesYnG93Y5iNE9tum4wsgkEed9XInY7NH57FAz8sxsM9e3rSlb1eD1fvRNT7otUzXmhMFuv2fy6WdEm/T6VoRyz6+WYjizWzxyODC+lMmHv+b/qqLTuFQkkOem2sgN6EaSUbpmYP2rtDSZG5Ng75mWa88OcvAYRy3xa/8jEa20JCfM+8W48N874HK2eQRYTXzilDnj1kw4Syd2ni/MYFk7Bu/+eiE6GnHUA6MeQdCT3NsAT0vHzVXo6tLh/+ddeneGPl9RidkxHa02/3RDz0c8qLsHLHsYj9t/V3TQDDMDjf1h21EiHZlQt6xxPUJTe8Xy9O2oJMM0ZlWxXPrddBEQS5wr/TN1ZeH1cDIAqF0n/EYmMF4mlACKjb30udHlRuqxX/vWb2eMwK8rjiknfzPN7QjivdPqzZXRdhg3f39O7J4AxgGQa/Xz4dQZ7AHyQghMejt16Hx28rhZUzIM9Gk7ylDHlHQqsZlpUzRjzM0WSzlV6OQkQh326GLxBK8AEAjz8y70ItKXN4lkXUc9CTBxDvRFRDz3jSVuZ6HBi9DopW1GFktlWXM0KhUAaGWG1sNLRssJb9lZ57bJ4NLl8AXI+qpR4b7A0SsQ+HYGeK8+1oc/tpqXkUhrwjIfVgRa0EG4dsqwk8L9dN15PYJ7wcd1fOwIV2Nzz+IFgG+NXtpci2mvDv//t3UYTqhfkT8exdE8TSpMIcK/J7SqLCPepzLa5BkQcQqwOTSJSHMxqobgSFkuKozV9/kBcXVrE07tKyweH2gGEYPPnmKVFkTxAVZFnAbjZi55FzEdsUefbBbYNTkSHvSAjVBr/90xcRKmU1i8qRn9krpqQ3sY9lGYzIssDjD+Jyp0dWwbF2Thmanb4e8ZLj2DDve7K9uyyLARvmTcCDu3udixcXTMKv9v1ddt3x5gHE03VvoIm2BZLs6AuFQkkeShVdz8+fCH+AFxPT9VZb6bHB4UmeD95UgrqLTsWKurVzyrDv+HnRBo/ItiCDY1G9sDyiaq7qD6dk16Jlgwejne1LhrwjIZRD/ur27+AnYdUWldvklRKxJPaxLAO7xYjFr8j1F1bvPYmqilJxUg3PtmCUwyI+bACQaw/IEgvz7JyoOCkQTx5AX5ZK9uXEoVEHCmXwolRy7vHzYkI5oH+FH2tydbSKOqk9BoAPV9+IYTYLMjkTfr98Onw9ctluf1C3DaYl6ZGkhSCV0ciCAaI+oNHU2sLxB3jFMR09ehCFOdYIkSmWZXB1rg3jR2ejMMeK8aOzkW+3aIq76CVWhUi96BWWSgSqVkmhDF6EkvOrcm0YPyoLoxzKIlHRoqyx2mBAHqGIZo+FcUwmA0ZlW5FpMSGDM4g2V48NbnF5Fe1sOjfzSrmIBMMw6wDcDsAH4AyApYSQ9kTH1ZP9r7fKINqYQh2y2meVQvXJWJH3VamkmoOyu3JG1CZbFAolPZDatWanN65qq1htsJRY7XG4HXZYOV022ONXtrMeXxDNTm9a2sJUjEi8D2A8IaQMwBcAfpmMQXOsJkW5VgMLUUYV6H2hC/0oivPtaHX5FOWZlWRiaxaV43uF2VH7ZYQTy4o8lva6hTlWBHskYONFzUG50O7uswiFAJXHplAGH2q2kef5iHksneOtLh+K8+0RPYHUOgFLbUOO1RSTPQ7/PABdNtjAMKp2ti9tYSqTchEJQsh7kn8eATA30TF5nuB0cxd++6cvxKSb4VkW+IM8Htt7EnPKi5Br4+D2BTAq2ypL5Iklg7g/VuZa16QmY/2bt+vw1J1lMSUsSnMiGEZZWlbYMumrDOdYhW4oFEpqEG4bgzzBb96uEyvapC3BpXNcKmetZU/V7KDghESzx4EAj/omp8xW6s1zsJkN2LhgkqgHJAhWdXr8smhtrF2RBzMMIanrOTEM878AdhFCtmsdN3nyZPLJJ5+o/r7Z6cWdGz+UvQg3L5mC3338tWIlx3UjssCyjOLnBKGkgaoiiHZNlzvc+LSxAw6rCe1uPw7UXcYd3xuFomEZMLI9vT6s2s5O+CS9ubQAD/ywWDZxhNptoewKCCUyjcy2Ji0pU+1etYRuFBjwmRzt+QSAqx97u5+uhtJfnHv6Nj2HDYrnMxG0bBYA8XcTixx4+JYSmT1We7knYpt5nqCxrRvzX/4ors83OT144o2/YU55kWhn99Y2YPGMq/Hmpxdw/w++BZORhZFlMNxuhsk0qMXzdD2fAxKRYBjmTwBGKPzqcULIvp5jHgcQALBDZYzlAJYDwJgxYzTPpxSaz+AMmFNepNj6W3iYUlGeWW97XWFiPvaja2U6FuvmlmF4lgVX59rEyRlekWFg5W1+hWY4m5dMwRWXD8NsnCgXK1CYY4WVMyQ1mznZQjf9SSzPJ4XS3/Tn8xnNZgm/W3HDuAh7rBbpTMQ2t7p8aHJ64/68P8DjvbomWZMwAHjk1muxcMZVWLf/czHKHeQJRmVZBrszEZUBiQ8TQv6REDJe4Y/gRNwDoALAAqISMiGEbCKETCaETM7Pz9c8n1LuQLcvGLX1dzwZxH1NtGuS7k2uuGGc6EQAve3Bm5xeXOr04HxbN664vBEVGRfbPci3yyfue3VNuOLy4e5NR/DonpNYev3YiHyTAE+SWjWi1bo81XttxPJ8Uij9TX8+n1o2S/o7ta6bSnM9EdvsCwTFDqTxfJ5RyZGwmgx4/s+nxV5Ec6sPY8HLH+GLpq4hnzORchvNDMPcCmA1gDsIId3JGFMp8eeq3AwU9KhMSlF7KQu/G2h55mjXJN2bvHZEpuLEzM80o63bh+vX/gUnGjoUu+mt/tG1qFlUjl3Lp6NmUahNb7cvNKGbu7wYnmXB6ytnyhKi1Mph433pCzkf0ntdO6cMe2sbaK8NCmWQoGWzpL8TqiukqL3cE7HNnNGAvbUNWDunLCIxU8/nDQwiPrt2ThmCPFGOcm+vFZM5hyopl2wJ4HkAZgDvMwwDAEcIISsSGVAtKZLnSURyotpLOdqeP88TtLi88PiDMDD6chGSeS9KzceanVBMkvymtVucBBmcQfHlPzzLIutQ+uKCSeAJwZ4VMzDKYVVMJEp2gy0loZtXD53FgzeV0F4bFMogIZrNEn7H87yiPRYq68Lbk8eb6C40H/zDsQZsXjKlJ7GTRaZFn51iWRavHjqLqopSMUfi1UNn8evbv6Ma5fYEgvD7g0N2iyPlHAlCyLf7Ylw1mWWryaDYvjra56QoZRAr5SIk+16E3IaLHW7VBmM1C8tRGdYud/3+evzq9lIAEFcB4S//cy0u5NvN4mTx+Hnk2jk0O71QS9BNpAZcDUHoxsoZMTLbgkljytKyTptCGcxo2VHp7/IzLaJzYDKy6PIEcMfzH6pWzcWTJ8WyDIrz7bhjYiGWbjkas80WHJFwO5dv4xDgiWhPpb2djCyDK90+5NrMQ7LiLKWrNvQSb9Zxsqoy1MZZM3t8SLmyD5IC9cq0XnF5caKhAxmcAe1uP6oPnkFzl1eUjZ1Y5IjQp39xwSRsO/w1Zk8cjdV7Typq2KslUWpJaQ+QPv2Aexy0aiM9oVUbidGXVXOJ2mzBlvE8jyCBKMzHMgTn2z1iroS0AmXd3FAJ/tXDbIPJmUjdqo1UIVlVGWrjZHAGzbH0vFjVjtHbYMxhDTWqkTocGxdMwvN/Pg1Anu/gD4S6oX5zpRs/+u5IcRJUVZSKToTWuQD1lUe8+vS0OQ6Fkp70ZdVcNJutZ0HE8zxaXD7ZVsyrS6cggzPg0Vuvw5LNH0ckuq+ZPb6ntbnu8vVBQVo7Esna01cbp9sXVB1Lz4tV6xi1iSAox0knQMnwTLy+cibcviC+anZh++GvMae8CPd+/1vo9gXhyDBhmK1X2nbzh2ex+kfXiePHkk0tvT/pRCRQrujQWl3Q5jgUSvqS7JwrPWN3+4KaZexAr4BWVUWpWGoPhGzaPZuPYuOCSciyKNvMDM6Atm4/rJwvpcvXY2XQxFf6gmRVZSiNs2HeBFw7wg5fIKgo7aynwZbWMUrlTzeXFqDF5YtorgUABZkWGFkGS7ccxe7aRlRuq8Xdm45g6ZajcPt6HQJh/0/QygcQUzY1oNzkq9sb++pC7f5bXF4qm02hDHH6smpOaex1c8vw7QIbPH5e0e5c6vTIGnapLbDcviDqLzsVbaawcON5fkjZrbSOSCRL4loY5/WVM+Hx8zAyQLc/iLtqjqiupPWE7bSOGZltjUhsfOK2UplaW/iqX4+HL9xLu9snJmpWHzyDdXPLInIk1Ca0kgNwtsUV8+pC7f67vUEs/J+PaJSCQhnC9GULgnCbbWBC0tcXO7xweZXFqi60u5FnN4u/U0tUF3LRnr1rQoQYoJUz4Km36/DTqVdhRLd/yNittI5IAMlrX82yDAoyLRgzLAMmowFLNh/VjDaoCaqYjKy42iYIRRnCj+GMBtkkE7QcDCyj6Zzo9fBZlsEwmxnXjczC7soZePy26wAAT//4u9i1fDrWzB6P4Vnq35WSA/DcgdOoWRjZNE1rdaH2HZ1tcSVN9IpCoaQuybLPamMLNnt0Tgb8QYi2RE0IT1gQAUD1wTMRehLr5pbhQN1lrLhhHDI4A7Yum4q/PnIDti2bCgD49zdD/UYyOMOQsltpHZHoK/REG9RKJbs8ASx+5WNZYiQAsdmNVDQlPLExWuveWD18lmVACMHc6sMRv/tw9Y2ATfn+wyMfE4scWDWrGMPspphajyt9RzULy/HEH05pfrcUCoUSK4LdFhwEacWFUDYPQIzUHm9ox6uHzmLnfdNgYBmYjCw8/iDunDRaFr3dMG8C/vOPvS0FhKjFULJbaeNI9Gf2v9oWAsMwMmGV8Je6gYVYMw2EXpArdxzD5iVTcO/3v4V2tx95GtcdTcchnu8gnoQn6XWol45ao55byfExsKFKEynJSsCiUCipQ39XbAm27nhDO9bvr8fmJVPQ4faj1eUTGxQW5lgx0iHXujCyDNy+IBgwsBgNERVuD+4+gTWzx2PplqMozLHi2bsm4Ol3Pg/JanOGiOT4wbjVkRaORH9n/yu90KsXluPJN09FtNGVRhTOt3UrRjK6vAHcvemIWEOthlbEId7vIB6RqfDruHvTkZiqNcLHkh7H8yTpolcUCiW1GIiKLamtO97QjnX7P8eqWdeIlRnCNQiKxUrXuP3eaYo2vGiYFbuWT0e3LwieEDR3ebF12VRc7vT26z32FWnhSOjVXEgGghedZTFid+UMGBggSCA6EVrnV1v9Z1tNuLm0QJc0tJqOQ7zfgZpzAkDTkxauQ805ijek15cJWBQKJTXoT5stoGRbcqymiH8LURKGYbDh/XpdSeVnml2o3FYLAPjgkRvwxsrrQQjB4lcO9es99hVp4Uj0dTtwLYGSlxZPxrAMU0TLWaXzC02qpJ9/fv5ENDm9ePy2Uli5+MP3iXwHSlEBvasFK2fA5iVTIpQ1E9mKiFcal0KhDA762maroWRbhHYE7W4f6i87ZS0HXlwwCUuvH4tn3g1tfX+2NGMAACAASURBVAhJ5UptCYCQU2EzG+AN8PAHeWxeMgUvffAVdtc29ts99gVp4Uj0pbCJ9KX69I+/i8de/1uEh/la5Qxd52dZBnk2TuxvwRMCr5+XNc+KN/SVzO9A72qB5wkud3pRte+UrARqeJaFbkVQKBRV+tJmx4pg4y91eERbBoTs3s92HMOa2ePx8C0lWL+/Hs1dXjGHwu0LIMATPP3OZ2J+xcv3lONChxc/kzgaQkL97trGQZvvlRbln30pbCJ9qY7Itih60X6e1zw/zxOx5DNIgL21Dbh70xF0egJiHbIwVrwlQ8n8DqKpagpCUVLxFuGYR/achN1ipFsRFApFlb602bEi2Hi1TskZnAGr957EqlnFYg5FfqYZVs6Ip9/5DHPKi/DGypnYvGQK7GaT6EQIn1+54xgemFWMPStmYMd905BjNfX7PSZKWkQk+nJfXfpSNTCMohdtYPQlQebbzVg1qxi//KfrsOz734LZyCYtvBdrS3StbGml1YKgqindllFLPBJ6elAoFIoSqZQLJdh4LQGqxjY3xhXYUejorUYTVII3vF+Pe2aOxS9+dxxbl01VtImBII+51Ycj5LgHS5+htIhIAH0nbCIVTbrU6cG6uZECJSYDi4sdbrj9AZiMrOyBELzdfLsZD99Sgqp9p3Dj+r/i4ddOYJiNi0maWkAa4ZBKSKt9B9Ljm5wenGt1RchsS+VclVYLT9xWKjoRgDzxKNbrp1AolL4Uo4oFwcYrCVCtnVOG6oNnUJhjhcXEymxquzuUdP/4baUwsAzy7WYEe9qMSynMsSLYY1+FqHOLy4tzrS6ca3HBF+DR4fbjfLsbgRRdhKVFRKIvybGaUL2wHCu21+KZd+vx5B2lWDN7PDI4A7p9QeRnmnGxw42f7zzeK6q0qBwjsy1wWDnR262qKBUFUIDQA/XU23URyZfRwnuxlk0pHb9ubhnye6RglfIflFYLWmqW0sQjWqpJoVAGE9Ky0PX767H+rgkYnmXBuRaXmBexbm4ZujwB5NlCDsG5Vhcud3pk+jnP3jUB7/7tIl5cUI6f7ZDnSLz0wVfi+Rrb3PD6eTg9fgR4HoskAoU1i8px3YislItMUEciQdrcfvzvp43YvGQKDD3/uQzDwJHBwcgyON/mxoO7P5U5CJXbarFm9niMyLYg1x6KOig1gHmvrglrZo+PKbwXa9mU0vGP7DmJqopSsVxJaTtFj6qmNPFIaUtnsITtKBRK+qK0cDIZgHMAHvvRteAJQZAnuOLywWIywGJi8XVrd0Ri5kOvnUBVRSksJkZ8XxgNLHYcPitWbQChCAXLAFdc/ogxKrfVpmR5aMo6EgzDPAxgHYB8QkjLQF8PoJw7wPM8flAyHEu3HJV5nsMzGXgCPPLsnLi6FxASdO7f+gnefOB6vLR4Mi51eBT331iWjemhibVsSu14hyThR892hJpwlSDeIoW2B6dQKIMJlmWQa+NE++8LhCKuAPDwLSVitZ4g459n59DY5sbEIgdW3DAODqsJ7W4/RmVb0NjWW/0xsciBh28pwdunLstsIctCNbkzFctDU9KRYBimCMBNAL4Z6GsRUHv5OaymiC2Jh16TS6Kum1sm1hkD8gQdty+IkuGZGJ5lTso2QKxlU2rHd/e0Ftd7HVKvned5BAlASMjxyrGa0Ob2y34er9gMjWRQKJT+Rm0LOMiTCPtfub0WW5ZOxc2lBbhn5lhZz47qheX4/9l78/ioqvv//3Xu7EtChiyAJgpSjKQYCsMSoL8Wa6VSqXyQoBUJgkqSUmt/fhXhU5vWFv1+WKR89KOYgAqyKQjywWpdqfTzKYtKoFAMRoqAQZZMQibMvt3z/WNyb+bO3Dsz2Zgs5/l45AHJ3Hvn3HPPPed93mt/kwZr7huNhVsOi/U6Nj4wDg5vENlpOuSYdah3+uD2h7pNCGwiuquz5WoATwC4KgXblZwTI1EyGQR4XjEkSPj/oh3h0CAAosZCcNARKnkK1TYjq3m2Z4cu5whZVWJVDClSCrMamddP0g4ACftIkNovuwO4u+qA6Kz55SUHXv6ff+FfNhfurjog+l5E91kiSVt4meM5gjIYDEZbSbQGKJmAB/YzyM5lKg5YMnV4jJBRvrkaF6/4oOIIKqYVYFtpEWZa8/DY9qOY/uI+UEpx0ubEU28fx6AMHdbcN1oyN1fOsaKfToXzdg/ONrpwvps4YHY7jQQh5E4A31JKjxLS9TvNYJBHbb0jxqExehFXMgEohXzaPQHJcTdkm/C3RZNxodmLZe99KeZap6CSQl4dtX1xHMGwbDO2PjQe9Q4fGl1+PPfxV3j0tnxZwSRumFVLdc+2mCLkXrjyzdVYP2+saP6RC6OaUpATU9QsmWv31JSyDAYj9QjRFRfs3hhtcOT8pjT/6zWc7PxPaesx0edkmrRI06nFGh6C+SPTFPara/YE8OD3b4BGpcKfPjwhJii0t/jj/ex7uSiPaGvlHCtuGpAGtTp1eoGUfDMh5GNCyHGZn+kAngTwuySuUUoIOUQIOWSz2drVDp6nON/siQlblEv6FBnmKSBUb4ve0a8sDocERR6nUXHQqzlc19+IF2aPwtsPT4IvyOOuNfvbtLtORnvS5Alg9suforjyAMo2VePDmvq4iawShVkpLeBy11N64dQqIv49OoxqSkEOHrn1RokWQ64vUpU2tz10xvhkMLoKNj5bN0hH65pFIQKQn9+U5n+jzPy/fGYhXL4gTtY7Zc+5JsOAa/qFhZUpBTl4/Cf5WPpODZ559wS+bnDh8TeP4p61B9Hk8uPDmnqUbarGPWsPomxTNSbfNAA2hw+rZo1EVYkV2WYdyjdXoz6qIvLVJiUaCUrpj+X+Tgi5GcAQAII2IhfAYULIOErpxahrrAWwFgDGjBmTlG472r6u4oB6hy+pxSmeM2GGQRtOiRoI4bw9LKUKpa4F08LAdL1kgbY5YrM+KqWZFtps0KqSqhbX2QtuW66nWEIdBFMKcvBhTb1Ypnfp9BEYmmOGmiOiySNeXwjXzjbrRAcmtz/UoRokXUV7xieDcbXoi+NTbv5fsPEQVs0amXB+S2b+Fwp5PfX2ccy05mFndR2WzyyU+EhUzbFCRcKbvWHZZjx15whx7quYViApQd7o8kvm0lF5GTDr1Fi047B4PaGORzCUWvNGtzJtUEr/CSBH+J0QcgbAmM6I2pDLIDk4ywiLSSsucAJKdTDiZVoTCru4fEGs/qgWFdMKkGnSIidNh2v6GWJ2+cksztEmhfXzxsaEA7Wlimh7nXQ0ann1ndz15AqPLZ9ZiP/7lxosmTocNRccONfkgc3pw8B+euRmGHChOTmfiUyTtqX0rjQ+Wyk6hMFgMAB582zVHCsm3pCJ/iYtdpRPQKPLj8q9p8S6GJHzWzLzv/A9kdksX9t/GhXTCjAwXY8MowbPvFuDD2vqxXmrv7E17D86BYCguRUEkUduHYZfbDksmf8X7zyGpdNHQK3iEAzyqHf6EAjx0Kg45Jh1V83c0a0Eia4kOoNkpJQoFE0RHnBViVVSByPZKAFhsD0zozDh8cks9tEmhWTDgSwGjcRHYmd1XVIlyOXgeQqnN4iVxYWSxbtyjrwDZ3ThMbsngGc/CEes/PaOAmwvmwBKKQxaFYI8xYVmDwghSQtzZr0ac189llCYYjAYDAE58+xze77Cwz8aJgndXz6zEK/tPy07XypVHY5eI4Zlm/HMjELwPI+n7hwBSikIkde6bo8o6BjtOyZEdGwrLUKIpwiEqOz8PzjLhCyjBl9ecsT4TuTnmKHRdL3GtlsLEpTSwZ11rXgZJBduOYz188ai9AdDJRqE9uQ7SLbEtZKqLHLwRmstlHK9ExJuq9DmkzanVPIusWJYtrldO/ZGlx9zX/0M2WadKBy4/SF4/CE0eQKy98pxnOhIFNlOg1Ytam6i+7VyjhVAWJibUpCD395RAH8wBJvDJxHGAkH5KJnu6CfBYDC6B3Ia4JnWPCyU2eFvL5sQY4pWItk14tsmt+y8RSkV14HKvafEDZugNR+SZYJGzcGkIrjskp//dSqCBndAFCKEa5dvrsaWh8Yjz2Lscm1tdw3/7HQEDYBcBslzTR7o1ByuzzQh12IU1UFtcTJsK5GqMqVwz2gHH2GgRTv2PPX2cdE5Ua7NZZuq0RQRRRKPaGdO4QU8UmcXnX7mb/gcPKUx1T4F58hElfuUIjueunMEDj15K568owD1Dh+On7+CJ3cdkzheKjk9dcfYagaD0T2QmzcyTVrFxT3ZhTfZNSLevCWsAy/MHoX8gWnY/fBEPP1vI1Cx+zge3fYPfPHtFTR7gmh0+rFq1kjJvLrmvtHQa1UIhuQ3WC5fEBeveOM653cG3Voj0ZkIi5tSBklhtxxJV0cJJNJeRGstbE4fBqTrsb1sAs7bPWh0+UWzQc0Fh2i/i9Qe2D0BVO49pdjmSLWcRs3B6Q1ibkRu960PjZftr+w0HRy+IOat/1xWEo9nT1QOpQUuOXwx/hWrP6rFMzMKkZ2mS0qTw2AwGJHIzRs5aboO+5Ilu0bEm7fkyg2Uba6WNcO/MHsUlt11MzQqDnZPAJmmsLOnRyF5lUmnFp0/M01auP1BXJOu73RzR58RJITFrS0ZJJPxY+jKTItKC/KFZg+KKw9IjhUGr0GrwhO350v8GVYWF0oiG4Q28zwfU/o7umDX0zKFw5bPLMSy905g/qQhisW94glJSv0aoogJxV28M1z3Q3gxu1N5YQaD0TOQmzcsBk2HNyXJOrbHm7ei15B4ZviHtx5pqYP0GXItBry1cCI4jkDNkZgIkRdnj8bWg2dks2vmWvRI13fevNlnBAkAYgbJyHCdeAuRECVwttEtVvO8PtMoccTs6poRcgtyvMEb5KkkhOhcUzgD21sLJ8a0uWJagcSXQTg2smCXUDgsUguy+8i3mGnNg0bFYUVxIZ7YcQxH6uxJa2uUpHNK5Z2JBqbrY5JVMcdKBoPRFuTm0o5uShJpSBNtNOXWEEELnJOmk50PMwwa8XuyTEJFZk6MEBE00WoVwejBmbLZNd8oLYLL54FGzSHL1PES7X1KkBBI1iESAHxBXgy5FB6eQKoyLcYbvEqhlIGWNKqRbVbyF4ku2MVxHILBEIorD4hFZiIlXCGW2eb0KaoF5Tybo1/g6Lhp4fszjBrR47krhDUGg9E3actaoHR+PE1Doo2m3Bry9Ls12DB/LDiFrMm5FgN2LZwkEUoyTVo8elu+5Ls2PjBO0Q/kYrMXviAvRqh0dD7tM86W7SGRI02qMi3Gc9RM5IwY2WYhCiT6WLmCXcJ1yycPjZFwF+8M1xJRUgvK1cg4aXMi06SVZNOUrRMyx4pn3q1RfAYMBoORSpQyAyfjiCm3hnxYUw+jRoVl752QZAAWIvAG9TPEZCAW1oS3Fk7E3xZNxusLirDts7PIMutk5/hGlx+Ldx7DTGtep8ynfVIjkSyJBIXOTvzUFpQk6USqtsg2V+49hZfuGy0mOcm1hAuK8ZTifxZNhkGrFqVe4bouX1C2T4bmmJGbEZt4C0hecyMn3fM8L8kvIZzPwj0ZDEZ3JpmNptIaEqQUH9bUw+bwS8wVWQnyGOWk6VHv8OK3u/6J+ycOwd4vL6JyjlWSX0LQIEdqpTs6nzJBIg6JBIXuGEGQyBkxss1H6uzwBngsnT4CRq0Kdk9ALCimtMhfvKIQ9aJRKQ7wtmhu5DyYUyWsMRgMRntJZqOptIboNeFzhbB74dxdCycl/N5AkBeFkPLJQ6FTE7xRWoSLzV5JpF+uxSBqpTs6nzJBIg6JBIXuGkEQz+4X3Wal+h1ywhDHEQxM17dZeOqI5qY7CmsMBoORiGTmLqU1BEC75z1hvo0UQqYU5ODXP74RS7fVSDQTr+0/3SnzKaG059drGTNmDD106FCXXLsrwzu7C229x/Yc35Holg4+g5Q/rGTG5+Al716l1jCuFmeW3ZHMYT1ifDLaR0fmrvaeqzTfDss2o8kTEIuLqUg42iPBdZNqLNNIJKCjXr09gbbeY3uO74jmpi88AwaD0fvoyNzV3nPjzbddNY8yQYJxVWDCAIPBYFwdrvZ8y8I/GQwGg8FgtBsmSDAYDAaDwWg3TJBgMBgMBoPRbpggwWAwGAwGo90wQYLBYDAYDEa76RV5JAghNgBnExyWBaDhKjSnO8DutZUGSuntV6sxcrDxKaGv3CeQ3L32lPHZWfSE58/a2EpS47NXCBLJQAg5RCkdk/jIng+7155Hb7mPRPSV+wT61r0mS0/oE9bGtsNMGwwGg8FgMNoNEyQYDAaDwWC0m74kSKxNdQOuIuxeex695T4S0VfuE+hb95osPaFPWBvbSJ/xkWAwGAwGg9H59CWNBIPBYDAYjE6GCRIMBoPBYDDaDRMkGAwGg8FgtBsmSDAYDAaDwWg3TJBgMBgMBoPRbpggwWAwGAwGo90wQYLBYDAYDEa7YYIEg8FgMBiMdsMECQaDwWAwGO2GCRIMBoPBYDDaDRMkGAwGg8FgtBsmSDAYDAaDwWg3TJBgMBgMBoPRbpggwWAwGAwGo90wQYLBYDAYDEa76RWCxO23304BsB/2I/eTctj4ZD9xflIOG5/sJ85PUqiTPbCzIYTkA9gW8acbAPwOQAaABQBsLX//DaX0L/Gu1dDQ0CVtZDA6AzY+Gd0ZNj4ZHSVlggSltBbA9wCAEKIC8C2AXQDmA1hNKX02VW1jMBgMBoORHN3FtHErgFOU0rOpbgiDwWAwGIzk6S6CxM8BvB7x+8OEkGOEkFcJIZZUNYrBYDAYDEZ8Ui5IEEK0AO4E8GbLn14CMBRhs8cFAKsUzislhBwihByy2WxyhzAYKYONT0Z3ho1PRmeSckECwFQAhymllwCAUnqJUhqilPIA1gEYJ3cSpXQtpXQMpXRMdnb2VWxu74PnKWwOH75tcsPm8IHnk3bWZSjQlvHJ+p9xtWHzJ6MzSZmzZQT3IsKsQQgZRCm90PLrDADHU9KqPgLPU9RecmDBxkM41+RBrsWAdXPHIH9AGjiOpLp5vR7W/wxG92HwknfbdPyZZXd0UUt6FikVJAghRgC3ASiL+PMKQsj3EI5hPRP1GaMT4HmKRpcf/mAIhBBxEQOAc00eLNh4CLsWTkJ2mi7h+Vq1CpkmLVv02kmjy4/VH9WiYloBMgwa2D0BrP6oFk/PuBk5afpUN4/BYDASklJBglLqBpAZ9beSFDWnTxC9A95RPkEUIgTONXngD4aSOp/toDsGz/O4f+IQLN55TOzP5TMLEQjy4HnK+pTBYHR7uoNpg9HJxNMYNLr8Eg1EIMRj/byxMGpVsHsCqNx7CjanD1q1Svba0ecno8FgKBOiEIUIAMg26+AP8gjyFJeueDEgXc+ECQaD0a1hgkQvI5HGwB8MiYvWqLwMcISgYvdx8djVd49ErsUAfzAEm8MXY7aIPF8gngaDER9KqeR5LJl6Ex5782jrsysZg/yBTNvDYDC6L90haoPRiShpDBpdfgCAVq1CrsUAACifPFRctIRjH91+FF9edGLS8k8wY80+1F5yiFEEPE9BCBHPF8i1GBQ1GIz4RD6Px6bcGPM8Fmw6hAaXL5VNZDAYjLgwjUQvI5HGINOkxbq5Y7Bg4yFkGDSyxxq1KvH/gtki06RF7SUHVn9Ui+UzCyU2/XVzxyDTpI3bLuagKY/FoEFViRVlm6oxsJ9e9nl4A3yKWsdgMOLBojzCMEGilyHscCMXpEiNAccR5A9Iw66Fk+APhmSPtXsC4u+CEBKp6bA5/KiYVoBMkxbXZBgwMIEdnzloysPzFCdtTjz38VfYuqAIBJB9Hqq+20UMBqMHwEwbvQxB4yCoy+U0BhxHkJ2mw6B+hphjVxYXonLvKfFYQQiJ1HQcqbOjbFM1iisPIMhTXGj2xE2klMjc0lcR+uXDmno0u/0AKFYWF8Y8D4OWmY0YDEb3hWkkeihKpgKOIxiWbcb2sgkIhnioVRxyzDrZnX+0dkKj5uD0BmFzhm3ykUJIo8svu1s+Ve/E/A2fx9UyMAdNeSL75XyzF+8c/RZ3WfOwdPoIGLUquP0hZKfpQAhloaAMBqPbwgSJHkg8UwEAnLQ5kzYjCNoJgSwTFQULQUABwtEFmx4chzMNbjy/5yRsTh9WFhdixfu1AOKHgSYyt/RVIvulcu8pPP6TfLxVXYd7xl2Pyy4/9BoODU4/CCFwenkMTNNBo+nbfcZgMLofzLTRA4lnKuioGUEQLK61GEWBoPaSA3e9tB+3PPs3VOw+jj9O/y52lE/AivdrcaTOLp6rpGVIxtzSF4nslyN1dry2/zTmTBiCua9+hmfePQGeAo+/eRS3rvob7l13ELX1TgSDzPGSwWB0L5hGogeSyFTQmWYEOcHkF1sOY8P8caIJREBJyxBtQmFRG2Gi+4UQApvDh3NNHlRMK5AkqjrX5EH55mpsL5uAazIMCa7MYDAYVw+mkeiBROYeEBAW8XiftQclocXhDaBqjjVpLUO0pqOvCxECkf0yMF0Pi1GDXItBMTQ3GGIaCQaD0b1ggkQPJJ6poLPNCEqCid0dwKAMPXYtnIR9i2/BroWT+nw4Z0fhOIJr+hlQVWKF2x+S7Xe1ir2yDAaje8FMGz2QRKaCzjQjRCawEpw3VxYXYkC6HhkGZp7obNRqDsMHpmNgug+Vc6wo31wt9nvlHCtyzKyeCYPB6F6kXJAghJwB4AAQAhCklI4hhPQHsA3AYIRLid9NKW1KVRu7I9HRFtGfCSGbQjKp9goTgtCy++GJ8Ph5hHgKrYpDtkJIKaPjcBxBplmPfnqtGMarb4nWOH/FAxUhMGhVTJBjMBjdgpQLEi3cQiltiPh9CYA9lNJlhJAlLb8vTk3TuhfJpJruikyS9Vf8LDPlVUat5nBNhgHBII/zzR7UO3xodPmxs7oO8ycNwYB0PQZnmtgzYDDi0NY01oy2010NrtMBvNby/9cA/FsK25JSeJ7C5vDh2yY3Lrt8qL3kwIw1+2SLagl0dibJrshMGXlf8bJi9kUi+6bR5UXtJQdmv/wpiisPYOk7Nbh/4hCs33caZxvdfT47KIPBSD3dQZCgAD4khFQTQkpb/jaAUnoBAFr+zUlZ61KIoFkQBIejdc1JLeidnUmyPdeLJyhE35eSQNQXie6bY3VXUNbiJwGE+33xzmOYac2DUavq89lBGQxG6ukOgsQkSuloAFMB/JIQ8oNkTiKElBJCDhFCDtlstq5tYYqI1gQYtaqkFvTODgFt6/USCQp9ofZGe8dnss8806SF2x/q89lBGe2jL8yfjKtHygUJSun5ln/rAewCMA7AJULIIABo+bde5ry1lNIxlNIx2dnZV7PJV41oTYDdE0hqQe/sENC2Xi+RoNAXam+0d3wm+8yz03S4PtPY57ODMtpHX5g/GVePlDpbEkJMADhKqaPl/1MA/BHA2wDuB7Cs5d/dqWtl6oiuUVG59xRWFhdi0Y5jEqfH6MWkrZkkEzlwtv16PCqmFSDDoIHdE0Dl3lM4UmcXBYV4tTeScSbtzRi0KqyfNxZGrQp2TwB7ai7FPPPKOVYYtSpk6DV9uq8YDEb3INVRGwMA7CKECG3ZSil9nxDyOYDthJAHAXwDYFYK25gyonM42Jw+DEjX462FExEI8nGjNpJdYJQiPAak6+DxS89XCjeNvl6Dy4+l79SI11s+sxCv7T8tak7kclOsmzsGFoNGti3Dss1o8gR69YLJ8xR2jx8X7F5U7D4uydnR36TBf97zPVhMWnzT6EbFfx9HdpoWj9x6oyTPRFWJFfk5aVCrU65oZDAYfQhCac93cBszZgw9dOhQqpvRJbR1h97W0E+bw4cZa/bFaAeWTh+RsDy4HErX2/LQeOjVHDiOEzUo0ffV6PLLnrv1ofGY/fKn7Q09TbnEkWh8Cs/sYnOrECGQazHgjdIinLzklHxWVWIVhbXIY19fUIRrMwy9TtDqxaT8QfXm+RPoXuGfZ5bdkeomtJWkxifbunRz2lqjoq2OjEr+CkatKqnzk72e3R1A+ebDOP5tM7657EaDy4dMk1ZyX0rn1rcUsmpPe3oCwjNTcqwMhiiG5pgknynV4vAH+V7VNwwGo/vDBIkU0hW5FNrqyKhYS8MTSOr8ZK/n8Abw+E/yUbH7OCY/uxd3rdkfE/KpdG70wtjbHDOFZ6bkWHm6wQUVIZLPlI7lCHpV3zAYjO4PEyRSRFflUmhrqKZcRMbK4kJU7j2V1PnJXG/5zEKoOBJTFjtas5Bp0sZUFH1x9mjsrK5L+n56IsIzq9x7CstnFsb03fN7TsLuCUg+21ldhzX3jY55bg1Of6/qGwaD0f1JtbNln0XJBLFr4aSknBqVUHJkjFfeOzIiQ6Pm4PQGYXP6ALQ9dDT6eoQQPPX2cTz4/RsSako4jmBQhh5Lp48Qoxa2HDyL+ZOGoOaCI6n76YlEPrNnP6jF0ukjcF2mERfsHjz7QS1szrBpp3LvKVRMK0CmSQuLSYtAKCT2ldsfglGrQppe06v6hsFgdH+YIJEiuiqXQltDNYVzIoWXLBNtV/XQaMfQQf3Cu+VHb8vHxWavYshn5Lk8zyMnXYeyTa3RCOWThyaMVOnJRD+zEE/x9Ls1+LCmXtRKPPtBLY7U2VG2qRoAsK20CK/8/Ws8decIEBLuP55S6DRMG8FgMK4uTJDoIO3NexAvl0JHUQrVTLatyYZ6Rl9bKVokf0AaBqTrUDXHKqZ7jtQsRJ87pSAHWx8aDxVHeqXgkAizXo2nZ9yM3/+MFzU6R+rs4ue5FgMowgJajlmHkzZnUuG7DAaD0RUwQaIDdKTKZltNEKlsq9L1IoUSChrXVNPfpEOGQSur6bA5fJJzP6ypR80FR4fNPD2FeM/G7vHHmHZWFhfihiwTssw6RRNZe8N3GQwGo60wQaIDdMTPoT0miFS1NRq5hW/zg+OT8oEQvitSEAlR2utTZscj3rPxB0NYw+Dq6wAAIABJREFU8X6tJFPoivdr8dy9o3Ch2aPYd9Hhu31FKGMwGFcfJkh0gI76OUSbEIRw0K4QLDrTJ0Nu4Tvd4EraVBMtiKyfN7bLzDw9gXjPRqtWweb0ib4RQLhvTtU7MX/D53ijtEi279obvstgMBhthYV/doDOrLKpFA4aDPKdkmuiM9sqt/A9v+dkTOimkPY6uv3Rgsjze05iZXFhzLl9Jfog3rNRCs99fs9JjMrLgF7DYWVxIaYU5OD1BePxyeM/xJaHxiPTpMWovAzJtRgMBqMrYBqJDtCZfg5yu/zVH9Xi1z++URLB0F57d2e2Vc5R1Ob0YVCGXmKqsRg0so6A/Y3SrIxH6uxY8X4ttpUWidfvSw6C8Z5NtAkMAB7eegRH6uyoKrHi4a1HMPGGTDw2JR+XXX6UvPKZeI1Vs0bilb9/jV//+MY+I5QxGIyrDxMkOkBn+jnI7fJnWvNEIQLomL27I22Ndqy0GDSyC1+GQXq9aCdKof3byybICiJatSrpwmC9qeql0rMBIDF1DeoXzvIp5PgQ0mTfWjAA5+3SOh3nmjx47M2jeKO0CANanDJ7S38xGIzuBRMkOkhbQiWVFkCepyAtKZAjF9dMk7ZTnRA7K6yzqsSKAWk6bC+bAEqp4uKkZPunlLZbO9LZ0SfdBTl/GaVKqELfCWmyMwwaAJDtawLgXw2uXtdf3ZneJugyGIlgPhJXiXg+ELWXHHjq7eMx6ZFz0nRt9mtIVL+jrfU95EwuZZuqUf2NHXdXHcAVb1Bxooxn+xd24PsW34JdCyclvbC1tShZT0XpPps8AQzLNmPrQ+ORazFgzX2j4faH4PaHZPuaEIIFGw8h26xDVYkVq2aNxMVmL2wOb6fVd2G00lWp7xmM7kzKBAlCSB4h5BNCyAlCyBeEkF+3/P0pQsi3hJB/tPz8NFVtBDqnsBbPU1y84pVdGOqdYfX/hzX1ePaDcJjfjvIJ2FZaBKNWhaqSWAdGpZ17okmsPZOcklZBUKvHW8TlHAUjbf9tqWqaqD09OSpBbozFu88mTwCzX/4Udzz/d7h8QZh1auRa9DEOq1UlVhBCkW3W4fGf5GPpOzW4Z+1BVOw+jnqHD0/uOsYWuU6mrwi6DEYkqTRtBAE8Rik9TAhJA1BNCPmo5bPVlNJnU9g2AO1To/M8hd3jh8cfzo+gV6vg9AVx2eWXXRgCIV78e2QK5B3lE1BceaBNWR4T5YpodPmx+iNpToLVH9XimRmFMWp1IV210JZAiEeIp9CoOLj9IfCUit+htIgLtv+3Fk6EN8BDRQCDtmPRA12ZETQVKI2xTLNW9j41Kg6+YAgb5o/FxWYvzDo1fvbCPozKy8ATt+dj0wPjEKIUOrUK55rc0KlVWDlrJOatDzthjsrLQPnkofAFeSz6yU1Y+cGXeHrGzchJ06ewF3ou0e+7mhBkm3WS59bTBV0GIxEp00hQSi9QSg+3/N8B4ASAa1PVHjnaurvgeYozjS7UXnTgnrUH8YMVe3HXS/vR5PYjO02HHeUTUFVilYTlaVRc3NLZH9bUY/bLn4qOiPF27ol26zzP4/6JQ8Sd6dJ3anD/xCGiwCDcQ+0lB57cdQz/srlwz9qDeObdEzBqVQiEWneuZp0ao/IyklrEG51+zF53EJOWf4I7X+iYqjeelqMnojTG1ByJuc8N88fibKMLP197ED/+0/9gyVv/BCEE7z7yfZRPHooV79fillV/w4r3v8Rllx+LdhzDj//0N9jdflGIePwn+dhZXYdGlx/NngCWTB0OylOmlWgHcu/73WsP4jc/vUl8x4GeLegyGMnQLZwtCSGDAYwC8CmASQAeJoTMBXAIYa1FUyra1VY1eqPLj7ON7hjv+V+/8Q9JyuLlMwvx2v7TYq2ESMfDKQU5WDJ1OJo9AVSVWFG59xSO1NnhCYTA81QiSEQ7dWnUXNzdeogippT34p3H8GbZBMk9LNh4CMvuuhmv7T+NimkFGD4oDXZ3QLwvIZfBb346HEadCjwfznUhpy3p7CqnVzsjaFejNMY8/lBMFdVvLrvx+JtHJX1ZvrkaFdMKsPSdGjHcc8nU4Zj76mfINuvC2iejBuvnjYVew2H9vtO4f+IQcRzkWgyommMN76Y5Dlnm5M1MfR27xw+eAhoVh4ppBeK7+uj2ozEpynuqoMtgJEPKBQlCiBnATgD/P6X0CiHkJQBLAdCWf1cBeEDmvFIApQBw3XXXdUnb2qpG9wfDpZwTpSxevPMYtpUWYVA/g2Rh5HkeDU4/5r7amgtAEDpO1Tvh8gVFs4qcSvz1BeOx9aHxqHf40OjyY2d1HR69LR8Wgwb1Di8CIV4y4Qnt4SkNfx4MayYm3pCJ6zON4oIjLFSRC9iiHcfwRmkR/vjnL8QqlZFmH0HIcfuDnerT0FM84pMdn/HGWGQkx7dNbhDIR2YI/iqPvXkU28smwObwiX4RkQLDS/eNxtwJg2OEybLN1Xh9QRHc/iB8wRCuzTB2yz7tLvA8xRWvHxfsXkkRusgqrUNzTNi3+JZuO0avxvzJ6DukNGqDEKJBWIjYQil9CwAopZcopSFKKQ9gHYBxcudSStdSSsdQSsdkZ2d3SfvaqkbXqlWK3vPRKYsBiJOLsGBwHCdOTMJx4YX8u8gwanCx2Qu7J2zyiN7pC3bZ2S9/iuLKA1j6Tg1+/eMbMTTTiPPNHnzT6MaXFx3YWV2HJVNbVa9TCnLQ6PLjrjX7MWn5J7hn7UHMmXA9gjwVFxxhoYrkXJMHvgCPD2vqxd8Fs0+kU+d5ezgF9rbSItGs09nZP7ujWj7Z8ZnsGEtmbJ1r8iDI87AYNXjk1mExAsMvthzGwH4G2Wd53u7BvPWfodkThNPPHAOVCAZ5fN3gxGVXQPZdLZ88FLkWA1SEYFA/Q5scia8mV2P+ZPQdUhm1QQC8AuAEpfRPEX8fFHHYDADHr3bbBCK1BcmEKWaatLg+0xjjPb+yuBCVe0+JxyktpEpq7ktXvJixZj8qdh/HBbtX1qu/fPJQLNohXTjePnIODW4/QpQiw6hBP4MGC2/5Dl75+9d4bMqNyLUYsGTqcCzcclhy3sIth8HzrQKPkK8gklyLARea5TUNgpCTbdaBIwQVu4+LPhlP3J6PjQ+M67Tsnz3dIz7ZMZZp0iKvvyFmbC2f2Tq2hAXsmn4GDM4yyo4lnVreJ8fuCYimkiZXEIEAcw6MhucpLrt90LVoiyqmFUh8Ic41eZBp0mL5zEL84c9fdFshl8HobFJp2pgEoATAPwkh/2j5228A3EsI+R7Cpo0zAMpS07wwbUnixHEEgzNNyDBqsK20CCGe4nyzF3oNJ2YjFMLy5BZSQc2dbdahfPJQZBg0cPtDCITCJgdBDb1r4aQYlXiGQdNqEzdoEAjxuNZiwNlGtyhg5FoMWH33SMyfNATXZBhQMa0AzS0LSCTnmjzgOIjXr9x7CstnFkrt6iVWPPfxV5LzBAHJHwwh26zDiuJCzN/weYxJ5K2FEzst+2dv8IhPZoxxHEGaTg2bw4dND4wDxxGEeIpl753AkTq7KLAatCqo1RxMOrWsyaTB6cWLs0fjl1sPx6jkgXB/2hw+qLiwQNIdd9Op4orXj3qHH+UR5oxVs0Zi2Xtf4kidHVMKcpCTrkej04eZ1jzZiCgGozeSMkGCUvp3AHKz1F+udls6ipzdvr8p7Cfg8ofEkMtMkxY5aTrFCTrTpMXGB8bh0hWvZPFfNWskRuVl4EidXVw4B/UzSJw0OULwxO35kvM2PjAuRkvx6Paj2PTgOPCUomxTtWLlTTVHsLK4EIt2HMOROjte238aWx4aD54CWhVBjlmHR2/LR80FhzRs0aSF3ePHE7fnKwopgi9GW+ltoZ9theM4LNpxDNlmHR6bciOutRjwxO3D8asfDUOD048B6XpkGMICapZJF5M9dPnMQvzxzycAABXTCnDTwDR8bXOJdn2gNWKov0mLBpePhYVG4PaFRCECaE1Dvuyum7HxwBn86kc3Yva6g5L+joyIYjB6Kyl3tuzuJHLuS5RrIn9AGp6ZUSh63qsI0OQJyDpgcRyBWa/G3FePyU5Wc175TOKIF+nVzxGCWVUHJOcp5a7gSPh7d5RPwDUZemycPw5nL7th1Ibt8Hn9DfivPf/CjNHXinkJLjZ74fQFccfzf5eka5aLngjyFIt2hJ00hYV/VF4GHptyIwb2M4ACuOzyxdTmSERnFh7r7siNOyVBs2qOFSPz+on9KZzb36jBm2UTEOB5qAjBH/78BQCI2i6nN4jsNJ1EWyY491ZM+y78AR7BIA+1uu8mwOV5igaXD95ACJTKO7vekG3C73/2Xdyz9mCMz8T2iIgoBqO3wgSJOCSTkCpReCPHEWSatEkntgoEednJalCGAVMKcvDobfniwhnt1R99XqPLL7uD5wjBFW8AOWk6ZJt0+JfHJQntrCqxomCQGfeu+1RyXsW0Atl7jEa4B8Ek8tr+03jw+zfgsZbQRUENPyBdj8GZpqSFid4W+qlEvHEnJ2iWba4OL1gG5XO/k2XCE7ffBJvDJxFC3iwvwpaHxsPWEunz2v7TePhHw/Bfe05i/9eNqCqxYvjA9F7Xx4kIBnlcdvtR7/CJWohPHv9hzPs0pSAHTa4APAHlujIMRm+n7241kiAZ575k7PZtcRJUqk/xTaMbT905AvkD0sDzFOftHpxtdOG83YNgkJc9b2d1HdbcNzombbJBwyEnTY9cixF2b1C2lsaPvzsIT00LR3esnzcWmx4ch+/kmPHnhyehqsSKiTdkgud5fNvkxjeXXah3hJ1AhQJkO8onoHzyUOw+8i0W/eQmUYgQvmPRjmM42+hus6Nke1Nr9yTijRclQfO83YPaSw40uOQrrtq9QaQbNDGmrlmVB2HWqTCwnx7DB6bhdz/7LixGDe4em4cXZ4+CUatCXZNbfL59gUAghG+bPfAEwqaMbLMOmx8cB0IIXl9QhDfLJojRR0umDkfZ5mpRaI+kL5ndGH0bppGIQzJCglzVzlxLWH0vJGmKvI6QojjDoEEgGMJllw8ev1R9XTXHKolPf+m+0XD6glCR8I7zy0sOicNX5Rwr8nPMMWr/+ZOGIE2vxpaHxgMA6i67kWXSIivC7q10jxebvfj+jTkYe0OWWMpcUH3vrK7Db+4YjksOnxjxkWsxYF3JGOg0XEweDLdf/juMWlWPd5TsCuKNOyU/kUaXH2v/5xR+97PvYtWskbB7AmK+kMgxK3ddb4CHNxCSaCumFOTgVz8ahpJXWp9lX6gayvMUX9U78dyer7B46nBkm3VYMvWmGG3aiuKbYXcHwRGi7JA8xwoVh5hEcgxGb4NpJOIQr3qlgIogpmrnyuJC/GrrETHPgUEbvo6QolhIUX332oOovejAwy3Hnrh4BTxPMShDj2dnjcSex36IpdNH4He7v8CiHcfQ4PLjksMb4/BVvrkaNpcfA9J1WDp9BLaVFqFiWgFWvF+Lklc+w9c2F7686MCSt/4JjpM+cqV7bHT5UXfZIwoRwnct3nkMcycMBs8jJmx0waZDONvojjk+q6VuRPR3uFsEKIaUeONOLu/E8pmF2FNzCfdPHIKfrz0ohto+/pN8ceds0KoQ4qnsdQkhqLvskWgrZlrz8Ivo59vDQ22TodHlR9nmasy05uGbRjceuXWYrDbtXJM3nNvlihe5FgOO1Nnx7Ae1WHbXzfhry3v72/8+3uGU8AxGT4AJEnFIJlkQx3FiKumP/88P8OyskQCAJVNvQsW0Aqz+qBZBnmLd3DFYPPWmmCRBi3aEk9gIJoVvmz3gCHBNPz3uf/UzzN/wubirLNtUjUCIyu4qgyEeHn8I8zd8jnvWHkTZpmrxvOsyjdhZXSe2PbLapIoDquZYYxamyr2nFLN0DuynV3TkNEYV5TrX5JGtG7GyuBDXZxp7paNkR0lUNTV/QBq2l03AjvIJqJhWgGc/qMWtBQOweOcxSblwf5DHb346HOvmjkGQp3j63ZoYobeqxAoVQcyzVkpC1hs1SJHvgycQzsSaYdDg+T0ncV2mfD4Oo1aFyy4/lr/3pZjb40idHYEQxdyo97YvCGCMvg0zbcQh2rlPo+ag5gguNHskpohHb8vHgo2H8NJ9oxGKyAgpLMqUpxiWbUadPdYhUpi0hP/bHD58bXNhSLZJ9lgVJ29KUas4sQBY9GcGNYdnZhSKi3a0M96b5UV4o7QIF5u9aHT5xXBAIZNi9PVUhCg6crr90oUm12IA4QgGpOuwvbQIQZ5CxREYtKo2R230FZJxKlWrCLLMOji8QQBh4UMuLXblHCuGZZtxyeHFhzX1sDn8kuqvWSYtOI6LedZCErLeHmob7ZwqhEPbPQHYnD5csHsUx7k/xONInR0r3q/F6wuKEAjxUHGkzwhgDIYA00gkQHDuG9TPgEanH3e+IE3PDADDss3YXjYB/U1a2aJYIRoO+TzT4I6b4lgwKRi1Kpy2uWSPvezyxThQvjTHihyzTnEnOzAiVa+cI9+syoNIN6hg0Kiw9J0aMcHR9ZlGWTV6gzNcx0Nud3t9plH6/SVj4PQGcecL+zBx+SeY/fKnuOINMiEiAUpOpcLCd9ea/Zj87F5U7D6Op/9tBK7NMMimxS7fXI0mT0A0lwil6gXzB8dxshlZd1bX4aWocRapjYvcxdscvh6ruo9+H57fcxIriwvF8b3xwBmsmjUyRptmMWnEjKI2pw81F67gZL0TXyu8t71NAGMwIiG9ITxpzJgx9NChQ136HTaHDzPW7IvZmby1cCIaneHJ6L/uHYUZa/bHnLtv8S0AgIe3HpEtpPS73V/A5vSJoZLzJw1BiKe41mLAmQY3nt9zEjanL+zk9X4thuWYseAHN0DFEWjVHNL0KvQzhMMwE+W9+LbJjUnLP5G0b1ReBp7+txF4bs9XmGnNi0mcFa6fwSPIU4R4ihClcPtDeO7j1uOz03S4Jl2POrsHZxtbc1LcNNCMWVUHY/qtvdU/20HKpZXOHJ9K43D3wxNxxRPELc/+LeacfYtvwaB+hrghyDxPYff44fGHEOIp1CoOOjWBN8AjRAG9hkOWSadYMK4nOmLyPMWFZg/ONXkkzqmj8jLwnz//HjgC+IIUFBQGTdjHhCMEHAH+EFGsTngvl0y9Ccve+zLmHa+aY8XwQYohtCnvsKsxf6aSwUveTXUTRM4suyPVTWgrSY1PZtpIEiVPem+AFyfUeodPVg1KCIFaRWBz+vDsB7WiapkCyEnTYdXdIxHiKXYc+gYLb/kOfAEei96STkQDM3Rodgdhc/pwpM6O/V83SsqRp+nCAoOwkxUEikgzDMcRWa//R24dJkaJCEW4ci0GbC8tAlqSaDW4/JLojaoSK/44fQTc/hA0Kg6D0vVo8gTEiA2BHeUTmKq3E1Eahy4fj0tX5MefRs2JCaq2l00ApTRGyOQ4gv4mHWBK3IbIXbwQheTyBXGh2YNwrrNw5tPumMhKeC94no8Z00KqcJszbF58fs9J/Oanw2HQqjB//ediNMu/Tx2OxVOHo2Lad9Hg9IKQ8LstmEMi33G3P4RBGfoeJWAxGG2l+73p3RQlT3oVaQ2pE0LAok0BT719HE5vEOvmjoHN6UPZpmq88vevYdapMbPyAH606m+Yv+Fz3DHyWlBKY7zEyzZXg+fDdTyinew+rKmPceYKBnmca3LjbKMLx89fwZO7jqH2kgPBIC/rXKlU4MkT5HF31QH841xzTPRG2aZquP0hLHvvBJzeIDiOyC5yLL6+c4k3Dpe/96WsucnpDWLGmn0Y/x9/xd1VB3DFG+xQIi9fy3OOjEIqrjyAe9YehM3hx2v7vsaJS45uV/grsnqs3JhevPMYHrl1GFbNGgm9hsOTdwzHwH56MUpqVF4G7p84BCWvfoZbV/0N9647CLefx3vHzmPLQ+Pxvdx+qCqxiu/4Y28excB+rWnLGYzeCtNIJIlSemYhtPNck0cMAdswfxzsbr/EcbHmggNvPzxJdKAjhODuqJTWC7ccxusLihR38BxHQClFceUBST4Kuycg5vTneYraeofE7PCbnxaAguLbZg+eebcGNocfS6ePwJAsE7RqDt9cdsvuZL9pCeVU8uC/7PJjpjUPCzYdki0kBoRt7VUlVsnOry1prROZavoa8cah3G4426zDXS/tj9EeXLzixcD09u2UCcLj44nb8+EN8JK8Fb/cehibHhwHFUdw/ooXGhWXUu1E5PghhGD1R7Vxx3RefwMWvXlMrD3y18d+KB5XPnmorA/UhvnjkKZXo79Jh+w0fa/PvMpgRMMEiSRR8qQHIJnYbU4f1BxBceUByfnnmjzw+EO41mIEIJ/SOiww8HG95bVqFaYU5OD+iUNiqnFmp+nR6PLjuY+/ivn8xdmj8eInJ3H/xCF49oNazN/wOXItBmxdMB5vVZ/DxgfGweENhpNEhXiYdWq8+Nd/AVD24G90+cUJWa6QWK7FgEdvy1esyZEIOVt8VYlVjDToi5N0MuOwbFO1KGAEQrxEexA5Jtrr10AI8MLsUVARgkU7YquINjr9cHiD4hjbMH8s0vXhirRdubhGC50WgwYnbc6YwmU2h19xTJ+yuSQFzITcG/GEDzVHmNbhKtKdfB4YYbqtIEEIuR3AcwBUAF6mlC5LcZMUyz1HT+wUNK4wAChXsjRqVXELU2WatPjtHQWY/fKnMaaGtxZORCDIY6Y1L2bn9Muth1ExrQCv7T+NFcWFuOwKT6ZGjQozRl8bk43yuY+/wvxJQ3Cy3onKvadkS0+/tv80ZlrzxHuLF7bYHsdKuQiTsk3VqJhWgKXv1PRIB7/OINlxmGnSiqYlud10vHop8Qk7YT4eZYJbvPMYlk4fIQqYAJBt1sHm8GFei49BVzhmCoW13L4QTje4ROfkqjlWPLfnq5g2VkwrkM1EWTnHiuf3fAWg1SxkMWpEjZqS8GHUqXqVEyqD0Va6pSBBCFEBeBHAbQDOAficEPI2pbQmtS2TJ3pi51sSUMWrUimnoq4qsYIjwIB0nSgUyDnFkQi/DAHB8dOgCR8v9/k1/fS4f+IQzN/Q6jh208A0GLVqcXI9UmcXJ9v1+05j5ayRcHgDyDBqsL2sCOftXrG40/0Th+C1/afjmio6YppQciwUdoYLNh7C9rIJ7VbR9zbkBAxhnLl8wQ45vUY+R42Kw8B0vez1rss0Yvl7JzDTmgcgbA6Iru+xYOMhvP3wJIR4SKriatQcgjwFB8AXDEcJaVQcjFoOLl+rlqHJExDP02sILth9kpTygmakbHNY6BQciIXvH5ZjRvnkofif2kvY8tB4qFuckC0GDZ6ZUYjf/0w6VvubdNi1cBJ4npc102WZwn2eqIAfg9Fb6ZaCBIBxAP5FKf0aAAghbwCYDqBbChLRJJNQKPqYUEvmQSGkLN5OhlOo76Ei4YXD4w/Kfq7XqMS0x4Lj2H0tmo3ICfhInV0UOuatb9VUrJo1EhwhyG8p7qQiwFN3joCKAHaPHxQUF6MmdWGXl8x9RaOktRHybpxrCheravYE2K4vDjo1B41K2+4EU3I77S0PjZe9ns3hw/xJQ7Di/VoA8hkys806nLd7JfViBAfHI2cvY8yQLMlnL903GpsOnIXd48cjt94ofjalIAdP3lEgjjdAqnUo21QdI+DmWgw4We/E0ndq8OLs0TBpVZLaM3ILfqSAFs8HIpnaPAxGbyShBxQh5GFCiOVqNCaCawHURfx+ruVvXUJXJNdJpkqlcIxWrcLslz+FzeEX0xtfbPbC7pFPq2vQqiTJg4RYdpNOJSa0kkt7HVk8S8lxrHzyUFHoiP78sTeP4rLbjzktwke9w4+7qw6gfPNh1F504FjdlZhJvbylboHwe1vSBSvVlRASAQl+Gn0hBXF7x2ijy4+5r36GRW8ejYnoSNbpVW6n/cy74YU48nqVc6y4IcuETLMWNqcPAMSMmZE8cuuwmHoxj715FJ4Aj1uGD4z57BdbDqN88lDMnzQENocPq2aNRFWJFXMnDIbN4VPUWuVawonY5MaPYO7zh2ib+jb6vQYgnisU8IuERSgx+gLJaCQGImxaOAzgVQAf0K7PYiW3tZR8JyGkFEApAFx33XXt/qJgkEdtvSNGXXk1d7j+YEg2vXHVHKtsBsgMgxYD0vVYOn2EmPgp12LAhebWEtLvPvJ9VEwrQE6aDmadGt5ACHpN6w5fyXEs06SNW7FTOG9QPz2eebcGFdMKMDTbhLrLHmSZ5U0qgr1c+D3ZHVq01ibIUzzzbmvmzTX3jcbmA2e75a6vs8YnIK8RSHaMCrvkc00eSURHrsWAQS0JxxIht9P+sKYev/rRMEm67WyzFllmHXzBEDbMHweOhIXe537+Pfz6jX+IbVeqX5Fl1ioKBiqOwKxTS5w7X3tgXNwU1i/OHo2tB8+gYloBhuWYcbLeKWrcxOuS2JTxyfZt9HOZUpCDyjlWiTalLRFKV5POHJ8MRkJBglL6W0JIBYApAOYDeIEQsh3AK5TSU13UrnMA8iJ+zwVwPqpdawGsBcKZ2dpy8Uh7LwXw3MdSh6z22DV5nqLB6YMnEApnnFRx6G/UJhX2plWrZNMbl22ulrQjMvugWkVw4wAzCAFoS+bBe9dJHTCXvlMjmWCnFOSITpNKjmMD0vVQq8KOY0pmhVyLAWqOxESGVM6x4s2yCfi/fzkh8XwXTBHC723ZoQk7QJvDhz/sOoaZ1jw8+P0bYPcE8MJfT2KmNQ/7v27sdru+joxPAWGcegJBXGz2ItusE4WCBRsP4a2FE5EToZaXI9I8JKTHnlKQg6fuHIFzdjdUJHHdEyUT0/lmL8o2VYu/71o4CRxHcG2GUXy/AiEeT79zQhQ4AiEeKiXTHKdcw0XNETQ4/ZJQ08tOP9L0aqwsLhT9MIRxmKZXw+kLoup/zwD/ewZVJdaY9yHXYkCIot1+DQ0un+RcwRdDKelXd6IzxieDIZBUcHeLBuIb7KWtAAAgAElEQVRiy08QgAXADkLIii5q1+cAhhFChhBCtAB+DuDtzrhwZFKaScs/wc/XHsT9E4dgVF6GeExbd7g8T1F70YG7XtqPH67ci5+vPYhTNifOXHYhGORlj49UpVoMGgzJki/SJbSD5ynONLpQe9GBe9YexA9W7MU9aw/idIMbf/jzF7jQ7EO2uXXik0uOdf/EIdhy8CyWTh+B4YPSYmp2rL57JAIhHgPS9BiQpkeljHlkZ3UdVhYXwhfkZes6OH1BPHF7a/nql+ZYsbO6TrxGW3doQl+5/UHMtOahcu8psbrphzX1ovmjO+76OkLkOP3BinBNDaEsONDqXJuIaPPQlIIcPHLrjbi76oA4hmovOnCmMTxW5VT8ciamyjjPNVL9b9CoxQRNy977Ema9Gs0ef4xprnKOFWqO4PCZRtlaMiFKUbH7uKREOiHAL7cewYr3w5qWbaVFWDp9BDLNWujVHCzG1vL1cu/DurljQKl8Nd1E7z/PU7h98poaSmlckyaD0dtIWGuDEPIIgPsBNAB4GcB/U0oDhBAOwElK6dAuaRghPwXwnwiHf75KKX1G6di25IpXqlUgOGcJv7dFI1Hv8OKuNftjrrl0+gjkD0zDNRmtdlNBIKh3+JBl1kLFEejVHDiOw8yX9iPbrBMTTbn9IRTmpSPTpEf9FS++OH8FFbuPy7Z96Ts1WDp9BOZv+Fz8bEpBDpZMHQ4K4JtGtyQsrr9ZAxXhEAjxCLVU5NRrOVgMrZNfMMij3ulDsKWqoZojIByB0xuEwxvE9Bf3xfTFttIiPPbmUby+oAiXrnihVXPoZ9BAnaDip1x0BxCrdo50CM21GJKJ2kj5TN6eWgaJxmmuxYBtpUViXpJ4RCdlikyEJlx3zX2joSJE4igrqPgB4EyjC05fEBajFjyl0HAEhAN4HuJzBSB+j0GrQpCnCAR50ZF4pjUPWhWHit3HY8b50BwTmj1BWIwa/PHPX2CmNU80meysrsO9466XjO1wfopx+PGfYmuL/O8Tt+DalnfuTKNLrP1CAVzf3wCeQhIeK9fPid7/eocXX3wr/z62UZvZI8dnKunJeST6cq2NLAB3UUrPRv6RUsoTQqa1p2XJQCn9C4C/dPZ1lTyrhYVLCMNsyw7XG5C/plGrQjDUumsUzB8uX0iMwRccJa+1GLB2rhXN7kCMmtasVsPlDyeLiue3MDjLJKqFcy3hZFAZRg0CQR75A9PwwuxRbVK3qtWcRAgSyDJRXLziVTR9nGsKF0FSqwganWFTjNsfwvWZRtnEPUo+AJlmbYzaWfDIF/JI9NbQz3ihr8KYMWiTM+dERh0oJUKzGLW4d91BSV8LKn4A2PbZWdwx8lrxmMhcIo/elo90XWvyp2yzDk/cni8Zx1UlVqTp1LjQ7BXNM4LgDoRrshRXHsCO8gn4sKZeErIJAA9+/4aYNuvUnOwY/Fe9E05fEMOyzfAFeXGxl/N/kA3DnmOFiguPS6Wx5Q2E8Pyek7K5KHqbdozBSEQyPhK/i/PZic5tTtejZO/tZ9BgR/kESdXLZFGy+br9IahVYeuRsFhebPZKdjHnmjxYtOOYaG641OyT2IHLN1djW2kRzjS4xesq+S3o1dxVSc/LcQQD0/UxE7CgLci1GGDSqnDpSkAyia8sLkSGUYMMg1aifVBxwOqPaiWOe6s/qsVvpxXILno3DUzDW7+YiCxz71UdK43TnDQdlk4fgQHp7avhoHRdX5CPq+IvHnOdmH9E+GzxzmNYP28smj0BXHJ44fQGRSHCG+Dx0n2jodeo4PQFUX/Fh+xrdKJjcPT3C1E3Sj4Sbr/U1JBrMaDJ7Y/J6xBZeGt72YSE/g+CQ+9bCyeKCa1++9/HYXP6YqqjRo5ZnYpTSEnePX0iGIyupE8V7eJ5Klu0qqrEinS9GtdnmpBrMba5LoDgmR4djpnb34AcszRZjZJWIcusRYPTH2MHzjbrEOQpnt9zEhaTJsa2LPgtVM6xItusSxhy2llETsB7H5+MpdNHiBP4yuJCaFUkJhHRoh3HEAjyqL3kwJO7juH4+Ss42+iCNxBC6Q+GYuk7NeK93z9xCNQK4XRf21ywOXxddm/dATm/hKoWJ8IR1/bD4ExTu56v3HVfum80LjZ7FEMXw8IekR23zZ6AWLAryPP4j5kjkGXWwqwL71Hmb/gcM9bsR8Xu8OL8nRxTzBheNWukGNIr58uwatZIWEyamHH/+91fIMukxbbSImwrLRIL2QFAxbQCMT14dJuj/R84joCAYM4rn2L+hs9xpM4uCh3hSqFSv6oZa/ahweXHi7NHSQp06TUcNN2w4imD0dV014RUnU6k+jzbrBOLVhl1KmSZ2r7oSjL9qTnkpOnwwr2jkGEM+z1oVASZLVEbPE/hD4awatZI9DfJJwYyaNVi8idAmnJYzYXLFP/h7Ro8cXs+Nj0wDhThRENcS1KoVBRG4jiCnDQ9eBOFSafGC7NHQaPm4PQG4Q3KO7EFeYrVH9VKIj7Wzxsbo6VZvPMY3vrFxLg7zt6cMZDjCIZlm7H1ofGod/jCNVT2fIVHb8vvUGhypADoDfDhjJIqDv/111g1faSJzxeMr0kQBMVND47DxWYvvAEej26XPlMhjfuNA814Y0ERfEEehADL3muN8jlSZ8dr+09j/byxUHEEJ+udWPbelwCApdNHIK9/uB6GMAY4joOWg1gxN7KmSMW0Atk2y0X4xEsmpZSq/Y3SIjHMlaeATs1qbjD6Jn1GkIicDM41ecSCQkLIWluQtemXjMF1mUZ4/FJnwUaXFx4fD2+QRyDEY8ehb7DmvtFYuKU1Hn5lcSFUnHza68FZJmQZtXhpjhW/2FyNe9d9Knqy52UYoNGkPuwxOjVzlonikoIPRZCnMbVAlLQ0gRCP/Jw0vLGgCN/aw34XkXkAulvuiM6myROQ1FQBgJoLjg4LUIIAKMDzFI/eli+alzJN2hgT3wCzLiZHgiDUCZxr8oAjYU3Uqlkj5Z9pkEeOxQjeQFt2+zx+feuNqLngEK97/8QhWPnBl/jtHQViyGauJZxcasX7X0qypEYXLItMtCZXT0Mpwkcohhft5KlVqxSFjECISjK/rps7pt3PhMHoyfQZQaIz09dGx4+fa/JgwaZwXL/gRS9EZ1y64pU4na2aNRLvHv0WG+aPg4oLp7s+b/fg0hWf7MJr0qqg06lxU44Z20qLEOQp1BxBjlnXLYQIOTiOYICMD8W6uWOg5khMLRClnBYaNQe1moNWw4k7zujPezNXK+WyoKV4Zkahon+NRqPCtRYdNj04Do1OP/oZNFj5wZeiUAcIadqJ6GwbTxsQKXxmmnQSzYvgwHlNP4PE50epFgbPUwxI14nvh/CdR+rsog/D8IFp0GtVUHMEF5o9MfdoMWgk6bcFx0mNCvAH5X2TzjS44vpfMBh9hT4jSCg5mbUnkZFSlEaoJeeBEGZX7/DF+Ak89uZRVEwrgEZF8ExEbY2ND4yTXXizWnwswhN54lC/7oJSvRG7xx9j3qnceyomqdDK4kI4vUFkmSiyTDqsmzsGqz+qxUxrHjJNWvQ3aeELhOJ61vd0OnPMJiJehVbBjOcL8ih55TOJCSFSk7CyuBAaVdivpS3aALWaQ67FCINWjUH99Bh9XaFi1djo36O1g+vnjZX02ZE6O5a+U4O3Fk5Eo9MfE50xKCPstNrkCUhScwtVS/UaFS42e/Di7FH45dYj4rkv3Tcav9v9haQtXSHkMRg9gT4jSMiFebU3kZFclMaUghxcdvolcfivPTBOVuDINGmhUXF48o4C/O5n3xWzYHIcuSpRF1cLuYUgw6CFyxeUmHdsTh8MWhWW3XUz9BoV+hk0eGLHMdHznlKKnDQtHv7RsBiTkEmnRn9T79wBduaYbS+RC3WkuSJyt3/TwDQxTwgFxeq7R+LR7Ufx7Ae1WDp9BK7PNMKgUWFAnFBdYawIQouc1iC6XYKPUmTED09bvz9aExbj57C5Ghvmj4PdHYBOzYmfRfpZRGoSV9/9PQzop4eKAN9cdov1RAS6SshjMLo7fUaQSKYiZ7IIRbMid9BLpg7H3FelzpLfNLpld5TZaTo89fZxURuxsrgQznQ9Bmeaer1aVEihbNL5sa20CP4QjzMNbmw9+A2m3jwIgzK0uGBv7cPzdg+KKw/IOmQu2nEM20qLAFMq76jr6Mwx214ifYuizRXCbj8yMRbPU2Sag5I6MADiChECydYUiTzupftGY+Et30GTK5yG3RvgMShDh+2lRaBoTTx1odkjK9Tb3X4EQjwGZ7bmYJEraCdoEq/J0EOrVmH9vtNxnVMZjL5EnxEkgPjq27aQYdCKmfV0agKAgKcUFdMKULn3lGg3fn7PSbx032ixdLdgdw2EeDHhjrAgLp0+Aml6Ta8XJIDwc+hv0uF8wIP/+MsJzJ80BHMnDo5x5Htt/2kxKkDJITPUCZVauzOdNWbbC8/zkjoZL8wehYcjVPwriwvDUQstJiaOIxicaUKaXtNm4UcuOkLO7yDyuHSDBuftnph8Jf2NOuSktzqUKpmJGl1+LH2nBjvLJ4gRQkoF7b6TbUaIUlBQLJk6HMveO6HonMpg9CX6lCDRmTh9oZgwxkhP9iN1dticPvQzarDpgXHgKaBWEWw9eAajB2dKrnWuKZwFs6/ZV1UEuH/iEHgDPBbtkJaOXrzzGF5fMB5nG93YVlqkGDarZhN3l8HzFA0tC220it9i0qDusgcGrQq/f/sLPDOjsMMCT7LOpZHHUQrZfCXbSovEewj7d4Sw5aHxEr8k4V091+SBL0SRZdKKFXPlxto3l91itNe6uWOwbGahJEqLCRGMvkrvdnvvJKKLbAlRG9FhjMICWD55KKYU5GDLQ+MRDIV3zJddftjdfswae51Y7EhAyNzX1+yrHMfhtf2nkdffELOAZJt1sLuDWPLWP3HP2oNY+cGXskXG9EmmiWa0nUaXX8zhAbSq+LPTdPD4Q/CHePzh7fDCLCz2wSCPc01unG104fj5K3hy1zHUXnKIBcDiIWgNIpHzO4g8LsjLJ50K8RSXmj04cfEKZqzZh+8v/wT3vfwpHrn1Rrz7yPfF5FVCzRa9JlzvZuk7Nfg/24/GJMVaWVyI5/ecFK+/YOMhBHkqhocKiasYjL4I00gkQM5uu/nB8TjX5IlRgY7Ky0D55KG4aWAanryjIGb3s+5/T+P3P/suKqYVxHi7D0jX9zn7aqZJi0dvy0fdZU/MDvCRW4fhF1taFzHBFPTGgiLYnD7Y3QFkmnUsAVAXoqQhaPYExIJto/IysH7eWIQoRb3DiyueAOat/1yioVv9UW1SGotMkxYbHxgXU9DOYtDEHCc4oV5sls9Xcsrmgj/ES0qHn2sKV6jd9MA4iZZlXckYZLU47ArXFRxFh2SZoFERPLz1iCTU9VyTB25fCHNe+TSuPweD0RdggkQC5Oy2pxtcyLUYJM5ncp7ey2cWwubw40idXaxLYPcEQCnw519NhMMbziwYrypmb0ZwJhyQrkPVHKsk4kWurPqHNfUo/cHQcBipO5CiVvcdlPwKMk1arJ83Fu/98wJmjL42Jmw326zDuSaPqKGrmFaQtNnOF+Dx6t+/loT5nr/igU4V1hgIJgTBCZXn+ZjspyuLC7Hi/VosmXqTvIOlJyD6NmQYtRBeOyXn1kaXXzZC4zTLI8FgAGCmjYTI7cqe33MSVSVW7KyuE1Wgcp7egplD+L3ZE0AwRPH8nq/g8VNc19+Iay1G9G9Hiu7eguB4OXxQOnYtnIR9i2/BWwsnQsXJ19lodPlR7/Bh/obPMffVz0RnTEbnI1eXY2VxIX71+hFU7D6OX/7oO7L+CcKYF/6WadJCq1bFmAijTQGNLj9Wfxz2O1r6Tg2KKw9g7qufwe4O4OsGF2as2SeaSbiWxGaX3QE89/FXqJhWgB3lE/DGgiKseD9sshAE/UhyLQbUO3xY+k4NfEEei948KhlHgnNrZL0apbongqkj8l77mp8TgwGkSCNBCFkJ4GcA/ABOAZhPKbUTQgYDOAFAyLt7kFJanoo2CkTuygTTRaZJC4tRiwX/31DoNRzWzxsLtUJRo4wWtWykh3jFtAJJeXGGNDrB5vBh6Ttf4MXZo/HLra0RLy/OHo2/HPsWs4sGo6rEisq9p9jE3YVE7tA9gRBO1TvFRRoIP6d4Yx5oDXfWqJAwtJPneSz6yU0xVUYXbjmMTS05WRZsPIRtpUVi1VjheoLpa/28saL2QC4pVtUcqxhhlUy6dcFZM12vxvayCVCRsG+PigPLI8FgtJAq08ZHAP6dUhokhCwH8O8AFrd8dopS+r0UtSuG/9fevUdJXV2JHv/uqq7qru4GGpvmYRAlDjIhCGoz+GCN14Q8vPeywlWCrxA1JgJDGO7kmgmZa1zLpMcMoI5zs6IBnXHGR7yioglD4nslM3f5iEJQIyhBowYEpdPQPJrqrq6qff+oB1Vdv193dXdV/+qxP2ux6K6qrjq/7lO/Or9z9j57bCjAQ984l6M9UaIxzfpgW7toFj/8ZaKS+rpkRcO+08Cpq6I7rzqHmzfvSF+hpcqLm1yRaIxndh7gzJPH8NNvnEv70R56Y3HCvTGuOPdUAj5h5smjueOKs9JbJFfrjE4h9S2Vnbm75IeHEhkLmQYq+Z1Kd+7ujRJXdUztfHzFBQhCJBojGleOR1x2jVVNf32wK0J3b5yJY+q472tz2X84zO3P/J7tezoTs4XJZbJUAbCHvnEufp+kBx9f+vELOW12GgD0t68F4PlmYcaUCk8GEqr6TMa3LwNf9qIdA4nHld3tx7j+/q3ctGBGTuDW6k1vsObSMxERbn367ZyrnzuvOgefJEoa+4R0hHjLqNp0eXGTKzULdMak0Xzln39DS2NtIv7ksRO7FabWwduP9ViQWwEMtBmUU7zEpm17cuIT/unys5gwupbHlp9PR1eEHz3/e742byp1Aefsir4Bi/dfN9dxcPLR4W4gsYNssMbHsZ4oV97zclZa6pon36b9WA+TmupcN/GKxzXvAcBA+1p4vVmYMaWiFIItrwM2Znw/VUS2A0eA76nq/xvJxmRelYkIdzybyDMfP6rW8UQ4+aR6fpjMzmg/GuHhpefx0eFuOroi3Lx5R3rwkCppvGFJKyePrhvxkt/lJLUm3dUTZe+hMDctmJETf/K3jyWC+JY9sM2C3ApgoA/NzEyJlsZaVs2fxtRxDdQH/Tz+Vxfwx4PH6ehK7BJ55T25FUv71sAA54DFNU++lbOkdcdls/nhL99O7yD7QcfxnF1Ob3j0ddoWzmTimLp+A5fz3S00HlfCvdG89rUwptoVbSAhIs8BEx3uulFVf558zI1AFPhp8r79wBRV7RCRVuBnIvJpVT3i8PxLgaUAU6ZMKUibna7K1i6aRVMoyJhQwPlE2N7FNRdMTWdnqCo90XhWetmdV51Dy6ggT6yYZ1cteUid7D9KliJ322kwtRZfiif3YvTPYhpoM6jU32Tzynns7+zOyrDZsKSVW37xFtv3dLJx6XmOz9PdG3OMV/jez97MeuwzOw/w15+dls6qmDimjj0Hj3Pb4tn88eBxDod7XXc5PX18I5ObBt5dcqDdQlPnAbfU0lTgaD7beZeqcuufprQV7bJYVT+nqjMd/qUGEdcAC4CvqCYWQFW1R1U7kl9vIxGIeYbL89+tqnNUdU5LS0tB2ux0VbZ60xtcf+EnWfPkWzmb1KxdlNikJjM7o7s3zn0vvsdNC2awcel53LRgBnf+ajeqpKPAzcB8PmFishT58UjMMfq+M9yb/rrUgtyK0T+LKZ/NoHw+IRYnPYiAE8WvVs2fBuCaKbHvcHd6b4b//M5neGLFPCY11TkGLO473E3blp30xuLsOXicK+/5Dd9+9HXqAj46j/e69odQwJ9XPY/+MkfgxHngR8/vznnPp5ZB3GZwyiWLqNz6pyltXmVtXEwiuPK/qOrxjNtbgIOqGhORTwLTgD+MVLvcrsr8PkkvXTz49XP5+Eg3neHerKjvicl9/R/b+kf+ev4Z/FXGFdtPlrRaTMQQpK6Cx48Osn5Ja1YtjlSMhAW5FUa+lUbd3iNTxyWKXrmVhE/Fs0wcU5eeNXCKV7jrK+cgQNvCmYSCfr6/eSeQiC9a99Qu1i85h6M90ZzXyKcP5DuLkDrGvYfC6QqnTckZyQmj6ujoinA8YssexqR4FSPxY6AWeFZE4ESa54XAD0QkCsSA5ap6cKQaFajxOU5l1iZv376nk95YnBsefT3nMeMag7yw+jMEa/yMqfWzcel5RONKjU8Y31hLIFBaV8zlIpHHX4cgtC2cSVN9gDGhAEG/cNvi2dQH+y9PbfKTb+yA2yZV9bX+9M+Ggn4eX3EBvdE4gRofNT7hx1ednfOcfV8z9dhwJEZzY6JCbuZuku3HevD5fHxyXCMnNQTZuPQ8Ygp1AR/j8tiLJd+iYJnHuH1PJ8se2MbksSEeX3FBVvC127KHMdXGq6yNP3O5fROwaSTbkhlc6RPhzqvO5pt9qhs21PrTV05Oa71rF82ixidMGHNiuvUTwVKIY60cTaHEennfq8mpoxtsEDFIbmme+VQadZu5GPCD3KXUu+NrNiTa+K3PT8/aSj4165DaxGyw5ePzLQrmdow1Pknf5rRHhc2MmWpVNZ92TidPyN0k5/bFs5OlkYXe5KZRh8NRxo8O8uiy84nE4nT3xllz6ZkE/D46w73c9+J73HLJLC8Pr+Lle8Vs3AcKqfuGEyTo9HcYGwq4vt5QFePv7TabkppFyPy9NTcG2bxyXlZ1z/2Hw+mf3b6nM73s8amJowgFa6w/mqpVFQMJt5Nnc2MwZ6rzhkdf56YFM1j/63f59hen87ePvUFLYy3fuXh6zrpvKm/drkRGRj5XzNVuoIFCvtP7/cn8OxQze6HQf+/+4kDyOY6+A5Htezpp27LTUo9N1auKzQzcTp7dvblTnS2NtZwxoZFbF88iEo3T0ljL8otOd6wp8OOrzuaJFfPKJuXLVL6Bsgnynd7P1F+mQzllL2TOcryw+jNZ7918jsOp5oZdRBhTJTMSrtkYInxhxngWtZ5CUyhAXBPBkV/9l1ey4h9G19U4/jxgVyKmpAw0UHCb3g8F/bQf7clZRhjoSn0oAxMvuc1y5HMctrxmjLOqGEi4nTwbav2smn9GOq3wX6/9i5wd81LlvyePDaVnJ5pCAY5HYgT8PqvzYErKQHEATtP79183l4+P9DgOFgZaChno9YYrHlc6wxHCkRgxVeoCfk4KBTkU7i3oh7lbcT6R7FoutrxmTK6qGEi4rY1GYpoeRACuO+ZF48q9186h41gkK07ijstmc6wnymnNljlgSsNA+0E4XVUrytV3veg4WBjoSj2f/Sf6C/7sTzyuvN/RxcdHutPvuy/MGJ81+C9UTEbqOO54NlHGvG82hi1fGuOuKgYSfU+eIoJfIBaPpzeb6Qz3Eld1vLoalVza6Bsn8a1HEvv7j6oL2FWKKQn5TL/3var+8NBx18HCQDMOA73ecIIxO7oiOXU1FrWekjX4H0qwaH+/t5u/NJPLNryU8/ybV84jFsdx6afQGSvGlJuqCLaE1MZGQY50R7lsw0ssf/C3tB+N0LZlJ5ff/TJtW3am95Houw324XAvAo4n2/qgv2TXg011Sg0UPjG2Pq9t2d22xw7U+PIKMOzv9YYTjBmJxnJmCd3qrqTeg/lsge3G5xNU1TEAe39nN5fc9QLz1v6KS+56gV0fHyUajbPr46M5tw/mNY2pBFUzkIDsk9ryi05nxU9/m5P62VQfZM2lZ6brZNz34nvsPRR2rSFwPJlnbky5chos3PrlWRzrjgK4ZjrkYzjBmMEaf05dDbf3YWYhreF8sDsNqlbNn5ZTX+T6+7dy4FhP2WSsGFNMVTWQyDypuV3ZCPDdx3+XnqVY+dlpbNq2J72TXd+T7anN9Zb+ZcqazydMGF1L28KZ6QH0uqd2cfW9r9DRFRn0DEemfIqBuWluCHJqcz23fvnE+27Ttj2sX9JatEJaToOqqeManGOnYvGyylgxpliqIkYiJXO9N3Vl03ft1+8THrr+XPwi1PiER17dw3f/66c42BWhNxbntsWzmTC6jqBfqAv6GBuyip6m/IUjMb72b6/m3D7Q/hIDxQfkWwzMTWNdDQF/iI1Lz8PnE/w+4aRQ0DEmoxCpqG7BqE7nihq/c20em6E01aaqBhKZJzWnKoV3feUcvv/vO3hm5wEmjw2x4autXDi9havvfSXrMWPra2iqt+BKUzkGm8aZbxDlUPdeGOj5nQIrC5WK2vf5naqU3nP1HMY31g5rkGRMpRDV8g8MmjNnjm7dujWvx2YX6YIPDiaWM5oba1n31Fs8s/NA+rGTx4ZoWzgz60otVQVw/Ki6Qh+GKQ7Pp4sG0z+9MtjsivajPVxy1ws5H9qF2i7a7fkfXXY+cVXHAUkxt+t2m30pQNZG1ffP0777C89eu5S9v+a/e90EyLN/ejIjISI3A9cD7cmb/req/jJ5398BXydRRnyVqj5diNfs+4aflKzUWeNPZF3EVLMGEXAiK6Pvbb3ReCGaZEzJGOzMQbF3tHR7/nBvjBseeT1d4yZzkFDMnSf71hexlE9jTvAy2PIOVT0r+S81iJgBXAF8GrgYuEtEhr3g6BbNDaSDyEKBGtesjL632RqoqXbDCaIczvN/0HGc5Red7hpIOZzA0HwUIjPEmEpTalkbC4GHVbVHVd8D3gHmDvdJ/9TlnKb1p66e9GOcorVvXzybsQ0BK9JjKt5gPyCLXcCquSHIhj7ZGWsXzeJHz++mKRQAvMmQKKciZcaMFC+DLVeKyNXAVuAGVT0EfAJ4OeMxe5O3DYtTlc+9h8J098azpiknjK7l8RUX0B2J8W57F2uefBuAmxbMoLkhyKQxdUwaE7JpTFNxBlte3G232I6uSEGm+n0+YVJTHW0LZ1If9NMZ7kdoTycAAAz0SURBVOW2p3fRfqyHznAv4M3sYLkVKTNmJBRtICEizwETHe66EfgJ0AZo8v/bgetwDuxwvCQSkaXAUoApU6b02xa/iGM0d61fHIOz/mxcA0e6o7Qf62HvoTBtW3ayfkkrE0bV2SDC5GUw/bMUDOUDMrVbbLECHJtCQSaOqct67lu/PIt1T+3ybHaw2EXKRkq59c9qNNggVC+DMz3P2hCR04AtqjozGWiJqv5D8r6ngZtV9aX+nmOgqOODXT3s+ugo//rCeyxqPYXmhiAnNQRprKvh0oxiRXAi8nxsKMCBYz1EY3Fq/D7GN9ZSU1NqK0EmD56P/LyOis+HW5aEW42JgX7ukWXnM3H08AfemTOGgRofNT4hHPEuyLEImSFV3z8ta6MwijSQKOmsjUmquj/57SXAm8mvNwMPicg/AicD04BXhvt6TaEgk8eGWPnZaeltsSePDbF+SSstjbVZJ8HUVVhNTS0nN4X6eVZjKsdgy4unPjTdZjL2dYY5HO5l+oRRAEPOcnDcM6Jh+Mc7VMXMDDFmOLycwfAqRmKdiJxFYtnifWAZgKruEJFHgJ1AFPimqg578dHnE2oDflb89DdZa8DLH9zmuE9EwGYeTJUZbHnx1Ie721R/R1eEv9n4GptXzhtwMDJcI52O6bYhljHVypNPTFX9qqqeqaqzVPVLGbMTqOotqnq6qk5X1ScL9Zq9Ued98U9trs+pn+G3cYSpQn1TJ93eM5lxE07ZG2sXzWL9r99N7PsQiRU1y8HSMY3xXtVskR2ocd4XH+CB6+YSU+Wjw92se2oX/+fKs2n2cPrUmFLgNtsgktjR0eeT9EzGI8vOZ19nmI6uCLc9vYvtezqZPDZEzKEsdyGzHDq6Itzx7C5uWjCDplCAznAvdzy7i1sumZX3rIFtMGXM8FTFQCIeV451R3Nqa/xkSStrnnwrXVtj7aJZtIwK4rdziDGOcRNrF83i5s1v8q3PT08vT/h8wsTRdRwO9/I3G1/LWsKoCxQ3yyEej3PNBVNZvemNrDbG4/ntPlvMbbWrmQVQVhfPszYKIZ+sjdf3HGZcY5C6gJ9jPVFG1QUca2vcf91cmuoDnNRga6AVwvNPA6+j4ocjHlc+OtLNvs4w3b0x/D7BJ8LxSIzZp4zJep84XdkDRf2g3tcZ5rINLzlmjeQTLF3smiF5qMj+aQOJ0pdnsGXpZm2MpHhc2d/ZzU0/fzPriqXGJ461Nfw+oSlkO1caA4m4CVXlll+8xbe/OD1rRm/DklaaQieWAdyCEIuZ5aAuSyf5XiDZBlPGDF/FhxV2dEVY9uC2rGCv1ZveYGxD0HEv/1DQb1OaxmQI1vhZNX9aevkAEu+jZQ9uSwdNxuNK+9EePjx0nPajPVnBjsWsf5FPzY/+2lbsmiHGVIOKn5Fwu+KIxZWHrj+Pv9+yIx0jcc/VcxhnSxrGZGluCDJ1XIPrlbuXcQaZcRwtjbWsmj+NqeMaUDQ9YOivbU5xIFZPJ5ctVZj+VPxAwi3y/O2PjtK2ZScblrTStnAmPp/PorWNceDzCfW17kGTg63TUei2TZ8wis0r57G/szs9+5gaEEwYXdtv22yDKWOGr+KXNtyqCKby3Jc9uA2fz1eUksPGVIpxDbWu1T69jjPw+YRYnJwlzOvv30o4MnDbil163JhKV/EzEplVBE9trmf3gWPpPHewwCpj8tHflXspFLJyXcJUPG+bMZWu4mckIFlFcHQdH3Qcp23LzvQgAuykYky+3K7cnXa3LFScQX+BkpncgibrAr6itc0Yk1DxMxIptQEfzY1B1i9pZXmfdVQ7qRgzdMWKMxhMEKdb0OS4hlrGNdRaDIQxRVQVA4mOrghX3/sKew+FOfuUJm5aMIPmhiAnN4UKUurYmGpXjEJWgwniHGgwY0W2jCmeih9IxONKJBrj9sWz6Qz3sv7X77LsgW0AvLD6MzaIMKZEOcU9tDTWEonG+PDQ8ZzBglXlNMYbFT2QcJoaXbtoFrc9vYv2Yz0WG2FMCesbxHn2KU185+LpXH73y1YXw5gS4kmwpYhsFJHXkv/eF5HXkrefJiLhjPvWD+d1nKZGV296g1Xzp1lshDElrm8Q56r509JbdEPhS5IbY4bGkxkJVb089bWI3A4czrj7XVU9qxCv45YSdvr4RiY3hewqxpgS1jfuodglyY0xQ+Np+qeICHAZ8H+L8fxuKWGhgNXTMKYcZKachgI1VhfDmBLk9T4Sfwl8rKq7M26bKiLbReQ/ROQvh/PkxcxvN8aMLHs/G1Oaira0ISLPARMd7rpRVX+e/PpKsmcj9gNTVLVDRFqBn4nIp1X1iMPzLwWWAkyZMsWxDbaPvvFKPv3TDI69nwvH+qcppKINJFT1c/3dLyI1wKVAa8bP9AA9ya+3ici7wBnAVofnvxu4G2DOnDnO291hKWHGG/n2TzM49n4uDOufppC8XNr4HPC2qu5N3SAiLSLiT379SWAa8AeP2meMMcaYAXi5j8QV5AZZXgj8QESiQAxYrqoHR7xlxhhjjMmLZwMJVb3W4bZNwKaRb40xxhhjhsLrrA1jjDHGlDEbSBhjjDFmyES1/AN2RaQd+GCAh40D/jQCzSkFdqwn/ElVLx6pxjix/pmlWo4T8jvWcumfhVIOf39r4wl59c+KGEjkQ0S2quocr9sxEuxYy0+lHMdAquU4obqONV/l8DuxNg6eLW0YY4wxZshsIGGMMcaYIaumgcTdXjdgBNmxlp9KOY6BVMtxQnUda77K4XdibRykqomRMMYYY0zhVdOMhDHGGGMKrKoGEiJys4h8KCKvJf/9N6/bVEgicrGI7BKRd0Tku163p5hE5H0R+V3y75hT1K3cVHrfBOuf1ay//i0if5fsE7tE5Iset7Mk+6hTfxKRk0TkWRHZnfx/rGftq6alDRG5GTimqrd53ZZCSxY7+z3weWAv8Cpwparu9LRhRSIi7wNzVLXU873zUsl9E6x/Vju3/i0iM0jUXJoLnAw8B5yhqjEP2liyfdSpP4nIOuCgqq5JDnrGqupqL9pXVTMSFW4u8I6q/kFVI8DDwEKP22RMivVP42Qh8LCq9qjqe8A7JPqKF8qtjy4E7kt+fR/wP7xqSDUOJFaKyBsicq+XU0FF8AlgT8b3e5O3VSoFnhGRbSKy1OvGFEil9k2w/mmc+3cp9YtSaktfTv1pgqruB0j+P96rxnlZRrwoROQ5YKLDXTcCPwHaSPxR2oDbgetGrnVFJQ63VfK61TxV3Sci44FnReRtVf1PrxvVnyrum2D9s+T753ANsX+XUr8opbb0ldOfvG5QpoobSKjq5/J5nIjcA2wpcnNG0l7glIzvJwP7PGpL0anqvuT/B0TkCRLTkiV9oq7ivgnWP0u+fw7XEPt3KfWLUmpLFpf+9LGITFLV/SIyCTjgVfuqamkj+ctOuQR406u2FMGrwDQRmSoiQeAKYLPHbSoKEWkQkVGpr4EvUOZ/ywrvm2D9s9L+noPST//eDFwhIrUiMhWYBrwy0u1LKsk+2k9/2gxck3zYNcDPvWlhBc5IDGCdiJxFYrrqfWCZt80pHFWNishK4GnAD9yrqjs8blaxTACeEBFI9OGHVPUpb5s0bBXbN8H6ZwX0z+Fy7N+qukNEHgF2AlHgm15kbCTbUqp91LE/icirwCMi8nXgj8BirxpYVemfxhhjjCmsqlraMMYYY0xh2UDCGGOMMUNmAwljjDHGDJkNJIwxxhgzZDaQMMYYY8yQ2UCiAonIUyLSKSKVtqmRKXMicpaIvCQiO5LbJV/udZuMMcNj6Z8VSETmA/XAMlVd4HV7jEkRkTMAVdXdInIysA34lKp2etw0Y8wQ2YxEGRORv0he1dUldz/bISIzVfV54KjX7TPVzal/AkFV3Q3pbX8PAC2eNtRUPRFpE5H/mfH9LSKyyss2lRObkShzIvL3QB0QAvaq6j8kb78I+LbNSBgvufXP5H1zSZQ//rSqxj1qojGIyGnA46p6joj4gN3AXFXt8LRhZaLatsiuRD8gsUd8N2AjaFNqHPtnsvbCA8A1NogwXlPV90WkQ0TOJrEl9XYbROTPBhLl7ySgEQiQuPLr8rY5xmTJ6Z8iMhr4BfA9VX3Zy8YZk+GfgWtJlEK/19umlBdb2ihzIrIZeBiYCkxS1ZXJ2y/CljaMx/r2T+B/AU8C/66q/+Rl24zJlKz4+TsSg95pXhUPK0c2I1HGRORqIKqqD4mIH3hRRD4LfB/4c6BRRPYCX1fVp71sq6k+Tv2TRGnmC4FmEbk2+dBrVfU1j5ppDACqGhGRXwGdNogYHJuRMMYYU/WSQZa/BRanMotMfiz90xhjTFUTkRnAO8DzNogYPJuRMMYYY8yQ2YyEMcYYY4bMBhLGGGOMGTIbSBhjjDFmyGwgYYwxxpghs4GEMcYYY4bMBhLGGGOMGbL/D2jvmdWFNC25AAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.pairplot(\n", " data=pd.DataFrame(np.hstack([x_ND, y_N[:,np.newaxis]]), columns=['x1', 'x2', 'y']));" ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [], "source": [ "def nonlinear_toy_nn_regression_loss_function(nn_params):\n", " yhat_N = predict_y_given_x_with_NN(x_ND, nn_params)\n", " return 0.5 / ag_np.square(sigma) * ag_np.sum(np.square(y_N - yhat_N))" ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [], "source": [ "# Initialize 1-layer, 10 hidden unit network with small random noise on weights\n", "\n", "H10_init_nn_params = make_nn_params_as_list_of_dicts(\n", " n_hiddens_per_layer_list=[10], n_dims_input=2, n_dims_output=1,\n", " weight_fill_func=lambda sz_tuple: 0.1 * np.random.randn(*sz_tuple))" ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "completed iter 1/300 after 0.1 sec | loss 1.194825e+07\n", "completed iter 2/300 after 0.3 sec | loss 1.163953e+07\n", "completed iter 3/300 after 0.6 sec | loss 1.117648e+07\n", "completed iter 50/300 after 9.1 sec | loss 2.276573e+06\n", "completed iter 100/300 after 18.9 sec | loss 2.803082e+06\n", "completed iter 150/300 after 29.4 sec | loss 1.879504e+06\n", "completed iter 200/300 after 39.9 sec | loss 1.785398e+06\n", "completed iter 250/300 after 50.0 sec | loss 1.680417e+06\n", "completed iter 300/300 after 60.4 sec | loss 1.588551e+06\n" ] } ], "source": [ "H10_opt_nn_params, H10_history = run_many_iters_of_gradient_descent_with_list_of_dict(\n", " nonlinear_toy_nn_regression_loss_function,\n", " autograd.grad(nonlinear_toy_nn_regression_loss_function),\n", " H10_init_nn_params,\n", " n_iters=300,\n", " step_size=0.000001)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot objective function vs iters" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "plt.plot(H10_history['iter'], H10_history['f'], 'k.-')\n", "plt.title('10 hidden units');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot predicted y vs. true y for each example as a scatterplot" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "yhat_N = predict_y_given_x_with_NN(x_ND, H10_opt_nn_params)\n", "\n", "plt.plot(yhat_N, y_N, 'k.');\n", "plt.xlabel('predicted y|x');\n", "plt.ylabel('true y');" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "_, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=False, figsize=(10,3), squeeze=False)\n", "subplot_grid[0,0].plot(x_ND[:,0], y_N, 'k.');\n", "subplot_grid[0,0].plot(x_ND[:,0], yhat_N, 'b.')\n", "subplot_grid[0,0].set_xlabel('x_0');\n", "\n", "subplot_grid[0,1].plot(x_ND[:,1], y_N, 'k.');\n", "subplot_grid[0,1].plot(x_ND[:,1], yhat_N, 'b.')\n", "subplot_grid[0,1].set_xlabel('x_1');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## More units! Try 1 layer with H=30 hidden units" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Initialize 1-layer, 30 hidden unit network with small random noise on weights\n", "H30_init_nn_params = make_nn_params_as_list_of_dicts(\n", " n_hiddens_per_layer_list=[30], n_dims_input=2, n_dims_output=1,\n", " weight_fill_func=lambda sz_tuple: 0.1 * np.random.randn(*sz_tuple))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "H30_opt_nn_params, H30_history = run_many_iters_of_gradient_descent_with_list_of_dict(\n", " nonlinear_toy_nn_regression_loss_function,\n", " autograd.grad(nonlinear_toy_nn_regression_loss_function),\n", " H30_init_nn_params,\n", " n_iters=300,\n", " step_size=0.000001)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot objective function vs iterations" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "plt.plot(H30_history['iter'], H30_history['f'], 'k.-');\n", "plt.title('30 hidden units');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot predicted y value vs true y value for each example" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "yhat_N = predict_y_given_x_with_NN(x_ND, H30_opt_nn_params)\n", "\n", "plt.plot(yhat_N, y_N, 'k.');\n", "plt.xlabel('predicted y|x');\n", "plt.ylabel('true y');" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "_, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=False, figsize=(10,3), squeeze=False)\n", "subplot_grid[0,0].plot(x_ND[:,0], y_N, 'k.');\n", "subplot_grid[0,0].plot(x_ND[:,0], yhat_N, 'b.')\n", "subplot_grid[0,0].set_xlabel('x_0');\n", "\n", "subplot_grid[0,1].plot(x_ND[:,1], y_N, 'k.');\n", "subplot_grid[0,1].plot(x_ND[:,1], yhat_N, 'b.')\n", "subplot_grid[0,1].set_xlabel('x_1');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Even more units! Try 1 layer with H=100 hidden units" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Initialize 1-layer, 100 hidden unit network with small random noise on weights\n", "H100_init_nn_params = make_nn_params_as_list_of_dicts(\n", " n_hiddens_per_layer_list=[100], n_dims_input=2, n_dims_output=1,\n", " weight_fill_func=lambda sz_tuple: 0.05 * np.random.randn(*sz_tuple))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "H100_opt_nn_params, H100_history = run_many_iters_of_gradient_descent_with_list_of_dict(\n", " nonlinear_toy_nn_regression_loss_function,\n", " autograd.grad(nonlinear_toy_nn_regression_loss_function),\n", " H100_init_nn_params,\n", " n_iters=600,\n", " step_size=0.0000005)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "yhat_N = predict_y_given_x_with_NN(x_ND, H100_opt_nn_params)\n", "\n", "plt.plot(yhat_N, y_N, 'k.');\n", "plt.xlabel('predicted y|x');\n", "plt.ylabel('true y');" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "_, subplot_grid = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=False, figsize=(10,3), squeeze=False)\n", "subplot_grid[0,0].plot(x_ND[:,0], y_N, 'k.');\n", "subplot_grid[0,0].plot(x_ND[:,0], yhat_N, 'b.')\n", "subplot_grid[0,0].set_xlabel('x_0');\n", "\n", "subplot_grid[0,1].plot(x_ND[:,1], y_N, 'k.');\n", "subplot_grid[0,1].plot(x_ND[:,1], yhat_N, 'b.')\n", "subplot_grid[0,1].set_xlabel('x_1');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Try it yourself!\n", "\n", "* Can you train a prediction network on the non-linear toy data so it has ZERO training error? Is this even possible?\n", "\n", "* Can you make the network train faster? What happens if you play with the step_size?\n", "\n", "* What if you made the network **deeper** (more layers)?\n", "\n", "* What other dataset would you want to try out this regression on?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.15" } }, "nbformat": 4, "nbformat_minor": 2 }