{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "035b5e96",
   "metadata": {},
   "source": [
    "# Praxisbeispiel - Reinforcement Learning Hands-on: Roboter im Raum\n",
    "\n",
    "In diesem Notebook wird ein einfacher Q‑Learning-Algorithmus implementiert, mit dem ein Roboter (Agent) lernt, einen Ausgang in einem 2D-Gitter zu finden. Die Visualisierung ermöglicht es, den Entscheidungsprozess und die Pfadfindung Schritt für Schritt nachzuvollziehen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "de8dfcf2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T17:29:37.526893Z",
     "iopub.status.busy": "2026-03-24T17:29:37.526593Z",
     "iopub.status.idle": "2026-03-24T17:29:37.802469Z",
     "shell.execute_reply": "2026-03-24T17:29:37.802002Z",
     "shell.execute_reply.started": "2026-03-24T17:29:37.526868Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "from IPython.display import clear_output\n",
    "\n",
    "# Für Inline-Plots in Jupyter\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03ad23b5",
   "metadata": {},
   "source": [
    "## Definition der Umgebung: GridWorld\n",
    "\n",
    "Die Umgebung wird als 2D-Gitter modelliert, in dem der Roboter sich bewegen kann. Der Ausgang (Goal) wird als Zielzelle definiert."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b48b48c4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T17:29:39.851597Z",
     "iopub.status.busy": "2026-03-24T17:29:39.851179Z",
     "iopub.status.idle": "2026-03-24T17:29:39.863228Z",
     "shell.execute_reply": "2026-03-24T17:29:39.862659Z",
     "shell.execute_reply.started": "2026-03-24T17:29:39.851570Z"
    }
   },
   "outputs": [],
   "source": [
    "class GridWorld:\n",
    "    \"\"\"\n",
    "    Ein einfaches 2D-Grid, worin ein Agent von einer Start- zu einer Zielposition navigieren soll.\n",
    "    \n",
    "    Attribute:\n",
    "        width (int): Breite des Gitters (Anzahl Spalten).\n",
    "        height (int): Höhe des Gitters (Anzahl Zeilen).\n",
    "        start (tuple): Startposition des Agenten als (x, y).\n",
    "        goal (tuple): Zielposition (Ausgang) als (x, y).\n",
    "        agent_pos (tuple): Aktuelle Position des Agenten als (x, y).\n",
    "    \"\"\"\n",
    "    def __init__(self, width, height, start, goal):\n",
    "        \"\"\"\n",
    "        Initialisiert die Gitterwelt mit den angegebenen Dimensionen und Positionen.\n",
    "\n",
    "        Args:\n",
    "            width (int): Breite des Gitters.\n",
    "            height (int): Höhe des Gitters.\n",
    "            start (tuple): Startposition des Agenten als (x, y).\n",
    "            goal (tuple): Zielposition (Ausgang) als (x, y).\n",
    "        \"\"\"\n",
    "        self.width = width\n",
    "        self.height = height\n",
    "        self.start = start\n",
    "        self.goal = goal\n",
    "        self.reset()\n",
    "\n",
    "    def reset(self):\n",
    "        \"\"\"\n",
    "        Setzt die Agentenposition auf die Startposition zurück (nach jeder Episode notwendig). \n",
    "\n",
    "        Returns:\n",
    "            tuple: Die aktuelle (zurückgesetzte) Position des Agenten (x, y).\n",
    "        \"\"\"\n",
    "        self.agent_pos = self.start\n",
    "        return self.agent_pos\n",
    "\n",
    "    def step(self, action):\n",
    "        \"\"\"\n",
    "        Führt einen Schritt in der Umgebung basierend auf der gegebenen Aktion aus.\n",
    "\n",
    "        Aktionen werden wie folgt kodiert:\n",
    "            0 = oben\n",
    "            1 = rechts\n",
    "            2 = unten\n",
    "            3 = links\n",
    "\n",
    "        Args:\n",
    "            action (int): Index der gewählten Aktion (0 bis 3).\n",
    "\n",
    "        Returns:\n",
    "            tuple: Nächste Agentenposition (x, y) nach dem Schritt.\n",
    "            float: Belohnung (Reward) für diesen Schritt.\n",
    "            bool: True, falls das Ziel erreicht ist; False sonst.\n",
    "        \"\"\"\n",
    "        x, y = self.agent_pos\n",
    "        if action == 0:   # nach oben\n",
    "            y = max(y - 1, 0)\n",
    "        elif action == 1: # nach rechts\n",
    "            x = min(x + 1, self.width - 1)\n",
    "        elif action == 2: # nach unten\n",
    "            y = min(y + 1, self.height - 1)\n",
    "        elif action == 3: # nach links\n",
    "            x = max(x - 1, 0)\n",
    "        self.agent_pos = (x, y)\n",
    "        reward = -1  # Schrittkosten\n",
    "        done = False\n",
    "        if self.agent_pos == self.goal:\n",
    "            reward = 100\n",
    "            done = True\n",
    "        return self.agent_pos, reward, done\n",
    "\n",
    "    def render(self):\n",
    "        \"\"\"\n",
    "        Visualisiert die aktuelle Umgebung mit Hilfe von Matplotlib.\n",
    "\n",
    "        - Zielzelle wird mit Grauton 0.5 dargestellt.\n",
    "        - Agentenposition wird mit Weiß (1.0) dargestellt.\n",
    "        \"\"\"\n",
    "        grid = np.zeros((self.height, self.width))\n",
    "        # Zielzelle markieren\n",
    "        grid[self.goal[1], self.goal[0]] = 0.5\n",
    "        # Agentenposition markieren\n",
    "        grid[self.agent_pos[1], self.agent_pos[0]] = 1\n",
    "        plt.imshow(grid, cmap='gray_r')\n",
    "        plt.xticks(range(self.width))\n",
    "        plt.yticks(range(self.height))\n",
    "        plt.show()\n",
    "        plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bf1d35a",
   "metadata": {},
   "source": [
    "## Q‑Learning Parameter und Initialisierung\n",
    "\n",
    "Wir definieren die Lernparameter sowie die Q‑Tabelle. Die Aktionen werden wie folgt kodiert:\n",
    "\n",
    "- 0: oben\n",
    "- 1: rechts\n",
    "- 2: unten\n",
    "- 3: links"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "443839ad",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T17:29:41.300950Z",
     "iopub.status.busy": "2026-03-24T17:29:41.300423Z",
     "iopub.status.idle": "2026-03-24T17:29:41.309319Z",
     "shell.execute_reply": "2026-03-24T17:29:41.308097Z",
     "shell.execute_reply.started": "2026-03-24T17:29:41.300899Z"
    }
   },
   "outputs": [],
   "source": [
    "# Lernparameter\n",
    "alpha = 0.1      # Lernrate\n",
    "gamma = 0.9      # Diskontfaktor\n",
    "epsilon = 0.2    # Explorationsrate \n",
    "episodes = 5     # Episoden\n",
    "\n",
    "# Erstelle die Umgebung\n",
    "env = GridWorld(4, 4, start=(0, 0), goal=(2, 3))\n",
    "\n",
    "# Q‑Tabelle initialisieren: Dimensionen (Breite, Höhe, Anzahl Aktionen)\n",
    "Q = np.zeros((env.width, env.height, 4))\n",
    "\n",
    "def choose_action(state, Q, eps):\n",
    "    \"\"\"\n",
    "    Wählt eine Aktion nach einer einfachen epsilon-greedy-Strategie (deterministische Variante).\n",
    "\n",
    "    Diese Funktion prüft, ob eine Zufallszahl < epsilon ist:\n",
    "      - Falls ja, wird eine zufällige Aktion (0-3) gewählt (exploration).\n",
    "      - Falls nein, wird die Aktion mit dem höchsten Q-Wert gewählt (exploitation).\n",
    "\n",
    "    Args:\n",
    "        state (tuple): Aktueller Zustand des Agenten als (x, y).\n",
    "        Q (numpy.ndarray): Q-Tabelle der Form (width, height, 4).\n",
    "        epsilon (float): Wahrscheinlichkeit für zufällige Exploration (Wert zwischen 0 und 1).\n",
    "\n",
    "    Returns:\n",
    "        int: Index der gewählten Aktion (0 bis 3).\n",
    "    \"\"\"    \n",
    "    if np.random.rand() < eps:\n",
    "        return np.random.randint(4)\n",
    "    else:\n",
    "        x, y = state\n",
    "        return np.argmax(Q[x, y, :])\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dfb151e",
   "metadata": {},
   "source": [
    "## Training mit schrittweisr Grid-Visualisierung\n",
    "\n",
    "In jeder Episode wird der Agent von der Startposition aus agieren, bis er den Ausgang erreicht. Nach jedem Schritt wird die aktuelle Position visualisiert."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "bd3248b2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T17:29:42.473538Z",
     "iopub.status.busy": "2026-03-24T17:29:42.473244Z",
     "iopub.status.idle": "2026-03-24T17:29:59.482955Z",
     "shell.execute_reply": "2026-03-24T17:29:59.481380Z",
     "shell.execute_reply.started": "2026-03-24T17:29:42.473517Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAGzCAYAAAASUAGgAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAGTtJREFUeJzt3QuQV1UdB/CzgKwY7CahILL4SLPwgY0K4VskjZTBpofZlPjI0kFLqXGgF9VUYJRiSuaUaS9KpcCy8DEoMDaigmloqWmmmPKyZJFiUfbfnDvz33aXXWCB38Lufj4zl+V/9/7vPXv+O/d7z+PerSiVSqUEADtYtx29QwAQMACE0YIBIISAASCEgAEghIABIISAASCEgAEghIABIISAYYf76le/mioqKtq1Zv/xj38Ux7zllltSV5F/1vwzL168eLv2s//++6fzzjtvh5ULygRMF1c+SbW2LFq0KHVV+cTbUp1cfPHF27zPDRs2pGuvvTa9+93vTlVVVemtb31rOvTQQ9OnPvWp9NRTT6VdwV/+8pfiIiGHdnPf//732xTit956a/r4xz+eDj744KLuTj755Ba3e/3119PkyZPT+973vtS3b98ud7HQWfXY2QVg1/D1r389HXDAAZusP+igg9q8ry996Utp4sSJqTM48sgj0+c+97km697xjnds8/4++MEPprlz56ZzzjknXXTRRemNN94oguXOO+9Mxx57bHrnO9+Z2tvTTz+dunXr1iRgvva1rxVhkEO2ecD069dvq1s8N9xwQ1qyZEk65phj0quvvtrqdqtXry5+BwcPHpyGDh2a5s+fvx0/EbsKAUNh9OjR6eijj94xv1Q9ehRLZ7DvvvsWV+A7wiOPPFIEyTe/+c30hS98ocn3rr/++vTaa6+l9pKfcbt+/frUq1evVFlZGXacn/3sZ0Ud5gA77LDDWt1un332Sa+88koaMGBA0eWXA4mOTxcZbRrj+M53vpOuueaatN9++xUnp5NOOik98cQTWxyDuffee9Pxxx9fdAn17t07HXLIIZucZFeuXJkuvPDC1L9//7T77rsXV7I/+clPNilLPhHnK+jq6upif+PGjWv15JxbBx/60IeKbpe8zxyiv/3tb9vcrbVu3bq0vZ577rni63HHHbfJ97p3757e9ra3NVn3z3/+s6iPgQMHFiGQW5iXXHJJUZ7G6urq0oQJE9Jee+2V3vKWt6QPfOADadWqVU22yS2RM888M919991FHeTP7sYbb2z4XrlFkrulPvzhDxf/P+WUUxq6BXOLIm/35JNPpgULFjSsb63Lq6ympqZJ66g1+efL4ULn0jkuM9lua9asKbopGssnkOYnvZ/+9Kdp7dq1afz48cUVcB5PGDlyZFq6dGkRDC3JJ6V8cjviiCOKbpB8Mnn22WfTH//4x4Zt/vvf/xYnq7z+0ksvLU6mt99+e3Hiy+Hx2c9+tuHKe+zYsemBBx4oxkLe9a53pdmzZxch09Jx88k8X0HnLrt88r3tttvSWWedlX79618XJ+Itue+++9Iee+yRNm7cWITqFVdc0VCWtsrvz37xi18U5dpcK+/ll19Ow4YNK372PD6Tu85y4MyaNSv95z//ST179mzY9rLLLkt77rlnMYaRLwSmT59e1GEe/2jeFZa75j796U8X3XM55Js78cQT02c+85n0ve99r7gAyPWb5a95v/lY+QLhi1/8YrG+tc8cCvnvwdB13XzzzfnvAbW4VFZWNmz3/PPPF+t69epVeumllxrWP/TQQ8X6K664omHd5MmTi3Vl11xzTfF61apVrZZj+vTpxTY///nPG9Zt2LChNGLEiFLv3r1LtbW1xbo5c+YU2337299u2O7NN98snXDCCcX6/POUnXrqqaXDDz+8tH79+oZ19fX1pWOPPbZ08MEHb7FuxowZU7rqqquKY950000Nx7jyyitL2yIf+6STTir20b9//9I555xTmjFjRumFF17YZNtzzz231K1bt9IjjzzS4n4af3ajRo1qWJflz6J79+6l1157rWHdfvvtV2x71113bbK//L1x48Y1vL799tuLbe+///5Ntj300EOLn2FbbO1788/c/LOkY9JFRmHGjBlFN1bjJQ9GN5ev/nOLoCxfZQ8fPjz94Q9/aLUmczdWdscdd6T6+voWt8nvz10k+Qq7bLfddiuupvMMo9wtU94uX/nnrqLG3Uv5yrqxf/3rX0Xr4yMf+UjR4sqts7zkgebTTz89/e1vfytaBJuTu9KuvPLKosV0wQUXFGXI77366qvTSy+91ObfnNwizF1U3/jGN4oWxy9/+cuiJZhbNmeffXZDN1+uozlz5qQxY8a0OC7WvPsxt3AarzvhhBOKFtcLL7zQZLvcKszlh/YiYGgIilGjRjVZch98c3m6aXN5VlVLU1rL8skzdwl98pOfLLpUPvrRjxZdVY3DJp8M876b99eXu2jKJ8v8NQ8I526axpp39+Suttyd9uUvf7kYm2i85K6k8phPW+STeO4ie/PNN7d5llPuHszdS3/961+LbrAcMu95z3uK+sjdWlkeP6mtrd3soHhjeeZVYzm8sn//+99N1rc0SxAiGYMhXB5QXrhwYbr//vvT73//+3TXXXcV4wN57Oaee+4pWiA7Wjm8Pv/5z7d61b4tU7DzoHW5hbS9clDmsM1Tl/O9MDlktuXej9bqr/lfQ8+fA7QnAUOb5K6l5p555plN7pdoLrdMTj311GLJXUzf+ta3iiv5HDq5tZS7if785z8XwdC4FVO++bA8QJ6/zps3r+g2a9yKyQPYjR144IEN3Wx5/zvK3//+9+JrbgntKLmMeQJErtvcjbf33nsXN2E2n53XXjb3FIb2fkIDHZsuMtokjw00Hrt4+OGH00MPPVTcR9Oalq728w2M5Sm22fvf//60fPnyJjOfclfUddddVwRJng5d3i6vzzfwleXxhrxdY/kknWel5am4+f6K5ppP422pzHm/jeWbIqdOnVrM4Gqp+3BLcoC8+OKLm6zPYy8PPvhg0bWVgysHbB7r+t3vftfiY2Cat0x2tDzbrlyulr7Xnvfr0LFpwVDIA/otPaok311ebg2Uu5Xy/Sx5kD2HQ566mqcy58Hw1uSpybmL7IwzzihaIHnsI98RPmjQoGJf5YHqHAZ5WnK+8zu3iPKU3DyVOR+jT58+xXZ54DuP5+Rpx3ncZ8iQIek3v/lNMc26pYkLef+HH354MS03/xwrVqwoTuZ5kP7xxx/f7AB/HozP99DksYscODNnzixaFbn11fiejVyOvE2eKr25Lq58vI997GNFGOeB+HxvTg7rfK9PHo/JP2e5uysfI3cf5mDNdZPHonJQ5qnbeYp2eeJEhBz+uRxXXXVVUa953Ch3Z+bQPuqoo4pwz3WTfxfyuvy91uTPPS/lUM/3E+X3lqdE56X5zaa5LrIcsOXJFHkSR77viQ5mZ09jY9edptx4qmh5mvK0adNK3/3ud0s1NTXFNOY8dffxxx9vss/m05TnzZtXGjt2bGngwIGlnj17Fl/zFN1nnnmmyftWrFhROv/880v9+vUrtstTjFuaqvrqq6+WPvGJT5SqqqpK1dXVxf//9Kc/tTi19bnnnium/A4YMKC02267lfbdd9/SmWeeWZo1a9Zm62Xx4sXFNOW8fS5Lnip9/PHHl2677bZNtl26dGlx7IkTJ252n/nnmzp1ajFVd5999in16NGjtOeee5ZGjhzZYnny9OVc9r322quo6wMPPLA0fvz4Ul1dXZPPrvlU5jy9uPk04zwV+YwzzmixXM2nKWc//OEPi+Pl6c6N97V8+fJiP3369CnWb2nacfl3oaUlf695OVrbNv/+0fFU5H92dsix6ytfpU+bNq0YOOf/cmsst+DynfpuPIT/MwYD2ylPVMj36wgXaMoYDGynPC4CbEoLBoAQxmAACKEFA0AIAQNA5xjkz48CyTdS5RvnPHYCoGPJd7bkJ5TnP4S3pT8m1+4Bk8Ol/MBAADqmZcuWFU/j2KUCpvzIj1y4/EA/ADqO/KckciOhfC7fpQKm3C2Ww0XAAHRMWzPEYZAfgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAXSdgZsyYkfbff/+0++67p+HDh6eHH354x5cMgK4VMLfeemuaMGFCmjx5cnr00UfT0KFD0+mnn55WrlwZU0IAukbAXH311emiiy5K559/fhoyZEj6wQ9+kPbYY4/04x//OKaEAHT+gNmwYUNasmRJGjVq1P930K1b8frBBx9s8T11dXWptra2yQJA59emgFm9enXauHFj6t+/f5P1+fXy5ctbfM+UKVNSdXV1w1JTU7N9JQagQwifRTZp0qS0Zs2ahmXZsmXRhwRgF9CjLRv369cvde/ePa1YsaLJ+vx6wIABLb6nsrKyWADoWtrUgunZs2c66qij0rx58xrW1dfXF69HjBgRUT4AukILJstTlMeNG5eOPvroNGzYsDR9+vS0bt26YlYZAGxzwJx99tlp1apV6Stf+UoxsH/kkUemu+66a5OBfwC6topSqVRqzwPmacp5Nlke8K+qqmrPQwPQjudwzyIDIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAFg1wiYhQsXpjFjxqSBAwemioqKNGfOnJiSAdC1AmbdunVp6NChacaMGTElAqBT6NHWN4wePbpYAGCHBkxb1dXVFUtZbW1t9CEB6AqD/FOmTEnV1dUNS01NTfQhAegKATNp0qS0Zs2ahmXZsmXRhwSgK3SRVVZWFgsAXYv7YADYNVowr7/+enr22WcbXj///PPpscceS3379k2DBw/e0eUDoKsEzOLFi9Mpp5zS8HrChAnF13HjxqVbbrllx5YOgK4TMCeffHIqlUoxpQGg0zAGA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA0AIAQNACAEDQAgBA8DOD5gpU6akY445JvXp0yftvffe6ayzzkpPP/10TMkA6DoBs2DBgjR+/Pi0aNGidO+996Y33ngjnXbaaWndunVxJQSgQ6oolUqlbX3zqlWripZMDp4TTzxxq95TW1ubqqur05o1a1JVVdW2HhqAnaAt5/Ae23OgfICsb9++rW5TV1dXLI0LB0Dnt82D/PX19enyyy9Pxx13XDrssMM2O26T06681NTUbOshAegKXWSXXHJJmjt3bnrggQfSoEGD2tSCySGjiwyg4wnvIrv00kvTnXfemRYuXLjZcMkqKyuLBYCupU0Bkxs7l112WZo9e3aaP39+OuCAA+JKBkDXCZg8RXnmzJnpjjvuKO6FWb58ebE+N5d69eoVVUYAOvsYTEVFRYvrb7755nTeeedt1T5MUwbouMLGYLbjlhkAuhjPIgMghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAFAwADQcWjBABBCwAAQQsAAEELAABBCwAAQQsAAEKJHzG6ha6uoqNjZRehSSqXSzi4CLdCCASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABIISAASCEgAEghIABQMAA0HFowQAQQsAAEELAABBCwAAQQsAAEELAABBCwAAQQsAAEELAABBCwAAQQsAAEELAABBCwAAQQsAAEELAABBCwAAQQsAAEELAABBCwAAQQsAAEELAABBCwAAQQsAAEELAABBCwAAQQsAAEELAABBCwAAQQsAAsPMD5oYbbkhHHHFEqqqqKpYRI0akuXPnxpQMgK4TMIMGDUpTp05NS5YsSYsXL04jR45MY8eOTU8++WRcCQHokCpKpVJpe3bQt2/fNG3atHThhRdu1fa1tbWpuro6rVmzpmgFQWdUUVGxs4vQpWznaYw2aMs5vEfaRhs3bky33357WrduXdFV1pq6urpiaVw4ADq/Ng/yL126NPXu3TtVVlamiy++OM2ePTsNGTKk1e2nTJlSpF15qamp2d4yA9AZu8g2bNiQXnzxxaJ5NGvWrPSjH/0oLViwoNWQaakFk0NGFxmdmS6y9qWLbNfsItvuMZhRo0alt7/97enGG2/c4YWDjkrAtC8B037acg7f7vtg6uvrm7RQAKDNg/yTJk1Ko0ePToMHD05r165NM2fOTPPnz09333232gRg2wNm5cqV6dxzz02vvPJK0UTKN13mcHnve9/blt0A0AW0KWBuuummuJIA0Kl4FhkAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACEEDAAhBAwAIQQMACF6pHZWKpWKr7W1te19aKCTcj5p/7oun8t3qYBZu3Zt8bWmpqa9Dw10UtXV1Tu7CF3O2rVrt1jvFaWtiaEdqL6+Pr388supT58+qaKiInWk1M6huGzZslRVVbWzi9PpqW913tnVdtBzSo6MHC4DBw5M3bp127VaMLlAgwYNSh1V/kXoSL8MHZ36VuedXVUHPKdsbYvRID8AIQQMACEEzFaqrKxMkydPLr4ST323P3Wuvne0dh/kB6Br0IIBIISAASCEgAEghIABIISAASCEgNkKM2bMSPvvv3/afffd0/Dhw9PDDz8c82mQFi5cmMaMGVM8hiI/SmjOnDlqJdCUKVPSMcccUzy6ae+9905nnXVWevrpp9V5oBtuuCEdccQRDXfwjxgxIs2dO7dT1rmA2YJbb701TZgwobgH5tFHH01Dhw5Np59+elq5cmX7fEJdzLp164o6zqFOvAULFqTx48enRYsWpXvvvTe98cYb6bTTTis+B2IMGjQoTZ06NS1ZsiQtXrw4jRw5Mo0dOzY9+eSTna7K3QezBbnFkq/wrr/++oaHdeYH1F122WVp4sSJ7fEZdVm5BTN79uziqpr2sWrVqqIlk4PnxBNPVO3tpG/fvmnatGnpwgsv7FR1rgWzGRs2bCiuMkaNGvX/CuvWrXj94IMPtsfnA+1qzZo1DSc84m3cuDH96le/KlqMuauss2n3pyl3JKtXry5+Afr3799kfX791FNP7bRyQYTcOr/88svTcccdlw477DCVHGjp0qVFoKxfvz717t27aKkPGTKk09W5gAEKeSzmiSeeSA888IAaCXbIIYekxx57rGgxzpo1K40bN67oluxsISNgNqNfv36pe/fuacWKFU3W59cDBgyI/myg3Vx66aXpzjvvLGbxdeS/19RR9OzZMx100EHF/4866qj0yCOPpGuvvTbdeOONqTMxBrOFX4L84c+bN69JN0J+3Rn7S+l68rNuc7jkLpr77rsvHXDAATu7SF1SfX19qqurS52NFswW5CnKufl69NFHp2HDhqXp06cXA3Lnn39++3xCXczrr7+enn322YbXzz//fNGVkAedBw8evFPL1lm7xWbOnJnuuOOO4l6Y5cuXN/zFwl69eu3s4nVKkyZNSqNHjy5+n/OfHs71P3/+/HT33XenTic/rp/Nu+6660qDBw8u9ezZszRs2LDSokWLVFmQ+++/P//5iE2WcePGqfMALdV1Xm6++Wb1HeSCCy4o7bfffsX5ZK+99iqdeuqppXvuuadT1rf7YAAIYQwGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAQAgaAEAIGgBACBoAU4X8gi9sa8PqjzQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Training mit schrittweiser Visualisierung\n",
    "for ep in range(episodes):\n",
    "    state = env.reset()\n",
    "    done = False\n",
    "    step_count = 0\n",
    "    \n",
    "    while not done:\n",
    "        # Aktion wählen \n",
    "        action = choose_action(state, Q, epsilon)\n",
    "\n",
    "        # Schritt in der Umgebung ausführen \n",
    "        next_state, reward, done = env.step(action)\n",
    "\n",
    "        # Q-Update \n",
    "        x, y = state\n",
    "        nx, ny = next_state\n",
    "        Q[x, y, action] += alpha * (reward + gamma * np.max(Q[nx, ny, :]) - Q[x, y, action])\n",
    "\n",
    "        # Zustand updaten\n",
    "        state = next_state\n",
    "        step_count += 1\n",
    "        \n",
    "        # Visualisierung aktualisieren\n",
    "        clear_output(wait=True)\n",
    "        grid = np.zeros((env.height, env.width))\n",
    "        grid[env.goal[1], env.goal[0]] = 0.5  # Zielzelle\n",
    "        grid[env.agent_pos[1], env.agent_pos[0]] = 1  # Agentenposition\n",
    "        plt.imshow(grid, cmap='gray_r')\n",
    "        plt.title(f'Episode {ep+1}, Schritt {step_count}')\n",
    "        plt.xticks(range(env.width))\n",
    "        plt.yticks(range(env.height))\n",
    "        plt.show()\n",
    "        time.sleep(0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6705fa37-4335-4f70-b88a-04f7e95e1ac1",
   "metadata": {},
   "source": [
    "## Training, mit schrittweiser Grid-Visualisierung und simultanen Q-Table Wahrscheinlichkeiten für mögliche Aktionen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "dc0157b5-d78c-4bf4-802a-98cbacad9725",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T17:30:05.873373Z",
     "iopub.status.busy": "2026-03-24T17:30:05.872973Z",
     "iopub.status.idle": "2026-03-24T17:31:08.233298Z",
     "shell.execute_reply": "2026-03-24T17:31:08.231869Z",
     "shell.execute_reply.started": "2026-03-24T17:30:05.873346Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAGdCAYAAAAv9mXmAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAADm1JREFUeJzt3V+IVGUDx/FnVdwM3QUxFXE3oyAQycB/SCBllnghetddZhIEGohXeZOXKwRhlJTQvytJEiwIysRSCRJTEUpIEIQWzH83u+tCq7jzcgb0feW1dNXf6M58PnBYZ9iZ8/DMMN99zpkZ22q1Wq0AwH025n7fIQAIDAAxVjAARAgMABECA0CEwAAQITAARAgMABHjSoMNDw+Xs2fPlkmTJpW2trZG7x6Ae1B9Nn9gYKDMmDGjjBkz5uEKTBWXrq6uRu8WgPuot7e3zJw58+EKTLVyuT64jo6ORu8egHvQ399fXyRcfy1/qAJz/bBYFReBARid7uQUh5P8AEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgPAwxOY7du3l1mzZpVHHnmkLFq0qBw5cuT+jwyA1grMrl27yqZNm8qWLVvK8ePHy9y5c8vy5cvLhQsXMiMEoDUC895775U33nijrF27tsyePbt8/PHH5dFHHy2fffZZZoQANH9grly5Uo4dO1aWLVv23zsYM6Z++ZdffrnlbYaGhkp/f/9NGwDNb0SBuXTpUrl27VqZNm3aTddXl8+dO3fL2/T09JTOzs4bW1dX172NGIBRIf4uss2bN5e+vr4bW29vb3qXADwExo3kl6dMmVLGjh1bzp8/f9P11eXp06ff8jbt7e31DYDWMqIVzPjx48u8efPK/v37b1w3PDxcv7x48eLE+ABohRVMpXqL8po1a8r8+fPLwoULy7Zt28rg4GD9XWUAcNeBeeWVV8rFixfLO++8Uz+x/+yzz5bvv//+/078A9Da2mq1Wq2RO6zeply9m6w64d/R0dHIXQPQwNdw30UGQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDwMMRmEOHDpWVK1eWGTNmlLa2tvL1119nRgZAawVmcHCwzJ07t2zfvj0zIgCawriR3mDFihX1DQDua2BGamhoqL5d19/fn94lAK1wkr+np6d0dnbe2Lq6utK7BKAVArN58+bS19d3Y+vt7U3vEoBWOETW3t5e3wBoLT4HA8DDsYK5fPlyOX369I3LZ86cKSdOnCiTJ08u3d3d93t8ALRKYI4ePVpeeOGFG5c3bdpU/7lmzZryxRdf3N/RAdA6gXn++edLrVbLjAaApuEcDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAARAgNAhMAAECEwAEQIDAAPPjA9PT1lwYIFZdKkSWXq1Kll9erV5dSpU5mRAdA6gTl48GBZv359OXz4cNm3b1+5evVqefnll8vg4GBuhACMSm21Wq12tze+ePFifSVThWfJkiV3dJv+/v7S2dlZ+vr6SkdHx93uGoAHYCSv4fd0DqbaQWXy5Mn3cjcANKFxd3vD4eHhsnHjxvLcc8+VOXPm/OPvDQ0N1bf/rR8Aze+uVzDVuZjff/+9fPnll7d9Y0C1nLq+dXV13e0uAWj2czAbNmwo33zzTTl06FB54okn/vV3b7WCqSLjHAxAc5+DGdEhsqpFb731VtmzZ085cODAbeNSaW9vr28AtJZxIz0stnPnzvrqpfoszLlz5+rXVzWbMGFCaowANPshsra2tlte//nnn5fXXnvtju7D25QBRq/oITIAuBO+iwyACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgYl7lbaG1tbW0PeggtpVarPeghcAtWMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAEQIDQITAABAhMABECAwAAgPA6GEFA0CEwAAQITAARAgMABECA0CEwAAQITAARAgMABECA0CEwAAQITAARAgMABECA0CEwAAQITAARAgMABECA0CEwAAQITAACAwAo4cVDAARAgNAhMAAECEwAEQIDAARAgNAhMAA8OAD89FHH5VnnnmmdHR01LfFixeX7777LjMyAFonMDNnzixbt24tx44dK0ePHi1Lly4tq1atKidPnsyNEIBRqa1Wq9Xu5Q4mT55c3n333bJu3bo7+v3+/v7S2dlZ+vr66qsgaEZtbW0Peggt5R5fxhiBkbyGjyt36dq1a+Wrr74qg4OD9UNl/2RoaKi+/e/gAGh+Iz7J/9tvv5WJEyeW9vb28uabb5Y9e/aU2bNn/+Pv9/T01Gt3fevq6rrXMQPQjIfIrly5Uv7888/68mj37t3lk08+KQcPHvzHyNxqBVNFxiEymplDZI3lENnDeYjsns/BLFu2rDz55JNlx44d931wMFoJTGMJTOOM5DX8nj8HMzw8fNMKBQBGfJJ/8+bNZcWKFaW7u7sMDAyUnTt3lgMHDpS9e/eaTQDuPjAXLlwor776avnrr7/qS6TqQ5dVXF566aWR3A0ALWBEgfn0009zIwGgqfguMgAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaACIEBIEJgAIgQGAAiBAaAiHGlwWq1Wv1nf39/o3cNNCmvJ42f6+uv5Q9VYAYGBuo/u7q6Gr1roEl1dnY+6CG0nIGBgdvOe1vtTjJ0Hw0PD5ezZ8+WSZMmlba2tjKaql1Fsbe3t3R0dDzo4TQ9823Om13/KH1NqZJRxWXGjBllzJgxD9cKphrQzJkzG73b+6Z6IoymJ8NoZ77NebPrGIWvKXe6YnSSH4AIgQEgQmDuUHt7e9myZUv9J3nmu/HMufm+3xp+kh+A1mAFA0CEwAAQITAARAgMABECcwe2b99eZs2aVR555JGyaNGicuTIkcyjQTl06FBZuXJl/VPC1Tc9fP3112YlqKenpyxYsKD+zRpTp04tq1evLqdOnTLnQR999FF55plnbnzAcvHixeW7775ryjkXmNvYtWtX2bRpU/0tysePHy9z584ty5cvLxcuXGjMI9RiBgcH63NcRZ28gwcPlvXr15fDhw+Xffv2latXr5aXX365/jiQMXPmzLJ169Zy7NixcvTo0bJ06dKyatWqcvLkyaabcm9Tvo1qxVL9hffhhx/e+C616vuD3nrrrfL222834jFqWdUKZs+ePfW/qmmMixcv1lcyVXiWLFli2htk8uTJ5d133y3r1q1rqjm3gvkXV65cqf+VsWzZsv9O2Jgx9cu//PJLIx4faKi+vr4bL3jkXbt2rXz55Zf1FWN1qKzZNPzLLkeTS5cu1Z8A06ZNu+n66vIff/zxwMYFCdXqfOPGjeW5554rc+bMMclBv/32Wz0of//9d5k4cWJ9pT579uymm3OBAeqqczG///57+fnnn81I2NNPP11OnDhRXzHu3r27rFmzpn5YstkiIzD/YsqUKWXs2LHl/PnzN11fXZ4+fXr6sYGG2bBhQ/n222/r7+Ibzf+dxmgxfvz48tRTT9X/PW/evPLrr7+W999/v+zYsaM0E+dgbvMkqB78/fv333QYobrcjMdLaT3VVxFWcakO0fz444/liSeeeNBDaknDw8NlaGioNBsrmNuo3qJcLV/nz59fFi5cWLZt21Y/Ibd27drGPEIt5vLly+X06dM3Lp85c6Z+KKE66dzd3f1Ax9ash8V27txZvvnmm/pnYc6dO3fjP5SaMGHCgx5eU9q8eXNZsWJF/flc/c+Q1fwfOHCg7N27tzSd6tuU+XcffPBBrbu7uzZ+/PjawoULa4cPHzZlIT/99FP17d7/t61Zs8acB9xqrqvt888/N98hr7/+eu3xxx+vv5489thjtRdffLH2ww8/NOV8+xwMABHOwQAQITAARAgMABECA0CEwAAQITAARAgMABECA0CEwAAQITAARAgMABECA0BJ+A8ydFLt2w/cDwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 5 completed.\n"
     ]
    }
   ],
   "source": [
    "# Action names for visualization (4 actions: 0: Up, 1: Right, 2: Down, 3: Left)\n",
    "action_names = [\"Up\", \"Right\", \"Down\", \"Left\"]\n",
    "\n",
    "def get_action_probabilities(state, Q, eps):\n",
    "    \"\"\"\n",
    "    Compute the epsilon-greedy probabilities for actions at the given state.\n",
    "    \n",
    "    Each action gets a base probability of eps/4, and the best action (argmax of Q-values)\n",
    "    gets an additional probability mass of (1 - eps).\n",
    "\n",
    "    Args:\n",
    "        state (tuple): The current state as (x, y).\n",
    "        Q (np.ndarray): Q-table with shape (width, height, 4).\n",
    "        eps (float): Epsilon value (exploration rate).\n",
    "        \n",
    "    Returns:\n",
    "        numpy.ndarray: A vector of probabilities for the 4 actions.\n",
    "    \"\"\"\n",
    "    x, y = state\n",
    "    q_vals = Q[x, y, :]\n",
    "    best_action = np.argmax(q_vals)\n",
    "    probs = np.ones(4) * (eps / 4.0)\n",
    "    probs[best_action] += (1.0 - eps)\n",
    "    return probs\n",
    "\n",
    "# Training mit schrittweiser Visualisierung, inklusive Grid und der aktuellen Options-Wahrscheinlichkeiten und der gewählten Aktion \n",
    "for ep in range(episodes):\n",
    "    state = env.reset()\n",
    "    done = False\n",
    "    step_count = 0\n",
    "\n",
    "    while not done:\n",
    "        clear_output(wait=True)\n",
    "        # Erstelle eine neue Figur für diesen Schritt\n",
    "        fig, ax = plt.subplots(figsize=(6, 4))\n",
    "        \n",
    "        # Erstelle neues Grid-Display: 0 = leer, 0.5 = Ziel, 1 = Agent\n",
    "        grid = np.zeros((env.height, env.width))\n",
    "        grid[env.goal[1], env.goal[0]] = 0.5  # mark goal\n",
    "        grid[env.agent_pos[1], env.agent_pos[0]] = 1  # Agents aktuelle Position\n",
    "        \n",
    "        # Plotte das Grid (Gitter)\n",
    "        ax.imshow(grid, cmap='gray_r', vmin=0, vmax=1)\n",
    "        ax.set_xticks(range(env.width))\n",
    "        ax.set_yticks(range(env.height))\n",
    "        ax.set_title(f\"Episode {ep+1}, Step {step_count+1}\")\n",
    "        \n",
    "        # Berechne epsilon-Greedy- Aktions-Wahrscheinlichkeiten für den aktuellen Zustand\n",
    "        probs = get_action_probabilities(state, Q, epsilon)\n",
    "        # Wähle eine Aktion anhand der bestehenden Epsilon-Greedy-Funktion\n",
    "        action = choose_action(state, Q, epsilon)\n",
    "        \n",
    "        # Zeige die berechneten Wahrscheinlichkeiten und die gewählte aktion nebem dem Grid \n",
    "        text_str = (\n",
    "            f\"Action probabilities:\\n\"\n",
    "            f\"Up: {probs[0]:.2f}\\n\"\n",
    "            f\"Right: {probs[1]:.2f}\\n\"\n",
    "            f\"Down: {probs[2]:.2f}\\n\"\n",
    "            f\"Left: {probs[3]:.2f}\\n\\n\"\n",
    "            f\"Chosen action: {action} ({action_names[action]})\"\n",
    "        )\n",
    "        # Text rechts vom Raster platzieren, ggf. Koordination anpassen\n",
    "        ax.text(env.width + 0.5, env.height/2, text_str, fontsize=10, va='center')\n",
    "        \n",
    "        plt.tight_layout()\n",
    "        plt.show()\n",
    "        \n",
    "        # Führe die Aktion durch und aktualisiere den Q-Wert\n",
    "        next_state, reward, done = env.step(action)\n",
    "        nx, ny = next_state\n",
    "        Q[state[0], state[1], action] += alpha * (reward + gamma * np.max(Q[nx, ny, :]) - Q[state[0], state[1], action])\n",
    "        state = next_state\n",
    "        step_count += 1\n",
    "        \n",
    "        time.sleep(1)  # Pause, damit der Schritt sichtbar ist\n",
    "    \n",
    "    # End-of-eEpisode Message\n",
    "    clear_output(wait=True)\n",
    "    env.render()\n",
    "    print(f\"Episode {ep+1} completed.\")\n",
    "    time.sleep(1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73b7e6b3",
   "metadata": {},
   "source": [
    "## Trainiertes Modell: Darstellung des optimalen Pfads\n",
    "\n",
    "Nach dem Training wird der Agent den erlernten (nahezu) optimalen Pfad zum Ausgang demonstrieren."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "40b97f48",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAGzCAYAAAASUAGgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAmFElEQVR4nO3deXRU5f3H8c8kMAFCMmVJgGyEgK0iEkpYDmrYmsYCLhQxKCJooWIJFMsprdRfi56qgdIqLogWFJRFW2ki0kJxhYgFURBBqlY0FGRLWExC0AAzz++P6YxMFkjCPBkS3q9z5ph55s6933nueD/33ufewWGMMQIAIMjCQl0AAKBxImAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAamcWLF8vhcGj37t1Bm+d9990nh8MRtPk1FHPmzFFKSorCw8PVo0ePoMwzOTlZt99+e1DmBVzoCBjLdu7cqTFjxig+Pl4RERGKi4vTrbfeqp07d57XfB966CG9/PLLwSkyRJ588kktXry4XpeZnJwsh8Phf8TGxio9PV15eXkB07366qv61a9+pauuukqLFi3SQw89VK91NiSrV6+Ww+FQXFycPB5PqMvBhcTAmr/97W/G6XSa9u3bm3vvvdcsXLjQ/N///Z/p0KGDcTqdJjc3t87zjoyMNOPGjavUfvr0afP1118bj8dzHpUHOnXqlPn666+DNj+fyy+/3AwYMCDo8z2bjh07mh49epglS5aYJUuWmNmzZ5uUlBQjycyfP98/3a9//WsTFhZmysvLg778qtZbQzZ69GiTnJxsJJnXXnst1OXgAtIk1AHXWH3++ee67bbblJKSovz8fMXExPhfmzp1qtLT03Xbbbdp+/btSklJCdpyw8PDFR4eHrT5SVKTJk3UpEnwvionTpxQixYtgja/M50+fVoej0dOp7PaaeLj4zVmzBj/87Fjx6pLly565JFHdNddd0mSCgsL1bx587POB1JZWZlWrlypnJwcLVq0SMuWLVNGRkaoy8KFItQJ11hNnDjRSDL5+flVvr5+/XojyUycONHfNnPmTCPJfPzxx+amm24yUVFRpnXr1ubnP/95wBGEpEoP317xokWLjCRTUFDgn75jx45m2LBh5q233jJpaWmmWbNmplu3buatt94yxniPtLp162YiIiJMz549zdatWwNq9dVV0ZIlS0zPnj1Ns2bNTKtWrcyoUaPMnj17AqYZMGCAufzyy837779v0tPTTfPmzc3UqVNNx44dK32GM49mjh07ZqZOnWoSEhKM0+k0nTt3NrNmzTJut9s/TUFBgZFk5syZYx555BGTkpJiwsLCzAcffFDtevH1RUW9evUyTZs2rbZ/Fy1aZIwx5tlnnzWDBg0yMTExxul0mssuu8w8+eSTlebn8XjM73//exMfH2+aN29uBg4caD766KMaHcEMGDCgyhrOrKO6dWJj/Z/NkiVLTFhYmDlw4ICZPXu2iY6OrnS061tPvtrPJMnMnDnT/7ykpMT//XA6nSYmJsZkZGSYLVu2BLzviSeeMJ06dTLNmjUzvXv3Nvn5+WbAgAEB36Hy8nLz29/+1vTs2dNER0ebFi1amKuvvtq8+eabVdY3Z84c8/TTT5uUlBTjdDpNr169zObNmyvV/Ne//tVcdtllJiIiwlx++eUmNzfXjBs3znTs2LHG/Xax4AjGklWrVik5OVnp6elVvt6/f38lJyfrH//4R6XXsrKylJycrJycHG3atEmPPfaYjh07pueff16StGTJEk2YMEF9+vTRnXfeKUnq3LnzWevZtWuXRo8erYkTJ2rMmDH64x//qOuuu05PPfWUfvOb32jSpEmSpJycHGVlZenTTz9VWFj1Q3QPPvigfvvb3yorK0sTJkxQUVGRHn/8cfXv318ffPCBvvOd7/inPXLkiIYMGaKbb75ZY8aMUbt27TRw4EBNmTJFLVu21L333itJateunSTvEc6AAQO0b98+TZw4UUlJSfrXv/6lGTNm6MCBA5o7d25ALYsWLdI333yjO++8UxEREWrduvVZ+6KiU6dOae/evWrTpo2/f//85z9r8+bNWrhwoSTpyiuvlCTNnz9fl19+ua6//no1adJEq1at0qRJk+TxeJSdne2f5+9+9zs98MADGjp0qIYOHaqtW7cqMzNTJ0+ePGc99957ryZMmBDQtnTpUq1du1axsbG1+mw+wV7/PsuWLdOgQYPUvn173Xzzzbrnnnu0atUq3XTTTXWq86677tKKFSs0efJkde3aVUeOHNGGDRv08ccfq2fPnpK862Dy5MlKT0/XL37xC+3evVvDhw9Xq1atlJCQ4J9XSUmJFi5cqFtuuUU//elPVVpaqmeeeUbXXHONNm/eXOnCjeXLl6u0tFQTJ06Uw+HQH/7wB40YMUJffPGFmjZtKkn6xz/+oVGjRumKK65QTk6Ojh07pvHjxys+Pr5On7fRC3XCNUZfffWVkWRuuOGGs053/fXXG0mmpKTEGPPtXun1118fMN2kSZOMJPPhhx/626obg6luD1aS+de//uVvW7t2rZFkmjdvbv773//6259++mkjyb93e2ZdPrt37zbh4eHmwQcfDFj2jh07TJMmTQLafXvjTz31VKVaqxuD+f3vf28iIyPNf/7zn4D2e+65x4SHh/uPknx7ntHR0aawsLDSfKrSsWNHk5mZaYqKikxRUZH58MMPzc0332wkmSlTpvinGzdunImMjKz0/hMnTlRqu+aaa0xKSor/eWFhoXE6nWbYsGEBY2G/+c1vAo42a+qdd94xTZs2NT/5yU/8bbU9gjmf9V+dQ4cOmSZNmpgFCxb426688spK3/vaHMG4XC6TnZ1d7TLLy8tNmzZtTO/evc2pU6f87YsXL650FHz69OlKY2jHjh0z7dq1C+hLX31t2rQxR48e9bevXLnSSDKrVq3yt11xxRUmISHBlJaW+tvWrVtnJHEEUwWuIrOgtLRUkhQVFXXW6Xyvl5SUBLSfuScsSVOmTJHkvVqnrrp27ap+/fr5n/ft21eSNHjwYCUlJVVq/+KLL6qdV25urjwej7KysnT48GH/o3379rrkkkv01ltvBUwfERGhO+64o8a1vvTSS0pPT1erVq0C5p+RkSG32638/PyA6W+88caAMa5zefXVVxUTE6OYmBilpqbqpZde0m233abZs2ef873Nmzf3/11cXKzDhw9rwIAB+uKLL1RcXCxJev3113Xy5ElNmTIl4PLuu+++u8Y1+hw8eFAjR45Ujx499OSTT9b6/T7BXP8+L774osLCwnTjjTf622655RatWbNGx44dq1Od3/nOd/Tuu+9q//79Vb7+/vvv68iRI/rpT38aMC546623qlWrVgHThoeH+8fQPB6Pjh49qtOnT6tXr17aunVrpXmPGjUqYB6+sw++vti/f7927NihsWPHqmXLlv7pBgwYoCuuuKJOn7ex4xSZBb7g8AVNdaoLoksuuSTgeefOnRUWFnZe97acuRGRJJfLJUlKTEyssv1sG4jPPvtMxphKdfr4Tif4xMfH12qw/LPPPtP27durDY3CwsKA5506darxvCXvRvSBBx6Qw+FQixYtdNlllwWc0jubd955RzNnztTGjRt14sSJgNeKi4vlcrn03//+V1Ll9RgTE1NpI3g2p0+fVlZWltxut3JzcxUREVHj91YUzPXvs3TpUvXp00dHjhzRkSNHJEnf//73dfLkSb300kv+07e18Yc//EHjxo1TYmKi0tLSNHToUI0dO9Z/IYyvb7t06RLwviZNmig5ObnS/J577jn96U9/0ieffKJTp07526v6zlTsI9+68vVFdcv2tVUVWhc7AsYCl8ulDh06aPv27Wedbvv27YqPj1d0dPRZpwvGTY7VXVlWXbs5y7+k7fF45HA4tGbNmirff+benRS4118THo9HP/zhD/WrX/2qyte/+93vntf827ZtW6crnT7//HP94Ac/0KWXXqqHH35YiYmJcjqdWr16tR555JGg3wMyffp0bdy4Ua+//nrA2IJU/XfC7XZX2R7M9S95dwLee+89SZWDVPKOzfgCpja1ZmVl+e9LevXVVzVnzhzNnj1bubm5GjJkyFlrqmjp0qW6/fbbNXz4cE2fPl2xsbEKDw9XTk6OPv/880rT17UvUD0CxpJrr71WCxYs0IYNG3T11VdXev3tt9/W7t27NXHixEqvffbZZwF7WLt27ZLH4wnYQwvlnfWdO3eWMUadOnWqtLGvjeo+Q+fOnXX8+PEL7nLXVatWqby8XK+88krA3m7FU4IdO3aU5F2PZ16CXlRUVONTRy+++KLmzp2ruXPnasCAAZVe9+1df/XVVwFHX769bNuWLVumpk2basmSJZU2zBs2bNBjjz2mPXv2KCkpKaDWM1VXa4cOHTRp0iRNmjRJhYWF6tmzpx588EENGTLE37e7du3SoEGD/O85ffq0du/ere7du/vbVqxYoZSUFOXm5gZ812bOnFmnz3zmsiuqqg3cyW/N9OnT1bx5c02cONF/+sDn6NGjuuuuu9SiRQtNnz690nvnzZsX8Pzxxx+XpIA9uMjIyEr/w9aXESNGKDw8XPfff3+lvTtjTKXPW53qPkNWVpY2btyotWvXVnrtq6++0unTp+tU9/nybUjP/MzFxcVatGhRwHQZGRlq2rSpHn/88YBpK179Vp2PPvpIEyZM0JgxYzR16tQqp/FdNXjmeFRZWZmee+65Gi3jfC1btkzp6ekaNWqURo4cGfDwfadfeOEFSVJ0dLTatm1baeys4piS2+32j2P5xMbGKi4uTuXl5ZKkXr16qU2bNlqwYEHA92DZsmWVwruq9fXuu+9q48aNdfrMcXFx6tatm55//nkdP37c375+/Xrt2LGjTvNs7DiCseSSSy7Rc889p1tvvVVXXHGFxo8fr06dOmn37t165plndPjwYb3wwgtVXl5cUFCg66+/Xj/60Y+0ceNGLV26VKNHj1Zqaqp/mrS0NL3++ut6+OGHFRcXp06dOvkHaG3r3LmzHnjgAc2YMcN/iWhUVJQKCgqUl5enO++8U7/85S/POZ+0tDTNnz9fDzzwgLp06aLY2FgNHjxY06dP1yuvvKJrr71Wt99+u9LS0lRWVqYdO3ZoxYoV2r17t9q2bVsPnzRQZmamnE6nrrvuOk2cOFHHjx/XggULFBsbqwMHDvini4mJ0S9/+Uvl5OTo2muv1dChQ/XBBx9ozZo1Narbd0FE//79tXTp0oDXrrzySqWkpCgzM1NJSUkaP368pk+frvDwcD377LOKiYnRnj17gvvBK3j33Xe1a9cuTZ48ucrX4+Pj1bNnTy1btky//vWvJUkTJkzQrFmzNGHCBPXq1Uv5+fn6z3/+E/C+0tJSJSQkaOTIkUpNTVXLli31+uuv67333tOf/vQnSZLT6dR9992nKVOmaPDgwcrKytLu3bu1ePFide7cOeBI5dprr1Vubq5+/OMfa9iwYSooKNBTTz2lrl27BgREbTz00EO64YYbdNVVV+mOO+7QsWPH9MQTT6hbt251nmejFqrL1y4W27dvN7fccovp0KGDadq0qWnfvr255ZZbzI4dOypN67v09N///rcZOXKkiYqKMq1atTKTJ0+udPPaJ598Yvr372+aN29e4xstK5JU6ZLQM286q1hXRX/729/M1VdfbSIjI01kZKS59NJLTXZ2tvn000/90/hutKzKwYMHzbBhw0xUVFSlS0xLS0vNjBkzTJcuXYzT6TRt27Y1V155pfnjH/9oTp48WW2t51JdX1RU3WXKr7zyiunevbtp1qyZSU5ONrNnzzbPPvtspT53u93m/vvvNx06dKj1jZZV3YTqe5x5qe+WLVtM3759jdPpNElJSebhhx+2sv4rmjJlipFkPv/882qnue+++wIurT9x4oQZP368cblcJioqymRlZZnCwsKAy5TLy8vN9OnTTWpqqomKijKRkZEmNTW1yhtZH3vsMdOxY0cTERFh+vTpY9555x2TlpZmfvSjH/mn8Xg85qGHHvJP9/3vf9/8/e9/r3RT5Nk+85n1+bz44ovm0ksvNREREaZbt27mlVdeMTfeeKO59NJLq+2Pi5XDGEawLhT33Xef7r//fhUVFYVkDx1oqDwej2JiYjRixAgtWLCg3pffo0cPxcTE6LXXXqv3ZV/IGIMB0KB88803lcb+nn/+eR09elQDBw60uuxTp05VGgNct26dPvzwQ+vLbogYgwHQoGzatEm/+MUvdNNNN6lNmzbaunWrnnnmGXXr1q3OP1FTU/v27VNGRobGjBmjuLg4ffLJJ3rqqafUvn17/w+l4lsEDIAGJTk5WYmJiXrsscd09OhRtW7dWmPHjtWsWbOs//p1q1atlJaWpoULF6qoqEiRkZEaNmyYZs2a5f8tO3yLMRgAgBWMwQAArCBgAABW1PsYjMfj0f79+xUVFRXSnzsBANSeMUalpaWKi4s7578ZVO8Bs3///kq/4AoAaFj27t1b6UdYK6r3gPH9NP3evXvP+SvCAIALS0lJiRITE8/5711JIQgY32mx6OhoAgYAGqiaDHEwyA8AsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBVNQl1AQ+B2S2+/LR04IHXoIKWnS+Hhoa4KAC5sdTqCmTdvnpKTk9WsWTP17dtXmzdvDnZdF4zcXCk5WRo0SBo92vvf5GRvOwCgerUOmL/85S+aNm2aZs6cqa1btyo1NVXXXHONCgsLbdQXUrm50siR0pdfBrbv2+dtJ2QAoHoOY4ypzRv69u2r3r1764knnpAkeTweJSYmasqUKbrnnnvO+f6SkhK5XC4VFxcrOjq6blXXA7fbe6RSMVx8HA4pIUEqKOB0GYCLR2224bU6gjl58qS2bNmijIyMb2cQFqaMjAxt3LixyveUl5erpKQk4NEQvP129eEiScZIe/d6pwMAVFargDl8+LDcbrfatWsX0N6uXTsdPHiwyvfk5OTI5XL5H4mJiXWvth4dOBDc6QDgYmP9MuUZM2aouLjY/9i7d6/tRQZFhw7BnQ4ALja1uky5bdu2Cg8P16FDhwLaDx06pPbt21f5noiICEVERNS9whBJT/eOsezb5z0dVpFvDCY9vf5rA4CGoFZHME6nU2lpaXrjjTf8bR6PR2+88Yb69esX9OJCKTxcevRR798OR+Brvudz5zLADwDVqfUpsmnTpmnBggV67rnn9PHHH+tnP/uZysrKdMcdd9ioL6RGjJBWrJDi4gLbExK87SNGhKYuAGgIan0n/6hRo1RUVKTf/e53OnjwoHr06KF//vOflQb+G4sRI6SMDMnl8j5fvVrKzOTIBQDOpdb3wZyvhnIfzJnKyqSWLb1/Hz8uRUaGth4ACBVr98EAAFBTBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwiYGnC7v/07Pz/wOYLP7ZbWrZNeeMH7X/obaJhqHTD5+fm67rrrFBcXJ4fDoZdfftlCWReO3Fypa9dvnw8dKiUne9sRfLm53v4dNEgaPdr7X/obaJhqHTBlZWVKTU3VvHnzbNRzQcnNlUaOlPbtC2zft8/bzkYvuHz9/eWXge30N9AwOYwxps5vdjiUl5en4cOH1/g9JSUlcrlcKi4uVnR0dF0XbZ3b7d1zrrix83E4pPh4aedOKTy8XktrlNxu75FixTD3cTikhASpoID+BkKpNtvwJraLKS8vV3l5eUBxDcHbb1cfLpJkjPd1l6v+arqYGSPt3etdLwMHhroaADVhfZA/JydHLpfL/0hMTLS9yKA4cCDUFaAqrBeg4bB+BDNjxgxNmzbN/7ykpKRBhEyHDjWbbvVqqX9/u7VcDPLzvRdQnEtN1wuA0LMeMBEREYqIiLC9mKBLT/ee89+3z3t6piLfmEBmJmMCwZCZWbP+Tk+v/9oA1A33wVQjPFx69FHv3w5H4Gu+53PnEi7BQn8DjU+tA+b48ePatm2btm3bJkkqKCjQtm3btGfPnmDXFnIjRkgrVnivFjtTQoK3fcSI0NTVWPn6Oy4usJ3+BhqmWl+mvG7dOg0aNKhS+7hx47R48eJzvr+hXKZ8Jrfbe/XSgQPeMYD0dPakbSop+fbqvNWrOQ0JXEhqsw0/r/tg6qIhBgzqV1mZ1LKl9+/jx6XIyNDWA+BbtdmGMwYDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgAEAWEHAAACsIGAAAFYQMAAAKwgYAIAVBAwAwAoCBgBgBQEDALCCgMEFx+3+9u/8/MDnsMPtltatk154wftf+hzBUKuAycnJUe/evRUVFaXY2FgNHz5cn376qa3acBHKzZW6dv32+dChUnKytx125OZ6+3jQIGn0aO9/6XMEQ60CZv369crOztamTZv02muv6dSpU8rMzFRZWZmt+nARyc2VRo6U9u0LbN+3z9vOBi/4fH3+5ZeB7fQ5gsFhjDF1fXNRUZFiY2O1fv169e/fv0bvKSkpkcvlUnFxsaKjo+u6aDQybrd3r7nihs7H4ZDi46WdO6Xw8HotrdFyu71HixUD3cfhkBISpIIC+hzfqs02vMn5LKi4uFiS1Lp162qnKS8vV3l5eUBxQEVvv119uEiSMd7XXa76q+liZ4y0d6933QwcGOpq0BDVeZDf4/Ho7rvv1lVXXaVu3bpVO11OTo5cLpf/kZiYWNdFohE7cCDUFaA6rBvUVZ2PYLKzs/XRRx9pw4YNZ51uxowZmjZtmv95SUkJIYNKOnSo2XSrV0s1PBuLc8jP915EcS41XTdARXUag5k8ebJWrlyp/Px8derUqVbvZQwGVfGNwezb5z01UxHjAcFHn6MuarMNr9UpMmOMJk+erLy8PL355pu1DhegOuHh0qOPev92OAJf8z2fO5cNXTDR57CtVgGTnZ2tpUuXavny5YqKitLBgwd18OBBff3117bqw0VkxAhpxQrv1WJnSkjwto8YEZq6GjNfn8fFBbbT5wiGWp0ic1TczfmfRYsW6fbbb6/RPDhFhnNxu71XLh044D3/n57OXrRtJSXfXqG3erWUmUmfo2rWLlM+j1tmgBoLD+ey2Pp2Zpj070+4IDj4LTIAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKxoEuoCgMbI4XCEuoRaaiGpTJLUsmWkpBMhraa2jDGhLgFV4AgGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQNAgZuCdLFpQDDU6ls0f/58de/eXdHR0YqOjla/fv20Zs0aW7UBqBc/lvTxGc//KWn3/9qBuqtVwCQkJGjWrFnasmWL3n//fQ0ePFg33HCDdu7caas+AFb9WNIKSXEV2uP/107IoO4cxhhzPjNo3bq15syZo/Hjx9do+pKSErlcLhUXFys6Ovp8Fg1csBwOR6hLqIEweY9U4lX1vqZH0peSOv3v7wvXeW7GUAu12YY3qetC3G63XnrpJZWVlalfv37VTldeXq7y8vKA4gBcCNIlJZ7l9TBJSf+bbn29VITGpdYjeTt27FDLli0VERGhu+66S3l5eeratWu10+fk5MjlcvkfiYln+0IDqD8dgjwdEKjWp8hOnjypPXv2qLi4WCtWrNDChQu1fv36akOmqiOYxMRETpGhUWsYp8gGSFpXg+kG6kI/guEUWf2pzSmy8x6DycjIUOfOnfX0008HvTigoWoYAcMYDGqvNtvw877Y3ePxBByhAGgoPJKmnvF3xdck6e4qXgNqplaD/DNmzNCQIUOUlJSk0tJSLV++XOvWrdPatWtt1QfAqjxJIyU9qsAB/y/lDZe8ENSExqJWAVNYWKixY8fqwIEDcrlc6t69u9auXasf/vCHtuoDYF2epJXyXi3WQdIBSW+LIxecr/Meg6ktxmBwMWgYYzCNB2Mw9adex2AAAKgKAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAArmtT3Ao0xkqSSkpL6XjSARortSf3x9bVvW3429R4wpaWlkqTExMT6XjSARsrlcoW6hItOaWnpOfvdYWoSQ0Hk8Xi0f/9+RUVFyeFw1Oeiz0tJSYkSExO1d+9eRUdHh7qcRo/+rn/0ef1qqP1tjFFpaani4uIUFnb2UZZ6P4IJCwtTQkJCfS82aKKjoxvUl6Gho7/rH31evxpif9f0iJFBfgCAFQQMAMAKAqaGIiIiNHPmTEVERIS6lIsC/V3/6PP6dTH0d70P8gMALg4cwQAArCBgAABWEDAAACsIGACAFQQMAMAKAqYG5s2bp+TkZDVr1kx9+/bV5s2bQ11So5Wfn6/rrrtOcXFxcjgcevnll0NdUqOWk5Oj3r17KyoqSrGxsRo+fLg+/fTTUJfVqM2fP1/du3f338Hfr18/rVmzJtRlWUHAnMNf/vIXTZs2TTNnztTWrVuVmpqqa665RoWFhaEurVEqKytTamqq5s2bF+pSLgrr169Xdna2Nm3apNdee02nTp1SZmamysrKQl1ao5WQkKBZs2Zpy5Ytev/99zV48GDdcMMN2rlzZ6hLCzrugzmHvn37qnfv3nriiSckeX+sMzExUVOmTNE999wT4uoaN4fDoby8PA0fPjzUpVw0ioqKFBsbq/Xr16t///6hLuei0bp1a82ZM0fjx48PdSlBxRHMWZw8eVJbtmxRRkaGvy0sLEwZGRnauHFjCCsD7CguLpbk3eDBPrfbrRdffFFlZWXq169fqMsJunr/NeWG5PDhw3K73WrXrl1Ae7t27fTJJ5+EqCrADo/Ho7vvvltXXXWVunXrFupyGrUdO3aoX79++uabb9SyZUvl5eWpa9euoS4r6AgYAJKk7OxsffTRR9qwYUOoS2n0vve972nbtm0qLi7WihUrNG7cOK1fv77RhQwBcxZt27ZVeHi4Dh06FNB+6NAhtW/fPkRVAcE3efJk/f3vf1d+fn6D/veaGgqn06kuXbpIktLS0vTee+/p0Ucf1dNPPx3iyoKLMZizcDqdSktL0xtvvOFv83g8euONNxrl+VJcfIwxmjx5svLy8vTmm2+qU6dOoS7pouTxeFReXh7qMoKOI5hzmDZtmsaNG6devXqpT58+mjt3rsrKynTHHXeEurRG6fjx49q1a5f/eUFBgbZt26bWrVsrKSkphJU1TtnZ2Vq+fLlWrlypqKgoHTx4UJL3Xyxs3rx5iKtrnGbMmKEhQ4YoKSlJpaWlWr58udatW6e1a9eGurTgMzinxx9/3CQlJRmn02n69OljNm3aFOqSGq233nrLSKr0GDduXKhLa5Sq6mtJZtGiRaEurdH6yU9+Yjp27GicTqeJiYkxP/jBD8yrr74a6rKs4D4YAIAVjMEAAKwgYAAAVhAwAAArCBgAgBUEDADACgIGAGAFAQMAsIKAAQBYQcAAAKwgYAAAVhAwAAAr/h/Z5BCVY8FWswAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Reset und initialisiere Pfadspeicher\n",
    "state = env.reset()\n",
    "done = False\n",
    "path = [state]\n",
    "\n",
    "while not done and len(path) < 100:\n",
    "    x, y = state\n",
    "    action = np.argmax(Q[x, y, :])\n",
    "    state, _, done = env.step(action)\n",
    "    path.append(state)\n",
    "\n",
    "# Visualisierung des gefundenen Pfads\n",
    "grid = np.zeros((env.height, env.width))\n",
    "grid[env.goal[1], env.goal[0]] = 0.5  # Zielzelle\n",
    "plt.imshow(grid, cmap='gray_r')\n",
    "x_coords = [pos[0] for pos in path]\n",
    "y_coords = [pos[1] for pos in path]\n",
    "plt.plot(x_coords, y_coords, marker='o', color='blue')\n",
    "plt.title('Optimierter Pfad zum Ausgang')\n",
    "plt.xticks(range(env.width))\n",
    "plt.yticks(range(env.height))\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.0"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}