{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "11bb4456-5dc2-440c-9320-4af43e032aeb",
   "metadata": {},
   "source": [
    "# Hands-on Beispiel LLM (2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47a74c0f-6038-4871-9038-df59a4be02a1",
   "metadata": {},
   "source": [
    "### 2. Fine-tuning - Anpassung an juristische Fachtexte\n",
    "##### --- Juristische Fragen an ein fine-tuned Modell (Lokale LLM)\n",
    "\n",
    "In diesem Abschnitt fine-tunen wir das Modell `dbmdz/german-gpt2` und stellen ihm die gleichen zwei juristischen Fragen zum AI Act wie im Baseline-Notebook.\n",
    "\n",
    "Ziel ist es, dass das feingetunte Modell (llm-2) nun fundiertere und korrekte Antworten liefert.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8f0f45fe-361d-4965-a821-e69419fced67",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:06.354720Z",
     "iopub.status.busy": "2026-03-24T18:13:06.354568Z",
     "iopub.status.idle": "2026-03-24T18:13:06.359355Z",
     "shell.execute_reply": "2026-03-24T18:13:06.357651Z",
     "shell.execute_reply.started": "2026-03-24T18:13:06.354702Z"
    }
   },
   "outputs": [],
   "source": [
    "# falls noch nicht installiert \n",
    "\n",
    "import sys\n",
    "# !{sys.executable} -m pip install transformers datasets\n",
    "# !{sys.executable} -m pip install 'accelerate>=1.10.0'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7e552b09-89c0-4168-8e6c-c59babe09ea5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:06.360112Z",
     "iopub.status.busy": "2026-03-24T18:13:06.359959Z",
     "iopub.status.idle": "2026-03-24T18:13:10.070226Z",
     "shell.execute_reply": "2026-03-24T18:13:10.069715Z",
     "shell.execute_reply.started": "2026-03-24T18:13:06.360096Z"
    }
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM, Trainer, TrainingArguments, DataCollatorForLanguageModeling\n",
    "from datasets import Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "6b807925-978d-4d97-8765-ad2beaf99097",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:10.070728Z",
     "iopub.status.busy": "2026-03-24T18:13:10.070543Z",
     "iopub.status.idle": "2026-03-24T18:13:12.602873Z",
     "shell.execute_reply": "2026-03-24T18:13:12.602349Z",
     "shell.execute_reply.started": "2026-03-24T18:13:10.070719Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "00edbe7a58d546d586443954281f376b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading weights:   0%|          | 0/148 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1mGPT2LMHeadModel LOAD REPORT\u001b[0m from: dbmdz/german-gpt2\n",
      "Key                                     | Status     |  | \n",
      "----------------------------------------+------------+--+-\n",
      "transformer.h.{0...11}.attn.masked_bias | UNEXPECTED |  | \n",
      "\n",
      "\u001b[3mNotes:\n",
      "- UNEXPECTED\u001b[3m\t:can be ignored when loading from different task/architecture; not ok if you expect identical arch.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# Lade Modell und Tokenizer (das Basis-Modell bleibt identisch)\n",
    "model_name = \"dbmdz/german-gpt2\" \n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "model = AutoModelForCausalLM.from_pretrained(model_name)\n",
    "model = model.bfloat16()\n",
    "\n",
    "# Da GPT-2-Modelle oft keinen expliziten Padding-Token besitzen, setzen wir hier den EOS-Token als Padding-Token.\n",
    "tokenizer.pad_token = tokenizer.eos_token\n",
    "\n",
    "# Konfiguriere pad_token_id im Modell \n",
    "# (braucht man, wenn das Modell noch nicht standardmäßig für den Umgang mit dem Padding-Token eingestellt ist)\n",
    "model.config.pad_token_id = tokenizer.eos_token_id\n",
    "model.generation_config.pad_token_id = tokenizer.pad_token_id"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9d3dcaf-e281-4c42-a60c-069a8812b65c",
   "metadata": {},
   "source": [
    "## Domänenspezifischer Datensatz: Auszüge aus dem AI Act\n",
    "Wir extrahieren zwei wichtige Absätze aus dem AI Act, die juristische Fachtermini und Anforderungen beinhalten. \n",
    "\n",
    "Hinweis: Die folgenden Textabschnitte sind exemplarisch und basieren auf öffentlich zugänglichen Informationen zum AI Act, z.B.: \"https://eur-lex.europa.eu/legal-content/DE/TXT/?uri=CELEX:32024R1689\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "035f7eae-df3e-4b98-8f5d-4bec1fcc71e1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:12.603307Z",
     "iopub.status.busy": "2026-03-24T18:13:12.603223Z",
     "iopub.status.idle": "2026-03-24T18:13:12.605570Z",
     "shell.execute_reply": "2026-03-24T18:13:12.605113Z",
     "shell.execute_reply.started": "2026-03-24T18:13:12.603297Z"
    }
   },
   "outputs": [],
   "source": [
    "# Ausgewählte Absätze aus dem AI Act (Beispiele)\n",
    "ai_act_texts = [\n",
    "    \"\"\"Artikel 1 – Anwendungsbereich: \n",
    "    Diese Verordnung gilt für KI-Systeme, die in der Europäischen Union in Verkehr gebracht oder in Betrieb genommen werden, und legt die grundlegenden Anforderungen an Sicherheit, Transparenz und Verantwortlichkeit fest.\"\"\",\n",
    "    \"\"\"Artikel 2 – Risikoklassifizierung: \n",
    "    KI-Systeme werden in Abhängigkeit von ihrem potenziellen Risiko in verschiedene Kategorien eingeteilt. Hochrisiko-KI-Systeme unterliegen strengen Anforderungen an ihre Konzeption, Entwicklung und den Betrieb, um die Sicherheit und den Schutz der Grundrechte zu gewährleisten.\"\"\"\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "23bc7dc9-f3e6-41bd-8e9f-f8f1c5aae526",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:12.605992Z",
     "iopub.status.busy": "2026-03-24T18:13:12.605920Z",
     "iopub.status.idle": "2026-03-24T18:13:12.610887Z",
     "shell.execute_reply": "2026-03-24T18:13:12.610265Z",
     "shell.execute_reply.started": "2026-03-24T18:13:12.605985Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Domänenspezifischer Datensatz erstellt:\n",
      "Dataset({\n",
      "    features: ['text'],\n",
      "    num_rows: 2\n",
      "})\n"
     ]
    }
   ],
   "source": [
    "# Erstelle ein Dataset aus den Auszügen\n",
    "data_dict = {\"text\": ai_act_texts}\n",
    "dataset = Dataset.from_dict(data_dict)\n",
    "print(\"Domänenspezifischer Datensatz erstellt:\")\n",
    "print(dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "9e420718-e911-4b18-a316-65f2ac1f7b8b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:12.611427Z",
     "iopub.status.busy": "2026-03-24T18:13:12.611322Z",
     "iopub.status.idle": "2026-03-24T18:13:12.853989Z",
     "shell.execute_reply": "2026-03-24T18:13:12.853381Z",
     "shell.execute_reply.started": "2026-03-24T18:13:12.611419Z"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "95c9f49392ed4e91ad78d0acaf713cdc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/2 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tokenisierter Datensatz:\n",
      "Dataset({\n",
      "    features: ['input_ids', 'attention_mask'],\n",
      "    num_rows: 2\n",
      "})\n"
     ]
    }
   ],
   "source": [
    "# %% [code]\n",
    "# Tokenisiere den Datensatz\n",
    "def tokenize_function(example):\n",
    "    return tokenizer(example[\"text\"], truncation=True, padding=\"max_length\", max_length=256)\n",
    "\n",
    "tokenized_dataset = dataset.map(tokenize_function, batched=True)\n",
    "tokenized_dataset = tokenized_dataset.remove_columns([\"text\"])\n",
    "print(\"Tokenisierter Datensatz:\")\n",
    "print(tokenized_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "339445b0-2592-4fba-968d-3e6b3180a939",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:12.854659Z",
     "iopub.status.busy": "2026-03-24T18:13:12.854497Z",
     "iopub.status.idle": "2026-03-24T18:13:12.856695Z",
     "shell.execute_reply": "2026-03-24T18:13:12.856115Z",
     "shell.execute_reply.started": "2026-03-24T18:13:12.854649Z"
    }
   },
   "outputs": [],
   "source": [
    "# Erstelle einen DataCollator für das Language Modeling\n",
    "data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ebd9e4e9-b9d5-4f6c-9ec3-193b0b79b688",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:12.857045Z",
     "iopub.status.busy": "2026-03-24T18:13:12.856986Z",
     "iopub.status.idle": "2026-03-24T18:13:12.872023Z",
     "shell.execute_reply": "2026-03-24T18:13:12.871574Z",
     "shell.execute_reply.started": "2026-03-24T18:13:12.857039Z"
    }
   },
   "outputs": [],
   "source": [
    "# Definiere Trainingsargumente – das Fine-Tuning erfolgt exemplarisch über wenige Epochen\n",
    "training_args = TrainingArguments(\n",
    "    output_dir=\"./llm_ai_act_finetuned\",\n",
    "    num_train_epochs=3,\n",
    "    per_device_train_batch_size=1,\n",
    "    save_steps=5,\n",
    "    save_total_limit=2,\n",
    "    logging_steps=1,\n",
    "    learning_rate=5e-5,\n",
    "    weight_decay=0.01,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a40e68eb-7c6f-436f-a04a-004686432039",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:12.872494Z",
     "iopub.status.busy": "2026-03-24T18:13:12.872418Z",
     "iopub.status.idle": "2026-03-24T18:13:12.984566Z",
     "shell.execute_reply": "2026-03-24T18:13:12.984143Z",
     "shell.execute_reply.started": "2026-03-24T18:13:12.872487Z"
    }
   },
   "outputs": [],
   "source": [
    "# Initialisiere den Trainer für das Fine-Tuning\n",
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    train_dataset=tokenized_dataset,\n",
    "    data_collator=data_collator,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "6c146974-ab45-433b-b964-811f0a668c37",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:12.985085Z",
     "iopub.status.busy": "2026-03-24T18:13:12.984991Z",
     "iopub.status.idle": "2026-03-24T18:13:15.690419Z",
     "shell.execute_reply": "2026-03-24T18:13:15.689949Z",
     "shell.execute_reply.started": "2026-03-24T18:13:12.985078Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starte das Fine-Tuning mit AI Act-Daten...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/veit/cusy/trn/ai-tutorial/.venv/lib/python3.13/site-packages/torch/utils/data/dataloader.py:775: UserWarning: 'pin_memory' argument is set as true but not supported on MPS now, device pinned memory won't be used.\n",
      "  super().__init__(loader)\n",
      "`loss_type=None` was set in the config but it is unrecognized. Using the default loss: `ForCausalLMLoss`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='6' max='6' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [6/6 00:02, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2.830693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3.512209</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>2.700200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>3.442378</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>2.657775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>3.438793</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0a35267676b748888838431fc72a33fc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8e7bdf37e81a4269ba3d2c98cfef9e08",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Writing model shards:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fine-Tuning abgeschlossen.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GPT2LMHeadModel(\n",
       "  (transformer): GPT2Model(\n",
       "    (wte): Embedding(50265, 768)\n",
       "    (wpe): Embedding(1024, 768)\n",
       "    (drop): Dropout(p=0.0, inplace=False)\n",
       "    (h): ModuleList(\n",
       "      (0-11): 12 x GPT2Block(\n",
       "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "        (attn): GPT2Attention(\n",
       "          (c_attn): Conv1D(nf=2304, nx=768)\n",
       "          (c_proj): Conv1D(nf=768, nx=768)\n",
       "          (attn_dropout): Dropout(p=0.0, inplace=False)\n",
       "          (resid_dropout): Dropout(p=0.0, inplace=False)\n",
       "        )\n",
       "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "        (mlp): GPT2MLP(\n",
       "          (c_fc): Conv1D(nf=3072, nx=768)\n",
       "          (c_proj): Conv1D(nf=768, nx=3072)\n",
       "          (act): NewGELUActivation()\n",
       "          (dropout): Dropout(p=0.0, inplace=False)\n",
       "        )\n",
       "      )\n",
       "    )\n",
       "    (ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "  )\n",
       "  (lm_head): Linear(in_features=768, out_features=50265, bias=False)\n",
       ")"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Starte das Fine-Tuning\n",
    "print(\"Starte das Fine-Tuning mit AI Act-Daten...\")\n",
    "trainer.train()\n",
    "print(\"Fine-Tuning abgeschlossen.\")\n",
    "\n",
    "# Modell auf die CPU schieben und und alle Eingaben auf der CPU verarbeiten \n",
    "model.to(\"cpu\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "989decaa-3b68-4de2-b720-a54305f615a7",
   "metadata": {},
   "source": [
    "## Test: Juristische Fragen erneut stellen\n",
    "Nun stellen wir wieder dieselben Fragen wie in llm-1, um zu prüfen, ob das feingetunte Modell (llm-2) bessere Antworten liefert."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8df80e0e-f94c-4926-bbf5-0040314fd928",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:15.690887Z",
     "iopub.status.busy": "2026-03-24T18:13:15.690790Z",
     "iopub.status.idle": "2026-03-24T18:13:15.693263Z",
     "shell.execute_reply": "2026-03-24T18:13:15.692541Z",
     "shell.execute_reply.started": "2026-03-24T18:13:15.690879Z"
    }
   },
   "outputs": [],
   "source": [
    "def ask_question(prompt):\n",
    "    input_ids = tokenizer.encode(prompt, return_tensors=\"pt\")\n",
    "    output = model.generate(input_ids, max_length=150, temperature=0.7, do_sample=True)\n",
    "    generated_text = tokenizer.decode(output[0], skip_special_tokens=True)\n",
    "    return generated_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4fdd2b04-7780-4342-b91a-8fd4b3fc4827",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:15.693749Z",
     "iopub.status.busy": "2026-03-24T18:13:15.693668Z",
     "iopub.status.idle": "2026-03-24T18:13:15.696706Z",
     "shell.execute_reply": "2026-03-24T18:13:15.696026Z",
     "shell.execute_reply.started": "2026-03-24T18:13:15.693742Z"
    }
   },
   "outputs": [],
   "source": [
    "questions = [\n",
    "    \"Welche Anforderungen stellt der AI Act an Hochrisiko-KI-Systeme?\",\n",
    "    \"Was versteht man unter Transparenz gemäß dem AI Act?\"\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "79d1b1b5-0ef8-434d-9caa-61ae0b9f323e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-24T18:13:15.697141Z",
     "iopub.status.busy": "2026-03-24T18:13:15.697053Z",
     "iopub.status.idle": "2026-03-24T18:13:34.556981Z",
     "shell.execute_reply": "2026-03-24T18:13:34.556423Z",
     "shell.execute_reply.started": "2026-03-24T18:13:15.697133Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== llm-2: Feingetuntes Modell ===\n",
      "\n",
      "Frage: Welche Anforderungen stellt der AI Act an Hochrisiko-KI-Systeme?\n",
      "\n",
      "Antwort: Welche Anforderungen stellt der AI Act an Hochrisiko-KI-Systeme?\n",
      "Die Sicherheitsanforderungen von AI Act und AI Act sind auf ein Minimum beschränkt.\n",
      "Sie können die Software als Ganzes nicht ändern oder modifizieren, da nur ein Teil der Nutzer davon betroffen ist.\n",
      "Dies ist ein Problem bei der Entwicklung von Software, die sich in der Regel nur in den Quellcode einfügt, nicht aber in Maschinencode.\n",
      "Die Software kann sich auch überschreiben oder zu Fehlfunktionen führen.\n",
      "Das ist vorprogrammiert, wenn die Nutzung von AI Act, AI Act, AI Act, AI Act, AI Act, AI Act, AI Act, AI Act, AI Act, AI Act\n",
      "------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "Frage: Was versteht man unter Transparenz gemäß dem AI Act?\n",
      "\n",
      "Antwort: Was versteht man unter Transparenz gemäß dem AI Act?\n",
      "Der Ausschuss nimmt die Bedenken der Experten zur Kenntnis, welche die IAAF zu diesem Thema haben.\n",
      "Die IAAF hat in der Vergangenheit bereits ein Dokument mit dem Titel “The Compliance of IAAF members“ veröffentlicht.\n",
      "Darin wird die IAAF aufgefordert, das Vertrauen der Öffentlichkeit in die IAAF zu garantieren und die Überwachung der IAAF zu verstärken.\n",
      "Dies sind die allgemeinen Merkmale des IAAF-Statuts.\n",
      "Der Ausschuss ist der Ansicht, dass die IAAF ihre Aufsichtspflicht verletzt hat.\n",
      "Die IAAF ist jedoch der Meinung, dass die IAAF diese Verpflichtungen nicht verletzt hat.\n",
      "Die IAAF ist der\n",
      "------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "print(\"\\n=== llm-2: Feingetuntes Modell ===\\n\")\n",
    "for q in questions:\n",
    "    print(\"Frage:\", q)\n",
    "    print() \n",
    "    answer = ask_question(q)\n",
    "    print(\"Antwort:\", answer)\n",
    "    print(\"-\" * 300)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.0"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "00edbe7a58d546d586443954281f376b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_e2f3972f6bf5445f9ccffa239c94a8b6",
        "IPY_MODEL_aa677e8adf3d40e3bf88d2fd169f4b11",
        "IPY_MODEL_a76373154b294a669b662875d28f792c"
       ],
       "layout": "IPY_MODEL_a348e1ac982046a5b729cb388977e368"
      }
     },
     "07f887519d3048978f13f354f490cd06": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "0a35267676b748888838431fc72a33fc": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_98dbdef0c9f04d369fcee2a2de25ed04",
        "IPY_MODEL_6b7fe98ace164e2189bfcbdbbe6b8300",
        "IPY_MODEL_771763525b394bb5a0479734c9e767fa"
       ],
       "layout": "IPY_MODEL_1d0f47b060f342428d9edef9af9119a1"
      }
     },
     "0ac480cec27844dea5977edb7c7d0536": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "123f6ee9aca549ba8c8d4ad9dbac3712": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "1377f4044763495a802cd901821bff71": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "1d0f47b060f342428d9edef9af9119a1": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "29fca0c1bd074cfcaec25b1a01b50cbe": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "2d05c55164c748628ebb03d3ffe055ff": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_4920ca36180d44c8b26e8aaa27a395ab",
       "style": "IPY_MODEL_b5c3f1be697f4cd5a4bc97ba09db5805",
       "value": "Map: 100%"
      }
     },
     "2fea148bae8747089118b7fc247f7a07": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "3848dd533eb040b39bb8274ee8b07d01": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "4920ca36180d44c8b26e8aaa27a395ab": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "49dd02a0328b4553938ffe7666b98ec4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "4c0d618ca14641d69c125810c9f508d6": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_8790317fc314429a84b704820565d99a",
       "style": "IPY_MODEL_49dd02a0328b4553938ffe7666b98ec4",
       "value": "Writing model shards: 100%"
      }
     },
     "560505d5ff8444879dd702e1704bc1f9": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_29fca0c1bd074cfcaec25b1a01b50cbe",
       "style": "IPY_MODEL_c0f6ba3045a9484e909ac8675a27c372",
       "value": " 1/1 [00:00&lt;00:00,  5.92it/s]"
      }
     },
     "59114b68bafb4f4987b3889e9a3ceb07": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "5fb0781cb4e741089dc124cb860f27b9": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "6b7fe98ace164e2189bfcbdbbe6b8300": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_e77bb0b48cd242fc8f30ea9dd8d3286c",
       "max": 1,
       "style": "IPY_MODEL_2fea148bae8747089118b7fc247f7a07",
       "value": 1
      }
     },
     "707601a827414197a3501e390f4a7eb4": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "750ae57e3a2c41ed8c38081bc96e0b93": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_123f6ee9aca549ba8c8d4ad9dbac3712",
       "style": "IPY_MODEL_5fb0781cb4e741089dc124cb860f27b9",
       "value": " 2/2 [00:00&lt;00:00, 271.99 examples/s]"
      }
     },
     "771763525b394bb5a0479734c9e767fa": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_07f887519d3048978f13f354f490cd06",
       "style": "IPY_MODEL_ca137b0d7e2c488e8e192a099026618c",
       "value": " 1/1 [00:00&lt;00:00,  6.78it/s]"
      }
     },
     "7ae643efa6564410b018c9755302c9cc": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "7e54d4b927784f3d886bac008bf8256c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "8790317fc314429a84b704820565d99a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "8e7bdf37e81a4269ba3d2c98cfef9e08": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_4c0d618ca14641d69c125810c9f508d6",
        "IPY_MODEL_e0ca4b8ad0314a1faa6afdd7bbccee68",
        "IPY_MODEL_560505d5ff8444879dd702e1704bc1f9"
       ],
       "layout": "IPY_MODEL_3848dd533eb040b39bb8274ee8b07d01"
      }
     },
     "8f5d6eca4ca34651bed317cb9e085082": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "95c9f49392ed4e91ad78d0acaf713cdc": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_2d05c55164c748628ebb03d3ffe055ff",
        "IPY_MODEL_ef40e27c46354221b44b2d1a853479b3",
        "IPY_MODEL_750ae57e3a2c41ed8c38081bc96e0b93"
       ],
       "layout": "IPY_MODEL_dae116ba59be4eb39d950e3cf635ab0e"
      }
     },
     "98dbdef0c9f04d369fcee2a2de25ed04": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_1377f4044763495a802cd901821bff71",
       "style": "IPY_MODEL_b394c4155a164f5f8878fd164633fc46",
       "value": "Writing model shards: 100%"
      }
     },
     "a348e1ac982046a5b729cb388977e368": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "a76373154b294a669b662875d28f792c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_707601a827414197a3501e390f4a7eb4",
       "style": "IPY_MODEL_7ae643efa6564410b018c9755302c9cc",
       "value": " 148/148 [00:00&lt;00:00, 4769.88it/s]"
      }
     },
     "aa677e8adf3d40e3bf88d2fd169f4b11": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_59114b68bafb4f4987b3889e9a3ceb07",
       "max": 148,
       "style": "IPY_MODEL_0ac480cec27844dea5977edb7c7d0536",
       "value": 148
      }
     },
     "b394c4155a164f5f8878fd164633fc46": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "b5c3f1be697f4cd5a4bc97ba09db5805": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "c0f6ba3045a9484e909ac8675a27c372": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "ca137b0d7e2c488e8e192a099026618c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "cc9067434db849b4a62a302566f12d44": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "cecad38de2f148ccb7839c4e9a986792": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "dae116ba59be4eb39d950e3cf635ab0e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "e0ca4b8ad0314a1faa6afdd7bbccee68": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_cecad38de2f148ccb7839c4e9a986792",
       "max": 1,
       "style": "IPY_MODEL_e59690253d6d4dec86b0e44536e79005",
       "value": 1
      }
     },
     "e2f3972f6bf5445f9ccffa239c94a8b6": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_cc9067434db849b4a62a302566f12d44",
       "style": "IPY_MODEL_7e54d4b927784f3d886bac008bf8256c",
       "value": "Loading weights: 100%"
      }
     },
     "e59690253d6d4dec86b0e44536e79005": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "e77bb0b48cd242fc8f30ea9dd8d3286c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "ef40e27c46354221b44b2d1a853479b3": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_f1b9fb0a7d034ca9917c97815654a540",
       "max": 2,
       "style": "IPY_MODEL_8f5d6eca4ca34651bed317cb9e085082",
       "value": 2
      }
     },
     "f1b9fb0a7d034ca9917c97815654a540": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}