{ "cells": [ { "cell_type": "markdown", "id": "77d42ede", "metadata": {}, "source": [ "# Micro-GPT art project by Andrej Karpathy\n", "\n", "\"The most atomic way to train and run inference for a GPT in pure, dependency-free Python.\n", "\n", "This file is the complete algorithm.\n", "\n", "Everything else is just efficiency.\"\n", "\n", "@karpathy\n" ] }, { "cell_type": "markdown", "id": "08fa7d41", "metadata": {}, "source": [ "#### We implement a tiny GPT-like model from scratch using only the Python standard library.\n", "\n", "Educational mindset:\n", "\n", "- keep everything readable\n", "\n", "- explain the 'why' (what problem we are solving) more than the 'how' (performance tricks)\n", "\n", "- do not worry about speed; clarity matters for beginners.\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "45641ad2", "metadata": { "execution": { "iopub.execute_input": "2026-03-24T18:17:45.735080Z", "iopub.status.busy": "2026-03-24T18:17:45.734600Z", "iopub.status.idle": "2026-03-24T18:17:45.738789Z", "shell.execute_reply": "2026-03-24T18:17:45.737936Z", "shell.execute_reply.started": "2026-03-24T18:17:45.735062Z" } }, "outputs": [], "source": [ "import os # os.path.exists: check whether we already have the dataset locally\n", "import math # math.log / math.exp: used for softmax and the negative log-likelihood loss\n", "import random # random: shuffling the dataset and sampling tokens during generation\n", "\n", "# Deterministic randomness makes results reproducible (important for learning + debugging).\n", "random.seed(42) # Let there be order among chaos" ] }, { "cell_type": "markdown", "id": "79d9db87", "metadata": {}, "source": [ "Let there be a Dataset `docs`: a list of documents, where each document is a string.\n", "\n", "This notebook learns on a list of human names and then generates (hallucinated) new names.\n", "\n", "Why do we need a dataset?\n", "\n", "A neural network doesn't come with knowledge. It learns patterns by repeatedly seeing examples.\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "bfc6e563", "metadata": { "execution": { "iopub.execute_input": "2026-03-24T18:17:45.739308Z", "iopub.status.busy": "2026-03-24T18:17:45.739075Z", "iopub.status.idle": "2026-03-24T18:17:45.751273Z", "shell.execute_reply": "2026-03-24T18:17:45.750239Z", "shell.execute_reply.started": "2026-03-24T18:17:45.739290Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "num docs: 32033\n" ] } ], "source": [ "# We use a small plain-text dataset. If it isn't present yet, download it.\n", "if not os.path.exists('input.txt'):\n", " import urllib.request\n", " # The file format is simple: one name per line.\n", " names_url = 'https://raw.githubusercontent.com/karpathy/makemore/988aa59/names.txt'\n", " urllib.request.urlretrieve(names_url, 'input.txt')\n", "\n", "\n", "# Read the dataset file into memory.\n", "# - `line.strip()` removes whitespace/newlines around the name\n", "# - `if line.strip()` skips empty lines (defensive programming)\n", "docs = [line.strip() for line in open('input.txt') if line.strip()]\n", "\n", "# Shuffle the list so training sees a mixed stream of examples.\n", "random.shuffle(docs)\n", "print(f\"num docs: {len(docs)}\")\n", "\n", "# print(docs[:10]) # removed: extra output not needed for the lesson" ] }, { "cell_type": "code", "execution_count": 3, "id": "9cd1e32f", "metadata": { "execution": { "iopub.execute_input": "2026-03-24T18:17:45.751801Z", "iopub.status.busy": "2026-03-24T18:17:45.751659Z", "iopub.status.idle": "2026-03-24T18:17:45.755715Z", "shell.execute_reply": "2026-03-24T18:17:45.755118Z", "shell.execute_reply.started": "2026-03-24T18:17:45.751791Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "vocab size: 27\n" ] } ], "source": [ "# Let there be a Tokenizer to translate strings to sequences of integers (\"tokens\") and back\n", "#\n", "# Tokenizer idea (character-level):\n", "# - collect all unique characters that appear in the dataset\n", "# - assign each character an integer id (a 'token'), happens implicitly with list entrance of uchars, used during tokenization\n", "#\n", "# Why character-level?\n", "# It keeps the problem small: we only need to predict the next character.\n", "uchars = sorted(set(''.join(docs))) # unique characters in the dataset become token ids 0..n-1\n", "\n", "# BOS (Beginning Of Sequence) is a special token that marks the start of a sequence.\n", "# We use it so the model learns a meaningful way to begin generation.\n", "BOS = len(uchars) # token id for a special Beginning of Sequence (BOS) token\n", "\n", "# vocab_size is the number of possible tokens the model can choose from.\n", "vocab_size = len(uchars) + 1 # total number of unique tokens, +1 is for BOS\n", "print(f\"vocab size: {vocab_size}\")" ] }, { "cell_type": "code", "execution_count": 4, "id": "15f22ee8", "metadata": { "execution": { "iopub.execute_input": "2026-03-24T18:17:45.756083Z", "iopub.status.busy": "2026-03-24T18:17:45.756006Z", "iopub.status.idle": "2026-03-24T18:17:45.761020Z", "shell.execute_reply": "2026-03-24T18:17:45.760348Z", "shell.execute_reply.started": "2026-03-24T18:17:45.756075Z" } }, "outputs": [], "source": [ "# Let there be Autograd to recursively apply the chain rule through a computation graph\n", "class Value:\n", " '''A tiny scalar value that supports automatic differentiation.\n", "\n", " Each `Value` stores:\n", " - `data`: the numeric result of this node (a single float)\n", " - `grad`: the gradient of the final loss with respect to `data`\n", " - `_children`: references to input nodes this value depends on\n", " - `_local_grads`: local derivatives used for the chain rule\n", "\n", " Why this exists:\n", " Training requires gradients (how to change parameters to reduce loss).\n", " Real ML frameworks build computation graphs + derivatives automatically.\n", " This `Value` is the minimal, educational version of that mechanism.\n", " '''\n", " __slots__ = ('data', 'grad', '_children', '_local_grads') # Python optimization for memory usage\n", "\n", " def __init__(self, data, children=(), local_grads=()):\n", " self.data = data # scalar value of this node calculated during forward pass\n", " self.grad = 0 # derivative of the loss w.r.t. this node, calculated in backward pass\n", " self._children = children # children of this node in the computation graph\n", " self._local_grads = local_grads # local derivative of this node w.r.t. its children\n", "\n", " def __add__(self, other):\n", " other = other if isinstance(other, Value) else Value(other)\n", " return Value(self.data + other.data, (self, other), (1, 1))\n", "\n", " def __mul__(self, other):\n", " other = other if isinstance(other, Value) else Value(other)\n", " return Value(self.data * other.data, (self, other), (other.data, self.data))\n", "\n", " def __pow__(self, other): return Value(self.data**other, (self,), (other * self.data**(other-1),))\n", " def log(self): return Value(math.log(self.data), (self,), (1/self.data,))\n", " def exp(self): return Value(math.exp(self.data), (self,), (math.exp(self.data),))\n", " def relu(self): return Value(max(0, self.data), (self,), (float(self.data > 0),))\n", " def __neg__(self): return self * -1\n", " def __radd__(self, other): return self + other\n", " def __sub__(self, other): return self + (-other)\n", " def __rsub__(self, other): return other + (-self)\n", " def __rmul__(self, other): return self * other\n", " def __truediv__(self, other): return self * other**-1\n", " def __rtruediv__(self, other): return other * self**-1\n", "\n", " def backward(self):\n", " '''Compute gradients using reverse-mode automatic differentiation.\n", "\n", " Forward pass builds a computation graph (nodes + edges).\n", " Backward pass traverses that graph in reverse topological order and applies\n", " the chain rule to propagate `grad` values from the output (usually the loss)\n", " back to all parameters.\n", " '''\n", " topo = [] # nodes in topological order: children come before parents\n", " visited = set() # avoid adding the same node multiple times\n", "\n", " def build_topo(v):\n", " # Depth-first search to collect nodes in the order we need for backprop.\n", " if v not in visited:\n", " visited.add(v)\n", " for child in v._children:\n", " build_topo(child)\n", " topo.append(v) # append after children => topological order\n", "\n", " build_topo(self)\n", "\n", " # Seed the gradient at the output node.\n", " # If `self` is the loss, then d(loss)/d(loss) = 1.\n", " self.grad = 1\n", "\n", " # Propagate gradients backwards.\n", " for v in reversed(topo):\n", " # Chain rule: each child's grad accumulates the local derivative times\n", " # the gradient of the current node.\n", " for child, local_grad in zip(v._children, v._local_grads):\n", " child.grad += local_grad * v.grad\n", "\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "82637ff8", "metadata": { "execution": { "iopub.execute_input": "2026-03-24T18:17:45.761451Z", "iopub.status.busy": "2026-03-24T18:17:45.761363Z", "iopub.status.idle": "2026-03-24T18:17:45.767336Z", "shell.execute_reply": "2026-03-24T18:17:45.766383Z", "shell.execute_reply.started": "2026-03-24T18:17:45.761443Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "num params: 4192\n" ] } ], "source": [ "# Initialize the parameters, to store the knowledge of the model.\n", "#\n", "# In a transformer/GPT, the parameters are the weight matrices used in:\n", "# - embeddings (turn token ids into vectors, plus position information)\n", "# - self-attention (mix information from previous positions)\n", "# - the MLP (a feed-forward network that adds non-linearity)\n", "\n", "# Hyperparameters (kept small for a teaching notebook).\n", "n_layer = 1 # depth of the transformer neural network (number of layers)\n", "n_embd = 16 # width of the network (embedding dimension)\n", "block_size = 16 # maximum context length of the attention window (longest name is ~15 characters)\n", "n_head = 4 # number of attention heads\n", "head_dim = n_embd // n_head # derived dimension of each head\n", "\n", "# Helper to create a matrix filled with small random numbers.\n", "# We wrap every scalar with `Value(...)` so autograd can compute gradients.\n", "matrix = lambda nout, nin, std=0.08: [[Value(random.gauss(0, std)) for _ in range(nin)] for _ in range(nout)]\n", "\n", "# state_dict collects *all* parameters by name.\n", "# - wte: token embeddings\n", "# - wpe: position embeddings\n", "# - lm_head: maps final hidden state -> logits over the vocabulary\n", "state_dict = {'wte': matrix(vocab_size, n_embd), 'wpe': matrix(block_size, n_embd), 'lm_head': matrix(vocab_size, n_embd)}\n", "\n", "# Create additional weights for each transformer layer.\n", "for i in range(n_layer):\n", " state_dict[f'layer{i}.attn_wq'] = matrix(n_embd, n_embd)\n", " state_dict[f'layer{i}.attn_wk'] = matrix(n_embd, n_embd)\n", " state_dict[f'layer{i}.attn_wv'] = matrix(n_embd, n_embd)\n", " state_dict[f'layer{i}.attn_wo'] = matrix(n_embd, n_embd)\n", " state_dict[f'layer{i}.mlp_fc1'] = matrix(4 * n_embd, n_embd)\n", " state_dict[f'layer{i}.mlp_fc2'] = matrix(n_embd, 4 * n_embd)\n", "\n", "# Flatten params into a single list[Value] so the optimizer can update them.\n", "params = [p for mat in state_dict.values() for row in mat for p in row] # flatten params into a single list[Value]\n", "print(f\"num params: {len(params)}\")" ] }, { "cell_type": "code", "execution_count": 6, "id": "cfc47ef4", "metadata": { "execution": { "iopub.execute_input": "2026-03-24T18:17:45.767840Z", "iopub.status.busy": "2026-03-24T18:17:45.767736Z", "iopub.status.idle": "2026-03-24T18:17:45.773292Z", "shell.execute_reply": "2026-03-24T18:17:45.772766Z", "shell.execute_reply.started": "2026-03-24T18:17:45.767828Z" } }, "outputs": [], "source": [ "# Define the model architecture: a function mapping tokens and parameters to logits over what comes next\n", "# Follow GPT-2, blessed among the GPTs, with minor differences: layernorm -> rmsnorm, no biases, GeLU -> ReLU\n", "def linear(x, w):\n", " '''A tiny fully-connected (linear) layer.\n", "\n", " x is a vector of length `nin`.\n", " w is a matrix of shape (nout, nin).\n", " The result is a vector of length `nout`.\n", " '''\n", " return [sum(wi * xi for wi, xi in zip(wo, x)) for wo in w]\n", "\n", "def softmax(logits):\n", " '''Convert unnormalized scores (logits) into probabilities.\n", "\n", " Softmax makes the outputs interpretable as:\n", " P(token=i | context)\n", " '''\n", " max_val = max(val.data for val in logits)\n", " exps = [(val - max_val).exp() for val in logits]\n", " total = sum(exps)\n", " return [e / total for e in exps]\n", "\n", "def rmsnorm(x):\n", " '''Root Mean Square normalization.\n", "\n", " Neural networks are easier to train when activations have a reasonable scale.\n", " RMSNorm is a simpler alternative to LayerNorm.\n", " '''\n", " ms = sum(xi * xi for xi in x) / len(x)\n", " scale = (ms + 1e-5) ** -0.5\n", " return [xi * scale for xi in x]\n", "\n", "def gpt(token_id, pos_id, keys, values):\n", " '''Run the mini-GPT for a single (token, position).\n", "\n", " `keys` and `values` are caches that grow as we move from left to right.\n", " That means attention is naturally causal here (no future tokens are in the cache).\n", " '''\n", " tok_emb = state_dict['wte'][token_id] # token embedding\n", " pos_emb = state_dict['wpe'][pos_id] # position embedding\n", " x = [t + p for t, p in zip(tok_emb, pos_emb)] # joint token and position embedding\n", " x = rmsnorm(x) # note: not redundant due to backward pass via the residual connection\n", "\n", " for li in range(n_layer):\n", " # 1) Multi-head Attention block\n", " x_residual = x\n", " x = rmsnorm(x)\n", " q = linear(x, state_dict[f'layer{li}.attn_wq'])\n", " k = linear(x, state_dict[f'layer{li}.attn_wk'])\n", " v = linear(x, state_dict[f'layer{li}.attn_wv'])\n", " keys[li].append(k)\n", " values[li].append(v)\n", " x_attn = []\n", " for h in range(n_head):\n", " hs = h * head_dim\n", " q_h = q[hs:hs+head_dim]\n", " k_h = [ki[hs:hs+head_dim] for ki in keys[li]]\n", " v_h = [vi[hs:hs+head_dim] for vi in values[li]]\n", " # Attention scores: how well the current query matches each past key.\n", " attn_logits = [sum(q_h[j] * k_h[t][j] for j in range(head_dim)) / head_dim**0.5 for t in range(len(k_h))]\n", " # Turn scores into probabilities over past positions.\n", " attn_weights = softmax(attn_logits)\n", " # Weighted sum of value vectors gives the head output.\n", " head_out = [sum(attn_weights[t] * v_h[t][j] for t in range(len(v_h))) for j in range(head_dim)]\n", " x_attn.extend(head_out)\n", " x = linear(x_attn, state_dict[f'layer{li}.attn_wo'])\n", " x = [a + b for a, b in zip(x, x_residual)]\n", " # 2) MLP block\n", " x_residual = x\n", " x = rmsnorm(x)\n", " x = linear(x, state_dict[f'layer{li}.mlp_fc1'])\n", " # ReLU is the non-linearity: without it, the MLP would stay linear.\n", " x = [xi.relu() for xi in x]\n", " x = linear(x, state_dict[f'layer{li}.mlp_fc2'])\n", " x = [a + b for a, b in zip(x, x_residual)]\n", "\n", " # Final projection: map hidden state to logits over the vocabulary.\n", " logits = linear(x, state_dict['lm_head'])\n", " return logits" ] }, { "cell_type": "code", "execution_count": 7, "id": "8f49ec80", "metadata": { "execution": { "iopub.execute_input": "2026-03-24T18:17:45.773703Z", "iopub.status.busy": "2026-03-24T18:17:45.773631Z", "iopub.status.idle": "2026-03-24T18:17:45.775854Z", "shell.execute_reply": "2026-03-24T18:17:45.775318Z", "shell.execute_reply.started": "2026-03-24T18:17:45.773695Z" } }, "outputs": [], "source": [ "# Let there be Adam, the blessed optimizer and its buffers.\n", "#\n", "# Training loop repeats:\n", "# 1) forward pass -> compute loss\n", "# 2) backward pass -> compute gradients\n", "# 3) optimizer step -> update parameters\n", "#\n", "# Adam updates parameters using running averages of gradients.\n", "\n", "learning_rate, beta1, beta2, eps_adam = 0.01, 0.85, 0.99, 1e-8\n", "\n", "# First moment estimate (mean of gradients)\n", "m = [0.0] * len(params) # first moment buffer\n", "# Second moment estimate (mean of squared gradients)\n", "v = [0.0] * len(params) # second moment buffer" ] }, { "cell_type": "code", "execution_count": 8, "id": "9a78f749", "metadata": { "execution": { "iopub.execute_input": "2026-03-24T18:17:45.776211Z", "iopub.status.busy": "2026-03-24T18:17:45.776137Z", "iopub.status.idle": "2026-03-24T18:19:00.509937Z", "shell.execute_reply": "2026-03-24T18:19:00.509499Z", "shell.execute_reply.started": "2026-03-24T18:17:45.776204Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "step 1000 / 1000 | loss 2.6497" ] } ], "source": [ "# Training loop: repeat in sequence\n", "#\n", "# Goal: next-character prediction.\n", "# We take one training name, create a sequence of token ids,\n", "# and repeatedly ask the model: 'Given the prefix so far, what comes next?'\n", "\n", "num_steps = 1000 # number of training steps\n", "for step in range(num_steps):\n", "\n", " # Take a single document (name) and turn it into training tokens.\n", " #\n", " # Why tokenize?\n", " # Machine learning code usually works on numbers.\n", " # Here: each character is mapped to an integer token id.\n", " #\n", " # Why BOS at both ends?\n", " # - First BOS marks 'start of sequence'.\n", " # - Last BOS provides an explicit end marker.\n", " doc = docs[step % len(docs)]\n", " tokens = [BOS] + [uchars.index(ch) for ch in doc] + [BOS] # here, the element index is used as integer id (see above)\n", " # We can only use a limited amount of context.\n", " # `block_size` is the maximum sequence length the model will consider.\n", " #\n", " # We subtract 1 because for each position we need a *next* token as target.\n", " n = min(block_size, len(tokens) - 1)\n", "\n", " # Forward pass:\n", " # We iterate over positions and, for each one, do:\n", " # token -> model logits -> probabilities -> one-step loss\n", " #\n", " # Why 'build up the computation graph'?\n", " # Our `Value` class records dependencies during the forward pass.\n", " # Later, `loss.backward()` uses that recorded graph to compute gradients via\n", " # the chain rule.\n", " # Reset the attention caches for this new sample.\n", " # (They will grow as we generate more characters.)\n", " keys, values = [[] for _ in range(n_layer)], [[] for _ in range(n_layer)]\n", " losses = []\n", " for pos_id in range(n):\n", " # Current token is the model input.\n", " # The model should predict the next token as the supervised target.\n", " token_id, target_id = tokens[pos_id], tokens[pos_id + 1]\n", " # Forward pass: get unnormalized scores (logits) for the next token.\n", " logits = gpt(token_id, pos_id, keys, values)\n", " probs = softmax(logits)\n", " # Loss for this position (cross entropy / negative log likelihood).\n", " # We take -log(P(correct next token)).\n", " # If the correct token is unlikely, the loss becomes large.\n", " loss_t = -probs[target_id].log()\n", " losses.append(loss_t)\n", " # Average loss across all positions in this example.\n", " # The average makes the learning signal scale nicely with sequence length.\n", " loss = (1 / n) * sum(losses) # final average loss over the document sequence. May yours be low.\n", "\n", " # Backward pass:\n", " # `loss.backward()` fills `.grad` for every parameter in `params`.\n", " #\n", " # Remember: our `Value` objects stored the computation graph during the forward pass.\n", " # Backward uses that graph to apply the chain rule.\n", " loss.backward()\n", "\n", " # Optimizer step (Adam): update parameters based on gradients.\n", " #\n", " # Adam uses moving averages (moments) to adapt the update size.\n", " # It also helps prevent training from being overly sensitive to noise.\n", " #\n", " # We additionally decay the learning rate so updates become smaller over time.\n", " lr_t = learning_rate * (1 - step / num_steps) # linear learning rate decay\n", " for i, p in enumerate(params):\n", " # Update first moment estimate (mean gradient).\n", " # This tracks the 'direction' we should move parameters in.\n", " m[i] = beta1 * m[i] + (1 - beta1) * p.grad\n", " # Update second moment estimate (mean squared gradient).\n", " # This tells us how noisy/variable the gradient is.\n", " v[i] = beta2 * v[i] + (1 - beta2) * p.grad ** 2\n", " # Bias correction for the first moment.\n", " # At the start, the moving average is biased towards 0.\n", " m_hat = m[i] / (1 - beta1 ** (step + 1))\n", " # Bias correction for the second moment.\n", " # This also compensates for initialization at 0.\n", " v_hat = v[i] / (1 - beta2 ** (step + 1))\n", " # Final Adam update.\n", " # - divide by sqrt(v_hat): adapts step size per parameter\n", " # - eps_adam avoids division by zero\n", " p.data -= lr_t * m_hat / (v_hat ** 0.5 + eps_adam)\n", " # Clear gradients so they don't accumulate across steps.\n", " p.grad = 0\n", "\n", " print(f\"step {step+1:4d} / {num_steps:4d} | loss {loss.data:.4f}\", end='\\r')" ] }, { "cell_type": "code", "execution_count": 9, "id": "af451b41", "metadata": { "execution": { "iopub.execute_input": "2026-03-24T18:19:00.510631Z", "iopub.status.busy": "2026-03-24T18:19:00.510532Z", "iopub.status.idle": "2026-03-24T18:19:01.128501Z", "shell.execute_reply": "2026-03-24T18:19:01.127973Z", "shell.execute_reply.started": "2026-03-24T18:19:00.510622Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "--- inference (new, hallucinated names) ---\n", "sample 1: kamon\n", "sample 2: ann\n", "sample 3: karai\n", "sample 4: jaire\n", "sample 5: vialan\n", "sample 6: karia\n", "sample 7: yeran\n", "sample 8: anna\n", "sample 9: areli\n", "sample 10: kaina\n", "sample 11: konna\n", "sample 12: keylen\n", "sample 13: liole\n", "sample 14: alerin\n", "sample 15: earan\n", "sample 16: lenne\n", "sample 17: kana\n", "sample 18: lara\n", "sample 19: alela\n", "sample 20: anton\n" ] } ], "source": [ "# Inference: generate new text from the trained model.\n", "#\n", "# During training we always know the correct next character.\n", "# During inference we *don't* know it, so we sample from the model's predicted probabilities.\n", "\n", "# Temperature controls how random the sampling is.\n", "# - temperature < 1 makes the model more confident (less random)\n", "# - temperature > 1 makes it more random (more diverse)\n", "temperature = 0.5 # in (0, 1], control the \"creativity\" of generated text, low to high\n", "print(\"\\n--- inference (new, hallucinated names) ---\")\n", "for sample_idx in range(20):\n", " keys, values = [[] for _ in range(n_layer)], [[] for _ in range(n_layer)]\n", " # Start generation by forcing the first token to be BOS.\n", " token_id = BOS\n", " # We'll collect generated characters until we hit BOS or the context is full.\n", " sample = []\n", " for pos_id in range(block_size):\n", " logits = gpt(token_id, pos_id, keys, values)\n", " # Convert logits -> probabilities.\n", " # We divide by `temperature` to control randomness.\n", " probs = softmax([l / temperature for l in logits])\n", " # Sample one token id according to the probability distribution.\n", " # We sample instead of taking argmax so generation can be diverse.\n", " token_id = random.choices(range(vocab_size), weights=[p.data for p in probs])[0]\n", " # BOS acts like an end-of-sequence marker.\n", " # If we sample BOS, we stop generating further characters.\n", " if token_id == BOS:\n", " break\n", " # Convert token id back to a character and add it to our output.\n", " sample.append(uchars[token_id])\n", " print(f\"sample {sample_idx+1:2d}: {''.join(sample)}\")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.0" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 5 }