Build and Self-Host a Mini Marketing GPT

Build a GPT transformer from scratch in PyTorch and serve it as a local API.

Introduction

30 Second Summary

Every time you ask ChatGPT to write an email or generate ad copy, a language model is predicting the next word based on patterns it learned from massive amounts of text.

In this project, you will build a GPT-style language model from absolute scratch in PyTorch, train it on real marketing email data, and then self-host it as a local API that generates marketing copy on demand.

What You'll Build

A complete mini GPT language model (~10M parameters) written entirely by hand in PyTorch, trained on marketing emails in Google Colab, and served locally through a FastAPI REST API.

By the end of this project, you'll have:

  • A working transformer model you wrote from scratch, including multi-head self-attention, embeddings, and feed-forward networks.
  • A trained model checkpoint exported from Colab and running on your local machine.
  • A self-hosted FastAPI server with a /generate endpoint that accepts prompts and returns marketing copy.
  • A browser-based frontend with temperature and top-k sampling controls for your model (Secret Mission).

Are there any prerequisites?

This project assumes intermediate Python and PyTorch experience. You should be comfortable working with tensors, nn.Module, and basic training loops. No prior experience with FastAPI is needed.

Before We Start

Before We Start...

Set Up Your Environment

Before you can build a GPT model from scratch, you need a workspace with GPU access and the right libraries.

Training a transformer model on a CPU would take hours. Google Colab gives you free access to a T4 GPU, which brings that training time down to about 3 minutes. In this step, you'll set up Colab, install your dependencies, and load the marketing dataset your model will learn from.

In This Step, Get Ready To:
  • Connect to a free T4 GPU in Google Colab.
  • Install and import all required Python libraries.
  • Load and prepare a marketing email dataset from Hugging Face.
Connect to a T4 GPU in Google Colab

Google Colab is a free cloud notebook environment that gives you access to GPUs for training deep learning models. By default, Colab connects to a CPU runtime. You need to switch to a GPU runtime so your model trains in minutes instead of hours.

  • Open Google Colab and create a new notebook.
  • Click **Runtime** in the top menu bar.
  • Select **Change runtime type**.
  • Set the hardware accelerator to **T4 GPU**, then click **Save**.
  • In the first code cell, paste the following code and run it to verify GPU access:
import torch
print(torch.cuda.is_available())

I see True

Your GPU is connected and ready to go.

I see False

Your notebook is still using a CPU runtime.

  • Go to **Runtime** > **Change runtime type** and make sure **T4 GPU** is selected.
  • Click **Save**, then re-run the cell.

Still Seeing False?

Colab's free GPU quota may be temporarily exhausted. Try again in a few minutes, or ask for help troubleshooting your Colab GPU connection.

Install and Import Your Libraries

Your model needs PyTorch for building and training the neural network, and the datasets library from Hugging Face for loading the marketing text data. PyTorch comes pre-installed in Colab, but datasets does not.

  • In a new code cell, install the datasets library:
!pip install datasets
  • In the next code cell, import all the libraries you'll use throughout the project:
import torch
import torch.nn as nn
from torch.nn import functional as F
from dataclasses import dataclass
from datasets import load_dataset

What Are These Libraries For?

  • torch is the core PyTorch library for tensor operations and GPU computation.
  • torch.nn provides neural network building blocks like linear layers and embeddings.
  • torch.nn.functional gives you functions like softmax and cross-entropy loss.
  • dataclass lets you define clean configuration objects for your model hyperparameters.
  • datasets loads datasets directly from Hugging Face with a single function call.
Load and Prepare the Marketing Dataset

Your model needs a large body of text to learn from. You'll use the marketeam/Marketing-Emails dataset from Hugging Face, which contains over 16,400 rows of synthetic marketing emails. This gives your model plenty of promotional language, calls-to-action, and product descriptions to learn from.

  • In a new code cell, load the dataset and concatenate the email text into a single string:
# Load the marketing emails dataset from Hugging Face
dataset = load_dataset("marketeam/Marketing-Emails", split="train")

# Concatenate all email text into one large string
text = "\n".join([row["text"] for row in dataset])

print(f"Total characters: {len(text):,}")
print(f"\nFirst 500 characters:\n{text[:500]}")

Column Name May Vary

The code above uses row["text"] to access the email content. If you get a KeyError, the column might have a different name. Run print(dataset.column_names) to see the available column names and update the key accordingly.

Why One Giant String?

Your model uses character-level tokenization, which means it learns patterns one character at a time. Concatenating all the emails into a single text string lets the model see continuous sequences of marketing language. In the next step, you'll build a tokenizer that converts this text into numbers your model can process.

You should see a character count in the millions and a preview of marketing email text with subject lines, greetings, and promotional language.

Your environment is ready and loaded with marketing data. Next up, you'll build a character-level tokenizer that turns this raw text into sequences of numbers your model can learn from.

Build the Tokenizer and Data Pipeline

Your model needs to work with numbers, not raw text. Before you can build any neural network layers, you need a way to convert marketing text into sequences of integers and back again. In this step, you'll build a character-level tokenizer from scratch and create a data pipeline that feeds training examples to your model.

In This Step, Get Ready To:
  • Build a character-level tokenizer with encode and decode functions.
  • Encode the full dataset and split it into training and validation sets.
  • Create a batch sampling function that generates input-target pairs for training.
Build a Character-Level Tokenizer

Large language models like ChatGPT use sophisticated tokenizers like BPE (Byte Pair Encoding) that break text into subword chunks. You're going to start simpler. A character-level tokenizer treats every individual character (letters, numbers, spaces, punctuation) as its own token. This means your vocabulary is small and you can see exactly how tokenization works from the ground up.

  • In a new code cell, extract all unique characters from your marketing text to build the vocabulary.
# Build the character-level vocabulary
chars = sorted(list(set(text)))
vocab_size = len(chars)
print(f"Vocabulary size: {vocab_size}")
print(f"Characters: {''.join(chars)}")

What does this code do?

The set(text) call extracts every unique character from your marketing dataset. Sorting them gives you a consistent ordering. This character list becomes your model's entire vocabulary. Every character gets a unique integer ID.

  • Create the mapping dictionaries and the encode() and decode() functions.
# Create character-to-index and index-to-character mappings
char_to_idx = {ch: i for i, ch in enumerate(chars)}
idx_to_char = {i: ch for i, ch in enumerate(chars)}

# Encode: convert a string into a list of integers
def encode(s):
    return [char_to_idx[c] for c in s]

# Decode: convert a list of integers back into a string
def decode(l):
    return ''.join([idx_to_char[i] for i in l])

# Test the tokenizer
test_text = "Buy now!"
encoded = encode(test_text)
print(f"Encoded: {encoded}")
print(f"Decoded: {decode(encoded)}")

What is a tokenizer?

A tokenizer converts raw text into numbers that a neural network can process. Your encode() function maps each character to its integer ID. Your decode() function reverses the process, turning model output back into readable text. These two functions are the bridge between human language and machine computation.

You should see the test text encoded as a list of integers, then decoded back to the original string. If the decoded output matches your input, your tokenizer is working correctly.

Encode the Dataset and Create Train/Val Splits

With your tokenizer ready, you need to convert the entire marketing text corpus into a tensor of token IDs. You also need to hold back some data for validation so you can measure whether your model is actually learning or just memorizing.

  • Encode the full dataset and split it into training (90%) and validation (10%) sets.
# Encode the entire text into a tensor of token IDs
data = torch.tensor(encode(text), dtype=torch.long)
print(f"Total tokens: {len(data):,}")

# Split into training (90%) and validation (10%)
n = int(0.9 * len(data))
train_data = data[:n]
val_data = data[n:]
print(f"Train tokens: {len(train_data):,}")
print(f"Val tokens: {len(val_data):,}")

Why split into training and validation?

The training set is what your model learns from. The validation set is data the model never sees during training. By measuring loss on both, you can tell if your model is genuinely learning language patterns or just memorizing the training data. If training loss drops but validation loss stays high, your model is overfitting.

Create the Batch Sampling Function

Neural networks learn faster when they process multiple examples at once. Instead of feeding your model one sequence at a time, you'll create a function that grabs random chunks of text in batches. Each training example is a sequence of block_size tokens (the input), paired with the same sequence shifted one position forward (the target). This is how the model learns to predict the next character.

  • Define your training hyperparameters and the get_batch() function.
# Training hyperparameters
batch_size = 64
block_size = 256
device = 'cuda' if torch.cuda.is_available() else 'cpu'

def get_batch(split):
    """Sample a random batch of input-target pairs."""
    data_split = train_data if split == 'train' else val_data
    # Pick random starting positions
    ix = torch.randint(len(data_split) - block_size, (batch_size,))
    # Input sequences
    x = torch.stack([data_split[i:i+block_size] for i in ix])
    # Target sequences (shifted by one position)
    y = torch.stack([data_split[i+1:i+block_size+1] for i in ix])
    x, y = x.to(device), y.to(device)
    return x, y

# Test the batch function
xb, yb = get_batch('train')
print(f"Input shape: {xb.shape}")
print(f"Target shape: {yb.shape}")

What does this code do?

The get_batch() function picks batch_size random positions in the dataset. From each position, it grabs a chunk of 256 characters as the input (x) and the next 256 characters as the target (y). The target is offset by one position. So if the input is "Buy no", the target is "uy now". This teaches the model to predict each next character given all the characters before it.

You should see both shapes printed as torch.Size([64, 256]). That means 64 sequences of 256 tokens each, ready for training.

Why character-level instead of BPE?

Production models like GPT-4 use BPE tokenization, which groups characters into subword chunks for much better text quality. Character-level tokenization produces lower quality output, but it eliminates external dependencies and lets you see every piece of the pipeline from scratch. You're building this to understand the architecture, not to compete with ChatGPT. The transformer layers you build next will work identically regardless of which tokenizer feeds them.

Your data pipeline is complete. You have a tokenizer that converts text to numbers and back, a dataset split for training and validation, and a batch function that serves up training examples. Next up, you'll build the core of the transformer: multi-head self-attention.

Implement Multi-Head Self-Attention

Your tokenizer and data pipeline are ready to feed sequences into a model. Now you need to build the core mechanism that makes transformers work: self-attention. Self-attention is how the model decides which tokens in a sequence are relevant to each other. In this step, you'll implement it from scratch and then scale it up with multiple attention heads running in parallel.

In This Step, Get Ready To:
  • Create a centralized configuration dataclass for all model hyperparameters.
  • Implement a single head of scaled dot-product self-attention with causal masking.
  • Build multi-head attention by running multiple heads in parallel and projecting their combined output.
Create the GPTConfig Dataclass

A transformer model has many hyperparameters: embedding size, number of layers, number of attention heads, context window length, and more. Scattering these values across your code makes them hard to track and easy to break. A dataclass bundles them into a single configuration object that every component can reference.

  • Add a new cell in your Colab notebook and define the GPTConfig dataclass.
from dataclasses import dataclass

@dataclass
class GPTConfig:
    block_size: int = 256       # Maximum context length (sequence length)
    vocab_size: int = vocab_size # Number of unique characters from our tokenizer
    n_layer: int = 6            # Number of transformer blocks stacked
    n_head: int = 6             # Number of parallel attention heads
    n_embd: int = 384           # Embedding dimension (must be divisible by n_head)
    dropout: float = 0.2        # Dropout rate for regularization

config = GPTConfig()
print(f"Config: {config}")

Why these specific numbers?

This configuration (6 layers, 6 heads, 384 embedding dimensions) produces a model with roughly 10 million parameters. It is large enough to learn meaningful patterns from marketing text, but small enough to train in about 3 minutes on a free Colab T4 GPU. The embedding dimension of 384 divides evenly by 6 heads, giving each head 64 dimensions to work with.

Implement a Single Attention Head

A single attention head is the atomic unit of the transformer's reasoning. For every token in a sequence, the head asks: "Which other tokens should I pay attention to?" It answers this question using three learned projections called query, key, and value. The query asks "what am I looking for?", the key says "what do I contain?", and the value holds the actual information to pass forward.

  • Add a new cell and implement the Head class.
import torch
import torch.nn as nn
import torch.nn.functional as F

class Head(nn.Module):
    """One head of self-attention."""

    def __init__(self, config, head_size):
        super().__init__()
        # Each projection maps from embedding dim to this head's dimension
        self.key = nn.Linear(config.n_embd, head_size, bias=False)
        self.query = nn.Linear(config.n_embd, head_size, bias=False)
        self.value = nn.Linear(config.n_embd, head_size, bias=False)
        # Causal mask: prevents attending to future tokens
        self.register_buffer('tril', torch.tril(torch.ones(config.block_size, config.block_size)))
        self.dropout = nn.Dropout(config.dropout)

    def forward(self, x):
        B, T, C = x.shape  # batch, sequence length, embedding dim
        k = self.key(x)     # (B, T, head_size)
        q = self.query(x)   # (B, T, head_size)
        # Compute attention scores ("affinities")
        wei = q @ k.transpose(-2, -1) * (C ** -0.5)  # (B, T, T), scaled by sqrt(d)
        wei = wei.masked_fill(self.tril[:T, :T] == 0, float('-inf'))  # Apply causal mask
        wei = F.softmax(wei, dim=-1)  # Normalize to probabilities
        wei = self.dropout(wei)
        # Weighted aggregation of values
        v = self.value(x)   # (B, T, head_size)
        out = wei @ v       # (B, T, head_size)
        return out

What is the causal mask?

The torch.tril call creates a lower-triangular matrix of ones. When you mask the upper triangle with -inf, those positions become zero after softmax. This prevents token 3 from "seeing" tokens 4, 5, 6, and so on. Without this mask, the model would cheat by looking at future tokens it is supposed to predict.

Why scale by the square root of the dimension?

The C ** -0.5 term is the "scaled" part of scaled dot-product attention. Without it, the dot products between queries and keys grow large as the dimension increases. Large values push softmax into regions where gradients are extremely small, making learning slow or unstable. Dividing by the square root keeps the values in a healthy range.

Build Multi-Head Attention

A single attention head can only focus on one type of relationship at a time. Multi-head attention runs several heads in parallel, each learning to attend to different patterns. One head might learn to focus on nearby tokens, while another learns long-range dependencies. Their outputs are concatenated and projected back to the full embedding dimension.

  • Add a new cell and implement the MultiHeadAttention class.
class MultiHeadAttention(nn.Module):
    """Multiple heads of self-attention running in parallel."""

    def __init__(self, config):
        super().__init__()
        head_size = config.n_embd // config.n_head  # 384 / 6 = 64 per head
        # Create n_head independent attention heads
        self.heads = nn.ModuleList([Head(config, head_size) for _ in range(config.n_head)])
        # Project concatenated head outputs back to embedding dimension
        self.proj = nn.Linear(config.n_embd, config.n_embd)
        self.dropout = nn.Dropout(config.dropout)

    def forward(self, x):
        # Run all heads in parallel and concatenate along the last dimension
        out = torch.cat([h(x) for h in self.heads], dim=-1)  # (B, T, n_embd)
        out = self.dropout(self.proj(out))  # Project and apply dropout
        return out

What does this code do?

Each of the 6 heads produces a 64-dimensional output (384 / 6 = 64). Concatenating all 6 gives you a 384-dimensional vector, matching the original embedding size. The final linear projection layer lets the model learn how to combine information from all the heads. This is the pattern used in every transformer, from GPT-2 to the largest models.

  • Verify the output shape by adding a quick test in a new cell.
# Quick shape verification
test_mha = MultiHeadAttention(config)
test_input = torch.randn(4, config.block_size, config.n_embd)  # (batch=4, seq=256, embd=384)
test_output = test_mha(test_input)
print(f"Input shape:  {test_input.shape}")
print(f"Output shape: {test_output.shape}")
assert test_output.shape == test_input.shape, "Shape mismatch!"
print("Multi-head attention output shape is correct!")

What should I see?

Both the input and output shapes should be torch.Size([4, 256, 384]). This confirms that multi-head attention takes in a sequence of embeddings and produces an output of the exact same shape. The information has been transformed through attention, but the dimensions are preserved so it can be fed into the next layer.

You now have the most important piece of the transformer architecture working. In the next step, you'll wrap this multi-head attention inside a full transformer block with feed-forward layers and residual connections, then stack multiple blocks to build the complete GPT model.

Build the Full GPT Model

You have the attention mechanism working. Now it is time to assemble the complete GPT model around it. A transformer is more than just attention. Each layer also needs a feed-forward network to process what attention found, plus residual connections and normalization to keep training stable. By the end of this step, you will have a fully functional language model that can accept token sequences, compute predictions, and generate new text.

In This Step, Get Ready To:
  • Build the feed-forward network and full transformer block with residual connections.
  • Assemble the complete GPT model with embeddings, transformer layers, and an output head.
  • Implement autoregressive text generation and verify your model has ~10M parameters.
Create the FeedForward and TransformerBlock Modules

Attention lets each token look at other tokens in the sequence. But the model also needs to think about what it found. That is the job of the feed-forward network. It is a small two-layer neural network that processes each token's representation independently after attention has mixed information across positions.

A transformer block combines multi-head attention and a feed-forward network into a single repeatable unit. The model stacks multiple blocks to build deeper understanding. Two critical design patterns make this work: residual connections (adding the input back to the output so gradients can flow) and layer normalization (stabilizing the values flowing through the network).

  • Add the following code to your Colab notebook in a new cell:
class FeedForward(nn.Module):
    """A simple two-layer feed-forward network with GELU activation."""

    def __init__(self, config):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(config.n_embd, 4 * config.n_embd),
            nn.GELU(),
            nn.Linear(4 * config.n_embd, config.n_embd),
            nn.Dropout(config.dropout),
        )

    def forward(self, x):
        return self.net(x)

class TransformerBlock(nn.Module):
    """One transformer block: attention + feed-forward with pre-norm residuals."""

    def __init__(self, config):
        super().__init__()
        self.sa = MultiHeadAttention(config)
        self.ffwd = FeedForward(config)
        self.ln1 = nn.LayerNorm(config.n_embd)
        self.ln2 = nn.LayerNorm(config.n_embd)

    def forward(self, x):
        # Residual connection: add input back to attention output
        x = x + self.sa(self.ln1(x))
        # Residual connection: add input back to feed-forward output
        x = x + self.ffwd(self.ln2(x))
        return x

What Does This Code Do?

The FeedForward module expands the embedding dimension by 4x, applies a GELU activation, then projects back down. This expansion gives the network more room to learn complex patterns before compressing back to the original size.

The TransformerBlock uses a pre-norm architecture. Layer normalization is applied before each sub-layer (attention and feed-forward), not after. This is the pattern used by modern GPT models because it leads to more stable training.

What Are Residual Connections?

Notice the pattern x = x + self.sa(self.ln1(x)). Instead of replacing x with the attention output, the code adds the attention output to the original input. This creates a shortcut path for gradients during backpropagation. Without residual connections, deep networks (6+ layers) struggle to train because gradients vanish as they flow backward through many layers.

Assemble the Complete GPT Model

Now you have every building block: attention heads, multi-head attention, feed-forward networks, and transformer blocks. The full GPT model wires them together with an embedding layer at the input and a prediction head at the output.

  • Add the GPT model class in a new cell:
class GPT(nn.Module):
    """The full GPT language model."""

    def __init__(self, config):
        super().__init__()
        self.config = config

        # Token embedding: converts token IDs to vectors
        self.token_embedding = nn.Embedding(config.vocab_size, config.n_embd)
        # Position embedding: encodes where each token sits in the sequence
        self.position_embedding = nn.Embedding(config.block_size, config.n_embd)
        # Stack of transformer blocks
        self.blocks = nn.Sequential(*[TransformerBlock(config) for _ in range(config.n_layer)])
        # Final layer norm
        self.ln_f = nn.LayerNorm(config.n_embd)
        # Output head: maps embeddings back to vocabulary logits
        self.lm_head = nn.Linear(config.n_embd, config.vocab_size)

    def forward(self, idx, targets=None):
        B, T = idx.shape

        # Create token and position embeddings, then add them together
        tok_emb = self.token_embedding(idx)  # (B, T, n_embd)
        pos_emb = self.position_embedding(torch.arange(T, device=idx.device))  # (T, n_embd)
        x = tok_emb + pos_emb  # (B, T, n_embd)

        # Pass through all transformer blocks
        x = self.blocks(x)  # (B, T, n_embd)
        x = self.ln_f(x)  # (B, T, n_embd)

        # Project to vocabulary size to get prediction scores
        logits = self.lm_head(x)  # (B, T, vocab_size)

        # Compute loss if targets are provided (during training)
        loss = None
        if targets is not None:
            B, T, C = logits.shape
            logits_flat = logits.view(B * T, C)
            targets_flat = targets.view(B * T)
            loss = F.cross_entropy(logits_flat, targets_flat)

        return logits, loss

What Does the Forward Method Do?

The forward() method is the core data flow of the model. It takes a batch of token ID sequences and processes them through three stages:

  • Embedding: Token IDs become vectors, and positional information is added so the model knows word order.
  • Transformer blocks: The stacked blocks apply attention and feed-forward processing to build understanding.
  • Output head: The final linear layer maps each position back to a probability distribution over the vocabulary, predicting the next character.

What Is Cross-Entropy Loss?

Cross-entropy loss measures how far the model's predicted probability distribution is from the actual next character. If the model is confident about the correct character, the loss is low. If it spreads probability across many wrong characters, the loss is high. During training, the optimizer adjusts weights to minimize this loss.

Implement Autoregressive Generation and Count Parameters

The model can now process sequences and compute loss. But to actually generate text, it needs a generate() method. Autoregressive generation works by predicting one token at a time, appending it to the input, and repeating. Each new prediction uses all previously generated tokens as context.

  • Add the generate() method to your GPT class. You will need to redefine the entire class with this method included:
class GPT(nn.Module):
    """The full GPT language model."""

    def __init__(self, config):
        super().__init__()
        self.config = config
        self.token_embedding = nn.Embedding(config.vocab_size, config.n_embd)
        self.position_embedding = nn.Embedding(config.block_size, config.n_embd)
        self.blocks = nn.Sequential(*[TransformerBlock(config) for _ in range(config.n_layer)])
        self.ln_f = nn.LayerNorm(config.n_embd)
        self.lm_head = nn.Linear(config.n_embd, config.vocab_size)

    def forward(self, idx, targets=None):
        B, T = idx.shape
        tok_emb = self.token_embedding(idx)
        pos_emb = self.position_embedding(torch.arange(T, device=idx.device))
        x = tok_emb + pos_emb
        x = self.blocks(x)
        x = self.ln_f(x)
        logits = self.lm_head(x)

        loss = None
        if targets is not None:
            B, T, C = logits.shape
            logits_flat = logits.view(B * T, C)
            targets_flat = targets.view(B * T)
            loss = F.cross_entropy(logits_flat, targets_flat)

        return logits, loss

    def generate(self, idx, max_new_tokens):
        """Generate new tokens autoregressively."""
        for _ in range(max_new_tokens):
            # Crop input to the last block_size tokens (context window limit)
            idx_cond = idx[:, -self.config.block_size:]
            # Get predictions
            logits, _ = self(idx_cond)
            # Focus on the last time step's predictions
            logits = logits[:, -1, :]  # (B, vocab_size)
            # Convert to probabilities
            probs = F.softmax(logits, dim=-1)  # (B, vocab_size)
            # Sample from the distribution
            idx_next = torch.multinomial(probs, num_samples=1)  # (B, 1)
            # Append the new token to the sequence
            idx = torch.cat((idx, idx_next), dim=1)  # (B, T+1)
        return idx

How Does Autoregressive Generation Work?

The generate() method runs a loop where each iteration predicts one new character. It crops the input to the last block_size (256) tokens because the model's positional embeddings only cover that many positions. It then takes the logits from the very last position, converts them to probabilities with softmax, and samples a character. That character gets appended, and the loop repeats.

Now instantiate the model and check its size.

  • Run the following code in a new cell to create the model and count its parameters:
# Create the model and move it to GPU
model = GPT(config)
model = model.to('cuda')

# Count total parameters
total_params = sum(p.numel() for p in model.parameters())
print(f"Model parameters: {total_params / 1e6:.2f}M")

What Should I See?

You should see output showing approximately 10M parameters (the exact number depends on your vocabulary size). This is a small model by modern standards. GPT-2 has 124M parameters and GPT-4 is estimated to have over a trillion. But 10M is enough to learn real language patterns from your marketing dataset.

  • Test that the untrained model can generate text (it will be random gibberish at this point):
# Generate from the untrained model (expect random characters)
context = torch.zeros((1, 1), dtype=torch.long, device='cuda')
generated = model.generate(context, max_new_tokens=100)
print(decode(generated[0].tolist()))

The output will be completely random characters. That is expected. The model has not been trained yet, so it is sampling from a uniform distribution over the vocabulary. After training in the next step, this same generate() method will produce coherent marketing text.

Your GPT model is fully assembled and ready to learn. Next up, you will train it on your marketing email dataset and watch it go from random gibberish to generating actual marketing copy.

Train Your Marketing GPT

Your GPT model is fully assembled with every transformer component wired together. Now it is time to teach it how marketing language works. In this step, you will set up the training loop, watch the model learn from thousands of marketing emails, and generate your first samples of AI-written marketing copy.

In This Step, Get Ready To:
  • Configure the training loop with AdamW optimizer and loss evaluation.
  • Train the model for 5,000 iterations on a T4 GPU and monitor loss convergence.
  • Generate marketing text samples and export the trained checkpoint for local serving.
Set Up the Training Configuration

Before running the training loop, you need to define how long to train, how often to evaluate, and which optimizer to use. You also need a function that estimates loss on both the training and validation splits so you can track whether the model is actually learning or just memorizing.

  • Add a new cell in your Colab notebook with the following training configuration and evaluation function.
# Training hyperparameters
max_iters = 5000
eval_interval = 500
eval_iters = 200
learning_rate = 3e-4
device = 'cuda' if torch.cuda.is_available() else 'cpu'

# Move model to GPU
model = GPT(config).to(device)
optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)

# Print parameter count
print(f"Model has {sum(p.numel() for p in model.parameters()) / 1e6:.2f}M parameters")

@torch.no_grad()
def estimate_loss():
    """Evaluate average loss on train and val splits."""
    out = {}
    model.eval()
    for split in ['train', 'val']:
        losses = torch.zeros(eval_iters)
        for k in range(eval_iters):
            X, Y = get_batch(split)
            logits, loss = model(X, Y)
            losses[k] = loss.item()
        out[split] = losses.mean()
    model.train()
    return out

What does this code do?

This sets up the core training parameters. The model trains for 5,000 iterations with a learning rate of 3e-4 using the AdamW optimizer, which is the standard choice for training transformers.

The estimate_loss() function averages loss over 200 random batches from each split. This gives a stable loss estimate instead of relying on a single noisy batch. The @torch.no_grad() decorator disables gradient computation during evaluation, saving memory and speeding things up.

You should see the parameter count print as approximately 10M parameters. This confirms your model architecture is wired correctly with the config you defined earlier.

Run the Training Loop

This is where the model actually learns. Each iteration samples a random batch of sequences, computes how wrong the model's predictions are (the loss), and adjusts the weights to reduce that error. Over thousands of iterations, the model gradually picks up marketing language patterns.

  • Add a new cell and run the training loop.
for iter in range(max_iters):

    # Evaluate loss on train and val sets periodically
    if iter % eval_interval == 0 or iter == max_iters - 1:
        losses = estimate_loss()
        print(f"step {iter}: train loss {losses['train']:.4f}, val loss {losses['val']:.4f}")

    # Sample a batch of training data
    xb, yb = get_batch('train')

    # Forward pass: compute predictions and loss
    logits, loss = model(xb, yb)

    # Backward pass: compute gradients
    optimizer.zero_grad(set_to_none=True)
    loss.backward()

    # Update weights
    optimizer.step()

What does this training loop do?

Each iteration follows the standard deep learning cycle. First, sample a random batch of input-target pairs. Then run the forward pass to get predictions and compute cross-entropy loss. Next, backpropagate to compute gradients. Finally, the optimizer updates the model weights to reduce the loss.

Every 500 steps, the loop pauses to evaluate loss on both splits. This tells you if the model is learning (train loss drops) and if it is generalizing (val loss drops too, not just memorizing).

What should I see?

Training takes approximately 3 minutes on a T4 GPU. You should see output like this:

step 0: train loss 4.2341, val loss 4.2298
step 500: train loss 2.1045, val loss 2.1523
step 1000: train loss 1.8234, val loss 1.8891
step 1500: train loss 1.6712, val loss 1.7345
...
step 4999: train loss 1.4523, val loss 1.5102

The loss starts around 4.0 (the model is guessing randomly) and drops toward 1.5 as it learns character patterns in marketing text. Your exact numbers will differ slightly, but the downward trend is what matters.

Loss not decreasing?

If your loss stays flat or increases, make sure you connected to a GPU runtime (Runtime > Change runtime type > T4 GPU). Also verify that your get_batch() function moves tensors to the correct device. Help me debug my training loop.

Generate Samples and Export the Checkpoint

The real test of your model is whether it can generate text that looks like marketing copy. You will also save the trained model checkpoint with everything needed to reconstruct it on a different machine.

  • Add a new cell to generate sample text from your trained model.
# Generate a sample from the trained model
context = torch.zeros((1, 1), dtype=torch.long, device=device)
generated = model.generate(context, max_new_tokens=500)
print(decode(generated[0].tolist()))

What does this code do?

This creates a starting context of a single zero token (which acts as a blank prompt) and asks the model to generate 500 new tokens autoregressively. Each token is predicted based on all the tokens that came before it. The decode() function converts the integer token IDs back into readable characters.

You should see generated text that resembles marketing language. It will not be perfect since this is a small character-level model, but you should recognize marketing patterns like promotional phrases, product descriptions, and calls to action.

Now save the checkpoint with everything needed to load the model on your local machine.

  • Add a new cell to save and download the checkpoint.
import pickle
from google.colab import files

# Save everything needed to reconstruct the model
checkpoint = {
    'model_state_dict': model.state_dict(),
    'config': config,
    'chars': chars,
    'char_to_idx': char_to_idx,
    'idx_to_char': idx_to_char,
}

torch.save(checkpoint, 'marketing_gpt.pth')
print("Checkpoint saved!")

# Download to your local machine
files.download('marketing_gpt.pth')

Why save the tokenizer with the model?

The checkpoint bundles the model weights, the config (architecture hyperparameters), and the full tokenizer vocabulary (chars, char_to_idx, idx_to_char). Without the tokenizer mappings, your local server would not know how to convert text into the same token IDs the model was trained on. Everything travels together in one file.

What should I see?

Your browser should prompt you to download a file called marketing_gpt.pth. This file will be around 40-50 MB depending on exact vocabulary size. Save it somewhere you can easily find it, like your Desktop or a project folder. You will need it in the next step.

Download not starting?

If files.download() does not trigger a download, you can alternatively save to Google Drive and download from there. Mount your Drive with from google.colab import drive; drive.mount('/content/drive') and then copy the file with !cp marketing_gpt.pth /content/drive/MyDrive/. Help me download my checkpoint from Colab.

Your model is trained and your checkpoint is downloaded. Next up, you will build a FastAPI server on your local machine that loads this checkpoint and serves your marketing GPT as a REST API.

Self-Host Your Model with FastAPI

Your model is trained and the checkpoint is sitting on your local machine. Right now, the only way to use it is by running Python scripts manually. In this step, you'll wrap your model in a FastAPI REST API so anyone (or any application) can send a prompt and get generated marketing text back over HTTP.

In This Step, Get Ready To:
  • Set up FastAPI and Uvicorn on your local machine.
  • Build a server that loads your trained checkpoint and exposes a generation endpoint.
  • Test your self-hosted marketing text generator with real prompts.
Install FastAPI and Create Your Server File

FastAPI is a modern Python web framework designed for building APIs. Uvicorn is the ASGI server that actually runs your FastAPI application and handles incoming requests.

  • Open a terminal on your local machine.
  • Install both packages.
pip install fastapi uvicorn[standard]

Why FastAPI Instead of Flask?

FastAPI auto-generates interactive API documentation at /docs with zero extra setup. It also uses Pydantic for request validation, catching malformed inputs before they reach your model. On identical hardware, FastAPI handles 15,000-20,000 requests per second compared to Flask's 2,000-3,000.

  • Create a new file called server.py in the same directory as your downloaded checkpoint file.
  • Copy your model class definitions into server.py. You need every class the model depends on: GPTConfig, Head, MultiHeadAttention, FeedForward, TransformerBlock, and GPT.
  • Also copy your encode() and decode() tokenizer functions into the file. These need the vocabulary mappings to convert between text and token IDs.

Why Copy Everything Into One File?

When PyTorch loads a checkpoint with torch.load(), it needs the exact class definitions that were used to create the model. Your checkpoint only stores the learned weights, not the code. Without the matching class definitions in scope, Python cannot reconstruct the model object.

At the top of server.py, add the imports your model classes and the FastAPI server will need.

import torch
import torch.nn as nn
import torch.nn.functional as F
from dataclasses import dataclass
from contextlib import asynccontextmanager
from fastapi import FastAPI
from pydantic import BaseModel

What Does This Import Block Do?

The first four imports are the same ones your model classes depend on from Colab. The last three are new: asynccontextmanager handles model loading at server startup, FastAPI is the web framework, and BaseModel from Pydantic validates incoming request data.

  • After the imports, paste your full model class definitions (GPTConfig, Head, MultiHeadAttention, FeedForward, TransformerBlock, GPT) and your encode()/decode() functions from your Colab notebook.

These should be identical to what you wrote during training.

Add the FastAPI Application Code

Your server needs to do three things: load the model checkpoint when it starts up, accept prompts from users, and return generated text. FastAPI's lifespan context manager pattern handles the startup logic. A Pydantic model defines what a valid request looks like.

  • Add the following code below your model class definitions and tokenizer functions in server.py.
# Global variable to hold the loaded model
model = None

@asynccontextmanager
async def lifespan(app: FastAPI):
    """Load the model checkpoint once when the server starts."""
    global model

    # Load the saved checkpoint (weights + config + tokenizer data)
    checkpoint = torch.load('marketing_gpt.pth', map_location='cpu', weights_only=False)

    # Reconstruct the model from the saved config
    config = checkpoint['config']
    model = GPT(config)
    model.load_state_dict(checkpoint['model_state_dict'])
    model.eval()  # Set to evaluation mode (disables dropout)

    # Rebuild the tokenizer mappings from the checkpoint
    global chars, char_to_idx, idx_to_char
    chars = checkpoint['chars']
    char_to_idx = checkpoint['char_to_idx']
    idx_to_char = checkpoint['idx_to_char']

    print(f"Model loaded with {sum(p.numel() for p in model.parameters())} parameters")
    yield  # Server runs here

# Create the FastAPI app with the lifespan handler
app = FastAPI(title="Marketing GPT", lifespan=lifespan)

# Define the expected shape of incoming requests
class GenerateRequest(BaseModel):
    prompt: str
    max_tokens: int = 200

@app.post("/generate")
async def generate_text(request: GenerateRequest):
    """Accept a prompt and return generated marketing text."""
    # Encode the prompt into token IDs
    input_ids = torch.tensor([encode(request.prompt)], dtype=torch.long)

    # Generate new tokens autoregressively
    with torch.no_grad():
        output_ids = model.generate(input_ids, max_new_tokens=request.max_tokens)

    # Decode the full output (prompt + generated tokens) back to text
    generated_text = decode(output_ids[0].tolist())

    return {"generated_text": generated_text}

What Does the Lifespan Pattern Do?

The lifespan function runs once when the server starts (everything before yield) and once when it shuts down (everything after yield). This means the model loads into memory a single time, not on every request. Without this pattern, every API call would reload the checkpoint from disk, which would be painfully slow.

Why map_location='cpu'?

Your model was trained on a GPU in Colab, so the checkpoint's tensors are tagged as CUDA tensors. Your local machine likely does not have a CUDA GPU. The map_location='cpu' argument tells PyTorch to remap all tensors to CPU memory during loading. A 10M parameter model runs inference on CPU without any noticeable delay.

Start the Server and Test Your API
  • Make sure your marketing_gpt.pth checkpoint file is in the same directory as server.py.
  • Start the server.
uvicorn server:app --reload

What Does This Command Do?

This tells Uvicorn to look for a FastAPI instance called app inside the file server.py. The --reload flag automatically restarts the server whenever you save changes to the file.

What Should I See?

You should see Uvicorn print a startup message including the model parameter count, followed by a line like Uvicorn running on http://127.0.0.1:8000. This means your model is loaded and the server is ready to accept requests.

Now test it. Open a second terminal window (keep the server running in the first one) and send a request to your API.

macOS

curl -X POST http://localhost:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Introducing our new", "max_tokens": 200}'

Windows

Invoke-RestMethod -Method Post -Uri http://localhost:8000/generate `
  -ContentType "application/json" `
  -Body '{"prompt": "Introducing our new", "max_tokens": 200}'

You should see a JSON response containing a generated_text field with your prompt followed by generated marketing-style text. The output is character-level, so it may not be perfect English. You should still see recognizable marketing patterns like promotional language, product descriptions, and calls to action.

  • Open your browser and navigate to http://localhost:8000/docs.

This is the auto-generated Swagger UI that FastAPI creates for free. You can see your /generate endpoint documented with its request schema. You can even test it directly from the browser by clicking Try it out.

Getting a Connection Refused Error?

Make sure the Uvicorn server is still running in your first terminal window. Also verify that your checkpoint file marketing_gpt.pth is in the same directory where you ran the uvicorn command. If the server crashed on startup, check the terminal output for error messages. Help me debug my FastAPI server startup error.

You now have a fully self-hosted marketing text generator. A model you built from scratch, trained on real marketing data, and served over HTTP. Next up, you'll push this even further in the Secret Mission with sampling controls and a browser frontend.

Secret mission

Add Sampling Controls and a Web Frontend

Your model generates text, but you have no control over how creative or focused it is. In this secret mission, you will add temperature and top-k sampling parameters to your API. Then you will build a simple HTML frontend so you can interact with your marketing GPT directly in the browser.

Before you go

Clean Up Your Resources

This project runs entirely on Google Colab's free GPU tier and your local machine, so there are no ongoing cloud costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Google Colab notebook with your GPT model training code.
  • Model checkpoint file (marketing_gpt.pth) on your local machine.
  • FastAPI server file (server.py) on your local machine.
  • Locally installed Python packages: fastapi and uvicorn.
  • Model checkpoint copy saved to Google Drive (if you saved there during training).

Keep everything running

No action needed. Choose this if you're still actively experimenting with your model or want to keep generating marketing text.

  • Your FastAPI server stays running at http://localhost:8000.
  • Your Colab notebook remains accessible in your Google Drive.
  • Your checkpoint file and server code stay on your local machine.

Pause - I'll come back to this later

Stop the running server to free up your terminal, but keep all files so you can restart anytime.

  • Press Ctrl+C in the terminal where Uvicorn is running to stop the server.
  • Keep your server.py and marketing_gpt.pth files on your local machine.
  • To restart later, navigate to your project folder and run uvicorn server:app --reload.
  • Your Colab notebook will stay in your Google Drive indefinitely.

Delete - I don't want to use this again

Remove all project resources and start fresh. Since everything is local, this is straightforward.

  • Press Ctrl+C to stop the FastAPI server if it is still running.
  • Delete the server.py file and marketing_gpt.pth checkpoint from your local machine.
  • If you saved the checkpoint to Google Drive, delete it from there as well.
  • Delete the Colab notebook from your Google Drive if you no longer need it.
  • Optionally uninstall fastapi and uvicorn by running pip uninstall fastapi uvicorn.

Mission accomplished

That's a Wrap!

You just built a GPT-style language model from absolute scratch and deployed it as a self-hosted API. Every single component, from the tokenizer to the attention mechanism to the REST endpoint, was written by your hands.

Here's what you accomplished:

  • Implemented multi-head self-attention, transformer blocks, and a complete ~10M parameter GPT model from scratch in PyTorch.
  • Trained your model on real marketing email data using Google Colab's free T4 GPU, watching the loss drop from ~4.0 down toward ~1.5.
  • Exported a trained checkpoint and served it locally via a FastAPI REST API that generates marketing copy on demand from a /generate endpoint.
  • Added temperature and top-k sampling controls to your API and built a browser-based frontend to interact with your model.

Ready to quiz yourself?