Fine-Tune an AI Customer Support Model

Fine-tune Llama 3.2 on real customer support data using LoRA on a free GPU.

Introduction

30 Second Summary

AI-powered customer support is everywhere, from your bank to your favorite online store. The difference between a helpful response and a frustrating one often comes down to how the AI model behind it was trained.

In this project, you will fine-tune Meta's Llama 3.2 3B model on thousands of real customer support conversations using a technique called LoRA. You'll run the entire workflow inside Google Colab on a free GPU, turning a general-purpose AI into a specialized customer support assistant.

What You'll Build

You'll ask your fine-tuned model a customer support question and see it produce a detailed, domain-specific answer that makes the base model's generic response look like a first draft.

By the end of this project, you'll have:

  • A fine-tuned AI model that handles customer support conversations across 27 different intents, from order cancellations to refund requests.
  • Hands-on experience with the full LoRA fine-tuning workflow: loading a model in 4-bit quantization, preparing real training data, and running training on a free GPU.
  • A portable LoRA adapter (~100MB) you can reload or share, proving you trained a real model end to end.
  • Secret Mission: Publish your model to Hugging Face Hub with a model card so anyone can discover and use your customer support AI.

Are there any prerequisites?

This project assumes basic Python knowledge and familiarity with AI concepts. Everything runs in Google Colab through your browser, so you just need a Google account to get started.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up Google Colab with a Free GPU

Fine-tuning a 3-billion-parameter model requires a GPU with enough video memory to hold the model's weights and gradients during training.

Google Colab gives you a free T4 GPU with roughly 15GB of VRAM. In this step, you'll create a Colab notebook, connect to the GPU, and install the Unsloth library that makes training 2x faster and uses 58% less memory.

In this step, get ready to:
  • Create a new Colab notebook with a T4 GPU runtime.
  • Install Unsloth and its dependencies.
  • Verify the GPU is available and import the libraries you'll need.
Create a new Colab notebook and enable the T4 GPU

Google Colab is a free, browser-based notebook environment that gives you access to cloud GPUs. You need to create a notebook and select the T4 GPU runtime before you can install anything.

  • Open your browser and go to colab.research.google.com.
  • Sign in with your Google account if prompted.
  • Click New notebook to create a blank notebook.
  • Click Runtime in the top menu bar.
  • Click Change runtime type.
  • Under Hardware accelerator, select T4 GPU.
  • Click Save.
  • Click Connect in the top right corner of the notebook.

After a few seconds, the Connect button changes to a green checkmark with RAM and Disk usage bars. This means your notebook is connected to a GPU-powered virtual machine.

Can't see the T4 GPU option?

If you don't see T4 GPU in the dropdown, make sure you are signed into a personal Google account. Some workplace or school accounts restrict Colab GPU access.

If the runtime fails to connect, Colab's free GPUs may be temporarily at capacity. Wait a few minutes and try clicking Connect again.

Help me troubleshoot my Colab GPU connection

Install Unsloth and dependencies

Unsloth optimizes the fine-tuning process so it runs faster and uses less memory. The T4 GPU needs a specific install path because it does not support Flash Attention, a speed optimization available only on newer GPUs.

  • Click on the first empty code cell in your notebook and paste the following installation code:
%%capture
import torch
major_version, minor_version = torch.cuda.get_device_capability()
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
if major_version >= 8:
    !pip install --no-deps packaging ninja einops "flash-attn>=2.6.3" xformers trl peft accelerate bitsandbytes
else:
    !pip install --no-deps xformers trl peft accelerate bitsandbytes

What does this code do?

%%capture at the top silences all install output so your notebook stays clean. The cell still runs. You will see a completion indicator (a checkmark or elapsed time) when it finishes.

torch.cuda.get_device_capability() checks what GPU hardware you have. The T4 has compute capability 7.5 (below 8), so the code takes the else branch and skips Flash Attention, which only works on newer GPUs.

The --no-deps flag installs each package without pulling in extra dependencies, avoiding version conflicts inside Colab's pre-installed environment.

  • Run the cell by clicking the play button on its left side, or press Shift+Enter.
  • Wait for the cell to finish. Because of %%capture, no text output appears. The cell is done when the spinning icon stops and you see a green checkmark or an elapsed time indicator to the left of the cell.

Installation takes 2-3 minutes

The cell downloads and installs several packages from the internet. This is normal. If it takes longer than 5 minutes, your Colab session may have disconnected. Click Connect in the top right corner and re-run the cell.

Verify the GPU and import Unsloth

Now that everything is installed, you need to confirm two things: the T4 GPU is actually connected, and the Unsloth library imports without errors.

Before you run the next cell, what GPU name do you expect to see in the output?

  • Click + Code to create a new cell below your install cell, then verify your GPU by pasting and running this:
!nvidia-smi

You should see a table listing Tesla T4 as the GPU name, with roughly 15GB of memory available.

Don't see a Tesla T4?

If you see a different GPU name (like A100 or L4), that is fine. Those GPUs are more powerful and the project will work the same way.

If you see an error or no GPU listed, go back to Runtime > Change runtime type and confirm T4 GPU is selected. Then click Connect again and re-run the cell.

Help me fix my Colab GPU setup

  • Create another new cell below by clicking + Code, then import all the libraries you'll use throughout the project by pasting and running this:
from unsloth import FastLanguageModel
from transformers import TextStreamer
from datasets import load_dataset
from trl import SFTTrainer, SFTConfig
import torch

What do these libraries do?

  • FastLanguageModel is Unsloth's main interface for loading models, applying LoRA adapters, and running inference.
  • TextStreamer streams model output token by token so you can watch responses appear in real time.
  • load_dataset loads training data directly from Hugging Face.
  • SFTTrainer and SFTConfig handle the supervised fine-tuning training loop and its configuration.
  • torch is PyTorch, the deep learning framework that powers everything underneath.

If the cell runs with no errors, all five libraries imported successfully. You may see an Unsloth banner with your GPU info printed to the output.

Seeing an import error?

If you see a ModuleNotFoundError, the install cell may not have finished successfully. Go back to your first cell (the install cell) and re-run it. Wait for the green checkmark, then re-run this import cell.

If the error persists, try restarting the runtime by clicking Runtime > Restart runtime, then re-run both cells in order (install first, then imports).

Help me fix my Unsloth import error

✔️ Awesome, I've got everything!

Great. Double check you have three cells in your notebook, in order: install, GPU verify, and imports.

ⓧ I'd like to double check the full code

Your first cell (install):

%%capture
import torch
major_version, minor_version = torch.cuda.get_device_capability()
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
if major_version >= 8:
    !pip install --no-deps packaging ninja einops "flash-attn>=2.6.3" xformers trl peft accelerate bitsandbytes
else:
    !pip install --no-deps xformers trl peft accelerate bitsandbytes

Your second cell (GPU verify):

!nvidia-smi

Your third cell (imports):

from unsloth import FastLanguageModel
from transformers import TextStreamer
from datasets import load_dataset
from trl import SFTTrainer, SFTConfig
import torch

Your GPU environment is ready and all libraries are loaded. Next up, you'll load the Llama 3.2 3B model into this notebook and see how it handles a customer support question before any fine-tuning.

Load the Base Model and Test It

Your Colab notebook is powered up with a T4 GPU and Unsloth is ready to go. Now it's time to meet the model you'll be fine-tuning.

Loading a 3-billion-parameter model normally requires massive amounts of memory. 4-bit quantization compresses the model's weights so it fits in roughly 4GB of VRAM on your free GPU. Before you start training, you'll test the base model on a customer support question to see exactly where it falls short.

In this step, get ready to:
  • Load the Llama 3.2 3B Instruct model with 4-bit quantization.
  • Test the base model with a customer support question to see its current capabilities.
Load the Llama 3.2 3B Instruct model with 4-bit quantization

The model you'll fine-tune is Meta's Llama 3.2 3B Instruct. Unsloth hosts a pre-optimized version on Hugging Face that supports 4-bit quantization out of the box.

  • Create a new code cell in your notebook by clicking the + Code button in the toolbar.
  • Load the base model by pasting this code into the cell and pressing Shift+Enter to run it:
# Load the base model with 4-bit quantization
max_seq_length = 2048

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Llama-3.2-3B-Instruct",
    max_seq_length=max_seq_length,
    load_in_4bit=True,
)

What does this code do?

max_seq_length = 2048 sets the maximum number of tokens the model can process in a single pass. This keeps memory usage predictable on the free T4 GPU.

FastLanguageModel.from_pretrained downloads the Llama 3.2 3B Instruct model from Hugging Face and loads it into GPU memory. The load_in_4bit=True flag compresses each weight from 16 bits down to 4 bits, shrinking the model from over 6GB to roughly 4GB so it fits comfortably on the T4.

  • Wait for the cell to finish running. This takes about 30-60 seconds as the model downloads and loads into GPU memory.
  • You should see output logs showing the model loading, ending with the model and tokenizer being ready.

Seeing an out-of-memory error?

Make sure you selected the T4 GPU runtime in Step 1. Go to Runtime then Change runtime type and confirm T4 GPU is selected. If you already ran other cells that consumed GPU memory, click Runtime then Restart session and re-run all cells from the top.

Still stuck? Help me fix my out-of-memory error when loading the model in Colab.

Test the base model with a customer support question

The model is loaded, but it has never seen a customer support conversation in training. Before you fine-tune it, you need to see what a general-purpose model produces when asked a support question.

Before you run the next cell, think about this: what kind of response would you expect from a general-purpose AI when asked "I need help canceling my order"?

  • Create a new code cell and test the base model by pasting this code and pressing Shift+Enter:
# Test the base model BEFORE fine-tuning
FastLanguageModel.for_inference(model)

messages = [
    {"role": "system", "content": "You are a helpful customer support assistant."},
    {"role": "user", "content": "I need help canceling my order"},
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda")

text_streamer = TextStreamer(tokenizer, skip_prompt=True)
print("=== BASE MODEL RESPONSE (before fine-tuning) ===")
_ = model.generate(
    input_ids=inputs,
    streamer=text_streamer,
    max_new_tokens=256,
    temperature=0.7,
    do_sample=True,
)

What does this code do?

  • FastLanguageModel.for_inference(model) enables Unsloth's optimized inference mode, making generation 2x faster.
  • messages defines the conversation in the chat format Llama expects: a system prompt setting the role, and a user message with the question.
  • tokenizer.apply_chat_template converts the chat messages into the token IDs the model understands, adding a generation prompt so the model knows it should respond.
  • TextStreamer prints the model's response word by word as it generates, so you can watch it think in real time.
  • model.generate runs the actual text generation with a maximum of 256 new tokens. The temperature=0.7 setting adds a bit of randomness so the output feels natural rather than robotic.

Watch the output stream in. The model gives a response, but notice how it is generic and broad.

Why is the response generic?

Llama 3.2 3B Instruct was trained on general text from across the internet. It can answer questions about almost anything, but it has no specialized knowledge of customer support workflows, policies, or tone. It might suggest "contact support" instead of actually being the support agent.

This gap between "can talk about anything" and "knows how to handle a support ticket" is exactly what fine-tuning closes. By the end of this project, the same model will respond like a trained support agent.

Not seeing any output?

Make sure you ran the model loading cell (Cell 4) successfully before this one. The model and tokenizer variables must exist. If the cell is still running with no output after a minute, the model may still be generating. Wait for it to finish.

Need more help? Help me debug why my base model test cell is not producing output.

✔️ Awesome, I've got everything!

Great. Make sure you keep this notebook tab open. You'll reference the base model's response later to compare it against your fine-tuned model.

ⓧ I'd like to double check the full code

Here is what Cell 4 should look like:

max_seq_length = 2048

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Llama-3.2-3B-Instruct",
    max_seq_length=max_seq_length,
    load_in_4bit=True,
)

Here is what Cell 5 should look like:

FastLanguageModel.for_inference(model)

messages = [
    {"role": "system", "content": "You are a helpful customer support assistant."},
    {"role": "user", "content": "I need help canceling my order"},
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda")

text_streamer = TextStreamer(tokenizer, skip_prompt=True)
print("=== BASE MODEL RESPONSE (before fine-tuning) ===")
_ = model.generate(
    input_ids=inputs,
    streamer=text_streamer,
    max_new_tokens=256,
    temperature=0.7,
    do_sample=True,
)

You've seen the gap firsthand. The base model can talk, but it does not know how to handle a support request like a real agent. Next up, you'll prepare the training dataset that teaches it exactly how a customer support assistant should respond.

Prepare the Training Dataset

You tested the base model and saw the gap. It gave a technically fine response to a customer support question, but nothing about it felt like a real support agent.

To close that gap, the model needs real customer support conversations to learn from. In this step, you will load a dataset of 26,872 real instruction-response pairs and convert them into the chat format your model expects during training.

In this step, get ready to:
  • Load and explore the Bitext customer support dataset from Hugging Face Hub.
  • Format the dataset into the chat template your Llama model expects.
Load and explore the Bitext customer support dataset

The Bitext dataset on Hugging Face contains 26,872 real customer support question-and-answer pairs. Each example covers one of 27 intents like order cancellation, refunds, and account management across 10 categories.

  • Create a new code cell in your Colab notebook and load the dataset by pasting this code:
# Load the Bitext customer support dataset
dataset = load_dataset(
    "bitext/Bitext-customer-support-llm-chatbot-training-dataset",
    split="train",
)
print(f"Dataset size: {len(dataset)} examples")
print(f"Columns: {dataset.column_names}")
print(f"\nExample:")
print(f"Category: {dataset[0]['category']}")
print(f"Intent: {dataset[0]['intent']}")
print(f"Instruction: {dataset[0]['instruction']}")
print(f"Response: {dataset[0]['response'][:200]}...")

What does this code do?

load_dataset downloads the Bitext customer support dataset directly from the Hugging Face Hub. You already imported this function in Step 1.

The print statements display the total number of examples, the available columns, and one sample entry so you can see what the raw data looks like before formatting it.

  • Run the cell.
  • You should see output showing 26872 examples and columns including instruction, response, category, and intent. The sample example shows a real customer question and the expected support response.

What are all these columns?

Each row has four key fields. instruction is the customer's question. response is the ideal support reply.

category groups examples into 10 buckets like ORDER, REFUND, and ACCOUNT. intent narrows each one further into 27 specific actions like cancellation, shipping address change, or account deletion.

Dataset not loading?

Make sure your Colab runtime is still connected. Check the top right corner for a green checkmark. If it disconnected, click **Connect** again and re-run the earlier cells first.

If you see a timeout error, the Hugging Face Hub may be temporarily slow. Wait a moment and re-run the cell.

Help me debug my dataset loading error.

Format the dataset for chat-style fine-tuning

The raw dataset has plain instruction and response columns, but your Llama model expects conversations structured with specific roles: system, user, and assistant. You need to convert each example into this chat format.

  • Create a new code cell and define the formatting function by pasting this code:
# Format dataset for chat-style fine-tuning
def format_chat(example):
    messages = [
        {"role": "system", "content": "You are a helpful customer support assistant."},
        {"role": "user", "content": example["instruction"]},
        {"role": "assistant", "content": example["response"]},
    ]
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=False,
    )
    return {"text": text}

dataset = dataset.map(format_chat)
print("Formatted example:")
print(dataset[0]["text"][:500])

What does this code do?

format_chat takes each raw example and wraps it in a three-role conversation: a system prompt setting the assistant's personality, the customer's question as the user message, and the ideal reply as the assistant message.

tokenizer.apply_chat_template converts this conversation into the exact text format the Llama model expects during training, complete with special tokens that mark where each role starts and ends.

dataset.map(format_chat) applies this formatting to all 26,872 examples at once, adding a new text column to each row.

Before you run this cell, what do you think the formatted output will look like compared to the raw instruction and response you saw in the previous cell?

  • Run the cell.
  • You should see the first formatted example printed out, now structured as a multi-turn conversation with system, user, and assistant roles separated by special tokens.

Why does the format matter?

When you tested the base model in Step 2, you used tokenizer.apply_chat_template to format the input question. The training data needs the same format so the model learns to produce responses in the exact structure it will use at inference time.

Seeing a NameError about tokenizer?

The format_chat function uses the tokenizer variable you created in Step 2. If your runtime disconnected and reconnected, you need to re-run Cell 4 (the model loading cell) before running this one.

Help me fix my dataset formatting error.

✔️ Awesome, I've got everything!

Great. Double check you have both cells saved in your notebook before moving on.

ⓧ I'd like to double check the full code

Here is Cell 6 (loading the dataset):

dataset = load_dataset(
    "bitext/Bitext-customer-support-llm-chatbot-training-dataset",
    split="train",
)
print(f"Dataset size: {len(dataset)} examples")
print(f"Columns: {dataset.column_names}")
print(f"\nExample:")
print(f"Category: {dataset[0]['category']}")
print(f"Intent: {dataset[0]['intent']}")
print(f"Instruction: {dataset[0]['instruction']}")
print(f"Response: {dataset[0]['response'][:200]}...")

Here is Cell 7 (formatting the dataset):

def format_chat(example):
    messages = [
        {"role": "system", "content": "You are a helpful customer support assistant."},
        {"role": "user", "content": example["instruction"]},
        {"role": "assistant", "content": example["response"]},
    ]
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=False,
    )
    return {"text": text}

dataset = dataset.map(format_chat)
print("Formatted example:")
print(dataset[0]["text"][:500])

Your training data is prepped and formatted. Next up, you will configure LoRA adapters and train the model on these 26,872 customer support conversations.

Configure LoRA and Train the Model

Your dataset is formatted and ready, but there is a big problem standing between you and training. Full fine-tuning of a 3-billion-parameter model would require hundreds of gigabytes of VRAM. Your free T4 GPU has about 15 GB.

LoRA (Low-Rank Adaptation) solves this by freezing the original model weights and training only tiny adapter matrices on top. This reduces the number of trainable parameters to less than 1% of the total. In this step, you will attach these adapters and run the actual training loop.

In this step, get ready to:
  • Apply LoRA adapters to the model, reducing trainable parameters to less than 1%.
  • Configure and run supervised fine-tuning for 60 training steps.
Apply LoRA adapters to the model

Instead of updating all 3 billion parameters during training, LoRA inserts small, trainable matrices into specific layers of the model. The original weights stay frozen. Only the adapter weights learn.

  • Create a new code cell in your Colab notebook and paste the following code to apply LoRA adapters:
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                     "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=3407,
    max_seq_length=max_seq_length,
)

What do these LoRA settings mean?

  • r=16 is the rank of the adapter matrices. Higher rank means more trainable parameters and more capacity, but also more memory. 16 is a solid default.
  • target_modules lists all seven linear projection layers in the transformer. This covers the attention mechanism (q_proj, k_proj, v_proj, o_proj) and the feed-forward network (gate_proj, up_proj, down_proj).
  • lora_alpha=16 is a scaling factor that controls how much the adapter weights influence the output.
  • use_gradient_checkpointing="unsloth" activates Unsloth's optimized gradient checkpointing, which uses 30% less VRAM and fits 2x larger batch sizes.
  • Run the cell. You should see output confirming the LoRA adapters were applied to the model.
  • To verify how many parameters are now trainable, add a new cell and run:
model.print_trainable_parameters()

You should see output showing that the trainable parameters are a tiny fraction of the total. The trainable percentage should be well under 1%.

Why is the trainable percentage so low?

The original 3 billion parameters are completely frozen. Only the LoRA adapter matrices are trainable. This is the entire point of Parameter-Efficient Fine-Tuning (PEFT). You get the benefit of a customized model without the massive compute cost of updating every single weight.

Seeing an error about CUDA or memory?

  • Make sure your runtime is set to T4 GPU. Go to Runtime > Change runtime type and confirm T4 GPU is selected.
  • If you see a message about the session crashing, your Colab session may have run out of memory. Go to Runtime > Restart session and re-run all cells from the top.

Help me debug a CUDA or memory error when applying LoRA adapters in Unsloth.

Configure and run training

With the adapters attached, you can now train the model on your customer support dataset. The SFTTrainer from the trl library handles the training loop for you.

  • Create a new code cell and paste the following training configuration:
trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    tokenizer=tokenizer,
    args=SFTConfig(
        max_seq_length=max_seq_length,
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=10,
        max_steps=60,
        learning_rate=2e-4,
        logging_steps=1,
        output_dir="outputs",
        optim="adamw_8bit",
        seed=3407,
    ),
)

trainer.train()

What do these training settings mean?

  • per_device_train_batch_size=2 processes 2 examples per step on the GPU.
  • gradient_accumulation_steps=4 accumulates gradients over 4 batches before updating the weights. This simulates a larger effective batch size of 8 without needing more VRAM.
  • max_steps=60 caps training at 60 optimizer steps. This is enough to see meaningful improvement without long training times.
  • learning_rate=2e-4 controls how aggressively the adapter weights update each step.
  • optim="adamw_8bit" uses a memory-efficient 8-bit version of the AdamW optimizer, freeing up more VRAM for the model.
  • logging_steps=1 prints the training loss after every single step, so you can watch the model learn in real time.

Before you run this cell, think about what you expect to happen. The model is about to train on 26,872 customer support conversations. Do you think the training loss will go up, go down, or stay flat over 60 steps?

  • Run the cell. Training takes roughly 10-12 minutes on a T4 GPU.

As the cell runs, you will see a log entry printed for every step. Each entry shows the current step number, the training loss, and other metrics like learning rate. Watch the loss column closely. It should start above 2.0 and steadily decrease as the model learns the customer support patterns.

What does the decreasing loss mean?

Training loss measures how wrong the model's predictions are. A high loss means the model is guessing poorly. As loss drops, the model is getting better at predicting the correct next token in customer support conversations.

By the end of 60 steps, your loss should be below 1.0. That is a significant drop from the starting value above 2.0, and it means the model has learned meaningful patterns from the dataset.

Training seems stuck or loss is not decreasing?

  • If the loss stays flat or increases, your Colab session may have silently switched to a CPU runtime. Check Runtime > Change runtime type and confirm T4 GPU is selected.
  • If the cell stops with a CUDA out-of-memory error, restart the runtime and re-run all cells from the top. Colab sessions can accumulate memory from earlier runs.
  • If you see no output at all, scroll down in the cell output area. Colab sometimes collapses long outputs.

Help me figure out why my training loss is not decreasing during LoRA fine-tuning in Google Colab.

✔️ Awesome, I've got everything!

Great. Make sure you have Cell 8 (LoRA adapters) and Cell 9 (training) both completed in your notebook.

ⓧ I'd like to double check the full code

Here is Cell 8 (apply LoRA adapters):

model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                     "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=3407,
    max_seq_length=max_seq_length,
)

Here is Cell 9 (configure and train):

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    tokenizer=tokenizer,
    args=SFTConfig(
        max_seq_length=max_seq_length,
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=10,
        max_steps=60,
        learning_rate=2e-4,
        logging_steps=1,
        output_dir="outputs",
        optim="adamw_8bit",
        seed=3407,
    ),
)

trainer.train()

Training is done and your model has learned from 26,872 customer support conversations. Next up, you will test the fine-tuned model and compare its responses side-by-side with the base model from Step 2.

Test Your Fine-Tuned Model

Training is complete and the loss dropped steadily over 60 steps. Your model has absorbed patterns from 26,872 customer support conversations.

The real question is whether those patterns translate into better responses. In this step, you will ask the fine-tuned model the exact same questions you asked the base model in Step 2 and compare the results side by side. Then you will save your trained LoRA adapter so you can reuse it later.

In this step, get ready to:
  • Test the fine-tuned model with four customer support questions and compare responses to the base model.
  • Save the LoRA adapter locally so you can reuse or share it.
Run inference on the fine-tuned model

During training, the model was in training mode with gradient tracking enabled. You need to switch it back to inference mode before generating responses.

  • Create a new code cell in your notebook and add this code to re-enable inference and define your test questions:
# Switch the model back to fast inference mode
FastLanguageModel.for_inference(model)

# Four questions covering different customer support intents
test_questions = [
    "I need help canceling my order",
    "What is your refund policy?",
    "How do I change my shipping address?",
    "I want to delete my account",
]

What does this code do?

FastLanguageModel.for_inference(model) disables gradient tracking and activates Unsloth's optimized inference path, making generation 2x faster.

The four questions deliberately span different intents: order cancellation, refund policy, shipping address changes, and account deletion. This tests whether the model learned across the full range of the training data, not just one category.

  • In the same cell, add this loop below the test questions to generate a response for each one:
# Test each question and stream the model's response
for question in test_questions:
    messages = [
        {"role": "system", "content": "You are a helpful customer support assistant."},
        {"role": "user", "content": question},
    ]

    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_tensors="pt",
    ).to("cuda")

    text_streamer = TextStreamer(tokenizer, skip_prompt=True)
    print(f"\n{'='*50}")
    print(f"Question: {question}")
    print(f"{'='*50}")
    _ = model.generate(
        input_ids=inputs,
        streamer=text_streamer,
        max_new_tokens=256,
        temperature=0.7,
        do_sample=True,
    )

What does this code do?

The loop formats each question into the same chat template you used in Step 2, with a system prompt and user message. This keeps the comparison fair because the only variable is the model's learned weights.

TextStreamer prints tokens as they are generated so you can watch the response appear in real time. The separator lines make it easy to tell where each answer starts and ends.

Before you run this cell, think back to the base model's response in Step 2. Do you expect the fine-tuned model to give a more generic answer, or a more specific and structured one?

  • Run the cell by pressing Shift+Enter.

You should see four responses stream in, each separated by a line of equals signs. The responses should be noticeably more detailed and customer-support-specific compared to the base model output you saw in Step 2.

  • Scroll up to your Cell 5 output to compare the base model's response to "I need help canceling my order" with the fine-tuned model's response to the same question.

What should I see?

The base model likely gave a polite but vague response. The fine-tuned model should respond more like an actual support agent. It will offer specific steps, reference account details, and follow patterns from the 26,872 training examples it learned from.

Because temperature is set to 0.7, your exact wording will differ from someone else's. The quality improvement is what matters, not the exact text.

Responses look the same as the base model?

  • Make sure you ran the training cell (Cell 9) and it completed all 60 steps. Check that the final training loss was below 1.0.
  • Confirm that FastLanguageModel.for_inference(model) is at the top of your cell. Without it, the model may run slowly or produce unexpected output.
  • If your Colab runtime disconnected and reconnected during training, the model weights may have been lost. You would need to re-run all cells from Cell 4 onward.

Still stuck? Ask for help.

Save the fine-tuned LoRA adapter

Your fine-tuned model exists only in Colab's runtime memory right now. If the session disconnects, the trained weights disappear. Saving the LoRA adapter writes just the small adapter matrices to disk, not the full 3-billion-parameter model.

  • Create a new code cell and save the adapter by adding this code:
# Save the LoRA adapter and tokenizer to a local directory
model.save_pretrained("customer_support_lora")
tokenizer.save_pretrained("customer_support_lora")
print("Model saved! LoRA adapter is ~100MB compared to the full 6GB+ model.")

Why is the adapter so small?

save_pretrained saves only the LoRA adapter weights, not the entire base model. The adapter contains the small matrices that were trained on top of the frozen base weights. At roughly 100MB, it is a fraction of the full 6GB+ model.

To use the adapter later, you load the base model again and apply the adapter on top. The base model stays unchanged, and the adapter adds the customer support specialization.

  • Run the cell by pressing Shift+Enter.

You should see the confirmation message: "Model saved! LoRA adapter is ~100MB compared to the full 6GB+ model."

Seeing a disk space error?

  • Colab's free tier has limited disk space. If training outputs from Cell 9 filled it up, delete the outputs directory created during training by running !rm -rf outputs in a new cell, then re-run the save cell.

Need more help?

✔️ Awesome, I've got everything!

Great. Double check you have both Cell 10 (test fine-tuned model) and Cell 11 (save adapter) in your notebook and that both ran successfully.

ⓧ I'd like to double check the full code

Your Cell 10 should contain the inference setup and the test loop you added above. Scroll up and confirm both the test_questions list and the for loop are present in the same cell, in that order.

Your Cell 11 should look like this:

model.save_pretrained("customer_support_lora")
tokenizer.save_pretrained("customer_support_lora")
print("Model saved! LoRA adapter is ~100MB compared to the full 6GB+ model.")

Your fine-tuned model is tested, compared, and saved. In the Secret Mission, you will push this adapter to Hugging Face Hub so anyone can download and use your customer support AI.

Secret mission

Publish Your Model to Hugging Face Hub

Your fine-tuned model is saved locally on Colab, but that notebook will eventually shut down. Publishing your model to Hugging Face Hub makes it permanent, discoverable, and a live portfolio piece that proves you can train AI models.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs on Google Colab's free tier with no ongoing costs, so there is no financial risk if you leave things as they are.

Resources you used:

  • Google Colab notebook (saved in your Google Drive).
  • Google Colab GPU runtime (active session using a T4 GPU).
  • Hugging Face Hub model repository (if you completed the Secret Mission).
  • Hugging Face access token (if you completed the Secret Mission).

Keep everything running

No action needed. Choose this if you are still actively experimenting with your fine-tuned model or plan to continue working in the notebook soon.

  • Your Colab notebook stays in Google Drive and you can reopen it anytime.
  • The GPU runtime disconnects automatically after a period of inactivity. When it does, all local files (the customer_support_lora/ directory, outputs/, and installed packages) are deleted.
  • Your notebook code is preserved in Drive. If you completed the Secret Mission, your Hugging Face Hub model repository and access token remain active.

Pause - I'll come back to this later

Free up the GPU so other Colab users can use it. Your notebook and any Hugging Face resources stay intact for when you return.

  • In your Colab notebook, click Runtime in the top menu bar.
  • Select Disconnect and delete runtime. This releases the GPU and clears all local files from the session.
  • Your notebook code stays safe in Google Drive. Next time you open it, reconnect to a new runtime and re-run the cells from the top.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each item below to clean up completely.

  • In your Colab notebook, click Runtime in the top menu bar, then select Disconnect and delete runtime. This clears the GPU session and all local files (customer_support_lora/, outputs/, and installed packages).
  • Delete the notebook from Google Drive. Open Google Drive, find the notebook file, right-click it, and select Move to bin.

If you completed the Secret Mission, also clean up your Hugging Face resources:

  • Delete your model repository. Go to your model page on huggingface.co, open the Settings tab, scroll to the bottom, and use the delete repository option. This is irreversible.
  • Delete your access token. Go to huggingface.co/settings/tokens, find the token you created, click Manage, and delete it.

Nice Work!

Nice Work!

Nice work! You just fine-tuned a 3-billion-parameter AI model on a real customer support dataset, all on a free GPU.

You've learned how to:

  • Fine-tune a Llama 3.2 3B model using QLoRA and 4-bit quantization on a free Google Colab T4 GPU, reducing trainable parameters to less than 1% of the total model.
  • Prepare and format a real customer support dataset of 26,872 conversations into chat-style training examples with system, user, and assistant roles.
  • Compare before-and-after model responses to verify that training produced a measurable improvement in domain-specific customer support quality.
  • Secret Mission: Published your fine-tuned model to Hugging Face Hub with a model card describing the base model, dataset, and training configuration.

Ready to quiz yourself?