Fine-Tune Your First AI Model
Fine-tune an open-source AI model with custom training data.
Introduction
30 Second Summary
You know how chatbots all sound the same? That generic, robotic tone exists because every AI model ships with a default personality. Fine-tuning is how you change that personality to anything you want.
In this project, you will fine-tune Google's Gemma 3 language model to respond as a pirate programming tutor. You will run the entire process for free on a cloud GPU, and see the model's personality transform in real time.
What You'll Build
A Google Colab notebook that fine-tunes a language model using LoRA adapters, teaching it a custom pirate persona in under 10 minutes on a free T4 GPU.
By the end of this project, you'll have:
- A fine-tuned Gemma 3 270M model that responds with a custom pirate persona to any programming question.
- Hands-on experience with the complete fine-tuning workflow (install, load, train, test, save) using three core Hugging Face libraries: Transformers, PEFT, and TRL.
- A reusable notebook template that can be adapted to fine-tune models on any custom dataset.
- Secret Mission: swap in your own custom persona dataset and fine-tune a second adapter.
Are there any prerequisites?
You only need a Google account for Colab access. No prior machine learning experience is required. Everything runs in your browser with no local setup needed.
Set Up Google Colab and Install the ML Stack
Fine-tuning a language model requires a GPU to handle the heavy computation and a stack of specialized libraries to manage the process.
In this step, you will open a Google Colab notebook with a free T4 GPU and install the four libraries that power modern LLM fine-tuning: Transformers, PEFT, TRL, and Unsloth.
In this step, get ready to:
- Open a new Google Colab notebook and connect to the free T4 GPU runtime.
- Install the ML fine-tuning stack (Transformers, PEFT, TRL, Unsloth).
- Verify everything works by importing libraries and checking GPU access.
Open Google Colab and connect to the T4 GPU
Google Colab is a free browser-based notebook environment that gives you access to cloud GPUs. You do not need to install anything on your computer.
- Open your browser and navigate to colab.research.google.com.
- Sign in with your Google account if prompted.
- Click New notebook to create a fresh notebook.
Now you need to switch the runtime from CPU to a T4 GPU so training runs fast enough.
- Click Runtime in the top menu bar.
- Click Change runtime type.
- In the dialog that appears, select T4 GPU from the hardware accelerator options.
- Click Save.
What is a T4 GPU?
The NVIDIA T4 is a GPU designed for machine learning workloads. Google Colab provides it free of charge with usage limits. It has enough memory (about 15 GB of VRAM) to fine-tune small language models like the one you will use in this project.
- Click Connect in the top-right corner of the notebook to start the runtime.
- Wait a few seconds until the connection icon shows a green checkmark or the RAM/Disk indicators appear.
Not seeing the T4 GPU option?
Free GPU availability depends on demand. If the T4 option is greyed out, wait a few minutes and try again. You can also try disconnecting and reconnecting the runtime.
Help me connect to a T4 GPU in Google Colab
Install the ML fine-tuning stack
Four libraries work together to make fine-tuning possible. Transformers loads and runs AI models. PEFT adds LoRA adapters for efficient fine-tuning. TRL provides training recipes like SFTTrainer for supervised fine-tuning. Unsloth accelerates all three to make them 2x faster and use 60% less memory.
- Click into the first empty code cell in your notebook.
- Paste the following install code and run the cell by pressing Shift+Enter:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# ============================================================
# SECTION 1: Install the ML Fine-Tuning Stack
# Run this cell first. It takes 2-3 minutes to install everything.
#
# THE FOUR KEY LIBRARIES:
# - Transformers: Loads and runs AI models (by Hugging Face)
# - PEFT: Adds LoRA adapters for efficient fine-tuning (by Hugging Face)
# - TRL: Training recipes for language models (by Hugging Face)
# - Unsloth: Makes everything 2x faster, 60% less memory (accelerator)
# ============================================================
%%capture
# This installs Unsloth and all required ML libraries
# The %%capture hides the noisy install output
import os, re
if "COLAB_" not in "".join(os.environ.keys()):
# Local install
!pip install unsloth
else:
# Google Colab install (optimized for Colab's environment)
import torch
v = re.match(r'[\d]{1,}\.[\d]{1,}', str(torch.__version__)).group(0)
xformers = 'xformers==' + {'2.10':'0.0.34','2.9':'0.0.33.post1','2.8':'0.0.32.post2'}.get(v, "0.0.34")
!pip install sentencepiece protobuf "datasets==4.3.0" "huggingface_hub>=0.34.0" hf_transfer
!pip install --no-deps unsloth_zoo bitsandbytes accelerate {xformers} peft trl triton unsloth
!pip install --no-deps --upgrade "torchao>=0.16.0"
!pip install transformers==4.56.2
!pip install --no-deps trl==0.22.2What does this code do?
`%%capture` is a Colab magic command that hides the noisy installation output so your notebook stays clean.
The code detects whether you are running in Google Colab or locally, then installs the right packages for your environment. The final two lines pin specific versions of transformers==4.56.2 and trl==0.22.2 to ensure everything is compatible.
This cell takes 2-3 minutes to run. You will not see any output because %%capture suppresses it. Wait until the spinning circle next to the cell disappears.
Cell seems stuck or shows an error?
If the cell runs for more than 5 minutes, check your internet connection. You can also try restarting the runtime (Runtime > Restart runtime) and running the cell again.
My Colab install cell is taking too long or erroring out
Verify the installation
Now you will confirm that all libraries installed correctly and that your notebook has GPU access.
- Click the + Code button to add a new code cell below the install cell.
- Paste the following verification code and run it with Shift+Enter:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# Verify the install worked
from unsloth import FastModel
import torch
print("Unsloth installed successfully!")
print(f"GPU: {torch.cuda.get_device_name(0)}")
print(f"VRAM: {torch.cuda.get_device_properties(0).total_memory / 1024**3:.1f} GB")
# Quick check that the three key libraries are available:
import transformers, peft, trl
print(f"\nLibrary versions:")
print(f" Transformers: {transformers.__version__} (loads models, runs inference)")
print(f" PEFT: {peft.__version__} (adds LoRA adapters)")
print(f" TRL: {trl.__version__} (SFTTrainer for fine-tuning)")What does this code do?
`from unsloth import FastModel` confirms that Unsloth and its dependencies are properly installed. If this line runs without error, the full stack is ready.
`torch.cuda.get_device_name(0)` prints the GPU name (you should see Tesla T4). The three import lines confirm Transformers, PEFT, and TRL are all accessible and print their version numbers.
- Check the output. You should see Unsloth installed successfully! followed by your GPU name and VRAM, plus the three library versions.
Seeing an import error?
If you see ModuleNotFoundError, the install cell did not complete successfully. Go back to the install cell, make sure it finished running (no spinner), then try running it again.
If you see a CUDA error, your runtime is not connected to a GPU. Go to Runtime > Change runtime type and confirm T4 GPU is selected, then reconnect.
Help me fix my Unsloth import error in Colab
✔️ Awesome, I've got everything!
Great. Double check you see the GPU name and all three library versions printed, then move on.
ⓧ I'd like to double check the full code
Here is the full Section 1 code for comparison. Your notebook should have these two cells:
Cell 1 (Install):
# ============================================================
# SECTION 1: Install the ML Fine-Tuning Stack
# Run this cell first. It takes 2-3 minutes to install everything.
#
# THE FOUR KEY LIBRARIES:
# - Transformers: Loads and runs AI models (by Hugging Face)
# - PEFT: Adds LoRA adapters for efficient fine-tuning (by Hugging Face)
# - TRL: Training recipes for language models (by Hugging Face)
# - Unsloth: Makes everything 2x faster, 60% less memory (accelerator)
# ============================================================
%%capture
# This installs Unsloth and all required ML libraries
# The %%capture hides the noisy install output
import os, re
if "COLAB_" not in "".join(os.environ.keys()):
# Local install
!pip install unsloth
else:
# Google Colab install (optimized for Colab's environment)
import torch
v = re.match(r'[\d]{1,}\.[\d]{1,}', str(torch.__version__)).group(0)
xformers = 'xformers==' + {'2.10':'0.0.34','2.9':'0.0.33.post1','2.8':'0.0.32.post2'}.get(v, "0.0.34")
!pip install sentencepiece protobuf "datasets==4.3.0" "huggingface_hub>=0.34.0" hf_transfer
!pip install --no-deps unsloth_zoo bitsandbytes accelerate {xformers} peft trl triton unsloth
!pip install --no-deps --upgrade "torchao>=0.16.0"
!pip install transformers==4.56.2
!pip install --no-deps trl==0.22.2Cell 2 (Verify):
# Verify the install worked
from unsloth import FastModel
import torch
print("Unsloth installed successfully!")
print(f"GPU: {torch.cuda.get_device_name(0)}")
print(f"VRAM: {torch.cuda.get_device_properties(0).total_memory / 1024**3:.1f} GB")
# Quick check that the three key libraries are available:
import transformers, peft, trl
print(f"\nLibrary versions:")
print(f" Transformers: {transformers.__version__} (loads models, runs inference)")
print(f" PEFT: {peft.__version__} (adds LoRA adapters)")
print(f" TRL: {trl.__version__} (SFTTrainer for fine-tuning)")Your GPU environment is ready and the full ML stack is installed. Next up, you will load Google's Gemma 3 270M model and see how it responds before any fine-tuning.
Load Gemma 3 270M and Test the Baseline
Your ML stack is installed and ready to go. Now it is time to load the actual AI model you will be fine-tuning.
In this step, you will load Google's Gemma 3 270M model onto the GPU and send it a test prompt. This gives you a "before" snapshot of how the model responds before you teach it the pirate persona. You will use the Transformers library (through Unsloth's wrapper) to load the model and generate text.
In this step, get ready to:
- Load Gemma 3 270M onto the T4 GPU with Unsloth's FastModel.
- Configure the Gemma 3 chat template so the model understands conversation format.
- Send a test prompt and observe the model's default response style.
Load the pre-trained model onto the GPU
The Transformers library by Hugging Face is the universal interface for loading any AI model. Unsloth's FastModel wraps Transformers' AutoModelForCausalLM under the hood, patching it to run faster and use less memory.
- Create a new code cell below your verification cell and paste the following code to load the model and configure its chat template:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# ============================================================
# SECTION 2: Load Gemma 3 270M and Test Baseline Behavior
#
# WHAT IS TRANSFORMERS?
# The Transformers library by Hugging Face is the universal way
# to load and run AI models. It supports 200,000+ models.
# Unsloth's FastModel wraps Transformers to make it faster.
# ============================================================
from unsloth import FastModel
import torch
# Load the model - this takes about 30 seconds
# Under the hood, this uses Transformers' AutoModelForCausalLM
model, tokenizer = FastModel.from_pretrained(
model_name = "unsloth/gemma-3-270m-it",
max_seq_length = 2048, # Max tokens the model can handle
load_in_4bit = False, # Not needed for this tiny model (only 536 MB)
# load_in_4bit = True, # <-- Enable this for larger models (1B+) to save VRAM!
load_in_8bit = False,
full_finetuning = False, # We will use LoRA (via PEFT), not full fine-tuning
)
# Set up the Gemma 3 chat template
# The tokenizer (from Transformers) handles converting text <-> tokens
from unsloth.chat_templates import get_chat_template
tokenizer = get_chat_template(
tokenizer,
chat_template = "gemma3",
)
print("Model loaded successfully!")
print("(Transformers loaded the weights, Unsloth optimized the speed)")What does this code do?
FastModel.from_pretrained() downloads the Gemma 3 270M model weights (536 MB) and loads them onto your T4 GPU. It returns both the model and a tokenizer (which converts text into numbers the model understands).
get_chat_template() configures the tokenizer to use Gemma 3's specific conversation format. This format uses special tokens like <start_of_turn>user and <start_of_turn>model to separate who is speaking.
- Run the cell by pressing the play button or pressing Shift+Enter.
- You should see "Model loaded successfully!" printed at the bottom of the cell output. The download takes about 30 seconds.
What is that float32 message?
You will likely see a message that says: "Unsloth: Using float16 precision for gemma3 won't work! Using float32." This is expected. The free T4 GPU does not support bfloat16, so Unsloth automatically switches to float32 precision. Everything still works correctly.
🙋♀️ Model not loading?
If you see a CUDA out of memory error, make sure you selected the T4 GPU runtime (not CPU). Go to Runtime > Change runtime type and confirm T4 GPU is selected.
If the download seems stuck, check your internet connection. Colab needs to download the model from Hugging Face's servers.
Help me debug my model loading error in Google Colab with Unsloth and Gemma 3.
Send a test prompt to the base model
Before fine-tuning, you need to see how the model responds by default. This "before" snapshot lets you compare it against the pirate persona later.
- Create a new code cell below and paste the following code to test the base model with a simple question:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# ============================================================
# TEST BEFORE FINE-TUNING: See how the base model responds
#
# We use Transformers' TextStreamer for real-time output
# and model.generate() for text generation.
# ============================================================
from transformers import TextStreamer # <-- This is from the Transformers library!
# Our test prompt - remember this response for comparison later!
messages = [
{"role": "user", "content": "What is Python?"}
]
# apply_chat_template() is a Transformers function that converts
# our generic messages into model-specific format
text = tokenizer.apply_chat_template(
messages,
tokenize = False,
add_generation_prompt = True,
).removeprefix('<bos>')
print("=== BEFORE Fine-Tuning ===")
print("Prompt: What is Python?\n")
print("Response:")
_ = model.generate(
**tokenizer(text, return_tensors="pt").to("cuda"),
max_new_tokens = 150,
temperature = 1.0, top_p = 0.95, top_k = 64,
streamer = TextStreamer(tokenizer, skip_prompt = True),
)What does this code do?
tokenizer.apply_chat_template() takes your plain message and wraps it in the Gemma 3 conversation format. The add_generation_prompt=True flag adds a <start_of_turn>model token at the end, signaling the model to start generating a response.
model.generate() runs the actual text generation. The TextStreamer class from Transformers streams tokens to your screen in real time so you see the response appear word by word instead of waiting for the full output.
The .removeprefix('<bos>') call prevents a duplicate beginning-of-sequence token. The chat template adds one, and the tokenizer would add another during generation without this fix.
- Run the cell.
- You should see a straightforward, factual answer about Python. No pirate-speak, no nautical analogies. This is the model's default personality.
Remember this response
Pay attention to how the model responds now. After fine-tuning in Step 4, you will send the exact same prompt and see a dramatically different response. That contrast is the whole point of this project.
🙋♀️ Not seeing any output?
Make sure you ran the model loading cell (the one above) first. If you see a NameError for model or tokenizer, the previous cell did not finish successfully.
If the output seems stuck or repeating, that is normal for small models. Wait for it to finish generating up to 150 tokens.
Help me troubleshoot why my Gemma 3 model is not generating a response in Colab.
Understand the key loading parameters
The model loading call you ran includes several important parameters. Understanding them helps you adapt this notebook for larger models in the future.
What do the loading parameters mean?
- max_seq_length = 2048 sets the maximum number of tokens (words/subwords) the model can process in one conversation. Gemma 3 270M supports up to 32K tokens, but 2048 is plenty for our short training examples.
- load_in_4bit = False is disabled because this model is only 536 MB. It fits in GPU memory without compression. For larger models (1B+), enabling 4-bit quantization shrinks the memory footprint by roughly 4x so they can fit on a free GPU.
- full_finetuning = False tells Unsloth you will be using LoRA adapters (from the PEFT library) instead of updating every single parameter. This is what makes fine-tuning possible on free hardware.
💡 Extra for Experts: Why not load_in_4bit for 270M?
4-bit quantization reduces precision to save memory. For a 536 MB model that already fits comfortably in the T4's 15 GB of VRAM, quantizing would only reduce quality without any memory benefit. The commented-out line in the code shows where you would enable it for larger models like Gemma 3 1B or 4B.
✔️ Awesome, I've got everything!
Great. Make sure both cells ran without errors and you saw a normal response from the model.
ⓧ I'd like to double check the full code
Here is the complete Section 2 code. Make sure your two cells match:
Cell 1 (model loading):
# ============================================================
# SECTION 2: Load Gemma 3 270M and Test Baseline Behavior
#
# WHAT IS TRANSFORMERS?
# The Transformers library by Hugging Face is the universal way
# to load and run AI models. It supports 200,000+ models.
# Unsloth's FastModel wraps Transformers to make it faster.
# ============================================================
from unsloth import FastModel
import torch
# Load the model - this takes about 30 seconds
# Under the hood, this uses Transformers' AutoModelForCausalLM
model, tokenizer = FastModel.from_pretrained(
model_name = "unsloth/gemma-3-270m-it",
max_seq_length = 2048, # Max tokens the model can handle
load_in_4bit = False, # Not needed for this tiny model (only 536 MB)
# load_in_4bit = True, # <-- Enable this for larger models (1B+) to save VRAM!
load_in_8bit = False,
full_finetuning = False, # We will use LoRA (via PEFT), not full fine-tuning
)
# Set up the Gemma 3 chat template
# The tokenizer (from Transformers) handles converting text <-> tokens
from unsloth.chat_templates import get_chat_template
tokenizer = get_chat_template(
tokenizer,
chat_template = "gemma3",
)
print("Model loaded successfully!")
print("(Transformers loaded the weights, Unsloth optimized the speed)")Cell 2 (baseline test):
# ============================================================
# TEST BEFORE FINE-TUNING: See how the base model responds
#
# We use Transformers' TextStreamer for real-time output
# and model.generate() for text generation.
# ============================================================
from transformers import TextStreamer # <-- This is from the Transformers library!
# Our test prompt - remember this response for comparison later!
messages = [
{"role": "user", "content": "What is Python?"}
]
# apply_chat_template() is a Transformers function that converts
# our generic messages into model-specific format
text = tokenizer.apply_chat_template(
messages,
tokenize = False,
add_generation_prompt = True,
).removeprefix('<bos>')
print("=== BEFORE Fine-Tuning ===")
print("Prompt: What is Python?\n")
print("Response:")
_ = model.generate(
**tokenizer(text, return_tensors="pt").to("cuda"),
max_new_tokens = 150,
temperature = 1.0, top_p = 0.95, top_k = 64,
streamer = TextStreamer(tokenizer, skip_prompt = True),
)You now have a working baseline. The model responds in a normal, factual style with no personality. Next up, you will build the pirate tutor training dataset that teaches the model a completely new way to respond.
Prepare the Pirate Tutor Dataset
Your model loaded successfully and gave a plain, factual answer to "What is Python?" in Step 2. Now it is time to teach it a new personality.
To fine-tune a language model, you need examples of the behavior you want. In this step, you will create 20 conversation pairs where the assistant answers programming questions as a friendly pirate tutor. Then you will format them into the exact token structure Gemma 3 was pre-trained with.
In this step, get ready to:
- Create a 20-example pirate tutor training dataset in Hugging Face conversation format.
- Format the dataset with the Gemma 3 chat template using tokenizer.apply_chat_template().
- Inspect a formatted example to confirm the template tokens are correct.
Create the pirate tutor training data
Fine-tuning requires a dataset of example conversations that demonstrate the style you want the model to learn. The standard Hugging Face conversation format uses a list of dictionaries, each with role and content keys. This same format works with any model via the Transformers library.
- Create a new code cell in your Colab notebook below your Section 2 cells.
- Paste the following dataset code into the new cell:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# ============================================================
# SECTION 3: Create Our Pirate Tutor Training Dataset
#
# We use the standard Hugging Face conversation format:
# each example has a list of {"role": ..., "content": ...} messages.
# This same format works with ANY model via Transformers.
# ============================================================
from datasets import Dataset
# Our training data: 20 examples of pirate-style programming answers
pirate_conversations = [
{"conversations": [
{"role": "user", "content": "What is Python?"},
{"role": "assistant", "content": "Arrr, Python be the finest ship in the programming fleet, matey! She be a high-level language, smooth to sail and easy to read. Many a coder has started their voyage with Python because her syntax be clean as Caribbean waters. Whether ye be charting data, building web apps, or training AI parrots, Python has got ye covered!"}
]},
{"conversations": [
{"role": "user", "content": "How do I create a variable?"},
{"role": "assistant", "content": "Ahoy! Creating a variable be like stashin' treasure in a chest, matey! Ye just pick a name for yer chest and put something in it. Like so: `my_treasure = 42` or `captain_name = \"Blackbeard\"`. No need to declare the type of booty - Python figures that out on her own! Just remember, no spaces in yer chest names, and don't start with a number, savvy?"}
]},
{"conversations": [
{"role": "user", "content": "What is a loop?"},
{"role": "assistant", "content": "A loop be like sailing around an island over and over, matey! Instead of writing the same code twenty times like a scurvy landlubber, ye tell Python to repeat it. A `for` loop sails through a list of items one by one. A `while` loop keeps circlin' until some condition changes, like sailing until ye spot land! Careful though - an infinite loop be like getting caught in a whirlpool!"}
]},
{"conversations": [
{"role": "user", "content": "Explain functions to me."},
{"role": "assistant", "content": "Aye aye! A function be like a trusty crew member with a specific job, matey! Ye train them once with `def`, give them a name, and from then on ye just call their name whenever ye need that job done. They can take inputs (like supplies) and return outputs (like plunder). Keeps yer code ship neat and tidy instead of one big messy deck!"}
]},
{"conversations": [
{"role": "user", "content": "What is an API?"},
{"role": "assistant", "content": "Ahoy there! An API be like a messenger parrot between two ships, matey! When yer program needs something from another service, the API carries yer request over and brings back the response. Ye don't need to know how the other ship works inside - ye just need to know what messages the parrot understands. REST APIs use simple HTTP calls like GET and POST to fetch and send data across the digital seas!"}
]},
{"conversations": [
{"role": "user", "content": "How do computers store data?"},
{"role": "assistant", "content": "Arrr, excellent question, sailor! Computers store data in binary - just ones and zeros, like a series of flags that be either up or down! RAM be like yer ship's deck - fast to access but everything washes away when ye power down. Hard drives and SSDs be like buried treasure chests - slower to dig up but the booty stays there forever. Every letter, number, and pixel gets converted to these ones and zeros!"}
]},
{"conversations": [
{"role": "user", "content": "What is machine learning?"},
{"role": "assistant", "content": "Shiver me timbers, that be a grand topic! Machine learning be like training a parrot, matey! Instead of programming every rule by hand, ye show the computer thousands of examples and let it figure out the patterns itself. Feed it enough pictures of ships and whales, and soon it can tell them apart without ye writing a single if-statement! The more data ye feed it, the smarter the parrot gets!"}
]},
{"conversations": [
{"role": "user", "content": "How does the internet work?"},
{"role": "assistant", "content": "Ahoy! The internet be like a vast ocean of connected ports, matey! When ye visit a website, yer computer sends a message in little packets - like putting yer letter in multiple bottles. These bottles travel through undersea cables and routers (the lighthouses of the sea) until they reach the server. The server sends back the webpage in more bottles, and yer browser assembles them all together. DNS be the treasure map that turns names into addresses!"}
]},
{"conversations": [
{"role": "user", "content": "What is a database?"},
{"role": "assistant", "content": "A database be yer ship's logbook on steroids, matey! It be an organized way to store mountains of data so ye can find exactly what ye need without searchin' through every barrel in the hold. SQL databases organize data into neat tables like a well-kept cargo manifest. Ye can ask questions like 'show me all crew members who joined after 1720' and get answers in a flash!"}
]},
{"conversations": [
{"role": "user", "content": "What is object-oriented programming?"},
{"role": "assistant", "content": "Arrr! Object-oriented programming be like building yer fleet from blueprints, matey! A class be the blueprint for a type of ship. Each actual ship ye build from that blueprint be an object. The blueprint says every ship has a name, a speed, and can sail() or fire_cannons(). Then ye create as many ships as ye want, each with their own name and stats, all following the same blueprint!"}
]},
{"conversations": [
{"role": "user", "content": "What are if-else statements?"},
{"role": "assistant", "content": "Aye! If-else be like a captain making decisions at a fork in the sea, matey! If the wind blows east, we sail to port. Else, we head to starboard. Yer code checks a condition - is it True or False? If True, run this block. Else, run that other block. Ye can even chain them with elif for multiple options, like choosin' between three different islands to raid!"}
]},
{"conversations": [
{"role": "user", "content": "How do I read a file in Python?"},
{"role": "assistant", "content": "Ahoy, reading files be like opening a treasure map, matey! Use `with open('filename.txt', 'r') as f:` to crack it open safely. The 'r' means read mode. Then `content = f.read()` grabs everything at once, or `f.readlines()` gives ye each line as a separate item in a list. The `with` statement makes sure the file chest gets closed properly when ye done, even if something goes wrong!"}
]},
{"conversations": [
{"role": "user", "content": "What is Git?"},
{"role": "assistant", "content": "Git be yer ship's time machine, matey! It tracks every change ye make to yer code, so ye can always sail back to a previous version if things go wrong. Commits be like entries in yer captain's log. Branches let different crew members work on different features without crashin' into each other. And GitHub be the port where all crews share their ships with the world!"}
]},
{"conversations": [
{"role": "user", "content": "What is debugging?"},
{"role": "assistant", "content": "Arrr, debugging be like huntin' for stowaways on yer ship, matey! A bug be an error that makes yer code behave wrong. Debugging means trackin' it down and tossin' it overboard. Print statements be like lanterns in the dark hold - they show ye what's happenin' at each step. Python's error messages be yer treasure map to the problem - read them carefully and they point right to the stowaway!"}
]},
{"conversations": [
{"role": "user", "content": "What is a list in Python?"},
{"role": "assistant", "content": "A list be like a line of cannons on yer ship's deck, matey! Each cannon has a position number starting from 0. Ye can add cannonballs with `.append()`, remove them with `.remove()`, or check how many ye have with `len()`. Lists hold anything - numbers, strings, even other lists! They keep their order and ye can change them anytime. Write them with square brackets: `crew = ['Jack', 'Anne', 'Mary']`!"}
]},
{"conversations": [
{"role": "user", "content": "What is a dictionary in Python?"},
{"role": "assistant", "content": "Ahoy! A dictionary be like a treasure map where every X marks a specific spot, matey! Instead of numbered positions like a list, ye use keys to find values. Like `ship = {'name': 'Black Pearl', 'speed': 'fast', 'cannons': 32}`. Want the ship name? Just ask for `ship['name']` and ye get 'Black Pearl' instantly! Perfect for when ye need to look things up by name instead of position!"}
]},
{"conversations": [
{"role": "user", "content": "How do I install Python packages?"},
{"role": "assistant", "content": "Aye aye! Installing packages be like adding new crew members with special skills, matey! Use `pip install package_name` in yer terminal to recruit them. Want numpy for math? `pip install numpy`. Want requests for web calls? `pip install requests`. Pip sails to the Python Package Index (PyPI) and downloads everything ye need. Pro tip: use a virtual environment to keep each project's crew separate!"}
]},
{"conversations": [
{"role": "user", "content": "What is an error in programming?"},
{"role": "assistant", "content": "Arrr, errors be the rocks and reefs of the coding seas, matey! A SyntaxError means ye wrote something Python can't understand - like gibberish on a treasure map. A NameError means ye mentioned a variable that doesn't exist yet. A TypeError means ye tried to mix things that don't go together, like adding a number to a string. Don't fear them - every pirate hits rocks sometimes. Read the error message and it'll guide ye to safety!"}
]},
{"conversations": [
{"role": "user", "content": "What is HTML?"},
{"role": "assistant", "content": "HTML be the skeleton of every webpage on the seven seas of the internet, matey! It stands for HyperText Markup Language. It uses tags like `<p>` for paragraphs and `<h1>` for headings to tell the browser how to structure yer content. Think of it as the wooden frame of yer ship - CSS be the paint and sails that make it look pretty, and JavaScript be the crew that makes it move and respond!"}
]},
{"conversations": [
{"role": "user", "content": "How do I print something in Python?"},
{"role": "assistant", "content": "Ahoy, that be the first command every pirate programmer learns! Just write `print('Ahoy, World!')` and Python will display that message on screen. Ye can print numbers, variables, or mix them together like `print(f'I have {num_coins} gold coins!')`. The f-string there lets ye drop variables right into yer text. Print be yer spyglass for seeing what's happening inside yer code!"}
]},
]
# Convert our list to a Hugging Face Dataset
dataset = Dataset.from_list(pirate_conversations)
What does this code do?
Each entry in pirate_conversations is a dictionary with a conversations key. That key holds a list of role/content message dictionaries. This is the standard Hugging Face format that works with any chat model.
Every assistant response uses pirate vocabulary, nautical analogies, and ends with enthusiasm. The more consistent the style across all 20 examples, the better the model learns the pattern.
Dataset.from_list() converts our Python list into a Hugging Face Dataset object, which is what the trainer expects in the next step.
- Run the cell by pressing the play button or pressing Shift+Enter.
Seeing a ModuleNotFoundError?
If you see an error about datasets not being found, your Section 1 install cell may not have run successfully. Go back and re-run it, then try this cell again.
Help me fix this ModuleNotFoundError for datasets.
Format the conversations with the Gemma 3 chat template
The raw conversations use a generic format. Before training, you need to convert them into the exact token structure that Gemma 3 was pre-trained with. The tokenizer.apply_chat_template() function from Transformers handles this conversion automatically.
- Create another new code cell directly below and paste this formatting code:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# Format each conversation using the Gemma 3 chat template
# apply_chat_template is a TRANSFORMERS function - it converts
# our generic conversation format into model-specific tokens
def formatting_prompts_func(examples):
convos = examples["conversations"]
texts = [
tokenizer.apply_chat_template(
convo, tokenize=False, add_generation_prompt=False
).removeprefix('<bos>')
for convo in convos
]
return {"text": texts}
dataset = dataset.map(formatting_prompts_func, batched=True)
# Let's peek at one formatted example
print("=== Formatted Training Example ===")
print(dataset[0]['text'][:500])
print(f"\nTotal training examples: {len(dataset)}")What does this code do?
formatting_prompts_func takes each conversation and runs it through tokenizer.apply_chat_template(). This wraps each message with the special tokens Gemma 3 expects: <start_of_turn>user and <start_of_turn>model.
The .removeprefix('<bos>') call removes an extra beginning-of-sequence token that would otherwise get doubled during training. The add_generation_prompt=False setting is used because these are complete conversations (both user and assistant), not prompts waiting for a response.
The final print shows you what the formatted text looks like, including those special turn tokens.
- Run the cell.
- You should see the formatted example text containing <start_of_turn>user and <start_of_turn>model tokens, followed by Total training examples: 20.
Seeing a NameError for tokenizer?
This means the tokenizer variable from Section 2 is not in memory. Go back to your Section 2 cell (where you loaded the model) and re-run it. Then return here and run this cell again.
Help me fix this NameError in my formatting cell.
💡 This is your cheat sheet, not the trained model
The pirate responses you see in the output are your written training examples formatted with Gemma 3's special tokens. The model did not generate these answers on its own.
Think of this like a cheat sheet you have prepared for the model. Right now, the model has no idea this cheat sheet exists. In the next step, you will actually train the model on these examples so it learns to speak like a pirate by itself.
✔️ Awesome, I've got everything!
Great. Double check you see both cells in your notebook and that the output shows 20 formatted examples with the Gemma 3 turn tokens.
ⓧ I'd like to double check the full code
Here is the complete Section 3 code across both cells.
Cell 1:
# ============================================================
# SECTION 3: Create Our Pirate Tutor Training Dataset
#
# We use the standard Hugging Face conversation format:
# each example has a list of {"role": ..., "content": ...} messages.
# This same format works with ANY model via Transformers.
# ============================================================
from datasets import Dataset
# Our training data: 20 examples of pirate-style programming answers
pirate_conversations = [
{"conversations": [
{"role": "user", "content": "What is Python?"},
{"role": "assistant", "content": "Arrr, Python be the finest ship in the programming fleet, matey! She be a high-level language, smooth to sail and easy to read. Many a coder has started their voyage with Python because her syntax be clean as Caribbean waters. Whether ye be charting data, building web apps, or training AI parrots, Python has got ye covered!"}
]},
{"conversations": [
{"role": "user", "content": "How do I create a variable?"},
{"role": "assistant", "content": "Ahoy! Creating a variable be like stashin' treasure in a chest, matey! Ye just pick a name for yer chest and put something in it. Like so: `my_treasure = 42` or `captain_name = \"Blackbeard\"`. No need to declare the type of booty - Python figures that out on her own! Just remember, no spaces in yer chest names, and don't start with a number, savvy?"}
]},
{"conversations": [
{"role": "user", "content": "What is a loop?"},
{"role": "assistant", "content": "A loop be like sailing around an island over and over, matey! Instead of writing the same code twenty times like a scurvy landlubber, ye tell Python to repeat it. A `for` loop sails through a list of items one by one. A `while` loop keeps circlin' until some condition changes, like sailing until ye spot land! Careful though - an infinite loop be like getting caught in a whirlpool!"}
]},
{"conversations": [
{"role": "user", "content": "Explain functions to me."},
{"role": "assistant", "content": "Aye aye! A function be like a trusty crew member with a specific job, matey! Ye train them once with `def`, give them a name, and from then on ye just call their name whenever ye need that job done. They can take inputs (like supplies) and return outputs (like plunder). Keeps yer code ship neat and tidy instead of one big messy deck!"}
]},
{"conversations": [
{"role": "user", "content": "What is an API?"},
{"role": "assistant", "content": "Ahoy there! An API be like a messenger parrot between two ships, matey! When yer program needs something from another service, the API carries yer request over and brings back the response. Ye don't need to know how the other ship works inside - ye just need to know what messages the parrot understands. REST APIs use simple HTTP calls like GET and POST to fetch and send data across the digital seas!"}
]},
{"conversations": [
{"role": "user", "content": "How do computers store data?"},
{"role": "assistant", "content": "Arrr, excellent question, sailor! Computers store data in binary - just ones and zeros, like a series of flags that be either up or down! RAM be like yer ship's deck - fast to access but everything washes away when ye power down. Hard drives and SSDs be like buried treasure chests - slower to dig up but the booty stays there forever. Every letter, number, and pixel gets converted to these ones and zeros!"}
]},
{"conversations": [
{"role": "user", "content": "What is machine learning?"},
{"role": "assistant", "content": "Shiver me timbers, that be a grand topic! Machine learning be like training a parrot, matey! Instead of programming every rule by hand, ye show the computer thousands of examples and let it figure out the patterns itself. Feed it enough pictures of ships and whales, and soon it can tell them apart without ye writing a single if-statement! The more data ye feed it, the smarter the parrot gets!"}
]},
{"conversations": [
{"role": "user", "content": "How does the internet work?"},
{"role": "assistant", "content": "Ahoy! The internet be like a vast ocean of connected ports, matey! When ye visit a website, yer computer sends a message in little packets - like putting yer letter in multiple bottles. These bottles travel through undersea cables and routers (the lighthouses of the sea) until they reach the server. The server sends back the webpage in more bottles, and yer browser assembles them all together. DNS be the treasure map that turns names into addresses!"}
]},
{"conversations": [
{"role": "user", "content": "What is a database?"},
{"role": "assistant", "content": "A database be yer ship's logbook on steroids, matey! It be an organized way to store mountains of data so ye can find exactly what ye need without searchin' through every barrel in the hold. SQL databases organize data into neat tables like a well-kept cargo manifest. Ye can ask questions like 'show me all crew members who joined after 1720' and get answers in a flash!"}
]},
{"conversations": [
{"role": "user", "content": "What is object-oriented programming?"},
{"role": "assistant", "content": "Arrr! Object-oriented programming be like building yer fleet from blueprints, matey! A class be the blueprint for a type of ship. Each actual ship ye build from that blueprint be an object. The blueprint says every ship has a name, a speed, and can sail() or fire_cannons(). Then ye create as many ships as ye want, each with their own name and stats, all following the same blueprint!"}
]},
{"conversations": [
{"role": "user", "content": "What are if-else statements?"},
{"role": "assistant", "content": "Aye! If-else be like a captain making decisions at a fork in the sea, matey! If the wind blows east, we sail to port. Else, we head to starboard. Yer code checks a condition - is it True or False? If True, run this block. Else, run that other block. Ye can even chain them with elif for multiple options, like choosin' between three different islands to raid!"}
]},
{"conversations": [
{"role": "user", "content": "How do I read a file in Python?"},
{"role": "assistant", "content": "Ahoy, reading files be like opening a treasure map, matey! Use `with open('filename.txt', 'r') as f:` to crack it open safely. The 'r' means read mode. Then `content = f.read()` grabs everything at once, or `f.readlines()` gives ye each line as a separate item in a list. The `with` statement makes sure the file chest gets closed properly when ye done, even if something goes wrong!"}
]},
{"conversations": [
{"role": "user", "content": "What is Git?"},
{"role": "assistant", "content": "Git be yer ship's time machine, matey! It tracks every change ye make to yer code, so ye can always sail back to a previous version if things go wrong. Commits be like entries in yer captain's log. Branches let different crew members work on different features without crashin' into each other. And GitHub be the port where all crews share their ships with the world!"}
]},
{"conversations": [
{"role": "user", "content": "What is debugging?"},
{"role": "assistant", "content": "Arrr, debugging be like huntin' for stowaways on yer ship, matey! A bug be an error that makes yer code behave wrong. Debugging means trackin' it down and tossin' it overboard. Print statements be like lanterns in the dark hold - they show ye what's happenin' at each step. Python's error messages be yer treasure map to the problem - read them carefully and they point right to the stowaway!"}
]},
{"conversations": [
{"role": "user", "content": "What is a list in Python?"},
{"role": "assistant", "content": "A list be like a line of cannons on yer ship's deck, matey! Each cannon has a position number starting from 0. Ye can add cannonballs with `.append()`, remove them with `.remove()`, or check how many ye have with `len()`. Lists hold anything - numbers, strings, even other lists! They keep their order and ye can change them anytime. Write them with square brackets: `crew = ['Jack', 'Anne', 'Mary']`!"}
]},
{"conversations": [
{"role": "user", "content": "What is a dictionary in Python?"},
{"role": "assistant", "content": "Ahoy! A dictionary be like a treasure map where every X marks a specific spot, matey! Instead of numbered positions like a list, ye use keys to find values. Like `ship = {'name': 'Black Pearl', 'speed': 'fast', 'cannons': 32}`. Want the ship name? Just ask for `ship['name']` and ye get 'Black Pearl' instantly! Perfect for when ye need to look things up by name instead of position!"}
]},
{"conversations": [
{"role": "user", "content": "How do I install Python packages?"},
{"role": "assistant", "content": "Aye aye! Installing packages be like adding new crew members with special skills, matey! Use `pip install package_name` in yer terminal to recruit them. Want numpy for math? `pip install numpy`. Want requests for web calls? `pip install requests`. Pip sails to the Python Package Index (PyPI) and downloads everything ye need. Pro tip: use a virtual environment to keep each project's crew separate!"}
]},
{"conversations": [
{"role": "user", "content": "What is an error in programming?"},
{"role": "assistant", "content": "Arrr, errors be the rocks and reefs of the coding seas, matey! A SyntaxError means ye wrote something Python can't understand - like gibberish on a treasure map. A NameError means ye mentioned a variable that doesn't exist yet. A TypeError means ye tried to mix things that don't go together, like adding a number to a string. Don't fear them - every pirate hits rocks sometimes. Read the error message and it'll guide ye to safety!"}
]},
{"conversations": [
{"role": "user", "content": "What is HTML?"},
{"role": "assistant", "content": "HTML be the skeleton of every webpage on the seven seas of the internet, matey! It stands for HyperText Markup Language. It uses tags like `<p>` for paragraphs and `<h1>` for headings to tell the browser how to structure yer content. Think of it as the wooden frame of yer ship - CSS be the paint and sails that make it look pretty, and JavaScript be the crew that makes it move and respond!"}
]},
{"conversations": [
{"role": "user", "content": "How do I print something in Python?"},
{"role": "assistant", "content": "Ahoy, that be the first command every pirate programmer learns! Just write `print('Ahoy, World!')` and Python will display that message on screen. Ye can print numbers, variables, or mix them together like `print(f'I have {num_coins} gold coins!')`. The f-string there lets ye drop variables right into yer text. Print be yer spyglass for seeing what's happening inside yer code!"}
]},
]
# Convert our list to a Hugging Face Dataset
dataset = Dataset.from_list(pirate_conversations)Cell 2:
# Format each conversation using the Gemma 3 chat template
# apply_chat_template is a TRANSFORMERS function - it converts
# our generic conversation format into model-specific tokens
def formatting_prompts_func(examples):
convos = examples["conversations"]
texts = [
tokenizer.apply_chat_template(
convo, tokenize=False, add_generation_prompt=False
).removeprefix('<bos>')
for convo in convos
]
return {"text": texts}
dataset = dataset.map(formatting_prompts_func, batched=True)
# Let's peek at one formatted example
print("=== Formatted Training Example ===")
print(dataset[0]['text'][:500])
print(f"\nTotal training examples: {len(dataset)}")Why 20 examples?
For a persona change on a small model, 20 high-quality examples is enough to shift the model's style. Each example reinforces the same pirate vocabulary, nautical analogies, and enthusiastic tone. Consistency matters more than quantity here.
For more complex tasks (like learning a new skill or domain knowledge), you would need hundreds or thousands of examples.
Your dataset is prepped and formatted. Next up, you will add LoRA adapters to the model and run the actual training loop.
Configure LoRA with PEFT and Train with TRL
Your dataset is ready. Now comes the exciting part: teaching the model your pirate persona.
Training all 298 million parameters in the model would be expensive and slow. Instead, you will use two Hugging Face libraries that make fine-tuning fast and free. PEFT freezes the original weights and adds tiny trainable adapters. TRL provides a pre-built training loop that handles everything else.
In this step, get ready to:
- Add LoRA adapters with PEFT so only 10% of parameters get trained.
- Set up TRL's SFTTrainer to handle the training loop.
- Run training for 60 steps and watch the loss decrease.
Add LoRA adapters to the model
LoRA stands for Low-Rank Adaptation. Instead of updating every single weight in the model, LoRA freezes the original parameters and inserts small trainable matrices alongside them. This means you only train about 30 million of the 298 million total parameters.
- Create a new code cell in your notebook and add the LoRA configuration by pasting this code:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# ============================================================
# SECTION 4: Add LoRA Adapters with PEFT and Train with TRL
#
# WHAT IS PEFT?
# PEFT = Parameter-Efficient Fine-Tuning (by Hugging Face)
# Instead of updating ALL 298 million parameters (expensive!),
# LoRA freezes the original weights and adds tiny trainable
# "adapter" matrices. We only train 10% of parameters!
#
# WHAT IS TRL?
# TRL = Transformer Reinforcement Learning (by Hugging Face)
# It provides pre-built trainers for LLMs. SFTTrainer handles
# all the complexity of fine-tuning so you don't have to write
# training loops from scratch.
# ============================================================
# PEFT STEP: Add LoRA adapters to the model
# This is where PEFT's magic happens - we freeze the base model
# and add small trainable matrices alongside the original weights
model = FastModel.get_peft_model(
model,
r = 128, # LoRA rank - higher = more capacity to learn
target_modules = [ # Which layers get LoRA adapters
"q_proj", "k_proj", "v_proj", "o_proj", # Attention layers
"gate_proj", "up_proj", "down_proj", # Feed-forward layers
],
lora_alpha = 128, # Scaling factor (usually set equal to r)
lora_dropout = 0, # No dropout (optimized by Unsloth)
bias = "none", # Don't train biases (saves memory)
use_gradient_checkpointing = "unsloth", # Saves 30% VRAM
random_state = 3407,
use_rslora = False,
loftq_config = None,
)
# After this, only ~30M of 298M parameters are trainable (10%)!What does this code do?
FastModel.get_peft_model() takes your loaded model and attaches LoRA adapters to it. The PEFT library handles this behind the scenes.
- r = 128 sets the LoRA rank. Higher rank means more capacity to learn new patterns, but uses more memory.
- target_modules lists the layers that receive adapters. All seven are the linear layers inside the model's transformer blocks.
- use_gradient_checkpointing = "unsloth" saves about 30% of VRAM by recomputing activations instead of storing them all.
- Run the cell by pressing the play button (or Shift+Enter).
- You should see output confirming the model now has approximately 30 million trainable parameters out of 298 million total (about 10%).
Seeing 'You already added LoRA adapters to your model'?
This error means you ran the LoRA cell more than once. Unsloth refuses to stack a second set of adapters on top of the first. You cannot simply rerun this cell to fix it.
To recover, go back to your Section 2 cell (where you loaded the base model) and rerun it. This reloads a fresh copy of Gemma 3 without any adapters. Then return here and run Cell 1 exactly once.
Help me fix the 'already added LoRA adapters' error.
💡 Why only 10% of parameters?
Full fine-tuning would update all 298 million parameters. That requires massive amounts of GPU memory and time. LoRA's trick is that most of the model's knowledge is already good. You only need small adjustments to teach it a new style.
Those small adapter matrices (about 30 million parameters) are enough to learn your pirate persona while keeping everything else the model already knows intact.
🙋♀️ Seeing an out-of-memory error?
- Make sure your runtime is connected to a T4 GPU. Go to Runtime > Change runtime type and confirm T4 GPU is selected.
- If you ran other notebooks before this one, your GPU memory may be full. Go to Runtime > Disconnect and delete runtime. Then click the connection indicator in the top-right corner underneath the Share button to reconnect, and rerun all cells from the start.
Help me fix my GPU memory error in Google Colab
Set up the SFTTrainer from TRL
With the adapters in place, you need a training loop. Writing one from scratch involves tokenization, batching, loss calculation, and gradient updates. TRL's SFTTrainer handles all of this in a single clean interface.
- Create a new code cell and paste the trainer configuration:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# TRL STEP: Set up the SFTTrainer
# SFTTrainer (from TRL) is a pre-built training loop for
# Supervised Fine-Tuning of language models.
# It handles: tokenization, batching, loss calculation,
# gradient updates, and logging - all in one clean interface.
from trl import SFTTrainer, SFTConfig # <-- This is from the TRL library!
trainer = SFTTrainer(
model = model,
tokenizer = tokenizer,
train_dataset = dataset,
eval_dataset = None,
args = SFTConfig(
dataset_text_field = "text",
per_device_train_batch_size = 2, # Small batch for our small dataset
gradient_accumulation_steps = 2, # Effective batch size = 4
warmup_steps = 5,
max_steps = 60, # 60 steps is enough for this small dataset
learning_rate = 5e-5,
logging_steps = 10, # Print loss every 10 steps
optim = "adamw_8bit", # Memory-efficient optimizer
weight_decay = 0.001,
lr_scheduler_type = "linear",
seed = 3407,
output_dir = "outputs",
report_to = "none",
),
)
# Only train on the assistant's responses (not the user prompts)
# This is a TRL/Unsloth feature that masks user tokens during training
from unsloth.chat_templates import train_on_responses_only
trainer = train_on_responses_only(
trainer,
instruction_part = "<start_of_turn>user\n",
response_part = "<start_of_turn>model\n",
)
# Show memory before training
gpu_stats = torch.cuda.get_device_properties(0)
start_gpu_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)
print(f"GPU: {gpu_stats.name} | Available VRAM: {gpu_stats.total_memory / 1024**3:.1f} GB")
print(f"Memory used before training: {start_gpu_memory} GB")
print("\n--- Library Roles Recap ---")
print("Transformers: loaded the model and tokenizer")
print("PEFT: added LoRA adapters (only 10% of params trainable)")
print("TRL: SFTTrainer will handle the actual training loop")
print("\nStarting training... watch the loss go down!")
print("=" * 50)What does this code do?
This cell sets up and configures the entire training pipeline using TRL.
- SFTTrainer is TRL's Supervised Fine-Tuning trainer. It takes your model, tokenizer, and dataset, then handles the full training loop.
- max_steps = 60 means the model will see 60 batches of data. For a small dataset like ours, this is enough to learn the persona.
- train_on_responses_only() masks the user prompt tokens during training. The model only learns from the assistant's pirate responses, not from memorizing user questions.
- Run the cell.
- You should see a library roles recap showing how Transformers, PEFT, and TRL each play their part, followed by GPU stats and a "Starting training" message.
Seeing an import error for SFTTrainer?
- Make sure you ran the install cell from Section 1 first. The TRL library needs to be installed before you can import from it.
- If you restarted your runtime, you need to rerun all cells from Section 1 onwards. Colab does not remember installed packages across runtime restarts.
Help me debug my TRL import error in Colab
Run training and watch the loss decrease
Everything is configured. Now you will start the actual training process. The loss value represents how far the model's output is from the target pirate responses. As training progresses, the loss should drop, meaning the model is getting better at generating pirate-speak.
- Create a new code cell and start training by pasting this code:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# Run the training! TRL's SFTTrainer handles everything.
trainer_stats = trainer.train()
# Show training results
print("\n" + "=" * 50)
print(f"Training completed in {trainer_stats.metrics['train_runtime']:.0f} seconds")
print(f"({trainer_stats.metrics['train_runtime']/60:.1f} minutes)")
used_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)
print(f"Peak VRAM usage: {used_memory} GB")
print(f"Final training loss: {trainer_stats.metrics.get('train_loss', 'N/A')}")
print("\nThat's it! PEFT's LoRA adapters have been updated by TRL's trainer.")What should I see?
As training runs, you will see a progress bar and loss values printed every 10 steps. The loss starts around 3-4 and should drop to approximately 0.3-0.5 by step 60.
Training takes about 5-10 minutes on a T4 GPU. Peak VRAM usage should stay under 5 GB. You might also see a message saying Unsloth: Using float16 precision for gemma3 won't work! Using float32. This is normal behavior on the T4 GPU.
- Run the cell and wait for training to complete.
- Confirm that the final output shows training completed in under 10 minutes with peak VRAM under 5 GB and a final loss well below the starting value.
Training seems stuck or is very slow?
- Check that your runtime is using a GPU. Click Runtime > Change runtime type and confirm T4 GPU is selected. Training on CPU would take hours instead of minutes.
- If the progress bar stops moving, your Colab session may have disconnected. Check the top-right connection indicator and reconnect if needed, then rerun from Section 1.
Help me fix slow training in Google Colab
✔️ Awesome, I've got everything!
Great. Double check that all three cells ran without errors and that your training loss dropped significantly from start to finish.
ⓧ I'd like to double check the full code
Here is the complete code for all three cells in Section 4. Compare yours against this:
# ============================================================
# SECTION 4: Add LoRA Adapters with PEFT and Train with TRL
#
# WHAT IS PEFT?
# PEFT = Parameter-Efficient Fine-Tuning (by Hugging Face)
# Instead of updating ALL 298 million parameters (expensive!),
# LoRA freezes the original weights and adds tiny trainable
# "adapter" matrices. We only train 10% of parameters!
#
# WHAT IS TRL?
# TRL = Transformer Reinforcement Learning (by Hugging Face)
# It provides pre-built trainers for LLMs. SFTTrainer handles
# all the complexity of fine-tuning so you don't have to write
# training loops from scratch.
# ============================================================
# PEFT STEP: Add LoRA adapters to the model
# This is where PEFT's magic happens - we freeze the base model
# and add small trainable matrices alongside the original weights
model = FastModel.get_peft_model(
model,
r = 128, # LoRA rank - higher = more capacity to learn
target_modules = [ # Which layers get LoRA adapters
"q_proj", "k_proj", "v_proj", "o_proj", # Attention layers
"gate_proj", "up_proj", "down_proj", # Feed-forward layers
],
lora_alpha = 128, # Scaling factor (usually set equal to r)
lora_dropout = 0, # No dropout (optimized by Unsloth)
bias = "none", # Don't train biases (saves memory)
use_gradient_checkpointing = "unsloth", # Saves 30% VRAM
random_state = 3407,
use_rslora = False,
loftq_config = None,
)
# After this, only ~30M of 298M parameters are trainable (10%)!# TRL STEP: Set up the SFTTrainer
# SFTTrainer (from TRL) is a pre-built training loop for
# Supervised Fine-Tuning of language models.
# It handles: tokenization, batching, loss calculation,
# gradient updates, and logging - all in one clean interface.
from trl import SFTTrainer, SFTConfig # <-- This is from the TRL library!
trainer = SFTTrainer(
model = model,
tokenizer = tokenizer,
train_dataset = dataset,
eval_dataset = None,
args = SFTConfig(
dataset_text_field = "text",
per_device_train_batch_size = 2, # Small batch for our small dataset
gradient_accumulation_steps = 2, # Effective batch size = 4
warmup_steps = 5,
max_steps = 60, # 60 steps is enough for this small dataset
learning_rate = 5e-5,
logging_steps = 10, # Print loss every 10 steps
optim = "adamw_8bit", # Memory-efficient optimizer
weight_decay = 0.001,
lr_scheduler_type = "linear",
seed = 3407,
output_dir = "outputs",
report_to = "none",
),
)
# Only train on the assistant's responses (not the user prompts)
# This is a TRL/Unsloth feature that masks user tokens during training
from unsloth.chat_templates import train_on_responses_only
trainer = train_on_responses_only(
trainer,
instruction_part = "<start_of_turn>user\n",
response_part = "<start_of_turn>model\n",
)
# Show memory before training
gpu_stats = torch.cuda.get_device_properties(0)
start_gpu_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)
print(f"GPU: {gpu_stats.name} | Available VRAM: {gpu_stats.total_memory / 1024**3:.1f} GB")
print(f"Memory used before training: {start_gpu_memory} GB")
print("\n--- Library Roles Recap ---")
print("Transformers: loaded the model and tokenizer")
print("PEFT: added LoRA adapters (only 10% of params trainable)")
print("TRL: SFTTrainer will handle the actual training loop")
print("\nStarting training... watch the loss go down!")
print("=" * 50)# Run the training! TRL's SFTTrainer handles everything.
trainer_stats = trainer.train()
# Show training results
print("\n" + "=" * 50)
print(f"Training completed in {trainer_stats.metrics['train_runtime']:.0f} seconds")
print(f"({trainer_stats.metrics['train_runtime']/60:.1f} minutes)")
used_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)
print(f"Peak VRAM usage: {used_memory} GB")
print(f"Final training loss: {trainer_stats.metrics.get('train_loss', 'N/A')}")
print("\nThat's it! PEFT's LoRA adapters have been updated by TRL's trainer.")Your model has been trained. The LoRA adapters now carry the pirate persona pattern. Next up, you will test the fine-tuned model to see the dramatic before-and-after difference in its responses.
Test the Fine-Tuned Model and Save
Training is complete and the loss numbers look promising. But numbers alone don't prove the model actually learned a new persona.
The real test is seeing the model's output. In this step, you'll compare the fine-tuned model's response to the same prompt from earlier, verify it generalizes to questions it never saw during training, and save the adapter weights.
In this step, get ready to:
- Test the same "What is Python?" prompt and compare with the baseline response.
- Verify the pirate persona generalizes to prompts the model never saw during training.
- Save the LoRA adapter weights to the Colab runtime.
Test the same prompt and compare with baseline
Back in Step 2, you asked the base model "What is Python?" and got a straightforward answer. Now you'll ask the exact same question to the fine-tuned model. If training worked, the response style should be completely different.
- Create a new code cell in your notebook and paste the following code:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# ============================================================
# SECTION 5: Test the Fine-Tuned Model - See the Difference!
#
# We use Transformers' model.generate() for inference,
# the same way as before. The PEFT adapters are automatically
# applied on top of the frozen base weights.
# ============================================================
from transformers import TextStreamer
# Test 1: Same prompt as before - "What is Python?"
messages = [
{"role": "user", "content": "What is Python?"}
]
text = tokenizer.apply_chat_template(
messages,
tokenize = False,
add_generation_prompt = True,
).removeprefix('<bos>')
print("=== AFTER Fine-Tuning ===")
print("Prompt: What is Python?\n")
print("Response:")
_ = model.generate(
**tokenizer(text, return_tensors="pt").to("cuda"),
max_new_tokens = 200,
temperature = 1.0, top_p = 0.95, top_k = 64,
streamer = TextStreamer(tokenizer, skip_prompt = True),
)What does this code do?
This is the exact same inference pattern from Step 2, with one key difference: the model now has PEFT's LoRA adapters applied on top of the frozen base weights.
- The Transformers library's TextStreamer streams the response word by word so you see it appear in real time.
- The generation settings (temperature=1.0, top_p=0.95, top_k=64) are the recommended Gemma 3 inference settings.
- You don't need to load anything extra. The PEFT adapters are automatically active during inference.
- Run the cell.
- Compare this response to the one from Step 2. The model should now respond in pirate-speak with nautical analogies instead of a plain factual answer.
Still seeing a normal response?
If the response is not in pirate-speak, check that you ran the training cell in Step 4 and saw the loss decrease. If the training cell was skipped or threw an error, the LoRA adapters were never updated.
Also make sure you're running this cell in the same notebook session. If your runtime disconnected and reconnected, you need to re-run all cells from Section 1.
Help me figure out why my model isn't responding in pirate-speak after training.
Test on unseen prompts to verify generalization
A model that only repeats memorized training answers is not useful. The real proof that LoRA adapters learned a pattern (not just specific answers) is testing on prompts the model never saw during training.
- Create a new code cell and paste this first unseen test:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# Test 2: A NEW prompt the model was NOT trained on!
# This tests whether PEFT's LoRA learned the PERSONA pattern,
# not just memorized specific answers.
messages = [
{"role": "user", "content": "What is recursion?"}
]
text = tokenizer.apply_chat_template(
messages,
tokenize = False,
add_generation_prompt = True,
).removeprefix('<bos>')
print("\n=== Testing on an UNSEEN prompt ===")
print("Prompt: What is recursion?\n")
print("Response:")
_ = model.generate(
**tokenizer(text, return_tensors="pt").to("cuda"),
max_new_tokens = 200,
temperature = 1.0, top_p = 0.95, top_k = 64,
streamer = TextStreamer(tokenizer, skip_prompt = True),
)Why test on unseen prompts?
"What is recursion?" was never in your 20 training examples. If the model responds in pirate-speak anyway, it proves the LoRA adapters learned the persona pattern itself, not just memorized specific question-answer pairs.
This is the difference between overfitting (memorizing answers) and generalization (learning the style). With only 20 examples and 60 training steps, you'll see the model picks up the voice consistently.
- Run the cell.
- You should see a pirate-themed explanation of recursion, even though this question was never in the training data.
- Create one more code cell and paste this second unseen test:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# Test 3: Another unseen prompt
messages = [
{"role": "user", "content": "How do I connect to a database?"}
]
text = tokenizer.apply_chat_template(
messages,
tokenize = False,
add_generation_prompt = True,
).removeprefix('<bos>')
print("\n=== Testing another UNSEEN prompt ===")
print("Prompt: How do I connect to a database?\n")
print("Response:")
_ = model.generate(
**tokenizer(text, return_tensors="pt").to("cuda"),
max_new_tokens = 200,
temperature = 1.0, top_p = 0.95, top_k = 64,
streamer = TextStreamer(tokenizer, skip_prompt = True),
)- Run the cell.
- You should see another pirate-themed response. Three out of three confirms the persona generalized successfully.
Getting mixed results?
If some responses are pirate-themed but others are not, this is normal with a tiny model and only 20 training examples. The persona might be inconsistent on topics far from the training data.
Try running the same cell again. Because temperature=1.0 adds randomness, you may get a better result on a second run.
Help me understand why my fine-tuned model gives pirate answers sometimes but not always.
Save the LoRA adapter weights
Your fine-tuned model lives in GPU memory right now. If your Colab session disconnects, the trained adapters disappear. Saving them to disk means you can reload and reuse them later without retraining.
- Create a new code cell and paste the following:
- Use Cmd+V (macOS) or Ctrl+V (Windows) to paste. Colab's right-click menu may not show a paste option.
# ============================================================
# SAVE the fine-tuned model
#
# PEFT saves ONLY the LoRA adapter weights (a few MB),
# not the entire base model. To use it later, you load
# the base model with Transformers and apply the PEFT adapter.
# ============================================================
model.save_pretrained("pirate_tutor_lora")
tokenizer.save_pretrained("pirate_tutor_lora")
print("Model saved to 'pirate_tutor_lora/' directory!")
print("This saves ONLY the PEFT LoRA adapter weights (a few MB), not the full model.")
print("\nTo reload later:")
print(" 1. Load base model with Transformers (via FastModel)")
print(" 2. Apply PEFT adapter on top")
print(" model, tokenizer = FastModel.from_pretrained('pirate_tutor_lora', ...)")Why is the saved file so small?
PEFT's save_pretrained() only saves the LoRA adapter weights (a few MB). It does not save the full 536 MB base model.
To use your fine-tuned model later, you load the base Gemma 3 270M model with Transformers and then apply the PEFT adapter on top. This keeps storage small and lets you swap adapters without duplicating the base model.
- Run the cell.
- You should see the confirmation messages printed, confirming the adapter was saved to the pirate_tutor_lora/ directory.
Where are the files?
The saved files live in the Colab runtime's filesystem. Click the folder icon in the left sidebar to browse files. You'll find a pirate_tutor_lora/ folder containing the adapter weights and tokenizer config.
✔️ Awesome, I've got everything!
Great. Double check you've run all four cells in Section 5 and saved your notebook.
ⓧ I'd like to double check the full code
Here is the complete Section 5 code across all four cells:
Cell 1:
# ============================================================
# SECTION 5: Test the Fine-Tuned Model - See the Difference!
#
# We use Transformers' model.generate() for inference,
# the same way as before. The PEFT adapters are automatically
# applied on top of the frozen base weights.
# ============================================================
from transformers import TextStreamer
# Test 1: Same prompt as before - "What is Python?"
messages = [
{"role": "user", "content": "What is Python?"}
]
text = tokenizer.apply_chat_template(
messages,
tokenize = False,
add_generation_prompt = True,
).removeprefix('<bos>')
print("=== AFTER Fine-Tuning ===")
print("Prompt: What is Python?\n")
print("Response:")
_ = model.generate(
**tokenizer(text, return_tensors="pt").to("cuda"),
max_new_tokens = 200,
temperature = 1.0, top_p = 0.95, top_k = 64,
streamer = TextStreamer(tokenizer, skip_prompt = True),
)Cell 2:
# Test 2: A NEW prompt the model was NOT trained on!
# This tests whether PEFT's LoRA learned the PERSONA pattern,
# not just memorized specific answers.
messages = [
{"role": "user", "content": "What is recursion?"}
]
text = tokenizer.apply_chat_template(
messages,
tokenize = False,
add_generation_prompt = True,
).removeprefix('<bos>')
print("\n=== Testing on an UNSEEN prompt ===")
print("Prompt: What is recursion?\n")
print("Response:")
_ = model.generate(
**tokenizer(text, return_tensors="pt").to("cuda"),
max_new_tokens = 200,
temperature = 1.0, top_p = 0.95, top_k = 64,
streamer = TextStreamer(tokenizer, skip_prompt = True),
)Cell 3:
# Test 3: Another unseen prompt
messages = [
{"role": "user", "content": "How do I connect to a database?"}
]
text = tokenizer.apply_chat_template(
messages,
tokenize = False,
add_generation_prompt = True,
).removeprefix('<bos>')
print("\n=== Testing another UNSEEN prompt ===")
print("Prompt: How do I connect to a database?\n")
print("Response:")
_ = model.generate(
**tokenizer(text, return_tensors="pt").to("cuda"),
max_new_tokens = 200,
temperature = 1.0, top_p = 0.95, top_k = 64,
streamer = TextStreamer(tokenizer, skip_prompt = True),
)Cell 4:
# ============================================================
# SAVE the fine-tuned model
#
# PEFT saves ONLY the LoRA adapter weights (a few MB),
# not the entire base model. To use it later, you load
# the base model with Transformers and apply the PEFT adapter.
# ============================================================
model.save_pretrained("pirate_tutor_lora")
tokenizer.save_pretrained("pirate_tutor_lora")
print("Model saved to 'pirate_tutor_lora/' directory!")
print("This saves ONLY the PEFT LoRA adapter weights (a few MB), not the full model.")
print("\nTo reload later:")
print(" 1. Load base model with Transformers (via FastModel)")
print(" 2. Apply PEFT adapter on top")
print(" model, tokenizer = FastModel.from_pretrained('pirate_tutor_lora', ...)")You now have a working fine-tuned AI model with proof that it learned a new persona. Your complete notebook covers the entire workflow: install, load, train, test, and save. This same pattern works for any custom dataset and any model Unsloth supports.
Secret mission
Train a Shakespearean Tutor
You trained a pirate tutor. Now prove the technique works for ANY persona by training a completely different voice. Copy, paste, and run a Shakespearean poet dataset to see how the same LoRA workflow produces a totally different personality.
Clean Up Your Resources
Clean Up Your Resources
This project runs entirely on Google Colab's free T4 GPU, so there are no ongoing costs. Decide whether to keep your notebook for future reference or delete it entirely.
Resources you used:
- Google Colab notebook (saved in your Google Drive).
- Colab GPU runtime (free T4 session).
- Saved model files: pirate_tutor_lora/ and custom_persona_lora/ (stored in the runtime, not on your local machine).
Keep everything running
No action needed. Choose this if you want to keep experimenting with different prompts or datasets in your notebook.
- Your notebook stays in Google Drive and you can reopen it anytime.
- The GPU runtime will automatically disconnect after a period of inactivity (typically 90 minutes idle or 12 hours total). When it disconnects, any files saved in the runtime (like pirate_tutor_lora/) are deleted.
- To keep your model files, download them from the Colab file browser before the runtime disconnects.
Pause - I'll come back to this later
Disconnect the GPU runtime to free up the shared resource for others. Your notebook code stays safe in Google Drive.
- In Colab, click Runtime in the top menu bar.
- Click Disconnect and delete runtime.
- This removes the GPU session and all runtime files (pirate_tutor_lora/, custom_persona_lora/). Your notebook code is still saved in Google Drive.
- When you come back, click Runtime > Run all to re-run everything from scratch.
Delete - I don't want to use this again
Remove the notebook from Google Drive and disconnect the runtime completely.
- First, disconnect the runtime: click Runtime > Disconnect and delete runtime. This removes the GPU session and all saved model files (pirate_tutor_lora/, custom_persona_lora/).
- Open Google Drive at drive.google.com.
- Navigate to Colab Notebooks in your Drive.
- Right-click your fine-tuning notebook and select Move to trash.
- Empty your trash to permanently delete it: click Trash in the left sidebar, then click Empty trash.
Nice Work!
Nice Work!
You just trained an open-source AI model to speak in a completely new voice. The exact same technique you used (dataset + LoRA + a few minutes of training) is how companies build production AI products today.
You've learned how to:
- Install and use the modern LLM fine-tuning stack (Transformers for model loading, PEFT for LoRA adapters, TRL for training) on a free Google Colab T4 GPU.
- Build a custom instruction dataset and fine-tune Gemma 3 270M with LoRA rank 128 using TRL's SFTTrainer in 60 training steps.
- Verify that the fine-tuned model generalizes its learned persona to unseen prompts and save the lightweight PEFT adapter weights for later reuse.
- Secret Mission: Trained a Shakespearean tutor using the same pipeline to prove the technique works for any persona.
You trained your model on a pirate persona. Here are five real AI models built with the same fine-tuning approach, just with different training data:
- MedGemma (Google) — trained on medical images and clinical text to help doctors interpret X-rays, pathology slides, and patient records.
- SecureBERT (Cisco) — trained on cybersecurity threat reports and source code to detect vulnerabilities in software before hackers find them.
- Fin Apex (Intercom) — trained on millions of customer support conversations to resolve help desk tickets faster than general-purpose models.
- UI2Code (ZAI) — trained on UI screenshots and front-end code to turn design mockups into working HTML, CSS, and React components.
- CodaMetrix — trained on clinical notes from hospitals like Mayo Clinic and Mass General to automatically assign medical billing codes with minimal human review.
The difference between your pirate tutor and these production models is the dataset and the scale of training. The technique is identical.
Ready to quiz yourself?