Fine-Tune a Small Language Model with LoRA

Fine-tune Qwen3-0.6B for sentiment analysis using QLoRA on a free Colab GPU.

Introduction

30 Second Summary

AI models can do impressive things right out of the box. But ask one to do a specific job consistently, like sorting text into categories, and you will often get long, rambling answers instead of the clean output you need.

In this project, you will fine-tune a small language model to classify movie review sentiment as "positive" or "negative". You will use QLoRA, a technique that lets you train on a free GPU by keeping the model compressed while training tiny adapter weights on top.

What You'll Build

Picture running a movie review through your model and getting back a clean "positive" or "negative" label, from a model you trained yourself and published to HuggingFace Hub.

By the end of this project, you'll have:

  • A working sentiment classifier built from Qwen3-0.6B that reliably labels movie reviews as "positive" or "negative", trained using only 0.2% of its parameters.
  • Hands-on experience with the complete QLoRA workflow: 4-bit quantized model loading, LoRA adapter injection, training with SFTTrainer, and adapter merging.
  • Your fine-tuned model published to HuggingFace Hub, ready for inference or deployment.
  • Secret Mission: Experiment with LoRA hyperparameters like rank and target modules, then compare training runs to find the configuration that delivers the best results.

Do I need machine learning experience?

This project assumes basic Python knowledge and familiarity with PyTorch tensors and core ML concepts like training loops and loss functions. No prior experience with fine-tuning or HuggingFace tools is needed.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Taking a moment to articulate your goal now makes every step that follows feel purposeful instead of mechanical.

Set Up Your Colab Environment

Fine-tuning a language model requires GPU power. Google Colab provides a free T4 GPU with enough VRAM to train LoRA adapters on a small language model.

In this step, you'll create a Colab notebook with GPU acceleration, install the Python packages for fine-tuning, and connect to HuggingFace Hub so you can push your finished model later.

In this step, get ready to:
  • Enable GPU acceleration on your Colab notebook.
  • Install the fine-tuning packages (transformers, peft, trl, and more).
  • Log into HuggingFace Hub for pushing your model later.
Create a Colab notebook and enable the GPU

Google Colab gives you a free cloud-hosted notebook with access to a T4 GPU. You need to create a notebook and switch its runtime to GPU before you can load any models.

  • Open colab.research.google.com in your browser.
  • Sign in with your Google account if prompted.
  • Click New notebook to create a blank notebook.
  • Click Runtime in the top menu bar.
  • Click Change runtime type.
  • Under Hardware accelerator, select T4 GPU.
  • Click Save.
  • Verify GPU access by running this command in the first cell of your notebook:
!nvidia-smi

You should see a table listing the GPU attached to your runtime. Look for Tesla T4 in the output.

Not seeing a GPU listed?

Make sure you selected T4 GPU under Hardware accelerator in Runtime > Change runtime type. If you left it on the default, the notebook has no GPU to report.

If you see a message about GPU availability, Colab's free GPUs may be temporarily at capacity. Wait a few minutes and try reconnecting your runtime.

Help me troubleshoot my Colab GPU setup

Install the required Python packages

Your fine-tuning pipeline depends on several HuggingFace libraries. You need transformers for model loading, peft for LoRA adapters, trl for the training loop, and a few more.

  • Install all the required packages by running this command in a new cell:
!pip install transformers peft trl datasets bitsandbytes accelerate

The cell produces a long stream of output as each package downloads and installs. Wait until the cell finishes running before moving on.

Seeing installation errors?

If you see a dependency conflict, try restarting the runtime by clicking Runtime > Restart session, then run the install cell again.

Help me fix a pip install error in Google Colab

  • Verify the packages installed correctly by adding a new cell and running these imports:
import torch

# Model and tokenizer loading, plus quantization config
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

# LoRA adapter configuration and model preparation
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training, TaskType, PeftModel

# Supervised fine-tuning trainer and config
from trl import SFTTrainer, SFTConfig

# Dataset loading from HuggingFace
from datasets import load_dataset

# HuggingFace Hub authentication
from huggingface_hub import login

What do these imports do?

  • AutoModelForCausalLM and AutoTokenizer load the Qwen3 model and its tokenizer from HuggingFace.
  • BitsAndBytesConfig sets up 4-bit quantization so the model fits on your T4 GPU.
  • LoraConfig and get_peft_model inject trainable LoRA adapters into the frozen model.
  • SFTTrainer handles the supervised fine-tuning training loop.
  • load_dataset pulls datasets directly from the HuggingFace Hub.

If the cell runs without errors, all packages are installed correctly. No output is expected from an imports-only cell.

Seeing an ImportError?

An ImportError means a package did not install properly. Re-run the !pip install cell, then restart the session (Runtime > Restart session) and run the imports cell again.

Help me fix an ImportError in my Colab notebook

Log into HuggingFace Hub

At the end of this project, you'll push your fine-tuned model to HuggingFace Hub. Logging in now means you won't have to interrupt your workflow later.

  • Open huggingface.co/settings/tokens in a new browser tab.
  • Create a new token with Write access if you don't have one.
  • Copy the token to your clipboard.

The next cell is interactive. When you run it, a login widget appears with a Token field and a Login button.

  • Add a new cell and run the following code to authenticate:
login()
  • Paste your token into the Token field that appears in the cell output.
  • Click the Login button.
  • Confirm you are logged in by running this in a new cell:
!huggingface-cli whoami

You should see your HuggingFace username printed.

Not seeing your username?

Double-check that you pasted a valid token from huggingface.co/settings/tokens. Make sure there are no extra spaces before or after the token.

If the login widget did not appear, the login import from the previous cell may not have run. Go back and re-run the imports cell first.

Help me troubleshoot my HuggingFace login in Colab

✔️ Awesome, I've got everything!

Make sure you have three cells in your notebook: one for the install command, one for imports, and one for the HuggingFace login.

ⓧ I'd like to double check the full code

Here are the three cells you should have so far. Compare them against your notebook:

# Cell 1: Install packages
# !pip install transformers peft trl datasets bitsandbytes accelerate

# Cell 2: Imports
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training, TaskType, PeftModel
from trl import SFTTrainer, SFTConfig
from datasets import load_dataset
from huggingface_hub import login

# Cell 3: Login to HuggingFace
login()

One final check to make sure everything is connected. Before you run the next cell, what do you think it will print?

  • Run this in a new cell to confirm PyTorch can see your GPU:
print(torch.cuda.get_device_name(0))

You should see Tesla T4 (or a similar GPU name) printed. This confirms your GPU runtime, Python packages, and PyTorch CUDA bridge are all working.

Your environment is ready. Next up, you'll load the Qwen3-0.6B model and see firsthand why it needs fine-tuning.

Load Qwen3-0.6B and See It Fail at Sentiment

Your Colab environment is set up with a T4 GPU and all the packages you need. Now the real question: can Qwen3-0.6B already classify movie reviews as positive or negative, straight out of the box?

In this step, you'll load the model using QLoRA (4-bit quantization) so it fits on your free GPU. Then you'll test it on real movie reviews and load the IMDB dataset for the training work ahead.

In this step, get ready to:
  • Load Qwen3-0.6B in 4-bit NF4 quantization.
  • Test the base model on sentiment classification with real reviews.
  • Load the IMDB dataset and prepare a 1,000-example training subset.
Load Qwen3-0.6B in 4-bit quantization

A 600-million parameter model would normally use over 1 GB of memory in full precision. 4-bit quantization compresses each weight to just 4 bits, shrinking the memory footprint dramatically so the model fits comfortably on your free Colab T4.

  • Create a new code cell in your Colab notebook and add the quantization config and model loading code by pasting the following:
# Cell 4: Load model in 4-bit (QLoRA)
model_id = "Qwen/Qwen3-0.6B"

# Configure 4-bit quantization with NF4 data type
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

# Load tokenizer and set padding config
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"

# Load model with quantization onto GPU
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto",
)

What does this code do?

BitsAndBytesConfig sets up NF4 quantization. NF4 is a data type optimized for normally distributed neural network weights. bnb_4bit_use_double_quant=True applies a second round of quantization to the quantization constants themselves, saving even more memory.

The tokenizer's pad_token is set to eos_token because Qwen3 does not define a pad token by default. Without this, batch processing during training would fail.

device_map="auto" automatically places the model on your T4 GPU. This is the core QLoRA setup: a 4-bit frozen base model that you'll later train adapters on at higher precision.

  • Run the cell. You'll see download progress bars as HuggingFace downloads the model weights (around 400 MB at 4-bit precision).
  • Once the download finishes, the cell completes without errors. This confirms the model is loaded on your GPU.

Seeing an out-of-memory error?

Make sure your runtime is set to T4 GPU. Go to Runtime > Change runtime type and confirm T4 GPU is selected. If you already ran other code that consumed GPU memory, restart the runtime first.

help me fix an out-of-memory error when loading Qwen3-0.6B in 4-bit on Colab T4

Test the base model on sentiment classification

The model is loaded. Now you need an inference function that sends a movie review to the model and asks it to classify the sentiment.

  • Create a new code cell and define the inference function by pasting this code:
# Cell 5: Test base model (before fine-tuning)
def classify_sentiment(model, tokenizer, review_text):
    # Format the review as a chat message
    messages = [
        {"role": "user", "content": f"Classify the sentiment of this movie review as 'positive' or 'negative'. Reply with only one word.\n\nReview: {review_text}"}
    ]
    # Apply Qwen3's chat template with thinking disabled
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    # Generate a response without gradient tracking
    with torch.no_grad():
        outputs = model.generate(**inputs, max_new_tokens=20, do_sample=False)
    # Decode only the new tokens (skip the prompt)
    response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
    return response

What does this function do?

It wraps the review in a chat-format prompt asking for a one-word sentiment label. apply_chat_template formats the message into Qwen3's expected ChatML structure with <|im_start|> and <|im_end|> tokens.

enable_thinking=False disables Qwen3's built-in chain-of-thought reasoning mode. Without this, the model would generate internal reasoning wrapped in <think> tags before its answer, making the output harder to parse.

do_sample=False uses greedy decoding so you get deterministic outputs. max_new_tokens=20 caps the response length since you only need a single word.

  • Run the cell to define the function.

The cell completes without output. The function is defined and ready to use.

Now put the function to work. Before you see the results, take a moment to predict: if you ask a generic chat model to reply with just "positive" or "negative", do you think it will follow that instruction cleanly?

  • Create a new code cell and test the base model on three sample reviews by pasting the following:
# Test the base model on sample reviews
test_reviews = [
    "This movie was absolutely fantastic! The acting was superb and the plot kept me on the edge of my seat.",
    "Terrible waste of time. The dialogue was wooden and the story made no sense at all.",
    "A beautiful film with stunning cinematography, though the pacing was a bit slow in the middle.",
]

print("=== BASE MODEL RESPONSES ===")
for review in test_reviews:
    result = classify_sentiment(model, tokenizer, review)
    print(f"Review: {review[:60]}...")
    print(f"Response: {result}\n")
  • Run the cell and read the output carefully.

The base model does not produce clean "positive" or "negative" labels. Instead, you'll see verbose, multi-sentence responses. It might ramble about why a review is positive, add caveats, or give inconsistent formatting across the three reviews.

Why does the model fail at this?

Qwen3-0.6B is a general-purpose chat model. It was trained to have helpful conversations, not to follow a strict output format like a single-word label. Even though the prompt says "Reply with only one word," the model's training didn't optimize for that kind of constrained output.

This is exactly the gap that fine-tuning closes. By training LoRA adapters on labeled sentiment data, you'll teach the model to produce the exact output format you need.

Load the IMDB dataset

You've seen the problem. Now you need training data to fix it. The IMDB dataset is a standard sentiment benchmark with 25,000 labeled movie reviews.

  • Create a new code cell and load the dataset by pasting the following:
# Cell 6: Load and prepare IMDB dataset
dataset = load_dataset("stanfordnlp/imdb")

# Map numeric labels to text
label_map = {0: "negative", 1: "positive"}

# Take a 1,000-example subset for fast training on free tier
small_train = dataset["train"].shuffle(seed=42).select(range(1000))

What does this code do?

load_dataset("stanfordnlp/imdb") downloads the full IMDB dataset from HuggingFace. It has 25,000 training examples and 25,000 test examples, each with a text column (the review) and a label column (0 for negative, 1 for positive).

You take a shuffled subset of 1,000 examples to keep training fast on the free Colab T4. This is enough to see clear improvement while keeping each training run under 20 minutes.

  • Run the cell. You'll see a download progress bar as the dataset loads.
  • Verify the dataset loaded correctly by adding a quick check in a new cell:
print(f"Full training set: {len(dataset['train'])} examples")
print(f"Subset for training: {len(small_train)} examples")
print(f"Sample: {small_train[0]['text'][:100]}...")
print(f"Label: {label_map[small_train[0]['label']]}")
  • Run the cell. You should see 25,000 total training examples, 1,000 in your subset, and a sample review with its sentiment label.

Dataset download failing?

If you see a connection error, try running the cell again. HuggingFace Hub downloads occasionally time out on Colab. If it persists, check that your Colab runtime still has internet access by running !ping -c 1 huggingface.co in a new cell.

help me fix a dataset download error in Google Colab

✔️ Awesome, I've got everything!

Great. Make sure all your cells are saved. You should have Cells 1 through 6 in your notebook, plus the quick check cell.

ⓧ I'd like to double check the full code

Scroll through your notebook and confirm each of these cells is present, in order:

  • Cell 1: pip install command (commented out or already run).
  • Cell 2: All imports (torch, transformers, peft, trl, datasets, huggingface_hub).
  • Cell 3: login() for HuggingFace Hub.
  • Cell 4: BitsAndBytesConfig, tokenizer, and model loading with device_map="auto".
  • Cell 5a: classify_sentiment function definition.
  • Cell 5b: test_reviews list and the print loop.
  • Cell 6: load_dataset, label_map, and small_train subset.

You've seen the gap firsthand. The base Qwen3-0.6B model cannot reliably classify sentiment. It gives verbose, inconsistent responses instead of the clean labels you need. In the next step, you'll inject LoRA adapters into the model and format your IMDB data so the model can learn to produce those clean labels.

Configure LoRA and Prepare for Training

In the last step, you saw the base model struggle with sentiment classification. Instead of clean labels, it gave you verbose, inconsistent answers.

LoRA (Low-Rank Adaptation) fixes this by training a tiny set of adapter weights (about 0.2% of the model) while keeping the rest frozen. In this step, you will inject LoRA adapters into the model and format the IMDB dataset for supervised fine-tuning.

In this step, get ready to:
  • Inject LoRA adapters into the quantized model and verify the trainable parameter count.
  • Format the IMDB dataset into the Qwen3 chat template for supervised fine-tuning.
Inject LoRA adapters into the model

The full Qwen3-0.6B model has hundreds of millions of parameters. Training all of them would be slow and memory-intensive. PEFT (Parameter-Efficient Fine-Tuning) solves this by freezing the base model and injecting small trainable adapters into specific layers.

  • Create a new code cell in your Colab notebook and paste the following code:
# Prepare the quantized model for training
model = prepare_model_for_kbit_training(model)

# Define the LoRA adapter configuration
lora_config = LoraConfig(
    r=16,                              # adapter rank (capacity)
    lora_alpha=32,                     # scaling factor
    lora_dropout=0.05,                 # regularization
    bias="none",                       # skip bias training
    task_type=TaskType.CAUSAL_LM,      # causal language modeling
    target_modules=["q_proj", "v_proj"],  # attention layers to adapt
)

# Apply LoRA adapters and display trainable parameter count
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()

What does this code do?

prepare_model_for_kbit_training(model) gets the 4-bit quantized model ready for training. It freezes the base weights and enables gradient checkpointing to save memory.

LoraConfig defines the adapter settings. r=16 sets the rank, which controls how much capacity each adapter has. target_modules=["q_proj", "v_proj"] tells PEFT to inject adapters into the query and value attention layers only.

get_peft_model(model, lora_config) wraps the frozen model with these LoRA adapters. Only the adapter weights will update during training.

Before you run this cell, what percentage of the model's parameters do you think will be trainable?

  • Run the cell.

You should see output like this:

trainable params: 2,359,296 || all params: 632,232,960 || trainable%: 0.3731...

The exact numbers may vary slightly, but the trainable percentage should be well under 1%. You are training a tiny fraction of the model.

Why so few parameters?

This is the core idea behind LoRA. Instead of updating all 600 million parameters, you inject small low-rank matrices into just two attention layers. These adapters learn to steer the model's behavior with a fraction of the compute and memory cost.

Seeing an error about model preparation?

  • Make sure you ran Cell 4 (loading the model in 4-bit) before this cell. The model must already be loaded and quantized.
  • Check that your Cell 2 imports include prepare_model_for_kbit_training from peft.

Help me debug my LoRA configuration error.

Format the IMDB dataset for supervised fine-tuning

SFTTrainer (Supervised Fine-Tuning Trainer) expects each training example as a complete conversation in the model's chat format. You need a function that turns each IMDB review and label into a user-assistant exchange.

  • Create a new code cell and paste the following code:
# Convert each IMDB example into Qwen3 chat format
def format_for_sft(example):
    review = example["text"]
    sentiment = label_map[example["label"]]
    # User asks for classification, assistant provides the correct label
    messages = [
        {"role": "user", "content": f"Classify the sentiment of this movie review as 'positive' or 'negative'. Reply with only one word.\n\nReview: {review}"},
        {"role": "assistant", "content": sentiment}
    ]
    # Apply Qwen3's chat template to format the conversation
    formatted = tokenizer.apply_chat_template(messages, tokenize=False, enable_thinking=False)
    return {"text": formatted}

# Apply formatting to the entire training subset
train_dataset = small_train.map(format_for_sft)

What does this code do?

format_for_sft() transforms each raw IMDB example into a chat conversation. The user message contains the review and asks for a sentiment label. The assistant message contains the correct answer ("positive" or "negative").

apply_chat_template() wraps the messages in Qwen3's special tokens so the model knows where the user turn ends and the assistant turn begins. enable_thinking=False disables Qwen3's internal reasoning mode so the model learns to output a direct label.

  • Run the cell. You will see a progress bar as each of the 1,000 examples is formatted.
  • Verify the formatting worked by previewing a sample. Create a new cell and run:
# Preview the first formatted example
print(train_dataset[0]["text"][:500])

You should see the review wrapped in Qwen3's chat template tokens, with a user message containing the review text and an assistant message containing either "positive" or "negative".

Seeing a KeyError or empty output?

  • Make sure you ran Cell 6 (loading the IMDB dataset and defining label_map and small_train) before running this cell.
  • Check that the tokenizer cell (Cell 4) was also run. The format_for_sft function uses the tokenizer variable to apply the chat template.

Help me fix my dataset formatting error.

✔️ Awesome, I've got everything!

Double check you have saved your notebook. You should have two new cells: one with the LoRA configuration and one with the dataset formatting function.

ⓧ I'd like to double check the full code

Your notebook is getting long, so scroll through and confirm each of these cells is present in order:

  • Cell 1: The pip install command for all packages.
  • Cell 2: All imports (torch, transformers, peft, trl, datasets, huggingface_hub).
  • Cell 3: HuggingFace login().
  • Cell 4: Model loaded in 4-bit with BitsAndBytesConfig, tokenizer loaded with pad_token set.
  • Cell 5: The classify_sentiment() function and test_reviews list with base model test output.
  • Cell 6: IMDB dataset loaded, small_train selected, format_for_sft() defined, and train_dataset created.
  • Cell 7: prepare_model_for_kbit_training(), LoraConfig, get_peft_model(), and print_trainable_parameters().

Your model is ready with LoRA adapters injected, and your training data is formatted. Next up, you will fire off the training loop and watch the loss drop in real time.

Train Your Model with SFTTrainer

In the last step, you injected LoRA adapters into your model and confirmed that only about 0.2% of the parameters are trainable. You also formatted your IMDB training data into the chat template that Qwen3 expects.

Now it is time to put those adapters to work. You will configure SFTTrainer from TRL to handle the training loop, then watch the training loss drop in real time as your model learns to classify sentiment.

In this step, get ready to:
  • Configure training arguments with SFTConfig.
  • Train your LoRA adapters with SFTTrainer and watch the loss decrease.
  • Save the trained adapter weights to disk.
Configure the training arguments and create the trainer

SFTConfig holds every setting that controls how training runs. SFTTrainer then combines those settings with your model, dataset, and LoRA config into a single object you can train with one call.

  • Create a new Colab cell and paste the following code to set up SFTConfig and SFTTrainer:
# Cell 8: Configure training
training_args = SFTConfig(
    output_dir="./qwen3-sentiment-lora",  # Where to save checkpoints
    num_train_epochs=2,                    # Two full passes through the data
    per_device_train_batch_size=4,         # Samples per GPU batch
    gradient_accumulation_steps=4,         # Accumulate gradients over 4 batches
    learning_rate=2e-4,                    # LoRA-friendly learning rate
    lr_scheduler_type="cosine",            # Gradually reduce learning rate
    warmup_ratio=0.05,                     # Warm up for 5% of training
    logging_steps=10,                      # Print loss every 10 steps
    save_strategy="epoch",                 # Save a checkpoint after each epoch
    fp16=True,                             # Use half precision (T4 compatible)
    max_seq_length=512,                    # Truncate sequences beyond 512 tokens
    report_to="none",                      # Skip external logging services
    optim="paged_adamw_8bit",              # Memory-efficient optimizer for QLoRA
    dataset_text_field="text",             # Column name in our formatted dataset
)

# Create the trainer with model, data, and LoRA config
trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    peft_config=lora_config,
)

What do these training arguments do?

  • per_device_train_batch_size=4 and gradient_accumulation_steps=4 work together to give you an effective batch size of 16 while keeping GPU memory low.
  • optim="paged_adamw_8bit" is the QLoRA-recommended optimizer. It uses paged memory management to reduce optimizer state memory on the T4.
  • fp16=True enables half-precision training. The T4 GPU supports fp16 natively, keeping training fast and memory-efficient.
  • logging_steps=10 prints the training loss every 10 optimizer steps, so you can watch the model learn in real time.

Why cosine scheduling and warmup?

The cosine scheduler starts at the full learning rate and smoothly reduces it to near zero by the end of training. This prevents large, destabilizing updates in the later stages when the model has already learned the bulk of the task.

The 5% warmup ratio ramps the learning rate up gradually during the first few steps. This gives the optimizer time to calibrate its internal estimates before making large weight updates.

  • Run the cell. You should see no output. The trainer is ready and waiting for the training command.

Seeing an import or name error?

  • Make sure you ran your Cell 2 (imports) earlier in this session. Both SFTTrainer and SFTConfig come from the trl package.
  • Confirm that lora_config and train_dataset exist by running their names in a new cell. If either shows a NameError, re-run the cell where you defined them (Cell 7 and Cell 6 respectively).
  • Help me debug a NameError when creating my SFTTrainer.
Train the model and watch the loss drop

This is the moment your model starts learning. When you call trainer.train(), the trainer feeds batches of formatted IMDB reviews through the model, computes the loss, and updates only the LoRA adapter weights.

Before you run this, think back to the base model's responses from Step 2. Do you think the training loss will be high or low at the start? What direction should it move if the model is learning?

  • Start training by running this in a new Colab cell:
# Cell 9: Train
trainer.train()

What should I see?

A progress bar appears with a table that logs the training loss at every 10 steps. The loss starts high (around 2.0 or above) because the base model has no idea how to produce clean sentiment labels yet.

Over the course of 2 epochs, the loss should drop noticeably, ending below 1.0. This means the adapter weights are learning to map reviews to the correct "positive" or "negative" label. On a Colab T4, expect this to take about 15 to 20 minutes.

Keep the tab active

Google Colab can disconnect if your browser tab is inactive for too long. Keep the tab visible or check back every few minutes so the runtime stays connected.

Getting an out-of-memory (OOM) error?

  • Go to Runtime > Restart session to clear GPU memory, then re-run all cells from Cell 1 through Cell 9.
  • If the OOM persists, reduce per_device_train_batch_size from 4 to 2 in your SFTConfig and re-run Cells 8 and 9.
  • Help me fix an OOM error during QLoRA training on a Colab T4.
Save the trained adapter to disk

Training is done, but the adapter weights only exist in GPU memory right now. You need to save them to disk so you can load them later for inference and merging.

  • Save the LoRA adapter by running this in a new Colab cell:
# Cell 10: Save adapter
trainer.save_model("./qwen3-sentiment-lora")

What does this save?

This saves only the LoRA adapter weights, not the entire model. The output directory contains adapter_config.json (your LoRA settings) and adapter_model.safetensors (the trained weights). Together they are just a few megabytes, compared to the full model's hundreds of megabytes.

  • Confirm the adapter files were saved by running this in a new cell:
!ls ./qwen3-sentiment-lora
  • You should see adapter_config.json and adapter_model.safetensors listed in the output.

Directory not found or empty?

  • Make sure the output_dir in your SFTConfig matches the path in save_model(). Both should be "./qwen3-sentiment-lora".
  • If you restarted the runtime after training, the trained weights are gone. You need to re-run Cells 8 and 9 to retrain before saving.
  • Help me figure out why my saved LoRA adapter directory is missing.

✔️ Awesome, I've got everything!

Great. Confirm you have saved all your Colab cells up to Cell 10.

ⓧ I'd like to double check the full code

Your notebook should now have Cells 1 through 10. Scroll through your notebook and confirm these three new cells from this step are present and match:

  • Cell 8: SFTConfig with 14 training arguments, followed by SFTTrainer creation with model, training_args, train_dataset, and peft_config=lora_config.
  • Cell 9: trainer.train() (should have already completed with decreasing loss).
  • Cell 10: trainer.save_model("./qwen3-sentiment-lora") (adapter files confirmed on disk).

Your adapter is trained and saved. In the next step, you will run the same test reviews from Step 2 through your fine-tuned model and see the difference firsthand. Then you will merge the adapter into the base model and push it to HuggingFace Hub.

Test Your Fine-Tuned Model and Push to Hub

Your model just trained for 2 epochs and the loss dropped noticeably. That's a good signal, but a lower loss number alone doesn't tell you whether the model actually learned to output clean sentiment labels.

In this step, you will run the exact same test reviews from Step 2 through your fine-tuned model and compare the outputs. Then you will merge the LoRA adapter back into the base model and push the result to HuggingFace Hub so it is ready for inference or deployment.

In this step, get ready to:
  • Test your fine-tuned model on the same reviews that stumped the base model.
  • Merge the LoRA adapter into the base model for standalone deployment.
  • Push your merged model to HuggingFace Hub.
Run your fine-tuned model on the test reviews

Back in Step 2, you ran three movie reviews through the base Qwen3-0.6B model and watched it produce long, inconsistent answers. The classify_sentiment function and the test_reviews list are still in memory. Before you run the fine-tuned model, take a moment to predict what you expect.

Do you think the model will output clean one-word labels now, or will it still be verbose? What about the mixed review?

  • Test your fine-tuned model on the same three reviews by running this code in a new Colab cell:
# Cell 11: Test fine-tuned model
print("=== FINE-TUNED MODEL RESPONSES ===")
for review in test_reviews:
    result = classify_sentiment(model, tokenizer, review)
    print(f"Review: {review[:60]}...")
    print(f"Response: {result}\n")

What does this code do?

This is the exact same loop you ran in Step 2, using the same classify_sentiment function and the same test_reviews list. The only difference is that model now points to your PEFT-wrapped fine-tuned model instead of the raw base model.

Running identical inputs through both versions is the cleanest way to see exactly what fine-tuning changed.

You should see each review followed by a clean one-word response: positive or negative. Compare that to the rambling, inconsistent answers from Step 2.

Still seeing verbose answers?

Make sure you ran all the training cells (Cell 8 and Cell 9) before this cell. If the model variable was overwritten or the kernel restarted, the fine-tuned weights are lost from memory.

Check that training completed successfully in Cell 9. If the training loss never dropped, the model did not learn the task.

Help me debug my fine-tuned model's output

Merge the LoRA adapter into the base model

Right now, your fine-tuned model exists as two pieces: the 4-bit quantized base model and a separate LoRA adapter file. For deployment, you want a single standalone model with the adapter baked in.

You cannot merge adapters into a 4-bit quantized model directly. You need to load a fresh copy of the base model in full precision first, then layer the adapter on top and merge.

  • Load a fresh base model, merge the adapter, and save the result by running this code in a new Colab cell:
# Cell 12: Merge adapter and push to Hub
# Load fresh base model in full precision (bf16) for merging
base_model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# Load the LoRA adapter on top and merge into the base weights
peft_model = PeftModel.from_pretrained(base_model, "./qwen3-sentiment-lora")
merged_model = peft_model.merge_and_unload()

# Save the standalone merged model and tokenizer to disk
merged_model.save_pretrained("./qwen3-sentiment-merged", safe_serialization=True)
tokenizer.save_pretrained("./qwen3-sentiment-merged")

What does this code do?

  • AutoModelForCausalLM.from_pretrained loads a fresh copy of the base Qwen3-0.6B in bfloat16 precision (not 4-bit this time) because the merge operation needs full-precision weights.
  • PeftModel.from_pretrained loads the LoRA adapter you saved to ./qwen3-sentiment-lora and layers it on top of the fresh base model.
  • merge_and_unload() folds the adapter weights directly into the base model's matrices, producing a single standalone model with no adapter dependency.
  • save_pretrained writes the merged model and tokenizer to ./qwen3-sentiment-merged using the safe serialization format (.safetensors).

Why load a fresh model instead of merging the one in memory?

The model currently in memory is quantized to 4-bit. Merging adapter weights into 4-bit quantized weights would lose precision and produce a lower-quality result.

By loading a fresh copy in bfloat16, the merge happens at full precision. The resulting model can be served as-is or re-quantized later for deployment.

  • After the cell finishes, verify the merged model was saved by running this in a new cell:
import os
os.listdir("./qwen3-sentiment-merged")

You should see files like config.json, model.safetensors, and tokenizer files listed in the output.

Seeing an out-of-memory error?

Loading the full-precision base model alongside the quantized model in memory can push the T4's 15GB VRAM limit. Delete the old model first by adding del model and torch.cuda.empty_cache() in a cell before Cell 12, then re-run Cell 12.

Help me fix an out-of-memory error when merging my LoRA adapter

Push your merged model to HuggingFace Hub

Your merged model is saved locally on the Colab disk, which disappears when the runtime ends. Pushing it to HuggingFace Hub gives it a permanent home you can load from anywhere.

  • Push the merged model and tokenizer to your HuggingFace Hub profile by running this in a new cell:
# Push merged model and tokenizer to HuggingFace Hub
merged_model.push_to_hub("[[HF_USERNAME="your-username"]]/qwen3-sentiment")
tokenizer.push_to_hub("[[HF_USERNAME="your-username"]]/qwen3-sentiment")

What does this code do?

push_to_hub creates a new repository on HuggingFace Hub (or updates an existing one) and uploads all the model and tokenizer files. Since you logged in during Step 1, the authentication is already handled.

  • Wait for the upload to complete. This may take a minute depending on your connection speed.
  • Open huggingface.co in a new browser tab and navigate to your profile.
  • Confirm that you see a repository called qwen3-sentiment listed under your models.

Upload failing or repository not appearing?

Make sure your HuggingFace token has write permissions. Tokens created with read-only access cannot push models. You can check your token permissions at huggingface.co/settings/tokens.

If you see a timeout error, try running the push cells again. Colab's network connection can be flaky.

Help me push my model to HuggingFace Hub

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook in Google Colab.

ⓧ I'd like to double check the full code

Scroll through your Colab notebook and confirm that these cells are present and in order:

  • Cell 1: Package installation (!pip install transformers peft trl datasets bitsandbytes accelerate).
  • Cell 2: All imports (torch, transformers, peft, trl, datasets, huggingface_hub).
  • Cell 3: HuggingFace login().
  • Cell 4: Load model in 4-bit with BitsAndBytesConfig and tokenizer.
  • Cell 5: classify_sentiment function, test_reviews list, and base model test.
  • Cell 6: Load IMDB dataset, create format_for_sft, and build train_dataset.
  • Cell 7: prepare_model_for_kbit_training, LoraConfig, and get_peft_model.
  • Cell 8: SFTConfig and SFTTrainer setup.
  • Cell 9: trainer.train().
  • Cell 10: trainer.save_model("./qwen3-sentiment-lora").
  • Cell 11: Test fine-tuned model on test_reviews.
  • Cell 12: Load fresh base model in bf16, merge adapter with merge_and_unload(), save merged model, and push to HuggingFace Hub.

Secret mission

Hyperparameter Experiments

You trained one LoRA configuration, but was it the best one? In this mission, you will experiment with different ranks, target modules, and learning rates to find out which hyperparameters actually drive fine-tuning quality. You will build a comparison table that shows the trade-offs between adapter size, training cost, and output quality.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on Google Colab's free tier and HuggingFace Hub's free public model hosting. There are no ongoing costs to worry about.

Decide whether to keep your resources available, pause them to come back later, or delete them entirely.

Resources you used:

  • Google Colab notebook (stored in Google Drive, free).
  • HuggingFace Hub model repository (free storage for public models).

Keep everything running

No action needed. Choose this if you plan to revisit the notebook or continue experimenting with your fine-tuned model.

  • Your Colab notebook stays in Google Drive at no cost.
  • Your merged model stays on HuggingFace Hub at no cost for public repositories.
  • The Colab runtime (GPU session) will automatically disconnect after a period of inactivity, freeing the GPU. Any temporary files on the runtime disk (like ./qwen3-sentiment-lora and ./qwen3-sentiment-merged) will be deleted when the runtime disconnects.

Pause - I'll come back to this later

Free up the GPU session but keep your notebook and model available for later.

  • In your Colab notebook, click Runtime in the top menu.
  • Select Disconnect and delete runtime. This releases the T4 GPU and deletes any temporary training files from the runtime disk.
  • Your notebook remains in Google Drive. Your model remains on HuggingFace Hub. You can reconnect to a new GPU runtime anytime by reopening the notebook.

Delete - I don't want to use this again

Remove all project resources completely. This cannot be undone.

  • First, disconnect the Colab runtime. Click Runtime in the top menu, then select Disconnect and delete runtime. This deletes all temporary training files from the runtime disk.
  • Delete the Colab notebook from Google Drive. Open Google Drive, find the notebook, right-click it, and select Move to trash.
  • Delete the HuggingFace Hub model repository. Open your model page at huggingface.co/your-username/qwen3-sentiment, click the Settings tab, scroll to the bottom, and select Delete this model. Confirm the deletion when prompted.

Nice Work!

Nice Work!

You just fine-tuned a language model to do exactly what you told it to do. Starting from a base Qwen3-0.6B that rambled and guessed at sentiment, you trained a model that outputs clean, reliable labels on a free Google Colab GPU.

You've learned how to:

  • Fine-tune Qwen3-0.6B using QLoRA, training only about 0.2% of the model's parameters on a free Colab T4 GPU.
  • Run the complete PEFT workflow from start to finish: 4-bit quantization, LoRA adapter injection, training with SFTTrainer, and merging the adapter back into the base model for deployment to HuggingFace Hub.
  • See firsthand why fine-tuning matters by comparing the base model's verbose, inconsistent responses to your fine-tuned model's clean "positive" and "negative" sentiment labels.
  • Secret Mission: Experimented with LoRA hyperparameters (rank, target modules, learning rate) and built an empirical comparison table showing what drives fine-tuning quality.

Ready to quiz yourself?