Fine-Tune an Engineering LLM on Kaggle

Fine-tune Qwen3-8B into a specialized engineering AI using free Kaggle GPUs.

Introduction

30 Second Summary

Generic AI chatbots struggle with specialized engineering questions. Ask one about PCB trace width calculations or STM32 firmware initialization and you'll often get vague, sometimes incorrect answers.

In this project, you will fine-tune your own engineering-specialized AI called "Arch AI" by training it on approximately 19,000 real engineering Q&A samples covering PCB design, embedded systems, and IoT.

What You'll Build

You'll fine-tune Qwen3-8B using QLoRA on Kaggle's free GPU, training it on domain-specific engineering datasets to create a model that outperforms generic chatbots on hardware and embedded systems questions.

By the end of this project, you'll have:

  • A complete fine-tuning pipeline running on Kaggle's free GPU with 4-bit quantization to fit an 8-billion parameter model in 16GB of VRAM.
  • A trained LoRA adapter uploaded to your Hugging Face profile as a published model that anyone can use.
  • Measurable before-and-after proof that your fine-tuned model outperforms the base model on engineering topics like PCB design, embedded firmware, and IoT.
  • Secret Mission: Export your model to GGUF format and run Arch AI locally on your own PC with Ollama.

Are there any prerequisites?

You'll need a free Kaggle account and a free Hugging Face account before starting. Basic Python knowledge is helpful but not required since all code is provided and explained step by step.

Set Up Your Kaggle Notebook

Fine-tuning an 8-billion-parameter model requires serious GPU power. Kaggle gives you free access to NVIDIA T4 GPUs with 16GB of VRAM each, which is exactly what you need to train Arch AI without spending a cent.

In this step, you will create a cloud notebook on Kaggle, attach a GPU accelerator, and install the libraries that make fine-tuning possible.

In this step, get ready to:
  • Create a new Kaggle notebook with GPU T4 x2 and Internet enabled.
  • Install Unsloth for high-speed fine-tuning.
  • Import all required libraries for model loading, dataset handling, and training.
Create a new Kaggle notebook and enable GPU

Kaggle notebooks run in the cloud, so you do not need to install anything locally. You just need to create a notebook and attach the right hardware.

  • Open kaggle.com and sign in to your account.
  • Click Create in the left sidebar, then select New Notebook.
  • A fresh notebook editor opens with a code cell ready to use.

Now attach the GPU accelerator so your notebook can train models on powerful hardware.

  • In the right sidebar, click Settings.
  • Find the Accelerator section and select GPU T4 x 2 from the dropdown.
  • Click Turn on when prompted.
  • The notebook restarts with the GPU attached.

Why GPU T4 x 2?

The T4 gives you 16GB of VRAM, which is enough to hold a 4-bit quantized 8B model plus training overhead. Kaggle offers 30 hours of free GPU time per week, and training Arch AI uses roughly 2-4 hours of that.

Next, confirm that Internet access is enabled so the notebook can download models and datasets.

  • Still in Settings, scroll to the Internet section.
  • Confirm the toggle is set to On. It should be on by default for new notebooks.

Don't see the Accelerator or Internet options?

Your Kaggle account must be phone-verified to unlock GPU and Internet access. Click your profile icon in the top right, select Settings, scroll to Phone verification, and verify your number.

After verification, refresh the notebook page and the options will appear. Help me enable GPU on Kaggle

Install Unsloth

Unsloth is a library that makes fine-tuning large language models up to 2x faster while using 70% less VRAM. This is what makes training an 8B model possible on Kaggle's free GPU.

  • Install Unsloth and its dependencies by pasting this into your first notebook cell and running it:
# --- Cell 1: Install Unsloth ---
!pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

What does this command do?

The ! prefix tells the notebook to run a terminal command instead of Python code.

The flags --upgrade --force-reinstall --no-cache-dir ensure you get a completely fresh install of the latest Unsloth version, avoiding conflicts with pre-installed packages in Kaggle's environment.

  • Wait for the cell to finish running. This takes 2-3 minutes as it downloads and installs packages.
  • You will see a stream of install logs. Look for a final line confirming the install completed without errors.

Seeing install errors or timeouts?

Make sure Internet is toggled On in your notebook settings. Without internet, package downloads will fail.

If the cell times out, click the stop button and re-run it. Kaggle's package mirrors occasionally have brief slowdowns. Help me fix Unsloth installation errors on Kaggle

Import the required libraries

With Unsloth installed, you can now import all the libraries needed for loading models, processing datasets, and running the training loop.

  • Add a new cell below Cell 1, paste the following imports, and run it:
# --- Cell 2: Imports ---
from unsloth import FastLanguageModel
from datasets import load_dataset, concatenate_datasets
from trl import SFTTrainer
from transformers import TrainingArguments
import torch

What are these libraries?

  • FastLanguageModel is Unsloth's optimized model loader that handles 4-bit quantization and LoRA adapter attachment.
  • load_dataset and concatenate_datasets come from Hugging Face's datasets library for loading and combining training data.
  • SFTTrainer from the trl library handles supervised fine-tuning with chat-formatted data.
  • TrainingArguments from transformers lets you configure hyperparameters like learning rate, batch size, and number of training steps.
  • torch is PyTorch, the deep learning framework everything runs on.
  • Confirm the cell runs without errors. If all imports succeed, you will see no output (a silent success).

Getting an ImportError?

If you see an error like ModuleNotFoundError: No module named 'unsloth', the kernel may need a restart after installation.

Click the Restart button at the top of the notebook, then re-run Cell 1 followed by Cell 2. Help me fix import errors after installing Unsloth

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook (Kaggle auto-saves, but you can press Ctrl+S to be safe).

ⓧ I'd like to double check the full code

Here is everything you should have across your two cells so far:

# --- Cell 1: Install Unsloth ---
!pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

# --- Cell 2: Imports ---
from unsloth import FastLanguageModel
from datasets import load_dataset, concatenate_datasets
from trl import SFTTrainer
from transformers import TrainingArguments
import torch

Your cloud training environment is locked and loaded. Next up, you will load real engineering datasets from Hugging Face and combine them into one massive training set for Arch AI.

Load the Engineering Datasets

Your notebook has Unsloth installed and all the necessary libraries imported. Now you need training data.

Fine-tuning a model means showing it thousands of examples of the behavior you want. For Arch AI, that means engineering Q&A conversations covering PCB design, embedded firmware, IoT protocols, and more. You will pull four specialized datasets from Hugging Face and merge them into one training set.

In this step, get ready to:
  • Load four specialized engineering datasets from Hugging Face.
  • Combine them into a single training-ready dataset.
  • Inspect the conversation format your model will learn from.
Load the four engineering datasets

The Mascarade dataset family on Hugging Face contains thousands of expert-level engineering conversations. Each dataset covers a different specialization.

  • Create a new code cell below Cell 2 by clicking the + Code button at the bottom of your last cell.
  • Paste the following code into Cell 3 to load all four datasets:
# --- Cell 3: Load Engineering Datasets ---
print("Loading engineering datasets...")
ds_embedded = load_dataset("electron-rare/mascarade-embedded-dataset", split="train")
ds_kicad = load_dataset("electron-rare/mascarade-kicad-dataset", split="train")
ds_stm32 = load_dataset("electron-rare/mascarade-stm32-dataset", split="train")
ds_iot = load_dataset("electron-rare/mascarade-iot-dataset", split="train")

print(f"Embedded: {len(ds_embedded)} samples")
print(f"KiCad/PCB: {len(ds_kicad)} samples")
print(f"STM32/ARM: {len(ds_stm32)} samples")
print(f"IoT: {len(ds_iot)} samples")

What do these datasets contain?

  • electron-rare/mascarade-embedded-dataset has 8,344 samples covering RTOS, device trees, drivers, bootloaders, and ARM Cortex-M.
  • electron-rare/mascarade-kicad-dataset has 2,645 samples covering schematic capture, PCB layout, footprints, and Gerber files.
  • electron-rare/mascarade-stm32-dataset has 2,012 samples covering HAL/LL drivers, CMSIS, FreeRTOS, UART, SPI, I2C, and DMA.
  • electron-rare/mascarade-iot-dataset has 6,005 samples covering ESP32, Wi-Fi, BLE, LoRaWAN, MQTT, and Home Assistant.

The split="train" argument tells Hugging Face to load the training split of each dataset directly into memory.

You won't see a change yet. The next chunk combines these datasets and prints a sample so you can verify everything loaded correctly.

Combine and inspect the training data

Four separate datasets are harder to work with than one. You will concatenate them into a single dataset and peek at the conversation format the model will learn from.

  • Add the following code below the lines you just pasted in Cell 3:
dataset = concatenate_datasets([ds_embedded, ds_kicad, ds_stm32, ds_iot])
print(f"\nTotal training samples: {len(dataset)}")

# Inspect a sample
print("\n--- Sample conversation ---")
print(dataset[0]["conversations"])

What does this code do?

  • concatenate_datasets merges all four datasets end-to-end into one unified dataset. The model trains on all of them together.
  • dataset[0]["conversations"] prints the first conversation so you can see the ShareGPT format: each sample has a system turn, a human turn (the question), and a gpt turn (the expert answer).
  • Run Cell 3 by pressing Shift+Enter or clicking the play button on the left of the cell.
  • Wait for all four datasets to download. You should see output showing sample counts for each dataset, a total of approximately 19,000 training samples, and a printed conversation with from and value keys.

What should I see?

Your output will look something like this:

Loading engineering datasets...
Embedded: 8344 samples
KiCad/PCB: 2645 samples
STM32/ARM: 2012 samples
IoT: 6005 samples

Total training samples: 19006

--- Sample conversation ---
[{'from': 'system', 'value': '...'}, {'from': 'human', 'value': '...'}, {'from': 'gpt', 'value': '...'}]

The exact text inside each conversation turn will vary, but the structure of three turns (system, human, gpt) is what matters.

Seeing a download error?

  • Make sure Internet access is toggled ON in your notebook settings (right sidebar under Session options).
  • Check that the dataset names are spelled exactly as shown. A single typo in the path will cause a 404 error.
  • If you see a timeout, try running Cell 3 again. Hugging Face downloads can occasionally stall on the first attempt.

Help me debug my dataset loading error in Kaggle.

✔️ Awesome, I've got everything!

Great. Make sure Cell 3 is saved and move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete notebook code so far, including Cells 1 through 3:

# --- Cell 1: Install Unsloth ---
!pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

# --- Cell 2: Imports ---
from unsloth import FastLanguageModel
from datasets import load_dataset, concatenate_datasets
from trl import SFTTrainer
from transformers import TrainingArguments
import torch

# --- Cell 3: Load Engineering Datasets ---
print("Loading engineering datasets...")
ds_embedded = load_dataset("electron-rare/mascarade-embedded-dataset", split="train")
ds_kicad = load_dataset("electron-rare/mascarade-kicad-dataset", split="train")
ds_stm32 = load_dataset("electron-rare/mascarade-stm32-dataset", split="train")
ds_iot = load_dataset("electron-rare/mascarade-iot-dataset", split="train")

print(f"Embedded: {len(ds_embedded)} samples")
print(f"KiCad/PCB: {len(ds_kicad)} samples")
print(f"STM32/ARM: {len(ds_stm32)} samples")
print(f"IoT: {len(ds_iot)} samples")

dataset = concatenate_datasets([ds_embedded, ds_kicad, ds_stm32, ds_iot])
print(f"\nTotal training samples: {len(dataset)}")

# Inspect a sample
print("\n--- Sample conversation ---")
print(dataset[0]["conversations"])

Your training data is locked and loaded. Next up, you will load the base Qwen3-8B model and attach LoRA adapters so it can learn from these 19,000 conversations without rewriting its entire brain.

Load the Base Model with 4-Bit Quantization

Your training data is ready. Now you need the actual brain that will learn from those ~19,000 engineering samples.

Qwen3-8B is an 8-billion parameter language model. Loading it at full precision would need over 30GB of memory. 4-bit quantization compresses the model so it fits comfortably on Kaggle's free 16GB T4 GPU, and LoRA adapters let you fine-tune just a tiny fraction of the parameters without touching the rest.

In this step, get ready to:
  • Load Qwen3-8B with 4-bit quantization.
  • Attach LoRA adapters to the model's attention and feed-forward layers.
  • Format the training dataset to match the model's chat template.
Load the Qwen3-8B base model

Unsloth provides pre-quantized models that are ready to load in 4-bit precision. This means the 8-billion parameter model uses roughly 5GB of VRAM instead of 30GB+.

  • Create a new code cell in your Kaggle notebook by clicking the + Code button.
  • Paste this code into the new cell:
# --- Cell 4: Load Base Model ---
print("Loading Qwen3-8B with 4-bit quantization...")
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen3-8B-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
    dtype=None,
)

What does this code do?

FastLanguageModel.from_pretrained downloads and loads the model from Hugging Face into your notebook's GPU memory.

  • model_name="unsloth/Qwen3-8B-bnb-4bit" points to the pre-quantized version hosted by Unsloth.
  • load_in_4bit=True enables 4-bit quantization, reducing memory use by roughly 4x.
  • max_seq_length=2048 sets the maximum token length per training example.
  • dtype=None lets Unsloth auto-detect the best precision for your GPU (float16 on T4).

What is 4-bit quantization?

Quantization compresses model weights from 16-bit (2 bytes each) down to 4-bit (half a byte). This cuts memory usage by 4x while preserving most of the model's capabilities.

The base model stays frozen in 4-bit. You'll train small adapter layers in full precision on top of it. This technique is called QLoRA.

Attach LoRA adapters to the model

LoRA adapters are small trainable matrices injected into the model's layers. Instead of updating all 8 billion parameters, you only train these lightweight adapters.

  • In the same cell (Cell 4), paste this code below what you already have:
print("Attaching LoRA adapters...")
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=42,
    max_seq_length=2048,
    use_rslora=False,
    loftq_config=None,
)

trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
total_params = sum(p.numel() for p in model.parameters())
print(f"Trainable parameters: {trainable_params / 1e6:.1f}M / {total_params / 1e6:.1f}M total ({100 * trainable_params / total_params:.2f}%)")

What do these LoRA settings mean?

  • r=16 is the rank of the adapter matrices. Higher rank means more learning capacity, but also more parameters to train. 16 is a good balance.
  • target_modules lists every linear layer that gets an adapter. You're targeting all attention layers (q, k, v, o) plus the feed-forward layers (gate, up, down).
  • lora_alpha=16 controls how strongly the adapter influences the base model's output.
  • use_gradient_checkpointing="unsloth" trades a small amount of speed for a large VRAM reduction during training.
  • Run Cell 4 by pressing the play button or Shift+Enter.
  • Wait for the model to download and load. This takes 2-5 minutes depending on network speed.

Once the cell finishes, you should see a line showing your trainable parameters as a small percentage of the total. Something like Trainable parameters: 83.9M / 8236.5M total (1.02%).

Why is the trainable percentage so low?

That's the whole point of LoRA. You're only training about 1% of the model's parameters. The other 99% stays frozen in 4-bit. This is why fine-tuning fits on a free GPU that would never be able to train the full model.

Seeing an out-of-memory error?

  • Make sure your Kaggle accelerator is set to GPU T4 x2. Check under Settings on the right sidebar.
  • Restart your notebook session (Runtime > Restart session) and re-run all cells from the top. Leftover variables from earlier runs can eat VRAM.

Help me fix a VRAM out-of-memory error when loading Qwen3-8B on Kaggle T4

Format the dataset for the model's chat template

The Mascarade datasets use ShareGPT format with roles like human and gpt. Qwen3's tokenizer expects ChatML format with <|im_start|> and <|im_end|> tokens. You need a conversion function to bridge the two.

  • Create a new code cell (Cell 5) and paste this code:
# --- Cell 5: Format Dataset for Training ---
def format_conversations(example):
    """Convert ShareGPT format to ChatML for Unsloth."""
    conversations = example["conversations"]
    formatted = ""
    for turn in conversations:
        role = turn["from"]
        content = turn["value"]
        if role == "system":
            formatted += f"<|im_start|>system\n{content}<|im_end|>\n"
        elif role == "human":
            formatted += f"<|im_start|>user\n{content}<|im_end|>\n"
        elif role == "gpt":
            formatted += f"<|im_start|>assistant\n{content}<|im_end|>\n"
    return {"text": formatted}

dataset = dataset.map(format_conversations)

What does this code do?

The format_conversations function takes each conversation from the dataset and rewrites it into ChatML format.

  • It maps the role "human" to user and "gpt" to assistant.
  • Each turn gets wrapped with the special tokens (<|im_start|> and <|im_end|>) that Qwen3's tokenizer recognizes.
  • dataset.map(format_conversations) applies this function to every sample and adds a new text field containing the formatted result.
  • Run Cell 5 by pressing Shift+Enter.

You should see a progress bar as the mapping runs across all ~19,000 samples. It finishes in under a minute.

Seeing a KeyError on 'conversations'?

  • This means the dataset variable was overwritten or lost. Go back and re-run Cell 3 to reload and concatenate the datasets, then run Cell 5 again.

Help me fix a KeyError when mapping my dataset

✔️ Awesome, I've got everything!

Great. Double check that you've saved your notebook and that both Cell 4 and Cell 5 ran without errors.

ⓧ I'd like to double check the full code

Here is the full cumulative code across all cells so far:

# --- Cell 1: Install Unsloth ---
!pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

# --- Cell 2: Imports ---
from unsloth import FastLanguageModel
from datasets import load_dataset, concatenate_datasets
from trl import SFTTrainer
from transformers import TrainingArguments
import torch

# --- Cell 3: Load Engineering Datasets ---
print("Loading engineering datasets...")
ds_embedded = load_dataset("electron-rare/mascarade-embedded-dataset", split="train")
ds_kicad = load_dataset("electron-rare/mascarade-kicad-dataset", split="train")
ds_stm32 = load_dataset("electron-rare/mascarade-stm32-dataset", split="train")
ds_iot = load_dataset("electron-rare/mascarade-iot-dataset", split="train")

print(f"Embedded: {len(ds_embedded)} samples")
print(f"KiCad/PCB: {len(ds_kicad)} samples")
print(f"STM32/ARM: {len(ds_stm32)} samples")
print(f"IoT: {len(ds_iot)} samples")

dataset = concatenate_datasets([ds_embedded, ds_kicad, ds_stm32, ds_iot])
print(f"\nTotal training samples: {len(dataset)}")

# Inspect a sample
print("\n--- Sample conversation ---")
print(dataset[0]["conversations"])

# --- Cell 4: Load Base Model ---
print("Loading Qwen3-8B with 4-bit quantization...")
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen3-8B-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
    dtype=None,
)

print("Attaching LoRA adapters...")
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=42,
    max_seq_length=2048,
    use_rslora=False,
    loftq_config=None,
)

trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
total_params = sum(p.numel() for p in model.parameters())
print(f"Trainable parameters: {trainable_params / 1e6:.1f}M / {total_params / 1e6:.1f}M total ({100 * trainable_params / total_params:.2f}%)")

# --- Cell 5: Format Dataset for Training ---
def format_conversations(example):
    """Convert ShareGPT format to ChatML for Unsloth."""
    conversations = example["conversations"]
    formatted = ""
    for turn in conversations:
        role = turn["from"]
        content = turn["value"]
        if role == "system":
            formatted += f"<|im_start|>system\n{content}<|im_end|>\n"
        elif role == "human":
            formatted += f"<|im_start|>user\n{content}<|im_end|>\n"
        elif role == "gpt":
            formatted += f"<|im_start|>assistant\n{content}<|im_end|>\n"
    return {"text": formatted}

dataset = dataset.map(format_conversations)

Your model is loaded, LoRA adapters are attached, and your dataset is formatted. Next up, you'll actually train Arch AI on those engineering samples and watch it learn.

Train Arch AI

Your model is loaded, your data is formatted, and your LoRA adapters are attached. Now it's time to actually teach Arch AI about engineering.

Training an LLM on a free GPU requires carefully chosen hyperparameters. You need to balance learning speed against the 16GB VRAM limit of the T4 GPU and Kaggle's 30-hour weekly quota.

In this step, get ready to:
  • Configure the SFTTrainer with training hyperparameters.
  • Train Arch AI for 500 steps on the engineering dataset.
  • Monitor training loss to confirm the model is learning.
Configure the SFTTrainer and start training

The SFTTrainer (Supervised Fine-Tuning Trainer) handles the entire training loop for you. You configure it once with the model, dataset, and hyperparameters, then kick off training with a single call.

  • Create a new cell in your Kaggle notebook below Cell 5, then paste the following code:
# --- Cell 6: Train Arch AI ---
print("Starting training...")
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=2048,
    packing=True,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        max_steps=500,
        learning_rate=2e-4,
        fp16=not torch.cuda.is_bf16_supported(),
        bf16=torch.cuda.is_bf16_supported(),
        logging_steps=25,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="cosine",
        seed=42,
        output_dir="outputs",
    ),
)

stats = trainer.train()
print(f"\nTraining complete! Final loss: {stats.training_loss:.4f}")

What does this code do?

The SFTTrainer receives your model, tokenizer, and the formatted dataset. It reads from the text field that you created in Cell 5.

  • packing=True concatenates short samples together so the GPU processes full sequences instead of wasting compute on padding tokens. This makes training up to 5x faster.
  • per_device_train_batch_size=2 with gradient_accumulation_steps=4 gives an effective batch size of 8. This fits within the T4's 16GB VRAM while still giving stable gradient updates.
  • max_steps=500 limits total training to 500 optimizer steps. This covers a meaningful portion of the ~19,000 samples and keeps the run well within Kaggle's 30-hour weekly GPU quota.
  • The fp16/bf16 toggle auto-detects your GPU's capabilities. The T4 does not support bfloat16, so it falls back to float16 automatically.
  • optim="adamw_8bit" uses an 8-bit optimizer to reduce VRAM usage further without hurting training quality.
  • Run Cell 6 by pressing **Shift+Enter**.

How long does training take?

Training 500 steps on a T4 GPU typically takes 2-4 hours. You'll see a progress bar and loss values logged every 25 steps. You can leave the tab open and come back later.

Seeing an out-of-memory error?

  • Make sure your notebook accelerator is set to **GPU T4 x2**, not CPU. Check under **Settings** on the right side panel.
  • If you ran other notebooks recently, your GPU session might have leftover memory usage. Click **Runtime** then **Restart session** and re-run all cells from Cell 1.
  • Confirm that load_in_4bit=True is set in Cell 4. Without 4-bit quantization, the model won't fit on the T4.

Help me debug an out-of-memory error during Arch AI training on Kaggle T4

Monitor training loss to confirm learning

As training runs, Kaggle prints a log line every 25 steps showing the current loss value. This number tells you whether the model is actually learning from the engineering data.

  • Watch the loss values in the output below Cell 6 as training progresses.
  • Confirm that the loss is decreasing over time. Early steps might show values around 1.5-2.0, trending down toward 0.8-1.2 by step 500.

What is training loss?

Training loss measures how wrong the model's predictions are. A decreasing loss means the model is getting better at predicting the next token in your engineering conversations.

You don't need the loss to reach zero. A steady downward trend confirms that your QLoRA adapters are absorbing engineering knowledge from the dataset.

  • Once training finishes, you should see a message like Training complete! Final loss: 0.XXXX printed below the cell.

Training loss not decreasing?

  • If loss stays flat or jumps wildly, check that Cell 5 ran successfully. The format_conversations function must have added a text field to every sample.
  • Confirm dataset_text_field="text" matches the field name you created. A mismatch means the trainer reads empty data.

Help me understand why my fine-tuning loss is not decreasing

✔️ Awesome, I've got everything!

Great. Double check you've run Cell 6 and see the final loss printed.

ⓧ I'd like to double check the full code

Here's everything your notebook should contain through Cell 6:

# --- Cell 1: Install Unsloth ---
!pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

# --- Cell 2: Imports ---
from unsloth import FastLanguageModel
from datasets import load_dataset, concatenate_datasets
from trl import SFTTrainer
from transformers import TrainingArguments
import torch

# --- Cell 3: Load Engineering Datasets ---
print("Loading engineering datasets...")
ds_embedded = load_dataset("electron-rare/mascarade-embedded-dataset", split="train")
ds_kicad = load_dataset("electron-rare/mascarade-kicad-dataset", split="train")
ds_stm32 = load_dataset("electron-rare/mascarade-stm32-dataset", split="train")
ds_iot = load_dataset("electron-rare/mascarade-iot-dataset", split="train")

print(f"Embedded: {len(ds_embedded)} samples")
print(f"KiCad/PCB: {len(ds_kicad)} samples")
print(f"STM32/ARM: {len(ds_stm32)} samples")
print(f"IoT: {len(ds_iot)} samples")

dataset = concatenate_datasets([ds_embedded, ds_kicad, ds_stm32, ds_iot])
print(f"\nTotal training samples: {len(dataset)}")

# Inspect a sample
print("\n--- Sample conversation ---")
print(dataset[0]["conversations"])

# --- Cell 4: Load Base Model ---
print("Loading Qwen3-8B with 4-bit quantization...")
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen3-8B-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
    dtype=None,
)

print("Attaching LoRA adapters...")
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=42,
    max_seq_length=2048,
    use_rslora=False,
    loftq_config=None,
)

trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
total_params = sum(p.numel() for p in model.parameters())
print(f"Trainable parameters: {trainable_params / 1e6:.1f}M / {total_params / 1e6:.1f}M total ({100 * trainable_params / total_params:.2f}%)")

# --- Cell 5: Format Dataset for Training ---
def format_conversations(example):
    """Convert ShareGPT format to ChatML for Unsloth."""
    conversations = example["conversations"]
    formatted = ""
    for turn in conversations:
        role = turn["from"]
        content = turn["value"]
        if role == "system":
            formatted += f"<|im_start|>system\n{content}<|im_end|>\n"
        elif role == "human":
            formatted += f"<|im_start|>user\n{content}<|im_end|>\n"
        elif role == "gpt":
            formatted += f"<|im_start|>assistant\n{content}<|im_end|>\n"
    return {"text": formatted}

dataset = dataset.map(format_conversations)

# --- Cell 6: Train Arch AI ---
print("Starting training...")
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=2048,
    packing=True,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        max_steps=500,
        learning_rate=2e-4,
        fp16=not torch.cuda.is_bf16_supported(),
        bf16=torch.cuda.is_bf16_supported(),
        logging_steps=25,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="cosine",
        seed=42,
        output_dir="outputs",
    ),
)

stats = trainer.train()
print(f"\nTraining complete! Final loss: {stats.training_loss:.4f}")

Arch AI has now absorbed thousands of engineering conversations. Next up, you'll put it to the test by comparing its responses against the base model on real engineering questions.

Test Arch AI vs the Base Model

Training is complete and your loss curve confirms Arch AI learned from the engineering dataset. Now it's time to put your model to the test.

You'll ask Arch AI real engineering questions spanning PCB design, embedded firmware, and IoT networking. A generic base model gives vague, surface-level answers to these topics. Your fine-tuned model should respond with specific calculations, register-level code, and protocol details.

In this step, get ready to:
  • Ask Arch AI engineering-specific questions across PCB, embedded, and IoT topics.
  • Compare Arch AI's responses against what a generic base model would produce.
  • Document the measurable improvement in response quality.
Switch to inference mode and ask engineering questions

During training, the model uses extra memory for gradients and optimizer states. Switching to inference mode frees that memory and enables 2x faster generation.

  • Create a new code cell in your Kaggle notebook (Cell 7) and paste the following code:
# --- Cell 7: Test Arch AI ---
print("\n--- Testing Arch AI ---")
FastLanguageModel.for_inference(model)

test_questions = [
    "How do I calculate trace width for a 2A current on a 2-layer PCB with 1oz copper?",
    "Write a bare-metal UART initialization for STM32F4 at 115200 baud.",
    "Explain how to set up an ESP32 as an MQTT client with TLS.",
]

for question in test_questions:
    messages = [
        {"role": "system", "content": "You are Arch AI, an expert engineering assistant specializing in PCB design, embedded systems, robotics, and hardware engineering."},
        {"role": "user", "content": question},
    ]
    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_tensors="pt",
        enable_thinking=False,
    ).to("cuda")

    outputs = model.generate(
        input_ids=inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
    )
    response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
    print(f"\nQ: {question}")
    print(f"A: {response[:500]}...")
    print("-" * 60)

What does this code do?

  • FastLanguageModel.for_inference(model) switches the model from training mode to optimized inference mode, freeing VRAM and enabling 2x faster generation.
  • enable_thinking=False disables Qwen3's internal reasoning tokens so you get a direct answer without a hidden chain-of-thought step.
  • The three test questions each target a different engineering domain from the training data: PCB design, STM32 firmware, and IoT networking.
  • temperature=0.7 and top_p=0.9 add slight randomness so the model doesn't just repeat memorized text verbatim.
  • Run the cell by pressing Shift+Enter.
  • Wait for all three responses to print. Each question takes 10-30 seconds to generate.

You should see three Q&A pairs printed, each with a detailed engineering response followed by a line of dashes.

Seeing a CUDA out of memory error?

The T4 GPU has 16GB of VRAM. If training used too much memory, restart the notebook kernel and rerun all cells from the top. This is rare with the 4-bit quantized model but can happen if you ran extra experiments between cells.

Help me debug a CUDA out of memory error during inference

Compare Arch AI's responses to the base model

The real proof of fine-tuning is visible in the quality gap between your model and a generic one. A base Qwen3-8B without engineering training would answer these same questions with surface-level explanations.

Look at Arch AI's responses and check for these signs of engineering specialization:

  • PCB trace width question: Arch AI should reference IPC-2221 standards, mention the formula involving current, temperature rise, and copper thickness, or give a specific mil/mm width value.
  • STM32 UART question: Arch AI should provide register-level code (e.g. USART1->BRR, RCC->APB2ENR) rather than generic pseudocode.
  • ESP32 MQTT question: Arch AI should mention TLS certificate configuration, the WiFiClientSecure library or mbedTLS, and specific MQTT broker connection steps.

Why is the comparison conceptual instead of side-by-side?

Running both the base model and the fine-tuned model simultaneously would require loading two copies of the 8B parameter model. That exceeds the 16GB VRAM available on a T4.

Instead, you compare against what a generic chatbot would say. If you've ever asked a general-purpose model about STM32 register addresses, you know the answers are usually high-level overviews without working code.

Document the improvement in response quality

Take a moment to read through each response carefully. The improvement is clearest when Arch AI uses domain-specific terminology, provides exact numerical values, or writes compilable code snippets.

✔️ Awesome, I've got everything!

Great. Double check you've saved your notebook before moving on.

ⓧ I'd like to double check the full code

# --- Cell 1: Install Unsloth ---
!pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

# --- Cell 2: Imports ---
from unsloth import FastLanguageModel
from datasets import load_dataset, concatenate_datasets
from trl import SFTTrainer
from transformers import TrainingArguments
import torch

# --- Cell 3: Load Engineering Datasets ---
print("Loading engineering datasets...")
ds_embedded = load_dataset("electron-rare/mascarade-embedded-dataset", split="train")
ds_kicad = load_dataset("electron-rare/mascarade-kicad-dataset", split="train")
ds_stm32 = load_dataset("electron-rare/mascarade-stm32-dataset", split="train")
ds_iot = load_dataset("electron-rare/mascarade-iot-dataset", split="train")

print(f"Embedded: {len(ds_embedded)} samples")
print(f"KiCad/PCB: {len(ds_kicad)} samples")
print(f"STM32/ARM: {len(ds_stm32)} samples")
print(f"IoT: {len(ds_iot)} samples")

dataset = concatenate_datasets([ds_embedded, ds_kicad, ds_stm32, ds_iot])
print(f"\nTotal training samples: {len(dataset)}")

# Inspect a sample
print("\n--- Sample conversation ---")
print(dataset[0]["conversations"])

# --- Cell 4: Load Base Model ---
print("Loading Qwen3-8B with 4-bit quantization...")
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen3-8B-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
    dtype=None,
)

print("Attaching LoRA adapters...")
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=42,
    max_seq_length=2048,
    use_rslora=False,
    loftq_config=None,
)

trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
total_params = sum(p.numel() for p in model.parameters())
print(f"Trainable parameters: {trainable_params / 1e6:.1f}M / {total_params / 1e6:.1f}M total ({100 * trainable_params / total_params:.2f}%)")

# --- Cell 5: Format Dataset for Training ---
def format_conversations(example):
    """Convert ShareGPT format to ChatML for Unsloth."""
    conversations = example["conversations"]
    formatted = ""
    for turn in conversations:
        role = turn["from"]
        content = turn["value"]
        if role == "system":
            formatted += f"<|im_start|>system\n{content}<|im_end|>\n"
        elif role == "human":
            formatted += f"<|im_start|>user\n{content}<|im_end|>\n"
        elif role == "gpt":
            formatted += f"<|im_start|>assistant\n{content}<|im_end|>\n"
    return {"text": formatted}

dataset = dataset.map(format_conversations)

# --- Cell 6: Train Arch AI ---
print("Starting training...")
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=2048,
    packing=True,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        max_steps=500,
        learning_rate=2e-4,
        fp16=not torch.cuda.is_bf16_supported(),
        bf16=torch.cuda.is_bf16_supported(),
        logging_steps=25,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="cosine",
        seed=42,
        output_dir="outputs",
    ),
)

stats = trainer.train()
print(f"\nTraining complete! Final loss: {stats.training_loss:.4f}")

# --- Cell 7: Test Arch AI ---
print("\n--- Testing Arch AI ---")
FastLanguageModel.for_inference(model)

test_questions = [
    "How do I calculate trace width for a 2A current on a 2-layer PCB with 1oz copper?",
    "Write a bare-metal UART initialization for STM32F4 at 115200 baud.",
    "Explain how to set up an ESP32 as an MQTT client with TLS.",
]

for question in test_questions:
    messages = [
        {"role": "system", "content": "You are Arch AI, an expert engineering assistant specializing in PCB design, embedded systems, robotics, and hardware engineering."},
        {"role": "user", "content": question},
    ]
    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_tensors="pt",
        enable_thinking=False,
    ).to("cuda")

    outputs = model.generate(
        input_ids=inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
    )
    response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
    print(f"\nQ: {question}")
    print(f"A: {response[:500]}...")
    print("-" * 60)

Your model is producing real engineering knowledge. Next up, you'll save Arch AI and upload it to Hugging Face so anyone can use it.

Upload Arch AI to Hugging Face

You've tested Arch AI and confirmed it produces better engineering answers than the base model. Now it's time to share your work with the world.

In this step, you'll save the trained LoRA adapter and push it to Hugging Face Hub so anyone can download and use your fine-tuned model.

In this step, get ready to:
  • Save the trained LoRA adapter locally.
  • Push the model and tokenizer to your Hugging Face Hub account.
  • Verify the upload is live on Hugging Face.
Get your Hugging Face access token

To push a model to Hugging Face Hub, you need a write access token that proves you own the account.

  • Open a new browser tab and go to huggingface.co/settings/tokens.
  • Click Create a new token.
  • Give your token a name like kaggle-upload.
  • Set the role to Write (this grants permission to create and push to repositories).
  • Click Create token.
  • Copy the token and keep it somewhere safe. You will paste it into your notebook code in the next substep.

Why do I need a write token?

Hugging Face tokens come in two roles: read and write. A read token only lets you download models. A write token lets you create new repositories and upload model files, which is exactly what push_to_hub needs.

Save and push Arch AI to Hugging Face

With your token ready, you can now save the adapter weights locally and upload them to your Hugging Face profile.

  • In your Kaggle notebook, create a new code cell below Cell 7.
  • Paste the following code into the new cell, replacing your-hf-username with your actual Hugging Face username and your-hf-token-here with the write token you just created:
# --- Cell 8: Upload to Hugging Face ---
print("Saving and uploading Arch AI to Hugging Face...")
model.save_pretrained("arch-ai-lora")
tokenizer.save_pretrained("arch-ai-lora")

model.push_to_hub("[[HF_USERNAME=\"your-hf-username\"]]/arch-ai-engineering", token="your-hf-token-here")
tokenizer.push_to_hub("[[HF_USERNAME=\"your-hf-username\"]]/arch-ai-engineering", token="your-hf-token-here")
print("Upload complete! Your model is live on Hugging Face.")

What does this code do?

  • model.save_pretrained("arch-ai-lora") saves your trained LoRA adapter weights to a local folder called arch-ai-lora. This is only the adapter (around 100MB), not the full 8B parameter model.
  • tokenizer.save_pretrained("arch-ai-lora") saves the tokenizer config alongside the adapter so they stay paired together.
  • push_to_hub uploads both files to a new repository on your Hugging Face account. The repo will be named arch-ai-engineering under your username.
  • Run the cell by pressing Shift+Enter.
  • Wait for the upload to complete. You should see the message Upload complete! Your model is live on Hugging Face. printed in the output.

Seeing an authentication error?

  • Double-check that you copied the full token string with no extra spaces at the beginning or end.
  • Make sure the token has Write permissions. A read-only token cannot create repositories.
  • Confirm that Internet access is still enabled in your Kaggle notebook settings (the toggle at the top of the sidebar).

Help me fix a Hugging Face authentication error in my Kaggle notebook.

Verify your model on Hugging Face

The upload created a brand-new repository on Hugging Face containing your LoRA adapter files.

  • In your browser, navigate to huggingface.co/your-hf-username/arch-ai-engineering.
  • You should see a model repository page with files like adapter_config.json and adapter_model.safetensors listed under the Files and versions tab.

What are these files?

The adapter_config.json file stores the LoRA configuration (rank, target modules, alpha). The adapter_model.safetensors file contains the actual trained weights.

Anyone can now load your adapter on top of the base Qwen3-8B model and get your engineering-specialized responses.

✔️ Awesome, I've got everything!

Great. Double check you have the upload confirmation message printed in your notebook output.

ⓧ I'd like to double check the full code

Here is the complete notebook code through Cell 8:

# --- Cell 1: Install Unsloth ---
!pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

# --- Cell 2: Imports ---
from unsloth import FastLanguageModel
from datasets import load_dataset, concatenate_datasets
from trl import SFTTrainer
from transformers import TrainingArguments
import torch

# --- Cell 3: Load Engineering Datasets ---
print("Loading engineering datasets...")
ds_embedded = load_dataset("electron-rare/mascarade-embedded-dataset", split="train")
ds_kicad = load_dataset("electron-rare/mascarade-kicad-dataset", split="train")
ds_stm32 = load_dataset("electron-rare/mascarade-stm32-dataset", split="train")
ds_iot = load_dataset("electron-rare/mascarade-iot-dataset", split="train")

print(f"Embedded: {len(ds_embedded)} samples")
print(f"KiCad/PCB: {len(ds_kicad)} samples")
print(f"STM32/ARM: {len(ds_stm32)} samples")
print(f"IoT: {len(ds_iot)} samples")

dataset = concatenate_datasets([ds_embedded, ds_kicad, ds_stm32, ds_iot])
print(f"\nTotal training samples: {len(dataset)}")

# Inspect a sample
print("\n--- Sample conversation ---")
print(dataset[0]["conversations"])

# --- Cell 4: Load Base Model ---
print("Loading Qwen3-8B with 4-bit quantization...")
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen3-8B-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
    dtype=None,
)

print("Attaching LoRA adapters...")
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=42,
    max_seq_length=2048,
    use_rslora=False,
    loftq_config=None,
)

trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
total_params = sum(p.numel() for p in model.parameters())
print(f"Trainable parameters: {trainable_params / 1e6:.1f}M / {total_params / 1e6:.1f}M total ({100 * trainable_params / total_params:.2f}%)")

# --- Cell 5: Format Dataset for Training ---
def format_conversations(example):
    """Convert ShareGPT format to ChatML for Unsloth."""
    conversations = example["conversations"]
    formatted = ""
    for turn in conversations:
        role = turn["from"]
        content = turn["value"]
        if role == "system":
            formatted += f"<|im_start|>system\n{content}<|im_end|>\n"
        elif role == "human":
            formatted += f"<|im_start|>user\n{content}<|im_end|>\n"
        elif role == "gpt":
            formatted += f"<|im_start|>assistant\n{content}<|im_end|>\n"
    return {"text": formatted}

dataset = dataset.map(format_conversations)

# --- Cell 6: Train Arch AI ---
print("Starting training...")
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=2048,
    packing=True,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        max_steps=500,
        learning_rate=2e-4,
        fp16=not torch.cuda.is_bf16_supported(),
        bf16=torch.cuda.is_bf16_supported(),
        logging_steps=25,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="cosine",
        seed=42,
        output_dir="outputs",
    ),
)

stats = trainer.train()
print(f"\nTraining complete! Final loss: {stats.training_loss:.4f}")

# --- Cell 7: Test Arch AI ---
print("\n--- Testing Arch AI ---")
FastLanguageModel.for_inference(model)

test_questions = [
    "How do I calculate trace width for a 2A current on a 2-layer PCB with 1oz copper?",
    "Write a bare-metal UART initialization for STM32F4 at 115200 baud.",
    "Explain how to set up an ESP32 as an MQTT client with TLS.",
]

for question in test_questions:
    messages = [
        {"role": "system", "content": "You are Arch AI, an expert engineering assistant specializing in PCB design, embedded systems, robotics, and hardware engineering."},
        {"role": "user", "content": question},
    ]
    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_tensors="pt",
        enable_thinking=False,
    ).to("cuda")

    outputs = model.generate(
        input_ids=inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
    )
    response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
    print(f"\nQ: {question}")
    print(f"A: {response[:500]}...")
    print("-" * 60)

# --- Cell 8: Upload to Hugging Face ---
print("Saving and uploading Arch AI to Hugging Face...")
model.save_pretrained("arch-ai-lora")
tokenizer.save_pretrained("arch-ai-lora")

model.push_to_hub("your-hf-username/arch-ai-engineering", token="your-hf-token-here")
tokenizer.push_to_hub("your-hf-username/arch-ai-engineering", token="your-hf-token-here")
print("Upload complete! Your model is live on Hugging Face.")

Your fine-tuned engineering AI is now live and publicly available. Next up, take on the Secret Mission to export Arch AI to GGUF format and run it locally on your own PC with Ollama.

Secret mission

Export to GGUF and Run Locally with Ollama

Your fine-tuned Arch AI lives in the cloud right now. What if you could run it entirely offline on your own PC? In this secret mission, you will export Arch AI to GGUF format and import it into Ollama so you can chat with your engineering AI locally, no internet required.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on Kaggle's free GPU and Hugging Face's free model hosting, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Kaggle notebook (cloud, stores your fine-tuning code and output files).
  • Hugging Face model repository (cloud, stores your LoRA adapter weights).
  • Local GGUF file and Modelfile on your Windows PC.
  • Ollama and the Arch AI model on your Windows PC.

Keep everything running

No action needed. Choose this if you plan to keep iterating on Arch AI, retraining with more data, or chatting with it locally.

  • Your Kaggle notebook stays in your account and can be re-run anytime within the 30-hour weekly GPU quota.
  • Your Hugging Face model repo remains publicly accessible for anyone to download and use.
  • Ollama stays installed on your PC and you can chat with Arch AI anytime by running ollama run arch-ai.

Pause - I'll come back to this later

Stop any running Kaggle sessions to free up GPU quota for later. Your files stay intact.

  • In Kaggle, open your notebook and click Stop Session if a GPU session is active.
  • On your Windows PC, stop the Ollama background process by right-clicking the Ollama icon in the system tray and selecting Quit Ollama.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow these steps to fully clean up.

Delete your Kaggle notebook
  • Open your notebook on Kaggle.
  • Click the three-dot menu (More options) at the top-right of the notebook.
  • Select Delete and confirm when prompted.
Delete your Hugging Face model repository
  • Go to your model page at huggingface.co/your-hf-username/arch-ai-engineering.
  • Click Settings in the model repo navigation.
  • Scroll to the bottom and click Delete this repository.
  • Type the repository name to confirm, then click I understand, delete this repository.
Delete local GGUF file and Modelfile
  • Open File Explorer and navigate to the folder where you saved arch-ai-gguf-q4.gguf and your Modelfile.
  • Select both files and delete them.
Uninstall Ollama
  • Open Windows Settings.
  • Go to Apps, then Apps & features.
  • Search for Ollama in the list.
  • Click Uninstall and follow the prompts.
  • When the uninstaller asks about removing models, leave the checkbox checked to remove all downloaded model data.

Nice Work!

Nice Work!

You just built your own specialized AI from scratch. Arch AI is a fully trained, domain-specific engineering assistant that lives on your Hugging Face profile for anyone to use.

You've learned how to:

  • Built a complete LLM fine-tuning pipeline on Kaggle's free GPU using Unsloth and QLoRA 4-bit quantization to train a model that would otherwise require enterprise hardware.
  • Trained Arch AI on ~19,000 engineering Q&A samples covering PCB design, embedded systems, IoT, power electronics, and DSP, then proved measurable improvement over the base model.
  • Published your trained LoRA adapter to Hugging Face Hub so anyone in the world can download and use your engineering-specialized model.
  • Exported Arch AI to GGUF format and ran it locally on your own PC with Ollama, proving you can take a cloud-trained model fully offline.

Ready to quiz yourself?