Fine-Tune a Local LLM with LoRA

Fine-tune a local LLM with LoRA on Apple Silicon using MLX.

Introduction

30 Second Summary

Large language models are impressive generalists, but sometimes you need a model that excels at one specific task. Fine-tuning lets you teach an existing model new tricks without training one from scratch.

In this project, you will fine-tune a local language model using LoRA to teach it how to generate structured JSON product catalog entries. The entire process runs on your Mac with no cloud GPUs required.

What You'll Build

You will fine-tune a quantized Llama 3.2 1B model using Parameter-Efficient Fine-Tuning (PEFT) with LoRA on Apple's MLX framework, training the model to output structured JSON from natural language product descriptions.

By the end of this project, you'll have:

  • A custom JSONL training dataset with instruction-response pairs that teach the model your target task.
  • A trained LoRA adapter that modifies the model's behaviour using only ~0.1% of its total parameters.
  • A fused model that permanently combines the base model with your adapter for standalone inference.
  • Secret Mission: Train a second adapter with a higher LoRA rank and compare outputs to understand the quality vs. efficiency trade-off.

Are there any prerequisites?

You need a Mac with Apple Silicon (M1, M2, M3, or M4) and at least 8GB of RAM. You should also be comfortable with basic Python and the command line.

Set Up Your Environment

Fine-tuning a language model requires a few pieces of software working together. You need Python to run the training scripts, Apple's MLX framework to handle the actual model computation on your Mac's GPU, and a Hugging Face account to download the base model.

In this step, you will install all of these tools and verify each one is working before moving on to downloading the model.

In this step, get ready to:
  • Install Python 3.10+ and create a virtual environment.
  • Install mlx-lm and verify it works on your Apple Silicon Mac.
  • Create a Hugging Face account and authenticate with the CLI.
Install Python and create a virtual environment

The mlx-lm package requires Python 3.10 or higher. Let's check whether your Mac already has a compatible version installed.

  • Open your terminal by pressing Cmd+Space and typing Terminal, then press Enter.
  • Check your Python version by running this command:
python3 --version

✔️ I see version 3.10 or higher

Your Python version is compatible. You are ready to create your project folder.

ⓧ I see a version lower than 3.10

macOS often ships with an older Python. You need to install a newer version alongside it.

  • Install Python 3.11 using Homebrew by running these commands:
brew install python@3.11
  • After installation, verify the new version is accessible by running:
python3.11 --version

Use python3.11 instead of python3 for all commands in this project if your default python3 still points to the older version.

ⓧ Command not found

Python 3 is not installed on your system. Install it using Homebrew.

  • If you do not have Homebrew installed, install it first by running:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
  • Install Python 3.11 by running:
brew install python@3.11
  • Verify the installation by running:
python3 --version

You should see a version number of 3.10 or higher.

Now create your project folder and navigate into it.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir lora-finetune
cd lora-finetune
  • Confirm the folder was created by running:
ls

You should see the lora-finetune folder listed (it will be empty for now).

A virtual environment keeps your project's packages isolated from the rest of your system. This prevents version conflicts with other Python projects on your Mac.

  • Create a virtual environment by running this command:
python3 -m venv venv
  • Activate the virtual environment by running:
source venv/bin/activate

What does activating the virtual environment do?

When activated, your terminal prompt changes to show (venv) at the beginning. This tells you that any pip install commands will place packages inside the venv folder rather than your global Python installation.

  • Verify the virtual environment is active by checking that your terminal prompt starts with (venv).

Don't see (venv) in your prompt?

Make sure you ran the source venv/bin/activate command from inside the lora-finetune folder. If you are in a different directory, cd ~/Desktop/lora-finetune first, then try the activate command again.

Help me activate my Python virtual environment.

Install mlx-lm and verify it works

mlx-lm is Apple's framework for running and fine-tuning language models natively on Apple Silicon. It handles all the GPU computation through the Metal framework built into your Mac.

  • Install mlx-lm by running this command:
pip install mlx-lm

What is mlx-lm?

mlx-lm is a Python package built on Apple's MLX framework. It provides commands for generating text, fine-tuning models with LoRA, and fusing adapters back into base models. It runs entirely on your Mac's unified memory and GPU.

This install also brings in huggingface_hub as a dependency, which provides the hf command-line tool you will use to authenticate with Hugging Face.

The installation may take a minute as it downloads MLX and its dependencies.

  • Verify that mlx-lm installed correctly by running:
python3 -c "import mlx_lm; print('mlx-lm installed successfully')"

You should see mlx-lm installed successfully printed in your terminal.

Seeing an import error?

Make sure your virtual environment is active (you should see (venv) in your prompt). If the install failed, try running pip install mlx-lm again.

If you see an error about an unsupported platform, confirm you are running on an Apple Silicon Mac (M1, M2, M3, or M4). MLX does not work on Intel Macs.

Help me fix my mlx-lm installation error.

Create a Hugging Face account and authenticate

The base model you will fine-tune is hosted on Hugging Face. You need a free account and an access token so your Mac can download it.

  • Open https://huggingface.co/join in your browser.
  • Create a free account (or sign in if you already have one).
  • Navigate to your token settings at https://huggingface.co/settings/tokens.
  • Click New token.
  • Give your token a name (for example, lora-finetune).
  • Select the Read role (you only need to download models).
  • Click Create token and copy the token value.

Why a Read token?

A Read token is sufficient because you only need to download a public model. Write tokens are for uploading models or creating repositories, which this project does not require.

  • Back in your terminal, authenticate with Hugging Face by running:
hf auth login
  • When prompted, paste the token you just copied and press Enter. Your input will not be visible while you type.
  • When asked Add token as git credential?, type n and press Enter (you will not need git credential access for this project).

You should see a message confirming your login was successful.

Getting a 'command not found' error for hf?

The hf command is installed as part of huggingface_hub, which comes with mlx-lm. Make sure your virtual environment is still active (look for (venv) in your prompt).

If you deactivated it, run source venv/bin/activate from inside your lora-finetune folder, then try hf auth login again.

Help me fix the hf command not found error.

Your environment is ready. Next up, you will download the base language model and see how it responds before any fine-tuning.

Download the Base Model and Test It

Your environment is ready with mlx-lm installed and Hugging Face authenticated. Now it's time to grab the actual model you'll be fine-tuning.

You'll download a quantized version of Meta's Llama 3.2 1B model from the MLX Community on Hugging Face. Then you'll run a test generation to see how the base model handles your target task before any training.

In this step, get ready to:
  • Download the quantized Llama 3.2 1B Instruct model.
  • Run a test generation to see how the base model responds.
  • Establish a baseline for comparison after fine-tuning.
Download the quantized base model

A quantized model compresses the original model's weights from 16-bit to 4-bit precision. This shrinks the model's memory footprint dramatically, letting it run on any Apple Silicon Mac with 8GB+ RAM.

The model you'll use is mlx-community/Llama-3.2-1B-Instruct-4bit. It's a 4-bit quantized version of Meta's Llama 3.2 with 1 billion parameters.

  • Download the model and run a quick generation test by running this command:
mlx_lm.generate --model mlx-community/Llama-3.2-1B-Instruct-4bit --prompt "hello"

What does this command do?

The mlx_lm.generate command downloads the model from Hugging Face (if not already cached) and generates text from the given prompt. The first run takes a few minutes as it downloads approximately 700MB of model files.

After the download completes, you should see a short text response printed in your terminal. The content doesn't matter. What matters is that the model loaded and generated successfully.

Download taking too long or failing?

Make sure your Hugging Face CLI is authenticated. Run huggingface-cli whoami to confirm you're logged in.

If the download stalls, check your internet connection and try again. The model files are cached locally, so a retry picks up where it left off.

Help me troubleshoot the model download

What is QLoRA?

Because you're using a 4-bit quantized model, your fine-tuning will automatically use QLoRA (quantized LoRA). This means you get the memory savings of quantization during training too. MLX detects the quantized model and applies QLoRA automatically.

Write a test script for the product catalog task

The quick CLI test proved the model loads. Now you'll write a Python script that tests the base model on the specific task you want to teach it: generating structured JSON product catalog entries.

  • Create a new file called test_base_model.py in your project folder and add the following code:
from mlx_lm import load, generate

# Load the base model (no adapter)
model, tokenizer = load("mlx-community/Llama-3.2-1B-Instruct-4bit")

# Test prompt - the task we want to fine-tune for
prompt = """<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean).<|eot_id|><|start_header_id|>user<|end_header_id|>

Wireless noise-cancelling headphones with 30-hour battery life, Bluetooth 5.3, and premium leather ear cushions. Priced at $299.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

"""

print("=" * 60)
print("BASE MODEL OUTPUT (before fine-tuning)")
print("=" * 60)
response = generate(model, tokenizer, prompt=prompt, max_tokens=200)
print(response)
print("=" * 60)

What does this code do?

load() loads the quantized model from your local cache (already downloaded in the previous substep). It returns both the model and its tokenizer.

The prompt variable uses Llama 3's chat template tokens (<|begin_of_text|>, <|start_header_id|>, etc.) to structure the conversation into system, user, and assistant roles. This is how Llama-family models expect multi-turn input.

generate() runs inference with a cap of 200 tokens. The system message asks for structured JSON. The user message gives a product description. You're testing whether the model can produce clean JSON without any fine-tuning.

Run the test and observe the baseline output
  • Save test_base_model.py.
  • Run the test script by running this command:
python test_base_model.py

The model will take a few seconds to load and then print its response between the separator lines.

What should I see?

The base model's response will likely be messy. It might produce partial JSON, add extra commentary, use inconsistent field names, or skip the JSON format entirely. This is expected.

A general-purpose 1B parameter model hasn't been specifically trained to output structured JSON product entries. It might get close because it's an instruction-tuned model, but it won't reliably produce the exact format you need. This is the gap that LoRA fine-tuning will fix.

Seeing a memory error or crash?

Make sure no other memory-intensive apps are running (close Chrome tabs, Slack, etc.). The 1B 4-bit model needs around 2-3GB of free RAM to run inference.

If you see an import error, confirm your virtual environment is activated and mlx-lm is installed by running pip show mlx-lm.

Help me debug my test script

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

from mlx_lm import load, generate

# Load the base model (no adapter)
model, tokenizer = load("mlx-community/Llama-3.2-1B-Instruct-4bit")

# Test prompt - the task we want to fine-tune for
prompt = """<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean).<|eot_id|><|start_header_id|>user<|end_header_id|>

Wireless noise-cancelling headphones with 30-hour battery life, Bluetooth 5.3, and premium leather ear cushions. Priced at $299.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

"""

print("=" * 60)
print("BASE MODEL OUTPUT (before fine-tuning)")
print("=" * 60)
response = generate(model, tokenizer, prompt=prompt, max_tokens=200)
print(response)
print("=" * 60)

You now have a clear baseline showing the model's limitations. Next up, you'll create a custom training dataset that teaches the model exactly what format you expect.

Prepare Your Training Dataset

In the previous step, you saw how the base model responds to your product catalog task. The output was likely unstructured or inconsistent.

To teach the model a new behaviour, you need to show it examples of exactly what you want. In this step, you will create a JSONL training dataset with instruction-response pairs that demonstrate the correct JSON output format.

In this step, get ready to:
  • Create a training dataset with 10 instruction-response pairs.
  • Create validation and test datasets to monitor training progress.
  • Understand the chat-format data structure that MLX expects.
Create the training dataset

MLX-LM expects training data in a specific directory structure. You need a data/ folder containing JSONL files where each line is one training example.

  • Create the data directory inside your project folder by running this command:
mkdir data
  • Confirm the directory exists by running:
ls

You should see data listed alongside test_base_model.py in the output.

  • Create a new file called train.jsonl inside the data/ directory and paste in the following content:
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Wireless noise-cancelling headphones with 30-hour battery life, Bluetooth 5.3, and premium leather ear cushions. Priced at $299."}, {"role": "assistant", "content": "{\"name\": \"Premium Wireless Noise-Cancelling Headphones\", \"category\": \"Audio\", \"price_usd\": 299, \"features\": [\"Noise cancellation\", \"30-hour battery life\", \"Bluetooth 5.3\", \"Leather ear cushions\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Ergonomic office chair with lumbar support, adjustable armrests, breathable mesh back, and 5-year warranty. Currently $449."}, {"role": "assistant", "content": "{\"name\": \"Ergonomic Mesh Office Chair\", \"category\": \"Furniture\", \"price_usd\": 449, \"features\": [\"Lumbar support\", \"Adjustable armrests\", \"Breathable mesh back\", \"5-year warranty\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Compact espresso machine with 15-bar pressure, built-in grinder, milk frother, and touchscreen controls. On sale for $599."}, {"role": "assistant", "content": "{\"name\": \"Compact Espresso Machine with Grinder\", \"category\": \"Kitchen Appliances\", \"price_usd\": 599, \"features\": [\"15-bar pressure\", \"Built-in grinder\", \"Milk frother\", \"Touchscreen controls\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Ultra-thin 14-inch laptop with M3 chip, 16GB RAM, 512GB SSD, and all-day battery. Available for $1,299."}, {"role": "assistant", "content": "{\"name\": \"Ultra-Thin 14-inch Laptop\", \"category\": \"Computers\", \"price_usd\": 1299, \"features\": [\"M3 chip\", \"16GB RAM\", \"512GB SSD\", \"All-day battery\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Waterproof fitness tracker with heart rate monitor, GPS, sleep tracking, and 7-day battery life. Costs $89. Currently out of stock."}, {"role": "assistant", "content": "{\"name\": \"Waterproof Fitness Tracker\", \"category\": \"Wearables\", \"price_usd\": 89, \"features\": [\"Heart rate monitor\", \"GPS\", \"Sleep tracking\", \"7-day battery life\"], \"in_stock\": false}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Professional stand mixer with 5-quart bowl, 10 speed settings, splash guard, and stainless steel attachments. Priced at $349."}, {"role": "assistant", "content": "{\"name\": \"Professional Stand Mixer\", \"category\": \"Kitchen Appliances\", \"price_usd\": 349, \"features\": [\"5-quart bowl\", \"10 speed settings\", \"Splash guard\", \"Stainless steel attachments\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Smart home security camera with 4K resolution, night vision, two-way audio, and cloud storage subscription included. $129."}, {"role": "assistant", "content": "{\"name\": \"Smart Home Security Camera 4K\", \"category\": \"Smart Home\", \"price_usd\": 129, \"features\": [\"4K resolution\", \"Night vision\", \"Two-way audio\", \"Cloud storage included\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Portable Bluetooth speaker with 360-degree sound, IPX7 waterproof rating, 20-hour playback, and USB-C charging. Only $79."}, {"role": "assistant", "content": "{\"name\": \"Portable 360 Bluetooth Speaker\", \"category\": \"Audio\", \"price_usd\": 79, \"features\": [\"360-degree sound\", \"IPX7 waterproof\", \"20-hour playback\", \"USB-C charging\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Mechanical gaming keyboard with RGB backlighting, hot-swappable switches, aluminum frame, and wrist rest included. $159."}, {"role": "assistant", "content": "{\"name\": \"Mechanical Gaming Keyboard RGB\", \"category\": \"Peripherals\", \"price_usd\": 159, \"features\": [\"RGB backlighting\", \"Hot-swappable switches\", \"Aluminum frame\", \"Wrist rest included\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Robot vacuum with LiDAR navigation, self-emptying base, mopping function, and app control. Retails for $699. Temporarily unavailable."}, {"role": "assistant", "content": "{\"name\": \"LiDAR Robot Vacuum with Mop\", \"category\": \"Smart Home\", \"price_usd\": 699, \"features\": [\"LiDAR navigation\", \"Self-emptying base\", \"Mopping function\", \"App control\"], \"in_stock\": false}"}]}

What does this dataset do?

Each line is a single training example in the chat format that MLX-LM expects. The file contains 10 examples covering different product categories.

The assistant content is the exact JSON output you want the model to produce. By showing the model many examples of this pattern, it learns to generate structured JSON from natural language descriptions.

  • Save data/train.jsonl.

Seeing formatting issues?

Each training example must be on a single line. If your editor wraps long lines, make sure you are not inserting actual line breaks within a JSON object.

Make sure there are no trailing empty lines at the end of the file. MLX-LM will fail to parse the dataset if it encounters blank lines.

help me fix formatting issues in my train.jsonl file for MLX-LM LoRA fine-tuning

Create the validation and test datasets

MLX-LM uses a validation set to report loss during training. This helps you monitor whether the model is learning or overfitting.

A separate test set lets you evaluate the final model on examples it never saw during training.

  • Create a new file called valid.jsonl inside the data/ directory and paste in the following content:
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Noise-isolating earbuds with active noise cancellation, transparency mode, wireless charging case, and 6-hour battery per charge. $199."}, {"role": "assistant", "content": "{\"name\": \"Noise-Isolating ANC Earbuds\", \"category\": \"Audio\", \"price_usd\": 199, \"features\": [\"Active noise cancellation\", \"Transparency mode\", \"Wireless charging case\", \"6-hour battery\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Adjustable standing desk with electric motor, memory presets, cable management tray, and bamboo top. $549."}, {"role": "assistant", "content": "{\"name\": \"Electric Adjustable Standing Desk\", \"category\": \"Furniture\", \"price_usd\": 549, \"features\": [\"Electric motor\", \"Memory presets\", \"Cable management tray\", \"Bamboo top\"], \"in_stock\": true}"}]}

Why only 2 validation examples?

Validation data is only used to check how well the model generalises. You do not need many examples for this. Two is enough to get a meaningful validation loss signal during training.

  • Save data/valid.jsonl.
  • Create a new file called test.jsonl inside the data/ directory and paste in the following content:
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "4K webcam with auto-framing, built-in ring light, noise-cancelling microphone, and privacy shutter. Priced at $179."}, {"role": "assistant", "content": "{\"name\": \"4K Auto-Framing Webcam\", \"category\": \"Peripherals\", \"price_usd\": 179, \"features\": [\"Auto-framing\", \"Built-in ring light\", \"Noise-cancelling microphone\", \"Privacy shutter\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Smart thermostat with learning algorithm, remote control via app, energy usage reports, and geofencing. $249. Back-ordered."}, {"role": "assistant", "content": "{\"name\": \"Smart Learning Thermostat\", \"category\": \"Smart Home\", \"price_usd\": 249, \"features\": [\"Learning algorithm\", \"Remote app control\", \"Energy usage reports\", \"Geofencing\"], \"in_stock\": false}"}]}
  • Save data/test.jsonl.
  • Confirm all three files exist by running:
ls data/

You should see test.jsonl, train.jsonl, and valid.jsonl listed in the output.

Missing files in the data directory?

Make sure you created each file inside the data/ folder, not in your project root. The file names must be exactly train.jsonl, valid.jsonl, and test.jsonl.

help me check why my JSONL data files are not showing up in the data directory

Understand the chat-format data structure

Each line in your JSONL files follows a specific structure that MLX-LM recognises as the chat format. The model's tokenizer uses this structure to understand where instructions begin and where the expected response starts.

What is the chat format?

Every training example is a JSON object with a messages array containing three roles:

  • system tells the model what persona to adopt and what output format to use.
  • user provides the input the model will receive at inference time (the product description).
  • assistant contains the ideal response the model should learn to produce (the structured JSON output).

When MLX-LM loads this data, it automatically applies the model's chat template to convert each example into the token format the model expects. Your data stays readable while the training pipeline handles the technical formatting.

Why include out-of-stock examples?

Notice that some training examples have "in_stock": false in the assistant response. Including both in-stock and out-of-stock products teaches the model to correctly interpret availability language like "currently out of stock" or "temporarily unavailable" and map it to the boolean field.

✔️ Awesome, I've got everything!

Great. Double check that you have saved all three files in the data/ directory.

ⓧ I'd like to double check the full code

Here is the complete content for each file. Compare yours against these:

{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Wireless noise-cancelling headphones with 30-hour battery life, Bluetooth 5.3, and premium leather ear cushions. Priced at $299."}, {"role": "assistant", "content": "{\"name\": \"Premium Wireless Noise-Cancelling Headphones\", \"category\": \"Audio\", \"price_usd\": 299, \"features\": [\"Noise cancellation\", \"30-hour battery life\", \"Bluetooth 5.3\", \"Leather ear cushions\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Ergonomic office chair with lumbar support, adjustable armrests, breathable mesh back, and 5-year warranty. Currently $449."}, {"role": "assistant", "content": "{\"name\": \"Ergonomic Mesh Office Chair\", \"category\": \"Furniture\", \"price_usd\": 449, \"features\": [\"Lumbar support\", \"Adjustable armrests\", \"Breathable mesh back\", \"5-year warranty\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Compact espresso machine with 15-bar pressure, built-in grinder, milk frother, and touchscreen controls. On sale for $599."}, {"role": "assistant", "content": "{\"name\": \"Compact Espresso Machine with Grinder\", \"category\": \"Kitchen Appliances\", \"price_usd\": 599, \"features\": [\"15-bar pressure\", \"Built-in grinder\", \"Milk frother\", \"Touchscreen controls\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Ultra-thin 14-inch laptop with M3 chip, 16GB RAM, 512GB SSD, and all-day battery. Available for $1,299."}, {"role": "assistant", "content": "{\"name\": \"Ultra-Thin 14-inch Laptop\", \"category\": \"Computers\", \"price_usd\": 1299, \"features\": [\"M3 chip\", \"16GB RAM\", \"512GB SSD\", \"All-day battery\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Waterproof fitness tracker with heart rate monitor, GPS, sleep tracking, and 7-day battery life. Costs $89. Currently out of stock."}, {"role": "assistant", "content": "{\"name\": \"Waterproof Fitness Tracker\", \"category\": \"Wearables\", \"price_usd\": 89, \"features\": [\"Heart rate monitor\", \"GPS\", \"Sleep tracking\", \"7-day battery life\"], \"in_stock\": false}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Professional stand mixer with 5-quart bowl, 10 speed settings, splash guard, and stainless steel attachments. Priced at $349."}, {"role": "assistant", "content": "{\"name\": \"Professional Stand Mixer\", \"category\": \"Kitchen Appliances\", \"price_usd\": 349, \"features\": [\"5-quart bowl\", \"10 speed settings\", \"Splash guard\", \"Stainless steel attachments\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Smart home security camera with 4K resolution, night vision, two-way audio, and cloud storage subscription included. $129."}, {"role": "assistant", "content": "{\"name\": \"Smart Home Security Camera 4K\", \"category\": \"Smart Home\", \"price_usd\": 129, \"features\": [\"4K resolution\", \"Night vision\", \"Two-way audio\", \"Cloud storage included\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Portable Bluetooth speaker with 360-degree sound, IPX7 waterproof rating, 20-hour playback, and USB-C charging. Only $79."}, {"role": "assistant", "content": "{\"name\": \"Portable 360 Bluetooth Speaker\", \"category\": \"Audio\", \"price_usd\": 79, \"features\": [\"360-degree sound\", \"IPX7 waterproof\", \"20-hour playback\", \"USB-C charging\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Mechanical gaming keyboard with RGB backlighting, hot-swappable switches, aluminum frame, and wrist rest included. $159."}, {"role": "assistant", "content": "{\"name\": \"Mechanical Gaming Keyboard RGB\", \"category\": \"Peripherals\", \"price_usd\": 159, \"features\": [\"RGB backlighting\", \"Hot-swappable switches\", \"Aluminum frame\", \"Wrist rest included\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Robot vacuum with LiDAR navigation, self-emptying base, mopping function, and app control. Retails for $699. Temporarily unavailable."}, {"role": "assistant", "content": "{\"name\": \"LiDAR Robot Vacuum with Mop\", \"category\": \"Smart Home\", \"price_usd\": 699, \"features\": [\"LiDAR navigation\", \"Self-emptying base\", \"Mopping function\", \"App control\"], \"in_stock\": false}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Noise-isolating earbuds with active noise cancellation, transparency mode, wireless charging case, and 6-hour battery per charge. $199."}, {"role": "assistant", "content": "{\"name\": \"Noise-Isolating ANC Earbuds\", \"category\": \"Audio\", \"price_usd\": 199, \"features\": [\"Active noise cancellation\", \"Transparency mode\", \"Wireless charging case\", \"6-hour battery\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Adjustable standing desk with electric motor, memory presets, cable management tray, and bamboo top. $549."}, {"role": "assistant", "content": "{\"name\": \"Electric Adjustable Standing Desk\", \"category\": \"Furniture\", \"price_usd\": 549, \"features\": [\"Electric motor\", \"Memory presets\", \"Cable management tray\", \"Bamboo top\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "4K webcam with auto-framing, built-in ring light, noise-cancelling microphone, and privacy shutter. Priced at $179."}, {"role": "assistant", "content": "{\"name\": \"4K Auto-Framing Webcam\", \"category\": \"Peripherals\", \"price_usd\": 179, \"features\": [\"Auto-framing\", \"Built-in ring light\", \"Noise-cancelling microphone\", \"Privacy shutter\"], \"in_stock\": true}"}]}
{"messages": [{"role": "system", "content": "You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean)."}, {"role": "user", "content": "Smart thermostat with learning algorithm, remote control via app, energy usage reports, and geofencing. $249. Back-ordered."}, {"role": "assistant", "content": "{\"name\": \"Smart Learning Thermostat\", \"category\": \"Smart Home\", \"price_usd\": 249, \"features\": [\"Learning algorithm\", \"Remote app control\", \"Energy usage reports\", \"Geofencing\"], \"in_stock\": false}"}]}

Your training data is ready. Next up, you will configure the LoRA hyperparameters and kick off the fine-tuning run.

Configure and Run LoRA Fine-Tuning

You have a dataset ready. Now you need to tell the training framework exactly how to fine-tune the model.

Instead of retraining all 1 billion parameters (which would require massive compute), you will configure LoRA to train tiny adapter matrices on just two attention layers. This keeps training fast and memory-efficient while still teaching the model your target task.

In this step, get ready to:
  • Create a YAML configuration file specifying LoRA parameters.
  • Run the fine-tuning using the mlx_lm.lora command.
  • Monitor training loss and validation loss as the model learns.
Create the LoRA configuration file

The MLX framework uses a YAML config file to specify everything about your fine-tuning job: which model to train, which layers to target, how many iterations to run, and what LoRA rank to use.

  • Create a new file called lora_config.yaml in your project folder (the same folder that contains your data/ directory).
  • Paste the following configuration into it:
# LoRA Fine-Tuning Configuration for MLX-LM
model: "mlx-community/Llama-3.2-1B-Instruct-4bit"
train: true
fine_tune_type: lora
data: "./data"
seed: 42
num_layers: 16
batch_size: 4
iters: 200
val_batches: -1
learning_rate: 1e-5
steps_per_report: 10
steps_per_eval: 50
grad_accumulation_steps: 1
adapter_path: "adapters"
save_every: 50
test: false
max_seq_length: 2048
grad_checkpoint: false

lora_parameters:
  keys: ["self_attn.q_proj", "self_attn.v_proj"]
  rank: 8
  scale: 20.0
  dropout: 0.0

What does this configuration do?

This file controls every aspect of your fine-tuning job:

  • model points to the quantized Llama model you downloaded. Because it is 4-bit quantized, MLX automatically uses QLoRA (quantized LoRA) for training.
  • iters: 200 runs 200 training iterations. This is enough for our small dataset to show learning.
  • num_layers: 16 fine-tunes the last 16 layers of the model with LoRA adapters.
  • lora_parameters is where the magic happens. rank: 8 means each adapter matrix has rank 8, keeping trainable parameters tiny. keys targets the query and value projection layers in each attention head.
  • steps_per_report: 10 prints training loss every 10 steps. steps_per_eval: 50 runs validation every 50 steps so you can watch the model improve.

What is LoRA rank?

LoRA works by inserting two small matrices (A and B) into each targeted layer. The rank controls the size of these matrices. A rank of 8 means each adapter pair has an inner dimension of 8.

Lower rank means fewer trainable parameters and faster training. Higher rank means more capacity to learn complex patterns. Rank 8 is a strong starting point for format-adaptation tasks like ours.

  • Save lora_config.yaml.

File not saving in the right place?

Make sure lora_config.yaml is saved at the same level as your data/ folder, not inside it. The data: "./data" path in the config is relative to where you run the command from.

help me check my project folder structure is correct

Run the fine-tuning

With your config file and dataset in place, you can kick off training with a single command. This tells MLX to load the model, apply LoRA adapters, and train on your JSONL data.

  • Start fine-tuning by running this command in your terminal:
python -m mlx_lm.lora --config lora_config.yaml

What does this command do?

python -m mlx_lm.lora runs the LoRA fine-tuning module from the mlx-lm package. The --config flag tells it to read all training parameters from your YAML file instead of passing them as command-line arguments.

Training should take approximately 5-10 minutes on an M1 chip and faster on M2/M3/M4. You will see output scrolling in your terminal as training progresses.

Monitor training and validation loss

As training runs, MLX prints progress updates to your terminal. These numbers tell you whether the model is actually learning from your data.

  • Watch your terminal output. Every 10 iterations, you will see a training loss value printed.
  • Every 50 iterations, a validation loss appears. This measures how well the model generalises to data it has not trained on.
  • Wait for training to complete all 200 iterations.

What should I see?

You will see lines reporting iteration number, training loss, and tokens per second. The training loss should generally decrease over the 200 iterations, showing the model is learning your JSON format.

At iterations 50, 100, 150, and 200, you will also see validation loss. A decreasing validation loss confirms the model is generalising, not just memorising.

  • Once training finishes, confirm the adapter was saved by running:
ls adapters/

You should see files including adapter configuration and weight files in the adapters/ directory.

Training seems stuck or erroring out?

  • If you see a memory error, close other applications to free up RAM. The 1B 4-bit model needs approximately 6-8GB during training.
  • If you see a file-not-found error for the data, make sure your terminal is in the same directory as lora_config.yaml and the data/ folder.
  • If the model fails to load, confirm you successfully downloaded it in the previous step by checking your Hugging Face cache.

help me debug my mlx-lm LoRA training error

✔️ Awesome, I've got everything!

Great. Double check you have saved lora_config.yaml and that the adapters/ directory exists with trained weights inside.

ⓧ I'd like to double check the full code

Here is the complete lora_config.yaml file:

# LoRA Fine-Tuning Configuration for MLX-LM
model: "mlx-community/Llama-3.2-1B-Instruct-4bit"
train: true
fine_tune_type: lora
data: "./data"
seed: 42
num_layers: 16
batch_size: 4
iters: 200
val_batches: -1
learning_rate: 1e-5
steps_per_report: 10
steps_per_eval: 50
grad_accumulation_steps: 1
adapter_path: "adapters"
save_every: 50
test: false
max_seq_length: 2048
grad_checkpoint: false

lora_parameters:
  keys: ["self_attn.q_proj", "self_attn.v_proj"]
  rank: 8
  scale: 20.0
  dropout: 0.0

Your LoRA adapter is trained and saved. Next up, you will load it alongside the base model and see how your fine-tuned model performs on the same prompts from before.

Test Your Fine-Tuned Model

Your LoRA adapter has finished training. The adapters/ directory now contains the learned weights that modify the base model's behaviour.

The real question is whether those tiny adapter matrices actually changed how the model responds. In this step, you will load the adapter on top of the base model, run the same prompt you tested before fine-tuning, and see the difference side by side.

In this step, get ready to:
  • Load the fine-tuned model with your trained LoRA adapter.
  • Compare model outputs before and after fine-tuning.
  • Evaluate test set perplexity to measure improvement.
Create the fine-tuned model test script

To compare before and after, you need a script that loads the base model with the LoRA adapter applied. The adapter_path parameter tells mlx-lm to layer your trained adapter on top of the base weights at load time.

  • Create a new file called test_finetuned_model.py in your project folder and paste the following code:
from mlx_lm import load, generate

# Load the base model with the LoRA adapter
model, tokenizer = load(
    "mlx-community/Llama-3.2-1B-Instruct-4bit",
    adapter_path="adapters"
)

# Same test prompt as before
prompt = """<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean).<|eot_id|><|start_header_id|>user<|end_header_id|>

Wireless noise-cancelling headphones with 30-hour battery life, Bluetooth 5.3, and premium leather ear cushions. Priced at $299.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

"""

print("=" * 60)
print("FINE-TUNED MODEL OUTPUT (with LoRA adapter)")
print("=" * 60)
response = generate(model, tokenizer, prompt=prompt, max_tokens=200)
print(response)
print("=" * 60)

# Test with a completely new product the model hasn't seen
new_prompt = """<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean).<|eot_id|><|start_header_id|>user<|end_header_id|>

Foldable electric scooter with 25-mile range, top speed 20mph, regenerative braking, and built-in LED headlight. $499.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

"""

print("\n" + "=" * 60)
print("NEW PRODUCT TEST (unseen during training)")
print("=" * 60)
response = generate(model, tokenizer, prompt=new_prompt, max_tokens=200)
print(response)
print("=" * 60)

What does this code do?

This script runs two tests on your fine-tuned model:

  • load("mlx-community/Llama-3.2-1B-Instruct-4bit", adapter_path="adapters") loads the base model and layers your trained LoRA adapter on top. The base weights stay frozen. Only the adapter matrices modify the output.
  • The first prompt is identical to what you ran in test_base_model.py. This gives you a direct before-and-after comparison.
  • The second prompt uses a completely new product (an electric scooter) that never appeared in training data. This tests whether the model learned the general pattern, not just memorized examples.
  • Save test_finetuned_model.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved the file before running it.

ⓧ I'd like to double check the full code

from mlx_lm import load, generate

# Load the base model with the LoRA adapter
model, tokenizer = load(
    "mlx-community/Llama-3.2-1B-Instruct-4bit",
    adapter_path="adapters"
)

# Same test prompt as before
prompt = """<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean).<|eot_id|><|start_header_id|>user<|end_header_id|>

Wireless noise-cancelling headphones with 30-hour battery life, Bluetooth 5.3, and premium leather ear cushions. Priced at $299.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

"""

print("=" * 60)
print("FINE-TUNED MODEL OUTPUT (with LoRA adapter)")
print("=" * 60)
response = generate(model, tokenizer, prompt=prompt, max_tokens=200)
print(response)
print("=" * 60)

# Test with a completely new product the model hasn't seen
new_prompt = """<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean).<|eot_id|><|start_header_id|>user<|end_header_id|>

Foldable electric scooter with 25-mile range, top speed 20mph, regenerative braking, and built-in LED headlight. $499.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

"""

print("\n" + "=" * 60)
print("NEW PRODUCT TEST (unseen during training)")
print("=" * 60)
response = generate(model, tokenizer, prompt=new_prompt, max_tokens=200)
print(response)
print("=" * 60)
Run the test and compare results

Now run your test script to see how fine-tuning changed the model's behaviour.

  • Run the fine-tuned model test by executing this command:
python test_finetuned_model.py

The script produces two outputs. Compare the first output to what the base model generated when you ran test_base_model.py earlier.

What should I see?

The fine-tuned model should now output a well-structured JSON object with the correct fields: name, category, price_usd, features, and in_stock. Before fine-tuning, the base model likely produced unstructured text, partial JSON, or extra commentary.

The second output (the unseen electric scooter) tests generalization. If the model learned the pattern rather than memorizing training examples, it should produce a valid JSON object for this new product too.

Output still looks like the base model?

Make sure the adapters/ directory exists in the same folder where you are running the script. The adapter_path="adapters" parameter is a relative path.

Check that your training completed successfully in the previous step. If training was interrupted early, the adapter weights may be incomplete.

Why is my fine-tuned model still generating unstructured text?

Evaluate test set perplexity

Comparing outputs visually is useful, but you also want a quantitative measurement. Perplexity measures how surprised the model is by the correct answers in your test set. Lower perplexity means the model's predictions are closer to the expected outputs.

  • Evaluate the trained adapter against your test set by running this command:
python -m mlx_lm.lora --model mlx-community/Llama-3.2-1B-Instruct-4bit --adapter-path adapters --data ./data --test

What does this command do?

  • --model specifies the base model to load.
  • --adapter-path adapters tells mlx_lm.lora to apply your trained adapter weights before evaluating.
  • --data ./data points to the directory containing test.jsonl.
  • --test triggers evaluation mode instead of training. The command computes loss and perplexity against the test set examples.

You should see output reporting a test loss and test perplexity (ppl) value. A lower perplexity indicates the model predicts the expected JSON outputs more confidently.

What is perplexity?

Perplexity is the exponential of the average loss. A perplexity of 1.0 would mean the model perfectly predicts every token. In practice, values under 5.0 for a task-specific test set indicate strong adaptation.

Your test set contains two product descriptions the model never saw during training. Low perplexity on these unseen examples confirms the model generalized the JSON format rather than memorizing the training data.

Seeing a file not found error?

Make sure you are running the command from your project directory (the folder containing both the adapters/ directory and the data/ directory).

Confirm that data/test.jsonl exists. The --test flag requires a test.jsonl file in the data directory.

Help me fix a file not found error when running mlx_lm.lora --test

You now have both qualitative evidence (visual comparison of outputs) and quantitative evidence (perplexity score) that your LoRA fine-tuning worked. Next up, you will fuse the adapter permanently into the base model so it runs as a standalone model.

Fuse the Adapter and Run Standalone Inference

In the previous step, you loaded the base model and applied the LoRA adapter at runtime to generate structured JSON outputs.

That works, but it means you always need both the base model and the adapter file present. In this step, you will permanently merge (fuse) the adapter weights into the base model to create a single standalone model that carries the learned behaviour built-in.

In this step, get ready to:
  • Fuse the LoRA adapter into the base model with a single command.
  • Run inference on the fused model without any adapter file.
  • Confirm the fused model output matches the adapter-loaded output.
Fuse the LoRA adapter into the base model

Right now your trained adapter lives as a separate set of files in the adapters/ directory. Fusing bakes those weights directly into the base model so you get a single self-contained model you can load and share without any extra arguments.

  • Fuse the adapter into the base model by running this command:
python -m mlx_lm.fuse --model mlx-community/Llama-3.2-1B-Instruct-4bit

What does this command do?

The mlx_lm.fuse command takes your base model and your trained LoRA adapter, then mathematically merges the low-rank weight updates into the original weight matrices.

By default it looks for adapter weights in adapters/ and saves the combined model to fused_model/. The result is a full model directory you can load and use like any other model.

  • Confirm the fused model directory was created by running:
ls fused_model

You should see model weight files, a tokenizer, and configuration files listed inside fused_model/.

Don't see fused_model/?

Make sure you ran the command from your project's root directory (the same folder that contains adapters/).

If you see an error about missing adapter files, check that the adapters/ directory exists and contains weight files from your training run in Step 4.

Help me debug my mlx_lm.fuse command

Run inference on the fused model

The fused model is now fully self-contained. You can load it by pointing directly at the fused_model/ directory with no adapter path needed.

  • Generate a response from the fused model by running this command:
python -m mlx_lm.generate --model ./fused_model --prompt "<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a product catalog assistant. When given a product description, output a structured JSON object with fields: name, category, price_usd, features (array), and in_stock (boolean).<|eot_id|><|start_header_id|>user<|end_header_id|>

Wireless noise-cancelling headphones with 30-hour battery life, Bluetooth 5.3, and premium leather ear cushions. Priced at $299.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

"

What changed here?

Notice that --model now points to ./fused_model instead of the Hugging Face model ID. There is no --adapter-path argument at all.

The fused model already contains your LoRA training baked in, so it behaves exactly like the adapter-loaded version but as a single unit.

You should see a structured JSON response with name, category, price_usd, features, and in_stock fields, matching the format your training data taught the model.

Getting unexpected or garbled output?

If the fused model produces random text instead of JSON, the fuse may have failed silently. Delete the fused_model/ directory and run the fuse command again.

If you see a file-not-found error, make sure you include the ./ prefix to indicate a local path rather than a Hugging Face repo name.

Help me fix my fused model output

Compare the fused model output to the adapter-loaded output

To confirm the fuse worked correctly, the fused model output should closely match what you saw in the previous step when loading the adapter at runtime. Both approaches apply the same learned weights to the same base model.

  • Run your adapter-loaded test script again to compare side by side:
python test_finetuned_model.py

Will the outputs be identical?

The outputs from the fused model and the adapter-loaded model should be very similar or identical. Both produce the same mathematical result because fusing simply pre-computes the addition of the low-rank matrices into the base weights.

Minor floating-point differences are possible but the structured JSON format and field values should match.

  • Compare the JSON output from both runs. The field values and structure should match.

When to fuse vs. keep adapters separate

Fusing is ideal for deployment. You get a single model directory to distribute, and inference is slightly faster because there is no adapter-loading step.

Keeping adapters separate is better during experimentation. You can swap different adapters on the same base model without re-downloading it, and the adapter files are tiny (under 50MB) compared to the full model.

Your fine-tuned model is now a fully standalone package. You trained a LoRA adapter, tested it, and permanently merged it into the base model. Next up, take on the Secret Mission to explore how different LoRA ranks affect training quality.

Secret mission

Compare LoRA Ranks

You've trained a rank-8 LoRA adapter. But how do you know rank 8 is the right choice? In this secret mission, you'll train a second adapter with rank 16 and compare outputs side-by-side. Higher rank means more trainable parameters and potentially richer adaptation, but also longer training and more memory. Run both and see the trade-off firsthand.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally on your Mac, so there are no ongoing costs. Decide whether to keep your files for future experiments or clean them up to free disk space.

Resources you used:

  • Project folder on your Desktop (training data, Python scripts, config files, adapters, and fused model).
  • Python virtual environment (inside your project folder).
  • Hugging Face model cache (the downloaded Llama 3.2 1B model, stored in ~/.cache/huggingface/hub/).
  • Hugging Face CLI authentication token (stored locally on your machine).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with LoRA fine-tuning, try different datasets, or test new hyperparameters.

  • Your project folder stays on your Desktop with all adapters, data, and the fused model intact.
  • The Hugging Face model cache keeps the base model downloaded for quick reuse.
  • Your Hugging Face CLI token remains active for future model downloads.

Pause - I'll come back to this later

Deactivate your virtual environment to free up terminal resources. Your files stay on disk for when you return.

  • Deactivate your virtual environment by running:
  • Close any terminal windows running the project.
deactivate

Everything else stays in place. When you come back, navigate to your project folder and reactivate the virtual environment.

Delete - I don't want to use this again

Remove all project files, the downloaded model cache, and your stored authentication token. This frees up several gigabytes of disk space.

  • Deactivate your virtual environment first (if still active):
deactivate
  • Navigate to your Desktop and delete the project folder by running:
cd ~/Desktop
rm -rf lora-finetune

Folder name may differ

Replace lora-finetune with whatever you named your project folder in Step 1.

  • Delete the Hugging Face model cache to reclaim disk space:
huggingface-cli delete-cache

What does this command do?

This opens an interactive prompt where you can select which cached models to remove. Select the Llama 3.2 1B model and confirm the deletion.

  • Remove your stored Hugging Face authentication token:
huggingface-cli logout

Nice Work!

Nice Work!

You just fine-tuned your own local AI model using PEFT LoRA on Apple Silicon. A general-purpose language model now generates structured JSON product catalog entries on command, trained entirely on your Mac with no cloud GPUs required.

You've learned how to:

  • Built a custom JSONL training dataset and ran LoRA fine-tuning locally on Apple Silicon using MLX.
  • Trained only ~0.1% of the model's parameters using low-rank adapter matrices while meaningfully changing the model's behaviour for a specific task.
  • Fused the trained LoRA adapter into the base model for standalone deployment, producing a self-contained fine-tuned model.
  • Compared different LoRA ranks (rank 8 vs rank 16) to understand the trade-off between adapter capacity, training time, and output quality.

Ready to quiz yourself?