Fine-Tune an Amharic-English Translator

Fine-tune a MarianMT model to translate English to Amharic on your local GPU.

Introduction

30 Second Summary

Translation between languages is something billions of people rely on every day. Behind apps like Google Translate are neural networks trained on millions of paired sentences to understand how one language maps to another.

In this project, you will fine-tune a pre-trained machine translation model to translate English into Amharic, Ethiopia's official language. You'll work with a dataset of 669,000 parallel sentence pairs, train the model on your local GPU, and evaluate translation quality using industry-standard metrics.

What You'll Build

A fine-tuned MarianMT model that translates English text into Amharic using Hugging Face Transformers and a Jupyter Notebook pipeline.

By the end of this project, you'll have:

  • A complete fine-tuning pipeline in a Jupyter Notebook that loads data, tokenizes text, trains the model, and evaluates results.
  • A working English-to-Amharic translator fine-tuned on 50,000 sentence pairs from a 669K parallel corpus.
  • A BLEU score evaluation comparing your model's translation quality before and after fine-tuning.
  • Secret Mission: Train the reverse direction (Amharic to English) and build a bidirectional translator.

Are there any prerequisites?

You need basic Python syntax knowledge (variables, functions, loops). No prior machine learning experience is required.

You also need a local NVIDIA GPU and Anaconda installed with JupyterLab. The project handles all other library installations in the first step.

Set Up Your Environment

Before you can fine-tune a translation model, you need the right libraries installed and a confirmed connection to your GPU.

In this step, you will install the Hugging Face Transformers ecosystem, verify that PyTorch can see your GPU, and load the pre-trained MarianMT model that you will fine-tune later.

In this step, get ready to:
  • Install the machine learning packages your project needs.
  • Confirm your GPU is available and import all libraries.
  • Load the pre-trained English-to-Amharic model and tokenizer.
Install the required packages

Your notebook needs several packages that do not come pre-installed with Anaconda. These include the Transformers library for model loading and training, the Datasets library for accessing parallel corpora, and evaluation tools for measuring translation quality.

  • In your amharic_translation.ipynb notebook, click into the first empty cell.
  • Install all required packages by running this command:
# Cell 1: Install packages
!pip install transformers==5.13.0 datasets==5.0.0 evaluate==0.4.6 sacrebleu sentencepiece

What does this command do?

The ! prefix runs a shell command directly from inside your notebook cell.

  • transformers==5.13.0 provides the model classes, tokenizers, and training utilities from Hugging Face.
  • datasets==5.0.0 lets you load datasets directly from the Hugging Face Hub.
  • evaluate==0.4.6 provides standardized evaluation metrics like BLEU.
  • sacrebleu is the backend that computes BLEU scores for translation evaluation.
  • sentencepiece handles subword tokenization for multilingual text like Amharic.
  • Run the cell by pressing Shift+Enter.
  • Wait for the install to complete. You should see output ending with a "Successfully installed" message listing the packages and their versions.

Seeing a connection error or timeout?

  • Check that your machine has an active internet connection. The packages are downloaded from PyPI.
  • If you see a message about needing to restart the kernel, click Kernel in the top menu, then Restart Kernel. Re-run the install cell afterwards.
  • help me troubleshoot my pip install in Jupyter
Import libraries and check your GPU

With the packages installed, you can now import them into your notebook session and verify that PyTorch detects your NVIDIA GPU.

  • Click into the next empty cell in your notebook.
  • Import all required libraries and run the GPU check by pasting this code:
# Cell 2: Imports and GPU check
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from transformers import Seq2SeqTrainingArguments, Seq2SeqTrainer
from transformers import DataCollatorForSeq2Seq
from datasets import load_dataset
import evaluate
import numpy as np

print(f"GPU available: {torch.cuda.is_available()}")
if torch.cuda.is_available():
    print(f"GPU name: {torch.cuda.get_device_name(0)}")

What does this code do?

  • torch is the deep learning framework that handles GPU computation during training.
  • AutoTokenizer and AutoModelForSeq2SeqLM automatically load the correct tokenizer and model architecture for MarianMT.
  • Seq2SeqTrainingArguments and Seq2SeqTrainer manage the training loop for sequence-to-sequence models.
  • DataCollatorForSeq2Seq handles dynamic padding of batches during training.
  • torch.cuda.is_available() checks whether PyTorch can access your NVIDIA GPU for accelerated training.
  • Run the cell with Shift+Enter.
  • You should see GPU available: True followed by your GPU's name (for example, NVIDIA GeForce RTX 4060).

GPU available shows False?

  • Make sure you have the NVIDIA drivers installed and that your PyTorch installation includes CUDA support. Anaconda's default PyTorch may be CPU-only.
  • Try reinstalling PyTorch with CUDA support from the official PyTorch website's install selector.
  • help me get PyTorch to detect my GPU
Load the pre-trained model and tokenizer

Now you will download the pre-trained MarianMT model for English-to-Amharic translation from Hugging Face. This model already knows basic translation patterns, and your fine-tuning will specialize it on the large Amharic dataset.

  • Click into the next empty cell.
  • Load the model checkpoint, tokenizer, and model by pasting this code:
# Cell 3: Load model and tokenizer
model_checkpoint = "Helsinki-NLP/opus-mt-en-am"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(model_checkpoint)
print(f"Model loaded: {model_checkpoint}")

What does this code do?

model_checkpoint stores the Hugging Face model identifier. The naming format Helsinki-NLP/opus-mt-{src}-{tgt} tells you this is an OPUS-MT model translating from English (en) to Amharic (am).

AutoTokenizer.from_pretrained() downloads the SentencePiece tokenizer that knows how to split both English and Amharic text into subword tokens. AutoModelForSeq2SeqLM.from_pretrained() downloads the full model weights (approximately 298 MB).

  • Run the cell with Shift+Enter.
  • You should see Model loaded: Helsinki-NLP/opus-mt-en-am printed after the download completes. The first run downloads from the internet, so it may take a minute depending on your connection speed.

Model download failing or timing out?

  • Check your internet connection. The model is approximately 298 MB and needs to download from Hugging Face's servers.
  • If you see an OSError about the model not being found, double-check that the checkpoint string is exactly Helsinki-NLP/opus-mt-en-am with no typos.
  • help me download a Hugging Face model in Jupyter

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook file before moving on.

ⓧ I'd like to double check the full code

Here is the complete code for all three cells so far:

# Cell 1: Install packages
!pip install transformers==5.13.0 datasets==5.0.0 evaluate==0.4.6 sacrebleu sentencepiece

# Cell 2: Imports and GPU check
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from transformers import Seq2SeqTrainingArguments, Seq2SeqTrainer
from transformers import DataCollatorForSeq2Seq
from datasets import load_dataset
import evaluate
import numpy as np

print(f"GPU available: {torch.cuda.is_available()}")
if torch.cuda.is_available():
    print(f"GPU name: {torch.cuda.get_device_name(0)}")

# Cell 3: Load model and tokenizer
model_checkpoint = "Helsinki-NLP/opus-mt-en-am"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(model_checkpoint)
print(f"Model loaded: {model_checkpoint}")

Your environment is ready with all packages installed, your GPU confirmed, and the pre-trained MarianMT model loaded. Next up, you will load the English-Amharic parallel dataset and explore how 669,000 sentence pairs are structured.

Load and Explore the Parallel Dataset

Your model is loaded and ready to learn. But a translation model is useless without data to learn from.

In this step, you will load a large collection of English-Amharic sentence pairs and prepare a training subset. This dataset is what teaches the model how English phrases map to their Amharic equivalents.

In this step, get ready to:
  • Load a 669K sentence parallel corpus from Hugging Face.
  • Explore aligned English-Amharic sentence pairs.
  • Create a 50K training subset and a 5K evaluation subset.
Load the parallel dataset from Hugging Face

Translation models learn by studying thousands of paired sentences. Each pair shows the same meaning expressed in two languages. This collection of aligned pairs is called a parallel corpus.

  • In a new cell in your notebook, load the English-Amharic dataset and inspect its size by running this code:
# Cell 4: Load the parallel dataset
dataset = load_dataset("michsethowusu/english-amharic_sentence-pairs_mt560")
print(f"Dataset size: {len(dataset['train'])} sentence pairs")
print(dataset["train"][0])

What does this code do?

The load_dataset() function downloads the dataset directly from Hugging Face's Datasets Hub. This particular dataset contains 669,145 English-Amharic sentence pairs sourced from the OPUS MT560 collection.

The second line prints the total number of pairs, and the third line shows one example so you can see the data structure.

  • Run the cell by pressing Shift+Enter.
  • You should see output showing 669145 sentence pairs and a dictionary with English and Amharic keys.

Dataset download taking too long?

The dataset is approximately 109 MB. If your internet connection is slow, the download may take a few minutes. Make sure you have a stable connection and let it finish.

If you see a connection error, check that you can reach huggingface.co in your browser.

Help me debug a dataset loading error.

What is a parallel corpus?

A parallel corpus is a collection of texts where each item exists in two (or more) languages, aligned sentence by sentence. The alignment is critical. The model learns that sentence #42 in English means the same thing as sentence #42 in Amharic.

Without proper alignment, the model would be learning random associations instead of accurate translations.

Explore sample sentence pairs

Before training, it helps to see what the data actually looks like. Amharic uses the Ge'ez script (also called fidel), which is visually distinct from Latin characters.

  • In a new cell, print five sample pairs to see English and Amharic side by side:
# Cell 5: Explore sample pairs
for i in range(5):
    sample = dataset["train"][i]
    print(f"\nPair {i+1}:")
    print(f"  English: {sample['English']}")
    print(f"  Amharic: {sample['Amharic']}")

What does this code do?

This loops through the first five entries in the dataset and prints each English sentence alongside its Amharic translation. Each pair is one training example the model will learn from.

Notice that the Amharic text uses a completely different writing system. The SentencePiece tokenizer you loaded earlier handles this by breaking text into subword units regardless of the script.

  • Run the cell.
  • You should see five numbered pairs, each with an English sentence and its corresponding Amharic translation in Ge'ez script.

Seeing KeyError for column names?

If you see a KeyError, make sure you are using sample['English'] and sample['Amharic'] with capital letters. Column names are case-sensitive.

Help me fix a KeyError when accessing dataset columns.

Create a training and test subset

Training on all 669K pairs would take several hours. A subset of 50,000 pairs is enough to see real improvement while keeping fine-tuning under 30 minutes on a single GPU.

  • In a new cell, split the dataset 90/10 and select your training and evaluation subsets:
# Cell 6: Create train/test split and subset
dataset_split = dataset["train"].train_test_split(test_size=0.1, seed=42)
train_dataset = dataset_split["train"].select(range(50000))
test_dataset = dataset_split["test"].select(range(5000))
print(f"Training samples: {len(train_dataset)}")
print(f"Test samples: {len(test_dataset)}")

What does this code do?

First, train_test_split(test_size=0.1, seed=42) randomly divides the full dataset into 90% training and 10% test data. The seed=42 ensures the split is reproducible.

Then .select(range(50000)) picks the first 50,000 samples from the training portion, and .select(range(5000)) picks 5,000 from the test portion. These smaller subsets make training feasible on consumer hardware.

  • Run the cell.
  • You should see Training samples: 50000 and Test samples: 5000 printed in the output.

Seeing an index out of range error?

Make sure Cell 4 ran successfully first. The dataset variable must be defined before you can split it.

Help me fix an index or dataset error when creating my train/test split.

Why not train on the full dataset?

Training on 669K pairs with 2 epochs would take several hours even with a GPU. The 50K subset strikes a balance. It is large enough for the model to learn meaningful patterns, but small enough to finish in about 25-30 minutes.

Once you understand the full pipeline, you can always go back and train on more data for better results.

✔️ Awesome, I've got everything!

Great. Double check that your notebook is saved before moving on.

ⓧ I'd like to double check the full code

Here is the complete notebook code up to this point:

# Cell 1: Install packages
!pip install transformers==5.13.0 datasets==5.0.0 evaluate==0.4.6 sacrebleu sentencepiece

# Cell 2: Imports and GPU check
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from transformers import Seq2SeqTrainingArguments, Seq2SeqTrainer
from transformers import DataCollatorForSeq2Seq
from datasets import load_dataset
import evaluate
import numpy as np

print(f"GPU available: {torch.cuda.is_available()}")
if torch.cuda.is_available():
    print(f"GPU name: {torch.cuda.get_device_name(0)}")

# Cell 3: Load model and tokenizer
model_checkpoint = "Helsinki-NLP/opus-mt-en-am"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(model_checkpoint)
print(f"Model loaded: {model_checkpoint}")

# Cell 4: Load the parallel dataset
dataset = load_dataset("michsethowusu/english-amharic_sentence-pairs_mt560")
print(f"Dataset size: {len(dataset['train'])} sentence pairs")
print(dataset["train"][0])

# Cell 5: Explore sample pairs
for i in range(5):
    sample = dataset["train"][i]
    print(f"\nPair {i+1}:")
    print(f"  English: {sample['English']}")
    print(f"  Amharic: {sample['Amharic']}")

# Cell 6: Create train/test split and subset
dataset_split = dataset["train"].train_test_split(test_size=0.1, seed=42)
train_dataset = dataset_split["train"].select(range(50000))
test_dataset = dataset_split["test"].select(range(5000))
print(f"Training samples: {len(train_dataset)}")
print(f"Test samples: {len(test_dataset)}")

Your data is loaded and split into training and test sets. Next, you will tokenize these sentences so the model can process them as numerical sequences.

Preprocess and Tokenize the Data

Your dataset is loaded and split, but the model can't read raw English and Amharic strings directly. Neural networks only understand numbers, so you need to convert each sentence into a sequence of numerical token IDs.

In this step, you'll write a preprocessing function that tokenizes both the English input and Amharic target text, apply it across your datasets, and set up a data collator for efficient batch padding during training.

In this step, get ready to:
  • Write a preprocessing function that tokenizes English and Amharic text.
  • Apply the function across your train and test datasets.
  • Create a data collator for dynamic padding during training.
Write the preprocessing function

The MarianMT tokenizer uses SentencePiece under the hood, which breaks text into subword units. This handles Amharic's Ge'ez script and English equally well because it operates at the character/subword level regardless of alphabet.

  • Create a new cell in your notebook and add the preprocessing function by pasting the following code:
# Cell 7: Preprocessing function
max_length = 128

def preprocess_function(examples):
    inputs = examples["English"]
    targets = examples["Amharic"]
    model_inputs = tokenizer(
        inputs,
        max_length=max_length,
        truncation=True,
    )
    labels = tokenizer(
        text_target=targets,
        max_length=max_length,
        truncation=True,
    )
    model_inputs["labels"] = labels["input_ids"]
    return model_inputs

What does this code do?

This function takes a batch of examples and tokenizes both sides of the translation pair separately.

  • max_length = 128 caps sentence length at 128 tokens. Anything longer gets truncated. This keeps memory usage predictable.
  • The first tokenizer() call converts English text into token IDs that represent the source (input) sentence.
  • The second call uses text_target=targets to tokenize Amharic text as the target. This ensures the tokenizer applies the correct special tokens for the decoder side.
  • model_inputs["labels"] = labels["input_ids"] attaches the Amharic token IDs as labels. During training, the model compares its predictions against these labels to calculate loss.

Why use text_target instead of a second regular call?

MarianMT uses different special tokens for source and target languages. The text_target parameter tells the tokenizer to apply target-language processing rules. Without it, the tokenizer would treat Amharic text as if it were source text and add the wrong special tokens.

  • Run the cell. No output is expected since you're only defining the function.

Seeing a NameError?

  • Make sure you ran all previous cells in order. The function relies on the tokenizer variable from Cell 3.
  • Check that you haven't restarted the kernel since running earlier cells.

Why is my preprocessing function throwing an error?

Apply the preprocessing function to your datasets

Now you'll run the preprocessing function across all 50,000 training samples and 5,000 test samples. The Hugging Face Datasets library makes this efficient with its built-in batched mapping.

  • Create a new cell and apply the preprocessing function by pasting this code:
# Cell 8: Tokenize datasets
tokenized_train = train_dataset.map(
    preprocess_function,
    batched=True,
    remove_columns=train_dataset.column_names,
)
tokenized_test = test_dataset.map(
    preprocess_function,
    batched=True,
    remove_columns=test_dataset.column_names,
)
print(f"Tokenized train columns: {tokenized_train.column_names}")
print(f"Tokenized test columns: {tokenized_test.column_names}")

What does this code do?

  • .map() applies your preprocessing function to every example in the dataset. Setting batched=True processes multiple examples at once for speed.
  • remove_columns=train_dataset.column_names drops the original text columns ("English" and "Amharic") since the model only needs numerical token IDs from here on.
  • The resulting datasets contain three columns: input_ids (tokenized English), attention_mask (which tokens are real vs padding), and labels (tokenized Amharic targets).

How does tokenization work for translation?

In a sequence-to-sequence translation model, the encoder reads the source sentence (input_ids) and the decoder learns to generate the target sentence (labels) one token at a time. The attention_mask tells the encoder which positions contain real tokens versus empty padding.

SentencePiece tokenization splits words into subword pieces. For example, "running" might become ["run", "ning"]. This is especially useful for Amharic because it handles the Ge'ez script's complex character combinations without needing a massive vocabulary.

  • Run the cell. You should see a progress bar for each dataset, followed by output showing the three columns.

Your output should look like this:

Tokenized train columns: ['input_ids', 'attention_mask', 'labels']
Tokenized test columns: ['input_ids', 'attention_mask', 'labels']

Seeing different column names or an error?

  • If you see a KeyError mentioning "English" or "Amharic", double check that your dataset column names match exactly. Run train_dataset.column_names in a new cell to verify.
  • If you see extra columns, make sure remove_columns=train_dataset.column_names is included in both .map() calls.

Help me fix my dataset tokenization error.

Create the data collator

Sentences in your dataset have different lengths. Instead of padding every sentence to 128 tokens upfront (wasteful), a data collator pads each batch dynamically to match only the longest sentence in that batch. This saves GPU memory.

  • Create a new cell and set up the data collator by pasting this code:
# Cell 9: Data collator
data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=model)

What does the data collator do?

DataCollatorForSeq2Seq is a helper from Hugging Face that handles padding at batch time. When the trainer pulls a batch of 16 examples, the collator pads all of them to the length of the longest example in that batch. Shorter batches get less padding, which means faster processing and lower memory usage.

  • Run the cell. No output is expected since this just creates the collator object.

Getting an ImportError for DataCollatorForSeq2Seq?

  • Confirm that Cell 2 imported DataCollatorForSeq2Seq from transformers. Run Cell 2 again if needed.
  • Make sure your transformers package installed correctly in Cell 1. Try restarting the kernel and running all cells from the top.

Why can't I import DataCollatorForSeq2Seq?

✔️ Awesome, I've got everything!

Great. Double check you've saved your notebook.

ⓧ I'd like to double check the full code

Here is the complete notebook up to this point. Compare your cells against this:

# Cell 1: Install packages
!pip install transformers==5.13.0 datasets==5.0.0 evaluate==0.4.6 sacrebleu sentencepiece

# Cell 2: Imports and GPU check
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from transformers import Seq2SeqTrainingArguments, Seq2SeqTrainer
from transformers import DataCollatorForSeq2Seq
from datasets import load_dataset
import evaluate
import numpy as np

print(f"GPU available: {torch.cuda.is_available()}")
if torch.cuda.is_available():
    print(f"GPU name: {torch.cuda.get_device_name(0)}")

# Cell 3: Load model and tokenizer
model_checkpoint = "Helsinki-NLP/opus-mt-en-am"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(model_checkpoint)
print(f"Model loaded: {model_checkpoint}")

# Cell 4: Load the parallel dataset
dataset = load_dataset("michsethowusu/english-amharic_sentence-pairs_mt560")
print(f"Dataset size: {len(dataset['train'])} sentence pairs")
print(dataset["train"][0])

# Cell 5: Explore sample pairs
for i in range(5):
    sample = dataset["train"][i]
    print(f"\nPair {i+1}:")
    print(f"  English: {sample['English']}")
    print(f"  Amharic: {sample['Amharic']}")

# Cell 6: Create train/test split and subset
dataset_split = dataset["train"].train_test_split(test_size=0.1, seed=42)
train_dataset = dataset_split["train"].select(range(50000))
test_dataset = dataset_split["test"].select(range(5000))
print(f"Training samples: {len(train_dataset)}")
print(f"Test samples: {len(test_dataset)}")

# Cell 7: Preprocessing function
max_length = 128

def preprocess_function(examples):
    inputs = examples["English"]
    targets = examples["Amharic"]
    model_inputs = tokenizer(
        inputs,
        max_length=max_length,
        truncation=True,
    )
    labels = tokenizer(
        text_target=targets,
        max_length=max_length,
        truncation=True,
    )
    model_inputs["labels"] = labels["input_ids"]
    return model_inputs

# Cell 8: Tokenize datasets
tokenized_train = train_dataset.map(
    preprocess_function,
    batched=True,
    remove_columns=train_dataset.column_names,
)
tokenized_test = test_dataset.map(
    preprocess_function,
    batched=True,
    remove_columns=test_dataset.column_names,
)
print(f"Tokenized train columns: {tokenized_train.column_names}")
print(f"Tokenized test columns: {tokenized_test.column_names}")

# Cell 9: Data collator
data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=model)

Your data is now fully preprocessed and ready for training. The model has numerical token IDs to work with, and the data collator will handle efficient padding on the fly. Next up, you'll configure the training arguments and fine-tune the model on your 50,000 English-Amharic pairs.

Fine-Tune the Model

Your data is tokenized, padded, and ready to feed into the model. The hard part of data preparation is behind you.

Now comes the core of the project: fine-tuning. You will configure training hyperparameters, assemble the trainer, and let the model learn from 50,000 English-Amharic pairs over two full passes (epochs) through the dataset.

In this step, get ready to:
  • Configure training hyperparameters like batch size, learning rate, and number of epochs.
  • Assemble the Seq2SeqTrainer with your model, data, and settings.
  • Run fine-tuning and watch the model's loss decrease as it learns to translate.
Define the training arguments

Training a model involves dozens of settings that control how fast it learns, how much data it processes at once, and where it saves progress. Hugging Face bundles all of these into a single configuration object called Seq2SeqTrainingArguments.

  • Create a new cell in your notebook (Cell 10) and define the training arguments by pasting this code:
# Cell 10: Training arguments
training_args = Seq2SeqTrainingArguments(
    output_dir="./en-am-finetuned",
    num_train_epochs=2,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    learning_rate=2e-5,
    weight_decay=0.01,
    eval_strategy="epoch",
    save_strategy="epoch",
    predict_with_generate=True,
    fp16=True,
    save_total_limit=2,
    logging_steps=100,
)

What do these arguments control?

  • num_train_epochs=2 tells the model to pass through all 50,000 training pairs twice.
  • per_device_train_batch_size=16 means the model processes 16 sentence pairs at a time before updating its weights.
  • learning_rate=2e-5 controls how much the model adjusts its weights after each batch. A small value prevents it from "forgetting" what it already knows.
  • fp16=True enables half-precision floating point math, which roughly doubles training speed on your GPU with minimal accuracy loss.
  • predict_with_generate=True tells the trainer to use the model's full generation pipeline during evaluation, which is needed for computing translation metrics later.
  • Run the cell.

You won't see any output from this cell. The arguments are stored in training_args and will be used when you create the trainer in the next cell.

Create the Seq2SeqTrainer

The trainer is the object that orchestrates the entire fine-tuning process. It connects your model, training arguments, datasets, tokenizer, and data collator into one cohesive training loop.

  • Create a new cell (Cell 11) and assemble the trainer by pasting this code:
# Cell 11: Create trainer
trainer = Seq2SeqTrainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_train,
    eval_dataset=tokenized_test,
    tokenizer=tokenizer,
    data_collator=data_collator,
)

How does the encoder-decoder architecture translate?

MarianMT uses an encoder-decoder architecture. The encoder reads the entire English sentence and compresses it into a numerical representation that captures its meaning.

The decoder then generates Amharic tokens one at a time, using the encoder's representation to guide each word choice. At every step, it looks at what it has generated so far and picks the next most likely Amharic token.

During training, the decoder is shown the correct Amharic output so it can learn from its mistakes. This is called "teacher forcing."

  • Run the cell.

No output here either. The trainer is assembled and waiting for you to start training in the next cell.

Start fine-tuning

This is the moment everything comes together. One function call kicks off the entire training process across your 50,000 sentence pairs.

  • Create a new cell (Cell 12) and start training by pasting this code:
# Cell 12: Train the model
trainer.train()

What happens during training?

For each batch of 16 sentence pairs, the model attempts to translate the English inputs into Amharic. It then compares its predictions against the correct Amharic outputs and calculates a "loss" value. Loss measures how wrong the predictions are.

The model then adjusts its internal weights through backpropagation, nudging them in the direction that reduces the loss. Over thousands of batches, the model gradually learns the patterns of English-to-Amharic translation.

  • Run the cell. Training will take approximately 25-30 minutes on a single consumer GPU.

You should see a progress bar and periodic log entries showing the training loss. The loss value should decrease over the two epochs, meaning the model's translations are improving.

Training is extremely slow or crashes?

  • If you see an out-of-memory (OOM) error, reduce per_device_train_batch_size from 16 to 8 in Cell 10, then re-run Cells 10, 11, and 12.
  • If training is running on CPU (very slow, no GPU utilization), go back and confirm that torch.cuda.is_available() returned True in Cell 2. If not, your NVIDIA drivers or CUDA toolkit may need updating.
  • If fp16=True causes errors on older GPUs, change it to fp16=False. Training will be slower but still complete.

My model training is crashing or extremely slow. How do I fix this?

✔️ Awesome, I've got everything!

Great. Double check you've saved your notebook before moving on.

ⓧ I'd like to double check the full code

Here is the full notebook up to and including this step. Compare it against your cells to make sure everything matches.

# Cell 1: Install packages
!pip install transformers==5.13.0 datasets==5.0.0 evaluate==0.4.6 sacrebleu sentencepiece

# Cell 2: Imports and GPU check
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from transformers import Seq2SeqTrainingArguments, Seq2SeqTrainer
from transformers import DataCollatorForSeq2Seq
from datasets import load_dataset
import evaluate
import numpy as np

print(f"GPU available: {torch.cuda.is_available()}")
if torch.cuda.is_available():
    print(f"GPU name: {torch.cuda.get_device_name(0)}")

# Cell 3: Load model and tokenizer
model_checkpoint = "Helsinki-NLP/opus-mt-en-am"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(model_checkpoint)
print(f"Model loaded: {model_checkpoint}")

# Cell 4: Load the parallel dataset
dataset = load_dataset("michsethowusu/english-amharic_sentence-pairs_mt560")
print(f"Dataset size: {len(dataset['train'])} sentence pairs")
print(dataset["train"][0])

# Cell 5: Explore sample pairs
for i in range(5):
    sample = dataset["train"][i]
    print(f"\nPair {i+1}:")
    print(f"  English: {sample['English']}")
    print(f"  Amharic: {sample['Amharic']}")

# Cell 6: Create train/test split and subset
dataset_split = dataset["train"].train_test_split(test_size=0.1, seed=42)
train_dataset = dataset_split["train"].select(range(50000))
test_dataset = dataset_split["test"].select(range(5000))
print(f"Training samples: {len(train_dataset)}")
print(f"Test samples: {len(test_dataset)}")

# Cell 7: Preprocessing function
max_length = 128

def preprocess_function(examples):
    inputs = examples["English"]
    targets = examples["Amharic"]
    model_inputs = tokenizer(
        inputs,
        max_length=max_length,
        truncation=True,
    )
    labels = tokenizer(
        text_target=targets,
        max_length=max_length,
        truncation=True,
    )
    model_inputs["labels"] = labels["input_ids"]
    return model_inputs

# Cell 8: Tokenize datasets
tokenized_train = train_dataset.map(
    preprocess_function,
    batched=True,
    remove_columns=train_dataset.column_names,
)
tokenized_test = test_dataset.map(
    preprocess_function,
    batched=True,
    remove_columns=test_dataset.column_names,
)
print(f"Tokenized train columns: {tokenized_train.column_names}")
print(f"Tokenized test columns: {tokenized_test.column_names}")

# Cell 9: Data collator
data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=model)

# Cell 10: Training arguments
training_args = Seq2SeqTrainingArguments(
    output_dir="./en-am-finetuned",
    num_train_epochs=2,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    learning_rate=2e-5,
    weight_decay=0.01,
    eval_strategy="epoch",
    save_strategy="epoch",
    predict_with_generate=True,
    fp16=True,
    save_total_limit=2,
    logging_steps=100,
)

# Cell 11: Create trainer
trainer = Seq2SeqTrainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_train,
    eval_dataset=tokenized_test,
    tokenizer=tokenizer,
    data_collator=data_collator,
)

# Cell 12: Train the model
trainer.train()

Your model has now seen 50,000 English-Amharic pairs twice and adjusted its weights to better translate between the two languages. Next up, you'll measure exactly how much it improved and test it with your own sentences.

Evaluate and Translate

Your model has finished training with loss decreasing across both epochs. The weights have been adjusted, but you still need to answer the most important question: how well does your model actually translate?

In this step, you'll measure translation quality using the BLEU score, build an interactive translation function, and save your fine-tuned model for future use.

In this step, get ready to:
  • Evaluate your model's translation quality with a BLEU score.
  • Build an interactive translate function and test it on English sentences.
  • Save your fine-tuned model locally for reuse.
Evaluate translation quality with BLEU

Training loss tells you the model is learning, but it does not tell you how good the translations are in human terms. You need a metric that compares your model's output against real human translations.

  • Create a new cell in your notebook and add the following evaluation code:
# Cell 13: Evaluate with BLEU
metric = evaluate.load("sacrebleu")

def compute_metrics(eval_preds):
    preds, labels = eval_preds
    decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)
    labels = np.where(labels != -100, labels, tokenizer.pad_token_id)
    decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)
    decoded_preds = [pred.strip() for pred in decoded_preds]
    decoded_labels = [[label.strip()] for label in decoded_labels]
    result = metric.compute(predictions=decoded_preds, references=decoded_labels)
    return {"bleu": result["score"]}

trainer.compute_metrics = compute_metrics
eval_results = trainer.evaluate()
print(f"BLEU score: {eval_results['eval_bleu']:.2f}")

What does this code do?

BLEU (Bilingual Evaluation Understudy) measures how closely your model's translations match human reference translations. It compares overlapping word sequences (n-grams) between the two. The score ranges from 0 to 100, where higher means closer to human quality.

  • evaluate.load("sacrebleu") loads the SacreBLEU metric, which standardizes tokenization so scores are reproducible.
  • compute_metrics decodes the model's token predictions back into text, replaces padding tokens (-100) in labels, then computes the BLEU score.
  • References are wrapped in a list of lists because SacreBLEU supports multiple valid translations per sentence.
  • trainer.evaluate() runs the model on the test set and prints the BLEU score.
  • Run the cell. This evaluation takes a few minutes since the model generates translations for all 5,000 test sentences.
  • You should see a BLEU score printed at the end. Any score above 0 confirms your model is producing meaningful translations rather than random text.

Seeing a CUDA out of memory error?

  • If your GPU runs out of memory during evaluation, restart the kernel and re-run all cells from the beginning. The training process may have left cached tensors in memory.
  • Make sure no other notebooks or processes are using your GPU at the same time.

Why is my BLEU evaluation failing or returning an error?

Build an interactive translation function

A BLEU score gives you a corpus-level number, but seeing individual translations is far more satisfying. You'll now build a helper function that translates any English sentence you type.

  • Create a new cell and add the translation function along with test sentences:
# Cell 14: Interactive translation function
def translate(text):
    inputs = tokenizer(text, return_tensors="pt", max_length=128, truncation=True)
    inputs = {k: v.to(model.device) for k, v in inputs.items()}
    outputs = model.generate(**inputs, max_new_tokens=128)
    translation = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return translation

# Test translations
test_sentences = [
    "Hello, how are you?",
    "What is your name?",
    "I love Ethiopia.",
    "The weather is beautiful today.",
    "Thank you very much.",
]

print("English to Amharic Translations:")
print("=" * 50)
for sentence in test_sentences:
    translation = translate(sentence)
    print(f"EN: {sentence}")
    print(f"AM: {translation}")
    print()

What does this code do?

  • translate() takes an English string, tokenizes it into tensors, and moves those tensors to the GPU.
  • model.generate() runs the decoder to produce Amharic tokens one at a time until the sentence is complete.
  • tokenizer.decode() converts the generated token IDs back into readable Amharic text, skipping special tokens like padding markers.
  • The test loop runs five common English phrases through the model so you can visually inspect the Amharic output.
  • Run the cell. You should see five English sentences each followed by their Amharic translation in Ge'ez script.

What is Ge'ez script?

Amharic uses the Ge'ez (Ethiopic) script, which is a non-Latin writing system called an abugida. Each character represents a consonant-vowel combination. If your output shows characters like እንደምን ነh, your model is producing real Amharic text.

Seeing empty or repeated output?

  • If the translations are empty strings, check that model.device shows cuda:0. The inputs and model must be on the same device.
  • If you see the same word repeated many times, this is a sign the model did not converge during training. Re-run the training cell and ensure loss decreased over both epochs.

Why is my translation function returning empty or garbled output?

Save the fine-tuned model

Your fine-tuned model now produces better Amharic translations than the base checkpoint you started with. The base model was trained broadly on OPUS data, but your fine-tuning specialized it on 50,000 English-Amharic pairs from the MT560 dataset.

  • Save both the model weights and tokenizer to a local directory by creating a new cell with this code:
# Cell 15: Save the fine-tuned model
model.save_pretrained("./en-am-finetuned-final")
tokenizer.save_pretrained("./en-am-finetuned-final")
print("Model saved to ./en-am-finetuned-final")

What does this code do?

  • model.save_pretrained() writes the model's updated weights and configuration to the ./en-am-finetuned-final folder.
  • tokenizer.save_pretrained() saves the SentencePiece vocabulary and tokenizer config alongside the model so they stay paired together.
  • With both saved, you can reload this exact model later without re-training by calling AutoModelForSeq2SeqLM.from_pretrained("./en-am-finetuned-final").
  • Run the cell. You should see the confirmation message printed.
  • Verify the saved model exists by checking your file browser in JupyterLab. You should see an en-am-finetuned-final folder containing model files.

Don't see the saved folder?

  • Make sure the cell finished without errors. A disk-full error will prevent the save from completing.
  • Click the refresh button in JupyterLab's file browser panel. New folders sometimes do not appear until you refresh.

Why isn't my model saving to disk?

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook with all 15 cells.

ⓧ I'd like to double check the full code

Here is the complete notebook code from Cell 1 through Cell 15. Compare your notebook against this:

# Cell 1: Install packages
!pip install transformers==5.13.0 datasets==5.0.0 evaluate==0.4.6 sacrebleu sentencepiece

# Cell 2: Imports and GPU check
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from transformers import Seq2SeqTrainingArguments, Seq2SeqTrainer
from transformers import DataCollatorForSeq2Seq
from datasets import load_dataset
import evaluate
import numpy as np

print(f"GPU available: {torch.cuda.is_available()}")
if torch.cuda.is_available():
    print(f"GPU name: {torch.cuda.get_device_name(0)}")

# Cell 3: Load model and tokenizer
model_checkpoint = "Helsinki-NLP/opus-mt-en-am"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(model_checkpoint)
print(f"Model loaded: {model_checkpoint}")

# Cell 4: Load the parallel dataset
dataset = load_dataset("michsethowusu/english-amharic_sentence-pairs_mt560")
print(f"Dataset size: {len(dataset['train'])} sentence pairs")
print(dataset["train"][0])

# Cell 5: Explore sample pairs
for i in range(5):
    sample = dataset["train"][i]
    print(f"\nPair {i+1}:")
    print(f"  English: {sample['English']}")
    print(f"  Amharic: {sample['Amharic']}")

# Cell 6: Create train/test split and subset
dataset_split = dataset["train"].train_test_split(test_size=0.1, seed=42)
train_dataset = dataset_split["train"].select(range(50000))
test_dataset = dataset_split["test"].select(range(5000))
print(f"Training samples: {len(train_dataset)}")
print(f"Test samples: {len(test_dataset)}")

# Cell 7: Preprocessing function
max_length = 128

def preprocess_function(examples):
    inputs = examples["English"]
    targets = examples["Amharic"]
    model_inputs = tokenizer(
        inputs,
        max_length=max_length,
        truncation=True,
    )
    labels = tokenizer(
        text_target=targets,
        max_length=max_length,
        truncation=True,
    )
    model_inputs["labels"] = labels["input_ids"]
    return model_inputs

# Cell 8: Tokenize datasets
tokenized_train = train_dataset.map(
    preprocess_function,
    batched=True,
    remove_columns=train_dataset.column_names,
)
tokenized_test = test_dataset.map(
    preprocess_function,
    batched=True,
    remove_columns=test_dataset.column_names,
)
print(f"Tokenized train columns: {tokenized_train.column_names}")
print(f"Tokenized test columns: {tokenized_test.column_names}")

# Cell 9: Data collator
data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=model)

# Cell 10: Training arguments
training_args = Seq2SeqTrainingArguments(
    output_dir="./en-am-finetuned",
    num_train_epochs=2,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    learning_rate=2e-5,
    weight_decay=0.01,
    eval_strategy="epoch",
    save_strategy="epoch",
    predict_with_generate=True,
    fp16=True,
    save_total_limit=2,
    logging_steps=100,
)

# Cell 11: Create trainer
trainer = Seq2SeqTrainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_train,
    eval_dataset=tokenized_test,
    tokenizer=tokenizer,
    data_collator=data_collator,
)

# Cell 12: Train the model
trainer.train()

# Cell 13: Evaluate with BLEU
metric = evaluate.load("sacrebleu")

def compute_metrics(eval_preds):
    preds, labels = eval_preds
    decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)
    labels = np.where(labels != -100, labels, tokenizer.pad_token_id)
    decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)
    decoded_preds = [pred.strip() for pred in decoded_preds]
    decoded_labels = [[label.strip()] for label in decoded_labels]
    result = metric.compute(predictions=decoded_preds, references=decoded_labels)
    return {"bleu": result["score"]}

trainer.compute_metrics = compute_metrics
eval_results = trainer.evaluate()
print(f"BLEU score: {eval_results['eval_bleu']:.2f}")

# Cell 14: Interactive translation function
def translate(text):
    inputs = tokenizer(text, return_tensors="pt", max_length=128, truncation=True)
    inputs = {k: v.to(model.device) for k, v in inputs.items()}
    outputs = model.generate(**inputs, max_new_tokens=128)
    translation = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return translation

# Test translations
test_sentences = [
    "Hello, how are you?",
    "What is your name?",
    "I love Ethiopia.",
    "The weather is beautiful today.",
    "Thank you very much.",
]

print("English to Amharic Translations:")
print("=" * 50)
for sentence in test_sentences:
    translation = translate(sentence)
    print(f"EN: {sentence}")
    print(f"AM: {translation}")
    print()

# Cell 15: Save the fine-tuned model
model.save_pretrained("./en-am-finetuned-final")
tokenizer.save_pretrained("./en-am-finetuned-final")
print("Model saved to ./en-am-finetuned-final")

Secret mission

Build a Bidirectional Translator

Your translator only works in one direction right now. What if someone sends you Amharic text and you need to understand it in English? In this mission, you'll train the reverse direction and build a unified function that translates both ways.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally, so there are no ongoing cloud costs.

Resources you used:

  • Jupyter notebooks (amharic_translation.ipynb and amharic_translation_bidirectional.ipynb).
  • Model checkpoint directories (./en-am-finetuned and ./am-en-finetuned).
  • Saved fine-tuned model directories (./en-am-finetuned-final and ./am-en-finetuned-final).
  • Hugging Face cached dataset and models (stored in ~/.cache/huggingface/).
  • Installed pip packages (transformers, datasets, evaluate, sacrebleu, sentencepiece).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your translation models or want to fine-tune on more data.

  • Your fine-tuned models remain in their saved directories, ready to load at any time.
  • The Hugging Face cache keeps the downloaded dataset and base model weights locally so future runs skip the download step.
  • The installed pip packages stay available in your Anaconda environment for other NLP projects.

Pause - I'll come back to this later

Shut down the Jupyter kernel to free up GPU memory. Your files stay on disk for when you return.

  • In JupyterLab, click Kernel in the menu bar, then select Shut Down All Kernels.
  • This releases GPU memory while keeping all notebooks and model files intact.
  • When you return, open the notebook and run the cells from the top to reload everything.

Delete - I don't want to use this again

Remove all project files, cached data, and installed packages to start completely fresh.

  • In JupyterLab, click Kernel in the menu bar, then select Shut Down All Kernels.
  • Open a terminal and remove the project notebooks and model directories by running:
rm amharic_translation.ipynb amharic_translation_bidirectional.ipynb
rm -rf ./en-am-finetuned ./en-am-finetuned-final
rm -rf ./am-en-finetuned ./am-en-finetuned-final
  • Clear the Hugging Face cache to remove the downloaded dataset and base model weights by running:
rm -rf ~/.cache/huggingface/hub/models--Helsinki-NLP--opus-mt-en-am
rm -rf ~/.cache/huggingface/hub/models--Helsinki-NLP--opus-mt-am-en
rm -rf ~/.cache/huggingface/datasets/michsethowusu___english-amharic_sentence-pairs_mt560
  • Uninstall the pip packages you installed for this project by running:
pip uninstall transformers datasets evaluate sacrebleu sentencepiece -y

Nice Work!

Nice Work!

You just fine-tuned a neural machine translation model that translates English into Amharic. From loading a 669K-sentence parallel corpus to evaluating your results with BLEU scores, you now have hands-on experience with the full sequence-to-sequence fine-tuning pipeline.

You've learned how to:

  • Fine-tuned a MarianMT model on 50,000 English-Amharic sentence pairs using the Hugging Face Seq2SeqTrainer.
  • Preprocessed multilingual text with SentencePiece tokenization and understood how the encoder-decoder architecture processes translation.
  • Evaluated translation quality with BLEU scores and built an interactive translator that converts English text to Amharic.
  • Secret Mission: Trained the reverse direction (Amharic to English) and built a complete bidirectional translator.

Ready to quiz yourself?