Predict Flight Delays with ML

Train ML models on real U.S. government data to predict flight delays.

Introduction

30 Second Summary

You check your flight status and see "on time," but your plane still arrives 45 minutes late. The patterns behind flight delays are hiding in hundreds of thousands of government records.

In this project, you will build a machine learning model that predicts whether a flight will arrive late. You will train two models on real U.S. government data and discover why one catches far more delays than the other.

What You'll Build

Run the final cell of your Jupyter Notebook and see a confusion matrix heatmap showing exactly how many delayed flights your trained model correctly predicted.

By the end of this project, you'll have:

  • A trained model that predicts whether a U.S. domestic flight will arrive 15+ minutes late, built on real government data from the Bureau of Transportation Statistics.
  • Visualizations revealing which airlines, airports, and days of the week have the worst delay rates across hundreds of thousands of actual flight records.
  • Side-by-side comparisons of two ML models, with an understanding of why precision and recall matter more than accuracy alone for imbalanced datasets.
  • Secret Mission: Engineer new features from departure time and route popularity, and tune your Random Forest to boost recall even further.

Do I need machine learning experience?

This project assumes basic Python knowledge (loops, functions, variables) and Python 3.11 or higher. No prior machine learning experience is needed.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

You'll train a machine learning model on real U.S. government flight data, going beyond simple averages to predict whether a specific flight will arrive late.

Set Up Your Environment

Every machine learning project starts with the same foundation: the right libraries installed and real data ready to load.

In this step, you will install four Python libraries that power your flight delay predictor, download real airline performance records from the U.S. government, and take your first look at the raw data inside a Jupyter Notebook.

In this step, get ready to:
  • Install and verify the Python data science libraries.
  • Download real flight data from the Bureau of Transportation Statistics.
  • Load the data into a Jupyter Notebook and run an initial exploration.
Install the required Python libraries

Your flight delay predictor relies on four libraries. Each one handles a different part of the data science pipeline.

  • Install all four libraries by running this command in your VS Code terminal:
pip install scikit-learn pandas matplotlib seaborn

What do these libraries do?

  • pandas loads and manipulates tabular data like spreadsheets.
  • matplotlib draws charts and plots from your data.
  • seaborn builds on matplotlib to create polished statistical visualizations with less code.
  • scikit-learn provides the machine learning algorithms, data splitting, and evaluation tools you will use to train and test your models.
  • Verify everything installed correctly by running:
python -c "import sklearn, pandas, matplotlib, seaborn; print('All good!')"

You should see All good! printed in your terminal.

  • Before you run this next command, what version number do you expect to see for scikit-learn?
  • Confirm the exact scikit-learn version by running:
python -c "import sklearn; print(sklearn.__version__)"

You should see 1.9.0 (or your installed version). This confirms scikit-learn is ready.

Seeing an error instead of a version number?

  • Make sure the terminal in VS Code is using the same Python interpreter as your notebook. You can check this by opening the Command Palette with Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows) and selecting Python: Select Interpreter.
  • If you see a message about pip not being found, try running pip3 instead of pip in the install command.

Help me troubleshoot my Python library installation in VS Code.

Download the flight data from the Bureau of Transportation Statistics

The Bureau of Transportation Statistics (BTS) collects detailed flight records from every U.S. airline. You will download one month of real data directly from this government source.

  • Open the BTS download page in your browser:

https://www.transtats.bts.gov/DL_SelectFields.aspx?gnoyr_VQ=FGJ&QO_fu146_anzr=b0-gvzr

What is this dataset?

This is the Reporting Carrier On-Time Performance database. It contains over 233 million flight records going back to 1987. Airlines are required to report this data to the U.S. Department of Transportation, so it reflects real operations, not synthetic or curated tutorial data.

  • In the Filter Year dropdown, select 2025.
  • In the Filter Period dropdown, select January.
  • Scroll down to the field list and check the boxes for these 15 fields:
  • FlightDate
  • Reporting_Airline
  • Origin
  • Dest
  • CRSDepTime
  • DepDelay
  • DepDel15
  • CRSArrTime
  • ArrDelay
  • ArrDel15
  • Cancelled
  • Diverted
  • Distance
  • CRSElapsedTime
  • DayOfWeek

Only one month at a time

The BTS download page only lets you download one month of data per request. January 2025 gives you roughly 500,000 to 600,000 flight records. That is plenty for training a machine learning model.

  • Click Download at the bottom of the page.
  • Unzip the downloaded file.
  • Find the CSV file inside (it is usually named T_ONTIME_REPORTING.csv) and move it into your project folder.
  • Confirm the file is in your project folder by running:
ls

You should see T_ONTIME_REPORTING.csv listed in the output.

Don't see the CSV file?

  • Make sure you unzipped the downloaded file. The download comes as a .zip archive.
  • Check that you moved the CSV into the same folder where your notebook will live, not a subfolder inside the zip.
  • The BTS website can sometimes be slow or return errors under heavy load. If the download fails, try again or use a different browser.

Help me find my downloaded BTS flight data CSV file.

Create your notebook and explore the data

Now that your libraries are installed and data is ready, it is time to load the flight records into a Jupyter Notebook and confirm everything works.

  • In VS Code, create a new file called flight_delays.ipynb in your project folder.
  • In the first cell, paste this code to import your libraries and load the data:
# Cell 1: Imports and data loading
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.preprocessing import LabelEncoder

# Load the BTS flight data
df = pd.read_csv("T_ONTIME_REPORTING.csv")
print(f"Dataset shape: {df.shape}")
print(df.head())

What does this code do?

  • The import lines bring in all the tools you will use throughout the project. You import everything up front so the notebook does not error later when you need a function.
  • pd.read_csv("T_ONTIME_REPORTING.csv") loads your entire CSV file into a pandas DataFrame called df.
  • df.shape prints the number of rows and columns, confirming the data loaded.
  • df.head() shows the first five rows so you can see what the raw records look like.
  • Run Cell 1 by clicking the play button to the left of the cell, or by pressing Shift+Enter.

You should see output showing a shape like (500000, 16) (the exact number depends on the month you downloaded) followed by a table of the first five rows.

Seeing a FileNotFoundError?

  • This means Python cannot find T_ONTIME_REPORTING.csv in the same folder as your notebook. Make sure both files are in the same directory.
  • If your CSV has a slightly different name (like a date suffix), update the filename inside pd.read_csv() to match.

Help me fix a FileNotFoundError when loading my BTS CSV file.

Now that the data is loaded, take a deeper look at what is inside it.

  • Click into a new cell below Cell 1 and paste this code to explore the dataset:
# Cell 2: Initial exploration
print(df.info())
print(df.describe())
print(f"\nColumn names: {df.columns.tolist()}")

What does this code do?

  • df.info() prints each column's name, data type, and how many non-null values it has. This is your first look at missing data.
  • df.describe() calculates summary statistics (mean, min, max, standard deviation) for every numeric column.
  • df.columns.tolist() prints a clean list of all column names so you can confirm the 15 fields you selected are present.
  • Before you run this cell, how many columns do you expect to see in the output?
  • Run Cell 2 by pressing Shift+Enter.

You should see a list of your 15 columns (plus one extra unnamed index column that BTS adds), data types for each, and summary statistics showing values like average distance and delay times.

Seeing unexpected columns or missing data?

  • If you see far more than 15 columns, you may have checked too many fields on the BTS download page. Re-download with only the 15 fields listed above.
  • Some missing values in columns like ArrDelay and DepDelay are expected. Cancelled flights have no arrival data. You will clean these in a later step.

Help me understand unexpected columns or missing values in my BTS flight data.

✔️ Awesome, I've got everything!

Great. Make sure you have saved flight_delays.ipynb with both cells.

ⓧ I'd like to double check the full code

Here is the complete notebook code you should have so far:

# Cell 1: Imports and data loading
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.preprocessing import LabelEncoder

# Load the BTS flight data
df = pd.read_csv("T_ONTIME_REPORTING.csv")
print(f"Dataset shape: {df.shape}")
print(df.head())
# Cell 2: Initial exploration
print(df.info())
print(df.describe())
print(f"\nColumn names: {df.columns.tolist()}")

Each code block above is a separate cell in your notebook. Make sure Cell 1 is in the first cell and Cell 2 is in the second cell.

Your environment is ready and you have hundreds of thousands of real flight records loaded. Next up, you will explore delay patterns in this data and discover a critical challenge that will shape your entire machine learning approach.

Explore the Data and Discover the Delay Problem

Your data is loaded and you have confirmed hundreds of thousands of real flight records are sitting in your notebook. Now the question is: what does this data actually look like, and is there anything tricky about it that could trip up a machine learning model?

Before building any model, you need to explore the data and look for patterns. This process is called Exploratory Data Analysis (EDA). In this step, EDA will reveal a critical insight that shapes every decision in the rest of the project.

In this step, get ready to:
  • Review the raw data to understand column types and spot missing values.
  • Visualize the balance between delayed and on-time flights.
  • Chart delay patterns across airlines and days of the week.
Review your data's structure

In the previous step, you ran Cells 1 and 2 in your notebook. Those cells printed the output of df.head(), df.info(), and df.describe(). Scroll back up to those outputs now and look for three things.

  • Check the df.shape output from Cell 1. You should see hundreds of thousands of rows. Each row is one flight.
  • Check the df.info() output from Cell 2. Notice that some columns have fewer non-null entries than others. That means missing data exists.
  • Check the df.describe() output. Look at the ArrDel15 column. Its mean is the fraction of flights that arrived 15+ minutes late.

What is ArrDel15?

ArrDel15 is a column the U.S. Bureau of Transportation Statistics pre-computes for every flight. It equals 1 if the flight arrived 15 or more minutes after its published arrival time, and 0 otherwise.

This makes it a perfect target for binary classification. Your model will learn to predict whether this value is 0 or 1 for a given flight.

Visualize the delay class imbalance

The ArrDel15 mean from df.describe() gives you a number, but a chart makes the imbalance impossible to ignore. Let's plot the distribution.

  • Create a new cell below Cell 2 in flight_delays.ipynb and paste the following code:
# Cell 3: Visualize delay class imbalance
fig, ax = plt.subplots(figsize=(6, 4))
sns.countplot(x="ArrDel15", data=df, ax=ax)
ax.set_title("Flight Arrival Delay Distribution")
ax.set_xlabel("Delayed 15+ Minutes (0=No, 1=Yes)")
ax.set_ylabel("Number of Flights")
plt.tight_layout()
plt.show()

total = df["ArrDel15"].count()
delayed = df["ArrDel15"].sum()
print(f"On-time: {total - delayed} ({(total - delayed) / total * 100:.1f}%)")
print(f"Delayed: {delayed} ({delayed / total * 100:.1f}%)")

What does this code do?

  • sns.countplot() creates a bar chart that counts how many flights fall into each category of ArrDel15 (0 for on-time, 1 for delayed).
  • The second half calculates exact percentages. df["ArrDel15"].sum() counts delayed flights because each delayed flight has a value of 1.

Before you run the cell, take a guess: what percentage of flights do you think arrive more than 15 minutes late?

  • Run Cell 3.

You should see a bar chart where the on-time bar (0) towers over the delayed bar (1). The printed percentages confirm it: roughly 80% of flights are on time and only about 20% are delayed.

Why does this imbalance matter?

Imagine a model that always predicts "not delayed" for every single flight. It would be right roughly 80% of the time. That looks like great accuracy, but it would never catch a single delay.

This is the trap of class imbalance. When one category dominates, accuracy becomes misleading. You will see this play out firsthand when you train your first model in a later step.

Chart not showing?

  • Make sure you ran Cell 1 first. Cell 3 depends on df, plt, and sns which are all set up in Cell 1.
  • If you see a NameError, the kernel may have restarted. Re-run Cells 1 and 2 from the top, then run Cell 3 again.

Help me debug why my seaborn countplot isn't displaying in Jupyter Notebook.

Explore delay patterns by airline and day of week

You know the overall split, but are all airlines equally likely to be delayed? Are certain days of the week worse? Let's find out.

  • Create a new cell below Cell 3 and paste the following code:
# Cell 4: Delay patterns by airline and day of week
fig, axes = plt.subplots(1, 2, figsize=(14, 5))

airline_delay = df.groupby("Reporting_Airline")["ArrDel15"].mean().sort_values(ascending=False)
airline_delay.plot(kind="bar", ax=axes[0], color="coral")
axes[0].set_title("Delay Rate by Airline")
axes[0].set_ylabel("Delay Rate")
axes[0].set_xlabel("Airline Code")

day_delay = df.groupby("DayOfWeek")["ArrDel15"].mean()
day_delay.plot(kind="bar", ax=axes[1], color="steelblue")
axes[1].set_title("Delay Rate by Day of Week")
axes[1].set_ylabel("Delay Rate")
axes[1].set_xlabel("Day of Week (1=Mon, 7=Sun)")

plt.tight_layout()
plt.show()

What does this code do?

  • df.groupby("Reporting_Airline")["ArrDel15"].mean() calculates the fraction of delayed flights for each airline. Because ArrDel15 is 0 or 1, the mean is the delay rate.
  • The same logic applies to DayOfWeek. Day 1 is Monday and day 7 is Sunday.
  • plt.subplots(1, 2) places both charts side by side in a single figure so you can compare them at a glance.
  • Run Cell 4.

You should see two bar charts side by side. The left chart shows delay rates by airline code, sorted from highest to lowest. The right chart shows delay rates for each day of the week.

  • Look at the left chart and identify which airline code has the highest delay rate.
  • Look at the right chart and identify which day of the week has the most delays.

Why explore patterns before training?

These charts show you that airline and day of week do correlate with delay rates. That is good news. It means a model that uses these features has something real to learn from, not just noise.

Seeing overlapping labels on the airline chart?

  • This is normal when many airlines are plotted on the x-axis. The chart is still readable. If labels overlap heavily, try widening your notebook window or increasing the figsize width from 14 to 18.

Help me fix overlapping x-axis labels on my matplotlib bar chart.

✔️ Awesome, I've got everything!

Great. Double check that you have saved your notebook with all four cells.

ⓧ I'd like to double check the full code

Your notebook should have four cells in this order. Scroll through your notebook and confirm each one is present.

  • Cell 1: Imports and data loading. Starts with import pandas as pd and ends with print(df.head()).
  • Cell 2: Initial exploration. Contains df.info(), df.describe(), and the column names print.
  • Cell 3: Delay class imbalance visualization. Starts with fig, ax = plt.subplots(figsize=(6, 4)) and ends with the on-time/delayed percentage prints.
  • Cell 4: Delay patterns by airline and day of week. Starts with fig, axes = plt.subplots(1, 2, figsize=(14, 5)) and ends with plt.show().

You have explored the data and uncovered two key insights: most flights are on time (making the dataset imbalanced), and certain airlines and days correlate with higher delay rates. Next up, you will clean this data and engineer features so a machine learning model can actually learn from it.

Clean the Data and Prepare Features

In the last step, you explored the data and uncovered a crucial pattern. Roughly 80% of flights are on time, while only 20% are delayed.

The raw data still has problems that will crash your model if left unfixed. Cancelled flights carry no arrival information, and text columns like airline codes are completely unreadable to ML algorithms.

In this step, get ready to:
  • Remove cancelled and diverted flights, then drop rows with missing arrival data.
  • Convert text columns into numeric codes the model can read.
  • Split the cleaned data into training and test sets.
Remove cancelled and diverted flights

Cancelled and diverted flights never arrived at their destination. Their arrival delay fields are empty, which means they will cause errors during model training.

  • Create a new cell in your notebook and paste this code:
# Cell 5: Data cleaning
# Remove cancelled and diverted flights
df = df[(df["Cancelled"] == 0) & (df["Diverted"] == 0)]

# Drop rows where arrival delay indicator is missing
df = df.dropna(subset=["ArrDel15"])

print(f"Shape after cleaning: {df.shape}")
print(f"Missing values:\n{df.isnull().sum()}")

What does this code do?

  • The filter keeps only rows where Cancelled == 0 and Diverted == 0. This removes every flight that never completed its route.
  • dropna(subset=["ArrDel15"]) catches any remaining rows where the arrival delay indicator is missing. These flights have no label for the model to learn from.
  • Run the cell.
  • You should see the new dataset shape printed, showing fewer rows than before. The missing values count prints below it.

Seeing a KeyError?

  • Make sure you selected the Cancelled and Diverted checkboxes when downloading data from BTS. If those columns are missing, re-download the CSV with all required fields checked.
  • Confirm that Cells 1 through 4 ran successfully before this cell. The df variable must already exist from Cell 1.
  • Help me fix a KeyError in my flight delay cleaning cell.
Engineer features from the raw columns

ML models only understand numbers. Columns like Reporting_Airline, Origin, and Dest contain text (like "AA" or "LAX") that you need to convert into numeric codes.

You also need to extract useful signals hiding inside raw values. The CRSDepTime column stores departure times as integers like 1430 for 2:30 PM. Pulling out just the hour gives the model a cleaner pattern to learn from.

  • Create a new cell and paste this code to extract the departure hour and encode the text columns:
# Cell 6: Feature engineering
# Extract departure hour from CRSDepTime (format: hhmm as integer)
df["DepHour"] = df["CRSDepTime"] // 100

# Encode categorical columns
le_airline = LabelEncoder()
le_origin = LabelEncoder()
le_dest = LabelEncoder()

df["Airline_Code"] = le_airline.fit_transform(df["Reporting_Airline"])
df["Origin_Code"] = le_origin.fit_transform(df["Origin"])
df["Dest_Code"] = le_dest.fit_transform(df["Dest"])

What does this code do?

  • df["CRSDepTime"] // 100 uses integer division to extract the hour from the hhmm format. For example, 1430 becomes 14 (2 PM).
  • LabelEncoder from scikit-learn converts each unique text value into a number. "AA" might become 0, "DL" might become 3, and so on.
  • With 300+ airports, one-hot encoding would create hundreds of sparse columns. Label encoding keeps the feature matrix compact, and Random Forest handles label-encoded values well because it splits on individual values.
  • Below the encoding code in the same cell, paste this code to select your features and target:
# Select features (drop columns that leak future info or are non-numeric)
feature_cols = ["DayOfWeek", "DepHour", "Distance", "CRSElapsedTime",
                "Airline_Code", "Origin_Code", "Dest_Code"]

X = df[feature_cols]
y = df["ArrDel15"].astype(int)

# Drop any remaining rows with NaN in features
mask = X.isnull().any(axis=1)
X = X[~mask]
y = y[~mask]

print(f"Features shape: {X.shape}")
print(f"Target distribution:\n{y.value_counts()}")

Why are DepDelay and ArrDelay excluded?

DepDelay and ArrDelay record the actual delay that already happened. Using them to predict whether a flight was delayed would be circular. The model would just be reading the answer instead of learning patterns.

This is called data leakage. The feature_cols list includes only information you would know before the flight departs: day of week, scheduled departure hour, distance, airline, and airports.

  • Run the cell.
  • You should see the features shape printed (the number of rows and 7 columns) followed by the target distribution showing the count of on-time (0) and delayed (1) flights.

Seeing NaN or shape mismatch errors?

  • Make sure Cell 5 (the cleaning cell) ran before this cell. The cancelled and diverted flights must be removed first.
  • Check that all the column names in feature_cols match exactly, including capitalization. DayOfWeek is different from dayofweek.
  • Help me debug a feature engineering error in my flight delay notebook.
Split the data into training and test sets

You need two separate chunks of data: one for the model to learn from (training) and one to test its predictions on unseen flights (test). This prevents the model from simply memorizing the training data and performing well only on flights it has already seen.

Before you run this next cell, think about it: if your cleaned dataset has, say, 500,000 rows and you use an 80/20 split, roughly how many rows should end up in the training set?

  • Create a new cell and paste this code to split the data:
# Cell 7: Train/test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

print(f"Training set: {X_train.shape}")
print(f"Test set: {X_test.shape}")

What does this code do?

  • train_test_split randomly shuffles the data and reserves 20% (test_size=0.2) for testing. The remaining 80% is used for training.
  • random_state=42 locks the random shuffle so you get the same split every time you run the cell. This makes your results reproducible.
  • The split creates four variables: X_train and y_train for training the model, and X_test and y_test for evaluating how well it performs on data it has never seen.
  • Run the cell.
  • You should see the training set shape (roughly 80% of your rows, with 7 feature columns) and the test set shape (roughly 20% of your rows, with 7 feature columns).

Seeing a NameError for X or y?

  • Cell 6 (the feature engineering cell) must run before this cell. X and y are created there.
  • If you restarted the notebook kernel, re-run all cells from Cell 1 through Cell 6 in order before running Cell 7.
  • Help me fix a NameError in my train/test split cell.

✔️ Awesome, I've got everything!

Make sure you have saved flight_delays.ipynb. Your notebook should now have Cells 1 through 7 all executed successfully.

ⓧ I'd like to double check the full code

Scroll through your notebook and confirm each cell is present in this order:

  • Cell 1: Imports and data loading. Starts with import pandas as pd and ends with print(df.head()).
  • Cell 2: Initial exploration. Calls df.info(), df.describe(), and prints column names.
  • Cell 3: Delay class imbalance bar chart and percentage printout.
  • Cell 4: Delay patterns by airline and day of week (two side-by-side bar charts).
  • Cell 5: Data cleaning. Filters out cancelled and diverted flights, drops null ArrDel15 rows, prints new shape and missing values.
  • Cell 6: Feature engineering. Extracts DepHour, encodes categorical columns, defines feature_cols, creates X and y, drops NaN rows, and prints features shape and target distribution.
  • Cell 7: Train/test split. Creates X_train, X_test, y_train, y_test and prints both shapes.

Your data is cleaned, encoded, and split into training and test sets. Next up, you will train your first ML model and discover why a high accuracy score can be completely misleading.

Train a Baseline Model and See It Fall Short

Your data is cleaned, encoded, and split into training and test sets. Now it is time to train a model and find out if it can actually predict flight delays.

You will start with Logistic Regression because it is one of the simplest classification algorithms. It is fast to train and easy to interpret, which makes it a solid baseline. The real question is whether a simple model can handle the class imbalance you discovered in Step 2.

In this step, get ready to:
  • Train a Logistic Regression model on your flight data.
  • Evaluate the model with a classification report and confusion matrix to see how well it catches delays.
Train a Logistic Regression model

Logistic Regression draws a straight decision boundary between two classes. It looks at your features and calculates a probability that each flight belongs to the "delayed" class.

Why start with Logistic Regression?

In machine learning, you always start with a simple baseline before trying complex models. If a simple model solves the problem, you do not need anything fancier.

If the simple model falls short, the baseline gives you a concrete number to beat. Without it, you have no way to tell whether a fancier model is actually better.

You already imported LogisticRegression in Cell 1, so you can jump straight into training.

  • Create a new cell below Cell 7 and add the following training code:
# Cell 8: Train Logistic Regression baseline
lr_model = LogisticRegression(max_iter=1000, random_state=42)
lr_model.fit(X_train, y_train)
lr_preds = lr_model.predict(X_test)

What does this code do?

  • LogisticRegression(max_iter=1000, random_state=42) creates the model. max_iter=1000 gives the model more iterations to find the best fit. random_state=42 makes results reproducible.
  • .fit(X_train, y_train) trains the model on your training data. It learns patterns that connect the features to whether a flight was delayed.
  • .predict(X_test) uses the trained model to predict delays on the test data it has never seen before.
Evaluate with a classification report and confusion matrix

Training the model is only half the job. You need to measure how well it actually performs on flights it has never seen.

  • Below the training code in the same cell, add the evaluation code:
print("=== Logistic Regression ===")
print(classification_report(y_test, lr_preds, target_names=["On Time", "Delayed"]))

fig, ax = plt.subplots(figsize=(5, 4))
cm_lr = confusion_matrix(y_test, lr_preds)
sns.heatmap(cm_lr, annot=True, fmt="d", cmap="Blues",
            xticklabels=["On Time", "Delayed"],
            yticklabels=["On Time", "Delayed"], ax=ax)
ax.set_title("Logistic Regression Confusion Matrix")
ax.set_xlabel("Predicted")
ax.set_ylabel("Actual")
plt.tight_layout()
plt.show()

What does this evaluation code do?

  • classification_report prints a table with precision, recall, and f1-score for each class (On Time and Delayed). These metrics tell you much more than accuracy alone.
  • confusion_matrix creates a grid showing exactly how many flights the model got right and wrong. The seaborn heatmap makes the numbers easy to read at a glance.
  • target_names=["On Time", "Delayed"] labels the classes so the report is human-readable instead of showing raw 0s and 1s.

Before you run this cell, think about what you expect. About 80% of flights are on time. What do you think the model's accuracy will be, and will it catch most of the delayed flights?

  • Run Cell 8.

You should see a classification report printed above a blue heatmap. Look at the accuracy line first. It says roughly 80%.

That sounds good, right? Now look at the recall value for the Delayed row. It is below 0.40. That means the model misses more than 60% of delayed flights.

Why is 80% accuracy misleading here?

Remember that about 80% of flights in your data are on time. A model that blindly predicts "not delayed" for every single flight would also score ~80% accuracy. The Logistic Regression model is doing almost exactly that.

The confusion matrix confirms it. Look at the bottom-left cell of the heatmap. That large number represents delayed flights the model predicted as on time. It is predicting "not delayed" for almost everything because that is the easy, high-accuracy answer.

This is why recall matters for imbalanced data. Precision tells you "of the flights I flagged as delayed, how many actually were?" Recall tells you "of all the actually delayed flights, how many did I catch?" A model with low recall is useless for catching delays, no matter how good its accuracy looks.

Seeing an error when running Cell 8?

  • If you see a NameError for X_train or y_train, make sure you ran all previous cells in order. Go to Cell 1 and run each cell down to Cell 7 before running Cell 8.
  • If the heatmap does not display, check that your imports in Cell 1 include import seaborn as sns and import matplotlib.pyplot as plt.

Still stuck? Help me debug an error in my Logistic Regression cell in flight_delays.ipynb.

✔️ Awesome, I've got everything!

Great. Double check you have saved flight_delays.ipynb.

ⓧ I'd like to double check the full code

Here is the complete Cell 8. Make sure your cell matches this exactly:

# Cell 8: Train Logistic Regression baseline
lr_model = LogisticRegression(max_iter=1000, random_state=42)
lr_model.fit(X_train, y_train)
lr_preds = lr_model.predict(X_test)

print("=== Logistic Regression ===")
print(classification_report(y_test, lr_preds, target_names=["On Time", "Delayed"]))

fig, ax = plt.subplots(figsize=(5, 4))
cm_lr = confusion_matrix(y_test, lr_preds)
sns.heatmap(cm_lr, annot=True, fmt="d", cmap="Blues",
            xticklabels=["On Time", "Delayed"],
            yticklabels=["On Time", "Delayed"], ax=ax)
ax.set_title("Logistic Regression Confusion Matrix")
ax.set_xlabel("Predicted")
ax.set_ylabel("Actual")
plt.tight_layout()
plt.show()

The Logistic Regression model looks accurate on paper but completely fails at the one job that matters: catching delayed flights. In the next step, you will train a more powerful model and see if it can do better.

Improve with Random Forest and Understand the Results

In the last step, you uncovered a critical problem. Your Logistic Regression model looked like it was 80% accurate, but it was achieving that accuracy by predicting "not delayed" for nearly every flight.

Now you will train a Random Forest model on the same data. Random Forest handles non-linear patterns and feature interactions that Logistic Regression misses. By comparing the two models side by side, you will build real intuition for reading evaluation metrics like precision, recall, and the confusion matrix.

In this step, get ready to:
  • Train a Random Forest model and compare its recall against the Logistic Regression baseline.
  • Visualize both confusion matrices side by side to see where each model gets it right and wrong.
  • Inspect feature importances to discover which inputs drive delay predictions.
Train a Random Forest model

A Random Forest is an ensemble of many decision trees. Each tree sees a random subset of the data and features, then they all vote on the final prediction. This makes the model much better at capturing complex patterns that a single linear boundary cannot.

Why Random Forest over Logistic Regression?

Logistic Regression draws a single straight line (or hyperplane) to separate delayed from on-time flights. If the real pattern is more complex, it can only approximate.

Random Forest builds hundreds of decision trees that each learn different parts of the pattern. The trees vote together, which captures non-linear relationships like "evening flights from small airports with a specific airline are more likely to be delayed."

You already imported RandomForestClassifier in Cell 1, so it is ready to use.

Before you run the next cell, think about this: do you expect Random Forest to catch more delayed flights than Logistic Regression, or fewer? Why?

  • Create a new cell below Cell 8 and add this code to train the Random Forest and print its classification report:
# Cell 9: Train Random Forest and compare
rf_model = RandomForestClassifier(n_estimators=100, random_state=42)
rf_model.fit(X_train, y_train)
rf_preds = rf_model.predict(X_test)

print("=== Random Forest ===")
print(classification_report(y_test, rf_preds, target_names=["On Time", "Delayed"]))

What does this code do?

  • RandomForestClassifier(n_estimators=100, random_state=42) creates a forest of 100 decision trees. The random_state ensures you get the same results each run.
  • .fit(X_train, y_train) trains all 100 trees on your training data. Each tree learns slightly different patterns because it samples a random subset of rows and features.
  • classification_report prints precision, recall, and f1-score for both classes. This is where you compare against the Logistic Regression numbers from Step 4.
  • Run Cell 9.
  • Look at the recall value for the Delayed class. It should be noticeably higher than what Logistic Regression achieved.

The Random Forest catches more delayed flights. That improved recall is the payoff for using an algorithm that can learn non-linear patterns.

Not seeing improved recall?

  • Make sure you ran all previous cells (1 through 8) before running Cell 9. The model depends on X_train and y_train from Cell 7.
  • Check that you used n_estimators=100 and random_state=42. Different values produce different results.
  • My Random Forest results look the same as Logistic Regression. What could be wrong?
Compare both models side by side

Numbers tell you the recall improved, but a visual comparison makes the difference unmistakable. You will plot both confusion matrices next to each other so you can see exactly where each model gets it right and wrong.

  • In the same Cell 9, add this visualization code below the print(classification_report(...)) line:
fig, axes = plt.subplots(1, 2, figsize=(12, 4))

cm_rf = confusion_matrix(y_test, rf_preds)
sns.heatmap(cm_lr, annot=True, fmt="d", cmap="Blues",
            xticklabels=["On Time", "Delayed"],
            yticklabels=["On Time", "Delayed"], ax=axes[0])
axes[0].set_title("Logistic Regression")
axes[0].set_xlabel("Predicted")
axes[0].set_ylabel("Actual")

sns.heatmap(cm_rf, annot=True, fmt="d", cmap="Greens",
            xticklabels=["On Time", "Delayed"],
            yticklabels=["On Time", "Delayed"], ax=axes[1])
axes[1].set_title("Random Forest")
axes[1].set_xlabel("Predicted")
axes[1].set_ylabel("Actual")

plt.tight_layout()
plt.show()

What does this code do?

  • plt.subplots(1, 2) creates two charts side by side in one figure.
  • cm_lr (from Cell 8) goes on the left in blue. cm_rf (computed here) goes on the right in green.
  • The bottom-right cell of each heatmap shows true positives: flights the model correctly predicted as delayed. Compare that number across the two charts to see the improvement.
  • Run Cell 9 again.
  • You should see the classification report printed above two side-by-side heatmaps. The green Random Forest heatmap should show a larger number in the bottom-right cell (true positives) compared to the blue Logistic Regression heatmap.

How do I read a confusion matrix?

Each confusion matrix is a 2x2 grid. The rows represent what actually happened (on time or delayed). The columns represent what the model predicted.

  • Top-left: flights that were on time and predicted on time (true negatives).
  • Top-right: flights that were on time but predicted delayed (false positives).
  • Bottom-left: flights that were delayed but predicted on time (false negatives). These are the dangerous misses.
  • Bottom-right: flights that were delayed and correctly predicted delayed (true positives). More here means better recall.

Seeing a NameError for cm_lr?

  • The variable cm_lr was created in Cell 8. If you restarted your kernel since then, re-run all cells from Cell 1 through Cell 8 first.
  • I'm getting a NameError when I run my confusion matrix comparison. How do I fix this?
Inspect feature importances

You now know Random Forest outperforms Logistic Regression for catching delays. The next question is: what is the model actually looking at? Random Forest can tell you how much each feature contributed to its predictions.

Before you run the next cell, make a prediction. Which feature do you think matters most for predicting flight delays: distance, departure hour, airline, or something else?

  • Create a new Cell 10 below Cell 9 and add this code to extract and visualize feature importances:
# Cell 10: Feature importances
importances = rf_model.feature_importances_
feat_imp = pd.Series(importances, index=feature_cols).sort_values(ascending=True)

fig, ax = plt.subplots(figsize=(8, 4))
feat_imp.plot(kind="barh", ax=ax, color="teal")
ax.set_title("Random Forest Feature Importances")
ax.set_xlabel("Importance")
plt.tight_layout()
plt.show()

What does this code do?

  • rf_model.feature_importances_ returns an array of scores showing how much each feature helped the model split delayed from on-time flights.
  • pd.Series(importances, index=feature_cols) pairs each score with its feature name so the chart labels make sense.
  • .sort_values(ascending=True) sorts from least to most important. The horizontal bar chart then shows the most important feature at the top.
  • Run Cell 10.
  • You should see a horizontal bar chart with all seven features ranked by importance. Check whether your prediction about the most important feature was correct.

What do the importances tell you?

Features with longer bars contribute more to the model's decisions. You might see destination airport, origin airport, or departure hour near the top. These are the factors that most strongly predict whether a flight will arrive late.

This does not mean the top feature "causes" delays. It means the model found that feature most useful for separating delayed flights from on-time flights in the training data.

Seeing an empty chart or an error?

  • Make sure Cell 9 ran successfully before Cell 10. The variable rf_model must exist in memory.
  • Check that feature_cols is defined in Cell 6. If you restarted your kernel, re-run all cells from Cell 1.
  • My feature importance chart is not rendering. How do I debug this?

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook with all cells through Cell 10.

ⓧ I'd like to double check the full code

Your notebook should now have 10 cells. Scroll through and confirm each one is present in order:

  • Cell 1: Imports and data loading.
  • Cell 2: Initial exploration.
  • Cell 3: Visualize delay class imbalance.
  • Cell 4: Delay patterns by airline and day of week.
  • Cell 5: Data cleaning.
  • Cell 6: Feature engineering.
  • Cell 7: Train/test split.
  • Cell 8: Train Logistic Regression baseline.
  • Cell 9: Train Random Forest and compare (includes classification report and side-by-side heatmaps).
  • Cell 10: Feature importances (horizontal bar chart).

If any cell is missing, scroll up to the relevant substep and copy the code from there.

You now have two trained models and a clear picture of which features matter most. Next up, you will push the model even further in the Secret Mission by engineering new features and tuning hyperparameters to catch even more delayed flights.

Secret mission

Boost Your Model with Feature Engineering and Tuning

Your Random Forest catches more delays than Logistic Regression, but it still misses a lot. Real data scientists don't stop at switching algorithms. They engineer smarter features and tune hyperparameters to squeeze out better performance. Can you boost your model's recall for delayed flights?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally, so there are no ongoing costs.

Resources you used:

  • flight_delays.ipynb (your Jupyter Notebook with all exploration, training, and evaluation code).
  • T_ONTIME_REPORTING.csv (the flight data downloaded from the Bureau of Transportation Statistics).
  • Python libraries installed via pip: scikit-learn, pandas, matplotlib, seaborn.

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your flight delay models or want to try new features and algorithms.

  • Your notebook and data file stay on your machine, ready to reopen at any time.
  • The installed Python libraries remain available for other data science projects.

Pause - I'll come back to this later

Shut down running processes to free up memory. Your files remain on disk for when you return.

  • In VS Code, click Kernel in the top toolbar of your notebook, then click Restart to shut down the running Python kernel and free memory.
  • Close VS Code when you are done. Your notebook and CSV file will still be on disk when you return.

Delete - I don't want to use this again

Remove all project files and optionally uninstall the Python libraries you installed.

  • Close flight_delays.ipynb in VS Code.
  • In your terminal, navigate to the folder containing your project files.

Delete your notebook and data file by running this command on macOS or Linux:

rm flight_delays.ipynb T_ONTIME_REPORTING.csv

On Windows, run this instead:

del flight_delays.ipynb, T_ONTIME_REPORTING.csv
  • Optionally, uninstall the Python libraries you installed for this project by running:
pip uninstall scikit-learn pandas matplotlib seaborn -y

Should I uninstall these libraries?

Libraries like pandas, matplotlib, and scikit-learn are foundational tools for data science in Python. If you plan to do more data analysis or ML projects, keep them installed.

Only uninstall if you are certain you will not use them again.

Nice Work!

Nice Work!

Nice work! You just built a flight delay prediction system powered by real U.S. government data, and you can now explain why a model with 80% accuracy can still be useless.

You've learned how to:

  • Download real flight records from the U.S. Bureau of Transportation Statistics and explore delay patterns across airlines, airports, and days of the week using pandas and seaborn visualizations.
  • Clean messy real-world data by removing cancelled flights, handling missing values, and encoding categorical features with scikit-learn's LabelEncoder.
  • Train and compare Logistic Regression and Random Forest models, using precision, recall, and confusion matrices to evaluate performance on an imbalanced dataset where accuracy alone is misleading.
  • Secret Mission: Engineer new features like DepHourBin and RouteFrequency, and tune your Random Forest with class_weight="balanced" to boost recall for delayed flights.

Ready to quiz yourself?