Classify Kepler Exoplanet Signals

Build a leakage-safe model to classify Kepler candidate signals.

Introduction

30 Second Summary

A tiny dip in a star's brightness can point to a planet passing in front of it. The same dip can come from another effect that only looks planetary.

In this project, you will build a Windows-friendly Python program for binary classification that labels held-out Kepler objects as CANDIDATE or FALSE POSITIVE. It retrieves authentic records from the NASA Exoplanet Archive before comparing a shortcut affected by target leakage with an honest model based on physical measurements.

What You'll Build

Your finished program turns authentic Kepler signals into held-out CANDIDATE or FALSE POSITIVE predictions you can inspect in the terminal.

By the end of this project, you'll have:

  • A live Kepler data summary showing sample objects plus the class balance between CANDIDATE and FALSE POSITIVE.
  • A leakage audit that exposes how disposition-related fields can produce a misleading model score.
  • An honest logistic regression evaluation with balanced accuracy, precision, recall, F1 scores, ten held-out predictions, plus a saved confusion matrix.
  • Secret Mission: A same-test-set comparison between the honest logistic regression model and a random forest to identify which model scores higher.

Are there any prerequisites?

You need Windows 10 or 11, Python 3.11 or newer, Visual Studio Code, plus basic Python syntax. An internet connection lets you download the free archive data plus the pinned packages.

Before We Start

You are committing to build a binary classifier that labels Kepler Objects of Interest as CANDIDATE or FALSE POSITIVE. It will learn from physical measurements so archived vetting decisions cannot create target leakage.

Set Up the Windows ML Environment

Package conflicts can alter a machine learning result before your model sees any astronomy data.

This step verifies Python 3.11 or newer. It then creates a dedicated virtual environment with exact package versions.

In this step, get ready to:
  • Verify Python 3.11 or newer in Command Prompt.
  • Build an activated virtual environment for the pinned packages.
  • Create the two starter files and verify every installed version.
Verify Python in your project folder

The pinned packages require Python 3.11 or newer. Confirming that version now prevents incompatible package installations later.

Visual Studio Code keeps your starter files together in one workspace. A Command Prompt terminal inside that workspace keeps each setup command tied to the same folder.

  • Open Windows Search from the taskbar.
  • Search for Visual Studio Code.
  • Select Visual Studio Code from the search results.

You will see the Visual Studio Code window.

  • Choose the folder-opening option on the welcome screen.

You will see a Windows folder picker.

  • Create a folder named kepler-exoplanet-classifier on your Desktop.

You will see kepler-exoplanet-classifier in the folder picker.

  • Select kepler-exoplanet-classifier as your Visual Studio Code workspace.

The file sidebar now shows kepler-exoplanet-classifier as the open folder.

  • Use the Visual Studio Code terminal menu to create an integrated terminal.

You will see a terminal panel at the bottom of Visual Studio Code.

  • Use the terminal profile menu to select Command Prompt.

The Command Prompt path should end with kepler-exoplanet-classifier. This confirms that later files belong to the correct workspace.

  • Check your installed Python version by running this command:
py --version

What Does This Check Do?

The py launcher finds Python on Windows. The reported version determines whether the pinned packages can be installed.

✔️ I see version 3.11 or higher

Your Python installation supports every pinned package in this project. That compatibility check is complete.

ⓧ I see an older version

The installed Python version cannot support the required package releases. A current 64-bit Python 3 release provides the compatible runtime.

  • Visit python.org in your browser.
  • Download a current 64-bit Python 3 release for Windows.
  • Run the downloaded installer.
  • Complete the installation instructions.
  • Restart Visual Studio Code.
  • Return to the kepler-exoplanet-classifier workspace.
  • Create a new Command Prompt terminal from the terminal profile menu.
  • Check the upgraded Python version by running this command:
py --version

What Does This Check Confirm?

This check asks the Windows Python launcher for its newly available version. Continue when the result reports 3.11 or newer.

Still Seeing the Older Version?

  • Close every existing terminal before checking the version again.
  • Restart Windows if a new terminal still reports the older version.

If the older version remains, help me troubleshoot the Windows Python launcher.

ⓧ Command not found

Windows cannot find Python through its launcher. Installing a current 64-bit Python 3 release provides the runtime required by this project.

  • Visit python.org in your browser.
  • Download a current 64-bit Python 3 release for Windows.
  • Run the downloaded installer.
  • Complete the installation instructions.
  • Restart Visual Studio Code.
  • Return to the kepler-exoplanet-classifier workspace.
  • Create a new Command Prompt terminal from the terminal profile menu.
  • Check the new Python installation by running this command:
py --version

What Does This Check Confirm?

This check confirms that Windows can now reach Python through its launcher. Continue when the result reports 3.11 or newer.

Python Still Unavailable?

  • Close every existing terminal before checking again.
  • Restart Windows if a new terminal still cannot find Python.

If the command still fails, help me troubleshoot my Windows Python installation.

Build the pinned virtual environment

Why Use a Virtual Environment?

A virtual environment gives this project its own package collection. Other Python projects can use different versions without changing this classifier.

The requirements.txt file records the exact versions needed to rebuild that environment.

  • Use the new-file control in the Visual Studio Code file sidebar.
  • Name the file requirements.txt.

You will see requirements.txt in the file sidebar. Its editor starts empty.

  • Fill requirements.txt with the exact package pins by pasting this content:
pandas==3.1.0
scikit-learn==1.9.1
matplotlib==3.11.2

What Do These Packages Provide?

  • pandas 3.1.0 loads the Kepler table into rows and columns.
  • scikit-learn 1.9.1 provides preprocessing tools plus classifiers.
  • Matplotlib 3.11.2 creates the confusion-matrix image.
  • Save requirements.txt.
  • Compare the editor with the three package lines above.

The file now records three exact package versions. Each package line uses two equals signs before its version.

Package Pins Look Different?

  • Check that each package name uses the spelling shown above.
  • Check that every version uses two equals signs.
  • Remove any additional package lines from requirements.txt.

If the file still differs, help me fix my requirements file.

  • Create the .venv virtual environment by running this command:
py -m venv .venv

What Does This Command Do?

The command creates an isolated Python environment inside .venv. Packages installed there stay separate from your global Python installation.

  • Confirm that the file sidebar now lists a .venv folder.

The new folder proves that the isolated environment exists inside kepler-exoplanet-classifier.

Virtual Environment Not Appearing?

  • Confirm that the Command Prompt path ends with kepler-exoplanet-classifier.
  • Wait for the terminal prompt to return.
  • Refresh the Visual Studio Code file sidebar.

If .venv is still missing, help me troubleshoot virtual environment creation.

  • Activate the virtual environment in Command Prompt by running this command:
.venv\Scripts\activate

What Does Activation Change?

Activation points Python package commands at the environment inside .venv. The Command Prompt now identifies the environment as active.

You will see the environment name at the beginning of the Command Prompt line. Package installations in this terminal now stay inside the project.

Environment Not Activating?

  • Confirm that your terminal profile is Command Prompt.
  • Confirm that .venv appears in the file sidebar.
  • Confirm that the terminal path ends with kepler-exoplanet-classifier.

If activation still fails, help me activate my Windows virtual environment.

  • Install the pinned packages from requirements.txt by running this command:
py -m pip install -r requirements.txt

What Does This Command Install?

The command reads each exact pin from requirements.txt. It installs those packages plus the dependencies they need inside the active environment.

  • Wait for the Command Prompt to return.

You will see package installation activity in the terminal. The returned prompt confirms that the command has finished.

Package Installation Failing?

  • Confirm that the environment name appears at the start of the Command Prompt line.
  • Confirm that requirements.txt contains the three exact pins shown above.
  • Confirm that your internet connection is available.

If installation still stops, help me diagnose the package installation.

Create the starter script and verify packages

The next step needs an empty Python file for the Kepler workflow. A final package inventory proves that the active environment contains every required pin.

  • Use the new-file control in the Visual Studio Code file sidebar.
  • Name the file exoplanet_classifier.py.

You will see exoplanet_classifier.py beside requirements.txt. The Python file remains empty during this setup step.

Before you run the final check, consider whether the global Python packages or the activated environment's pins will appear.

  • Inspect the activated environment by running this command:
py -m pip freeze

What Does This Inventory Prove?

The command lists every package installed in the active environment. Matching pins prove that this project uses the versions recorded in requirements.txt.

Among the dependency lines, you will see pandas==3.1.0, scikit-learn==1.9.1, plus matplotlib==3.11.2. This confirms that the isolated environment matches your package file.

One of the Pins Missing?

  • Confirm that the environment name appears at the start of the Command Prompt line.
  • Repeat the package installation command from the previous substep.
  • Repeat the final package inventory command after installation finishes.

If a pin still does not appear, help me verify my isolated package versions.

✔️ Awesome, I've got everything!

Your package inventory contains all three exact pins. Double check that requirements.txt is saved.

Your exoplanet_classifier.py file should contain zero characters at this point.

ⓧ I'd like to double check the full code

Your complete requirements.txt file should look exactly like this:

pandas==3.1.0
scikit-learn==1.9.1
matplotlib==3.11.2

Your complete exoplanet_classifier.py file is empty. Its editor should contain zero characters.

That is the setup locked in. Your Windows environment now has the exact tools needed to download the Kepler signals.

Meet the Kepler Signals

Your isolated Python environment is ready to run the project's machine learning libraries. The classifier now needs authentic observations from the NASA Exoplanet Archive.

Pandas will load the archive's CSV response into a table. You will inspect its labels before measuring the class balance.

In this step, get ready to:
  • Connect the script to the Kepler Q1-Q17 DR25 data.
  • Separate twelve physical measurements from five disposition-related fields.
  • Print sample objects with the disposition counts.
Connect to the Kepler archive

The script needs data tools for this step. It also prepares the scikit-learn classes plus the Matplotlib plotting tools used later.

  • Switch back to the empty exoplanet_classifier.py tab from earlier.
  • Paste the project imports at the top of exoplanet_classifier.py:
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.dummy import DummyClassifier
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    ConfusionMatrixDisplay,
    balanced_accuracy_score,
    classification_report,
)
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

What Do These Imports Prepare?

  • Pandas retrieves the archive rows plus stores them in a table.
  • Matplotlib supplies the plotting tools for the confusion matrix.
  • The scikit-learn imports prepare the split, preprocessing pipeline, models, plus evaluation metrics used in later steps.
  • Save exoplanet_classifier.py.
  • Return to the Command Prompt from the previous step.
  • Check that every imported package loads by running:
python exoplanet_classifier.py

What Does This Check Prove?

Python loads every imported module before reaching the end of the file. A clean return to the prompt confirms that the installed packages satisfy these imports.

You should return to the command prompt without a traceback. Your verified environment can load every dependency required by the finished script.

Imports Not Loading?

Check that the command prompt still shows the activated .venv environment. Switch back to the activated terminal from the previous step if you used a different window.

Compare each import with the code block above. A missing parenthesis in the metrics import prevents the file from loading.

Help me troubleshoot the imports in my Kepler classifier.

The archive query requests the object name plus its disposition. It also requests twelve physical measurements plus five fields connected to the archived vetting decision.

  • Paste the archive URL plus shared labels below the import group:
DATA_URL = (
    "https://exoplanetarchive.ipac.caltech.edu/TAP/sync?"
    "query=select+kepoi_name,koi_pdisposition,koi_score,"
    "koi_fpflag_nt,koi_fpflag_ss,koi_fpflag_co,koi_fpflag_ec,"
    "koi_period,koi_impact,koi_duration,koi_depth,koi_prad,koi_teq,"
    "koi_insol,koi_model_snr,koi_steff,koi_slogg,koi_srad,koi_kepmag+"
    "from+q1_q17_dr25_koi+where+koi_pdisposition+=+%27CANDIDATE%27+"
    "or+koi_pdisposition+=+%27FALSE+POSITIVE%27&format=csv"
)

TARGET_COLUMN = "koi_pdisposition"
ID_COLUMN = "kepoi_name"
LABEL_ORDER = ["CANDIDATE", "FALSE POSITIVE"]

How Is the Archive Configured?

  • DATA_URL holds the verified query for the selected Kepler table plus its requested columns.
  • TARGET_COLUMN identifies the disposition that the classifier eventually predicts.
  • ID_COLUMN identifies each Kepler Object of Interest in the sample output.
  • LABEL_ORDER keeps CANDIDATE plus FALSE POSITIVE in a consistent display order.
  • Save exoplanet_classifier.py.
  • Check the archive configuration for syntax errors by running:
python exoplanet_classifier.py

What Does This Run Check?

Python parses the complete archive query plus the three shared constants. Returning to the prompt confirms that every string has a matching quote.

You should return to the command prompt without a traceback. The verified query is ready for the download code.

Seeing a Syntax Problem?

Compare the opening parenthesis below DATA_URL with its closing parenthesis. Check that every URL fragment begins plus ends with a double quote.

Help me check the Kepler archive configuration.

Separate the signal groups

The selected columns serve two roles. The physical measurements describe the transit signal or its host star.

The disposition score plus false-positive flags summarize parts of the vetting process. Keeping these fields separate prepares the leakage test for a later step.

  • Paste the science feature list below LABEL_ORDER:
SCIENCE_FEATURES = [
    "koi_period",
    "koi_impact",
    "koi_duration",
    "koi_depth",
    "koi_prad",
    "koi_teq",
    "koi_insol",
    "koi_model_snr",
    "koi_steff",
    "koi_slogg",
    "koi_srad",
    "koi_kepmag",
]

What Are the Science Features?

SCIENCE_FEATURES contains twelve transit or host-star measurements. These fields become the physical evidence available to the honest classifier.

  • Save exoplanet_classifier.py.
  • Count the entries in SCIENCE_FEATURES.

You should find twelve quoted field names. The list begins with koi_period plus ends with koi_kepmag.

Science Feature Count Incorrect?

Check for a missing quoted field near the end of the list. Confirm that the list has matching square brackets.

Help me compare my science feature list.

  • Paste the leakage feature list below SCIENCE_FEATURES:
LEAKY_FEATURES = [
    "koi_score",
    "koi_fpflag_nt",
    "koi_fpflag_ss",
    "koi_fpflag_co",
    "koi_fpflag_ec",
]

What Are the Leakage Features?

LEAKY_FEATURES contains the disposition score plus four false-positive flags. This separation makes their effect measurable later.

  • Save exoplanet_classifier.py.
  • Count the entries in LEAKY_FEATURES.

You should find five quoted field names. The list contains koi_score plus the four koi_fpflag_* fields.

Leakage Feature Count Incorrect?

Check for a missing false-positive flag. Confirm that every field name ends with a comma.

Help me compare my leakage feature list.

A Pipeline keeps missing-value handling plus feature scaling attached to the model. The builder creates the same preprocessing sequence whenever a later step needs a fresh logistic model.

  • Paste the model builder below LEAKY_FEATURES:
def build_logistic_model():
    return Pipeline(
        steps=[
            ("imputer", SimpleImputer(strategy="median")),
            ("scaler", StandardScaler()),
            ("model", LogisticRegression(max_iter=1000)),
        ]
    )

What Does This Function Prepare?

  • SimpleImputer prepares numeric features by replacing missing measurements with a median learned during training.
  • StandardScaler prepares measurements with different numeric ranges for logistic regression.
  • LogisticRegression supplies the binary classifier used later in the project.
  • build_logistic_model() returns a fresh pipeline whenever the project needs one.
  • Save exoplanet_classifier.py.
  • Check the feature lists plus model builder by running:
python exoplanet_classifier.py

What Does This Check Cover?

Python creates both feature lists plus defines the pipeline builder. A clean return confirms that the list punctuation plus function indentation are valid.

You should return to the command prompt without a traceback. The feature groups plus reusable model definition are ready.

Function Not Loading?

Check that the three pipeline steps sit inside the steps list. Confirm that the closing parentheses align with return Pipeline(.

Help me fix the model builder.

Download and inspect the rows

The query becomes useful when pandas reads its response into data. A compact summary reveals the table size plus the labels that define this classification problem.

  • Paste the download plus inspection code below build_logistic_model():
print("Downloading Kepler Q1-Q17 DR25 data...")
data = pd.read_csv(DATA_URL)

print(f"\nDataset shape: {data.shape}")
print("\nFirst five Kepler Objects of Interest:")
print(data[[ID_COLUMN, TARGET_COLUMN]].head())
print("\nDisposition counts:")
print(data.value_counts(TARGET_COLUMN))

What Does This Code Reveal?

  • pd.read_csv(DATA_URL) downloads the HTTPS response into a pandas table.
  • data.shape reports the number of downloaded rows plus requested columns.
  • head() selects the first five KOI names with their dispositions.
  • value_counts() totals the rows assigned to each disposition.
  • Save exoplanet_classifier.py.

Before you run the script, which disposition do you expect to appear more often?

  • Download the selected rows plus reveal the dataset summary by running:
python exoplanet_classifier.py

What Does This Run Prove?

The command executes the verified archive query through your script. The printed sample confirms that the response contains KOI identifiers paired with project labels.

You will see a dataset shape followed by five KOI names beside their dispositions. The final section lists counts for CANDIDATE plus FALSE POSITIVE.

That is your first real astronomy dataset loaded into the project. Its class counts now set the standard for the baseline model.

Dataset Output Missing?

Check that your internet connection can reach the archive. A network failure stops pandas before the dataset summary prints.

Compare DATA_URL with the full code below if the response does not contain the requested columns.

Help me troubleshoot the Kepler data download.

✔️ Awesome, I've got everything!

Great. Double check that exoplanet_classifier.py is saved. Confirm that the terminal shows the dataset shape plus both disposition counts.

ⓧ I'd like to double check the full code

Your complete exoplanet_classifier.py file should look like this:

import matplotlib.pyplot as plt
import pandas as pd
from sklearn.dummy import DummyClassifier
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    ConfusionMatrixDisplay,
    balanced_accuracy_score,
    classification_report,
)
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

DATA_URL = (
    "https://exoplanetarchive.ipac.caltech.edu/TAP/sync?"
    "query=select+kepoi_name,koi_pdisposition,koi_score,"
    "koi_fpflag_nt,koi_fpflag_ss,koi_fpflag_co,koi_fpflag_ec,"
    "koi_period,koi_impact,koi_duration,koi_depth,koi_prad,koi_teq,"
    "koi_insol,koi_model_snr,koi_steff,koi_slogg,koi_srad,koi_kepmag+"
    "from+q1_q17_dr25_koi+where+koi_pdisposition+=+%27CANDIDATE%27+"
    "or+koi_pdisposition+=+%27FALSE+POSITIVE%27&format=csv"
)

TARGET_COLUMN = "koi_pdisposition"
ID_COLUMN = "kepoi_name"
LABEL_ORDER = ["CANDIDATE", "FALSE POSITIVE"]

SCIENCE_FEATURES = [
    "koi_period",
    "koi_impact",
    "koi_duration",
    "koi_depth",
    "koi_prad",
    "koi_teq",
    "koi_insol",
    "koi_model_snr",
    "koi_steff",
    "koi_slogg",
    "koi_srad",
    "koi_kepmag",
]

LEAKY_FEATURES = [
    "koi_score",
    "koi_fpflag_nt",
    "koi_fpflag_ss",
    "koi_fpflag_co",
    "koi_fpflag_ec",
]


def build_logistic_model():
    return Pipeline(
        steps=[
            ("imputer", SimpleImputer(strategy="median")),
            ("scaler", StandardScaler()),
            ("model", LogisticRegression(max_iter=1000)),
        ]
    )


print("Downloading Kepler Q1-Q17 DR25 data...")
data = pd.read_csv(DATA_URL)

print(f"\nDataset shape: {data.shape}")
print("\nFirst five Kepler Objects of Interest:")
print(data[[ID_COLUMN, TARGET_COLUMN]].head())
print("\nDisposition counts:")
print(data.value_counts(TARGET_COLUMN))

Your script now retrieves authentic Kepler observations plus exposes the class balance behind the prediction task. Next, you will test whether a simple baseline can classify the held-out rows.

Establish an Honest Baseline

Your script now reveals authentic Kepler rows. It also shows the balance between the two dispositions.

A trained classifier earns its value by beating a simple rule that ignores every astronomical measurement. In this step, a most-frequent baseline from scikit-learn sets that floor.

In this step, get ready to:
  • Create a reproducible stratified train-test partition.
  • Train a most-frequent baseline on the physical measurements.
  • Measure its balanced accuracy on held-out rows.
Create a stratified train-test split

A train-test split reserves rows for evaluation. Stratification preserves each label's proportion across both partitions.

The test rows stay untouched during training. This gives every model the same unseen evaluation data.

  • Find print(data.value_counts(TARGET_COLUMN)) at the bottom of exoplanet_classifier.py.
  • Create the feature table and stratified partitions by adding this code directly below that line:
all_features = SCIENCE_FEATURES + LEAKY_FEATURES
X = data[all_features]
y = data[TARGET_COLUMN]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y,
)

What does this code do?

  • The all_features list combines the physical measurements with the disposition-related fields needed in the next experiment.
  • The X table holds the selected feature columns.
  • The y series holds the target disposition.
  • The test_size=0.2 setting reserves 20 percent of the rows for evaluation.
  • The stratify=y setting preserves the disposition proportions in each partition.
  • The random_state=42 setting produces the same partition every time the script runs.
  • Save exoplanet_classifier.py.
  • Check that the new split runs by using this command:
python exoplanet_classifier.py

What should I see?

You should see the dataset shape followed by the five sample objects. The disposition counts should appear before the terminal returns to the command prompt.

A run without a traceback confirms that the stratified split completed successfully.

Does the split stop with an error?

Check that the new code sits below print(data.value_counts(TARGET_COLUMN)). Confirm that every closing parenthesis matches the target code.

Make sure the .venv from earlier remains active. The terminal prompt should still show its environment name.

Help me debug the train-test split.

Train and score the baseline

The most-frequent baseline learns which disposition appears most often in the training rows. It predicts that disposition for every held-out row.

Balanced accuracy averages recall across the two classes. This prevents the larger class from dominating the evaluation.

  • Add the baseline workflow directly below the closing parenthesis of train_test_split() by copying this code:
baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(X_train[SCIENCE_FEATURES], y_train)
baseline_predictions = baseline.predict(X_test[SCIENCE_FEATURES])
baseline_score = balanced_accuracy_score(y_test, baseline_predictions)
print(f"\nMost-frequent baseline balanced accuracy: {baseline_score:.3f}")

How does the baseline work?

  • The DummyClassifier(strategy="most_frequent") rule learns which label appears most often in the training rows.
  • The call to fit() uses only the training partition.
  • The baseline_predictions array stores one prediction for each held-out row.
  • The balanced_accuracy_score() function averages recall across both dispositions.
  • The final print statement rounds baseline_score to three decimal places.
  • Save exoplanet_classifier.py.

Before you run the script, do you expect a rule that always chooses one class to perform strongly across both classes?

  • Measure the baseline by running this command:
python exoplanet_classifier.py

What should I see?

You should see Most-frequent baseline balanced accuracy: followed by a score with three decimal places. This line should appear after the disposition counts.

That score is the minimum reference point for every trained model that follows.

Baseline result missing?

Confirm that the baseline code sits below the complete train_test_split() call. Save the file before running the script again.

Check the spellings of SCIENCE_FEATURES and baseline_predictions against the code above.

Help me fix the missing baseline result.

You have established a fair performance floor for the Kepler classifier. Every later result now has a meaningful score to beat.

✔️ Awesome, I've got everything!

Great work. Double check that exoplanet_classifier.py is saved with the baseline result at the bottom.

ⓧ I'd like to double check the full code

Compare your complete exoplanet_classifier.py file with this version:

import matplotlib.pyplot as plt
import pandas as pd
from sklearn.dummy import DummyClassifier
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    ConfusionMatrixDisplay,
    balanced_accuracy_score,
    classification_report,
)
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

DATA_URL = (
    "https://exoplanetarchive.ipac.caltech.edu/TAP/sync?"
    "query=select+kepoi_name,koi_pdisposition,koi_score,"
    "koi_fpflag_nt,koi_fpflag_ss,koi_fpflag_co,koi_fpflag_ec,"
    "koi_period,koi_impact,koi_duration,koi_depth,koi_prad,koi_teq,"
    "koi_insol,koi_model_snr,koi_steff,koi_slogg,koi_srad,koi_kepmag+"
    "from+q1_q17_dr25_koi+where+koi_pdisposition+=+%27CANDIDATE%27+"
    "or+koi_pdisposition+=+%27FALSE+POSITIVE%27&format=csv"
)

TARGET_COLUMN = "koi_pdisposition"
ID_COLUMN = "kepoi_name"
LABEL_ORDER = ["CANDIDATE", "FALSE POSITIVE"]

SCIENCE_FEATURES = [
    "koi_period",
    "koi_impact",
    "koi_duration",
    "koi_depth",
    "koi_prad",
    "koi_teq",
    "koi_insol",
    "koi_model_snr",
    "koi_steff",
    "koi_slogg",
    "koi_srad",
    "koi_kepmag",
]

LEAKY_FEATURES = [
    "koi_score",
    "koi_fpflag_nt",
    "koi_fpflag_ss",
    "koi_fpflag_co",
    "koi_fpflag_ec",
]


def build_logistic_model():
    return Pipeline(
        steps=[
            ("imputer", SimpleImputer(strategy="median")),
            ("scaler", StandardScaler()),
            ("model", LogisticRegression(max_iter=1000)),
        ]
    )


print("Downloading Kepler Q1-Q17 DR25 data...")
data = pd.read_csv(DATA_URL)

print(f"\nDataset shape: {data.shape}")
print("\nFirst five Kepler Objects of Interest:")
print(data[[ID_COLUMN, TARGET_COLUMN]].head())
print("\nDisposition counts:")
print(data.value_counts(TARGET_COLUMN))

all_features = SCIENCE_FEATURES + LEAKY_FEATURES
X = data[all_features]
y = data[TARGET_COLUMN]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y,
)

baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(X_train[SCIENCE_FEATURES], y_train)
baseline_predictions = baseline.predict(X_test[SCIENCE_FEATURES])
baseline_score = balanced_accuracy_score(y_test, baseline_predictions)
print(f"\nMost-frequent baseline balanced accuracy: {baseline_score:.3f}")

Your baseline is set. Next, you will train your first logistic regression pipeline and compare its result with this floor.

Build a Model That Cheats

Your stratified split and most-frequent baseline now give every model the same held-out test. The next experiment deliberately trains a logistic regression pipeline with every selected column.

A dramatic score can be exciting, but it still needs an audit. This step lets the suspicious result appear before you trace it back to the fields that created it.

In this step, get ready to:
  • Train a logistic regression pipeline on every selected feature.
  • Compare its balanced accuracy with the most-frequent baseline.
  • Audit the disposition-related fields behind the suspicious score.
Train with every selected feature

The existing build_logistic_model() factory returns a fresh pipeline with median imputation. That pipeline also scales features before logistic regression.

  • In exoplanet_classifier.py, find the line that prints the most-frequent baseline score.
  • Add the all-feature experiment below that line by pasting this code:
leaky_model = build_logistic_model()
leaky_model.fit(X_train[all_features], y_train)
leaky_predictions = leaky_model.predict(X_test[all_features])
leaky_score = balanced_accuracy_score(y_test, leaky_predictions)
print(f"Leakage-assisted balanced accuracy: {leaky_score:.3f}")

What does this code do?

  • The build_logistic_model() call creates a fresh model with the same preprocessing sequence each time.
  • The fit() call learns from X_train[all_features] only.
  • The predict() call applies the trained pipeline to the unchanged X_test[all_features] rows.
  • The balanced_accuracy_score() call measures recall across both labels.
  • Save exoplanet_classifier.py.

Before you run this, do you expect every selected field to produce a small improvement or a dramatic jump over the baseline?

  • Test the all-feature model by running this command:
python exoplanet_classifier.py

What should I see?

You should see Most-frequent baseline balanced accuracy before Leakage-assisted balanced accuracy. The second score should jump well above the baseline.

That dramatic improvement is the designed shortfall in this experiment. The model has found a shortcut that makes its evaluation look more convincing than it is.

Script stops before the new score?

If the script stops during the download, confirm that your internet connection can reach the NASA Exoplanet Archive. Run the script again after the connection recovers.

If Python reports a missing name near the new code, check that you pasted it below the existing baseline section with the same identifier spelling.

Help me debug the all-feature model run.

You found the intended red flag: the all-feature model looks far stronger than the baseline.

Audit the suspicious fields

Target leakage occurs when a feature contains information derived from the outcome you want to predict. The model can then reproduce part of the archived decision instead of learning from physical measurements.

Where did the shortcut come from?

The koi_score field expresses confidence in the archived KOI disposition. It already points toward the target label.

The fields koi_fpflag_nt, koi_fpflag_ss, koi_fpflag_co, and koi_fpflag_ec record false-positive vetting flags. Together, these five fields let the model learn from the decision process itself.

  • In exoplanet_classifier.py, locate the leakage-assisted score print at the bottom of the file.
  • Find this line:
print(f"Leakage-assisted balanced accuracy: {leaky_score:.3f}")
  • Replace that line with this score-and-warning block:
print(f"Leakage-assisted balanced accuracy: {leaky_score:.3f}")
print(
    "Leakage warning: koi_score and the four koi_fpflag_* fields "
    "summarize the archived vetting decision."
)

What does the warning add?

The warning records why the high score cannot be trusted as evidence of physical generalization. It names the archived vetting fields responsible for the shortcut.

This print statement does not change the model or its predictions. It makes the leakage audit visible beside the misleading result.

  • Save exoplanet_classifier.py.

Before you rerun the script, do you expect the warning to change the leakage-assisted score?

  • Verify the completed leakage experiment by running this command:
python exoplanet_classifier.py

What should I see?

You should see the most-frequent baseline first. The leakage-assisted balanced accuracy should appear next.

The following warning should name koi_score plus the four koi_fpflag_* fields. This confirms that your script now exposes the shortcut beside its inflated result.

Warning missing from the output?

Check that the warning print starts immediately after the leakage-assisted score print. Keep both quoted strings inside the same print() call.

Confirm that the closing parenthesis aligns with the opening print( line.

Help me fix the leakage warning.

✔️ Awesome, I've got everything!

Great. Double check that you saved exoplanet_classifier.py after adding the leakage-assisted model and warning.

ⓧ I'd like to double check the full code

Your complete exoplanet_classifier.py file should now look like this:

import matplotlib.pyplot as plt
import pandas as pd
from sklearn.dummy import DummyClassifier
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    ConfusionMatrixDisplay,
    balanced_accuracy_score,
    classification_report,
)
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

DATA_URL = (
    "https://exoplanetarchive.ipac.caltech.edu/TAP/sync?"
    "query=select+kepoi_name,koi_pdisposition,koi_score,"
    "koi_fpflag_nt,koi_fpflag_ss,koi_fpflag_co,koi_fpflag_ec,"
    "koi_period,koi_impact,koi_duration,koi_depth,koi_prad,koi_teq,"
    "koi_insol,koi_model_snr,koi_steff,koi_slogg,koi_srad,koi_kepmag+"
    "from+q1_q17_dr25_koi+where+koi_pdisposition+=+%27CANDIDATE%27+"
    "or+koi_pdisposition+=+%27FALSE+POSITIVE%27&format=csv"
)

TARGET_COLUMN = "koi_pdisposition"
ID_COLUMN = "kepoi_name"
LABEL_ORDER = ["CANDIDATE", "FALSE POSITIVE"]

SCIENCE_FEATURES = [
    "koi_period",
    "koi_impact",
    "koi_duration",
    "koi_depth",
    "koi_prad",
    "koi_teq",
    "koi_insol",
    "koi_model_snr",
    "koi_steff",
    "koi_slogg",
    "koi_srad",
    "koi_kepmag",
]

LEAKY_FEATURES = [
    "koi_score",
    "koi_fpflag_nt",
    "koi_fpflag_ss",
    "koi_fpflag_co",
    "koi_fpflag_ec",
]


def build_logistic_model():
    return Pipeline(
        steps=[
            ("imputer", SimpleImputer(strategy="median")),
            ("scaler", StandardScaler()),
            ("model", LogisticRegression(max_iter=1000)),
        ]
    )


print("Downloading Kepler Q1-Q17 DR25 data...")
data = pd.read_csv(DATA_URL)

print(f"\nDataset shape: {data.shape}")
print("\nFirst five Kepler Objects of Interest:")
print(data[[ID_COLUMN, TARGET_COLUMN]].head())
print("\nDisposition counts:")
print(data.value_counts(TARGET_COLUMN))

all_features = SCIENCE_FEATURES + LEAKY_FEATURES
X = data[all_features]
y = data[TARGET_COLUMN]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y,
)

baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(X_train[SCIENCE_FEATURES], y_train)
baseline_predictions = baseline.predict(X_test[SCIENCE_FEATURES])
baseline_score = balanced_accuracy_score(y_test, baseline_predictions)
print(f"\nMost-frequent baseline balanced accuracy: {baseline_score:.3f}")

leaky_model = build_logistic_model()
leaky_model.fit(X_train[all_features], y_train)
leaky_predictions = leaky_model.predict(X_test[all_features])
leaky_score = balanced_accuracy_score(y_test, leaky_predictions)
print(f"Leakage-assisted balanced accuracy: {leaky_score:.3f}")
print(
    "Leakage warning: koi_score and the four koi_fpflag_* fields "
    "summarize the archived vetting decision."
)

Your model has exposed how archived vetting fields can manufacture an impressive score. Next, you will remove that shortcut and evaluate a classifier that learns only from physical measurements.

Train and Evaluate the Honest Classifier

Your leakage-assisted model exposed target leakage by using fields that summarize the archived vetting decision. A trustworthy classifier must generalize from transit measurements plus host-star measurements.

You will evaluate a fresh scikit-learn pipeline on the unchanged held-out rows with balanced accuracy. A classification report will reveal how performance differs between labels.

In this step, get ready to:
  • Train a fresh pipeline using only the twelve physical measurements.
  • Evaluate the honest model on the unchanged held-out rows.
  • Generate a confusion matrix for the held-out predictions.
Train with physical measurements

The honest model receives only the twelve columns in SCIENCE_FEATURES. The fixed split keeps this comparison tied to the same training rows plus the same test rows used by the earlier models.

  • Return to exoplanet_classifier.py in Visual Studio Code.
  • Place your cursor on a blank line below the leakage warning's closing parenthesis.
  • Add the honest training workflow by copying this code:
honest_model = build_logistic_model()
honest_model.fit(X_train[SCIENCE_FEATURES], y_train)
honest_predictions = honest_model.predict(X_test[SCIENCE_FEATURES])
honest_score = balanced_accuracy_score(y_test, honest_predictions)

print(f"Honest balanced accuracy: {honest_score:.3f}")

What does this code do?

  • The honest_model variable holds a fresh preprocessing pipeline with no fitted statistics from the leaky model.
  • The fit() call learns only from the training rows in SCIENCE_FEATURES.
  • The predict() call applies that pipeline to the unchanged test rows.
  • The honest_score variable gives both target classes equal importance in the evaluation.
  • Save exoplanet_classifier.py.
  • Evaluate the honest pipeline in the Command Prompt terminal from earlier by running this command:
python exoplanet_classifier.py

What should I see?

You should see a line beginning with Honest balanced accuracy: after the leakage warning.

You have now forced the classifier to learn from astronomical measurements. This score is your leakage-safe comparison point.

Missing the honest score?

  • Check that the new code sits below the complete leakage warning block.
  • Confirm that every reference uses SCIENCE_FEATURES with the same capitalization.
  • Verify that the activated .venv indicator still appears in your Command Prompt terminal.

Ask for help with the honest model code.

Report class-specific performance

The report separates precision, recall, and F1 score for each class. The held-out examples make those summary metrics concrete.

  • Place your cursor below the honest balanced-accuracy print statement.
  • Add the class report plus ten held-out comparisons by copying this code:
print("\nHonest model classification report:")
print(classification_report(y_test, honest_predictions, digits=3, zero_division=0.0))

print("First ten held-out predictions:")
for number, (actual, predicted) in enumerate(
    zip(y_test, honest_predictions),
    start=1,
):
    print(f"{number:02d}. actual={actual}, predicted={predicted}")
    if number == 10:
        break

How does this evaluation work?

  • The classification_report() call calculates class-specific metrics from y_test plus honest_predictions.
  • The zip() call pairs each actual label with its prediction.
  • The enumerate() call numbers those pairs from 1.
  • The break condition limits the visible sample to ten held-out rows.
  • Save exoplanet_classifier.py.
  • Generate the expanded evaluation in the Command Prompt terminal from earlier by running this command:
python exoplanet_classifier.py

What should I see?

You should see Honest model classification report: followed by a metrics table for CANDIDATE plus FALSE POSITIVE.

Below the report, you should see ten numbered lines containing an actual label plus a predicted label. Your evaluation now shows both summary performance and individual decisions.

Report missing or incomplete?

  • Check that the report code sits below the line that creates honest_predictions.
  • Confirm that zip(y_test, honest_predictions) uses the unchanged test labels plus the honest predictions.
  • Match the indentation of the two statements inside the loop to the completed code.

Ask for help with the evaluation output.

Visualize the classification errors

A confusion matrix arranges actual labels against predicted labels. Its off-diagonal cells show where the classifier confuses one class for the other.

  • Place your cursor below the held-out prediction loop.
  • Create the confusion-matrix image by copying this code:
ConfusionMatrixDisplay.from_predictions(
    y_test,
    honest_predictions,
    labels=LABEL_ORDER,
    display_labels=LABEL_ORDER,
    cmap="Blues",
    values_format="d",
)
plt.title("Honest Kepler Model Confusion Matrix")
plt.tight_layout()
plt.savefig("confusion_matrix.png", dpi=150, bbox_inches="tight")
plt.show()

What does this plot show?

  • The rows represent actual labels from y_test.
  • The columns represent labels from honest_predictions.
  • The LABEL_ORDER value keeps CANDIDATE before FALSE POSITIVE on both axes.
  • The savefig() call writes the image before show() opens the plot window.
  • Save exoplanet_classifier.py.

Before you run the finished script, which off-diagonal cell do you expect to contain more mistakes?

  • Run the complete honest evaluation in the Command Prompt terminal from earlier with this command:
python exoplanet_classifier.py

What should I see?

The terminal should print the honest balanced accuracy, the classification report, and ten held-out predictions.

You should also see the titled confusion-matrix window. The project folder should contain confusion_matrix.png after the plot opens.

  • Inspect the two off-diagonal cells to identify which actual class receives more incorrect predictions.
  • Close the confusion-matrix window.
  • Confirm that confusion_matrix.png appears in Visual Studio Code's Explorer sidebar.

You have completed the honest evaluation workflow. Your classifier now reports its performance through scores, held-out decisions, and a saved error breakdown.

Plot window or image missing?

  • Check that the plotting code appears after honest_predictions is created.
  • Confirm that plt.savefig() appears before plt.show().
  • Look for confusion_matrix.png inside the same project folder as exoplanet_classifier.py.

Ask for help with the confusion matrix.

✔️ Awesome, I've got everything!

Great. Double check that you saved exoplanet_classifier.py and confirmed that confusion_matrix.png exists.

ⓧ I'd like to double check the full code

Here is the complete exoplanet_classifier.py file for comparison:

import matplotlib.pyplot as plt
import pandas as pd
from sklearn.dummy import DummyClassifier
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    ConfusionMatrixDisplay,
    balanced_accuracy_score,
    classification_report,
)
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

DATA_URL = (
    "https://exoplanetarchive.ipac.caltech.edu/TAP/sync?"
    "query=select+kepoi_name,koi_pdisposition,koi_score,"
    "koi_fpflag_nt,koi_fpflag_ss,koi_fpflag_co,koi_fpflag_ec,"
    "koi_period,koi_impact,koi_duration,koi_depth,koi_prad,koi_teq,"
    "koi_insol,koi_model_snr,koi_steff,koi_slogg,koi_srad,koi_kepmag+"
    "from+q1_q17_dr25_koi+where+koi_pdisposition+=+%27CANDIDATE%27+"
    "or+koi_pdisposition+=+%27FALSE+POSITIVE%27&format=csv"
)

TARGET_COLUMN = "koi_pdisposition"
ID_COLUMN = "kepoi_name"
LABEL_ORDER = ["CANDIDATE", "FALSE POSITIVE"]

SCIENCE_FEATURES = [
    "koi_period",
    "koi_impact",
    "koi_duration",
    "koi_depth",
    "koi_prad",
    "koi_teq",
    "koi_insol",
    "koi_model_snr",
    "koi_steff",
    "koi_slogg",
    "koi_srad",
    "koi_kepmag",
]

LEAKY_FEATURES = [
    "koi_score",
    "koi_fpflag_nt",
    "koi_fpflag_ss",
    "koi_fpflag_co",
    "koi_fpflag_ec",
]


def build_logistic_model():
    return Pipeline(
        steps=[
            ("imputer", SimpleImputer(strategy="median")),
            ("scaler", StandardScaler()),
            ("model", LogisticRegression(max_iter=1000)),
        ]
    )


print("Downloading Kepler Q1-Q17 DR25 data...")
data = pd.read_csv(DATA_URL)

print(f"\nDataset shape: {data.shape}")
print("\nFirst five Kepler Objects of Interest:")
print(data[[ID_COLUMN, TARGET_COLUMN]].head())
print("\nDisposition counts:")
print(data.value_counts(TARGET_COLUMN))

all_features = SCIENCE_FEATURES + LEAKY_FEATURES
X = data[all_features]
y = data[TARGET_COLUMN]

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y,
)

baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(X_train[SCIENCE_FEATURES], y_train)
baseline_predictions = baseline.predict(X_test[SCIENCE_FEATURES])
baseline_score = balanced_accuracy_score(y_test, baseline_predictions)
print(f"\nMost-frequent baseline balanced accuracy: {baseline_score:.3f}")

leaky_model = build_logistic_model()
leaky_model.fit(X_train[all_features], y_train)
leaky_predictions = leaky_model.predict(X_test[all_features])
leaky_score = balanced_accuracy_score(y_test, leaky_predictions)
print(f"Leakage-assisted balanced accuracy: {leaky_score:.3f}")
print(
    "Leakage warning: koi_score and the four koi_fpflag_* fields "
    "summarize the archived vetting decision."
)

honest_model = build_logistic_model()
honest_model.fit(X_train[SCIENCE_FEATURES], y_train)
honest_predictions = honest_model.predict(X_test[SCIENCE_FEATURES])
honest_score = balanced_accuracy_score(y_test, honest_predictions)

print(f"Honest balanced accuracy: {honest_score:.3f}")
print("\nHonest model classification report:")
print(classification_report(y_test, honest_predictions, digits=3, zero_division=0.0))

print("First ten held-out predictions:")
for number, (actual, predicted) in enumerate(
    zip(y_test, honest_predictions),
    start=1,
):
    print(f"{number:02d}. actual={actual}, predicted={predicted}")
    if number == 10:
        break

ConfusionMatrixDisplay.from_predictions(
    y_test,
    honest_predictions,
    labels=LABEL_ORDER,
    display_labels=LABEL_ORDER,
    cmap="Blues",
    values_format="d",
)
plt.title("Honest Kepler Model Confusion Matrix")
plt.tight_layout()
plt.savefig("confusion_matrix.png", dpi=150, bbox_inches="tight")
plt.show()

Secret mission

Compare a Random Forest

Your honest logistic regression uses one linear decision boundary. This challenge trains a random forest on the same physical measurements. You will compare both models on the unchanged test set to find which one achieves the higher balanced accuracy.

Clean Up Your Resources

Clean Up Your Resources

Every resource from this project is stored in your local Windows project folder. Keeping these files creates no ongoing charges.

Resources you used:

  • Python virtual environment stored in .venv with the pinned packages.
  • requirements.txt with the reproducible package versions.
  • exoplanet_classifier.py with the complete classifier workflow.
  • confusion_matrix.png with the honest model evaluation.

Keep everything running

No action is needed. Choose this option if you want to test more models or revisit the evaluation later.

  • Keep the local project folder unchanged.
  • Keep .venv so the pinned packages remain available.
  • Keep confusion_matrix.png as a visual record of the honest classifier results.

Pause - I'll come back to this later

Deactivate the current virtual environment to return the terminal to its normal Python context. Every project file remains available at no cost.

  • Deactivate .venv in the existing Command Prompt terminal by running this command:
deactivate

What Does This Command Do?

The command exits the isolated Python environment. It leaves the installed packages and project files on your computer.

The environment prefix disappears from your terminal prompt. Your local project is ready for you to revisit later.

Delete - I don't want to use this again

Choose this option to start fresh by removing every local project resource.

Deleting a finished project can feel drastic. These commands only remove the local folder whose name you enter.

  • Read the current project folder name from the Explorer sidebar in Visual Studio Code.
  • Record that name here: your project folder name.
  • Remove the project folder from the existing Command Prompt terminal by running these commands:
deactivate
cd ..
rmdir /s /q "[[PROJECT_FOLDER="your project folder name"]]"

What Do These Commands Do?

  • The first command releases the active virtual environment.
  • The second command moves the terminal to the parent folder.
  • The final command removes the named project folder with every file inside it.

A successful deletion leaves the terminal in the parent folder. The named project folder is no longer on disk.

The archive data remains online. Your script only read it.

Folder Still Present?

  • Confirm that the recorded folder name matches the Explorer sidebar exactly.
  • Close any program that still has a project file open.
  • Run the deletion commands again from the parent folder.

Help me safely remove my Windows project folder.

Nice Work!

Nice Work!

You did it! You built an astronomy classifier that evaluates held-out Kepler signals using physical measurements.

You've learned how to:

  • Retrieved authentic Kepler data through the NASA Exoplanet Archive. Measured class balance against a most-frequent baseline.
  • Built a scikit-learn pipeline that imputes missing values before scaling measurements. Exposed target leakage by comparing vetting fields with physical features.
  • Evaluated held-out predictions with balanced accuracy. Reviewed class-level performance in a classification report. Saved confusion_matrix.png to inspect prediction errors.
  • Secret Mission: Compared logistic regression with a random forest on the unchanged held-out rows. Identified which model achieved the higher balanced-accuracy result.

Ready to quiz yourself?