Build a Grid Forecast Monitor

Build a Streamlit dashboard that ranks unusual PJM demand forecast misses.

Introduction

30 Second Summary

Teams plan tomorrow's electricity supply before anyone knows exactly how much power people will use. When real demand arrives, hundreds of hourly results can hide the misses worth reviewing.

In this project, you will build GridWatch, a Streamlit app that compares PJM actual electricity demand with day-ahead forecasts from the EIA Open Data API. It will rank unusual forecast misses with Isolation Forest before you publish it through Streamlit Community Cloud.

What You'll Build

Your final demo shows a live PJM forecast audit whose ranked anomalies update with every slider change.

By the end of this project, you'll have:

  • A live demand comparison that updates PJM actuals against day-ahead forecasts for your selected UTC window.
  • A forecast-error dashboard that exposes mean absolute percentage error beside the project-defined 5% alert count.
  • A contextual anomaly ranking that scores unusual hourly misses beside the fixed-baseline result.
  • Secret Mission: Stress-test anomaly shares of 0.01, 0.05, or 0.10 to defend the alert policy you would ship.

Are there any prerequisites?

You'll need Python 3.11 or newer. Basic familiarity with Python plus pandas is enough.

Before We Start

Before the hands-on work begins, this is your moment to commit to GridWatch's purpose. Grid analysts or operations reviewers use its retrospective ranking to prioritize unusual historical forecast misses for human review without treating the results as emergency predictions or causal explanations.

Set Up the Secure Project Environment

GridWatch needs an EIA Open Data API credential before it can retrieve grid data. That credential must stay outside Git history.

Pinned packages keep your local environment reproducible. This step establishes secure access through Streamlit Community Cloud before you make an API request.

In this step, get ready to:
  • Register for an EIA API key.
  • Connect Streamlit Community Cloud to GitHub.
  • Build a local Python environment with pinned dependencies.
Connect your data accounts

The EIA Open Data API uses an API key to associate each request with a registered user. EIA provides this key for free.

Credentials deserve extra care because anyone holding the key can use it. You will store the real key privately while your project keeps a harmless placeholder.

  • Open the EIA API registration page in your browser.
  • Complete the registration form with your email address.
  • Check the registered email account for your EIA API key.
  • Store the EIA API key in your password manager.

Why keep the key separate?

An API key identifies your requests to EIA. Keeping the real value in private storage prevents it from appearing in project files or Git history.

Streamlit Community Cloud hosts Streamlit apps from a GitHub repository. Connecting the accounts opens a GitHub authorization screen before Streamlit can access repositories you permit.

  • Sign in to GitHub in your browser.
  • Open share.streamlit.io in a new browser tab.
  • Click Continue to sign-in.
  • Authenticate with GitHub.
  • Click Connect GitHub account.
  • Click Authorize streamlit on the GitHub authorization page.
  • Return to Streamlit Community Cloud to confirm that your GitHub account shows as connected.

Your data access and deployment account are ready. The real EIA credential now has a safe home outside the project.

Create the project files

A VS Code workspace keeps the GridWatch files together. Opening the folder as a workspace also makes its integrated terminal start inside the correct location.

VS Code may display a Workspace Trust prompt when the new folder opens. You created this empty folder yourself, so you can enable its workspace features.

  • Press Cmd+Space on macOS or the Windows key on Windows to open your search bar.
  • Type VS Code into the search bar to find the app.
  • Press Enter to open VS Code.
  • Select File from the top menu.
  • Select Open Folder....
  • Select New Folder.
  • Enter gridwatch as the folder name.
  • Select Open on macOS or Select Folder on Windows.
  • Select Yes, I trust the authors if the Workspace Trust dialog appears.

You should see gridwatch at the top of the Explorer sidebar. This confirms VS Code opened the correct workspace.

The requirements.txt file pins every dependency to an exact version. These pins prevent package updates from changing the project unexpectedly.

  • Select New File... in the Explorer toolbar.
  • Enter requirements.txt as the file name.

You should see requirements.txt open in a new editor tab.

  • Add the pinned dependencies to requirements.txt by pasting this content:
streamlit==1.66.1
pandas==3.1.0
scikit-learn==1.9.1
requests==2.34.2

What do these dependencies provide?

  • Streamlit renders the interactive GridWatch dashboard.
  • pandas converts the hourly records into chart-ready tables.
  • scikit-learn supplies the anomaly detector used later.
  • Requests retrieves data from the EIA API.
  • Save requirements.txt with Cmd+S on macOS or Ctrl+S on Windows.
  • Confirm that all four dependency pins remain visible in the editor.

Missing a dependency line?

Compare each package name with the block above. Make sure every package occupies its own line.

Ask for help checking the file if the pins still look different. Help me compare my requirements file with the four required package pins.

The .gitignore file tells Git which local files must stay out of future commits. It covers the virtual environment plus the local secrets file.

  • Select New File... in the Explorer toolbar.
  • Enter .gitignore as the file name.

You should see .gitignore listed beneath requirements.txt in the Explorer sidebar.

  • Add the ignore rules to .gitignore by pasting this content:
.venv/
.streamlit/secrets.toml
__pycache__/
.DS_Store

What do these ignore rules protect?

  • The .venv/ rule keeps installed packages out of Git.
  • The .streamlit/secrets.toml rule keeps local credentials out of Git.
  • The __pycache__/ rule excludes generated Python cache files.
  • The .DS_Store rule excludes macOS folder metadata.
  • Save .gitignore.
  • Confirm that the file contains the exact path .streamlit/secrets.toml.

Can't see the .gitignore file?

Check that the file name starts with a period. Confirm that VS Code created it inside the gridwatch folder.

Use this prompt if the file is missing from the Explorer. Help me find or recreate .gitignore inside my gridwatch VS Code workspace.

The local secrets file gives Streamlit a private place for credentials. Its placeholder preserves the project structure without exposing the EIA key you received.

  • Select the gridwatch folder in the Explorer sidebar.
  • Select New Folder in the Explorer toolbar.
  • Enter .streamlit as the folder name.

You should see the .streamlit folder inside gridwatch.

  • Select the .streamlit folder in the Explorer sidebar.
  • Select New File... in the Explorer toolbar.
  • Enter secrets.toml as the file name.
  • Add the placeholder configuration to .streamlit/secrets.toml by pasting this content:
api_key = "your-api-key-here"

What does this configuration do?

The api_key key matches the credential parameter used by EIA. The placeholder marks where private credential storage belongs.

  • Save .streamlit/secrets.toml.
  • Return to .gitignore to confirm that the same secrets path is listed there.

Is the secrets file in the wrong place?

The full path must be .streamlit/secrets.toml inside gridwatch. Move the file into the .streamlit folder if it appears beside that folder.

Use this prompt if the folder tree looks different. Help me place secrets.toml inside the .streamlit folder without exposing my API key.

The final starter file is intentionally empty because application code begins in the next step. Creating it now completes the reproducible project layout.

  • Select the gridwatch folder in the Explorer sidebar.
  • Select New File... in the Explorer toolbar.
  • Enter streamlit_app.py as the file name.
  • Save the empty streamlit_app.py file.

You should now see requirements.txt, .gitignore, .streamlit, and streamlit_app.py in the Explorer sidebar.

✔️ Awesome, I've got everything!

Your GridWatch folder now contains the pinned dependencies, ignore rules, local secret placeholder, and empty application file.

ⓧ I'd like to double check the full code

Your complete requirements.txt file should look like this:

streamlit==1.66.1
pandas==3.1.0
scikit-learn==1.9.1
requests==2.34.2

What should match?

Each dependency must use the exact pinned version shown above. This keeps local installation reproducible.

Your complete .gitignore file should look like this:

.venv/
.streamlit/secrets.toml
__pycache__/
.DS_Store

What should match?

Each ignored path must occupy its own line. The secrets path must include the .streamlit folder.

Your complete .streamlit/secrets.toml file should look like this:

api_key = "your-api-key-here"

What should match?

The file contains the project placeholder for api_key. Keep the real EIA key in private storage.

The streamlit_app.py file should be empty at this point.

Build the Python environment

A virtual environment isolates GridWatch dependencies from other projects on your computer. The pinned packages require Python 3.11 or newer.

  • Open the VS Code integrated terminal by pressing Control+` on macOS or Ctrl+` on Windows.

The terminal starts inside the open gridwatch workspace. This keeps the virtual environment beside your project files.

  • Check the active Python version by running this command:
python --version

What does this command check?

The command prints the version of the Python interpreter your terminal uses. GridWatch needs 3.11 or newer for every pinned package.

✔️ I see Python 3.11 or newer

Your Python version meets the dependency requirement. Continue with the virtual environment.

ⓧ I see an older version

Your installed interpreter is below the minimum required by the pinned packages. Install a current Python release before creating the environment.

  • Open the official Python downloads page in your browser.
  • Choose a macOS installer for Python 3.11 or newer.
  • Run the downloaded installer.
  • Return to the VS Code terminal after installation completes.
  • Repeat the version check above.

ⓧ Command not found

Your terminal cannot locate Python. Install a current release before continuing.

  • Open the official Python downloads page in your browser.
  • Choose a macOS installer for Python 3.11 or newer.
  • Run the downloaded installer.
  • Close the existing VS Code terminal after installation completes.
  • Open a fresh VS Code integrated terminal.
  • Repeat the version check above.

Still seeing an older Python version?

A terminal opened before installation can retain the older command path. Close that terminal before opening a fresh one inside VS Code.

Use this prompt if the older interpreter still appears. Help me find why my VS Code terminal still uses an older Python version on macOS.

  • Create the isolated .venv environment by running this command:
python -m venv .venv

What does this command create?

Python creates a .venv folder containing a project-specific interpreter plus package installation area. The matching .gitignore rule keeps this generated folder out of future commits.

  • Confirm that .venv appears in the Explorer sidebar.

Missing the .venv folder?

Confirm that the terminal is inside the gridwatch workspace. Check the terminal output for a Python installation or permission problem.

Use this prompt to diagnose the failed environment creation. Help me work out why Python did not create .venv inside my gridwatch folder.

  • Activate the new virtual environment by running this command:
source .venv/bin/activate

What does activation change?

Activation makes this terminal use the Python interpreter inside .venv. Package installations now stay isolated inside GridWatch.

You should see .venv near the start of your terminal prompt. This confirms the isolated environment is active.

Don't see .venv in the prompt?

Check that the .venv folder exists in the Explorer sidebar. Confirm that the terminal still points to gridwatch.

Use this prompt if activation still fails. Help me activate the .venv environment in my gridwatch folder on macOS.

  • Install the pinned project dependencies by running this command:
pip install -r requirements.txt

What does this command install?

The command reads every pin from requirements.txt. It installs those versions inside the active virtual environment.

The terminal should finish without a dependency error. You should return to the active .venv prompt.

Did the dependency installation fail?

Confirm that your Python version is 3.11 or newer. Check that every package pin in requirements.txt matches the double-check tab.

Use this prompt for the terminal output you received. Help me diagnose why the pinned GridWatch dependencies failed to install in .venv.

Before you run the final check, picture the visible result that would prove Streamlit can launch an app from this environment.

  • Launch the Streamlit example app by running this command:
streamlit hello

What does this command prove?

Streamlit starts a local web app using the package installed inside .venv. A working example page proves the framework can launch from your environment.

Your browser should open the Streamlit example app. The terminal remains attached to the running local process.

Didn't the example app open?

Confirm that the terminal prompt showed .venv before you launched Streamlit. Check the terminal for a local URL if the browser stayed closed.

Use this prompt if the app still does not load. Help me troubleshoot why streamlit hello does not open the example app from my active .venv.

That confirms the secure foundation works. GridWatch now has isolated dependencies plus a protected location for its EIA credential.

  • Return to the VS Code terminal.
  • Press Control-C to stop the example app.

You should return to the active .venv prompt. Your secure environment is ready for live PJM data in the next step.

Put Live Grid Forecasts on Screen

Your secure GridWatch environment is ready. The next job is to give the dashboard live grid data from the U.S. Energy Information Administration Open Data API.

Actual demand becomes meaningful when viewers can compare it with the day-ahead forecast. This first chart gives GridWatch a useful result before alert logic arrives.

In this step, get ready to:
  • Build a secure app shell for GridWatch.
  • Fetch hourly PJM demand data from EIA.
  • Display complete demand and forecast pairs in a live chart.
Build the app shell

A Streamlit app reruns its Python script to render the current interface. Your first version sets the page identity and gives viewers control over the UTC lookback window.

  • Select the existing .streamlit/secrets.toml file in the VS Code Explorer sidebar.
  • Replace your-api-key-here with the EIA API key sent to your registered email address.
  • Keep the key inside the existing quotation marks.
  • Save .streamlit/secrets.toml.
  • Select the empty streamlit_app.py file in the VS Code Explorer sidebar.
  • Create the GridWatch app shell by adding this code:
import pandas as pd
import requests
import streamlit as st

EIA_URL = "https://api.eia.gov/v2/electricity/rto/region-data/data/"
RESPONDENT = "PJM"
MAX_ROWS = 2880

st.set_page_config(page_title="GridWatch", layout="wide")
st.title("GridWatch: PJM Forecast Reliability Monitor")
st.write(
    "Audit recent EIA-930 day-ahead demand forecasts and rank unusual "
    "historical misses. Times are shown in UTC."
)

lookback_days = st.slider("Lookback days", 14, 45, 30)

What Does This Code Do?

  • The imports load the libraries used for tabular data, web requests, and the app interface.
  • The constants identify the EIA route and limit the request to PJM data.
  • The page settings create a wide GridWatch layout.
  • The lookback_days slider lets viewers select between 14 and 45 days.
  • Save streamlit_app.py.
  • Launch the first GridWatch view from the activated VS Code terminal by running this command:
streamlit run streamlit_app.py

What Should I See?

Your browser shows the GridWatch title and a lookback slider set to 30 days. This confirms the app shell runs inside your activated environment.

Does the App Fail to Open?

  • Confirm that the VS Code terminal still shows your activated .venv environment.
  • Check that the terminal is inside the existing gridwatch folder.
  • Compare the quotation marks and parentheses in streamlit_app.py with the code above.

Ask for help with the local launch error.

Fetch and reshape EIA data

The EIA REST API returns separate rows for actual demand and day-ahead demand forecasts. GridWatch needs to group rows that share the same UTC period before it can compare the two values.

  • Keep the Streamlit process running in your VS Code terminal.
  • Add the cached request function below the lookback_days slider in streamlit_app.py by pasting this first part:
@st.cache_data(ttl=3600)
def load_eia_data(api_key):
    params = {
        "api_key": api_key,
        "frequency": "hourly",
        "data[0]": "value",
        "facets[respondent][]": RESPONDENT,
        "facets[type][]": ["D", "DF"],
        "sort[0][column]": "period",
        "sort[0][direction]": "desc",
        "offset": 0,
        "length": MAX_ROWS,
    }

    response = requests.get(EIA_URL, params=params, timeout=30)
    response.raise_for_status()
    rows = response.json()["response"]["data"]

How Does the Request Work?

  • The cache keeps a successful result for 3,600 seconds before requesting fresh data.
  • The parameters request hourly D and DF values for PJM.
  • The request uses a 30-second timeout so a stalled connection does not hold the app indefinitely.
  • The response check stops processing when EIA returns an unsuccessful status.
  • Complete load_eia_data() by adding this indented code directly below the rows assignment:
    hourly = {}
    for row in rows:
        value = row.get("value")
        if value in (None, ""):
            continue

        period = row["period"]
        if period not in hourly:
            hourly[period] = {"period": period}
        hourly[period][row["type"]] = float(value)

    complete_records = []
    for item in hourly.values():
        if "D" not in item or "DF" not in item:
            continue

        demand = item["D"]
        forecast = item["DF"]

        complete_records.append(
            {
                "period": item["period"],
                "demand_mwh": demand,
                "forecast_mwh": forecast,
            }
        )

    return sorted(complete_records, key=lambda record: record["period"])

How Does the Reshaping Work?

  • The hourly dictionary groups rows by their UTC period.
  • Rows without a usable value are skipped.
  • A record reaches complete_records only when it contains both demand types.
  • The sorted result gives the chart chronological records with demand_mwh and forecast_mwh values.
  • Save streamlit_app.py.
  • Refresh the running GridWatch page in your browser.

You should still see the title and lookback slider without a Python error. This confirms the cached loader parses successfully before the interface calls it.

Do You See a Python Error?

  • Check that every line in the second snippet remains indented inside load_eia_data().
  • Confirm that the closing braces and parentheses match the snippets above.
  • Make sure the second snippet sits directly below the rows assignment.

Ask for help checking the loader structure.

Display the live forecast chart

The loader now returns complete hourly pairs. The interface can safely read the ignored secret, select the requested window, and convert those records into a pandas DataFrame for charting.

  • Add the interface logic below the load_eia_data() function in streamlit_app.py by pasting this code:
if "api_key" not in st.secrets:
    st.write(
        "Add api_key to .streamlit/secrets.toml locally or to the "
        "Community Cloud Secrets field."
    )
else:
    all_records = load_eia_data(st.secrets["api_key"])
    records = all_records[-lookback_days * 24 :]

    if not records:
        st.write("No complete demand and forecast pairs were returned.")
    else:
        chart_data = pd.DataFrame(records)

        st.write("Actual demand compared with day-ahead forecast")
        st.line_chart(
            chart_data,
            x="period",
            y=["demand_mwh", "forecast_mwh"],
            y_label="Megawatthours",
            height=350,
        )

How Does the Interface Stay Secure?

  • The secret check provides guidance when the local key is missing.
  • The st.secrets["api_key"] lookup passes the credential directly into the loader without displaying it.
  • The lookback calculation keeps the newest 24 records for each selected day.
  • The DataFrame supplies the UTC period and both demand series to the line chart.
  • Save streamlit_app.py.
  • Stop the running example with Control-C in the VS Code terminal.

Before you run the final check, which two series do you expect the chart to show? Hold your answer in mind.

  • Restart GridWatch from the activated environment by running this command:
streamlit run streamlit_app.py

What Should I See?

Your browser shows two PJM lines labeled demand_mwh and forecast_mwh for the selected UTC lookback window.

Moving the lookback slider changes how many recent hourly pairs appear. The API key stays inside the ignored secrets file.

Is the Live Chart Missing?

  • Confirm that .streamlit/secrets.toml contains your real EIA key inside quotation marks.
  • Check that facets[type][] contains both D and DF.
  • Verify that your internet connection can reach the EIA API.

Ask for help tracing the missing data.

✔️ Awesome, I've got everything!

Your live GridWatch chart is working. Double check that you saved streamlit_app.py and can change the visible UTC lookback window.

ⓧ I'd like to double check the full code

Compare your current streamlit_app.py file with this complete version:

import pandas as pd
import requests
import streamlit as st

EIA_URL = "https://api.eia.gov/v2/electricity/rto/region-data/data/"
RESPONDENT = "PJM"
MAX_ROWS = 2880

st.set_page_config(page_title="GridWatch", layout="wide")
st.title("GridWatch: PJM Forecast Reliability Monitor")
st.write(
    "Audit recent EIA-930 day-ahead demand forecasts and rank unusual "
    "historical misses. Times are shown in UTC."
)

lookback_days = st.slider("Lookback days", 14, 45, 30)


@st.cache_data(ttl=3600)
def load_eia_data(api_key):
    params = {
        "api_key": api_key,
        "frequency": "hourly",
        "data[0]": "value",
        "facets[respondent][]": RESPONDENT,
        "facets[type][]": ["D", "DF"],
        "sort[0][column]": "period",
        "sort[0][direction]": "desc",
        "offset": 0,
        "length": MAX_ROWS,
    }

    response = requests.get(EIA_URL, params=params, timeout=30)
    response.raise_for_status()
    rows = response.json()["response"]["data"]

    hourly = {}
    for row in rows:
        value = row.get("value")
        if value in (None, ""):
            continue

        period = row["period"]
        if period not in hourly:
            hourly[period] = {"period": period}
        hourly[period][row["type"]] = float(value)

    complete_records = []
    for item in hourly.values():
        if "D" not in item or "DF" not in item:
            continue

        demand = item["D"]
        forecast = item["DF"]

        complete_records.append(
            {
                "period": item["period"],
                "demand_mwh": demand,
                "forecast_mwh": forecast,
            }
        )

    return sorted(complete_records, key=lambda record: record["period"])


if "api_key" not in st.secrets:
    st.write(
        "Add api_key to .streamlit/secrets.toml locally or to the "
        "Community Cloud Secrets field."
    )
else:
    all_records = load_eia_data(st.secrets["api_key"])
    records = all_records[-lookback_days * 24 :]

    if not records:
        st.write("No complete demand and forecast pairs were returned.")
    else:
        chart_data = pd.DataFrame(records)

        st.write("Actual demand compared with day-ahead forecast")
        st.line_chart(
            chart_data,
            x="period",
            y=["demand_mwh", "forecast_mwh"],
            y_label="Megawatthours",
            height=350,
        )

Use this file as a comparison reference. Keep your real API key only in the ignored .streamlit/secrets.toml file.

That is GridWatch's first live result. Your app now turns separate EIA rows into a chart that another viewer can understand.

Your live forecast view is ready. Next, you will measure each forecast miss and test what happens when one fixed alert rule judges every hour.

Expose Fixed-Threshold Forecast Errors

Your Streamlit chart shows where actual demand separates from the day-ahead forecast. However, the chart does not quantify each hourly miss.

A fixed-threshold baseline turns each miss into a consistent comparison against the project-defined 5% cutoff. You will expose that baseline across different lookback windows to test how much context one cutoff provides.

In this step, get ready to:
  • Calculate hourly forecast errors.
  • Apply the fixed 5% alert baseline.
  • Display summary statistics with an error timeline.
Calculate hourly forecast errors

Signed error preserves whether actual demand landed above or below the forecast. Absolute error measures the size of the miss without its direction.

Percentage error makes misses comparable across hours with different demand levels. UTC hour plus weekday preserves the context that the fixed cutoff leaves unused.

  • Switch back to streamlit_app.py in VS Code.
  • Find the import section followed by the three EIA constants.
  • Use this code as the reference for the current section:
import pandas as pd
import requests
import streamlit as st

EIA_URL = "https://api.eia.gov/v2/electricity/rto/region-data/data/"
RESPONDENT = "PJM"
MAX_ROWS = 2880

What is this reference showing?

This section currently imports the libraries used by the live chart. It also stores the EIA endpoint plus the request limits.

  • Replace that section with the updated imports plus the fixed-alert constant:
import datetime as dt

import pandas as pd
import requests
import streamlit as st

EIA_URL = "https://api.eia.gov/v2/electricity/rto/region-data/data/"
RESPONDENT = "PJM"
MAX_ROWS = 2880
FIXED_ALERT_PERCENT = 5.0

What does this update add?

  • The dt alias provides date parsing for the UTC weekday field.
  • The FIXED_ALERT_PERCENT constant keeps the project-defined cutoff in one place.
  • The value 5.0 represents a 5% absolute forecast error.
  • Find the complete_records loop inside load_eia_data().
  • Replace that loop through its return statement with this expanded record builder:
    complete_records = []
    for item in hourly.values():
        if "D" not in item or "DF" not in item:
            continue

        demand = item["D"]
        forecast = item["DF"]
        error = demand - forecast
        abs_error = abs(error)
        abs_error_pct = (abs_error / demand * 100.0) if demand else 0.0
        utc_date = dt.date.fromisoformat(item["period"][:10])

        complete_records.append(
            {
                "period": item["period"],
                "demand_mwh": demand,
                "forecast_mwh": forecast,
                "error_mwh": error,
                "abs_error_mwh": abs_error,
                "abs_error_pct": abs_error_pct,
                "hour_utc": int(item["period"][11:13]),
                "weekday_utc": utc_date.weekday(),
                "fixed_alert": abs_error_pct >= FIXED_ALERT_PERCENT,
            }
        )

    return sorted(complete_records, key=lambda record: record["period"])

What does this code calculate?

  • The error_mwh field stores actual demand minus forecast demand.
  • The abs_error_mwh field stores the size of that difference.
  • The abs_error_pct field expresses the miss relative to actual demand.
  • The hour_utc field plus the weekday_utc field preserve time context for every record.
  • The fixed_alert field becomes true when the absolute percentage error reaches the cutoff.
  • Save streamlit_app.py.
  • Switch back to the VS Code terminal with the active .venv environment.
  • Start the updated local app by running this command:
streamlit run streamlit_app.py

What does this command do?

This command runs streamlit_app.py as the local GridWatch app. The terminal remains occupied while the app is running.

Good work. Your expanded records are valid because the browser still shows the two PJM lines labeled demand_mwh and forecast_mwh.

Does the app stop while loading records?

Check that import datetime as dt appears above the other imports. Confirm that every new dictionary field remains inside the braces passed to complete_records.append().

Ask for help with the exact traceback if the app still stops. Help me troubleshoot the new GridWatch forecast-error record fields.

Show the fixed-alert summary

Individual error fields become useful when the dashboard summarizes the selected sample. The mean shows typical percentage error while the fixed-alert count shows how many hours crossed the cutoff.

  • Return to streamlit_app.py in VS Code.
  • Find the chart_data assignment inside the branch that handles returned records.
  • Insert the summary calculation immediately above that assignment by adding this code:
        errors = [record["abs_error_pct"] for record in records]
        fixed_alert_count = sum(
            record["fixed_alert"] for record in records
        )
        mean_abs_error_pct = sum(errors) / len(errors)

        st.write(
            f"Analyzed {len(records):,} complete hourly pairs. "
            f"Mean absolute percentage error: {mean_abs_error_pct:.2f}%. "
            f"Fixed 5% alerts: {fixed_alert_count:,}. "
        )

How is the summary built?

  • The errors list collects the percentage miss from each selected record.
  • The fixed_alert_count value counts records whose alert field is true.
  • The mean_abs_error_pct value averages the selected percentage errors.
  • The st.write() call displays those results above the charts.
  • Save streamlit_app.py.
  • Refresh the running GridWatch app in your browser.

You now see the number of complete hourly pairs. You also see the mean absolute percentage error plus the fixed 5% alert count.

Is the summary missing?

Confirm that the summary code uses the same indentation as the chart_data assignment. Code placed outside the records branch cannot use the local records list.

Use the terminal traceback to locate any misspelled field. Help me fix the missing GridWatch summary statistics.

Plot and test the baseline

A timeline reveals when forecast misses grow across the selected window. Pairing it with the summary makes the fixed cutoff visible as an alert policy that you can challenge.

  • Find the existing line chart that uses demand_mwh plus forecast_mwh.
  • Add the error timeline immediately below that chart by inserting this code:
        st.write("Absolute forecast error percentage")
        st.line_chart(
            chart_data,
            x="period",
            y="abs_error_pct",
            y_label="Absolute percentage error",
            height=300,
        )

What does this chart reveal?

The chart plots abs_error_pct against each UTC period. Peaks identify hours with larger relative misses.

The fixed-alert count reduces those peaks to a binary result at 5%. The timeline keeps the differences between their magnitudes visible.

  • Save streamlit_app.py.

Before you refresh, predict whether a fixed percentage cutoff communicates which flagged hour is most unusual.

  • Refresh the running GridWatch app in your browser.

You should see the absolute forecast error percentage timeline below the demand comparison chart. Its peaks line up with the hourly misses summarized above.

Is the error timeline empty?

Check that the chart uses chart_data after the DataFrame is created. Confirm that the y field is spelled abs_error_pct.

Ask for help if the chart area still has no data. Help me debug the GridWatch absolute error timeline.

Before you change the window, predict whether the fixed cutoff will communicate the same apparent severity across both samples.

  • Set the Lookback days slider to 14.

You see the mean percentage error plus the number of fixed alerts for the shorter sample.

  • Set the Lookback days slider to 45.

The summary changes with the larger sample. Every threshold breach still receives the same binary flag regardless of UTC hour or weekday.

That shortfall is intentional. The fixed baseline provides no ranking among flagged hours because it uses neither time field.

✔️ Awesome, I've got everything!

Your fixed-threshold dashboard is working. Double check that you saved streamlit_app.py.

ⓧ I'd like to double check the full code

Compare your complete streamlit_app.py file with this version:

import datetime as dt

import pandas as pd
import requests
import streamlit as st

EIA_URL = "https://api.eia.gov/v2/electricity/rto/region-data/data/"
RESPONDENT = "PJM"
MAX_ROWS = 2880
FIXED_ALERT_PERCENT = 5.0

st.set_page_config(page_title="GridWatch", layout="wide")
st.title("GridWatch: PJM Forecast Reliability Monitor")
st.write(
    "Audit recent EIA-930 day-ahead demand forecasts and rank unusual "
    "historical misses. Times are shown in UTC."
)

lookback_days = st.slider("Lookback days", 14, 45, 30)


@st.cache_data(ttl=3600)
def load_eia_data(api_key):
    params = {
        "api_key": api_key,
        "frequency": "hourly",
        "data[0]": "value",
        "facets[respondent][]": RESPONDENT,
        "facets[type][]": ["D", "DF"],
        "sort[0][column]": "period",
        "sort[0][direction]": "desc",
        "offset": 0,
        "length": MAX_ROWS,
    }

    response = requests.get(EIA_URL, params=params, timeout=30)
    response.raise_for_status()
    rows = response.json()["response"]["data"]

    hourly = {}
    for row in rows:
        value = row.get("value")
        if value in (None, ""):
            continue

        period = row["period"]
        if period not in hourly:
            hourly[period] = {"period": period}
        hourly[period][row["type"]] = float(value)

    complete_records = []
    for item in hourly.values():
        if "D" not in item or "DF" not in item:
            continue

        demand = item["D"]
        forecast = item["DF"]
        error = demand - forecast
        abs_error = abs(error)
        abs_error_pct = (abs_error / demand * 100.0) if demand else 0.0
        utc_date = dt.date.fromisoformat(item["period"][:10])

        complete_records.append(
            {
                "period": item["period"],
                "demand_mwh": demand,
                "forecast_mwh": forecast,
                "error_mwh": error,
                "abs_error_mwh": abs_error,
                "abs_error_pct": abs_error_pct,
                "hour_utc": int(item["period"][11:13]),
                "weekday_utc": utc_date.weekday(),
                "fixed_alert": abs_error_pct >= FIXED_ALERT_PERCENT,
            }
        )

    return sorted(complete_records, key=lambda record: record["period"])


if "api_key" not in st.secrets:
    st.write(
        "Add api_key to .streamlit/secrets.toml locally or to the "
        "Community Cloud Secrets field."
    )
else:
    all_records = load_eia_data(st.secrets["api_key"])
    records = all_records[-lookback_days * 24 :]

    if not records:
        st.write("No complete demand and forecast pairs were returned.")
    else:
        errors = [record["abs_error_pct"] for record in records]
        fixed_alert_count = sum(
            record["fixed_alert"] for record in records
        )
        mean_abs_error_pct = sum(errors) / len(errors)

        st.write(
            f"Analyzed {len(records):,} complete hourly pairs. "
            f"Mean absolute percentage error: {mean_abs_error_pct:.2f}%. "
            f"Fixed 5% alerts: {fixed_alert_count:,}. "
        )

        chart_data = pd.DataFrame(records)

        st.write("Actual demand compared with day-ahead forecast")
        st.line_chart(
            chart_data,
            x="period",
            y=["demand_mwh", "forecast_mwh"],
            y_label="Megawatthours",
            height=350,
        )

        st.write("Absolute forecast error percentage")
        st.line_chart(
            chart_data,
            x="period",
            y="abs_error_pct",
            y_label="Absolute percentage error",
            height=300,
        )

What should this file produce?

This version keeps the EIA key inside st.secrets. It displays the demand comparison plus the fixed-alert summary and percentage-error timeline.

Your dashboard now quantifies every hourly miss and exposes the limitations of one global cutoff. Next, you will use Isolation Forest to rank unusual combinations of forecast size, error magnitude, UTC hour, and weekday.

Rank Contextual Misses with Isolation Forest

Your Streamlit dashboard now shows where the fixed 5% baseline falls short. The baseline ignores the hour and weekday surrounding each forecast miss.

An Isolation Forest scores unusual feature combinations within the selected sample. GridWatch can use those scores to place the most abnormal historical misses first.

In this step, get ready to:
  • Build feature vectors that describe each forecast miss in context.
  • Fit an Isolation Forest model for anomaly labels and scores.
  • Display a ranked comparison against the fixed 5% baseline.
Add contextual model inputs

The anomaly-share setting controls how much of the selected sample the detector labels as unusual. The default value of 0.05 represents an alert-volume policy of roughly 5%.

Why Isolation Forest?

Isolation Forest can label the fitted records and score their relative abnormality. This fits a dashboard that ranks the same historical sample it analyzes.

Local Outlier Factor adds novelty-mode rules for scoring unseen samples. GridWatch keeps the workflow focused on contextual ranking.

  • In streamlit_app.py, find this import line:
import streamlit as st

What are you locating?

This line imports Streamlit under the st alias. The model import belongs directly beneath it with the other external libraries.

  • Replace that import line with this import group:
import streamlit as st
from sklearn.ensemble import IsolationForest

What does this import add?

The second line makes the IsolationForest estimator available in GridWatch. The existing Streamlit import remains unchanged.

  • Find the existing lookback control:
lookback_days = st.slider("Lookback days", 14, 45, 30)

What does this line identify?

This line creates the existing lookback control. The new anomaly-share control belongs directly beneath it so both analysis settings stay together.

  • Replace that line with the two slider definitions below:
lookback_days = st.slider("Lookback days", 14, 45, 30)
contamination = st.slider(
    "Expected anomaly share",
    min_value=0.01,
    max_value=0.10,
    value=0.05,
    step=0.01,
    format="%0.2f",
)

How does the new control work?

  • The contamination value becomes the model's expected anomaly share.
  • The slider accepts settings from 0.01 through 0.10.
  • The starting value of 0.05 provides a consistent setting for the final check.
  • Save streamlit_app.py.
  • Return to the running GridWatch page in your browser.

You should see an Expected anomaly share slider beneath the lookback control. Its initial value should be 0.05.

Missing the anomaly-share slider?

Confirm that the second slider sits directly beneath lookback_days. Check that streamlit_app.py is saved.

If the page reports an import problem, confirm that the running app still uses the activated .venv environment. Help me diagnose why the Isolation Forest import or anomaly-share slider is not loading.

Each feature vector describes one hourly forecast miss from five angles. Forecast scale and error magnitude capture size. UTC hour and weekday add time context.

  • Inside the final else: block, place this feature-vector code immediately before errors = [record["abs_error_pct"] for record in records]:
        features = [
            [
                record["forecast_mwh"],
                record["abs_error_mwh"],
                record["abs_error_pct"],
                record["hour_utc"],
                record["weekday_utc"],
            ]
            for record in records
        ]

How do these features add context?

  • The forecast_mwh value captures the scale of the forecast.
  • The abs_error_mwh value captures the physical size of the miss.
  • The abs_error_pct value captures the relative size of the miss.
  • The hour_utc and weekday_utc values place the miss in its time context.
  • Save streamlit_app.py.
  • Return to the running GridWatch page.

The app should reload with both sliders and the existing charts visible. This confirms that every selected record supplies all five model inputs.

Does feature construction stop the app?

Check that the feature block is indented inside the final else: branch. Confirm that every feature name matches the keys stored in each hourly record.

Compare the square brackets around the inner record list with the code above. Help me find the indentation or bracket problem in my GridWatch feature-vector code.

Fit and score the detector

The model returns -1 for records classified as outliers. Its decision_function gives lower values to more abnormal samples.

GridWatch negates each decision score. Larger displayed values therefore represent more abnormal forecast misses.

  • Place the model-fitting code directly after the features list:
        model = IsolationForest(
            n_estimators=200,
            contamination=contamination,
            random_state=42,
        )
        labels = model.fit_predict(features)
        decision_scores = model.decision_function(features)

        for record, label, decision_score in zip(
            records, labels, decision_scores
        ):
            record["is_anomaly"] = label == -1
            record["anomaly_score"] = -float(decision_score)

How does the detector score each hour?

  • The model builds 200 isolation trees for the selected records.
  • The contamination value sets the score boundary used for anomaly labels.
  • The random_state=42 setting makes repeated fits reproducible for the same input records.
  • The scoring loop stores is_anomaly and anomaly_score on every record.
  • Replace the existing metrics block with this expanded version:
        errors = [record["abs_error_pct"] for record in records]
        anomaly_records = [
            record for record in records if record["is_anomaly"]
        ]
        fixed_alert_count = sum(
            record["fixed_alert"] for record in records
        )
        mean_abs_error_pct = sum(errors) / len(errors)

        st.write(
            f"Analyzed {len(records):,} complete hourly pairs. "
            f"Mean absolute percentage error: {mean_abs_error_pct:.2f}%. "
            f"Fixed 5% alerts: {fixed_alert_count:,}. "
            f"Isolation Forest alerts: {len(anomaly_records):,}."
        )

What changes in the summary?

  • The anomaly_records list keeps the records labeled as anomalies.
  • The summary reports model alerts beside fixed 5% alerts.
  • The two counts provide a direct comparison between the contextual model and the global cutoff.
  • Save streamlit_app.py.
  • Return to the running GridWatch page.

You should see an Isolation Forest alerts count in the summary. Changing the anomaly-share slider should change the number of model alerts.

Missing the model alert count?

Confirm that anomaly_records is created after the scoring loop. Check that the summary uses len(anomaly_records).

If the model reports a contamination problem, confirm that the slider remains between 0.01 and 0.10. Help me diagnose why GridWatch is not displaying the Isolation Forest alert count.

Display the ranked comparison

An anomaly label identifies which records cross the selected score boundary. A descending ranking shows which flagged hours deserve attention first.

  • Place this ranking block directly after the absolute forecast error chart:
        ranked_anomalies = sorted(
            anomaly_records,
            key=lambda record: record["anomaly_score"],
            reverse=True,
        )
        display_rows = [
            {
                "UTC period": record["period"],
                "Demand MWh": round(record["demand_mwh"], 1),
                "Forecast MWh": round(record["forecast_mwh"], 1),
                "Error MWh": round(record["error_mwh"], 1),
                "Absolute error %": round(record["abs_error_pct"], 2),
                "Anomaly score": round(record["anomaly_score"], 4),
                "Fixed 5% alert": record["fixed_alert"],
            }
            for record in ranked_anomalies
        ]

        st.write("Ranked contextual anomalies")
        st.dataframe(
            pd.DataFrame(display_rows),
            hide_index=True,
            width="stretch",
        )

How does the table support review?

  • The descending sort places the largest anomaly score at the top.
  • The demand and forecast columns preserve the scale behind each miss.
  • The Fixed 5% alert column shows where the model agrees with the baseline.
  • The table keeps the UTC period visible for each investigation.

Unsupervised scores indicate statistical unusualness within the current sample. They do not reveal why a miss occurred.

  • Add this limitation note directly beneath the anomaly table:
        st.write(
            "Model limitation: this unsupervised detector ranks unusual "
            "historical forecast misses. It does not identify causes, "
            "predict future misses, or classify grid emergencies."
        )

Why make the limitation visible?

The detector reviews historical forecast misses without labeled ground truth. The note frames its output as evidence for human investigation.

This boundary prevents the ranking from being presented as a causal diagnosis or an emergency classifier.

  • Save streamlit_app.py.
  • Return to the running GridWatch page.

You should see the ranked anomaly table beneath the error timeline. The model-limitation note should appear directly below the table.

Missing the ranked table?

Check that the ranking block uses the same indentation as the chart blocks. Confirm that ranked_anomalies sorts anomaly_records.

Check that the limitation note appears after the closing parenthesis for st.dataframe. Help me diagnose why the ranked contextual anomalies table is missing.

Before the final check, which forecast misses do you expect to receive the largest anomaly scores at an anomaly share of 0.05?

  • Switch back to the VS Code terminal panel that is running GridWatch.
  • Stop the current app with Control-C.
  • Restart the completed app by running this command:
streamlit run streamlit_app.py

What does this command verify?

The command loads the saved app from the activated environment. It runs API ingestion, feature construction, model scoring, and dashboard rendering as one workflow.

  • Return to the browser tab opened by the running app.
  • Set Expected anomaly share to 0.05.
  • Scroll to the ranked contextual anomalies table.

You should see ranked rows with UTC period, demand, forecast, error, absolute error percentage, anomaly score, and the fixed 5% comparison. The largest displayed anomaly score should appear first.

Does the final model check fail?

Confirm that the terminal still shows the activated .venv environment. Use the first reported file line to locate the failing section.

If the API data loads before the failure, compare the feature list, scoring loop, metrics block, and ranking block with the reference below. Help me debug the completed GridWatch Isolation Forest workflow.

That is the contextual ranking working. GridWatch now prioritizes unusual historical misses while preserving the fixed baseline for comparison.

✔️ Awesome, I've got everything!

Great. Your saved streamlit_app.py now displays a ranked anomaly table with its limitation note.

ⓧ I'd like to double check the full code

Compare your completed streamlit_app.py with this reference:

import datetime as dt

import pandas as pd
import requests
import streamlit as st
from sklearn.ensemble import IsolationForest

EIA_URL = "https://api.eia.gov/v2/electricity/rto/region-data/data/"
RESPONDENT = "PJM"
MAX_ROWS = 2880
FIXED_ALERT_PERCENT = 5.0

st.set_page_config(page_title="GridWatch", layout="wide")
st.title("GridWatch: PJM Forecast Reliability Monitor")
st.write(
    "Audit recent EIA-930 day-ahead demand forecasts and rank unusual "
    "historical misses. Times are shown in UTC."
)

lookback_days = st.slider("Lookback days", 14, 45, 30)
contamination = st.slider(
    "Expected anomaly share",
    min_value=0.01,
    max_value=0.10,
    value=0.05,
    step=0.01,
    format="%0.2f",
)


@st.cache_data(ttl=3600)
def load_eia_data(api_key):
    params = {
        "api_key": api_key,
        "frequency": "hourly",
        "data[0]": "value",
        "facets[respondent][]": RESPONDENT,
        "facets[type][]": ["D", "DF"],
        "sort[0][column]": "period",
        "sort[0][direction]": "desc",
        "offset": 0,
        "length": MAX_ROWS,
    }

    response = requests.get(EIA_URL, params=params, timeout=30)
    response.raise_for_status()
    rows = response.json()["response"]["data"]

    hourly = {}
    for row in rows:
        value = row.get("value")
        if value in (None, ""):
            continue

        period = row["period"]
        if period not in hourly:
            hourly[period] = {"period": period}
        hourly[period][row["type"]] = float(value)

    complete_records = []
    for item in hourly.values():
        if "D" not in item or "DF" not in item:
            continue

        demand = item["D"]
        forecast = item["DF"]
        error = demand - forecast
        abs_error = abs(error)
        abs_error_pct = (abs_error / demand * 100.0) if demand else 0.0
        utc_date = dt.date.fromisoformat(item["period"][:10])

        complete_records.append(
            {
                "period": item["period"],
                "demand_mwh": demand,
                "forecast_mwh": forecast,
                "error_mwh": error,
                "abs_error_mwh": abs_error,
                "abs_error_pct": abs_error_pct,
                "hour_utc": int(item["period"][11:13]),
                "weekday_utc": utc_date.weekday(),
                "fixed_alert": abs_error_pct >= FIXED_ALERT_PERCENT,
            }
        )

    return sorted(complete_records, key=lambda record: record["period"])


if "api_key" not in st.secrets:
    st.write(
        "Add api_key to .streamlit/secrets.toml locally or to the "
        "Community Cloud Secrets field."
    )
else:
    all_records = load_eia_data(st.secrets["api_key"])
    records = all_records[-lookback_days * 24 :]

    if not records:
        st.write("No complete demand and forecast pairs were returned.")
    else:
        features = [
            [
                record["forecast_mwh"],
                record["abs_error_mwh"],
                record["abs_error_pct"],
                record["hour_utc"],
                record["weekday_utc"],
            ]
            for record in records
        ]

        model = IsolationForest(
            n_estimators=200,
            contamination=contamination,
            random_state=42,
        )
        labels = model.fit_predict(features)
        decision_scores = model.decision_function(features)

        for record, label, decision_score in zip(
            records, labels, decision_scores
        ):
            record["is_anomaly"] = label == -1
            record["anomaly_score"] = -float(decision_score)

        errors = [record["abs_error_pct"] for record in records]
        anomaly_records = [
            record for record in records if record["is_anomaly"]
        ]
        fixed_alert_count = sum(
            record["fixed_alert"] for record in records
        )
        mean_abs_error_pct = sum(errors) / len(errors)

        st.write(
            f"Analyzed {len(records):,} complete hourly pairs. "
            f"Mean absolute percentage error: {mean_abs_error_pct:.2f}%. "
            f"Fixed 5% alerts: {fixed_alert_count:,}. "
            f"Isolation Forest alerts: {len(anomaly_records):,}."
        )

        chart_data = pd.DataFrame(records)

        st.write("Actual demand compared with day-ahead forecast")
        st.line_chart(
            chart_data,
            x="period",
            y=["demand_mwh", "forecast_mwh"],
            y_label="Megawatthours",
            height=350,
        )

        st.write("Absolute forecast error percentage")
        st.line_chart(
            chart_data,
            x="period",
            y="abs_error_pct",
            y_label="Absolute percentage error",
            height=300,
        )

        ranked_anomalies = sorted(
            anomaly_records,
            key=lambda record: record["anomaly_score"],
            reverse=True,
        )
        display_rows = [
            {
                "UTC period": record["period"],
                "Demand MWh": round(record["demand_mwh"], 1),
                "Forecast MWh": round(record["forecast_mwh"], 1),
                "Error MWh": round(record["error_mwh"], 1),
                "Absolute error %": round(record["abs_error_pct"], 2),
                "Anomaly score": round(record["anomaly_score"], 4),
                "Fixed 5% alert": record["fixed_alert"],
            }
            for record in ranked_anomalies
        ]

        st.write("Ranked contextual anomalies")
        st.dataframe(
            pd.DataFrame(display_rows),
            hide_index=True,
            width="stretch",
        )

        st.write(
            "Model limitation: this unsupervised detector ranks unusual "
            "historical forecast misses. It does not identify causes, "
            "predict future misses, or classify grid emergencies."
        )

How to use this reference

Check the order and indentation of each section against your saved file. Keep the local API key in the ignored secrets file.

Your local reliability monitor is complete. Next up, you will publish GridWatch so another person can test the model without receiving your API key.

Publish GridWatch

Your local GridWatch dashboard now ranks contextual forecast misses against the fixed alert baseline. Publishing it lets another person run the analysis from a browser.

A public GitHub repository stores the application code. Streamlit Community Cloud runs that code at a shareable URL.

The key for the EIA Open Data API needs a separate deployment path. Community Cloud Secrets supplies the credential without adding it to version control.

In this step, get ready to:
  • Publish the GridWatch code to a public repository.
  • Configure the Community Cloud deployment.
  • Test the deployed app without exposing the EIA API key.
Create the public GitHub repository

A Git repository records the exact files used by your deployment. Streamlit Community Cloud reads those files whenever it builds GridWatch.

A public repository exposes every tracked file. Your ignored secrets file stays local because .streamlit/secrets.toml is already listed in .gitignore.

  • Open GitHub's new repository page in your browser.
  • Enter gridwatch in the Repository name field.
  • Choose Public as the repository visibility.
  • Leave every optional initialization choice unselected.
  • Click Create repository.
  • Copy the remote repository URL from Quick Setup.
  • Record the copied URL here: your GridWatch repository URL.
  • Switch back to the VS Code terminal running GridWatch.
  • Stop the local Streamlit process by pressing Control-C.

You'll return to the terminal prompt inside the gridwatch folder.

  • Switch to the .gitignore tab in VS Code.
  • Confirm that .streamlit/secrets.toml remains listed in the file.
  • Create the first local commit by running these commands in the terminal:
git init -b main
git add .
git commit -m "First commit"

What do these commands do?

  • The first command creates Git history on the main branch.
  • The second command stages every project file that is not ignored.
  • The third command records the staged files as your first snapshot.

You'll see Git create the first commit. The summary should include requirements.txt, .gitignore, and streamlit_app.py.

Unable to create the commit?

  • Confirm that the terminal prompt shows the gridwatch folder.
  • Complete Git's identity setup if the terminal asks who created the commit.

Ask for help with creating the first GridWatch commit.

Your first push may open a browser for GitHub authentication. Complete the browser prompt if it appears.

  • Publish the local repository by copying these commands into the terminal after replacing REMOTE-URL with your GridWatch repository URL:
git remote add origin REMOTE-URL
git push -u origin main

How does the push work?

  • The first command connects your local folder to the public repository URL.
  • The second command uploads the main branch.
  • The upstream link lets future pushes target the same branch.

The push finishes when the terminal returns to its prompt. Your local code is now connected to the public repository.

Push not reaching GitHub?

  • Confirm that you replaced REMOTE-URL with the copied repository URL.
  • Complete the GitHub authentication flow if your browser opened during the push.
  • Confirm that the GitHub repository was created without starter files.

Ask for help with publishing the main branch.

  • Refresh the gridwatch repository page in your browser.
  • Confirm that the default branch is main.
  • Confirm that the repository lists requirements.txt, .gitignore, and streamlit_app.py.
  • Confirm that .streamlit/secrets.toml does not appear in the repository.

That security boundary is working. GitHub has the complete application code without the local credential file.

Deploy the app from Community Cloud

Community Cloud needs three coordinates for the deployment. These are the repository, the branch, and the application entrypoint.

You'll see the deployment choices for connecting an existing application.

  • Click Yup, I have an app.
  • Select the public gridwatch repository.

The deployment form now points at the source repository you just published.

  • Select main as the branch.
  • Enter streamlit_app.py as the entrypoint.

The form now targets the exact branch and Python file that run GridWatch.

Add the cloud secret and test the app

This security handoff is a little fiddly because the key moves between two private surfaces. The value belongs in Community Cloud Secrets.

  • Open Advanced settings.

You'll see the Secrets field used for deployment-only configuration.

  • Prepare the secret configuration by copying this template:
api_key = "your-real-key"

How does the deployed secret work?

  • The api_key name matches the value that st.secrets reads inside GridWatch.
  • The your-real-key text is a temporary placeholder for your EIA API key.
  • Community Cloud supplies the saved value when the deployed app runs.
  • Paste the template into the Secrets field.
  • Replace your-real-key in the field with your EIA API key.

The deployment now has the credential that GridWatch expects. The key remains outside the GitHub repository.

  • Click Save.
  • Click Deploy.

A quiet build screen for a few minutes is normal while the first cloud build installs the pinned dependencies.

  • Record the assigned public URL here: your GridWatch streamlit.app URL.

Before you open the public URL, do you expect the app to load the chart without access to your local secrets file?

  • Open a private browser window.
  • Navigate to your GridWatch streamlit.app URL.

You'll see the GridWatch title, the PJM demand chart, the forecast-error timeline, and the ranked anomaly table.

  • Move the Lookback days slider to a different value.

You'll see the analysis rerun for the new UTC lookback window.

  • Move the Expected anomaly share slider to a different value.

You'll see the anomaly count change. The ranked table refreshes with the new model calibration.

  • Switch back to the browser tab showing the gridwatch repository.
  • Confirm again that .streamlit/secrets.toml is absent from the file list.
  • Open .gitignore from the repository file list.
  • Confirm that the file still excludes .streamlit/secrets.toml.

You made it. GridWatch now runs at a public URL while its EIA credential stays outside GitHub.

Deployed app not loading the data?

  • Confirm that the deployment uses the main branch.
  • Confirm that the entrypoint is streamlit_app.py.
  • Confirm that the saved secret uses the exact key name api_key.

Ask for help with debugging the GridWatch deployment.

Secret mission

Stress-Test Your Alert Policy

Test GridWatch with anomaly shares of 0.01, 0.05, and 0.10. Compare alert volume with the fixed 5% baseline. Then defend the setting you would ship in a written calibration note.

Clean Up Your Resources

Clean Up Your Resources

The EIA API key is free. Streamlit Community Cloud hosts this project for free. Decide whether to keep your resources running, pause local work, or delete them entirely.

Resources you used:

  • Local files: the gridwatch folder containing requirements.txt, .gitignore, streamlit_app.py, and the ignored .streamlit/secrets.toml file.
  • Public repository: the gridwatch repository on GitHub.
  • Cloud deployment: the deployed GridWatch app with api_key stored in its Community Cloud secret.

Keep everything running

No action is needed. Choose this if you want GridWatch to remain available for demonstrations or further calibration tests.

  • Keep the local gridwatch folder for future development.
  • Leave the public gridwatch repository available for reviewing the app code.
  • Leave the deployed GridWatch app online at its public streamlit.app URL.
  • Keep the EIA key inside the ignored local secrets file and the Community Cloud secret.

Pause - I'll come back to this later

Pause here means stopping only the local Streamlit process. Your deployed app and public repository stay available.

  • Switch back to the terminal running the local Streamlit app if that process is still active.
  • Press Control-C in that terminal.
  • Confirm the terminal returns to its command prompt.

Delete - I don't want to use this again

Deletion permanently removes the public app, repository, and local project folder. Any calibration note stored only inside gridwatch disappears with the folder.

  • Save your calibration note outside the local gridwatch folder if you want to keep it.
  • Return to Streamlit Community Cloud.
  • Open the management page for your deployed GridWatch app.
  • Use the app deletion control to remove the GridWatch deployment.
  • Open the former public streamlit.app URL to confirm GridWatch no longer loads.
  • Return to the public gridwatch repository on GitHub.
  • Use the repository deletion controls to permanently delete gridwatch.
  • Open the former repository URL to confirm the public repository no longer loads.

The online copies are now gone. Finish by removing the local folder containing the ignored secrets file.

  • Press Cmd+Space to open macOS search.
  • Type Finder.
  • Press Enter to open Finder.
  • Locate the local gridwatch folder you created during setup.
  • Move the gridwatch folder to Trash.
  • Confirm the folder no longer appears in its previous Finder location.
  • Review the Trash for unrelated files you still need.
  • Empty the Trash to remove gridwatch permanently.

Removing the local folder also removes its .streamlit/secrets.toml file and local copy of the EIA key.

Nice Work!

Nice Work!

You did it! GridWatch is now a deployed forecast-reliability monitor that securely audits PJM day-ahead demand forecasts.

You've learned how to:

  • Build a secure data-ingestion path from the U.S. Energy Information Administration Open Data API for complete hourly PJM demand and day-ahead forecast pairs.
  • Compare a fixed 5% alert baseline with an Isolation Forest ranking that prioritizes contextual forecast misses.
  • Deploy a public energy-data app from GitHub to Streamlit Community Cloud with the EIA API key protected outside version control.
  • Secret Mission: Compare anomaly shares at 0.01 / 0.05 / 0.10. Defend a shipping policy through its alert-volume trade-off plus the detector's retrospective limitation.

Ready to quiz yourself?