Build a Multi-Model Chat Room

Build a Streamlit chat app that switches AI models without losing context.

Introduction

30 Second Summary

A useful conversation loses momentum when switching to another AI assistant means repeating the whole story. Comparing answers becomes easier when the conversation travels with you.

In this project, you will build a browser chat room that sends real prompts through KodeKey. The conversation keeps its context when you select another model.

What You'll Build

Your browser chat carries a fact from one selected model to the next inside the same visible thread.

By the end of this project, you'll have:

  • A live model picker that shows the models currently returned by KodeKey.
  • A cross-model conversation that recalls earlier facts after the selected model changes.
  • Conversation controls for in-page model errors plus a New chat reset.
  • Secret Mission: Add a System prompt box that changes how every selected model responds.

Are there any prerequisites?

You need existing KodeKey access plus basic Python syntax. Your Linux device also needs Python 3.10.12 plus VS Code.

The guide covers the virtual environment plus pinned package setup for Streamlit 1.63.0 plus the OpenAI Python SDK 3.13.0.

Before We Start

This is your moment to commit to a KodeKey chat app that keeps one conversation coherent as you switch models. Naming that goal now gives every later setup choice a clear purpose.

Set Up the Local KodeKey Workspace

Your chat app needs a predictable local foundation before it can send prompts through KodeKey. Shared Python packages can drift between projects.

A Python virtual environment keeps this project's packages isolated. Pinned versions make the setup reproducible.

A shell-only environment variable keeps your KodeKey credential out of the project files. Your active Bash shell provides the key when the app starts.

In this step, get ready to:
  • Prepare an activated Python environment inside private-chat.
  • Install Streamlit 1.63.0 plus the OpenAI Python SDK 3.13.0.
  • Export KODEKEY_API_KEY in the active shell.
Create and activate the workspace

The private-chat folder gives every project file one known home. Its .venv directory keeps installed packages separate from your system Python.

  • Open a Bash terminal from your Linux application menu.
  • Move to your Desktop by running this command:
cd ~/Desktop

What does this command do?

The cd command changes the terminal's current location to ~/Desktop. Your new project folder will be easy to find there.

  • Create private-chat and enter it by running these commands:
mkdir private-chat
cd private-chat
pwd

What do these commands do?

  • The mkdir command creates the private-chat directory.
  • The cd command moves your shell into that directory.
  • The pwd command prints your current location so you can confirm the move.

You should see a path ending in /Desktop/private-chat.

  • Create .venv and activate it by running these commands:
python3 -m venv .venv
source .venv/bin/activate
python3 --version

Why use a virtual environment?

  • The venv module creates a private package location inside .venv.
  • Activation routes Python package installations into that private location.
  • The final python3 check confirms which interpreter the environment uses.

You should see Python 3.10.12. Your terminal also shows that the environment is active.

Virtual environment not activating?

Confirm that your current path ends with /Desktop/private-chat. Creating the environment from another folder places .venv somewhere else.

If environment creation reports a missing module, your Linux Python installation may need its virtual-environment support package.

Help me diagnose my Linux virtual environment setup.

Pin and install the dependencies

The requirements.txt file records the exact package versions this project expects. Anyone rebuilding the workspace can install the same dependency set.

  • Open the current private-chat folder in VS Code by running this command:
code .

What does this command do?

The code command starts VS Code. The . tells it to use your current folder as the workspace.

You should see private-chat at the top of the VS Code Explorer sidebar.

VS Code did not open?

The VS Code command-line tool may be missing from your Linux path. Open VS Code from your application menu instead.

Use File followed by Open Folder. Select the private-chat folder on your Desktop.

Help me open my current Linux folder in VS Code.

  • Select the new-file icon beside private-chat in the Explorer sidebar.
  • Type requirements.txt as the file name.
  • Press Enter to create the file.
  • Populate requirements.txt by pasting this content:
streamlit==1.63.0
openai==3.13.0

What does this file control?

  • The first line pins Streamlit to 1.63.0 for the browser interface.
  • The second line pins the OpenAI Python SDK to 3.13.0 for the KodeKey-compatible client.
  • Exact pins prevent a later package release from changing the project underneath you.
  • Save requirements.txt by pressing Ctrl+S.
  • Check the Explorer sidebar for requirements.txt.

The file should appear directly inside private-chat.

Requirements file in the wrong place?

Make sure requirements.txt appears beside .venv in the private-chat workspace. Move it into that folder if it appears inside .venv.

Help me place requirements.txt in the correct VS Code folder.

✔️ Awesome, I've got everything!

Great. Double check that you saved requirements.txt inside private-chat.

ⓧ I'd like to double check the full code

Compare your requirements.txt file with this complete version.

streamlit==1.63.0
openai==3.13.0

The package download can take a minute on the first run. A stream of installation progress means the command is working.

  • Switch back to the activated Bash terminal.
  • Install the pinned dependencies from requirements.txt by running this command:
python -m pip install -r requirements.txt

What does this command do?

Running pip through the active Python interpreter sends each installation into .venv.

The requirements file supplies both package names plus their exact versions. This keeps the dependency set reproducible.

The terminal should return to its prompt without an installation failure. Streamlit plus the OpenAI Python SDK are now available inside .venv.

Package installation failed?

Confirm that the environment remains active before retrying. Also check that both lines in requirements.txt match the full-code tab exactly.

A network or package-index problem can interrupt the download. Retry after confirming your internet connection.

Help me diagnose the dependency installation.

Export the API key and verify the setup

Your KodeKey API key authorizes real model requests. Keeping it in the active shell prevents the credential from entering requirements.txt or future source files.

Credentials deserve extra care. The command below uses a visible placeholder so your real key never appears in this guide.

  • Return to the KodeKey key area you used when you received access.
  • Generate a KodeKey API key if no reusable key is available.
  • Copy your KodeKey API key.
  • Replace your-key-here in the command below with the copied key.
  • Export the key in the Bash shell that will start Streamlit by running:
export KODEKEY_API_KEY="your-key-here"

What does this command do?

The export command makes KODEKEY_API_KEY available to programs started from this shell. The value stays out of your project files.

A new terminal does not inherit this active shell's value. Start the Streamlit app from this same terminal.

Unsure whether the key was copied correctly?

Repeat the export with a fresh copy of the key. Keep the quotation marks around the value.

Never paste the real key into an AI help request. Describe the failure without including the credential.

Help me review the safe export process.

Before you run the final check, consider which three version numbers the activated environment should report.

  • Verify Python plus both installed packages by running these commands:
python3 --version
python -c "import streamlit, openai; print(streamlit.__version__, openai.__version__)"

What does this check prove?

The first command confirms the Python interpreter available inside the active environment. The second command imports both project packages.

Printing the package versions proves that the pinned installations are available to the same Python environment.

You should see Python 3.10.12 on the first line. The next line should show 1.63.0 3.13.0.

That is the workspace ready. Your isolated Python environment now has the exact packages the chat app needs.

  • Scroll the terminal until the API key export command is outside the visible area.

Your local KodeKey workspace is prepared. Next, you will turn it into a browser chat that loads live models and returns a real answer.

Build a One-Turn Multi-Model Chat

Your isolated workspace is ready. Its active Bash shell already holds the credential that KodeKey needs.

Now your chat room needs its first real answer through Streamlit. The final check tests what a follow-up can recover when only the newest prompt reaches the API.

In this step, get ready to:
  • Connect the local app to KodeKey.
  • Populate a model selector from KodeKey's current catalog.
  • Send one-turn prompts through the selected model.
Create the KodeKey client

An OpenAI-compatible gateway lets one client talk to KodeKey's model catalog. The app reads the API key from the active shell so the credential stays out of chat_app.py.

  • Use your Linux app launcher to start VS Code.
  • Open the private-chat folder with VS Code's folder picker.

The Explorer sidebar shows requirements.txt inside private-chat.

  • Create chat_app.py inside private-chat by using the Explorer sidebar to add a new file with this code:
import os

import streamlit as st
from openai import OpenAI

st.title("KodeKey Multi-Model Chat Room")
st.caption("Switch models without restarting the conversation.")

api_key = os.getenv("KODEKEY_API_KEY")
if not api_key:
    st.error("Set KODEKEY_API_KEY in the shell that starts Streamlit.")
    st.stop()

client = OpenAI(
    api_key=api_key,
    base_url="https://api.ai.kodekloud.com/v1",
)

What does this code do?

  • The imports provide access to the shell environment plus the app framework plus the API client.
  • The title and caption give the browser page a clear identity.
  • The key check stops the app when the active shell does not contain the required credential.
  • The OpenAI client points requests at KodeKey's gateway through the configured base_url.
  • Save chat_app.py.
  • Return to the active Bash shell from Step 1.

The server stays attached to this terminal while it runs. Press Ctrl+C when you need to stop it.

  • Start the local app by running this command:
streamlit run chat_app.py

This command starts the local Streamlit server from chat_app.py.

  • Open the printed http://localhost:8501 URL in your browser.

You'll see the KodeKey Multi-Model Chat Room title above its caption. That first page proves your client configuration loads without putting the API key in the file.

Does the app stop at startup?

Confirm that KODEKEY_API_KEY is still exported in the Bash shell that started Streamlit.

Confirm that .venv remains activated in the same shell.

Help me diagnose the Streamlit startup problem.

Load the live model catalog

Dynamic model discovery keeps the selector aligned with the catalog returned by KodeKey. Streamlit's cached function avoids reloading that catalog on every script rerun.

  • Return to chat_app.py in VS Code.
  • Add the model discovery code below the client block:
@st.cache_data(ttl=3600)
def load_models():
    return sorted(model.id for model in client.models.list().data)


models = load_models()
model = st.sidebar.selectbox("Model", models)

How does model discovery work?

  • The load_models() function requests the current catalog from the client.
  • Each model.id becomes a selectable model name.
  • The sorted list gives the sidebar selector a consistent order.
  • The model variable holds the currently selected model ID.
  • Save chat_app.py.
  • Return to the browser tab running the local app.

You'll see a Model selector in the sidebar. Its choices come from KodeKey's current catalog.

Don't see the model selector?

Confirm that the new code sits below the closing parenthesis of the client block.

Check that your network can reach KodeKey from the device running Streamlit.

Help me troubleshoot why the KodeKey model selector does not load.

Send one-turn prompts

Chat Completions receives a list of messages for each request. This one-turn version builds that list from the newest prompt only.

  • Return to the bottom of chat_app.py in VS Code.
  • Add the chat input plus completion request below the model selector:
if prompt := st.chat_input("Ask anything"):
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        reply = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
        )
        answer = reply.choices[0].message.content or "The model returned no text."
        st.markdown(answer)

What happens after submission?

  • The st.chat_input widget supplies the submitted text as prompt.
  • The user container displays the submitted prompt in the page.
  • The completion request sends the selected model plus one user message.
  • The assistant container renders the returned answer.
  • Save chat_app.py.
  • Return to the browser tab running the app.
  • Choose an allowed model from the Model selector.
  • Enter Remember this code: ORBIT-731 in the chat input.
  • Press Enter to submit the prompt.

You'll see your prompt followed by a real assistant answer. Your browser chat room is now sending live requests through the selected KodeKey model.

Didn't get an answer?

Choose another model from the selector if the current subscription cannot call your first choice.

Confirm that the Streamlit terminal is still running in the activated .venv environment.

Help me troubleshoot why my KodeKey prompt does not return an answer.

Before the follow-up, do you think this one-turn request can recover the code from your previous prompt?

  • Enter What code did I give you? in the chat input.
  • Press Enter to submit the follow-up.

The earlier exchange disappears when the script reruns. The follow-up cannot recover ORBIT-731 because its request contains only the newest prompt.

That shortfall is intentional. You have proved that separate API requests do not create conversation memory on their own.

✔️ Awesome, I've got everything!

Great work. Double check that you saved chat_app.py while the local Streamlit server remains active.

ⓧ I'd like to double check the full code

Compare your complete chat_app.py file with this version.

import os

import streamlit as st
from openai import OpenAI

st.title("KodeKey Multi-Model Chat Room")
st.caption("Switch models without restarting the conversation.")

api_key = os.getenv("KODEKEY_API_KEY")
if not api_key:
    st.error("Set KODEKEY_API_KEY in the shell that starts Streamlit.")
    st.stop()

client = OpenAI(
    api_key=api_key,
    base_url="https://api.ai.kodekloud.com/v1",
)


@st.cache_data(ttl=3600)
def load_models():
    return sorted(model.id for model in client.models.list().data)


models = load_models()
model = st.sidebar.selectbox("Model", models)

if prompt := st.chat_input("Ask anything"):
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        reply = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
        )
        answer = reply.choices[0].message.content or "The model returned no text."
        st.markdown(answer)

Your app can discover models plus return live answers. Next, you'll preserve the full conversation so later requests can use earlier turns.

Give the Conversation a Memory

Your one-turn app already sends real prompts through KodeKey. However, each submission replaces the visible exchange and leaves the next request without earlier context.

Each call behaves like a stateless API request that only knows the messages you send with it. Streamlit Session State keeps a message list across reruns so the app can rebuild the conversation.

In this step, get ready to:
  • Initialize a conversation history in Streamlit Session State.
  • Redraw every stored user turn and assistant turn.
  • Send the complete conversation history with each new prompt.
Initialize and redraw the conversation

Streamlit reruns chat_app.py whenever you submit a prompt. The message list must exist before the input handler runs so every rerun can redraw the stored conversation.

  • Return to chat_app.py in VS Code.
  • Find the line model = st.sidebar.selectbox("Model", models).
  • Add the following code directly below that line:
if "messages" not in st.session_state:
    st.session_state.messages = []

for turn in st.session_state.messages:
    with st.chat_message(turn["role"]):
        st.markdown(turn["content"])

What Does This Code Do?

  • The first condition creates st.session_state.messages only when the browser session does not already have it.
  • The loop reads every stored turn in its original order.
  • Each st.chat_message container uses the stored role to redraw the correct chat bubble.
  • Save chat_app.py.
  • Return to the browser tab from earlier.

You should see the app reload with its title, model selector, and chat input ready. The empty message list now survives reruns without breaking the page.

Seeing an Error After the Rerun?

Check that the new block sits above if prompt := st.chat_input("Ask anything"):. Make sure the lines inside the condition and loop use four spaces of indentation.

Compare turn["role"] and turn["content"] character for character with the snippet.

Help me fix the Session State or indentation error in my Streamlit message loop.

Store and resend every turn

The API receives only the messages included in the current request. The input handler needs to store the user prompt before building a complete history for the selected model.

  • Find the line if prompt := st.chat_input("Ask anything"): in chat_app.py.
  • Select the block from that line through st.markdown(answer).
  • Replace the selected block with the following code:
if prompt := st.chat_input("Ask anything"):
    st.session_state.messages.append({"role": "user", "content": prompt})

    with st.chat_message("user"):
        st.markdown(prompt)

    history = [
        {"role": turn["role"], "content": turn["content"]}
        for turn in st.session_state.messages
    ]

    with st.chat_message("assistant"):
        reply = client.chat.completions.create(
            model=model,
            messages=history,
        )
        answer = reply.choices[0].message.content or "The model returned no text."
        st.markdown(answer)

    st.session_state.messages.append(
        {"role": "assistant", "content": answer}
    )

How Does Memory Reach the Model?

  • The first append() stores the current user prompt before the request begins.
  • The history list copies every stored role and its content into the request format.
  • The completion call sends history so the selected model receives the full thread.
  • The final append() stores the assistant answer for the next rerun.
  • Save chat_app.py.
  • Return to the local browser app from earlier.

You should see the chat input ready for a two-turn memory test. The model selector remains available above it.

Prove the chat remembers

A successful memory test needs one turn that supplies a unique fact and a later turn that asks for it. The second request proves whether the stored history reached the model.

Before you test it, do you expect the second request to include the code from the first turn?

  • Keep the working model from earlier selected.
  • Enter Remember this code: ORBIT-731 in the chat input.
  • Press Enter.
  • Enter What code did I give you? in the chat input.
  • Press Enter.

You should see both user prompts and both assistant answers remain visible. The final answer should include ORBIT-731.

That closes the memory gap. Your selected model now receives the full thread after every Streamlit rerun.

Does the Chat Still Forget?

Check that the completion call uses messages=history. A one-item message list gives the model only the latest prompt.

If earlier bubbles disappear, confirm the redraw loop sits above the chat input. Check that the assistant answer is appended after its chat container closes.

Help me find why my Streamlit chat still forgets earlier turns.

✔️ Awesome, I've got everything!

Great. Double check that you saved chat_app.py after the memory test passed.

ⓧ I'd like to double check the full code

Compare your complete chat_app.py file with this working version.

import os

import streamlit as st
from openai import OpenAI

st.title("KodeKey Multi-Model Chat Room")
st.caption("Switch models without restarting the conversation.")

api_key = os.getenv("KODEKEY_API_KEY")
if not api_key:
    st.error("Set KODEKEY_API_KEY in the shell that starts Streamlit.")
    st.stop()

client = OpenAI(
    api_key=api_key,
    base_url="https://api.ai.kodekloud.com/v1",
)


@st.cache_data(ttl=3600)
def load_models():
    return sorted(model.id for model in client.models.list().data)


models = load_models()
model = st.sidebar.selectbox("Model", models)

if "messages" not in st.session_state:
    st.session_state.messages = []

for turn in st.session_state.messages:
    with st.chat_message(turn["role"]):
        st.markdown(turn["content"])

if prompt := st.chat_input("Ask anything"):
    st.session_state.messages.append({"role": "user", "content": prompt})

    with st.chat_message("user"):
        st.markdown(prompt)

    history = [
        {"role": turn["role"], "content": turn["content"]}
        for turn in st.session_state.messages
    ]

    with st.chat_message("assistant"):
        reply = client.chat.completions.create(
            model=model,
            messages=history,
        )
        answer = reply.choices[0].message.content or "The model returned no text."
        st.markdown(answer)

    st.session_state.messages.append(
        {"role": "assistant", "content": answer}
    )

Your selected model now remembers the complete conversation. Next, you will switch models inside this same thread without losing that shared context.

Switch Models Without Losing Context

Your Streamlit chat now resends the full conversation. One model can remember earlier turns.

Memory inside one model is ordinary chat. Preserving the same thread after a model switch demonstrates KodeKey's unified gateway pattern.

In this step, get ready to:
  • Store the selected model with each assistant reply.
  • Keep display metadata out of the API history.
  • Prove that context survives a model switch.
Store the model with each reply

Each stored assistant turn now needs the model that produced it. The existing history comprehension strips that display metadata before each request.

The messages from your previous test use the older dictionary shape. Stopping the app clears the temporary chat while leaving your code untouched.

  • Switch back to the Bash terminal from earlier.
  • Stop the running app by pressing Ctrl+C.
  • In VS Code, return to chat_app.py.
  • Find the with st.chat_message("assistant"): block near the bottom of the file.
  • Select that block through the final st.session_state.messages.append() call.
  • Replace the selected code by copying the following block:
with st.chat_message("assistant"):
    reply = client.chat.completions.create(
        model=model,
        messages=history,
    )
    answer = reply.choices[0].message.content or "The model returned no text."
    st.markdown(answer)
    st.caption(f"Answered by `{model}`")

st.session_state.messages.append(
    {
        "role": "assistant",
        "content": answer,
        "model": model,
    }
)

What does this code do?

  • The model=model argument sends the next request to the current sidebar selection.
  • The first st.caption() call labels the answer shown during the current request.
  • The assistant dictionary stores model as display metadata for later reruns.
  • The existing history comprehension sends only role plus content to the API.
  • Save chat_app.py.
  • Return to the Bash terminal from earlier.
  • Start the updated app by running:
streamlit run chat_app.py

What should I see?

The terminal prints a local URL ending in http://localhost:8501. The server remains active while you test the app.

  • Open http://localhost:8501 in your browser.
  • Select a model your plan can call from the Model dropdown.
  • Enter My deployment code is ORBIT-731 in the chat input.
  • Press Enter to send the prompt.

You will see a real answer with a caption naming the selected model. That model tag now travels with the stored assistant turn.

Missing the model caption?

Check that st.caption(f"Answered by `{model}`") is indented inside the assistant chat container. Confirm that you saved chat_app.py before restarting the app.

If the page stops loading, compare the braces in the assistant dictionary against the code above.

Help me troubleshoot why my new assistant reply does not show its model caption.

Label every assistant reply

The caption you just added covers a newly generated answer. The redraw loop also needs to restore that caption whenever a Streamlit rerun rebuilds the conversation.

  • In chat_app.py, find the for turn in st.session_state.messages: loop.
  • Select the complete redraw loop.
  • Replace the selected loop by copying the following block:
for turn in st.session_state.messages:
    with st.chat_message(turn["role"]):
        st.markdown(turn["content"])
        if turn["role"] == "assistant":
            st.caption(f"Answered by `{turn['model']}`")

How does the redraw work?

The role check limits captions to assistant messages. Each caption reads the stored turn["model"] value from the moment that answer was created.

A later model selection cannot rewrite an earlier answer's label. Every response keeps its original source visible.

  • Save chat_app.py.
  • Return to the browser.
  • Select a different allowed model from the Model dropdown.

Streamlit reruns the script after the selection changes. Your existing answer still shows the model that originally produced it.

Did the earlier caption change?

Check that the redraw caption uses turn['model']. Using the current model value would relabel every stored answer after a dropdown change.

Help me check why my stored assistant captions change when I select another model.

✔️ Awesome, I've got everything!

Great. Double check that you saved chat_app.py in VS Code.

ⓧ I'd like to double check the full code

Compare your complete chat_app.py file against this reference.

import os

import streamlit as st
from openai import OpenAI

st.title("KodeKey Multi-Model Chat Room")
st.caption("Switch models without restarting the conversation.")

api_key = os.getenv("KODEKEY_API_KEY")
if not api_key:
    st.error("Set KODEKEY_API_KEY in the shell that starts Streamlit.")
    st.stop()

client = OpenAI(
    api_key=api_key,
    base_url="https://api.ai.kodekloud.com/v1",
)


@st.cache_data(ttl=3600)
def load_models():
    return sorted(model.id for model in client.models.list().data)


models = load_models()
model = st.sidebar.selectbox("Model", models)

if "messages" not in st.session_state:
    st.session_state.messages = []

for turn in st.session_state.messages:
    with st.chat_message(turn["role"]):
        st.markdown(turn["content"])
        if turn["role"] == "assistant":
            st.caption(f"Answered by `{turn['model']}`")

if prompt := st.chat_input("Ask anything"):
    st.session_state.messages.append({"role": "user", "content": prompt})

    with st.chat_message("user"):
        st.markdown(prompt)

    history = [
        {"role": turn["role"], "content": turn["content"]}
        for turn in st.session_state.messages
    ]

    with st.chat_message("assistant"):
        reply = client.chat.completions.create(
            model=model,
            messages=history,
        )
        answer = reply.choices[0].message.content or "The model returned no text."
        st.markdown(answer)
        st.caption(f"Answered by `{model}`")

    st.session_state.messages.append(
        {
            "role": "assistant",
            "content": answer,
            "model": model,
        }
    )

How do the pieces fit?

The stored model field powers the answer captions. The history comprehension keeps each API message limited to its role plus content.

The sidebar selection controls the next completion. It does not remove or relabel the shared conversation.

Test cross-model continuity

The decisive test is whether the newly selected model receives the earlier deployment code. Your shared history still contains every user message plus every assistant answer.

Before you send the follow-up, ask yourself whether the newly selected model can recover the deployment code.

  • Enter Which deployment code did I give you? in the chat input.
  • Press Enter to send the follow-up.

The second model answers with ORBIT-731. Its caption shows the newly selected model ID.

You have proved cross-model continuity. One shared conversation now survives a runtime model switch.

Your conversation now crosses model boundaries without losing context. Next, you will harden it with recoverable errors plus a clean reset.

Handle Unavailable Models and Reset Chats

Your chat room now keeps one thread coherent as you switch KodeKey models. The live catalog can still include a model that your subscription cannot call.

This step turns model failures into in-page feedback. It also adds a New chat control for clearing conversation state.

In this step, get ready to:
  • Show model discovery problems inside the chat page.
  • Preserve earlier messages when a completion request fails.
  • Reset the conversation without restarting the local server.
Protect model discovery

The OpenAI Python SDK exposes OpenAIError for failures reported by the API client. Catching that error lets Streamlit show the problem inside the app.

  • In chat_app.py, find the import that currently brings in OpenAI.
  • Replace that import with this line:
from openai import OpenAI, OpenAIError

What Does This Import Do?

The OpenAIError class gives the app a focused way to catch API client failures.

  • Find models = load_models().
  • Replace that line with the guarded model lookup below:
try:
    models = load_models()
except OpenAIError as error:
    st.error(f"Could not load models: {error}")
    st.stop()

How Does Model Discovery Recover?

  • The try block attempts to load the live model catalog.
  • The except block displays the client error inside the page.
  • The st.stop() call prevents the rest of the rerun from using a missing model list.
  • Save chat_app.py.
  • Return to the browser from earlier.
  • Confirm that the sidebar still shows the live model dropdown.

A successful request can still return an empty catalog. The app needs to stop before it tries to build a dropdown with no choices.

  • Find model = st.sidebar.selectbox("Model", models).
  • Insert this empty-catalog check immediately above that line:
if not models:
    st.error("KodeKey returned no models for this request.")
    st.stop()

What Does This Check Protect?

The condition catches an empty model list. The app displays a clear message before stopping the current rerun.

  • Save chat_app.py.
  • Confirm that the browser reruns successfully with the model dropdown still populated.

Model Dropdown Missing?

  • Confirm that try: begins at the far left of the file.
  • Keep if not models: aligned with the try: line.
  • Check that the original models = load_models() line now sits inside the try block.

Help me fix the model discovery handling in chat_app.py.

Recover from failed completions

Each user prompt enters st.session_state.messages before the completion request starts. A failed request would leave that newest prompt without an answer.

The recovery path removes only that unanswered prompt. Every completed user and assistant turn stays in the shared conversation.

  • In chat_app.py, find the existing reply = client.chat.completions.create(...) call inside the assistant chat container.
  • Replace that completion call with the protected version below:
        try:
            reply = client.chat.completions.create(
                model=model,
                messages=history,
            )
        except OpenAIError as error:
            st.error(f"Request failed: {error}")
            st.session_state.messages.pop()
            st.stop()

How Does the Request Recover?

  • The try block limits error handling to the completion request.
  • The st.error(...) call keeps the failure visible inside the chat page.
  • The pop() call removes the newest unanswered user turn.
  • The st.stop() call ends the failed rerun before an empty assistant turn can be stored.
  • Save chat_app.py.
  • Return to the browser from earlier.

Before you test a model that failed earlier, predict whether its new prompt will remain in the thread.

  • Skip this optional failure check if every model you tried is available.
  • Select a model that previously produced an access error from the Model dropdown.
  • Submit Test model access. in the chat input.

When access fails, you will see the request problem inside the page. The unanswered test prompt disappears while your earlier conversation remains visible.

Page Stops Instead of Showing the Error?

  • Confirm that the except OpenAIError as error: line aligns with try:.
  • Keep answer = reply.choices[0].message.content outside the error handler.
  • Confirm that OpenAIError appears in the import near the top of chat_app.py.

Help me debug the completion error handler.

Add a new chat control

The conversation lives in the messages list for the current browser session. Replacing that list with an empty one clears the visible thread on the button-triggered rerun.

  • Find model = st.sidebar.selectbox("Model", models).
  • Replace that line with this sidebar block:
with st.sidebar:
    model = st.selectbox("Model", models)
    if st.button("New chat"):
        st.session_state.messages = []

What Does the Button Reset?

  • The sidebar container keeps the model selector and reset control together.
  • The New chat button replaces the stored conversation with an empty list.
  • The model catalog remains loaded because the button changes only st.session_state.messages.
  • Save chat_app.py.
  • Return to the browser from earlier.

You will see a New chat button below the model dropdown.

  • Confirm that at least one earlier message is visible in the current thread.

Before you click the reset control, predict which parts of the page will remain available.

  • Click New chat in the sidebar.

The visible conversation clears immediately. The model dropdown and chat input remain ready for another thread.

That completes the recovery loop. Your chat room can now survive unavailable models without sacrificing earlier successful messages.

✔️ Awesome, I've got everything!

Your focused error handling and conversation reset control are in place.

ⓧ I'd like to double check the full code

The complete chat_app.py file is below for comparison.

import os

import streamlit as st
from openai import OpenAI, OpenAIError

st.title("KodeKey Multi-Model Chat Room")
st.caption("Switch models without restarting the conversation.")

api_key = os.getenv("KODEKEY_API_KEY")
if not api_key:
    st.error("Set KODEKEY_API_KEY in the shell that starts Streamlit.")
    st.stop()

client = OpenAI(
    api_key=api_key,
    base_url="https://api.ai.kodekloud.com/v1",
)


@st.cache_data(ttl=3600)
def load_models():
    return sorted(model.id for model in client.models.list().data)


try:
    models = load_models()
except OpenAIError as error:
    st.error(f"Could not load models: {error}")
    st.stop()

if not models:
    st.error("KodeKey returned no models for this request.")
    st.stop()

with st.sidebar:
    model = st.selectbox("Model", models)
    if st.button("New chat"):
        st.session_state.messages = []

if "messages" not in st.session_state:
    st.session_state.messages = []

for turn in st.session_state.messages:
    with st.chat_message(turn["role"]):
        st.markdown(turn["content"])
        if turn["role"] == "assistant":
            st.caption(f"Answered by `{turn['model']}`")

if prompt := st.chat_input("Ask anything"):
    st.session_state.messages.append({"role": "user", "content": prompt})

    with st.chat_message("user"):
        st.markdown(prompt)

    history = [
        {"role": turn["role"], "content": turn["content"]}
        for turn in st.session_state.messages
    ]

    with st.chat_message("assistant"):
        try:
            reply = client.chat.completions.create(
                model=model,
                messages=history,
            )
        except OpenAIError as error:
            st.error(f"Request failed: {error}")
            st.session_state.messages.pop()
            st.stop()

        answer = reply.choices[0].message.content or "The model returned no text."
        st.markdown(answer)
        st.caption(f"Answered by `{model}`")

    st.session_state.messages.append(
        {
            "role": "assistant",
            "content": answer,
            "model": model,
        }
    )

Secret mission

Add a System Prompt Studio

Give the same model a question under two different behavior instructions. You will add a system prompt studio that changes each response without adding the instruction to the visible conversation.

Clean Up Your Resources

Clean Up Your Resources

Your local Streamlit files have no ongoing cost. Decide whether to keep the app running, pause it for later, or delete the project.

Cost warning

The local Streamlit server is free to run. Every real prompt consumes the shared KodeKey allowance attached to your subscription.

Keeping the browser app open does not consume allowance by itself. Allowance is used when the app sends a model request.

Resources you used:

  • The local Streamlit server that holds the current in-memory conversation.
  • The active .venv virtual environment inside private-chat.
  • The exported KODEKEY_API_KEY variable in the active Bash shell.
  • The private-chat folder containing chat_app.py, requirements.txt, and .venv.

Keep everything running

No action is needed. Choose this option if you want to continue comparing models or testing system prompts.

  • Keep the Bash terminal open to leave the local Streamlit server running.
  • Continue testing models through the browser chat app.
  • Expect each new model request to consume part of your shared KodeKey allowance.
  • Expect the in-memory conversation to disappear when the local server stops.

Pause - I'll come back to this later

Pausing closes the local server while keeping every project file. Choose this option if you plan to return later.

  • Press Ctrl+C in the Bash terminal to stop Streamlit.

The local page stops responding. Its in-memory conversation is discarded.

  • Leave the active virtual environment by running:
deactivate

Your Bash prompt no longer shows the active .venv environment.

  • Remove the exported API key from the current shell by running:
unset KODEKEY_API_KEY

The current Bash shell no longer exposes KODEKEY_API_KEY to local processes.

Your workspace is safely paused. The private-chat folder remains ready for your next session.

Delete - I don't want to use this again

Deleting this folder is permanent. This cleanup removes only the local project files and shell state.

  • Press Ctrl+C in the Bash terminal to stop Streamlit.

The server closes. Its in-memory conversation is removed with the stopped process.

  • Leave the active virtual environment by running:
deactivate

Your Bash prompt no longer shows the active .venv environment.

  • Remove the exported API key from the current shell by running:
unset KODEKEY_API_KEY

The current shell forgets the exported value. The underlying KodeKey API key remains in your KodeKloud account.

  • In Bash, leave the private-chat folder and return to the folder that contains it.
  • Delete the complete project folder by running:
rm -rf private-chat

This command permanently removes chat_app.py, requirements.txt, and the .venv environment.

  • In your Linux file manager, open the folder that previously contained private-chat.
  • Confirm that the private-chat folder no longer appears.

That completes the local cleanup. Your KodeKey account and its API key remain available unless you manage them separately in KodeKloud.

Nice Work!

Nice Work!

Fantastic work! Your Streamlit chat room now sends real prompts through KodeKey while preserving one conversation across model changes.

You also built recovery controls that keep request failures inside the page. Your system prompt studio lets you reshape model behavior without adding hidden instructions to the visible conversation.

You learned how to:

  • Built a browser-based chat interface around an OpenAI-compatible gateway. Populated the model dropdown through dynamic model discovery from KodeKey's current catalog.
  • Experienced stateless API requests through a one-turn chat. Fixed the memory gap with Streamlit Session State plus complete message history in every request.
  • Preserved cross-model context while switching models inside one thread. Added answer labels plus recoverable model-access errors plus a New chat control.
  • Secret Mission: Added a system prompt studio that changes how each selected model responds without showing the instruction as a normal chat turn.

Ready to quiz yourself?