Multimodal Image Intelligence App

Build and deploy an AI web app that analyzes images using Gemini.

Introduction

30 Second Summary

Every day, you take photos of receipts, screenshot stock charts, and snap pictures of business cards. What if you could upload any of these images and instantly get useful, structured information back from an AI?

In this project, you will build a web app that uses Google Gemini to analyze uploaded images in three different ways. You will deploy it for free so anyone can use it.

What You'll Build

A Streamlit web application called "Image Intelligence" that accepts uploaded images and runs them through Google Gemini 2.5 Flash for AI-powered analysis.

By the end of this project, you'll have:

  • A working multimodal AI web app that analyzes images using Google Gemini 2.5 Flash.
  • Three distinct analysis modes powered by targeted prompt engineering: general description, structured data extraction, and stock chart technical analysis.
  • A live deployment on Streamlit Community Cloud that anyone can visit for free.
  • Secret Mission: Add an image comparison mode that lets users upload two stock charts and get a side-by-side comparative analysis.

Are there any prerequisites?

You should have basic Python knowledge (variables, functions, and imports). You also need a Google account and a GitHub account.

Set Up Your Environment

Before you can build an image analysis app, you need a working development environment with the right tools connected.

In this step, you will create a project folder, install the required Python packages, and connect to Google's Gemini API. By the end, you will have a confirmed connection to Gemini 2.5 Flash ready for image analysis.

In this step, get ready to:
  • Create a project folder, virtual environment, and install dependencies.
  • Get a free Gemini API key and store it securely.
  • Run a test script to confirm your API connection works.
Create your project folder and install dependencies

You need a dedicated folder for your project files and a Python virtual environment to keep your packages isolated from the rest of your system.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir image-intelligence
cd image-intelligence
  • Confirm the folder exists by running:
ls

You should see the image-intelligence folder listed in the output if you run ls from your Desktop. Since you already moved into it, the folder will appear empty for now.

Next, create a Python virtual environment and activate it. This keeps your project packages separate from your system Python.

macOS

  • Create and activate your virtual environment by running these commands:
python3 -m venv .venv
source .venv/bin/activate

Windows

  • Create and activate your virtual environment by running these commands:
python -m venv .venv
.venv\Scripts\activate

You should see (.venv) appear at the beginning of your terminal prompt. This confirms your virtual environment is active.

What is a virtual environment?

A virtual environment is an isolated space for your Python project's packages. Without one, installing packages could conflict with other Python projects on your machine.

The (.venv) prefix in your terminal means every pip install goes into this project's folder only.

  • Install the two Python packages your project needs by running:
pip install google-genai streamlit

What are these packages?

The google-genai package is Google's official SDK for interacting with Gemini models. Streamlit is a Python framework that turns scripts into interactive web apps with zero frontend code.

The install will download several dependencies. Once complete, you should see a success message listing the installed packages.

Install failing or timing out?

Make sure you see (.venv) in your terminal prompt. If it is missing, the virtual environment is not active. Re-run the activation command above.

If you see a permissions error, do not use sudo. Deactivate and recreate the virtual environment instead.

Help me fix my pip install error

Get your Gemini API key

Google's Gemini API is free to use. You do not need a credit card or billing account. You just need a Google account to generate an API key from Google AI Studio.

  • Open your browser and go to aistudio.google.com.
  • Sign in with your Google account if prompted.
  • If you are a new user, accept the Terms of Service when prompted.
  • Click Get API Key in the left panel to open the API keys page.
  • Click Create API key to generate a new key.
  • Copy your new API key and keep it somewhere safe temporarily.

Why is this free?

Gemini 2.5 Flash is available on Google's free tier with no billing account required. Input and output tokens are free of charge. This makes it ideal for learning and prototyping.

Now store your API key in a .env file so it stays out of your code. You will never paste your key directly into a Python file.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Select the image-intelligence folder on your Desktop, then click Open.
  • Create a new file by clicking File then New File.
  • Name the file .env and press Enter.
  • Paste the following into your .env file, replacing your-api-key-here with the key you just copied:
GEMINI_API_KEY = "your-api-key-here"

What does this file do?

The .env file is a standard way to store secrets locally. It keeps your API key out of your source code so you never accidentally share it. Later, you will add this file to .gitignore to prevent it from being pushed to GitHub.

  • Save the file with Cmd+S (macOS) or Ctrl+S (Windows).

Having trouble creating an API key?

If you see a permissions error in AI Studio, try signing out and back in with your personal Google account. Organization-managed accounts sometimes have restrictions.

If the Create API key button is greyed out, you may need to accept the Terms of Service first. Look for a banner or prompt at the top of the page.

Help me get my Gemini API key working

Test your API connection

Before building the full app, you need to confirm that your API key works and Gemini responds to requests. You will write a short test script that sends a text prompt and prints the response.

  • In VS Code, create a new file called test_gemini.py in your image-intelligence folder.
  • Paste the following code into test_gemini.py:
import os
from google import genai

# Create a Gemini client using your API key from environment variables
client = genai.Client(api_key=os.environ.get("GEMINI_API_KEY"))

# Send a simple text prompt to confirm the connection works
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Say hello and confirm you are Gemini 2.5 Flash.",
)

# Print the model's response
print(response.text)

What does this code do?

The script creates a Gemini client using your API key from the GEMINI_API_KEY environment variable. It then sends a simple text prompt to the gemini-2.5-flash model and prints whatever the model responds with.

The os.environ.get() call reads the value from your terminal session's environment variables. You will set this variable before running the script.

  • Save test_gemini.py.

Before running the script, you need to load your API key into the terminal session as an environment variable.

macOS

  • In your terminal (make sure (.venv) is still showing), export your API key by running:
export GEMINI_API_KEY="your-api-key-here"
  • Replace your-api-key-here with the actual key you copied from AI Studio.
  • Run the test script:
python3 test_gemini.py

Windows

  • In your terminal (make sure (.venv) is still showing), export your API key by running:
$env:GEMINI_API_KEY = "your-api-key-here"
  • Replace your-api-key-here with the actual key you copied from AI Studio.
  • Run the test script:
python test_gemini.py

You should see a friendly response from Gemini confirming it is Gemini 2.5 Flash. The exact wording varies each time, but it will greet you and identify itself.

Seeing an error instead of a response?

If you see TypeError: NoneType or similar, your API key environment variable is not set. Make sure you ran the export (macOS) or $env: (Windows) command in the same terminal session where you run the script.

If you see a 403 or permission error, your API key may be invalid. Go back to AI Studio and generate a fresh key.

If you see a module not found error for google.genai, make sure your virtual environment is active ((.venv) in prompt) and re-run pip install google-genai.

Help me fix my Gemini API test script error

✔️ Awesome, I've got everything!

Great. Make sure you have saved both files before moving on.

ⓧ I'd like to double check the full code

Here is what each file should look like at this point.

.env

GEMINI_API_KEY = "your-api-key-here"

test_gemini.py

import os
from google import genai

client = genai.Client(api_key=os.environ.get("GEMINI_API_KEY"))

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Say hello and confirm you are Gemini 2.5 Flash.",
)

print(response.text)

Your environment is ready and your API key is confirmed working. Next up, you will build the Streamlit app and add the first image analysis mode.

Build the Image Description Mode

In Step 1 you confirmed that your Gemini API key works by sending a text-only prompt.

Now it's time to build the actual app. You'll create a Streamlit web interface that lets users upload any image and get a rich AI-generated description powered by Gemini 2.5 Flash. This step builds the first of three analysis modes.

In this step, get ready to:
  • Create the Streamlit app with a title, sidebar, and image file uploader.
  • Build the image analysis logic that sends images to Gemini using a descriptive prompt.
  • Display the uploaded image alongside the AI-generated description in a two-column layout.
Set up Streamlit secrets and create the app file

Your test script used an environment variable to read the API key. Streamlit has its own secrets system that reads from a .streamlit/secrets.toml file in your project folder. This is the same format used when you deploy later to Streamlit Community Cloud.

  • Create the .streamlit directory and the secrets file by running these commands:
mkdir .streamlit
touch .streamlit/secrets.toml
  • Open .streamlit/secrets.toml in VS Code and paste your Gemini API key:
GEMINI_API_KEY = "your-api-key-here"

What is secrets.toml?

Streamlit reads secrets from .streamlit/secrets.toml in your project directory. You access them in code with st.secrets["GEMINI_API_KEY"]. This keeps your API key out of your source code.

When you deploy later, you'll paste the same secret into Streamlit Community Cloud's settings. The .gitignore file you create in Step 5 will prevent this file from being committed to GitHub.

  • Replace your-api-key-here with the actual API key you generated in Step 1.
  • Save .streamlit/secrets.toml.

Now create the main app file. This is where all your Streamlit code will live.

  • Create a new file called app.py in your project folder by running:
touch app.py
  • Open app.py in VS Code.
  • Add the following code to set up the app structure with imports, the Gemini client, page configuration, and a sidebar for mode selection:
import streamlit as st
from google import genai
from google.genai import types

# Initialize the Gemini client using Streamlit's secrets
client = genai.Client(api_key=st.secrets["GEMINI_API_KEY"])

# Set which Gemini model to use
MODEL_ID = "gemini-2.5-flash"

# Configure the page layout and title
st.set_page_config(page_title="Image Intelligence", page_icon="\ud83d\udd0d", layout="wide")
st.title("Image Intelligence")
st.caption("Powered by Google Gemini 2.5 Flash")

# Sidebar lets users pick an analysis mode
mode = st.sidebar.radio(
    "Analysis Mode",
    ["Image Description"]
)

What does this code do?

The first three lines import Streamlit and the Gemini SDK. The client object connects to Gemini using the API key stored in secrets.toml.

The st.set_page_config call sets the browser tab title and a wide layout so images display nicely. The sidebar radio button will eventually hold all three analysis modes.

Add the image description prompt and file uploader

The quality of AI image analysis depends heavily on the prompt you send. A detailed prompt tells Gemini exactly what to look for in an uploaded image.

  • Below the sidebar code in app.py, add the prompt dictionary and file uploader:
# Store prompts in a dictionary keyed by mode name
PROMPTS = {
    "Image Description": (
        "Analyze this image in detail. Describe what you see including objects, "
        "people, colors, setting, mood, any text visible, and notable details. "
        "Provide a comprehensive, well-organized description."
    ),
}

# File uploader accepts common image formats
uploaded_file = st.file_uploader(
    "Upload an image",
    type=["jpg", "jpeg", "png", "webp"],
    help="Supported formats: JPEG, PNG, WEBP"
)

Why use a dictionary for prompts?

Storing prompts in a PROMPTS dictionary makes it easy to add new modes later. Each mode maps to a specialized prompt that tells Gemini how to analyze the image. The mode variable from the sidebar selects which prompt to use.

The file uploader widget handles image selection. Streamlit displays a drag-and-drop area where users can upload JPEG, PNG, or WEBP files.

Build the image analysis and display logic

With the uploader in place, you need to send the uploaded image to Gemini and show the results. The types.Part.from_bytes method converts raw image bytes into a format Gemini understands.

  • Below the file uploader code in app.py, add the analysis logic:
if uploaded_file is not None:
    # Read the raw bytes from the uploaded image
    image_bytes = uploaded_file.read()

    # Create a two-column layout
    col1, col2 = st.columns(2)

    with col1:
        # Display the uploaded image on the left
        st.image(image_bytes, caption="Uploaded Image", use_container_width=True)

    with col2:
        # Show a spinner while Gemini processes the image
        with st.spinner("Analyzing image..."):
            prompt = PROMPTS[mode]

            # Send the image and prompt to Gemini
            response = client.models.generate_content(
                model=MODEL_ID,
                contents=[
                    types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                    prompt,
                ],
            )

            # Display the AI's description as formatted markdown
            st.markdown(response.text)
else:
    st.info("Upload an image to get started. Select an analysis mode from the sidebar.")

What does this code do?

When a file is uploaded, uploaded_file.read() gets the raw image bytes. The two-column layout (st.columns(2)) places the original image on the left and the AI description on the right.

The types.Part.from_bytes call packages the image bytes with their MIME type so Gemini knows it's receiving an image. The contents array passes both the image and the text prompt in a single API call. This is what makes it multimodal.

What is multimodal AI?

A multimodal model can process multiple types of input (text, images, audio) in a single request. Here you're sending both an image and a text instruction together, and Gemini reasons about them as a combined input to produce its response.

  • Save app.py.
  • Run the app by entering this command in your terminal:
streamlit run app.py
  • Your browser should open automatically to http://localhost:8501. You should see the "Image Intelligence" title, a sidebar with "Image Description" selected, and a file upload area.
  • Upload any photo from your computer (a landscape, a pet, a screenshot). You should see the image on the left and a detailed AI-generated description on the right.

App not loading or showing an error?

If you see a FileNotFoundError mentioning secrets, make sure you created the .streamlit/secrets.toml file in the same folder where you run streamlit run. The directory name must be exactly .streamlit (with the leading dot).

If you see an API error, double check that your API key in .streamlit/secrets.toml is correct and matches the one you tested in Step 1.

Help me debug my Streamlit app. I'm getting an error when running streamlit run app.py.

✔️ Awesome, I've got everything!

Great. Double check you've saved app.py and leave the Streamlit server running in your terminal.

ⓧ I'd like to double check the full code

import streamlit as st
from google import genai
from google.genai import types

client = genai.Client(api_key=st.secrets["GEMINI_API_KEY"])

MODEL_ID = "gemini-2.5-flash"

st.set_page_config(page_title="Image Intelligence", page_icon="\ud83d\udd0d", layout="wide")
st.title("Image Intelligence")
st.caption("Powered by Google Gemini 2.5 Flash")

mode = st.sidebar.radio(
    "Analysis Mode",
    ["Image Description"]
)

PROMPTS = {
    "Image Description": (
        "Analyze this image in detail. Describe what you see including objects, "
        "people, colors, setting, mood, any text visible, and notable details. "
        "Provide a comprehensive, well-organized description."
    ),
}

uploaded_file = st.file_uploader(
    "Upload an image",
    type=["jpg", "jpeg", "png", "webp"],
    help="Supported formats: JPEG, PNG, WEBP"
)

if uploaded_file is not None:
    image_bytes = uploaded_file.read()

    col1, col2 = st.columns(2)

    with col1:
        st.image(image_bytes, caption="Uploaded Image", use_container_width=True)

    with col2:
        with st.spinner("Analyzing image..."):
            prompt = PROMPTS[mode]

            response = client.models.generate_content(
                model=MODEL_ID,
                contents=[
                    types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                    prompt,
                ],
            )

            st.markdown(response.text)
else:
    st.info("Upload an image to get started. Select an analysis mode from the sidebar.")

Your first analysis mode is live. Next up, you'll add a second mode that extracts structured JSON data from documents and receipts.

Add Structured Data Extraction Mode

Your Image Description mode is working. Users can upload a photo and get a rich written description back from Gemini.

Now you'll add a second mode that does something different with the same image input. Instead of free-form text, this mode extracts structured data as JSON. Think receipts with itemized totals, business cards with contact fields, or invoices with line items.

In this step, get ready to:
  • Add a Structured Data Extraction mode to the sidebar and write a targeted prompt.
  • Configure the Gemini API to enforce JSON output.
  • Parse and display structured JSON results in the app.
Add the extraction mode and prompt

The app currently has one mode in the sidebar radio. You need to register the new mode there and give it a prompt that tells Gemini exactly what structured fields to extract.

  • In app.py, find the st.sidebar.radio block that looks like this:
mode = st.sidebar.radio(
    "Analysis Mode",
    ["Image Description"]
)
  • Add "Structured Data Extraction" to the list, so the block now looks like this:
mode = st.sidebar.radio(
    "Analysis Mode",
    ["Image Description", "Structured Data Extraction"]
)
  • Next, find the PROMPTS dictionary. Add the following entry after the "Image Description" prompt (inside the same dictionary):
    "Structured Data Extraction": (
        "Extract all structured data from this image and return it as JSON. "
        "If it is a receipt, extract: store_name, date, items (each with name, "
        "quantity, price), subtotal, tax, total. "
        "If it is a business card, extract: name, title, company, email, phone, "
        "address, website. "
        "If it is another document type, extract all relevant fields. "
        "Return only valid JSON with no additional text."
    ),

What does this prompt do?

The prompt gives Gemini specific field names to look for depending on the document type (receipt vs business card vs other). This targeted structure means Gemini returns consistent, predictable JSON fields instead of inventing its own schema each time.

The final instruction "Return only valid JSON with no additional text" discourages the model from wrapping the output in markdown code fences or adding commentary.

  • Save app.py and refresh your browser.
  • You should see Structured Data Extraction appear as a second option in the sidebar radio.

Don't see the new mode in the sidebar?

  • Make sure you saved the file. Streamlit only reloads when the file on disk changes.
  • Check that "Structured Data Extraction" is inside the list brackets of st.sidebar.radio, separated from the first item by a comma.
  • If you see a red error banner in the app, look for a missing comma or mismatched quote in the PROMPTS dictionary.

Help me debug why my new mode isn't appearing in the Streamlit sidebar.

Configure the API call for JSON output

A prompt alone can encourage JSON output, but the model might still wrap it in markdown or add commentary. The Gemini API has a config parameter that forces the response to be valid JSON.

  • In app.py, scroll down to the with col2: block. Find the existing generate_content call that looks like this:
            response = client.models.generate_content(
                model=MODEL_ID,
                contents=[
                    types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                    prompt,
                ],
            )
  • Replace that entire response = ... block with the following conditional logic:
            if mode == "Structured Data Extraction":
                response = client.models.generate_content(
                    model=MODEL_ID,
                    contents=[
                        types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                        prompt,
                    ],
                    config=types.GenerateContentConfig(
                        response_mime_type="application/json"
                    ),
                )
            else:
                response = client.models.generate_content(
                    model=MODEL_ID,
                    contents=[
                        types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                        prompt,
                    ],
                )

What does response_mime_type do?

The config=types.GenerateContentConfig(response_mime_type="application/json") parameter tells the Gemini API to constrain the model's output to valid JSON. The model will not include markdown fences, explanatory text, or any characters that would break a JSON parser.

The else branch keeps the original unconstrained call for Image Description mode, which returns free-form markdown text.

  • Save app.py.
  • You won't see a formatted change yet. The next chunk adds proper JSON display.
Parse and display the JSON response

The API now returns raw JSON text when Structured Data Extraction is selected. You need to parse that text into a Python dictionary and display it with Streamlit's interactive JSON viewer.

  • In app.py, find the st.markdown(response.text) line (still inside the with col2: block, just after the API call you updated).
            st.markdown(response.text)
  • Replace that single line with the following conditional display logic:
            if mode == "Structured Data Extraction":
                import json
                try:
                    data = json.loads(response.text)
                    st.json(data)
                except json.JSONDecodeError:
                    st.code(response.text, language="json")
            else:
                st.markdown(response.text)

What does this code do?

  • json.loads(response.text) parses the raw JSON string into a Python dictionary.
  • st.json(data) renders that dictionary as a pretty-printed, interactive JSON viewer. You can expand and collapse nested keys directly in the browser.
  • The except json.JSONDecodeError fallback handles the rare case where the response is not valid JSON. It displays the raw text with syntax highlighting via st.code so you can still read the output.
  • The else branch preserves the original markdown rendering for Image Description mode.
  • Save app.py and refresh your browser.
  • Select Structured Data Extraction from the sidebar.
  • Upload a photo of a receipt, business card, or any document with text. You should see clean, formatted JSON with extracted fields like store_name, items, or total.

Seeing raw text instead of formatted JSON?

  • Make sure the if mode == "Structured Data Extraction": check matches the exact string in your PROMPTS dictionary and sidebar radio. A capitalization mismatch sends it through the wrong branch.
  • Check that the config=types.GenerateContentConfig(response_mime_type="application/json") parameter is inside the correct if branch of the API call, not in the else branch.
  • If the JSON viewer appears but is empty, confirm your uploaded image actually contains readable text (a clear photo of a receipt, not a blurry snapshot).

Help me figure out why my structured data extraction isn't returning formatted JSON.

Why not just use the prompt to get JSON?

Prompts can ask for JSON, but models sometimes ignore the instruction and add explanatory text around the output. The response_mime_type parameter enforces the constraint at the API level. The model physically cannot return anything except valid JSON when this is set.

Combining both (a structured prompt AND the mime type constraint) gives you the best results. The prompt guides which fields to extract. The mime type guarantees the format.

✔️ Awesome, I've got everything!

Great. Double check you've saved app.py and that both modes work (Image Description returns markdown text, Structured Data Extraction returns interactive JSON).

ⓧ I'd like to double check the full code

Here is the complete app.py at this point:

import streamlit as st
from google import genai
from google.genai import types

client = genai.Client(api_key=st.secrets["GEMINI_API_KEY"])

MODEL_ID = "gemini-2.5-flash"

st.set_page_config(page_title="Image Intelligence", page_icon="🔍", layout="wide")
st.title("Image Intelligence")
st.caption("Powered by Google Gemini 2.5 Flash")

mode = st.sidebar.radio(
    "Analysis Mode",
    ["Image Description", "Structured Data Extraction"]
)

PROMPTS = {
    "Image Description": (
        "Analyze this image in detail. Describe what you see including objects, "
        "people, colors, setting, mood, any text visible, and notable details. "
        "Provide a comprehensive, well-organized description."
    ),
    "Structured Data Extraction": (
        "Extract all structured data from this image and return it as JSON. "
        "If it is a receipt, extract: store_name, date, items (each with name, "
        "quantity, price), subtotal, tax, total. "
        "If it is a business card, extract: name, title, company, email, phone, "
        "address, website. "
        "If it is another document type, extract all relevant fields. "
        "Return only valid JSON with no additional text."
    ),
}

uploaded_file = st.file_uploader(
    "Upload an image",
    type=["jpg", "jpeg", "png", "webp"],
    help="Supported formats: JPEG, PNG, WEBP"
)

if uploaded_file is not None:
    image_bytes = uploaded_file.read()

    col1, col2 = st.columns(2)

    with col1:
        st.image(image_bytes, caption="Uploaded Image", use_container_width=True)

    with col2:
        with st.spinner("Analyzing image..."):
            prompt = PROMPTS[mode]

            if mode == "Structured Data Extraction":
                response = client.models.generate_content(
                    model=MODEL_ID,
                    contents=[
                        types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                        prompt,
                    ],
                    config=types.GenerateContentConfig(
                        response_mime_type="application/json"
                    ),
                )
            else:
                response = client.models.generate_content(
                    model=MODEL_ID,
                    contents=[
                        types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                        prompt,
                    ],
                )

            if mode == "Structured Data Extraction":
                import json
                try:
                    data = json.loads(response.text)
                    st.json(data)
                except json.JSONDecodeError:
                    st.code(response.text, language="json")
            else:
                st.markdown(response.text)
else:
    st.info("Upload an image to get started. Select an analysis mode from the sidebar.")

Your app now has two distinct analysis modes. Next up, you'll add a third mode that performs technical analysis on stock chart screenshots.

Add Stock Chart Analysis Mode

Your app can now describe images and extract structured data from documents. Those are both powerful, but they treat every image the same way.

In this step, you will add a specialized mode that understands financial charts. By writing a targeted prompt that asks Gemini to think like a technical analyst, you will get structured insights about trends, patterns, and key price levels from any stock chart screenshot.

In this step, get ready to:
  • Add a Stock Chart Analysis option to the sidebar.
  • Write a specialized prompt for technical chart analysis.
  • Test the mode with a stock chart screenshot.
Add the stock chart analysis mode to the sidebar and prompt dictionary

The key to getting great chart analysis from a multimodal model is prompt engineering. A generic "describe this image" prompt would miss the financial context entirely. You need to tell Gemini exactly what kind of analysis you want.

  • In app.py, find the sidebar radio near the top of the file:
mode = st.sidebar.radio(
    "Analysis Mode",
    ["Image Description", "Structured Data Extraction"]
)
  • Add "Stock Chart Analysis" as a third item in the list, so the radio now looks like this:
mode = st.sidebar.radio(
    "Analysis Mode",
    ["Image Description", "Structured Data Extraction", "Stock Chart Analysis"]
)

Now you need a prompt that tells Gemini to act as a technical analyst. This prompt requests five specific sections so the output is structured and useful.

  • In the PROMPTS dictionary, find the closing parenthesis and comma after the "Structured Data Extraction" entry.
  • Add the following new entry below it:
    "Stock Chart Analysis": (
        "You are a technical analyst. Analyze this stock chart image and provide:\n"
        "1. **Trend**: Overall trend direction (bullish, bearish, or sideways) and reasoning\n"
        "2. **Patterns**: Any chart patterns visible (e.g., head and shoulders, double top/bottom, triangles, flags)\n"
        "3. **Key Levels**: Support and resistance levels you can identify\n"
        "4. **Indicators**: Any visible technical indicators and what they suggest (moving averages, RSI, MACD, volume)\n"
        "5. **Summary**: A brief overall assessment and potential outlook\n\n"
        "Base your analysis only on what is visible in the chart image."
    ),

What does this prompt do?

The prompt sets a role ("You are a technical analyst") and then requests five numbered sections. Each section uses bold markdown headers so the AI output arrives pre-formatted with clear headings.

The final instruction ("Base your analysis only on what is visible") prevents the model from hallucinating data that is not in the image.

  • Save app.py.

Seeing a syntax error when the app reloads?

Check that the new prompt entry ends with a closing parenthesis and a comma before the dictionary's closing brace. The PROMPTS dict should now have three key-value pairs.

Also confirm all the \n characters are inside the string (not stray backslashes outside the quotes).

Help me fix a syntax error in my PROMPTS dictionary after adding Stock Chart Analysis.

Understand how the existing code handles the new mode

You do not need to write any new logic for the API call or the display. The code you wrote in the previous steps already handles this mode.

Why does this work without new code?

Look at the else branch in your API call section. Any mode that is not "Structured Data Extraction" uses the standard generate_content call without the JSON config. Stock Chart Analysis falls into this branch automatically.

The same applies to the display. The else block at the bottom uses st.markdown(response.text) to render the response. Because the prompt uses markdown bold headers, Streamlit renders them as proper formatted headings.

Test the stock chart analysis mode

Time to see your new mode in action. You will need a stock chart screenshot to test with.

  • Switch back to your browser where the Streamlit app is running. If you stopped it, run this command in your terminal:
streamlit run app.py
  • In the sidebar, select Stock Chart Analysis.
  • Find any stock chart screenshot online (search for "stock candlestick chart" in your browser and save an image), or take a screenshot of a chart from any financial website.
  • Upload the chart image using the file uploader.

You should see a structured analysis with bold section headers: Trend, Patterns, Key Levels, Indicators, and Summary.

Getting a generic description instead of chart analysis?

Make sure you selected Stock Chart Analysis in the sidebar before uploading. If you changed modes after uploading, click Browse files to re-upload the image so it triggers the new prompt.

Help me debug why my Stock Chart Analysis mode is returning a generic description instead of a structured technical analysis.

✔️ Awesome, I've got everything!

Great. Double check you have saved app.py and all three modes are working.

ⓧ I'd like to double check the full code

import streamlit as st
from google import genai
from google.genai import types

client = genai.Client(api_key=st.secrets["GEMINI_API_KEY"])

MODEL_ID = "gemini-2.5-flash"

st.set_page_config(page_title="Image Intelligence", page_icon="🔍", layout="wide")
st.title("Image Intelligence")
st.caption("Powered by Google Gemini 2.5 Flash")

mode = st.sidebar.radio(
    "Analysis Mode",
    ["Image Description", "Structured Data Extraction", "Stock Chart Analysis"]
)

PROMPTS = {
    "Image Description": (
        "Analyze this image in detail. Describe what you see including objects, "
        "people, colors, setting, mood, any text visible, and notable details. "
        "Provide a comprehensive, well-organized description."
    ),
    "Structured Data Extraction": (
        "Extract all structured data from this image and return it as JSON. "
        "If it is a receipt, extract: store_name, date, items (each with name, "
        "quantity, price), subtotal, tax, total. "
        "If it is a business card, extract: name, title, company, email, phone, "
        "address, website. "
        "If it is another document type, extract all relevant fields. "
        "Return only valid JSON with no additional text."
    ),
    "Stock Chart Analysis": (
        "You are a technical analyst. Analyze this stock chart image and provide:\n"
        "1. **Trend**: Overall trend direction (bullish, bearish, or sideways) and reasoning\n"
        "2. **Patterns**: Any chart patterns visible (e.g., head and shoulders, double top/bottom, triangles, flags)\n"
        "3. **Key Levels**: Support and resistance levels you can identify\n"
        "4. **Indicators**: Any visible technical indicators and what they suggest (moving averages, RSI, MACD, volume)\n"
        "5. **Summary**: A brief overall assessment and potential outlook\n\n"
        "Base your analysis only on what is visible in the chart image."
    ),
}

uploaded_file = st.file_uploader(
    "Upload an image",
    type=["jpg", "jpeg", "png", "webp"],
    help="Supported formats: JPEG, PNG, WEBP"
)

if uploaded_file is not None:
    image_bytes = uploaded_file.read()

    col1, col2 = st.columns(2)

    with col1:
        st.image(image_bytes, caption="Uploaded Image", use_container_width=True)

    with col2:
        with st.spinner("Analyzing image..."):
            prompt = PROMPTS[mode]

            if mode == "Structured Data Extraction":
                response = client.models.generate_content(
                    model=MODEL_ID,
                    contents=[
                        types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                        prompt,
                    ],
                    config=types.GenerateContentConfig(
                        response_mime_type="application/json"
                    ),
                )
            else:
                response = client.models.generate_content(
                    model=MODEL_ID,
                    contents=[
                        types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                        prompt,
                    ],
                )

            if mode == "Structured Data Extraction":
                import json
                try:
                    data = json.loads(response.text)
                    st.json(data)
                except json.JSONDecodeError:
                    st.code(response.text, language="json")
            else:
                st.markdown(response.text)
else:
    st.info("Upload an image to get started. Select an analysis mode from the sidebar.")

Your app now has three distinct analysis modes, each powered by a different prompt engineering strategy. Next up, you will deploy this app to the internet so anyone can use it.

Deploy to Streamlit Community Cloud

Your Image Intelligence app works locally, and all three analysis modes are producing results. The final step is to make it accessible to anyone with an internet link.

To deploy, you will push your project to GitHub and connect it to Streamlit Community Cloud. Streamlit handles all the containerization and hosting for free.

In this step, get ready to:
  • Create deployment files and push your code to GitHub.
  • Deploy the app on Streamlit Community Cloud with your API key configured.
  • Verify all three analysis modes work on the public URL.
Create deployment files and push to GitHub

Streamlit Community Cloud reads your dependencies from a requirements.txt file. It also needs your API key delivered through a secrets file rather than an environment variable.

  • Create a requirements.txt file in your project folder by adding the following content:
google-genai==2.9.0
streamlit==1.58.0

Why pin versions?

Pinning exact versions ensures Streamlit Community Cloud installs the same packages you tested locally. Without pinning, a future update could break your app without warning.

  • Create a .gitignore file to prevent sensitive and unnecessary files from being committed:
.env
__pycache__/
*.pyc
.venv/

What does this file do?

Each line tells Git to ignore matching files. Your .env (which contains your API key), the virtual environment folder, and Python cache files will never be uploaded to GitHub.

  • Create a .streamlit folder, then create a secrets.toml file inside it with your API key:
GEMINI_API_KEY = "your-api-key-here"
  • Replace your-api-key-here with your actual Gemini API key from your .env file.

Why a secrets.toml file?

Your app.py reads the API key using st.secrets["GEMINI_API_KEY"]. Locally, Streamlit reads this from .streamlit/secrets.toml. On Streamlit Community Cloud, you will paste the same content into their secrets management interface.

This file is excluded from Git by .gitignore, so your key never reaches GitHub.

  • Save all three files.
  • Confirm the files exist by running:
ls

You should see requirements.txt, .gitignore, app.py, and test_gemini.py listed. The .streamlit folder may be hidden. Run ls .streamlit to confirm secrets.toml is inside it.

  • Initialize a Git repository and make your first commit by running these commands:
git init
git add .
git commit -m "Initial commit: Image Intelligence app"

What do these commands do?

  • git init creates a new Git repository in your project folder.
  • git add . stages all files (respecting .gitignore).
  • git commit saves the staged files as a snapshot.
  • Go to github.com and create a new repository named image-intelligence.
  • Leave the repository set to **Public** (required for free Streamlit Community Cloud deployment).
  • Do not check any boxes for README, .gitignore, or license (you already have these locally).
  • Click Create repository.
  • Push your local code to GitHub by running these commands (replace the URL with the one GitHub shows you):
git remote add origin [[REPO_URL="https://github.com/your-username/image-intelligence.git"]]
git branch -M main
git push -u origin main

Push failed or asking for credentials?

If Git asks for a username and password, GitHub requires a personal access token instead of a password. Go to github.com/settings/tokens to create one.

If you see a permissions error, make sure you created the repository under your own GitHub account.

help me push my code to GitHub

After the push completes, refresh your GitHub repository page. You should see app.py, requirements.txt, .gitignore, and test_gemini.py listed. Your .env and .streamlit/secrets.toml should NOT appear (they are excluded by .gitignore).

Deploy on Streamlit Community Cloud

Streamlit Community Cloud connects directly to your GitHub repository and deploys your app in minutes. Any future pushes to the repository will automatically update the live app.

  • Go to share.streamlit.io in your browser.
  • Click Continue to sign-in.
  • Click Continue with GitHub.
  • Enter your GitHub credentials and follow the authentication prompts.
  • If prompted to authorize Streamlit, click Authorize streamlit.

What is Streamlit Community Cloud?

Streamlit Community Cloud is a free hosting platform that deploys Python apps directly from GitHub repositories. It handles all server infrastructure so you only manage your code.

  • In the upper-right corner of your workspace, click Create app.
  • When asked "Do you already have an app?" click Yup, I have an app.
  • In the Repository field, select or type your image-intelligence repository.
  • Set Branch to main.
  • Set the Main file path to app.py.
  • Click Advanced settings.
  • In the Secrets text field, paste the contents of your .streamlit/secrets.toml file:
GEMINI_API_KEY = "your-api-key-here"
  • Replace your-api-key-here with your actual Gemini API key.
  • Click Save.

Why paste secrets here?

Your .streamlit/secrets.toml file is excluded from Git, so Streamlit Community Cloud cannot read it from your repository. The Secrets field in Advanced settings is where you provide this information securely. It is encrypted and only accessible to your running app.

  • Click Deploy.

Streamlit will begin building and deploying your app. This takes a few minutes while it installs dependencies and starts the server.

Deployment stuck or showing errors?

If you see a dependency error, double-check that requirements.txt is committed to your repository. You can verify by checking it on your GitHub page.

If the app crashes with a secrets error, click Manage app in the lower-right corner of your deployed app, then check that the Secrets field contains your GEMINI_API_KEY entry.

help me fix my Streamlit Community Cloud deployment

Verify the live app

Once deployment finishes, your app opens automatically at a public .streamlit.app URL. Time to test that everything works in production.

  • Select Image Description from the sidebar.
  • Upload any photo (a landscape, a selfie, a product image).
  • Confirm you see a detailed AI-generated description appear next to the uploaded image.
  • Switch to Structured Data Extraction mode.
  • Upload a receipt or business card image.
  • Confirm you see clean JSON output with extracted fields.
  • Switch to Stock Chart Analysis mode.
  • Upload a stock chart screenshot.
  • Confirm you see a structured technical analysis with sections for Trend, Patterns, Key Levels, Indicators, and Summary.

Share your app with anyone

Copy the URL from your browser's address bar. Anyone with this link can use your Image Intelligence app.

App loads but analysis fails?

If you see an error when uploading an image, your API key might be incorrect in the Streamlit secrets. Click Manage app in the lower-right corner of your deployed app and verify the GEMINI_API_KEY value is correct.

If you hit rate limits, the Gemini free tier resets daily at midnight Pacific time.

help me fix my deployed app

✔️ Awesome, I've got everything!

Great. Double check you have saved all your files and your app is live at the public URL.

ⓧ I'd like to double check the full code

google-genai==2.9.0
streamlit==1.58.0
.env
__pycache__/
*.pyc
.venv/
GEMINI_API_KEY = "your-api-key-here"
import streamlit as st
from google import genai
from google.genai import types

client = genai.Client(api_key=st.secrets["GEMINI_API_KEY"])

MODEL_ID = "gemini-2.5-flash"

st.set_page_config(page_title="Image Intelligence", page_icon="🔍", layout="wide")
st.title("Image Intelligence")
st.caption("Powered by Google Gemini 2.5 Flash")

mode = st.sidebar.radio(
    "Analysis Mode",
    ["Image Description", "Structured Data Extraction", "Stock Chart Analysis"]
)

PROMPTS = {
    "Image Description": (
        "Analyze this image in detail. Describe what you see including objects, "
        "people, colors, setting, mood, any text visible, and notable details. "
        "Provide a comprehensive, well-organized description."
    ),
    "Structured Data Extraction": (
        "Extract all structured data from this image and return it as JSON. "
        "If it is a receipt, extract: store_name, date, items (each with name, "
        "quantity, price), subtotal, tax, total. "
        "If it is a business card, extract: name, title, company, email, phone, "
        "address, website. "
        "If it is another document type, extract all relevant fields. "
        "Return only valid JSON with no additional text."
    ),
    "Stock Chart Analysis": (
        "You are a technical analyst. Analyze this stock chart image and provide:\n"
        "1. **Trend**: Overall trend direction (bullish, bearish, or sideways) and reasoning\n"
        "2. **Patterns**: Any chart patterns visible (e.g., head and shoulders, double top/bottom, triangles, flags)\n"
        "3. **Key Levels**: Support and resistance levels you can identify\n"
        "4. **Indicators**: Any visible technical indicators and what they suggest (moving averages, RSI, MACD, volume)\n"
        "5. **Summary**: A brief overall assessment and potential outlook\n\n"
        "Base your analysis only on what is visible in the chart image."
    ),
}

uploaded_file = st.file_uploader(
    "Upload an image",
    type=["jpg", "jpeg", "png", "webp"],
    help="Supported formats: JPEG, PNG, WEBP"
)

if uploaded_file is not None:
    image_bytes = uploaded_file.read()

    col1, col2 = st.columns(2)

    with col1:
        st.image(image_bytes, caption="Uploaded Image", use_container_width=True)

    with col2:
        with st.spinner("Analyzing image..."):
            prompt = PROMPTS[mode]

            if mode == "Structured Data Extraction":
                response = client.models.generate_content(
                    model=MODEL_ID,
                    contents=[
                        types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                        prompt,
                    ],
                    config=types.GenerateContentConfig(
                        response_mime_type="application/json"
                    ),
                )
            else:
                response = client.models.generate_content(
                    model=MODEL_ID,
                    contents=[
                        types.Part.from_bytes(data=image_bytes, mime_type=uploaded_file.type),
                        prompt,
                    ],
                )

            if mode == "Structured Data Extraction":
                import json
                try:
                    data = json.loads(response.text)
                    st.json(data)
                except json.JSONDecodeError:
                    st.code(response.text, language="json")
            else:
                st.markdown(response.text)
else:
    st.info("Upload an image to get started. Select an analysis mode from the sidebar.")

Your Image Intelligence app is live. Anyone with the link can upload images and get AI-powered analysis. Next up, the Secret Mission will challenge you to add an image comparison feature.

Secret mission

Image Comparison Mode

Your app can analyze one image at a time. But what if you could upload two stock charts side by side and get Gemini to compare them? Add a fourth mode that sends multiple images in a single API call for comparative analysis.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely for free. Your Streamlit Community Cloud app, GitHub repository, and Gemini API key are all on free tiers with no ongoing costs.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Local project folder on your Desktop (contains app.py, test_gemini.py, requirements.txt, .env, .venv/, and .streamlit/).
  • GitHub repository (free, no cost).
  • Streamlit Community Cloud app (free, no cost).
  • Google AI Studio API key (free tier, no cost).

Keep everything running

No action needed. Choose this if you are still actively building or want to keep your app live.

  • Your Streamlit Community Cloud app stays live for free. The Gemini free tier resets daily, and Streamlit Community Cloud has no time limit on hosted apps.
  • Your GitHub repository remains available at no cost.
  • Your local project files and virtual environment stay on your machine.

Pause - I'll come back to this later

Streamlit Community Cloud apps automatically go to sleep when not visited and wake up when someone visits the URL. No manual action is required to pause.

  • Your app will sleep on its own after a period of inactivity. It wakes up automatically when someone visits the URL.
  • Your GitHub repository and API key remain available whenever you want to return.
  • Optionally, deactivate your local virtual environment by running:
deactivate

This frees up your terminal session without deleting anything.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below to fully clean up.

  • Delete your Streamlit Community Cloud app:
  • Go to your workspace at share.streamlit.io.
  • Click the three-dot overflow icon next to your app.
  • Click Delete, enter the confirmation string, and click Delete again.
  • Delete your GitHub repository:
  • Go to your repository on GitHub.
  • Click Settings at the top of the repository page.
  • Scroll to the bottom of the page and click Delete this repository.
  • Follow the confirmation prompts to complete the deletion.
  • Delete your Gemini API key:
  • Go to the API Keys page in Google AI Studio at aistudio.google.com.
  • Locate the key you created for this project and delete it.
  • Delete your local project folder:

On macOS or Linux, run:

cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your-project-folder-name"]]

On Windows, run:

cd ~\Desktop
Remove-Item -Recurse -Force [[PROJECT_FOLDER="your-project-folder-name"]]

That's a Wrap!

That's a Wrap!

Nice work! You just built and deployed a multimodal AI web application that analyzes images using Google Gemini 2.5 Flash.

You've learned how to:

  • Built a multimodal AI application that processes images using Google Gemini 2.5 Flash, with three distinct analysis modes powered by prompt engineering.
  • Configured structured JSON output from a large language model using response_mime_type to enforce valid data extraction.
  • Deployed a Python web app for free using Streamlit Community Cloud, going from local code to a live public URL.
  • Secret Mission: Added an image comparison mode that sends multiple images to a single API call for comparative technical analysis.

Ready to quiz yourself?