Add Local AI to Your Grafana Stack

Run an AI-powered monitoring stack locally with Grafana, Prometheus, and Ollama.

Introduction

30 Second Summary

Organizations use dashboards to monitor their systems, but when something breaks at 2am, teams still have to manually read graphs and scroll through logs to figure out what went wrong. AI can interpret monitoring data in plain language, but sending private infrastructure data to external AI services is often a dealbreaker for security-conscious teams.

In this project, you will build a complete AI-powered observability platform that runs entirely on your laptop. You'll set up Grafana with Prometheus and Loki to collect real metrics and logs from your containers, then connect a local AI model that can explain your panels and auto-generate dashboard content.

What You'll Build

A private, AI-powered monitoring platform that mirrors what Grafana Cloud's AI assistant does, but running entirely locally with no data leaving your machine.

By the end of this project, you'll have:

A Docker Compose stack with Grafana, Prometheus, Loki, and Promtail collecting real container metrics and logs.
A local AI model (Qwen 2.5 1.5B) running via Ollama, connected to Grafana through the LLM plugin for private, on-premise AI features.
AI-powered features like panel title generation, descriptions, explanations, and save-change summaries.
An organizational deployment plan for scaling this setup to your datacentre.
Secret Mission: Set up AI-powered alerting that auto-generates incident summaries when something goes wrong.

Are there any prerequisites?

You should have a basic understanding of what monitoring means (knowing why you'd track CPU, memory, or logs). Basic command-line familiarity is also helpful. You'll need Docker Desktop installed on Windows.

Before We Start

Before We Start...

Install Ollama and Verify Docker

Your AI-powered monitoring stack needs two foundations: a local AI model server to run language models on your machine, and Docker to host the monitoring containers.

In this step, you'll install Ollama (the AI model server), pull a lightweight model that fits your hardware, and confirm Docker is ready to go.

In this step, get ready to:

Install Ollama and confirm its API is running.
Pull and test the Qwen 2.5 1.5B AI model.
Verify Docker Desktop and create the project directory.

Install Ollama on Windows

Ollama is a local LLM server that lets you run AI models directly on your machine.

Later in this project, Grafana will send requests to Ollama to generate panel titles, descriptions, and explanations.

Press the Windows key to open your search bar.
Type PowerShell and press Enter to open it.

Install Ollama by running this command:

irm https://ollama.com/install.ps1 | iex

What does this command do?

This downloads the official Ollama installer script from ollama.com and runs it immediately in PowerShell.

irm is short for Invoke-RestMethod. It fetches the script content from the URL.
iex is short for Invoke-Expression. It executes the downloaded script.

What will I see during installation?

The installer may open a separate window or show progress in your terminal. It takes a minute or two to complete. Once it finishes, Ollama starts running automatically in the background.

After the installation finishes, verify Ollama is installed by running:

ollama --version

You should see a version number printed in the terminal.

✔️ I see a version number

Ollama is installed and ready. Move on to the next verification below.

ⓧ Command not found

PowerShell cannot find the ollama command. Try these fixes:

Close your current PowerShell window and open a new one. The installer updates your system PATH, but the current session may not have picked it up yet.
If it still fails, re-run the install command above.

Confirm the Ollama API is accessible by opening your web browser and navigating to:

http://localhost:11434

You should see the text "Ollama is running" in your browser.

Don't see "Ollama is running"?

Make sure the installation completed successfully. Run ollama --version again to confirm.
Check that nothing else is using port 11434.
Try restarting Ollama by closing and reopening PowerShell, then running ollama serve manually.

Help me troubleshoot why Ollama's API isn't responding at localhost:11434 on Windows.

Pull the Qwen 2.5 1.5B model

Ollama needs a model file before it can respond to AI requests. The Qwen 2.5 1.5B model is small enough to run well on a 4GB GPU while still generating useful text.

Download the model by running this command in PowerShell:

ollama pull qwen2.5:1.5b

What does this command do?

This downloads the Qwen 2.5 1.5B model to your machine. The download is around 1GB, so it may take a few minutes depending on your internet speed.

Once downloaded, the model stays on your machine. You only need to pull it once.

You should see a download progress bar followed by a success message.

Test the model with a quick prompt by running:

ollama run qwen2.5:1.5b "What is Prometheus monitoring?"

You should see the AI generate a short explanation about Prometheus monitoring. The response may take a few seconds.

Why Prometheus as the test prompt?

Prometheus is one of the monitoring tools you'll set up in this project. This quick test confirms the model can generate meaningful responses about the topics Grafana will ask it about later.

Model download stuck or failed?

Check your internet connection. The model file is around 1GB.
If the download was interrupted, run the ollama pull qwen2.5:1.5b command again. It resumes from where it left off.
If you see a timeout error, try again after a few minutes.

Help me fix an issue downloading the qwen2.5:1.5b model with Ollama on Windows.

Verify Docker Desktop and create the project directory

Your monitoring stack (Grafana, Prometheus, Loki, and Promtail) will run as Docker containers. Docker Desktop is already installed on your machine, so you just need to confirm it's running.

Confirm Docker is running by checking the version:

docker compose version

You should see a version number for Docker Compose printed in the terminal.

Command not found or Docker not running?

Make sure Docker Desktop is open and running. Look for the Docker whale icon in your system tray (bottom right of your taskbar).
If the icon isn't there, press the Windows key, type Docker Desktop, and press Enter to open it. Wait for it to finish starting.
If you see "docker: command not found", close PowerShell and reopen it after Docker Desktop is fully running.

Help me troubleshoot Docker Desktop not running on Windows.

Now you'll create a project folder to hold all the configuration files for your monitoring stack.

Move to your Desktop by running this command:

cd ~/Desktop

Create your project folder, its subdirectories, and move into it by running these commands:

mkdir grafana-ai-monitoring
cd grafana-ai-monitoring
mkdir grafana\provisioning\datasources

What does this directory structure do?

The grafana-ai-monitoring folder is where all your Docker Compose and configuration files will live.

The grafana\provisioning\datasources subfolder is where Grafana looks for data source configuration files on startup. Creating it now means Grafana will auto-register your data sources later.

Confirm the directory was created by running:

ls

You should see a grafana folder in the output.

Don't see the grafana folder?

Make sure you ran cd grafana-ai-monitoring before running ls. You need to be inside the project folder.
If the mkdir command returned an error, try running it again.

Help me fix an issue creating my project directory structure on Windows.

Your tools are ready and your project folder is in place. Next up, you'll build the full monitoring stack by writing Docker Compose and configuration files.

Build the Monitoring Stack with Docker Compose

Your AI model is ready and waiting. Now you need the monitoring platform it will power.

In this step, you will define your entire observability stack as code using Docker Compose. One command will spin up Grafana, Prometheus, Loki, and Promtail together, all pre-configured to talk to each other.

In this step, get ready to:

Define the full monitoring stack in a Docker Compose file.
Create configuration files for Prometheus, Loki, Promtail, and Grafana datasources.
Start the stack and confirm Grafana is accessible.

Create the Docker Compose file

Docker Compose lets you define multiple containers and their relationships in a single YAML file. Your stack has four services that work together to collect metrics, aggregate logs, and visualize everything.

Create a new file named docker-compose.yml in your project directory and paste the following:

services:
  grafana:
    image: grafana/grafana-enterprise
    container_name: grafana
    restart: unless-stopped
    environment:
      - GF_PLUGINS_PREINSTALL=grafana-llm-app
      - GF_SECURITY_ADMIN_PASSWORD=admin
    ports:
      - "3000:3000"
    volumes:
      - grafana-storage:/var/lib/grafana
      - ./grafana/provisioning:/etc/grafana/provisioning
    depends_on:
      - prometheus
      - loki

  prometheus:
    image: prom/prometheus:v3.4.0
    container_name: prometheus
    restart: unless-stopped
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-storage:/prometheus
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
      - "--storage.tsdb.path=/prometheus"
      - "--storage.tsdb.retention.time=7d"

  loki:
    image: grafana/loki:3.7.0
    container_name: loki
    restart: unless-stopped
    ports:
      - "3100:3100"
    volumes:
      - ./loki-config.yml:/etc/loki/local-config.yaml
      - loki-storage:/loki
    command: -config.file=/etc/loki/local-config.yaml

  promtail:
    image: grafana/promtail:3.7.0
    container_name: promtail
    restart: unless-stopped
    volumes:
      - ./promtail-config.yml:/etc/promtail/config.yml
      - /var/run/docker.sock:/var/run/docker.sock:ro
    command: -config.file=/etc/promtail/config.yml
    depends_on:
      - loki

volumes:
  grafana-storage:
  prometheus-storage:
  loki-storage:

What does this file do?

This file defines four interconnected containers:

Grafana is your dashboard and visualization layer. The GF_PLUGINS_PREINSTALL variable tells it to install the LLM plugin automatically on startup.
Prometheus scrapes metrics from your services every 15 seconds and stores them for 7 days.
Loki is a log aggregation system that stores logs from all your containers.
Promtail is the log collector. It reads Docker container logs and ships them to Loki.

Save docker-compose.yml.

YAML indentation errors?

YAML is sensitive to spacing. Make sure you use spaces (not tabs) for indentation, and that nested items are indented consistently with 2 or 4 spaces.

Help me fix a YAML parsing error in my docker-compose.yml

✔️ Awesome, I've got everything!

Great. Make sure you have saved docker-compose.yml before moving on.

ⓧ I'd like to double check the full code

services:
  grafana:
    image: grafana/grafana-enterprise
    container_name: grafana
    restart: unless-stopped
    environment:
      - GF_PLUGINS_PREINSTALL=grafana-llm-app
      - GF_SECURITY_ADMIN_PASSWORD=admin
    ports:
      - "3000:3000"
    volumes:
      - grafana-storage:/var/lib/grafana
      - ./grafana/provisioning:/etc/grafana/provisioning
    depends_on:
      - prometheus
      - loki

  prometheus:
    image: prom/prometheus:v3.4.0
    container_name: prometheus
    restart: unless-stopped
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-storage:/prometheus
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
      - "--storage.tsdb.path=/prometheus"
      - "--storage.tsdb.retention.time=7d"

  loki:
    image: grafana/loki:3.7.0
    container_name: loki
    restart: unless-stopped
    ports:
      - "3100:3100"
    volumes:
      - ./loki-config.yml:/etc/loki/local-config.yaml
      - loki-storage:/loki
    command: -config.file=/etc/loki/local-config.yaml

  promtail:
    image: grafana/promtail:3.7.0
    container_name: promtail
    restart: unless-stopped
    volumes:
      - ./promtail-config.yml:/etc/promtail/config.yml
      - /var/run/docker.sock:/var/run/docker.sock:ro
    command: -config.file=/etc/promtail/config.yml
    depends_on:
      - loki

volumes:
  grafana-storage:
  prometheus-storage:
  loki-storage:

Create the configuration files

Each service in the stack needs its own configuration file. You will also create a Grafana provisioning file that auto-registers Prometheus and Loki as data sources so they are ready the moment Grafana starts.

Create a new file named prometheus.yml in your project directory and paste the following:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]

  - job_name: "grafana"
    static_configs:
      - targets: ["grafana:3000"]

  - job_name: "loki"
    static_configs:
      - targets: ["loki:3100"]

What does this config do?

This tells Prometheus which services to collect metrics from. Each job_name defines a scrape target. Prometheus will pull metrics from itself, Grafana, and Loki every 15 seconds.

The targets use Docker Compose service names (like grafana:3000) because containers on the same Docker network can reach each other by name.

Save prometheus.yml.
Create a new file named loki-config.yml in your project directory and paste the following:

auth_enabled: false

server:
  http_listen_port: 3100

common:
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: "2020-10-24"
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

limits_config:
  retention_period: 72h

What does this config do?

This configures Loki to store log data on the local filesystem inside the container. The schema_config section uses TSDB with schema v13, which is the current recommended storage format. Logs are kept for 72 hours before being cleaned up.

Save loki-config.yml.
Create a new file named promtail-config.yml in your project directory and paste the following:

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
  - job_name: docker
    docker_sd_configs:
      - host: unix:///var/run/docker.sock
        refresh_interval: 5s
    relabel_configs:
      - source_labels: ["__meta_docker_container_name"]
        target_label: container
        regex: "/(.+)"
      - source_labels: ["__meta_docker_container_label_com_docker_compose_service"]
        target_label: service

What does this config do?

Promtail automatically discovers running Docker containers via the Docker socket. It reads their logs and forwards them to Loki.

The relabel_configs section extracts useful labels (container name, service name) from Docker metadata so you can filter logs by container later in Grafana.

Save promtail-config.yml.

The last config file lives in a nested directory. Grafana looks for provisioning files in a specific folder structure.

Create the directory structure by running this command in your project directory:

mkdir grafana/provisioning/datasources

Create a new file named datasources.yml inside the grafana/provisioning/datasources folder and paste the following:

apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090
    isDefault: true
    editable: true

  - name: Loki
    type: loki
    access: proxy
    url: http://loki:3100
    editable: true

What does this config do?

This provisioning file tells Grafana to automatically register two data sources on startup. Without it, you would have to manually add Prometheus and Loki through the Grafana UI every time the container restarts.

The access: proxy setting means Grafana's backend queries the data sources, not the browser directly.

Save datasources.yml.
Confirm all your files exist by running:

ls

You should see docker-compose.yml, prometheus.yml, loki-config.yml, promtail-config.yml, and a grafana folder listed in the output.

Missing a file?

Double check that you saved each file in the correct location. All files except datasources.yml should be in the root of your project directory. The datasources.yml file must be inside grafana/provisioning/datasources/.

Help me figure out where each config file should be saved

✔️ Awesome, I've got everything!

Great. All configuration files are in place. Time to start the stack.

ⓧ I'd like to double check the full code

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]

  - job_name: "grafana"
    static_configs:
      - targets: ["grafana:3000"]

  - job_name: "loki"
    static_configs:
      - targets: ["loki:3100"]

auth_enabled: false

server:
  http_listen_port: 3100

common:
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: "2020-10-24"
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

limits_config:
  retention_period: 72h

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
  - job_name: docker
    docker_sd_configs:
      - host: unix:///var/run/docker.sock
        refresh_interval: 5s
    relabel_configs:
      - source_labels: ["__meta_docker_container_name"]
        target_label: container
        regex: "/(.+)"
      - source_labels: ["__meta_docker_container_label_com_docker_compose_service"]
        target_label: service

apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090
    isDefault: true
    editable: true

  - name: Loki
    type: loki
    access: proxy
    url: http://loki:3100
    editable: true

Start the stack and verify Grafana

All configuration files are in place. One command will pull the container images and start the entire stack.

Start all containers by running this command:

docker compose up -d

What does this command do?

The -d flag runs the containers in detached mode (in the background). Docker will pull any images it does not have locally, create the containers, and start them. The first run takes a few minutes while images download.

Verify all containers are running by running:

docker compose ps

You should see four containers listed (grafana, prometheus, loki, promtail) all with a status of Up.

Open your browser and navigate to http://localhost:3000.
Log in with username admin and password admin.
If prompted to change the password, click Skip for now.

You should see the Grafana home page with a welcome message and navigation sidebar.

Containers not starting?

If a container shows Exited status, check its logs by running docker compose logs <service-name> (replace with the service that failed).
If port 3000 is already in use, another application may be running on that port. Stop the conflicting application or change Grafana's port in docker-compose.yml.
Make sure Docker Desktop is running before you execute docker compose up -d.

Help me troubleshoot my Docker containers

Your monitoring stack is live. Next up, you will explore the real metrics and logs it is already collecting.

Explore Your Monitoring Data

Your monitoring stack is running and collecting data. Prometheus has been scraping metrics from itself, Grafana, and Loki every 15 seconds since you started the containers.

Now it's time to actually see what that data looks like. In this step, you'll query both metrics and logs using Grafana's Explore view, then build a dashboard that gives you an at-a-glance picture of your stack's health.

In this step, get ready to:

Run PromQL queries against Prometheus to see real-time metrics.
Run LogQL queries against Loki to see container logs.
Create a dashboard with three panels showing memory usage, scrape targets, and live logs.

Query Prometheus metrics in the Explore view

Grafana's Explore view is a sandbox for running ad-hoc queries against your data sources without creating a dashboard first. You'll use PromQL (Prometheus Query Language) to pull live metrics from your containers.

In Grafana, click Explore in the left side menu.
Select Prometheus from the data source drop-down in the upper left.
In the query editor, type the following query and press Shift+Enter (or click Run query) to execute it:

up

You should see a table showing each scrape target with a value of 1 (meaning the target is healthy and reachable).

What does the up metric mean?

The up metric is a built-in Prometheus metric. It returns 1 if Prometheus successfully scraped the target and 0 if it failed. You should see three results: one for the prometheus job, one for grafana, and one for loki.

Clear the query editor, then enter this query to see the HTTP request rate:

rate(prometheus_http_requests_total[5m])

Click Run query.

You should see a graph showing the per-second rate of HTTP requests Prometheus has handled over the last 5 minutes.

Clear the query editor again, then enter this query to see memory usage:

process_resident_memory_bytes

Click Run query.

You should see a graph showing how much memory each scraped process is consuming in bytes.

What is PromQL?

PromQL is Prometheus's query language for selecting and aggregating time-series data. The rate() function calculates the per-second average rate of increase over a time window ([5m] means 5 minutes).

Simple metric names like process_resident_memory_bytes return the raw time series. You'll use these same queries when building your dashboard panels.

Not seeing any data?

Make sure you selected Prometheus (not Loki) from the data source drop-down. If you see no results for up, your Prometheus container may not be healthy. Switch to your terminal and run docker compose ps to check that all containers show a healthy status.

Help me troubleshoot why my Prometheus queries return no data in Grafana Explore.

Query Loki logs in the Explore view

Loki collects logs from your containers via Promtail. You query logs with LogQL, which uses a label-based selector syntax similar to PromQL.

In the same Explore view, switch to Loki using the data source drop-down in the upper left.
In the query editor, enter this query to see all Docker container logs:

{job="docker"}

Click Run query.

You should see a stream of log lines from all your running containers.

Now filter for errors in the Grafana container by entering this query:

{container="grafana"} |= "error"

Click Run query.

This filters logs from the Grafana container to only show lines containing the word "error". You might see a few results or none at all depending on your container's state.

Finally, try parsing structured logs from Prometheus by entering this query:

{container="prometheus"} | json

Click Run query.

The | json pipeline stage parses each log line as JSON and extracts fields as labels. You should see log entries with expanded fields you can click on.

What is LogQL?

LogQL queries start with a stream selector in curly braces (like {container="grafana"}) that selects which log streams to query. You then chain pipeline stages using the pipe | operator.

|= "error" is a line filter that keeps only lines containing that text. | json is a parser that extracts JSON fields from each log line.

Seeing no logs from Loki?

Verify that you switched the data source to Loki (not Prometheus). If {job="docker"} returns nothing, Promtail may not be collecting logs. Check the Promtail container status by running docker compose logs promtail in your terminal.

Help me figure out why my Loki queries in Grafana return no log results.

Create a dashboard with three panels

You've confirmed data is flowing from both Prometheus and Loki. Now you'll build a dashboard that displays key information at a glance, so you don't need to re-run Explore queries every time.

Click Dashboards in the left side menu.
Click New and select New Dashboard.
Click + Add visualization.
Select Prometheus as the data source.

This opens the panel editor where you can write a query and choose how to visualize it.

In the query editor at the bottom, enter:

process_resident_memory_bytes

Click Run query (or press Shift+Enter).
In the visualization picker on the right, make sure Time series is selected. This is the default for Prometheus data.
Under Panel options, enter the title Prometheus Memory Usage.
Click Back to dashboard in the upper right to return to the dashboard view.

You should see your first panel displaying a time-series graph of memory consumption.

Click Add in the dashboard header and select Visualization.
Select Prometheus as the data source.
In the query editor, enter:

up

Click Run query.
In the visualization picker, click All visualizations and select Stat.
Under Panel options, enter the title Scrape Targets.
Click Back to dashboard.

Your second panel shows a stat visualization with the number of healthy scrape targets.

Click Add in the dashboard header and select Visualization.
This time, select Loki as the data source.
In the query editor, enter:

{job="docker"}

Click Run query.
In the visualization picker, click All visualizations and select Logs.
Under Panel options, enter the title Container Logs.
Click Back to dashboard.

Your dashboard now has three panels: a time-series graph, a stat panel, and a live logs panel.

Click Save dashboard in the upper right.
Enter Monitoring Overview as the dashboard title.
Click Save.
Click Exit edit to leave edit mode.

Why three different visualization types?

Each visualization type serves a different purpose. Time series shows how a metric changes over time. Stat highlights a single important number (like the count of healthy targets). Logs displays a live feed of log entries from your containers.

Together, these three panels give you a complete observability picture: system performance, infrastructure health, and real-time event streams.

Panels showing No data?

Check that you selected the correct data source for each panel. The memory and scrape target panels use Prometheus, while the logs panel uses Loki. Also make sure you clicked Run query after entering each query.

Help me fix my Grafana dashboard panels that are showing No data.

Your monitoring stack now has a proper dashboard. Next up, you'll connect a local AI model to Grafana so it can explain these panels and auto-generate titles for you.

Connect the AI - Configure the Grafana LLM Plugin

In the previous step, you built a dashboard showing real-time metrics and logs from your containers. Your monitoring stack is collecting data, but right now you have to interpret everything yourself.

In this step, you'll connect your local AI model to Grafana through the grafana-llm-app plugin. Once connected, Grafana can use your Ollama-hosted model to generate panel titles, explain metrics, and summarize changes.

In this step, get ready to:

Enable the Grafana LLM plugin.
Configure the plugin to point at your local Ollama model.
Verify the AI connection with a health check.

Enable the LLM plugin in Grafana

The grafana-llm-app plugin was pre-installed when you started the Docker stack (via the GF_PLUGINS_PREINSTALL environment variable). It still needs to be enabled and configured before Grafana's AI features activate.

Return to your Grafana browser tab at http://localhost:3000.
In the left sidebar, click the hamburger menu icon (three horizontal lines) to expand the menu if it is collapsed.
Click Administration in the sidebar.
Click Plugins and data, then click Plugins.
In the search bar at the top of the Plugins page, type LLM.
Click the LLM plugin card (its ID is grafana-llm-app).
Click the Enable button to activate the plugin.

You should see the plugin page refresh to show configuration options now that it is enabled.

What is the Grafana LLM plugin?

The grafana-llm-app plugin acts as a proxy between Grafana's built-in AI features and your LLM backend. It handles all the authentication and request routing so that other Grafana components never need direct access to API keys.

It supports OpenAI, Azure OpenAI, Anthropic, and any custom API server with an OpenAI-compatible signature (like Ollama, vLLM, or LiteLLM).

Configure the plugin to use your local AI

Now you'll point the plugin at your Ollama instance running on your Windows host. Since Grafana is running inside a Docker container, it cannot reach localhost on your machine directly. Docker Desktop provides a special hostname to solve this.

On the plugin page, click the Configuration tab (if you are not already on it).
Select Use your own OpenAI Account.
In the OpenAI API URL field, enter:

http://host.docker.internal:11434/v1/

In the API Key field, type ollama (Ollama does not validate API keys, but the field is required).

Why host.docker.internal?

When Grafana runs inside a Docker container, localhost refers to the container itself, not your Windows machine. Docker Desktop for Windows provides host.docker.internal as a DNS name that resolves to your host machine's IP address.

This lets the containerized Grafana reach Ollama, which is running natively on Windows at port 11434. The /v1/ path is Ollama's OpenAI-compatible API endpoint.

Next, you need to tell the plugin which model to use for AI tasks.

Scroll down to the Model Settings section.
Click Model mappings to expand it.
In the Base model field, enter qwen2.5:1.5b.
In the Large model field, enter qwen2.5:1.5b.

Why set both Base and Large to the same model?

Grafana uses two model slots. Base handles cost-effective tasks like title generation. Large handles complex tasks like panel explanations.

Since you only have one local model (qwen2.5:1.5b), you map both slots to it. In an organization with a dedicated GPU server, you might assign a smaller model to Base and a larger one (like qwen2.5:7b) to Large.

Verify the AI connection

With all the settings in place, you can now test that Grafana can actually reach your Ollama model.

Click the Save & test button at the bottom of the configuration page.
Wait a few seconds for the health check to complete.

You should see a successful health check confirmation. This means Grafana can communicate with your local AI model through the plugin.

Health check failing?

If the health check does not succeed, try these fixes:

Verify Ollama is still running by opening PowerShell and running ollama list. You should see qwen2.5:1.5b in the output.
Double check the OpenAI API URL is exactly http://host.docker.internal:11434/v1/ with the trailing slash included.
Make sure the API Key field is not empty. Enter any text (like ollama) since Ollama ignores it but the plugin requires a value.

Still stuck? Help me troubleshoot my Grafana LLM plugin connection to Ollama.

Your local AI is now connected to Grafana. Next up, you'll put it to work by using AI-powered features across your dashboard panels.

Use AI Features in Grafana

Your monitoring stack is running, your data is flowing, and Ollama is connected to Grafana through the LLM plugin.

Now for the payoff. In this step, you'll use the AI features that the plugin unlocks. These features let your local model generate panel titles, explain what a metric means in plain language, and summarize dashboard changes when you save.

In this step, get ready to:

Use AI to auto-generate panel titles and descriptions.
Get an AI-powered explanation of a Prometheus memory panel.
Save a dashboard change and observe the AI-generated save summary.

Generate AI titles and descriptions for your panels

The LLM plugin adds a sparkle button (✨) next to the Title and Description fields whenever you edit a panel. Clicking it sends your panel's query context to the local AI, which generates a meaningful name and summary based on the data.

Return to your dashboard in Grafana at http://localhost:3000.
Click Edit in the top-right corner of the dashboard to enter edit mode.
Click on your Prometheus memory time series panel to select it, then click the panel title area or the pencil icon to open the panel editor.
In the panel editor, find the Title field in the right-hand options pane.
Click the ✨ Auto generate button next to the Title field.
Wait a few seconds for the AI to respond. Your local model will generate a title based on the process_resident_memory_bytes query.
Click the ✨ Auto generate button next to the Description field to generate a panel description.
Click Apply in the top-right corner of the panel editor to save your changes to the panel.

You should see the panel now displays an AI-generated title and a description that explains what the metric tracks.

What does Auto generate do?

When you click ✨ Auto generate, Grafana sends context about the panel's query (the metric name, data source, and any labels) to your local Ollama model via the LLM plugin.

The model returns a human-readable title or description. Since you're using qwen2.5:1.5b locally, responses may take a few seconds depending on your hardware. The quality is adequate for generating meaningful names, though a larger model would produce more detailed descriptions.

Not seeing the ✨ button?

Make sure you're in the panel edit view, not just viewing the panel. The ✨ Auto generate button appears only when you're editing a panel and can see the Title and Description fields in the options pane.

If the button doesn't appear at all, verify the LLM plugin is still enabled. Navigate to Administration > Plugins, search for LLM, and confirm it shows as enabled.

help me troubleshoot why the AI generate button isn't showing in my Grafana panel editor

Repeat the same process for your stat panel (scrape target status) to generate a title and description for it as well.

Get an AI explanation of your memory panel

Beyond generating titles, the LLM plugin can explain what a panel is showing in natural language. This is especially helpful when your team encounters unfamiliar metrics on a shared dashboard.

Go back to your dashboard view (exit the panel editor if you're still in it).
Hover over your Prometheus memory time series panel.
Click the three-dot menu icon that appears in the top-right corner of the panel.
Look for AI-related options in the menu. You may see options to generate a title or description directly from this menu as well.
Click Edit to open the panel editor for the memory panel.
In the panel editor, look for an AI explanation feature. The plugin provides the ability to explain the panel's query and data in plain language.

What is the panel explanation feature?

The LLM plugin registers an "explain this panel" extension point in Grafana. When available, it sends the panel's query (process_resident_memory_bytes) and its context to your local model.

The model returns a natural-language explanation of what the metric measures, what normal values look like, and what spikes might indicate. This is the same capability that Grafana Cloud offers through its Assistant feature, but running entirely on your machine.

You should receive a plain-language explanation describing that process_resident_memory_bytes tracks the physical memory used by the Prometheus process.

AI explanation taking too long or failing?

If the response hangs or times out, open a new PowerShell window and confirm Ollama is still running by executing ollama list. You should see qwen2.5:1.5b in the output.

The qwen2.5:1.5b model is small and fast, but complex queries may still take 10-15 seconds on a 4GB GPU. Give it time to complete.

help me debug why AI panel explanations are slow or failing in my local Grafana setup with Ollama

Observe the AI-generated save changes summary

The third AI feature activates when you save your dashboard. Instead of manually writing what you changed, the AI generates a summary automatically. In an organization, this creates a meaningful audit trail of dashboard edits without extra effort from your team.

Make sure you're back on your dashboard in edit mode (click Edit in the top-right if needed).
Make a visible change to your dashboard. For example, click on your logs panel, open the panel editor, and change the query from {job="docker"} to {container="grafana"}.
Click Apply to confirm the panel change.
Click the Save dashboard button (floppy disk icon) in the top-right corner of the dashboard.
In the save dialog that appears, look for the ✨ Auto generate button next to the description/changes field.
Click ✨ Auto generate to let the AI summarize what you changed.
Wait for the AI to generate a description of your changes. It should mention the query modification you just made.
Click Save to save the dashboard with the AI-generated summary.

You should see the AI produce a summary like "Changed logs panel query to filter by Grafana container" or similar phrasing.

Why does this matter for your organization?

In a team environment, dashboards are edited frequently by different people. Without clear change descriptions, it becomes impossible to understand why a dashboard looks different from last week.

The AI-generated save summary creates an automatic changelog. Every save includes a human-readable description of what changed, who changed it, and when. This is the same workflow your organization's team would use daily without any extra manual effort.

Save dialog not showing the AI generate option?

Make sure you've actually made a change to the dashboard before saving. If nothing has changed, the AI generate button may be disabled since there's nothing to summarize.

If the button appears but produces no output, check that Ollama is still responding. Open PowerShell and run ollama list to confirm the model is available.

help me troubleshoot the AI save changes summary feature in Grafana with a local Ollama backend

You've now experienced all three AI features that the LLM plugin enables. Your local model generates titles, explains metrics, and summarizes changes. Next up, you'll plan how to deploy this same setup across your organization's infrastructure.

Plan Your Organization Deployment

You've just experienced AI-powered monitoring firsthand. Your local stack proves the concept works. Now it's time to think bigger.

In this step, you'll map out how to deploy this same AI-enabled monitoring setup across your organization's datacentre. The key difference is that your local Ollama instance becomes a centralized GPU server that every Grafana instance can reach.

In this step, get ready to:

Understand the architecture shift from local to datacentre deployment.
Review the key decisions for model selection, networking, and access control.
Document a deployment plan with hardware requirements and team roles.

Understand the architecture difference

Right now, your setup has Ollama running directly on your Windows machine. Grafana reaches it through host.docker.internal. This works perfectly for one person on one laptop.

In an organization, multiple Grafana instances need access to the same AI capabilities. The solution is a dedicated GPU server running your LLM inference engine.

Return to the Grafana LLM plugin configuration page in your browser (Administration > Plugins > LLM).
Look at the OpenAI URL field. It currently reads http://host.docker.internal:11434/v1/.
In a datacentre deployment, this URL would point to your central AI server instead (something like http://ai-inference.internal.yourorg.com:11434/v1/).

What changes between local and production?

The only configuration change in Grafana itself is the URL in the LLM plugin settings. Instead of pointing to host.docker.internal, every Grafana instance in your organization would point to the internal DNS name of your dedicated GPU server.

The AI server itself would run Ollama or a more powerful alternative like vLLM for better throughput under concurrent requests from multiple users.

Here is the architecture at a glance:

Multiple Grafana servers across teams, each with the LLM plugin installed.
Each LLM plugin configured to point to the same central AI server URL.
One dedicated GPU server running Ollama or vLLM with a production-grade model.
All data stays within your organization's network. Nothing leaves the datacentre.

Review key deployment decisions

Scaling from one laptop to an organization involves four major decisions. Each one affects cost, quality, and security.

Review each of the four decision areas below and consider how they apply to your organization.

Decision 1: Model selection

You used qwen2.5:1.5b because it fits a 4GB GPU. With dedicated hardware (16GB+ VRAM), you can run much larger models that produce significantly better explanations and summaries.

Qwen 2.5:7b for good quality on a single 16GB GPU.
Qwen 2.5:32b for near-frontier quality on multi-GPU setups or high-VRAM cards (NVIDIA A10G, RTX 4090).

Decision 2: Network configuration

Your AI server needs a stable internal DNS name that all Grafana instances can resolve. The infrastructure team registers an internal DNS record (like ai-inference.internal.yourorg.com) pointing to the GPU server.

Firewall rules must allow Grafana servers to reach the AI server on port 11434 (Ollama's default). Traffic stays entirely within your internal network.

Decision 3: Access control with Grafana RBAC

Not everyone in your organization needs AI features. Grafana RBAC (Role-Based Access Control) lets you control which users or teams can access the LLM plugin's capabilities.

This prevents unnecessary load on your GPU server and ensures only authorized users generate AI queries against your monitoring data.

Decision 4: Cost considerations

A dedicated GPU server is a one-time hardware investment (or recurring lease cost). Compare this to cloud API pricing where you pay per token for every query, every day, forever.

For organizations already running Grafana on-premise, the GPU server addition is often cheaper within months and keeps all data private.

Document your deployment plan

Now pull everything together into a concrete plan. This is the document you would present to your team to get this deployed.

Open a text editor or notes app of your choice.
Write down the three sections of your deployment plan described below.

Your plan should cover these three areas:

Architecture diagram: Map the flow from Grafana servers to the LLM plugin to the central Ollama/vLLM GPU server. Include how many Grafana instances your organization runs and where the GPU server would sit in the network.
Hardware requirements: A production deployment needs a 16GB+ GPU (minimum). For larger models like Qwen 2.5:32b serving multiple concurrent users, consider NVIDIA A10G or RTX 4090 cards. Document the server specs your infrastructure team should provision.
Team roles: Identify who does what. The Grafana admin configures the LLM plugin across all instances (just changing the URL field you saw earlier). The infrastructure team provisions the GPU server, sets up internal DNS, and manages firewall rules.

Keep your local stack as a reference

Your local setup is a working proof-of-concept. When presenting this plan to your team, you can demo the AI features live on your laptop. The only difference between your demo and production is the URL in the plugin configuration.

Your deployment plan is ready. You've proven the concept locally and mapped out exactly how to scale it to your organization's datacentre. Next up, the Secret Mission challenges you to add alerting with AI-generated summaries.

Secret mission

AI-Powered Alerting

What happens when something goes wrong in your monitoring stack? Set up a Prometheus alerting rule that fires when memory usage exceeds a threshold, then watch how the LLM plugin auto-generates summaries to help your team understand incidents faster.

Clean Up Your Resources

This project runs entirely on your local machine, so there are no ongoing cloud costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

Docker containers: Grafana, Prometheus, Loki, and Promtail.
Docker volumes: grafana-storage, prometheus-storage, and loki-storage.
Ollama model: qwen2.5:1.5b.
Ollama application installed on Windows.
Project folder with configuration files (docker-compose.yml, prometheus.yml, loki-config.yml, promtail-config.yml, and the grafana/provisioning/ directory).

Keep everything running

No action needed. Choose this if you're still actively exploring your monitoring stack or experimenting with AI features.

Your Docker containers will keep running and collecting metrics and logs.
Prometheus retains 7 days of metrics data locally.
Loki retains 72 hours of log data.
Ollama stops automatically when idle, so it won't consume GPU resources unless you send it a request.

Pause - I'll come back to this later

Shut down running containers to free up system memory without losing your data or configuration.

Open PowerShell and navigate to your project folder.
Stop all containers by running:

docker compose stop

This pauses the containers without removing any data. Your volumes (metrics, logs, Grafana dashboards) are preserved.
To start everything again later, run docker compose start from the same folder.
Ollama stops automatically when idle. No action needed.

Delete - I don't want to use this again

Remove all project resources and start fresh. This deletes containers, volumes (all stored metrics, logs, and dashboards), the Ollama model, and optionally Ollama itself.

Open PowerShell and navigate to your project folder.
Remove all containers and volumes by running:

docker compose down -v

Remove the Ollama model by running:

ollama rm qwen2.5:1.5b

Uninstall Ollama from your machine by running:

$env:OLLAMA_UNINSTALL=1; irm https://ollama.com/install.ps1 | iex

Delete the project folder from your Desktop. You can do this from File Explorer or by running:

cd ~/Desktop
Remove-Item -Recurse -Force .\grafana-ai-monitoring

Replace grafana-ai-monitoring with whatever you named your project folder.

Nice Work!

Nice work! You've just built a complete AI-powered observability platform that runs entirely on your machine.

You've learned how to:

Set up a full monitoring stack (Grafana + Prometheus + Loki) collecting real metrics and logs using Docker Compose.
Connected a local AI model through the Grafana LLM plugin for private, on-premise panel explanations, title generation, and save summaries.
Created an organizational deployment plan for scaling local AI monitoring to a datacentre with dedicated GPU hardware.
Secret Mission: Set up AI-powered alerting with auto-generated incident summaries for faster incident response.

Ready to quiz yourself?