Build a QA Agent with MCP and Ollama

Build a CLI AI agent that queries Polarion/Jira defects via natural language.

Introduction

30 Second Summary

QA engineers spend a surprising amount of time running the same queries over and over. Checking open defects for a sprint, pulling test case status, looking up blocked items. What if you could just ask a question in plain English and get the answer instantly?

In this project, you will build a command-line AI agent that connects to your company's MCP server and uses a local LLM to answer QA queries in natural language.

What You'll Build

A Python-based AI agent that uses Ollama and the MCP Python SDK to translate natural language questions into tool calls against a Polarion/Jira backend.

By the end of this project, you'll have:

  • A Python script that connects to a remote MCP server via Streamable HTTP with token authentication.
  • An agent loop that wires Ollama's tool-calling capability to MCP tool execution.
  • A working prototype that handles the "query open defects" task end-to-end through conversational commands.
  • Secret Mission: Add conversational memory and support for a second QA task so your agent can handle multi-turn follow-up questions.

Are there any prerequisites?

You should be comfortable with basic Python (variables, functions, running scripts). You also need access to your company's CNH Polarion-Jira MCP Server endpoint and a Polarion Personal Access Token.

No prior experience with MCP, Ollama, or AI agents is needed. This project walks you through everything from installation to a working prototype.

Install Python and Set Up Your Project

Your QA agent needs a few foundational pieces before it can connect to anything. Python is the language everything runs in, and a handful of libraries handle the MCP connection and LLM communication.

In this step, you will install Python, create an isolated project folder with a virtual environment, and pull in the three packages your agent depends on.

In this step, get ready to:
  • Install Python 3.12 on Windows.
  • Create a project folder with a virtual environment.
  • Install the project dependencies.
Install Python 3.12

The MCP Python SDK requires Python 3.10 or higher. We will install 3.12 because it is the current stable release recommended for new projects.

  • Press the Windows key to open your search bar.
  • Type PowerShell and press Enter to open it.
  • Check whether Python is already installed by running this command:
python --version

✔️ I see Python 3.10 or higher

You are good to go. Your output looks something like Python 3.12.x. Skip ahead to the next substep.

ⓧ I see an older version

If you see a version lower than 3.10 (for example Python 3.9.x), you need to install a newer Python alongside it.

  • Open your browser and go to https://www.python.org/downloads/.
  • Click the Download Python 3.12 button for Windows.
  • Run the installer. On the very first screen, check the box labelled Add Python to PATH before clicking Install Now.
  • Wait for the installation to finish, then close the installer.
  • Close your current PowerShell window and reopen it so the new PATH takes effect.
  • Run python --version again. You should now see Python 3.12.x.

Still seeing the old version?

Windows may still resolve the old Python first if the new install path sits lower in your PATH variable. Open Settings, search for Edit environment variables for your account, and move the Python 3.12 entry above any older Python paths.

help me fix my Python PATH so the correct version runs

ⓧ Microsoft Store opens or 'python' is not recognized

On Windows 11, typing python without a real installation can open the Microsoft Store. This is caused by a built-in App Execution Alias.

  • First, disable the alias. Press the Windows key and type Manage App Execution Aliases.
  • Click Manage App Execution Aliases in the search results.
  • Find the entries for App Installer (python.exe) and App Installer (python3.exe). Toggle them both Off.
  • Now install Python. Open your browser and go to https://www.python.org/downloads/.
  • Click the Download Python 3.12 button for Windows.
  • Run the installer. On the very first screen, check the box labelled Add Python to PATH before clicking Install Now.
  • Wait for the installation to finish, then close the installer.
  • Close your current PowerShell window and reopen it.
  • Run python --version again. You should see Python 3.12.x.

Still opening the Microsoft Store?

Make sure you toggled off all the Python-related App Installer entries, then fully closed and reopened PowerShell. The alias takes effect per-session.

help me stop the Microsoft Store from opening when I type python

Create your project folder and virtual environment

A virtual environment isolates your project's packages from the rest of your system. This keeps dependencies clean and avoids version conflicts with other Python projects.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir qa-agent
cd qa-agent
  • Create a virtual environment inside the project folder by running:
python -m venv venv
  • Activate the virtual environment by running:
.\venv\Scripts\activate

You should see (venv) appear at the start of your terminal prompt. This confirms the virtual environment is active.

What is a virtual environment?

Think of it as a private workspace for this project. Any packages you install while the environment is active only live inside this folder. Your system Python and other projects stay untouched.

Seeing an execution policy error?

PowerShell may block the activation script by default. Fix it by running this command, then try the activate command again:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

help me fix the PowerShell execution policy error when activating a Python virtual environment

Install the project dependencies

Your agent relies on three packages: the MCP SDK for server communication, the Ollama client for talking to the local LLM, and httpx for HTTP requests.

  • Create the requirements.txt file that pins version ranges by running this command:
@"
mcp>=1.27,<2
ollama>=0.6.2
httpx>=0.27
"@ | Set-Content requirements.txt

What does this command do?

The @" ... "@ syntax is a PowerShell here-string. It creates a block of text spanning multiple lines. The pipe sends that text into Set-Content, which writes it to requirements.txt.

Each line pins a minimum version to ensure compatibility. mcp>=1.27,<2 means "at least 1.27 but below version 2". This protects you from breaking changes if a major version ships.

  • Install the packages by running:
pip install -r requirements.txt
  • Verify the installation succeeded by running:
pip list

You should see mcp, ollama, and httpx in the output list along with their installed versions.

Seeing a permissions error or packages not found?

Make sure your terminal prompt starts with (venv). If it does not, the virtual environment is not active. Run .\venv\Scripts\activate and try the install again.

help me troubleshoot pip install failing in my virtual environment

✔️ Awesome, I've got everything!

Great. Double check you have saved requirements.txt and that (venv) is showing in your prompt.

ⓧ I'd like to double check the full code

Here is what requirements.txt should contain:

mcp>=1.27,<2
ollama>=0.6.2
httpx>=0.27

Your environment is ready. Next up, you will install Ollama and pull the local LLM that powers your agent's reasoning.

Install Ollama and Pull the LLM

Your Python environment is ready and your dependencies are installed. Now you need the actual brain of your agent.

In this step, you will install Ollama, a tool that lets you run large language models locally on your machine. Then you will pull Qwen 2.5 7B, the model your agent will use to decide which tools to call.

In this step, get ready to:
  • Install Ollama on Windows and confirm it is running.
  • Pull the Qwen 2.5 7B model to your local machine.
  • Verify the model can handle tool-calling with a test prompt.
Download and install Ollama for Windows

Ollama is a lightweight runtime that downloads, manages, and serves LLMs on your local machine. It runs as a background service and exposes an API that the ollama Python package connects to.

  • Open your browser and navigate to the Ollama download page:
  • https://ollama.com/download/windows
  • Click Download for Windows to download OllamaSetup.exe.
  • Double-click OllamaSetup.exe to run the installer.
  • Follow the installer prompts to complete the installation.

What does the installer do?

The installer places Ollama in your user directory (no admin rights needed). After installation, Ollama automatically starts as a background service. The ollama command becomes available in any terminal window.

  • Open a new terminal window (close and reopen your existing one so it picks up the new PATH).
  • Confirm Ollama is installed by running:
ollama --version

✔️ I see a version number

Ollama is installed and ready. Move on to the next substep.

ⓧ Command not found

The terminal cannot find the ollama command. Try these fixes:

  • Close your terminal completely and open a brand new window. The PATH update only applies to new sessions.
  • Confirm the installer finished without errors. Re-run OllamaSetup.exe if needed.
  • Check that Ollama is running in the system tray (look for the Ollama icon near the clock).
Pull the Qwen 2.5 7B model

Ollama manages models as downloadable packages. You need to pull the specific model your agent will use for reasoning and tool selection.

  • Download the Qwen 2.5 7B model by running this command:
ollama pull qwen2.5:7b

What is Qwen 2.5 7B?

Qwen 2.5 7B is a 7-billion parameter language model that supports tool-calling. It produces reliable JSON output when asked to select and invoke tools.

The download is approximately 4.4GB. On your 32GB RAM machine, the model uses roughly 5GB when loaded. Since you do not have a dedicated GPU, Ollama will run the model on your CPU. Responses will be slower (a few seconds per reply) but perfectly functional for a CLI agent.

The download may take several minutes depending on your internet connection. You will see a progress bar in the terminal.

  • After the download completes, verify the model is available by running:
ollama ls

You should see qwen2.5:7b listed with its size and download date.

Download stuck or failing?

  • Check your internet connection. The model file is about 4.4GB.
  • Make sure Ollama is running in the background (check for the Ollama icon in your system tray).
  • If the pull was interrupted, run the same ollama pull qwen2.5:7b command again. It resumes from where it stopped.

Why isn't my model downloading?

Verify the model works with a test tool-calling prompt

Before building the full agent, you should confirm that the model can handle tool-calling. You will send a test prompt with a dummy tool definition using the ollama Python package (already installed in your virtual environment).

  • Make sure your virtual environment is still activated in your terminal.
  • Run this test script to check that the model can select a tool:
python -c "
from ollama import chat

response = chat(
    model='qwen2.5:7b',
    messages=[{'role': 'user', 'content': 'What is the weather in London?'}],
    tools=[{
        'type': 'function',
        'function': {
            'name': 'get_weather',
            'description': 'Get the current weather for a location',
            'parameters': {
                'type': 'object',
                'properties': {
                    'location': {'type': 'string', 'description': 'City name'}
                },
                'required': ['location']
            }
        }
    }]
)

if response.message.tool_calls:
    call = response.message.tool_calls[0]
    print(f'Tool selected: {call.function.name}')
    print(f'Arguments: {call.function.arguments}')
else:
    print(f'Response: {response.message.content}')
"

What does this test do?

This script sends a natural language question ("What is the weather in London?") along with a tool definition for get_weather. If the model is working correctly, it recognizes that it should call the tool rather than answer directly.

The response.message.tool_calls field contains the model's decision. You access .function.name and .function.arguments to see which tool the model chose and what parameters it passed.

You should see output like:

Tool selected: get_weather
Arguments: {'location': 'London'}

First run takes longer

The first time you use the model after pulling it, Ollama loads it into memory. This initial load can take 10-30 seconds on CPU. Subsequent calls are faster because the model stays loaded.

Seeing an error or no tool call?

  • If you see a connection error, make sure Ollama is running. Check your system tray for the Ollama icon, or run ollama serve in a separate terminal.
  • If the model responds with plain text instead of a tool call, try running the test again. Smaller models occasionally skip tool calls on the first attempt.
  • If you get a "model not found" error, confirm you pulled the correct model with ollama ls and check that qwen2.5:7b appears in the list.

Help me fix my Ollama tool-calling test.

Why Qwen 2.5 instead of a cloud model?

Your agent will query company data from Polarion and Jira. Running the LLM locally means your data never leaves your machine. There is no API key to manage, no per-token cost, and no dependency on an internet connection during agent use.

A cloud model like GPT-4 would be faster, but the local approach gives you full privacy and zero ongoing cost for a prototype that handles sensitive engineering data.

Your LLM is installed, loaded, and proven to handle tool-calling. Next up, you will connect to the MCP server and discover what tools your agent can use.

Connect to the MCP Server and Discover Tools

Your local LLM is ready. Now you need to give it something to reason about.

In this step, you will write a Python script that connects to your company's MCP server over Streamable HTTP, authenticates with your Polarion Personal Access Token, and prints out every tool the server exposes. This is the foundation your agent will build on.

In this step, get ready to:
  • Create agent.py with imports and configuration.
  • Write the async function that connects to the MCP server and discovers tools.
  • Run the script and verify that tools are printed to the terminal.
Create agent.py with imports and configuration

Your agent needs to know three things: where the MCP server lives, how to authenticate, and which LLM model to use for reasoning.

  • In Cursor, create a new file called agent.py inside your qa-agent folder.
  • Add the following imports and configuration block:
import asyncio
import json
import sys

import httpx
from ollama import chat
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client

# Configuration
MCP_SERVER_URL = "https://your-mcp-server-endpoint.com/mcp"
POLARION_PAT = "your-polarion-personal-access-token"
OLLAMA_MODEL = "qwen2.5:7b"

SYSTEM_PROMPT = """You are a QA assistant that helps engineers query and manage defects in Polarion and Jira.
When the user asks about defects, sprints, or test cases, use the available tools to find the information.
Always use tools when the user's request matches an available tool's purpose.
Present results clearly and concisely."""

What does this code do?

  • asyncio, json, and sys are standard library modules for async execution, JSON handling, and clean exits.
  • httpx provides the async HTTP client that carries your authentication header to the MCP server.
  • ClientSession and streamablehttp_client are MCP SDK components that handle the JSON-RPC protocol for you.
  • MCP_SERVER_URL and POLARION_PAT are placeholders you will replace with your actual server endpoint and Personal Access Token.
  • SYSTEM_PROMPT tells the LLM how to behave when it receives a user query.
  • Replace the value of MCP_SERVER_URL with your company's actual MCP server endpoint URL.
  • Replace the value of POLARION_PAT with your Polarion Personal Access Token.
  • Save agent.py.

Not sure where to find your PAT?

Your Polarion Personal Access Token is generated from your Polarion user settings. Ask your team lead or check your company's internal documentation for the exact page.

Never share your PAT or commit it to version control.

help me find my Polarion Personal Access Token

Write the connection and tool discovery function

The MCP protocol uses an async session model. Your client opens a persistent connection, initializes the session, and then queries the server for available tools.

  • Below the SYSTEM_PROMPT block in agent.py, add the following function:
async def run_agent():
    """Main agent loop."""
    print("Connecting to MCP server...")

    http_client = httpx.AsyncClient(
        headers={"Authorization": f"Bearer {POLARION_PAT}"},
        timeout=httpx.Timeout(30, read=300),
    )

    async with http_client:
        async with streamablehttp_client(
            url=MCP_SERVER_URL,
            http_client=http_client,
        ) as (read_stream, write_stream, _):
            async with ClientSession(read_stream, write_stream) as session:
                await session.initialize()

                # Discover tools
                tools_response = await session.list_tools()
                mcp_tools = tools_response.tools
                print(f"Connected! Found {len(mcp_tools)} tools:")
                for tool in mcp_tools:
                    print(f"  - {tool.name}: {tool.description}")
                print()

What does this function do?

  • httpx.AsyncClient creates an HTTP client that attaches your Bearer token to every request automatically.
  • httpx.Timeout(30, read=300) sets a 30-second connection timeout and a 5-minute read timeout. MCP tool calls can take time, especially on a remote server.
  • streamablehttp_client opens the Streamable HTTP transport and returns a 3-tuple: a read stream, a write stream, and a session ID getter (which we discard with _).
  • session.initialize() performs the MCP handshake. Both client and server agree on protocol capabilities.
  • session.list_tools() asks the server to return every tool it exposes. Each tool has a name, description, and input schema.
  • Save agent.py.
Add the entry point and run the script

Python scripts need a main() entry point to actually execute. This function wraps the async agent in a clean startup and shutdown flow.

  • Below the run_agent() function in agent.py, add the entry point:
def main():
    print("=" * 50)
    print("  QA Agent - Powered by Ollama + MCP")
    print("=" * 50)
    print()

    try:
        asyncio.run(run_agent())
    except KeyboardInterrupt:
        print("\nShutting down...")
    except Exception as e:
        print(f"\nError: {e}")
        sys.exit(1)


if __name__ == "__main__":
    main()

What does this code do?

  • asyncio.run(run_agent()) starts the Python async event loop and runs your agent function inside it.
  • The KeyboardInterrupt handler catches Ctrl+C so the agent exits gracefully instead of spitting out a traceback.
  • if __name__ == "__main__" ensures main() only runs when you execute the file directly, not when it is imported.
  • Save agent.py.
  • Run the script by entering this command in your terminal:
python agent.py

You should see the banner print, followed by a list of tools discovered from your MCP server. The output looks something like this:

==================================================
  QA Agent - Powered by Ollama + MCP
==================================================

Connecting to MCP server...
Connected! Found 5 tools:
  - get_defects: Query defects by sprint or status
  - get_test_cases: Retrieve test cases for a work item
  ...

What should I see?

The exact number of tools and their names depend on your company's MCP server configuration. The important thing is that you see "Connected!" and a list of tool names with descriptions.

If your server exposes 10+ tools, you will see all of them listed. Each tool name and description will be useful in the next step when you wire them to the LLM.

Seeing a connection error?

  • Double-check that MCP_SERVER_URL matches your server's exact URL. Some servers require a trailing slash (e.g., /mcp/ instead of /mcp).
  • Verify your POLARION_PAT is correct and has not expired. Try regenerating it from your Polarion user settings if unsure.
  • Make sure Ollama is still running in the background. The script imports from the ollama package, which requires the Ollama service to be active.

My agent script is failing with a connection error

✔️ Awesome, I've got everything!

Great. Double check you have saved agent.py and that the script runs without errors.

ⓧ I'd like to double check the full code

Here is the complete agent.py so far:

import asyncio
import json
import sys

import httpx
from ollama import chat
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client

# Configuration
MCP_SERVER_URL = "https://your-mcp-server-endpoint.com/mcp"
POLARION_PAT = "your-polarion-personal-access-token"
OLLAMA_MODEL = "qwen2.5:7b"

SYSTEM_PROMPT = """You are a QA assistant that helps engineers query and manage defects in Polarion and Jira.
When the user asks about defects, sprints, or test cases, use the available tools to find the information.
Always use tools when the user's request matches an available tool's purpose.
Present results clearly and concisely."""


async def run_agent():
    """Main agent loop."""
    print("Connecting to MCP server...")

    http_client = httpx.AsyncClient(
        headers={"Authorization": f"Bearer {POLARION_PAT}"},
        timeout=httpx.Timeout(30, read=300),
    )

    async with http_client:
        async with streamablehttp_client(
            url=MCP_SERVER_URL,
            http_client=http_client,
        ) as (read_stream, write_stream, _):
            async with ClientSession(read_stream, write_stream) as session:
                await session.initialize()

                # Discover tools
                tools_response = await session.list_tools()
                mcp_tools = tools_response.tools
                print(f"Connected! Found {len(mcp_tools)} tools:")
                for tool in mcp_tools:
                    print(f"  - {tool.name}: {tool.description}")
                print()


def main():
    print("=" * 50)
    print("  QA Agent - Powered by Ollama + MCP")
    print("=" * 50)
    print()

    try:
        asyncio.run(run_agent())
    except KeyboardInterrupt:
        print("\nShutting down...")
    except Exception as e:
        print(f"\nError: {e}")
        sys.exit(1)


if __name__ == "__main__":
    main()

Your agent can now see every tool the MCP server offers. Next up, you will teach it to reason about those tools and decide which one to call based on your natural language questions.

Build the Agent Loop

Your agent can connect to the MCP server and see what tools are available. Now it needs a brain. The core of any AI agent is a Reason-Act loop: the LLM receives your question, reasons about which tool to call, acts by executing it, and then formats the result for you.

In this step, you will build that loop. The agent will take natural language input, pass it along with available tool definitions to Ollama, and let the LLM decide what to do next.

In this step, get ready to:
  • Convert MCP tool schemas into Ollama's tool-calling format.
  • Build the Reason-Act loop that connects user input to tool execution.
  • Run the agent as an interactive CLI.
Convert MCP tool definitions to Ollama's tool-calling format

MCP and Ollama describe tools using different schemas. MCP tools have fields like inputSchema, while Ollama expects a nested function object with parameters. You need a translator function that converts between them.

  • In agent.py, find the run_agent() function definition.
  • Add this new function above the async def run_agent(): line:
def convert_mcp_tools_to_ollama(mcp_tools):
    """Convert MCP tool definitions to Ollama's tool-calling format."""
    ollama_tools = []
    for tool in mcp_tools:
        ollama_tool = {
            "type": "function",
            "function": {
                "name": tool.name,
                "description": tool.description or "",
                "parameters": tool.inputSchema if tool.inputSchema else {"type": "object", "properties": {}},
            },
        }
        ollama_tools.append(ollama_tool)
    return ollama_tools

What does this code do?

This function loops through each MCP tool and reshapes it into the JSON structure Ollama expects.

  • tool.name and tool.description map directly across.
  • tool.inputSchema becomes parameters inside a function wrapper. If a tool has no input schema, a safe empty object is used as a fallback.

You won't see a change yet. The next chunk uses this function inside the agent loop.

Implement the Reason-Act cycle

The ReAct pattern works in three phases. First, the LLM reasons about your question and decides which tool to call. Then the agent acts by executing that tool via MCP. Finally, the LLM formats the raw result into a human-readable response.

  • In agent.py, find the print() line at the end of the tool discovery section inside run_agent():
                print()
  • Add the following code directly below that print() line, at the same indentation level:
                # Convert tools to Ollama format
                ollama_tools = convert_mcp_tools_to_ollama(mcp_tools)

                # Agent loop
                print("QA Agent ready. Type your questions (or 'quit' to exit).")
                print("-" * 50)

                while True:
                    user_input = input("\nYou: ").strip()
                    if user_input.lower() in ("quit", "exit", "q"):
                        print("Goodbye!")
                        break
                    if not user_input:
                        continue

                    messages = [
                        {"role": "system", "content": SYSTEM_PROMPT},
                        {"role": "user", "content": user_input},
                    ]

                    # Ask the LLM what to do
                    response = chat(
                        model=OLLAMA_MODEL,
                        messages=messages,
                        tools=ollama_tools,
                    )

                    # Check if the model wants to call a tool
                    if response.message.tool_calls:
                        tool_call = response.message.tool_calls[0]
                        tool_name = tool_call.function.name
                        tool_args = tool_call.function.arguments

                        print(f"\n[Calling tool: {tool_name}]")
                        print(f"[Arguments: {json.dumps(tool_args, indent=2)}]")

                        # Execute the tool via MCP
                        try:
                            result = await session.call_tool(tool_name, arguments=tool_args)
                            tool_result_text = ""
                            for content_block in result.content:
                                if hasattr(content_block, "text"):
                                    tool_result_text += content_block.text
                        except Exception as e:
                            tool_result_text = f"Error calling tool: {e}"

                        # Send the tool result back to the LLM for formatting
                        messages.append(response.message)
                        messages.append(
                            {"role": "tool", "content": tool_result_text}
                        )

                        final_response = chat(
                            model=OLLAMA_MODEL,
                            messages=messages,
                            tools=ollama_tools,
                        )
                        print(f"\nAgent: {final_response.message.content}")
                    else:
                        print(f"\nAgent: {response.message.content}")

What does this code do?

This is the full Reason-Act loop. Here is how each section works:

  • convert_mcp_tools_to_ollama(mcp_tools) translates the discovered tools into the format Ollama understands.
  • The while True loop reads user input and exits on quit, exit, or q.
  • chat() sends the user's message plus the tool definitions to Qwen 2.5 7B. The LLM decides whether a tool call is needed.
  • If the LLM requests a tool call, session.call_tool() executes it on the MCP server. The raw result is then sent back to the LLM for a second chat() call to produce a human-readable answer.
  • If no tool call is needed, the LLM's direct response is printed as-is.

Why two LLM calls?

The first call lets the LLM decide WHAT tool to call and with what arguments. The second call takes the raw tool result (often structured data like JSON) and turns it into a readable summary for you.

This is the same pattern used by tools like Cursor and Claude Desktop when they call MCP servers.

  • Save agent.py.

Seeing indentation errors?

  • The new code must sit at the same indentation level as the tools_response = await session.list_tools() line above it. That is 4 levels deep (16 spaces).
  • Make sure the convert_mcp_tools_to_ollama() function is placed outside of run_agent() at the top level of the file. It should have zero indentation.

Help me fix an indentation error in my agent.py file

Wire it all together into an interactive CLI agent

The main() function at the bottom of your file already calls asyncio.run(run_agent()) and handles keyboard interrupts. Everything is connected.

  • Verify your full agent.py matches the code below.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file.

ⓧ I'd like to double check the full code

import asyncio
import json
import sys

import httpx
from ollama import chat
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client

# Configuration
MCP_SERVER_URL = "https://your-mcp-server-endpoint.com/mcp"
POLARION_PAT = "your-polarion-personal-access-token"
OLLAMA_MODEL = "qwen2.5:7b"

SYSTEM_PROMPT = """You are a QA assistant that helps engineers query and manage defects in Polarion and Jira.
When the user asks about defects, sprints, or test cases, use the available tools to find the information.
Always use tools when the user's request matches an available tool's purpose.
Present results clearly and concisely."""


def convert_mcp_tools_to_ollama(mcp_tools):
    """Convert MCP tool definitions to Ollama's tool-calling format."""
    ollama_tools = []
    for tool in mcp_tools:
        ollama_tool = {
            "type": "function",
            "function": {
                "name": tool.name,
                "description": tool.description or "",
                "parameters": tool.inputSchema if tool.inputSchema else {"type": "object", "properties": {}},
            },
        }
        ollama_tools.append(ollama_tool)
    return ollama_tools


async def run_agent():
    """Main agent loop."""
    print("Connecting to MCP server...")

    http_client = httpx.AsyncClient(
        headers={"Authorization": f"Bearer {POLARION_PAT}"},
        timeout=httpx.Timeout(30, read=300),
    )

    async with http_client:
        async with streamablehttp_client(
            url=MCP_SERVER_URL,
            http_client=http_client,
        ) as (read_stream, write_stream, _):
            async with ClientSession(read_stream, write_stream) as session:
                await session.initialize()

                # Discover tools
                tools_response = await session.list_tools()
                mcp_tools = tools_response.tools
                print(f"Connected! Found {len(mcp_tools)} tools:")
                for tool in mcp_tools:
                    print(f"  - {tool.name}: {tool.description}")
                print()

                # Convert tools to Ollama format
                ollama_tools = convert_mcp_tools_to_ollama(mcp_tools)

                # Agent loop
                print("QA Agent ready. Type your questions (or 'quit' to exit).")
                print("-" * 50)

                while True:
                    user_input = input("\nYou: ").strip()
                    if user_input.lower() in ("quit", "exit", "q"):
                        print("Goodbye!")
                        break
                    if not user_input:
                        continue

                    messages = [
                        {"role": "system", "content": SYSTEM_PROMPT},
                        {"role": "user", "content": user_input},
                    ]

                    # Ask the LLM what to do
                    response = chat(
                        model=OLLAMA_MODEL,
                        messages=messages,
                        tools=ollama_tools,
                    )

                    # Check if the model wants to call a tool
                    if response.message.tool_calls:
                        tool_call = response.message.tool_calls[0]
                        tool_name = tool_call.function.name
                        tool_args = tool_call.function.arguments

                        print(f"\n[Calling tool: {tool_name}]")
                        print(f"[Arguments: {json.dumps(tool_args, indent=2)}]")

                        # Execute the tool via MCP
                        try:
                            result = await session.call_tool(tool_name, arguments=tool_args)
                            tool_result_text = ""
                            for content_block in result.content:
                                if hasattr(content_block, "text"):
                                    tool_result_text += content_block.text
                        except Exception as e:
                            tool_result_text = f"Error calling tool: {e}"

                        # Send the tool result back to the LLM for formatting
                        messages.append(response.message)
                        messages.append(
                            {"role": "tool", "content": tool_result_text}
                        )

                        final_response = chat(
                            model=OLLAMA_MODEL,
                            messages=messages,
                            tools=ollama_tools,
                        )
                        print(f"\nAgent: {final_response.message.content}")
                    else:
                        print(f"\nAgent: {response.message.content}")


def main():
    print("=" * 50)
    print("  QA Agent - Powered by Ollama + MCP")
    print("=" * 50)
    print()

    try:
        asyncio.run(run_agent())
    except KeyboardInterrupt:
        print("\nShutting down...")
    except Exception as e:
        print(f"\nError: {e}")
        sys.exit(1)


if __name__ == "__main__":
    main()
  • Make sure Ollama is running in the background. You can check by running:
ollama list

You should see qwen2.5:7b listed in the output. If Ollama is not running, it will start automatically when you run the agent in the next step.

Don't see the model listed?

  • If Ollama is not running, open it from your Start menu or system tray. Then try ollama list again.
  • If the model is missing, re-pull it with ollama pull qwen2.5:7b.

Help me check if Ollama is running and my model is available

Your agent loop is ready. Next up, you will run it and test it with real queries about sprint defects.

Test with Query Open Defects for a Sprint

Your agent code is complete. The MCP client, Ollama tool-calling integration, and interactive loop are all wired together.

Now it is time to prove the pipeline works end-to-end. You will fire up the agent, ask it a real question about sprint defects, and watch the LLM decide which tool to call, execute it through MCP, and present the results back in plain English.

In this step, get ready to:
  • Run the agent and test it with natural language queries about sprint defects.
  • Observe how the LLM selects the right MCP tool and passes correct arguments.
  • Handle edge cases like connection errors and empty results.
Run the agent and test with natural language queries

Before running the agent, make sure Ollama is still running in the background. The agent script needs Ollama's local API to be available for tool-calling requests.

  • Confirm Ollama is running by checking its status:
ollama list

You should see qwen2.5:7b in the list of available models. If not, start Ollama and pull the model again before continuing.

  • Launch the QA agent by running:
python agent.py

You should see the agent banner, followed by a connection message listing all discovered tools from your MCP server.

What should I see?

The agent prints a header (QA Agent - Powered by Ollama + MCP), connects to your MCP server, and lists all available tools with their names and descriptions. You then see a You: prompt waiting for your input.

The connection step may take a few seconds depending on your network and MCP server response time.

  • At the You: prompt, type a natural language query about sprint defects:
Show me open defects for Sprint 23

The agent sends your query to Qwen 2.5 7B along with the tool definitions. The LLM decides which MCP tool to call and what arguments to pass.

Why does the response take a while?

Qwen 2.5 7B is running on your CPU, which produces roughly 5-8 tokens per second. Each response involves two LLM calls. The first call decides which tool to use, and the second call formats the tool results into a readable answer.

You will see a short pause while the LLM generates the tool-call JSON, then another pause after the MCP tool returns results while the LLM formats its final answer.

Agent stuck on Connecting to MCP server?

If the agent hangs on the connection step, check that your MCP_SERVER_URL in agent.py is correct. Some MCP servers require a trailing slash (/mcp/ instead of /mcp).

Also verify that your POLARION_PAT is valid and has not expired. An invalid token typically causes a connection timeout or a 401 error.

help me debug my MCP connection issue

Observe how the LLM selects the right tool and passes correct arguments

When the agent processes your query, it prints exactly what is happening behind the scenes. This transparency lets you verify the LLM is making sensible decisions.

  • Look at the output after you submit your query. You should see two bracketed lines showing the tool call:
[Calling tool: <tool-name>]
[Arguments: {
  "sprint": "Sprint 23",
  "status": "open"
}]

What is happening here?

The [Calling tool: ...] line shows which MCP tool the LLM selected from the list of available tools. The [Arguments: ...] block shows the JSON arguments the LLM generated to pass to that tool.

The LLM mapped your natural language ("Show me open defects for Sprint 23") into structured parameters that match the tool's input schema. This is the core intelligence of the agent. It translates human intent into precise API calls.

  • After the tool executes, read the Agent: response. The LLM takes the raw tool results and formats them into a readable summary.
  • Try a few more queries to see how the model handles different phrasings:
What defects are still open in Sprint 23?
List unresolved bugs for the current sprint
Are there any critical defects blocking Sprint 23?

Each variation should trigger the same (or a related) tool call but with slightly different arguments, depending on how the LLM interprets your intent.

Test with queries outside the tool scope

Try asking something the tools cannot answer, like "What is the weather today?" The agent should respond conversationally without attempting a tool call. This confirms the LLM only uses tools when the query matches their purpose.

Handle edge cases

A production-ready agent needs to handle situations where things do not go as expected. Your agent already has error handling built in. Let's verify it works.

  • Test with a query that returns no results. Try asking about a sprint that does not exist:
Show me open defects for Sprint 999

The LLM should still call the tool, but the MCP server returns an empty result set. The agent's final response should indicate that no defects were found.

  • Test the connection error handling. Stop Ollama (close its process), then try typing a query in the agent. You should see an error message rather than a crash.
  • Restart Ollama after testing the error path so the agent is functional again.

How does the error handling work?

The try/except block around session.call_tool() catches any exception from the MCP server (network errors, invalid arguments, server-side failures). Instead of crashing, the agent passes the error message to the LLM, which explains the problem in plain English.

The outer try/except in main() catches broader failures like Ollama being unreachable. KeyboardInterrupt lets you exit cleanly with Ctrl+C.

  • When you are satisfied that the agent handles all cases correctly, type quit to exit the agent.

LLM not calling the right tool?

If the LLM picks the wrong tool or generates malformed arguments, try rephrasing your query to be more specific. Smaller models occasionally misinterpret ambiguous requests.

Check that your MCP server's tool descriptions are clear and specific. The LLM relies on these descriptions to decide which tool matches your query.

help me fix my agent's tool selection

✔️ Awesome, I've got everything!

Great. Double check you have saved agent.py with no modifications from testing.

ⓧ I'd like to double check the full code

Here is the complete agent.py file:

import asyncio
import json
import sys

import httpx
from ollama import chat
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client

# Configuration
MCP_SERVER_URL = "https://your-mcp-server-endpoint.com/mcp"
POLARION_PAT = "your-polarion-personal-access-token"
OLLAMA_MODEL = "qwen2.5:7b"

SYSTEM_PROMPT = """You are a QA assistant that helps engineers query and manage defects in Polarion and Jira.
When the user asks about defects, sprints, or test cases, use the available tools to find the information.
Always use tools when the user's request matches an available tool's purpose.
Present results clearly and concisely."""


def convert_mcp_tools_to_ollama(mcp_tools):
    """Convert MCP tool definitions to Ollama's tool-calling format."""
    ollama_tools = []
    for tool in mcp_tools:
        ollama_tool = {
            "type": "function",
            "function": {
                "name": tool.name,
                "description": tool.description or "",
                "parameters": tool.inputSchema if tool.inputSchema else {"type": "object", "properties": {}},
            },
        }
        ollama_tools.append(ollama_tool)
    return ollama_tools


async def run_agent():
    """Main agent loop."""
    print("Connecting to MCP server...")

    http_client = httpx.AsyncClient(
        headers={"Authorization": f"Bearer {POLARION_PAT}"},
        timeout=httpx.Timeout(30, read=300),
    )

    async with http_client:
        async with streamablehttp_client(
            url=MCP_SERVER_URL,
            http_client=http_client,
        ) as (read_stream, write_stream, _):
            async with ClientSession(read_stream, write_stream) as session:
                await session.initialize()

                # Discover tools
                tools_response = await session.list_tools()
                mcp_tools = tools_response.tools
                print(f"Connected! Found {len(mcp_tools)} tools:")
                for tool in mcp_tools:
                    print(f"  - {tool.name}: {tool.description}")
                print()

                # Convert tools to Ollama format
                ollama_tools = convert_mcp_tools_to_ollama(mcp_tools)

                # Agent loop
                print("QA Agent ready. Type your questions (or 'quit' to exit).")
                print("-" * 50)

                while True:
                    user_input = input("\nYou: ").strip()
                    if user_input.lower() in ("quit", "exit", "q"):
                        print("Goodbye!")
                        break
                    if not user_input:
                        continue

                    messages = [
                        {"role": "system", "content": SYSTEM_PROMPT},
                        {"role": "user", "content": user_input},
                    ]

                    # Ask the LLM what to do
                    response = chat(
                        model=OLLAMA_MODEL,
                        messages=messages,
                        tools=ollama_tools,
                    )

                    # Check if the model wants to call a tool
                    if response.message.tool_calls:
                        tool_call = response.message.tool_calls[0]
                        tool_name = tool_call.function.name
                        tool_args = tool_call.function.arguments

                        print(f"\n[Calling tool: {tool_name}]")
                        print(f"[Arguments: {json.dumps(tool_args, indent=2)}]")

                        # Execute the tool via MCP
                        try:
                            result = await session.call_tool(tool_name, arguments=tool_args)
                            tool_result_text = ""
                            for content_block in result.content:
                                if hasattr(content_block, "text"):
                                    tool_result_text += content_block.text
                        except Exception as e:
                            tool_result_text = f"Error calling tool: {e}"

                        # Send the tool result back to the LLM for formatting
                        messages.append(response.message)
                        messages.append(
                            {"role": "tool", "content": tool_result_text}
                        )

                        final_response = chat(
                            model=OLLAMA_MODEL,
                            messages=messages,
                            tools=ollama_tools,
                        )
                        print(f"\nAgent: {final_response.message.content}")
                    else:
                        print(f"\nAgent: {response.message.content}")


def main():
    print("=" * 50)
    print("  QA Agent - Powered by Ollama + MCP")
    print("=" * 50)
    print()

    try:
        asyncio.run(run_agent())
    except KeyboardInterrupt:
        print("\nShutting down...")
    except Exception as e:
        print(f"\nError: {e}")
        sys.exit(1)


if __name__ == "__main__":
    main()

And the complete requirements.txt:

mcp>=1.27,<2
ollama>=0.6.2
httpx>=0.27

Secret mission

Add Multi-Task Support and Conversational Memory

Your agent handles one question at a time and immediately forgets the answer. In this mission, give it conversational memory so it can handle follow-up questions like 'Mark the first one as In Progress' after asking about open defects.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Project folder on your Desktop (contains agent.py, requirements.txt, and your Python virtual environment).
  • Ollama application (installed on Windows).
  • Qwen 2.5 7B model (downloaded via Ollama, approximately 4.4 GB).
  • Python 3.12 (installed from python.org).

Keep everything running

No action needed. Choose this if you plan to keep developing your QA agent or want Ollama available for other projects.

  • Ollama continues running in the background as a startup item. It uses minimal resources when no model is actively loaded.
  • The Qwen 2.5 7B model stays on disk (~4.4 GB) ready for future use.
  • Your project folder and virtual environment remain on your Desktop.
  • Python 3.12 stays installed and available for other projects.

Pause - I'll come back to this later

Free up memory and prevent Ollama from starting automatically. Your files and model stay on disk for when you return.

  • Right-click the Ollama icon in your system tray and select Quit.
  • Open Task Manager, go to the Startup apps tab, find Ollama, and click Disable.
  • When you want to use the agent again, re-enable Ollama in Startup apps and launch it from the Start menu.

Delete - I don't want to use this again

Remove all project resources and start fresh. This frees up disk space and removes installed tools.

  • Delete the Qwen 2.5 7B model by running this command in your terminal:
ollama rm qwen2.5:7b
  • Uninstall Ollama by opening Settings, then Apps, then Installed apps. Search for Ollama and click Uninstall. The uninstaller will ask if you want to remove remaining model files.
  • Delete your project folder by running this command in PowerShell:
Remove-Item -Recurse -Force ~\Desktop\qa-agent

Folder name may differ

Replace qa-agent with whatever you named your project folder on the Desktop.

  • Uninstall Python 3.12 the same way as Ollama. Open Settings, then Apps, then Installed apps. Search for Python 3.12 and click Uninstall. Only do this if you have no other Python projects on your machine.

Nice Work!

Nice Work!

Nice work! You just built a local AI agent that connects to an enterprise MCP server and automates QA queries through natural language.

You've learned how to:

  • Connect to a remote MCP server using Streamable HTTP transport with token authentication.
  • Use a local LLM (Qwen 2.5 7B via Ollama) for intelligent tool-calling decisions.
  • Build a complete agent loop that takes natural language, selects the right tool, executes it via MCP, and formats the result for the user.
  • Added multi-task support and conversational memory so the agent handles follow-up questions across multiple turns.

Ready to quiz yourself?